How to Split a Multi-Line String by Newline
The aim of this pageđź“ť is to explain how to split a string by newlines and loop over each line in Bash based on the particular example of processing a multi-line string.
2 min readAug 26, 2024
Use while
loop to read each line.
- Example string required for data wrangling, say to prepend it with an increasing integer starting from
1
a="project/xyz_abc_xyz/data_analysis/
project/xyz_abc_xyz/data_cleaning/
project/xyz_abc_xyz/data_migration/
project/xyz_abc_xyz/data_visualization/
project/xyz_abc_xyz/data_backup/
project/xyz_abc_xyz/data_archiving/
project/xyz_abc_xyz/data_processing/
project/xyz_abc_xyz/data_reporting/
project/xyz_abc_xyz/data_validation/
project/xyz_abc_xyz/data_integration/
project/xyz_abc_xyz/data_transformation/
project/xyz_abc_xyz/data_storage/
project/xyz_abc_xyz/data_retrieval/
project/xyz_abc_xyz/data_security/
project/xyz_abc_xyz/data_management/"
- Use
echo
andwhile
loop:
echo "$a" | while IFS= read -r line; do
echo "Processing: $line"
# Add your processing logic here
done
- Use
for
loop withIFS
set to newline:
IFS=$'\n'
for line in $a; do
echo "Processing: $line"
# Add your processing logic here
done
- The issue with variable persistence in
while
loop:
i=1
echo "$a" | while IFS= read -r line; do
echo "$i $line"
((i+=1))
done
echo $i # Output: 1 !
- Solution using
for
loop:
i=1
IFS=$'\n'
for line in $a; do
echo "$i $line"
((i+=1))
done
echo $i # Output: 15
- This is of subshell issue in
while
loop: The reason$i
was not 16 at the end of script is because the variable$i
is being modified inside a subshell created by thewhile
loop. When the loop finishes, the changes to$i
are lost because they were made in the subshell, not in the parent shell.
Desired Output
1 project/xyz_abc_xyz/data_analysis/
2 project/xyz_abc_xyz/data_cleaning/
3 project/xyz_abc_xyz/data_migration/
4 project/xyz_abc_xyz/data_visualization/
5 project/xyz_abc_xyz/data_backup/
6 project/xyz_abc_xyz/data_archiving/
7 project/xyz_abc_xyz/data_processing/
8 project/xyz_abc_xyz/data_reporting/
9 project/xyz_abc_xyz/data_validation/
10 project/xyz_abc_xyz/data_integration/
11 project/xyz_abc_xyz/data_transformation/
12 project/xyz_abc_xyz/data_storage/
13 project/xyz_abc_xyz/data_retrieval/
14 project/xyz_abc_xyz/data_security/
15 project/xyz_abc_xyz/data_management/