How to Split a Multi-Line String by Newline

The aim of this page📝 is to explain how to split a string by newlines and loop over each line in Bash based on the particular example of processing a multi-line string.

Use while loop to read each line.

• Example string required for data wrangling, say to prepend it with an increasing integer starting from 1

a="project/xyz_abc_xyz/data_analysis/
project/xyz_abc_xyz/data_cleaning/
project/xyz_abc_xyz/data_migration/
project/xyz_abc_xyz/data_visualization/
project/xyz_abc_xyz/data_backup/
project/xyz_abc_xyz/data_archiving/
project/xyz_abc_xyz/data_processing/
project/xyz_abc_xyz/data_reporting/
project/xyz_abc_xyz/data_validation/
project/xyz_abc_xyz/data_integration/
project/xyz_abc_xyz/data_transformation/
project/xyz_abc_xyz/data_storage/
project/xyz_abc_xyz/data_retrieval/
project/xyz_abc_xyz/data_security/
project/xyz_abc_xyz/data_management/"

• Use echo and while loop:

echo "$a" | while IFS= read -r line; do
    echo "Processing: $line"
    # Add your processing logic here
done

• Use for loop with IFS set to newline:

IFS=$'\n'
for line in $a; do
    echo "Processing: $line"
    # Add your processing logic here
done

• The issue with variable persistence in while loop:

i=1
echo "$a" | while IFS= read -r line; do
    echo "$i $line"
    ((i+=1))
done
echo $i  # Output: 1 !

• Solution using for loop:

i=1
IFS=$'\n'
for line in $a; do
    echo "$i $line"
    ((i+=1))
done
echo $i  # Output: 15

• This is of subshell issue in while loop: The reason $i was not 16 at the end of script is because the variable $i is being modified inside a subshell created by the while loop. When the loop finishes, the changes to $i are lost because they were made in the subshell, not in the parent shell.

Desired Output

1 project/xyz_abc_xyz/data_analysis/
2 project/xyz_abc_xyz/data_cleaning/
3 project/xyz_abc_xyz/data_migration/
4 project/xyz_abc_xyz/data_visualization/
5 project/xyz_abc_xyz/data_backup/
6 project/xyz_abc_xyz/data_archiving/
7 project/xyz_abc_xyz/data_processing/
8 project/xyz_abc_xyz/data_reporting/
9 project/xyz_abc_xyz/data_validation/
10 project/xyz_abc_xyz/data_integration/
11 project/xyz_abc_xyz/data_transformation/
12 project/xyz_abc_xyz/data_storage/
13 project/xyz_abc_xyz/data_retrieval/
14 project/xyz_abc_xyz/data_security/
15 project/xyz_abc_xyz/data_management/