How to Split a Multi-Line String by Newline

The aim of this pageđź“ť is to explain how to split a string by newlines and loop over each line in Bash based on the particular example of processing a multi-line string.

Pavol Kutaj
2 min readAug 26, 2024

Use while loop to read each line.

  • Example string required for data wrangling, say to prepend it with an increasing integer starting from 1
a="project/xyz_abc_xyz/data_analysis/
project/xyz_abc_xyz/data_cleaning/
project/xyz_abc_xyz/data_migration/
project/xyz_abc_xyz/data_visualization/
project/xyz_abc_xyz/data_backup/
project/xyz_abc_xyz/data_archiving/
project/xyz_abc_xyz/data_processing/
project/xyz_abc_xyz/data_reporting/
project/xyz_abc_xyz/data_validation/
project/xyz_abc_xyz/data_integration/
project/xyz_abc_xyz/data_transformation/
project/xyz_abc_xyz/data_storage/
project/xyz_abc_xyz/data_retrieval/
project/xyz_abc_xyz/data_security/
project/xyz_abc_xyz/data_management/"
  • Use echo and while loop:
echo "$a" | while IFS= read -r line; do
echo "Processing: $line"
# Add your processing logic here
done
  • Use for loop with IFS set to newline:
IFS=$'\n'
for line in $a; do
echo "Processing: $line"
# Add your processing logic here
done
  • The issue with variable persistence in while loop:
i=1
echo "$a" | while IFS= read -r line; do
echo "$i $line"
((i+=1))
done
echo $i # Output: 1 !
  • Solution using for loop:
i=1
IFS=$'\n'
for line in $a; do
echo "$i $line"
((i+=1))
done
echo $i # Output: 15
  • This is of subshell issue in while loop: The reason $i was not 16 at the end of script is because the variable $i is being modified inside a subshell created by the while loop. When the loop finishes, the changes to $i are lost because they were made in the subshell, not in the parent shell.

Desired Output

1 project/xyz_abc_xyz/data_analysis/
2 project/xyz_abc_xyz/data_cleaning/
3 project/xyz_abc_xyz/data_migration/
4 project/xyz_abc_xyz/data_visualization/
5 project/xyz_abc_xyz/data_backup/
6 project/xyz_abc_xyz/data_archiving/
7 project/xyz_abc_xyz/data_processing/
8 project/xyz_abc_xyz/data_reporting/
9 project/xyz_abc_xyz/data_validation/
10 project/xyz_abc_xyz/data_integration/
11 project/xyz_abc_xyz/data_transformation/
12 project/xyz_abc_xyz/data_storage/
13 project/xyz_abc_xyz/data_retrieval/
14 project/xyz_abc_xyz/data_security/
15 project/xyz_abc_xyz/data_management/

--

--

Pavol Kutaj

Today I Learnt | Infrastructure Support Engineer at snowplow.io with a passion for cloud infrastructure/terraform/python/docs. More at https://pavol.kutaj.com