Explaining Process Substitution in Bash Using diff and icdiff
The aim of this pageđź“ť is to explain process substitution in Bash, focusing on its usage if the built-in diff and custom icdiff commands. I just need to quickly compare a couple of JSONs pulled from Consul. For this, I am assigning them to vars and using process substitution and diff and icdiff commands.
2 min readFeb 27, 2024
- I need process substitution in Bash because it treats command output as file input — and my tools require files as input
- It is denoted by
<(command)syntax. - As said, process substitution is useful for commands that expect file input but need to work with command output.
diff <(command1) <(command2)compares the output ofcommand1andcommand2- Commonly used with commands like
diff,grep,sort, etc. - Provides a way to work with dynamic or command output as if it were a file.
- Offers a convenient way to compare, process, or manipulate command output.
- Process substitution creates named pipes for command output.
default diff
# Example using diff with process substitution
diff <(echo $prod1) <(echo $qa1)
1c1
< customer_qa1/...:[{"type":"spEnrichedFilter"...}]
---
> customer_prod1/...:[{"type":"spEnrichedFilter"...}]- In the context of the
diffcommand output,1c1indicates a change between the first file (in this case, the output ofecho $prod1) and the second file (the output ofecho $qa1). - The
1before thecindicates the line number in the first file where the change is located. - The
1after thecindicates the line number in the second file where the change is located. - So,
1c1means that there is a change between line 1 of the first file and line 1 of the second file.
custom icdiff
- the custom
diffis not very helpful as I have a couple of very long oneliners there so1c1is useless — as I only have 1 line… - so, I turned to icdiff (for more alteratives, see https://news.ycombinator.com/item?id=8712417)
- my aim is to highlight the substring that’s different in the variables assigned to
$prod1and$qa1
icdiffprints out the file descriptors- the values are pulled from Hashicorp Consul there they are in the form of JSON object defining a filter for stream replication using Snowplow’s Snowbridge
- notably, in the output provided,
/dev/fd/62and/dev/fd/61are special paths in Unix-like systems that represent file descriptors. /dev/fd/62: This path represents the file descriptor 62./dev/fd/61: This path represents the file descriptor 61.- When you use process substitution in bash (e.g.,
<(...)) to create temporary files for command substitution, bash creates these temporary files in the/dev/fddirectory and assigns file descriptors to them. - These file descriptors are then used as arguments to the commands being executed.
