Bash Scripting: Everything you need to know about Bash-shell programming
By: Uday Hiwarale
Tags: Bash Script, Unix, Bash, Programming, Terminal

Other important topics

The Bash scripting language has much more features and things to worry about. Here are some things to watch out for and learn.

1. Aliases in Bash

We can create an alias of a command using alias command. For example, to list files in greater details, we use ls -alh command. We can shorten it by defining ll alias like below.

~$ alias ll="ls -alh"

~$ ll ⥤

drwxr-xr-x 11 Uday.Hiwarale staff 352B 7 Sep 20:10 .

drwxr-xr-x 4 Uday.Hiwarale staff 128B 4 Sep 06:38 ..

-rw-r--r--@ 1 Uday.Hiwarale staff 6.0K 6 Sep 12:45 photo.txt

2. type command

type is a built-in shell command to see the type of a keyword or a command.

3. Executing multiple commands at once

Bash gives us the ability to run multiple commands at once. You might have used && to combine two command to run at once. Let’s see other variants.

# && : Run next command if previous is command successful

true && echo "1: Got printed"

false && echo "1: This won't get printed" # || : Run commands until one is successful

false || echo "2: Got printed"

true || echo "2: This won't get printed" # ; : Run all commands (just a statement separator)

false ; echo "2A: Got printed"

true ; echo "2B: Got printed" ⥤ 1: Got printed

⥤ 2: Got printed

⥤ 2A: Got printed

⥤ 2B: Got printed

We can pass STDOUT of command to STDIN of another command using | operator. For example, STDOUT of ls -alh * command can be passed to less command when we execute ls -alh * | less statement.

💡 for more information about less command, execute man less command in your terminal. man is also a command that provides manual of a command distributed in UNIX-like operating systems.

~$ ls -alh * | less ⥤

Applications:

total 16

drwx------@ 5 Uday staff 160B Aug 31 18:15 .

drwxr-xr-x+ 31 Uday staff 992B Sep 9 17:16 ..

-rw-r--r--@ 1 Uday staff 6.0K Aug 31 18:16 .DS_Store

-rw-r--r--@ 1 Uday staff 0B Aug 28 16:23 .localized

drwx------@ 11 Uday staff 352B Sep 4 23:03 Chrome Apps.localized Creative Cloud Files:

total 47936

drwxrwxr-x@ 14 Uday staff 448B Sep 8 00:46 .

drwxr-xr-x+ 31 Uday staff 992B Sep 9 17:16 ..

-rw-r--r--@ 1 Uday staff 0B Sep 8 00:43 Icon

-rw-r--r-- 1 Uday staff 707K Jul 11 00:44 post-1.png

-rw-r--r-- 1 Uday staff 1.6M Jul 11 00:44 post-2.ai

-rw-r--r-- 1 Uday staff 716K Jul 15 21:07 post-3-01.png

-rw-r--r-- 1 Uday staff 1.7M Jul 15 21:07 post-3.ai

-rw-r--r-- 1 Uday staff 756K Jul 20 23:28 post-4-01.png

-rw-r--r-- 1 Uday staff 1.7M Jul 20 23:28 post-4.ai

:▐

The main purpose of passing the output of ls -alh * to less is to prevent ls command to dump too much information to the terminal that terminal can not handle. Using the above command, we can feed the output of ls to less and less will let us read that output one or few lines at a time.

Basically, STDIN , STDERR and STDOUT are streams, which means they are like pipes through which data flows. They are always open and any program can pass data through it. Anybody like terminal shell listening to these streams will get the data whenever somebody sends through it.

| is called the pipe operator, because it connects two streams and passes the data from one stream to another. For example, it can connect STDOUT of program to STDIN of another program. So | in ls -alh * | less basically connecting STDOUT of ls to STDIN of less.

If a command starts a continuously running program like infinite while loop and that program pipes data to another command, the connection between STDOUT and STDIN remain open. This can be dangerous in multiple ways. For example, if the program receiving a continuous stream of data from another program can not consume fast enough, the pipe buffer will eventually be full.

Let’s create a simple infinitely running while loop that prints a random number, which means sends to STDOUT stream. But instead printing it to the terminal directly, we will redirect or pipe the output to less command which will read it one line at a time.

(while true; do sleep 1; echo $RANDOM; done) | less

The above program will never exit as while loop is continuously running. However, in this case, since the pipe operator connects the output stream of the left-side program to the input stream of less and it remains connected forever, whenever the left-side program sends data through the output stream, it ends up in input stream of less and less outputs it to STDOUT which is our terminal console.

If we want to write the output stream of a command to a file, we can use > operator AKA redirection operator. This will overwrite a file content if the file already exists, else a new file will be created.

echo "Hello" > random.txt

cat random.txt ⥤ Hello echo "World" > random.txt

cat random.txt ⥤ World

If you want to append the output of a command to a file, use >> instead.

echo "Hello " >> random.txt # adds a newline as well

cat random.txt ⥤ Hello echo "World" >> random.txt # adds a newline as well

cat random.txt ⥤

Hello

World

If you are writing the output of a command that starts an infinitely running program, the pipe between file’s STDIN and program’s STDOUT will remain open. Any data sent in STDIN will be appended continuously in the file.

We can also use < operator which is same as > but it redirects the content of a file to a command. A typical use case syntax will look like command < file.txt which means send data of file.txt into the command .

~$ cat < random.txt ⥤

Hello

World

So far, we have dealt with STDOUT , but if we need STDERR stream as well, there is &> and &>> syntax for it. It is explained very well in this answer.

4. Environment variables

A shell variable is a variable defined inside a Bash script. When you source an external Bash script, variables defined inside the external Bash script will be available inside the current Bash script. These are short-lived variables.

Environmental variables are also Bash variable but defined with export command. Let’s take a small example of defining such a variable.

# env-test.sh

export MY_IP='192.168.1.7'

echo "My IP is: $MY_IP" ------------------------------- ~$ bash env-test.sh

⥤ My IP is: 192.168.1.7

When we use export comand, Bash registers this variable and saves in the namespace of the current terminal session and sub-sessions (sub-processes) created by it. To see the list of all available environment variables, we use printenv command.

~$ printenv ⥤ TERM_PROGRAM=vscode

⥤ TERM=xterm-256color

...

⥤ HOME=/Users/Uday.Hiwarale

But from the above result, we can’t see our MY_IP variable. As said before, environment variables are only accessible inside the current session and sub-sessions created by it. Since bash command runs a script in a separate session, the environment variable will be created there and killed at the end.

# env-test.sh

export MY_IP='192.168.1.7'

printenv ------------------------------- ~$ bash env-test.sh

⥤ MY_IP=192.168.1.7

⥤ TERM_PROGRAM=vscode

⥤ TERM=xterm-256color

...

⥤ HOME=/Users/Uday.Hiwarale

As you can see from the above results, MY_IP environmental variable exists inside the session started by bash env-test.sh command.

The main purpose of the environment is to set global values for sub-processes (sub-sessions) created by the main Bash script. Let’s see an example.

# child.sh

echo "LOCAL_VAR inside child.sh: $LOCAL_VAR"

echo "MY_IP inside child.sh : $MY_IP" # main.sh

LOCAL_VAR="MAIN"

export MY_IP='192.168.1.7'

bash ./child.sh # starts a sub-session ------------------------------- ~$ bash main.sh

⥤ LOCAL_VAR inside child.sh:

⥤ MY_IP inside child.sh : 192.168.1.7

A sub-session or sub-process is little different from sub-shell we have seen before. Any code inside parentheses () runs in sub-shell. A sub-shell is also a separate process (with an exception in KSh) started by the main shell process but in contrast with sub-session, it is an identical copy of the main shell process. This article explains the differenrce between sub-shell and sub-process.

We can also pass an environmental variable to a process directly from the command which started it using below syntax. This way, our environment variables can be portable and we can avoid writing unnecessary boilerplate.

# main.sh

MY_IP='192.168.1.7' bash ./child.sh ------------------------------- ~$ bash main.sh

⥤ MY_IP inside child.sh : 192.168.1.7

If you need to set an environmental variable for all the process started by the current terminal session, users can directly execute export command. But once you open a new terminal, it won’t have that environmental variable. To make environmental variables accessible across all terminal sessions, export them from .bash_profile or any other startup script.

💡 It is recommended to write environment variables in all uppercase and shell variables in all lowercase, but my personal choice is to use all uppercase for the both kinds. If you want to check if an environmental variable exists in the terminal session, you can just echo it using echo $SOME_ENV_VAR command.

5. Code Block (statements block)

If we need to execute some code as a block, then we can put our code in {} curly braces. In contrast with () block which executes the code inside it in a sub-shell, code-block executes the code in the same shell, hence in the same shell process. Let’s see a quick example.

MAIN_VAR="main" {

sleep 1;

echo "code-block: $MAIN_VAR"

} ⥤ code-block: main

If we want to run some code as a block on a single line, we need to terminate the statements wit ; character (unlike a sub-shell).

MAIN_VAR="main" { sleep 1; echo "code-block: $MAIN_VAR"; }

( sleep 1; echo "sub-shell: $MAIN_VAR" ) ⥤ code-block: main

⥤ sub-shell: main

In the example, code-block and sub-shell both have access to MAIN_VAR because code-block runs in the same environment of the main shell while sub-shell might run in the different process but it has an identical copy of main process which also contains the variables from the main process.

The difference comes where we try to set or update a variable in the main process from a sub-shell. Here is a demonstration of that.

VAR_CODE_BLOCK="INIT"

VAR_SUB_SHELL="INIT" { VAR_CODE_BLOCK="MODIFIED"; echo "code-block: $VAR_CODE_BLOCK"; }

( VAR_SUB_SHELL="MODIFIED"; echo "sub-shell: $VAR_SUB_SHELL" ) echo "main/code-block: $VAR_CODE_BLOCK"

echo "main/sub-shell: $VAR_SUB_SHELL" ⥤ code-block: MODIFIED

⥤ sub-shell: MODIFIED

⥤ main/code-block: MODIFIED

⥤ main/sub-shell: INIT

From the above example, we can interpret that when we try to update or set a variable in the sub-shell, it will set a new variable in its own environment.

A good use case of code block would be to pipe (|) or redirect (>) the output of some statements as a whole.

{ echo -n "Hello"; sleep 1; echo " World!"; } > hello.txt ------------------------------- ~$ bash main.sh && cat hello.txt

⥤ Hello World!

6. Special characters

In many examples, we have seen that it is not safe to pass a variable as is to a command or use it inside an expression, because Bash expands it in series of arguments (or series of words).

There is no harm as long as our string contains plain words. The problem comes when a word means something different to Bash, like * . Such characters are called as special characters and they should be either escaped using \ like \* or enclosed inside quotes like "*" .