Bash Execution Tips for Shell Jockeys and Script Fabricators
Tags: linux, devops, bash, tutorial
Published: 2021-02-09 18:24:20

Have you ever wanted to execute one command if another failed? Or one command only if the first one succeeded? What about background execution?

[Bash] and other popular [shells][shell] such as [Zsh] and [Ash/Dash][dash] have some useful but sometimes confusing operators for command execution.

What is the difference between `&&`, `&`, `||` and `;`?

A very brief cheatsheet:

- Use `&&` to execute one command only when the previous one succeeds.
- Use `||` to execute one command only when the previous one fails.
- Combine the above for conditional branching.
- Use `;` to join two commands when you want the second to execute _no matter_ the result of the first one.
- Use `&` to run the first job in the background while the next executes. Follow all with `wait` for a clean return to the command prompt

## Execute if previous command succeeds: `&&`

When running one command, sometimes it is desirable for another command to execute if and only if the first one was successful.

Let's say we want to create a user's home directory if and only if that user already exists. We can test for the user's existence with `id`, then, if successful, create the home directory, then, if _that_ is successful, write a `.bashrc` file:

```sh
id fredflintstone && mkdir /home/fredflintstone && echo 'echo "Welcome, $USER"' > /home/fredflintstone/.bashrc
```

Note the use of the `&&`, a logical "and", for conditional execution. If _this_, then _that_.

The `&&` operator should not be confused with the background execution operator `&` noted below.

## Execute if previous command fails: `||`

Sometimes, the opposite of the above is desired: execute the next command if the previous one _failed_.

For instance, perhaps we want to add a DNS server to `/etc/resolv.conf` only if it doesn't already exist. We can use grep to test if the server name already exists, then write back to the file if it wasn't already there:

```sh
grep 1.1.1.1 /etc/resolv.conf || echo "nameserver 1.1.1.1" | sudo tee -a /etc/resolv.conf
```

Note the use of the `||`, a logical "or", for conditional execution. If not _this_, then _that_.

Also note the pipe `|` operator. It should not be confused with the `||` operator. The pipe `|` means "send the output of this command to the input of the next." In this case, the `echo` command pipes output to the `tee` command, which appends (due to the `-a` option) the input to `/etc/resolv.conf`.

## Branching logic with both `&&` and `||`

A pleasant combination of the above is indeed possible. Imagine that you want to execute a command, then, if it succeeds, execute one command, but if it fails, execute a different command. Some creative chaining is possible, as in this example:

```sh
id fredflintstone && mkdir -p /home/fredflintstone || id bettyrubble && mkdir /home/bettyrubble
```

In the above, if one user does not exist, another will be tried. However, it should be pointed out that this is not equivalent to an if/then/else statement. If the 2nd command `mkdir -p /home/fredflintsone` would fail (not likely, with the `-p` flag), `id bettyrubble` would still run. In other words, when executing `true && false || true` all three commands run, because the failure of the 2nd command triggers the 3rd.

Sometimes, I use this method to explicitly set human-readable variables in a script, so that later readers (i.e. me) will easily understand what is going on:

```sh
sudo passwd -S $USER && PWD_IS_SET=true || PWD_IS_SET=false
```

In effect, the above "caches" a test so that the script can later test for the value of `$PWD_IS_SET` numerous times, in a memorable and readable way.

## Unconditional execution with `;`

To mash a couple commands together in one line, use the semicolon. In human language, a semicolon says, "Do this, wait until it completes, then do that." For instance:

```sh
echo Hello ; echo World
```

The execution is _unconditional_; no matter what the first command does, the second will also execute afterward:

```sh
cat filename_that_does_not_exist ; echo Continue anyway
```

## Background execution with `&`

Try this:

```sh
echo Hello & echo World
```

Yes, both commands executed, but the behavior may well be quite different than the use of `;` above. The above two commands were executed in _parallel_. In other words, at the same time. There is no guarantee regarding which will complete first. In fact using just `&` at the end of a long running command will allow it to run in the background indefinitely.

The main point in the context of this article: `&` is not `&&` nor is it `;` (the order and conditions for execution are different for each).

## Testing teaser

A thorough exploration of the builtin shell command `test` is better left for a future article. That said, let's at least dabble a bit, as the concept is quite applicable to conditional execution.

> Note: in [POSIX]-derived shells such as Bash, Zsh, and Ash/Dash, the `test` command and the `[` command are the same command, with the exception that when `[` is used, it should end with a `]`. While `[` works well in a pattern like "`if [ -d some_directory ]; then`" for multi-line readability (the last line should be "`fi`" to end the if statement), for succinct one-liners I prefer "`test -d some_directory`" and the like.

A quick cheatsheet with some commonly used tests using the `test` command:

- `test -d some_directory` will be true if a directory exists
- `test -r some_file` will be true if a regular file is readable
- `test -n "$SOME_STRING"` will be true if a string (such as a variable) is non-empty
- `test -z "$SOME_NONEXISTENT_STRING"` will be true if a string is empty

See [the POSIX spec for test][test] for many more options. You might also browse [Bash Conditional Expressions][bashconditional] or [Zsh Conditional Expressions][zshconditional] for shell-specific docs. When possible, I try to write shell scripts in POSIX-compliant ways for portability (scripts that work across a variety of shells). That said, sometimes you may prefer to use the full power of your shell's specific features. Browse the relevant docs to consider your options.

The above can be very useful for conditional execution. Something like this works well:

```sh
test -r /etc/resolv.conf || echo "nameserver 1.1.1.1" | sudo tee /etc/resolv.conf
```

In other words, if `/etc/resolv.conf` does not exist, create it with the appropriate contents. But if it already exists, do nothing.

## The possibility of idempotence (alternate title: put a little Dev in your Ops)

Perhaps you can see a clear use case here for system configuration. If you have ever used [Ansible](https://www.ansible.com/) and similar configuration tools, you may note that it is desirable for every task to be [_idempotent_][idempotency]; in other words, even when run more than once, the desired result is the same.

I believe it is good for shell scripts to be as idempotent as possible, when they are intended to configure a system in a certain state. The above examples that reference `/etc/resolv.conf` do this well, so I will reference them again here:

```sh
test -f /etc/resolv.conf || echo "nameserver 1.1.1.1" | sudo tee /etc/resolv.conf
grep 1.1.1.1 /etc/resolv.conf || echo "nameserver 1.1.1.1" | sudo tee -a /etc/resolv.conf
```

These two lines are very similar, and only one is needed, depending on the desired outcome. The first is a great example of creating a new file with the desired initial state. It uses `test -f` to first determine if the file exists. Sure, you could always overwrite the file and ensure state that way, but that would update the file stats, such as timestamp, needlessly. In addition, this example may be useful on configuration files with a desired _initial_ state, when future modifications are expected and should not be altered.

The second is instead an example of ensuring that a certain _line_ exists in a file. It uses `grep` to test for a word or line in the file, then adds the line if and only if it isn't already there.

These methods achieve [idempotency] with the combination of testing and conditional execution. A worthy goal.

## One-liners and robust scripts

The examples and concepts in this article are equally at home in a terminal window or in a substantial configuration script. Shell execution logic should be your friend. Feel free to post examples and tips in the comments below.

[bash]: https://www.gnu.org/software/bash/
[bashconditional]: https://www.gnu.org/software/bash/manual/html_node/Bash-Conditional-Expressions.html#Bash-Conditional-Expressions
[dash]: http://gondor.apana.org.au/~herbert/dash/
[idempotency]: https://en.wikipedia.org/wiki/Idempotence
[posix]: https://pubs.opengroup.org/onlinepubs/9699919799/
[shell]: https://en.wikipedia.org/wiki/Shell_%28computing%29
[test]: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/test.html
[zsh]: http://zsh.sourceforge.net/
[zshconditional]: http://zsh.sourceforge.net/Doc/Release/Conditional-Expressions.html