Writing Bash Scripts that are not only Bash: Checking for Bashisms and testing with Dash
Tags: bash, tutorial, linux, devops
Published: 2021-02-21 20:52:25


Shells like [Bash] or [Zsh] are advanced and user-friendly, and include features beyond what a simpler POSIX-compliant shell might offer. You will do well to utilize the full features of your shell when writing scripts.

There are situations, however, when _portability_ should be a valued feature, allowing the script to run on a variety of shells.

Bash scripts are most portable when "bashisms" are avoided. Let's explore writing POSIX-compliant shell scripts that work on [Ash/Dash][dash] and other shells.

## A summary checklist

Here is a checklist I use to keep tabs on my own script writing. Does my script:

1. Have `#!/bin/sh` as the first ("[shebang]") line of the script, not `#!/usr/bin/bash` or other shell
2. Avoid double-bracket tests `[[ ]]` and instead use single-brackets `[ ]`
3. Use `printf` instead of `echo -e` when newlines `'\n'` need to be printed
4. Use no other `read` flag other than `-r`, as in `read -r`
5. Avoid Bash's convenience redirects: use `>myfile 2>&1` to redirect stdout and stderr to a file rather than `&>myfile`
6. Test accurately with [dash] or [posh: Policy-compliant Ordinary SHell][posh]
7. Only use standard flags and options with common utilities such as [sed], [grep], [cut], [test], [and others][posix-shell]
8. Avoid issues discovered by [shellcheck]

## An example

```bash
#!/bin/sh

read -p "Who would you like to greet? "

if [[ -z $REPLY ]] ; then
  recipient="$REPLY"
else
  recipient="World"
fi

echo -e "Hello\n$recipient\n"

```

You might save the following in the current working directory of your choice, as `example-noncompliant.sh`

In human language, the above script prompts for a greeting recipient, then sets the recipient to "World" if none was given, then greets the recipient on multiple lines.

The above works on Bash, but has issues on other shells. You may wish to run it once in Bash, just to feel good. Even in [Zsh], though, it may raise some complaints.

## Dash, a POSIX compliant shell

[Dash] is a derivative of the [Ash] (Almquist, named for the original creator) shell. It is meant to be [POSIX]-compliant.

Debian and Ubuntu come with dash installed. In fact, scripts invoked with `/bin/sh` will run with dash by default. On [Alpine], [Tiny Core Linux][tinycore], [OpenWRT], and other distros that use [BusyBox] by default, the standard shell is also dash (although labeled as `ash`). On Fedora, dash can be installed with `sudo dnf install dash`. Other distros may also include dash in their repositories.

Using [Docker] or [Podman], running dash is easy, as in this example:

```bash
docker run -it debian dash
```

You may also try my [POSIX playground container][article2], with a variety of tools, including [dash]. By default, it uses [posh: Policy-compliant Ordinary SHell][posh], which is slightly stricter than dash. It can be launched with:

```bash
docker run -it docker.io/bowmanjd/posix-playground
```

See [the article][article2] for a deeper explanation.

In all of the above, `podman` can replace `docker` without a problem.

## Testing the example

Can you try running the `example-noncompliant.sh` script above, but with dash, not Bash?

```bash
dash example-noncompliant.sh
```

Or, using Docker or Podman:

```bash
docker run -it -v "$(pwd):/work" debian dash /work/example-noncompliant.sh
```

The output is likely something resembling:

```
dash: 3: read: arg count
dash: 5: [[: not found
-e Hello
```

A few learning points can be derived from that output.

## Avoid double-bracket tests `[[ ]]`

The `[[` construct is a safe one if using Bash or another shell that supports bashisms. It has some convenient features, like regex matching using `=~`, and has less risks with string matching.

That said, you will generally not go wrong with the single bracket approach: `[ ]` (an alias for [`test`][test]). Always be sure to quote variables, but that is good advice anyway. If you need regular express matching, use [`grep`][grep].

Bottom line: not all shells support `[[`; use `[` instead.

## Use vanilla `read`

When using the `read` command to get input, here are a few suggestions:

1. Always specify the variable, rather than relying on Bash's default `$REPLY`
2. Use `read -r` and no other flags. Using `-r` prohibits the user from using backslash to escape characters, which can cause issues later. And no other flag is [supported by POSIX `read`][read].
3. Instead of specifying a prompt with `-p`, just use a `printf` call prior to the `read` command. Again, [POSIX `read`][read] does not support such a flag, plus the `-p` option means something different to [Zsh's `read`](http://zsh.sourceforge.net/Doc/Release/Shell-Builtin-Commands.html).

Given these rules, our script should not use `read -p "Who would you like to greet? "` but rather:

```bash
printf "Who would you like to greet? "
read -r recipient
```

## Use `printf` when newlines are at issue

The `echo` command works great when we know we want to output a simple string, followed by a newline.

However, if we have newlines in a string we want to print, or if printing without a trailing newline is desired, then `printf`, not `echo -e` will be our friend.

So, instead of `echo -e "Hello\n$recipient\n"` in our code above, this would be better:

```bash
printf "Hello\n%s\n\n" "$recipient"
```

Note the variable substitution going on with `%s` in the first string (the format string). Do not put shell variables like `$recipient` in the format string. This is the way.

## Refactoring the example

Given the above concerns, let's completely rewrite our greeting script:

```bash
#!/bin/sh

printf "Who would you like to greet? "
read -r recipient

if [ -z "$recipient" ] ; then
  recipient="World"
fi

printf "Hello\n%s\n\n" "$recipient"
```

You might save the above with the filename `example-posix.sh` or similar.

When you run it, does it behave the same as the noncompliant script? Hopefully not; try it out.

Satisfying.

## Finding the bashisms with `checkbashisms`

There is a tool embedded in the [Debian devscripts][devscripts] project, called [checkbashisms]. It is a simple but powerful [Perl script][checkbashisms-script] that ferrets out any bashisms in a shell script that begins with the `#!/bin/sh` [shebang] line.

On Debian and Ubuntu, it can be installed with `sudo apt install devscripts` and on Fedora with `sudo dnf install devscripts-checkbashisms` while Alpine is `sudo apk add checkbashisms`. Other distros may have something similar. You might also try installing Perl, then downloading and running [the checkbashisms Perl script][checkbashisms-script] itself.

What happens when you run it on `example-noncompliant.sh` or `example-posix.sh`? So telling...

## Pursuing best practices with `shellcheck`

My new favorite shell scripting helper is [Shellcheck]. You can paste your shell script online and check it there, or install shellcheck in the usual way (Debian, Ubuntu, Fedora, Alpine, Archlinux, and others have it readily available in the standard package repositories.)

It does raise the POSIX-compliance flag on any lines that need it, but many other issues are checked as well. Your code might run just fine, but have gotchas that need some attention. [Shellcheck] will help you there. I integrate it into my editor, so that I can lint while I type.

## Measure against the POSIX specification

Thankfully, the [POSIX.1-2017 standard][posix] is openly documented. Consider this: when discovering and testing options for a given tool like [`read`][read], [`grep`][grep], or [`sed`][sed], instead of going to the GNU pages, the distro man pages, or the Bash or Zsh docs, why not go to the POSIX spec itself? [The list of utilities and their options][posix-shell] is plainly explained.

In instances where you really need an enhancement provided by the extended tools, you can make that choice. With the POSIX spec in hand, it becomes an informed decision.

Often, I find that I don't need `sed -E` or `grep -E` as badly as I thought. A few extra escape characters, and I am there.

## Consider other languages and configuration tools

Sometimes, when the extended syntax provided by GNU utilities is warranted, it may be a sign that the right tool for the job isn't the shell and its compatriots at all.

If your system has [Python](https://www.python.org/), [Ruby](https://www.ruby-lang.org/), [NodeJS](https://nodejs.org/), or other favorite language, might that be a more robust, flexible, and consistent option? Even in circumstances (embedded systems) in which those runtimes would be too bulky, perhaps remote scripting from another machine is in order. For instance, one could use Python to SSH to the remote machine, gain the information necessary, perform some logic, then send the appropriate commands back, without Python being necessary on the target machine.

This is the reason such tools as [Ansible](https://www.ansible.com/), [Saltstack](https://saltproject.io/), [Chef](https://www.chef.io/), and [Puppet](https://puppet.com/) exist. These, too, can be quite bloated if the needs are simple. But they are unbeatable for flexibility and repeatability.

## Other resources

In my research for this article, I encountered some resources you may find at least as interesting as this one:

- The informative and well-written [A Brief POSIX Advocacy: Shell Script Portability][advocacy] by Arnaud Tomeï
- The [Autoconf portable shell guide](https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.70/autoconf.html#Portable-Shell)
- [Sven Mascheck's remarks on various Unix tools](https://www.in-ulm.de/~mascheck/various/)
- [How to make bash scripts work in dash][bashism] by Greg Wooledge

Please feel free to share your tips, questions, corrections in the comments!

[advocacy]: https://www.usenix.org/system/files/login/articles/login_spring16_09_tomei.pdf
[alpine]: https://alpinelinux.org/
[article2]: https://www.bowmanjd.com/posix-playground-container/
[ash]: https://en.wikipedia.org/wiki/Almquist_shell
[bash]: https://www.gnu.org/software/bash/
[bashconditional]: https://www.gnu.org/software/bash/manual/html_node/Bash-Conditional-Expressions.html#Bash-Conditional-Expressions
[bashism]: https://mywiki.wooledge.org/Bashism
[busybox]: https://busybox.net/
[checkbashisms-script]: https://github.com/Debian/devscripts/blob/master/scripts/checkbashisms.pl
[checkbashisms]: https://manpages.debian.org/unstable/devscripts/checkbashisms.1.en.html
[cut]: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/cut.html
[dash]: http://gondor.apana.org.au/~herbert/dash/
[devscripts]: https://github.com/Debian/devscripts
[docker]: https://www.docker.com/
[grep]: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/grep.html
[idempotency]: https://en.wikipedia.org/wiki/Idempotence
[openwrt]: https://openwrt.org/
[podman]: https://podman.io/
[posh]: https://manpages.debian.org/unstable/posh/posh.1.en.html
[posix-shell]: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/contents.html
[posix]: https://pubs.opengroup.org/onlinepubs/9699919799/
[read]: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/read.html
[sed]: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/sed.html
[shebang]: https://en.wikipedia.org/wiki/Shebang_(Unix)
[shell]: https://en.wikipedia.org/wiki/Shell_%28computing%29
[shellcheck]: https://www.shellcheck.net/
[test]: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/test.html
[tinycore]: http://tinycorelinux.net/
[zsh]: http://zsh.sourceforge.net/