Bash Command Completion - Finding all the cats in your $PATH!
Tags: bash, linux, productivity, python
Published: 2018-11-15 11:50:01

---
title: Bash Command Completion - Finding all the cats in your $PATH!
published: true
description: How to search for programs
tags: bash,linux,productivity,python
---

Today, I was curious to find out **how many programs ending with `cat`** I had on my system. I remember using `cat`, `zcat` and wanted to know if there are similar programs. Pressing TAB after `cat` only gives programs which start with `cat`.

### 1. `apropos`

In my [last post](https://dev.to/rrampage/getting-help-in-the-terminal-4cmo), I had mentioned that apropos is a way to search for what programs are available on your system. The search string for `apropos` can be any regex. So, `apropos 'cat$'` should solve the problem. `cat$` means all words ending with 'cat'

The output has 
```
STAILQ_CONCAT (3)     - implementations of singly-linked lists, singly-linked tail queues, lists and tail queues
OPENSSL_strlcat (3ssl) - Memory allocation functions
..
bzcat (1)            - decompresses files to stdout
cat (1)              - concatenate files and print on the standard output
fc-cat (1)           - read font information cache files
gencat (1)           - Generate message catalog

```

Clearly, the top 2 do not look like programs. Why is `apropos` then returning them?
Let's have a look at the `apropos` manual

```bash
man apropos
# apropos - search the manual page names and descriptions
```
So `apropos` searches the `man` pages. And looks like there are `man` pages for other things and not just programs...

Digging deeper, let's try manual for the `man` pages!
```
man man
# The table below shows the section numbers of the manual followed by the types of pages they contain.
#       1   Executable programs or shell commands
#       2   System calls (functions provided by the kernel)
#       3   Library calls (functions within program libraries)
#       4   Special files (usually found in /dev)
#       5   File formats and conventions eg /etc/passwd
#       6   Games
#       7   Miscellaneous (including macro packages and conventions), e.g. man(7), groff(7)
#       8   System administration commands (usually only for root)
#       9   Kernel routines [Non standard]

```

Ok. We are interested in are executable programs i.e section 1 of the man pages. `apropos` has a way to limit which sections we search using`-s` flag.
**`apropos -s 1 'cat$'`** gives us all programs ending with name `cat` which have an entry in the man pages but it does not show us any programs which do not have a man page.

### 2. List all programs on your path

The way Bash knows which programs can be called directly by their name (e.g `ls`) and not by their full path (e.g `/usr/bin/ls`) is by looking at the `$PATH` environment variable.

** Listing all executable files on path **

Here's a small bash snippet which lists the executable files in PATH (let's call it `paths.sh`)
```bash
#!/bin/bash
# The directories in $PATH are separated by ":", so we split by it to get individual directories
for pdir in $(echo "$PATH" | tr ":" "\n")
do
    # We `find` all files in the directory which are executable and print the filename
    find "$pdir" -maxdepth 1 -executable -type f -printf "%f\n"
done

```

If you prefer Python, here's a small Python program for the same (let's call it `paths.py`)
```python
from itertools import chain
import os
path_dirs = os.environ['PATH'].split(':') # Split PATH by ':'
print(path_dirs)
all_files = chain(*(os.walk(p) for p in path_dirs)) # Iterable of all files in the directories contained in PATH
is_exec = lambda x : os.access(x, os.X_OK) # Function to check if a filename is executable
execs = chain(*([f for f in fs if is_exec(os.path.join(r,f))] for r,_,fs in all_files))
for x in execs:
    print(x)
```

Running either our Bash or Python scripts will give us the correct output!

```bash
sh paths.sh | grep 'cat$'
python3 paths.py | grep 'cat$'
```

### 3. Power of Bash Completion!
When I press TAB TAB after typing a letter, I get a list of suggestions. How does Bash do that? The [Bash manual](https://www.gnu.org/software/bash/manual/html_node/Programmable-Completion.html) says that it uses `complete` and `compgen` built-ins for suggesting completions.

`compgen` generates completions using a list of words (-W) or list of commands (-c). The latter is of particular interest to us. `compgen -c` prints every executable on our path and all shell built-ins and shell-functions.

`compgen --help` prints following message:
`compgen: compgen [-abcdefgjksuv] [-o option] [-A action] [-G globpat] [-W wordlist]  [-F function] [-C command] [-X filterpat] [-P prefix] [-S suffix] [word]
    Display possible completions depending on the options.`

The options stand for:
- a : aliases
- b : shell builtins
- **c : executable commands**
- d : directories in current directory
- e : **export** variables
- f : files in current directory
- g : groups in system
- j : pending jobs (in background / stopped)
- k : Bash **keywords**
- s : System services
- u : users
- v : All shell variables

So **`compgen -c | grep 'cat$'`** should give us every single executable ending with `cat`.

### Epilogue

Diving into this rabbit-hole has given me a better understanding of how Bash completion works, how apropos finds relevant programs and why man pages are organized into various sections.