A Senior Dev's List of Key Unix Utilities, Explained by a Junior Dev
Published: 2020-08-07 19:40:27

If you are a developer working on a Mac, chances are you've used a Unix utility or two. At least to `cd` around to different directories or `touch` a new file. But chances are you haven't used the vast majority of Unix programs you have on your machine - I have 1325 on mine! Luckily, there's no need to sit down and memorize them all. What is important is to recognize the power of the Unix command line, and to know a few commands that will make your life in the terminal much easier. Below is a list of those commands. But first, a couple notes about using Unix commands in general. 

### General Rules of Unix Utilities
* Commands are lowercase
* Options modify the command. They are often single letters added in short form with a single dash `-`. Some have long form options added with a double dash `--`.
* Options can be combined. For example, adding the `-a` and `-l` options to `ls` can be written as `ls -a -l` or `ls -al`
* Commands can take arguments, often filenames
* Options come before filenames
* There must be spaces between commands, options, and filenames

### Unix Utilities to Know by Heart

#### ls
Lists all your files in the current directory. There are a few important option flags to know as well. 
* `-a` will also show all the hidden files in a directory (those that start with a `.`). 
* `-l` will list the files in long format, showing permissions, size, and modified date
* `-lh` lists in long format with readable file size (eg 1K, 23M)
* `-s` lists the allocated size of each file in blocks

#### cp *source destination*
Copies the source to the destination. Source can be one or more files or directories. Destination is the name of the directory to copy to or the name of the new file. When copying directories, the `-r` (or `-R`) flag must be used. This copies recursively, and will ensure that the contents of a folder is also copied.

#### cd
Change directory. `cd` with no arguments will bring you to your home directory, which is represented by `~`. You can also pass a pathname or make smaller "steps" with `cd ..`, which will move to your parent directory. `cd` will take relative or absolute pathnames, which can be built with `~` (home directory), `..` (parent directory), `.` (current directory) and `/` within directories. Note that if you are in a directory with a subfolder `test`, `cd test` is equivalent to `cd ./test` and both will move you into the test folder.

#### mv *source destination*
Moves (or renames) source to destination. If the destination is an existing directory, source(s) will be moved there. 

#### rm *source*
Removes files or directories. Similarly to `cp`, to remove directories the `-r` or `-R` flag must be used which will recursively removed the contents of the directory.

#### mkdir *directory*
Create a new directory. The `-p` flag is used to create parent directories if they don't already exist. For example, if I am in a directory with a subdirectory `test` and want to create a director `example` within `test`, I could run `mkdir -p test/example`. Since test already exists, a new folder `example` would be created in `test`. However, if `test` didn't exist, the command would simply create `test` and then create `example` within test. Either way there are no errors.

#### grep *patterns file*
Searches for patterns in each file, and prints each line that matches. This is a really useful command for finding other commands you've used if they are in your logs. The `-r` flag reads files in subdirectories recursively. `-n` adds to the output the line number in the file where the match occurred. 

#### cat *filename*
Used to view the contents of a file. With an argument of a filename, `cat` will simply print the content of that file. Other options can be added to view only parts of the file or to write contents to a new file.

#### echo
Displays a line of text. Most often used in scripts and other commands where you need to insert text.

#### head *filename*
Output the first part of the file. By default, `head` prints the first 10 lines. Useful when you only need to see the beginning of a file with lots of content.

#### tail *filename*
Output the last part of the file. Like `head` but for the end of the file.

#### less *filename*
Opens a file in the terminal and allows forward and backward navigation within the file. In this mode, patterns can be searched for with `/pattern` and using `n` and `N` to move backwards and forwards between matches. There are a bunch more navigation options to use as well. `q` will exit navigation.

### Unix Utilities to Know About

#### find
Searches for files based on some user specified criteria, and returns a list of the found files. The search criteria could be pattern matching, a file type, or file name to name a few. For example, `find . -name 'test*'` would search the current directory for filenames that started with 'test', with `*` being a placeholder for anything that might follow. 

#### curl
A tool to transfer data to or from a server, using any of its supported protocols (FTP or HTTP for example). There are whole bunch of options for using `curl` but know that it exists so you can search for the specifics when you need it! `wget` is a similar command with a couple differences, mainly its ability to download recursively.

#### vim *filename*
Opens a file with the vim text editor. Using vim is a whole other blog post, but it is a powerful tool that many developers swear by.

#### sed
Used to perform text transformations on an input stream (a file or input from a pipeline). It can be used to search for strings in a file, update content, and replace content.

#### awk
Another big topic, `awk` can run programs written in the AWK language that are used to transform text in a file. `awk` can pattern match and perform operations on matched lines, useful for file formatting.

#### pipes
Piping is a process of sending the output of one command as the input to another. The syntax for a pipe is `|` and information flows from left to right. So, to pipe the output of `ls` into `less` for example, you could write `ls | less`, and enter the `less` navigation with all the filenames in the directory - the output of `ls`. 

#### xargs
Converts standard input to arguments. Useful with commands that only accept arguments and do not read standard input such as `echo` and `rm`. With piping, input can be piped to `xargs` which will execute a given command for each input. A common use of this is with the `find` command. Results of `find` can be piped to `xargs` to run an operation on each result, such as `rm`. For example, to remove any file that begin with `test`, I could write `find . -name 'test*' | xargs rm`.

### And there's so much more

The internet is your friend when it comes to learning the details of Unix utilities! Don't be afraid to play around in your terminal either and see what you can do. Hopefully this is a good start for some useful commands to know about and the basics of how to use them. Thanks for reading!