Everything is a File
Tags: unix, linux, history, operatingsystems
Published: 2019-01-02 21:37:19

One of the defining features of UNIX and UNIX-like operating systems is that "everything is a file" [[1]].* Documents, directories, links, input and output devices and so on, are all just sinks for or sources of streams of bytes from the point of view of the OS [[2]]. You can verify that a directory is just a special kind of file with the command:

```bash
$ view /etc
```

...which will show the contents of the `/etc` file (directory). Note that `view` is shorthand for running `vi` in read-only mode, `vi -R` [[7]], so you can exit it by typing `:q!`. Be extremely careful when opening directories as files -- you can cause serious damage to your system if you accidentally edit something you shouldn't have. On a Linux file system, the `/dev` directory contains files that represent devices, `/proc` contains special files that represent process and system information, and so on:

```
$ ls -l /dev/std*
lrwxrwxrwx 1 root 15 Nov  3  2017 /dev/stderr -> /proc/self/fd/2
lrwxrwxrwx 1 root 15 Nov  3  2017 /dev/stdin -> /proc/self/fd/0
lrwxrwxrwx 1 root 15 Nov  3  2017 /dev/stdout -> /proc/self/fd/1

$ cat /proc/uptime  # shows current system uptime
36710671.21 1127406622.14

$ cat /dev/random  # returns randomly-generated bytes
{IGnh▒I侨▒▒Ұ>j▒L▒▒%▒=▒▒U@▒Q▒2▒;▒l▒q▒$▒r▒1▒U񝾎...
```

Other common `/dev` files include `/dev/zero` which produces a constant stream of zeros, and `/dev/null`, which accepts all input and does nothing with it (think of it like a rubbish bin) [[8]]. Note that "files" can show the same information each time you open them, or they may be constantly changing -- reflecting the current state of the system, or displaying the results of some constantly-running process.

## But _why_ is everything a file in UNIX?

As Dennis Ritchie and Ken Thompson outlined in [their 1974 Communications of the ACM paper, "The UNIX Time-Sharing System"](https://people.eecs.berkeley.edu/~brewer/cs262/unix.pdf), there are three main advantages to the "everything is a file" approach:

1. file and device I/O are as similar as possible
1. file and device names have the same syntax and meaning, so that a program expecting a file name as a parameter can be passed a device name
1. special files [`/dev/random`, etc.] are subject to the same protection mechanism as regular files

The advantages of this approach are illustrated in [this 1982 Bell Labs video](https://youtu.be/tc4ROCJYbm0?t=330), in which Brian Kernighan, creator of the AWK programming language and co-creator of UNIX, describes how programs can be pipelined to reduce unnecessary code repetition (through modularisation) and increase flexibility (jump to 05:30):

{% youtube tc4ROCJYbm0 %}

```
$ makewords sentence | lowercase | sort | unique | mismatch
```

The code Kernighan types is reproduced above. In it, `makewords` splits the text file `sentence` into words (separated by whitespace characters) and returns one word per line. `makewords` was passed a file as input and its output would normally be sent to the terminal, but we've _piped_ it (using `|`) as input to the next method, `lowercase`. `lowercase` treats each line as a separate argument, converting all uppercase characters to lowercase, and then pipes its output to `sort`, which sorts the list of words alphabetically. `sort` pipes its output to `unique`, which removes duplicate words from the list, and sends its output to `mismatch`, which checks the list of unique, all-lowercase words against a dictionary file. Any misspelled words (words not appearing in the dictionary) are then printed to the terminal by default. By connecting these five separate functions, we've easily created a brand new function which spell-checks a file.

Note that input can come from regular files, but also special files like disks and other input devices like the terminal itself. Output can be sent to the terminal, or to other programs, or written to the disk as files. This ability to pipeline together functions, treating files and disks and special I/O devices identically, greatly increases the power and flexibility of UNIX relative to other systems which treat these things differently from one another.

---

\* This is more correctly written as "everything is a file descriptor or a process" [[3]]. (File descriptors are also sometimes called "handles".) File descriptors are simply non-negative indices which refer to files, directories, input and output devices, etc. [[5]] The file descriptors of `stdin`, `stdout`, and `stderr` are 0, 1, and 2, respectively. This is why, when we want to suppress error output, we redirect it (with the `>` command [[4]]) to `/dev/null` with

```bash
$ command 2>/dev/null
```

We can send all output (`stdout` and `stderr`) of a particular command as input to another command with `2>&1 |`, or, more simply, '|&' [[6]]:

```bash
$ command1 2>&1 | command2
$ command1 |& command2
```

The OS maps file descriptors to files (the actual locations of the bytes on disk) using the _inode_ (index node) [[10]].

Processes [[9]] are separate from file descriptors. A _process_ is an instance of a program which is currently being executed. A process contains an _image_ (read-only copy) of the code to be executed, some memory and heap space to use during execution, relevant file descriptors, and so on. Processes also have their own separate indexing system (process ids, or `pid`s). 

---

### Related:

[Introduction to Linux (2008), Machtelt Garrels](ftp://ftp.wayne.edu/ldp/en/Intro-Linux/Intro-Linux.pdf)

[An Introduction to UNIX Processes (2014), Brian Storti](https://www.brianstorti.com/an_introduction_to_unix_processes/)

[Ghosts of UNIX Past: A Historical Search for Design Patterns (2010), Neil Brown](https://lwn.net/Articles/411845/)

[unix-history-repo (continuous UNIX commit history from 1970 until today)](https://github.com/dspinellis/unix-history-repo)

[1]: https://en.wikipedia.org/wiki/Everything_is_a_file
[2]: https://www.computerhope.com/jargon/f/file-descriptor.htm
[3]: https://yarchive.net/comp/linux/everything_is_file.html
[4]: https://www.computerhope.com/jargon/r/redirect.htm
[5]: https://www.computerhope.com/jargon/f/file-descriptor.htm
[6]: https://www.computerhope.com/unix/ubash.htm#pipelines
[7]: https://www.lifewire.com/view-linux-command-4093565
[8]: https://www.howtogeek.com/117939/htg-explains-what-everything-is-a-file-means-on-linux
[9]: https://www.tecmint.com/linux-process-management
[10]: https://en.wikipedia.org/wiki/Inode