Introduction

Hello, and welcome back to CS631 "Advanced Programming in the UNIX Environment". In our last segment, we looked at creating links, both hard links and symbolic links. We discussed how hard links - file names - are merely entries in a directory, so now let's take a look at how we create directories.

mkdir(2)

It's really quite straightforward: you simply call mkdir(2), which will create a new directory, which is empty, containing only the necessary entries '.' and '..'. The permissions on the new directory are similar as when we create a new file, as specified by the mode_t argument, but modified by the process's current umask, as we illustrated in a previous video segment. Similarly, ownership of the new directory follows the same semantics as we also previously discussed, and based on the version of Unix in question.

rmdir(2)

Likewise, removing a directory is not very complicated either: call rmdir(2). rmdir(2) removes the given directory if it is empty; that is, it only contains '.' and '..'. If so, then it is removed and the st_nlink count is decremented. If that link count is now zero - or rather, will be zero after this call - and no other process has the directory open, then the directory is removed.

Removing directories examples

Now with directories, there's also the issue of a process having it open as a current working directory, and removing such a directory can lead to confusing situations. Here, we create a directory and change into it, then, from another terminal, we yank the directory out under our feet. Which works just fine, since the directory is empty. But our existing process still thinks that /tmp/dir is its current working directory, but if we try to list its contents -- that is, we try to open it and iterate over the entries -- that fails, and creating a new file in the directory can't work either, since the directory doesn't really exist anymore. Now if we recreate the directory, we won't magically be back in it; we'll have created a brand new directory and trying to access the entries can't work either, since '..', for example, is still in the directory, which doesn't exist. An absolute path, however, does work and all operations now behave as expected.

Reading directories

Reading directories is something that should look familiar from our simple ls(1) clone from our very first lecture: first, we call opendir(2) to open a new handle on the directory, then we iterate over the entries found in the directory by calling readdir(2) repeatedly, finally calling closedir(2) here when we're done. The system calls look like so. One thing to note here is that the order in which we get entries from the directory is opaque to us. The entries are not sorted in any way, or if they are, in a file system implementation dependent way that we cannot rely on. Okay, so opening a directory and listing its contents requires read permissions on the directory, but accessing any files inside the directory will, as we discussed in a previous segment, require exec or search permissions on the directory. Reading a directory should always be done using readdir(2) or getdents(2), since the implementation of the directory entries is file system dependent. The structures involved are documented in the dirent(3) manual page. When you open the directory you get back a DIR handle, which represents the directory as a stream, meaning the entries are returned to you in an ordered fashion, but you must not make the assumption that this happens in an order you can predict, such as alphabetical or by directory entry creation; the ordering inside the directory is opaque to you. Whether or not an open directory counts towards the open file handle or file descriptor limit is not something you can rely on. The implementations and standards are very unclear or variable here; see the COMPATIBILITY section of the opendir(2) manual page for more information. Finally, while you can perform file system hierarchy traversal with the careful sequence of opendir(2), readdir(2), and closedir(2), this gets very hairy real quick. Instead, I recommend that you look at the fts(3) library functions, especially for your 'ls' midterm project. As you might have guessed, the fts(3) library functions do in fact call the opendir(2)/readdir(2) syscalls underneath, but provide a lot of additional convenience. These file hierarchy traversal functions are not guaranteed to be available in all Unix versions, but fortunately for you they are available on your target platforms, so please give the manual pages a read and use that to handle recursive listing of directories.

Directory permission examples

Here, let's take a quick look at some permission edge cases when handling directories. Here's a file inside a directory. We remove execute permissions from the directory, but we are still able to list its contents. That is, opening the directory and calling readdir(2) works without exec permissions. However, accessing a file inside of the directory fails, since we do not have permissions to exec or search the directory. If we change the permissions to allow exec and remove read permissions, then we can access the file inside the directory, despite not being able to read the directory, but listing the directory contents fails. Perhaps unexpectedly, removing the directory fails, since we are unable to open it to see if there are any files in it, and if we flip the permissions, we can open the directory, see that there's a file in it, but we can't remove that file because we can't exec the directory; since we can't remove the file, the directory won't be empty, so we can't remove it. Weird, huh? In other words, to recursively remove a directory, we need both read and exec permissions.

getcwd(2)

Okay, we've already seen that we may have an open file handle on the directory after we changed into it, which introduces the concept of a current working directory, which a bit earlier in the semester we had mentioned every process has. So to get the current working directory, you can call the getcwd(2) syscall. This is done by, for example, the pwd(1) command as well and also by built into most shells a special variable.

chdir(2)

We also just saw how to change the current working directory via the 'cd' command. Now how does that work? To change the current working directory of the current process, you call chdir(2). As we saw earlier, you need to have exec permissions on the directory in question, otherwise you can change into it. Note, though, that the current working directory is set on a per-process basis. That is, if we recall our simple shell from lecture one, when we execute commands in the shell, we generally fork a new process, then exec the binary, and then return. But this has some interesting implications on commands like 'cd'. Here, let's implement 'cd'. Not particularly difficult, right? Let's see if it works. Okay, changing the current working directory into /tmp seems to work. Let's confirm with our pwd(1) and then... wait, what's that? Why are we still in the directory we started out? Let's try something else: '..' Seems to work. Except... it didn't. But we are able to change directories when running 'cd', aren't we? Here, cd /tmp - done, no problem. So what's going on here? Well, as explained a minute ago, when we run our program 'a.out', we begin in our current working directory - /home/jschauma/04. The shell will fork a new process; the new process will have a current working directory of /home/jschauma/04. Then that process calls chdir(2) and the new current working directory will be /tmp. The a.out program exits, and our parent process, which never changed its current working directory, prints our command prompt again, where we then run pwd(1), which reports the current working directory. As I said, chdir(2) can only influence the current process, not a parent process. But then why does 'cd' work? Wel,l there isn't actually a 'cd' executable; rather, the 'cd' command is built into the shell, meaning that the shell does not fork a new process to run it. It is called within the current shell process. In fact, 'cd' has to be a shell built-in for this to work. But now it gets weird: let's look at macOS. There, we do have a /usr/bin/cd command. Let's give that a try! Nope, still no go. Same problem. Then what the hell good is this program? Well, turns out that POSIX requires a standalone utility named 'cd' to exist, so if you want to be POSIX compliant and a trademark UNIX, then you have to provide this command, even if it doesn't work and thus is completely useless. You still have to provide a 'cd' builtin in your shell, so that you can actually change your current working directory however, but you do ship with a useless utility to appease the standard. Ah yes, computers - they're great, because things always make sense with computers!

Recap

On that, let's break here. To recap: to create a directory, call mkdir(2); to remove a directory, call rmdir(2); to traverse file system hierarchies, opendir(2)/readdir(2) can be used, but for your midterm project you should use the fts(3) library. The current working directory is specific to the current process, and changing the current working directory can only work within the same process, which is why 'cd' must always be a shell built-in, and cannot be a standalone executable - even if your os may ship with one. In our next segment, we'll take a look at directory sizes. You should find that they're a bit less obvious than you might initially think. Thanks for watching, until then - cheers!