Introduction

Hello, and welcome back to CS631 "Advanced Programming in the UNIX Environment". With this video, we complete our discussion of interprocess communication. We've come a long way from when we started with signals as the most trivial, asynchronous method to inform a process that some condition occurred to full, bi-directional, connection-oriented communications between two systems spread out over the internet using a modern network protocol such as IPv6. In our last two videos, we moved from simple interprocess communications to a client-server communications model, but we still used primarily one-to-one communications. That is, we had a single process accepting connections and another process send messages. But this isn't a realistic communications model on the internet, so let's see how we can build on this to move towards a more typical client-server model with a server side process capable of handling multiple simultaneous clients.

Example: Opening two sockets

So let's quickly revisit our last example and see where we might run into problems: Here's our streamreader code, where we create a socket, bind it, listen, and then accept connections. Suppose that we want to allow multiple connections. We might consider opening additional sockets, and so we might split out the functionality to create a socket and to then handle a socket and accept connections into subroutines, so that our new 'main' might look like so: We create two sockets, then handle both of them in succession, hoping to allow multiple connections to our server. So now when we run this program, we will have two sockets open. They are port 65446 and port 65445. So now we should be able to connect to either port and send messages to our server. Let's give it a try. Here we create one client connection from our laptop to port 65445 and send a message. But we don't see our server receiving it! Why is that? If we try to connect to the other port - 65446 - and send a message, it is immediately received. So what happened to the other connection? It seems like we can send messages, but nothing happens, because our program is still blocked inside the first call to the 'handleSocket' function. Only when we terminate this connection can our program move on and handle the second socket, which it then does immediately, as shown here. So our model of creating two sockets and then trying to process them sequentially is not going to work so well -- the 'accept' in the first call to 'handleSocket' will block, even if there is already a connection waiting on the second socket.

I/O Multiplexing options

So what can we do when we have multiple file descriptors that we wish to perform I/O on? Well, the first option is what we've just seen: we simply open the file descriptor, block, waiting for I/O, and then move on to the next. But that's clearly not a very good option. We could fork a new process for each file descriptor we want to handle. That would seem reasonable, but what if you want to have information be passed between the different channels? Then you need additional IPC, so that seems a bit cumbersome, too. We could try non-blocking mode, something we'll discuss in a bit more detail in a future lecture. But in a nutshell, if we mark the file descriptor as non-blocking, then any call we make will return immediately, and we can move on if it's not ready for I/O. Or we can use asynchronous I/O, another form of I/O we'll discuss in more detail in the future as well. This approach requires us to get notified by the kernel when the file descriptors we're interested in are ready for I/O. But each of these options has at least some drawbacks: Blocking forever clearly is undesirable, as we've seen. Using non-blocking mode, on the other hand, means that if there is nothing connecting to us, we end up busy-polling. That is, we loop and test and loop and test etc. etc., constantly making calls and wasting CPU cycles. Asynchronous I/O, while sounding promising, is often times very limited, as we will see in the future. But let's instead consider another option: We build a set of file descriptors we care about, and then call a special function that will tell us whether or not any of them are ready for I/O.

select(2)

And that is precisely what the select(2) syscall does. We pass it the file descriptor number up to which we want it to inspect. That is, the descriptors from 0 through nfds - 1 is examined, which is why you will always see the argument given being set as a file descriptor plus one. Then we pass in a file descriptor set for each of the fields readfds, writefds, and exceptfds, and select(2) will, respectively, check whether any file descriptors in those sets are ready for reading, writing, or if an exceptional condition has occurred. Exceptional conditions include for example out-of-band messages arriving on a socket, something we may revisit in a future lecture on advanced I/O. Finally, we can control for how long select(2) should block when waiting for such conditions: we can have it block forever by passing NULL we can specify a granularity of seconds and microseconds to wait or we can have it not block at all by setting both seconds and microseconds of the struct timeval to zero. When using select(2) for this form of synchronous I/O multiplexing, we will operate on file descriptor _sets_, which we manipulate using the provided FD_ macros, FD_SET, FD_CLR, FD_ISSET, and FD_ZERO, which we'll see in action in a moment. As mentioned, the read, write, and except fd sets are used to check for read, write, and exceptional conditions. One thing to remember about reading from a file descriptor is that EOF is a form of I/O -- that is, the file descriptor set would be marked as ready for I/O only for you to then get back 0, but that is not really surprising. If you need more granularity when blocking, you can use the pselect(2) variation, which provides nanosecond granularity as well as an option to specify a signal mask to apply while blocking.

Using select(2) with two sockets

But let's see if we can use this in action to resolve our problem of having multiple sockets that we'd like to be able to handle without blocking communications via the second socket while waiting on the first. So here we updated our example program from before to add a call to select(2). We initialize a file descriptor set, then add our sockets s1 and s2 to that set. Then we call select(2), providing the max file descriptor of the two plus one as the first argument. After select returns, we can then test each socket individually using FD_ISSET to see if either is ready for reading. So let's see if this helps us. When we start, we again create two sockets, one listening on port 65441 and one on port 65440. Let's pull up the code while we make our connections. Ok, now we can connect to our server on port 65440. In our previous example, this connection to the second socket would be blocked, as we'd be waiting for a client to connect to the first socket before moving on to the second, but now we notice that we were able to pick up the second socket because select(2) identified that this socket is ready for I/O, while the first socket did not yet have any connections. Of course we can also connect to the first socket and have that connection not be blocked. Ok, so here we've seen how the use of select(2) allowed our program to pick up whichever socket was ready, which is useful if you have multiple file descriptors to juggle I/O on. But more often, if you're writing a server, for example, you wouldn't want to have multiple sockets open. Think about a web server -- it's listening on port 80 (or on port 443 for TLS connections), but you still have multiple clients connecting at the same time.

Using select(2) with a single socket

So if we use select(2) in this example, we only have one socket, but we won't be blocked waiting for connections, meaning if there is no client connecting, we can do something else. This is what this example looks like. Now running the program, we'll see that while we're waiting for connections the server has determined that no connections are ready, so it can print this incredibly useful message while waiting. When we connect to the server, it will now handle our connection and not enter the other code block to print the 'waiting' message. Only after we disconnect does the server again get a chance to do other things as now no file descriptors are ready for I/O until the next client connects. Ok, now let's see what happens when we have two simultaneous client connections: We connect from localhost, and of course that connection is picked up right away. But if we then initiate a second connection -- over here from our laptop -- that connection remains blocked since the first connection is still active, and to have our second connection get handled, we need to finish and terminate the first connection. At that point, all the messages from the second client are delivered, since our second socket was still in the backlog of our server. We'll see that over here in a second. There. All messages from the second client were immediately delivered. So that's not going to be very helpful if you're writing a web server: imagine a website that can only handle a single connection at a time and you have to wait until the other person has finished. So that's no good. Let's think how we could improve on this...

Combining select(2) and fork(2)

Suppose we redo our program such that whenever a connection is ready on our socket, we fork a new child process, let _that_ handle the request and do whatever the server is supposed to do while the parent goes back to waiting for new connections... For this, we first need to establish a signal handler to let the parent wait(3) for any children that have terminated, as otherwise whenever a client disconnects we have a zombie left. Then we create out socket and make our select(2) call. Since we might get interrupted when a client disconnects, we simply ignore those conditions and loop around, calling select(2) again. Our function 'handleSocket' now gets invoked whenever there's a client connecting; it can then accept(2) the new connection, but instead of handling it directly, we now fork a new process and let _that_ process handle it. The parent returns and can then proceed to pick up the next connection whenever it comes in. The 'handleConnection' function then does what we're used to, with the difference to before being that after we are done and disconnect, we exit. Ok, let's see if that helped us. The server is now listening on port 65427. While no client connections are ready, the server can go and do other stuff, and when our first client connects, of course it is being handled as expected. But now note that while we are communicating with the first client, the server can _still_ go and do other things. That is, it's not blocked from moving on since the client connection is now handled by a dedicated child process. And while we are still connected to the server in the upper right, we can now initiate a second connection in the bottom right, and _this_ time, the server immediately picks it up. So now both clients can simultaneously communicate with the server without being blocked, and the server is still ready to accept new connections. If we terminate the server, all client connections are of course terminated as well.

Recap

Ok, so we've seen how to handle I/O multiplexing in this video: We can use select(2) to check a set of file descriptors for specific conditions, which allows us to either juggle multiple file descriptors or sockets, or to avoid blocking on a single socket and do other things while we wait for clients to connect. We've also seen how we can then handle multiple simultaneous clients on a single socket by spawning child processes or separate threads to have them perform the interactions. This works well if each child process does not need to communicate with any of the other clients, and is a common pattern in standard server design. Sometimes you may come across server programs that pre-fork a bunch of processes to handle the connections, which may improve on this approach. There are a few other alternative interfaces an options for being notified of when I/O is ready or, more generically speaking, an event occurred and you want to handle it. Each call or API listed here has its own advantages, and you can read up on their use cases and comparison with others on your own. They are: poll(2) epoll(2) on Linux The kqueue(2) kernel event notification mechanism available on the various BSD systems, including macOS. And libevent or libev, which may wrap some of these interfaces to allow for a platform agnostic I/O multiplexing mechanism. Each of those tries to overcome some of the shortcomings of the others in an effort to improve efficiency when handling large numbers of simultaneous connections, which isn't a trivial problem to solve. This link over here describes the problem space and the various approaches well. Alright, I think that about concludes our coverage of interprocess communications, and with the discussion from this video, you're now well prepared to begin work on your final group project: writing an HTTP server. We'll discuss the project in class, and then continue our video lecture series with a look at daemon processes and shared libraries. Until next time - thanks for watching. Cheers!