Put all the file descriptors into the fdset every loop.

That's because when select() returns, only the one that caused the return will
still be in the set. Thus, the old behaviour was that only the fd that woke
select would ever be monitored again. The upshot of which was that once we got
one IP4 packet we could happily keep getting IP4 packets but were no longer
watching the control pipe...
TODO.