Make epoll_ctl errors fatal errors

We see this every so often and they result in a bad state, so it is
best to terminate.