epoll: Ignore permission errors at unregister