bugfix: ensure setting both the maxxfd and the minfd properly
when the last fd is removed from the mux

otherwise we would endup passing invalid arguments to epoll which would
return immediately wasting cpu time