close the poller before the fd.