Same logic as uv__fsevents_poller_destroy().  Could be moved into a separate function.