Use a separate list for changes to be able to addfd() and delfd() while dispatchevents() is happening