Fix another scheduling bug where we were running an extra callback
before removing an event.