Make sure we unschedule on the main run loop since that's where we scheduled.