Returning without calling setTimeout if we abort