Got to ensure any end of teardown functions get called. We have a removeTrackEvent you can call that's handle this all for you. Should just call that. Forgot about it. Not sure if you need to worry about it when you remove the initial padding events, but should look into it.

Also I think that means you don't need to loop through start times as I think removeTrackEvent removes it form both byStart and byEnd arrays.