Remove the entities when we cancel the event to hopefully prevent entity build-up.