Actually cancel the pan gesture if it's cancelled.