Only call the gesture end when we have a recognized gesture