Ungrab touch only if it's our touch.