Pinching to shrink the image scale to < 1.0 adds the UIPanGesture back, even if you're still zooming

This caused the image to flicker between two positions on screen.

Reproducible by pinching/zooming out (to smaller than 1.0 scale) in the center of the screen and then dragging away from the center of the screen while holding two fingers/still pinching.