Don't limit panning to viewport bounds