Sped up panning with arrows by caching when possible to prevent redundant iterations.