Fixed checks to setup panning only once
Fixes #1, store the view on which the panning happens.

This is not that good. Could need a deeper investigation.
Version bump