If the page has moved during a click by more than a small amount, ignore it