Ignore scroll changes when not scrolling