Don't limit the scroll diff amount going down/right