Enhance scroller dragging experience

- Stop moving the scroller when the click happens inside it (prevents "jumping" before drag)
- When dragging, make the cursor stay at the same position relative to the scroller
- Properly set/remove event listener on mouseup/down to improve performance