Allow the user to drag faster

This is a simple fix for issues #4920 and #5999.

cc @njx @peterflynn