Fixing logic issue with toggling drag on mousedown