DragManager should only claim mouse release if we're dragging