Don't track selection to mouse in exclusive mode

The selection should not extend to the mouse when the selection mode is
exclusive.