Deactivate the marker that has been moused off of, not the active marker.