Only drop if the element is in the correct place