Allow the de-selection of contact users