Fix a bug when trying to remove a line while the selected element is not the key but the label