Fix several issues with remove_pair