Add some logic that avoids adding duplicates but replaces duplicates with the key that's being moved