Modify charmers duplicate removal to be more robust