Added missing implementations of _get_duplicates and _clean_duplicates