added remove duplicates algorithm