got rid of unicode, duplicates