De duplication and sorting algorithms done