added functionality to respect unique keys, simplified code