fixed issue where it was not correctly removing duplicates, and also cleaned the code up a bit