added dupe checking & small refactor