made some changes to how it checks for dupes, etc