Added more intelligent duplication checking