added dedup check