Added more optimizations for checking duplicates