added duplicate detection