Use a more efficient method of duplicate detection

See, interview questions *are* useful in the real world!