fix Issue 11009 - Regression (2.064 git-head): DMD consumes huge memory when it compiles enum containing many items

This is a fixup of pull #1540.

Turns out NoDuplicates is scary inefficient for large input.

This cleans up and simplifies the pull somewhat.
Merge pull request #1565 from monarchdodra/fix11009

fix Issue 11009 - Regression (2.064 git-head): DMD consumes huge memory ...