add duplicate detection
add duplicate detection
Merge pull request #18 from gt-big-data/no_dups

No dups