Exclude files that are only redirects, and where the real file exists elsewhere in the data set. Fixes #13.
Fixed typo
Merge pull request #14 from commonsmachinery/duplicate-check

Duplicate check