refactor dedupe
refactor dedupe
Merge pull request #38 from iplayer/dedupe_refactor

Dedupe refactor