Update for new Dedupe version. More clever file processing