Numerous bugs in the code I supplied you

Hi

After testing, including on real data, I have now finished the real deduplicator. As it stands, whenever it encounters a file with -metadata- in the name, it migrates it twice. The first migration is a regular warc migration. The second is a deduplication migration. For examle, the file 99-metadata-1.arc will become 99-metadata-1.warc and 99-duplications-1.warc.