dups removed and file gzipped