Provides tools for deduping a set of files and sync the unique set to S3.