documentcloud/documentcloud — pull request #36 by nathanstitt (2012-10-01 16:05:40+00:00)
Title: Track Document's file hash to speed duplicate processing

Stores a the file hash of each file processed by cloud_crowd's document_import action. The file_hash is then checked by the process_images step for duplicates. If a duplicate is found the per-page images are copied from it, skipping the expensive Docsplit.extract_images step.