Patent ID: 8898120

Claim:
A computer-implemented method for distributed data deduplication, at least a portion of the method being performed by a computing device comprising at least one processor, the method comprising: identifying a deduplicated data system, the deduplicated data system comprising a plurality of nodes, wherein each node within the plurality of nodes is configured to deduplicate data stored on the node; identifying a data object to store within the deduplicated data system; generating a similarity hash of the data object, the similarity hash representing a probabilistic dimension-reduction of the data object; determining, based on at least one characteristic of the data object, that block-level deduplication is more efficient for the data object that file-level deduplication; selecting a target node from the plurality of nodes that uses block-level deduplication instead of a candidate node that uses file-level deduplication based at least in part on the similarity hash and at least in part on a determination that the target node uses block-level deduplication instead of file-level deduplication; routing the data object for storage on the target node based on the selection of the target node.