Patent ID: 9110936
Filing Date: 2015-08-18
Classification: G06F

Abstract:
1. In a computing environment, a method performed at least in part on at least one processor, comprising: loading a subspace index comprising less than all index entries of a hash index service from a secondary media into a primary memory cache in which the subspace index corresponds to a partitioning of a global index into separately optimized subspaces; reconciling at least two of the subspaces to remove at least one duplicate chunk, including using a resemblance metric to compare a subspace and another subspace by determining a similarity between at least one signature representative of the subspace and at least one signature representative of the other subspace; and using the subspace index to deduplicate a dataset, including chunking the dataset into one or more chunks, and for each chunk, determining whether a hash value computed for that chunk matches a hash value of an entry in the primary memory cache, and if not, storing the chunk and adding an entry for that hash value into the subspace index in association with a reference to that chunk, and if so, returning information by which the existing chunk is locatable.