Patent ID: 9665610
Date: 2017-05-30
CPC Classifications: G06F

Claim:
1. A method for reducing digests storage consumption by tracking numbers of similarity elements in a similarity search structure in a data deduplication system using a processor device in a computing environment, comprising: partitioning, using the processor device, input data into chunks and grouping the chunks into chunk sets, the partitioned chunks each being at least 16 Megabytes (MB) in size; calculating, using the processor device, digests for the input data and storing the digests in sets corresponding to the chunk sets; calculating, using the processor device, similarity elements for the input data and storing the similarity elements in a similarity search structure; maintaining, using the processor device, for each one of the chunk sets a number of the similarity elements associated with the chunk set which are currently contained in the similarity search structure; removing, using the processor device, a digests set associated with a chunk set from a repository when the number of similarity elements of that chunk set becomes lower than a predetermined threshold; and deduplicating, using the processor device, the input data using the calculated similarity elements of the chunk set from the repository.