Patent ID: 9128951
Filing Date: 2015-09-08
Classification: G06F

Abstract:
1. A computer-implemented method for variable-length chunking for deduplication, at least a portion of the computer-implemented method being performed by a computing device comprising at least one processor, the computer-implemented method comprising: identifying a first data stream subject to deduplication using variable-length chunking; identifying a predetermined chunk of the first data stream, the predetermined chunk starting at a first location within the first data stream and ending at a second location within the first data stream; identifying, based on the predetermined chunk, a second data stream that comprises a matching chunk that matches the predetermined chunk, the matching chunk starting at a third location within the second data stream and ending at a fourth location within the second data stream; identifying a subsequent chunk within the second data stream, the subsequent chunk starting at the fourth location within the second data stream and ending at a fifth location within the second data stream; calculating a candidate boundary offset within the first data stream based on exceeding the second location within the first data stream by a difference between the fifth location within the second data stream and the fourth location within the second data stream; performing a boundary test at the candidate boundary offset to determine whether the candidate boundary offset is a location of a chunk boundary within the first data stream, wherein the boundary test indicates a likelihood that a candidate chunk beginning at the second location within the first data stream and ending at the candidate boundary offset matches the subsequent chunk; and chunking the first data stream based on the boundary test by beginning a chunk at the second location and ending the chunk at the candidate boundary offset.