Patent Document ID: 20170277599
Application ID: 15622023
Patent Status: 0

Claim One:
1. A method comprising: determining chunk boundaries in a data set to determine variable size data chunks from the data set, wherein determining each of the chunk boundaries comprises, progressively shifting a window over the data set; for each data subset of the data set within the window, calculating, with a first hash function that is a rolling hash function, a boundary identifying hash based on the data sub set; after calculating the boundary identifying hash, determining whether the boundary identifying hash matches a first boundary indicating value; based on a determination that the boundary identifying hash does not match the first boundary indicating value and before shifting the window, determining whether a size of the data subset within the window and of data trailing the window since a preceding chunk boundary is equal to a maximum chunk size to determine whether the maximum chunk size has been reached; identifying the data subset within the window as a chunk boundary based on a determination that the boundary identifying hash matches the first boundary indicating value or a determination that the maximum chunk size has been reached; and for each identified chunk boundary, determining a variable size data chunk as the data subset within the window corresponding to the identified chunk boundary and data trailing the window since a preceding chunk boundary.