Patent ID: 8442956
Filing Date: 2013-05-14
Classification: G06F

Abstract:
1. A method for controlling a data de-duplication computer to selectively store or transmit a sub-block of data based on sub-block level sampling-based de-duplication, comprising: controlling the data de-duplication computer to select a number of locations in the sub-block to be sampled and positions for the locations to be sampled, where the selecting is based on one or more of whether an entropy measure associated with the sub-block exceeds an entropy threshold, whether a sub-block size associated with the sub-block exceeds a size threshold, and whether the sub-block was chunked using fixed boundaries or data dependent boundaries; controlling the data de-duplication computer to compute a sampling sequence for the sub-block of data, where the sampling sequence comprises data from the selected number of locations at the selected positions in the sub-block; controlling the data de-duplication computer to use the sampling sequence to locate a stored sub-block known to the data de-duplication computer, comprising locating an entry point to a set of sampling sequence related sub-blocks, where the entry point is identified in a sampling sequence keyed index; controlling the data de-duplication computer to determine a degree of similarity between the sub-block and the stored sub-block, where the degree of similarity is determined by one or more of: and based on the degree of similarity, controlling: