Patent ID: 8150851

Claim:
Data processing apparatus comprising: a chunk store partitioned into a plurality of chunk sections, at least one of the chunk sections storing specimen data chunks, a processor to: process input data into input data chunks; identify a chunk section already containing a specimen data chunk corresponding to a particular one of the input data chunks; and store the particular input data chunk in another chunk section as a specimen data chunk if the identified chunk section has a predetermined characteristic, wherein the predetermined characteristic is selected from the group consisting of: a characteristic that specimen data chunks corresponding to other input data chunks of the input data being processed are stored in more than a predetermined number of chunk sections, a characteristic that specimen data chunks in the identified chunk section have been referenced by fewer than a predetermined number of the most recent M manifests to be compiled, a characteristic that fewer than a predetermined number of specimen data chunks in the identified chunk section correspond to input data chunks of the input data being processed, and a characteristic that fewer than a predetermined number of manifests in a manifest store contain a reference to the said corresponding specimen chunk in the identified chunk section.