Patent ID: 8484214

Claim:
An apparatus comprising: an interface operable to receive a plurality of source data; and a processor operable to: create a plurality of active templates based upon common data from the plurality of source data, wherein the source data is read as an unordered set, determine, for each of the plurality of active templates, a match ratio calculated as a source data size minus a template delta size divided by the source data size; determine a total match ratio of the plurality of active templates, wherein the total match ratio is calculated as a weighted average of average match ratios of the active templates and a weight is determined by a popularity metric of each active template; compare the total match ratio of the plurality of active templates to a target total match ratio, determine whether a limit of active templates has been reached when the total match ratio is less than the target total match ratio, retire a template having a smallest relative match ratio from the plurality of active templates when the limit of active templates has been reached to limit the number of active templates considered for use in compression, create a best active template when the total match ratio is less than the target total match ratio, and create a compressed record including a reference to the best active template and a template delta that represents a difference between the best active template and one of the plurality of source data.