Patent ID: 8380688

Claim:
A data compression method comprising: processing a first input stream of uncompressed data for a first file, including dividing the input stream into a plurality of segments, the size of the segments defined by a dividing algorithm determining segment boundaries based on data content; for each segment, applying a hash to a segment and associating an offset and length with the hashed segment for identifying the location and size of the segment; identifying whether the segment is unique by comparing the hash of the segment with all other hashes previously stored in a hash table; storing the hash and corresponding offset and length for the segment into the hash table responsive to determining that the segment is unique; streaming data for the unique segment into an output stream; and compressing data in the output stream, wherein an uncompressed segment of a second input stream is appended to a first compressed input stream based upon data in the hash table, including employing the hash table from the first file for creating a unique hash and associated offset for all non-duplicate segments.