Patent ID: 8868575

Claim:
A method of transforming a logical data object for storage in a storage device operable with at least one storage protocol, said method comprising: dividing the logical data object into a plurality of segments, each segment characterized by respective start and end offsets, wherein the storage device is one of a network-attached storage (NAS) file server, a storage area network (SAN) storage device, and a stream storage device; receiving a first plurality of variable size data chunks at a first time and corresponding to the logical data object; processing the first plurality of variable size data chunks to generate a first plurality of processed data chunks; receiving a second plurality of variable size data chunks at a second time and corresponding to the logical data object; processing the second plurality of variable size data chunks to generate a second plurality of processed data chunks, wherein: processing the first and second plurality of variable size data chunks comprises compressing the first and second plurality of variable size data chunks by combining a time-based compression technique and a position-based compression technique, the time-based compression technique comprises compressing the first plurality of variable size data chunks to form a first compressed stream and compressing the second plurality of variable size data chunks to form a second compressed stream, and the position-based compression technique comprises dividing at least one data chunk in the first compressed stream into at least two portions including different sizes, dividing at least one data chunk in the second compressed stream into at least two portions including different sizes, and interleaving the divided portions from the first compressed stream and the second compressed stream, wherein interleaving the portions comprises interleaving at least two portions of text data; repeating the processing technique for each subsequently received plurality of variable size data chunks; bounding the amount of processed data by controlling a size of a recovery list and a flushing frequency of a segment index section, a global index section, or both for data recovery; associating each processed data chunk with a respective segment of the logical data object; and sequentially accommodating each plurality of processed data chunks in accordance with an order in which each plurality of variable size data chunks were received while maintaining each association with the respective segments.