Patent ID: 8538936

Claim:
A computer-implemented method of compressing data, the method comprising: receiving a data set in a data stream, the data set including a set of data descriptor fields; partitioning the data set into one or more data subsets using the set of data descriptor fields; generating one or more tabular slices and an index for at least one of the one or more data subsets using the set of data descriptor fields, the one or more tabular slices identified by the index, and each tabular slice comprising a plurality of rows from the one or more data subsets, wherein the number of rows comprised in each tabular slice is variable; compressing the one or more tabular slices into a compressed data block by a data compression scheme using a hardware compressor; generating a compression data file in a database, the compression data file having a header that stores information about the data compression scheme; and storing the compressed data block in the compression data file.