Patent ID: 8108361

Claim:
A computer-implemented method for encoding data to simultaneously compact and organize the data in a manner that facilitates efficient data access operations, including: an act of a computer system, which includes at least one processing unit, organizing the data according to a set of column based sequences of values corresponding to different data fields of the data; an act of the computer system transforming the set of column based sequences of values to a set of column based integer sequences of values according to at least one encoding algorithm; and an act of the computer system compressing the set of column based integer sequences according to an iterative hybrid compression algorithm, the iterative hybrid compression algorithm comprising, in at least one iterative encoding step: an act of analyzing the set of column based integer sequences to determine which encoding technique to apply to compress the set of column based integer sequences, the encoding technique comprising at least one of: a run length encoding (RLE) technique that replaces one or more sequences of a same data value with a single data value and a count of the number of times the data value appears in each sequence, or a bit packing encoding technique that minimizes a number of bits used to represent each value in a set values, by determining a numeric range over which the set of values span and applying one or more optimizations, including one or more of: (i) eliminating any bits not needed to represent numbers within the numeric range, (ii) removing common powers, or (iii) shifting the set of values, the act of analyzing including comparing a first computed bit savings of applying the RLE technique against a second computed bit savings of applying the bit packing encoding technique; and an act of applying the RLE technique or the bit packing encoding technique on at least a portion of the set of column based integer sequences, based on the analysis.