Patent ID: 8478771

Claim:
A computer-implemented method including executing, using at least one processor, instructions recorded on a non-transitory computer-readable storage medium, the method comprising: receiving a request to insert a data record within a database of a data collection system; placing the data record within a buffer in a main memory of the data collection system; selecting a record data structure and a record index structure associated with the data record; storing the record data structure and the record index structure within a particular storage chunk of a log file comprising a plurality of storage chunks, including the particular storage chunk, in a storage medium of the database, the storage medium being different than the main memory, wherein the particular storage chunk includes: at least one associated coarse index that stores two index values, a low index value representing the lowest value stored in the chunk and a high index value representing the highest value stored in the chunk, and at least one associated index structure that includes the record index structure that can be used to retrieve the data record, and other data records different than the data record associated with the received request; receiving a second request to retrieve data records from the data collection system, the second request including a first search parameter corresponding to the coarse index; examining the coarse indexes of the plurality of storage chunks by looping through directory entries for the coarse indexes to identify storage chunks that meet the first search parameter of the second request; and when a storage chunk from the plurality of storage chunks is identified, examining the record index structure to determine data records responsive to the second request and retrieve the data records responsive to the second request.