Patent ID: 8028007

Claim:
A method, implemented at least in part on a computing device, for processing a data stream embodying a hierarchically structured document, said method comprising: querying the hierarchical structure of the document embodied in the data stream; determining an offset of the data stream, the offset determined during the querying, and the offset including one or more bits; partitioning said data stream into respective fixed length segments utilizing said queried hierarchical structure and the offset to determine a respective length of each fixed length segment; processing said fixed length segments in a pipeline fashion, the processing the fixed length segments including decoding the fixed length segments; parsing the decoded fixed length segments; partitioning the parsed fixed length segments into fragments, the fragments having at least one size, the at least one fragment size being cached in a memory of the computing device; inserting database persistence boundaries between the fragments; storing the fragments in a storage medium, the storage medium including the database and the at least one fragment size being determined in accordance with characteristics of the database, the characteristics including a native unit for holding data; creating a database table, the table comprising: meta data associated with the document; queries over the document and respective results; a first fragment of the document; sizes of all fragments of the document other than the first fragment; and storing the database table in the database.