Patent ID: 8229916

Claim:
In a parallel pipelined structure on a multi-core device, a method for parallel pipelined multi-core indexing, comprising the steps of: generating one or more single document indexes respectively corresponding to one or more single documents of a given data stream; generating one or more multi-document interval-based hash tables from the one or more single document indexes; modeling at least one of the multi-document interval-based hash tables in a fixed array and a residual array, wherein overflows in the fixed array are represented in the residual array using offsets; and generating one or more global hash tables formed from merging one or more of the multi-document interval-based hash tables, all of the one or more global hash tables representing a collective index for all of the single documents for which the one or more single document indexes were generated, wherein merging one or more of the multi-document interval-based hash tables comprises using term-based partitioning to reduce a compute unit load relating to the merging by distributing the compute unit load onto at least some of a plurality of compute units of the multi-core device.