Patent ID: 8661037

Claim:
A method for indexing documents, comprising: generating a single document hash table in storage memory for a single document using an index construction in a multithreaded and scalable configuration wherein multiple threads per processing core are each concurrently assigned work to reduce synchronization between threads wherein generating a single document hash table includes: partitioning the single document into a plurality of subparts and indexing strings of partitioned subparts of the single document to create a minor hash table for each subpart; generating a document level hash table from the minor hash tables; generating an interval hash table in storage memory for a plurality of single document hash tables; and generating a global hash table in storage memory for a plurality of interval hash tables such that single documents or portions thereof can be searched by employing the global hash table wherein the interval hash table and the global hash table maintain the index construction for the threads; updating a stream level hash table for the strings which maps every string to a global identifier, wherein the multiple threads per processing core are configured to read the strings from a buffer into the stream level hash table; and generating a term reordered array from the document level hash table.