Patent ID: 8423532

Claim:
A non-transitory computer-readable medium embodying a program executable in a computing device, the program comprising: code that assigns an identifier to each of a plurality of data records, the identifiers being randomly selected from a set of identifiers; code that provides a set of queries for retrieving the data records from a first search index, a first one of the set of queries encompassing all of the set of identifiers; and code that generates a second search index from the first search index by repeatedly: selecting one of the set of queries as an active query; querying the first search index for at most a predetermined number of unmarked ones of the data records using the active query, thereby obtaining a result set of data records, the predetermined number of unmarked ones of the data records being less than a total number of the data records indexed by the first search index; inserting a transformed version of each one of the result set of data records into the second search index; marking the result set of data records in the first search index; removing the active query from the set of queries when the result set of data records associated with the active query is empty; estimating an inconsistency rate by comparing the result set of data records with at least one previously obtained result set of data records, the inconsistency rate representing a proportion of unmarked ones of the result set of data records that have been previously marked; replacing the active query in the set of queries with a first query and a second query when the inconsistency rate meets a minimum threshold, the first query encompassing a first bisection of a set of identifiers associated with the active query, the second query encompassing a second bisection of a set of identifiers associated with the active query; and replacing the active query in the set of queries with a merged query when the inconsistency rate meets a maximum threshold, the merged query being constructed from a set of identifiers associated with the active query and an adjacent subset of a set of identifiers associated with another query.