Patent Document ID: 8725666
Application ID: 13037284

Base Claim:
1. A method of operating an information extraction system, comprising: providing the information extraction system, the information extraction system comprising i. a processor master device, configured to receive input data and experimental options; ii. a plurality of processor extractor devices, configured to transform input data into extractor outputs; iii. a processor aggregator device, configured to aggregate extracted entities of the plurality of extractor outputs to form meta-entities, dispatch meta-entities to aggregation algorithms, form hypotheses for each meta-entity, calculate probability for each hypothesis, and reconstruct a truth entity based on each hypothesis, wherein each of the meta-entities is formed from the extracted entities of at least two processor extractor devices of the plurality of processor extractor devices; iv. a processor storage device, configured to store input data, extractor outputs, and other files; and v. a processor communication device, configured to enable high bandwidth communication of data between the devices of the information extraction system; calibrating the information extraction system by: calibrating a plurality of aggregation algorithms executed by the aggregator device; and calibrating a dispatcher of the aggregator device, wherein the calibrating comprises (a) providing a training corpus of text containing a plurality of known entities to train a plurality of entity extractors, thus forming a plurality of trained entity extractors; (b) transforming, using each of a plurality of trained entity extractors, a corresponding evaluation corpus to extractor outputs of extracted entities; (c) combining the extracted entities from the plurality of trained entity extractors to form meta-entities, wherein each of the meta-entities is formed from the extracted entities of at least two trained entity extractors of the plurality of trained entity extractors; (d) mapping each meta-entity to one or more known entities, thus revealing the errors induced by the transforming of known entities into extracted entities by the plurality of entity extractors; (e) determining and storing a probability distribution for the errors induced by the plurality of entity extractors across the plurality of meta-entities, wherein determining and storing a probability distribution for the errors induced by the plurality of entity extractors across the plurality of meta-entities further comprises a. estimating the probability distribution over an implicit space of the errors; b. constructing a pattern dictionary from the evaluation corpus by determining a meta-entity pattern for each meta-entity and a truth pattern for each known entity; c. recording a first tally of votes for the number of times each truth pattern is associated with the meta-entity pattern; and d. recording a second tally of votes for the number of times each truth pattern is associated with a subset sub-pattern of the meta-entity pattern; and (f) determining an aggregate probability distribution for the errors induced by a plurality of entity extractors over the training corpus, thus calibrating each aggregation algorithm; providing an input corpus of texts utilizing the master device and the communication device; transforming, using each of a plurality of extractors executed by the extractor devices, the input corpus to extractor outputs of extracted entities; combining the extracted entities from the plurality of extractors to form meta-entities by the aggregator device; dispatching the meta-entities, via the calibrated dispatcher executed by the aggregator device, to one or more aggregation algorithms based on defined features of the meta-entities; forming a plurality of hypotheses for each meta-entity; calculating a value for each hypothesis; and reconstructing a truth entity based on one or more hypothesis ranked by the value calculated for each hypothesis.

---

Claim 6:
6. The method according to claim 1 , wherein the calculating of a value for each hypothesis further comprises: converting each hypothesis to a truth pattern relative to the associated meta-entity pattern; assigning a prior probability for each model in a class of models; computing a probability estimate for each hypothesis with each model; estimating a posterior probability distribution over the class of models based on the first tally for the truth patterns in the pattern dictionary for each model; and combining the probability estimates for each hypothesis based on the posterior probability distribution to calculate the value for each hypothesis.