Patent ID: 8346702

Claim:
A technical system for determining the accuracy of expert forecasts embodied in a target natural language document, the system comprising: a data storage area attached to a prediction server, wherein the data storage area stores a plurality of structured models and decision models; a communication interface device that receives information from a data communication network, said information including natural language documents; an opinion extraction module that receives a natural language document comprising an opinion, parses the words in said natural language document, identifies an expert who authored the natural language document, identifies a named entity in the natural language document, determines a polarity for each reference to the named entity in the natural language document, aggregates each determined polarity into a single aggregate polarity for the named entity utilizing a voting process, determines an opinion based on the aggregate polarity, creates a structured model of said natural language document, wherein the structured model identifies the expert and the named entity and includes said opinion, and stores the structured model in the data storage area; a decision model module that obtains a plurality of structured models for an expert, obtains a plurality of outcomes for the opinions of the plurality of structured models, processes the plurality of structured models and the plurality of outcomes to produce a decision model for said expert, and stores said decision model in the data storage area; and a prediction module that receives a target natural language document, identifies a target expert who authored the target natural language document, extracts a target opinion from the target natural language document, obtains a decision model for said identified target expert from the data storage area based on a lookup of said identified target expert, and applies the obtained decision model to the target opinion to generate a prediction of the accuracy of the target opinion.