Patent Document ID: 9965548
Application ID: 14097655

Base Claim:
1. A method, in a data processing system comprising a processor and a memory configured to implement a question and answer system (QA), for improving answers to an input question, the method comprising: receiving, in the data processing system, the input question; processing, by the data processing system, the input question to generate at least one query; extracting, by the data processing system, one or more selections of evidence portions from a corpus of information which match the at least one query; generating, by the data processing system, one or more candidate answers based on the one or more selections of evidence portions; for at least one candidate answer in the one or more candidate answers, determining, by the data processing system, whether there are one or more missing or ambiguous pieces of information that would refine an inclusion or exclusion of the candidate answer in the one or more candidate answers; outputting, by the data processing system, the one or more candidate answers, the one or more missing or ambiguous pieces of information that would refine the inclusion or exclusion of the candidate answer in the one or more candidate answers, and the selections of evidence portions for evaluation by the user; outputting, by the data processing system, one or more pieces of criteria that identifies conditions met by each candidate answer in the one or more candidate answers in relation to the input question; generating, by the data processing system, one or more confidence scores for each of the one or more candidate answers by applying one or more reasoning algorithms to language of the input question and language of the one or more candidate answers; comparing, by the data processing system, results of each of the one or more reasoning algorithms to a statistical model; based on the comparison of the results to the statistical model, modifying, by the data processing system, the statistical model; applying, by the data processing system, statistical weights to confidence scores associated with each of the one or more reasoning algorithms in accordance with the statistical model; and synthesizing, by the data processing system, the one or more confidence scores for each of the one or more candidate answers, thereby creating a one or more final answers.

---

Claim 3:
3. The method of claim 1 , further comprising: parsing, by the data processing system, the input question for a known set of critical information helpful in answering the input question; determining, by the data processing system, whether all specific annotators with regard to the known set of critical information are provided based on a predefined set of critical annotators; responsive to one or more specific annotators of the predefined set of critical annotators being unidentifiable in the input question, thereby forming a set of missing annotators, determining, by the data processing system, whether the input question is submitted in relation to one or more stored data files associated with a subject of the input question; responsive to the input question being submitted in relation to one or more stored data files associated with the subject of the input question, parsing, by the data processing system, the one or more stored data files associated with the subject of the input question for the set of missing annotators; and responsive to the set of missing annotators being unidentifiable via parsing the one or more stored data files associated with the subject of the input question, prompting, by the data processing system, a user who submitted the input question for the set of missing annotators.