Patent ID: 9536444
Date: 2017-01-03
CPC Classifications: G09B,H04L

Claim:
1. A method, in an information handling system comprising a processor and a memory, the method comprising: receiving, from a computer network, a plurality of responses at a question and answering (QA) system, wherein each of the responses includes an evaluation of a passage selected from a plurality of passages corresponding to a question and an answer, wherein each of the responses is received from one of a plurality of subject matter experts (SMEs), and wherein the responses are stored in the memory; calculating a plurality of passage evaluation scores, each of the plurality of passage evaluation scores based on the received responses, wherein each of the passage evaluation scores correspond to a different one of the plurality of passages; associating the plurality of passage evaluation scores to the question and the answer, wherein each of the passage evaluation scores signifies a relevance of one of the plurality of passages to the question and the answer; and training a model utilized by the QA system, wherein the training is based in part on the plurality of passage evaluation scores associated with the question and the answer.