Patent ID: 9613317
Date: 2017-04-04
CPC Classifications: G06F,G06N

Claim:
1. A method, in a data processing system comprising a processor and a memory configured to implement a question and answer system (QA), for generating answers to an input question, comprising: training a justifying passage model (JPM) based on a JPM ground truth data structure that comprises a justification indicator for each question-answer-evidence passage (QAP) triplet in a plurality of QAP triplets of the JPM ground truth data structure, wherein the justification indicator indicates whether or not an answer in the QAP triplet is justified by the evidence passage of the QAP triplet as being a correct answer for a question in the QAP triplet, and wherein an answer of the QAP triplet is justified by the evidence passage of the QAP triplet when content of the evidence passage explicitly states the answer to be a correct answer for the question of the QAP triplet; receiving, in the data processing system, the input question; generating, by the data processing system, a set of candidate answers for the input question and, for each candidate answer in the set of candidate answers, a corresponding selection of one or more selected evidence portions from a corpus of information providing evidence in support of the candidate answer being a correct answer for the input question; ranking, by the data processing system, the candidate answers based on an application of the trained JPM to the selected evidence portions for each of the candidate answers in the set of candidate answers, wherein the JPM identifies whether a candidate answer is justified by a selected evidence portion corresponding to the candidate answer, and wherein application of the trained JPM to the selected evidence portions causes the ranking of the candidate answers to be modified based on whether or not a selected evidence portion is justifying of a corresponding candidate answer; and outputting, by the data processing system, a candidate answer in the set of candidate answers as the correct answer for the input question based on the ranking of the candidate answers.