Patent ID: 11921761
Assignee: OPTUM, INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 13:
14. A computing system comprising a non-transitory computer readable storage medium and one or more processors, the computing system configured to:
receive an input question originating from a user computing entity;
generate, using a deep question-answering (QA) application, a deep learning answer and a deep learning confidence score for the deep learning answer, wherein (a) the deep QA application comprises an artificial neural network, and (b) the deep learning answer is generated by the artificial neural network for the input question;
determining that the deep learning confidence score for the deep learning answer does not satisfy a configurable deep QA threshold associated with a likelihood that the deep learning answer generated using the deep QA application is a correct answer to the input question;
query a retrieval QA application for a retrieved answer, wherein (a) the retrieval QA application identifies the retrieved answer at least in part by ranking a plurality of documents responsive to the query and generates a plurality of retrieval confidence scores for the plurality of documents respectively, and (b) the retrieved answer comprises a highest retrieval confidence score of the plurality of retrieval confidence scores; and
responsive to determining that the highest retrieval confidence score for the retrieved answer satisfies a retrieval confidence score threshold, provide the retrieved answer from the retrieval QA application as an answer from a hybrid QA platform to the input question, wherein the hybrid QA platform comprises the retrieval QA application and the deep QA application; and
responsive to determining that the highest retrieval confidence score for the retrieved answer does not satisfy the retrieval confidence score threshold, request an expert user answer by transmitting the input question to a user computing entity associated with an expert user;

identify a plurality of retrieved answers, wherein (a) one or more of the plurality of retrieved answers satisfies a training threshold associated with a quality of the plurality of retrieved answers being suitable for generating training data for the deep QA application and (b) the plurality of retrieved answers comprises the retrieved answer;
generate training data for the artificial neural network, wherein the training data comprises one or more training question-answer pairs, each comprising a respective question and a corresponding respective answer, for respective one or more retrieved answers in the plurality of retrieved answers, wherein one of the one or more training question-answer pairs comprises the input question and the retrieved answer; and
initiate the performance of training the artificial neural network based at least in part on the training data, wherein initiating the performance of training comprises, for the one or more training question-answer pairs in the training data: (i) generating a comparison measure between the corresponding respective answer and the deep learning answer, and (ii) updating trainable weights of the artificial neural network based at least in part on the comparison measure.