Patent ID: 11972211
Assignee: WELLS FARGO BANK, N.A.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 12:
13. A computing device comprising:
a processor;
a memory, the memory storing instructions, which when executed by the processor, causes the computing device to perform operations comprising:
identifying a sentence in a document corpus, the sentence used to train or test a first machine learned model used to provide answers to user questions, the sentence producing a first result when used as input to the first machine learned model, the first machine learned model being a natural language processing machine-learned model;

searching the sentence using parameters of the first machine learned model to identify a token in the sentence;
identifying a set of one or more replacement tokens by executing a second machine learned model using the token as an input;
filtering the set of one or more replacement tokens to remove one or more of the replacement tokens that, if used to replace the token, would change a semantic meaning of the sentence;
creating a new sentence by replacing the token with one of the set of one or more replacement tokens that passes the filter, the new sentence producing a second result when used as input to the first machine learned model; and
retraining the first machine learned model using the new sentence and the first result, the retrained first machine learned model producing the first result when the new sentence is input to the retrained first machine learned model; and
providing a question answering service using the retrained first machine learned model.