Patent ID: 11966697
Assignee: BANK OF AMERICA CORPORATION
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 5:
6. A machine-learning (“ML”)-based language-processing system with automated tuning, said system comprising:
an ML model that is configured to derive a comprehension of an input utterance, wherein:
said ML model is trained using a training set;
the training set is stored in a database and comprises a plurality of utterances mapped to a plurality of intents; and
each of the plurality of utterances is assigned a set of features; and

a tuning module, said tuning module for improving the comprehension of the ML model while resisting degradation in the comprehension, said tuning module configured to:
identify a set of inoperative input utterances and a set of operative input utterances, wherein an inoperative utterance is an utterance that returns a null intent response;
iterate through the set of inoperative input utterances and the set of operative input utterances based on the derived accuracy score, such that when the highest scoring intent for an utterance is flagged as inaccurate, the tuning module is further configured to attempt to determine whether the second highest scoring intent is inaccurate;
derive a first accuracy score for the set of inoperative input utterances and for the set of operative input utterances;
retrieve a feature set associated with each inoperative input utterance from the set of inoperative input utterances;
generate, based on the feature set, an updated utterance-feature-intent (“UFI”) mapping;
retrain the ML model, in a controlled environment, using the updated UFI mapping;
generate an updated accuracy score for the set of inoperative input utterances and for the set of operative input utterances, said updated accuracy score based on the retrained ML model;

when the updated accuracy score for the set of inoperative input utterances is less than or equal to the first accuracy score for the set of inoperative input utterances, and the updated accuracy score for the set of operative input utterances is greater than or equal to the first accuracy score for the set of operative input utterances, amplify the impact of the updated UFI mapping by retraining the ML model with a plurality of instances of the updated UFI mapping; and
when the updated accuracy score for the set of inoperative input utterances is greater than the first accuracy score for the set of inoperative input utterances, and the updated accuracy score for the set of operative input utterances is greater than or equal to the first accuracy score for the set of operative input utterances, deploy the updated UFI mapping;, wherein the tuning module is further configured to amplify the impact of the updated UFI mapping incrementally, wherein the tuning module is configured to:
retrain the ML model with a first instance of the updated UFI mapping;
generate a new updated accuracy score for the set of inoperative input utterances and for the set of operative input utterances;
when the new updated accuracy score for the set of inoperative input utterances is still less than or equal to the first accuracy score for the set of inoperative input utterances, and the updated accuracy score for the set of operative input utterances is still greater than or equal to the first accuracy score for the set of operative input utterances, retrain the ML model with another instance of the updated UFI mapping; and
wherein the ML model is retrained with a maximum of three instances of the updated UFI mapping.