Patent ID: 11934784
Assignee: VERINT AMERICAS INC.
Field: Pharmaceuticals (Chemistry)
Classification: CPC G  A | IPC A  G

Claim 0:
1. A computer-implemented method of automatically removing terms from a language model based on use, the method comprising:
ingesting language data from internet-based content streams, wherein the internet-based content streams are specific to a domain of an intelligent virtual assistant (IVA);
processing the language data using a language model toolkit to extract vocabulary from the language data, the extracted vocabulary comprising terms;
identifying select terms by counting the terms in the extracted vocabulary over a sliding window;
comparing the select terms from the extracted vocabulary to terms stored in a language model specific to the IVA to determine if the select terms have a match to the terms in the language model;
determining usage of the select terms in context with respect to the language data using an embedding model;
for any of the select terms that have a match in the language model, determining a frequency of use of the select term in the ingested language data in its associated usage in context;
automatically deleting one or more first terms of the select terms from a training example for the language model without further human intervention based on the frequency of use of the one or more first terms falls below a predetermined threshold;
recompiling the language model based on the training example after deleting the one or more first terms; and
adopting the recompiled language model into the IVA.