Patent ID: 11914966
Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
Field: Computer technology (Electrical engineering)
Classification: CPC G  H | IPC G  H

Claim 8:
9. A method for generating a topic model for use in natural language processing, the method comprising:
processing, via a processor, a set of documents to generate training data, wherein each document in the set of documents is associated with one or more users and one or more terms are identified from each of the documents based at least upon occurrence frequency;
performing an iterative topic model generation routine for a plurality of iterations, each of the iterations generating another topic model, the performing comprising:
including an initial number of topics in a first topic model in a first initial iteration of the plurality of iterations,
evaluating in each of the plurality of iterations, each specific topic of the topics of the particular topic model to determine when the specific topic is classified as user referenced,
wherein determining whether the specific topic is user referenced comprises calculating for each specific user of the one or more users a topic distribution value that is based on the frequency of the terms appearing in the specific user's associated documents that correspond to the specific topic, and the topic distribution values for the one or more users are collectively compared to a first threshold to identify whether the specific topic is classified as user referenced or not,
evaluating in each of the plurality of iterations a user-referenced criterion for a current topic model, the user-referenced criterion comprising an evaluation score comprising a number of the classified user referenced topics in the current topic model divided by a total number of topics in the current topic model, and
checking whether the user-referenced criterion is identified to not satisfy a second threshold and terminating the iterative topic model generation routine, otherwise, when the user-referenced criterion of the current topic model satisfies the second threshold, modifying the particular topic model by adding a prespecified step size amount of topics to the included topics within the particular topic model to generate another topic model and repeating the evaluation and checking steps in the iterative topic model generation routine; and

upon termination, identifying the particular topic model that was most recently generated as the final topic model and store the final topic model to be used in natural language processing.