ITMT : Interactive Topic Model Trainer (2023)

Abstract: Topic Modeling is a commonly used technique for analyzing unstructured data in various fields, but achieving accurate results and useful models can be challenging, especially for domain experts who lack the knowledge needed to optimize the parameters required by this natural language processing technique. From this perspective, we introduce an Interactive Topic Model Trainer (ITMT) developed within the EU-funded project IntelComp. ITMT is a user-in-the-loop topic modeling tool presented with a graphical user interface that allows the training and curation of different state-of-the-art topic extraction libraries, including some recent neural-based methods, oriented toward the usage by domain experts. This paper reviews ITMT’s functionalities and key implementation aspects in this paper, including a comparison with other tools for topic modeling analysis.

## introduction in the growing information age, today mostly dominated by an unprecedented interest in artificial intelligence (ai), as well as its deployment in a multitude of applications, topic modeling is still mostly preferred over other ai techniques for the automatic extraction of the main themes concurring in a collection of documents. nonetheless, the blind application of these topic extraction tools entails some difficulties, from which we can cite the presence of garbage topics (i.e., topics that describe the corpus under analysis as a whole, but not the relevant topics it consists of); the complicated adjustment of flat topic models when the corpus is characterized by topics with very different sizes, as they do not support hierarchical modeling; or challenges associated with finding a suitable tuning for each algorithm, which requires expertise and a good knowledge of their hyperparameters, just to mention some. moreover, when the knowledge of domain experts is available, it is worthwhile to offer tools that enable the incorporation of such understanding into the building of topic models, providing both appliances for visualization and guided adjustment of the model, specially designed for the usage of non-ai practitioners that are experts in their area. nevertheless, it is essential that the models created for this purpose are easily interpretable by end users, i.e., avoid garbage or too broad topics, etc. hence, we present in this paper intelcomp's interactive topic model trainer (itmt), a tool developed within the h2020 european project in-telcomp1 for this purpose. intelcomp seeks the development of a platform that makes use of the latest generation of artificial intelligence and natural language processing (nlp) tools to provide relevant information to assist public policies in science, technology, and innovation (sti), geared toward aiding decision-making over the policy cycle. this requires a thorough analysis of documentary sources, which can entail up to hundreds of millions of documents (e.g., scientific articles, patents, etc.); therefore, here becomes fundamental the use of topic modeling to extract information with a level of detail greater than attainable through the inspection of these sources' metadata. itmt consists of a python-based toolbox integrated within a pyqt6-based graphical user inter-face2 for the training of topic models following an expert-in-the-loop approach that ultimately contributes to models that are more aligned with the prior experience and needs of intelcomp end users. the software package includes several state-of-theart topic modeling solutions, seeking adequacy to the needs of each possible scenario, due to both the characteristics of the data sets and the scalability of the algorithms, but also the infrastructure available for training. besides, the software contains a series of proprietary algorithmic improvements that allow features itmt gensim mallet standfordtmt sttm familia topicnet tomodapi octis bigml pre-processing tools bayesian based models neural topic models level-2 htms for granularity inspection topic visualization tools topic curation tools topic annotation tools hyper-parameters tuning global topic statistics coherence metrics entropy metrics diversity metrics significance metrics classification metrics domain-experts oriented compared to other current topic modeling frameworks @xcite @xcite , which typically focus on putting out topic modeling algorithms but ignore their interpretability and adequateness for the needs of end users, itmt stands out as a tool for training topic models while including the knowledge of experts in the creation and curation of such models. a comparison summary between itmt and other available frameworks is available in the main contributions of our framework's current release are: itmt has been published under a permissive mit license in the github project https:// github.com/intelcomph2020/topicmodeler . ## system overview itmt consists both of a pyqt6-based graphical user interface (gui) as the front-end and a back-end service supporting all the operations that need to be carried out as a response to user interactions. the visualization itself and the actual training and optimization of the models are completely decoupled. the state management is performed on the back-end side, and it is sustained by the use of external folders given as input to the application, as described in section 2.1. this provides the itmt with both persistence and portability capabilities, as all structures (training datasets, models, etc.) created during the application's execution can be accessed and modified at a later time. ## software components as we have covered in the former section, the construction of a topic model is a procedure that requires the sequential execution of various tasks, each of them managed by a different component of the topic modeling service underlying the gui. we present in this section each of these components. ## existing frameworks probably the most widely used topic modeling frameworks are gensim @xcite and java-based package mallet @xcite , which include implementations of a handful of popular btms. they also provide pre-processing pipelines, hyper-parameters optimization, and the calculation of some coherence metrics. in the same direction, the stanford tmt @xcite is a set of topic modeling tools, including, inter alia, features such as the training of btms, the selection of parameters via a data-driven process, and the manipulation of texts from different spreadsheets. also released as a java framework, sttm @xcite focuses on the integration of short text topic modeling algorithms, but it includes as well some long-text implementations and evaluation metrics. in @xcite the authors proposed a configurable framework named familia that performs automatic parameter inference for a variety of topic models and supports the design of new topic models to best suit specific problems at hand. aiming to bring additive regularization for topic modeling (artm) @xcite ; @xcite accessible for the general public, victor @xcite proposed topicnet, a python module including a modular approach to topic model training and several visualization techniques, as well as semi-automated model selection and support for user-defined goal metrics. other state-of-the-art frameworks include to-modapi @xcite , a python-based api for the training, inference, and evaluation of different topic models; and octis @xcite , also a python-based framework + dashboard for the training of topic models, which additionally supports its analysis and comparison over several datasets and evaluation metrics, besides a bayesian-based hyperparameters optimization strategy. lastly, bigml is a general tool for machine learning, that incorporates some topic modeling functionalities4, including an optimized implementation of lda for any text in seven languages, with preprocessing, training, inference, and visualization of models in a user-friendly dashboard, as well as the possibility of creating, configuring, and updating topic models programmatically via the bigml api and bindings. nonetheless, from the latter, only tomodapi and octis support the training of ntms. moreover, none of them allow the actual incorporation of knowledge expertise into the model building, nor allow for a thematic analysis with different levels of resolution. hence, itmt excels as an expert-in-the-loop oriented tool for the training at different resolution levels, curation, evaluation, and visualization of both btms and ntms. ## conclusions in this paper, we have presented intelcomp's interactive topic model trainer (itmt), a python-based tool that includes implementations of several stateof-the-art topic modeling algorithms orientated towards the usage of domain experts and a novelty implementation of second-level hierarchical topic models for granularity exploration. moreover, the framework is provided with a set of tools for the evaluation, visualization, annotation, and curation of topic models, and a preprocessing pipeline. for future work, we are active in offering each of the itmt's components as a docker container to transform the gui into a web service. tion program under grant agreement no 101004870, and from grant ted2021-132366b-i00 funded by mcin/aei/10.13039/501100011033 and by the "european union nextgenerationeu/prtr".