Enhanced TextNetTopics for Text Classification Using the G-S-M Approach with Filtered fastText-Based LDA Topics and RF-Based Topic Scoring: faSTNT  
[PERSON]  
1 Computer Engineering Department, AI-Quds University, Jerusalem 20002, Palestine 1,* 1  
The wrapper method uses search strategies to find candidate feature subsets and selects the best subset based on the predictive performance of a pre-chosen classifier. Due to their iterative learning steps and cross-validation, wrapper methods are more computationally expensive than filter methods but are considered more accurate. Examples of wrapper methods include genetic algorithms, sequential feature selection algorithms, and recursive feature elimination algorithms [4]. The embedded method embeds feature selection within the training process of a machine learning model. Random forest is an example of an embedded method. Unlike wrapper methods, embedded methods are computationally less intensive because they do not need to assess feature sets iteratively. They also outperform filter methods by considering interactions with the learning algorithm. However, a limitation is their model-specific nature, which restricts their applicability to other learning models [5]. A recent trend along this line is to employ topic modeling (TM) as an innovative feature reduction approach to avoid the well-known \"curse of dimensionality\". Topic modeling is an unsupervised machine learning procedure for extracting latent variables or abstract topics from large training datasets. Topic modeling reveals concealed patterns in data and provides a meaningful, latent representation of documents. This automated approach facilitates the organization, understanding, and summarization of vast textual data, thereby enriching our understanding of the underlying themes within the dataset [6]. Topic modeling has been widely used in various domains, particularly in text classification, as a method for feature projection. It works by calculating topics as distributions of words and then representing each document as a distribution across these topics. This topic-to-document distribution matrix serves as an input for training document classifiers [7]. Despite its widespread use in document representation, topic modeling in the context of feature selection has only been explored in the existing literature to a limited extent. A pioneering algorithm that fills this gap, drawing inspiration from [8], is TextNetTopics [9], a topic modeling-based feature selection method that is specifically crafted for text classification. It relies on Latent Dirichlet Allocation (LDA) as its core topic modeling technique, which identifies latent topics. These latent topics are composed of words that are semantically related. The algorithm assigns a score to each topic, reflecting its discriminative power based on a machine learning algorithm, and selects the top topics with the highest scores. These selected topics form a subset of distinctive words for binary classification scenarios. Subsequently, this subset of topics is used to train the classifier. A more refined version, TextNetTopics Pro [10], has also been developed explicitly for short-text classification. Both algorithms are based on the Grouping-Scoring-Modeling (G-S-M) approach [11]. For an extensive review of feature selection approaches based on the grouping of features, readers are encouraged to refer to [12,13]. However, TextNetTopics comes with three main shortcomings:  
* Inclusion of Redundant or Irrelevant Features. TextNetTopics selects \(r\) top-ranked topics extracted by Latent Dirichlet Allocation (LDA) to train a machine learning model, where a topic consists of a set of words. However, a significant drawback of this approach is that the selected topics may include redundant or irrelevant features, which increase the size of the final feature set while negatively impacting classification performance. * Computational Burden in Topic Scoring.