# Integrating Text Classification into Topic Discovery Using Semantic Embedding Models
## 1 Introduction  
_Natural language_ is the mechanism that a human being uses to communicate and transmit an idea, opinion, or feeling [1]. Understanding natural language is a complex task and requires time because millions of connections between neurons are necessary to learn it. However, a computer needs structure and logic to understand a programming or natural language. Therefore, a mathematical formula or a predefined pattern will be necessary for the computer to learn the required knowledge [1]. For a computer to recognize the data it receives, it must generate an adequate numerical representation. Therefore, _natural language processing_ (_NLP_) is an area that is in constant development; it seeks to generate efficient algorithms for a computer to understand the spontaneous language of a human being. Some of the characteristics of natural language involve strictrules, which facilitate their computerized analysis [1]. Hence, _NLP_ encompasses techniques and tools for developing systems that can interpret and utilize _natural language_ to perform desired tasks, such as news classification or spam identification [1]. Mechanisms, such as the _extraction of semantic relationships_, _named entity recognition_, _topic discovery_, and _word embedding_, are essential to provide a computer with the necessary knowledge to process information and display inevitable results. Therefore, _topic discovery_ is defined as the task of finding specific topics present in a set of documents as input. This task can be applied to any text [2]. The purpose is to identify, without needing a dictionary, the main themes that implicitly exist within a collection of texts [2]. For a computer to understand _natural language_, it is necessary to create vectors of numbers. The embedding vectors or patterns may be subject to operations, such as addition, subtraction, and distance measurements. The literature shows that some word-embedding models are based on neural networks or context matrices [3]. The advancement of technology has made it possible to streamline processes, such as:  
1. Searching for the subject of a document. 2. Searching for a specific document. 3. Generating a summary or extracting key phrases from a text. In the literature, some _word embedding_ models are _word2 vec_, _glove_, _fastText_, and _BERT_. The embedding or word embedding model came to fruition in 2013 when [PERSON] and his team at Google developed the first embedding model, named _word2 vec_. The model has the following sub-models:  
1. Continuous bag-of-words (_CBOW_[4]): receives a context and predicts a target word [4]. 2. Skip-gram [5]: each word is represented as a bag of \(n\)-grams of [6] characters. The _GloVe_ embedding model was developed in 2014 by [PERSON] [7]. This model combines the advantages of the two main family models in the literature: _global matrix factorization_ and _local context window_. _GloVe_ works with non-zero elements in a word-word co-occurrence matrix rather than the entire sparse matrix or separate context windows in a large corpus [7]. On the other hand, in 2015, _Facebook_ researchers created the embedding model called _fastText_. The _fastText_ model has pre-trained models for 294 languages. The authors relied on the _skip-gram_[8] model. In 2018, _BERT_ (_bidirectional encoder representations from transformers_) was designed to pre-train deep bidirectional representations from the unlabeled text by jointly conditioning left and right contexts in all layers [9]. In [3], the authors applied GloVe and _fastText_ for the text classification of two text corpora, and compared the results that were obtained with a semantics embedding model. Hence, _text classification_ involves processing data without a person's intervention. The computer must have access to the knowledge necessary to carry out tasks such as medical diagnoses, analysis of social networks, or the search for fake news.