# Method of Feature Reduction in Short Text Classification Based on Feature Clustering
## 1 Introduction  
Communication on the internet is increasingly frequent, resulting in a considerable amount of information data. Most of these data are short texts, such as microblogs and BBSs (Bulletin Board Systems). The short text classification has encountered new challenges due to their limited length, the prevalence of internet vocabulary, abbreviations, and intensifying synonyms [1, 2]. Therefore, one research hotspot in natural language processing is precisely classifying these short texts and effectively analyzing their meanings. The early work of short text classification involves converting texts into data representation, a form that computers can process, which is crucial in text classification and can directly affect classification [3]. A basic representation of a text is bag of words in which each text is signified as a vector of words. In traditional short text classification tasks, a VSM (vector space model) or a word embedding model usually represent text vectors. However, the following problems exist when applying these methods to short text classification: (1) The spatial distance between words is not considered, and dimensional disasters are easily triggered when constructing text vectors by the VSM and TF-IDF (term frequency-inverse document frequency). The VSM only pays attention to statistical information, such as word frequency. Moreover, it ignores the influence of other factors, such as synonyms on text similarity, thus reducing the accuracy of short text classification [4, 5, 6]; (2) By modeling the context and semantic relations of words, the word embedding model maps words to an abstract low-dimensional real space and generates a corresponding word vector. This is an effective way ofconstructing a semantic association of words [7; 8; 9]. However, in the task of short text classification, the improvement of classification accuracy is limited when applying the word embedding model alone, primarily because a short text is concise and it contains many polysemous words, noise words, and internet words. The absence of effective features makes it difficult to extract enough semantic information [10]. Feature reduction can solve the above problems. Feature reduction removes irrelevant features and extracts a feature subset without reducing the classification accuracy [11]. It is a key preprocessing step that can improve classification accuracy and reduce computation. In general, there are two ways to achieve feature reduction [12]. One approach is to only select the most important subset of features in a category and ignore the rest. However, applying this path alone does not take advantage of the relationships between features that are captured from external knowledge. Another feature reduction method is to cluster the related features and then generate a compact feature set based on the original features. This type of feature reduction builds relationships between features [13; 14; 15]. Feature clustering can, to a certain extent, achieve the semantic extension of texts. Its purpose is to enrich the semantic representation of texts without reducing the classification accuracy [16; 17]. Semantic expansion changes the expression and dimension of feature vectors via auxiliary information, such as an external corpus [18], context [16], characters [19], core words [20], and attention [21], so as to improve the accuracy of text classification. There are two common routes of extending text semantics. The first way is to build an external knowledge base to enrich the semantic representation of short texts. Researchers have enriched the short texts by employing external resources, such as WordNet [22] and Wikipedia [18; 23; 24]. These improvements involve complex natural language processing for syntactic analysis and complex semantics and demand a mass of external data, which is rare for short text classification. The second method is to optimize the feature representation by, for example, improved word embedding [25] or feature clustering [26].