# Mining the Frequent Patterns of Named Entities for Long Document Classification
## 1 Introduction  
Text classification is a classic topic in the field of natural language processing. With the development of information technology, a large amount of document-level text data have emerged on different topics, such as news articles [1], patent documents [2], e-commerce [3], construction specification [4], and medical narratives [5]. Effective classification of those long documents is beneficial to the efficient acquisition of target information. Differently from short texts, long documents have more characters and contain more topically irrelevant information, which usually affects text classification tasks. The core information in the document's text is often hidden in several keywords or sentences, as shown in Figure 1. Hence, how to excavate this information is a challenging problem for long document classification. Current mainstream methods of text classification can be divided into two categories, i.e, machine learning-based methods and deep learning-based methods. Both of them generally follow a 2-stage paradigm: Firstly, converting text data to vectors which contain the key information of text, and secondly feeding them into the classification model. How to construct a vector containing the key features of a long document is a key step to improving classification performance. For machine learning-based methods, statistics-based text representation methods are the most commonly used, such as one-hot [6], bag-of-words [7], and n-grams with TF-IDF [8]. These methods are simple and effective, with low computational costs. However, they often suffer from data sparsity, have dimensional disasters, and ignore the semantic information, especially when the text is long and containsa lot of information. Deep learning-based methods typically employ a neural network-based approach, such as word vector, which transforms the text into a low-dimensional vector. In addition, neural network models can capture more abstract semantic features between words and sentences. However, the neural network methods usually need a large scale dataset to achieve good performance, and their time consumption is significant. Besides, they are generally considered black-box models, so they are not well interpretable. Motivated by a novel software test case recommendation method [9], which utilizes a correlation identification method to identify and recommend the proper test cases in the test case knowledge graph, we demonstrate a long document classification model, MIPELD. Our work aims to mine the core features and find the keywords in a long document. Specifically, we first apply the named entity recognition pattern to the text in preprocessing, replacing the specific name of each entity in the text with a unified entity label. This process can enhance the robustness of the extracted features for generalization among various documents. Different from [9], we adapt a distance constraint function adapted to the Apriori algorithm to mine the frequent itemsets of words which contain local features hidden in the key sentences. Furthermore, all the words and frequent itemsets are further screened as the feature words under the direction of the information gain metrics calculated using the \"one vs. rest\" method since the document types in long document classification tasks are pretty numerous. Subsequently, based on the selected words and frequent itemsets, all the text is converted into a discrete vector by bag-of-words. Finally, we employ a simple and effective model, a Naive Bayesian model, as the classification model. In addition, according to the selected key frequent itemsets and the feature weight of the Naive Bayesian, MIPELD provides the key sentences of each text, which can improve the interpretability of the classification results.