# Mining the Frequent Patterns of Named Entities for Long Document Classification
## 2 Related Work
### Machine Learning-Based Methods  
With the development of machine learning, text classification started to utilize machine learning methods, such as Naive Bayesian (NB) [10; 11; 12; 13; 14], support vector machines  
Figure 1: An exampleof a long document’s key sentences. These key sentences can be very effective in helping people to classify the document.  
(SVM) [15; 16; 17; 18; 19; 20], k-nearest neighbors (KNN) [21; 22; 23; 24; 25; 26], decision trees (DT) [27; 28; 29], and random forest (RF) [30; 31; 32; 33].  
For an NB classifier, [PERSON] and [PERSON] [13] described the differences and details between a Bayesian network and a polynomial model which made the Naive Bayesian assumptions, and compared their classification performances on five text corpora. [PERSON] et al. [12] proposed some effective techniques for Naive Bayesian text classification, which use two empirical heuristics: per-document text normalization and a feature weighting method. [PERSON] et al. [14] proposed an improved Bayes method for Chinese information classification, which applies TF-IDF into its conditional probability and applies grade factor feature weights into the classification formula of Naive Bayes.  
For an SVM classifier, [PERSON] et al. [19] proposed an optimal SVM-based text classification algorithm, using multiple optimal strategies, such as the weight definition, the entropy weighting scheme for feature selection, and optimal parameter settings. [PERSON] et al. [20] reported the active learning approach, employing SVM to select a set of documents to reduce the labeling effort.  
For a KNN classifier, [PERSON] et al. [24] proposed a weight adjusted k-nearest neighbor (WAKNN) classification model, which gains feature weights with a greedy hill climbing method. [PERSON] et al. [25] presented an improved KNN model for text categorization that combines a constrained one-pass clustering algorithm and KNN text classification. [PERSON] et al. [26] presented a method of representative sample optimization based on the CURE algorithm, and proposed a quick QKNN (Quick k-nearest neighbor) on the basis of gaining the k-nearest neighbor samples.  
For DT and RF classifier, [PERSON] et al. [32] proposed an improved random forest algorithm for text classification and developed a novel feature weighting method and tree selection method to make the random forest model well suited to document categorization, using dozens of topics. The extreme gradient boosting (XGBoost) [34] algorithm proposed by [PERSON] et al. an the improvement on a gradient boosting decision tree (GBDT) algorithm, which has the advantages of regularization, parallelization, and flexibility; and its performance is remarkable. The light gradient boosting machine [35] (LGB) is another improvement on the GBDT algorithm, which has several advantages, such as high accuracy and fast training speed. The decision trees in LightGBM are grown leaf-wise, instead of checking all the previous leaves for each new leaf.  
These methods have performed well in text classification and showed robustness, so many researchers still use them in many fields, such as spam filtering [36] and information retrieval [37]. However, it is worth noting that machine learning-based models cannot extract features directly from text data, so they generally incorporate text representation approaches, such as bag-of-words (BOWS) [7] with n-grams [38]. Such traditional statistical-based representation approaches only employ a word frequency method to get the text representations, without considering the semantic relationships between words, which leads to the problem of data sparsity and dimensionality.