# Semantic-Based Classification of Long Texts on Higher Education in China

Authors: Chun Li; Yanying Fei
Journal: Discrete Dynamics in Nature and Society
 (2021)

---

## Abstract

The development level of higher education (HE) is an important indicator of the development level and development potential of a country. The HE-related document is the mirror to reflect the develop process of the HE. The research of high education (HE) has been developing rapidly in China, resulting in a huge number of texts, such as relevant policies, speech drafts, and yearbooks. The traditional manual classification of HE texts is inefficient and unable to deal with the huge number of HE texts. Besides, the effect of direct classification is rather poor because HE texts tend to be long and exist as an imbalanced dataset. To solve these problems, this paper improves the convolutional neural network (CNN) into the HE-CNN classification model for HE texts. Firstly, Chinese HE policies, speech drafts, and yearbooks (1979–2020) were downloaded from the official website of Chinese Ministry of Education. In total, 463 files were collected and divided into four classes, namely, definition, task, method, and effect evaluation. To handle the huge number of HE texts, the Twitter-latent Dirichlet allocation (LDA) topic model was employed to extract word frequency and critical information, such as age and author, enhancing the training effect of CNN. To address the dataset imbalance problem, CNN parameters were optimized repeatedly through comparative experiments, which further improve the training effect. Finally, the proposed HE-CNN model was found more effective and accurate than other classification models.


---

## Body

## 1. Introduction

Higher education (HE) is a crucial part of the country education system [1] and the foundation of national talent training. In recent years, with the continuous development of China’s national power, higher education has also developed rapidly and the HE serves the national development. Therefore, through semantic analysis of higher education-related documents, the development trend of higher education can be discovered, and future planning and research can be carried out.The HE has been developing rapidly in China, resulting in a huge number of texts, such as relevant policies, speech drafts, and yearbooks. The traditional research almost uses the manual method (some statistic methods) in analysis of higher education-related data in the field of humanities and social sciences. However, with the development of HE research, the semantic-based HE document analysis model should be used in future HE research aspect to solve the problem of HE texts, such as inefficient and hard to process. What is worse, there is no classification model for HE files data in any academic database. In order to solve this problem, this paper intends to develop a semantic-based classification model for HE texts.In 2020s, the social networking, which uses the semantic-based text analysis, becomes a hot topic in the field of computer science [2]. Various accurate text mining models have emerged, including convolutional neural network (CNN) and long short-term memory (LSTM) model [3–12]. However, these traditional classification models cannot be directly used for training and applying directly to process the HE files because the HE files are much longer and richer in contents, and the whole HE dataset is more imbalanced than common social network texts (e.g., Tweets) [13].To address the above problems, this paper firstly sets up a standard Chinese HE dataset, using a Python crawler. The dataset includes the policies, yearbooks, and speed drafts on HE in 1979–2020. In total, there are 466 files in the dataset. Every text file is long text style, which contain thousands of words at least. So, although the number of text file is not too large, it is still hard to build the text analysis model. The code and data will be uploaded to GitHub in future. Next, the CNN was extended into an HE classification model called HE-CNN. (1) The huge number of HE files makes it hard to train the classification model. To solve this problem, the Twitter-latent Dirichlet allocation (LDA) topic model was adopted to extract and compress text data, convert long texts into short texts, and then compress the texts, without sacrificing the critical contents. (2) To solve the dataset imbalance problem, the CNN training effect was improved with a mixture of texts and special attributes (e.g., period and high-frequency words). In addition, the parameters of the HE-CNN model were optimized experimentally through cross validation, making the classification more accurate and training more efficient. Experimental results show that the optimized model strikes a good balance between accuracy and efficiency, compared with unoptimized classification models. The main contributions of this research are as follows:(1)
The CNN classification model was improved into the HE-CNN classification for HE files. The proposed model handles the long texts in HE files through keyword extraction and overcomes the dataset imbalance problem by expanding the training set with text attributes. Moreover, the model parameters were tuned to balance training time with classification accuracy, thereby improving the model training effect. This is unachievable with traditional CNN.(2)
A standard Chinese HE dataset of 466 files, including speech drafts, policies, and yearbooks, was established, laying a solid data basis for future attempts of HE classification.


## 2. Literature Review

For the semantic analysis problem, there are some works which have achieved great successes results in processing the short social network text (Twitter and Weibo). For instance, Yue et al. [14] designed a classification model for short texts like tweets and tax invoices. Relying on Chinese knowledge graph, their model solves the sparsity of data labels and facilitates model training. Gultepe et al. [15] presented a CNN-based simple classifier for text files: a new CNN architecture is adopted to utilize locally trained latent semantic analysis (LSA) word vectors. Qiu et al. [16] proposed a multichannel semantic synthesis CNN (SFCNN). To complete the task of emotion classification, the emotional weights of word vectors are determined through multichannel semantic synthesis, and the model parameters are optimized through gradient descent with an adaptive learning rate. Shi and Zhao [17] developed a semantic classifier based on a neural network. The theoretical findings of the axiom fuzzy set theory are incorporated to the neural network, and complex concepts are extracted by the neural network to enhance classification accuracy. Wang et al. [18] put forward a dual feature training support vector machine (SVM) model to classify texts and images. Wu et al. [19] proposed a deep semantic matching model, which fine-tunes CNN parameters through the generation of candidate entities. Yang et al. [20] came up with two semantic-based Chinese file classification strategies: the ambiguity problem is solved by a novel semantic similarity calculation (SSC) method and the problem of synonyms is overcome through a robust correlation analysis method (SCM).Apart from semantic-based text classification, the text mining model becomes a new hotspot in the field of education. For instance, Chen et al. [21] classified questionnaire data with a semantic analysis model to solve the educational backwardness of ethnic minorities. Goncalves et al. [22] used a semantic model to evaluate massive open online courses (MOOC) and improved the course quality against the classification results. Niu et al. [23] proposed a novel theory of semantic cohesion for Chinese airworthiness regulations and specified four critical elements of the theory, namely, definition, model, theorem, and rules. Koutsomitropoulos et al. [24] combined explicit knowledge graph representations with vector-based learning of formal thesaurus terms into a hybrid semantic classification model and demonstrated the good effect of the hybrid model on the classification of biological files in library terminology learning. With the aid of the enhanced learning model, Shen and Ho [25] evaluated HE teaching effects and quickly grasped the development state of HE.In summary, semantic-based text analysis is relatively mature in computer science but remains in the early stage in the field of HE. Therefore, this paper aims to improve semantic-based analysis on HE text classification.


## 3. Data Collection and Preprocessing

Our data fall into three categories: yearbooks, speech drafts, and policies. The original data were mainly downloaded from the open datasets provided on the official website of Chinese Ministry of Education (https://www.moe.gov.cn/jyb_sjzl/moe_364/zgjynj_2015/), using a self-designed Python crawler. The HyperText Markup Language (HTML) tags were removed to generate the final experimental dataset. In total, the dataset contains 466 HE-related files of speech drafts and policies and 331 HE yearbook files, all of which were released between 1988 and 2019 (Figures 1 and 2).Figure 1
Data on HE speech drafts and policies.Figure 2
Data on HE yearbooks.There are two primary attributes in the data: period and high-frequency words. The period words are the label that can reflect the key words changed with the time, and high-frequency words can partially summarize the main topic of the document.The two attributes were adopted to enhance the classification accuracy of our semantic-based model. The entire experimental dataset is illustrated in Table1.Table 1
The entire experimental dataset.
DescriptionDefinitionTaskMethodEffect evaluationComprehensive experimentTraining set160156151171Validation set40443543Test set40444630File feature extraction797 texts


## 4. Text Attributes and Model Framework

This section introduces the overall framework of the proposed HE-CNN model. As its name suggests, our model consists of two parts: one is the traditional CNN and the other is the attributes of HE files (i.e., year and high-frequency words). The framework of the proposed model is shown in Figure3.Figure 3
Framework of HE-CNN.


### 4.1. Text Mining Model

In 2014, Kim [26] proposed the CNN model for text classification. Their model converts the original text into multidimensional vectors for further analysis and achieves relatively good classification results. However, the model requires a huge amount of training data and does poorly in fault tolerance. In recent years, several novel methods have been introduced to optimize the CNN model. For example, Zhang et al. [27] proposed a three-way model that improves the classification accuracy of CNN for emotional texts. Yang et al. [28] employed a multichannel SFCNN to overcome the emotional ambiguity caused by the changing text context.In this paper, 100 convolutional filters (size: 1) are adopted to process text vectors, each text is segmented into words by Python Jieba, and max pooling is performed to analyze the output, which is cascaded from the results of all filters. The text representation model is shown in Figure4.Figure 4
Text representation model.


### 4.2. File Feature Extraction

This paper relies on the unique features of each HE file (i.e., publish year and main topic) to improve HE-CNN training performance and overcome dataset imbalance.The publish year was selected for two reasons. (1) The HE development in China is greatly affected by government policy. The HE files usually reflect the concept of governance. In the 1990s, HE mainly emphasized on vocational education, a booster of industrialization. In the 2000s, the focus of HE gradually shifted towards science and technology. The shift is a response to China’s requirements on HE in different periods. (2) The publish year is readily available in the files. Here, the attribute of publish year is divided by decade into the 1980s, the 1990s, the 2000s, and the 2010s.The file features cannot be directly extracted from the files. Thus, the semisupervised Twitter-LDA topic model [29] was selected to extract the topic from each file. In this paper, in order to highly generalize the document, the first five high-frequency keywords are chosen as the topic words to describe the file. Once extracted, the eigenvector of each file was taken as the topic vector, and the topic vectors of all files were merged and divided by publishing year to facilitate model training.Finally, a soft layer was added to HE-CNN to output the result. All layers of our model were processed by a normalization algorithm, such that the parameters between different layers can be dynamically adjusted with the training data. The model parameters are listed in Table2.Table 2
Model parameters.
DescriptionValueWord representationStatic 300-dimensional word2vecSize of convolution kernel1Number of convolution kernels100Pooling1-max poolingDropout0.5


## 4.1. Text Mining Model

In 2014, Kim [26] proposed the CNN model for text classification. Their model converts the original text into multidimensional vectors for further analysis and achieves relatively good classification results. However, the model requires a huge amount of training data and does poorly in fault tolerance. In recent years, several novel methods have been introduced to optimize the CNN model. For example, Zhang et al. [27] proposed a three-way model that improves the classification accuracy of CNN for emotional texts. Yang et al. [28] employed a multichannel SFCNN to overcome the emotional ambiguity caused by the changing text context.In this paper, 100 convolutional filters (size: 1) are adopted to process text vectors, each text is segmented into words by Python Jieba, and max pooling is performed to analyze the output, which is cascaded from the results of all filters. The text representation model is shown in Figure4.Figure 4
Text representation model.


## 4.2. File Feature Extraction

This paper relies on the unique features of each HE file (i.e., publish year and main topic) to improve HE-CNN training performance and overcome dataset imbalance.The publish year was selected for two reasons. (1) The HE development in China is greatly affected by government policy. The HE files usually reflect the concept of governance. In the 1990s, HE mainly emphasized on vocational education, a booster of industrialization. In the 2000s, the focus of HE gradually shifted towards science and technology. The shift is a response to China’s requirements on HE in different periods. (2) The publish year is readily available in the files. Here, the attribute of publish year is divided by decade into the 1980s, the 1990s, the 2000s, and the 2010s.The file features cannot be directly extracted from the files. Thus, the semisupervised Twitter-LDA topic model [29] was selected to extract the topic from each file. In this paper, in order to highly generalize the document, the first five high-frequency keywords are chosen as the topic words to describe the file. Once extracted, the eigenvector of each file was taken as the topic vector, and the topic vectors of all files were merged and divided by publishing year to facilitate model training.Finally, a soft layer was added to HE-CNN to output the result. All layers of our model were processed by a normalization algorithm, such that the parameters between different layers can be dynamically adjusted with the training data. The model parameters are listed in Table2.Table 2
Model parameters.
DescriptionValueWord representationStatic 300-dimensional word2vecSize of convolution kernel1Number of convolution kernels100Pooling1-max poolingDropout0.5


## 5. Experimental Results

This section mainly reports the experimental results on our model. Firstly, the experimental results and effects are measured by the low function as follows:(1)L=−∑i=1Nyilogpi,where N is the number of semantic classes; pi is the value of the i-th output vector; and yi is the ground truth.


### 5.1. Parameter Configuration

Before any experiment, HE-CNN parameters must be initialized and optimized, for classification accuracy hinges on feature representation. In this paper, the model parameters are optimized by the word2vec model. The input layer of our model was trained separately in multiple modes, namely, 100-dimensional, 200-dimensional, 300-dimensional, and 400-dimensional vectors, using the text data on yearbooks, speech drafts, and policies. The training results indicate that the 300-dimensional experiment had the best effect. Hence, the 300-dimensional vector was taken as the parameter of the input layer.For the convolution layer, 100 kernels were selected after multiple experiments. Once the number of kernels surpassed 100 and continued to grow, the classification accuracy did not increase, but the training time surged up (Table3).Table 3
Classification performance at different numbers of kernels.
Number of kernelsAccuracy (%)Time cost (s)1069.7019452072.4620874075.5521586078.7822748080.28239910082.15245812082.22273214082.463058For the pooling layer, 1-max pooling achieved the best performance. The dropout rate had a small effect on the model and was thus set to 0.5 for our experiments. Table2 lists the model parameters for experiments. In addition, the LSTM model was also adopted to extract text features in our experiments.


### 5.2. Attribute Extraction

The feature distribution of each class was extracted from all 463 files in four periods. Firstly, the Twitter-LDA topic model was employed to extract the distribution of class preferences. Focusing on all the files published in a period, the model holds that the features of each class are reflected by high-frequency words. After adjusting the number of topicsk to 6, each of the six topics was labeled (i.e., education, development, construction, reform, school, and party). Under the six topics, 30 keywords (translated literally into English) were sorted by frequency. The results in Table 4 suggest that these keywords are reasonably grouped under corresponding topics.Table 4
Keyword distribution of each topic.
TopicResultsEducationCountry0.00008Important0.00315Training0.00979Employment0.00475Focus0.00507Society0.0013Question0.0044Economy0.0036Management0.0057National0.0074Student0.0019Research0.0098Science0.0012Quality0.007Need0.0046Innovation0.005Teacher0.0094Thought0.0034Morality0.0067Government0.0041Our country0.0097Rural0.0035China0.0058Culture0.0077Mechanism0.0043Improvement0.0033Talent0.0016System0.0052Services0.0092Level0.0097DevelopmentEducation0.0092Improve0.00248Economic0.00875Implement0.0043Focus0.00462Construction0.00103Cause0.0005Socialism0.00778Talent0.00313Quality0.0056Reform0.00665Problem0.00225Innovation0.00786Level0.00105Service0.00393Society0.00905China0.00509Science0.00759Need0.00979Student0.0094Country0.00207School0.00584Bring up0.00378The study0.00544System0.00425Our country0.00222Promote0.00988Colleges0.00703Features0.00787People0.00238ConstructionStrategy0.00131Must0.0053Advance0.00721Postgraduate0.00128Modernization0.00528Cooperation0.00693Science and technology0.00297Comprehensive0.00613Communication0.00174Our country0.00724Strengthen0.00388Cadre0.00871Xi Jinping0.00405Philosophy0.00396Innovation0.00489Organization0.00868Intellectuals0.00023System0.00395Social science0.00688Reform0.00095Nation0.00332Produce0.00461Accelerate0.00468Theory0.00054Development0.00593Nationwide0.00231System reform0.00795General secretary0.00868Socialism0.00531Thought0.00652ReformNationwide0.00339Society0.00578Thought0.00154People0.00273Colleges0.00335Education0.002Modernization0.00696Practice0.00969Service0.00453Profession0.00165China0.00102Construction0.00083Problem0.00794Political0.00394System0.00453Features0.00597Our country0.00638Science0.00354Central0.00588School0.00419Socialism0.00719Innovation0.00819Leadership0.00215Solve0.00775All levels0.009Cause0.00865Development of0.00546School0.00532Management0.00889The study0.00325SchoolBring up0.0097Thought0.00096Our country0.00225Modernization0.00785Improve0.00428Cause0.00642People0.00533Leadership0.00675School0.00135Central0.00076China0.00047Development of0.00988Country0.00112Implement0.00184Political0.0027Culture0.00694Socialism0.00953System0.00851Government0.00617All levels0.00372Practice0.0029Education0.00374Features0.00566Society0.00686Reform0.00363Promote0.0015Problem0.00264Talent0.00098Student0.00866The study0.00068PartyImportant0.00625Cause0.00326Leadership0.00747Solve0.00217Practice0.00675Employment0.00312Implement0.00959Promote0.00786Teaching0.00399Enterprise0.00369Mechanism0.00897School0.00547Technology0.00651Technology0.0028International0.00578Colleges0.00245Political0.00667Plan0.0003Central0.00668Major0.00833Profession0.00418People0.00536Modernization0.00724Department0.00222World0.00601Socialism0.00844Area0.00169All levels0.00835Features0.00498Engineering0.00913To make each topic more intuitive and facilitate the analysis of period distribution, the topic word distribution was fixed, regardless of periods, to analyze preference distribution. As shown in Figure5, the preference distribution across periods varied greatly from topic to topic. For example, Figure 6 displays the preference distribution on various topics in the 2010s. The preference distribution was adopted as the vectorized representation of a file feature and combined with publish year (four dimensions: 1980s, 1990s, 2000s, and 2010s) and high-frequency words (five dimensions) of the files. In this way, a 20-dimensional feature was obtained to depict file attributes.Figure 5
Preference distribution of the six topics across periods.Figure 6
Preference distribution in the 2010s.


### 5.3. Comparative Experiments

Random cross validation was implemented in model training. Specifically, 25% of the training data were used as the cross-validation set, and 40% standard files were used as the validation set for each validation. The fusion model was trained for 200 generations, and each trained model was validated for 204 iterations. After a total of 40,800 iterations, the stability of the fusion model was evaluated against the test set by(2)Acc=∑i=1NTP+TN/TP+TN+FP+FNN,where N is the number of semantic classes; TP is the true positive; TN is the true negative; FP is the false positive; and FN is the false negative.For comparison, our model was contrasted with CNN, decision tree (DT), Naïve Bayes (NB), k-nearest neighbors (KNN), random forest (RF), multilayer perceptron (MLP), SVM, and logistics regression (LR). The results (Table5) show that our model far outperformed these traditional classification models on the same HE dataset.Table 5
Classification performance of different models on our HE dataset.
ModelAccuracy (%)DT72.746NB73.856KNN60.555RF52.568MLP34.583SVM61.734LR72.786CNN78.56HE-CNN83.516As mentioned before, HE files are generally composed of a few long texts, each of which is very large. To ensure the sufficiency of training, standard text dataset and text features were combined in the training set. The performance of different classification models on the HE dataset with text features is compared in Table6. It can be seen that for the long text style document, the CNN model obtains a better result than other models. In addition, using the extracted text features can significantly improved the performance of the HE-CNN model than traditional CNN models.Table 6
Classification performance of different models on our HE dataset with text features.
ModelAccuracy (%)DT77.424NB75.824KNN64.236RF54.358MLP33.528SVM68.157LR75.856CNN81.356HE-CNN88.782Furthermore, many combinations of text-feature presentation dimensions were tested to optimize the CNN parameters. In order to keep the balance between the training speed and model accuracy, the optimal experiment is processed and the result is shown in Table7. The results in Table 7 suggest that the number of the text-feature dimension is 300 which can get the balance.Table 7
Accuracy of different presentation dimensions of the text features.
ModelAccuracy (%)Text-10046.213Text 100-fusion79.745Text 20069.532Text 200-fusion79.212Text 30076.332Text 300-fusion82.345Text 40069.712Text 400-fusion78.662In addition, the experiment also uses the text-feature LSTM model to extract the text features which could increase the number of the text features. However, the experiment reflected that the extracted futures using text-feature LSTM model cannot be significantly used for improving the accuracy and indicated that the LSTM model is not suitable for processing long texts. The final experimental results are given in Table8. Apparently, our HE-CNN is applicable to semantic-based HE text analysis and is fast in dividing the texts into different classes.Table 8
Final experimental results.
ModelAccuracy (%)Text CNN73.323Text LSTM66.755Feature CNN31.875Text-feature CNN77.587Text-feature LSTM67.976Our model88.782


## 5.1. Parameter Configuration

Before any experiment, HE-CNN parameters must be initialized and optimized, for classification accuracy hinges on feature representation. In this paper, the model parameters are optimized by the word2vec model. The input layer of our model was trained separately in multiple modes, namely, 100-dimensional, 200-dimensional, 300-dimensional, and 400-dimensional vectors, using the text data on yearbooks, speech drafts, and policies. The training results indicate that the 300-dimensional experiment had the best effect. Hence, the 300-dimensional vector was taken as the parameter of the input layer.For the convolution layer, 100 kernels were selected after multiple experiments. Once the number of kernels surpassed 100 and continued to grow, the classification accuracy did not increase, but the training time surged up (Table3).Table 3
Classification performance at different numbers of kernels.
Number of kernelsAccuracy (%)Time cost (s)1069.7019452072.4620874075.5521586078.7822748080.28239910082.15245812082.22273214082.463058For the pooling layer, 1-max pooling achieved the best performance. The dropout rate had a small effect on the model and was thus set to 0.5 for our experiments. Table2 lists the model parameters for experiments. In addition, the LSTM model was also adopted to extract text features in our experiments.


## 5.2. Attribute Extraction

The feature distribution of each class was extracted from all 463 files in four periods. Firstly, the Twitter-LDA topic model was employed to extract the distribution of class preferences. Focusing on all the files published in a period, the model holds that the features of each class are reflected by high-frequency words. After adjusting the number of topicsk to 6, each of the six topics was labeled (i.e., education, development, construction, reform, school, and party). Under the six topics, 30 keywords (translated literally into English) were sorted by frequency. The results in Table 4 suggest that these keywords are reasonably grouped under corresponding topics.Table 4
Keyword distribution of each topic.
TopicResultsEducationCountry0.00008Important0.00315Training0.00979Employment0.00475Focus0.00507Society0.0013Question0.0044Economy0.0036Management0.0057National0.0074Student0.0019Research0.0098Science0.0012Quality0.007Need0.0046Innovation0.005Teacher0.0094Thought0.0034Morality0.0067Government0.0041Our country0.0097Rural0.0035China0.0058Culture0.0077Mechanism0.0043Improvement0.0033Talent0.0016System0.0052Services0.0092Level0.0097DevelopmentEducation0.0092Improve0.00248Economic0.00875Implement0.0043Focus0.00462Construction0.00103Cause0.0005Socialism0.00778Talent0.00313Quality0.0056Reform0.00665Problem0.00225Innovation0.00786Level0.00105Service0.00393Society0.00905China0.00509Science0.00759Need0.00979Student0.0094Country0.00207School0.00584Bring up0.00378The study0.00544System0.00425Our country0.00222Promote0.00988Colleges0.00703Features0.00787People0.00238ConstructionStrategy0.00131Must0.0053Advance0.00721Postgraduate0.00128Modernization0.00528Cooperation0.00693Science and technology0.00297Comprehensive0.00613Communication0.00174Our country0.00724Strengthen0.00388Cadre0.00871Xi Jinping0.00405Philosophy0.00396Innovation0.00489Organization0.00868Intellectuals0.00023System0.00395Social science0.00688Reform0.00095Nation0.00332Produce0.00461Accelerate0.00468Theory0.00054Development0.00593Nationwide0.00231System reform0.00795General secretary0.00868Socialism0.00531Thought0.00652ReformNationwide0.00339Society0.00578Thought0.00154People0.00273Colleges0.00335Education0.002Modernization0.00696Practice0.00969Service0.00453Profession0.00165China0.00102Construction0.00083Problem0.00794Political0.00394System0.00453Features0.00597Our country0.00638Science0.00354Central0.00588School0.00419Socialism0.00719Innovation0.00819Leadership0.00215Solve0.00775All levels0.009Cause0.00865Development of0.00546School0.00532Management0.00889The study0.00325SchoolBring up0.0097Thought0.00096Our country0.00225Modernization0.00785Improve0.00428Cause0.00642People0.00533Leadership0.00675School0.00135Central0.00076China0.00047Development of0.00988Country0.00112Implement0.00184Political0.0027Culture0.00694Socialism0.00953System0.00851Government0.00617All levels0.00372Practice0.0029Education0.00374Features0.00566Society0.00686Reform0.00363Promote0.0015Problem0.00264Talent0.00098Student0.00866The study0.00068PartyImportant0.00625Cause0.00326Leadership0.00747Solve0.00217Practice0.00675Employment0.00312Implement0.00959Promote0.00786Teaching0.00399Enterprise0.00369Mechanism0.00897School0.00547Technology0.00651Technology0.0028International0.00578Colleges0.00245Political0.00667Plan0.0003Central0.00668Major0.00833Profession0.00418People0.00536Modernization0.00724Department0.00222World0.00601Socialism0.00844Area0.00169All levels0.00835Features0.00498Engineering0.00913To make each topic more intuitive and facilitate the analysis of period distribution, the topic word distribution was fixed, regardless of periods, to analyze preference distribution. As shown in Figure5, the preference distribution across periods varied greatly from topic to topic. For example, Figure 6 displays the preference distribution on various topics in the 2010s. The preference distribution was adopted as the vectorized representation of a file feature and combined with publish year (four dimensions: 1980s, 1990s, 2000s, and 2010s) and high-frequency words (five dimensions) of the files. In this way, a 20-dimensional feature was obtained to depict file attributes.Figure 5
Preference distribution of the six topics across periods.Figure 6
Preference distribution in the 2010s.


## 5.3. Comparative Experiments

Random cross validation was implemented in model training. Specifically, 25% of the training data were used as the cross-validation set, and 40% standard files were used as the validation set for each validation. The fusion model was trained for 200 generations, and each trained model was validated for 204 iterations. After a total of 40,800 iterations, the stability of the fusion model was evaluated against the test set by(2)Acc=∑i=1NTP+TN/TP+TN+FP+FNN,where N is the number of semantic classes; TP is the true positive; TN is the true negative; FP is the false positive; and FN is the false negative.For comparison, our model was contrasted with CNN, decision tree (DT), Naïve Bayes (NB), k-nearest neighbors (KNN), random forest (RF), multilayer perceptron (MLP), SVM, and logistics regression (LR). The results (Table5) show that our model far outperformed these traditional classification models on the same HE dataset.Table 5
Classification performance of different models on our HE dataset.
ModelAccuracy (%)DT72.746NB73.856KNN60.555RF52.568MLP34.583SVM61.734LR72.786CNN78.56HE-CNN83.516As mentioned before, HE files are generally composed of a few long texts, each of which is very large. To ensure the sufficiency of training, standard text dataset and text features were combined in the training set. The performance of different classification models on the HE dataset with text features is compared in Table6. It can be seen that for the long text style document, the CNN model obtains a better result than other models. In addition, using the extracted text features can significantly improved the performance of the HE-CNN model than traditional CNN models.Table 6
Classification performance of different models on our HE dataset with text features.
ModelAccuracy (%)DT77.424NB75.824KNN64.236RF54.358MLP33.528SVM68.157LR75.856CNN81.356HE-CNN88.782Furthermore, many combinations of text-feature presentation dimensions were tested to optimize the CNN parameters. In order to keep the balance between the training speed and model accuracy, the optimal experiment is processed and the result is shown in Table7. The results in Table 7 suggest that the number of the text-feature dimension is 300 which can get the balance.Table 7
Accuracy of different presentation dimensions of the text features.
ModelAccuracy (%)Text-10046.213Text 100-fusion79.745Text 20069.532Text 200-fusion79.212Text 30076.332Text 300-fusion82.345Text 40069.712Text 400-fusion78.662In addition, the experiment also uses the text-feature LSTM model to extract the text features which could increase the number of the text features. However, the experiment reflected that the extracted futures using text-feature LSTM model cannot be significantly used for improving the accuracy and indicated that the LSTM model is not suitable for processing long texts. The final experimental results are given in Table8. Apparently, our HE-CNN is applicable to semantic-based HE text analysis and is fast in dividing the texts into different classes.Table 8
Final experimental results.
ModelAccuracy (%)Text CNN73.323Text LSTM66.755Feature CNN31.875Text-feature CNN77.587Text-feature LSTM67.976Our model88.782


## 6. Conclusions

To solve the classification problem of HE texts, this paper builds a standard HE dataset and proposes the HE-CNN model, which combines text features with optimal CNN parameters. The proposed dataset lays the foundation for future studies on HE classification models, while our model was designed to handle large HE datasets containing long texts. The proposed model was proved effective through comprehensive experiments.



---

*Source: 9237713-2021-10-20.xml*