# Mining the Frequent Patterns of Named Entities for Long Document Classification
## 4 Experiments
### Experiments Results
#### 4.3.2 Time Consumption Analysis  
We compared the model training times and the inference decoding times of all the methods on the CCF-BDCI 2020 dataset, and we summarize the results in Table 3. The evaluation of MIPELD and traditional machine learning-based models was based on an Intel i7-10700 CPU, whereas the evaluation of deep learning-based methods was based on a single Nvidia P100 GPU.  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{2}{c}{**CCKS 2021-Level1**} & \multicolumn{2}{c}{**CCKS 2021-Level2**} & \multicolumn{2}{c}{**CCF-BDCI 2020**} & \multicolumn{2}{c}{**THUNews**} \\ \cline{2-7}  & **ACC (\%)** & **Macro-F1** (\%)** & **ACC (\%)** & **Macro-F1** (\%)** & **ACC (\%)** & **ACC (\%)** & **ACC (\%)** & **Macro-F1** (\%)** \\ \hline KNN & 59.75 & 38.48 & 42.92 & 31.79 & 88.52 & 87.26 & 84.85 & 82.95 \\ RandomForest & 84.58 & 56.55 & 52.25 & 39.38 & 88.80 & 88.00 & 81.01 & 72.04 \\ DT & 80.42 & 53.76 & 50.16 & 42.26 & 86.48 & 85.68 & 81.61 & 77.66 \\ SVM & 89.25 & 59.70 & 55.67 & 41.54 & 91.39 & 90.38 & 85.41 & 70.82 \\ XGBoost & 89.19 & 61.38 & 62.17 & 52.26 & 88.91 & 88.61 & 82.83 & 84.36 \\ LGB & 90.42 & 63.12 & 61.84 & 54.13 & 89.92 & 90.81 & 84.59 & 85.69 \\ \hline Word2 Vec & 71.42 & 50.80 & 38.50 & 30.94 & 80.19 & 77.82 & 70.36 & 62.87 \\ Doc2 Vec & 80.17 & 53.91 & 50.08 & 42.12 & 89.84 & 88.34 & 85.57 & 83.13 \\ RNN & 85.50 & 57.17 & 38.60 & 25.27 & 75.27 & 69.63 & 78.40 & 63.98 \\ TextCNN & 89.40 & 59.73 & 58.40 & 46.39 & 86.58 & 84.17 & 92.41 & **90.67** \\ UniLSTM & 88.60 & 59.26 & 46.30 & 33.96 & 55.27 & 42.21 & 84.56 & 66.22 \\ BiLSTM & 89.20 & 59.67 & 61.80 & 52.09 & 88.01 & 86.50 & **93.05** & 90.44 \\ HAN & 74.73 & 50.15 & 44.87 & 36.47 & 70.75 & 51.88 & 90.36 & 87.27 \\ BERT & **92.70** & 66.96 & 37.80 & 16.56 & 30.74 & 20.85 & 50.63 & 48.84 \\ \hline MIPELD & 89.42 & **68.17** & **62.33** & **55.11** & **92.01** & **90.84** & 87.97 & 85.88 \\ \hline \hline \end{tabular}
\end{table}
Table 2: The main results on the datasets. The best results are in bold, and the second best ones are underlined.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Model** & **Training** & **Inference** \\ \hline KNN & 1.26 s & 10.98 s \\ RandomForest & 2.98 s & 2.35 s \\ DT & 4.08 s & 0.78 s \\ SVM & 16.72 s & 6.26 s \\ XGBoost & 29.62 s & 4.90 s \\ LGB & 22.17 s & 5.10 s \\ \hline Word2 Vec & 36.13 s & 26.01 s \\ Doc2 Vec & 106.21 s & 28.80 s \\ RNN & 59 min & 7 min \\ TextCNN & 73 min & 9 min \\ UniLSTM & 60 min & 7 min \\ BiLSTM & 82 min & 10 min \\ HAN & 89 min & 5 min \\ BERT & 102 min & 13 min \\ \hline MIPELD & 33.09 s & 8.12 s \\ \hline \hline \end{tabular}
\end{table}
Table 3: The running-time comparison results.  
It is obvious that the time consumed by the machine learning algorithms was less than that consumed by the deep learning algorithms. In particular, BERT had the highest time consumption due to the complexity of its architecture, and the RNN and the LSTM cannot process data in parallel, resulting in high time consumption as well. In addition, for the time consumption of Word2 Vec and Doc2 Vec, we only counted the training and inference time taken by the Naive Bayesian classifier, so their time consumption is on the same level as that of the traditional machine learning models. By comparing the time consumption results with baselines, we can conclude that MIPELD possesses the advantage of low time consumption, yet it has comparable classification performance.