## 4 Empirical Analysis
### Results of Empirical Analysis
5.1 Analysis of Empirical Results of Financial Crisis Early Warning Based on Traditional Financial Indicators  
Using traditional financial indicators as the benchmark, this paper tests the performance effects of thirteen machine learning models based on traditional financial indicators, and the results are shown in Table 3. The ranking is based on the value of AUC, where the five models with the highest value are AdaBoost, random forest, Bagging, DBGT, and CatBoost. Bagging has the highest prediction accuracy of 0.6839 for the sample of ST listed companies, followed by the random forest model and the DBGT model, with a value of 0.6821 and 0.6710, respectively. Among the thirteen machine learning models, the decision tree has the lowest value of AUC and SPE. All thirteen models have an AUC of 0.7 or higher, and the average AUC of the thirteen models is 0.7292. The average score of ACC, SEN and SPE is 0.6647, 0.6707 and 0.6586, respectively. 5.2 Analysis of the Empirical Results of Financial Crisis Early Warning Based on Traditional Financial Indicators and Emotional Tone of MD&A  
According to the RQ1 proposed by this paper, this section examines the early warning effects of thirteen machine learning models using traditional financial indicators and the emotional tone of MD&A as input feature variables. The results are shown in Table 4, and the ROC curves are detailed in Figure A2 of Appendix A. From the comparison of the values of AUC, it can be found that AdaBoost, random forest, DBGT, and Bagging and CatBoost models are the five models with the best performance. Compared to the warning effect of thirteen machine learning models based on traditional financial indicators, there is a slight decrease in the average value of AUC, ACC, SEN and SPE in thirteen machine learning models merging emotional tone of MD&A, \begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & **AUC** & **ACC** & **SEN** & **SPE** & **Rank** \\ \hline Logistic regression & 0.7136 & 0.6525 & 0.6359 & 0.6691 & 11 \\ Ridge regression & 0.7135 & 0.6525 & 0.6359 & 0.6691 & 12 \\ Lasso regression & 0.7141 & 0.6525 & 0.6340 & 0.6710 & 10 \\ DBGT & 0.7434 & 0.6719 & 0.6821 & 0.6617 & 3 \\ XGBost & 0.7301 & 0.6571 & 0.6747 & 0.6396 & 7 \\ LightGBM & 0.7212 & 0.6516 & 0.6636 & 0.6396 & 8 \\ CatBoost & 0.7355 & 0.6691 & 0.6932 & 0.6451 & 5 \\ AdaBoost & 0.7514 & 0.6774 & 0.7006 & 0.6543 & 1 \\ SVM & 0.7164 & 0.6599 & 0.6433 & 0.6765 & 9 \\ BPNN & 0.7343 & 0.6811 & 0.6543 & 0.7079 & 6 \\ Decision tree & 0.7071 & 0.6349 & 0.6913 & 0.5786 & 13 \\ Random forest & 0.7464 & 0.6682 & 0.6691 & 0.6673 & 2 \\ Bagging & 0.7432 & 0.6747 & 0.6839 & 0.6654 & 4 \\
**Mean** & 0.7285 & 0.6618 & 0.6663 & 0.6573 & \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Early warning effects of thirteen machine learning models based on traditional financial indicators and emotional tone of MD&A. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & **AUC** & **ACC** & **SEN** & **SPE** & **Rank** \\ \hline Logistic regression & 0.7149 & 0.6488 & 0.6322 & 0.6654 & 11 \\ Ridge regression & 0.7144 & 0.6488 & 0.6322 & 0.6654 & 12 \\ Lasso regression & 0.7152 & 0.6497 & 0.6322 & 0.6673 & 10 \\ DBGT & 0.7429 & 0.6784 & 0.6858 & 0.6710 & 4 \\ XGBost & 0.7300 & 0.6590 & 0.6599 & 0.6580 & 7 \\ LightGBM & 0.7230 & 0.6580 & 0.6617 & 0.6543 & 8 \\ CatBoost & 0.7413 & 0.6728 & 0.6969 & 0.6488 & 5 \\ AdaBoost & 0.7462 & 0.6867 & 0.7061 & 0.6673 & 1 \\ SVM & 0.7165 & 0.6534 & 0.6488 & 0.6580 & 9 \\ BPNN & 0.7397 & 0.6728 & 0.6913 & 0.6543 & 6 \\ Decision tree & 0.7065 & 0.6442 & 0.7024 & 0.5860 & 13 \\ Random forest & 0.7455 & 0.6774 & 0.6728 & 0.6821 & 2 \\ Bagging & 0.7431 & 0.6904 & 0.6969 & 0.6839 & 3 \\
**Mean** & 0.7292 & 0.6647 & 0.6707 & 0.6586 & \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Early warning effects of thirteen machine learning models based on traditional financial indicators. but the magnitude of decrease was less than 0.01.