## 4 Empirical Analysis
### Results of Empirical Analysis
In addition, there are seven machine learning models with slightly decreasing AUC and six machine learning models with slightly increasing AUC, but the magnitude was less than 0.01. As for these six machine learning models with slightly higher AUC, a further comparison of the value of ACC, SEN and SPE shows that the values of the five models are lower than those only based on traditional financial indicators. Then this paper studies the distribution of emotional tone of MD&A for ST and non-ST listed companies and finds that the distribution is more consistent and does not have significant differentiation. The results are shown in Table 5. Because of the existence of management manipulation and modification of emotional tone in ST listed companies, the emotional tone of MD&A in ST and non-ST listed companies is not as distinguishable as it is expected to be. Through the empirical results, it can be clearly seen that the emotional tone of MD&A does not have a significant effect on the early warning of financial crises of listed companies and even brings some noise. 5.3 Analysis of Empirical Results of Financial Crisis Early Warning Based on Traditional Financial Indicators and Emotional Tone of Financial News  
Financial news text, as an external text, having a characteristic of objectivity, is not susceptible to manipulation by the companies. Based on this, this paper studies whether the emotional tone of financial news texts can enhance the effect of traditional financial indicators for the identification of financial crises in listed companies. Based on traditional financial indicators and the emotional tone of financial news, the empirical results of financial crisis early warning are shown in Table 6. The ROC curve is detailed in Figure A3 of Appendix A. By comparing the values of AUC, it can be found that CatBoost, AdaBoost, random forest, DBGT and Bagging remain to be the five models with the best performance. Figure 2 is plotted based on the values of the AUC of thirteen machine learning models under three groups of input feature variables. According to Figure 2, the values of the AUC of the group with the emotional tone indicators of financial news are higher than the effects of the other two groups of input feature variables on each machine learning model. Moreover, according to the average value of the four evaluation indexes of the thirteen machine learning models, the prediction effect of the group of input feature variables with the emotional tone indicators of financial news is significantly better than those of the other two groups of input features. \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & **AUC** & **ACC** & **SEN** & **SPE** & **Rank** \\ \hline Logistic regression & 0.7359 & 0.6765 & 0.6617 & 0.6913 & 11 \\ Ridge regression & 0.7349 & 0.6747 & 0.6636 & 0.6885 & 13 \\ Lasso regression & 0.7358 & 0.6765 & 0.6599 & 0.6932 & 12 \\ DBGT & 0.7732 & 0.7135 & 0.7190 & 0.7079 & 4 \\ XGBoost & 0.7648 & 0.6969 & 0.7024 & 0.6913 & 6 \\ LightGBM & 0.7550 & 0.6839 & 0.6858 & 0.6821 & 8 \\ CatBoost & 0.7795 & 0.7033 & 0.7375 & 0.6691 & 1 \\ AdaBoost & 0.7761 & 0.7098 & 0.7043 & 0.7153 & 2 \\ SVM & 0.7368 & 0.6895 & 0.6821 & 0.6969 & 9 \\ BPNN & 0.7587 & 0.6904 & 0.6691 & 0.7116 & 7 \\ Decision tree & 0.7367 & 0.6765 & 0.7301 & 0.6229 & 10 \\ Random forest & 0.7751 & 0.7107 & 0.7098 & 0.7116 & 3 \\ Bagging & 0.7679 & 0.7043 & 0.7116 & 0.6969 & 5 \\
**Mean** & 0.7562 & 0.6928 & 0.6952 & 0.6905 & \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Early warning effects of thirteen machine learning models based on traditional financial indicators and the emotional tone of financial news. \begin{table}
\begin{tabular}{c c c} \hline \hline  & **ST** & **Non-ST** \\ \hline Average value & 0.4435 & 0.4548 \\
25 th percentile & 0.3420 & 0.3312 \\ Median & 0.4725 & 0.4930 \\
75 th percentile & 0.5607 & 0.5885 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Descriptive statistics of the emotional tone of MD&A for ST and non-ST listed companies. 4.5.4.