# Are You Depressed? Analyze User Utterances to Detect Depressive Emotions Using DistilBERT
## 5 Analysis of Results
### Comparison with Other Algorithms  
We conduct performance evaluations of the BWS and DSM-5 models using 27,774 Reddit test data points and compare them with other machine learning and deep learning algorithms. All models have been trained with the same epoch (5), learning rate (\(5\times 10^{-5}\)), and batch size (8). The performance evaluation results of each model are presented in Table 10. Our findings indicate that the BERT model outperforms other models for regression problems, while the DistilBERT model is the best for classification problems. Furthermore, the deep learning models perform better than the machine learning models. Among the deep learning models, the pre-trained models, such as BERT, DeBERTa [38], RoBERTa [39], and ELECTRA, perform better than non-pre-trained models, such as DNN, BiLSTM, and CNN. We select the DistilBERT model for our tasks based on these results.  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline \multirow{2}{*}{**Models**} & \multirow{2}{*}{**Models**} & \multicolumn{2}{c}{**Regression**} & \multicolumn{3}{c}{**Classification**} \\ \cline{3-8}  & & **RMSE** & **R2** & **Precision** & **Recall** & **Specificity** & **F1** \\ \hline \multirow{8}{*}{Machine Learning} & Support Vector & \multirow{8}{*}{3.944} & \multirow{8}{*}{0.3518} & \multirow{8}{*}{-} & \multirow{8}{*}{-} & \multirow{8}{*}{-} & \multirow{8}{*}{-} \\  & Regressor [40] & & & & & & \\ \cline{1-1} \cline{2-8}  & K-NN [41] & & & & & & \\ \cline{1-1}  & Regression & 3.9181 & 0.3603 & - & - & - & - \\ \cline{1-1}  & Naïve-Bayes [42] & - & - & 0.9156 & 0.573 & 0.9763 & 0.8089 \\ \cline{1-1}  & K-NN Classification & - & - & 0.8504 & 0.6506 & 0.9699 & 0.7483 \\ \cline{1-1}  & Random Forest [43] & - & - & 0.9586 & 0.8795 & 0.9927 & 0.9396 \\ \hline \multirow{8}{*}{Deep Learning} & DNN & 3.4472 & 0.5048 & 0.9718 & 0.9602 & 0.9956 & 0.9631 \\ \cline{1-1}  & DNN + Bi-LSTM & 2.59 & 0.72 & 0.9893 & 0.9874 & 0.9983 & 0.9863 \\ \cline{1-1}  & CNN-1D [44] & 3.7623 & 0.4101 & 0.9723 & 0.9642 & 0.9963 & 0.969 \\ \cline{1-1} \cline{2-8}  & BERT & 2.0557 & 0.8239 & 0.9984 & 0.9984 & 0.9998 & 0.9987 \\ \cline{1-1} \cline{2-8}  & DeBERTa & 2.3020 & 0.7792 & 0.9983 & 0.9977 & 0.9998 & 0.9988 \\ \cline{1-1} \cline{2-8}  & RoBERTa & 2.6558 & 0.7061 & 0.9974 & 0.997 & 0.9997 & 0.9981 \\ \cline{1-1} \cline{2-8}  & ELECTRA & 2.5173 & 0.7360 & 0.9976 & 0.9983 & 0.9998 & 0.9983 \\ \cline{1-1} \cline{2-8}  & **DistilBERT (ours)** & 2.1601 & 0.8056 & 0.9989 & 0.9988 & 0.9998 & 0.999 \\ \hline \hline \end{tabular}
\end{table}
Table 10: Performance evaluation of machine learning and deep learning algorithms.  
Figure 13: Visualization of attention weights using BertViz.  
We conduct an inference speed evaluation for each model on the same GPU environment. The dataset used for the evaluation is the size of 27,774 Reddit DSM-5 test data points, and the results are presented in Table 11. Our findings show that the DistilBERT model has a faster inference speed than half of the other pre-trained deep learning models, indicating that our model can be applied even in real-time environments that require a fast response.