# Text-Based Emotion Recognition in English and Polish for Therapeutic Chatbot
## 6 Discussion  
The envisioned study objectives have been met: we have created and tested a sentiment (3-class) and emotion (9-class) text-based classification engine for a therapeutic dialogue system, working in Polish. To achieve this, we had to create our own emotion-labeled corpus, which we generated using a neural MT system and two source English corpora. For sentiment and emotion recognition, we employed the state-of-the-art deep-learning classifier based on the BERT model, which outperformed the classic models, such as Naive Bayes or Support Vector Machines. We analyzed the misclassifications made by the best model (BERT) in more detail by looking at the examples related to the highest values from the confusion matrix (Table 4), especially the cases when a positive emotion label was confused with a negative emotion prediction and vice versa. The most problematic class was _other_positive_, as it was quite frequently predicted for sentences labeled with negative emotions, such as _anger_, _sadness_, and _other_negative_. The models for both languages did well in distinguishing between neutral and emotional texts; we obtained high F1-scores for the neutral class: 97.6% and 96.8% for English and Polish, respectively. We can explain some of the failed predictions as errors in the translation, others by prompt difficulty, e.g., the emotion was not reflected in the prompt itself (example: _I have some friends who are traveling all over Europe_ taken from a dialogue labeled _jealous_) or multiple emotions were present in the utterance (example: _I recently said goodbye to a good friend for a \begin{table}
\begin{tabular}{l c c c c c c c c c c} \hline \hline
**Emotion** & **Happin.** & **Conf.** & **o\_pos** & **Anger** & **Fear** & **Sadness** & **Guilt** & **o\_neg** & **neutral** & **F1-Score [\%]** \\ \hline happiness & **247** & 15 & 27 & 4 & 5 & 10 & 8 & 5 & 7 & \(75.08\%\pm 1.53\) \\ confidence & 24 & **158** & 15 & 2 & 7 & 7 & 3 & 2 & 4 & \(74.70\%\pm 1.54\) \\ other\_pos & 39 & 16 & **180** & 5 & 6 & 8 & 10 & 10 & 2 & \(67.04\%\pm 1.67\) \\ \hline anger & 3 & 0 & 7 & **159** & 12 & 13 & 10 & 18 & 4 & \(70.04\%\pm 1.62\) \\ fear & 0 & 4 & 4 & 8 & **171** & 10 & 7 & 3 & 0 & \(80.28\%\pm 1.41\) \\ sadness & 10 & 0 & 8 & 17 & 9 & **215** & 11 & 14 & 5 & \(75.84\%\pm 1.52\) \\ guilt & 0 & 3 & 7 & 15 & 2 & 10 & **148** & 13 & 1 & \(73.45\%\pm 1.56\) \\ other\_neg & 3 & 1 & 11 & 18 & 3 & 5 & 7 & **111** & 0 & \(66.07\%\pm 1.68\) \\ \hline neutral & 4 & 4 & 2 & 0 & 4 & 0 & 0 & 1 & **223** & \(92.15\%\pm 0.95\) \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Confusion matrix for emotion classification in Polish with BERT model for test subset. Confidence intervals given for confidence level 90%. \begin{table}
\begin{tabular}{l l c c} \hline \hline
**Language** & **Sentiment** & **Precision** & **Recall** & **F1-Score** \\ \hline \multirow{3}{*}{English} & positive & \(89.49\pm 0.94\) & \(92.74\pm 0.80\) & \(91.08\pm 0.87\) \\  & negative & \(93.54\pm 0.75\) & \(91.20\pm 0.87\) & \(92.36\pm 0.81\) \\  & neutral & \(97.79\pm 0.45\) & \(97.48\pm 0.48\) & \(97.64\pm 0.47\) \\ \hline \multirow{3}{*}{Polish} & positive & \(87.80\pm 1.00\) & \(88.86\pm 0.96\) & \(88.33\pm 0.98\) \\  & negative & \(90.68\pm 0.89\) & \(91.94\pm 0.83\) & \(91.31\pm 0.86\) \\  & neutral & \(98.06\pm 0.42\) & \(95.49\pm 0.64\) & \(96.76\pm 0.53\) \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Evaluation metrics (in percentages) for sentiment classification with BERT model for test subsets, for both languages. Confidence intervals given for confidence level 90%. while. I love her!_). Sometimes the label itself was just wrong, e.g., some of the neutral texts from DailyDialog seemed to be missing emotional labels (_I'm happy with that price_--labeled with _no emotion_--for which the model predicted _l happiness_). One of the limitations of our study is the accuracy of employed MT. Translation errors are the inevitable cost of our fast method of creating an emotion-labeled corpus for a new language.