## 4 Experimental Results
### Supervised Evaluation
\begin{table}
\begin{tabular}{l c c c c c c} \hline \hline  & \multicolumn{3}{c}{**Hard Labels**} & \multicolumn{3}{c}{**Soft Labels**} \\ \hline
**Emotion** & **Precision** & **Recall** & **F1** & **Precision** & **Recall** & **F1** \\ \hline admiration & 0.74 & 0.31 & 0.45 & 0.72 & 0.31 & 0.43 \\ amusement & 0.73 & 0.87 & 0.79 & 0.75 & 0.87 & 0.8 \\ anger & 0.70 & 0.65 & 0.68 & 0.71 & 0.68 & 0.69 \\ annoyance & 0.51 & 0.74 & 0.60 & 0.5 & 0.72 & 0.59 \\ approval & 0.83 & 0.51 & 0.63 & 0.8 & 0.49 & 0.61 \\ boredom & 0.67 & 0.94 & 0.78 & 0.64 & 0.92 & 0.75 \\ calmness & 0.65 & 0.82 & 0.73 & 0.63 & 0.82 & 0.71 \\ caring & 0.73 & 0.83 & 0.77 & 0.69 & 0.8 & 0.74 \\ courage & 0.47 & 0.67 & 0.55 & 0.46 & 0.67 & 0.54 \\ curiosity & 0.76 & 0.82 & 0.79 & 0.76 & 0.84 & 0.79 \\ desire & 0.82 & 0.79 & 0.81 & 0.81 & 0.77 & 0.78 \\ despair & 0.72 & 0.70 & 0.71 & 0.7 & 0.7 & 0.7 \\ disappointment & 0.44 & 0.46 & 0.45 & 0.4 & 0.44 & 0.42 \\ disapproval & 0.47 & 0.23 & 0.31 & 0.48 & 0.25 & 0.33 \\ disgust & 0.84 & 0.38 & 0.52 & 0.79 & 0.36 & 0.5 \\ doubt & 0.72 & 0.49 & 0.58 & 0.61 & 0.44 & 0.51 \\ embarrassment & 0.58 & 0.64 & 0.61 & 0.5 & 0.73 & 0.59 \\ envy & 0.28 & 0.86 & 0.42 & 0.28 & 0.93 & 0.43 \\ excitement & 0.55 & 0.68 & 0.61 & 0.57 & 0.68 & 0.62 \\ faith & 0.39 & 0.85 & 0.54 & 0.45 & 0.77 & 0.57 \\ fear & 0.48 & 0.41 & 0.44 & 0.42 & 0.41 & 0.42 \\ frustration & 0.54 & 0.57 & 0.56 & 0.53 & 0.61 & 0.57 \\ gratitude & 0.28 & 0.79 & 0.42 & 0.26 & 0.71 & 0.38 \\ greed & 0.57 & 0.64 & 0.60 & 0.55 & 0.68 & 0.61 \\ grief & 0.27 & 0.86 & 0.41 & 0.31 & 0.93 & 0.46 \\ guilt & 0.43 & 0.69 & 0.53 & 0.45 & 0.77 & 0.57 \\ indifference & 0.66 & 0.89 & 0.76 & 0.65 & 0.84 & 0.73 \\ joy & 0.84 & 0.43 & 0.57 & 0.77 & 0.44 & 0.56 \\ love & 0.69 & 0.66 & 0.67 & 0.69 & 0.72 & 0.71 \\ nervousness & 0.54 & 0.54 & 0.54 & 0.55 & 0.46 & 0.5 \\ nostalgia & 0.29 & 0.97 & 0.44 & 0.27 & 0.97 & 0.42 \\ optimism & 0.52 & 0.43 & 0.47 & 0.5 & 0.38 & 0.43 \\ pain & 0.33 & 0.55 & 0.41 & 0.42 & 0.73 & 0.53 \\ pride & 0.46 & 0.59 & 0.52 & 0.48 & 0.59 & 0.53 \\ relief & 0.54 & 0.84 & 0.66 & 0.51 & 0.84 & 0.64 \\ sadness & 0.70 & 0.60 & 0.65 & 0.67 & 0.62 & 0.64 \\ surprise & 0.68 & 0.69 & 0.68 & 0.71 & 0.69 & 0.70 \\ trust & 0.71 & 0.63 & 0.67 & 0.76 & 0.65 & 0.70 \\ \hline macro- & & & & & & \\ average & 0.58 & 0.66 & 0.59 & 0.58 & 0.66 & 0.59 \\ std & 0.17 & 0.18 & 0.13 & 0.16 & 0.18 & 0.13 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Results on the human-annotated portion of the EmoLit dataset. The soft and hard label models had the same average F1 score. The results are in line with related work (e.g., GoEmotions). \begin{table}
\begin{tabular}{l c} \hline \hline
**Hyperparameter** & **Value** \\ \hline Batch Size & \(16\) \\ Learning Rate & \(2\times 10^{-5}\) \\ Max Epochs & \(10\) \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Hyperparameter values used for training the supervised model on our dataset. Finally, in Table 11, we compare with the results provided by related work, showing that the baseline results in our dataset, created by our semi-supervised approach, are similar to crowdsourced multi-label emotion datasets. This is true even though our taxonomy is more fine-grained and thus needs to distinguish between very similar emotions that are grouped together in the other datasets.