## 5 Experimental Results
### Discussions  
Table 4 displays the outcomes of testing the RoBERTa models, trained on five classes of sentiments, against each dataset. Table 5 is similar, but the models in this case were trained on four sentiment classes. The performance is evaluated using the F1 score. Every row represents the model's evaluation on all datasets. Each column displays the dataset evaluated on all models. The \"Average\" rows and columns symbolize the average F1 score \begin{table}
\begin{tabular}{c c} \hline \hline
**Environment** & **Description** \\ \hline Model & roberta-base \\ \hline Transformer & Hugging Face \\ \hline \multirow{2}{*}{Programming language \& tools} & PyTorch, PyTorch Lightning, Anaconda \\  & Distribution, Python 3.10 \\ \hline GPU & NVIDIA GeForce GTX 1650 Ti \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Tools and technologies. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Dataset** & **Labeled Comments** & **Labels** \\ \hline Jira & 5.6 K & joy, love, surprise, neutral, sadness, anger, fear \\ \hline Twitter & 16 K & joy, love, surprise, neutral, sadness, anger, fear \\ \hline \multirow{4}{*}{Reddit} & \multirow{4}{*}{58 K} & admiration, amusement, anger, annoyance, \\  & & approval, caring, confusion, curiosity, desire, \\ \cline{1-1}  & & disappointment, disapproval, disgust, \\ \cline{1-1}  & & embarrassment, excitement, fear, gratitude, grief, \\ \cline{1-1}  & & joy, love, nervousness, optimism, pride, realization, \\ \cline{1-1}  & & relief, remorse, sadness, surprise, neutral \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Dataset specifications. on that particular row or column. A higher F1 score indicates a better-performing model. We consider a low-performing model if its F1 score is below the score of SentiStrength-SE (0.774), underlining the value in the tables. The highest performance achieved on each column is represented in bold font. By investigating the results presented in Tables 4 and 5, the following observations are worth mentioning:  
**(1) Models struggle to predict well in the SE domain, possibly due to differences in technical language and content. ** Based on the table columns, it is evident that Jira and Reddit datasets show the poorest performance. The Reddit dataset is extensive and has numerous labels. Despite using the Ekman grouping method, we can assume that the diverse data can impact the training of models. Upon analyzing the averages of both columns and rows, it is clear that the Reddit dataset performed the worst. It should be noted that although Reddit had the lowest performance, the Jira dataset produced results that were nearly as poor. This could be because technical communications between software engineers often include industry-specific terminology and jargon, which is not typically present in general conversation or news articles. As a result, the base models, which are trained on more general data, may struggle to accurately interpret these technical messages. On the other hand, Twitter-based models achieved the best performance, maybe due to their common data insight. **(2) Single-domain DSSM models, tested on general datasets, showed an expected decrease in performance, but the performance can be significantly improved if the model is trained with data from DISMs. ** By incorporating a variety of data from different domains, multi-domain datasets can integrate numerous models. For example, combining a rigorous domain dataset with a common domain dataset creates a model familiar with more challenging data, improving the performance over multi-domains. When examining the J + T row, we observe that by merging technical messages from Jira with day-to-day messages from Twitter, we built a model that is nearly as effective as the three-domain model.