## 4 Experiments and Results
### Results  
In the first part of Table 4, the performance of various transformer models is evaluated, where only the text of Reddit posts is used. The results show that the roberta-based model alone achieved the highest averaged macro precision. In contrast, the client-based models had the highest averaged macro recall and accuracy, along with the roberta-based model. The google/electroa-based model had the lowest averaged macro precision and recall, with an averaged macro F1-score of 0.70. The bert-based, uncased model performed similarly to the roberta-based and xlnet-based, cased model, with slightly less precision and recall. The albert v2-based and distilbert-based, uncased models also had identical and promising performances. In the second part of Table 4, the performance of the transformer models improved significantly when the titles of the Reddit posts were included. The roberta-based model again had the highest scores, with an averaged macro F1-score of 0.83. The xlnet-based, cased model, the second-best performer, had a similar performance to the roberta-based model; however, it had a slightly lower precision and recall, and obtained an averaged-macro F1-score of 0.82. The bert-based, uncased, distilbert-based, uncased, and albert v2-based models had similar performances, with slight differences in their scores. These models had lower precision and recall than the roberta-based and xlnet-based, cased models, resulting in lower F1 scores. The google/electroa-based model again had the lowest precision and recall, but still performed relatively well, with an F1-score of 0.76, indicating an improvement of 10% when compared to previous experiments utilizing only text. When comparing the results in Table 4 with the previous state-of-the-art results in Table 5, it can be seen that the transformer models generally outperform the machine learning and deep learning models. In the first part of Table 4, where only the text of the Reddit posts is used, the transformer models have averaged macro F1-scores, ranging from 0.70 to 0.76, while the best-performing machine learning model (LR) has an averaged macro F1-score of 0.56 and the deep learning model (CNN with GloVe) hs an averaged macro F1-score of 0.61. In the second part of Table 4, where the text and title of the Reddit posts are used, the transformer models have averaged macro F1-scores ranging from 0.78 to 0.83, while the machine learning model (LR) has an averaged macro F1-score of 0.55 and the deep learning model (BiLSTM with GloVe) has an averaged macro F1-score of 0.72; this was the previous state-of-the-art model. \begin{table}
\begin{tabular}{l c c c c c c c} \hline \hline \multirow{2}{*}{**Transformers**} & \multicolumn{3}{c}{**Avg. Weighted Scores**} & \multicolumn{3}{c}{**Avg. Macro Scores**} & \multirow{2}{*}{**Accuracy**} \\  & **Precision** & **Recall** & **F1-Score** & **Precision** & **Recall** & **F1-Score** \\ \hline \hline \multicolumn{7}{c}{Regret Detection (only text)} \\ \hline bert-base-uncased & 0.78 & 0.78 & 0.78 & 0.75 & 0.71 & 0.73 & 0.78 \\ roberta-base & 0.81 & 0.81 & 0.81 & 0.78 & 0.75 & 0.76 & 0.81 \\ albert-base-v2 & 0.76 & 0.76 & 0.76 & 0.73 & 0.69 & 0.70 & 0.76 \\ xlnet-base-cased & 0.80 & 0.80 & 0.80 & 0.77 & 0.75 & 0.76 & 0.80 \\ google/electroa-base & 0.73 & 0.72 & 0.72 & 0.68 & 0.66 & 0.66 & 0.72 \\ distilbert-base-uncased & 0.76 & 0.76 & 0.76 & 0.72 & 0.69 & 0.70 & 0.76 \\ \hline \hline \multicolumn{7}{c}{Regret Detection (text + title)} \\ \hline bert-base-uncased & 0.82 & 0.82 & 0.82 & 0.80 & 0.78 & 0.79 & 0.82 \\ roberta-base & 0.86 & 0.86 & 0.85 & 0.84 & 0.83 & 0.83 & 0.85 \\ albert-base-v2 & 0.81 & 0.81 & 0.81 & 0.80 & 0.77 & 0.78 & 0.81 \\ xlnet-base-cased & 0.84 & 0.84 & 0.84 & 0.83 & 0.81 & 0.82 & 0.84 \\ google/electroa-base & 0.79 & 0.79 & 0.79 & 0.77 & 0.75 & 0.76 & 0.79 \\ distilbert-base-uncased & 0.81 & 0.77 & 0.78 & 0.79 & 0.77 & 0.78 & 0.81 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Results regarding regret detection using transformers.