## 4 Experiments
### Performance Evaluation  
The performance of our fine-tuned models on the Hellopeter dataset, which served as the primary dataset for fine-tuning, is summarised in Table 6. All models yielded good performance, and it was evident that all models were capable of capturing sentiment in the financial context. The _gpt-4o-mini-2024-07-18_ model achieved the highest accuracy of 89.15%, followed by _gpt-3.5-turbo-0125_ at 88.95%. The _Llama-2-7b-lift_ model, while slightly lower in accuracy at 81.15%, showed competitive performance with a precision of 88.05% and an F1-score of 88.10%. The _Meta-Llama-3-8B-Instruct_ model also performed well, with an accuracy of 87.05%, indicating that both Llama models are effective in aligning with the sentiment nuances specific to South African financial reviews.  
The confusion matrices for the Llama models, as depicted in Figure 8, also show the results on the Hellopeter dataset. Hence, the findings show that the classification is evenly distributed between the positive and negative sentiment classes with a low misclassification level, as highlighted by their high recall and precision. The confusion matrices of the GPT models are presented in Figure 9, and it can be seen that both models achieve comparable results, with the sentiment trends in the financial product reviews being predicted well by both models, with the GPT models having a slightly better accuracy than the Llama models.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Model** & **Dataset** & **Domain** & **Accuracy** & **Precision** & **Recall** & **F1-Score** \\ \hline Llama-2-7b-hf & Hellopeter & Financial Products & 0.8115 & **0.8805** & 0.8244 & 0.8810 \\ Meta-Llama-3-8B-Instruct & Hellopeter & Financial Products & 0.8705 & 0.8138 & 0.8265 & 0.8558 \\ gpt-3.5-turbo-0125 & Hellopeter & Financial Products & 0.8895 & 0.8203 & 0.8195 & 0.8793 \\ gpt-4o-mini-2024-07-18 & Hellopeter & Financial Products & **0.8915** & 0.8316 & **0.8315** & **0.8815** \\ \hline \hline \end{tabular}
\end{table}
Table 6: Performance evaluation on the Hellopeter dataset.  
Table 7 presents the cross-domain evaluation results across multiple datasets, allowing us to assess the adaptability of the models to data they were not primarily fine-tuned on. The results demonstrate that the models retained competitive performance across domains, highlighting their generalisation ability. For instance,the _gpt-3. 5-turbo-0125_ model had an accuracy of 85 percent. On the Financial PhraseBank dataset, the score is 95%, and on the SemEval-2014 Restaurant dataset, it is 77. 80%, which shows that it works well in other areas in addition to the financial industry. On the other hand, the _Llama-2-7b-hf_ model performed well in terms of adaptability, especially in the IMDB dataset, and an F1-score of 88 was attained, 50% of which is quite good considering the fact that the dataset mainly contains movie reviews.  
The cross-domain evaluation results indicate that although there is a small drop in performance as compared to the Hellopeter dataset, the models are still able to generalise well. This flexibility is important in order to make sure that the models work not only in the financial domain in which they were trained but also in different areas such as movie reviews, hotel feedback, political speeches, and restaurant reviews. These results also confirm the effectiveness of the fine-tuning process that allows the models to retain high accuracy, precision, recall, and F1-score in different domains, which proves the applicability of the models in various sentiment analysis tasks.  
Figure 8: Confusion matrices for Llama models on the Hellopeter dataset. (**a**) Llama-2-7b-hf confusion matrix; (**b**) Meta-Llama-3-8B-Instruct confusion matrix.  
Figure 9: Confusion matrices for GPT Models on the Hellopeter Dataset. (**a**) gpt-3.5-turbo-0125 confusion matrix; (**b**) gpt-4o-mini-2024-07-18 confusion matrix.