## 4 Experiments
### Evaluation of Inference Model  
The proposed inference model was evaluated on the Hellopeter dataset to assess its effectiveness in the sentiment analysis of financial products. As depicted in Table 8, the model achieved an impressive accuracy of 92.76%, which is much higher than the previously fine-tuned LLMs discussed in the previous sections, and the model achieved a precision of 87.65% and a recall of 88. Its accuracy was 94.06%, while its F1-score was 91.23%, thus proving the model's strong capacity to keep a good balance between precision and recall. These results confirm that the proposed inference model is well suited for sentiment analysis tasks in the financial context and provides accurate and reproducible predictions. Figure 10 provides a confusion matrix that visualises the performance of the proposed inference model. The model demonstrates strong classification abilities, as evidenced by the distribution of correct and incorrect predictions. In addition to traditional performance metrics, the model's effectiveness was further assessed using RAGAS-specific metrics, focusing on aspects such as answer quality and context retrieval. The RAGAS metrics, as shown in Table 9, demonstrate the model's proficiency in producing relevant and contextually accurate responses. The model achieved an average answer similarity of 98.45% and an answer correctness score of 93.85%, accompanied by a context recall of 78.97% and a context precision of 97.69%. These outcomes also provide evidence that the developed inference model outperforms not only in terms of sentiment analysis but also in the ability to generate contextually relevant and semantically accurate responses, thus supporting the model's credibility. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Model** & **Dataset** & **Domain** & **Accuracy** & **Precision** & **Recall** & **F1-Score** \\ \hline Llama-2-7b-hf & Financial PhraseBank & Financial News & 0.8118 & 0.8043 & 0.7632 & 0.7768 \\ Meta-Llama-3-88-Instruct & Financial PhraseBank & Financial News & 0.7897 & 0.8341 & 0.7026 & 0.7207 \\ gpt-3.5-turbo-0125 & Financial PhraseBank & Financial News & **0.8595** & **0.8607** & 0.8595 & 0.8600 \\ gpt-4o-mini-2024-07-18 & Financial PhraseBank & Financial News & 0.7828 & 0.7785 & 0.7828 & 0.7794 \\ \hline Llama-2-7b-hf & IMDB 50K & Movies & 0.7034 & 0.7466 & **0.8968** & 0.8850 \\ Meta-Llama-3-88-Instruct & IMDB 50K & Movies & 0.7145 & 0.7455 & 0.7992 & 0.8357 \\ gpt-3.5-turbo-0125 & IMDB 50K & Movies & 0.7330 & 0.7629 & 0.7330 & 0.7237 \\ gpt-4o-mini-2024-07-18 & IMDB 50K & Movies & 0.7110 & 0.7609 & 0.7110 & 0.7941 \\ \hline Llama-2-7b-hf & TripAdvisor & Hotels & 0.7210 & 0.7964 & 0.7163 & 0.7992 \\ Meta-Llama-3-88-Instruct & TripAdvisor & Hotels & 0.7593 & 0.7872 & 0.6528 & 0.8110 \\ gpt-3.5-turbo-0125 & TripAdvisor & Hotels & 0.8490 & 0.7849 & 0.6490 & **0.8962** \\ gpt-4o-mini-2024-07-18 & TripAdvisor & Hotels & 0.8210 & 0.7777 & 0.7210 & 0.8523 \\ \hline Llama-2-7b-hf & SemEval-2016 Stance & Political & 0.7124 & 0.7336 & 0.7010 & 0.6973 \\ Meta-Llama-3-88-Instruct & SemEval-2016 Stance & Political & 0.7146 & 0.7630 & 0.8891 & 0.7223 \\ gpt-3.5-turbo-0125 & SemEval-2016 Stance & Political & 0.7450 & 0.7460 & 0.7450 & 0.7435 \\ gpt-4o-mini-2024-07-18 & SemEval-2016 Stance & Political & 0.7140 & 0.7312 & 0.7140 & 0.7040 \\ \hline Llama-2-7b-hf & SemEval-2014 & Restaurant & 0.7277 & 0.7637 & 0.7277 & 0.7180 \\ Meta-Llama-3-88-Instruct & SemEval-2014 & Restaurant & 0.7926 & 0.7446 & 0.6926 & 0.8177 \\ gpt-3.5-turbo-0125 & SemEval-2014 & Restaurant & 0.7780 & 0.7962 & 0.7780 & 0.7745 \\ gpt-4o-mini-2024-07-18 & SemEval-2014 & Restaurant & 0.7736 & 0.7903 & 0.7736 & 0.7703 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Cross-domain evaluation results on different LLMs.