## 4 Results and Discussion  
Table 3 reports the scores obtained by each considered model on the whole novel dataset contributed with this work. VADER exhibits the lowest performance across all criteria, highlighting its constraints in target-level sentiment analysis. VADER demonstrates a macro precision of 0.5093, a macro F1-score of 0.4753, a macro recall of 0.4736, and an overall accuracy of 0.4939. The weighted metrics provide more evidence of its insufficiency with a weighted F1-score of 0.5010, a weighted precision of 0.5336, and a weighted recall of 0.4939. VADER, a lexicon-based technique originally designed for general sentiment analysis, particularly in social media contexts, faces difficulties in handling the subtle language and specialist terminology commonly found in financial literature. The system's static and rule-based nature hinders its ability to accurately understand the changing linguistic nuances commonly seen in financial discussions. This limitation restricts its usefulness in dynamic and contextually sensitive situations. Pre-trained models such as DistilFinRoBERTa and DeBERTa-v3-base-ABSA-v1.1 exhibit some improvement compared to VADER. DistilFinRoBERTa demonstrates a marginal improvement in accuracy, reaching a value of 0.5129. This improvement is accompanied by enhanced macro measures, including a recall of 0.5357, precision of 0.6182, and F1-score of 0.5010. DeBERTa-v3-base-ABSA-v1.1 demonstrates improved performance, with an accuracy of 0.5169, macro recall of 0.5833, macro precision of 0.6736, and macro F1-score of 0.5367. These models perform better than VADER, which is likely because of their sophisticated structures and specialized training. DistilFinRoBERTa benefits from effective pre-training on various datasets in the financial domain, while DeBERTa-v3-base-ABSA-v1.1 uses a unique disentangled attention mechanism that is crucial for understanding target-level sentiments. Moreover, we recall that DeBERTa-v3-base-ABSA-v1.1 is specifically fine-tuned for target/aspect-level sentiment analysis, while VADER and DistilFinRoBERTa are meant for document/sentiment-level sentiment analysis. The fine-tuned DeBERTa-v3-base-ABSA-v1.1 model stands out by achieving the highest recorded accuracy of 0.8679 in this investigation. This model underwent thorough training and fine-tuning using our financial dataset, employing a 5-fold cross-validation approach. The model attains a macro recall of 0.8414, macro precision of 0.8499, and macro F1-score of 0.8448. The weighted metrics also demonstrate its exceptional performance. The results underscore the significant advantages of refining this model on domain-specific data. Among the LLMs, ChatGPT-4 achieves an outstanding performance, ranking second only to the highly optimized DeBERTa-v3-base-ABSA-v1.1. It achieves a macro aver  
\begin{table}
\begin{tabular}{l c c c c c c c} \hline \hline
**Model** & **Accuracy** & \begin{tabular}{c} **Macro** \\ **Precision** \\ \end{tabular}  & \begin{tabular}{c} **Macro** \\ **Recall** \\ \end{tabular}  & \begin{tabular}{c} **Macro** \\ **F1-Score** \\ \end{tabular}  & \begin{tabular}{c} **Weighted** \\ **Precision** \\ \end{tabular}  & \ \begin{tabular}{c} **Weighted** \\ **Recall** \\ \end{tabular}  & \begin{tabular}{c} **Weighted** \\ **F1-Score** \\ \end{tabular}  \\ \hline VADER & 0.4939 & 0.5093 & 0.4736 & 0.4753 & 0.5336 & 0.4939 & 0.5010 \\ DistilFinRoBERTa & 0.5129 & 0.6182 & 0.5357 & 0.5010 & 0.6757 & 0.5129 & 0.5283 \\ DeBERTa-v3 & 0.5169 & 0.6736 & 0.5833 & 0.5367 & 0.7451 & 0.5169 & 0.5324 \\ \hline DeBERTa-v3 (fine-tuned) & **0.8679** & **0.8499** & **0.8414** & **0.8448** & **0.8657** & **0.8679** & **0.8662** \\ \hline ChatGPT-3.5 & 0.7527 & 0.7716 & 0.7470  
age F1-score of 0.8337 and an accuracy of 0.8591. The performance of Gemini models, specifically Gemini 1.5 Pro and Gemini 1.5 Flash, is noteworthy, demonstrating significant improvements compared to previous versions such as Gemini 1 Pro.