## 1 Introduction
The objective of this research is to fill these gaps by assessing the capability of LLMs to analyze sentiment at the target level in the financial field with a specific focus on news related to the stock market. We believe that such an assessment is both timely and relevant with the potential to influence the development of professional financial reporting software. We compare the effectiveness of LLMs, such as ChatGPT and Gemini, to traditional models, such as VADER, as well as to fine-tuned transformer-based models, including DistilFinRoBERTa and Deberta-v3-base-absa-v1.1, to identify and categorize sentiment at the target level within financial news headlines. More specifically, we address the following research questions:  
1. How effectively do zero-shot LLMs perform in target-level financial sentiment analysis, and how do they compare to other state-of-the-art sentiment analysis techniques? 2. How do the performances of different versions of ChatGPT (specifically, ChatGPT 3.5, ChatGPT 4, and ChatGPT 4o) compare to those of Gemini models (including Gemini 1 Pro, Gemini 1.5 Flash, and Gemini 1.5 Pro) in target-level sentiment analysis tasks under zero-shot scenarios?The work detailed in this paper makes the following novel key contributions compared to existing studies:  
1. We present and publicly release a novel dataset that is specifically manually annotated for target-level sentiment analysis. This dataset consists of 1476 headlines from the financial sector. Studying target-level sentiment in news headlines is crucial because headlines shape immediate perceptions, influence public opinion, and dominate social media sharing, offering a concise and impactful representation of the news that often carries more bias and emotional weight than the full text. This resource is a critical instrument for the development and evaluation of sentiment analysis models that differentiate and evaluate sentiments directed at specific entities within the financial domain, as it uniquely addresses the nuanced requirements of target-level analysis in financial texts. To the best of our knowledge, no other dataset with the given characteristics is currently available;
2. Utilizing the contributed dataset, we present a thorough comparative analysis of LLMs compared to conventional sentiment analysis methods, emphasizing their strengths and weaknesses in the context of target-level financial sentiment analysis. This study is one of the first to systematically assess the efficacy of LLMs, such as different versions of Gemini and ChatGPT, in conducting target-level financial sentiment analysis within a zero-shot learning approach. Our work thus contributes to the development of NLP tools in the financial sector, enabling stakeholders to make more informed decisions by utilizing target-level sentiment analysis. The paper is organized as follows: Section 2 reviews related literature. Section 3 delineates the empirical methodology and the dataset creation and annotation process. The findings and discussion are reported in Section 4. Finally, Section 5 concludes the study and offers possible future research directions in the field of financial sentiment analysis.