EFSA : Towards Event-Level Financial Sentiment Analysis (2024)

Abstract: In this paper, we extend financial sentiment analysis (FSA) to event-level since events usually serve as the subject of the sentiment in financial text. Though extracting events from the financial text may be conducive to accurate sentiment predictions, it has specialized challenges due to the lengthy and discontinuity of events in a financial text. To this end, we reconceptualize the event extraction as a classification task by designing a categorization comprising coarse-grained and fine-grained event categories. Under this setting, we formulate the Event-Level Financial Sentiment Analysis(EFSA for short) task that outputs quintuples consisting of (company, industry, coarse-grained event, fine-grained event, sentiment) from financial text. A large-scale Chinese dataset containing 12,160 news articles and 13,725 quintuples is publicized as a brand new testbed for our task. A four-hop Chain-of-Thought LLM-based approach is devised for this task. Systematically investigations are conducted on our dataset, and the empirical results demonstrate the benchmarking scores of existing methods and our proposed method can reach the current state-of-the-art. Our dataset and framework implementation are available at https://github.com/cty1934/EFSA

## introduction since robert f. engle was awarded the nobel prize because he researched the influence of news on financial market volatility @xcite , it catalyzed a surge in academic interest in financial sentiment analysis (fsa) @xcite @xcite . fsa holds significant importance within the application domain of sentiment analysis @xcite , encompassing the study of financial textual sentiment @xcite within news to forecast financial market title: stock market news, nov. 20, 2023: microsoft stock sets record high; openai in turmoil after sam altman's ouster news body: artificial intelligence was back in the spotlight monday, this time with some human drama attached to it. microsoft shares closed at an all-time high after the software giant said sam altman-openai's fired chief executive-would lead its new advanced ai research team. altman was pushed out on friday. now more than 700 openai employees have threatened to leave the company if the board doesn't resign and reinstate altman. the ai news will continue tuesday, when nvidia reports its quarterly results. the chip maker's stock-which has been a big contributor to the s&p 500's gains this year-also closed at a record high. from this example, we can be aware that the event usually serves as the subject of the sentiment in financial text, while the entity is the target of the emotional impact. extracting events from financial text may be conducive to accurate sentiment predictions. to this end, we aim to discover the events in financial text to provide an easier financial sentiment prediction. an intuitive way to identify an event from financial text is extracting a specific text span from the original text just like existing aspect-based sentiment analysis (absa) tasks that find both aspect and opinion terms from customer's comments @xcite @xcite . however, direct adapting approaches of absa for this task could be ineffective even infeasible due to the events in the financial text being overlong and discontinuous (c.f. figure 1 ). we will provide a further discussion about the similarities and differences between absa and our task in section 2.3. in this paper, based on our observations, we provide an alternative setting for fsa and propose a novel task, named event-level financial sentiment analysis (efsa), involving the prediction of quintuples (company, industry, coarsegrained event, fine-grained event, sentiment). here we have enhanced the fsa tasks in two facets. first, to overcome the difficulties associated with extracting events from financial texts, we reconceptualize the event extraction task as a classification task. we design a categorization system that comprises both coarse-grained and fine-grained event categories, specifically tailored for various event types in financial news. besides, we construct a knowledge-based rule to classify companies by industry, enabling fsa to elevate to higher dimensions, such as indices or sector markets. our task setting can offer substantial practical value in financial applications, including stock trading, stock market anomaly attribution analysis, enterprise risk management, etc. figure 1 presents example quintuples in our efsa task. to support this task, we annotated a large-scale dataset from chinese financial news. the dataset includes 12, 160 news articles, selected from an initial set of over 50, 000 articles collected from mainstream chinese financial news websites. detailed annotations were conducted based on the above task settings. to the best of our knowledge, this dataset is a large-scale fine-grained annotated dataset for fsa and the largest chinese dataset in the event-level fsa domain. we conducted comprehensive experiments to benchmark our dataset. empirical results demonstrate the efsa task presents a significant challenge, even for advanced large language models (llms) such as gpt-4, primarily due to the complexity of simultaneously predicting two categories and the fact that the sentiments in financial texts are primarily implicit. recent studies on implicit sentiments underscore the efficacy of the chain-of-thought (cot) approach in reasoning implicit sentiments in fine-grained sentiment analysis @xcite . consequently, we devise a framework utilizing a 4-hop chain of thought (cot) prompt based on llms, achieving the highest level of performance recorded for our task. the main contributions of our work can be summarized as follows: ## problem formulation and discussion in this section, we formulate the efsa task and differentiate it from the traditional absa task. ## dataset financial sentiment indicators are categorized into market-derived sentiment and human-annotated sentiments @xcite . the market-derived sentiment is estimated based on market dynamics such as stock price changes and trading volume, potentially incorporating noise from other sources @xcite . therefore, we employed manual labeling conducted by financial experts to ensure the creation of a high-quality dataset. specifically, the data construction process is elaborated in detail from the following three sections: data collection, data annotation, and data distribution. ## experimental evaluation in this section, we benchmark efsa with some widely used language models. in the following part, we will introduce the benchmark methods first, then detail our proposed framework, and finally present the evaluation metrics. ## related work previous research on the fsa datasets has concentrated on sentence or document level @xcite @xcite . this is based on an assumption that the given text conveys a single sentiment towards a certain topic. recent fsa datasets exhibit a trend towards progressively finer granularity. however, fine-grained datasets @xcite mostly focus on entities and sentiments, neglecting the concern of events. furthermore, the resources for fine-grained fsa datasets are still limited @xcite . following @xcite , we summarize the most widely used and recent fsa benchmark datasets in appendix e. event-level sentiment analysis is designed to identify user emotions on social platforms regarding current events @xcite . in this paper, we broaden the scope of event-level sentiment analysis by applying it to fsa, enhancing its relevance and utility in financial contexts. ## conclusion in this paper, we present efsa, a novel task for financial sentiment analysis (fsa), which deepens fsa to the event level. to support this task, we constructed the largest chinese dataset annotated for event-level fsa from a large-scale financial news corpus. we evaluated this task on widely used language models to present benchmark scores on the proposed dataset. additionally, we designed a 4-hop reasoning prompting framework based on existing llms to resolve this task. our experiments demonstrate the challenge of efsa and the effectiveness of our proposed approach. our work opens a new avenue in fsa and offers significant value to the entire financial domain.