A Fine-Grained Annotated Corpus for Target-Based Opinion Analysis of Economic and Financial Narratives (2021)

Abstract: In this paper about aspect-based sentiment analysis (ABSA), we present the first version of a fine-grained annotated corpus for target-based opinion analysis (TBOA) to analyze economic activities or financial markets. We have annotated, at an intra-sentential level, a corpus of sentences extracted from documents representative of financial analysts’ most-read materials by considering how financial actors communicate about the evolution of event trends and analyze related publications (news, official communications, etc.). Since we focus on identifying the expressions of opinions related to the economy and financial markets, we annotated the sentences that contain at least one subjective expression about a domain-specific term. Candidate sentences for annotations were randomly chosen from texts of specialized press and professional information channels over a period ranging from 1986 to 2021. Our annotation scheme relies on various linguistic markers like domain-specific vocabulary, syntactic structures, and rhetorical relations to explicitly describe the author’s subjective stance. We investigated and evaluated the recourse to automatic pre-annotation with existing natural language processing technologies to alleviate the annotation workload. Our aim is to propose a corpus usable on the one hand as training material for the automatic detection of the opinions expressed on an extensive range of domain-specific aspects and on the other hand as a gold standard for evaluation TBOA. In this paper, we present our pre-annotation models and evaluations of their performance, introduce our annotation scheme and report on the main characteristics of our corpus.

## introduction financial markets are places where the exchange and the processing of information are crucial in determining prices. thus, the knowledge about the market participants' opinions is an essential driver of markets. understanding how these opinions evolve is therefore valuable for financial participants and regulators. the interaction of market participants determines market dynamics, and how market participants act is largely determined by their beliefs on the outlook. the importance of beliefs in shaping market dynamics reflects an element of a selffulfilling prophecy: beliefs guide decisions that then validate the underlying belief. one of the classic models of this socio-psychological phenomenon is the diamond and dybvig's model (1983) about bank runs. the model demonstrates that even healthy banks can go bankrupt because of depositors' expectations about the behavior of others. if a large number of depositors expect the others to withdraw their funds, the only rational option for them is to be the first one to withdraw the money. if all the depositors decide to withdraw their money, the bank will go bankrupt despite of its initial financial health. in other words, unlike natural science, market dynamics are also determined by its participants' beliefs about the future. an investor buys or sells an asset based on his view about its expected future price. a recent study, for example, finds some evidence that investors' beliefs can be reflected in respondents' multi-asset allocation strategy @xcite . people's beliefs are formed as a result of changes in business activities, economic data, and financial outlook. specialized press and communication of corporates are essential inputs in belief formation. economists have been using sentiment indices obtained from surveys, such as the purchasing manager index, to gauge the underlying economic activity. one limitation of sentiment indices is that information is restricted by the questionnaire design and update frequency. consequently, these surveys provide an indication for the general direction of the economy but do not provide much additional color in terms of "why" things are changing or what the underlying narrative is. one approach to get a better sense of the underlying factors is to analyze news articles. numerous studies on news articles have been conducted in the past, though without using any algorithms @xcite , @xcite ) or designing algorithms dependent on sentiment lexicons. for example, @xcite use the loughran mcdonald dictionary @xcite to extract negative emotion from financial newspapers as explainable variable to predict sovereign debt spread. thanks to recent progress in natural language understanding, it becomes possible to monitor economic narratives and opinions expressed in written texts. there are many approaches to address the problem, e.g., opinion classification (classifying opinions into positive, negative, and neutral) @xcite , topic modeling (probability models for discovering clusters based on the frequency of words in a collection of texts) @xcite . although model accuracy of opinion categorization is crucial for its downstream economic and financial analysis tasks, performing fine-grained level target-based opinion analysis on texts written by domain experts is still a relatively unexploited field. in the literature about text mining applications in finance, one encounters most often dictionary-based or unsupervised learning approaches, and the performance of the language model is rarely discussed. one of the reasons for this state of affairs could be the lack of relevant annotated corpora. to fill this gap, we introduce an annotated corpus that includes both information labeled by human and by algorithms. the corpus consists of texts from different reliable sources; its contents cover corporate news, macroeconomic statements, and comments relevant to financial markets. each sentence in our corpus is annotated with the following information: (1) domain-specific concepts (2) span of words that potentially indicate the author's opinion (3) named entities (4) negative patterns. if the targets of opinion can be identified, we annotated the pair (target, polarity). in addition to these labels and relations, we also propose to consider specific rhetorical modes like domain experts do. ## related works opinion analysis is the task of natural language understanding for classifying texts into positive, negative and neutral polarities. it sees increasing importance in the banking industry among regulators and financial participants. most existing approaches of opinion analysis applied to economics and finance, however, attempt to detect the overall polarity of a sentence @xcite , or text @xcite , leaving aside opinion targets. a study on twitter sentiment classification showed that 40% of errors resulted from the ignorance of target @xcite . to the best of our knowledge, fiqa foot_0 task 1 is the first corpus that labels opinion polarities together with its targets from sentences of microblog and headlines. nevertheless, its size is relatively small (1,313 samples), and it contains only relatively short sentences foot_1 . in the texts in which we are interested, sentences written by financial experts are likely to be more complex; as detailed in in this work, we are interested in sentences containing at least one expression of opinion about evolution in the economic and financial landscape foot_3 , called subjective sentences. their particularity is that writers use language to monitor and judge the flow of events or assess their impacts. authors may: a in the down & low category, plummet and decrease convey the notion of scaling rapid and median, respectively. ## methodology supervised learning approaches have showed their efficiency to predict target-oriented opinions in a similar task, i.e. semeval-2014 task 4 @xcite which is a widely used benchmark for tboa. however, to the best of our knowledge, no publicly available annotated corpus corresponds to our research problem. therefore, we follow the standard scheme of corpus creation in natural language processing to create our corpus for tboa specific to economy and finance. the overall picture of our research is illustrated in figure 2 . it is mainly divided into three phases: data collection and pre-annotation (i.e., machineassisted annotation), data annotation, and model training. only the first two phrases will be fully examined in this paper, especially our work in preannotation, which delivers satisfying accuracy and reduces laborious annotation workload. future work will investigate appropriate neural network architecture to solve the task tboa (see definition 1) by using this corpus. before discussing technical details, we want to specify why we use machine-assisted preannotations prior to applying manual annotations. ## conclusion this paper defines an annotation scheme for identifying target-based opinions from sentences related to the economy and financial markets, by incorporating specific-domain use of language, namely formulations of argumentative and conditional opinions, opinions expressed in cause-effect relations, and opinions with a low level of commitment. the contribution of this paper can be summarized as follows: 1. our annotation scheme considers rhetorical modes, which is, as far as we know, a novelty in opinion analysis applied to economic and financial narratives. it enables our corpus to describe faithfully how financial professionals communicate and analyze the evolution of events and changes in business data. 2. we designed a tailor-made pipeline by leveraging a set of techniques in natural language processing (nlp) to pre-annotate informative features for identifying candidates relevant to tboa and alleviating the workload of annotators. at this stage of our project, these pre-annotation models evaluation scores are significant enough to assume that most pertinent sentences are retained for tboa. 3. our manually annotated corpus responds to the need for high-quality training and evaluation data for the development of supervised learning algorithms in the research field of tboa applied to economy and finance. in the future, we want to develop neural models adapted to our corpus by incorporating fundamental approaches in nlp and domain-specific knowledge, in particular, we investigate the contribution of data augmentation approach to augment the size of our corpus. labeled groups tags definition arrow by algorithms named entities person people's name none geo regions, countries org companies or institutions other elements negation absolute, partial or verbal negation from [negation] to its relevant [oee] toi terms of interest in economics and finance none appraisal opinion and emotion expressions by human opinion & emotion expressions oee word of the span of words which indicated opinion holder's appraisal from [oee] toward its corresponding target targets -target toward which a negative opinion is expressed none + target toward which a positive opinion is expressed non-committal target toward which opinion holder do not express their thoughts, attitude or intentions clearly. unknown annotators do not know which polarity to attribute for a particular concept/jargon, but this tag should be used with caution rhetorical modes result the span of words that declares: " something produces another thing as a result"; relation (cause, consequence) expressed by author from [result] to its relevant targets guess explicit speculation about future from [guess] to its relevant targets premise arguments used by opinion holder to support his opinion none conditional expressed opinion are conditional on a specific clause words negative pattern scarcely, scarce, fail, no more, nowhere, not, no, none, neither, little, deficiency, no longer, few, nowise, seldom, miscarry, in no manner, miscarry, nor, to no degree, hardly, zilch, nobody, cipher, by no means, lack, rarely, barely, nothing, deny, without, null, in no way, never, nada, cypher, naught, nix, zero, absence, nil, negate