UOA at the F in NLP -2022 ERAI Task: Leveraging the Class Label Description for Financial Opinion Mining (2022)

Abstract: Evaluating the Rationales of Amateur Investors (ERAI) is a task about mining expert-like viewpoints from social media. This paper summarizes our solutions to the ERAI shared task, which is co-located with the FinNLP workshop at EMNLP 2022. There are 2 sub-tasks in ERAI. Sub-task 1 is a pair-wised comparison task, where we propose a BERT-based pre-trained model projecting opinion pairs in a common space for classification. Sub-task 2 is an unsupervised learning task ranking the opinions’ maximal potential profit (MPP) and maximal loss (ML), where our model leverages the regression method and multi-layer perceptron to rank the MPP and ML values. The proposed approaches achieve competitive accuracy of 54.02% on ML Accuracy and 51.72% on MPP Accuracy for pairwise tasks, also 12.35% and -9.39% regression unsupervised ranking task for MPP and ML.

## introduction using textual information to guide investment decisions is not a novel topic in either financial or fintech settings. many researchers have devoted endeavors to social media posts and tried to dig out the rationale underlying the standpoints. however, these works struggle to cope with a considerable amount of data in the information explosion era, which brings an unnecessary expense to computation efficiency. moreover, posts with high rationality have more probability of leading to profitable outcomes than those less rational. thus, selecting high-quality analytical opinions can be a meaningful first step in investment opinion mining. the erai shared task @xcite proposes the rationale evaluation challenge with the goal of mining opinions leading to higher maximal potential profit (mpp) and lower maximal loss (ml). this challenge uses forecasting skills as a proxy and focuses on amateur investors' viewpoints. two settings are involved in this challenge, including 1) pairwise comparison, which aims to find posts with more rationality; 2) unsupervised ranking, which aims to sort out the posts leading to the highest mpp and lowest ml. several related works have launched good pilots for high-quality mining reviews. the bert model proposed by @xcite has been proven efficient in many nlp tasks since it was published. @xcite presented and summarized the opinion mining methods. @xcite provides methods to measure forecasting skills from the text. @xcite creatively introduces the mpp and ml values to support digging into the review quality. moreover, their proposed dataset, which is utilized in this paper, is the first dataset focusing on revealing text rationals. our model is based on a pre-trained language model, and for the binary classification task, we propose a method that utilizes the class-label information, and then we fine-tuned bert for the regression task. the official results show that our models achieve competitive performance on both tasks, indicating our approaches' effectiveness. we introduce the tasks and present our work as follows. section 2 elaborates on the shared task erai and the datasets for sub-tasks pairwise comparison and unsupervised ranking. we introduce our methodology and models in section 3 and present the experimental setup and official results in section 4. finally, we conclude our work in section 5. ## shared tasks the erai shared tasks aim to spark interest from nlp and financial communities and to launch a novel pilot with the perspective of text rationality evaluation. the shared tasks have two sub-tasks focusing on digging into investors' posts and sorting out those with higher possibilities leading to mpp and ml. ## conclusion this work presents the uoa team with how to tackle the erai shared tasks. for sub-task 1, we proposed a model by appending the class-label description from a pre-trained language model to accomplish the classification task. this suggests that our model is able to learn more discriminative features. specifically, in sub-task 1, our proposed system achieved the second position considering the average of mpp and ml accuracy by statistical manually. for sub-task 2, we leveraged a regression framework to rank ml and mpp values. the official results show that our approaches could effectively solve the two tasks. our models are simple but effective, and we achieved competitive performance on the shared tasks.