ai ML at the F in NLP -2022 ERAI Task: Combining Classification and Regression Tasks for Financial Opinion Mining (2022)

Abstract: Identifying posts of high financial quality from opinions is of extraordinary significance for investors. Hence, this paper focuses on evaluating the rationales of amateur investors (ERAI) in a shared task, and we present our solutions. The pairwise comparison task aims at extracting the post that will trigger higher MPP and ML values from pairs of posts. The goal of the unsupervised ranking task is to find the top 10% of posts with higher MPP and ML values. We initially model the shared task as text classification and regression problems. We then propose a multi-learning approach applied by financial domain pre-trained models and multiple linear classifiers for factor combinations to integrate better relationships and information between training data. The official results have proved that our method achieves 48.28% and 52.87% for MPP and ML accuracy on pairwise tasks, 14.02% and -4.17% regarding unsupervised ranking tasks for MPP and ML. Our source code is available.

## introduction the fast-growing financial social media has become a mainstream information source for investors. they prefer to follow high quality viewpoints with persuasive rationales. however, browsing numerous and noisy posts is time-consuming and inefficient. therefore, automatically identifying high quality posts in the financial field is vital. finnlp workshop of emnlp-2022 @xcite publishes a shared task regarding the above problem focusing on evaluating the rationales of amateur investors. there are two sub-tasks: pairwise comparison and unsupervised ranking for online posts. the posts are all from the financial social platforms of chinese. regarding sub-task1, we are asked to select high financial quality posts from pairs of posts. regarding sub-task2, all given posts are required to rank by their potential financial quality. as evaluation, it is difficult to assess the quality of a post directly. therefore, we propose and utilize the maximum possible profit (mpp) and the maximum loss (ml) in a certain period @xcite as the evaluation metric of opinion quality. several recent findings evaluate user-generated social media content, such as posts, tweets, and blogs, by nlp technology. @xcite explores using ai models for detecting financial fine-grained sentiment tendencies. they proved the importance of mining the premises and evaluating the rationales of a financial opinion. moreover, patel and ezeife (2021) investigated that bidirectional encoder representations from transformers (bert) @xcite is an advanced deep learning model for fine-grained aspect-based opinion mining on social media posts. besides, many natural language processing (nlp) technologies have been widely used to analyze financial-related domain information, for instance, stock price prediction @xcite and financial sentiment analysis @xcite . in our work, we leverage several pre-trained language models (plm) for two tasks, roughly described as (1) posts classification: comparing financial quality for two given posts. (2) posts ranking: ranking all the test posts by the financial quality and selecting the top 10% of posts. we propose two strategies which are financial domain pre-training and multi-task learning. we are mainly concerned with shared word embedding learning through extracting financial semantic information on a large financial corpus for the pre-training process. financial embedding would give precise and helpful semantic information for models in order to settle downstream financial tasks. also, we introduce a multi-task learning approach that combines regression and classification tasks. in addition, multi-task learning can make good use of the relationship between tasks. specifically, we combine our models with multiple linear classifiers regarding both tasks and optimize the models by joint weighted loss calculation. at last, we introduce our methodology and solve the task as follows. section 2 elaborates on the erai shared task and the datasets for sub-tasks pairwise comparison and unsupervised ranking. we introduce our methodology and models in section 3 and present the experimental setup and official results in section 4. finally, we conclude our work in the final section 5. ## datasets we conduct experiments on two different datasets corresponding to two sub-tasks. for the mpp and ml values comparison settings, mpp label "1" : "mpp1" < "mpp2"; mpp label "0": "mpp1" > "mpp2" , on the other hand, ml label "1": "ml1" < "ml2"; ml label "0": "ml1" > "ml2". @xcite . erai-dataset-pairwise test dataset has 87 pieces data and identical settings with the train dataset except without the mpp and ml labels. ## conclusion this paper describes a multi-task learning approach based on financial domain plm for dealing with pairwise comparison and unsupervised ranking derived from rationales of amateur investors dataset. we demonstrate that joint loss optimization based on plm can achieve competitive results. we also observed that chinese-based plm performs better than english-based plm because the english translation cannot accurately express the exact meaning represented by the original chinese version. for the results regarding both tasks, our models obtain an accuracy of 48.28% and 52.87% for mpp and ml labels in the first task. besides, our second task achieves 14.02% and -4.17% for mpp and ml in the second task.