Yet at the F in NLP -2022 ERAI Task: Modified models for evaluating the Rationales of Amateur Investors (2022)

Abstract: The financial reports usually reveal the recent development of the company and often cause the volatility in the company’s share price. The opinions causing higher maximal potential profit and lower maximal loss can help the amateur investors choose rational strategies. FinNLP-2022 ERAI task aims to quantify the opinions’ potentials of leading higher maximal potential profit and lower maximal loss. In this paper, different strategies were applied to solve the ERAI tasks. Valinna ‘RoBERTa-wwm’ showed excellent performance and helped us rank second in ‘MPP’ label prediction task. After integrating some tricks, the modified ‘RoBERTa-wwm’ outperformed all other models in ‘ML’ ranking task.

## introduction with the development of data mining and natural language processing techniques, more and more people are looking at textual information in various fields. one such area is finance. based on the financial corpora, researchers have pre-trained several models, like mengzi-fin @xcite and various versions of finbert @xcite @xcite , which help better learn the semantic layer of financial domain knowledge and more comprehensively learn the feature distribution of financial domain words and phrases. besides, there are a number of researchers who predict future events with texts @xcite , like mining the sentiment of financial posts to predict which stock has better returns @xcite . @xcite compares the rationales of experts and those of the crowd from stylistic and semantic perspectives to find the top-ranked opinions, and find they can increase potential returns and reduce downside risk. in addition, finnlp teams holds a series of workshops to help collect the research related to * *corresponding author ai in fintech @xcite @xcite , and to handle some frontier financial problems. this year they have partnered with emnlp and hold erai shared task to evaluate the rationales of amateur investors by predicting the maximal potential profit (mpp) and maximal loss (ml) of the given analytical opinions @xcite . we participated it and came up with several solutions like changing the optimizer, using 'stochastic weight averaging' method, which helped us rank 2nd in the 'mpp' classification subtask and 1st in the 'ml' ranking subtask. ## task setting and datasets there are two subtasks in the evaluating the rationales of amateur investors (erai) shared task @xcite , namely 'pairwise comparison' and 'unsupervised ranking'. the former one includes two binary classification tasks. one aims to determine, given the opinion pairs, whether the given opinion 1 will lead to higher maximal potential profit (mpp) than the given opinion 2, while another requires to determine whether the opinion 1 will to higher maximal loss (ml) than the given opinion 2. 'unsupervised ranking' task requires to find out the top 10% of the given posts that will lead to higher mpp. the datasets are collected from one of the largest financial social media platforms in taiwan, ptt stock foot_0 and mobile01 @xcite . and the posts are available in both english and chinese. there are 200 post pairs and their corresponding 'mpp' values and 'ml' values in the training phase, while 87 post pairs in testing phase of 'pairwise comparison' task and 210 posts in testing phase of 'unsupervised ranking' task. ## method we applied different strategies to handle the erai tasks. in both subtasks, we used a bert-type pretrained model @xcite , but we treated 'pairwise comparison' subtask as a sentence pair classification task and 'unsupervised ranking' subtask as a regression task. we processed chinese posts in one more step than english posts, i.e. turning traditional chinese into simplified chinese using 'zhconv' library foot_1 for that many models were pre-trained on simplified chinese corpus @xcite . then we removed the '\n' characters, urls, and emoticons in the posts. finally, we used a max length of 128 truncation of the posts and fed the cleaned posts into the models. ## experiments and results seven models were adapted in 'pairwise comparison' task and accuracy was selected as the evaluation metric, while three strategies were applied in 'unsupervised ranking' task and average mpp and ml are used as the evaluation metric, just as