DCU - ML at the F in NLP -2022 ERAI Task: Investigating the Transferability of Sentiment Analysis Data for Evaluating Rationales of Investors (2022)

Abstract: In this paper, we describe our system for the FinNLP-2022 shared task: Evaluating the Rationales of Amateur Investors (ERAI). The ERAI shared tasks focuses on mining profitable information from financial texts by predicting the possible Maximal Potential Profit (MPP) and Maximal Loss (ML) based on the posts from amateur investors. There are two sub-tasks in ERAI: Pairwise Comparison and Unsupervised Rank, both target on the prediction of MPP and ML. To tackle the two tasks, we frame this task as a text-pair classification task where the input consists of two documents and the output is the label of whether the first document will lead to higher MPP or lower ML. Specifically, we propose to take advantage of the transferability of Sentiment Analysis data with an assumption that a more positive text will lead to higher MPP or higher ML to facilitate the prediction of MPP and ML. In experiment on the ERAI blind test set, our systems trained on Sentiment Analysis data and ERAI training data ranked 1st and 8th in ML and MPP pairwise comparison respectively. Code available in this link.

## introduction financial opinion mining @xcite , the focus of the finnlp-2022 shared task erai, has attracted the attention of the natural language processing (nlp) community in recent years @xcite @xcite for its potential use in financial analytic such as stock movement and volatility prediction @xcite . the finnlp-2022 shared task erai @xcite targets at extracting profitable information from financial documents particularly the posts from amateur investors. in the shared task, erai aims to predict the maximal potential profit (mpp) and maximal loss (ml) conveyed by the posts from amateur investors as such mined opinions could be possibly used to analyse the financial market. to tackle this task, we firstly frame it as a textpair classification task where the input consists of two documents from different amateur investors. and the output is the label of whether the first document will lead to higher mpp or lower ml. second, we take advantage of sentiment analysis data that have been shown to be useful in financial nlp @xcite @xcite . moreover, sentiment data are richresource and can be easily obtained @xcite and the erai data that is in relatively small scale could benefit from it. specifically, we use sentiment analysis data with an assumption that more positive text would give a higher profitable outcome. practically, we build the erai-like dataset based on sentiment analysis data via iteratively sample two documents from sentiment analysis corpus, if the sentiment polarity of the first document is more positive than the second document then we think the first document would lead to higher mpp as well as higher ml (as a more positive document could mean a more aggressive action which could lead to higher mpp but also with high risk resulting in higher ml). then we use the erai-like sentiment data to pre-train our model, of which the basic architecture is a text-pair classification model, and further fine-tune it with the erai training data. in experiment, we employ bert-chinese @xcite as our base model since erai data is in chinese. we experimented with three different strategies for training our model: 1) bert-senti: only use sentiment data thus fully relying on the transferability of sentiment data; 2) bert-erai: only use erai training data; 3) bert-senti+erai: fine-tune our model after training it using sentiment data. we submit the three systems trained based on the above strategies to erai pairwise comparison blind test set. we submit the first system for erai unsupervised ranking evaluation as it is only trained on sentiment data thus it's an unsupervised system. the experimental results on erai blind test set show that our bert-senti achieves 1st and 8th in ml and mpp pairwise comparison with accuracy of 59.77% and 52.87% respectively. our bert-senti achieves average mpp and ml of 13.97% and -8.25% for unsupervised ranking, which ranked at 10th and 13th respectively. ## methodology the main architecture of our proposed approach is shown in figure 1 , where we take advantage of the sentiment data to construct a erai-like dataset based on an assumption that a more positive document would lead to higher mpp and a more negative document would lead to higher ml. we pretrain our model based on the erai-like sentiment data followed by fine-tuning with the erai training data. the resulting systems are submitted to erai pairwise comparison and unsupervised ranking for evaluation.