F in S enti A : Sentiment Analysis in E nglish Financial Microblogs (2018)

Abstract: FinSentiA: Sentiment Analysis in English Financial Microblogs The objective of this paper is to report on the building of a Sentiment Analysis (SA) system dedicated to financial microblogs in English. The purpose of our work is to build a financial classifier that predicts the sentiment of stock investors in microblog platforms such as StockTwits and Twitter. Our contribution shows that it is possible to conduct such tasks in order to provide fine grained SA of financial microblogs. We extracted financial entities with relevant contexts and assigned scores on a continuous scale by adopting a deep learning method for the classification.

## introduction stock investment platforms like stocktwits 1 provide their users with many indicators of live events occurring on markets. to anticipate volatility, seen as a risk, investors use indexes such as the vix 2 (volatility index) which is interpreted as a measure of investor confidence with respect to s&p 500 stock index option prices @xcite . the vix is calculated from numerical values which module investor sentiment but there is an increasing interest in capturing opinions on finance from social media as an alternative source of sentiment. in a highly competitive and volatile domain such as investing, acquiring an insight into the public opinion of relevant and valuable economic signals can give a competitive edge and allow more informed investment decisions to be executed. microblog messages posted on social media such as twitter or stocktwits are central to determining these economic signals. sentiment analysis (sa) plays a central role for this purpose. intersecting between the fields of computational linguistics and natural language processing, the field has a long tradition in the use of tools to automatically determine sentiments in documents @xcite . it is admitted that sa tends to perform best when restricted to a specific domain and much work, so far, has focused on domains such as hotel and product reviews. in the financial domain some studies have used press articles with economic and financial focuses @xcite but few studies were conducted on financial microblog messages @xcite . and yet, due to the specific style of this type of messages, financial microblog sa requires adapted nlp methods. in addition, despite the need of financial experts to have a more revealing finegrained analysis, most studies so far have concentrated on classifying sentiments in categories at sentence or document level. recently, there have been some studies focusing on entities on a continuous scale @xcite , but they were based on alreadyidentified text spans which simplified the sentiment assignment task. finally, many studies have used supervised learning approaches but, as far as we know, few have applied deep learning methods. the purpose of our research is to develop a finegrained financial sa classifier for the ssix foot_0 project. in this paper, we present a supervised learning approach foot_1 which relies on a financial microblog gold standard for the training of a neural network. the sa is carried out in a fine grained fashion: i) by extracting entities, i.e. stocks, along with relevant contexts ii) by assigning sentiment scores on a continuous polarity scale ranging from 1 to 1. the remaining of this paper is divided into four sections. in section 2, we provide a short review of related work in the domain of financial sa. section 3 covers the method used to build the financial classifier. in section 4, we present the results followed by a short discussion. we conclude in section 5. 1 see https://stocktwits.com 2 cboe created the volatility index® (vix® index) which is a benchmark index to measure the market's expectation of future volatility. given the broad diversity of studies conducted in the domain of sa to date (see @xcite for a detailed overview of the domain) and due to the scope of the ssix project, we choose to focus this review of related work to the financial domain only. there have been two types of approaches: rulebased and supervised learning approaches. rulebased approaches consist in dictionary lookups for the assignment of polarities. typically, the first step of the task consists in building a lexicon of terms from a domainspecific corpus together with their respective polarities @xcite @xcite . the second step of the rulebased approach is to create rules for polarity lookups. (see @xcite @xcite @xcite for variations in the approach). the second type of approach in sa relies on supervised learning methods. when applied to financial texts, these machine learning (ml) methods make use of annotated documents that are used to "learn" specific financial features in order to subsequently classify. some studies use common feature engineering strategies relying on internal text features such as bagofwords, part of speech (pos) tags and named entities @xcite @xcite . other studies add polarities from lexicons as features of vectors @xcite and assign sentiment classes at sentence level. in terms of results, @xcite reported best results with a 0.869 threeclass accuracy. more recently, as part of the ssix project @xcite , @xcite gave the opportunity for semeval2017 candidates to conduct experiments based on the data set used in our experiment. the best scores were obtained by combining features such as pos, word embeddings @xcite and financial lexicons in ml approaches including neural network architectures @xcite . the presented approaches focus on sa at sentence/document level while our finergrained approach targets entities on a continuous scale. we use a different implementation of cosine similarity that targets entities, and we do not use manually selected text spans for sentiment assignment. ## method this section covers the method used to build the financial classifier. we describe the evaluation corpus before detailing the experimental setup. ## results and discussion figure 2 shows the distribution of the predicted (right) and the test set (left) of sentiment scores at entity level. we can see clearly that the scores predicted by the classifier have the same distribution pattern as the ones from the test set from the gold standard. we notice, though, that there is a lack of predicted scores in the intervals [0.5, 1] and [0.5, 1]. this can be explained by the inability of our model to learn from the training set due to the very small number of scores in these intervals in the gs. this can be seen in the scatter plot of the training set occurrences (figure 3 ). as the closest existing work to ours is @xcite , we computed our system performance using the exact same evaluation method they used. we decided not to use the predefined manually selected spans they use. instead, we kept the real world configuration, which means dealing with entire messages, as they are, even if that implies adding a preprocessing step and dealing with the noise. our system had 0.713 versus 0.733 for @xcite . as their evaluation method calculate the average of the cosine similarities at message level and not at entity level (both for the predicted and the test set), we decided to modify the method (see section 3.2) for it to take into consideration the entity scores distinctively, which keeps the finegrained characteristic. as we cannot compare totally our work to theirs we decided to have an internal baseline. we built a baseline model with word2vec and sentiwordnet as features run through a 1hiddenlayer neural network. the cosine similarity function gives results taking into consideration the continuous values of the scores (see one point of discussion is about the cosine similarity measure which is the most used in the sentiment analysis domain. a more suitable evaluation metric that takes into consideration not only the semantic similarity but also the sentiment similarity @xcite as not only the semantic space is important but also the emotional space. to add another perspective to our results we computed as well the accuracy and the f1scores of our model on a negative and positive classes basis ( ]1,0] and ]0,1] ). our main evaluation metric still the one described above (section 3.2) as it maintains the fine grained aspect of our work. one point of discussion is about the cosine similarity measure. it may be relevant for semantic similarity but, in the case of sentiment similarity, it is not only the proximity of the vectors that is important but also the polarities of the values. two vectors may be close to each other and yet have opposite values. this distinction is important in the measurement of sentiments. ## conclusion building a financial classifier for sa in the domain of financial microblogs faces many challenges involving linguistic engineering and machine learning tasks. our contribution shows that it is possible to conduct such tasks in order to provide finegrained sa of financial microblogs. we extracted financial entities with relevant contexts and assigned scores on a continuous scale by adopting a deep learning method for the classification. further steps involve exploring a more suitable evaluation metric to our case, broadening the classifier features, increasing the training sets and applying some sensitivity analysis.