prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. Which stock market sector achieved the best performance?
Context 1: Introduction
Natural Language Processing (NLP) has increasingly attracted the attention of the financial community  This trend can be explained by at least three major factors  The first factor refers to the business perspective  It is the economics of gaining competitive advantage using alternative sources of data and going beyond historical stock prices, thus, trading by analyzing market news automatically  The second factor is the major advancements in the technologies to collect, store, and query massive amounts of user-generated data almost in real-time  The third factor refers to the progress made by the NLP community in understanding unstructured text  Over the last decades the number of studies using NLP for financial forecasting has experienced exponential growth  According to BIBREF0 , until 2008, less than five research articles were published per year mentioning both “stock market” and “text mining” or “sentiment analysis” keywords  In 2012, this number increased to slightly more than ten articles per year  The last numbers available for 2016 indicates this has increased to sixty articles per year 
The ability to mechanically harvest the sentiment from texts using NLP has shed light on conflicting theories of financial economics  Historically, there has been two differing views on whether disagreement among market participants induces more trades  The “non-trade theorem” BIBREF1 states that assuming all market participants have common knowledge about a market event, the level of disagreement among the participants does not increase the number of trades but only leads to a revision of the market quotes  In contrast, the theoretically framework proposed in BIBREF2 advocates that disagreement among market participants increases trading volume  Using textual data from Yahoo and RagingBull com message boards to measure the dispersion of opinions (positive or negative) among traders, it was shown in BIBREF3 that disagreement among users' messages helps to predict subsequent trading volume and volatility  Similar relation between disagreement and increased trading volume was found in BIBREF4 using Twitter posts  Additionally, textual analysis is adding to the theories of medium-term/long-term momentum/reversal in stock markets BIBREF5   The unified Hong and Stein model BIBREF6 on stock's momentum/reversal proposes that investors underreact to news, causing slow price drifts, and overreact to price shocks not accompanied by news, hence inducing reversals  This theoretical predicated behaviour between price and news was systematically estimated and supported in BIBREF7 , BIBREF8 using financial media headlines and in BIBREF9 using the Consumer Confidence Index® published by The Conference Board BIBREF10   Similarly, BIBREF11 uses the Harvard IV-4 sentiment lexicon to count the occurrence of words with positive and negative connotation of the Wall Street Journal showing that negative sentiment is a good predictor of price returns and trading volumes 
Accurate models for forecasting both price returns and volatility are equally important in the financial domain  Volatility measures how wildly the asset is expected to oscillate in a given time period and is related to the second moment of the price return distribution  In general terms, forecasting price returns is relevant to take speculative positions  The volatility, on the other hand, measures the risk of these positions  On a daily basis, financial institutions need to assess the short-term risk of their portfolios  Measuring the risk is essential in many aspects  It is imperative for regulatory capital disclosures required by banking supervision bodies  Moreover, it is useful to dynamically adjust position sizing accordingly to market conditions, thus, maintaining the risk within reasonable levels 
Although, it is crucial to predict the short-term volatility from the financial markets application perspective, much of the current NLP research on volatility forecasting focus on the volatility prediction for very long-term horizons (see BIBREF12 , BIBREF13 , BIBREF14 , BIBREF15 , BIBREF16 )  Predominately, these works are built on extensions of the bag-of-words representation that has the main drawback of not capturing word order  Financial forecasting, however, requires the ability to capture semantics that is dependent upon word order  For example, the headline “Qualcomm sues Apple for contract breach” and “Apple sues Qualcomm for contract breach” trigger different responses for each stock and for the market aggregated index, however, they share the same bag-of-words representation  Additionally, these works use features from a pretrained sentiment analyis model to train the financial forecasting model  A key limitation of this process is that it requires a labelled sentiment dataset  Additionally, the error propagation is not end-to-end  In this work, we fill in the gaps of volatility prediction research in the following manner:
Related work
Previous work in BIBREF12 incorporates sections of the “Form 10-K” to predict the volatility twelve months after the report is released  They train a Support Vector Regression model on top of sparse representation (bag-of-words) with standard term weighting (e g  Term-Frequency)  This work was extended in BIBREF13 , BIBREF14 , BIBREF15 , BIBREF16 by employing the Loughran-McDonald Sentiment Word Lists BIBREF20 , which contain three lists where words are grouped by their sentiments (positive, negative and neutral)  In all these works, the textual representation is engineered using the following steps: 1) For each sentiment group, the list is expanded by retrieving 20 most similar words for each word using Word2Vec word embeddings BIBREF21   2) Finally, each 10-K document is represented using the expanded lists of words  The weight of each word in this sparse representation is defined using Information Retrieval (IR) methods such as term-frequency (tf) and term-frequency with inverted document frequency (tfidf)  Particularly, BIBREF16 shows that results can be improved using enhanced IR methods and projecting each sparse feature into a dense space using Principal Component Analysis (PCA) 
The works described above ( BIBREF13 , BIBREF14 , BIBREF15 , BIBREF16 ) target long-horizon volatility predictions (one year or quarterly BIBREF16 )  In particular, BIBREF16 and BIBREF15 uses market data (price) features along with the textual representation of the 10-K reports  These existing works that employ multi-modal learning BIBREF22 are based on a late fusion approach  For example, stacking ensembles to take into account the price and text predictions BIBREF16   In contrast, our end-to-end trained model can learn the joint distribution of both price and text 
Predicting the price direction rather than the volatility was the focus in BIBREF23   They extracted sentiment words from Twitter posts to build a time series of collective Profile of Mood States (POMS)  Their results show that collective mood accurately predicts the direction of Down Jones stock index (86 7% accuracy)  In BIBREF24 handcrafted text representations including term count, noun-phrase tags and extracted named entities are employed for predicting stock market direction using Support Vector Machine (SVM)  An extension of Latent Dirichlet Allocation (LDA) is proposed in BIBREF25 to learn a joint latent space of topics and sentiments 
Our deep learning models bear a close resemblance to works focused on directional price forecasting BIBREF26 , BIBREF27   In BIBREF26 , headline news are processed using Stanford OpenIE to generate triples that are fed into a Neural Tensor Network to create the final headline representation  In BIBREF27 , a character-level embedding is pre-trained in an unsupervised manner  The character embedding is used as input to a sequence model to learn the headline representation  Particularly, both works average all headline representations in a given day, rather than attempting to weight the most relevant ones  In this work, we propose a neural attention mechanism to capture the News Relevance and provide experimental evidence that it is a key component of the end-to-end learning process  Our attention extends the previous deep learning methods from BIBREF26 , BIBREF27  
Despite the fact that end-to-end deep learning models have attained state-of-the-art performance, the large number of parameters make them prone to overfitting  Additionally, end-to-end models are trained from scratch requiring large datasets and computational resources  Transfer learning (TL) alleviates this problem by adapting representations learnt from a different and potentially weakly related source domain to the new target domain  For example, in computer vision tasks the convolutional features learnt from ImageNet BIBREF28 dataset (source domain) have been successfully transferred to multiple domain target tasks with much smaller datasets such as object classification and scene recognition BIBREF29   In this work, we consider TL in our experiments for two main reasons  First, it address the question whether our proposed dataset is suitable for end-to-end training since the performance of the transferred representations can be compared with end-to-end learning  Second, it is still to be investigated which dataset transfers better to the forecasting problem  Recently, the NLP community has focused on universal representations of sentences BIBREF17 , BIBREF19 , which are dense representations that carry the meaning of a full sentence  BIBREF17 found that transferring the sentence representation trained on the Stanford Natural Language Inference (SNLI) BIBREF30 dataset achieves state-of-the-art sentence representations to multiple NLP tasks (e g  sentiment analysis, question-type and opinion polarity)  Following BIBREF17 , in this work, we investigate the suitability of SNLI and Reuters RCV1 BIBREF31 datasets to transfer learning to the volatility forecasting task  To the best of our knowledge, the hierarchical attention mechanism at headline level, proposed in our work, has not being applied to volatility prediction so far; neither has been investigated the ability to transfer sentence encoders from source datasets to the target forecasting problem (Transfer Learning) 
Our dataset
Our corpus covers a broad range of news including news around earnings dates and complements the 10-K reports content  As an illustration, the headlines “Walmart warns that strong U S  dollar will cost $15B in sales” and “Procter & Gamble Co raises FY organic sales growth forecast after sales beat” describe the company financial conditions and performance from the management point of view – these are also typical content present in Section 7 of the 10-K reports 
In this section, we describe the steps involved in compiling our dataset of financial news at stock level, which comprises a broad range of business sectors 
Sectors and stocks
The first step in compiling our corpus was to choose the constituents stocks  Our goal was to consider stocks in a broad range of sectors, aiming a diversified financial domain corpus  We found that Exchange Traded Funds (ETF) provide a mechanical way to aggregate the most relevant stocks in a given industry/sector  An ETF is a fund that owns assets, e g  stock shares or currencies, but, unlike mutual funds are traded in stock exchanges  These ETFs are extremely liquid and track different investment themes  We decided to use SPDR Setcor Funds constituents stocks in our work since the company is the largest provider of sector funds in the United States  We included in our analysis the top 5 (five) sector ETFs by financial trading volume (as in Jan/2018)  Among the most traded sectors we also filtered out the sectors that were similar to each other  For example, the Consumer Staples and Consumer Discretionary sectors are both part of the parent Consumer category  For each of the top 5 sectors we selected the top 10 holdings, which are deemed the most relevant stocks  tbl:stockuniverse, details our dataset sectors and its respective stocks 
Stock specific data
We assume that an individual stock news as the one that explicitly mention the stock name or any of its surface forms in the headline  As an illustration, in order to collect all news for the stock code PG, Procter & Gamble company name, we search all the headlines with any of these words: Procter&Gamble OR Procter and Gamble OR P&G  In this example, the first word is just the company name and the remaining words are the company surface forms 
We automatically derived the surface forms for each stock by starting with a seed of surface forms extracted from the DBpedia Knowledge Base (KB)  We then applied the following procedure:
Relate each company name with the KB entity unique identifier 
Retrieve all values of the wikiPageRedirects property  The property holds the names of different pages that points to the same entity/company name  This step sets the initial seed of surface forms 
Manually, filter out some noisy property values  For instance, from the Procter & Glamble entity page we were able to automatically extract dbr:Procter_and_gamble and dbr:P_&_G, but had to manually exclude the noisy associations dbr:Female_pads and dbr:California_Natural 
The result of the steps above is a dictionary of surface forms $wd_{sc}$  
Stock headlines
Our corpus is built at stock code level by collecting headlines from the Reuters Archive  This archive groups the headlines by date, starting from 1 January 2007  Each headline is a html link (<a href> tag) to the full body of the news, where the anchor text is the headline content followed by the release time  For example, the page dated 16 Dec 2016 has the headline “Procter & Gamble appoints Nelson Peltz to board 5:26PM UTC” 
For each of the 50 stocks (5 sectors times 10 stocks per sector) selected using the criteria described in sub:corpussecstock, we retrieved all the headlines from the Reuters Archive raging from 01/01/2007 to 30/12/2017  This process takes the following steps:
For a given stock code ( $sc$ ) retrieve all surface forms $wd_{sc}$  
For each day, store only the headlines content matching any word in $wd_{sc}$   For each stored headline we also store the time and timezone 
Convert the news date and time to Eastern Daylight Time (EDT) 
Categorize the news release time  We consider the following category set: {before market, during market , after market, holidays, weekends}  during market contains news between 9:30AM and 4:00PM  before market before 9:30AM and after market after 4:00PM 
The time categories prevents any misalignment between text and stock price data  Moreover, it prevents data leakage and, consequently, unrealistic predictive model performance  In general, news released after 4:00PM EDT can drastically change market expectations and the returns calculated using close to close prices as in the GARCH(1,1) model (see eq:closingreturn)  Following BIBREF3 , to deal with news misalignment, news issued after 4:00PM (after market) are grouped with the pre-market (before market) on the following trading day 
tbl:stocktimecat shows the distribution of news per sector for each time category  We can see a high concentration of news released before the market opens (55% on average)  In contrast, using a corpus compiled from message boards, a large occurrence of news during market hours was found BIBREF3   This behaviour indicating day traders' activity  Our corpus comprise financial news agency headlines, a content more focused on corporate events (e g  lawsuits, merges & acquisitions, research & development) and on economic news (see tbl:stockheadlinesexmaples for a sample of our dataset)  These headlines are mostly factual  On the other hand, user-generated content such as Twitter and message boards (as in BIBREF3 , BIBREF4 ) tends to be more subjective 
U S  macroeconomic indicators such as Retail Sales, Jobless Claims and GDP are mostly released around 8:30AM (one hour before the market opens)  These numbers are key drivers of market activity and, as such, have a high media coverage  Specific sections of these economic reports impact several stocks and sectors  Another factor that contribute to the high activity of news outside regular trading hours are company earnings reports  These are rarely released during trading hours  Finally, before the market opens news agencies provide a summary of the international markets developments, e g  the key facts during the Asian and Australian trading hours  All these factors contribute to the high concentration of pre-market news 
Background
We start this section by reviewing the GARCH(1,1) model, which is a strong benchmark used to evaluate our neural model  We then review the source datasets proposed in the literature that were trained independently and transfered to our volatility prediction model  Finally, we review the general architectures of sequence modelling and attention mechanisms 
GARCH model
Financial institutions use the concept of “Value at risk” to measure the expected volatility of their portfolios  The widespread econometric model for volatility forecasting is the Generalized Autoregressive Conditional Heteroskedasticity (GARCH) BIBREF32 , BIBREF33   Previous research shows that the GARCH(1,1) model is hard to beat  For example, BIBREF34 compared GARCH(1,1) with 330 different econometric volatility models showing that they are not significantly better than GARCH(1,1)  Let $p_t$ be the price of an stock at the end of a trading period with closing returns $r_t$ given by
$$r_t = \frac{p_t}{p_{t-1}} - 1 $$   (Eq  29)
The GARCH process explicitly models the time-varying volatility of asset returns  In the GARCH(1,1) specification the returns series $r_t$ follow the process:
$$r_t &= \mu + \epsilon _t  \\ \epsilon _t &= \sigma _t z_t  \\ \sigma ^2_t &= a_0 + a_1 \epsilon _{t-1}^2 + b_1 \sigma _{t-1}^2$$   (Eq  30)
where $\mu $ is a constant (return drift) and $z_t$ is a sequence of i i d  random variables with mean zero and unit variance  It is worth noting that although the conditional mean return described in eq:garchcondmean has a constant value, the conditional volatility $\sigma _t$ is time-dependent and modeled by eq:att 
The one-step ahead expected volatility forecast can be computed directly from eq:garchcondvariance and is given by
$$E_T[\sigma _{T+1}^2] = a_0 + a_1 E_T[\epsilon ^2] + b_1 E_T[\sigma _{T}^2] $$   (Eq  32)
In general, the $t^{\prime }$ -steps ahead expected volatility $E_T[\sigma _{T+t^{\prime }}^2]$ can be easily expressed in terms of the previous step expected volatility  It is easy to prove by induction that the forecast for any horizon can be represented in terms of the one-step ahead forecast and is given by
$$E_T[\sigma _{T+t^{\prime }}^2] - \sigma _u^2 = (a_1 + b_1)^{(t^{\prime } -1)} \left(E_T[\sigma _{T+1}^2] - \sigma _u^2\right)$$   (Eq  33)
where $\sigma _u$ is the unconditional volatility:
$$\sigma _u = \sqrt{a_0 / (1 - a_1 - b_1)} $$   (Eq  34)
From the equation above we can see that for long horizons, i e  $t^\prime \rightarrow \infty $ , the volatility forecast in eq:forecastrecursive converges to the unconditional volatility in eq:unvar 
All the works reviewed in sec:introduction ( BIBREF12 , BIBREF13 , BIBREF14 , BIBREF15 , BIBREF16 ) consider GARCH(1,1) benchmark  However, given the long horizon of their predictions (e g  quarterly or annual), the models are evaluated using the unconditional volatility $\sigma _u$ in eq:unvar  In this work, we focus on the short-term volatility prediction and use the GARCH(1,1) one-day ahead conditional volatility prediction in eq:forecastoneperiod to evaluate our models 
Let $\sigma _{t+1}$ denote the ex-post “true” daily volatility at a given time $t$   The performance on a set with $N$ daily samples can be evaluated using the standard Mean Squared Error ( $MSE$ ) and Mean Absolute Error ( $MAE$ )
$$MSE &= \frac{1}{N} \sum _{t=1}^{N} \left( E_t[\sigma _{t+1}] - \sigma _{t+1}\right)^2  \\ MAE &= \frac{1}{N} \sum _{t=1}^{N}\left|E_t[\sigma _{t+1}] - \sigma _{t+1} \right|$$   (Eq  36)
Additionally, following BIBREF35 , the models are also evaluated using the coefficient of determination $R^2$ of the regression
$$\sigma _{t+1} = a + b E_t[\sigma _{t+1}] + e_t$$   (Eq  37)
where
$$R^2 = 1 - \frac{\sum _{t=1}^{N}e^{2}_{t}}{\sum _{t=1}^{N}\left(E_t[\sigma _{t+1}] - \frac{1}{N} \sum _{t=1}^{N}E_t[\sigma _{t+1}]\right)^{2}}$$   (Eq  38)
One of the challenges in evaluating GARCH models is the fact that the ex-post volatility $\sigma _{t+1}$ is not directly observed  Apparently, the squared daily returns $r_{t+1}^{2}$ in eq:closingreturn could stand as a good proxy for the ex-post volatility  However, the squared returns yield very noisy measurements  This is a direct consequence of the term $z^t$ that connects the squared return to the latent volatility factor in eq:garchwhitenoise  The use of intraday prices to estimate the ex-post daily volayility was first proposed in BIBREF35   They argue that volatility estimators using intraday prices is the proper way to evaluate the GARCH(1,1) model, as opposed to squared daily returns  For example, considering the Deutsche Mark the GARCH(1,1) model $R^2$ improves from $0 047$ (squared returns) to $0 33$ (intraday returns) BIBREF35  
It is clear from the previous section that any volatility model evaluation using the noisy squared returns as the ex-post volatility proxy will lead to very poor performance  Therefore, high-frequency intraday data is fundamental to short-term volatility performance evaluation  However, intraday data is difficult to acquire and costly  Fortunately, there are statistically efficient daily volatility estimators that only depend on the open, high, low and close prices  These price “ranges” are widely available  In this section, we discuss these estimators 
Let $O_t$ , $H_t$ , $L_t$ , $C_t$ be the open, high, low and close prices of an asset in a given day $t$   Assuming that the daily price follows a geometric Brownian motion with zero drift and constant daily volatility $\sigma $ , Parkinson (1980) derived the first daily volatility estimator
$$\widehat{\sigma _{PK,t}^2} = \frac{\ln \left(\frac{H_t}{L_t}\right)^2}{4\ln (2)} $$   (Eq  41)
which represents the daily volatility in terms of its price range  Hence, it contains information about the price path  Given this property, it is expected that $\sigma _{PK}$ is less noisy than the volatility calculated using squared returns  The Parkinson's volatility estimator was extended by Garman-Klass (1980) which incorporates additional information about the opening ( $O_t$ ) and closing ( $C_t$ ) prices and is defined as
$$\widehat{\sigma _{GK,t}^{2}} = \frac{1}{2} \ln \left(\frac{H_t}{L_t}\right)^2 - (2\ln (2) - 1) \ln \left(\frac{C_t}{O_t}\right)^2 $$   (Eq  42)
The relative noisy of different estimators $\hat{\sigma }$ can be measured in terms of its relative efficiency to the daily volatility $\sigma $ and is defined as
$$e\left(\widehat{\sigma ^{2}}, \sigma ^2\right) \equiv \frac{Var[\sigma ^2]}{Var[\widehat{\sigma ^{2}}]}$$   (Eq  43)
where $Var[\cdot ]$ is the variance operator  It follows directly from eq:garchwhitenoise that the squared return has efficiency 1 and therefore, very noisy  BIBREF36 reports Parkinson ( $\widehat{\sigma _{PK,t}^2}$ ) volatility estimator has 4 9 relative efficiency and Garman-Klass ( $\widehat{\sigma _{GK,t}^2}$ ) 7 4  Additionally, all the described estimators are unbiased 
Many alternative estimators to daily volatility have been proposed in the literature  However, experiments in BIBREF36 rate the Garman-Klass volatility estimator as the best volatility estimator based only on open, high, low and close prices  In this work, we train our models to predict the state-of-the-art Garman-Klass estimator  Moreover, we evaluate our models and GARCH(1,1) using the metrics described in sub:evalution, but with the appropriate volatility proxies, i e  Parkinson and Garman-Klass estimators 
Transfer Learning from other source domains
Vector representations of words, also known as Word embeddings BIBREF21 , BIBREF37 , that represent a word as a dense vector has become the standard building blocks of almost all NLP tasks  These embeddings are trained on large unlabeled corpus and are able to capture context and similarity among words 
Some attempts have been made to learn vector representations of a full sentence, rather than only a single word, using unsupervised approaches similar in nature to word embeddings  Recently, BIBREF17 showed state-of-the-art performance when a sentence encoder is trained end-to-end on a supervised source task and transferred to other target tasks  Inspired by this work, we investigate the performance of sentence encoders trained on the Text categorization and Natural Language Inference (NLI) tasks and use these encoders in our main short-term volatility prediction task 
A generic sentence encoder $S_e$ receives the sentence words as input and returns a vector representing the sentence  This can be expressed as a mapping
$$S_e \colon \mathbb {R}^{T^{S} \times d_w} \rightarrow \mathbb {R}^{d_S}$$   (Eq  45)
from a variable size sequence of words to a sentence vector $S$ of fixed-size $d_S$ , where $T^{S}$ is the sentence number of words and $d_w$ is the pre-trained word embedding dimension 
In the following sections, we describe the datasets and architectures to train the sentence encoders of the auxiliary transfer learning tasks 
The Reuters Corpus Volume I (RCV1) is corpus containing 806,791 news articles in the English language collected from 20/08/1996 to 19/08/1997 BIBREF31   The topic of each news was human-annotated using a hierarchical structure  At the top of the hierarchy, lies the coarse-grained categories: CCAT (Corporate), ECAT (Economics), GCAT (Government), and MCAT (Markets)  A news article can be assigned to more than one category meaning that the text categorization task is mutilabel  Each news is stored in a separate XML file  lst:rcv1xmlexample shows the typical structure of an article 
<?xml version="1 0" encoding="iso-8859-1" ?>
<newsitem itemid="6159" id="root" date="1996-08-21" xml:lang="en">
<headline>Colombia raises internal coffee price </headline>
<dateline>BOGOTA 1996-08-21</dateline>
<copyright>(c) Reuters Limited 1996</copyright>
<metadata>
<codes class="bip:topics:1 0">
<code code="C13">
<editdetail attribution="Reuters BIP Coding Group" action="confirmed" date="1996-08-21"/>
</code>
<code code="C31">
<editdetail attribution="Reuters BIP Coding Group" action="confirmed" date="1996-08-21"/>
</code>
<code code="CCAT">
<editdetail attribution="Reuters BIP Coding Group" action="confirmed" date="1996-08-21"/>
</code>
<code code="M14">
<editdetail attribution="Reuters BIP Coding Group" action="confirmed" date="1996-08-21"/>
</code>
<code code="M141">
<editdetail attribution="Reuters BIP Coding Group" action="confirmed" date="1996-08-21"/>
</code>
<code code="MCAT">
<editdetail attribution="Reuters BIP Coding Group" action="confirmed" date="1996-08-21"/>
</code>
</codes>
</metadata>
</newsitem>
The RCV1 dataset is not released with a standard train, validation, test split  In this work, we separated 15% of samples as a test set for evaluation purposes  The remaining samples were further split leaving 70% and 15% for training and validation, respectively 
Regarding the categories distribution, we found that, from the original 126 categories, 23 categories were never assigned to any news; therefore, were disregarded  From the 103 classes left we found a high imbalance among the labels with a large number of underrepresented categories having less than 12 samples  The very low number of samples for these minority classes brings a great challenge to discriminate the very fine-grained categories  Aiming to alleviate this problem, we grouped into a same class all categories below the second hierarchical level  For example, given the root node CCAT (Corporate) we grouped C151 (ACCOUNTS/EARNINGS), C1511 (ANNUAL RESULTS) and C152 (COMMENT/FORECASTS) into the direct child node C15 (PERFORMANCE)  Using this procedure the original 103 categories where reduced to 55  One of the benefits of this procedure was that the less represented classes end up having around thousand samples compared with only 12 samples in the original dataset 
fig:rcv1arch, shows the architecture for the end-to-end text categorization task  On the bottom of the architecture $S_e$ receives word embeddings and outputs a sentence vector $S$   The $S$ vector pass through a fully connected (FC) layer with sigmoid activation function that outputs a vector $\hat{y} \in \mathbb {R}^{55}$ with each element $\hat{y}_j \in [0,1]$  
The architecture described above is trained under the assumption that each category is independent but not mutually exclusive since a sample can have more than one category assigned (multilabel classification)  The loss per sample is the average log loss across all labels:
$$\mathcal {L}(\hat{y}, y) = - \sum _{i=1}^{55}\left( y_i \log (\hat{y}_i) + (1-y_{i}) \log (1-\hat{y}_{i}) \right)$$   (Eq  48)
where the index $i$ runs over the elements of the predicted and true vectors 
Given the high categories imbalance, during the training we monitor the $F_1$ metric of the validation set and choose the model with the highest value 
Stanford Natural Language Inference (SNLI) dataset BIBREF30 consist of 570,000 pairs of sentences  Each pair has a premise and a hypothesis, manually labeled with one of the three labels: entailment, contradiction, or neutral  The SNLI has many desired properties  The labels are equally balanced, as opposed to the RCV1 dataset  Additionally, language inference is a complex task that requires a deeper understanding of the sentence meaning making this dataset suitable for learning supervised sentence encoders that generalize well to other tasks BIBREF17   tbl:snliexmaples, shows examples of SNLI dataset sentence pairs and its respective labels 
In order to learn sentence encoders that can be transfered to other tasks unambiguously, we consider a neural network architecture for the sentence encoder with shared parameters between the premise and hypothesis pairs as in BIBREF17  
fig:snliarch, describes the neural network architecture  After each premise and hypothesis is encoded into $S_p$ and $S_h$ , respectively, we have a fusion layer  This layer has no trainable weights and just concatenate each sentence embedding  Following BIBREF17 , we add two more matching methods: the absolute difference $\vert S_p - S_h \vert $ and the element-wise $S_p \odot S_h$   Finally, in order to learn the pair representation, $S_ph$ is feed into and FC layer with rectified linear unit (ReLU) activation function, which is expressed as $f(x) = \log (1 + e^x)$   The last softmax layer outputs the probability of each class 
Finally, the NLI classifier weights are optimized in order to minimize the categorical log loss per sample
$$\mathcal {L}(\hat{y}, y) = - \sum _{j=1}^{3}y_i \log (\hat{y}_i)$$   (Eq  52)
During the training, we monitor the validation set accuracy and choose the model with the highest metric value 
Sequence Models
We start this section by reviewing the Recurrent Neural Network (RNN) architecture and its application to encode a sequence of words 
RNN's are capable of handling variable-length sequences, this being a direct consequence of its recurrent cell, which shares the same parameters across all sequence elements  In this work, we adopt the Long Short-Term Memory (LSTM) cell BIBREF38 with forget gates $f_t$ BIBREF39   The LSTM cell is endowed with a memory state that can learn representations that depend on the order of the words in a sentence  This makes LSTM more fit to find relations that could not be captured using standard bag-of-words representations 
Let $x_1, x_2, \cdots , x_T$ be a series of observations of length $T$ , where $x_t \in \mathbb {R}^{d_w}$   In general terms, the LSTM cell receives a previous hidden state $h_{t-1}$ that is combined with the current observation $x_t$ and a memory state $C_t$ to output a new hidden state $h_t$   This internal memory state $C_{t}$ is updated depending on its previous state and three modulating gates: input, forget, and output  Formally, for each step $t$ the updating process goes as follows (see fig:lstmcell for a high level schematic view): First, we calculate the input $i_t$ , forget $T$0 , and output $T$1 gates:
$$i_t &= \sigma _s\left(W_i x_t + U_i h_{t-1} + b_i\right) \\ f_t &= \sigma _s\left(W_f x_t + U_f h_{t-1} + b_f\right) \\ o_t &= \sigma _s\left(W_o x_t + U_o h_{t-1} + b_o\right)$$   (Eq  54)
where $\sigma _s$ is the sigmoid activation  Second, a candidate memory state $\widetilde{C}_t$ is generated:
$$\widetilde{C}_t = \tanh \left(W_c x_t + U_c h_{t-1} + b_c\right)$$   (Eq  55)
Now we are in a position to set the final memory state $C_t$   Its value is modulated based on the input and forget gates of eq:inputforgetgates and is given by:
$$C_t = i_t \odot \widetilde{C}_t + f_t \odot C_{t-1}$$   (Eq  56)
Finally, based on the memory state and output gate of eq:inputforgetgates, we have the output hidden state
$$h_t = o_t \odot \tanh \left(C_t\right)$$   (Eq  57)
Regarding the trainable weights, let $n$ be the LSTM cell number of units  It follows that $W$ 's and $U$ 's matrices of the affine transformations have ${n \times d_w}$ and ${n \times n}$ dimensions, respectively  Its bias terms $b$ 's are vectors of size $n$   Consequently, the total number of parameters is $4 (n d_w + n^2 + n)$ and does not depend on the sequence number of time steps $T$  
We see that the LSTM networks are able to capture temporal dependencies in sequences of arbitrary length  One straightforward application is to model the Sentence encoder discussed in sec:transferlearning, which outputs a sentence vector representation using its words as input 
Given a sequence of words $\left\lbrace w_t\right\rbrace _{t=1}^{T}$ we aim to learn the words hidden state $\left\lbrace h_t\right\rbrace _{t=1}^{T}$ in a way that each word captures the influence of its past and future words  The Bidirectional LSTM (BiLSTM) proposed in BIBREF40 is an LSTM that “reads” a sentence, or any sequence in general, from the beginning to the end (forward) and the other way around (backward)  The new state $h_t$ is the concatenation
$$h_t = [\overrightarrow{h_t}, \overleftarrow{h_t}]$$   (Eq  59)
where
$$\overrightarrow{h_t} &= \text{LSTM}\left(w_1, \cdots , w_T\right) \\ \overleftarrow{h_t} &= \text{LSTM}\left(w_T, \cdots , w_1\right) \\$$   (Eq  60)
Because sentences have different lengths, we need to convert the $T$ concatenated hidden states of the BiLSTM into a fixed-length sentence representation  One straightforward operation is to apply any form of pooling  Attention mechanism is an alternative approach where the sentence is represented as an weighted average of hidden states where the weights are learnt end-to-end 
In the next sections we describe the sentence encoders using pooling and attention layers 
The max-pooling layer aims to extract the most salient word features all over the sentence  Formally, it outputs a sentence vector representation $S_{MP} \in \mathbb {R}^{2n}$ such that
$$S_{MP} = \max _{t=1}^{T} h_t$$   (Eq  62)
where $h_t$ is defined in eq:htconcat and the $\max $ operator is applied over the time steps dimension  fig:bilstmmaxpool illustrates the BiLSTM max-pooling (MP) sentence encoder 
The efficacy of the max-pooling layer was assessed in many NLP studies  BIBREF41 employed a max-pooling layer on top of word representations and argues that it performs better than mean pooling  Experimental results in BIBREF17 show that among three types of pooling (max, mean and last) the max-pooling provides the most universal sentence representations in terms of transferring performance to other tasks  Grounded on these studies, in this work, we choose the BiLSTM max-pooling as our pooling layer of choice 
Attention mechanisms were introduced in the deep learning literature to overcome some simplifications imposed by pooling operators  When we humans read a sentence, we are able to spot its most relevant parts in a given context and disregard information that is redundant or misleading  The attention model aims to mimic this behaviour 
Attention layers were proposed for different NLP tasks  For example, NLI, with cross-attention between premise and hypothesis, Question & Answering and Machine Translation (MT)  Specifically in the Machine Translation task, each word in the target sentence learns to attend the relevant words of the source sentence in order to generate the sentence translation 
A sentence encoder with attention (or self-attentive) BIBREF42 , BIBREF43 , BIBREF44 assigns different weights to the own words of the sentence; therefore, converting the hidden states into a single sentence vector representation 
Considering the word hidden vectors set $\lbrace h_1, \cdots , h_T\rbrace $ where $h_t \in \mathbb {R}^n$ , the attention mechanism is defined by the equations:
$$\tilde{h}_t &= \sigma \left(W h_t + b \right) \\ \alpha _{t} &= \frac{\exp ({v^{\intercal } \cdot \tilde{h}_t} )}{\sum _{t} \exp ({v \cdot \tilde{h}_t})} \\ S_{A_w} &= \sum _{t} \alpha _{t} h_t$$   (Eq  66)
where $W \in \mathbb {R}^{d_a \times n}$ , $b \in \mathbb {R}^{d_a \times 1}$ , and $v \in \mathbb {R}^{d_a \times 1}$ are trainable parameters 
We can see that the sentence representation $S_{A_w}$ is a weighted average of the hidden states  fig:bilstminneratt provides a schematic view of the BiLSTM attention, where we can account the attention described in eq:att as a two layer model with a dense layer ( $d_a$ units) followed by another dense that predicts $\alpha _t$ (single unit) 
Methodology
In this section, we first introduce our problem in a deep multimodal learning framework  We then present our neural architecture, which is able to address the problems of news relevance and novelty  Finally, we review the methods applied to learn commonalities between stocks (global features) 
Problem statement
Our problem is to predict the daily stock volatility  As discussed in subsub:rangevolestimators, the Gaman-Klass estimator $\widehat{\sigma _{GK,t}}$ in eq:volgk is a very efficient short-term volatility proxy, thus, it is adopted as our target variable 
Our goal is to learn a mapping between the next day volatility $\sigma _{t+1}$ and historical multimodal data available up to day $t$   To this aim, we use a sliding window approach with window size $T$   That is, for each stock $sc$ a sample on day $t$ is expressed as a sequence of historical prices $P^{sc}_t$ and corpus headlines $N^{sc}_t$   The price sequence is a vector of Daily Prices (DP) and expressed as
$$P^{sc}_t = \left[DP^{sc}_{t-T}, DP^{sc}_{t-T+1}, \cdots , DP^{sc}_t \right]$$   (Eq  69)
where $DP^{sc}_{t^{\prime }}$ is a vector of price features  In order to avoid task-specific feature engineering, the daily price features are expressed as the simple returns:
$$DP^{sc}_t = \left[ \frac{O^{sc}_{t}}{C^{sc}_{t-1}} - 1, \frac{H^{sc}_{t}}{C^{sc}_{t-1}} - 1, \frac{L^{sc}_{t}}{C^{sc}_{t-1}} - 1, \frac{C^{sc}_{t}}{C^{sc}_{t-1}} - 1 \right]$$   (Eq  70)
The sequence of historical corpus headlines $N^{sc}_t$ is expressed as
$$N^{sc}_t = \left[n^{sc}_{t-T}, n^{sc}_{t-T+1}, \cdots , n^{sc}_{t} \right]$$   (Eq  71)
where $n^{sc}_{t^{\prime }}$ is a set containing all headlines that influence the market on a given day $t^{\prime }$  
Aiming to align prices and news modes, we consider the explicit alignment method discussed in subsec:stockheadlines  That is, $n^{sc}_{t^{\prime }}$ contains all stock headlines before the market opens ( $\texttt {before market}_{t}$ ), during the trading hours
( $\texttt {during market}_{t}$ ), and previous day after-markets
( $\texttt {after market}_{t-1}$ ) 
As a text preprocessing step, we tokenize the headlines and convert each word to an integer that refers to its respective pre-trained word embedding  This process is described as follows: First, for all stocks of our corpus we tokenize each headline and extract the corpus vocabulary set $V$   We then build the embedding matrix $E_w \in \mathbb {R}^{\vert V \vert \times d_w}$ , where each row is a word embedding vector $d_w$ dimensions  Words that do not have a corresponding embedding, i e  out of vocabulary words, are skipped 
Finally, the input sample of the text mode is a tensor of integers with $T \times l_n \times l_s$ dimensions, where $l_n$ is the maximum number of news occurring in a given day and $l_s$ is the maximum length of a corpus sentence  Regarding the price mode, we have a $T \times 4$ tensor of floating numbers 
Global features and stock embedding
Given the price and news histories for each stock $sc$ we could directly learn one model per stock  However, this approach suffers from two main drawbacks  First, the market activity of one specific stock is expected to impact other stocks, which is a widely accepted pattern named “spillover effect”  Second, since our price data is sampled on a daily basis, we would train the stock model relying on a small number of samples  One possible solution to model the commonality among stocks would be feature enrichment  For example, when modeling a given stock $X$ we would enrich its news and price features by concatenating features from stock $Y$ and $Z$   Although the feature enrichment is able to model the effect of other stocks, it still would consider only one sample per day 
In this work, we propose a method that learns an global model 
The global model is implemented using the following methods:
Multi-Stock batch samples: Since our models are trained using Stochastic Gradient Descent, we propose at each mini-batch iteration to sample from a batch set containing any stock of our stocks universe  As a consequence, the mapping between volatility and multimodal data is now able to learn common explanatory factors among stocks  Moreover, adopting this approach increases the total number of training samples, which is now the sum of the number of samples per stock 
Stock Embedding: Utilizing the Multi-Stock batch samples above, we tackle the problem of modeling commonality among stocks  However, it is reasonable to assume that stocks have part of its dynamic driven by idiosyncratic factors  Nevertheless, we could aggregate stocks per sector or rely on any measure of similarity among stocks  In order to incorporate information specific to each stock, we propose to equip our model with a “stock embedding” mode that is learnt jointly with price and news modes  That is to say, we leave the task of distinguishing the specific dynamic of each stock to be learnt by the neural network  Specifically, this stock embedding is modeled using a discrete encoding as input, i e  $\mathcal {I}^{sc}_t$ is a vector with size equal to the number of stocks of the stocks universe and has element 1 for the i-th coordinate and 0 elsewhere, thus, indicating the stock of each sample 
Formally, we can express the one model per stock approach as the mapping
$$\begin{split} \sigma ^{sc}_{t+1} = f^{sc} ( DN^{sc}_{t-T}, DN^{sc}_{t-T+1}, \cdots , DN^{sc}_t ; \\ DP^{sc}_{t-T}, DP^{sc}_{t-T+1}, \cdots , DP^{sc}_t ) \end{split}$$   (Eq  75)
where $DN^{sc}_{t^{\prime }}$ is a fixed-vector representing all news released on a given day for the stock $sc$ and $DP^{sc}_{t^{\prime }}$ is defined in eq:pricemodevec 
The global model attempts to learn a single mapping $f$ that at each mini-batch iteration randomly aggregates samples across all the universe of stocks, rather than one mapping $f^{sc}$ per stock  The global model is expressed as
$$\begin{split} \sigma ^{sc}_{t+1} = f ( DN^{sc}_{t-T}, DN^{sc}_{t-T+1}, \cdots , DN^{sc}_t ; \\ DP^{sc}_{t-T}, DP^{sc}_{t-T+1}, \cdots , DP^{sc}_t ; \\ \mathcal {I}^{sc}_t) \end{split}$$   (Eq  77)
In the next section, we describe our hierarchical neural model and how the news, price and stock embedding are fused into a joint representation 
Our multimodal hierarchical network
In broad terms, our hierarchical neural architecture is described as follows  First, each headline released on a given day $t$ is encoded into a fixed-size vector $S_t$ using a sentence encoder  We then apply our daily New Relevance Attention (NRA) mechanism that attends each news based on its content and converts a variable size of news released on a given day into a single vector denoted by Daily News ( $DN$ )  We note that this representation take account of the overall effect of all news released on a given day  This process is illustrated in fig:DNencoder  We now are in a position to consider the temporal effect of the past $T$ days of market news and price features  fig:nntimeseriesarch illustrates the neural network architecture from the temporal sequence to the final volatility prediction  For each stock code $sc$ the temporal encoding for news is denoted by Market News $MN^{sc}_t$ and for the price by Market Price $MP^{sc}_t$ and are a function of the past $T$ Daily News representations ${\lbrace DN^{sc}_{t-T}, \cdots , DN^{sc}_t \rbrace }$ (Text mode) and Daily Prices features $S_t$0 (Price mode), where each Daily Price $S_t$1 feature is given by eq:pricemodevec and the $S_t$2 representation is calculated using Daily New Relevance Attention  After the temporal effects of $S_t$3 past days of market activity were already encoded into the Market News $S_t$4 and Market Price $S_t$5 , we concatenate feature-wise $S_t$6 , $S_t$7 and the Stock embedding $S_t$8   The stock embedding $S_t$9 represents the stock code of the sample on a given day $t$   Finally, we have a Fully Connected (FC) layer that learns the Joint Representation of all modes  This fixed-sized joint representation is fed into a FC layer with linear activation that predicts the next day volatility $\hat{\sigma }_{t+1}$  
Below, we detail, for each mode separately, the layers of our hierarchical model 
– Text mode
Word Embedding Retrieval
Standard embedding layer with no trainable parameters  It receives a vector of word indices as input and returns a matrix of word embeddings 
News Encoder
This layer encodes all news on a given day and outputs a set news embeddings $\lbrace S^{1}_t, \cdots , S^{l_n}_t \rbrace $   Each encoded sentence has dimension $d_S$ , which is a hyperparameter of our model  This layer constitutes a key component of our neural architectures and, as such, we evaluate our models considering sentence encoders trained end-to-end, using the BiLSTM attention (subsec:bilstminneratt) and BiLSTM max-pooling (subsec:bilstmmaxpool) architectures, and also transferred from the RCV1 and SNLI as fixed features 
Daily news relevance attention
Our proposed news relevance attention mechanism for all news released on a given day  The attention mechanism is introduced to tackle information overload  It was designed to “filter out” redundant or misleading news and focus on the relevant ones based solely on the news content  Formally, the layer outputs a Daily News (DN) embedding $DN^{sc}_t = \sum _{i=1}^{l_n} \beta _i S^{sc^{i}}_t$ , which is a linear combination of all encoded news on a given day $t$   This news-level attention uses the same equations as in eq:att, but with trainable weights $\lbrace W_{R}, b_{R}, v_{R}\rbrace $ , i e  the weights are segregated from the sentence encoder  fig:DNencoder, illustrates our relevance attention  Note that this layer was deliberately developed to be invariant to headlines permutation, as is the case with the linear combination formula above  The reason is that our price data is sampled daily and, as a consequence, we are not able to discriminate the market reaction for each intraday news 
News Temporal Context
Sequence layer with daily news embeddings $DN^{sc}_t$ as time steps  This layer aims to learn the temporal context of news, i e  the relationship between the news at day $t$ and the $T$ past days  It receives as input a chronologically ordered sequence of $T$ past Daily News embeddings ${\lbrace DN^{sc}_{t-T}, \cdots , DN^{sc}_t \rbrace }$ and outputs the news mode encoding Market News $MN^{sc}_t \in d_{MN}$   The sequence with $T$ time steps is encoded using a BiLSTM attention  The layer was designed to capture the temporal order that news are released and the current news novelty  i e  news that were repeated in the past can be “forgotten” based on the modulating gates of the LSTM network 
– Price mode
Price Encoder
Sequence layer analogous to News Temporal Context, but for the price mode  The input is the ordered sequence Daily Prices ${\lbrace DP^{sc}_{t-T}, \cdots , DP^{sc}_t \rbrace }$ of size $T$ , where each element the price feature defined in eq:pricemodevec  Particularly, the architecture consists of two stacked LSTM's  The first one outputs for each price feature time step a hidden vector that takes the temporal context into account  Then these hidden vectors are again passed to a second independent LSTM  The layer outputs the price mode encoding Market Price $MP^{sc}_t \in d_{MP}$   This encoding is the last hidden vector of the second LSTM Market 
– Stock embedding
Stock Encoder
Stock dense representation  The layer receives the discrete encoding $\mathcal {I}^{sc}_t$ indicating the sample stock code pass through a FC layer and outputs a stock embedding $E_{sc}$  
– Joint Representation
Merging
Feature-wise News, Price, and Stock modes concatenation  No trainable parameters 
Joint Representation Encoder
FC layer of size $d_{JR}$  
Multimodal learning with missing modes
During the training we feed into our neural model the price, news, and stock indicator data  The price and stock indicator modes data occur in all days  However, at the individual stock level we can have days that the company is not covered by the media  This feature imposes challenges to our multimodal training since neural networks are not able to handle missing modes without special intervention  A straightforward solution would be to consider only days with news released, disregarding the remaining samples  However, this approach has two main drawbacks  First, the “missing news” do not happen at random, or are attributed to measurement failure as is, for example, the case of multimodal tasks using mechanical sensors data  Conversely, as highlighted in BIBREF7 , BIBREF8 the same price behaviour results in distinct market reactions when accompanied or not by news  In other words, specifically to financial forecasting problems the absence or existence of news are highly informative 
Some methods were proposed in the multimodal literature to effectively treat informative missing modes or “informative missingness”, which is a characteristic refereed in the literature as learning with missing modalities BIBREF22   In this work, we directly model the news missingness as a feature of our text model temporal sequence by using the method initially proposed in BIBREF45 , BIBREF46 for clinical data with missing measurements and applied in the context of financial forecasting in BIBREF47   Specifically, we implement the Zeros & Imputation (ZI) method BIBREF46 in order to jointly learn the price mode and news relationship across all days of market activity 
The ZI implementation is described as follows: Before the daily news sequence is processed by the text temporal layer (described in itm:newstclayer) we input a 0 vector for all time steps with missing news and leave the news encoding unchanged otherwise  This step is called zero imputation  In addition, we concatenate feature-wise an indicator vector with value 1 for all vectors with zero imputation and 0 for the days with news 
As described in BIBREF47 , the ZI method endow a temporal sequence model with the ability to learn different representations depending on the news history and its relative time position  Moreover, it allows our model to predict the volatility for all days of our time series and, at the same time, take into account the current and past news informative missingness  Furthermore, the learnt positional news encoding works differently than a typical “masking”, where days without news are not passed through the LSTM cell  Masking the time steps would be losing information about the presence or absence of news concomitant with prices 
Experimental results and discussions
We aim to evaluate our hierarchical neural model in the light of three main aspects  First, we asses the importance of the different sentence encoders to our end-to-end models and how it compares to transferring the sentence encoder from our two auxiliary TL tasks  Second, we ablate our proposed news relevance attention (NRA) component to evaluate its importance  Finally, we consider a model that takes into consideration only the price mode (unimodal), i e  ignoring any architecture related to the text mode 
Before we define the baselines to asses the three aspects described above, we review in the next section the scores of the trained TL tasks 
Auxiliary transfer learning tasks
This section reports the performance of the auxiliary TL tasks considered in this work  Our ultimate goal is to indicate that our scores are in line with previous works All the architectures presented in sec:transferlearning are trained for a maximum of 50 epochs using mini-batch SGD with Adam optimizer BIBREF48   Moreover, at the end of each epoch, we evaluate the validation scores, which are accuracy (Stanfor SNLI dataset) and F1 (RCV1 dataset), and save the weights with the best values  Aiming to seeped up training, we implement early stopping with patience set to 8 epochs  That is, if the validation scores do not improve for more than 10 epochs we halt the training  Finally, we use Glove pre-trained word embeddings BIBREF37 as fixed features 
tbl:tlevaluation compares our test scores with state-of-the-art (SOTA) results reported in previous works  We can see that our scores for the SNLI task are very close to state-of-the-art 
Regarding the RCV1 dataset, our results consider only the headline content for training, while the refereed works consider both the news headline and message body  The reason for training using only the headlines is that both tasks are learnt with the sole purpose of transferring the sentence encoders to our main volatility prediction task, whose textual input is restricted to headlines 
Training setup
During the training of our hierarchical neural model described in sub:HAN we took special care to guard against overfitting  To this aim, we completely separate 2016 and 2017 as the test set and report our results on this “unseen” set  The remaining data is further split into training (2007 to 2013) and validation (2014 to 2015)  The model convergence during training is monitored in the validation set  We monitor the validation score of our model at the end of each epoch and store the network weights if the validation scores improves between two consecutive epochs  Additionally, we use mini-batch SGD with Adam optimizer and early stopping with patience set to eight epochs  The hyperparameter tunning is performed using grid search 
All training is performed using the proposed global model approach described in sub:globalmodel, which learns a model that takes into account the features of all the 40 stocks of our corpus  Using this approach our training set has a total of 97,903 samples  Moreover, during the SGD mini-batch sampling the past $T$ days of price and news history tensors and each stock sample stock indicator are randomly selected from the set of all 40 stocks 
Stocks universe result
In order to evaluate the contributions of each component of our neural model described in sub:HAN and the effect of using textual data to predict the volatility, we report our results using the following baselines:
- News (unimodal price only): This baseline completely ablates (i e  removes) any architecture related to the news mode, considering only the price encoding and the stock embedding components  Using this ablation we aim to evaluate the influence of news to the volatility prediction problem 
+ News (End-to-end Sentence Encoders) - NRA: This baseline ablates our proposed new relevance attention (NRA) component, and instead, makes use of the same Daily Averaging method in BIBREF26 , BIBREF27 , where all fixed-sized headline representations on a given day are averaged without taking into account the relevance of each news  We evaluate this baseline for both BiLSTM attention (Att) and BiLSTM max-pooling (MP) sentence encoders  Here, our goal is to asses the true contribution of our NRA component in the case SOTA sentence encoders are taken into account 
+ News (End-to-End W-L Att Sentence Encoder) + NRA: The Word-Level Attention (W-L Att) sentence encoder implements an attention mechanism directly on top of word embeddings, and, as such, does not consider the order of words in a sentence  This baseline complements the previous one, i e  it evaluates the influence of the sentence encoder when our full specification is considered 
+ News (TL Sentence Encoders) + NRA: Makes use of sentence encoders of our two auxiliary TL tasks as fixed features  This baseline aims to address the following questions, namely: What dataset and models are more suitable to transfer to our specific volatility forecasting problem; How End-to-End models, which are trained on top of word embeddings, perform compared to sentence encoders transferred from other tasks 
tbl:comparativeallsectors summarizes the test scores for the ablations discussed above  Our best model is the + News (BiLSTM Att) + NRA, which is trained end-to-end and uses our full architecture  The second best model, i e  + News (BiLSTM MP) + NRA, ranks slightly lower and only differs form the best model in terms of the sentence encoder  The former sentence encoder uses an attention layer (subsec:bilstminneratt) and the the last a max-pooling layer (subsec:bilstmmaxpool), where both layers are placed on top of the LSTM hidden states of each word 
Importantly, our experiments show that using news and price (multimodal) to predict the volatility improves the scores by 11% (MSE) and 9% (MAE) when compared with the – News (price only unimodal) model that considers only price features as explanatory variables 
When comparing the performance of End-to-End models and the TL auxiliary tasks the following can be observed: The end-to-end models trained with the two SOTA sentence encoders perform better than transferring sentence encoder from both auxiliary tasks  However, our experiments show that the same does not hold for models trained end-to-end relying on the simpler WL-Att sentence encoder, which ignores the order of words in a sentence  In other words, considering the appropriate TL task, it is preferable to transfer a SOTA sentence encoder trained on a larger dataset than learning a less robust sentence encoder in an end-to-end fashion  Moreover, initially, we thought that being the RCV1 a financial domain corpus it would demonstrate a superior performance when compared to the SNLI dataset  Still, the SNLI transfers better than RCV1  We hypothesize that the text categorization task (RCV1 dataset) is not able to capture complex sentence structures at the same level required to perform natural language inference  Particularly to the volatility forecasting problem, our TL results corroborates the same findings in BIBREF17 , where it was shown that SNLI dataset attains the best sentence encoding for a broad range of pure NLP tasks, including, among other, text categorization and sentiment analysis 
Significantly, experimental results in tbl:comparativeallsectors clearly demonstrate that our proposed news relevance attention (NRA) outperforms the News Averaging method proposed in previous studies BIBREF26 , BIBREF27   Even when evaluating our NRA component in conjunction with the more elementary W-L Att sentence encoder it surpass the results of sophisticated sentence encoder using a News Averaging approach  In other words, our results strongly points to the advantage of discriminating noisy from impacting news and the effectiveness of learning to attend the most relevant news 
Having analyzed our best model, we now turn to its comparative performance with respect to the widely regarded GARCH(1,1) model described in sec:GARCH 
We asses our model performance relative to GARCH(1,1) using standard loss metrics (MSE and MAE) and the regression-based accuracy specified in eq:regressionloss and measured in terms of the coefficient of determination $R^2$   In addition, we evaluate our model across two different volatility proxies: Garman-Klass ( $\widehat{\sigma _{GK}}$ ) (eq:volgk) and Parkinson ( $\widehat{\sigma _{PK}}$ ) (eq:volpk)  We note that, as reviewed in sub:evalution, these two volatility proxies are statically efficient and proper estimators of the next day volatility 
tbl:garchallsectors reports the comparative performance among our best Price + News model (+ News BiLSTM (MP) + NRA), our Price only (unimodal) model and GARCH(1,1)  The results clearly demonstrate the superiority of our model, being more accurate than GRACH for both volatility proxies  We note that evaluating the GARCH(1,1) model relying on standard MSE and MAE error metrics should be taken with a grain of salt  BIBREF35 provides the background theory and arguments supporting $R^2$ as the metric of choice to evaluate the predictive power of a volatility model  In any case, the outperformance or our model with respect to GARCH(1,1) permeates all three metrics, name $R^2$ , $MSE$ and $MAE$  
Sector-level results
Company sectors are expected to have different risk levels, in the sense that each sector is driven by different types of news and economic cycles  Moreover, by performing a sector-level analysis we were initially interested in understanding if the outperformance of our model with respect to GARCH(1,1) was the result of a learning bias to a given sector or if, as turned out to be the case, the superior performance of our model spreads across a diversified portfolio of sectors 
In order to evaluate the performance per sector, we first separate the constituents stocks for each sector in tbl:stockuniverse  Then, we calculate the same metrics discussed in the previous section for each sector individually 
tbl:garcheachsector reports our experimental results segregated by sector  We observe that the GRACH model accuracy, measured using the $R^2$ score, has a high degree of variability among sectors  For example, the accuracy ranges from 0 15 to 0 44 for the HealthCare and Energy sector, respectively  This high degree of variability is in agreement with previous results reported in BIBREF16 , but in the context of long-term (quarterly) volatility predictions  Although the GARCH(1,1) accuracy is sector-dependent, without any exception, our model using price and news as input clearly outperforms GRACH sector-wise  This fact allow us to draw the following conclusions:
Our model outperformance is persistent across sectors, i e  the characteristics of the results reported in tbl:garchallsectors permeates all sectors, rather than being composed of a mix of outperforming and underperforming sector contributions  This fact provides a strong evidence that our model is more accurate than GARCH(1,1) 
The proposed Global model approach discussed in sub:globalmodel is able to generalize well, i e  the patterns learnt are not biased to a given sector or stock 
One of the limitations of our work is to rely on proxies for the volatility estimation  Although these proxies are handy if only open, high, low and close daily price data is available, having high frequency price data we could estimate the daily volatility using the sum of squared intraday returns to measure the true daily latent volatility  For example, in evaluating the performance for the one-day-ahead GARCH(1,1) Yen/Dollar exchange rate BIBREF35 reports $R^2$ values of 0 237 and 0 392 using hourly and five minutes sampled intraday returns, respectively  However, we believe that utilizing intraday data would further improve our model performance 
Since our experimental results demonstrate the key aspect of the news relevance attention to model architecture we observe that intraday data would arguably ameliorate the learning process  Having intraday data would allow us to pair each individual news release with the instantaneous market price reaction  Using daily data we are losing part of this information by only measuring the aggregate effect of all news to the one-day-ahead prediction 
Conclusion
We study the joint effect of stock news and prices on the daily volatility forecasting problem  To the best of our knowledge, this work is one of the first studies aiming to predict short-term (daily) rather than long-term (quarterly or yearly) volatility taking news and price as explanatory variables and using a comprehensive dataset of news headlines at the individual stock level  Our hierarchical end-to-end model benefits from state-of-the-art approaches to encode text information and to deal with two main challenges in correlating news with market reaction: news relevance and novelty  That is, to address the problem of how to attend the most important news based purely on its content (news relevance attention) and to take into account the temporal information of past news (temporal context)  Additionally, we propose a multi-stock mini-batch + stock embedding method suitable to model commonality among stocks 
The experimental results show that our multimodal approach outperforms the GARCH(1,1) volatility model, which is the most prevalent econometric model for daily volatility predictions  The outperformance being sector-wise and demonstrates the effectiveness of combining price and news for short-term volatility forecasting  The fact that we outperform GARCH(1,1) for all analyzed sectors confirms the robustness of our proposed architecture and evidences that our global model approach generalizes well 
We ablated (i e  removed) different components of our neural architecture to assess its most relevant parts  To this aim, we replaced our proposed news relevance attention layer, which aims to attend the most important news on a given day, with a simpler architecture proposed in the literature, which averages the daily news  We found that our attention layer improves the results  Additionally, we ablated all the architecture related to the news mode and found that news enhances the forecasting accuracy 
Finally, we evaluated different sentence encoders, including those transfered from other NLP tasks, and concluded that they achieve better performance as compared to a plain Word-level attention sentence encoder trained end-to-end  However, they do not beat state-of-the-art sentence encoders trained end-to-end 
In order to contribute to the literature of Universal Sentence Encoders, we evaluated the performance of transferring sentence encoders from two different tasks to the volatility prediction problem  We showed that models trained on the Natural Language Inference (NLI) task are more suitable to forecasting problems than a financial domain dataset (Reuters RCV1)  By analyzing different architectures, we showed that a BiLSTM with max-pooling for the SNLI dataset provides the best sentence encoder 
In the future, we plan to make use of intraday prices to better assess the predictive power of our proposed models  Additionally, we would further extend our analysis to other stock market sectors 
Context 2: Introduction
Data imbalance is a common issue in a variety of NLP tasks such as tagging and machine reading comprehension  Table TABREF3 gives concrete examples: for the Named Entity Recognition (NER) task BIBREF2, BIBREF3, most tokens are backgrounds with tagging class $O$  Specifically, the number of tokens tagging class $O$ is 5 times as many as those with entity labels for the CoNLL03 dataset and 8 times for the OntoNotes5 0 dataset; Data-imbalanced issue is more severe for MRC tasks BIBREF4, BIBREF5, BIBREF6, BIBREF7, BIBREF8 with the value of negative-positive ratio being 50-200 
Data imbalance results in the following two issues: (1) the training-test discrepancy: Without balancing the labels, the learning process tends to converge to a point that strongly biases towards class with the majority label  This actually creates a discrepancy between training and test: at training time, each training instance contributes equally to the objective function while at test time, F1 score concerns more about positive examples; (2) the overwhelming effect of easy-negative examples  As pointed out by meng2019dsreg, significantly large number of negative examples also means that the number of easy-negative example is large  The huge number of easy examples tends to overwhelm the training, making the model not sufficiently learned to distinguish between positive examples and hard-negative examples  The cross-entropy objective (CE for short) or maximum likelihood (MLE) objective, which is widely adopted as the training objective for data-imbalanced NLP tasks BIBREF9, BIBREF10, BIBREF11, BIBREF12, BIBREF13, BIBREF14, BIBREF15, handles neither of the issues 
To handle the first issue, we propose to replace CE or MLE with losses based on the Sørensen–Dice coefficient BIBREF0 or Tversky index BIBREF1  The Sørensen–Dice coefficient, dice loss for short, is the harmonic mean of precision and recall  It attaches equal importance to false positives (FPs) and false negatives (FNs) and is thus more immune to data-imbalanced datasets  Tversky index extends dice loss by using a weight that trades precision and recall, which can be thought as the approximation of the $F_{\beta }$ score, and thus comes with more flexibility  Therefore, We use dice loss or Tversky index to replace CE loss to address the first issue 
Only using dice loss or Tversky index is not enough since they are unable to address the dominating influence of easy-negative examples  This is intrinsically because dice loss is actually a hard version of the F1 score  Taking the binary classification task as an example, at test time, an example will be classified as negative as long as its probability is smaller than 0 5, but training will push the value to 0 as much as possible  This gap isn't a big issue for balanced datasets, but is extremely detrimental if a big proportion of training examples are easy-negative ones: easy-negative examples can easily dominate training since their probabilities can be pushed to 0 fairly easily  Meanwhile, the model can hardly distinguish between hard-negative examples and positive ones  Inspired by the idea of focal loss BIBREF16 in computer vision, we propose a dynamic weight adjusting strategy, which associates each training example with a weight in proportion to $(1-p)$, and this weight dynamically changes as training proceeds  This strategy helps to deemphasize confident examples during training as their $p$ approaches the value of 1, makes the model attentive to hard-negative examples, and thus alleviates the dominating effect of easy-negative examples 
Combing both strategies, we observe significant performance boosts on a wide range of data imbalanced NLP tasks  Notably, we are able to achieve SOTA results on CTB5 (97 92, +1 86), CTB6 (96 57, +1 80) and UD1 4 (96 98, +2 19) for the POS task; SOTA results on CoNLL03 (93 33, +0 29), OntoNotes5 0 (92 07, +0 96)), MSRA 96 72(+0 97) and OntoNotes4 0 (84 47,+2 36) for the NER task; along with competitive results on the tasks of machine reading comprehension and paraphrase identification 
The rest of this paper is organized as follows: related work is presented in Section 2  We describe different training objectives in Section 3  Experimental results are presented in Section 4  We perform ablation studies in Section 5, followed by a brief conclusion in Section 6 
Related Work ::: Data Resample
The idea of weighting training examples has a long history  Importance sampling BIBREF17 assigns weights to different samples and changes the data distribution  Boosting algorithms such as AdaBoost BIBREF18 select harder examples to train subsequent classifiers  Similarly, hard example mining BIBREF19 downsamples the majority class and exploits the most difficult examples  Oversampling BIBREF20, BIBREF21 is used to balance the data distribution  Another line of data resampling is to dynamically control the weights of examples as training proceeds  For example, focal loss BIBREF16 used a soft weighting scheme that emphasizes harder examples during training  In self-paced learning BIBREF22, example weights are obtained through optimizing the weighted training loss which encourages learning easier examples first  At each training step, self-paced learning algorithm optimizes model parameters and example weights jointly  Other works BIBREF23, BIBREF24 adjusted the weights of different training examples based on training loss  Besides, recent work BIBREF25, BIBREF26 proposed to learn a separate network to predict sample weights 
Related Work ::: Data Imbalance Issue in Object Detection
The background-object label imbalance issue is severe and thus well studied in the field of object detection BIBREF27, BIBREF28, BIBREF29, BIBREF30, BIBREF31  The idea of hard negative mining (HNM) BIBREF30 has gained much attention recently  shrivastava2016ohem proposed the online hard example mining (OHEM) algorithm in an iterative manner that makes training progressively more difficult, and pushes the model to learn better  ssd2016liu sorted all of the negative samples based on the confidence loss and picking the training examples with the negative-positive ratio at 3:1  pang2019rcnn proposed a novel method called IoU-balanced sampling and aploss2019chen designed a ranking model to replace the conventional classification task with a average-precision loss to alleviate the class imbalance issue  The efforts made on object detection have greatly inspired us to solve the data imbalance issue in NLP 
Losses ::: Notation
For illustration purposes, we use the binary classification task to demonstrate how different losses work  The mechanism can be easily extended to multi-class classification 
Let $\lbrace x_i\rbrace $ denote a set of instances  Each $x_i$ is associated with a golden label vector $y_i = [y_{i0},y_{i1} ]$, where $y_{i1}\in \lbrace 0,1\rbrace $ and $y_{i0}\in \lbrace 0,1\rbrace $ respectively denote the positive and negative classes, and thus $y_i$ can be either $[0,1]$ or $[0,1]$  Let $p_i = [p_{i0},p_{i1} ]$ denote the probability vector, and $p_{i1}$ and $p_{i0}$ respectively denote the probability that a model assigns the positive and negative label to $x_i$ 
Losses ::: Cross Entropy Loss
The vanilla cross entropy (CE) loss is given by:
As can be seen from Eq DISPLAY_FORM8, each $x_i$ contributes equally to the final objective  Two strategies are normally used to address the the case where we wish that not all $x_i$ are treated equal: associating different classes with different weighting factor $\alpha $ or resampling the datasets  For the former, Eq DISPLAY_FORM8 is adjusted as follows:
where $\alpha _i\in [0,1]$ may be set by the inverse class frequency or treated as a hyperparameter to set by cross validation  In this work, we use $\lg (\frac{n-n_t}{n_t}+K)$ to calculate the coefficient $\alpha $, where $n_t$ is the number of samples with class $t$ and $n$ is the total number of samples in the training set  $K$ is a hyperparameter to tune  The data resampling strategy constructs a new dataset by sampling training examples from the original dataset based on human-designed criteria, e g , extract equal training samples from each class  Both strategies are equivalent to changing the data distribution and thus are of the same nature  Empirically, these two methods are not widely used due to the trickiness of selecting $\alpha $ especially for multi-class classification tasks and that inappropriate selection can easily bias towards rare classes BIBREF32 
Losses ::: Dice coefficient and Tversky index
Sørensen–Dice coefficient BIBREF0, BIBREF33, dice coefficient (DSC) for short, is a F1-oriented statistic used to gauge the similarity of two sets  Given two sets $A$ and $B$, the dice coefficient between them is given as follows:
In our case, $A$ is the set that contains of all positive examples predicted by a specific model, and $B$ is the set of all golden positive examples in the dataset  When applied to boolean data with the definition of true positive (TP), false positive (FP), and false negative (FN), it can be then written as follows:
For an individual example $x_i$, its corresponding DSC loss is given as follows:
As can be seen, for a negative example with $y_{i1}=0$, it does not contribute to the objective  For smoothing purposes, it is common to add a $\gamma $ factor to both the nominator and the denominator, making the form to be as follows:
As can be seen, negative examples, with $y_{i1}$ being 0 and DSC being $\frac{\gamma }{ p_{i1}+\gamma }$, also contribute to the training  Additionally, milletari2016v proposed to change the denominator to the square form for faster convergence, which leads to the following dice loss (DL):
Another version of DL is to directly compute set-level dice coefficient instead of the sum of individual dice coefficient  We choose the latter due to ease of optimization 
Tversky index (TI), which can be thought as the approximation of the $F_{\beta }$ score, extends dice coefficient to a more general case  Given two sets $A$ and $B$, tversky index is computed as follows:
Tversky index offers the flexibility in controlling the tradeoff between false-negatives and false-positives  It degenerates to DSC if $\alpha =\beta =0 5$  The Tversky loss (TL) for the training set $\lbrace x_i,y_i\rbrace $ is thus as follows:
Losses ::: Self-adusting Dice Loss
Consider a simple case where the dataset consists of only one example $x_i$, which is classified as positive as long as $p_{i1}$ is larger than 0 5  The computation of $F1$ score is actually as follows:
Comparing Eq DISPLAY_FORM14 with Eq DISPLAY_FORM22, we can see that Eq DISPLAY_FORM14 is actually a soft form of $F1$, using a continuous $p$ rather than the binary $\mathbb {I}( p_{i1}>0 5)$  This gap isn't a big issue for balanced datasets, but is extremely detrimental if a big proportion of training examples are easy-negative ones: easy-negative examples can easily dominate training since their probabilities can be pushed to 0 fairly easily  Meanwhile, the model can hardly distinguish between hard-negative examples and positive ones, which has a huge negative effect on the final F1 performance 
To address this issue, we propose to multiply the soft probability $p$ with a decaying factor $(1-p)$, changing Eq DISPLAY_FORM22 to the following form:
One can think $(1-p_{i1})$ as a weight associated with each example, which changes as training proceeds  The intuition of changing $p_{i1}$ to $(1-p_{i1}) p_{i1}$ is to push down the weight of easy examples  For easy examples whose probability are approaching 0 or 1, $(1-p_{i1}) p_{i1}$ makes the model attach significantly less focus to them  Figure FIGREF23 gives gives an explanation from the perspective in derivative: the derivative of $\frac{(1-p)p}{1+(1-p)p}$ with respect to $p$ approaches 0 immediately after $p$ approaches 0, which means the model attends less to examples once they are correctly classified 
A close look at Eq DISPLAY_FORM14 reveals that it actually mimics the idea of focal loss (FL for short) BIBREF16 for object detection in vision  Focal loss was proposed for one-stage object detector to handle foreground-background tradeoff encountered during training  It down-weights the loss assigned to well-classified examples by adding a $(1-p)^{\beta }$ factor, leading the final loss to be $(1-p)^{\beta }\log p$ 
In Table TABREF18, we show the losses used in our experiments, which is described in the next section 
Experiments
We evaluate the proposed method on four NLP tasks: part-of-speech tagging, named entity recognition, machine reading comprehension and paraphrase identification  Baselines in our experiments are optimized by using the standard cross-entropy training objective 
Experiments ::: Part-of-Speech Tagging
Part-of-speech tagging (POS) is the task of assigning a label (e g , noun, verb, adjective) to each word in a given text  In this paper, we choose BERT as the backbone and conduct experiments on three Chinese POS datasets  We report the span-level micro-averaged precision, recall and F1 for evaluation  Hyperparameters are tuned on the corresponding development set of each dataset 
Experiments ::: Part-of-Speech Tagging ::: Datasets
We conduct experiments on the widely used Chinese Treebank 5 0, 6 0 as well as UD1 4 
CTB5 is a Chinese dataset for tagging and parsing, which contains 507,222 words, 824,983 characters and 18,782 sentences extracted from newswire sources 
CTB6 is an extension of CTB5, containing 781,351 words, 1,285,149 characters and 28,295 sentences 
UD is the abbreviation of Universal Dependencies, which is a framework for consistent annotation of grammar (parts of speech, morphological features, and syntactic dependencies) across different human languages  In this work, we use UD1 4 for Chinese POS tagging 
Experiments ::: Part-of-Speech Tagging ::: Baselines
We use the following baselines:
Joint-POS: shao2017character jointly learns Chinese word segmentation and POS 
Lattice-LSTM: lattice2018zhang constructs a word-character lattice 
Bert-Tagger: devlin2018bert treats part-of-speech as a tagging task 
Experiments ::: Part-of-Speech Tagging ::: Results
Table presents the experimental results on the POS task  As can be seen, the proposed DSC loss outperforms the best baseline results by a large margin, i e , outperforming BERT-tagger by +1 86 in terms of F1 score on CTB5, +1 80 on CTB6 and +2 19 on UD1 4  As far as we are concerned, we are achieving SOTA performances on the three datasets  Weighted cross entropy and focal loss only gain a little performance improvement on CTB5 and CTB6, and the dice loss obtains huge gain on CTB5 but not on CTB6, which indicates the three losses are not consistently robust in resolving the data imbalance issue  The proposed DSC loss performs robustly on all the three datasets 
Experiments ::: Named Entity Recognition
Named entity recognition (NER) refers to the task of detecting the span and semantic category of entities from a chunk of text  Our implementation uses the current state-of-the-art BERT-MRC model proposed by xiaoya2019ner as a backbone  For English datasets, we use BERT$_\text{Large}$ English checkpoints, while for Chinese we use the official Chinese checkpoints  We report span-level micro-averaged precision, recall and F1-score  Hyperparameters are tuned on the development set of each dataset 
Experiments ::: Named Entity Recognition ::: Datasets
For the NER task, we consider both Chinese datasets, i e , OntoNotes4 0 BIBREF34 and MSRA BIBREF35, and English datasets, i e , CoNLL2003 BIBREF36 and OntoNotes5 0 BIBREF37 
CoNLL2003 is an English dataset with 4 entity types: Location, Organization, Person and Miscellaneous  We followed data processing protocols in BIBREF14 
English OntoNotes5 0 consists of texts from a wide variety of sources and contains 18 entity types  We use the standard train/dev/test split of CoNLL2012 shared task 
Chinese MSRA performs as a Chinese benchmark dataset containing 3 entity types  Data in MSRA is collected from news domain  Since the development set is not provided in the original MSRA dataset, we randomly split the training set into training and development splits by 9:1  We use the official test set for evaluation 
Chinese OntoNotes4 0 is a Chinese dataset and consists of texts from news domain, which has 18 entity types  In this paper, we take the same data split as wu2019glyce did 
Experiments ::: Named Entity Recognition ::: Baselines
We use the following baselines:
ELMo: a tagging model from peters2018deep 
Lattice-LSTM: lattice2018zhang constructs a word-character lattice, only used in Chinese datasets 
CVT: from kevin2018cross, which uses Cross-View Training(CVT) to improve the representations of a Bi-LSTM encoder 
Bert-Tagger: devlin2018bert treats NER as a tagging task 
Glyce-BERT: wu2019glyce combines glyph information with BERT pretraining 
BERT-MRC: The current SOTA model for both Chinese and English NER datasets proposed by xiaoya2019ner, which formulate NER as machine reading comprehension task 
Experiments ::: Named Entity Recognition ::: Results
Table shows experimental results on NER datasets  For English datasets including CoNLL2003 and OntoNotes5 0, our proposed method outperforms BERT-MRCBIBREF38 by +0 29 and +0 96 respectively  We observe huge performance boosts on Chinese datasets, achieving F1 improvements by +0 97 and +2 36 on MSRA and OntoNotes4 0, respectively  As far as we are concerned, we are setting new SOTA performances on all of the four NER datasets 
Experiments ::: Machine Reading Comprehension
Machine reading comprehension (MRC) BIBREF39, BIBREF40, BIBREF41, BIBREF40, BIBREF42, BIBREF15 has become a central task in natural language understanding  MRC in the SQuAD-style is to predict the answer span in the passage given a question and the passage  In this paper, we choose the SQuAD-style MRC task and report Extract Match (EM) in addition to F1 score on validation set  All hyperparameters are tuned on the development set of each dataset 
Experiments ::: Machine Reading Comprehension ::: Datasets
The following five datasets are used for MRC task: SQuAD v1 1, SQuAD v2 0 BIBREF4, BIBREF6 and Quoref BIBREF8 
SQuAD v1 1 and SQuAD v2 0 are the most widely used QA benchmarks  SQuAD1 1 is a collection of 100K crowdsourced question-answer pairs, and SQuAD2 0 extends SQuAD1 1 allowing no short answer exists in the provided passage 
Quoref is a QA dataset which tests the coreferential reasoning capability of reading comprehension systems, containing 24K questions over 4 7K paragraphs from Wikipedia 
Experiments ::: Machine Reading Comprehension ::: Baselines
We use the following baselines:
QANet: qanet2018 builds a model based on convolutions and self-attention  Convolution to model local interactions and self-attention to model global interactions 
BERT: devlin2018bert treats NER as a tagging task 
XLNet: xlnet2019 proposes a generalized autoregressive pretraining method that enables learning bidirectional contexts 
Experiments ::: Machine Reading Comprehension ::: Results
Table shows the experimental results for MRC tasks  With either BERT or XLNet, our proposed DSC loss obtains significant performance boost on both EM and F1  For SQuADv1 1, our proposed method outperforms XLNet by +1 25 in terms of F1 score and +0 84 in terms of EM and achieves 87 65 on EM and 89 51 on F1 for SQuAD v2 0  Moreover, on QuoRef, the proposed method surpasses XLNet results by +1 46 on EM and +1 41 on F1  Another observation is that, XLNet outperforms BERT by a huge margin, and the proposed DSC loss can obtain further performance improvement by an average score above 1 0 in terms of both EM and F1, which indicates the DSC loss is complementary to the model structures 
Experiments ::: Paraphrase Identification
Paraphrases are textual expressions that have the same semantic meaning using different surface words  Paraphrase identification (PI) is the task of identifying whether two sentences have the same meaning or not  We use BERT BIBREF11 and XLNet BIBREF43 as backbones and report F1 score for comparison  Hyperparameters are tuned on the development set of each dataset 
Experiments ::: Paraphrase Identification ::: Datasets
We conduct experiments on two widely used datasets for PI task: MRPC BIBREF44 and QQP 
MRPC is a corpus of sentence pairs automatically extracted from online news sources, with human annotations of whether the sentence pairs are semantically equivalent  The MRPC dataset has imbalanced classes (68% positive, 32% for negative) 
QQP is a collection of question pairs from the community question-answering website Quora  The class distribution in QQP is also unbalanced (37% positive, 63% negative) 
Experiments ::: Paraphrase Identification ::: Results
Table shows the results for PI task  We find that replacing the training objective with DSC introduces performance boost for both BERT and XLNet  Using DSC loss improves the F1 score by +0 58 for MRPC and +0 73 for QQP 
Ablation Studies ::: The Effect of Dice Loss on Accuracy-oriented Tasks
We argue that the most commonly used cross-entropy objective is actually accuracy-oriented, whereas the proposed dice loss (DL) performs as a hard version of F1-score  To explore the effect of the dice loss on accuracy-oriented tasks such as text classification, we conduct experiments on the Stanford Sentiment Treebank sentiment classification datasets including SST-2 and SST-5  We fine-tune BERT$_\text{Large}$ with different training objectives  Experiment results for SST are shown in   For SST-5, BERT with CE achieves 55 57 in terms of accuracy, with DL and DSC losses slightly degrade the accuracy performance and achieve 54 63 and 55 19, respectively  For SST-2, BERT with CE achieves 94 9 in terms of accuracy  The same as SST-5, we observe a slight performance drop with DL and DSC, which means that the dice loss actually works well for F1 but not for accuracy 
Ablation Studies ::: The Effect of Hyperparameters in Tversky index
As mentioned in Section SECREF10, Tversky index (TI) offers the flexibility in controlling the tradeoff between false-negatives and false-positives  In this subsection, we explore the effect of hyperparameters (i e , $\alpha $ and $\beta $) in TI to test how they manipulate the tradeoff  We conduct experiments on the Chinese OntoNotes4 0 NER dataset and English QuoRef MRC dataset to examine the influence of tradeoff between precision and recall  Experiment results are shown in Table   The highest F1 for Chinese OntoNotes4 0 is 84 67 when $\alpha $ is set to 0 6 while for QuoRef, the highest F1 is 68 44 when $\alpha $ is set to 0 4  In addition, we can observe that the performance varies a lot as $\alpha $ changes in distinct datasets, which shows that the hyperparameters $\alpha ,\beta $ play an important role in the proposed method 
Conclusion
In this paper, we alleviate the severe data imbalance issue in NLP tasks  We propose to use dice loss in replacement of the standard cross-entropy loss, which performs as a soft version of F1 score  Using dice loss can help narrow the gap between training objectives and evaluation metrics  Empirically, we show that the proposed training objective leads to significant performance boost for part-of-speech, named entity recognition, machine reading comprehension and paraphrase identification tasks 

ground_truth: Energy with accuracy of 0.538

Energy

prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. How are weights dynamically adjusted?
Context 3: Introduction
Deep neural networks (DNNs), in particular convolutional and recurrent neural networks, with huge architectures have been proven successful in wide range of tasks including audio processing such as speech to text [1 - 4], emotion recognition [5 - 8], speech/non-speech (e g , of non-speech include noise, music, etc ,) classification [9 - 12], etc 
Training these deep architectures require large amount of annotated data, as a result, they cannot be used in low data resource scenarios which is common in speech-based applications [13 - 15]  Apart from collecting large data corpus, annotating the data is also very difficult, and requires manual supervision and efforts  Especially, annotation of speech for tasks like emotion recognition also suffer from lack of agreement among the annotators [16]  Hence, there is a need to build reliable systems that can work in low resource scenario 
In this work, we propose a novel approach to address the task of classification in low data resource scenarios  Our approach involves simultaneously considering more than one sample (in this work, two samples are considered) to train the classifier  We call this approach as simultaneous two sample learning (s2sL)  The proposed approach is also applicable to low resource data suffering with data imbalance  The contributions of this paper are:
Proposed approach
The s2sL approach proposed to address low data resource problem is explained in this Section  In this work, we use MLP (modified to handle our data representation) as the base classifier  Here, we explain the s2sL approach by considering two-class classification task 
Data representation
Consider a two-class classification task with INLINEFORM0 denoting the set of class labels, and let INLINEFORM1 and INLINEFORM2 be the number of samples corresponding to INLINEFORM3 and INLINEFORM4 , respectively  In general, to train a classifier, the samples in the train set are provided as input-output pairs as follows  DISPLAYFORM0
where INLINEFORM0 refers to the INLINEFORM1 -dimensional feature vector representing the INLINEFORM2 sample corresponding to INLINEFORM3 class label, and INLINEFORM4 , refers to output label of INLINEFORM5 class  In the proposed data representation format, called simultaneous two sample (s2s) representation, we will simultaneously consider two samples as follows  DISPLAYFORM0
where INLINEFORM0 , INLINEFORM1 are the INLINEFORM2 -dimensional feature vectors representing the INLINEFORM3 sample in INLINEFORM4 class and INLINEFORM5 sample in the INLINEFORM6 class, respectively; and INLINEFORM7 , INLINEFORM8 refers to the output labels of INLINEFORM9 and INLINEFORM10 class, respectively 
Hence, in s2s data representation, we will have an input feature vector of length INLINEFORM0 i e , INLINEFORM1 , and output class labels as either INLINEFORM2 or INLINEFORM3   Each sample can be combined with all the samples (i e , with ( INLINEFORM4 ) samples) in the dataset  Therefore, by representing the data in the s2s format, the number of samples in the train set increases to INLINEFORM5 from INLINEFORM6 samples  We hypothesize that the s2s format is expected to help the network not only to learn the characteristics of the two classes separately, but also the difference and similarity in characteristics of the two classes 
Classifier training
MLP, the most commonly used feed forward neural network, is considered as the base classifier to validate our proposed s2s framework  Generally, MLPs are trained using the data format given by eq  INLINEFORM0   But to train the MLP on our s2s based data representation (as in eq  INLINEFORM1 ), the following modifications are made to the MLP architecture (refer to Figure FIGREF4 ) 
We have INLINEFORM0 units (instead of INLINEFORM1 units) in the input layer to accept the two samples i e , INLINEFORM2 and INLINEFORM3 , simultaneously 
The structure of the hidden layer in this approach is similar to that of a regular MLP  The number of hidden layers and hidden units can be varied depending upon the complexity of the problem  The number of units in the hidden layer is selected empirically by varying the number of hidden units from 2 to twice the length of the input layer (i e , 2 to INLINEFORM0 ) and the unit at which the highest performance is obtained are selected  In this paper, we considered only a single hidden layer  Rectified linear units (ReLU) are used for hidden layer 
The output layer will consist of units equal to twice the considered number of classes in the classification task i e, the output layer will have four units for two-class classification task  The sigmoid activation function (not softmax) is used on the output layer units  Unlike regular MLP, we use sigmoid activation units in the output layer, with binary cross-entropy as the cost function, because the output labels in the proposed s2s based data representation will have more than one unit active at a time (not one-hot encoded output) and this condition cannot be handled using softmax function 
As can be seen from Figure FIGREF4 , the output layer in our proposed method has outputs INLINEFORM0 and INLINEFORM1 which corresponds to the outputs associated with the input feature vectors INLINEFORM2 and INLINEFORM3 , respectively  For a two-class classification problem, there will be four units in the output layer and the possible output labels are INLINEFORM4 , INLINEFORM5 , INLINEFORM6 , INLINEFORM7 corresponding to the class labels INLINEFORM8 and INLINEFORM9 , respectively  This architecture is referred to as s2s-MLP  In s2sL, s2s-MLP is trained using the s2s data representation format  Further, the s2s-MLP is trained using adam optimizer 
Classifier testing
Generally, the feature vector corresponding to the test sample is provided as input to the trained MLP in the testing phase and the class label is decided based on the obtained output 
However, in s2sL method, the feature vector corresponding to the test sample should also be converted to the s2s data representation format to test the trained s2s-MLP  We propose a testing approach, where the given test sample is combined with a set of preselected reference samples, whose class label is known a priori, to generate multiple instances of the same test sample as follows  DISPLAYFORM0
where INLINEFORM0 , INLINEFORM1 refer to the INLINEFORM2 -dimensional feature vector corresponding to the test sample and the INLINEFORM3 reference sample, respectively  INLINEFORM4 refers to the considered number of reference samples  These reference samples can be selected from any of the two classes 
For testing the s2s-MLP (as shown in Figure FIGREF4 ), each test sample INLINEFORM0 (same as ' INLINEFORM1 ' in (3)) is combined with all the INLINEFORM2 reference samples ( INLINEFORM3 ) to form INLINEFORM4 different instances of the same test sample INLINEFORM5   The corresponding outputs ( INLINEFORM6 ) obtained from s2s-MLP for the INLINEFORM7 generated instances of INLINEFORM8 are combined by voting-based decision approach to obtain the final decision INLINEFORM9   The class label that gets maximum votes is considered as the predicted output label 
Experiments
We validate the performance of the proposed s2sL by providing the preliminary results obtained on two different tasks namely, Speech/Music discrimination and emotion classification  We considered the GTZAN Music-Speech dataset [17], consisting of 120 audio files (60 speech and 60 music), for task of classifying speech and music  Each audio file (of 2 seconds duration) is represented using a 13-dimensional mel-frequency cepstral coefficient (MFCC) vector, where each MFCC vector is the average of all the frame level (frame size of 30 msec and an overlap of 10 msec) MFCC vectors  It is to be noted that our main intention for this task is not better feature selection, but to demonstrate the effectiveness of our approach, in particular for low data scenarios 
The standard Berlin speech emotion database (EMO-DB) [18] consisting of 535 utterances corresponding to 7 different emotions is considered for the task of emotion classification  Each utterance is represented by a 19-dimensional feature vector obtained by using the feature selection algorithm from WEKA toolkit [19] on the 384-dimensional utterance level feature vector obtained using openSMILE toolkit [20]  For two class classification, we considered the two most confusing emotion pairs i e , (Neutral,Sad) and (Anger, Happy)  Data corresponding to Speech/Music classification (60 speech and 60 music samples) and Neutral/Sad classification (79 neutral and 62 sad utterances) is balanced whereas Anger/Happy classification task has data imbalance, with anger forming the majority class (127 samples) and happy forming the minority class (71 samples)  Therefore, we show the performance of s2sL on both, balanced and imbalanced datasets 
All experimental results are validated using 5-fold cross validation (80% of data for training and 20% for testing in each fold)  Further, to analyze the effectiveness of s2sL in low resource scenarios, different proportions of training data, within each fold, are considered to train the system  For this analysis, we considered 4 different proportions i e , INLINEFORM0 , INLINEFORM1 , INLINEFORM2 and INLINEFORM3 of the training data to train the classifier  For instance, INLINEFORM4 means considering only half of the original training data to train the classifier, and INLINEFORM5 means considering the complete training data  5-fold cross validation is considered for all data proportions  Accuracy (in %) is used as a performance measure for balanced data classification tasks (i e , Speech/Music classification and Neutral/Sad emotion classification), whereas the more preferred INLINEFORM6 measure [21] is used as a measure for imbalanced data classification task (i e , Anger/Happy emotion classification) 
Table TABREF14 show the results obtained for proposed s2sL approach in comparison to that of MLP for the tasks of Speech/Music and Neutral/Sad classification, by considering different proportions of training data  The values in Table TABREF14 are mean accuracies (in %) obtained by 5-fold cross validation  It can be observed from Table TABREF14 that for both tasks, s2sL method outperforms MLP, especially at low resource conditions  s2sL shows an absolute improvement in accuracy of INLINEFORM0 % and INLINEFORM1 % over MLP for Speech/Music and Neutral/Sad classification tasks, respectively, when INLINEFORM2 of the original training data is used in experiments 
Table TABREF14 show the results (in terms of INLINEFORM0 values) obtained for proposed s2sL approach in comparison to that of MLP for Anger/Happy classification (data imbalance problem)  Here, state-of-the-art methods i e , Eusboost [22] and MWMOTE [23] are also considered for comparison  It can be observed from Table TABREF14 that the s2sL method outperforms MLP, and also performs better than Eusboost and MWMOTE techniques on imbalanced data (around 3 % absolute improvement in INLINEFORM1 value for s2sL compared to MWMOTE, when INLINEFORM2 of the training data is considered)  In particular, at lower amounts of training data, s2sL outperforms all the other methods, illustrating its effectiveness even for low resourced data imbalance problems  s2sL method shows an absolute improvement of 6% ( INLINEFORM3 ) in INLINEFORM4 value over the second best ( INLINEFORM5 for MWMOTE), when only INLINEFORM6 of the training data is used 
Conclusions
In this paper, we introduced a novel s2s framework to effectively learn the class discriminative characteristics, even from low data resources  In this framework, more than one sample (here, two samples) are simultaneously considered to train the classifier  Further, this framework allows to generate multiple instances of the same test sample, by considering preselected reference samples, to achieve a more profound decision making  We illustrated the significance of our approach by providing the experimental results for two different tasks namely, speech/music discrimination and emotion classification  Further, we showed that the s2s framework can also handle the low resourced data imbalance problem 
References
[1] Hinton, G , Deng, L , Yu, D , Dahl, G  E , Mohamed, A  R , Jaitly, N , Senior, A , Vanhoucke, V , Nguyen, P , Sainath, T  N  & Kingsbury, B  (2012) Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups  IEEE Signal Processing Magazine, pp  82–97 
[2] Vinyals, O , Ravuri, S  V  & Povey, D  (2012) Revisiting recurrent neural networks for robust ASR  In Proc  IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp  4085–4088 
[3] Lu, L , Zhang, X , Cho K  & Renals, S  (2015) A study of the recurrent neural network encoder-decoder for large vocabulary speech recognition  In Proc  INTERSPEECH, pp  3249–3253 
[4] Zhang, Y , Pezeshki, M , Brakel, P , Zhang, S , Bengio, C L Y  & Courville, A  (2017) Towards end-to-end speech recognition with deep convolutional neural networks  arXiv preprint, arXiv:1701 02720 
[5] Han, K , Yu, D  & Tashev, I  (2014) Speech emotion recognition using deep neural network and extreme learning machine  In Proc  Interspeech, pp  223–227 
[6] Trigeorgis, G , Ringeval, F , Brueckner, R , Marchi, E , Nicolaou, M  A , Schuller, B , & Zafeiriou, S  (2016) Adieu features? End-to-end speech emotion recognition using a deep convolutional recurrent network  In Proc  ICASSP, pp  5200–5204) 
[7] Huang, Che-Wei, & Shrikanth S  Narayanan  (2016) Attention Assisted Discovery of Sub-Utterance Structure in Speech Emotion Recognition  In Proc  INTERSPEECH, pp  1387–1391 
[8] Zhang, Z , Ringeval, F , Han, J , Deng, J , Marchi, E  & Schuller, B  (2016) Facing realism in spontaneous emotion recognition from speech: Feature enhancement by autoencoder with LSTM neural networks  In Proc  INTERSPEECH, pp  3593–3597 
[9] Scheirer, E  D  & Slaney, M  (2003) Multi-feature speech/music discrimination system  U S  Patent 6,570,991 
[10] Pikrakis, A  & Theodoridis S  (2014) Speech-music discrimination: A deep learning perspective  In Proc  European Signal Processing Conference (EUSIPCO), pp  616–620 
[11] Choi, K , Fazekas, G  & Sandler, M  (2016) Automatic tagging using deep convolutional neural networks  arXiv preprint, arXiv:1606 00298 
[12] Zazo Candil, R , Sainath, T N , Simko, G  & Parada, C  (2016) Feature learning with raw-waveform CLDNNs for Voice Activity Detection  In Proc  Interspeech, pp  3668–3672 
[13] Tian, L , Moore, JD  & Lai, C  (2015) Emotion recognition in spontaneous and acted dialogues  In Proc  International Conference on Affective Computing and Intelligent Interaction (ACII), pp  698–704 
[14] Dumpala, S H , Chakraborty, R  & Kopparapu, S K  (2017) k-FFNN: A priori knowledge infused feed-forward neural networks  arXiv preprint, arXiv:1704 07055 
[15] Dumpala, S H , Chakraborty, R  & Kopparapu, S K  (2018) Knowledge-driven feed-forward neural network for audio affective content analysis  To appear in AAAI-2018 workshop on Affective Content Analysis 
[16] Deng, J , Zhang, Z , Marchi, E  & Schuller, B  (2013) Sparse autoencoder-based feature transfer learning for speech emotion recognition  In Proc  Humaine Association Conference on Affective Computing and Intelligent Interaction (ACII), pp  511–516 
[17] George Tzanetakis, Gtzan musicspeech  Availabe on-line at http://marsyas info/download/datasets 
[18] Burkhardt, F , Paeschke, A , Rolfes, M , Sendlmeier, W F  & Weiss, B  (2005) A database of german emotional speech  In Proc  Interspeech, pp  1517–1520 
[19] Hall, M , Frank, E , Holmes, G , Pfahringer, B , Reutemann, P  & Witten, I H  (2009) The WEKA data mining software: an update  ACM SIGKDD explorations newsletter, 11(1), pp  10–18 
[20] Eyben, F , Weninger, F , Gross, F  & Schuller, B  (2013) Recent developments in opensmile, the munich open-source multimedia feature extractor  In Proc  ACM international conference on Multimedia, pp  835–838 
[21] Maratea, A , Petrosino, A  & Manzo, M  (2014) Adjusted f-measure and kernel scaling for imbalanced data learning  Information Sciences 257:331–341 
[22] Galar, M , Fernandez, A , Barrenechea, E  & Herrera, F  (2013) Eusboost: Enhancing ensembles for highly imbalanced data-sets by evolutionary undersampling  Pattern Recognition 46(12):3460–3471 
[23] Barua, S , Islam, M  M , Yao, X  & Murase, K  (2014) Mwmote–majority weighted minority oversampling technique for imbalanced data set learning  IEEE Transactions on Knowledge and Data Engineering 26(2):405–425 

ground_truth: One can think $(1-p_{i1})$ as a weight associated with each example, which changes as training proceeds. The intuition of changing $p_{i1}$ to $(1-p_{i1}) p_{i1}$ is to push down the weight of easy examples. For easy examples whose probability are approaching 0 or 1, $(1-p_{i1}) p_{i1}$ makes the model attach significantly less focus to them. Figure FIGREF23 gives gives an explanation from the perspective in derivative: the derivative of $\frac{(1-p)p}{1+(1-p)p}$ with respect to $p$ approaches 0 immediately after $p$ approaches 0, which means the model attends less to examples once they are correctly classified.

associates each training example with a weight in proportion to $(1-p)$, and this weight dynamically changes as training proceeds

prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. Which models/frameworks do they compare to?
Context 4: 
The tree creation is performed without processing the content of comments, which allows us to process posts and comments of any length efficiently  The heuristic used in the process is based on three simplifying assumptions:
Unless there is a specific reference to another comment or a user, comments are attached to the original post 
When replying, the commenting poster is always replying to the original post or some other comment  Unless specified otherwise, it is assumed that it is a response to the previous (in time) post/comment 
Subsequent comments by the same poster are part of the same thread 
To evaluate the performance of the thread extraction algorithm, 2 annotators have manually constructed the trees for 100 conversations  The performance of the algorithm on this set of 100 conversations is evaluated using accuracy and standard Information Retrieval evaluation metrics of precision, recall, and F$_1$ measure  The results are reported in Table TABREF28 together with random and majority baselines  The turn-level percent agreement between the 2 annotators is 97 99% and Cohen's Kappa Coefficient is 83 80% 
Datasets ::: Data Representation
For both data sources, Twitter and OSG with extracted threads, posts and comments are tokenized and sentence split  Each sentence is passed through Sentiment Analysis and Dialogue Act tagging  Since a post or a comment can contain multiple sentences, therefore multiple Dialogue Acts, it is represented as as a one-hot encoding, where each position represents a Dialogue Act 
For Sentiment Analysis we use a lexicon-based sentiment analyser introduced by BIBREF25  For Dialogue Act tagging, on the other hand, we make use of a model trained on NPSChat corpus BIBREF22 following the approach of BIBREF26 
Analysis
As we mentioned in Section 3, the presence of each of the therapeutic conditions under analysis is a necessary for a conversation to be considered to have therapeutic factors  In this section we present the results of our analysis with respect to these conditions 
Analysis ::: Change in Sentiment score of Original Posters
The first condition which we test is the sentiment change in conversation threads, comparing the initial and final sentiment scores (i e  posts' scores) of the original poster  The results of the analysis are presented in Figure FIGREF33  In the figure we can observe that the distribution of the sentiment change in the two datasets is different  While in Twitter the amount of conversations that lead to the increase of sentiment score is roughly equal to the amount of conversations that lead to the decrease of sentiment score; the situation is different for OSG  In OSG, the amount of conversations that lead to the increase of sentiment score is considerably higher 
Figure FIGREF34 provides a more fine grained analysis, where we additionally analyse the sentiment change in nominal polarity terms – negative and positive  In OSG, the number of users that changed polarity from negative to positive is more than the double of the users that have changed the polarity from positive to negative  In Twitter, on the other hand, the users mostly changed polarity from positive to negative  Results of the analysis suggest that in OSG , sentiment increases and users tend to change polarity from negative to positive, whereas in Twitter sentiment tends to decrease  Verification of this condition alone indicates that the ratio of potentially therapeutic conversations in Twitter is lower 
Analysis ::: Structure of Posts and Comments
Table TABREF36 presents the distribution of automatically predicted per-sentence Dialogue Acts in the datasets  The most frequent tag is statement in both  In Table TABREF37, on the other hand, we present the distribution of post and comment structures in terms of automatically predicted Dialogue Act tags  The structure is an unordered set of tags in the post or comment  From the table we can observe that the distribution of tag sets is similar between posts and comments  In both cases the most common set is statement only  However, conversations containing only statement, emphasis or question posts and comments predominantly appear in Twitter  Which is expected due to the shorter length of Twitter posts and comments 
We can also observe that the original posters tend to ask more questions than the commenting posters – 19 83% for posts vs  11 21% for comments (summed)  This suggests that the original posters frequently ask either for suggestion or confirmation of their points of view or their disconfirmation  However, the high presence of personal experiences is supported by the high number of posts containing only statements 
High number of statement tags in comments suggests that users reply either with supporting or empathic statements or personal experience  However, 6 39% of comments contain accept and reject tags, which mark the degree to which a speaker accepts some previous proposal, plan, opinion, or statement BIBREF20  The described Dialogue Act tags are often used when commenting posters discuss original poster's point of view  For instance, “It's true  I felt the same ” – {Accept, Statement} or “Well no  You're not alone” – {Reject, Statement}  The datasets differ with respect to the distribution of these Dialogue Acts tags, they appear more frequently in OSG 
Analysis ::: Sentiment of Posts and Comments
Table TABREF39 presents the distribution of sentiment polarity in post and comment statements (i e  sentences tagged as statement)  For OSG, the predominant sentiment label of statements is positive and it is the highest for both posts and comments  However, the difference between the amounts of positive and negative statements is higher for the replying comments (34 5% vs  42 5%)  For Twitter, on the other hand, the predominant sentiment label of statements is neutral and the polarity distribution between posts and comments is very close  One particular observation is that the ratio of negative statements is higher in OSG for both posts and comments than in Twitter, which supports the idea of sharing negative experiences 
Further we analyze whether the sentiment of a comment (i e  the replying user) is affected by the sentiment of the original post (i e  the user being replied to), which will imply that the users adapt their behaviour with respect to the post's sentiment  For the analysis, we split the datasets into three buckets according to the posts' sentiment score – negative, neutral, or positive, and represent each conversation in terms of percentages of comments (replies) with each sentiment label  The buckets are then compared using t-test for statistically significant differences 
Table TABREF40 presents the distribution of sentiment labels with respect to the post's sentiment score  The patterns of distribution are similar across the datasets  We can observe that overall, replies tend to have a positive sentiment, which suggests that replying posters tend to have a positive attitude  However, the ratio of positive comments is higher for OSG than for Twitter 
The results of the Welch's t-test on OSG data reveal that there are statistically significant differences in the distribution of replying comments' sentiment between conversations with positive and negative starting posts  A positive post tends to get significantly more positive replies  Similarly, a negative post tends to get significantly more negative replies (both with $p < 0 01$) 
Table TABREF41 presents the distribution of the sentiment labels of the final text provided by the original poster with respect to the sentiment polarity of the comments  The results indicate that OSG participants are more supportive, as the majority of conversations end in a positive final sentiment regardless of the sentiment of comments  We can also observe that negative comments in OSG lead to positive sentiment, which supports the idea of sharing the negative experiences, thus presence of therapeutic factors  For Twitter, on the other hand, only positive comments lead to the positive final sentiments, whereas other comments lead predominantly to neutral final sentiments 
Our analysis in terms of sentiment and Dialogue Acts supports the presence of the three selected therapeutic factors – Universality, Altruism and Instillation of Hope – in OSG more than in Twitter  The main contributors to this conclusion are the facts that there is more positive change in the sentiment of the original posters in OSG (people seeking support) and that in OSG even negative and neutral comments are likely to lead to positive changes 
Conclusion
In this work, we propose a methodology to automatically analyse online social platforms for the presence of therapeutic factors (i e  Universality, Altruism and Instillation of Hope)  We evaluate our approach on two on-line platforms, Twitter and an OSG web forum  We apply NLP techniques of Sentiment Analysis and Dialogue Act tagging to automatically verify the presence of therapeutic factors, which allows us to analyse larger amounts of conversational data (as compared to previous studies) 
Our analysis indicates that OSG conversations satisfy higher number of conditions approximating therapeutic factors than Twitter conversations  Given this outcome, we postulate that users who join support group websites spontaneously seem to benefit from it  Indeed, as shown in Section SECREF5, the original posters who interact with others by replying to comments, have benefited from an improvement of their emotional state 
We would like to reemphasise that the conditions for the therapeutic factors are necessary but not sufficient; since our analysis focuses on the structure of conversations, being agnostic to the content  NLP, however, allows us to strengthen our approximations even further  Thus, the further extension of our work is also augmentation of our study with other language analysis metrics and their correlation with human annotation 
It should be noted that the proposed approach is an approximation of the tedious tasks of annotation of conversations by experts versed in the therapeutic factors and their associated theories  Even though we can use Sentiment Analysis to detect the existence of therapeutic factors, we cannot differentiate between Altruism and Instillation of Hope, as this requires differentiation between emotional state of the user and the intended emotional communication  Thus, the natural extensions of this work are differentiation between different therapeutic factors and comparison of the proposed analysis to the human evaluation 
Although we acknowledge that the proposed methodology does not serve as a replacement of manual analysis of OSG for the presence of therapeutic factors, we believe that it could facilitate and supplement this process  The method can serve as a tool for general practitioners and psychologists who can use it as an additional source of information regarding their patients’ condition and, in turn, offer a more personalised support that is better tailored to individual therapeutic needs 

ground_truth: MLP

Eusboost, MWMOTE