prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. Which baselines are used for evaluation?
Context 1: Introduction
Headline generation is the process of creating a headline-style sentence given an input article  The research community has been regarding the task of headline generation as a summarization task BIBREF1, ignoring the fundamental differences between headlines and summaries  While summaries aim to contain most of the important information from the articles, headlines do not necessarily need to  Instead, a good headline needs to capture people's attention and serve as an irresistible invitation for users to read through the article  For example, the headline “$2 Billion Worth of Free Media for Trump”, which gives only an intriguing hint, is considered better than the summarization style headline “Measuring Trump’s Media Dominance” , as the former gets almost three times the readers as the latter  Generating headlines with many clicks is especially important in this digital age, because many of the revenues of journalism come from online advertisements and getting more user clicks means being more competitive in the market  However, most existing websites naively generate sensational headlines using only keywords or templates  Instead, this paper aims to learn a model that generates sensational headlines based on an input article without labeled data 
To generate sensational headlines, there are two main challenges  Firstly, there is a lack of sensationalism scorer to measure how sensational a headline is  Some researchers have tried to manually label headlines as clickbait or non-clickbait BIBREF2, BIBREF3  However, these human-annotated datasets are usually small and expensive to collect  To capture a large variety of sensationalization patterns, we need a cheap and easy way to collect a large number of sensational headlines  Thus, we propose a distant supervision strategy to collect a sensationalism dataset  We regard headlines receiving lots of comments as sensational samples and the headlines generated by a summarization model as non-sensational samples  Experimental results show that by distinguishing these two types of headlines, we can partially teach the model a sense of being sensational 
Secondly, after training a sensationalism scorer on our sensationalism dataset, a natural way to generate sensational headlines is to maximize the sensationalism score using reinforcement learning (RL)  However, the following shows an example of a RL model maximizing the sensationalism score by generating a very unnatural sentence, while its sensationalism scorer gave a very high score of 0 99996: UTF8gbsn 十个可穿戴产品的设计原则这消息消息可惜说明 Ten design principles for wearable devices, this message message pity introduction  This happens because the sensationalism scorer can make mistakes and RL can generate unnatural phrases which fools our sensationalism scorer  Thus, how to effectively leverage RL with noisy rewards remains an open problem  To deal with the noisy reward, we introduce Auto-tuned Reinforcement Learning (ARL)  Our model automatically tunes the ratio between MLE and RL based on how sensational the training headline is  In this way, we effectively take advantage of RL with a noisy reward to generate headlines that are both sensational and fluent 
The major contributions of this paper are as follows: 1) To the best of our knowledge, we propose the first-ever model that tackles the sensational headline generation task with reinforcement learning techniques  2) Without human-annotated data, we propose a distant supervision strategy to train a sensationalism scorer as a reward function 3) We propose a novel loss function, Auto-tuned Reinforcement Learning, to give dynamic weights to balance between MLE and RL  Our code will be released  
Sensationalism Scorer
To evaluate the sensationalism intensity score $\alpha _{\text{sen}}$ of a headline, we collect a sensationalism dataset and then train a sensationalism scorer  For the sensationalism dataset collection, we choose headlines with many comments from popular online websites as positive samples  For the negative samples, we propose to use the generated headlines from a sentence summarization model  Intuitively, the summarization model, which is trained to preserve the semantic meaning, will lose the sensationalization ability and thus the generated negative samples will be less sensational than the original one, similar to the obfuscation of style after back-translation BIBREF4  For example, an original headline like UTF8gbsn“一趟挣10万？铁总增开申通、顺丰专列" (One trip to earn 100 thousand? China Railway opens new Shentong and Shunfeng special lines) will become UTF8gbsn“中铁总将增开京广两列快递专列" (China Railway opens two special lines for express) from the baseline model, which loses the sensational phrases of UTF8gbsn“一趟挣10万？" (One trip to earn 100 thousand?)   We then train the sensationalism scorer by classifying sensational and non-sensational headlines using a one-layer CNN with a binary cross entropy loss $L_{\text{sen}}$  Firstly, 1-D convolution is used to extract word features from the input embeddings of a headline  This is followed by a ReLU activation layer and a max-pooling layer along the time dimension  All features from different channels are concatenated together and projected to the sensationalism score by adding another fully connected layer with sigmoid activation  Binary cross entropy is used to compute the loss $L_{\text{sen}}$ 
Sensationalism Scorer ::: Training Details and Dataset
For the CNN model, we choose filter sizes of 1, 3, and 5 respectively  Adam is used to optimize $L_{sen}$ with a learning rate of 0 0001  We set the embedding size as 300 and initialize it from qiu2018revisiting trained on the Weibo corpus with word and character features  We fix the embeddings during training  For dataset collection, we utilize the headlines collected in qin2018automatic, lin2019learning from Tencent News, one of the most popular Chinese news websites, as the positive samples  We follow the same data split as the original paper  As some of the links are not available any more, we get 170,754 training samples and 4,511 validation samples  For the negative training samples collection, we randomly select generated headlines from a pointer generator BIBREF0 model trained on LCSTS dataset BIBREF5 and create a balanced training corpus which includes 351,508 training samples and 9,022 validation samples  To evaluate our trained classifier, we construct a test set by randomly sampling 100 headlines from the test split of LCSTS dataset and the labels are obtained by 11 human annotators  Annotations show that 52% headlines are labeled as positive and 48% headlines as negative by majority voting (The detail on the annotation can be found in Section SECREF26) 
Sensationalism Scorer ::: Results and Discussion
Our classifier achieves 0 65 accuracy and 0 65 averaged F1 score on the test set while a random classifier would only achieve 0 50 accuracy and 0 50 averaged F1 score  This confirms that the predicted sensationalism score can partially capture the sensationalism of headlines  On the other hand, a more natural choice is to take headlines with few comments as negative examples  Thus, we train another baseline classifier on a crawled balanced sensationalism corpus of 84k headlines where the positive headlines have at least 28 comments and the negative headlines have less than 5 comments  However, the results on the test set show that the baseline classifier gets 60% accuracy, which is worse than the proposed classifier (which achieves 65%)  The reason could be that the balanced sensationalism corpus are sampled from different distributions from the test set and it is hard for the trained model to generalize  Therefore, we choose the proposed one as our sensationalism scorer  Therefore, our next challenge is to show that how to leverage this noisy sensationalism reward to generate sensational headlines 
Sensational Headline Generation
Our sensational headline generation model takes an article as input and output a sensational headline  The model consists of a Pointer-Gen headline generator and is trained by ARL  The diagram of ARL can be found in Figure FIGREF6 
We denote the input article as $x=\lbrace x_1,x_2,x_3,\cdots ,x_M\rbrace $, and the corresponding headline as $y^*=\lbrace y_1^*,y_2^*,y_3^*,\cdots ,y_T^*\rbrace $, where $M$ is the number of tokens in an article and $T$ is the number of tokens in a headline 
Sensational Headline Generation ::: Pointer-Gen Headline Generator
We choose Pointer Generator (Pointer-Gen) BIBREF0, a widely used summarization model, as our headline generator for its ability to copy words from the input article  It takes a news article as input and generates a headline  Firstly, the tokens of each article, $\lbrace x_1,x_2,x_3,\cdots ,x_M\rbrace $, are fed into the encoder one-by-one and the encoder generates a sequence of hidden states $h_i$  For each decoding step $t$, the decoder receives the embedding for each token of a headline $y_t$ as input and updates its hidden states $s_t$  An attention mechanism following luong2015effective is used:
where $v$, $W_h$, $W_s$, and $b_{attn}$ are the trainable parameters and $h_t^*$ is the context vector  $s_t$ and $h_t^*$ are then combined to give a probability distribution over the vocabulary through two linear layers:
where $V$, $b$, $V^{^{\prime }}$, and $b^{^{\prime }}$ are trainable parameters  We use a pointer generator network to enable our model to copy rare/unknown words from the input article, giving the following final word probability:
where $x^t$ is the embedding of the input word of the decoder, $w_{h^*}^T$, $w_s^T$, $w_x^T$, and $b_{ptr}$ are trainable parameters, and $\sigma $ is the sigmoid function 
Sensational Headline Generation ::: Training Methods
We first briefly introduce MLE and RL objective functions, and a naive way to mix these two by a hyper-parameter $\lambda $  Then we point out the challenge of training with noisy reward, and propose ARL to address this issue 
Sensational Headline Generation ::: Training Methods ::: MLE and RL
A headline generation model can be trained with MLE, RL or a combination of MLE and RL  MLE training is to minimize the negative log likelihood of the training headlines  We feed $y^*$ into the decoder word by word and maximize the likelihood of $y^*$  The loss function for MLE becomes
For RL training, we choose the REINFORCE algorithm BIBREF6  In the training phase, after encoding an article, a headline $y^s = \lbrace y_1^s, y_2^s, y_3^s, \cdots , y_T^s\rbrace $ is obtained by sampling from $P(w)$ from our generator, and then a reward of sensationalism or ROUGE(RG) is calculated 
We use the baseline reward $\hat{R_t}$ to reduce the variance of the reward, similar to ranzato2015sequence  To elaborate, a linear model is deployed to estimate the baseline reward $\hat{R_t}$ based on $t$-th state $o_t$ for each timestep $t$  The parameters of the linear model are trained by minimizing the mean square loss between $R$ and $\hat{R_t}$:
where $W_r$ and $b_r$ are trainable parameters  To maximize the expected reward, our loss function for RL becomes
A naive way to mix these two objective functions using a hyper-parameter $\lambda $ has been successfully incorporated in the summarization task BIBREF7  It includes the MLE training as a language model to mitigate the readability and quality issues in RL  The mixed loss function is shown as follows:
where $*$ is the reward type  Usually $\lambda $ is large, and paulus2017deep used 0 9984 
Sensational Headline Generation ::: Training Methods ::: Auto-tuned Reinforcement Learning
Applying the naive mixed training method using sensationalism score as the reward is not obvious/trivial in our task  The main reason is that our sensationalism reward is notably more noisy and more fragile than the ROUGE-L reward or abstractive reward used in the summarization task BIBREF7, BIBREF8  A higher ROUGE-L F1 reward in summarization indicates higher overlapping ratio between generation and true summary statistically, but our sensationalism reward is a learned score which is fragile to be fooled with unnatural samples 
To effectively train the model with RL under noisy sensationalism reward, our idea is to balance RL with MLE  However, we argue that the weighted ratio between MLE and RL should be sample-dependent, instead of being fixed for all training samples as in paulus2017deep, kryscinski2018improving  The reason is that, RL and MLE have inconsistent optimization objectives  When the training headline is non-sensational, MLE training will encourage our model to imitate the training headline (thus generating non-sensational headlines), which counteracts the effects of RL training to generate sensational headlines 
The sensationalism score is, therefore, used to give dynamic weight to MLE and RL  Our ARL loss function becomes:
If $\alpha _{\text{sen}}(y^*)$ is high, meaning the training headline is sensational, our loss function encourages our model to imitate the sample more using the MLE training  If $\alpha _{\text{sen}}(y^*)$ is low, our loss function replies on RL training to improve the sensationalism  Note that the weight $\alpha _{\text{sen}}(y^*)$ is different from our sensationalism reward $\alpha _{\text{sen}}(y^s)$ and we call the loss function Auto-tuned Reinforcement Learning, because the ratio between MLE and RL are well “tuned” towards different samples 
Sensational Headline Generation ::: Dataset
We use LCSTS BIBREF5 as our dataset to train the summarization model  The dataset is collected from the Chinese microblogging website Sina Weibo  It contains over 2 million Chinese short texts with corresponding headlines given by the author of each text  The dataset is split into 2,400,591 samples for training, 10,666 samples for validation and 725 samples for testing  We tokenize each sentence with Jieba and a vocabulary size of 50000 is saved 
Sensational Headline Generation ::: Baselines and Our Models
We experiment and compare with the following models 
Pointer-Gen is the baseline model trained by optimizing $L_\text{MLE}$ in Equation DISPLAY_FORM13 
Pointer-Gen+Pos is the baseline model by training Pointer-Gen only on positive examples whose sensationalism score is larger than 0 5
Pointer-Gen+Same-FT is the model which fine-tunes Pointer-Gen on the training samples whose sensationalism score is larger than 0 1
Pointer-Gen+Pos-FT is the model which fine-tunes Pointer-Gen on the training samples whose sensationalism score is larger than 0 5
Pointer-Gen+RL-ROUGE is the baseline model trained by optimizing $L_\text{RL-ROUGE}$ in Equation DISPLAY_FORM17, with ROUGE-L BIBREF9 as the reward 
Pointer-Gen+RL-SEN is the baseline model trained by optimizing $L_\text{RL-SEN}$ in Equation DISPLAY_FORM17, with $\alpha _\text{sen}$ as the reward 
Pointer-Gen+ARL-SEN is our model trained by optimizing $L_\text{ARL-SEN}$ in Equation DISPLAY_FORM19, with $\alpha _\text{sen}$ as the reward 
Test set is the headlines from the test set 
Note that we didn't compare to Pointer-Gen+ARL-ROUGE as it is actually Pointer-GEN  Recall that $\alpha _{\text{sen}}(y^*)$ in Equation DISPLAY_FORM19 measures how good (based on reward function) is $y^*$  Then the loss function for Pointer-Gen+ARL-ROUGE will be
We also tried text style transfer baseline BIBREF10, but the generated headlines were very poor (many unknown words and irrelevant) 
Sensational Headline Generation ::: Training Details
MLE training: An Adam optimizer is used with the learning rate of 0 0001 to optimize $L_{\text{MLE}}$  The batch size is set as 128 and a one-layer, bi-directional Long Short-Term Memory (bi-LSTM) model with 512 hidden sizes and a 350 embedding size is utilized  Gradients with the l2 norm larger than 2 0 are clipped  We stop training when the ROUGE-L f-score stops increasing 
Hybrid training: An Adam optimizer with a learning rate of 0 0001 is used to optimize $L_{\text{RL-*}}$ (Equation DISPLAY_FORM17) and $L_\text{{ARL-SEN}}$ (Equation DISPLAY_FORM19)  When training Pointer-Gen+RL-ROUGE, the best $\lambda $ is chosen based on the ROUGE-L score on the validation set  In our experiment, $\lambda $ is set as 0 95  An Adam optimizer with a learning rate of 0 001 is used to optimize $L_b$  When training Pointer-Gen+ARL-SEN, we don't use the full LCSTS dataset, but only headlines with a sensationalism score larger than 0 1 as we observe that Pointer-Gen+ARL-SEN will generate a few unnatural phrases when using full dataset  We believe the reason is the high ratio of RL during training  Figure FIGREF23 shows that the probability density near 0 is very high, meaning that in each batch, many of the samples will have a very low sensationalism score  On expectation, each sample will receive 0 239 MLE training and 0 761 RL training  This leads to RL dominanting the loss  Thus, we propose to filter samples with a minimum sensationalism score with 0 1 and it works very well  For Pointer-Gen+RL-SEN, we also set the minimum sensationalism score as 0 1, and $\lambda $ is set as 0 5 to remove unnatural phrases, making a fair comparison to Pointer-Gen+ARL-SEN 
We stop training Pointer-Gen+Same-FT, Pointer-Gen+Pos-FT, Pointer-Gen+RL-SEN and Pointer-Gen+ARL-SEN, when $\alpha _\text{sen}$ stops increasing on the validation set  Beam-search with a beam size of 5 is adopted for decoding in all models 
Sensational Headline Generation ::: Evaluation Metrics
We briefly describe the evaluation metrics below 
ROUGE: ROUGE is a commonly used evaluation metric for summarization  It measures the N-gram overlap between generated and training headlines  We use it to evaluate the relevance of generated headlines  The widely used pyrouge toolkit is used to calculate ROUGE-1 (RG-1), ROUGE-2 (RG-2), and ROUGE-L (RG-L) 
Human evaluation: We randomly sample 50 articles from the test set and send the generated headlines from all models and corresponding headlines in the test set to human annotators  We evaluate the sensationalism and fluency of the headlines by setting up two independent human annotation tasks  We ask 10 annotators to label each headline for each task  For the sensationalism annotation, each annotator is asked one question, “Is the headline sensational?”, and he/she has to choose either `yes' or `no'  The annotators were not told which system the headline is from  The process of distributing samples and recruiting annotators is managed by Crowdflower  After annotation, we define the sensationalism score as the proportion of annotations on all generated headlines from one model labeled as `yes'  For the fluency annotation, we repeat the same procedure as for the sensationalism annotation, except that we ask each annotator the question “Is the headline fluent?” We define the fluency score as the proportion of annotations on all headlines from one specific model labeled as `yes'  We put human annotation instructions in the supplemental material 
UTF8gbsn
Results
We first compare all four models, Pointer-Gen, Pointer-Gen-RL+ROUGE, Pointer-Gen-RL-SEN, and Pointer-Gen-ARL-SEN, to existing models with ROUGE in Table TABREF25 to establish that our model produces relevant headlines and we leave the sensationalism for human evaluation  Note that we only compare our models to commonly used strong summarization baselines, to validate that our implementation achieves comparable performance to existing work  In our implementation, Pointer-Gen achieves a 34 51 RG-1 score, 22 21 RG-2 score, and 31 68 RG-L score, which is similar to the results of gu2016incorporating  Pointer-Gen+ARL-SEN, although optimized for the sensationalism reward, achieves similar performance to our Pointer-Gen baseline, which means that Pointer-Gen+ARL-SEN still keeps its summarization ability  An example of headlines generated from different models in Table TABREF29 shows that Pointer-Gen and Pointer-Gen+RL-ROUGE learns to summarize the main point of the article: “The Nikon D600 camera is reported to have black spots when taking photos”  Pointer-Gen+RL-SEN makes the headline more sensational by blaming Nikon for attributing the damage to the smog  Pointer-Gen+ARL-SEN generates the most sensational headline by exaggerating the result “Getting a serious trouble!” to maximize user's attention 
We then compare different models using the sensationalism score in Table TABREF30  The Pointer-Gen baseline model achieves a 42 6% sensationalism score, which is the minimum that a typical summarization model achieves  By filtering out low-sensational headlines, Pointer-Gen+Same-FT and Pointer-Gen+Pos-FT achieves higher sensationalism scores, which implies the effectiveness of our sensationalism scorer  Our Pointer-Gen+ARL-SEN model achieves the best performance of 60 8%  This is an absolute improvement of 18 2% over the Pointer-Gen baseline  The Chi-square test on the results confirms that Pointer-Gen+ARL-SEN is statistically significantly more sensational than all the other baseline models, with the largest p-value less than 0 01  Also, we find that the test set headlines achieves 57 8% sensationalism score, much larger than Pointer-Gen baseline, which also supports our intuition that generated headlines will be less sensational than the original one  On the other hand, we found that Pointer-Gen+Pos is much worse than other baselines  The reason is that training on sensational samples alone discards around 80% of the whole training set that is also helpful for maintaining relevance and a good language model  It shows the necessity of using RL 
UTF8gbsn
In addition, both Pointer-Gen+RL-SEN and Pointer-Gen+ARL-SEN, which use the sensationalism score as the reward, obtain statistically better results than Pointer-Gen+RL-ROUGE and Pointer-Gen, with a p-value less than 0 05 by a Chi-square test  This result shows the effectiveness of RL to generate more sensational headlines  The reason is that even though our noisy classifier could also learn to classify domains, the generator during RL training is not allowed to increase the reward by shifting domains but encouraged to generate more sensational headlines, due to the consistency constraint on the domains of the headline and the article  Furthermore, Poiner-Gen+ARL-SEN gets better performance than Pointer-Gen+RL-SEN, which confirms the superiority of the ARL loss function  We also visualize in Figure FIGREF31 a comparison between Pointer-Gen+ARL-SEN and Pointer-Gen+RL-SEN according to how sensational the test set headlines are  The blue bars denote the smaller scores between the two models  For example, if the blue bar is 0 6, it means that the worse model between Pointer-Gen+RL-SEN and Pointer-Gen+ARL-SEN achieves 0 6  And the color of orange/black further indicates the better model and its score  We find that Pointer-Gen+ARL-SEN outperforms Pointer-Gen+RL-SEN for most cases  The improvement is higher when the test set headlines are not sensational (the sensationalism score is less than 0 5), which may be attributed to the higher ratio of RL training on non-sensational headlines 
Apart from the sensationalism evaluation, we measure the fluency of the headlines generated from different models  Fluency scores in Table TABREF30 show that Pointer-Gen+RL-SEN and Pointer-Gen+ARL-SEN achieve comparable fluency performance to Pointer-Gen and Pointer-Gen+RL-ROUGE  Test set headlines achieve the best performance among all models, but the difference is not statistically significant  Also, we observe that fine-tuning on sensational headlines will hurt the performance, both in sensationalism and fluency 
After manually checking the outputs, we observe that our model is able to generate sensational headlines using diverse sensationalization strategies  These strategies include, but are not limited to, creating a curiosity gap, asking questions, highlighting numbers, being emotional and emphasizing the user  Examples can be found in Table TABREF32 
Related Work
Our work is related to summarization tasks  An encoder-decoder model was first applied to two sentence-level abstractive summarization tasks on the DUC-2004 and Gigaword datasets BIBREF12  This model was later extended by selective encoding BIBREF13, a coarse to fine approach BIBREF14, minimum risk training BIBREF1, and topic-aware models BIBREF15  As long summaries were recognized as important, the CNN/Daily Mail dataset was used in nallapati2016abstractive  Graph-based attention BIBREF16, pointer-generator with coverage loss BIBREF0 are further developed to improve the generated summaries  celikyilmaz2018deep proposed deep communicating agents for representing a long document for abstractive summarization  In addition, many papers BIBREF17, BIBREF18, BIBREF19 use extractive methods to directly select sentences from articles  However, none of these work considered the sensationalism of generated outputs 
RL is also gaining popularity as it can directly optimize non-differentiable metrics BIBREF20, BIBREF21, BIBREF22  paulus2017deep proposed an intra-decoder model and combined RL and MLE to deal with summaries with bad qualities  RL has also been explored with generative adversarial networks (GANs) BIBREF23  liu2017generative applied GANs on summarization task and achieved better performance  niu2018polite tackles the problem of polite generation with politeness reward  Our work is different in that we propose a novel function to balance RL and MLE 
Our task is also related to text style transfer  Implicit methods BIBREF10, BIBREF24, BIBREF4 transfer the styles by separating sentence representations into content and style, for example using back-translationBIBREF4  However, these methods cannot guarantee the content consistency between the original sentence and transferred output BIBREF25  Explicit methods BIBREF26, BIBREF25 transfer the style by directly identifying style related keywords and modifying them  However, sensationalism is not always restricted to keywords, but the full sentence  By leveraging small human labeled English dataset, clickbait detection has been well investigated BIBREF2, BIBREF27, BIBREF3  However, these human labeled dataset are not available for other languages, such as Chinese 
Modeling sensationalism is also related to modeling emotion  Emotion has been well investigated in both word levelBIBREF28, BIBREF29 and sentence levelBIBREF30, BIBREF31, BIBREF32, BIBREF33, BIBREF34  It has also been considered an important factor in engaging interactive systemsBIBREF35, BIBREF36, BIBREF37  Although we observe that sensational headlines contain emotion, it is still not clear which emotion and how emotions will influence the sensationalism 
Conclusion and Future Work
In this paper, we propose a model that generates sensational headlines without labeled data using Reinforcement Learning  Firstly, we propose a distant supervision strategy to train the sensationalism scorer  As a result, we achieve 65% accuracy between the predicted sensationalism score and human evaluation  To effectively leverage this noisy sensationalism score as the reward for RL, we propose a novel loss function, ARL, to automatically balance RL with MLE  Human evaluation confirms the effectiveness of both our sensationalism scorer and ARL to generate more sensational headlines  Future work can be improving the sensationalism scorer and investigating the applications of dynamic balancing methods between RL and MLE in textGANBIBREF23  Our work also raises the ethical questions about generating sensational headlines, which can be further explored 
Acknowledgments
Thanks to ITS/319/16FP of Innovation Technology Commission, HKUST 16248016 of Hong Kong Research Grants Council for funding  In addition, we thank Zhaojiang Lin for helpful discussion and Yan Xu, Zihan Liu for the data collection 
Context 3: Introduction
Word embeddings are representations of words in numerical form, as vectors of typically several hundred dimensions  The vectors are used as an input to machine learning models; for complex language processing tasks these are typically deep neural networks  The embedding vectors are obtained from specialized learning tasks, based on neural networks, e g , word2vec BIBREF0, GloVe BIBREF1, FastText BIBREF2, ELMo BIBREF3, and BERT BIBREF4  For training, the embeddings algorithms use large monolingual corpora that encode important information about word meaning as distances between vectors  In order to enable downstream machine learning on text understanding tasks, the embeddings shall preserve semantic relations between words, and this is true even across languages 
Probably the best known word embeddings are produced by the word2vec method BIBREF5  The problem with word2vec embeddings is their failure to express polysemous words  During training of an embedding, all senses of a given word (e g , paper as a material, as a newspaper, as a scientific work, and as an exam) contribute relevant information in proportion to their frequency in the training corpus  This causes the final vector to be placed somewhere in the weighted middle of all words' meanings  Consequently, rare meanings of words are poorly expressed with word2vec and the resulting vectors do not offer good semantic representations  For example, none of the 50 closest vectors of the word paper is related to science 
The idea of contextual embeddings is to generate a different vector for each context a word appears in and the context is typically defined sentence-wise  To a large extent, this solves the problems with word polysemy, i e  the context of a sentence is typically enough to disambiguate different meanings of a word for humans and so it is for the learning algorithms  In this work, we describe high-quality models for contextual embeddings, called ELMo BIBREF3, precomputed for seven morphologically rich, less-resourced languages: Slovenian, Croatian, Finnish, Estonian, Latvian, Lithuanian, and Swedish  ELMo is one of the most successful approaches to contextual word embeddings  At time of its creation, ELMo has been shown to outperform previous word embeddings BIBREF3 like word2vec and GloVe on many NLP tasks, e g , question answering, named entity extraction, sentiment analysis, textual entailment, semantic role labeling, and coreference resolution 
This report is split into further five sections  In section SECREF2, we describe the contextual embeddings ELMo  In Section SECREF3, we describe the datasets used and in Section SECREF4 we describe preprocessing and training of the embeddings  We describe the methodology for evaluation of created vectors and results in Section SECREF5  We present conclusion in Section SECREF6 where we also outline plans for further work 
ELMo
Typical word embeddings models or representations, such as word2vec BIBREF0, GloVe BIBREF1, or FastText BIBREF2, are fast to train and have been pre-trained for a number of different languages  They do not capture the context, though, so each word is always given the same vector, regardless of its context or meaning  This is especially problematic for polysemous words  ELMo (Embeddings from Language Models) embedding BIBREF3 is one of the state-of-the-art pretrained transfer learning models, that remedies the problem and introduces a contextual component 
ELMo model`s architecture consists of three neural network layers  The output of the model after each layer gives one set of embeddings, altogether three sets  The first layer is a CNN layer, which operates on a character level  It is context independent, so each word always gets the same embedding, regardless of its context  It is followed by two biLM layers  A biLM layer consists of two concatenated LSTMs  In the first LSTM, we try to predict the following word, based on the given past words, where each word is represented by the embeddings from the CNN layer  In the second LSTM, we try to predict the preceding word, based on the given following words  It is equivalent to the first LSTM, just reading the text in reverse 
In NLP tasks, any set of these embeddings may be used; however, a weighted average is usually used  The weights of the average are learned during the training of the model for the specific task  Additionally, an entire ELMo model can be fine-tuned on a specific end task 
Although ELMo is trained on character level and is able to handle out-of-vocabulary words, a vocabulary file containing most common tokens is used for efficiency during training and embedding generation  The original ELMo model was trained on a one billion word large English corpus, with a given vocabulary file of about 800,000 words  Later, ELMo models for other languages were trained as well, but limited to larger languages with many resources, like German and Japanese 
ELMo ::: ELMoForManyLangs
Recently, ELMoForManyLangs BIBREF6 project released pre-trained ELMo models for a number of different languages BIBREF7  These models, however, were trained on a significantly smaller datasets  They used 20-million-words data randomly sampled from the raw text released by the CoNLL 2017 Shared Task - Automatically Annotated Raw Texts and Word Embeddings BIBREF8, which is a combination of Wikipedia dump and common crawl  The quality of these models is questionable  For example, we compared the Latvian model by ELMoForManyLangs with a model we trained on a complete (wikidump + common crawl) Latvian corpus, which has about 280 million tokens  The difference of each model on the word analogy task is shown in Figure FIGREF16 in Section SECREF5  As the results of the ELMoForManyLangs embeddings are significantly worse than using the full corpus, we can conclude that these embeddings are not of sufficient quality  For that reason, we computed ELMo embeddings for seven languages on much larger corpora  As this effort requires access to large amount of textual data and considerable computational resources, we made the precomputed models publicly available by depositing them to Clarin repository 
Training Data
We trained ELMo models for seven languages: Slovenian, Croatian, Finnish, Estonian, Latvian, Lithuanian and Swedish  To obtain high-quality embeddings, we used large monolingual corpora from various sources for each language  Some corpora are available online under permissive licences, others are available only for research purposes or have limited availability  The corpora used in training datasets are a mix of news articles and general web crawl, which we preprocessed and deduplicated  Below we shortly describe the used corpora in alphabetical order of the involved languages  Their names and sizes are summarized in Table TABREF3 
Croatian dataset include hrWaC 2 1 corpus BIBREF9, Riznica BIBREF10, and articles of Croatian branch of Styria media house, made available to us through partnership in a joint project  hrWaC was built by crawling the  hr internet domain in 2011 and 2014  Riznica is composed of Croatian fiction and non-fiction prose, poetry, drama, textbooks, manuals, etc  The Styria dataset consists of 570,219 news articles published on the Croatian 24sata news portal and niche portals related to 24sata 
Estonian dataset contains texts from two sources, CoNLL 2017 Shared Task - Automatically Annotated Raw Texts and Word Embeddings BIBREF8, and news articles made available to us by Ekspress Meedia due to partnership in the project  Ekspress Meedia dataset is composed of Estonian news articles between years 2009 and 2019  The CoNLL 2017 corpus is composed of Estonian Wikipedia and webcrawl 
Finnish dataset contains articles by Finnish news agency STT, Finnish part of the CoNLL 2017 dataset, and Ylilauta downloadable version BIBREF11  STT news articles were published between years 1992 and 2018  Ylilauta is a Finnish online discussion board; the corpus contains parts of the discussions from 2012 to 2014 
Latvian dataset consists only of the Latvian portion of the ConLL 2017 corpus 
Lithuanian dataset is composed of Lithuanian Wikipedia articles from 2018, DGT-UD corpus, and LtTenTen  DGT-UD is a parallel corpus of 23 official languages of the EU, composed of JRC DGT translation memory of European law, automatically annotated with UD-Pipe 1 2  LtTenTen is Lithuanian web corpus made up of texts collected from the internet in April 2014 BIBREF12 
Slovene dataset is formed from the Gigafida 2 0 corpus BIBREF13  It is a general language corpus composed of various sources, mostly newspapers, internet pages, and magazines, but also fiction and non-fiction prose, textbooks, etc 
Swedish dataset is composed of STT Swedish articles and Swedish part of CoNLL 2017  The Finnish news agency STT publishes some of its articles in Swedish language  They were made available to us through partnership in a joint project  The corpus contains those articles from 1992 to 2017 
Preprocessing and Training
Prior to training the ELMo models, we sentence and word tokenized all the datasets  The text was formatted in such a way that each sentence was in its own line with tokens separated by white spaces  CoNLL 2017, DGT-UD and LtTenTen14 corpora were already pre-tokenized  We tokenized the others using the NLTK library and its tokenizers for each of the languages

ground_truth: Pointer-Gen, Pointer-Gen+Pos, Pointer-Gen+Same-FT, Pointer-Gen+Pos-FT, Pointer-Gen+RL-ROUGE, Pointer-Gen+RL-SEN

prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. What are the sources of the datasets?
Context 2: Introduction
Emotion detection has long been a topic of interest to scholars in natural language processing (NLP) domain  Researchers aim to recognize the emotion behind the text and distribute similar ones into the same group  Establishing an emotion classifier can not only understand each user's feeling but also be extended to various application, for example, the motivation behind a user's interests BIBREF0  Based on releasing of large text corpus on social media and the emotion categories proposed by BIBREF1, BIBREF2, numerous models have provided and achieved fabulous precision so far  For example, DeepMoji BIBREF3 which utilized transfer learning concept to enhance emotions and sarcasm understanding behind the target sentence  CARER BIBREF4 learned contextualized affect representations to make itself more sensitive to rare words and the scenario behind the texts 
As methods become mature, text-based emotion detecting applications can be extended from a single utterance to a dialogue contributed by a series of utterances  Table TABREF2 illustrates the difference between single utterance and dialogue emotion recognition  The same utterances in Table TABREF2, even the same person said the same sentence, the emotion it convey may be various, which may depend on different background of the conversation, tone of speaking or personality  Therefore, for emotion detection, the information from preceding utterances in a conversation is relatively critical 
In SocialNLP 2019 EmotionX, the challenge is to recognize emotions for all utterances in EmotionLines dataset, a dataset consists of dialogues  According to the needs for considering context at the same time, we develop two classification models, inspired by bidirectional encoder representations from transformers (BERT) BIBREF5, FriendsBERT and ChatBERT  In this paper, we introduce our approaches including causal utterance modeling, model pre-training, and fine-turning 
Dataset
EmotionLines BIBREF6 is a dialogue dataset composed of two subsets, Friends and EmotionPush, according to the source of the dialogues  The former comes from the scripts of the Friends TV sitcom  The other is made up of Facebook messenger chats  Each subset includes $1,000$ English dialogues, and each dialogue can be further divided into a few consecutive utterances  All the utterances are annotated by five annotators on a crowd-sourcing platform (Amazon Mechanical Turk), and the labeling work is only based on the textual content  Annotator votes for one of the seven emotions, namely Ekman’s six basic emotions BIBREF1, plus the neutral  If none of the emotion gets more than three votes, the utterance will be marked as “non-neutral” 
For the datasets, there are properties worth additional mentioning  Although Friends and EmotionPush share the same data format, they are quite different in nature  Friends is a speech-based dataset which is annotated dialogues from the TV sitcom  It means most of the utterances are generated by the a few main characters  The personality of a character often affects the way of speaking, and therefore “who is the speaker" might provide extra clues for emotion prediction  In contrast, EmotionPush does not have this trait due to the anonymous mechanism  In addition, features such as typo, hyperlink, and emoji that only appear in chat-based data will need some domain-specific techniques to process 
Incidentally, the objective of the challenge is to predict the emotion for each utterance  Just, according to EmotionX 2019 specification, there are only four emotions be selected as our label candidates, which are Joy, Sadness, Anger, and Neutral  These emotions will be considered during performance evaluation  The technical detail will also be introduced and discussed in following Section SECREF13 and Section SECREF26 
Model Description
For this challenge, we adapt BERT which is proposed by BIBREF5 to help understand the context at the same time  Technically, BERT, designed on end-to-end architecture, is a deep pre-trained transformer encoder that dynamically provides language representation and BERT already achieved multiple state-of-the-art results on GLUE benchmark BIBREF7 and many tasks  A quick recap for BERT's architecture and its pre-training tasks will be illustrated in the following subsections 
Model Description ::: Model Architecture
BERT, the Bidirectional Encoder Representations from Transformers, consists of several transformer encoder layers that enable the model to extract very deep language features on both token-level and sentence-level  Each transformer encoder contains multi-head self-attention layers that provide ability to learn multiple attention feature of each word from their bidirectional context  The transformer and its self-attention mechanism are proposed by BIBREF8  This self-attention mechanism can be interpreted as a key-value mapping given query  By given the embedding vector for token input, the query ($Q$), key ($K$) and value ($V$) are produced by the projection from each three parameter matrices where $W^Q \in \mathbb {R}^{d_{{\rm model}} \times d_{k}}, W^K \in \mathbb {R}^{d_{\rm model} \times d_{k}}$ and $W^V \in \mathbb {R}^{d_{\rm model} \times d_{v}}$  The self-attention BIBREF8 is formally represented as:
The $ d_k = d_v = d_{\rm model} = 1024$ in BERT large version and 768 in BERT base version  Once model can extract attention feature, we can extend one self-attention into multi-head self-attention, this extension makes sub-space features can be extracted in same time by this multi-head configuration  Overall, the multi-attention mechanism is adopt for each transformer encoder, and several of encoder layer will be stacked together to form a deep transformer encoder 
For the model input, BERT allow us take one sentence as input sequence or two sentences together as one input sequence, and the maximum length of input sequence is 512  The way that BERT was designed is for giving model the sentence-level and token-level understanding  In two sentences case, a special token ([SEP]) will be inserted between two sentences  In addition, the first input token is also a special token ([CLS]), and its corresponding ouput will be vector place for classification during fine-tuning  The outputs of the last encoder layer corresponding to each input token can be treated as word representations for each token, and the word representation of the first token ([CLS]) will be consider as classification (output) representation for further fine-tuning tasks  In BERT, this vector is denoted as $C \in \mathbb {R}^{d_{\rm model}} $, and a classification layer is denoted as $ W \in \mathbb {R}^{K \times d_{\rm model}}$, where $K$ is number of classification labels  Finally, the prediction $P$ of BERT is represented as $P = {\rm softmax}(CW^T)$ 
Model Description ::: Pre-training Tasks
In pre-training, intead of using unidirectional language models, BERT developed two pre-training tasks: (1) Masked LM (cloze test) and (2) Next Sentence Prediction  At the first pre-training task, bidirectional language modeling can be done at this cloze-like pre-training  In detail, 15% tokens of input sequence will be masked at random and model need to predict those masked tokens  The encoder will try to learn contextual representations from every given tokens due to masking tokens at random  Model will not know which part of the input is going to be masked, so that the information of each masked tokens should be inferred by remaining tokens  At Next Sentence Prediction, two sentences concatenated together will be considered as model input  In order to give model a good nature language understanding, knowing relationship between sentence is one of important abilities  When generating input sequences, 50% of time the sentence B is actually followed by sentence A, and rest 50% of the time the sentence B will be picked randomly from dataset, and model need to predict if the sentence B is next sentence of sentence A  That is, the attention information will be shared between sentences  Such sentence-level understanding may have difficulties to be learned at first pre-training task (Masked LM), therefore, the pre-training task (NSP) is developed as second training goal to capture the cross sentence relationship 
In this competition, limited by the size of dataset and the challenge in contextual emotion recognition, we consider BERT with both two pre-training tasks can give a good starting point to extract emotion changing during dialogue-like conversation  Especially the second pre-training task, it might be more important for dialogue-like conversation where the emotion may various by the context of continuous utterances  That is, given a set of continues conversations, the emotion of current utterance might be influenced by previous utterance  By this assumption and with supporting from the experiment results of BERT, we can take sentence A as one-sentence context and consider sentence B as the target sentence for emotion prediction  The detail will be described in Section SECREF4 
Methodology
The main goal of the present work is to predict the emotion of utterance within the dialogue  Following are four major difficulties we concern about:
The emotion of the utterances depends not only on the text but also on the interaction happened earlier 
The source of the two datasets are different  Friends is speech-based dialogues and EmotionPush is chat-based dialogues  It makes datasets possess different characteristics 
There are only $1,000$ dialogues in both training datasets which are not large enough for the stability of training a complex neural-based model 
The prediction targets (emotion labels) are highly unbalanced 
The proposed approach is summarized in Figure FIGREF3, which aims to overcome these challenges  The framework could be separated into three steps and described as follow:
Methodology ::: Causal Utterance Modeling
Given a dialogue $D^{(i)}$ which includes sequence of utterances denoted as $D^{(i)}=(u^{(i)}_{1}, u^{(i)}_{2},    , u^{(i)}_{n})$, where $i$ is the index in dataset and $n$ is the number of utterances in the given dialogue  In order to conserve the emotional information of both utterance and conversation, we rearrange each two consecutive utterances $u_{t}, u_{t-1}$ into a single sentence representation $x_{t}$ as
The corresponding sentence representation corpus $X^{(i)}$ are denoted as $X^{(i)}=(x^{(i)}_{1}, x^{(i)}_{2},    , x^{(i)}_{n})$  Note that the first utterance within a conversation does not have its causal utterance (previous sentence), therefore, the causal utterance will be set as [None]  A practical example of sentence representation is shown in Table TABREF11 
Since the characteristics of two datasets are not identical, we customize different causal utterance modeling strategies to refine the information in text 
For Friends, there are two specific properties  The first one is that most dialogues are surrounding with the six main characters, including Rachel, Monica, Phoebe, Joey, Chandler, and Ross  The utterance ratio of given by the six roles is up to $83 4\%$  Second, the personal characteristics of the six characters are very clear  Each leading role has its own emotion undulated rule  To make use of these features, we introduce the personality tokenization which help learning the personality of the six characters  Personality tokenization concatenate the speaker and says tokens before the input utterance if the speaker is one of the six characters  The example is shown in Table TABREF12 
For EmotionPush, the text are informal chats which including like slang, acronym, typo, hyperlink, and emoji  Another characteristic is that the specific name entities are tokenized with random index  (e g  “organization_80”, “person_01”, and “time_12”)  We consider some of these informal text are related to expressing emotion such as repeated typing, purposed capitalization, and emoji (e g  “:D”, “:(”, and “<3”))  Therefore, we keep most informal expressions but only process hyperlinks, empty utterance, and name entities by unifying the tokens 
Methodology ::: Model Pre-training
Since the size of both datasets are not large enough for complex neural-based model training as well as BERT model is only pre-train on formal text datasets, the issues of overfitting and domain bias are important considerations for design the pre-training process 
To avoid our model overfitting on the training data and increase the understanding of informal text, we adapted BERT and derived two models, namely FriendsBERT and ChatBERT, with different pre-training tasks before the formal training process for Friends and EmotionPush dataset, respectively  The pre-training strategies are described below 
For pre-training FriendsBERT, we collect the completed scripts of all ten seasons of Friends TV shows from emorynlp which includes 3,107 scenes within 61,309 utterances  All the utterances are followed the preprocessing methods mentions above to compose the corpus for Masked language model pre-training task  The consequent utterances in the same scenes are considered as the consequent sentences to pre-train the Next Sentence Prediction task  In the pre-training process, the training loss is the sum of the mean likelihood of two pre-train tasks 
For pre-training ChatBERT, we pre-train our model on the Twitter dataset, since the text and writing style on Twitter are close to the chat text where both may involved with many informal words or emoticons as well  The Twitter emotion dataset, 8 basic emotions from emotion wheel BIBREF1, was collected by twitter streaming API with specific emotion-related hashtags, such as #anger, #joy, #cry, #sad and etc  The hashtags in tweets are treated as emotion label for model fine-tuning  The tweets were fine-grined processing followed the rules in BIBREF9, BIBREF4, including duplicate tweets removing, the emotion hashtags must appearing in the last position of a tweet, and etc  The statis of tweets were summarized in Table TABREF17  Each tweet and corresponding emotion label composes an emotion classification dataset for pre-training 
Methodology ::: Fine-tuning
Since our emotion recognition task is treated as a sequence-level classification task, the model would be fine-tuned on the processed training data  Following the BERT construction, we take the first embedding vector which corresponds to the special token [CLS] from the final hidden state of the Transformer encoder  This vector represents the embedding vector of the corresponding conversation utterances which is denoted as $\mathbf {C} \in \mathbb {R}^{H}$, where $H$ is the embedding size  A dense neural layer is treated as a classification layer which consists of parameters $\mathbf {W} \in \mathbb {R}^{K\times H}$ and $\mathbf {b} \in \mathbb {R}^{K}$, where $K$ is the number of emotion class  The emotion prediction probabilities $\mathbf {P} \in \mathbb {R}^{K}$ are computed by a softmax activation function as
All the parameters in BERT and the classification layer would be fine-turned together to minimize the Negative Log Likelihood (NLL) loss function, as Equation (DISPLAY_FORM22), based on the ground truth emotion label $c$ 
In order to tackle the problem of highly unbalanced emotion labels, we apply weighted balanced warming on NLL loss function, as Equation (DISPLAY_FORM23), in the first epoch of fine-tuning procedure 
where $\mathbf {w}$ are the weights of corresponding emotion label $c$ which are computed and normalize by the frequency as
By adding the weighted balanced warming on NLL loss, the model could learn to predict the minor emotions (e g  anger and sadness) earlier and make the training process more stable  Since the major evaluation metrics micro F1-score is effect by the number of each label, we only apply the weighted balanced warming in first epoch to optimize the performance 
Experiments
Since the EmotionX challenge only provided the gold labels in training data, we pick the best performance model (weights) to predict the testing data  In this section, we present the experiment and evaluation results 
Experiments ::: Experimental Setup
The EmotionX challenge consists of $1,000$ dialogues for both Friends and EmotionPush  In all of our experiments, each dataset is separated into top 800 dialogues for training and last 200 dialogues for validation  Since the EmotionX challenge considers only the four emotions (anger, joy, neutral, and sadness) in the evaluation stage, we ignore all the data point corresponding to other emotions directly  The details of emotions distribution are shown in Table TABREF18 
The hyperparameters and training setup of our models (FriendsBERT and ChatBERT) are shown in Table TABREF25  Some common and easily implemented methods are selected as the baselines embedding methods and classification models  The baseline embedding methods are including bag-of-words (BOW), term frequency–inverse document frequency (TFIDF), and neural-based word embedding  The classification models are including Logistic Regression (LR), Random Forest (RF), TextCNN BIBREF10 with initial word embedding as GloVe BIBREF11, and our proposed model  All the experiment results are based on the best performances of validation results 
Experiments ::: Performance
The experiment results of validation on Friends are shown in Table TABREF19  The proposed model and baselines are evaluated based on the Precision (P ), Recall (R ), and F1-measure (F1) 
For the traditional baselines, namely BOW and TFIDF, we observe that they achieve surprising high F1 scores around $0 81$, however, the scores for Anger and Sadness are lower  This explains that traditional approaches tend to predict the labels with large sample size, such as Joy and Neutral, but fail to take of scarce samples even when an ensemble random forest classifier is adopted  In order to prevent the unbalanced learning, we choose the weighted loss mechanism for both TextCNN and causal modeling TextCNN (C-TextCNN), these models suffer less than the traditional baselines and achieve a slightly balance performance, where there are around 15% and 7% improvement on Anger and Sadness, respectively  We following adopt the casual utterance modeling to original TextCNN, mapping previous utterance as well as target utterance into model  The causal utterance modeling improve the C-TextCNN over TextCNN for 6%, 2% and 1% on Anger, Joy and overall F1 score  Motivated from these preliminary experiments, the proposed FriendsBERT also adopt the ideas of both weighted loss and causal utterance modeling  As compared to the original BERT, single sentence BERT (FriendsBERT-base-s), the proposed FriendsBERT-base improve 1% for Joy and overall F1, and 2% for Sadness  For the final validation performance, our proposed approach achieves the highest scores, which are $0 85$ and $0 86$ for FriendsBERT-base and FriendsBERT-large, respectively 
Overall, the proposed FriendsBERT successfully captures the sentence-level context-awarded information and outperforms all the baselines, which not only achieves high performance on large sample labels, but also on small sample labels  The similar settings are also adapted to EmotionPush dataset for the final evaluation 
Experiments ::: Evaluation Results
The testing dataset consists of 240 dialogues including $3,296$ and $3,536$ utterances in Friends and EmotionPush respectively  We re-train our FriendsBERT and ChatBERT with top 920 training dialogues and predict the evaluation results using the model performing the best validation results  The results are shown in Table TABREF29 and Table TABREF30  The present method achieves $81 5\%$ and $88 5\%$ micro F1-score on the testing dataset of Friends and EmotionPush, respectively 
Conclusion and Future work
In the present work, we propose FriendsBERT and ChatBERT for the multi-utterance emotion recognition task on EmotionLines dataset  The proposed models are adapted from BERT BIBREF5 with three main improvement during the model training procedure, which are the causal utterance modeling mechanism, specific model pre-training, and adapt weighted loss  The causal utterance modeling takes the advantages of the sentence-level context information during model inference  The specific model pre-training helps to against the bias in different text domain  The weighted loss avoids our model to only predict on large size sample  The effectiveness and generalizability of the proposed methods are demonstrated from the experiments 
In future work, we consider to include the conditional probabilistic constraint $P ({\rm Emo}_{B} | \hat{\rm Emo}_{A})$  Model should predict the emotion based on a certain understanding about context emotions  This might be more reasonable for guiding model than just predicting emotion of ${\rm Sentence}_B$ directly  In addition, due to the limitation of BERT input format, ambiguous number of input sentences is now becoming an important design requirement for our future work  Also, personality embedding development will be another future work of the emotion recognition  The personality embedding will be considered as sentence embedding injected into word embedding, and it seems this additional information can contribute some improvement potentially 

ground_truth: Friends TV sitcom, Facebook messenger chats

prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. What is the improvement in performance for Estonian in the NER task?
Context 3:  There is no tokenizer for Croatian in NLTK library, so we used Slovene tokenizer instead  After tokenization, we deduplicated the datasets for each language separately, using the Onion (ONe Instance ONly) tool for text deduplication  We applied the tool on paragraph level for corpora that did not have sentences shuffled and on sentence level for the rest  We considered 9-grams with duplicate content threshold of 0 9 
For each language we prepared a vocabulary file, containing roughly one million most common tokens, i e  tokens that appear at least $n$ times in the corpus, where $n$ is between 15 and 25, depending on the dataset size  We included the punctuation marks among the tokens  We trained each ELMo model using default values used to train the original English ELMo (large) model 
Evaluation
We evaluated the produced ELMo models for all languages using two evaluation tasks: a word analogy task and named entity recognition (NER) task  Below, we first shortly describe each task, followed by the evaluation results 
Evaluation ::: Word Analogy Task
The word analogy task was popularized by mikolov2013distributed  The goal is to find a term $y$ for a given term $x$ so that the relationship between $x$ and $y$ best resembles the given relationship $a : b$  There are two main groups of categories: 5 semantic and 10 syntactic  To illustrate a semantic relationship, consider for example that the word pair $a : b$ is given as “Finland : Helsinki”  The task is to find the term $y$ corresponding to the relationship “Sweden : $y$”, with the expected answer being $y=$ Stockholm  In syntactic categories, the two words in a pair have a common stem (in some cases even same lemma), with all the pairs in a given category having the same morphological relationship  For example, given the word pair “long : longer”, we see that we have an adjective in its base form and the same adjective in a comparative form  That task is then to find the term $y$ corresponding to the relationship “dark : $y$”, with the expected answer being $y=$ darker, that is a comparative form of the adjective dark 
In the vector space, the analogy task is transformed into vector arithmetic and search for nearest neighbours, i e  we compute the distance between vectors: d(vec(Finland), vec(Helsinki)) and search for word $y$ which would give the closest result in distance d(vec(Sweden), vec($y$))  In the analogy dataset the analogies are already pre-specified, so we are measuring how close are the given pairs  In the evaluation below, we use analogy datasets for all tested languages based on the English dataset by BIBREF14   Due to English-centered bias of this dataset, we used a modified dataset which was first written in Slovene language and then translated into other languages BIBREF15 
As each instance of analogy contains only four words, without any context, the contextual models (such as ELMo) do not have enough context to generate sensible embeddings  We therefore used some additional text to form simple sentences using the four analogy words, while taking care that their noun case stays the same  For example, for the words "Rome", "Italy", "Paris" and "France" (forming the analogy Rome is to Italy as Paris is to $x$, where the correct answer is $x=$France), we formed the sentence "If the word Rome corresponds to the word Italy, then the word Paris corresponds to the word France"  We generated embeddings for those four words in the constructed sentence, substituted the last word with each word in our vocabulary and generated the embeddings again  As typical for non-contextual analogy task, we measure the cosine distance ($d$) between the last word ($w_4$) and the combination of the first three words ($w_2-w_1+w_3$)  We use the CSLS metric BIBREF16 to find the closest candidate word ($w_4$)  If we find the correct word among the five closest words, we consider that entry as successfully identified  The proportion of correctly identified words forms a statistic called accuracy@5, which we report as the result 
We first compare existing Latvian ELMo embeddings from ELMoForManyLangs project with our Latvian embeddings, followed by the detailed analysis of our ELMo embeddings  We trained Latvian ELMo using only CoNLL 2017 corpora  Since this is the only language, where we trained the embedding model on exactly the same corpora as ELMoForManyLangs models, we chose it for comparison between our ELMo model with ELMoForManyLangs  In other languages, additional or other corpora were used, so a direct comparison would also reflect the quality of the corpora used for training  In Latvian, however, only the size of the training dataset is different  ELMoForManyLangs uses only 20 million tokens and we use the whole corpus of 270 million tokens 
The Latvian ELMo model from ELMoForManyLangs project performs significantly worse than EMBEDDIA ELMo Latvian model on all categories of word analogy task (Figure FIGREF16)  We also include the comparison with our Estonian ELMo embeddings in the same figure  This comparison shows that while differences between our Latvian and Estonian embeddings can be significant for certain categories, the accuracy score of ELMoForManyLangs is always worse than either of our models  The comparison of Estonian and Latvian models leads us to believe that a few hundred million tokens is a sufficiently large corpus to train ELMo models (at least for word analogy task), but 20-million token corpora used in ELMoForManyLangs are too small 
The results for all languages and all ELMo layers, averaged over semantic and syntactic categories, are shown in Table TABREF17  The embeddings after the first LSTM layer perform best in semantic categories  In syntactic categories, the non-contextual CNN layer performs the best  Syntactic categories are less context dependent and much more morphology and syntax based, so it is not surprising that the non-contextual layer performs well  The second LSTM layer embeddings perform the worst in syntactic categories, though still outperforming CNN layer embeddings in semantic categories  Latvian ELMo performs worse compared to other languages we trained, especially in semantic categories, presumably due to smaller training data size  Surprisingly, the original English ELMo performs very poorly in syntactic categories and only outperforms Latvian in semantic categories  The low score can be partially explained by English model scoring $0 00$ in one syntactic category “opposite adjective”, which we have not been able to explain 
Evaluation ::: Named Entity Recognition
For evaluation of ELMo models on a relevant downstream task, we used named entity recognition (NER) task  NER is an information extraction task that seeks to locate and classify named entity mentions in unstructured text into pre-defined categories such as the person names, organizations, locations, medical codes, time expressions, quantities, monetary values, percentages, etc  To allow comparison of results between languages, we used an adapted version of this task, which uses a reduced set of labels, available in NER datasets for all processed languages  The labels in the used NER datasets are simplified to a common label set of three labels (person - PER, location - LOC, organization - ORG)  Each word in the NER dataset is labeled with one of the three mentioned labels or a label 'O' (other, i e  not a named entity) if it does not fit any of the other three labels  The number of words having each label is shown in Table TABREF19 
To measure the performance of ELMo embeddings on the NER task we proceeded as follows  We embedded the text in the datasets sentence by sentence, producing three vectors (one from each ELMo layer) for each token in a sentence  We calculated the average of the three vectors and used it as the input of our recognition model  The input layer was followed by a single LSTM layer with 128 LSTM cells and a dropout layer, randomly dropping 10% of the neurons on both the output and the recurrent branch  The final layer of our model was a time distributed softmax layer with 4 neurons 
We used ADAM optimiser BIBREF17 with the learning rate 0 01 and $10^{-5}$ learning rate decay  We used categorical cross-entropy as a loss function and trained the model for 3 epochs  We present the results using the Macro $F_1$ score, that is the average of $F_1$-scores for each of the three NE classes (the class Other is excluded) 
Since the differences between the tested languages depend more on the properties of the NER datasets than on the quality of embeddings, we can not directly compare ELMo models  For this reason, we take the non-contextual fastText embeddings as a baseline and predict named entities using them  The architecture of the model using fastText embeddings is the same as the one using ELMo embeddings, except that the input uses 300 dimensional fastText embedding vectors, and the model was trained for 5 epochs (instead of 3 as for ELMo)  In both cases (ELMo and fastText) we trained and evaluated the model five times, because there is some random component involved in initialization of the neural network model  By training and evaluating multiple times, we minimise this random component 
The results are presented in Table TABREF21  We included the evaluation of the original ELMo English model in the same table  NER models have little difficulty distinguishing between types of named entities, but recognizing whether a word is a named entity or not is more difficult  For languages with the smallest NER datasets, Croatian and Lithuanian, ELMo embeddings show the largest improvement over fastText embeddings  However, we can observe significant improvements with ELMo also on English and Finnish, which are among the largest datasets (English being by far the largest)  Only on Slovenian dataset did ELMo perform slightly worse than fastText, on all other EMBEDDIA languages, the ELMo embeddings improve the results 
Conclusion
We prepared precomputed ELMo contextual embeddings for seven languages: Croatian, Estonian, Finnish, Latvian, Lithuanian, Slovenian, and Swedish  We present the necessary background on embeddings and contextual embeddings, the details of training the embedding models, and their evaluation  We show that the size of used training sets importantly affects the quality of produced embeddings, and therefore the existing publicly available ELMo embeddings for the processed languages are inadequate  We trained new ELMo embeddings on larger training sets and analysed their properties on the analogy task and on the NER task  The results show that the newly produced contextual embeddings produce substantially better results compared to the non-contextual fastText baseline  In future work, we plan to use the produced contextual embeddings on the problems of news media industry  The pretrained ELMo models will be deposited to the CLARIN repository by the time of the final version of this paper 
Acknowledgments
The work was partially supported by the Slovenian Research Agency (ARRS) core research programme P6-0411  This paper is supported by European Union's Horizon 2020 research and innovation programme under grant agreement No 825153, project EMBEDDIA (Cross-Lingual Embeddings for Less-Represented Languages in European News Media)  The results of this publication reflects only the authors' view and the EU Commission is not responsible for any use that may be made of the information it contains 

ground_truth: 5 percent points.

0.05 F1