prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. How large is their data set?
Context 1: Introduction
The task of document quality assessment is to automatically assess a document according to some predefined inventory of quality labels  This can take many forms, including essay scoring (quality = language quality, coherence, and relevance to a topic), job application filtering (quality = suitability for role + visual/presentational quality of the application), or answer selection in community question answering (quality = actionability + relevance of the answer to the question)  In the case of this paper, we focus on document quality assessment in two contexts: Wikipedia document quality classification, and whether a paper submitted to a conference was accepted or not 
Automatic quality assessment has obvious benefits in terms of time savings and tractability in contexts where the volume of documents is large  In the case of dynamic documents (possibly with multiple authors), such as in the case of Wikipedia, it is particularly pertinent, as any edit potentially has implications for the quality label of that document (and around 10 English Wikipedia documents are edited per second)  Furthermore, when the quality assessment task is decentralized (as in the case of Wikipedia and academic paper assessment), quality criteria are often applied inconsistently by different people, where an automatic document quality assessment system could potentially reduce inconsistencies and enable immediate author feedback 
Current studies on document quality assessment mainly focus on textual features  For example, BIBREF0 examine features such as the article length and the number of headings to predict the quality class of a Wikipedia article  In contrast to these studies, in this paper, we propose to combine text features with visual features, based on a visual rendering of the document  Figure 1 illustrates our intuition, relative to Wikipedia articles  Without being able to read the text, we can tell that the article in Figure 1 has higher quality than Figure 1 , as it has a detailed infobox, extensive references, and a variety of images  Based on this intuition, we aim to answer the following question: can we achieve better accuracy on document quality assessment by complementing textual features with visual features?
Our visual model is based on fine-tuning an Inception V3 model BIBREF1 over visual renderings of documents, while our textual model is based on a hierarchical biLSTM  We further combine the two into a joint model  We perform experiments on two datasets: a Wikipedia dataset novel to this paper, and an arXiv dataset provided by BIBREF2 split into three sub-parts based on subject category  Experimental results on the visual renderings of documents show that implicit quality indicators, such as images and visual layout, can be captured by an image classifier, at a level comparable to a text classifier  When we combine the two models, we achieve state-of-the-art results over 3/4 of our datasets 
This paper makes the following contributions:
All code and data associated with this research will be released on publication 
Related Work
A variety of approaches have been proposed for document quality assessment across different domains: Wikipedia article quality assessment, academic paper rating, content quality assessment in community question answering (cQA), and essay scoring  Among these approaches, some use hand-crafted features while others use neural networks to learn features from documents  For each domain, we first briefly describe feature-based approaches and then review neural network-based approaches  Wikipedia article quality assessment: Quality assessment of Wikipedia articles is a task that assigns a quality class label to a given Wikipedia article, mirroring the quality assessment process that the Wikipedia community carries out manually  Many approaches have been proposed that use features from the article itself, meta-data features (e g , the editors, and Wikipedia article revision history), or a combination of the two  Article-internal features capture information such as whether an article is properly organized, with supporting evidence, and with appropriate terminology  For example, BIBREF3 use writing styles represented by binarized character trigram features to identify featured articles  BIBREF4 and BIBREF0 explore the number of headings, images, and references in the article  BIBREF5 use nine readability scores, such as the percentage of difficult words in the document, to measure the quality of the article  Meta-data features, which are indirect indicators of article quality, are usually extracted from revision history, and the interaction between editors and articles  For example, one heuristic that has been proposed is that higher-quality articles have more edits BIBREF6 , BIBREF7   BIBREF8 use the percentage of registered editors and the total number of editors of an article  Article–editor dependencies have also been explored  For example, BIBREF9 use the authority of editors to measure the quality of Wikipedia articles, where the authority of editors is determined by the articles they edit  Deep learning approaches to predicting Wikipedia article quality have also been proposed  For example, BIBREF10 use a version of doc2vec BIBREF11 to represent articles, and feed the document embeddings into a four hidden layer neural network  BIBREF12 first obtain sentence representations by averaging words within a sentence, and then apply a biLSTM BIBREF13 to learn a document-level representation, which is combined with hand-crafted features as side information  BIBREF14 exploit two stacked biLSTMs to learn document representations 
Academic paper rating: Academic paper rating is a relatively new task in NLP/AI, with the basic formulation being to automatically predict whether to accept or reject a paper  BIBREF2 explore hand-crafted features, such as the length of the title, whether specific words (such as outperform, state-of-the-art, and novel) appear in the abstract, and an embedded representation of the abstract as input to different downstream learners, such as logistic regression, decision tree, and random forest  BIBREF15 exploit a modularized hierarchical convolutional neural network (CNN), where each paper section is treated as a module  For each paper section, they train an attention-based CNN, and an attentive pooling layer is applied to the concatenated representation of each section, which is then fed into a softmax layer 
Content quality assessment in cQA: Automatic quality assessment in cQA is the task of determining whether an answer is of high quality, selected as the best answer, or ranked higher than other answers  To measure answer content quality in cQA, researchers have exploited various features from different sources, such as the answer content itself, the answerer's profile, interactions among users, and usage of the content  The most common feature used is the answer length BIBREF16 , BIBREF17 , with other features including: syntactic and semantic features, such as readability scores  BIBREF18 ; similarity between the question and the answer at lexical, syntactic, and semantic levels BIBREF18 , BIBREF19 , BIBREF20 ; or user data (e g , a user's status points or the number of answers written by the user)  There have also been approaches using neural networks  For example, BIBREF21 combine CNN-learned representations with hand-crafted features to predict answer quality  BIBREF22 use a 2-dimensional CNN to learn the semantic relevance of an answer to the question, and apply an LSTM to the answer sequence to model thread context  BIBREF23 and BIBREF24 model the problem similarly to machine translation quality estimation, treating answers as competing translation hypotheses and the question as the reference translation, and apply neural machine translation to the problem  Essay scoring: Automated essay scoring is the task of assigning a score to an essay, usually in the context of assessing the language ability of a language learner  The quality of an essay is affected by the following four primary dimensions: topic relevance, organization and coherence, word usage and sentence complexity, and grammar and mechanics  To measure whether an essay is relevant to its “prompt” (the description of the essay topic), lexical and semantic overlap is commonly used BIBREF25 , BIBREF26   BIBREF27 explore word features, such as the number of verb formation errors, average word frequency, and average word length, to measure word usage and lexical complexity  BIBREF28 use sentence structure features to measure sentence variety  The effects of grammatical and mechanic errors on the quality of an essay are measured via word and part-of-speech $n$ -gram features and “mechanics” features BIBREF29 (e g , spelling, capitalization, and punctuation), respectively  BIBREF30 , BIBREF31 , and BIBREF32 use an LSTM to obtain an essay representation, which is used as the basis for classification  Similarly, BIBREF33 utilize a CNN to obtain sentence representation and an LSTM to obtain essay representation, with an attention layer at both the sentence and essay levels 
The Proposed Joint Model
We treat document quality assessment as a classification problem, i e , given a document, we predict its quality class (e g , whether an academic paper should be accepted or rejected)  The proposed model is a joint model that integrates visual features learned through Inception V3 with textual features learned through a biLSTM  In this section, we present the details of the visual and textual embeddings, and finally describe how we combine the two  We return to discuss hyper-parameter settings and the experimental configuration in the Experiments section 
Visual Embedding Learning
A wide range of models have been proposed to tackle the image classification task, such as VGG BIBREF34 , ResNet BIBREF35 , Inception V3 BIBREF1 , and Xception BIBREF36   However, to the best of our knowledge, there is no existing work that has proposed to use visual renderings of documents to assess document quality  In this paper, we use Inception V3 pretrained on ImageNet (“Inception” hereafter) to obtain visual embeddings of documents, noting that any image classifier could be applied to our task  The input to Inception is a visual rendering (screenshot) of a document, and the output is a visual embedding, which we will later integrate with our textual embedding 
Based on the observation that it is difficult to decide what types of convolution to apply to each layer (such as 3 $\times $ 3 or 5 $\times $ 5), the basic Inception model applies multiple convolution filters in parallel and concatenates the resulting features, which are fed into the next layer  This has the benefit of capturing both local features through smaller convolutions and abstracted features through larger convolutions  Inception is a hybrid of multiple Inception models of different architectures  To reduce computational cost, Inception also modifies the basic model by applying a 1 $\times $ 1 convolution to the input and factorizing larger convolutions into smaller ones 
Textual Embedding Learning
We adopt a bi-directional LSTM model to generate textual embeddings for document quality assessment, following the method of BIBREF12 (“biLSTM” hereafter)  The input to biLSTM is a textual document, and the output is a textual embedding, which will later integrate with the visual embedding 
For biLSTM, each word is represented as a word embedding BIBREF37 , and an average-pooling layer is applied to the word embeddings to obtain the sentence embedding, which is fed into a bi-directional LSTM to generate the document embedding from the sentence embeddings  Then a max-pooling layer is applied to select the most salient features from the component sentences 
The Joint Model
The proposed joint model (“Joint” hereafter) combines the visual and textual embeddings (output of Inception and biLSTM) via a simple feed-forward layer and softmax over the document label set, as shown in Figure 2   We optimize our model based on cross-entropy loss 
Experiments
In this section, we first describe the two datasets used in our experiments: (1) Wikipedia, and (2) arXiv  Then, we report the experimental details and results 
Datasets
The Wikipedia dataset consists of articles from English Wikipedia, with quality class labels assigned by the Wikipedia community  Wikipedia articles are labelled with one of six quality classes, in descending order of quality: Featured Article (“FA”), Good Article (“GA”), B-class Article (“B”), C-class Article (“C”), Start Article (“Start”), and Stub Article (“Stub”)  A description of the criteria associated with the different classes can be found in the Wikipedia grading scheme page  The quality class of a Wikipedia article is assigned by Wikipedia reviewers or any registered user, who can discuss through the article's talk page to reach consensus  We constructed the dataset by first crawling all articles from each quality class repository, e g , we get FA articles by crawling pages from the FA repository: https://en wikipedia org/wiki/Category:Featured_articles  This resulted in around 5K FA, 28K GA, 212K B, 533K C, 2 6M Start, and 3 2M Stub articles 
We randomly sampled 5,000 articles from each quality class and removed all redirect pages, resulting in a dataset of 29,794 articles  As the wikitext contained in each document contains markup relating to the document category such as {Featured Article} or {geo-stub}, which reveals the label, we remove such information  We additionally randomly partitioned this dataset into training, development, and test splits based on a ratio of 8:1:1  Details of the dataset are summarized in Table 1  
We generate a visual representation of each document via a 1,000 $\times $ 2,000-pixel screenshot of the article via a PhantomJS script over the rendered version of the article, ensuring that the screenshot and wikitext versions of the article are the same version  Any direct indicators of document quality (such as the FA indicator, which is a bronze star icon in the top right corner of the webpage) are removed from the screenshot 
The arXiv dataset BIBREF2 consists of three subsets of academic articles under the arXiv repository of Computer Science (cs), from the three subject areas of: Artificial Intelligence (cs ai), Computation and Language (cs cl), and Machine Learning (cs lg)  In line with the original dataset formulation BIBREF2 , a paper is considered to have been accepted (i e  is positively labeled) if it matches a paper in the DBLP database or is otherwise accepted by any of the following conferences: ACL, EMNLP, NAACL, EACL, TACL, NIPS, ICML, ICLR, or AAAI  Failing this, it is considered to be rejected (noting that some of the papers may not have been submitted to one of these conferences)  The median numbers of pages for papers in cs ai, cs cl, and cs lg are 11, 10, and 12, respectively  To make sure each page in the PDF file has the same size in the screenshot, we crop the PDF file of a paper to the first 12; we pad the PDF file with blank pages if a PDF file has less than 12 pages, using the PyPDF2 Python package  We then use ImageMagick to convert the 12-page PDF file to a single 1,000 $\times $ 2,000 pixel screenshot  Table 2 details this dataset, where the “Accepted” column denotes the percentage of positive instances (accepted papers) in each subset 
Experimental Setting
As discussed above, our model has two main components — biLSTM and Inception— which generate textual and visual representations, respectively  For the biLSTM component, the documents are preprocessed as described in BIBREF12 , where an article is divided into sentences and tokenized using NLTK BIBREF38   Words appearing more than 20 times are retained when building the vocabulary  All other words are replaced by the special UNK token  We use the pre-trained GloVe BIBREF39 50-dimensional word embeddings to represent words  For words not in GloVe, word embeddings are randomly initialized based on sampling from a uniform distribution $U(-1, 1)$   All word embeddings are updated in the training process  We set the LSTM hidden layer size to 256  The concatenation of the forward and backward LSTMs thus gives us 512 dimensions for the document embedding  A dropout layer is applied at the sentence and document level, respectively, with a probability of 0 5 
For Inception, we adopt data augmentation techniques in the training with a “nearest” filling mode, a zoom range of 0 1, a width shift range of 0 1, and a height shift range of 0 1  As the original screenshots have the size of 1,000 $\times 2$ ,000 pixels, they are resized to 500 $\times $ 500 to feed into Inception, where the input shape is (500, 500, 3)  A dropout layer is applied with a probability of 0 5  Then, a GlobalAveragePooling2D layer is applied, which produces a 2,048 dimensional representation 
For the Joint model, we get a representation of 2,560 dimensions by concatenating the 512 dimensional representation from the biLSTM with the 2,048 dimensional representation from Inception  The dropout layer is applied to the two components with a probability of 0 5  For biLSTM, we use a mini-batch size of 128 and a learning rate of 0 001  For both Inception and joint model, we use a mini-batch size of 16 and a learning rate of 0 0001  All hyper-parameters were set empirically over the development data, and the models were optimized using the Adam optimizer BIBREF40  
In the training phase, the weights in Inception are initialized by parameters pretrained on ImageNet, and the weights in biLSTM are randomly initialized (except for the word embeddings)  We train each model for 50 epochs  However, to prevent overfitting, we adopt early stopping, where we stop training the model if the performance on the development set does not improve for 20 epochs  For evaluation, we use (micro-)accuracy, following previous studies BIBREF5 , BIBREF2  
Baseline Approaches
We compare our models against the following five baselines:
Majority: the model labels all test samples with the majority class of the training data 
Benchmark: a benchmark method from the literature  In the case of Wikipedia, this is BIBREF5 , who use structural features and readability scores as features to build a random forest classifier; for arXiv, this is BIBREF2 , who use hand-crafted features, such as the number of references and TF-IDF weighted bag-of-words in abstract, to build a classifier based on the best of logistic regression, multi-layer perception, and AdaBoost 
Doc2Vec: doc2vec BIBREF11 to learn document embeddings with a dimension of 500, and a 4-layer feed-forward classification model on top of this, with 2000, 1000, 500, and 200 dimensions, respectively 
biLSTM: first derive a sentence representation by averaging across words in a sentence, then feed the sentence representation into a biLSTM and a maxpooling layer over output sequence to learn a document level representation with a dimension of 512, which is used to predict document quality 
Inception $_{\text{fixed}}$ : the frozen Inception model, where only parameters in the last layer are fine-tuned during training 
The hyper-parameters of Benchmark, Doc2Vec, and biLSTM are based on the corresponding papers except that: (1) we fine-tune the feed forward layer of Doc2Vec on the development set and train the model 300 epochs on Wikipedia and 50 epochs on arXiv; (2) we do not use hand-crafted features for biLSTM as we want the baselines to be comparable to our models, and the main focus of this paper is not to explore the effects of hand-crafted features (e g , see BIBREF12 ) 
Experimental Results
Table 3 shows the performance of the different models over our two datasets, in the form of the average accuracy on the test set (along with the standard deviation) over 10 runs, with different random initializations 
On Wikipedia, we observe that the performance of biLSTM, Inception, and Joint is much better than that of all four baselines  Inception achieves 2 9% higher accuracy than biLSTM  The performance of Joint achieves an accuracy of 59 4%, which is 5 3% higher than using textual features alone (biLSTM) and 2 4% higher than using visual features alone (Inception)  Based on a one-tailed Wilcoxon signed-rank test, the performance of Joint is statistically significant ( $p<0 05$ )  This shows that the textual and visual features complement each other, achieving state-of-the-art results in combination 
For arXiv, baseline methods Majority, Benchmark, and Inception $_{\text{fixed}}$ outperform biLSTM over cs ai, in large part because of the class imbalance in this dataset (90% of papers are rejected)  Surprisingly, Inception $_{\text{fixed}}$ is better than Majority and Benchmark over the arXiv cs lg subset, which verifies the usefulness of visual features, even when only the last layer is fine-tuned  Table 3 also shows that Inception and biLSTM achieve similar performance on arXiv, showing that textual and visual representations are equally discriminative: Inception and biLSTM are indistinguishable over cs cl; biLSTM achieves 1 8% higher accuracy over cs lg, while Inception achieves 1 3% higher accuracy over cs ai  Once again, the Joint model achieves the highest accuracy on cs ai and cs cl by combining textual and visual representations (at a level of statistical significance for cs ai)  This, again, confirms that textual and visual features complement each other, and together they achieve state-of-the-art results  On arXiv cs lg, Joint achieves a 0 6% higher accuracy than Inception by combining visual features and textual features, but biLSTM achieves the highest accuracy  One characteristic of cs lg documents is that they tend to contain more equations than the other two arXiv datasets, and preliminary analysis suggests that the biLSTM is picking up on a correlation between the volume/style of mathematical presentation and the quality of the document 
Analysis
In this section, we first analyze the performance of Inception and Joint  We also analyze the performance of different models on different quality classes  The high-level representations learned by different models are also visualized and discussed  As the Wikipedia test set is larger and more balanced than that of arXiv, our analysis will focus on Wikipedia 
Inception
To better understand the performance of Inception, we generated the gradient-based class activation map BIBREF41 , by maximizing the outputs of each class in the penultimate layer, as shown in Figure 3   From Figure 3 and Figure 3 , we can see that Inception identifies the two most important regions (one at the top corresponding to the table of contents, and the other at the bottom, capturing both document length and references) that contribute to the FA class prediction, and a region in the upper half of the image that contributes to the GA class prediction (capturing the length of the article body)  From Figure 3 and Figure 3 , we can see that the most important regions in terms of B and C class prediction capture images (down the left and right of the page, in the case of B and C), and document length/references  From Figure 3 and Figure 3 , we can see that Inception finds that images in the top right corner are the strongest predictor of Start class prediction, and (the lack of) images/the link bar down the left side of the document are the most important for Stub class prediction 
Joint
Table 4 shows the confusion matrix of Joint on Wikipedia  We can see that more than 50% of documents for each quality class are correctly classified, except for the C class where more documents are misclassified into B  Analysis shows that when misclassified, documents are usually misclassified into adjacent quality classes, which can be explained by the Wikipedia grading scheme, where the criteria for adjacent quality classes are more similar 
We also provide a breakdown of precision (“ $\mathcal {P}$ ”), recall (“ $\mathcal {R}$ ”), and F1 score (“ $\mathcal {F}_{\beta =1}$ ”) for biLSTM, Inception, and Joint across the quality classes in Table 5   We can see that Joint achieves the highest accuracy in 11 out of 18 cases  It is also worth noting that all models achieve higher scores for FA, GA, and Stub articles than B, C and Start articles  This can be explained in part by the fact that FA and GA articles must pass an official review based on structured criteria, and in part by the fact that Stub articles are usually very short, which is discriminative for Inception, and Joint  All models perform worst on the B and C quality classes  It is difficult to differentiate B articles from C articles even for Wikipedia contributors  As evidence of this, when we crawled a new dataset including talk pages with quality class votes from Wikipedia contributors, we found that among articles with three or more quality labels, over 20% percent of B and C articles have inconsistent votes from Wikipedia contributors, whereas for FA and GA articles the number is only 0 7% 
We further visualize the learned document representations of biLSTM, Inception, and Joint in the form of a t-SNE plot BIBREF42 in Figure 4   The degree of separation between Start and Stub achieved by Inception is much greater than for biLSTM, with the separation between Start and Stub achieved by Joint being the clearest among the three models  Inception and Joint are better than biLSTM at separating Start and C  Joint achieves slightly better performance than Inception in separating GA and FA  We can also see that it is difficult for all models to separate B and C, which is consistent with the findings of Tables 4 and 5  
Conclusions
We proposed to use visual renderings of documents to capture implicit document quality indicators, such as font choices, images, and visual layout, which are not captured in textual content  We applied neural network models to capture visual features given visual renderings of documents  Experimental results show that we achieve a 2 9% higher accuracy than state-of-the-art approaches based on textual features over Wikipedia, and performance competitive with or surpassing state-of-the-art approaches over arXiv  We further proposed a joint model, combining textual and visual representations, to predict the quality of a document  Experimental results show that our joint model outperforms the visual-only model in all cases, and the text-only model on Wikipedia and two subsets of arXiv  These results underline the feasibility of assessing document quality via visual features, and the complementarity of visual and textual document representations for quality assessment 
Context 2: Introduction
Modelling the relationship between sequences is extremely significant in most retrieval or classification problems involving two sequences  Traditionally, in Siamese networks, Hadamard product or concatenation have been used to fuse two vector representations of two input sequences to form a final representation for tasks like semantic similarity, passage retrieval  This representation, subsequently, has been used to compute similarity scores which has been used in a variety of training objectives like margin loss for ranking or cross-entropy error in classification 
We have also witnessed word or phrase level similarity to create alignment matrices between two sequences BIBREF0 , BIBREF1   These alignment matrices has proved to be very useful to model the relationship between two word representations as well fuse the relevant information of one sequence into another  Empirical evidences have shown this alignment procedures have significantly performed better then simple concatenation or element-wise multiplication, especially for long sentences or paragraphs 
Attention works on creating neural alignment matrix using learnt weights without pre-computing alignment matrix and using them as features  The main objective of any attentive or alignment process is to look for matching words or phrases between two sequences and assign a high weight to the most similar pairs and vice-versa  The notion of matching or similarity maybe not semantic similarity but based on whatever task we have at hand  For example, for a task that requires capturing semantic similarity between two sequences like "how rich is tom cruise" and "how much wealth does tom cruise have", an attentive model shall discover the high similarity between "rich" and "wealthy" and assign a high weight value to the pair  Likewise, for a different task like question answering, a word "long" in a question like "how long does it take to recover from a mild fever" might be aligned with the phrase "a week" from the candidate answer "it takes almost a week to recover fully from a fever"  Thus, attention significantly aids in better understanding the relevance of a similar user query in a similar measurement task or a candidate answer in a question answering task  The final prediction score is dependent on how well the relationship between two sequences are modeled and established 
The general process of matching one sequence with another through attention includes computing the alignment matrix containing weight value between every pair of word representations belonging to both of the sequences  Subsequently, softmax function is applied on all the elements of one of the two dimensions of the matrix to represent the matching probabilities of all the word of a sequence with respect to one particular word in the other sequence 
Since attention always looks for matching word representations, it operates under the assumption that there is always a match to be found inside the sequences  We provide a theoretical limitation to it and propose another technique called conflict that looks for contrasting relationship between words in two sequences  We empirically verify that our proposed conflict mechanism combined with attention can outperform the performance of attention working solely 
Related Work
Bahdanau et al  BIBREF2 introduced attention first in neural machine translation  It used a feed-forward network over addition of encoder and decoder states to compute alignment score  Our work is very similar to this except we use element wise difference instead of addition to build our conflict function  BIBREF3 came up with a scaled dot-product attention in their Transformer model which is fast and memory-efficient  Due to the scaling factor, it didn't have the issue of gradients zeroing out  On the other hand, BIBREF4 has experimented with global and local attention based on the how many hidden states the attention function takes into account  Their experiments have revolved around three attention functions - dot, concat and general  Their findings include that dot product works best for global attention  Our work also belongs to the global attention family as we consider all the hidden states of the sequence 
Attention has been widely used in pair-classification problems like natural language inference  Wang et al  BIBREF5 introduced BIMPM which matched one sequence with another in four different fashion but one single matching function which they used as cosine  Liu et al  BIBREF6 proposed SAN for language inference which also used dot-product attention between the sequences 
Summarizing, attention has helped in achieving state-of-the-art results in NLI and QA  Prior work in attention has been mostly in similarity based approaches while our work focuses on non-matching sequences 
How attention works
Let us consider that we have two sequences INLINEFORM0 and INLINEFORM1 each with M and N words respectively  The objective of attention is two-fold: compute alignment scores (or weight) between every word representation pairs from INLINEFORM2 and INLINEFORM3 and fuse the matching information of INLINEFORM4 with INLINEFORM5 thus computing a new representation of INLINEFORM6 conditioned on INLINEFORM7  
The word representations that attention operates on can be either embeddings like GloVe or hidden states from any recurrent neural network  We denote these representations as u = INLINEFORM0 and v = INLINEFORM1   We provide a mathematical working of how a general attention mechanism works between two sequences, followed by a explanation in words: DISPLAYFORM0
Explanation: Both are sequences are non-linearly projected into two different spaces (eqn 1) and each word representation in INLINEFORM0 is matched with that in INLINEFORM1 by computing a dot-product (eqn 2)  INLINEFORM2 is a M X N matrix that stores the alignment scores between word INLINEFORM3 and INLINEFORM4 (eqn 2)  Since, the scores are not normalized, a softmax function is applied on each row to convert them to probabilities (eqn  3)  Thus, each row contains relative importance of words in INLINEFORM5 to a particular word INLINEFORM6   Weighted sum of INLINEFORM7 is taken (eqn  4) and fused with the word representation INLINEFORM8 using concatenation (eqn 5) 
Limits of using only Attention
Attention operates by using dot product or sometimes addition followed by linear projection to a scalar which models the similarity between two vectors  Subsequently, softmax is applied which gives high probabilities to most matching word representations  This assumes that there is some highly matched word pairs already existing and high scores will be assigned to them  Given a vector INLINEFORM0 =( INLINEFORM1 ,   , INLINEFORM2 ) on which softmax function is applied, each INLINEFORM3 INLINEFORM4 (0, 1)  It is observable that the average value of INLINEFORM5 is always INLINEFORM6   In other words, it is impossible to produce a vector having all INLINEFORM7 < INLINEFORM8 when two sequences have no matching at all 
In cases, where one or more word pairs from two different sequences are highly dissimilar, it is impossible to assign a very low probability to it without increasing the probability of some other pair somewhere else since INLINEFORM0 = 1 
For example, when we consider two sequences "height of tom cruise" and "age of sun", while computing the attention weights between the word "height" and all the words in the second sequence it can be observed that their no matching word in the latter  In this case, a standard dot-product based attention with softmax won't be able to produce weights which is below 0 33 (=1/3) for all the words in the second sequence with respect to the word "height" in the first sequence 
Conflict model
We propose a different mechanism that does the opposite of what attention does that is computing how much two sequences repel each other  This works very similar to how attention works but inversely 
We demonstrate a general model but we also realize that there can be other variants of it which may be worked out to perform better  Our approach consists of using element wise difference between two vectors followed by a linear transformation to produce a scalar weight  The remaining of the process acts similar to how attention works  Mathematically, we can express it as: DISPLAYFORM0
where INLINEFORM0 INLINEFORM1 INLINEFORM2 is a parameter that we introduce to provide a weight for the pair
Context 3: Introduction
Named Entity Recognition (NER) is a foremost NLP task to label each atomic elements of a sentence into specific categories like "PERSON", "LOCATION", "ORGANIZATION" and othersBIBREF0  There has been an extensive NER research on English, German, Dutch and Spanish language BIBREF1, BIBREF2, BIBREF3, BIBREF4, BIBREF5, and notable research on low resource South Asian languages like HindiBIBREF6, IndonesianBIBREF7 and other Indian languages (Kannada, Malayalam, Tamil and Telugu)BIBREF8  However, there has been no study on developing neural NER for Nepali language  In this paper, we propose a neural based Nepali NER using latest state-of-the-art architecture based on grapheme-level which doesn't require any hand-crafted features and no data pre-processing 
Recent neural architecture like BIBREF1 is used to relax the need to hand-craft the features and need to use part-of-speech tag to determine the category of the entity  However, this architecture have been studied for languages like English, and German and not been applied to languages like Nepali which is a low resource language i e limited data set to train the model  Traditional methods like Hidden Markov Model (HMM) with rule based approachesBIBREF9,BIBREF10, and Support Vector Machine (SVM) with manual feature-engineeringBIBREF11 have been applied but they perform poor compared to neural  However, there has been no research in Nepali NER using neural network  Therefore, we created the named entity annotated dataset partly with the help of Dataturk to train a neural model  The texts used for this dataset are collected from various daily news sources from Nepal around the year 2015-2016 
Following are our contributions:
We present a novel Named Entity Recognizer (NER) for Nepali language  To best of our knowledge we are the first to propose neural based Nepali NER 
As there are not good quality dataset to train NER we release a dataset to support future research
We perform empirical evaluation of our model with state-of-the-art models with relative improvement of upto 10%
In this paper, we present works similar to ours in Section SECREF2  We describe our approach and dataset statistics in Section SECREF3 and SECREF4, followed by our experiments, evaluation and discussion in Section SECREF5, SECREF6, and SECREF7  We conclude with our observations in Section SECREF8 
To facilitate further research our code and dataset will be made available at github com/link-yet-to-be-updated
Related Work
There has been a handful of research on Nepali NER task based on approaches like Support Vector Machine and gazetteer listBIBREF11 and Hidden Markov Model and gazetteer listBIBREF9,BIBREF10 
BIBREF11 uses SVM along with features like first word, word length, digit features and gazetteer (person, organization, location, middle name, verb, designation and others)  It uses one vs rest classification model to classify each word into different entity classes  However, it does not the take context word into account while training the model  Similarly, BIBREF9 and BIBREF10 uses Hidden Markov Model with n-gram technique for extracting POS-tags  POS-tags with common noun, proper noun or combination of both are combined together, then uses gazetteer list as look-up table to identify the named entities 
Researchers have shown that the neural networks like CNNBIBREF12, RNNBIBREF13, LSTMBIBREF14, GRUBIBREF15 can capture the semantic knowledge of language better with the help of pre-trained embbeddings like word2vecBIBREF16, gloveBIBREF17 or fasttextBIBREF18 
Similar approaches has been applied to many South Asian languages like HindiBIBREF6, IndonesianBIBREF7, BengaliBIBREF19 and In this paper, we present the neural network architecture for NER task in Nepali language, which doesn't require any manual feature engineering nor any data pre-processing during training  First we are comparing BiLSTMBIBREF14, BiLSTM+CNNBIBREF20, BiLSTM+CRFBIBREF1, BiLSTM+CNN+CRFBIBREF2 models with CNN modelBIBREF0 and Stanford CRF modelBIBREF21  Secondly, we show the comparison between models trained on general word embeddings, word embedding + character-level embedding, word embedding + part-of-speech(POS) one-hot encoding and word embedding + grapheme clustered or sub-word embeddingBIBREF22  The experiments were performed on the dataset that we created and on the dataset received from ILPRL lab  Our extensive study shows that augmenting word embedding with character or grapheme-level representation and POS one-hot encoding vector yields better results compared to using general word embedding alone 
Approach
In this section, we describe our approach in building our model  This model is partly inspired from multiple models BIBREF20,BIBREF1, andBIBREF2
Approach ::: Bidirectional LSTM
We used Bi-directional LSTM to capture the word representation in forward as well as reverse direction of a sentence  Generally, LSTMs take inputs from left (past) of the sentence and computes the hidden state  However, it is proven beneficialBIBREF23 to use bi-directional LSTM, where, hidden states are computed based from right (future) of sentence and both of these hidden states are concatenated to produce the final output as $h_t$=[$\overrightarrow{h_t}$;$\overleftarrow{h_t}$], where $\overrightarrow{h_t}$, $\overleftarrow{h_t}$ = hidden state computed in forward and backward direction respectively 
Approach ::: Features ::: Word embeddings
We have used Word2Vec BIBREF16, GloVe BIBREF17 and FastText BIBREF18 word vectors of 300 dimensions  These vectors were trained on the corpus obtained from Nepali National Corpus  This pre-lemmatized corpus consists of 14 million words from books, web-texts and news papers  This corpus was mixed with the texts from the dataset before training CBOW and skip-gram version of word2vec using gensim libraryBIBREF24  This trained model consists of vectors for 72782 unique words 
Light pre-processing was performed on the corpus before training it  For example, invalid characters or characters other than Devanagari were removed but punctuation and numbers were not removed  We set the window context at 10 and the rare words whose count is below 5 are dropped  These word embeddings were not frozen during the training session because fine-tuning word embedding help achieve better performance compared to frozen oneBIBREF20 
We have used fasttext embeddings in particular because of its sub-word representation ability, which is very useful in highly inflectional language as shown in Table TABREF25  We have trained the word embedding in such a way that the sub-word size remains between 1 and 4  We particularly chose this size because in Nepali language a single letter can also be a word, for example e, t, C, r, l, n, u and a single character (grapheme) or sub-word can be formed after mixture of dependent vowel signs with consonant letters for example, C + O + = CO, here three different consonant letters form a single sub-word 
The two-dimensional visualization of an example word npAl is shown in FIGREF14  Principal Component Analysis (PCA) technique was used to generate this visualization which helps use to analyze the nearest neighbor words of a given sample word  84 and 104 nearest neighbors were observed using word2vec and fasttext embedding respectively on the same corpus 
Approach ::: Features ::: Character-level embeddings
BIBREF20 and BIBREF2 successfully presented that the character-level embeddings, extracted using CNN, when combined with word embeddings enhances the NER model performance significantly, as it is able to capture morphological features of a word  Figure FIGREF7 shows the grapheme-level CNN used in our model, where inputs to CNN are graphemes  Character-level CNN is also built in similar fashion, except the inputs are characters  Grapheme or Character -level embeddings are randomly initialized from [0,1] with real values with uniform distribution of dimension 30 
Approach ::: Features ::: Grapheme-level embeddings
Grapheme is atomic meaningful unit in writing system of any languages  Since, Nepali language is highly morphologically inflectional, we compared grapheme-level representation with character-level representation to evaluate its effect  For example, in character-level embedding, each character of a word npAl results into n + + p + A + l has its own embedding  However, in grapheme level, a word npAl is clustered into graphemes, resulting into n + pA + l  Here, each grapheme has its own embedding  This grapheme-level embedding results good scores on par with character-level embedding in highly inflectional languages like Nepali, because graphemes also capture syntactic information similar to characters  We created grapheme clusters using uniseg package which is helpful in unicode text segmentations 
Approach ::: Features ::: Part-of-speech (POS) one hot encoding
We created one-hot encoded vector of POS tags and then concatenated with pre-trained word embeddings before passing it to BiLSTM network  A sample of data is shown in figure FIGREF13 
Dataset Statistics ::: OurNepali dataset
Since, we there was no publicly available standard Nepali NER dataset and did not receive any dataset from the previous researchers, we had to create our own dataset  This dataset contains the sentences collected from daily newspaper of the year 2015-2016  This dataset has three major classes Person (PER), Location (LOC) and Organization (ORG)  Pre-processing was performed on the text before creation of the dataset, for example all punctuations and numbers besides ',', '-', '|' and ' ' were removed  Currently, the dataset is in standard CoNLL-2003 IO formatBIBREF25 
Since, this dataset is not lemmatized originally, we lemmatized only the post-positions like Ek, kO, l, mA, m, my, jF, sg, aEG which are just the few examples among 299 post positions in Nepali language  We obtained these post-positions from sanjaalcorps and added few more to match our dataset  We will be releasing this list in our github repository  We found out that lemmatizing the post-positions boosted the F1 score by almost 10% 
In order to label our dataset with POS-tags, we first created POS annotated dataset of 6946 sentences and 16225 unique words extracted from POS-tagged Nepali National Corpus and trained a BiLSTM model with 95 14% accuracy which was used to create POS-tags for our dataset 
The dataset released in our github repository contains each word in newline with space separated POS-tags and Entity-tags

ground_truth: a sample of  29,794 wikipedia articles and 2,794 arXiv papers 

prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. On which tasks do they test their conflict method?
Context 2:  The two word representations INLINEFORM3 and INLINEFORM4 are projected to a space where their element wise difference can be used to model their dissimilarity and softmax applied on them can produce high probability to more dissimilar word pairs 
It is good to note that conflict suffers from the same limitation that attention suffers from  This is when a pair of sentences are highly matching especially with multiple associations  But when the two methods work together, each compensates for the other's shortcomings 
Combination of attention and conflict
We used two weighted representations of INLINEFORM0 using weights of attention and conflict as computed in Eqn  (4) and (8) respectively  Our final representation of a word representation INLINEFORM1 conditioned on INLINEFORM2 can be expressed as: DISPLAYFORM0
where A and C denote that they are from attention and conflict models respectively 
Relation to Multi-Head attention
Multi-head attention, as introduced in BIBREF3 , computes multiple identical attention mechanism parallelly on multiple linear projections of same inputs  The parameters of each attention and projections are different in each head  Finally, they concatenate all the attentions which is similar to how we concatenate conflict and attention  However, they use dot-product to compute each of the attention 
Our combined model that contains both attention and conflict can be thought of as a 2-head attention model but both heads are different  Our conflict head explicitly captures difference between the inputs 
Visualizing attention and conflict
We observe how our conflict model learns the dissimilarities between word representations  We achieve that by visualizing the heatmap of the weight matrix INLINEFORM0 for both attention and conflict from eqns  (3) and (8)  While attention successfully learns the alignments, conflict matrix also shows that our approach models the contradicting associations like "animal" and "lake" or "australia" and "world"  These two associations are the unique pairs which are instrumental in determining that the two queries are not similar 
The model
We create two models both of which constitutes of three main parts: encoder, interaction and classifier and take two sequences as input  Except interaction, all the other parts are exactly identical between the two models  The encoder is shared among the sequences simply uses two stacked GRU layers  The interaction part consists of only attention for one model while for the another one it consists of attention and conflict combined as shown in (eqn 11)   The classifier part is simply stacked fully-connected layers  Figure 3 shows a block diagram of how our model looks like 
Task 1: Quora Duplicate Question Pair Detection
The dataset includes pairs of questions labelled as 1 or 0 depending on whether a pair is duplicate or not respectively  This is a popular pair-level classification task on which extensive work has already been done before like BIBREF7 , BIBREF8   For this task, we make the output layer of our model to predict two probabilities for non-duplicate and duplicate  We sample the data from the original dataset so that it contains equal positive and negative classes  Original dataset has some class imbalance but for sake simplicity we don't consider it  The final data that we use has roughly 400,000 question pairs and we split this data into train and test using 8:2 ratio 
We train all our models for roughly 2 epochs with a batch size of 64  We use a hidden dimension of 150 throughout the model  The embedding layer uses ELMO BIBREF9 which has proven to be very useful in various downstream language understanding tasks  Our FC layers consists of four dense layers with INLINEFORM0 activation after each layer  The dropout rate is kept as 0 2 for every recurrent and FC linear layers  We use Adam optimizer in our experiment with epsilon=1e-8, beta=0 9 and learning rate=1e-3 
Task 2: Ranking questions in Bing's People Also Ask
People Also Ask is a feature in Bing search result page where related questions are recommended to the user  User may click on a question to view the answer  Clicking is a positive feedback that shows user's interest in the question  We use this click logs to build a question classifier using the same model in Figure 3  The problem statement is very similar to BIBREF10 where they use logistic regression to predict whether an user would click on ad  Our goal is to classify if a question is potential high-click question or not for a given query  For this, we first create a labelled data set using the click logs where any question having CTR lower than 0 3 is labelled as 0 and a question having CTR more than 0 7 as 1 
Our final data resembles that of a pair-level classifier, as in Task 1, where user query and candidate questions are input  With these data set, we train a binary classifier to detect high-click and low-click questions 
Quantitative Analysis
For both tasks, we compute classification accuracy using three model variants and report the results in Table 1 and Table 2  We observe that model with both attention and conflict combined gives the best results 
We also show the training loss curve for both the models having attention and attention combined with conflict respectively  Figure 4 and 5 shows these curves for Task 1 and Task 2 respectively  The curves are smoothed using moving average having an window size of 8  We notice that the conflict model has much steeper slope and converges to a much better minima in both the tasks  It can also be noticed that in the training procedure for the model which has both attention and conflict, the updates are much smoother 
Qualitative Comparison
We also show qualitative results where we can observe that our model with attention and conflict combined does better on cases where pairs are non-duplicate and has very small difference  We have observed that the conflict model is very sensitive to even minor differences and compensates in such cases where attention poses high bias towards similarities already there in the sequences 
Sequence 1: What are the best ways to learn French ?
Sequence 2: How do I learn french genders ?
Attention only: 1
Attention+Conflict: 0
Ground Truth: 0
Sequence 1: How do I prevent breast cancer ?
Sequence 2: Is breast cancer preventable ?
Attention only: 1
Attention+Conflict: 0
Ground Truth: 0
We provide two examples with predictions from the models with only attention and combination of attention and conflict  Each example is accompanied by the ground truth in our data 
Analyzing the gains
We analyzed the gains in Task 1 which we get from the attention-conflict model in order to ensure that they are not due to randomness in weight initialization or simply additional parameters  We particularly focused on the examples which were incorrectly marked in attention model but correctly in attention-conflict model  We saw that 70% of those cases are the ones where the pair was incorrectly marked as duplicate in the previous model but our combined model correctly marked them as non-duplicate 
Conclusion
In this work, we highlighted the limits of attention especially in cases where two sequences have a contradicting relationship based on the task it performs  To alleviate this problem and further improve the performance, we propose a conflict mechanism that tries to capture how two sequences repel each other  This acts like the inverse of attention and, empirically, we show that how conflict and attention together can improve the performance 
Future research work should be based on alternative design of conflict mechanism using other difference operators other than element wise difference which we use 
Context 3:  The sentences are separated by empty newline  A sample sentence from the dataset is presented in table FIGREF13 
Dataset Statistics ::: ILPRL dataset
After much time, we received the dataset from Bal Krishna Bal, ILPRL, KU  This dataset follows standard CoNLL-2003 IOB formatBIBREF25 with POS tags  This dataset is prepared by ILPRL Lab, KU and KEIV Technologies  Few corrections like correcting the NER tags had to be made on the dataset  The statistics of both the dataset is presented in table TABREF23 
Table TABREF24 presents the total entities (PER, LOC, ORG and MISC) from both of the dataset used in our experiments  The dataset is divided into three parts with 64%, 16% and 20% of the total dataset into training set, development set and test set respectively 
Experiments
In this section, we present the details about training our neural network  The neural network architecture are implemented using PyTorch framework BIBREF26  The training is performed on a single Nvidia Tesla P100 SXM2  We first run our experiment on BiLSTM, BiLSTM-CNN, BiLSTM-CRF BiLSTM-CNN-CRF using the hyper-parameters mentioned in Table TABREF30  The training and evaluation was done on sentence-level  The RNN variants are initialized randomly from $(-\sqrt{k},\sqrt{k})$ where $k=\frac{1}{hidden\_size}$ 
First we loaded our dataset and built vocabulary using torchtext library  This eased our process of data loading using its SequenceTaggingDataset class  We trained our model with shuffled training set using Adam optimizer with hyper-parameters mentioned in table TABREF30  All our models were trained on single layer of LSTM network  We found out Adam was giving better performance and faster convergence compared to Stochastic Gradient Descent (SGD)  We chose those hyper-parameters after many ablation studies  The dropout of 0 5 is applied after LSTM layer 
For CNN, we used 30 different filters of sizes 3, 4 and 5  The embeddings of each character or grapheme involved in a given word, were passed through the pipeline of Convolution, Rectified Linear Unit and Max-Pooling  The resulting vectors were concatenated and applied dropout of 0 5 before passing into linear layer to obtain the embedding size of 30 for the given word  This resulting embedding is concatenated with word embeddings, which is again concatenated with one-hot POS vector 
Experiments ::: Tagging Scheme
Currently, for our experiments we trained our model on IO (Inside, Outside) format for both the dataset, hence the dataset does not contain any B-type annotation unlike in BIO (Beginning, Inside, Outside) scheme 
Experiments ::: Early Stopping
We used simple early stopping technique where if the validation loss does not decrease after 10 epochs, the training was stopped, else the training will run upto 100 epochs  In our experience, training usually stops around 30-50 epochs 
Experiments ::: Hyper-parameters Tuning
We ran our experiment looking for the best hyper-parameters by changing learning rate from (0,1, 0 01, 0 001, 0 0001), weight decay from [$10^{-1}$, $10^{-2}$, $10^{-3}$, $10^{-4}$, $10^{-5}$, $10^{-6}$, $10^{-7}$], batch size from [1, 2, 4, 8, 16, 32, 64, 128], hidden size from [8, 16, 32, 64, 128, 256, 512 1024]  Table TABREF30 shows all other hyper-parameter used in our experiment for both of the dataset 
Experiments ::: Effect of Dropout
Figure FIGREF31 shows how we end up choosing 0 5 as dropout rate  When the dropout layer was not used, the F1 score are at the lowest  As, we slowly increase the dropout rate, the F1 score also gradually increases, however after dropout rate = 0 5, the F1 score starts falling down  Therefore, we have chosen 0 5 as dropout rate for all other experiments performed 
Evaluation
In this section, we present the details regarding evaluation and comparison of our models with other baselines 
Table TABREF25 shows the study of various embeddings and comparison among each other in OurNepali dataset  Here, raw dataset represents such dataset where post-positions are not lemmatized  We can observe that pre-trained embeddings significantly improves the score compared to randomly initialized embedding  We can deduce that Skip Gram models perform better compared CBOW models for word2vec and fasttext  Here, fastText_Pretrained represents the embedding readily available in fastText website, while other embeddings are trained on the Nepali National Corpus as mentioned in sub-section SECREF11  From this table TABREF25, we can clearly observe that model using fastText_Skip Gram embeddings outperforms all other models 
Table TABREF35 shows the model architecture comparison between all the models experimented  The features used for Stanford CRF classifier are words, letter n-grams of upto length 6, previous word and next word  This model is trained till the current function value is less than $1\mathrm {e}{-2}$  The hyper-parameters of neural network experiments are set as shown in table TABREF30  Since, word embedding of character-level and grapheme-level is random, their scores are near 
All models are evaluated using CoNLL-2003 evaluation scriptBIBREF25 to calculate entity-wise precision, recall and f1 score 
Discussion
In this paper we present that we can exploit the power of neural network to train the model to perform downstream NLP tasks like Named Entity Recognition even in Nepali language  We showed that the word vectors learned through fasttext skip gram model performs better than other word embedding because of its capability to represent sub-word and this is particularly important to capture morphological structure of words and sentences in highly inflectional like Nepali  This concept can come handy in other Devanagari languages as well because the written scripts have similar syntactical structure 
We also found out that stemming post-positions can help a lot in improving model performance because of inflectional characteristics of Nepali language  So when we separate out its inflections or morphemes, we can minimize the variations of same word which gives its root word a stronger word vector representations compared to its inflected versions 
We can clearly imply from tables TABREF23, TABREF24, and TABREF35 that we need more data to get better results because OurNepali dataset volume is almost ten times bigger compared to ILPRL dataset in terms of entities 
Conclusion and Future work
In this paper, we proposed a novel NER for Nepali language and achieved relative improvement of upto 10% and studies different factors effecting the performance of the NER for Nepali language 
We also present a neural architecture BiLSTM+CNN(grapheme-level) which turns out to be performing on par with BiLSTM+CNN(character-level) under the same configuration  We believe this will not only help Nepali language but also other languages falling under the umbrellas of Devanagari languages  Our model BiLSTM+CNN(grapheme-level) and BiLSTM+CNN(G)+POS outperforms all other model experimented in OurNepali and ILPRL dataset respectively 
Since this is the first named entity recognition research in Nepal language using neural network, there are many rooms for improvement  We believe initializing the grapheme-level embedding with fasttext embeddings might help boosting the performance, rather than randomly initializing it  In future, we plan to apply other latest techniques like BERT, ELMo and FLAIR to study its effect on low-resource language like Nepali  We also plan to improve the model using cross-lingual or multi-lingual parameter sharing techniques by jointly training with other Devanagari languages like Hindi and Bengali 
Finally, we would like to contribute our dataset to Nepali NLP community to move forward the research going on in language understanding domain  We believe there should be special committee to create and maintain such dataset for Nepali NLP and organize various competitions which would elevate the NLP research in Nepal 
Some of the future works are listed below:
Proper initialization of grapheme level embedding from fasttext embeddings 
Apply robust POS-tagger for Nepali dataset
Lemmatize the OurNepali dataset with robust and efficient lemmatizer
Improve Nepali language score with cross-lingual learning techniques
Create more dataset using Wikipedia/Wikidata framework
Acknowledgments
The authors of this paper would like to express sincere thanks to Bal Krishna Bal, Kathmandu University Professor for providing us the POS-tagged Nepali NER data 

ground_truth: Task 1: Quora Duplicate Question Pair Detection, Task 2: Ranking questions

Quora Duplicate Question Pair Detection, Ranking questions in Bing's People Also Ask

prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. How many sentences does the dataset contain?
Context 4: 0 are clipped  We stop training when the ROUGE-L f-score stops increasing 
Hybrid training: An Adam optimizer with a learning rate of 0 0001 is used to optimize $L_{\text{RL-*}}$ (Equation DISPLAY_FORM17) and $L_\text{{ARL-SEN}}$ (Equation DISPLAY_FORM19)  When training Pointer-Gen+RL-ROUGE, the best $\lambda $ is chosen based on the ROUGE-L score on the validation set  In our experiment, $\lambda $ is set as 0 95  An Adam optimizer with a learning rate of 0 001 is used to optimize $L_b$  When training Pointer-Gen+ARL-SEN, we don't use the full LCSTS dataset, but only headlines with a sensationalism score larger than 0 1 as we observe that Pointer-Gen+ARL-SEN will generate a few unnatural phrases when using full dataset  We believe the reason is the high ratio of RL during training  Figure FIGREF23 shows that the probability density near 0 is very high, meaning that in each batch, many of the samples will have a very low sensationalism score  On expectation, each sample will receive 0 239 MLE training and 0 761 RL training  This leads to RL dominanting the loss  Thus, we propose to filter samples with a minimum sensationalism score with 0 1 and it works very well  For Pointer-Gen+RL-SEN, we also set the minimum sensationalism score as 0 1, and $\lambda $ is set as 0 5 to remove unnatural phrases, making a fair comparison to Pointer-Gen+ARL-SEN 
We stop training Pointer-Gen+Same-FT, Pointer-Gen+Pos-FT, Pointer-Gen+RL-SEN and Pointer-Gen+ARL-SEN, when $\alpha _\text{sen}$ stops increasing on the validation set  Beam-search with a beam size of 5 is adopted for decoding in all models 
Sensational Headline Generation ::: Evaluation Metrics
We briefly describe the evaluation metrics below 
ROUGE: ROUGE is a commonly used evaluation metric for summarization  It measures the N-gram overlap between generated and training headlines  We use it to evaluate the relevance of generated headlines  The widely used pyrouge toolkit is used to calculate ROUGE-1 (RG-1), ROUGE-2 (RG-2), and ROUGE-L (RG-L) 
Human evaluation: We randomly sample 50 articles from the test set and send the generated headlines from all models and corresponding headlines in the test set to human annotators  We evaluate the sensationalism and fluency of the headlines by setting up two independent human annotation tasks  We ask 10 annotators to label each headline for each task  For the sensationalism annotation, each annotator is asked one question, “Is the headline sensational?”, and he/she has to choose either `yes' or `no'  The annotators were not told which system the headline is from  The process of distributing samples and recruiting annotators is managed by Crowdflower  After annotation, we define the sensationalism score as the proportion of annotations on all generated headlines from one model labeled as `yes'  For the fluency annotation, we repeat the same procedure as for the sensationalism annotation, except that we ask each annotator the question “Is the headline fluent?” We define the fluency score as the proportion of annotations on all headlines from one specific model labeled as `yes'  We put human annotation instructions in the supplemental material 
UTF8gbsn
Results
We first compare all four models, Pointer-Gen, Pointer-Gen-RL+ROUGE, Pointer-Gen-RL-SEN, and Pointer-Gen-ARL-SEN, to existing models with ROUGE in Table TABREF25 to establish that our model produces relevant headlines and we leave the sensationalism for human evaluation  Note that we only compare our models to commonly used strong summarization baselines, to validate that our implementation achieves comparable performance to existing work  In our implementation, Pointer-Gen achieves a 34 51 RG-1 score, 22 21 RG-2 score, and 31 68 RG-L score, which is similar to the results of gu2016incorporating  Pointer-Gen+ARL-SEN, although optimized for the sensationalism reward, achieves similar performance to our Pointer-Gen baseline, which means that Pointer-Gen+ARL-SEN still keeps its summarization ability  An example of headlines generated from different models in Table TABREF29 shows that Pointer-Gen and Pointer-Gen+RL-ROUGE learns to summarize the main point of the article: “The Nikon D600 camera is reported to have black spots when taking photos”  Pointer-Gen+RL-SEN makes the headline more sensational by blaming Nikon for attributing the damage to the smog  Pointer-Gen+ARL-SEN generates the most sensational headline by exaggerating the result “Getting a serious trouble!” to maximize user's attention 
We then compare different models using the sensationalism score in Table TABREF30  The Pointer-Gen baseline model achieves a 42 6% sensationalism score, which is the minimum that a typical summarization model achieves  By filtering out low-sensational headlines, Pointer-Gen+Same-FT and Pointer-Gen+Pos-FT achieves higher sensationalism scores, which implies the effectiveness of our sensationalism scorer  Our Pointer-Gen+ARL-SEN model achieves the best performance of 60 8%  This is an absolute improvement of 18 2% over the Pointer-Gen baseline  The Chi-square test on the results confirms that Pointer-Gen+ARL-SEN is statistically significantly more sensational than all the other baseline models, with the largest p-value less than 0 01  Also, we find that the test set headlines achieves 57 8% sensationalism score, much larger than Pointer-Gen baseline, which also supports our intuition that generated headlines will be less sensational than the original one  On the other hand, we found that Pointer-Gen+Pos is much worse than other baselines  The reason is that training on sensational samples alone discards around 80% of the whole training set that is also helpful for maintaining relevance and a good language model  It shows the necessity of using RL 
UTF8gbsn
In addition, both Pointer-Gen+RL-SEN and Pointer-Gen+ARL-SEN, which use the sensationalism score as the reward, obtain statistically better results than Pointer-Gen+RL-ROUGE and Pointer-Gen, with a p-value less than 0 05 by a Chi-square test  This result shows the effectiveness of RL to generate more sensational headlines  The reason is that even though our noisy classifier could also learn to classify domains, the generator during RL training is not allowed to increase the reward by shifting domains but encouraged to generate more sensational headlines, due to the consistency constraint on the domains of the headline and the article  Furthermore, Poiner-Gen+ARL-SEN gets better performance than Pointer-Gen+RL-SEN, which confirms the superiority of the ARL loss function  We also visualize in Figure FIGREF31 a comparison between Pointer-Gen+ARL-SEN and Pointer-Gen+RL-SEN according to how sensational the test set headlines are  The blue bars denote the smaller scores between the two models  For example, if the blue bar is 0 6, it means that the worse model between Pointer-Gen+RL-SEN and Pointer-Gen+ARL-SEN achieves 0 6  And the color of orange/black further indicates the better model and its score  We find that Pointer-Gen+ARL-SEN outperforms Pointer-Gen+RL-SEN for most cases  The improvement is higher when the test set headlines are not sensational (the sensationalism score is less than 0 5), which may be attributed to the higher ratio of RL training on non-sensational headlines 
Apart from the sensationalism evaluation, we measure the fluency of the headlines generated from different models  Fluency scores in Table TABREF30 show that Pointer-Gen+RL-SEN and Pointer-Gen+ARL-SEN achieve comparable fluency performance to Pointer-Gen and Pointer-Gen+RL-ROUGE  Test set headlines achieve the best performance among all models, but the difference is not statistically significant  Also, we observe that fine-tuning on sensational headlines will hurt the performance, both in sensationalism and fluency 
After manually checking the outputs, we observe that our model is able to generate sensational headlines using diverse sensationalization strategies  These strategies include, but are not limited to, creating a curiosity gap, asking questions, highlighting numbers, being emotional and emphasizing the user  Examples can be found in Table TABREF32 
Related Work
Our work is related to summarization tasks  An encoder-decoder model was first applied to two sentence-level abstractive summarization tasks on the DUC-2004 and Gigaword datasets BIBREF12  This model was later extended by selective encoding BIBREF13, a coarse to fine approach BIBREF14, minimum risk training BIBREF1, and topic-aware models BIBREF15  As long summaries were recognized as important, the CNN/Daily Mail dataset was used in nallapati2016abstractive  Graph-based attention BIBREF16, pointer-generator with coverage loss BIBREF0 are further developed to improve the generated summaries  celikyilmaz2018deep proposed deep communicating agents for representing a long document for abstractive summarization  In addition, many papers BIBREF17, BIBREF18, BIBREF19 use extractive methods to directly select sentences from articles  However, none of these work considered the sensationalism of generated outputs 
RL is also gaining popularity as it can directly optimize non-differentiable metrics BIBREF20, BIBREF21, BIBREF22  paulus2017deep proposed an intra-decoder model and combined RL and MLE to deal with summaries with bad qualities  RL has also been explored with generative adversarial networks (GANs) BIBREF23  liu2017generative applied GANs on summarization task and achieved better performance  niu2018polite tackles the problem of polite generation with politeness reward  Our work is different in that we propose a novel function to balance RL and MLE 
Our task is also related to text style transfer  Implicit methods BIBREF10, BIBREF24, BIBREF4 transfer the styles by separating sentence representations into content and style, for example using back-translationBIBREF4  However, these methods cannot guarantee the content consistency between the original sentence and transferred output BIBREF25  Explicit methods BIBREF26, BIBREF25 transfer the style by directly identifying style related keywords and modifying them  However, sensationalism is not always restricted to keywords, but the full sentence  By leveraging small human labeled English dataset, clickbait detection has been well investigated BIBREF2, BIBREF27, BIBREF3  However, these human labeled dataset are not available for other languages, such as Chinese 
Modeling sensationalism is also related to modeling emotion  Emotion has been well investigated in both word levelBIBREF28, BIBREF29 and sentence levelBIBREF30, BIBREF31, BIBREF32, BIBREF33, BIBREF34  It has also been considered an important factor in engaging interactive systemsBIBREF35, BIBREF36, BIBREF37  Although we observe that sensational headlines contain emotion, it is still not clear which emotion and how emotions will influence the sensationalism 
Conclusion and Future Work
In this paper, we propose a model that generates sensational headlines without labeled data using Reinforcement Learning  Firstly, we propose a distant supervision strategy to train the sensationalism scorer  As a result, we achieve 65% accuracy between the predicted sensationalism score and human evaluation  To effectively leverage this noisy sensationalism score as the reward for RL, we propose a novel loss function, ARL, to automatically balance RL with MLE  Human evaluation confirms the effectiveness of both our sensationalism scorer and ARL to generate more sensational headlines  Future work can be improving the sensationalism scorer and investigating the applications of dynamic balancing methods between RL and MLE in textGANBIBREF23  Our work also raises the ethical questions about generating sensational headlines, which can be further explored 
Acknowledgments
Thanks to ITS/319/16FP of Innovation Technology Commission, HKUST 16248016 of Hong Kong Research Grants Council for funding  In addition, we thank Zhaojiang Lin for helpful discussion and Yan Xu, Zihan Liu for the data collection 

ground_truth: 3606

6946