prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. How does Gaussian-masked directional multi-head attention works?
Context 1: Introduction
Chinese word segmentation (CWS) is a task for Chinese natural language process to delimit word boundary  CWS is a basic and essential task for Chinese which is written without explicit word delimiters and different from alphabetical languages like English  BIBREF0 treats Chinese word segmentation (CWS) as a sequence labeling task with character position tags, which is followed by BIBREF1, BIBREF2, BIBREF3  Traditional CWS models depend on the design of features heavily which effects the performance of model  To minimize the effort in feature engineering, some CWS models BIBREF4, BIBREF5, BIBREF6, BIBREF7, BIBREF8, BIBREF9, BIBREF10, BIBREF11 are developed following neural network architecture for sequence labeling tasks BIBREF12  Neural CWS models perform strong ability of feature representation, employing unigram and bigram character embedding as input and approach good performance 
The CWS task is often modeled as one graph model based on a scoring model that means it is composed of two parts, one part is an encoder which is used to generate the representation of characters from the input sequence, the other part is a decoder which performs segmentation according to the encoder scoring  Table TABREF1 summarizes typical CWS models according to their decoding ways for both traditional and neural models  Markov models such as BIBREF13 and BIBREF4 depend on the maximum entropy model or maximum entropy Markov model both with a Viterbi decoder  Besides, conditional random field (CRF) or Semi-CRF for sequence labeling has been used for both traditional and neural models though with different representations BIBREF2, BIBREF15, BIBREF10, BIBREF17, BIBREF18  Generally speaking, the major difference between traditional and neural network models is about the way to represent input sentences 
Recent works about neural CWS which focus on benchmark dataset, namely SIGHAN Bakeoff BIBREF21, may be put into the following three categories roughly 
Encoder  Practice in various natural language processing tasks has been shown that effective representation is essential to the performance improvement  Thus for better CWS, it is crucial to encode the input character, word or sentence into effective representation  Table TABREF2 summarizes regular feature sets for typical CWS models including ours as well  The building blocks that encoders use include recurrent neural network (RNN) and convolutional neural network (CNN), and long-term memory network (LSTM) 
Graph model  As CWS is a kind of structure learning task, the graph model determines which type of decoder should be adopted for segmentation, also it may limit the capability of defining feature, as shown in Table 2, not all graph models can support the word features  Thus recent work focused on finding more general or flexible graph model to make model learn the representation of segmentation more effective as BIBREF9, BIBREF11 
External data and pre-trained embedding  Whereas both encoder and graph model are about exploring a way to get better performance only by improving the model strength itself  Using external resource such as pre-trained embeddings or language representation is an alternative for the same purpose BIBREF22, BIBREF23  SIGHAN Bakeoff defines two types of evaluation settings, closed test limits all the data for learning should not be beyond the given training set, while open test does not take this limitation BIBREF21  In this work, we will focus on the closed test setting by finding a better model design for further CWS performance improvement 
Shown in Table TABREF1, different decoders have particular decoding algorithms to match the respective CWS models  Markov models and CRF-based models often use Viterbi decoders with polynomial time complexity  In general graph model, search space may be too large for model to search  Thus it forces graph models to use an approximate beam search strategy  Beam search algorithm has a kind low-order polynomial time complexity  Especially, when beam width $b$=1, the beam search algorithm will reduce to greedy algorithm with a better time complexity $O(Mn)$ against the general beam search time complexity $O(Mnb^2)$, where $n$ is the number of units in one sentences, $M$ is a constant representing the model complexity  Greedy decoding algorithm can bring the fastest speed of decoding while it is not easy to guarantee the precision of decoding when the encoder is not strong enough 
In this paper, we focus on more effective encoder design which is capable of offering fast and accurate Chinese word segmentation with only unigram feature and greedy decoding  Our proposed encoder will only consist of attention mechanisms as building blocks but nothing else  Motivated by the Transformer BIBREF24 and its strength of capturing long-range dependencies of input sentences, we use a self-attention network to generate the representation of input which makes the model encode sentences at once without feeding input iteratively  Considering the weakness of the Transformer to model relative and absolute position information directly BIBREF25 and the importance of localness information, position information and directional information for CWS, we further improve the architecture of standard multi-head self-attention of the Transformer with a directional Gaussian mask and get a variant called Gaussian-masked directional multi-head attention  Based on the newly improved attention mechanism, we expand the encoder of the Transformer to capture different directional information  With our powerful encoder, our model uses only simple unigram features to generate representation of sentences 
For decoder which directly performs the segmentation, we use the bi-affinal attention scorer, which has been used in dependency parsing BIBREF26 and semantic role labeling BIBREF27, to implement greedy decoding on finding the boundaries of words  In our proposed model, greedy decoding ensures a fast segmentation while powerful encoder design ensures a good enough segmentation performance even working with greedy decoder together  Our model will be strictly evaluated on benchmark datasets from SIGHAN Bakeoff shared task on CWS in terms of closed test setting, and the experimental results show that our proposed model achieves new state-of-the-art 
The technical contributions of this paper can be summarized as follows 
We propose a CWS model with only attention structure  The encoder and decoder are both based on attention structure 
With a powerful enough encoder, we for the first time show that unigram (character) featues can help yield strong performance instead of diverse $n$-gram (character and word) features in most of previous work 
To capture the representation of localness information and directional information, we propose a variant of directional multi-head self-attention to further enhance the state-of-the-art Transformer encoder 
Models
The CWS task is often modelled as one graph model based on an encoder-based scoring model  The model for CWS task is composed of an encoder to represent the input and a decoder based on the encoder to perform actual segmentation  Figure FIGREF6 is the architecture of our model  The model feeds sentence into encoder  Embedding captures the vector $e=(e_1,   ,e_n)$ of the input character sequences of $c=(c_1,   ,c_n)$  The encoder maps vector sequences of $ {e}=(e_1,  ,e_n)$ to two sequences of vector which are $ {v^b}=(v_1^b,   ,v_n^b)$ and ${v^f}=(v_1^f,   v_n^f)$ as the representation of sentences  With $v^b$ and $v^f$, the bi-affinal scorer calculates the probability of each segmentation gaps and predicts the word boundaries of input  Similar as the Transformer, the encoder is an attention network with stacked self-attention and point-wise, fully connected layers while our encoder includes three independent directional encoders 
Models ::: Encoder Stacks
In the Transformer, the encoder is composed of a stack of N identical layers and each layer has one multi-head self-attention layer and one position-wise fully connected feed-forward layer  One residual connection is around two sub-layers and followed by layer normalization BIBREF24  This architecture provides the Transformer a good ability to generate representation of sentence 
With the variant of multi-head self-attention, we design a Gaussian-masked directional encoder to capture representation of different directions to improve the ability of capturing the localness information and position information for the importance of adjacent characters  One unidirectional encoder can capture information of one particular direction 
For CWS tasks, one gap of characters, which is from a word boundary, can divide one sequence into two parts, one part in front of the gap and one part in the rear of it  The forward encoder and backward encoder are used to capture information of two directions which correspond to two parts divided by the gap 
One central encoder is paralleled with forward and backward encoders to capture the information of entire sentences  The central encoder is a special directional encoder for forward and backward information of sentences  The central encoder can fuse the information and enable the encoder to capture the global information 
The encoder outputs one forward information and one backward information of each positions  The representation of sentence generated by center encoder will be added to these information directly:
where $v^{b}=(v^b_1,   ,v^b_n)$ is the backward information, $v^{f}=(v^f_1,   ,v^f_n)$ is the forward information, $r^{b}=(r^b_1,   ,r^b_n)$ is the output of backward encoder, $r^{c}=(r^c_1,   ,r^c_n)$ is the output of center encoder and $r^{f}=(r^f_1,   ,r^f_n)$ is the output of forward encoder 
Models ::: Gaussian-Masked Directional Multi-Head Attention
Similar as scaled dot-product attention BIBREF24, Gaussian-masked directional attention can be described as a function to map queries and key-value pairs to the representation of input  Here queries, keys and values are all vectors  Standard scaled dot-product attention is calculated by dotting query $Q$ with all keys $K$, dividing each values by $\sqrt{d_k}$, where $\sqrt{d_k}$ is the dimension of keys, and apply a softmax function to generate the weights in the attention:
Different from scaled dot-product attention, Gaussian-masked directional attention expects to pay attention to the adjacent characters of each positions and cast the localness relationship between characters as a fix Gaussian weight for attention  We assume that the Gaussian weight only relys on the distance between characters 
Firstly we introduce the Gaussian weight matrix $G$ which presents the localness relationship between each two characters:
where $g_{ij}$ is the Gaussian weight between character $i$ and $j$, $dis_{ij}$ is the distance between character $i$ and $j$, $\Phi (x)$ is the cumulative distribution function of Gaussian, $\sigma $ is the standard deviation of Gaussian function and it is a hyperparameter in our method  Equation (DISPLAY_FORM13) can ensure the Gaussian weight equals 1 when $dis_{ij}$ is 0  The larger distance between charactersis, the smaller the weight is, which makes one character can affect its adjacent characters more compared with other characters 
To combine the Gaussian weight to the self-attention, we produce the Hadamard product of Gaussian weight matrix $G$ and the score matrix produced by $Q{K^{T}}$
where $AG$ is the Gaussian-masked attention  It ensures that the relationship between two characters with long distances is weaker than adjacent characters 
The scaled dot-product attention models the relationship between two characters without regard to their distances in one sequence  For CWS task, the weight between adjacent characters should be more important while it is hard for self-attention to achieve the effect explicitly because the self-attention cannot get the order of sentences directly  The Gaussian-masked attention adjusts the weight between characters and their adjacent character to a larger value which stands for the effect of adjacent characters 
For forward and backward encoder, the self-attention sublayer needs to use a triangular matrix mask to let the self-attention focus on different weights:
where $pos_i$ is the position of character $c_i$  The triangular matrix for forward and backward encode are:
$\left[ \begin{matrix} 1 & 0 & 0 & \cdots &0\\ 1 & 1 & 0 & \cdots &0\\ 1 & 1 & 1 & \cdots &0\\ \vdots &\vdots &\vdots &\ddots &\vdots \\ 1 & 1 & 1 & \cdots & 1\\ \end{matrix} \right]$ $\left[ \begin{matrix} 1 & 1 & 1 & \cdots &1 \\ 0 & 1 & 1 & \cdots &1 \\ 0 & 0& 1 & \cdots &1 \\ \vdots &\vdots &\vdots &\ddots &\vdots \\ 0 & 0 & 0 & \cdots & 1\\ \end{matrix}\right]$
Similar as BIBREF24, we use multi-head attention to capture information from different dimension positions as Figure FIGREF16 and get Gaussian-masked directional multi-head attention  With multi-head attention architecture, the representation of input can be captured by
where $MH$ is the Gaussian-masked multi-head attention, ${W_i^q, W_i^k,W_i^v} \in \mathbb {R}^{d_k \times d_h}$ is the parameter matrices to generate heads, $d_k$ is the dimension of model and $d_h$ is the dimension of one head 
Models ::: Bi-affinal Attention Scorer
Regarding word boundaries as gaps between any adjacent words converts the character labeling task to the gap labeling task  Different from character labeling task, gap labeling task requires information of two adjacent characters  The relationship between adjacent characters can be represented as the type of gap  The characteristic of word boundaries makes bi-affine attention an appropriate scorer for CWS task 
Bi-affinal attention scorer is the component that we use to label the gap  Bi-affinal attention is developed from bilinear attention which has been used in dependency parsing BIBREF26 and SRL BIBREF27  The distribution of labels in a labeling task is often uneven which makes the output layer often include a fixed bias term for the prior probability of different labels BIBREF27  Bi-affine attention uses bias terms to alleviate the burden of the fixed bias term and get the prior probability which makes it different from bilinear attention  The distribution of the gap is uneven that is similar as other labeling task which fits bi-affine 
Bi-affinal attention scorer labels the target depending on information of independent unit and the joint information of two units  In bi-affinal attention, the score $s_{ij}$ of characters $c_i$ and $c_j$ $(i < j)$ is calculated by:
where $v_i^f$ is the forward information of $c_i$ and $v_i^b$ is the backward information of $c_j$  In Equation (DISPLAY_FORM21), $W$, $U$ and $b$ are all parameters that can be updated in training  $W$ is a matrix with shape $(d_i \times N\times d_j)$ and $U$ is a $(N\times (d_i + d_j))$ matrix where $d_i$ is the dimension of vector $v_i^f$ and $N$ is the number of labels 
In our model, the biaffine scorer uses the forward information of character in front of the gap and the backward information of the character behind the gap to distinguish the position of characters  Figure FIGREF22 is an example of labeling gap  The method of using biaffine scorer ensures that the boundaries of words can be determined by adjacent characters with different directional information  The score vector of the gap is formed by the probability of being a boundary of word  Further, the model generates all boundaries using activation function in a greedy decoding way 
Experiments ::: Experimental Settings ::: Data
We train and evaluate our model on datasets from SIGHAN Bakeoff 2005 BIBREF21 which has four datasets, PKU, MSR, AS and CITYU  Table TABREF23 shows the statistics of train data  We use F-score to evaluate CWS models  To train model with pre-trained embeddings in AS and CITYU, we use OpenCC to transfer data from traditional Chinese to simplified Chinese 
Experiments ::: Experimental Settings ::: Pre-trained Embedding
We only use unigram feature so we only trained character embeddings  Our pre-trained embedding are pre-trained on Chinese Wikipedia corpus by word2vec BIBREF29 toolkit  The corpus used for pre-trained embedding is all transferred to simplified Chinese and not segmented  On closed test, we use embeddings initialized randomly 
Experiments ::: Experimental Settings ::: Hyperparameters
For different datasets, we use two kinds of hyperparameters which are presented in Table TABREF24  We use hyperparameters in Table TABREF24 for small corpora (PKU and CITYU) and normal corpora (MSR and AS)  We set the standard deviation of Gaussian function in Equation (DISPLAY_FORM13) to 2  Each training batch contains sentences with at most 4096 tokens 
Experiments ::: Experimental Settings ::: Optimizer
To train our model, we use the Adam BIBREF30 optimizer with $\beta _1=0 9$, $\beta _2=0 98$ and $\epsilon =10^{-9}$  The learning rate schedule is the same as BIBREF24:
where $d$ is the dimension of embeddings, $step$ is the step number of training and $warmup_step$ is the step number of warmup  When the number of steps is smaller than the step of warmup, the learning rate increases linearly and then decreases 
Experiments ::: Hardware and Implements
We trained our models on a single CPU (Intel i7-5960X) with an nVidia 1080 Ti GPU  We implement our model in Python with Pytorch 1 0 
Experiments ::: Results
Tables TABREF25 and TABREF26 reports the performance of recent models and ours in terms of closed test setting  Without the assistance of unsupervised segmentation features userd in BIBREF20, our model outperforms all the other models in MSR and AS except BIBREF18 and get comparable performance in PKU and CITYU  Note that all the other models for this comparison adopt various $n$-gram features while only our model takes unigram ones 
With unsupervised segmentation features introduced by BIBREF20, our model gets a higher result  Specially, the results in MSR and AS achieve new state-of-the-art and approaching previous state-of-the-art in CITYU and PKU  The unsupervised segmentation features are derived from the given training dataset, thus using them does not violate the rule of closed test of SIGHAN Bakeoff 
Table TABREF36 compares our model and recent neural models in terms of open test setting in which any external resources, especially pre-trained embeddings or language models can be used  In MSR and AS, our model gets a comparable result while our results in CITYU and PKU are not remarkable 
However, it is well known that it is always hard to compare models when using open test setting, especially with pre-trained embedding  Not all models may use the same method and data to pre-train  Though pre-trained embedding or language model can improve the performance, the performance improvement itself may be from multiple sources  It often that there is a success of pre-trained embedding to improve the performance, while it cannot prove that the model is better 
Compared with other LSTM models, our model performs better in AS and MSR than in CITYU and PKU  Considering the scale of different corpora, we believe that the size of corpus affects our model and the larger size is, the better model performs  For small corpus, the model tends to be overfitting 
Tables TABREF25 and TABREF26 also show the decoding time in different datasets  Our model finishes the segmentation with the least decoding time in all four datasets, thanks to the architecture of model which only takes attention mechanism as basic block 
Related Work ::: Chinese Word Segmentation
CWS is a task for Chinese natural language process to delimit word boundary  BIBREF0 for the first time formulize CWS as a sequence labeling task  BIBREF3 show that different character tag sets can make essential impact for CWS  BIBREF2 use CRFs as a model for CWS, achieving new state-of-the-art  Works of statistical CWS has built the basis for neural CWS 
Neural word segmentation has been widely used to minimize the efforts in feature engineering which was important in statistical CWS  BIBREF4 introduce the neural model with sliding-window based sequence labeling  BIBREF6 propose a gated recursive neural network (GRNN) for CWS to incorporate complicated combination of contextual character and n-gram features  BIBREF7 use LSTM to learn long distance information  BIBREF9 propose a neural framework that eliminates context windows and utilize complete segmentation history  BIBREF33 explore a joint model that performs segmentation, POS-Tagging and chunking simultaneously  BIBREF34 propose a feature-enriched neural model for joint CWS and part-of-speech tagging  BIBREF35 present a joint model to enhance the segmentation of Chinese microtext by performing CWS and informal word detection simultaneously  BIBREF17 propose a character-based convolutional neural model to capture $n$-gram features automatically and an effective approach to incorporate word embeddings  BIBREF11 improve the model in BIBREF9 and propose a greedy neural word segmenter with balanced word and character embedding inputs  BIBREF23 propose a novel neural network model to incorporate unlabeled and partially-labeled data  BIBREF36 propose two methods that extend the Bi-LSTM to perform incorporating dictionaries into neural networks for CWS  BIBREF37 propose Switch-LSTMs to segment words and provided a more flexible solution for multi-criteria CWS which is easy to transfer the learned knowledge to new criteria 
Related Work ::: Transformer
Transformer BIBREF24 is an attention-based neural machine translation model  The Transformer is one kind of self-attention networks (SANs) which is proposed in BIBREF38  Encoder of the Transformer consists of one self-attention layer and a position-wise feed-forward layer  Decoder of the Transformer contains one self-attention layer, one encoder-decoder attention layer and one position-wise feed-forward layer  The Transformer uses residual connections around the sublayers and then followed by a layer normalization layer 
Scaled dot-product attention is the key component in the Transformer  The input of attention contains queries, keys, and values of input sequences  The attention is generated using queries and keys like Equation (DISPLAY_FORM11)  Structure of scaled dot-product attention allows the self-attention layer generate the representation of sentences at once and contain the information of the sentence which is different from RNN that process characters of sentences one by one  Standard self-attention is similar as Gaussian-masked direction attention while it does not have directional mask and gaussian mask  BIBREF24 also propose multi-head attention which is better to generate representation of sentence by dividing queries, keys and values to different heads and get information from different subspaces 
Conclusion
In this paper, we propose an attention mechanism only based Chinese word segmentation model  Our model uses self-attention from the Transformer encoder to take sequence input and bi-affine attention scorer to predict the label of gaps  To improve the ability of capturing the localness and directional information of self-attention based encoder, we propose a variant of self-attention called Gaussian-masked directional multi-head attention to replace the standard self-attention  We also extend the Transformer encoder to capture directional features  Our model uses only unigram features instead of multiple $n$-gram features in previous work  Our model is evaluated on standard benchmark dataset, SIGHAN Bakeoff 2005, which shows not only our model performs segmentation faster than any previous models but also gives new higher or comparable segmentation performance against previous state-of-the-art models 
Context 2: Introduction
Natural text generation, as a key task in NLP, has been advanced substantially thanks to the flourish of neural models BIBREF0 , BIBREF1   Typical frameworks such as sequence-to-sequence (seq2seq) have been applied to various generation tasks, including machine translation BIBREF2 and dialogue generation BIBREF3   The standard paradigm to train such neural models is maximum likelihood estimation (MLE), which maximizes the log-likelihood of observing each word in the text given the ground-truth proceeding context BIBREF4  
Although widely used, MLE suffers from the exposure bias problem BIBREF5 , BIBREF6 : during test, the model sequentially predicts the next word conditioned on its previous generated words while during training conditioned on ground-truth words  To tackle this problem, generative adversarial networks (GAN) with reinforcement learning (RL) training approaches have been introduced to text generation tasks BIBREF7 , BIBREF8 , BIBREF9 , BIBREF10 , BIBREF11 , BIBREF12 , BIBREF13 , where the discriminator is trained to distinguish real and generated text samples to provide reward signals for the generator, and the generator is optimized via policy gradient BIBREF7  
However, recent studies have shown that potential issues of training GANs on discrete data are more severe than exposure bias BIBREF14 , BIBREF15   One of the fundamental issues when generating discrete text samples with GANs is training instability  Updating the generator with policy gradient always leads to an unstable training process because it's difficult for the generator to derive positive and stable reward signals from the discriminator even with careful pre-training BIBREF8   As a result, the generator gets lost due to the high variance of reward signals and the training process may finally collapse BIBREF16  
In this paper, we propose a novel adversarial training framework called Adversarial Reward Augmented Maximum Likelihood (ARAML) to deal with the instability issue of training GANs for text generation  At each iteration of adversarial training, we first train the discriminator to assign higher rewards to real data than to generated samples  Then, inspired by reward augmented maximum likelihood (RAML) BIBREF17 , the generator is updated on the samples acquired from a stationary distribution with maximum likelihood estimation (MLE), weighted by the discriminator's rewards  This stationary distribution is designed to guarantee that training samples are surrounding the real data, thus the exploration space of our generator is indeed restricted by the MLE training objective, resulting in more stable training  Compared to other text GANs with RL training techniques, our framework acquires samples from the stationary distribution rather than the generator's distribution, and uses RAML training paradigm to optimize the generator instead of policy gradient  Our contributions are mainly as follows:
Related Work
Recently, text generation has been widely studied with neural models trained with maximum likelihood estimation BIBREF4   However, MLE tends to generate universal text BIBREF18   Various methods have been proposed to enhance the generation quality by refining the objective function BIBREF18 , BIBREF19 or modifying the generation distribution with external information like topic BIBREF20 , sentence type BIBREF21 , emotion BIBREF22 and knowledge BIBREF23  
As mentioned above, MLE suffers from the exposure bias problem BIBREF5 , BIBREF6   Thus, reinforcement learning has been introduced to text generation tasks such as policy gradient BIBREF6 and actor-critic BIBREF24   BIBREF17 proposed an efficient and stable approach called Reward Augmented Maximum Likelihood (RAML), which connects the log-likelihood and expected rewards to incorporate MLE training objective into RL framework 
Since some text generation tasks have no explicit metrics to be directly optimized, adversarial training has been applied to generating discrete text samples with a discriminator to learn a proper reward  For instance, SeqGAN BIBREF7 devised a discriminator to distinguish the real data and generated samples, and a generator to maximize the reward from the discriminator via policy gradient  Other variants of GANs have been proposed to improve the generator or the discriminator  To improve the generator, MaliGAN BIBREF8 developed a normalized maximum likelihood optimization target for the generator to stably model the discrete sequences  LeakGAN BIBREF11 guided the generator with reward signals leaked from the discriminator at all generation steps to deal with long text generation task  MaskGAN BIBREF10 employed an actor-critic architecture to make the generator fill in missing text conditioned on the surrounding context, which is expected to mitigate the problem of mode collapse  As for the discriminator, RankGAN BIBREF9 replaced traditional discriminator with a ranker to learn the relative ranking information between the real texts and generated ones  Inverse reinforcement learning BIBREF12 used a trainable reward approximator as the discriminator to provide dense reward signals at each generation step  DPGAN BIBREF13 introduced a language model based discriminator and regarded cross-entropy as rewards to promote the diversity of generation results 
The most similar works to our model are RAML BIBREF17 and MaliGAN BIBREF8 : 1) Compared with RAML, our model adds a discriminator to learn the reward signals instead of choosing existing metrics as rewards  We believe that our model can adapt to various text generation tasks, particularly those without explicit evaluation metrics  2) Unlike MaliGAN, we acquire samples from a fixed distribution near the real data rather than the generator's distribution, which is expected to make the training process more stable 
Task Definition and Model Overview
Text generation can be formulated as follows: given the real data distribution INLINEFORM0 , the task is to train a generative model INLINEFORM1 where INLINEFORM2 can fit INLINEFORM3 well  In this formulation, INLINEFORM4 and INLINEFORM5 denotes a word in the vocabulary INLINEFORM6  
Figure FIGREF3 shows the overview of our model ARAML  This adversarial training framework consists of two phases: 1) The discriminator is trained to assign higher rewards to real data than to generated data  2) The generator is trained on the samples acquired from a stationary distribution with reward augmented MLE training objective  This training paradigm of the generator indeed constrains the search space with the MLE training objective, which alleviates the issue of unstable training 
Discriminator
The discriminator INLINEFORM0 aims to distinguish real data and generated data like other GANs  Inspired by Least-Square GAN BIBREF25 , we devise the loss function as follows: DISPLAYFORM0
This loss function forces the discriminator to assign higher rewards to real data than to generated data, so the discriminator can learn to provide more proper rewards as the training proceeds 
Generator
The training objective of our generator INLINEFORM0 is derived from the objective of other discrete GANs with RL training method: DISPLAYFORM0
where INLINEFORM0 denotes the rewards from the discriminator INLINEFORM1 and the entropy regularized term INLINEFORM2 encourages INLINEFORM3 to generate diverse text samples  INLINEFORM4 is a temperature hyper-parameter to balance these two terms 
As mentioned above, discrete GANs suffer from the instability issue due to policy gradient, thus they are consequently difficult to train  Inspired by RAML BIBREF17 , we introduce an exponential payoff distribution INLINEFORM0 to connect RL loss with RAML loss: DISPLAYFORM0
where INLINEFORM0   Thus, we can rewrite INLINEFORM1 with INLINEFORM2 and INLINEFORM3 as follows: DISPLAYFORM0
Following RAML, we remove the constant term and optimize the KL divergence in the opposite direction: DISPLAYFORM0
where INLINEFORM0 is a constant in the training phase of the generator  It has been proved that INLINEFORM1 and INLINEFORM2 are equivalent up to their first order Taylor approximations, and they have the same global optimum BIBREF17   INLINEFORM3 can be trained in a MLE-like fashion but sampling from the distribution INLINEFORM4 is intractable in the adversarial setting, because INLINEFORM5 varies with the discriminator INLINEFORM6   Thus, we introduce importance sampling to separate sampling process from INLINEFORM7 and obtain the final loss function: DISPLAYFORM0
where INLINEFORM0 denotes a stationary distribution and INLINEFORM1   To optimize this loss function, we first construct the fixed distribution INLINEFORM2 to get samples, and devise the proper reward function INLINEFORM3 to train the generator in a stable and effective way 
We construct the distribution INLINEFORM0 based on INLINEFORM1 : DISPLAYFORM0
In this way, INLINEFORM0 can be designed to guarantee that INLINEFORM1 is near INLINEFORM2 , leading to a more stable training process  To obtain a new sample INLINEFORM3 from a real data sample INLINEFORM4 , we can design three steps which contain sampling an edit distance INLINEFORM5 , the positions INLINEFORM6 for substitution and the new words INLINEFORM7 filled into the corresponding positions  Thus, INLINEFORM8 can be decomposed into three terms: DISPLAYFORM0
The first step is to sample an edit distance based on a real data sample INLINEFORM0 , where INLINEFORM1 is a sequence of length INLINEFORM2   The number of sentences which have the edit distance INLINEFORM3 to some input sentence can be computed approximately as below: DISPLAYFORM0
where INLINEFORM0 denotes the number of sentences which have an edit distance INLINEFORM1 to a sentence of length INLINEFORM2 , and INLINEFORM3 indicates the size of vocabulary  We then follow BIBREF17 to re-scale the counts by INLINEFORM4 and do normalization, so that we can sample an edit distance INLINEFORM5 from: DISPLAYFORM0
where INLINEFORM0 , as a temperature hyper-parameter, restricts the search space surrounding the original sentence  Larger INLINEFORM1 brings more samples with long edit distances 
The next step is to select positions for substitution based on the sampled edit distance INLINEFORM0   Intuitively, we can randomly choose INLINEFORM1 distinct positions in INLINEFORM2 to be replaced by new words  The probability of choosing the position INLINEFORM3 is calculated as follows: DISPLAYFORM0
Following this sampling strategy, we can obtain the position set INLINEFORM0   This strategy approximately guarantees that the edit distance between a new sentence and the original sentence is INLINEFORM1  
At the final step, our model determines new words for substitution at each sampled position INLINEFORM0   We can formulate this sampling process from the original sequence INLINEFORM1 to a new sample INLINEFORM2 as a sequential transition INLINEFORM3   At each step from INLINEFORM4 to INLINEFORM5 INLINEFORM6 , we first sample a new word INLINEFORM7 from the distribution INLINEFORM8 , then replace the old word at position INLINEFORM9 of INLINEFORM10 to obtain INLINEFORM11   The whole sampling process can be decomposed as follows: DISPLAYFORM0
There are two common sampling strategies to model INLINEFORM0 , i e  random sampling and constrained sampling  Random sampling strategy samples a new word INLINEFORM1 according to the uniform distribution over the vocabulary INLINEFORM2 BIBREF17 , while constrained sampling strategy samples INLINEFORM3 to maximize the language model score of the target sentence INLINEFORM4 BIBREF26 , BIBREF27   Here, we adopt constrained sampling in our model and compare the performances of two strategies in the experiment 
We devise the reward function INLINEFORM0 according to the discriminator's output INLINEFORM1 and the stationary distribution INLINEFORM2 : DISPLAYFORM0
Intuitively, this reward function encourages the generator to generate sentences with large sampling probability and high rewards from the discriminator  Thus, the weight of samples INLINEFORM0 can be calculated as follows: DISPLAYFORM0
So far, we can successfully optimize the generator's loss INLINEFORM0 via Equation EQREF12   This training paradigm makes our generator avoid possible variances caused by policy gradient and get more stable reward signals from the discriminator, because our generator is restricted to explore the training samples near the real data 
[htb] Adversarial Reward Augmented Maximum Likelihood [1]
Total adversarial training iterations: INLINEFORM0
Steps of training generator: INLINEFORM0
Steps of training discriminator: INLINEFORM0
Pre-train the generator INLINEFORM0 with MLE loss Generate samples from INLINEFORM1 Pre-train the discriminator INLINEFORM2 via Eq ( EQREF6 ) Construct INLINEFORM3 via Eq ( EQREF14 ) - Eq ( EQREF19 ) each INLINEFORM4 each INLINEFORM5 Update INLINEFORM6 via Eq ( EQREF12 ) each INLINEFORM7 Update INLINEFORM8 via Eq ( EQREF6 )
Extension to Conditional Text Generation
We have shown our adversarial training framework for text generation tasks without an input  Actually, it can also be extended to conditional text generation tasks like dialogue generation  Given the data distribution INLINEFORM0 where INLINEFORM1 denote contexts and responses respectively, the objective function of ARAML's generator can be modified as below: DISPLAYFORM0
where INLINEFORM0 and INLINEFORM1 is trained to distinguish whether INLINEFORM2 is the true response to INLINEFORM3  
Comparison with RAML and MaliGAN
The most similar works to our framework are RAML BIBREF17 and MaliGAN BIBREF8   The main difference among them is the training objective of their generators  We have shown different objective functions in Table TABREF26   For comparison, we use the form with no input for all the three models 
Our model is greatly inspired by RAML, which gets samples from a non-parametric distribution INLINEFORM0 constructed based on a specific reward  Compared to RAML, our reward comes from a learnable discriminator which varies as the adversarial training proceeds rather than a specific reward function  This difference equips our framework with the ability to adapt to the text generation tasks with no explicit evaluation metrics as rewards 
Our model is also similar to MaliGAN, which gets samples from the generator's distribution  In MaliGAN's training objective, INLINEFORM0 also indicates the generator's distribution but it's used in the sampling phase and fixed at each optimization step  The weight of samples INLINEFORM1   Different from our model, MaliGAN acquires samples from the generator's distribution INLINEFORM2 , which usually brings samples with low rewards even with careful pre-training for the generator, leading to training instability  Instead, our framework gets samples from a stationary distribution INLINEFORM3 around real data, thus our training process is more stable 
Datasets
We evaluated ARAML on three datasets: COCO image caption dataset BIBREF28 , EMNLP2017 WMT dataset and WeiboDial single-turn dialogue dataset BIBREF29 
Context 3: Introduction
Question answering (QA) has been a blooming research field for the last decade  Selection-based QA implies a family of tasks that find answer contexts from large data given questions in natural language  Three tasks have been proposed for selection-based QA  Given a document, answer extraction BIBREF0 , BIBREF1 finds answer phrases whereas answer selection BIBREF2 , BIBREF3 , BIBREF4 , BIBREF5 and answer triggering BIBREF6 , BIBREF7 find answer sentences instead, although the presence of the answer context is not assumed within the provided document for answer triggering but it is for the other two tasks  Recently, various QA tasks that are not selection-based have been proposed BIBREF8 , BIBREF9 , BIBREF10 , BIBREF11 ; however, selection-based QA remains still important because of its practical value to real applications (e g , IBM Watson, MIT Start) 
Several datasets have been released for selection-based QA  wang:07a created the QASent dataset consisting of 277 questions, which has been widely used for benchmarking the answer selection task  feng:15a presented InsuranceQA comprising 16K+ questions on insurance contexts  yang:15a introduced WikiQA for answer selection and triggering  jurczyk:16 created SelQA for large real-scale answer triggering  rajpurkar2016squad presented SQuAD for answer extraction and selection as well as for reading comprehension  Finally, morales-EtAl:2016:EMNLP2016 provided InfoboxQA for answer selection 
These corpora make it possible to evaluate the robustness of statistical question answering learning  Although all of these corpora target on selection-based QA, they are designed for different purposes such that it is important to understand the nature of these corpora so a better use of them can be made  In this paper, we make both intrinsic and extrinsic analyses of four latest corpora based on Wikipedia, WikiQA, SelQA, SQuAD, and InfoboxQA  We first give a thorough intrinsic analysis regarding contextual similarities, question types, and answer categories (Section SECREF2 )  We then map questions in all corpora to the current version of English Wikipedia and benchmark another selection-based QA task, answer retrieval (Section SECREF3 )  Finally, we present an extrinsic analysis through a set of experiments cross-testing these corpora using a convolutional neural network architecture (Section SECREF4 ) 
Intrinsic Analysis
Four publicly available corpora are selected for our analysis  These corpora are based on Wikipedia, so more comparable than the others, and have already been used for the evaluation of several QA systems 
WikiQA BIBREF6 comprises questions selected from the Bing search queries, where user click data give the questions and their corresponding Wikipedia articles  The abstracts of these articles are then extracted to create answer candidates  The assumption is made that if many queries lead to the same article, it must contain the answer context; however, this assumption fails for some occasions, which makes this dataset more challenging  Since the existence of answer contexts is not guaranteed in this task, it is called answer triggering instead of answer selection 
SelQA BIBREF7 is a product of five annotation tasks through crowdsourcing  It consists of about 8K questions where a half of the questions are paraphrased from the other half, aiming to reduce contextual similarities between questions and answers  Each question is associated with a section in Wikipedia where the answer context is guaranteed, and also with five sections selected from the entire Wikipedia where the selection is made by the Lucene search engine  This second dataset does not assume the existence of the answer context, so can be used for the evaluation of answer triggering 
SQuAD BIBREF12 presents 107K+ crowdsourced questions on 536 Wikipedia articles, where the answer contexts are guaranteed to exist within the provided paragraph  It contains annotation of answer phrases as well as the pointers to the sentences including the answer phrases; thus, it can be used for both answer extraction and selection  This corpus also provides human accuracy on those questions, setting up a reasonable upper bound for machines  To avoid overfitting, the evaluation set is not publicly available although system outputs can be evaluated by their provided script 
InfoboxQA BIBREF13 gives 15K+ questions based on the infoboxes from 150 articles in Wikipedia  Each question is crowdsourced and associated with an infobox, where each line of the infobox is considered an answer candidate  This corpus emphasizes the gravity of infoboxes, which summary arguably the most commonly asked information about those articles  Although the nature of this corpus is different from the others, it can also be used to evaluate answer selection 
Analysis
All corpora provide datasets/splits for answer selection, whereas only (WikiQA, SQuAD) and (WikiQA, SelQA) provide datasets for answer extraction and answer triggering, respectively  SQuAD is much larger in size although questions in this corpus are often paraphrased multiple times  On the contrary, SQuAD's average candidates per question ( INLINEFORM0 ) is the smallest because SQuAD extracts answer candidates from paragraphs whereas the others extract them from sections or infoboxes that consist of bigger contexts  Although InfoboxQA is larger than WikiQA or SelQA, the number of token types ( INLINEFORM1 ) in InfoboxQA is smaller than those two, due to the repetitive nature of infoboxes 
All corpora show similar average answer candidate lengths ( INLINEFORM0 ), except for InfoboxQA where each line in the infobox is considered a candidate  SelQA and SQuAD show similar average question lengths ( INLINEFORM1 ) because of the similarity between their annotation schemes  It is not surprising that WikiQA's average question length is the smallest, considering their questions are taken from search queries  InfoboxQA's average question length is relatively small, due to the restricted information that can be asked from the infoboxes  InfoboxQA and WikiQA show the least question-answer word overlaps over questions and answers ( INLINEFORM2 and INLINEFORM3 in Table TABREF2 ), respectively  In terms of the F1-score for overlapping words ( INLINEFORM4 ), SQuAD gives the least portion of overlaps between question-answer pairs although WikiQA comes very close 
Fig

ground_truth: pays attentions to adjacent characters and casts a localness relationship between the characters as a fixed Gaussian weight assuming the weight relies on the distance between characters

Gaussian-masked directional attention can be described as a function to map queries and key-value pairs to the representation of input, Gaussian-masked directional attention expects to pay attention to the adjacent characters of each positions and cast the localness relationship between characters as a fix Gaussian weight for attention, Gaussian weight only relys on the distance between characters

prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. How much improvement is gained from Adversarial Reward Augmented Maximum Likelihood (ARAML)?
Context 2:  COCO and EMNLP2017 WMT are the common benchmarks with no input to evaluate the performance of discrete GANs, and we followed the existing works to preprocess these datasets BIBREF12 , BIBREF11   WeiboDial, as a dialogue dataset, was applied to test the performance of our model with input trigger  We simply removed post-response pairs containing low-frequency words and randomly selected a subset for our training/test set  The statistics of three datasets are presented in Table TABREF28  
Baselines
We compared our model with MLE, RL and GAN baselines  Since COCO and EMNLP2017 WMT don't have input while WeiboDial regards posts as input, we chose the following baselines respectively:
MLE: a RNN model trained with MLE objective BIBREF4   Its extension, Seq2Seq, can work on the dialogue dataset BIBREF2  
SeqGAN: The first text GAN model that updates the generator with policy gradient based on the rewards from the discriminator BIBREF7  
LeakGAN: A variant of SeqGAN that provides rewards based on the leaked information of the discriminator for the generator BIBREF11  
MaliGAN: A variant of SeqGAN that optimizes the generator with a normalized maximum likelihood objective BIBREF8  
IRL: This inverse reinforcement learning method replaces the discriminator with a reward approximator to provide dense rewards BIBREF12  
RAML: A RL approach to incorporate MLE objective into RL training framework, which regards BLEU as rewards BIBREF17  
DialogGAN: An extension of SeqGAN tuned to dialogue generation task with MLE objective added to the adversarial objective BIBREF16  
DPGAN: A variant of DialogGAN which uses a language model based discriminator and regards cross-entropy as rewards BIBREF13  
Note that MLE, SeqGAN, LeakGAN, MaliGAN and IRL are the baselines on COCO and EMNLP2017 WMT, while MLE, RAML, DialogGAN, and DPGAN on WeiboDial  The original codes are used to test the baselines 
Implementation Details
The implementation details of our model are shown in Table TABREF31   For COCO / EMNLP2017, the generator is a LSTM unit BIBREF30 with 128 cells, and the discriminator is implemented based on BIBREF7   For WeiboDial, the generator is an encoder-decoder structure with attention mechanism, where both the encoder and the decoder consist of a two-layer GRU BIBREF31 with 128 cells  The discriminator is implemented based on BIBREF32   The language model used in the constrained sampling of ARAML is implemented in the same setting as the generators, and is pre-trained on the training set of each dataset  The codes and the datasets are available at https://github com/kepei1106/ARAML 
As for the details of the baselines, the generators of all the baselines except LeakGAN are the same as ours  Note that the generator of LeakGAN consists of a hierarchical LSTM unit, thus we followed the implementation in the original paper  In terms of the differences, the discriminators of GAN baselines are implemented based on the original papers  Other hyper-parameters of baselines including batch size, learning rate, and pre-training epochs, were set based on the original codes, because the convergence of baselines is sensitive to these hyper-parameters 
Language Generation on COCO and EMNLP2017 WMT
We adopted forward/reverse perplexity BIBREF33 and Self-BLEU BIBREF34 to evaluate the quality of generated texts  Forward perplexity (PPL-F) indicates the perplexity on the generated data provided by a language model trained on real data to measure the fluency of generated samples  Reverse perplexity (PPL-R) switches the roles of generated data and real data to reflect the discrepancy between the generated distribution and the data distribution  Self-BLEU (S-BLEU) regards each sentence in the generated collection as hypothesis and the others as reference to obtain BLEU scores, which evaluates the diversity of generated results 
Results are shown in Table TABREF33   LeakGAN performs best on forward perplexity because it can generate more fluent samples  As for reverse perplexity, our model ARAML beats other baselines, showing that our model can fit the data distribution better  Other GANs, particularly LeakGAN, obtain high reverse perplexity due to mode collapse BIBREF12 , thus they only capture limited fluent expressions, resulting in large discrepancy between the generated distribution and data distribution  ARAML also outperforms the baselines in terms of Self-BLEU, indicating that our model doesn't fall into mode collapse with the help of the MLE training objective and has the ability to generate more diverse sentences 
We also provide standard deviation of each metric in Table TABREF33 , reflecting the stability of each model's performance  Our model ARAML nearly achieves the smallest standard deviation in all the metrics, indicating that our framework outperforms policy gradient in the stability of adversarial training 
Dialogue Generation on WeiboDial
Dialogue evaluation is an open problem and existing works have found that automatic metrics have low correlation to human evaluation BIBREF35 , BIBREF36 , BIBREF37   Thus, we resorted to manual evaluation to assess the generation quality on WeiboDial  We randomly sampled 200 posts from the test set and collected the generated results from all the models  For each pair of responses (one from ARAML and the other from a baseline, given the same input post), five annotators were hired to label which response is better (i e  win, lose or tie) in terms of grammaticality (whether a response itself is grammatical and logical) and relevance (whether a response is appropriate and relevant to the post)  The two metrics were evaluated independently 
The evaluation results are shown in Table TABREF35   To measure the inter-annotator agreement, we calculated Fleiss' kappa BIBREF38 for each pair-wise comparison where results show moderate agreement ( INLINEFORM0 )  We also conducted sign test to check the significance of the differences 
As shown in Table TABREF35 , ARAML performs significantly better than other baselines in all the cases  This result indicates that the samples surrounding true responses provide stable rewards for the generator, and stable RAML training paradigm significantly enhances the performance in both metrics 
Further Analysis on Stability
To verify the training stability, we conducted experiments on COCO many times and chose the best 5 trials for SeqGAN, LeakGAN, IRL, MaliGAN and ARAML, respectively  Then, we presented the forward/reverse perplexity in the training process in Figure FIGREF38   We can see that our model with smaller standard deviation is more stable than other GAN baselines in both metrics  Although LeakGAN reaches the best forward perplexity, its standard deviation is extremely large and it performs badly in reverse perplexity, indicating that it generates limited expressions that are grammatical yet divergent from the data distribution 
Ablation Study
The temperature INLINEFORM0 controls the search space surrounding the real data as we analyze in Section UID13   To investigate its impact on the performance of our model, we fixed all the other hyper-parameters and test ARAML with different temperatures on COCO 
The experimental results are shown in Figure FIGREF41   We can see that as the temperature becomes larger, forward perplexity increases gradually while Self-BLEU decreases  As mentioned in Section UID13 , large temperatures encourage our generator to explore the samples that are distant from real data distribution, thus the diversity of generated results will be improved  However, these samples distant from the data distribution are more likely to be poor in fluency, leading to worse forward perplexity  Reverse perplexity is influenced by both generation quality and diversity, so the correlation between temperature and reverse perplexity is not intuitive  We can observe that the model with INLINEFORM0 reaches the best reverse perplexity 
We have mentioned two common sampling strategies in Section UID13 , i e  random sampling and constrained sampling  To analyze their impact, we keep all the model structures and hyper-parameters fixed and test ARAML with these two strategies on COCO 
Table TABREF45 shows the results  It's obvious that random sampling hurts the model performance except Self-BLEU-1, because it indeed allows low-quality samples available to the generator  Exploring these samples degrades the quality and diversity of generated results  Despite the worse performance on automatic metrics, random sampling doesn't affect the training stability of our framework  The standard deviation of ARAML-R is still smaller than other GAN baselines 
Case Study
Table TABREF47 presents the examples generated by the models on COCO  We can find that other baselines suffer from grammatical errors (e g  “in front of flying her kite" from MLE), repetitive expressions (e g  “A group of people" from IRL) and incoherent statements (e g  “A group of people sitting on a cell phone” from IRL)  By contrast, our model performs well in these sentences and has the ability to generate grammatical and coherent results 
Table TABREF48 shows the generated examples on WeiboDial  It's obvious that other baselines don't capture the topic word “late" in the post, thus generate irrelevant responses  ARAML can provide a response that is grammatical and closely relevant to the post 
Conclusion
We propose a novel adversarial training framework to deal with the instability problem of current GANs for text generation  To address the instability issue caused by policy gradient, we incorporate RAML into the advesarial training paradigm to make our generator acquire stable rewards  Experiments show that our model performs better than several state-of-the-art GAN baselines with lower training variance, yet producing better performance on three text generation tasks 
Acknowledgments
This work was supported by the National Science Foundation of China (Grant No  61936010/61876096) and the National Key R&D Program of China (Grant No  2018YFC0830200)  We would like to thank THUNUS NExT Joint-Lab for the support 

ground_truth: ARAM has achieved improvement over all baseline methods using reverese perplexity and  slef-BLEU metric.  The maximum reverse perplexity improvement 936,16 is gained for EMNLP2017  WMT  dataset and  48,44 for COCO dataset.

Compared to the baselines, ARAML does not do better in terms of perplexity on COCO and EMNLP 2017 WMT datasets, but it does by up to 0.27 Self-BLEU points on COCO and 0.35 Self-BLEU on EMNLP 2017 WMT. In terms of Grammaticality and Relevance, it scores better than the baselines on up to 75.5% and 73% of the cases respectively.

prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. Do they employ their indexing-based method to create a sample of a QA Wikipedia dataset?
Context 3:  FIGREF4 shows the distributions of seven question types grouped deterministically from the lexicons  Although these corpora have been independently developed, a general trend is found, where the what question type dominates, followed by how and who, followed by when and where, and so on 
Fig  FIGREF6 shows the distributions of answer categories automatically classified by our Convolutional Neural Network model trained on the data distributed by li:02a  Interestingly, each corpus focuses on different categories, Numeric for WikiQA and SelQA, Entity for SQuAD, and Person for InfoboxQA, which gives enough diversities for statistical learning to build robust models 
Answer Retrieval
This section describes another selection-based QA task, called answer retrieval, that finds the answer context from a larger dataset, the entire Wikipedia  SQuAD provides no mapping of the answer contexts to Wikipedia, whereas WikiQA and SelQA provide mappings; however, their data do not come from the same version of Wikipedia  We propose an automatic way of mapping the answer contexts from all corpora to the same version of Wikipeda so they can be coherently used for answer retrieval 
Each paragraph in Wikipedia is first indexed by Lucene using {1,2,3}-grams, where the paragraphs are separated by WikiExtractor and segmented by NLP4J (28 7M+ paragraphs are indexed)  Each answer sentence from the corpora in Table TABREF3 is then queried to Lucene, and the top-5 ranked paragraphs are retrieved  The cosine similarity between each sentence in these paragraphs and the answer sentence is measured for INLINEFORM0 -grams, say INLINEFORM1   A weight is assigned to each INLINEFORM2 -gram score, say INLINEFORM3 , and the weighted sum is measured: INLINEFORM4   The fixed weights of INLINEFORM5 are used for our experiments, which can be improved 
If there exists a sentence whose INLINEFORM0 , the paragraph consisting of that sentence is considered the silver-standard answer passage  Table TABREF3 shows how robust these silver-standard passages are based on human judgement ( INLINEFORM1 ) and how many passages are collected ( INLINEFORM2 ) for INLINEFORM3 , where the human judgement is performed on 50 random samples for each case  For answer retrieval, a dataset is created by INLINEFORM4 , which gives INLINEFORM5 accuracy and INLINEFORM6 coverage, respectively  Finally, each question is queried to Lucene and the top- INLINEFORM7 paragraphs are retrieved from the entire Wikipedia  If the answer sentence exists within those retrieved paragraphs according to the silver-standard, it is considered correct 
Finding a paragraph that includes the answer context out of the entire Wikipedia is an extremely difficult task (128 7M)  The last row of Table TABREF3 shows results from answer retrieval  Given INLINEFORM0 , SelQA and SQuAD show about 34% and 35% accuracy, which are reasonable  However, WikiQA shows a significantly lower accuracy of 12 47%; this is because the questions in WikiQA is about twice shorter than the questions in the other corpora such that not enough lexicons can be extracted from these questions for the Lucene search 
Answer Selection
Answer selection is evaluated by two metrics, mean average precision (MAP) and mean reciprocal rank (MRR)  The bigram CNN introduced by yu:14a is used to generate all the results in Table TABREF11 , where models are trained on either single or combined datasets  Clearly, the questions in WikiQA are the most challenging, and adding more training data from the other corpora hurts accuracy due to the uniqueness of query-based questions in this corpus  The best model is achieved by training on W+S+Q for SelQA; adding InfoboxQA hurts accuracy for SelQA although it gives a marginal gain for SQuAD  Just like WikiQA, InfoboxQA performs the best when it is trained on only itself  From our analysis, we suggest that to use models trained on WikiQA and InfoboxQA for short query-like questions, whereas to use ones trained on SelQA and SQuAD for long natural questions 
Answer Triggering
The results of INLINEFORM0 from the answer retrieval task in Section SECREF13 are used to create the datasets for answer triggering, where about 65% of the questions are not expected to find their answer contexts from the provided paragraphs for SelQA and SQuAD and 87 5% are not expected for WikiQA  Answer triggering is evaluated by the F1 scores as presented in Table TABREF11 , where three corpora are cross validated  The results on WikiQA are pretty low as expected from the poor accuracy on the answer retrieval task  Training on SelQA gives the best models for both WikiQA and SelQA  Training on SQuAD gives the best model for SQuAD although the model trained on SelQA is comparable  Since the answer triggering datasets are about 5 times larger than the answer selection datasets, it is computationally too expensive to combine all data for training  We plan to find a strong machine to perform this experiment in near future 
Related work
Lately, several deep learning approaches have been proposed for question answering  yu:14a presented a CNN model that recognizes the semantic similarity between two sentences  wang-nyberg:2015:ACL-IJCNLP presented a stacked bidirectional LSTM approach to read words in sequence, then outputs their similarity scores  feng:15a applied a general deep learning framework to non-factoid question answering  santos:16a introduced an attentive pooling mechanism that led to further improvements in selection-based QA 
Conclusion
We present a comprehensive comparison study of the existing corpora for selection-based question answering  Our intrinsic analysis provides a better understanding of the uniqueness or similarity between these corpora  Our extrinsic analysis shows the strength or weakness of combining these corpora together for statistical learning  Additionally, we create a silver-standard dataset for answer retrieval and triggering, which will be publicly available  In the future, we will explore different ways of improving the quality of our silver-standard datasets by fine-tuning the hyper-parameters 

ground_truth: Yes

No