prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. How much improvement is gained from Adversarial Reward Augmented Maximum Likelihood (ARAML)?
Context 1: Introduction
Natural text generation, as a key task in NLP, has been advanced substantially thanks to the flourish of neural models BIBREF0 , BIBREF1   Typical frameworks such as sequence-to-sequence (seq2seq) have been applied to various generation tasks, including machine translation BIBREF2 and dialogue generation BIBREF3   The standard paradigm to train such neural models is maximum likelihood estimation (MLE), which maximizes the log-likelihood of observing each word in the text given the ground-truth proceeding context BIBREF4  
Although widely used, MLE suffers from the exposure bias problem BIBREF5 , BIBREF6 : during test, the model sequentially predicts the next word conditioned on its previous generated words while during training conditioned on ground-truth words  To tackle this problem, generative adversarial networks (GAN) with reinforcement learning (RL) training approaches have been introduced to text generation tasks BIBREF7 , BIBREF8 , BIBREF9 , BIBREF10 , BIBREF11 , BIBREF12 , BIBREF13 , where the discriminator is trained to distinguish real and generated text samples to provide reward signals for the generator, and the generator is optimized via policy gradient BIBREF7  
However, recent studies have shown that potential issues of training GANs on discrete data are more severe than exposure bias BIBREF14 , BIBREF15   One of the fundamental issues when generating discrete text samples with GANs is training instability  Updating the generator with policy gradient always leads to an unstable training process because it's difficult for the generator to derive positive and stable reward signals from the discriminator even with careful pre-training BIBREF8   As a result, the generator gets lost due to the high variance of reward signals and the training process may finally collapse BIBREF16  
In this paper, we propose a novel adversarial training framework called Adversarial Reward Augmented Maximum Likelihood (ARAML) to deal with the instability issue of training GANs for text generation  At each iteration of adversarial training, we first train the discriminator to assign higher rewards to real data than to generated samples  Then, inspired by reward augmented maximum likelihood (RAML) BIBREF17 , the generator is updated on the samples acquired from a stationary distribution with maximum likelihood estimation (MLE), weighted by the discriminator's rewards  This stationary distribution is designed to guarantee that training samples are surrounding the real data, thus the exploration space of our generator is indeed restricted by the MLE training objective, resulting in more stable training  Compared to other text GANs with RL training techniques, our framework acquires samples from the stationary distribution rather than the generator's distribution, and uses RAML training paradigm to optimize the generator instead of policy gradient  Our contributions are mainly as follows:
Related Work
Recently, text generation has been widely studied with neural models trained with maximum likelihood estimation BIBREF4   However, MLE tends to generate universal text BIBREF18   Various methods have been proposed to enhance the generation quality by refining the objective function BIBREF18 , BIBREF19 or modifying the generation distribution with external information like topic BIBREF20 , sentence type BIBREF21 , emotion BIBREF22 and knowledge BIBREF23  
As mentioned above, MLE suffers from the exposure bias problem BIBREF5 , BIBREF6   Thus, reinforcement learning has been introduced to text generation tasks such as policy gradient BIBREF6 and actor-critic BIBREF24   BIBREF17 proposed an efficient and stable approach called Reward Augmented Maximum Likelihood (RAML), which connects the log-likelihood and expected rewards to incorporate MLE training objective into RL framework 
Since some text generation tasks have no explicit metrics to be directly optimized, adversarial training has been applied to generating discrete text samples with a discriminator to learn a proper reward  For instance, SeqGAN BIBREF7 devised a discriminator to distinguish the real data and generated samples, and a generator to maximize the reward from the discriminator via policy gradient  Other variants of GANs have been proposed to improve the generator or the discriminator  To improve the generator, MaliGAN BIBREF8 developed a normalized maximum likelihood optimization target for the generator to stably model the discrete sequences  LeakGAN BIBREF11 guided the generator with reward signals leaked from the discriminator at all generation steps to deal with long text generation task  MaskGAN BIBREF10 employed an actor-critic architecture to make the generator fill in missing text conditioned on the surrounding context, which is expected to mitigate the problem of mode collapse  As for the discriminator, RankGAN BIBREF9 replaced traditional discriminator with a ranker to learn the relative ranking information between the real texts and generated ones  Inverse reinforcement learning BIBREF12 used a trainable reward approximator as the discriminator to provide dense reward signals at each generation step  DPGAN BIBREF13 introduced a language model based discriminator and regarded cross-entropy as rewards to promote the diversity of generation results 
The most similar works to our model are RAML BIBREF17 and MaliGAN BIBREF8 : 1) Compared with RAML, our model adds a discriminator to learn the reward signals instead of choosing existing metrics as rewards  We believe that our model can adapt to various text generation tasks, particularly those without explicit evaluation metrics  2) Unlike MaliGAN, we acquire samples from a fixed distribution near the real data rather than the generator's distribution, which is expected to make the training process more stable 
Task Definition and Model Overview
Text generation can be formulated as follows: given the real data distribution INLINEFORM0 , the task is to train a generative model INLINEFORM1 where INLINEFORM2 can fit INLINEFORM3 well  In this formulation, INLINEFORM4 and INLINEFORM5 denotes a word in the vocabulary INLINEFORM6  
Figure FIGREF3 shows the overview of our model ARAML  This adversarial training framework consists of two phases: 1) The discriminator is trained to assign higher rewards to real data than to generated data  2) The generator is trained on the samples acquired from a stationary distribution with reward augmented MLE training objective  This training paradigm of the generator indeed constrains the search space with the MLE training objective, which alleviates the issue of unstable training 
Discriminator
The discriminator INLINEFORM0 aims to distinguish real data and generated data like other GANs  Inspired by Least-Square GAN BIBREF25 , we devise the loss function as follows: DISPLAYFORM0
This loss function forces the discriminator to assign higher rewards to real data than to generated data, so the discriminator can learn to provide more proper rewards as the training proceeds 
Generator
The training objective of our generator INLINEFORM0 is derived from the objective of other discrete GANs with RL training method: DISPLAYFORM0
where INLINEFORM0 denotes the rewards from the discriminator INLINEFORM1 and the entropy regularized term INLINEFORM2 encourages INLINEFORM3 to generate diverse text samples  INLINEFORM4 is a temperature hyper-parameter to balance these two terms 
As mentioned above, discrete GANs suffer from the instability issue due to policy gradient, thus they are consequently difficult to train  Inspired by RAML BIBREF17 , we introduce an exponential payoff distribution INLINEFORM0 to connect RL loss with RAML loss: DISPLAYFORM0
where INLINEFORM0   Thus, we can rewrite INLINEFORM1 with INLINEFORM2 and INLINEFORM3 as follows: DISPLAYFORM0
Following RAML, we remove the constant term and optimize the KL divergence in the opposite direction: DISPLAYFORM0
where INLINEFORM0 is a constant in the training phase of the generator  It has been proved that INLINEFORM1 and INLINEFORM2 are equivalent up to their first order Taylor approximations, and they have the same global optimum BIBREF17   INLINEFORM3 can be trained in a MLE-like fashion but sampling from the distribution INLINEFORM4 is intractable in the adversarial setting, because INLINEFORM5 varies with the discriminator INLINEFORM6   Thus, we introduce importance sampling to separate sampling process from INLINEFORM7 and obtain the final loss function: DISPLAYFORM0
where INLINEFORM0 denotes a stationary distribution and INLINEFORM1   To optimize this loss function, we first construct the fixed distribution INLINEFORM2 to get samples, and devise the proper reward function INLINEFORM3 to train the generator in a stable and effective way 
We construct the distribution INLINEFORM0 based on INLINEFORM1 : DISPLAYFORM0
In this way, INLINEFORM0 can be designed to guarantee that INLINEFORM1 is near INLINEFORM2 , leading to a more stable training process  To obtain a new sample INLINEFORM3 from a real data sample INLINEFORM4 , we can design three steps which contain sampling an edit distance INLINEFORM5 , the positions INLINEFORM6 for substitution and the new words INLINEFORM7 filled into the corresponding positions  Thus, INLINEFORM8 can be decomposed into three terms: DISPLAYFORM0
The first step is to sample an edit distance based on a real data sample INLINEFORM0 , where INLINEFORM1 is a sequence of length INLINEFORM2   The number of sentences which have the edit distance INLINEFORM3 to some input sentence can be computed approximately as below: DISPLAYFORM0
where INLINEFORM0 denotes the number of sentences which have an edit distance INLINEFORM1 to a sentence of length INLINEFORM2 , and INLINEFORM3 indicates the size of vocabulary  We then follow BIBREF17 to re-scale the counts by INLINEFORM4 and do normalization, so that we can sample an edit distance INLINEFORM5 from: DISPLAYFORM0
where INLINEFORM0 , as a temperature hyper-parameter, restricts the search space surrounding the original sentence  Larger INLINEFORM1 brings more samples with long edit distances 
The next step is to select positions for substitution based on the sampled edit distance INLINEFORM0   Intuitively, we can randomly choose INLINEFORM1 distinct positions in INLINEFORM2 to be replaced by new words  The probability of choosing the position INLINEFORM3 is calculated as follows: DISPLAYFORM0
Following this sampling strategy, we can obtain the position set INLINEFORM0   This strategy approximately guarantees that the edit distance between a new sentence and the original sentence is INLINEFORM1  
At the final step, our model determines new words for substitution at each sampled position INLINEFORM0   We can formulate this sampling process from the original sequence INLINEFORM1 to a new sample INLINEFORM2 as a sequential transition INLINEFORM3   At each step from INLINEFORM4 to INLINEFORM5 INLINEFORM6 , we first sample a new word INLINEFORM7 from the distribution INLINEFORM8 , then replace the old word at position INLINEFORM9 of INLINEFORM10 to obtain INLINEFORM11   The whole sampling process can be decomposed as follows: DISPLAYFORM0
There are two common sampling strategies to model INLINEFORM0 , i e  random sampling and constrained sampling  Random sampling strategy samples a new word INLINEFORM1 according to the uniform distribution over the vocabulary INLINEFORM2 BIBREF17 , while constrained sampling strategy samples INLINEFORM3 to maximize the language model score of the target sentence INLINEFORM4 BIBREF26 , BIBREF27   Here, we adopt constrained sampling in our model and compare the performances of two strategies in the experiment 
We devise the reward function INLINEFORM0 according to the discriminator's output INLINEFORM1 and the stationary distribution INLINEFORM2 : DISPLAYFORM0
Intuitively, this reward function encourages the generator to generate sentences with large sampling probability and high rewards from the discriminator  Thus, the weight of samples INLINEFORM0 can be calculated as follows: DISPLAYFORM0
So far, we can successfully optimize the generator's loss INLINEFORM0 via Equation EQREF12   This training paradigm makes our generator avoid possible variances caused by policy gradient and get more stable reward signals from the discriminator, because our generator is restricted to explore the training samples near the real data 
[htb] Adversarial Reward Augmented Maximum Likelihood [1]
Total adversarial training iterations: INLINEFORM0
Steps of training generator: INLINEFORM0
Steps of training discriminator: INLINEFORM0
Pre-train the generator INLINEFORM0 with MLE loss Generate samples from INLINEFORM1 Pre-train the discriminator INLINEFORM2 via Eq ( EQREF6 ) Construct INLINEFORM3 via Eq ( EQREF14 ) - Eq ( EQREF19 ) each INLINEFORM4 each INLINEFORM5 Update INLINEFORM6 via Eq ( EQREF12 ) each INLINEFORM7 Update INLINEFORM8 via Eq ( EQREF6 )
Extension to Conditional Text Generation
We have shown our adversarial training framework for text generation tasks without an input  Actually, it can also be extended to conditional text generation tasks like dialogue generation  Given the data distribution INLINEFORM0 where INLINEFORM1 denote contexts and responses respectively, the objective function of ARAML's generator can be modified as below: DISPLAYFORM0
where INLINEFORM0 and INLINEFORM1 is trained to distinguish whether INLINEFORM2 is the true response to INLINEFORM3  
Comparison with RAML and MaliGAN
The most similar works to our framework are RAML BIBREF17 and MaliGAN BIBREF8   The main difference among them is the training objective of their generators  We have shown different objective functions in Table TABREF26   For comparison, we use the form with no input for all the three models 
Our model is greatly inspired by RAML, which gets samples from a non-parametric distribution INLINEFORM0 constructed based on a specific reward  Compared to RAML, our reward comes from a learnable discriminator which varies as the adversarial training proceeds rather than a specific reward function  This difference equips our framework with the ability to adapt to the text generation tasks with no explicit evaluation metrics as rewards 
Our model is also similar to MaliGAN, which gets samples from the generator's distribution  In MaliGAN's training objective, INLINEFORM0 also indicates the generator's distribution but it's used in the sampling phase and fixed at each optimization step  The weight of samples INLINEFORM1   Different from our model, MaliGAN acquires samples from the generator's distribution INLINEFORM2 , which usually brings samples with low rewards even with careful pre-training for the generator, leading to training instability  Instead, our framework gets samples from a stationary distribution INLINEFORM3 around real data, thus our training process is more stable 
Datasets
We evaluated ARAML on three datasets: COCO image caption dataset BIBREF28 , EMNLP2017 WMT dataset and WeiboDial single-turn dialogue dataset BIBREF29   COCO and EMNLP2017 WMT are the common benchmarks with no input to evaluate the performance of discrete GANs, and we followed the existing works to preprocess these datasets BIBREF12 , BIBREF11   WeiboDial, as a dialogue dataset, was applied to test the performance of our model with input trigger  We simply removed post-response pairs containing low-frequency words and randomly selected a subset for our training/test set  The statistics of three datasets are presented in Table TABREF28  
Baselines
We compared our model with MLE, RL and GAN baselines  Since COCO and EMNLP2017 WMT don't have input while WeiboDial regards posts as input, we chose the following baselines respectively:
MLE: a RNN model trained with MLE objective BIBREF4   Its extension, Seq2Seq, can work on the dialogue dataset BIBREF2  
SeqGAN: The first text GAN model that updates the generator with policy gradient based on the rewards from the discriminator BIBREF7  
LeakGAN: A variant of SeqGAN that provides rewards based on the leaked information of the discriminator for the generator BIBREF11  
MaliGAN: A variant of SeqGAN that optimizes the generator with a normalized maximum likelihood objective BIBREF8  
IRL: This inverse reinforcement learning method replaces the discriminator with a reward approximator to provide dense rewards BIBREF12  
RAML: A RL approach to incorporate MLE objective into RL training framework, which regards BLEU as rewards BIBREF17  
DialogGAN: An extension of SeqGAN tuned to dialogue generation task with MLE objective added to the adversarial objective BIBREF16  
DPGAN: A variant of DialogGAN which uses a language model based discriminator and regards cross-entropy as rewards BIBREF13  
Note that MLE, SeqGAN, LeakGAN, MaliGAN and IRL are the baselines on COCO and EMNLP2017 WMT, while MLE, RAML, DialogGAN, and DPGAN on WeiboDial  The original codes are used to test the baselines 
Implementation Details
The implementation details of our model are shown in Table TABREF31   For COCO / EMNLP2017, the generator is a LSTM unit BIBREF30 with 128 cells, and the discriminator is implemented based on BIBREF7   For WeiboDial, the generator is an encoder-decoder structure with attention mechanism, where both the encoder and the decoder consist of a two-layer GRU BIBREF31 with 128 cells  The discriminator is implemented based on BIBREF32   The language model used in the constrained sampling of ARAML is implemented in the same setting as the generators, and is pre-trained on the training set of each dataset  The codes and the datasets are available at https://github com/kepei1106/ARAML 
As for the details of the baselines, the generators of all the baselines except LeakGAN are the same as ours  Note that the generator of LeakGAN consists of a hierarchical LSTM unit, thus we followed the implementation in the original paper  In terms of the differences, the discriminators of GAN baselines are implemented based on the original papers  Other hyper-parameters of baselines including batch size, learning rate, and pre-training epochs, were set based on the original codes, because the convergence of baselines is sensitive to these hyper-parameters 
Language Generation on COCO and EMNLP2017 WMT
We adopted forward/reverse perplexity BIBREF33 and Self-BLEU BIBREF34 to evaluate the quality of generated texts  Forward perplexity (PPL-F) indicates the perplexity on the generated data provided by a language model trained on real data to measure the fluency of generated samples  Reverse perplexity (PPL-R) switches the roles of generated data and real data to reflect the discrepancy between the generated distribution and the data distribution  Self-BLEU (S-BLEU) regards each sentence in the generated collection as hypothesis and the others as reference to obtain BLEU scores, which evaluates the diversity of generated results 
Results are shown in Table TABREF33   LeakGAN performs best on forward perplexity because it can generate more fluent samples  As for reverse perplexity, our model ARAML beats other baselines, showing that our model can fit the data distribution better  Other GANs, particularly LeakGAN, obtain high reverse perplexity due to mode collapse BIBREF12 , thus they only capture limited fluent expressions, resulting in large discrepancy between the generated distribution and data distribution  ARAML also outperforms the baselines in terms of Self-BLEU, indicating that our model doesn't fall into mode collapse with the help of the MLE training objective and has the ability to generate more diverse sentences 
We also provide standard deviation of each metric in Table TABREF33 , reflecting the stability of each model's performance  Our model ARAML nearly achieves the smallest standard deviation in all the metrics, indicating that our framework outperforms policy gradient in the stability of adversarial training 
Dialogue Generation on WeiboDial
Dialogue evaluation is an open problem and existing works have found that automatic metrics have low correlation to human evaluation BIBREF35 , BIBREF36 , BIBREF37   Thus, we resorted to manual evaluation to assess the generation quality on WeiboDial  We randomly sampled 200 posts from the test set and collected the generated results from all the models  For each pair of responses (one from ARAML and the other from a baseline, given the same input post), five annotators were hired to label which response is better (i e  win, lose or tie) in terms of grammaticality (whether a response itself is grammatical and logical) and relevance (whether a response is appropriate and relevant to the post)  The two metrics were evaluated independently 
The evaluation results are shown in Table TABREF35   To measure the inter-annotator agreement, we calculated Fleiss' kappa BIBREF38 for each pair-wise comparison where results show moderate agreement ( INLINEFORM0 )  We also conducted sign test to check the significance of the differences 
As shown in Table TABREF35 , ARAML performs significantly better than other baselines in all the cases  This result indicates that the samples surrounding true responses provide stable rewards for the generator, and stable RAML training paradigm significantly enhances the performance in both metrics 
Further Analysis on Stability
To verify the training stability, we conducted experiments on COCO many times and chose the best 5 trials for SeqGAN, LeakGAN, IRL, MaliGAN and ARAML, respectively  Then, we presented the forward/reverse perplexity in the training process in Figure FIGREF38   We can see that our model with smaller standard deviation is more stable than other GAN baselines in both metrics  Although LeakGAN reaches the best forward perplexity, its standard deviation is extremely large and it performs badly in reverse perplexity, indicating that it generates limited expressions that are grammatical yet divergent from the data distribution 
Ablation Study
The temperature INLINEFORM0 controls the search space surrounding the real data as we analyze in Section UID13   To investigate its impact on the performance of our model, we fixed all the other hyper-parameters and test ARAML with different temperatures on COCO 
The experimental results are shown in Figure FIGREF41   We can see that as the temperature becomes larger, forward perplexity increases gradually while Self-BLEU decreases  As mentioned in Section UID13 , large temperatures encourage our generator to explore the samples that are distant from real data distribution, thus the diversity of generated results will be improved  However, these samples distant from the data distribution are more likely to be poor in fluency, leading to worse forward perplexity  Reverse perplexity is influenced by both generation quality and diversity, so the correlation between temperature and reverse perplexity is not intuitive  We can observe that the model with INLINEFORM0 reaches the best reverse perplexity 
We have mentioned two common sampling strategies in Section UID13 , i e  random sampling and constrained sampling  To analyze their impact, we keep all the model structures and hyper-parameters fixed and test ARAML with these two strategies on COCO 
Table TABREF45 shows the results  It's obvious that random sampling hurts the model performance except Self-BLEU-1, because it indeed allows low-quality samples available to the generator  Exploring these samples degrades the quality and diversity of generated results  Despite the worse performance on automatic metrics, random sampling doesn't affect the training stability of our framework  The standard deviation of ARAML-R is still smaller than other GAN baselines 
Case Study
Table TABREF47 presents the examples generated by the models on COCO  We can find that other baselines suffer from grammatical errors (e g  “in front of flying her kite" from MLE), repetitive expressions (e g  “A group of people" from IRL) and incoherent statements (e g  “A group of people sitting on a cell phone” from IRL)  By contrast, our model performs well in these sentences and has the ability to generate grammatical and coherent results 
Table TABREF48 shows the generated examples on WeiboDial  It's obvious that other baselines don't capture the topic word “late" in the post, thus generate irrelevant responses  ARAML can provide a response that is grammatical and closely relevant to the post 
Conclusion
We propose a novel adversarial training framework to deal with the instability problem of current GANs for text generation  To address the instability issue caused by policy gradient, we incorporate RAML into the advesarial training paradigm to make our generator acquire stable rewards  Experiments show that our model performs better than several state-of-the-art GAN baselines with lower training variance, yet producing better performance on three text generation tasks 
Acknowledgments
This work was supported by the National Science Foundation of China (Grant No  61936010/61876096) and the National Key R&D Program of China (Grant No  2018YFC0830200)  We would like to thank THUNUS NExT Joint-Lab for the support 
Context 3: Introduction
Neural networks have been successfully used to describe images with text using sequence-to-sequence models BIBREF0  However, the results are simple and dry captions which are one or two phrases long  Humans looking at a painting see more than just objects  Paintings stimulate sentiments, metaphors and stories as well  Therefore, our goal is to have a neural network describe the painting artistically in a style of choice  As a proof of concept, we present a model which generates Shakespearean prose for a given painting as shown in Figure FIGREF1  Accomplishing this task is difficult with traditional sequence to sequence models since there does not exist a large collection of Shakespearean prose which describes paintings: Shakespeare's works describes a single painting shown in Figure FIGREF3  Fortunately we have a dataset of modern English poems which describe images BIBREF1 and line-by-line modern paraphrases of Shakespeare's plays BIBREF2  Our solution is therefore to combine two separately trained models to synthesize Shakespearean prose for a given painting 
Introduction ::: Related work
A general end-to-end approach to sequence learning BIBREF3 makes minimal assumptions on the sequence structure  This model is widely used in tasks such as machine translation, text summarization, conversational modeling, and image captioning  A generative model using a deep recurrent architecture BIBREF0 has also beeen used for generating phrases describing an image  The task of synthesizing multiple lines of poetry for a given image BIBREF1 is accomplished by extracting poetic clues from images  Given the context image, the network associates image attributes with poetic descriptions using a convolutional neural net  The poem is generated using a recurrent neural net which is trained using multi-adversarial training via policy gradient 
Transforming text from modern English to Shakespearean English using text "style transfer" is challenging  An end to end approach using a sequence-to-sequence model over a parallel text corpus BIBREF2 has been proposed based on machine translation  In the absence of a parallel text corpus, generative adversarial networks (GANs) have been used, which simultaneously train two models: a generative model which captures the data distribution, and a discriminative model which evaluates the performance of the generator  Using a target domain language model as a discriminator has also been employed BIBREF4, providing richer and more stable token-level feedback during the learning process  A key challenge in both image and text style transfer is separating content from style BIBREF5, BIBREF6, BIBREF7  Cross-aligned auto-encoder models have focused on style transfer using non-parallel text BIBREF7  Recently, a fine grained model for text style transfer has been proposed BIBREF8 which controls several factors of variation in textual data by using back-translation  This allows control over multiple attributes, such as gender and sentiment, and fine-grained control over the trade-off between content and style 
Methods
We use a total three datasets: two datasets for generating an English poem from an image, and Shakespeare plays and their English translations for text style transfer 
We train a model for generating poems from images based on two datasets BIBREF1  The first dataset consists of image and poem pairs, namely a multi-modal poem dataset (MultiM-Poem), and the second dataset is a large poem corpus, namely a uni-modal poem dataset (UniM-Poem)  The image and poem pairs are extended by adding the nearest three neighbor poems from the poem corpus without redundancy, and an extended image and poem pair dataset is constructed and denoted as MultiM-Poem(Ex)BIBREF1 
We use a collection of line-by-line modern paraphrases for 16 of Shakespeare’s plays BIBREF2, for training a style transfer network from English poems to Shakespearean prose  We use 18,395 sentences from the training data split  We keep 1,218 sentences in the validation data set and 1,462 sentences in our test set 
Methods ::: Image To Poem Actor-Critic Model
For generating a poem from images we use an existing actor-critic architecture BIBREF1  This involves 3 parallel CNNs: an object CNN, sentiment CNN, and scene CNN, for feature extraction  These features are combined with a skip-thought model which provides poetic clues, which are then fed into a sequence-to-sequence model trained by policy gradient with 2 discriminator networks for rewards  This as a whole forms a pipeline that takes in an image and outputs a poem as shown on the top left of Figure FIGREF4  A CNN-RNN generative model acts as an agent  The parameters of this agent define a policy whose execution determines which word is selected as an action  When the agent selects all words in a poem, it receives a reward  Two discriminative networks, shown on the top right of Figure FIGREF4, are defined to serve as rewards concerning whether the generated poem properly describes the input image and whether the generated poem is poetic  The goal of the poem generation model is to generate a sequence of words as a poem for an image to maximize the expected return 
Methods ::: Shakespearizing Poetic Captions
For Shakespearizing modern English texts, we experimented with various types of sequence to sequence models  Since the size of the parallel translation data available is small, we leverage a dictionary providing a mapping between Shakespearean words and modern English words to retrofit pre-trained word embeddings  Incorporating this extra information improves the translation task  The large number of shared word types between the source and target sentences indicates that sharing the representation between them is beneficial 
Methods ::: Shakespearizing Poetic Captions ::: Seq2Seq with Attention
We use a sequence-to-sequence model which consists of a single layer unidrectional LSTM encoder and a single layer LSTM decoder and pre-trained retrofitted word embeddings shared between source and target sentences  We experimented with two different types of attention: global attention BIBREF9, in which the model makes use of the output from the encoder and decoder for the current time step only, and Bahdanau attention BIBREF10, where computing attention requires the output of the decoder from the prior time step  We found that global attention performs better in practice for our task of text style transfer 
Methods ::: Shakespearizing Poetic Captions ::: Seq2Seq with a Pointer Network
Since a pair of corresponding Shakespeare and modern English sentences have significant vocabulary overlap we extend the sequence-to-sequence model mentioned above using pointer networks BIBREF11 that provide location based attention and have been used to enable copying of tokens directly from the input  Moreover, there are lot of proper nouns and rare words which might not be predicted by a vanilla sequence to sequence model 
Methods ::: Shakespearizing Poetic Captions ::: Prediction
For both seq2seq models, we use the attention matrices returned at each decoder time step during inference, to compute the next word in the translated sequence if the decoder output at the current time step is the UNK token  We replace the UNKs in the target output with the highest aligned, maximum attention, source word  The seq2seq model with global attention gives the best results with an average target BLEU score of 29 65 on the style transfer dataset, compared with an average target BLEU score of 26 97 using the seq2seq model with pointer networks 
Results
We perform a qualitative analysis of the Shakespearean prose generated for the input paintings  We conducted a survey, in which we presented famous paintings including those shown in Figures FIGREF1 and FIGREF10 and the corresponding Shakespearean prose generated by the model, and asked 32 students to rate them on the basis of content, creativity and similarity to Shakespearean style on a Likert scale of 1-5  Figure FIGREF12 shows the result of our human evaluation 
The average content score across the paintings is 3 7 which demonstrates that the prose generated is relevant to the painting  The average creativity score is 3 9 which demonstrates that the model captures more than basic objects in the painting successfully using poetic clues in the scene  The average style score is 3 9 which demonstrates that the prose generated is perceived to be in the style of Shakespeare 
We also perform a quantitative analysis of style transfer by generating BLEU scores for the model output using the style transfer dataset  The variation of the BLEU scores with the source sentence lengths is shown in Figure FIGREF11  As expected, the BLEU scores decrease with increase in source sentence lengths 
Results ::: Implementation
All models were trained on Google Colab with a single GPU using Python 3 6 and Tensorflow 2 0  The number of hidden units for the encoder and decoder is 1,576 and 256 for seq2seq with global attention and seq2seq with pointer networks respectively  Adam optimizer was used with the default learning rate of 0 001  The model was trained for 25 epochs  We use pre-trained retrofitted word embeddings of dimension 192 
Results ::: Limitations
Since we do not have an end-to-end dataset, the generated English poem may not work well with Shakespeare style transfer as shown in Figure FIGREF12 for "Starry Night" with a low average content score  This happens when the style transfer dataset does not have similar words in the training set of sentences  A solution would be to expand the style transfer dataset, for a better representation of the poem data 
Results ::: Conclusions and Future Work
In conclusion, combining two pipelines with an intermediate representation works well in practice  We observe that a CNN-RNN based image-to-poem net combined with a seq2seq model with parallel text corpus for text style transfer synthesizes Shakespeare-style prose for a given painting  For the seq2seq model used, we observe that it performs better in practice using global attention as compared with local attention  We make our models and code publicly available BIBREF12  In future work we would like to experiment with GANs in the absence of non-parallel datasets, so that we can use varied styles for text style transfer  We would also like to experiment with cross aligned auto-encoders, which form a latent content representation, to efficiently separate style and content 

ground_truth: ARAM has achieved improvement over all baseline methods using reverese perplexity and  slef-BLEU metric.  The maximum reverse perplexity improvement 936,16 is gained for EMNLP2017  WMT  dataset and  48,44 for COCO dataset.

Compared to the baselines, ARAML does not do better in terms of perplexity on COCO and EMNLP 2017 WMT datasets, but it does by up to 0.27 Self-BLEU points on COCO and 0.35 Self-BLEU on EMNLP 2017 WMT. In terms of Grammaticality and Relevance, it scores better than the baselines on up to 75.5% and 73% of the cases respectively.

prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. How many users do they look at?
Context 2: Introduction
Over the past two decades, the emergence of social media has enabled the proliferation of traceable human behavior  The content posted by users can reflect who their friends are, what topics they are interested in, or which company they are working for  At the same time, users are listing a number of profile fields to define themselves to others  The utilization of such metadata has proven important in facilitating further developments of applications in advertising BIBREF0 , personalization BIBREF1 , and recommender systems BIBREF2   However, profile information can be limited, depending on the platform, or it is often deliberately omitted BIBREF3   To uncloak this information, a number of studies have utilized social media users' footprints to approximate their profiles 
This paper explores the potential of predicting a user's industry –the aggregate of enterprises in a particular field– by identifying industry indicative text in social media  The accurate prediction of users' industry can have a big impact on targeted advertising by minimizing wasted advertising BIBREF4 and improved personalized user experience  A number of studies in the social sciences have associated language use with social factors such as occupation, social class, education, and income BIBREF5 , BIBREF6 , BIBREF7 , BIBREF8   An additional goal of this paper is to examine such findings, and in particular the link between language and occupational class, through a data-driven approach 
In addition, we explore how meaning changes depending on the occupational context  By leveraging word embeddings, we seek to quantify how, for example, cloud might mean a separate concept (e g , condensed water vapor) in the text written by users that work in environmental jobs while it might be used differently by users in technology occupations (e g , Internet-based computing) 
Specifically, this paper makes four main contributions  First, we build a large, industry-annotated dataset that contains over 20,000 blog users  In addition to their posted text, we also link a number of user metadata including their gender, location, occupation, introduction and interests 
Second, we build content-based classifiers for the industry prediction task and study the effect of incorporating textual features from the users' profile metadata using various meta-classification techniques, significantly improving both the overall accuracy and the average per industry accuracy 
Next, after examining which words are indicative for each industry, we build vector-space representations of word meanings and calculate one deviation for each industry, illustrating how meaning is differentiated based on the users' industries  We qualitatively examine the resulting industry-informed semantic representations of words by listing the words per industry that are most similar to job related and general interest terms 
Finally, we rank the different industries based on the normalized relative frequencies of emotionally charged words (positive and negative) and, in addition, discover that, for both genders, these frequencies do not statistically significantly correlate with an industry's gender dominance ratio 
After discussing related work in Section SECREF2 , we present the dataset used in this study in Section SECREF3   In Section SECREF4 we evaluate two feature selection methods and examine the industry inference problem using the text of the users' postings  We then augment our content-based classifier by building an ensemble that incorporates several metadata classifiers  We list the most industry indicative words and expose how each industrial semantic field varies with respect to a variety of terms in Section SECREF5   We explore how the frequencies of emotionally charged words in each gender correlate with the industries and their respective gender dominance ratio and, finally, conclude in Section SECREF6  
Related Work
Alongside the wide adoption of social media by the public, researchers have been leveraging the newly available data to create and refine models of users' behavior and profiling  There exists a myriad research that analyzes language in order to profile social media users  Some studies sought to characterize users' personality BIBREF9 , BIBREF10 , while others sequenced the expressed emotions BIBREF11 , studied mental disorders BIBREF12 , and the progression of health conditions BIBREF13   At the same time, a number of researchers sought to predict the social media users' age and/or gender BIBREF14 , BIBREF15 , BIBREF16 , while others targeted and analyzed the ethnicity, nationality, and race of the users BIBREF17 , BIBREF18 , BIBREF19   One of the profile fields that has drawn a great deal of attention is the location of a user  Among others, Hecht et al  Hecht11 predicted Twitter users' locations using machine learning on nationwide and state levels  Later, Han et al  Han14 identified location indicative words to predict the location of Twitter users down to the city level 
As a separate line of research, a number of studies have focused on discovering the political orientation of users BIBREF15 , BIBREF20 , BIBREF21   Finally, Li et al  Li14a proposed a way to model major life events such as getting married, moving to a new place, or graduating  In a subsequent study, BIBREF22 described a weakly supervised information extraction method that was used in conjunction with social network information to identify the name of a user's spouse, the college they attended, and the company where they are employed 
The line of work that is most closely related to our research is the one concerned with understanding the relation between people's language and their industry  Previous research from the fields of psychology and economics have explored the potential for predicting one's occupation from their ability to use math and verbal symbols BIBREF23 and the relationship between job-types and demographics BIBREF24   More recently, Huang et al  Huang15 used machine learning to classify Sina Weibo users to twelve different platform-defined occupational classes highlighting the effect of homophily in user interactions  This work examined only users that have been verified by the Sina Weibo platform, introducing a potential bias in the resulting dataset  Finally, Preotiuc-Pietro et al  Preoctiuc15 predicted the occupational class of Twitter users using the Standard Occupational Classification (SOC) system, which groups the different jobs based on skill requirements  In that work, the data collection process was limited to only users that specifically mentioned their occupation in their self-description in a way that could be directly mapped to a SOC occupational class  The mapping between a substring of their self-description and a SOC occupational class was done manually  Because of the manual annotation step, their method was not scalable; moreover, because they identified the occupation class inside a user self-description, only a very small fraction of the Twitter users could be included (in their case, 5,191 users) 
Both of these recent studies are based on micro-blogging platforms, which inherently restrict the number of characters that a post can have, and consequently the way that users can express themselves 
Moreover, both studies used off-the-shelf occupational taxonomies (rather than self-declared occupation categories), resulting in classes that are either too generic (e g , media, welfare and electronic are three of the twelve Sina Weibo categories), or too intermixed (e g , an assistant accountant is in a different class from an accountant in SOC)  To address these limitations, we investigate the industry prediction task in a large blog corpus consisting of over 20K American users, 40K web-blogs, and 560K blog posts 
Dataset
We compile our industry-annotated dataset by identifying blogger profiles located in the U S  on the profile finder on http://www blogger com, and scraping only those users that had the industry profile element completed 
For each of these bloggers, we retrieve all their blogs, and for each of these blogs we download the 21 most recent blog postings  We then clean these blog posts of HTML tags and tokenize them, and drop those bloggers whose cumulative textual content in their posts is less than 600 characters  Following these guidelines, we identified all the U S  bloggers with completed industry information 
Traditionally, standardized industry taxonomies organize economic activities into groups based on similar production processes, products or services, delivery systems or behavior in financial markets  Following such assumptions and regardless of their many similarities, a tomato farmer would be categorized into a distinct industry from a tobacco farmer  As demonstrated in Preotiuc-Pietro et al  Preoctiuc15 such groupings can cause unwarranted misclassifications 
The Blogger platform provides a total of 39 different industry options  Even though a completed industry value is an implicit text annotation, we acknowledge the same problem noted in previous studies: some categories are too broad, while others are very similar  To remedy this and following Guibert et al  Guibert71, who argued that the denominations used in a classification must reflect the purpose of the study, we group the different Blogger industries based on similar educational background and similar technical terminology  To do that, we exclude very general categories and merge conceptually similar ones  Examples of broad categories are the Education and the Student options: a teacher could be teaching in any concentration, while a student could be enrolled in any discipline  Examples of conceptually similar categories are the Investment Banking and the Banking options 
The final set of categories is shown in Table TABREF1 , along with the number of users in each category  The resulting dataset consists of 22,880 users, 41,094 blogs, and 561,003 posts  Table TABREF2 presents additional statistics of our dataset 
Text-based Industry Modeling
After collecting our dataset, we split it into three sets: a train set, a development set, and a test set  The sizes of these sets are 17,880, 2,500, and 2,500 users, respectively, with users randomly assigned to these sets  In all the experiments that follow, we evaluate our classifiers by training them on the train set, configure the parameters and measure performance on the development set, and finally report the prediction accuracy and results on the test set  Note that all the experiments are performed at user level, i e , all the data for one user is compiled into one instance in our data sets 
To measure the performance of our classifiers, we use the prediction accuracy  However, as shown in Table TABREF1 , the available data is skewed across categories, which could lead to somewhat distorted accuracy numbers depending on how well a model learns to predict the most populous classes  Moreover, accuracy alone does not provide a great deal of insight into the individual performance per industry, which is one of the main objectives in this study  Therefore, in our results below, we report: (1) micro-accuracy ( INLINEFORM0 ), calculated as the percentage of correctly classified instances out of all the instances in the development (test) data; and (2) macro-accuracy ( INLINEFORM1 ), calculated as the average of the per-category accuracies, where the per-category accuracy is the percentage of correctly classified instances out of the instances belonging to one category in the development (test) data 
Leveraging Blog Content
In this section, we seek the effectiveness of using solely textual features obtained from the users' postings to predict their industry 
The industry prediction baseline Majority is set by discovering the most frequently featured class in our training set and picking that class in all predictions in the respective development or testing set 
After excluding all the words that are not used by at least three separate users in our training set, we build our AllWords model by counting the frequencies of all the remaining words and training a multinomial Naive Bayes classifier  As seen in Figure FIGREF3 , we can far exceed the Majority baseline performance by incorporating basic language signals into machine learning algorithms (173% INLINEFORM0 improvement) 
We additionally explore the potential of improving our text classification task by applying a number of feature ranking methods and selecting varying proportions of top ranked features in an attempt to exclude noisy features  We start by ranking the different features, w, according to their Information Gain Ratio score (IGR) with respect to every industry, i, and training our classifier using different proportions of the top features  INLINEFORM0 INLINEFORM1
Even though we find that using the top 95% of all the features already exceeds the performance of the All Words model on the development data, we further experiment with ranking our features with a more aggressive formula that heavily promotes the features that are tightly associated with any industry category  Therefore, for every word in our training set, we define our newly introduced ranking method, the Aggressive Feature Ranking (AFR), as: INLINEFORM0
In Figure FIGREF3 we illustrate the performance of all four methods in our industry prediction task on the development data  Note that for each method, we provide both the accuracy ( INLINEFORM0 ) and the average per-class accuracy ( INLINEFORM1 )  The Majority and All Words methods apply to all the features; therefore, they are represented as a straight line in the figure  The IGR and AFR methods are applied to varying subsets of the features using a 5% step 
Our experiments demonstrate that the word choice that the users make in their posts correlates with their industry  The first observation in Figure FIGREF3 is that the INLINEFORM0 is proportional to INLINEFORM1 ; as INLINEFORM2 increases, so does INLINEFORM3   Secondly, the best result on the development set is achieved by using the top 90% of the features using the AFR method  Lastly, the improvements of the IGR and AFR feature selections are not substantially better in comparison to All Words (at most 5% improvement between All Words and AFR), which suggest that only a few noisy features exist and most of the words play some role in shaping the “language" of an industry 
As a final evaluation, we apply on the test data the classifier found to work best on the development data (AFR feature selection, top 90% features), for an INLINEFORM0 of 0 534 and INLINEFORM1 of 0 477 
Leveraging User Metadata
Together with the industry information and the most recent postings of each blogger, we also download a number of accompanying profile elements  Using these additional elements, we explore the potential of incorporating users' metadata in our classifiers 
Table TABREF7 shows the different user metadata we consider together with their coverage percentage (not all users provide a value for all of the profile elements)  With the exception of the gender field, the remaining metadata elements shown in Table TABREF7 are completed by the users as a freely editable text field  This introduces a considerable amount of noise in the set of possible metadata values  Examples of noise in the occupation field include values such as “Retired”, “I work ”, or “momma” which are not necessarily informative for our industry prediction task 
To examine whether the metadata fields can help in the prediction of a user's industry, we build classifiers using the different metadata elements  For each metadata element that has a textual value, we use all the words in the training set for that field as features  The only two exceptions are the state field, which is encoded as one feature that can take one out of 50 different values representing the 50 U S  states; and the gender field, which is encoded as a feature with a distinct value for each user gender option: undefined, male, or female 
As shown in Table TABREF9 , we build four different classifiers using the multinomial NB algorithm: Occu (which uses the words found in the occupation profile element), Intro (introduction), Inter (interests), and Gloc (combined gender, city, state) 
In general, all the metadata classifiers perform better than our majority baseline ( INLINEFORM0 of 18 88%)  For the Gloc classifier, this result is in alignment with previous studies BIBREF24   However, the only metadata classifier that outperforms the content classifier is the Occu classifier, which despite missing and noisy occupation values exceeds the content classifier's performance by an absolute 3 2% 
To investigate the promise of combining the five different classifiers we have built so far, we calculate their inter-prediction agreement using Fleiss's Kappa BIBREF25 , as well as the lower prediction bounds using the double fault measure BIBREF26   The Kappa values, presented in the lower left side of Table TABREF10 , express the classification agreement for categorical items, in this case the users' industry  Lower values, especially values below 30%, mean smaller agreement  Since all five classifiers have better-than-baseline accuracy, this low agreement suggests that their predictions could potentially be combined to achieve a better accumulated result 
Moreover, the double fault measure values, which are presented in the top-right hand side of Table TABREF10 , express the proportion of test cases for which both of the two respective classifiers make false predictions, essentially providing the lowest error bound for the pairwise ensemble classifier performance  The lower those numbers are, the greater the accuracy potential of any meta-classification scheme that combines those classifiers  Once again, the low double fault measure values suggest potential gain from a combination of the base classifiers into an ensemble of models 
After establishing the promise of creating an ensemble of classifiers, we implement two meta-classification approaches  First, we combine our classifiers using features concatenation (or early fusion)  Starting with our content-based classifier (Text), we successively add the features derived from each metadata element  The results, both micro- and macro-accuracy, are presented in Table TABREF12   Even though all these four feature concatenation ensembles outperform the content-based classifier in the development set, they fail to outperform the Occu classifier 
Second, we explore the potential of using stacked generalization (or late fusion) BIBREF27   The base classifiers, referred to as L0 classifiers, are trained on different folds of the training set and used to predict the class of the remaining instances  Those predictions are then used together with the true label of the training instances to train a second classifier, referred to as the L1 classifier: this L1 is used to produce the final prediction on both the development data and the test data  Traditionally, stacking uses different machine learning algorithms on the same training data  However in our case, we use the same algorithm (multinomial NB) on heterogeneous data (i e , different types of data such as content, occupation, introduction, interests, gender, city and state) in order to exploit all available sources of information 
The ensemble learning results on the development set are shown in Table TABREF12   We notice a constant improvement for both metrics when adding more classifiers to our ensemble except for the Gloc classifier, which slightly reduces the performance  The best result is achieved using an ensemble of the Text, Occu, Intro, and Inter L0 classifiers; the respective performance on the test set is an INLINEFORM0 of 0 643 and an INLINEFORM1 of 0 564  Finally, we present in Figure FIGREF11 the prediction accuracy for the final classifier for each of the different industries in our test dataset  Evidently, some industries are easier to predict than others  For example, while the Real Estate and Religion industries achieve accuracy figures above 80%, other industries, such as the Banking industry, are predicted correctly in less than 17% of the time  Anecdotal evidence drawn from the examination of the confusion matrix does not encourage any strong association of the Banking class with any other  The misclassifications are roughly uniform across all other classes, suggesting that the users in the Banking industry use language in a non-distinguishing way 
Qualitative Analysis
In this section, we provide a qualitative analysis of the language of the different industries 
Top-Ranked Words
To conduct a qualitative exploration of which words indicate the industry of a user, Table TABREF14 shows the three top-ranking content words for the different industries using the AFR method 
Not surprisingly, the top ranked words align well with what we would intuitively expect for each industry  Even though most of these words are potentially used by many users regardless of their industry in our dataset, they are still distinguished by the AFR method because of the different frequencies of these words in the text of each industry 
Industry-specific Word Similarities
Next, we examine how the meaning of a word is shaped by the context in which it is uttered  In particular, we qualitatively investigate how the speakers' industry affects meaning by learning vector-space representations of words that take into account such contextual information  To achieve this, we apply the contextualized word embeddings proposed by Bamman et al  Bamman14, which are based on an extension of the “skip-gram" language model BIBREF28  
In addition to learning a global representation for each word, these contextualized embeddings compute one deviation from the common word embedding representation for each contextual variable, in this case, an industry option  These deviations capture the terms' meaning variations (shifts in the INLINEFORM0 -dimensional space of the representations, where INLINEFORM1 in our experiments) in the text of the different industries, however all the embeddings are in the same vector space to allow for comparisons to one another 
Using the word representations learned for each industry, we present in Table TABREF16 the terms in the Technology and the Tourism industries that have the highest cosine similarity with a job-related word, customers  Similarly, Table TABREF17 shows the words in the Environment and the Tourism industries that are closest in meaning to a general interest word, food  More examples are given in the Appendix SECREF8  
The terms that rank highest in each industry are noticeably different  For example, as seen in Table TABREF17 , while food in the Environment industry is similar to nutritionally and locally, in the Tourism industry the same word relates more to terms such as delicious and pastries  These results not only emphasize the existing differences in how people in different industries perceive certain terms, but they also demonstrate that those differences can effectively be captured in the resulting word embeddings 
Emotional Orientation per Industry and Gender
As a final analysis, we explore how words that are emotionally charged relate to different industries  To quantify the emotional orientation of a text, we use the Positive Emotion and Negative Emotion categories in the Linguistic Inquiry and Word Count (LIWC) dictionary BIBREF29   The LIWC dictionary contains lists of words that have been shown to correlate with the psychological states of people that use them; for example, the Positive Emotion category contains words such as “happy,” “pretty,” and “good ”
For the text of all the users in each industry we measure the frequencies of Positive Emotion and Negative Emotion words normalized by the text's length  Table TABREF20 presents the industries' ranking for both categories of words based on their relative frequencies in the text of each industry 
We further perform a breakdown per-gender, where we once again calculate the proportion of emotionally charged words in each industry, but separately for each gender  We find that the industry rankings of the relative frequencies INLINEFORM0 of emotionally charged words for the two genders are statistically significantly correlated, which suggests that regardless of their gender, users use positive (or negative) words with a relative frequency that correlates with their industry  (In other words, even if e g , Fashion has a larger number of women users, both men and women working in Fashion will tend to use more positive words than the corresponding gender in another industry with a larger number of men users such as Automotive )
Finally, motivated by previous findings of correlations between job satisfaction and gender dominance in the workplace BIBREF30 , we explore the relationship between the usage of Positive Emotion and Negative Emotion words and the gender dominance in an industry  Although we find that there are substantial gender imbalances in each industry (Appendix SECREF9 ), we did not find any statistically significant correlation between the gender dominance ratio in the different industries and the usage of positive (or negative) emotional words in either gender in our dataset 
Conclusion
In this paper, we examined the task of predicting a social media user's industry  We introduced an annotated dataset of over 20,000 blog users and applied a content-based classifier in conjunction with two feature selection methods for an overall accuracy of up to 0 534, which represents a large improvement over the majority class baseline of 0 188 
We also demonstrated how the user metadata can be incorporated in our classifiers  Although concatenation of features drawn both from blog content and profile elements did not yield any clear improvements over the best individual classifiers, we found that stacking improves the prediction accuracy to an overall accuracy of 0 643, as measured on our test dataset  A more in-depth analysis showed that not all industries are equally easy to predict: while industries such as Real Estate and Religion are clearly distinguishable with accuracy figures over 0 80, others such as Banking are much harder to predict 
Finally, we presented a qualitative analysis to provide some insights into the language of different industries, which highlighted differences in the top-ranked words in each industry, word semantic similarities, and the relative frequency of emotionally charged words 
Acknowledgments
This material is based in part upon work supported by the National Science Foundation (#1344257) and by the John Templeton Foundation (#48503)  Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation or the John Templeton Foundation 
Additional Examples of Word Similarities

ground_truth: 22,880 users

20,000

prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. What models are used for painting embedding and what for language style transfer?
Context 4: 
Among the extractive evaluation metrics, CS, which also only relies on the reference, has poor correlation in the hyperparams category  RG-F, and both variants of the PARENT metric achieve the highest correlation for both settings  There is no significant difference among these for the hyperparams category, but for systems, PARENT-W is significantly better than the other two  While RG-F needs a full information extraction pipeline in its implementation, PARENT-C only relies on co-occurrence counts, and PARENT-W can be used out-of-the-box for any dataset  To our knowledge, this is the first rigorous evaluation of using information extraction for generation evaluation 
On this dataset, the word-overlap model showed higher correlation than the co-occurrence model for entailment  In § SECREF47 we will show that for the WebNLG dataset, where more paraphrasing is involved between the table and text, the opposite is true  Lastly, we note that the heuristic for selecting INLINEFORM0 is sufficient to produce high correlations for PARENT, however, if human annotations are available, this can be tuned to produce significantly higher correlations (PARENT*-W/C) 
Analysis
In this section we further analyze the performance of PARENT-W under different conditions, and compare to the other best metrics from Table TABREF37  
To study the correlation as we vary the number of divergent references, we also collected binary labels from workers for whether a reference is entailed by the corresponding table  We define a reference as entailed when it mentions only information which can be inferred from the table  Each table and reference pair was judged by 3 independent workers, and we used the majority vote as the label for that pair  Overall, only INLINEFORM0 of the references were labeled as entailed by the table  Fleiss' INLINEFORM1 was INLINEFORM2 , which indicates a fair agreement  We found the workers sometimes disagreed on what information can be reasonably entailed by the table 
Figure FIGREF40 shows the correlations as we vary the percent of entailed examples in the evaluation set of WikiBio  Each point is obtained by fixing the desired proportion of entailed examples, and sampling subsets from the full set which satisfy this proportion  PARENT and RG-F remain stable and show a high correlation across the entire range, whereas BLEU and BLEU-T vary a lot  In the hyperparams category, the latter two have the worst correlation when the evaluation set contains only entailed examples, which may seem surprising  However, on closer examination we found that this subset tends to omit a lot of information from the tables  Systems which produce more information than these references are penalized by BLEU, but not in the human evaluation  PARENT overcomes this issue by measuring recall against the table in addition to the reference 
We check how different components in the computation of PARENT contribute to its correlation to human judgments  Specifically, we remove the probability INLINEFORM0 of an n-gram INLINEFORM1 being entailed by the table from Eqs  EQREF19 and EQREF23   The average correlation for PARENT-W drops to INLINEFORM5 in this case  We also try a variant of PARENT with INLINEFORM6 , which removes the contribution of Table Recall (Eq  EQREF22 )  The average correlation is INLINEFORM7 in this case  With these components, the correlation is INLINEFORM8 , showing that they are crucial to the performance of PARENT 
BIBREF28 point out that hill-climbing on an automatic metric is meaningless if that metric has a low instance-level correlation to human judgments  In Table TABREF46 we show the average accuracy of the metrics in making the same judgments as humans between pairs of generated texts  Both variants of PARENT are significantly better than the other metrics, however the best accuracy is only INLINEFORM0 for the binary task  This is a challenging task, since there are typically only subtle differences between the texts  Achieving higher instance-level accuracies will require more sophisticated language understanding models for evaluation 
WebNLG Dataset
To check how PARENT correlates with human judgments when the references are elicited from humans (and less likely to be divergent), we check its correlation with the human ratings provided for the systems competing in the WebNLG challenge BIBREF6   The task is to generate text describing 1-5 RDF triples (e g  John E Blaha, birthPlace, San Antonio), and human ratings were collected for the outputs of 9 participating systems on 223 instances  These systems include a mix of pipelined, statistical and neural methods  Each instance has upto 3 reference texts associated with the RDF triples, which we use for evaluation 
The human ratings were collected on 3 distinct aspects – grammaticality, fluency and semantics, where semantics corresponds to the degree to which a generated text agrees with the meaning of the underlying RDF triples  We report the correlation of several metrics with these ratings in Table TABREF48   Both variants of PARENT are either competitive or better than the other metrics in terms of the average correlation to all three aspects  This shows that PARENT is applicable for high quality references as well 
While BLEU has the highest correlation for the grammar and fluency aspects, PARENT does best for semantics  This suggests that the inclusion of source tables into the evaluation orients the metric more towards measuring the fidelity of the content of the generation  A similar trend is seen comparing BLEU and BLEU-T  As modern neural text generation systems are typically very fluent, measuring their fidelity is of increasing importance  Between the two entailment models, PARENT-C is better due to its higher correlation with the grammaticality and fluency aspects 
The INLINEFORM0 parameter in the calculation of PARENT decides whether to compute recall against the table or the reference (Eq  EQREF22 )  Figure FIGREF50 shows the distribution of the values taken by INLINEFORM1 using the heuristic described in § SECREF3 for instances in both WikiBio and WebNLG  For WikiBio, the recall of the references against the table is generally low, and hence the recall of the generated text relies more on the table  For WebNLG, where the references are elicited from humans, this recall is much higher (often INLINEFORM2 ), and hence the recall of the generated text relies more on the reference 
Related Work
Over the years several studies have evaluated automatic metrics for measuring text generation performance BIBREF29 , BIBREF30 , BIBREF31 , BIBREF32 , BIBREF33 , BIBREF34 , BIBREF35   The only consensus from these studies seems to be that no single metric is suitable across all tasks  A recurring theme is that metrics like BLEU and NIST BIBREF36 are not suitable for judging content quality in NLG  Recently, BIBREF37 did a comprehensive study of several metrics on the outputs of state-of-the-art NLG systems, and found that while they showed acceptable correlation with human judgments at the system level, they failed to show any correlation at the sentence level  Ours is the first study which checks the quality of metrics when table-to-text references are divergent  We show that in this case even system level correlations can be unreliable 
Hallucination BIBREF38 , BIBREF39 refers to when an NLG system generates text which mentions extra information than what is present in the source from which it is generated  Divergence can be viewed as hallucination in the reference text itself  PARENT deals with hallucination by discounting n-grams which do not overlap with either the reference or the table 
PARENT draws inspiration from iBLEU BIBREF26 , a metric for evaluating paraphrase generation, which compares the generated text to both the source text and the reference  While iBLEU penalizes texts which match the source, here we reward such texts since our task values accuracy of generated text more than the need for paraphrasing the tabular content BIBREF40   Similar to SARI for text simplification BIBREF41 and Q-BLEU for question generation BIBREF42 , PARENT falls under the category of task-specific metrics 
Conclusions
We study the automatic evaluation of table-to-text systems when the references diverge from the table  We propose a new metric, PARENT, which shows the highest correlation with humans across a range of settings with divergent references in WikiBio  We also perform the first empirical evaluation of information extraction based metrics BIBREF1 , and find RG-F to be effective  Lastly, we show that PARENT is comparable to the best existing metrics when references are elicited by humans on the WebNLG data 
Acknowledgements
Bhuwan Dhingra is supported by a fellowship from Siemens, and by grants from Google  We thank Maruan Al-Shedivat, Ian Tenney, Tom Kwiatkowski, Michael Collins, Slav Petrov, Jason Baldridge, David Reitter and other members of the Google AI Language team for helpful discussions and suggestions  We thank Sam Wiseman for sharing data for an earlier version of this paper  We also thank the anonymous reviewers for their feedback 
Information Extraction System
For evaluation via information extraction BIBREF1 we train a model for WikiBio which accepts text as input and generates a table as the output  Tables in WikiBio are open-domain, without any fixed schema for which attributes may be present or absent in an instance  Hence we employ the Pointer-Generator Network (PG-Net) BIBREF19 for this purpose  Specifically, we use a sequence-to-sequence model, whose encoder and decoder are both single-layer bi-directional LSTMs  The decoder is augmented with an attention mechanism over the states of the encoder  Further, it also uses a copy mechanism to optionally copy tokens directly from the source text  We do not use the coverage mechanism of BIBREF19 since that is specific to the task of summarization they study  The decoder is trained to produce a linearized version of the table where the rows and columns are flattened into a sequence, and separate by special tokens  Figure FIGREF53 shows an example 
Clearly, since the references are divergent, the model cannot be expected to produce the entire table, and we see some false information being hallucinated after training  Nevertheless, as we show in § SECREF36 , this system can be used for evaluating generated texts  After training, we can parse the output sequence along the special tokens INLINEFORM0 R INLINEFORM1 and INLINEFORM2 C INLINEFORM3 to get a set of (attribute, value) pairs  Table TABREF54 shows the precision, recall and F-score of these extracted pairs against the ground truth tables, where the attributes and values are compared using an exact string match 
Hyperparameters
After tuning we found the same set of hyperparameters to work well for both the table-to-text PG-Net, and the inverse information extraction PG-Net  The hidden state size of the biLSTMs was set to 200  The input and output vocabularies were set to 50000 most common words in the corpus, with additional special symbols for table attribute names (such as “birth-date”)  The embeddings of the tokens in the vocabulary were initialized with Glove BIBREF43   Learning rate of INLINEFORM0 was used during training, with the Adam optimizer, and a dropout of INLINEFORM1 was also applied to the outputs of the biLSTM  Models were trained till the loss on the dev set stopped dropping  Maximum length of a decoded text was set to 40 tokens, and that of the tables was set to 120 tokens  Various beam sizes and length normalization penalties were applied for the table-to-text system, which are listed in the main paper  For the information extraction system, we found a beam size of 8 and no length penalty to produce the highest F-score on the dev set 
Sample Outputs
Table TABREF55 shows some sample references and the corresponding predictions from the best performing model, PG-Net for WikiBio 

ground_truth: generating a poem from images we use an existing actor-critic architecture, various types of sequence to sequence models