Question: Which model is most similar to the authors'?
Evidence: Recently, text generation has been widely studied with neural models trained with maximum likelihood estimation BIBREF4 . However, MLE tends to generate universal text BIBREF18 . Various methods have been proposed to enhance the generation quality by refining the objective function BIBREF18 , BIBREF19 or modifying the generation distribution with external information like topic BIBREF20 , sentence type BIBREF21 , emotion BIBREF22 and knowledge BIBREF23 .
As mentioned above, MLE suffers from the exposure bias problem BIBREF5 , BIBREF6 . Thus, reinforcement learning has been introduced to text generation tasks such as policy gradient BIBREF6 and actor-critic BIBREF24 . BIBREF17 proposed an efficient and stable approach called Reward Augmented Maximum Likelihood (RAML), which connects the log-likelihood and expected rewards to incorporate MLE training objective into RL framework.
Since some text generation tasks have no explicit metrics to be directly optimized, adversarial training has been applied to generating discrete text samples with a discriminator to learn a proper reward. For instance, SeqGAN BIBREF7 devised a discriminator to distinguish the real data and generated samples, and a generator to maximize the reward from the discriminator via policy gradient. Other variants of GANs have been proposed to improve the generator or the discriminator. To improve the generator, MaliGAN BIBREF8 developed a normalized maximum likelihood optimization target for the generator to stably model the discrete sequences. LeakGAN BIBREF11 guided the generator with reward signals leaked from the discriminator at all generation steps to deal with long text generation task. MaskGAN BIBREF10 employed an actor-critic architecture to make the generator fill in missing text conditioned on the surrounding context, which is expected to mitigate the problem of mode collapse. As for the discriminator, RankGAN BIBREF9 replaced traditional discriminator with a ranker to learn the relative ranking information between the real texts and generated ones. Inverse reinforcement learning BIBREF12 used a trainable reward approximator as the discriminator to provide dense reward signals at each generation step. DPGAN BIBREF13 introduced a language model based discriminator and regarded cross-entropy as rewards to promote the diversity of generation results.
The most similar works to our model are RAML BIBREF17 and MaliGAN BIBREF8 : 1) Compared with RAML, our model adds a discriminator to learn the reward signals instead of choosing existing metrics as rewards. We believe that our model can adapt to various text generation tasks, particularly those without explicit evaluation metrics. 2) Unlike MaliGAN, we acquire samples from a fixed distribution near the real data rather than the generator's distribution, which is expected to make the training process more stable.
The most similar works to our model are RAML (Norouzi et al., 2016) and MaliGAN (Che et al., 2017): 1) Compared with RAML, our model adds a discriminator to learn the reward signals instead of choosing existing metrics as rewards. We believe that our model can adapt to various text gen_x0002_eration tasks, particularly those without explicit evaluation metrics.
Answer: The most similar works to the author's model are RAML and MaliGAN.