user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: SALSA-TEXT : SELF ATTENTIVE LATENT SPACE BASED ADVERSARIAL TEXT GENERATION

Paper content:
Inspired by the success of self attention mechanism and Transformer architecture
in sequence transduction and image generation applications, we propose novel self
attention-based architectures to improve the performance of adversarial latent code-
based schemes in text generation.
Adversarial latent code-based text generation
has recently gained a lot of attention due to their promising results.
In this paper,
we take a step to fortify the architectures used in these setups, specifically AAE
and ARAE.
We benchmark two latent code-based methods (AAE and ARAE)
designed based on adversarial setups.
In our experiments, the Google sentence
compression dataset is utilized to compare our method with these methods using
various objective and subjective measures.
The experiments demonstrate the
proposed (self) attention-based models outperform the state-of-the-art in adversarial
code-based text generation.
Text generation is of particular interest in many natural language processing (NLP) applications such as dialogue systems, machine translation, image captioning and text summarization.
Recent deep learning-based approaches to this problem can be categorized into three classes: auto-regressive or maximum likelihood estimation (MLE)-based, generative adversarial network (GAN)-based and reinforcement learning (RL)-based approaches.
BID26 ) model the text (language) as an auto-regressive process, commonly using RNNs.
RNNs compactly represent the samples history in the form of recurrent states.
In these models, text is generated by predicting next token (character, word, etc) based on the previously generated ones BID9 .
The results of objective and subjective evaluations are presented in Tables 2 to 5.
As seen, the proposed self attention-based (SALSA) architectures consistently outperform the non-attention-based benchmarks in terms of diversity (measured by reverse perplexity).
Moreover, they often show better performance in terms of output quality (measured by BLEU, self BLEU, preplexity and human evaluations) on the long and complicated sentences of the GSC dataset.As seen in the generated samples TAB0 , human evaluation TAB3 ) and objective metrics (Tables 2 to 4), the original AAE and ARAE setups perform very poorly on GSC with long sentences.
With reverse perplexities of over 8000 and high self-BLEU scores close to 0.9, they suffer from a high level of mode collapse (repeated sentences).Human
evaluations do not account for lack of diversity. The reason
is humans are presented with a number of shuffled sentences and asked to evaluate them independently (without knowing which sentence coming from which model). Hence, in
our experiments for the original AAE and ARAE, a model can generate similar sentences (maybe due to mode collapse) and still receives high subjective scores. It seems
that, in our experiments, the original ARAE model suffers from mode collapse. We can see
that it has slightly higher human evaluation scores, but extremely poor diversity metrics, i.e. very high reverse perplexity and self-BLEU scores. It can also
be seen in the randomly selected generated sentences TAB0 , where all the sentences start with "A man" and invariably mention he is being arrested or accused of grievous crimes. This is likely
because the sentences in the GSC dataset are long and that their structure is elaborate. SALSA-ARAE on
the other hand reliably produces sentences of quality with great diversity.SALSA-AAE has both considerably higher individual quality metrics than the original AAE and much better diversity metrics. It is the strongest
pure adversarial text model. As seen in TAB3 , SALSA-AAE
provides the best grammaticality, semantic consistency and Fluency performance.
In this paper, we introduced SALSA-TEXT, a Transformer-based architecture for adversarial codebased text generation.
It incorporates self-attention mechanism by utilizing Transformer architecture in autoencoder and GAN setups.
Our extensive experiments demonstrate the better performance of our models compared to the state-of-the-art in adversarial code-based text generation (without self-attention).
The proposed architectures provide diverse, long and high quality output sentences as confirmed by objective metrics and human evaluations in extensive experiments.As a future direction, it is beneficial to study the performance of self attention in other text generation methods including variational code-based and reinforcement learning-based approaches.
Another interesting direction is to experiment with deeper Transformer-based autoencoders to better capture the underlying language model and perform unsupervised pre-training isnpired by the success of BID0 and Radford et al..

assistant: text: We propose a self-attention based GAN architecture for unconditional text generation and improve on previous adversarial code-based results.