# CTGGAN: Controllable Text Generation with Generative Adversarial Network
## 2 Related Work  
There are two categories of algorithms to solve the CTG problem: conditional language models and decoding-time methods. * **Conditional Language Models**: The conditional language models (CLMs) can be achieved with different implementations. For instance:CTLR model [7] splices control codes with the text, and expects to learn the association between the generated text and the control codes through the language model. The control codes usually indicate a particular domain of the training sample, e.g., \"Books\", \"Reviews\", etc., or more complex, such as \"Reviews Rating:5.0\". The model used in CTLR is exactly a language model without any improvement. GSUM [8] encodes the control codes (i.e., the guidance signals mentioned in the paper) and the texts separately with a shared transformer encoder. During decoding, both features of the control codes and the texts are used for attention calculations. Concretely, they are successively introduced into two consecutive cross-attention structures to complete the guided abstractive summarization. GeDi [9] trains a conditional language model to guide another language model, e.g., GPT-2, to generate controlled texts. It uses a pair of opposite control codes, such as \"positive\" and \"negative\", to obtain the discriminative probabilities on both sentiments. When decoding, the Bayes rule is employed to get the true probabilities of tokens under the specific constraint. Attribute alignment [10] holds the idea that adding control codes ahead of texts may break the originally learned sequential dependencies; therefore, it learns a projection function for both the key and value features of the control codes (which are derived from a language model) and fuses them into the key/value representations of the text in the corresponding layer for a modified attention calculation. Contrastive Prefixes [11] uses a categorical distribution to map the text to attribute category \(Z\), and further learns a projection network to represent it as \(H(Z)\) as attribute prefix representation. For decoding, the prefix representation is attached ahead of the text feature so as to generate controlled results. [PERSON] [12] and [PERSON] [13] employs samples with attribute labels to train a VAE model which maps the attributes to latent space \(Z\). When generating, it first obtains the appropriate \(z\) through the ODE diffusion process, and finally uses the language model, i.e., the VAE decoder, to generate texts that satisfy the constraints. [PERSON] et al. [14] trains an auto-encoder framework where the encoder projects a sentence into latent space and the decoder, i.e., a fixed language model, reconstructs the same sentence with the latent values. For controllable generation, it establishes links between the latent space and a prior space (which obeys the Gaussian distribution) using normalizing flow and samples from the prior space to derive the final CTG results. * **Decoding-time Methods**: Different from CLMs, decoding-time methods require no pre-trained language model training. PPLM [15] uses an attribute model \(p(a|x)\) which is learned from an autoregressive LM's top-level hidden layer to measure current context and modify the LM's gradient via gradient ascent with increasing the probability of the desired attribute at each step. In case of problems of repetitive and low-quality texts, the authors compute the mean over the Dist-N (\(N\in\{1,2,3\}\)) scores and discard samples with a mean score that is less than the preset threshold. Based on Bayesian factorization, [PERSON] [16] multiplies the output probability of the original language model with that of the attribute discriminator (i.e., a constraint classifier) to re-rank the possibility of the next token. At each step, [PERSON] first samples 200 top tokens according to the language model's output, and for each token calculates its probability (the current context) with a discriminator.