# CTGGAN: Controllable Text Generation with Generative Adversarial Network
## 3 Materials and Methods
### Proposed Method
If continuing with the BOLT [22] method to satisfy constraints, which utilizes an energy function during inference to gradually adjust energy values to be as small as possible, we cannot learn \(W(X,A_{i})\) in Equation (6). Moreover, if using the CLM methods which model the true probability as \(p=p_{LM}*p_{A_{1}}^{W(X,A_{1})}*...*p_{A_{n}}^{W(X,A_{n})}/Z\), large-scale negative sampling will be required to approximate the normalization value \(Z\). Therefore, we apply GANs to separate the \(logits_{b}\) network from the constraint combination network \(W(X,A_{i})*A_{i}\). This not only avoids the issue of extensive negative sampling but also, after the model has been trained, allows expeditious CTG by combining the \(logits_{b}\) network with the original output of the language model; namely, the efficiency of the decoding process would align perfectly with that of the original language model, without the need for additional operations such as negative sampling, making it more convenient. Specifically, we use a language model, e.g., GPT-2, with biased \(logits_{b}\) (modeled as Equation (7)) as the Generator part to generate text that satisfies the constraints combination. Additionally, the parameters of the original language model are frozen, and only those of the \(logits_{b}\) network participate in back-propagation updates. The Discriminator part is designed based on the contrast learning approach where the positive samples come from the training corpus itself, while the negative samples are obtained by sampling text from the Generator network. We employ \(W(X,A_{i})\) for samples' scores calculation. * **Generator and Optimization**: For the Generator part, i.e., \(logits=logits_{LM}+logits_{b}\), the probability of output text can be expressed as: \[ \begin{split} p(Y|X,A)&=softmax(logits_{LM_{1}}+logits_{b_{1}})*...\\ softmax(logits_{LM_{i}}+logits_{b_{i}})*...*softmax(logits_{LM_{n}}+logits_{b_{n}}).\end{split} \] (8) The goal of the Generator part is to make the distribution of generated text as close as possible to that of real corpora to deceive the Discriminator. In this context, greedy search is used to obtain the Generator's sampled result of current state, i.e., \[ \begin{split} Gen(p_{sample})&=max(softmax(logits_{LM_{1}}+logits_{b_{1}}))*...\\ max(softmax(logits_{LM_{i}}+logits_{b_{i}}))*...*max(softmax(logits_{LM_{n}}+logits_{b_{n}})).\end{split} \] (9) The text generated by \(argmax\) is denoted as \(Y_{gen}\). As \(argmax\) operator is not differentiable, the corresponding parameters cannot be updated according to the original GANs loss function. Therefore, we use the reinforcement learning approach (REINFORCE) [25] to redefine the loss function: \[\
abla Gen=\frac{1}{n}\sum(logp(Y_{gen}|X,A))* Reward(Y_{gen}).\] (10) Here, the reward function is calculated using \(sigmoid(score_{gen}-score_{true})\), where \(score_{gen}\) is the score of the generated sample \(Y_{gen}\), and \(score_{true}\) is the score of the real corpus data \(Y_{true}\). The scores are determined through the Discriminator's constraint combination, i.e., \(\sum W(X,A_{i})*A_{i}\). Nevertheless, using this loss function alone may lead to instability in GANs training, failing to achieve the expected generation effect or even disrupting the distribution quality of the original language model. Therefore, without affecting the GANs loss function, we add the language model's inherent loss function (Equation (3)) as a supplement to the Generator loss function: \[loss_{LM}=CE(p(Y|X,A),label)\] (11) where \(p(Y|X,A)\) is the language model with logits bias (Equation (7)) from the previous text, and label is the real corpus tokens. * **Discriminator and Optimization**: The Discriminator is responsible for scoring the qualities of texts, achieved through the combination of various constraint conditions. In this paper, for simplicity, two constraint conditions are set: positive sentiment and fluency.