A Causal Lens for Controllable Text Generation
Authors: Zhiting Hu
Li Erran Li
Venue: NeurIPS 2021 Poster (2021)
Keywords: causal inference
controllable text generation
text style transfer
natural language processing

Abstract: Controllable text generation concerns two fundamental tasks of wide applications, namely generating text of given attributes (i.e., attribute-conditional generation), and minimally editing existing text to possess desired attributes (i.e., text attribute transfer). Extensive prior work has largely studied the two problems separately, and developed different conditional models which, however, are prone to producing biased text (e.g., various gender stereotypes). This paper proposes to formulate controllable text generation from a principled causal perspective which models the two tasks with a unified framework. A direct advantage of the causal formulation is the use of  rich causality tools to mitigate generation biases and improve control. We treat the two tasks as interventional and counterfactual causal inference based on a structural causal model, respectively. We then apply the framework to the challenging practical setting where confounding factors (that induce spurious correlations) are observable only on a small fraction of data. Experiments show significant superiority of the causal approach over previous conditional models for improved control accuracy and reduced bias.

A Causal Lens for Controllable Text Generation
Zhiting Hu Li Erran Li
'UC San Diego, "AWS AI, Amazon
2hh019Qucsd.eed edu, Lilimam@amazon.com com
Abstract
Controllable text generation concerns two fundamental tasks of wide applications, namely generating text of given attributes (i.e., attribute-conditional generation), and minimally editing existing text to possess desired attributes (i.e., text attribute transfer). Extensive prior work has largely studied the problems separately, and developed different conditional models which, however, are prone to producing biased text (c.g., various gender stereotypes). This paper proposes to formulate controllable text generation from a principled causal perspective which models the two tasks with a unified framework. direct advantage of the causal formulation is the use of rich causality tools to mitigate generation biases and improve control. We treat the two tasks as interventional and counterfactual causal inference based on a structural causal model, respectively. We then apply the framework to the challenging practical setting where confounding factors (that induce spurious correlations) are observable only on a small fraction of data. Experiments show significant superiority of the causal approach over previous conditional models for improved control accuracy and reduced bias.
Introduction
Controllable text generation aims at producing fluent language with control over various attributes,
ranging from sentiment, topic, politeness, to gender, persona, and so forth (60 22]. The problem lies at the heart of many NI.P applications such as emotional chatbot, news article writing, language detoxification, etc. Of particular interest in this increasingly significant area are two settings for control, namely (1)
generation [14] 30) which generates sentences that entail given attribute, and (2) text attribute transfer [64] (26] which rewrites given sentence to possess a desired attribute while preserving all other original characteristics (Figure D. The goal is to learn the control in each setting with (attribute, text) training pairs]
The two settings have usually been considered as separate tasks and each led to various solutions, respectively. Let æ denote sentence and an attribute. Previous courbbtee-oodittoon generation work typically concerns the conditional distribution p(æ]uu) [14
Despite the success of simulating observed real text, the conditional distribution known to be susceptible to capture spurious correlations or biases in the data [48] (84). For example, when generating biographical text given a gender attribute, the conditional model tends to generate text related to specific occupations such nurse and yoga teacher for female, and architect and attorney for male [57] 69) (Figure []. The learned biases could impair the model generalization to new domains, and make negative social impact in downstream applications. A few very recent attempts have been made mitigate the biases in the model with various machine learning techniques. Yet those methods are often specific to particular attribute (e.g., gender) [68 [6). rely access to additional resources, such as fully observed confounding labels or a priori debiased classifiers 23 40, 67], which can be costly obtain in real applications. Furthermore, it is unclear how the diverse methods designed for
30
33
TThus, for text attribute transfer (a.k.a., style transfer), there is direct supervision data, i.e., (original text, attribute, target text) triples.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

generation could also be applied to debias text attribute transfer that has been formulated with distinct training objectives.
This paper studies controllable text generation from principled causal perspective, that of- fers a unifying formulation of the two central tasks, and enables mitigation of spurious correla- tions with well-established causality techniques, growing number of recent work has used causality with machine learning [62 for disen- tangled representation (5)778) model explana- tion 6 [3], and robust prediction [61] [24] [3). Yet most approaches have focused on the vi- sion domain, taking advantage of image spatial structures, and thus are not directly applicable to text with abstract attributes (such as sentiment), Though previous research on text modeling has also studied related concepts such as counterfac- tuals, it either handles only correlation instead of causation [58 39 76 23
Attribute transfer (ours)
Counterfactual
She in women's basketball team ...
and mother of two girls.
p(x| [x.))))
"male"
He is in men's basketball team and a father girls.
Attribute-conditoon gen. (ours)
Intervention
p(X do(a))
Army worked as lawyer in Toranto, where she focuses on privacy issuc.
female
Attribute-conditionaa gen. (conventional)
Association ((||a)
female
She previously worked as nurse practitioner in
male
went to law school and
86 [66], focuses on different applications such as data augmenta- tion 885 28 82 and classification [3] 29]. We
became a plaintifs'' attorney.
The causal ladder (56) and the formulations of controllable generation tasks corresponding to different rungs of the ladder.
Figure 1:
discuss more related work in $
We develop the first unified causal framework for text generation under control. In particular, we devise a structural causal model (SCM) [56] that describes the causal relationships between different variables, where the text æ is outcome and the attribute a to control (e.g., sentiment) is treatment. The SCM further accounts for spurious correlations with confounders (e.g.. category) with latent variables. The resulting SCM enables us to formulate the two control tasks as performing causal inference at different rungs of the causal ladder go beyond
(56) (Figure [T) respectively. Specifically, (1) for sttribue-cooddition the association- based conditional p( (a) and propose to instead use p(æ) (a)) corresponding to the
generation,
the intervention rung. The do-operation effectively eliminates the effect of confounders control, leading to unbiased text outputs: (2) for text attribute transfer, the task naturally maps to counterfactual prediction the SCM, which answers the question "what the text would have been if the attribute had been different" through the standard causal inference procedure [56]] (55). The unifying perspective also allows to draw from existing successful techniques and train the SCM for accurate control and confounder balancing
Previous causal work typically assumes access to confounding labels relevant proxy information for the entire observed data [43] 44 48). In many real applications, however, it is prohibitively expensive impossible to measure all the confounding factors for unbiased training. For example, is often not affordable to annotate massively the confounding labels for the entire (attribute, text) corpus. We thus consider more practical yet challenging scenario where observe confounding information for only a small subset (e.g.. 1% - 5%) of samples [5). We experiment on difficult datasets where the target attributes and confounding factors have strong correlations. Results show the causal approach substantially improves over conventional conditional models with enhanced control accuracy and reduced bias, both attribute-oonditonall generation and attribute transfer.
2 Background
We first briefly review the causal concepts most relevant to the paper. A structural causal model (SCM) [56] defined by a directed graph consisting of nodes (variables) and edges (direct causal dependence between variables), e.g., Figure 2 Different inference questions on an SCM correspond to different levels of the causal ladder (Figure T and require different reasoning tools: (1) "Association" deals with correlations in observed data with coin//marginal/coniiioon distributions. (2) "Intervention"
concerns what would happen were some actions been performed. typical question is to estimate the distribution of an outcome variable x given an intervention a treulment variable a: (a))). where the do-operation represents an action on a by setting it to given value. With randomized experimental data (i.e.. collected by randomly assigning treatment), p(æ (d((a)) equals to the standard

---

2
2
a
(a)
(b)
(c)
Figure 2: Illustration of causal graphs: (a) The proposed structural causal model (SCM, (3.1). where the outcome variable x denotes the text, treatment variable a denotes the attribute to control, z is the latent confounder, and is the proxy variable for the confounder. A hollow circle indicates the variable is latent, and shaded circle indicates the variable observed. The proxy information which we indicate with a dashed circle. Note the difference of the SCM compared to previous latent-variable controllable generation models 22 which do not explicitly model the confounder or its proxy information, making it impossible to identify the causal effects. (h) Intervention
observed only for a subset of examples,
the attribute ($3.2). represented as blue circle. eliminates the dependence between and a, leading the intervened SCM wherein the æ arrow is removed. (c) Counterfactual prediction ($3.3), where red dashed arrows represent abduction from original factual data (a, æ, c), and æ' is the counterfactual outcome given the new attribute a'.
conditional p(zla), Yet in practice, we usually only have access passively observed data, such as the (attribute, text) pairs from existing corpus, and have to adjust for confounders (i.e., variables that correlate with both treatment and outcome) in order to estimate p( do(a)) from observational distributions. For example, we apply backdoor adjustment (56) in $3.2 for attribute-conditionn generation. Finally, (3) "Counterfactuals" involves queries about what would have happened, given the knowledge of what in fact happened. We next show how the controllable text generation tasks are bridged together as different levels of causal inference, operationalized by the proposed SCM.
3 The Causal Framework for Controllable Text Generation
We now describe the unified causal perspective. We first develop the structural causal model that characterizes the causal structure in the controlled generation process ($3.1) We then show that intervention the SCM leads to attribue-coonditinn generation ($3.2). while counterfactual prediction makes attribute transfer ($3.3). At last, $3.4 describes the training of the SCM with new objectives encourage confounder balancing and correatlltionn
Compared to previous causal modeling in other domains (e.g.. images), modeling text as the outcome challenging due to the complex unstructured information encoded in the text. We show here that the unifying perspective enables us to bring to bear rich tools and inspirations from causal inference, disentangled representation, and controllable generation, for effective text causal modeling.
Figure shows the SCM graphs as detailed below. the appendix we illustrate the model architecture used in our experimental studies.
3.1 The Structural Causal Model
Figure ([a) shows the SCM that describes the controlled generation process of text. Here the attribute of interest serves the treatment, and the text is the outcome. For simplicity, assume is binary (e.g., positive or negative sentiment), though the framework can straightforwardly to more general cases where has multiple classes and dimensions. Note that as the condition for generating æ can be instantiated in different forms depending on the concrete application. For example, it can be a scalar positive", " "[sentimentt] negative") that acts as a prompt for the
applied
{0, 1} an input to the generator, a word sequence such as
E ['''sntiment generator to produce text continuation [5 18 30 81].
In general, the confounder that induces spurious correlations between a and æ infeasible to be fully specified or observed. For example, to control the sentiment of restaurant review, the confounder could involve popularity of the restaurant, personal preferences the customer, and other factors, whose values cannot be directly measured. Thus, following the recent causal approaches in other we model the unobserved confounder as high dimensional latent variable z, and infer from "indirect" confounding variables that are measurable in practice (such as food type).
domains 43

---

The "indirect" variables are also called proxy variables in causality [T) which we denote as c. More background of confounder and proxy is provided in the appendix.
The goal of controllable text generation is thus to generate coherent text (given original text in the case of text attribute transfer) with accurate target attribute a while unbiased in terms of the confounders.
Previous causal studies [43 48 44] have usually assumed the confounder proxy c is available for all data. Similarly, recent work of debiasing atrribue-condittionll generation (based machine learning) has relied on access to those extensive proxy labels [23 40]. However, the assumption is often impractical due to the time and financial cost for obtaining the massive additional information beyond the common (attribute. text) data. We thus consider a more practical setting [15] where only have access to the proxy information for small subset (e.g., 1% 5%) of examples. In Figure 2aa a), we use dashed circle of e to denote the new challenging setting.
The resulting SCM thus defines joint distribution
(1 a, 2, c) = po(ar (a, ())((a)))pp((())))))
(I)
where the component ps (c]x) applies when the proxy c is observed for the example; po(=) [z) is a standard Gaussian prior following the common practice; and all components with free parameters are modeled as deep neural networks. We use the amortized variational inference as in variational auto-encoders (VAEs) [31 infer the latent confounder from observations. Specifically, introduce a variational distribution (z(2 xx a, e) with parameters Φ. To infer z for those examples whose proxy is not available, we could apply auxiliary predictor that estimates e from the observed (æc, a). The auxiliary predictor can be trained on the subset of examples with available e. In this work, we instead the default to a dummy value when inferring for simplicity.
Note that previous work has also used VAEs for controllable text generation [22] [4. However, they reside purely at the association level. In particular, despite the latent variables, they do not explicitly model the confounder and/or its proxy information, rendering the causal effects between components not identifiable [56]. As result, those models vulnerable to biases, shown in the experiments.
3.2 Inference (I): Intervention for Attribate-Conditionn Generation
We now discuss how to perform causal inference given the SCM for attribute-conditional generation. As mentioned in şI in contrast to the conventional association-leve methods based the conditional p(zla), here we formulate the task with the interventional conditional (o)))) The do-operation sets a to given value independently of ($2). which eliminates the dependence between a and leading to the new intervened causal graph in Figure (bb), where the arrow from z to a is removed. text outcome without confounding bias. We can use the backdoor adjustment [56] to estimate pa (u()do((a) from the observed data:
Thus pa (z) (d(()) captures the causal effect of attribute
(=)
Σ,
That is, we adjust for confounder by making fair considerations of every possible z values and aver- aging the results by the distribution p(z) discussed below. The difference from the previous methods becomes even clearer if we similarly decompose the conditional (x Σx Pa 2 ())o((z()) (z(a). which as we can see depends on pe (z (a) and inherits the correlations between a and z in the data.
We generate text samples from po (z)))(a)) approximately by first drawing ~ p(2) and then decoding with æ ~ (a a, 2). To sample from the marginal p(z) which does not have a clean analytic form, we use similar approach as in [6)] by fitting a simple generative adversarial network (GAN) 16), (2),(2), on the learned latent space (i.e., z ~ qo on all training data). We found sufficient to use single-layer GAN which is fast to train.
3.3 Inference (II): Counterfactual for Text Attribute Transfer
Given an observed text æ, text attribute transfer seeks to produce new text x' that possesses the given new attribute o' and preserves many characteristics of the original x as possible. The task can naturally be mapped to counterfactual prediction on the SCM, i.e., imagining the alternative outcome for should its attribute have been a'. The resulting inference procedure looks similar to the previous VAE-based attribute transfer method 222. However, besides the key modeling difference of confounder/proxy above, counterfactual based interpretation offers principled causal account

---

for the attribute transfer task. Moreover, the causal perspective inspires new training techniques that substantially improve the performance and reduce generation bias, as presented in $3.4
Figure c) illustrates the inference process. Specifically, from the causal perspective, counterfactual prediction is mathematically formulated as a three-step procedure [56] 555: (1) Abduction that infers the "context" compatible with the observation æ. In our problem, it is sufficient to infer 2 as the context, as we would only intervene its descendant node a in the SCM. Thus the step is done by computing (= xt, a, c); (2) Action that performs intervention on variable a by setting a = a'; and (3) Prediction that computes the counterfactual outcome based on the SCM, i.e., x' pa(2"(el [a', z). where we set x to the mean (vector) of the above abduction distribution 96 for simplicity.
3.4 Learning
With the causal model and the inferences on it, we now discuss model training, which integrates variational learning and counterfactual reasoning for confounder balancing and disentanglement.
Variational auto-encoding objective The base objective for learning the causal model is built on the common VAE approach [31]. Briefly, since the model's marginal log-likelihood (that marginalizes out the latent z) is intractable, VAEs derive a lower bound with the variational distribution (s- Formally, given a training example (xx,a) a) with the optional proxy c:
(3) where the first term is the reconstruction that aims to recover the observations (æ, c) given the inferred from go: the second term is Kullback-Leibler regularizer that enforces the variational distribution to stay close to the prior po Pz(z). We refer readers to [31] for more details of VAEs. In the objective, Au de. and ARd > 0 are balancing hyperparameters. We set de to when proxy is not available. and otherwise select from {0.01, 0.1, } based on validation, same as Xa We use the cyclic schedule from 36 to anneal AR1 from 0 to 1 to avoid excessive regularization of the KL term.
Luee (0, Φ) = Exwag po la, =) + Aa (o) + A pogg(()))) - AL (gel1po).
Counterfactual objectives Training with the above base objective alone can lead to model collapse where the attribute variable a is ignored in the generation process, i.e., text sampled from po po(a (a, 2) is not effectively controlled by a. This is because the training text æ has already contained the attribute information, allowing both the inference of and the subsequent reconstruction of not to depend on the attribute value a. This issue highlights& a key difference of our model compared to previous latent-confounder causal models in other domains (e.g., medication effect prediction), where the outcome is typically a simple binary variable (e.g., cured or not) that does not "leak" the treatment
information (c.g., medication) [43 48 44]. Causal controllable text generation thus requires new solutions to encourage effective control.
Besides, key ingredient for accurate causal inference is achieve balance of confounders between That is, we want to match the confounder representation z of the examples whose 0 and those of the examples whose a = 1, in order to enhance the generalization performance for inferring counterfactual outcomes (27]. The concept is closely related to disentangled representation in machine learning which seeks to keep most dimensions of representation invariant to the change of a particular dimension 41 [9].
treatment groups 22 63 52
The above two desiderata can be resolved with suite of counterfactual objectives that are based on the counterfactual outcomes x' inferred in $3.3 We now describe those intuitive objectives, which are related to the attribute a, confounder z, and proxy respectively. We also discuss how are able to draw inspirations from previous literature of disentangled representation and text attribute transfer, thanks to their connections with causal inference above.
The first objective concerns the attribute a to correctly learn its influence on the outcome. Intuitively, given the counterfactual outcome x' given the counterfactual attribute a', we want to make sure x' truly entails a'. This can be achieved by using a pretrained attribute classifier [ar, a) that estimates the likelihood of text æ possessing attribute a. More specifically, we train the model such that its predicted x' possesses a' with a high likelihood measured by the classifier:
(o. () Eg~qg, s' e'>ppl(a" (=))) [[xx' aa)]
(4) 47 25 to the discrete text x'' to enable
As in 22 79
we use Gumbel-softmax approximation
gradient backpropagation for optimizing (0. Φ). Similar objective has been used in previous condi-
tional generation of text 22 45] and image 21 35). A crucial caveat is that, here the classifier itself
5

---

pretrained with the (attribute, text) data can also be biased due to confounding factors. Thus relying only on this objective as the previous work is not sufficient for accurate unbiased attribute control, as shown in our experiments. To this end, we further devise the following counterfactual objectives.
The second objective focuses balancing the confounder z. Intuitively, by the definition of counterfactuals, x' must have the same confounder representation as the original a. We thus minimize the distance between the respective z' and z:
Cef Φ) = EE, [d(z', (((x',z)) x)]]
(5)
where, with slight abuse of notation, 2 is the mean (vector) of go a, c). x' is the mean (vector) of (ol==l"", a', c) on the counterfactual x', and d(---) is distance metric. Though the vectors & and z' have continuous values, we draw inspiration from the recent disentangled representation work 4 and use binary cross-entropy loss to match z' to i.e.,
d(z' .)) mean (= log(o(z))) (1 2) log(1 a(2)))
(6)
where & = min( x)) / ///x/==ax= min( x))) normalizes x to [0, 1). a()) is the logistic function applied to z' element-win wise, and mean(-) takes the average distance across dimensions. The distance is shown to be more effective [1 than the common L.2 loss | - =12 as used in earlier work 22
The third objective carries similar intuition above, though uses the proxy when it is available. Specifically, we want z' to be able to reconstruct e (as is z):
Lef-o(0, Φ) Bz' po (=2||a") where z' is the mean (vector) of (2 |æl, , al, c), same as in Eq. In sum, the overall objective for training the causal model is:
[( Φ) Leae + Tafef-a + Tafeff YoLef-es
(8)
with balancing hyperparameters Ya. Y2, and % 2 0. In practice, we found the model is not sensitive to the choices of those hyperparameters. We set each of them to either 0.5 or 1.0 based on validation.
Related Work
Causal modeling for generation There is an emerging interest in integrating causality with ma- chine learning [62] in various problems. Several latest works have studied causal inference combined with deep generative models for images, to learn causal structures between attributes (78, 65 [1],
and augment unbiased classifier training [61] The spatial structure of images can make it easier learn causal mechanisms, e.g., the work (6 specified independent modules for image background and texture. In contrast, text with abstract concepts (e.g., sentiment, topics) exhibits less independent structure. Previous causal modeling for usually focuses on language understanding 229 74 75
synthesize novel images (32 3
49). Recent work has also studied text as outcome or generating text in specific domains (e.g., court view (77)) We make the first study of causal modeling for the general problem of text generation under control and demonstrate the effectiveness for bias mitigation.
71
13
in causal inference [12 for data augmentation 85 28
Controllable text generation Various approaches have been developed for attribute-conditional generation, by learning conditional language models (LMs) (30 [4] 81], guided inference [33 [
or prompts [5 72 Recent work has focused on reducing gender bias in machine translation and Other work studied more general unbiased generation with ML assuming access to unbiased classifiers [40] (3). We use causal techniques to address different and challenging setting where only limited confounding labels are observed. Unsupervised text attribute transfer has gained increasing attention [64] 22 26],, with the primary focus on learning to disentangle target attribute with other factors. We study the new challenge of attribute transfer the presence of strong bias in the data, and show greatly improved performance.
generation 68
66
69
5 Experiments
We study the challenging generation tasks with strong spurious correlations in training data, The causal framework substantially reduces bias and improves control accuracy.

---

We describe detailed model configurations in appendix. Briefly, the main model components, including the decoder Po (a) (z]a, z). inference network (,(2) a, c), and classifier f(x, a) (Eq(4) 4 are all based on the GPT-2 (1 17M) architecture [59 with pretrained weights, respectively. In qa and J. we use the GPT-2 final-step output feature as the representation of input sentence æ. We implement other components (No (1) 22) and po (clz)) as simple MLPs. The model is trained with Adam optimizer
42
using an initial learning rate of le-6, All experiments were conducted on 8 Tesla V100 GPUs.
5.1 Attribute-Conditionn Generation
We first evaluate the interventional inference for attribute-conditional generation ($3.2). We use two datasets where the target attribute has a correlation strength of over 90% with the confounding factor, following the challenging settings of the latest work on visual bias 773 17 61 65]. That is, the target attribute and the confounding factor of over 90% examples are both positive or negative, while those of the rest 10% examples are opposite. Differing from previous studies, we further assume the model can observe the confounding labels of a small subset of data, a more practical setting as in $3.1
Datasets Our first dataset is derived from the YELP challenge that contains customer reviews of different categories. Sentiment (1:positive vs. Ornegative) is the attribute we aim to control, and the category of review object (I:restaurant vs. 0:others) is the confounding factor. Specifically, we extract a subset of data where 90% restaurant reviews are of positive sentiment, while 90% reviews to other entities (e.g., shopping) are of negative sentiment (thus 90% correlation strength). We keep the category labels for less than 2% of training data. The resulting data has 510K/6K training/validation examples, wherein 10K training examples have observable confounding category l For evaluation, we further create a balanced test set of 13K examples with correlation strength 50% (i.e., no correlation). Following the previous controllable generation [22] 64), we focus on generating short text, by truncating the output text in the data to 20 tokens at maximum.
The second dataset is from the BIOS corpus [10] that contains online biographies with gender and occupation labels. We use gender (female/male in the corpus) as the attribute to control. Thus the goal is to generate biographical text of given gender. For occupation which is the confounding factor, we subsample and merge the occupations into two groups, i.c., /nurse, dietitian, paralegal, ...J and (rapper, DJ, surgeon, / (see appendix for more details). The correlation strength of the resulting dataset is 95%. For example, 95% female biographies are about the occupations in group one. We randomly split the dataset into 43K training and 2K validation examples, and keep the binary occupation labels for only 3K randomly selected training examples (among which only 5% x x3K= examples have opposite gender and occupation labels). As above, further create a balanced test set of 2K examples for evaluation, and truncate the output text to no more than 20 tokens.
Baselines and setup We compare with the conditional language models that people would com- monly train for the task. The first model, and generates text accordingly. The second model, Conditional LM (full), makes full use of the attribute and confounding labels in hope of better de-correlating the two. Since the confounding labels are available only on small subset of examples, we first train a classifier the subset with data-reweighting (see appendix for details), and use it to predict confounding labels for the remaining examples. The language model is then trained the resulting complete data, conditioning on both the attribute and the (real or estimated) confounding label. We also compare with latest attribute-conditione generation approaches, such as GeDi 33 where a language model conditioning on the confounding information Predi (ø|c) is used to reshape the generation distribution of the above Condit tinal LM. We include comparison with more baseline methods in the appendix.
LM, conditions only on the target attribute
For our approach, the available confounding labels serve as the proxy c. The attribute classifier f used to train our model (Eq 4 is pretrained on the biased training data. On YELP, the resulting (sentiment) classifier has a mediocre accuracy of 83% on the balanced test set; On BIOS, the (gender) classifier has accuracy of 91%.
'Due to the strong correlation, only 10% x 1K examples have opposite sentiment and category labels. posing significant challenge for the model to de-correlate the two factors.
7

---

Methods
Control accuracy (1)
Bias (1)
Fluency (4)
Diversity (1)
Conditional LM
79.1
78.7
50.4
41.4
Conditional LM (full)
80.3
78.9
50.8
41.9
YELP
GeDi
33
80.9
74.3
83.2
41.7
Ablation: Ours w/o ef-z/c Ours
91.1
89.2
54.1
40.4
96.3
59.8
51.3
39.1
Conditional LM
95.51
84.73
17.0
46.5
Conditional LM (full)
93.28
72.34
18.5
48.5
B10s
GeDi
86.0
75.2
27.8
43.5
Ablation: Ours w/o ef-z/c
97.3
70.1
29.4
42.1
Ours
99.2
62.4
32.0
40.6
Table I: Automatic evaluation of attrbbut-conddiional generation on YELP and BIOS. Control accuracy is measured by the attribute classifier accuracy; Bias by the confounding classifier accuracy: For fluency, we report perplexity, thus lower score indicates more fluent text; Diversity measured by the Distinct-2 metric. For each evaluation aspect, we highlight the best result that has significant improvements over others.
Methods
Control accuracy (t)
Bias (L)
Fluency (1)
Conditional LM (full)
80.0
73.0
3.90
YELP
Ours
97.0
56.0
3.85
Conditional LM (full)
96.0
82.0
4.43
BIOs
Ours
99.0
60.0
4.25
Table Human evaluation of attribute-condiiional generation on YELP and BiOs.
Evaluation We conduct both automatic and human evaluation. For the former, we follow the common practice and evaluate the generations in terms of various aspects following: (1) Control accuracy for which we use an "evaluation attribute classifier" that takes as inputs the generated sentences and measures how accurate they entail the input attributes. The evaluation attribute classifier trained on a large unbiased set of examples from the original corpus and of high test accuracy (87% for YELP and 95% BIOS) for evaluation purpose (note the difference from the above classifier trained with only biased training data); (2) Bias which is measured by another classifier for the confounding factor. Intuitively, the better the predicted confounding labels match the input attributes, the more correlated the two factors the generation. 50% match indicates no correlation. The classifiers trained similarly the evaluation attribute classifiers, and achieve accuracy 85% YELP and 90% for BIOS; (3) Fluency which is measured by applying GPT-2 language models (LMs) on the generated text and computing the perplexity; The LMs obtain perplexity of 32.4 and 18.0 on the real text of YELP and BiOs, respectively. (4) Diversity with the common Distinct-n metric [37 that measures the ratio of unique n-gramm against total number of n-gram in the generation set. We evaluate 10K generated samples by each model.
For human evaluation, ask human raters to annotate for each generated text the attribute label and confounding factor label, based on which we compute the control accuracy and bias as above. We also annotate language fluency using 5-point Likert scale. On each dataset, compare Conditional LM (full) and our approach, with 100 sentences from each model annotated by raters. The Pearson correlation coefficient of human scores is 0.67, showing strong inter-rater agreement.
Results Table 1 shows the automatic evaluation results on both YELP and BIOS. Our causal approach significantly improves over the association- -based conditional models. For example, on YELP, our model achieves 16% absolute improvement in terms of control accuracy, and at the same time reduces the bias (spurious correlation with the confounder) by 19%. In contrast, the conditional LMS mostly inherit the bias from the training data. As an ablation study, we also evaluate a simplified variant of our full approach by omitting the counterfactual objectives w.r.t and c (Eqs 5 and g (which reduces to a training strategy similar to the previous methods 22g. The variant improves the control accuracy results show the crucial role of confounder balancing in bias reduction. On the BIOS dataset, our approach also obtains consistent improvement both accuracy and bias.
the conditional LMs, but fails effectively reduce the generation bias. The
Table shows the human evaluation results on both datasets, which largely confirm the above observations with automatic evaluation.

---

Methods
Control accuracy (f)
Bias (L)
Preservation (1)
Fluency (L)
Hu et al. 22
44.1
68.4
77.7
132.7
He et al. 20
35.3
60.2
80.1
57.7
Ablation: Ours w/o ef-2/c
75.0
67.8
36.3
34.2
Ours
77.0
61.4
42.3
29.6
Table Results of attribute transfer on the biased YELP, Baselines [22 20 with the public code, and they to rewrite the text on most instances, leading to very low control accuracy and high preservation.
Preservation (1)
Methods
Control accuracy (f)
Fluency (J)
BelE-BEUU
ref BLEU
Hu ct al.
22
86.7
58.4
177.7
Shen et al.
64
73.9
20.7
7.8
72.0
He al.
87.9
48.4
18.7
31.7
et al. 8
87.7
54.9
20.3
73.0
Ablation: Ours w/o cf-z/c
87.1
57.2
24.3
46.6
Ours
91.9
57.3
25.5
47.1
Table 4: Results of text attribute transfer on the common unhiased YELP data.
Text Attribute Transfer
We next study text attribute transfer (3.3) as the second core task of controllable generation. The proposed causal approach also achieves substantial improvement in terms accurate control and bias reduction. Besides, for broader comparison, also apply approach to another unbiased dataset widely studied in previous text attribute transfer research, showing superior performance.
Datasets We use the above biased YELP dataset ($5.1) to study the attribute transfer, where aim to modify sentence to possess the opposite sentiment (e.g., from negative to positive), and at the same time preserve all other characteristics. In particular, we the new sentence keep the category unchanged, which difficult for previous association-based controllable models given the strong correlation in the data between sentiment and category. Besides, since most previous attribute transfer studies have focused only on unbiased setting, we additionally evaluate our approach on the popular unbiased YELP data (reviews with sentiment for restaurants only) 664 for comparison.
Evaluation We follow the standard practice for evaluation. For the biased setting, we measure control accuracy, bias, and fluency as in $5.1 We also assess the common aspect preservation, which evaluates the BLEU score (53] between the generated and original sentences (i.e., self-BL.BU), higher score indicates better preservation of sentence properties. For the unbiased setting, we omit the bias evaluation, and additionally compute another preservation metric, ref-BLEU, which is the BLEU score between the generation and human- written golden text on subset of test examples (38). We also conduct human evaluation which shows the same conclusions the automatic evaluation in terms of model performance. We put the results in appendix due to space limitation.
Results Table shows results the biased YELP data, a substantially than the popular unbiased one (Table We compare two of the previous best-performing methods with public code. Our approach again manages reduce the bias while achieving decent transfer accuracy. The previous methods struggie to edit the text on many instances (c.g., generating the same sentences as inputs), leading to low control accuracy. Ablation comparison with our simplified variant (our w/o cf-z/c) further validates the effect of counterfactual objectives for confounder balancing ($3.4), shown by the improved accuracy and mitigated bias of the full approach.
challenging setting
Finally, Table shows the results on the common unbiased YELP sentiment data. The results show approach generates fluent output with improved accuracy and preservation.
6 Conclusions and Future Work
We have presented a principled causal perspective for the two core tasks of controllable text generation. Based on the proposed structural causal model, attribute-conditional generation is modeled as interventional inference, and text attribute transfer performs counterfactual prediction. We connect

---

rich techniques in causality, disentangled representation, and text generative modeling, and develop learning objectives for accurate control and confounder balancing. Focusing on the challenging setting with partially available confounding information, the experiments show our approach achieves accurate control and mitigates the strong correlations in the data.
The proposed causal framework opens up a range of new opportunities for further improving and enriching controllable text generation. For example, though this work has focused on single control attribute and confounding factor, it would be interesting to generalize the approach for structured control a richer set of text attributes, by modeling the underlying causal graph between attributes (as explored similarly in image generation 178, 651). Besides, we are interested in importing more causality tools through the causal perspective propensity reweighting technique in causality can potentially be used to debias pretrained lan- guage models Peretrain (a)(a)) with the following known equation between the unbiased interventional conditional p( (d)( and the biased standard conditional p(ær(a):
enable new applications. For instance, the inverse
p(æ]do(a)) = Σ. p(ala, z)p(z)= = Σ p(ala)p(z)mm,d) p(a)
(9)
p(alz)
where p(a [z) is known as the propensity score [56], i.c., the propensity (probability) of the = being assigned the particular treatment a. Plugging in the Ppretrain (ar a) together with the parameterized estimates of po (z (ee a) and pa [a|=)) as learned in } we would effectively convert the pretrained LM into the unbiased p(æ [do(a). Further, rich studies the causality literature have proposed stabilized and enhanced variants of the above inverse propensity reweighting [e.g.. see [8), all of which present interesting topics explore in the controllable generation setting the future.
Ethical considerations We would like to note that automatic text generation could be used mali- ciously generate fake, toxic, or offensive content [34 72 could offer techniques to alleviate potential issues.
We hope the unbiased modeling study
References
[1] J. D. Angrist and J.-S. Pischke. Mostly harmless econometrics; An empiricist's companion. Princeton university press, 2008
[2] F. M. Bender, T. Gebru, A. McMillan-Major, and S. Shmitchell, On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pages 610-623, 2021.
[3] M. Besserve, A. Mehrjou, R. Sun, and B. Schölkopf. Counterfactuals uncover the modular struc- ture of deep generative models. In Eighth International Conference on Learning Representations (ICLR 2020), 2020.
[4] S. R. Bowman, L. Vilnis, O. Vinyals, A. Dai, R. Jozefowicz, and S. Bengio. Gener- ating sentences from a continuous space. In Proceedings of The 20th SIGNLL Confer- ence on Computational Natural Language Leurning, pages 10-21, Berlin, Germany, Aug. 2016. Association for Computational Linguistics. doi: 10.18653/v1/K16- 1/02210002 URL https://wwwe wwtp:://wwwwwwlwb.b.b org/ Saathhlogy/KK6-10000
[5] T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, Shyam, G. Sastry, A. Askell, et al. Language models are few-shot learners, arXiv preprint arXiv:2005.14165, 2020.
[6] A. Chattopadhyay, P. Manupriya, A. Sarkar, and V. N. Balasubramanian. Neural network attributions: A causal perspective. In International Conference on Machine Learning, pages 981-990. PMLR, 2019.
[7] W. Chen, J. Tian, L. Xiao, H. He, and Y. Jin. Exploring logically dependent multi-task learning with causal inference. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020.
[8] N. Dai, J. Liang, X. Qiu, and X.-J Huang. Style transformer: Unpaired text style transfer without disentangled latent representation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5997-6007, 2019.
10

---

[9] S. Dathathri, A. Madotto. J. Lan, J. Hung. E. Frank, P. Molino, J. Yosinski, and R. Liu. Plug and play language models: A simple approach to controlled text generation. In 8th International Conference on Learning Representations, ICLR 2020, Addis Abuha, Ethiopia, April 26-30, 2020. OpenReview.net, 2020. URI. htttt:://operreiiiiiiee http:////pppppaeeue.eeeeoro////a
[10] M. De-Arteaga, A. Romanov, H. Wallach. J. Chayes, C. Borgs, A. Chouldechova, S. Geyik, K. Kenthapadi, and A. T. Kalai. Bias in bios: A case study of semantic representation bias in a high-stakes setting. In proceedings of the Conference on Fairness, Accountability, and Transporency, pages 120-128, 2019.
[11] E. Dinan, A. Fan, Williams, J. Urbanek, D. Kiela, and Weston. Queens are powerful too: Mitigating gender bias in dialogue generation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8173-8188 2020.
[12] N. Egami, C. J. Fong, J. Grimmer, M. E. Roberts, and B. M. Stewart. How to make causal inferences using texts. arXiv preprint arXi::
2018.
[13] A. Feder, N. Oved, U. Shalit, and R. Reichart. CausaLM: Causal model explanation through counterfactual language models. arXiv preprint arXiv:2005. 13407, 2020.
[14] J. Ficler and Y. Goldberg. Controlling linguistic style aspects in neural language generation. CoRR, abs/ 707.00263, 2017. Uttht:t://arxxt wttp:ooo//////////oe 1707.02633
[15] K. Gan, A. Li, 2. Lipton, and S. Tayur. Causal inference with selectively deconfounded data. In International Conference on Artificial Intelligence and Statistics, pages 2791-2799. PMLR, 2021.
[16] J. Goodfellow, J. and Y. Bengio. Generative adversarial nets. In NIPS, 2014.
Pouglltt Pougeet-badie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. C. Courville,
[17] Y. Goyal, A. Feder, Shalit, and B. Kim. Explaining classifiers with causal concept effect (cace). arXiv preprint arXiv:190.007165, 2019.
[18] H. Guo, B. Tan, Z. Liu, E. P. Xing, and Z. Hu. Text generation with efficient (soft) Q-learning. arXiv preprini arXiv: arXiv:2106.07704 106.07704, 2021.
[19] N. Hassanpour and R. Greiner. Learning disentangled representations for counterfactual regression. In International Conference on Learning Representations, 2020.
[20] J. He, X. Wang, G. Neubig, and T. Berg-Kirkpatrick. A probabilistic formulation of unsupervised text style transfer. In International Conference on Learning Representations (ICLR), 2019.
[21] Z. He, W. Zuo, M. Kan, Shan, and X. Chen. Attgan: Facial attribute editing by only changing what you want. IEEE Transactions on Image Processing, 28( 28(11)) (11011)::444447777 2019.
[22] Z. Hu, Z. Yang, X. Liang, R. Salakhutdinov, and E. Xing. Toward controlled generation of text. In International Conference on Machine Learning (ICML), 2017.
[23] P.-S. Huang, H. Zhang, R. Jiang, R. Stanforth, J. Welbl, J. Rae, V. Maini, D. Yogatama, and Kohli. Reducing sentiment bias in language models via counterfactual evaluation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings, pages 65-83, 2020.
[24] M. Ilse, J. Tomezak, P. Forré, et al. Selecting data augmentation for simulating interventions. In AAAI, 2021.
[25] E. Jang, S. Gu, and B. Poole. Categorical reparameterization with gumbel-softmax. arXiv preprint
2016.
[26] D. Jin, Z. Jin, Z. Hu, O. Vechtomova, and R. Mihalcea. Deep learning for text style transfer: A survey, arXiv preprint arXiv:2011.00411 2020.
[27] Johansson, U. Shalit, and D. Sontag. Learning representations for counterfactual inference.
In International conference on machine learning, pages 3020-3029. PMLR, 2016.
11

---

[28] D. Kaushik, E. Hovy, and Z. Lipton. Learning the difference that makes difference with counterfactually-augeeee augmented data. In International Conference on Learning Representations. 2019.
[29] K. Keith, D. Jensen, and B. O'Connor. Text and causal inference: A review of using text to remove confounding from causal estimates. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguisfics, pages 5332-5344, 2020.
[30] N. S. Keskar, B. McCann, L. Varshney, C. Xiong. and R. Socher. Ctrl: A conditional transformer language model for controllable generation. arXiv preprint arXiv:199.0060 1909.05888, 2019.
[31] D. P. Kingma and M. Welling, Auto-encoding variational bayes. arXiv preprint arXiv: 1312.61141 2013.
[32] M. Kocaoglu, C. Snyder, A. G. Dimakis, and Vishwanath. CausalGAN: Learning causal implicit generative models with adversarial training. In International Conference on Learning Representations, 2018.
[33] B. Krause, A. D. Gotmare, B. McCann, N. S. Keskar, S. Joty, R. Socher, and N. F. Rajani. Gedi: Generative discriminator guided sequence generation. arXiv preprint arXiv:2009.06360 2020.
[34] S. Kreps, R. M. McCain, and M. Brundage. All the news that's fit to fabricate: Ai-generated text as tool of media misinformation. Journal of Experimental Political Science, pages 1-14, 2020.
[35] G. Lample, N. Zeghidour, N. Usunier, A. Bordes, L.. Denoyer, and M. Ranzato. Fader networks: Manipulating images by sliding attributes. Guyon, U. von Luxburg, Bengio, H. M. Wal- lach, R. Fergus, S. V. N. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4-9 December 2017, Long Beach, CA, USA, pages cc/paper/71788 1 www.ttowrkssaappaatiiino..
2017. URL http://///ppapr nips - by -ea--i--eessse
[36] C. Li, Gao, Y. Li, X. Li, B. Peng, Y. Zhang, and J. Gao. Optimus: Organizing sentences via pre-trained modeling of latent space. arXiv preprint arXiv:2004.04092 2020.
[37] Li, M. Galley, C. Brockett, Gao, and W. B. Dolan. A diversity-promoting objective function for neural conversation models. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 110-119, 2016.
[38] R. H. He, and P. Liang. Delete, retrieve, generate: A simple approach sentiment and style transfer. In 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. NAACL HLT 2018, pages 1865-1874. Association for Computational Linguistics (ACL), 2018
[39] S. Li, S. Yavuz, K. Hashimoto, J. Li, T. Niu, N. Rajani, X. Yan, Y. Zhou, and C. Xiong. Coco: Controllable counterfactuals for evaluating dialogue state trackers. ICLR, 2021.
[40] R. Liu, C. Jia, J. Wei, G. Xu, L. Wang, and S. Vosoughi. Mitigating political bias in language models through reinforced calibration. In Proceedings of the AAAI Conference on Artificial Intelligence, 2021.
[41] F. Locatello, M. Tschannen, S. Bauer, G. Rätsch, B. Schölkopf, and O. Bachem. Disentangling factors of variations using few labels. In International Conference on Learning Representations, 2019.
[42] I. Loshchilov and F. Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations, 2018.
[43] C. Louizos, U. Shalit, J. M. Mooij, D. Sontag, R. Zemel, and M. Welling. Causal effect inference with deep latent-variable models. Advances in neural information processing systems (NeurIPS), pages 6446-6456, 2017.
12

---

[44] D. Lu, C. Tao, J. Chen, F. Li, F. Guo, and L. Carin. Reconsidering generative objectives for counterfactual reasoning. NeurIPS, 2020.
[45] Luo, P. Li, J. Zhou, P. Yang, B. Chang, Z. Sui, and X. Sun. A dual reinforcement learning framework for unsupervised text style transfer. In IJCAI, 2019.
[46] N. Madaan, Padhi, N. Panwar, and D. Saha. Generate your counterfactuals: Towards controlled counterfactual generation for text. In AAAI, 2021.
[47] C. J. Maddison, A. Mnih, and W. Teh. The concrete distrihution: continuous relaxation of discrete random variables. In ICIR, 2017.
[48] D. Madras, E. Creager, T. Pitassi, and R. Zemel. Fairness through causal awareness: Learning causal latent-variable models for biased data. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pages 349-358, 2019.
[49] Mani and F. Cooper. Causal discovery from medical textual data. In Proceedings of the AMIA Symposium, page 542, American Medical Informatics Association, 2000.
[50] M. R. Montgomery, M. Gragnolati, K. A. Burke, and E. Paredes. Measuring living standards with proxy variables. Demography, 37(2):155-174, 2000.
[51] R. Moraffah, B. Moraffah, M. Karami, A. Raglin, and H. Liu. Can: A causal adversarial network for learning observational and interventional distributions. arXiv preprint arXiv:2008. /3376, 2020.
[52] M. Ozery-Flato, P. Thodoroff, M. Ninio, M. Rosen-Zvi, and T. El-Hay. Adversarial balancing for causal inference. arXiv preprint arXiv:1810.07406. 2018.
[53] K. Papineni, S. Roukos, Ward, and W.-J. Zhu. Bleu: method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311-318, 2002.
[54] G. Parascandolo, Kilbertus, M. Rojas-Carulla and B. Schölkopf. Learning independent causal mechanisms. In International Conference Machine Learning, pages 4036-4044. PMLR, 2018
[55] N. Pawlowski, D. Coelho de Castro, and B. Glocker. Deep structural causal models for tractable counterfactual inference. Advances in Neural Information Processing Systems, 33, 2020.
[56] J. Pearl. Causality. Cambridge university press, 2009.
[57] M. O. Prates, P. H. Avelar, and L. C. Lamb. Assessing gender bias in machine translation: case study with google translate. Neural Computing and Applications, pages 1-19, 2019.
[58] L. Qin, A. Bosselut, A. Holtzman, C. Bhagavatula, E. Clark, and Y. Choi. Counterfactual story reasoning and generation. In EMNLP, pages (046-5056, 2019.
[59] A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever. Language models are unsupervised multitask learners. OpenAl Blog, 1(8):9, 2019.
[60] E. Reiter and R. Dale. Building applied natural language generation systems. Natural Language Engineering. 3(1:577-7,, 1997.
[61] A. Sauer and A. Geiger. Counterfactual generative networks. In ICLR, 2021.
[62] B. Schölkopf. Causality for machine learning. arXiv preprint (tti::9911.000000 2019.
[63] U. Shalit, F. D. Johansson, and D. Sontag. Estimating individual treatment effect: generalization bounds and algorithms. In International Conference on Machine Learning, pages 3076-3085
PMLR, 2017.
[64] T. Shen, T. Lei, R. -Bazzilay, and T. Jaakkola. Style transfer from non-parallel text by cross- alignment. In Advances neural information processing systems (NeurIPS), pages 6830-6841, 2017.
13

---

[65] X. Shen, F. Liu, H. Dong, Q. Lian, Z. Chen, and T. Zhang. Disentangled generative causal representation learning. arXiv preprint arXiv:2010.02637, 2020.
[66] E. Sheng, K.-W. Chang, P. Natarajan, and N. Peng. The woman worked as a babysitter: On biases in language generation. In EMNLP, pages 1398-34008 2019.
[67] E. Sheng, K.-W. Chang, P. Natarajan, and N. Peng. Towards controllable biases in language generation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Findings, pages 3239-3254, 2020.
Processing:
[68] A. Stafanovićs, M. Pinnis, and T. Bergmanis. Mitigating gender bias in machine translation with target gender annotations, In Proceedings of the Fifth Conference on Machine Translation, pages 629-638, 2020.
[69] G. Stanovsky, N. A. Smith, and L.. Zeutlemoyer. Evaluating gender bias in machine translation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1679-1684, 2019.
[70] J. H. Stock, M. W. Watson, et al. Introduction to econometrics, volume 3. Pearson New York, 2012.
[71] C. Tan, L. Lee, and B. Pang. The effect of wording on message propagation: Topic-and author- controlled natural experiments on twitter. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguisties ( (Volume I: Long Papers), pages 175-185, 2014.
[72] E. Wallace, S. Feng, N. Kandpal, M. Gardner, and S. Singh. Universal adversarial triggers for attacking and analyzing nlp. In EMNLP, 2019.
[73] Z. Wang, K. Qinami, I. C. Karakozis, K. Genova, P. Nair, K. Hata, and O. Russakovsky, Towards fairness in visual recognition: Effective strategies for bias mitigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8919-8928, 2020.
[74] N. Weber, R. Rudinger, and B. Van Durme. Causal inference of script knowledge. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7583-7596, 2020.
[75] Z. Wood-Doughty, I. Shpitser, and M. Dredze. Challenges of using text classifiers for causal inference. In EMNLP, volume 2018, page 4586. NIH Public Access, 2018.
[76] T. Wu, M. T. Ribeiro, J. Heer, and D. S. Weld. Polyjuice: Automated, general-purposs counterfactual generation. arXiv preprint arX1v:2001.00288
[77] Y. Wu, K. Kuang, Y. Zhang, X. Liu, C. Sun, J. Xiao, Y. Zhuang, L. Si, and Wu. De-biased court's ' view generation with causality. In Proceedings of the 2020 Conference on Empirical Methods Natural Language Processing (EMNLP), pages 763-780, 2020.
[78] M. Yang, F. Liu, Z. Chen, X. Shen, J. Hao, and J. Wang. CausalVAE: Structured causal disentanglement in variational autoencoder. arXiv preprint arXiv:2004.08697, 2020.
[79] Z. Yang, Z. Hu, C. Dyer, E. P. Xing, and T. Kirk-tiik......c. Unsupervised text style transfer using language models as discriminators. In NeurlPS, pages 7298-7309, 2018.
[80] L. Yao, Z. Chu, S. Li, Y. Li, J. Gao, and A. Zhang. A survey on causal inference. arXiv preprint arXiv:2002.02777 027700 2020.
[81] R. Zellers, A. Holtzman, H. Rashkin, Y. Bisk, A. Farhadi, F, Roesner, and Y. Choi. Defending against neural fake news. Neurips, 2020.
[82] X. Zeng, Y. Li, Y. Zhai, and Y. Zhang. Counterfactual generator: A weakly- method for named entity recognition. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7270-7280, 2020.
[83] D. Zhang, H. Zhang, J. Tang, X.-S Hua, and Q. Sun. Causal intervention for weakly-supervised semantic segmentation, Advances in Neural Information Processing Systems, 33, 2020.
14

---

[84] J. Zhao, T. Wang. M. Yatskar. V. Ordonez, and K.-W Chang. Men also like shopping: Reducing gender bias amplification using corpus-level constraints. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 2017.
[85] Q Zhu, W. Zhang, T. Liu, and W. Y. Wang. Counterfactual off-policy training for neural dialogue generation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3438 -448,, 2020.
[86] R. Zmigrod, S. J. Mielke, H. Wallach, and R. Cotterell. Counterfactual data augmentation for mitigating gender stereotypes in languages with rich morphology. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1651-1661, 2019.
15


Reviews:
[{"review_id": "cPo69oLUxT", "reviewer": "Reviewer_dfNU", "summary": "The goal of the paper is to control an attribute of the text without changing another associated attribute. Let's use the biased YELP dataset the authors created as an exampe. In the dataset, 90% of positive reviews come from restaurants and 90% of negative reviews come from the other categories (e.g., shopping). Their goal is to control the sentiment (attribute a) without changing its category (confounding factor c). This is difficult because the conditional generator might just generate a restaurant review when the task is to generate a positive review. Similarly, if most of the nurses are female, the text generator might just generate nurses to satisfy the requirement of generating a female character.\n\nIn conditional generation such as a seq2seq model, even the controlled attribute is given as the condition, the text generator might still generate the text that might imply confounding factors or some other attributes because the confounding factors and controlled attribute are associated or the prior probabilities of those factors/attributes are high. The paper solves the problem using backdoor adjustment (removing the dependency between the attribute a and the hidden representation of the sentence z). My understanding of their method is like assuming confounding factors and controlled attributes are independent or ignore the prior probability of observing other attributes. This would make the generated text less fluent but can generate the text that would have the attribute in a high probability.\n\nThe experiments show that the proposed method is significantly better than conditional LM or other text attribute transferring approaches when the training data is biased (e.g., the review category is highly correlated with the sentiment of the review in YELP). When the dataset is unbias, one experiment seems to show that the proposed method is also better than other approaches.\n", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I think the problem setting is practical. For example, people might want to remove an undesired attribute that has a high correlation with other desired attributes (e.g., remove the toxic attribute but keep the negative sentiment). The proposed method also seems to work pretty well. I did not know any other direct-related work. Thus, the originality and significance are good, which might lead to high impact. \n\nNevertheless, the experiments and presentation still have sufficient room for improvement. First, the paper does not say very clearly whether the proposed method is better than existing approaches in the popular unbiased benchmarks. The evaluation setting of the unbiased YELP experiment is not specified clearly. Is the proposed model trained on the unbiased dataset in this experiment? How do you tune the hyperparameters of your model and other methods? Do you think where the improvement comes from? Do all the models use the same amount of resources and are trained on the same data? For example, does the proposed method use category information but other methods do not? Is your model size (number of parameters) larger than previous approaches (I suggest you put all the model sizes into your table)? Is the classifier given when you train other models? Do all the models use the same classifier? Could you also compare the diversity in the text attribute transfer task? Could you also compare your methods with other approaches in a more standard evaluation (e.g., Table 9 in https://arxiv.org/pdf/2010.12742.pdf)? You use the predicted confounding labels to train Conditional LM (full), so I am not sure whether it performs worse than your model due to the noise in the predicted labels or the effectiveness of your model. Did you try to train your casual model on such predicted labels rather than the ground truth label subsets?\n\nIf the authors cannot show that the proposed method is consistently better than other approaches in unbiased datasets in a fair comparison. I think emphasizing the focus of this paper earlier would make the paper much easier to read. For example, the current title and introduction give me the impression that this paper proposes a general approach to all types of controllable text generation tasks. I do not understand the main motivation behind the paper until I saw the creation of the biased YELP dataset in the experiment section. I recommend the author to mention some concrete examples (e.g., in your biased YELP dataset) in the introduction or Figure 2 to explain what a, c, x, and z mean and emphasize the focus of this paper. In addition, in Tables 1 and 2, the YELP dataset should be called biased YELP to avoid confusion.\n\nFinally, I think more ablation studies are needed (The paper only tries another baseline in the ablation study). The architecture is complicated (e.g., line 162 says it even uses GAN) and you have seven loss terms. Why do we need to use GAN here? If the proposed method is better than other approaches in the unbiased setting, we would like to know how much each model design or each loss term contributes to the improvement.\n\nIf the rebuttal addresses my concerns about experiments and presentations well, I will vote acceptance (>=7) for this work. You don't have to answer all the questions I asked. I just want to know whether this method is generally better than state-of-the-art results in unbiased datasets in a fair comparison. If no, majorly revise your presentation. If yes, provide more pieces of evidence (and minorly revise the paper to make your motivation more concrete). \n\n\nSome minor suggestions:\n1. More detailed description of model architectures should be included in the appendix. It would be even better to add a figure to explain architecture. This is especially important if you want to claim the proposed method is generally better than others in unbiased datasets. I only have a very high-level understanding of the model after reading the whole paper.\n2. A paper I saw usually won't describe the baselines inside the result section/paragraph. Explain cf-z/c baseline before the result paragraph and mention the cf-z/c keywords that allow the readers to search.\n3. In the paper, you mentioned that your method is related to disentangled representation studies. The relation of your work to the related text style transfer studies such as [1] could be discussed in the paper.\n4. I think the perplexity numbers are usually positive values and smaller is better.\n5. What's the n value you used in the distinct-n diversity metric?\n\n[1] John, Vineet, et al. \"Disentangled Representation Learning for Non-Parallel Text Style Transfer.\" Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "One potential negative societal: Although neural style transferring could be used to remove toxic attributes but also could be used to inject toxic attributes, I do not think explicitly mentioning this possibility would have a positive social impact. Thus, I think the paper should keep as it is on this part.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "OjKw5FsiwTK", "reviewer": "Reviewer_BdYj", "summary": "The paper introduces methods to integrate causality in the task of controllable text generation, specifically on attribute-conditioned generation and text attribute transfer. The goal is to eliminate biases from spurious correlations with confounders (e.g., jobs when controlling the gender of the text). They do this by introducing a latent confounder variable and an observed proxy variable. They trained their models using a VAE objective, as well as three counterfactual objectives to help mitigate model collapse.", "questions": "", "limitations": "", "rating": 7, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths\n* Controlling text while removing biases and spurious correlations is one of the most important problems in controllable text generation. The method introduced in the paper is intuitive and the results show that it works better than competing baselines. \n\nWeaknesses\n* It would be very helpful to readers who are more familiar with NLG but less familiar with work on causality to provide definitions to terms on this area (e.g., outcome, treatment, confounders, and their relations).\n* The ablation could have been more extensive. For example, what happens when only one of the three counterfactual objectives is used?\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The paper does not have separate sections for limitations and societal impact.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "8"}, "extra_text": {}}, {"review_id": "Nji1daMJitK", "reviewer": "Reviewer_rETv", "summary": "The authors propose a unified causal framework for controllable text generation tasks (attribute-conditional text generation and attribute text transfer). The authors propose a structural causal model (SCM) to formulate the two tasks as causal intervention and counterfactual reasoning in the context of causal latter, which thus provides a unified framework for studying controllable text generation.  The learning stage aims to integrate variational learning (VAE) and confounder disentanglement (and balancing). \n\nFor the evaluation, the authors reused Yelp and Bios datasets, and then added synthetic spurious correlations, creating challenging data scenarios.  They evaluate the proposed model with a few simple baseline methods in terms of control accuracy, bias, fluency, and diversity. To support the automatic evaluation of these metrics, they use pre-trained attribute classifiers. They also conduct human evaluations to show the effectiveness of their proposed methods.  \n\nOverall, the main contribution of the paper is a casual perspective for two main controllable text generation tasks via casual interventional inference and counterfactual reasoning. The method can be used to reduce toxic content (e.g., gender bias) in downstream applications that require automatic text generation. \n", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "***Originality: Are the tasks or methods new? Is the work a novel combination of well-known techniques? (This can be valuable!) Is it clear how this work differs from previous contributions? Is related work adequately cited?***\n\nThe controllable text generation tasks,  particular causal framework, and learning methods are all existing techniques. The work is rather a novel combination of them and provides a novel and valuable perspective. The casualty-centric framework is adequately different from prior works in the same direction, most of which are also well cited. \n\n\n***Quality: Is the submission technically sound? Are claims well supported (e.g., by theoretical analysis or experimental results)? Are the methods used appropriate? Is this a complete piece of work or work in progress? Are the authors careful and honest about evaluating both the strengths and weaknesses of their work?***\n\nThe proposed method itself is sound and complete. The evaluation section is rather inadequate. There are only two simple datasets and the attribute and confounding factors are very simple. The baseline methods for controlling attributes and debiasing are not compared. The authors only used a simple \"Conditional LM\" as the key baseline method. Therefore, the comparisons are not that convincing to me. \n\n\n\n***Clarity: Is the submission clearly written? Is it well organized? (If not, please make constructive suggestions for improving its clarity.) Does it adequately inform the reader? (Note that a superbly written paper provides enough information for an expert reader to reproduce its results.)***\n\nMost of the parts are clearly written, while I would suggest the authors add more concrete examples of attributes vs confounding factors in the method section. Maybe the authors can simply add a problem formulation section and use the cases in the dataset description paragraphs (e.g., sentiment vs categories). Another improvement of the presentation could be adding a figure for the overview architecture of the proposed method, i.e., connecting the GPT2 and the attribute classifier and so on to illustrate how the method works in terms of the implementation. \n\n\n***Significance: Are the results important? Are others (researchers or practitioners) likely to use the ideas or build on them? Does the submission address a difficult task in a better way than previous work? Does it advance the state of the art in a demonstrable way? Does it provide unique data, unique conclusions about existing data, or a unique theoretical or experimental approach?***\n\nAs stated above, the empirical results in the paper are inadequate, although the reported numbers show that the proposed method is much better than a simple baseline. It is, however, not convincing enough that the proposed method is significantly better than more sophisticated baseline methods --- either recent conditional text generation methods or the methods to remove spurious correlation (for NLG and other tasks). The style transfer experiment does not use other popular datasets in the community, so it is hard to directly compare it in a broader context. \n\nAlso, the analysis of the results is also superficial and does not give any insights and implications why such a casual framework is significantly better. The experiment section is barely a report of the results. Adding more dedicated analysis with concrete examples would improve the paper a lot. \n\n***Other weaknesses. ***\n\nThe proposed method (and the evaluation) is only limited to a single target attribute and a single confounding factor. What are the key challenges for extending the method (and the experiments) to multiple target attributes with multiple confounding factors? The authors should have explicitly discussed the limitation and the possible ways to address them as this is a key weakness of the paper. \n\nMinor question: Would it be possible to apply the method to lexical-constrained text generation tasks such as CommonGen? ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "***Have the authors adequately addressed the limitations and potential negative societal impact of their work?***\n\nThis proposed method in this paper can be used to reduce the bias in automatic text generation tasks and thus can lead to a positive social impact. However, it is also possible that someone can use the method to generate even toxic text by controlling the attribute in a bad way.  In this aspect, I would suggest the authors add a simple paragraph in the final version to discuss the social impact from both aspects. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "0Tol3gmb5A", "reviewer": "Reviewer_nG9H", "summary": "This paper investigates the problem of the attribute (sentiment, gender) controlled text generation or text style transfer. The authors formulate the problem from a principled causal perspective, which models the two tasks with a unified framework. A framework of structural causal model is introduced to ground the idea. Experiments on text generation and style transfer are conducted and the experimental results show that the proposed framework can indeed improve the control accuracy.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper investigates the problem of the attribute (sentiment, gender) controlled text generation or text style transfer. The authors formulate the problem from a principled causal perspective, which models the two tasks with a unified framework. A framework of structural causal model is introduced to ground the idea. Experiments on text generation and style transfer are conducted and the experimental results show that the proposed framework can indeed improve the control accuracy.\n\n\nStrength:\n\n- The paper is well written and easy to follow. The authors describe the technical content detailly and readers can easily understand the implementation tricks.\n\n- The proposed framework unifies the tasks of attribute-controlled text generation and style transfer which using one model can conduct two tasks simultaneously.\n\n\nWeakness:\n\n- Although the authors present the story from a principled causal perspective which is good certainly, it is not easy to figure out the novelty and technical contribution compared with the previous methods (such as CVAE model). The basic and main idea is still a VAE-based framework with some more variables or tricks to improve the performance, therefore, from the technical derivatives and descriptions, it is hard to feel excited and benefit more.  \n\n- It is good to see that Conditional LM is introduced as the baseline method to conduct comparisons. I am wondering what is the performance of CVAE model which GPT2 is employed as the language decoder. This comparison is more important and convincing. \n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors have addressed the limitations and potential negative societal impact. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The submission proposes a method for debiasing controllable text generation, allowing text to be generated that satisfies one desired attribute, without also introducing confounding attributes. The reviewers agree that this is an important problem, and that a causal perspective is valuable here. The major concerns raised by the reviewers focused on the evaluation, however the authors have provided additional experimental results in their response that significantly strengthen the conclusions. Therefore, I recommend acceptance.
Decision: Accept (Poster)