Text Style Transfer with Confounders

1 INTRODUCTION . Despite advances in neural text generation ( Brown et al. , 2020 ) , fine-grained control over generated outputs remains challenging . Indeed , the ability to easily transfer output style by altering attributes such as sentiment , formality , genre , and personal styles would make text generation tools more appealing ( Hu et al. , 2017 ; Shen et al. , 2017 ; Rao & Tetreault , 2018 ; Xu et al. , 2012 ) . Solving style transfer typically requires the method to be able to disentangle what should be transferred from orthogonal aspects of sentences that ought to be kept intact . This disentanglement problem can be largely avoided in simple supervised scenarios with access to parallel sentences differing only in style ( e.g. , sentiment transfer with parallel negative and positive sentences , Figure 1.a ) . Recent approaches address a more difficult version of the task by dispensing with parallel sentences ( Shen et al. , 2017 ) . Nevertheless , they assume that the corpora differing in style remain distributionally matched in other ways ( e.g. , sentiment transfer with negative and positive reviews from the same category of products , Figure 1.b ) . However , data available for training style transfer models is rarely distributionally matched , and often involves changes other than style as well ( e.g. , sentiment transfer where negative and positive reviews come from different product categories , Figure 1.c ) . This means that the desired style difference is no longer illustrated directly as the difference between the two corpora , making the task substantially more challenging . More subtly , the model is asked to generate sentences it has not seen during training , since the generated sentences in a new style should not also reproduce those confounding differences present in the training data . Solving style transfer with confounding cues requires us to also learn what the desired style difference is . This can be facilitated by dividing the data into two groups of different styles , while the sets within each group illustrate variation we do not wish to alter . In the example in Figure 1.c , reviews are divided into two groups according to their sentiment . Within a group , each dataset corresponds to a different product category which should be preserved . Besides sentiment transfer , this setup easily generalizes to other style transfer tasks , e.g. , dialectal transfer . In this case , the groups will be divided according to the dialects , and sets within each group will represent different speakers whose personal style should be preserved . Our model builds on invariant risk minimization ( Arjovsky et al. , 2019 ) to infer the style as an invariant distinction across different datasets from the two groups . The resulting style classifier leaves complementary aspects of sentences to be controlled ( preserved ) . We can illustrate aspects orthogonal to style with new sets of environments and learn another invariant classifier . Together , 1Our code will be publicly released after the review process . the two classifiers are used to guide sentence generation in a style transfer model along the desired direction . Combining with back-translation ( Sennrich et al. , 2015 ; Zhu et al. , 2017 ) and language model regularization ( He et al. , 2020 ) techniques , we can generate new types of sentences that have not been seen during training . We empirically evaluate our proposed model in two sentiment transfer settings that involve confounding factors . In the first setting , we augment original review data with special tokens , creating a spurious correlation with sentiment . In the second setting , we consider sentiment transfer where negative and positive reviews come from non-overlapping product categories . In both cases , we assess the ability of the model to transfer the sentiment , while preserving other aspects – special tokens in the first case , and product category in the second case . Our experiments demonstrate that our model successfully achieves both tasks , bringing significant gains over baselines that do not consider confounding factors . For instance , on the the task of sentiment transfer from different product categories , the model yields 28.4 % increase in category preservation ; according to human evaluation , its success rate is 6.2 % higher than the best previous system . 2 RELATED WORK . The task of style transfer is related to paraphrasing , whose goal is to generate multiple linguistic realizations of the same underlying content ( Androutsopoulos & Malakasiotis , 2010 ) . However , style transfer adds an additional complexity – the requirement to control for a specific realization characteristic during rewriting . While paraphrasing models have been used in the past for the task of style transfer , these models are impacted by arbitrary variations present in paraphrasing datasets ( Preotiuc-Pietro et al. , 2016 ; Gröndahl & Asokan , 2019 ; Krishna et al. , 2020 ) . We instead take a data-driven approach for discerning style from content , where the style is not strictly limited to paraphrasing and can involve more general attribute transfer such as sentiment and political slant transfer ( Prabhumoye et al. , 2018 ) . Recent work in non-parallel style transfer proposes different techniques such as cross-alignment ( Shen et al. , 2017 ) , delete and retrieval ( Li et al. , 2018 ) , or parallel latent sequences ( He et al. , 2020 ) . The generation of these approaches all takes place “ in-distribution ” , i.e. , realizing sentences of the type already seen during training . The sentences we generate are by design not seen during training ( Figure 1.c ) . Subramanian et al . ( 2018 ) proposed multi-attribute style transfer that controls both the sentiment and the category of a sentence . However , similar to other prior work , they assumed access to all sentiment-category combinations for training . In contrast , we assume that negative and positive sentences come from non-overlapping categories . Since we have to infer style from datasets with confounding cues , we make use of invariant learning ( Peters et al. , 2016 ; Arjovsky et al. , 2019 ) to estimate style as an invariant direction . Moreover , our classifiers need to perform well across different combinations of source/target datasets and generalize to generated text ( out-of-distribution samples ) to properly guide the transfer model . Therefore , our problem is also related to domain adaptation ( Ben-David et al. , 2010 ; Ganin & Lempitsky , 2015 ; Ganin et al. , 2016 ) and domain generalization ( Blanchard et al. , 2011 ; Muandet et al. , 2013 ) . Our main contribution is the utilization of invariance for guiding the generation process . 3 STYLE TRANSFER WITH CONFOUNDERS . We consider two groups of datasets : GA = { A1 , . . . , An } where each Ai ( i = 1 , . . . , n ) is a dataset consisting of sentences with style sA ( e.g. , negative sentiment ) , and GB = { Bn+1 , . . . , Bn+m } where each Bj ( j = n+ 1 , . . . , n+m ) similarly conforms to style sB ( e.g. , positive sentiment ) . In addition to style , a dataset has its own characteristics different from each other ( e.g. , category ) . Our goal is to transfer a sentence x of style sA into style sB ( and vice versa ) without changing its content or other characteristics . One attempt is to aggregate collections of sentences in each group A = A1 ∪ · · · ∪ An , B = Bn+1 ∪ · · · ∪Bn+m and perform style transfer between them . However , the specific characteristics of Ai and Bj will become confounding factors and will be changed along with style . Instead , we notice that style is an invariant distinction between group GA and group GB . In other words , the style difference is stable across different Ai and Bj . Therefore , we can learn to isolate it by taking out intra-group variations . Once we have access to style , we can learn to transfer sentences along this direction while preserving other sentence characteristics . We take a two-step procedure to accomplish this task . We first learn a pair of invariant classifiers to detect style and orthogonal characteristics . Then we use the classifiers to guide the learning of a style transfer model . Each part of the model is described in detail below . 3.1 INVARIANT CLASSIFIERS . We make use of Invariant Risk Minimization ( Arjovsky et al. , 2019 , IRM ) to learn our style and orthogonal classifiers . IRM requires us to specify a set of environments E = { e1 , . . . , eE } , where each e ∈ E represents data { ( xei , yei ) } ne i=1 collected under a certain environment . The different environments account for nuisance variation that the classifier should not pay attention to . The IRM objective is to learn a feature representation that enables a classifier to be simultaneously optimal for all environments . The rationale is that such representation likely involves primarily causal features that remain stable regardless of the nuisance variation . Therefore , the classifier can better generalize to new , unseen test environments compared to the standard Empirical Risk Minimization ( ERM ) classifier trained on the pooled data from all environments . We adopt the IRMv1 formulation , which does not explicitly separate the representation from the classifier but treats the classifier output itself as the representation . In this vein , the objective becomes to minimize the classifier loss across all the data while penalizing per-environment gradients with respect to any multiplier of the classifier output : min Φ : X→Y ∑ e∈E Re ( Φ ) + λ‖∇w|w=1.0Re ( w · Φ ) ‖2 ( 1 ) where Re ( f ) : = EXe , Y e [ ` ( f ( Xe ) , Y e ) ] is the risk of f under environment e ( ` can be any loss function ) , and λ is a hyperparameter weighting the gradient penalty term . Gradients would be zero if Φ is per-environment optimal . It remains to define suitable environments for the invariant classifiers to separate style and confounding factors in our task . 3.1.1 STYLE CLASSIFIER . To learn a classifier to distinguish between styles sA and sB without relying the specific characteristics of each dataset , we pair each Ai and each Bj to form environments : ei , j = { ( x , y = 0 ) | x ∈ Ai } ∪ { ( x , y = 1 ) | x ∈ Bj } ( 2 ) and learn an IRM classifier Cs : X → Y to predict the group label across environments { ei , j } 1≤i≤n , n+1≤j≤n+m ( cf . Figure 2.a ) . Since all Ai datasets share style sA , and all Bj datasets share style sB , style is a feature representation that elicits an invariant classifier across different environments . Conversely , if the classifier uses any features specific to Ai/Bj , it will not be optimal in another environment consisting of a different pair Ai′/Bj′ , thus violating the IRM constraint . 3.1.2 ORTHOGONAL CLASSIFIER . In addition to requiring the transferred output to have a different style from the input , we also require it to retain the other characteristics of the input . To detect style-independent characteristics , we construct environments based on predictions of the style classifier Cs . These style-dependent environments serve to illustrate to the second invariant classifier Co what it should not rely on . Let D = { ( x , y = 0 ) | x ∈ A } ∪ { ( x , y = 1 ) | x ∈ B } be the entire dataset , aggregate of all corpora . We create two environments2 : e1 = { ( x , y ) ∈ D | Cs ( y|x ) > 0.5 } ( 3 ) e2 = { ( x , y ) ∈ D | Cs ( y|x ) ≤ 0.5 } and learn an IRM classifier Co : X → Y across { e1 , e2 } . In this way , Co can not depend on the direction quantified by Cs ( i.e. , the inferred style ) , and must find other orthogonal features to distinguish A from B ( cf . Figure 2.b ) . Note that if we have successfully transferred a sentence in Ai to style sB , then Cs should predict label 1 for the output sentence because its style has been changed , while Co should continue to assign label 0 as the orthogonal characteristics ought to have remained intact .

Summary: Style transfer is a text generation task where a certain attribute of a sentence (like formality) is modified while preserving sentence content. It is generally studied in "unsupervised" settings --- no access to parallel data of sentences differing only in the target attribute. Nevertheless, algorithms assume access to corpora of unpaired sentences in each of the attributes, which may significantly differ in content. Prior works [1, 2, 3, 5] have noted this difference in content, which is encouraging models to modify non-target attributes like semantics (See Table 1 in [4]).  This paper is an attempt to fix this issue using invariant risk minimization (IRM) [6]. IRM encourages classifiers to be invariant across "environments", or datasets which share a target class but differ in other aspects which are not essential for classification ("spurious correlations"). In this work two spurious correlations are studied in sentiment transfer --- (1) a synthetic punctuation correlation; (2) product category of Amazon reviews. To perform style transfer, the authors first train two orthogonal IRM classifiers --- the first classifies the target attribute ("sentiment") irrespective of the confounder ("product category"), and the second for the confounder ("product category") irrespective of sentiment. These two classifiers are used to guide model generation, along with an LM smoothing and backtranslation objective (similar to [4]).  The authors compare their proposed approach against strong baselines and report promising results.  [1] - https://arxiv.org/abs/1910.03747   [2] - https://arxiv.org/abs/2010.05700   [3] - https://dl.acm.org/doi/10.5555/3016100.3016326   [4] - https://arxiv.org/abs/1811.00552   [5] - https://arxiv.org/abs/1905.13464   [6] - https://arxiv.org/abs/1907.02893