Benchmarking and Improving Compositional Generalization of Multi-aspect Controllable Text Generation (2024)

Abstract: Compositional generalization, representing the model’s ability to generate text with new attribute combinations obtained by recombining single attributes from the training data, is a crucial property for multi-aspect controllable text generation (MCTG) methods. Nonetheless, a comprehensive compositional generalization evaluation benchmark of MCTG is still lacking. We propose CompMCTG, a benchmark encompassing diverse multi-aspect labeled datasets and a crafted three-dimensional evaluation protocol, to holistically evaluate the compositional generalization of MCTG approaches. We observe that existing MCTG works generally confront a noticeable performance drop in compositional testing. To mitigate this issue, we introduce Meta-MCTG, a training framework incorporating meta-learning, where we enable models to learn how to generalize by simulating compositional generalization scenarios in the training phase. We demonstrate the effectiveness of Meta-MCTG through achieving obvious improvement (by at most 3.64%) for compositional testing performance in 94.4% cases1.

heading: 1 Introduction
text: Multi-aspect Controllable Text Generation aims to generate fluent text with a combination of attributes from diverse aspects (e.g. sentiment, topic, tense, person, and stuff). In comparison with single-aspect controllable text generation (Zhang and Song, 2022), it is more challenging and calls for increasing attention in recent years (Gu et al., 2022; Yang et al., 2023).
Current MCTG methods involve decoding-timebased (Dathathri et al., 2019; Yang and Klein, 2021) that modulate output distribution by a welltrained classifier, separate-training-based (Gu et al.,
∗The first two authors contributed equally to this work. †Corresponding author: Zhendong Mao.
1The code implementation is available at https:// github.com/tqzhong/CG4MCTG.
2022; Huang et al., 2023; Gu et al., 2023; Yang et al., 2023) that train multiple single-aspect modules in turn with single-aspect data and generating multi-aspect text by fusing them, and joint-trainingbased (Keskar et al., 2019; Qian et al., 2022a; Zeng et al., 2023), which train multiple single-aspect modules simultaneously or multi-aspect modules with multi-aspect data. These methods based on pre-trained language models (Radford et al., 2019) have achieved promising results on this task.
However, seldom works focus on the investigation of compositional generalization, a crucial property of MCTG approaches, which refers to the model’s ability to generate text with new attribute combinations obtained by recombining single attributes from the training data. For example, we aim for the model to generate text with the attribute combination (negative, male) after training on data with (positive, male) and (negative, female). Due to the difficulties in collecting data with all possible attribute combinations in most real-world scenarios, the capability for compositional generalization is paramount.
To this end, We propose CompMCTG, a compre-
6486
hensive benchmark to evaluate the compositional generalization of MCTG approaches (Section 3.1). We first collect four popular datasets (from a minimum of two-aspect, eight attribute combinations to a maximum of four-aspect, forty attribute combinations) in the MCTG field to comprise CompMCTG. The next crucial issue is how to split the dataset to better unveil the compositional generalization risk of MCTG methods. Generally, we split the whole dataset C into two disjoints sets: indistribution set Ci.d. and compositional set Ccomp, where the MCTG model is trained on Ci.d. and tested on both Ci.d. (in-distribution testing) and Ccomp (compositional testing). For an all-sided evaluation, we propose a three-dimensional evaluation protocol containing Hold-Out, ACD, and Few-Shot, which is depicted in Figure 1. Among them, Hold-Out is an easy protocol, which holds a few attribute combinations out from C as Ccomp and uses the remaining combinations as Ci.d.. FewShot is the hardest protocol, in which we guarantee every single attribute appears in the Ci.d. while minimizing |Ci.d.|2. To better reflect the capacity of models in cases that |Ccomp| is comparable to |Ci.d.|, which are closer to real-world scenarios, we design Attribute Compound Divergence (ACD), where we make |Ci.d.| = |Ccomp|. The core idea of ACD is to maximize the distributional divergence between Ci.d. and Ccomp. Compared with random sampling that contributes to similar distributions between Ci.d. and Ccomp easily (Zeng et al., 2023), ACD can better amplify the compositional generalization risk while random-based splits often lead to gross under-estimation (Section 3.4).
Through the results on CompMCTG (Section 3.3), we observe that all of the evaluated MCTG baseline approaches are faced with a noticeable performance drop between in-distribution and compositional testing. To further enhance the compositional generalization performance of joint-trainingbased methods which generally perform the best among all baselines, we propose Meta-MCTG (Section 4), a training framework incorporating metalearning (Finn et al., 2017), in which we enable models to learn how to generalize by simulating compositional generalization scenarios in the training phase. Firstly, we train the original model on a training batch Btrain, perform one step of gradient descent, and save the updated parame-
2We define |C| as the number of attribute combinations in C
ters to a backup model without updating the original model’s parameters. Secondly, we sample a “pseudo compositional” batch Bpcomp from the training set where the attribute combinations are the re-combination of those in Btrain and train the backup model on Bpcomp. Finally, we combine the losses from both steps and perform one step of gradient descent to update the original model’s parameters. Compared with solely training the model on Btrain, introducing Bpcomp enables the model’s parameters to update in a direction that not only focuses on fitting the training data but also takes outof-distribution data into account, which helps to elevate model’s capability of compositional generalization. We implement Meta-MCTG on three topperforming joint-training-based MCTG baselines and conduct extensive experiments on CompMCTG, demonstrating the effectiveness of MetaMCTG through achieving obvious improvement (by at most 3.64%) for compositional testing in 94.4% cases.
Our main contributions are three-fold: (1) We propose CompMCTG, the first holistic benchmark targeting compositional generalization for MCTG, incorporating four popular datasets and a crafted three-dimensional evaluation protocol. (2) We conduct extensive experiments on CompMCTG with eight representative MCTG baselines and two additional LLMs, unveiling noticeable compositional generalization risk in them and demonstrating the necessity of designs in CompMCTG. (3) We propose Meta-MCTG, incorporating meta-learning into the MCTG training process, to mitigate MCTG models’ over-fitting to attribute combinations seen in the training phase and improve their capacity for compositional generalization. To the best of our knowledge, we are the first to comprehensively evaluate MCTG on compositional generalization and introduce meta-learning into MCTG to improve composition generalization.

heading: 2 Related Work
text: Multi-aspect Controllable Text Generation Existing works on MCTG primarily fall into the following three categories: The first is decoding-timebased (Dathathri et al., 2019; Yang and Klein, 2021; Krause et al., 2021), which uses a welltrained classifier or conditional language model to adjust the output probability distribution of a frozen causal language model. The second is separatetraining-based, which trains single-attribute mod-
ules (Yang et al., 2023; Huang et al., 2023), Energybased Models (Mireshghallah et al., 2022; Qin et al., 2022) or latent space representations (Gu et al., 2022, 2023) using single-attribute label data, and controls the generation by concatenating individual modules, Energy-based Models or seeking the intersection of different attribute representations in the latent space. The third is jointtraining-based, which trains multi-attribute modules (Keskar et al., 2019; Zeng et al., 2023; Qian et al., 2022b) simultaneously using multi-attribute label data. Qian et al. (2022b) add a prefix (Li and Liang, 2021) for each attribute and train these prefixes using a contrastive loss. Zeng et al. (2023) encode different control codes (word embedding of attribute tokens) into prompts (Lester et al., 2021) using a fully connected layer and train this layer using a contrastive loss similar to Qian et al. (2022b).
Compositional Generalization Existing works on compositional generalization involve various NLP topics: Semantic Parsing (Herzig and Berant, 2021; Ontanon et al., 2022; Drozdov et al., 2023; Li et al., 2023), Machine Translation (Li et al., 2021; Zheng and Lapata, 2022; Lin et al., 2023), Text Classification (Kim et al., 2021; Chai et al., 2023), Complex Reasoning (Zhou et al., 2023a; Press et al., 2023; Li et al., 2024; Lin et al., 2024) and stuff. Nonetheless, in the field of open-domain controllable text generation, compositional generalization, which we target and reveal as the necessity for the robustness of neural language generators in this paper, remains under-explored. (Zeng et al., 2023) investigates compositional generalization focusing on a neighboring topic, controllable dialogue generation. We regard their work as a starting point of our research and further depict the deficiency of its naive evaluation protocol, for the underestimation of the compositionality gap in more realistic scenarios (Keysers et al., 2020).

heading: 3 Benchmark: CompMCTG
text: We propose CompMCTG, a novel benchmark to comprehensively evaluate the compositional generalization capacity of MCTG approaches. The superiority and novelty of CompMCTG are out of its scale of dataset and its three-dimensional evaluation protocol (Section 3.1). We select eight representative baseline approaches (Section 3.2), evaluate their performance on our CompMCTG benchmark, and unveil their struggling on compositional testing (Section 3.3). Moreover, system-
atic analysis towards exploring the behaviors of baseline approaches under different evaluation protocols of CompMCTG is provided in Section 3.4, which highlights: 1) its capacity to dig out the potential generalization risk of evaluated approaches and 2) the undervalued compositionality gap in the previous work (Zeng et al., 2023) as well.

heading: 3.1 On the Construction of CompMCTG
text: Data Source We collect commonly used and open-sourced datasets for our usage. Consequently, we select a shopping review dataset: Amazon Review (He and McAuley, 2016), a mixture of movie(IMDB (Maas et al., 2011)), tablet, automobile(Sentube (Uryupina et al., 2014)) and hotel(OpenNER (Agerri et al., 2013)) review dataset: Mixture (Liu et al., 2022), and two restaurant review datasets: YELP (Shen et al., 2017; YELP, 2014) and Fyelp (Lample et al., 2019). Details of these datasets are concluded in Appendix A.
Three-Dimensional evaluation Protocol We design a three-dimensional(Hold-Out, ACD and FewShot) evaluation protocol, aiming to sufficiently explore the compositional generalization capacity of existing approaches. Supposing that dataset D3 contains m distinct aspect sets: A1, A2, ..., Am and a specific aspect Ai (1 ≤ i ≤ m) has ai kinds of different attribute values in its set: Ai = {A1i , A2i , ..., Aaii }, we denote the whole attribute combination set as the continued Cartesian product C = A1 × A2 × ... × Am = {(Atii )1≤i≤m|1 ≤ ti ≤ ai}. The core of constructing CompMCTG is to split the attribute combination set C into in-distribution set Ci.d. and compositional set Ccomp. Basically, Ccomp has no intersection with Ci.d. and any attribute combination in Ccomp can be derived through recombining single attributes in Ci.d.. Hence we have the formal definition of an eligible split s(C) = Ci.d., Ccomp as following:
C = Ci.d. ∪ Ccomp, Ci.d. ∩ Ccomp = ∅ {attribute|∃c ∈ Ccomp, attribute ∈ c} ⊆ {attribute|∃c ∈ Ci.d., attribute ∈ c}
(1)
Hold-Out is an easy evaluation protocol, which holds a few attribute combinations out from C as
3Each datum in D consists of two components: (c, x), where c denotes the condition part, a combination of several attributes of different aspects (e.g., sentiment:“positive”, tense:“past”, and topic:“basketball”) and x denotes the text part, a span of text corresponding to these conditions. For brevity, we omit the text part and use the condition part to represent the data in this section.
Ccomp and uses the remaining attribute combinations as Ci.d.. Supposing |Ccomp| equals to k (k is relatively small so that the split is eligible), there are (|C| k ) different kinds of splits. In our benchmark, we set k = 1, and the final result is the average across (|C| k ) scenarios to eliminate bias.
Few-Shot is the hardest evaluation protocol, in which we guarantee every single attribute appears in the Ci.d. while minimizing |Ci.d.|, which simulate the scenarios of the low-data regime.
While in most real-world scenarios, |Ccomp| is comparable to |Ci.d.|. A crucial issue to this situation is how we divide C into Ci.d. and Ccomp
as the exponential complexity of sweeping over all of the eligible possibilities (We discuss this point in Appendix C). Thus focusing on a representative subset of them is a feasible solution. Inspired by (Keysers et al., 2020), we propose ACD, where we keep |Ci.d.| = |Ccomp| and construct representative splits by maximizing the Attribute Compound Divergence between Ci.d. and Ccomp. The term attribute compound refers to a specific tuple of two attributes: (Atii , A tj j ), i ≤ j, 1 ≤ ti ≤ ai, 1 ≤ tj ≤ aj , which characterizes the cooccurrence of two attributes in one attribute combination c ∈ C. Firstly, we calculate the frequency density of the attribute compound (Atii , A tj j ) in the combination sets C ∈ {Ci.d., Ccomp} and obtain two frequency distributions (fCi.d.((A ti i , A tj j )))i,j,ti,tj and (fCcomp((A ti i , A tj j )))i,j,ti,tj :
fC((A ti i , A tj j )) =
∑ c∈C I(A ti i ∈ c ∧A
tj j ∈ c)∑
c∈C ∑ x∈c,y∈c,x ̸=y 1
= 2 ∑ c∈C I(A ti i ∈ c ∧A tj j ∈ c)
m(m− 1)|C|
(2)
Then we introduce the Chernoff Coefficient S(P,Q) (Chung et al., 1989) to measure the scale of similarity between two probability distributions P and Q (i.e., P = (p1, p2, ..., pn) and Q = (q1, q2, ..., qn), S(P,Q) = ∑n i=1 p α i q 1−α i ∈ [0, 1])4. Finally, we define the Attribute Compound Divergence as D(Pi.d., Pcomp) = 1 − S(Pi.d., Pcomp) ∈ [0, 1] to measure the divergence between Ci.d. and Ccomp, where distribution Pi.d.
4α ∈ [0, 1] is a hyperparameter to control our tolerance on the difference between P and Q:
and Pcomp represent (fCi.d.((A ti i , A tj j )))i,j,ti,tj and (fCcomp((A ti i , A tj j )))i,j,ti,tj , respectively. In the real construction of ACD splits, we adopt a greedybased hill climbing algorithm (Russell and Norvig, 2010)5 to sample satisfactory splits which maximize D(Pi.d., Pcomp).
Note that for Amazon Review and Mixture datasets, ACD protocol degenerates to Few-Shot protocol as these datasets only contain two aspects and we can not optimize the attribute compound divergence in that situation.

heading: 3.2 Baseline and Evaluation Metric
text: We select eight representative baseline methods to study: 1) for Joint-Training based methods, we choose CTRL (Keskar et al., 2019), a classic and powerful baseline, Contrastive Prefix (Con.Prefix) (Qian et al., 2022a), CatPrompt (Yang et al., 2023), and DCG (Zeng et al., 2023), a related work targeting on reducing the compositionality gap, as our baseline methods, 2) for SeperateTraining based, we select two state-of-the-art baselines: Distribution-Lens (Gu et al., 2022) and Prior (Gu et al., 2023), 3) for Decoding-Time based methods, we choose PPLM (Dathathri et al., 2019) and Fudge (Yang and Klein, 2021). In addition, we adopt LLaMA-2 (Touvron et al., 2023) and ChatGPT (OpenAI, 2023) to study the compositional generalization of large language models (LLMs) with In-context Learning paradigm (Brown et al., 2020). Following (Sun et al., 2023), we attach five demonstrations in the input prompt for LLMs to follow. One can find more details about our implementations in Appendix D.
Grounded on the MCTG task, we adopt the evaluation metrics (note that the subfixes “i.d." and “comp" refer to the in-distribution and compositional testing respectively.) of 1) ACCi.d. and ACCcomp: the averaged prediction accuracies6 for all of the control aspects to measure the controllability of generated text, 2) PPLi.d. and PPLcomp: perplexity calculated by GPT-2 Large to measure the fluency of generated text in all of our experiments, and 3) Dist-3: 3-gram distinctness to evaluate the diversity of the text generated by approaches mentioned above. We also adopt Humanevaluation to measure the relevance and fluency of
5The algorithm pseudo-code is available in Appendix H. 6For each aspect in each dataset, we train a Roberta classifier (Liu et al., 2019) to evaluate its accuracy (details in Appendix D.3).
the generated text for each approach7.

heading: 3.3 Evaluation Result
text: The main evaluation results on CompMCTG benchmark are shown in Table 1, where values in “Original” column refer the performance where text data of all attribute combinations are available in the training set and hence there is no compositional testing; values in “Hold-Out” and “ACD” columns refer to in-distribution and compositional testing performance through the evaluation protocols of “Hold-Out” and “ACD” mentioned in Section 3.1 respectively; values in “Aavg” and “Pavg” column refer to overall performance which is the arithmetic mean of results under different evaluation protocols mentioned here (Originali.d., Hold-Outi.d., HoldOutcomp, ACDi.d. and ACDcomp), which are formulated as:
Aavg = 1
5 (A
original i.d. + A holdout i.d. + A holdout comp + A acd i.d. + A acd comp)
Pavg = 1
5 (P
original i.d. + P holdout i.d. + P holdout comp + P acd i.d. + P acd comp)
(3)
The “gap” (Gavg) is used to assess the average compositional generalization risk and a lower Gavg indicates better robustness under compositional testing, which is formulated as:
Gavg = 1
2 (Gholdout +Gacd)
= 1 2 ( Aholdouti.d. −Aholdoutcomp
Aholdouti.d. + Aacdi.d. −Aacdcomp Aacdi.d. )
(4)
Among all the evaluated baselines, joint-trainingbased approaches generally exhibit higher attribute accuracy, better fluency (lower perplexity, only inferior to LLM+ICL), and better robustness to compositional testing (lower Gavg). Though seperate-training-based methods perform acceptably in in-distribution testing, their performance drops drastically in compositional testing and we discuss the inherent reason for their failures in Appendix I.1. Decoding-time-based methods perform poorly overall, despite PPLM owning the lowest Gavg, both its average accuracy and perplexity are unusable. LLMs can generate more fluent text while the controllability of the generated text (54.04%) falls behind joint-training-based methods (79.82%). At the same time, LLMs (+ICL) also suffer from a large performance drop in compositional testing (Gavg is 23.5% for LLaMA and ChatGPT).
7Due to the page limit, please find the result of Dist-3 and Human-evaluation in Appendix E and F.
Additionally, We evaluate all of the baseline approaches with Few-Shot evaluation protocol in Table 2, to reflect their performance when only limited attribute combinations are available. Again, joint-training-based approaches hold the best average performance and compositional generalization capacity among them. We provide the details of our benchmark in Appendix B.

heading: 3.4 Insight
text: In this section, we conduct analysis experiments to show the effect of our key designs in CompMCTG: 1) the three-dimensional evaluation protocol (Hold-Out, ACD and Few-Shot) and 2) the effectiveness of ACD in amplifying the compositional generalization gap.
Compositional gaps with different evaluation protocols. In Figure 2, we show compositional gaps (G = Ai.d.−AcompAi.d. ) for approaches: CTRL, CatPrompt and DCG, with three evaluation protocols on YELP and Fyelp datasets. We observe that the compositional gaps on the same approach and dataset vary a lot with different evaluation protocols: Gholdout < Gacd < Gfewshot generally holds. Notably, Hold-Out can not properly unveil the compositional generalization gap for a specific approach. For instance: On Fyelp dataset, CatPrompt has the compositional gap of 0.91% on Hold-Out protocol, while it drastically increases to 10.96% on ACD protocol. Moreover, different approaches have different preferences for these protocols. By way of example, The compositional gap (e.g., on Fyelp) of DCG with ACD (1.97%) is lower than CTRL (5.95%) while its gap with Few-Shot (25.91%) is much higher than CTRL (13.95%),
demonstrating that the deficiency of DCG in lowdata regime. Hence jointly leveraging these three evaluation protocols evaluates MCTG approaches more comprehensively.
Does the ACD better unveil the compositional generalization risk in comparison with Random Sampling? To demonstrate the effectiveness of ACD, where we maximize the divergence of attribute compound distributions between indistribution and compositional sets, we design two other protocols in which we still keep |Ci.d.| = |Ccomp|: Random Sampling (random divergence) and minimizing the divergence (minimum divergence). We compare the compositional gaps among the three protocols (on Fyelp dataset) in Figure 3. We observe that gaps of ACD are consistently higher than two comparison protocols by large margins. Notably, using baseline approaches of CTRL and DCG, compositional gaps with Random Sampling are near zero while they are 5.65% and 1.97% with ACD. Hence we conclude that ACD generally better unveils the compositional generalization risk while Random Sampling often causes gross under-estimation of such risk.

heading: 4 Methodlogy: Meta-MCTG
text: In Section 3.4, we observe that joint-training-based (both parameter-efficient fine-tuning based and allparameter fine-tuning based) baselines generally achieve better overall performance. Nonetheless, there still exist non-negligible compositional generalization gaps for all these baselines, which highly calls for our attention. To this end, we propose Meta-MCTG, a novel Meta-learning (Finn et al.,
2017) based MCTG training framework, to further improve compositional generalization capabilities of existing joint-training baselines. The framework is easy to implement and can be directly combined with any joint-training-based methods. We discuss the design of Meta-MCTG in Section 4.1 and demonstrate its effectiveness through experiment results for Meta-MCTG in combination with three competitive joint-training baselines (CTRL (Keskar et al., 2019), ContrastivePrefix (Qian et al., 2022a) and DCG (Zeng et al., 2023)) in Section 4.2.

heading: 4.1 Design
text: Overall Motivation The overall framework of Meta-MCTG is depicted in Figure 4. We analyze that the failure of generating text satisfying control conditions in compositional testing can be attributed to the over-fitting of language models to local optima of control conditions in the training set. Thus when trained language models are fed with recomposed attribute combinations as the control conditions in the compositional testing (e.g., In Figure 4, “positive-sport-present”), it will potentially encode and distribute those new attribute combinations in the neighbor area of similar ones (e.g., “positive-sport-past”) that they have seen in the training phase. In this way, previous MCTG approaches fail to generate text that perfectly meets the requirements of all given conditions. As depicted in Figure 4, when given the recomposed attribute combination of “positive-sport-present”, models may generate text like “The book sparked my love for sports.”, neglecting the “present” con-
dition (As models only sees “positive-sport-past” attribute combination in the training phase).
Meta-MCTG training procedure Inspired by previous meta-learning works targeting generalization (Li et al., 2018; Wang et al., 2021; Conklin et al., 2021), we aim to leverage Model-Agnostic Meta Learning (MAML) (Finn et al., 2017) to mitigate the overfitting problem.
First of all, given a specific joint-training-based approach M, we denote its training objective as LMtrain(θ;ϕ;B) where θ represents the learnable parameters of encoding control conditions, ϕ represents the parameters of the language model (e.g., GPT-2), which are frozen during training (Note that in CTRL, ϕ is also updated while it still suits for the Meta-MCTG.), and B denotes a batch of data. In general, the training objective can be derived as:
min θ
LMtrain(θ;ϕ;B) =
min θ
∑
(ci,xi)∈B [− log p(xi|ci; θ;ϕ)] + LM(θ;ϕ;B) (5)
The first term refers to the basic LM loss (Radford et al., 2018) which maximizes the likelihood of generating target text xi and the second term refers to the auxiliary loss added by baseline M (e.g., contrastive loss (Qian et al., 2022a)).
In the Meta-MCTG framework, we first sample a batch of training data, denoted as Btrain = (ctraini , x train i ) m i=1 and a batch of pseudo-comp data, denoted as Bpcomp = (cpcompi , x pcomp i ) m i=1 where {ctraini }mi=1 ∩ {cpcompi }mi=1 = ∅ and each attribute combination of {cpcompi }mi=1 must be the
recombination of single attributes appearing in the {ctraini }mi=1. For instance, in Figure 4 the pseudocomp conditions “positive-movie-past” and “negative-sport-present” are the recombinations of conditions “positive-sport-past” and “negative-moviepresent” in the training batch.
We train model on Btrain and perform one step of gradient descent to update θ with Objective 5 (α is the learning-rate):
θ1 = θ − α∇θLMtrain(θ;ϕ;Btrain) (6)
Then we maintain θ unchanged in the original model, temporarily store θ1 to a backup model, and feed Bpcomp to the backup model to obtain the loss on pseudo-comp data:
LMpseudo−comp(θ;ϕ;Bpcomp) = LMtrain(θ1;ϕ;Bpcomp) = LMtrain(θ − α∇θLMtrain(θ;ϕ;Btrain);ϕ;Bpcomp)
(7)
According to the construction of Bpcomp, we use LMpseudo−comp(θ;ϕ;Bpcomp) to simulate the compositional generalization scenario, evaluating the compositional generalization capacity of model updated by Eq 6. We hope the updated model (with θ1) performs as well as possible on these pseudocomp data rather than merely overfitting Btrain. Taking both the original training Objective 5 and the compositional generalization Objective 7 into consideration, Meta-MCTG is to minimize the following objective:
LMtotal(θ;ϕ;Btrain;Bpcomp) = LMtrain(θ;ϕ;Btrain) + λLMpseudo−comp(θ;ϕ;Bpcomp) (8)
Where λ is a hyper-parameter to make a trade-off between the above two terms. Finally, we perform one step of gradient descent to update θ in the original model with Objective 8:
θ′ = θ − β∇θLMtotal(θ;ϕ;Btrain;Bpcomp) (9)
Where β is the learning rate. We summarize the pseudo-code of the Meta-MCTG training procedure in Algorithm 2 in Appendix H.

heading: 4.2 Experiment Results and Analysis
text: Experiment Results of Meta-MCTG We train CTRL, ContrastivePrefix and DCG with the MetaMCTG algorithm and aim to demonstrate that Meta-MCTG can generally improve their compositional generalization capacity. The compositional
testing results for all four datasets are shown in Table 38. For most cases (94.4% of the total), we can observe that baseline approaches trained with Meta-MCTG have an obvious improvement in compositional testing performance on controllability of generated text (i.e., attribute accuracy) over the original versions (by at most 3.64%). Besides, the introduction of the Meta-MCTG framework has almost no impact on text fluency (i.e., perplexity). We additionally show the in-distribution testing results in Table 4, demonstrating that Meta-MCTG nearly has no negative effect on in-distribution testing. Instead, it improves the in-distribution testing over the original baselines on 72.2% cases. We also provide a separate analysis of the compositional generalization gap variations for each dataset and protocol before and after incorporating the MetaMCTG framework in Table 5, where the gap is calculated by gap = Ai.d.−AcompAi.d. . From the results, it can be observed that in the majority of cases, the Meta-MCTG framework is able to reduce the compositional generalization gap.
Visualization and Case Study Previously we hypothesize that Meta-MCTG mitigates the problem that overfitted baseline approaches distribute recomposed novel attribute combinations in the neighbor of in-distribution ones in the representation space. We now calculate the difference in the distance of any two attribute combinations of the original version of baselines and baselines trained with Meta-MCTG. An example result for CTRL is shown in Figure 5. We observe that nearly all
of the distances between Ci.d. and Ccomp increase with Meta-MCTG and are notably larger than the
8We do not apply Meta-MCTG to Few-Shot settings, for we can not construct Bpseudo−comp when each attribute only appears once in Ci.d..
distances within Ci.d.. The results demonstrate that Meta-MCTG can distribute the hidden representations of attribute combinations more sparsely and thus possibly make them more distinguishable. Calculation details and more relevant results are available in Appendix I.2. Besides, we also present case study to compare the generation results of the original version of baselines and baselines trained with Meta-MCTG in Appendix G, highlighting the better controllability of the latter ones.

heading: 5 Conclusion
text: We propose CompMCTG, the first holistic benchmark targeting compositional generalization for Multi-Aspect Controllable Text Generation (MCTG), and conduct extensive experiments on CompMCTG with eight representative MCTG baselines and two LLM baselines, unveiling noticeable compositional generalization risk in them and demonstrating the effectiveness of CompMCTG. In addition, we propose Meta-MCTG, a framework incorporating meta-learning into the MCTG training process to improve its compositional generalization ability, which can be combined with any joint-training-based MCTG methods.

heading: Limitations
text: Our proposed Meta-MCTG framework improves the compositional generalization performance of MCTG methods in most scenarios. However, when attribute combinations of data in the training set are extremely scarce (e.g., the Few-Shot protocol in CompMCTG), we cannot build the pseudo-comp batch to utilize the Meta-MCTG framework. Besides, though Meta-MCTG is generally effective, current MCTG methods still have considerable room for improvement in compositional generalization. Both of these limitations will be areas for our future research.
Frankly speaking, the experimental workload of Hold-Out protocol in the CompMCTG benchmark is overly cumbersome, and the average results in our main table do not include Few-Shot, which we believe are discrepancies. For researchers with limited resources who want to follow our work, we recommend focusing on the performance of models under the ACD and Few-Shot protocols. These protocols are relatively more challenging and facilitate distinguishing models based on different capabilities.

heading: Ethics Statement
text: Multi-aspect controllable text generation is widely used in social media. However, improper use can cause serious negative effects, such as using this technology to spread inappropriate remarks (political attributes) or create rumors. Therefore this kind of technology should be subject to certain regulations.

heading: Acknowledgements
text: We sincerely thank all the anonymous reviewers for their constructive comments. This work is supported by the National Science Fund for Excellent Young Scholars under Grant 62222212, the National Key R&D Program of China (No.2021ZD0111801), the Research Grants Council of the Hong Kong SAR under Grant GRF 11217823 and Collaborative Research Fund C104223GF, the National Natural Science Foundation of China under Grant 62371411 and InnoHK initiative, the Government of the HKSAR, Laboratory for AI-Powered Financial Technologies.

heading: Appendix

heading: A Datasets
text: We select a shopping review dataset: Amazon Review (He and McAuley, 2016), a mixture of movie(IMDB (Maas et al., 2011)), tablet, automobile(Sentube (Uryupina et al., 2014)) and hotel(OpenNER (Agerri et al., 2013)) review dataset: Mixture (Liu et al., 2022), and two restaurant review datasets: YELP (Shen et al., 2017; YELP, 2014) and FYelp (Lample et al., 2019). In this section, we mainly introduce the four datasets that make up our benchmark as mentioned above.
Fyelp Following previous work (Yang et al., 2023; Huang et al., 2023; Lample et al., 2019), we adopt the widely used Fyelp dataset, which contains restaurant reviews with the sentiment (positive and negative), the cuisine (American, Mexican, Asian, Bar, and dessert), and the gender (Male and Female). To evaluate the extensibility of methods, we add one additional aspect of constraints: the tense (Past and Present) (Ficler and Goldberg, 2017), where its label is automatically extracted from the reviews. Thus far, the Fyelp dataset is the one with the largest scale of attribute combinations in our benchmark. In total, there are 2×2×5×2 = 40 possible attribute combinations.
Amazon Review Amazon Review (He and McAuley, 2016) is a dataset containing reviews for Amazon products, which is widely used in previous academic works around text rewriting, controllable text generation, and stuff (Li and Tuzhilin, 2019; Lample et al., 2019; Zhou et al., 2023b). Following (Lample et al., 2019), we process the dataset and label the data with two aspects: the sentiment (positive and negative) and the topic (Books, Clothing, Music, Electronics, Movies and Sports) with the meta-data in the original Amazon Review9 dataset. Hence there are 2× 6 = 12 different attribute combinations.
YELP YELP business reviews dataset (YELP, 2014) contains the three aspects of attributes: the tense (Past and Present), the sentiment (positive and negative), and the person (singular and plural). We process the dataset in alignment with (John et al., 2019) and (Russo et al., 2020) and randomly re-split the whole dataset for our usage. There are
9https://jmcauley.ucsd.edu/data/ amazon/
2× 2× 2 = 8 different attribute combinations in this dataset.
Mixture Mixture is the combination of three individual datasets: IMDb (Maas et al., 2011) (movie reviews) OpenNER (Agerri et al., 2013) (hotel reviews) and SenTube (Uryupina et al., 2014) (tablet and automobile reviews), constructed by (Liu et al., 2022). Hence each datum in Mixture has two aspects of attributes: sentiment (positive and negative) and topic (movie, hotel, tablet, and automobile) and there are in total 2 × 4 = 8 possible attribute combinations.
We summarize all details and statistics of these datasets in Table 6.

heading: B Details of CompMCTG Benchmark

heading: B.1 Two Types of Testing
text: Our CompMCTG benchmark contains four datasets: Fyelp, Amazon, Yelp, and Mixture. For each dataset, we divide it into two disjoint subsets: in-distribution set and the compositional set. The in-distribution set contains the data that is visible during training, while the compositional set contains the data that is not visible during training. The sets of attribute combinations contained in the in-distribution set and compositional set are defined as Ci.d. and Ccomp, respectively. We first train the model on the in-distribution set and then there are two types of testing. The first type that involves generating text attribute combinations from Ci.d. is referred to as in-distribution testing, which tests the model’s learning ability within the distribution of the training data. The second type that involves generating text with attribute combinations from Ccomp is referred to as compositional testing, which tests the model’s compositional generalization ability beyond the distribution of the training data.

heading: B.2 Datasets Details
text: For each dataset, the total number of data points N , the number of attribute combinations |C|, and the number of data points per attribute combinations Ni are related as N = |C|×Ni (the data points per attribute combination are equal for all datasets).
For the Hold-Out protocol, we define the indistribution set as the subset obtained by removing one attribute combination from the total dataset. Therefore, the size of the in-distribution set for the Hold-Out protocol is given by N × (|C| − 1)/|C|.
For the ACD protocol, we designed it such that the ratio between the in-distribution set and the compositional set is 1:1. Therefore, the size of the in-distribution set for the ACD protocol is N/2.
For the Few-Shot protocol, our requirement for the in-distribution set is: 1) Each individual attribute must appear at least once, and 2) The total number of attribute combinations should be minimal. Therefore, for the Few-Shot protocol, the number attribute combinations in the in-distribution set is equal to the number of attributes in the aspect with the most attributes. Let’s assume that the aspect with the most attributes in the dataset contains M attributes. In this case, the size of the in-distribution set for the Few-Shot protocol is N ×M/|C|.
Take the Fyelp dataset as an example. The total number of data points for training generator is 70000, and the number of attribute combinations is 2 × 2 × 2 × 5 = 40. Therefore, N = 70000, |C| = 40, M = 5. Hence, the size of the in-distribution set for the Hold-Out protocol is 70000 × (40 − 1)/40 = 68250, for the ACD protocol is 70000/2 = 35000, and for the FewShot protocol is 70000× 5/40 = 8750. Similarly, we can calculate the corresponding sizes of the in-distribution sets for the other three datasets.

heading: B.3 Why Few-Shot not in Average Results?
text: In Table 1, the calculation of the Average does not include results from the Few-Shot protocol. There are two reasons for this approach: 1) According to the design principles of our ACD and Few-Shot protocols, the partitioning results for the datasets Mixture and Amazon are consistent between ACD and Few-Shot; 2) The difficult level of the Few-Shot protocol is relatively high for current models, and we aim to present this category as a direction for
future research within the community.

heading: B.4 Results in CompMCTG
text: As previously mentioned, the results in Table 1 and Table 2 represent the average outcomes across four datasets. In fact, for each dataset, the results for each protocol are derived from the average of multiple experiments.
For the Hold-Out protocol, we define it as randomly selecting one attribute combination from the complete dataset. In order to eliminate bias during the experiments, we iterate over all attribute combinations, and the final result for each dataset in the Hold-Out protocol is the average of all these results.
For the ACD protocol, we maximize attribute divergence to partition the datasets. In our experiments, there are usually multiple optimal partitioning methods, hence we also average over all cases for the final results.
Similarly, for the Few-Shot protocol, we partition the datasets by maximizing attribute divergence and take the average of all optimal partitioning results.
We present the number of partitioning methods included in different protocols across four datasets in Table 7.

heading: C Complexity discussion
text: In this section, we discuss the complexity of sweeping over all possibilities for “Half&Half” splitting (i.e.,|Ci.d.| = |Ccomp|) in Section 3.1. Following the denotations in Section 3.1: m refers to the number of different aspects; Ai, (1 ≤ i ≤ m) is the set of attribute values for the i-th aspect; min1≤i≤m |Ai| = a; the total number of attribute combinations is O(am).
Sweeping over all possible “Half&Half” splitting methods requires O( ( am
am/2
) ) kinds of situa-
tions, which can be derived as follows (using Stirling’s formula (Robbins, 1955)):
( am
am/2
) =
(am)!
(a m 2 )! · (a m 2 )! ≈ √ 2πam · (ame )a m πam · (am2e )a m
=
√ 2πam · 2am
πam
Hence O( ( am
am/2
) ) ≈ O( √ 2πam·2am πam ) = O((2 −
η)a m ) where η → 0. This complexity is exponential to am and thus unacceptable, which highly calls for an effective sampling strategy (i.e., ACD in Section 3.1).
D Implementation Details
Our implementation is based on Hugging Face Transformer models10 and we use GPT-2 Medium as our backbone for all baselines (except two LLM baselines). In this section, we provide all the hyperparameters for the baselines and our Meta-MCTG method, as well as the training hyperparameters for the classifiers used for evaluation.
First of all, we unify the settings for all experiments during the generation phase. Following previous work (Gu et al., 2022, 2023), we use the 35 prompts from PPLM (Dathathri et al., 2019) for testing. For all MCTG baselines, we generate 10 texts for each prompt and each attribute combination, each text with a length of 50, and we adopt topk=200, topp=1.0, and temperature=1.0. For two LLM baselines, due to time and financial costs, we generate only one text for each prompt and each attribute combination. All experiments are completed on an NVIDIA A100 (80G) GPU.

heading: D.1 MCTG Baselines
text: Fudge Fudge (Yang and Klein, 2021) uses a future discriminator to guide the GPT-2 for the gener-
10https://github.com/huggingface/ transformers
ation. Following previous work (Zeng et al., 2023), for each dataset, we train a Multilayer Perceptron (MLP) of dimension dembd ×m as the future discriminator, where dembd is the embedding dimension of GPT-2 Medium, and m is the number of all attribute combinations in the dataset. We set batch size to 8, epoch to 5, and learning rate to 3e-5 in the training phase for all datasets and all protocols. As for the generation, we set control strength α to 20 for all datasets and all settings.
PPLM PPLM (Dathathri et al., 2019) uses a discriminator to calculate gradient to update the states of a language model and guide the model to generate texts with a certain attribute. We train a Multilayer Perceptron of dimension dembd ×m as the discriminator-like fudge to guide the model. For each dataset and each protocol, we set the batch size to 8, epoch to 5, and learning rate to 3e-5 in the training phase. As for the generation, we followed the hyperparameters in Dathathri et al. (2019). We set γ to 1.5, num-iterations to 3, num-samples to 10, stepsize to 0.03, window-length to 5, fusion-klscale to 0.01, and fusion-gm-scale to 0.99.
Distributional Lens During the training phase, we follow all the hyperparameters of the original work (Gu et al., 2022), with the only change made to the number of training steps. We sweep across training steps from {2000,4000,6000, ...,30000} and select the minimum number of steps for convergence as our experimental setup. We summarize it in the Table 8. In the generation phase, for simplicity and fairness, we set all aspect weights to 1, and all other settings are consistent with the original paper.
Prior Proposed by (Gu et al., 2023), this method is based on the model trained in Gu et al. (2022), with the training loss of the Normalizing Flows added for further training. Therefore, during the training phase, we further train based on all models trained by method Gu et al. (2022), with the hyperparameters consistent with the original work and
only a change made to the number of training steps. Like experiments in Gu et al. (2022), we sweep across training steps from {5000, 10000, ..., 50000} and select the minimum number of steps for convergence as our experimental setup. We summarize it in the Table 9. In the generation phase, we find that aspect weights setting to 1 for the Fyelp dataset do not yield satisfactory results. Therefore, we attempt to adjust the aspect weights on this dataset and finally set weights to [12,4,24,12] corresponding to aspect ["sentiment", "gender", "cuisine", "tense"] and std to 0.1. For the other three datasets, we set weight to 1 for all aspects and set std to 1.
Catprompt As this is a naive method derived from Yang et al. (2023), there is no clear experiment setup for reference. We sweep across prompt length from {10,20,40,60,80,100,120}, selecting the length with the best test results for each attribute as our experimental hyperparameters. The specific results are as follows. For the Fyelp dataset, in the non-FewShot protocols, we set prompt length to 120, batch size to 16, epochs to 20, and learning rate to 5e-5, and in the FewShot protocol, we set prompt length to 100, batch size to 16, epochs to 40, and learning rate to 5e-5. For the Amazon dataset, we set prompt length to 10, batch size to 16, epochs to 5, and learning rate to 5e-5 for all settings. For the YELP dataset, in the non-FewShot protocols, we set prompt length to 20, batch size to 16, epochs to 20, and learning rate to 5e-5, and in the FewShot protocol, we set prompt length to 20, batch size to 16, epochs to 40, and learning rate to 5e-5. For the Mixture dataset, we set prompt length to 10, batch size to 16, epochs to 50, and learning rate to 5e-5 for all settings.
DCG Following previous work (Zeng et al., 2023), for all settings across all datasets, prompt length is set to 50 (where attribute prompt length is set to 6 and task prompt length is set to 44), the disentanglement loss weight is set to 0.1, the batch size is set to 8, and the number of Pseudo Combinations is set to 7. For the setting of epochs, we set
epochs to 3 for dataset Fyelp and Amazon, epochs to 8 for dataset YELP, and epochs to 7 for dataset Mixture. And for all datasets and protocols, we set the learning rate to 7.5e-5.
CTRL Following previous work (Zeng et al., 2023), we concatenate multi-attribute control codes with training datasets to fine-tune the GPT-2. Since we find that CTRL is not sensitive to hyperparameters, we set the batch size to 8, epochs to 5, and learning rate to 3e-5, which converges well for all datasets and protocols.
Contrastive Prefix-Tuning Following previous work (Qian et al., 2022a), we set each attribute’s prefix length to 10. For the dataset Fyelp and Amazon, we set the batch size to 8 and epochs to 2 for all protocols. For the dataset YELP, we set the batch size to 8 and epochs to 5 for all protocols. For the dataset Mixture, we set the batch size to 8 and epochs to 5 for non-FewShot protocols. For the FewShot protocol of the dataset Mixture, we set the batch size to 8 and the epoch to 10. And for all datasets and protocols, we set the learning rate to 3e-5.

heading: D.2 LLM Baselines and Prompts
text: In this section, we introduce the LLMs we use in Section 3.3 and the prompt template we used for In-Context Learning.
Prompt Following (Sun et al., 2023), we use 5-shot in context learning prompt template to evaluate the compositional generalization capacity of LLMs regarding ICL. Namely, we insert five demonstrations (Input, Output) for each time of controllable generation. Here is our prompt template:
\\5-shot in-context-learning \\prompt template "Task: write a sentence that meets the
requirement of input control conditions.
Below are some examples (Input, Output) for the task: Input: <attribute combination 1>. Output: <text 1> # demonstration_1 Input: <attribute combination 2>. Output: <text 2> # demonstration_2 Input: <attribute combination 3>. Output: <text 3> # demonstration_3 Input: <attribute combination 4>. Output: <text 4> # demonstration_4 Input: <attribute combination 5>. Output: <text 5> # demonstration_5 Input: <testing attribute combination>. Output: <a head of text>" \\ generation
For in-distribution testing, we insert five demonstrations that share the control conditions (in the attribute combinations) with the testing one. For compositional testing, we uniformly sample five demonstrations (of different attribute combinations) from the whole training set.
Another point that is worth noting is that we encode the control conditions in a standard format (e.g., in Yelp we use “cuisine-0” to represent Asian cuisine, “cuisine-1” to represent Mexican cuisine, “gender-0” to represent gender Male, “gender-1” to represent gender Female and so on). The underlying reason is that we aim to test the LLM’s ability to understand the relationship between control attributes and target text content, as well as their capacity to generalize to new combinations of previously seen control attributes.
LLM For LLaMA-2 (Touvron et al., 2023), we adopt the version of “LLaMA-2-7B-hf”11. Our generation configuration is following the default configuration provided by Meta:
\\LLaMA-2-7B generation configuration GEN_CONFIGS["llama2-7b"]={ "bos_token_id": 1, "do_sample": True, "eos_token_id": 2, "pad_token_id": 0, "temperature": 0.6, "max_length": 50, "top_p": 0.9, "transformers_version": "4.31.0.dev0" }
For ChatGPT (OpenAI, 2023), we use the OpenAIapi12 and adpot the version of “gpt-3.5-turbo0613”. The default generation configuration is as follows:
\\gpt-3.5 generation configuration GEN_CONFIGS["gpt-3.5-turbo-0613"]={ "temperature": 1.0, "max_length": 50, "top_p": 0.9, "openai_version": "0.28.0" }
Cost For the evaluation of LLaMA-2-7B, we do experiments on a NVIDIA A100 GPU for around 60 hours. For the evaluation of ChatGPT, we spend around 3.5e7 tokens in total, costing 70 dollars.
11https://huggingface.co/meta-llama/ Llama-2-7b-hf
12https://openai.com/blog/openai-api

heading: D.3 Classifiers
text: To avoid the impact of domain differences among different datasets on the accuracy of the classifier, we train a classifier using RobertaLarge (Liu et al., 2019) for each aspect of each dataset. We sweep over batch sizes from {4,8,16,32,64,128,256,512,1024} and epochs from {1,2,3,4,5,6,7,8,9,10}, choosing the settings that yield the highest accuracy on the test set as our experimental configuration. The specific configuration results and the performance of the classifiers on the test set for all datasets and all attribute aspects are shown in Table 10.

heading: D.4 Meta-MCTG
text: In the experiments of Meta-MCTG, we select the three best-performing joint-training-based methods from the baselines, namely CTRL (Keskar et al., 2019) , DCG (Zeng et al., 2023), and Contrastive Prefix (Qian et al., 2022b). For different datasets and protocols in our benchmark, we search λ from {0.01,0.05,0.1,0.2} based on the original experimental hyperparameters, and further refine the value of λ based on the results. For the majority of cases, we opt for λ to be 0.01. For the learning rate β in all MCTG experiments, we set β to be the same as the learning rate α of each baseline.

heading: E Evaluation on diversity
text: Following previous work (Li et al., 2016), we use distinctness to measure the generated text’s diversity. For each text, we calculate 3-grams named Dist-3 to evaluate distinctness. We choose to conduct diversity evaluation on the data under the three protocols of Original, Hold-Out, and ACD. The whole results are shown in Table 11.

heading: F Human Evaluation
text: Following previous work (Zhang and Song, 2022; Zhong et al., 2023), we evaluate generated texts from two aspects: Relevance (R) which reflects the degree of achievement for the desired control attribute combination and Fluency (F) which evaluates the text’s fluency. Unlike automated evaluation, where the accuracy of individual attributes is measured and averaged, human evaluation directly scores the satisfaction of the given control condition (attribute combination). For each dataset and baseline in each protocol (Original, HoldOut, and ACD), we randomly sample 10 texts (for HoldOut and ACD, we sample 10 texts from in-distribution result and 10 texts from compositional result) and employ three annotators to score them on the two metrics on a scale from 1 (very bad) to 5 (very good). Finally, we calculate the average of these scores and get the final result shown in Table 12. We can find that the results of human evaluation are consistent with the results of automated evaluation.

heading: F.1 Specific Scoring Guidelines
text: In this subsection, we provide specific scoring guidelines for each human evaluation metric.

heading: Relevance
text: • 5: The generated texts are perfectly aligned with the desired attribute combination.
• 4: The generated texts are very related to the desired attribute combination.
• 3: The generated texts are related to the desired attribute combination. At most one attribute does not match.
• 2: The generated texts are less related to the desired attribute combination. At most two attributes do not match.
• 1: The generated texts are not aligned with the desired attribute combination. None of the attributes meet the requirements.

heading: Fluency
text: • 5: The generated texts are grammatically correct, fluent, and easy to understand.
• 4: The generated texts are grammatically correct, but slightly less smooth, yet still easily understandable.
• 3: The generated texts have a few grammar errors, but do not hinder understanding.
• 2: The generated texts have a few grammar errors and are not very easy to understand.
• 1: The generated texts have many grammar errors, lack coherence, and are difficult to understand.
F.2 Inter-Annotator Agreement Score
We also use Fleiss’Kappa coefficient (Fleiss, 1971) to measure the inter-annotator agreement score for each human evaluation metric. The result is shown in Table 13.

heading: G Case Study
text: In this section, we show some specific generation examples, primarily to compare the difference in generation results before and after using the MetaMCTG framework. Cases in this section are from the compositional result of ACD protocol of dataset Fyelp. The specific results are shown in Table 14.

heading: H Algorithm Pseudo-Code
text: We conclude the pseudo-code of constructing ACD splits in Algorithm 1 and the pseudo-code of MetaMCTG training in Algorithm 2.
Following the denotations in Section 3.1: m refers to the number of different aspects; Ai, (1 ≤ i ≤ m) is the set of attribute values for the ith aspect; min1≤i≤m |Ai| = a; the total number of attribute combinations is O(am). The time complexity of Algorithm 1 (Greedily constructing ACD splits) is O(T1 · T2 · am) (linearly increasing with am) which is much better than O((2 − ϵ)am), (ϵ ← 0) (exponentially increasing with am) in Appendix C.

heading: I Additional Results
text: I.1 Why do Separate-Training-based methods perform badly in compositional testing?
In this section, we briefly discuss the reasons why the seperate-training-based MCTG methods fail in compositional testing. We take Dis-Lens (Gu et al., 2022) as an example to illustrate. This type
Algorithm 1 Constructing ACD splits Require: Attribute combination set C. Require: Divergence function D(·, ·). Require: Maximum step T1, T2, maximum diver-
gence threshold η ∈ (0, 1). 1: Initialization: current step t1 = 0; maximum
divergence dm = 0. 2: A set of ACD splits result = ∅. 3: while t1 < T1 do 4: t1 = t1 + 1 5: Randomly split C into Ci.d. and Ccomp where |Ci.d.| = |Ccomp|. 6: t2 = 0 7: Compute current divergence d: d = D(Ci.d., Ccomp). 8: Update maximum divergence: dm = d. 9: while t2 < T2 do 10: t2 = t2 + 1 11: c1 = None. 12: for c ∈ Ci.d. do 13: if dm < D(Ci.d. − {c}, Ccomp + {c}) then 14: c1 = c. 15: dm = D(Ci.d.−{c}, Ccomp+ {c}). 16: break 17: end if 18: end for 19: if c1 == None then 20: continue 21: end if 22: Ci.d. = Ci.d. − {c1}. 23: Ccomp = Ccomp + {c1}. 24: for c ∈ Ccomp do 25: if dm < D(Ci.d. + {c}, Ccomp − {c}) then 26: dm = D(Ci.d.+ {c}, Ccomp−{c}). 27: Ci.d. = Ci.d. + {c1}. 28: Ccomp = Ccomp − {c1}. 29: break 30: end if 31: end for 32: end while 33: for dm ≥ η do 34: Add (Ci.d.,Ccomp) into result. 35: end for 36: end while 37: return result
Algorithm 2 Meta-MCTG Require: Training set Dtrain Require: Base MethodM Require: Learning rate α, β, batch size m
1: while not done do 2: Sample m data as the training batch Btrain = (ctraini , xtraini )mi=1 from Dtrain. 3: Construct pseudo-compositional batch
Bpcomp = (cpcompi , x comp i ) m i=1 by sampling another m data from Dtrain, where {ctraini }mi=1 ∩ {cpcompi }mi=1 = ∅ while each single attribute condition in Bpseudo−comp must appear in the Btrain.
4: Compute training loss LMtrain through Objective 5. 5: Compute θ1 through Equation 6. (while not really update θ to θ1) 6: Temporarily use θ1 in the language model. 7: Compute pseduo compositional generalization loss LMp−comp through Objective 7. 8: Compute total loss LMtotal through Objective 8. 9: Update θ to θ′ through Equation 9
10: end while
of method encodes each single attribute data into a latent vector space, and then constructs the intersection of different attribute latent vector areas through loss function constraints, and finally guides GPT-2 to generate multi-aspect text by searching for the intersection of different attribute spaces. The essential reason why this method can work is that the training dataset itself has multiple attributes. For example, the data corresponding to the latent space intersection constructed with positive emotion data and sports theme data actually has these two attributes. Therefore, when using a multi-attribute dataset to train the latent vector space, the attribute combinations corresponding to the constrained intersection space are the attribute combinations contained in the training set, and will not produce attribute combinations that do not exist in the training set.
Specifically, we use a Few-Shot split of the dataset Mixture to conduct experiments, reducing the dimensionality of hidden vectors to a two-dimensional plane through PCA and performing visualization processing. There are four attribute combinations in the training set which are "Negative-movies", "Negative-opener", "Negativetablets", and "Positive-auto". Figure 6, 7 depict
the results of pre-training visualizations, while Figure 8, 9 show the results of post-training’s counterpart. Figure 6, 8 are annotated with multi-aspect labels, whereas Figure 7, 9 are annotated with singleaspect labels. From these four figures, we can find that after training, the hidden vector spaces corresponding to different single attributes have converged, and the intersection of four multi-attribute latent vector spaces has been formed. However, through Figure 8, it can be found that these four intersections exactly correspond to the four attribute combinations contained in the training set, and the intersection of the latent vector spaces of the four compositional attribute combinations ("Negativeauto", "Positive-movies", "Positive-opener", and "Positive-tablets") in Figure 9 basically does not exist. This explains why such methods fail in compositional testing.

heading: I.2 Analysis Experiments
text: In this section, we conduct visualization experiments on the Meta-MCTG framework we proposed, indirectly verifying its effectiveness. Consider-
ing that the joint-training-based MCTG methods tend to overfit the control parameters to the indistribution (I.D.) attribute combinations, this implies that for compositional (Comp.) attribute combinations, their control parameters are relatively close to those of in-distribution. Therefore, we approach this from the perspective of control parameters, calculating the L1 distance L1base, L1meta and cosine similarity Cosbase, Cosmeta between the control parameters before and after the introduction of the Meta-MCTG framework, and use the difference diffL1 = L1meta−L1baseL1meta × 100, diffCos = −Cosmeta−CosbaseCosmeta × 100 between the two as the data for visualization.
We select CTRL (Keskar et al., 2019) , DCG (Zeng et al., 2023), and Contrastive Prefix (Qian et al., 2022b) and conduct our visualization experiments on ACD protocol of YELP (YELP, 2014) and Fyelp (Lample et al., 2019) datasets. For CTRL, we use the mean embeddings of its attribute tokens (i.e., control codes) as the control parameters. For DCG, we use the mean embedding obtained by encoding the attribute tokens through a
fully connected layer as the control parameters. For Contrastive Prefix, we use the mean embedding of the prefix keys and prefix values of the corresponding attributes in the last layer of the GPT-2 as the control parameters. On the YELP dataset, there are a total of 8 attribute combinations, including 4 in-distribution and 4 compositional. For the control parameters under 8 control conditions, we compute the difference diffL1 and diffCos between each pair and obtain two 4× 8 heatmaps for each baseline. Similarly, for the Fyelp dataset, we can get two 20 × 40 heatmaps for each baseline. The results are shown in Figure 10 and Figure 11. The visual results show that the control parameters after the Meta-MCTG training framework can better distinguish between the in-distribution and compositional parts, thus confirming the effectiveness of the Meta-MCTG framework.

heading: I.3 Detailed Results on the Single Dataset
text: In this section, we provide detailed experimental results of all baselines (eight MCTG baselines and two LLMs) in CompMCTG Benchmark in 4
datasets. In these tables, the first column contains the protocol, including Original, HoldOut, ACD, and FewShot (Amazon and Mixture datasets do not have ACD protocol). Holdout, ACD, and FewShot respectively divide the in-distribution (I.D.) results and compositional (Comp.) results. The second column is the method name and the next two to four columns are the accuracy of the corresponding attributes. Specifically, Accs, Accg, Accc, Acct in Fyelp are the accuracy of sentiment, gender, cuisine, and tense respectively. Accs, Acct in Amazon are the accuracy of sentiment and topic respectively. Accs, Accp, Acct in YELP are the accuracy of sentiment, person, and tense respectively. Accs, Acct in Mixture are the accuracy of sentiment and topic respectively. Accavg is the average of accuracy in previous columns. Accgap is calculated from the I.D. and Comp. of the corresponding protocol. The specific formula is Accgap =
AccI.D.−AccComp. AccI.D.
× 100%. PPL represents perplexity and Dist3 is the value of 3-ngrams. All the detailed experimental results can be found in Table 15 to Table 54.

heading: Future Work
text: Fudge Fudge (Yang and Klein, 2021) uses a future discriminator to guide the GPT-2 for the gener-
10https://github. com/huggingface/ transformers
ation. Following previous work (Zeng et al. , 2023), for each dataset, we train a Multilayer Perceptron (MLP) of dimension dembd ×m as the future discriminator, where dembd is the embedding dimension of GPT-2 Medium, and m is the number of all attribute combinations in the dataset. We set batch size to 8, epoch to 5, and learning rate to 3e-5 in the training phase for all datasets and all protocols. As for the generation, we set control strength α to 20 for all datasets and all settings. PPLM PPLM (Dathathri et al. , 2019) uses a discriminator to calculate gradient to update the states of a language model and guide the model to generate texts with a certain attribute. We train a Multilayer Perceptron of dimension dembd ×m as the discriminator-like fudge to guide the model. For each dataset and each protocol, we set the batch size to 8, epoch to 5, and learning rate to 3e-5 in the training phase. As for the generation, we followed the hyperparameters in Dathathri et al. (2019). We set γ to 1. 5, num-iterations to 3, num-samples to 10, stepsize to 0. 03, window-length to 5, fusion-klscale to 0. 01, and fusion-gm-scale to 0. 99. Distributional Lens During the training phase, we follow all the hyperparameters of the original work (Gu et al. , 2022), with the only change made to the number of training steps. We sweep across training steps from {2000,4000,6000,. ,30000} and select the minimum number of steps for convergence as our experimental setup. We summarize it in the Table 8. In the generation phase, for simplicity and fairness, we set all aspect weights to 1, and all other settings are consistent with the original paper. Prior Proposed by (Gu et al. , 2023), this method is based on the model trained in Gu et al. (2022), with the training loss of the Normalizing Flows added for further training. Therefore, during the training phase, we further train based on all models trained by method Gu et al. (2022), with the hyperparameters consistent with the original work and
only a change made to the number of training steps. Like experiments in Gu et al. (2022), we sweep across training steps from {5000, 10000,. , 50000} and select the minimum number of steps for convergence as our experimental setup. We summarize it in the Table 9. In the generation phase, we find that aspect weights setting to 1 for the Fyelp dataset do not yield satisfactory results. Therefore, we attempt to adjust the aspect weights on this dataset and finally set weights to [12,4,24,12] corresponding to aspect ["sentiment", "gender", "cuisine", "tense"] and std to 0. 1. For the other three datasets, we set weight to 1 for all aspects and set std to 1. Catprompt As this is a naive method derived from Yang et al. (2023), there is no clear experiment setup for reference. We sweep across prompt length from {10,20,40,60,80,100,120}, selecting the length with the best test results for each attribute as our experimental hyperparameters. The specific results are as follows. For the Fyelp dataset, in the non-FewShot protocols, we set prompt length to 120, batch size to 16, epochs to 20, and learning rate to 5e-5, and in the FewShot protocol, we set prompt length to 100, batch size to 16, epochs to 40, and learning rate to 5e-5. For the Amazon dataset, we set prompt length to 10, batch size to 16, epochs to 5, and learning rate to 5e-5 for all settings. For the YELP dataset, in the non-FewShot protocols, we set prompt length to 20, batch size to 16, epochs to 20, and learning rate to 5e-5, and in the FewShot protocol, we set prompt length to 20, batch size to 16, epochs to 40, and learning rate to 5e-5. For the Mixture dataset, we set prompt length to 10, batch size to 16, epochs to 50, and learning rate to 5e-5 for all settings. DCG Following previous work (Zeng et al. , 2023), for all settings across all datasets, prompt length is set to 50 (where attribute prompt length is set to 6 and task prompt length is set to 44), the disentanglement loss weight is set to 0. 1, the batch size is set to 8, and the number of Pseudo Combinations is set to 7. For the setting of epochs, we set
epochs to 3 for dataset Fyelp and Amazon, epochs to 8 for dataset YELP, and epochs to 7 for dataset Mixture. And for all datasets and protocols, we set the learning rate to 7. 5e-5. CTRL Following previous work (Zeng et al. , 2023), we concatenate multi-attribute control codes with training datasets to fine-tune the GPT-2. Since we find that CTRL is not sensitive to hyperparameters, we set the batch size to 8, epochs to 5, and learning rate to 3e-5, which converges well for all datasets and protocols. Contrastive Prefix-Tuning Following previous work (Qian et al. , 2022a), we set each attribute’s prefix length to 10. For the dataset Fyelp and Amazon, we set the batch size to 8 and epochs to 2 for all protocols. For the dataset YELP, we set the batch size to 8 and epochs to 5 for all protocols. For the dataset Mixture, we set the batch size to 8 and epochs to 5 for non-FewShot protocols. For the FewShot protocol of the dataset Mixture, we set the batch size to 8 and the epoch to 10. And for all datasets and protocols, we set the learning rate to 3e-5