Controlled Text Generation for Large Language Model with Dynamic Attribute Graphs (2024)

Abstract: Controlled Text Generation (CTG) aims to produce texts that exhibit specific desired attributes. In this study, we introduce a pluggable CTG framework for Large Language Models (LLMs) named Dynamic Attribute Graphs-based controlled text generation (DATG)1. This framework utilizes an attribute scorer to evaluate the attributes of sentences generated by LLMs and constructs dynamic attribute graphs. DATG modulates the occurrence of key attribute words and key anti-attribute words, achieving effective attribute control without compromising the original capabilities of the model. We conduct experiments across four datasets in two tasks: toxicity mitigation and sentiment transformation, employing five LLMs as foundational models. Our findings highlight a remarkable enhancement in control accuracy, achieving a peak improvement of 19.29% over baseline methods in the most favorable task across four datasets. Additionally, we observe a significant decrease in perplexity, markedly improving text fluency. CONTENT WARNING: This document, for the purpose of illustrating tasks related to toxicity in CTG, may contain examples that are offensive. Please read selectively.

text: Findings of the Association for Computational Linguistics ACL 2024, pages 5797–5814 August 11-16, 2024 ©2024 Association for Computational Linguistics
CONTENT WARNING: This document, for the purpose of illustrating tasks related to toxicity in CTG, may contain examples that are offensive. Please read selectively.

heading: 1 Introduction
text: Controlled Text Generation (CTG) focuses on generating text adhering to specific conditions or attributes, such as sentiment, non-toxicity(Liu et al., 2021; Pei et al., 2023) and style(Konen et al., 2024; Tao et al., 2024). In the realm of CTG, achieving precise control over specific attributes of the generated content is a significant challenge. This
*Equal contribution †Corresponding author 1Our code is available at https://github.com/
IAAR-Shanghai/DATG
Instance: The novel is a masterpiece / success / disappointment / disaster of storytelling, with a complex narrative.
must be accomplished without compromising the generative capabilities and text quality of LLMs.
Traditionally, CTG methods have employed small language models to influence the decoding process of larger models(Dathathri et al., 2020; Krause et al., 2021; Yang and Klein, 2021). Though this approach provides a degree of control, it may compromise the inherent quality and variability of the output. Recent studies (Zhong et al., 2023) highlight that an overemphasis on control can detrimentally affect text fluency, rendering the content less effective. This issue underscores a critical insight: excessive reliance on smaller language models to steer the outputs of LLMs can diminish the decoding capabilities inherent to LLMs. When small-scale models assume control, they effectively overshadow the original performance of LLMs during the inference and decoding phase. This process not only masks the vast capabilities of LLMs but also relegates them to a subordinate role, essentially transforming these sophisticated generative models into mere “puppets” of their smaller counterparts.
In light of our exploration, we think the specific attributes of a text are predominantly determined by a limited number of words that bear close relation
5797
to those attributes (Zhong et al., 2023). Despite these key words being sparse within the text, their impact on the overall attributes is decisive. For instance, changing the word “masterpiece” to “failure” in the sentence “The novel is a masterpiece of storytelling, with a complex narrative.” shifts the sentiment from positive to negative. This change alters the entire sentence’s sentiment and meaning. In the conceptual framework of semantic space, these attributes can be seen as dimensions within this space. By strategically adjusting these key words, we can guide the text generated by LLMs to move in the desired direction within the semantic space, controlling its attributes without significant alterations to the overall content (See Figure 1).
Based on these observations, we propose a pluggable CTG approach, Dynamic Attribute Graphsbased controlled text generation (DATG), which employs dynamic attribute graphs to identify key words aligned or opposed to target attribute dimensions. By modulating the occurrence of these key words, our method precisely controls text attributes without compromising the inherent capabilities of LLMs. This strategy allows for targeted movement
within the semantic space.
As described in Figure 2. Our work begins with Contextual Corpus Construction, where LLMs generate text sequences from specific prompts. Subsequently, Attribute Classifier Scoring assesses these texts with classifiers, such as toxicity or sentiment classifiers, to evaluate alignment with the target attribute. The core of our method, Dynamic Attribute Graphs Construction, transforms the text sequences into directed weighted graphs, informed by classifier scores. This process leads to the creation of two distinct graphs: a positive attribute graph, weighted by the consistency scores from the classifier, and a negative attribute graph, weighted by the complements of these scores. The attribute graphs represent the text’s adherence to and deviation from the target attribute dimension within the semantic space. During the ReGeneration with Dynamic Boundary Controlling process, the graph ranking algorithm selects key nodes that propel the generated text towards the upper boundary of the control attribute dimension in the semantic space. Adjustments of the occurrence of these key nodes, facilitated by
logits-boost and prefix-prompt strategies, enable the regeneration of text.
The key contributions of our study are summarized as follows:
• We introduce a pluggable DATG framework that integrates dynamic attribute graphs with LLMs for CTG, providing a novel, flexible approach to attribute-driven text generation.
• DATG achieves a peak improvement of 19.29% in performance over baseline methods, according to comprehensive experiments across various datasets, and significantly enhances text fluency.
• We reintroduce the application of graph models in CTG tasks, offering new insights for controlled text generation with LLMs.

heading: 2 Methodology

heading: 2.1 Problem Definition
text: The generative capability of LLMs is characterized by the probability distribution over a sequence X:
P (xn|X1:n−1) = p(xn|x1, x2, . . . , xn−1), (1)
where xn represents the token currently being generated, and X1:n−1 includes the sequence of tokens generated prior to xn. This probabilistic framework allows LLMs to produce text sequences that are diverse and coherent.
In the domain of CTG, control conditions C are integrated into the generative process to steer the text towards exhibiting specific attributes, such as sentiment and toxicity. This can be formulated as:
P (X|C) = n∏
i=1
p(xi|x<i, C), (2)
where C signifies the desired attributes to be reflected in the generated text. The key challenge in CTG is to integrate C into the generative process seamlessly, maintaining the LLMs’ inherent generative quality.
We consider the problem within the framework of a semantic space S ⊂ Rd, where outputs of LLMs are mapped as vectors. In this semantic space S , our goal is to adjust dimensions associated with control conditions C, directing the distribution of text towards desired attributes while preserving the integrity of other semantic dimensions. This
objective is achieved through a transformation function f , designed to delicately shift semantic vectors without altering their inherent characteristics:
J(f) = Ex∼P (S)[s(f(x))], (3)
where J(f) evaluates the effectiveness of f in aligning text generation with control conditions C, and s(·) measures the semantic vector’s conformity to these conditions. To depict the vector transition within S towards desired attributes, we employ the transformation equation:
xafter = f(xbefore) = xbefore +∆x, (4)
Leveraging attribute graphs, we identify key words that significantly influence the LLMgenerated sentences in the semantic space S , along the control attribute dimension. By adjusting the occurrence of just a few key words, we not only preserve the original performance of LLMs but also effectively steer the regenerated text towards desired conditions. This method effectively guides the text towards specified attributes, maintaining semantic integrity and coherence.

heading: 2.2 Contextual Corpus Construction
text: Recent studies, including LIMA (Zhou et al., 2023a) and Re-Align (Lin et al., 2024), affirm that the foundational knowledge and capabilities of LLMs are established predominantly during the pre-training phase. This evidence suggests that unaligned base models already possess the capacity to generate the desired texts.
Guided by the principles of the LIMA hypothesis and findings from Re-Align, our approach commences with the generation of a sentence set, symbolized as X, using an LLM prompted by a query that is intricately tied to the desired context. This initial phase leverages the LLM’s pre-trained knowledge to generate text sequences closely aligned with the prompt’s context, reflecting the inherent distribution of text in the semantic space produced by large language models.
The set comprises individual sentences, Xj , each generated in response to the initial prompt, represented as X = {X1, X2, . . . , Xm}. Each sentence Xj is a sequence of tokens {x1j , x2j , . . . , xnjj}, where nj denotes the sentence’s token count. This constructs a contextual corpus foundational for subsequent manipulations.

heading: 2.3 Attribute Classifier Scoring
text: To align generated texts with specific attributes like toxicity or sentiment levels, we employ a pretrained language model enhanced with a classification layer. This classifier is fine-tuned on data tailored to the target attribute, enabling a conditionspecific classifier to precisely evaluate and quantify attribute presence and intensity.
The classifier model scores each text Xi in X = {X1, X2, . . . , Xm} as:
s(Xi) = ClassifierModel(Xi), (5)
where s(Xi), between 0 and 1, reflects how well Xi exhibits the target attribute and assesses text distribution along the control condition in the semantic space. This scoring, a quantitative metric, aids in evaluating attribute representation in X and understanding text alignment with control conditions.

heading: 2.4 Dynamic Attribute Graphs Construction
text: In the dynamic attribute graphs construction phase, each sentence Xj in X is tokenized into discrete tokens, forming vertex sets Vj = {v1,j , v2,j , . . . , vnj ,j} for each sentence:
V = m⋃
j=1
Vj , (6)
where vi,j represents a distinct token from sentence Xj , and V is the union of all vertex sets Vj .
Directed edges within each Vj are defined by sequentially linking tokens to reflect their order in the sentence:
Ej = {(vi,j , vi+1,j) | vi,j , vi+1,j ∈ Vj}, (7)
The overall edge set E is then defined as the union of all Ej , reflecting the aggregation of directed edges from all sentences:
E = m⋃
j=1
Ej , (8)
In the dynamic attribute graphs (G+ for positive influence and G− for negative influence), the framework is defined to encapsulate the relationships tokens have with the control attribute, representing the semantic space boundaries shaped by these influences. The cumulative weights for
each edge, reflecting the total influence across all sentences, are formalized for both graphs as:
G± = (V,E,W±), (9)
where W± is the set of cumulative weights for edges, determined by aggregating attribute classifier scores, and is calculated as:
W± =   w ± ik | w±ik = ∑
j
w±ik,j    , (10)
with the weights w+ik,j = s(Xj) for G + and w−ik,j = 1 − s(Xj) for G−, corresponding to the direct and inverse classifier score influences of sentence Xj on the edge from token vi to vk.
Applying a graph ranking algorithm to the dynamic attribute graphs, G+ and G−, identifies key tokens that affect the text’s alignment with the target attribute. This method evaluates the importance of tokens based on their connectivity and the weights of their connections, distinguishing tokens’ positive or negative influence on the attributes.
For G+, the graph ranking algorithm highlights tokens that positively influence the attribute through W+; for G−, it identifies with negative impacts using W−. Key tokens are identified as:
VPos = {vi ∈ V |GraphRanking(G+) > θp}, (11) VNeg = {vi ∈ V |GraphRanking(G−) > θn}, (12) Thresholds θp and θn are used to identify key tokens with a significant influence from G+ and G−, respectively:
• Boost the occurrence of key tokens identified in G+ during text regeneration.
• Suppress the occurrence of key tokens identified in G− during text regeneration.
By enhancing or reducing the occurrence of key tokens, we facilitate the movement of text within the semantic space towards the desired attribute direction.

heading: 2.5 ReGeneration with Dynamic Boundary Controlling
text: Positive and Negative Nodes in dynamic attribute graphs inherently represent the semantic space boundaries of LLM’s generative capabilities. These nodes act as natural boundary anchors, directing
the text’s semantic trajectory towards or away from specific attributes. Activating Positive Nodes aligns the text with desired attributes, moving it closer to the upper boundary, while suppressing Negative Nodes helps avoid undesired attributes, distancing it from the lower boundary. Through logits-boost and prefix-prompt strategies, we precisely manipulate these boundaries to control the text’s semantic orientation, ensuring alignment with desired attributes or distancing from undesired ones.
Logits-Boost Strategy. The Logits-Boost method influences token probabilities associated with Positive and Negative Nodes by adjusting logits in the LLM’s generation algorithm. By enhancing logits for Positive Nodes and reducing those for Negative Nodes before the softmax operation, we achieve precise control over the model’s output:
P̃ (Xt|x<t) = softmax(zt + α · 1Pos − β · 1Neg) (13) Here, zt is the original logits, 1Pos and 1Neg indicate Positive and Negative Nodes, and α, β control the adjustment extent. This selective logits modification aligns the output with control conditions without significantly affecting text fluency, as it only dynamically adjusts the probabilities of a few attribute-related words.
Prefix-Prompt Strategy. Alongside logits adjustment, we employ the Prefix-Prompt strategy to guide LLM towards highlighting Positive Nodes and avoiding Negative Nodes. By appending specific prefixes to prompts, like “The following passage often discusses [Positive Words] but does not mention [Negative Words].”, we steer content generation in line with control conditions. This approach, combined with logits modification, ensures that generated text aligns with desired attributes while maintaining fluency and coherence.

heading: 3 Experiments

heading: 3.1 Tasks Setup
text: Inspired by the CTG capabilities demonstrated in PREADD (Pei et al., 2023), we designed our experiments around two principal tasks, utilizing datasets annotated for specific attributes. (1) Toxicity Mitigation Task: We employ the RealToxicityPrompts dataset (Gehman et al., 2020) to evaluate our method’s ability to reduce toxicity in generated texts. We use two evaluation sets: RandomToxic and TopToxic, focusing on broad toxicity mitigation and critical toxicity reduction, respectively.
(2) Sentiment Transformation Task: Utilizing the SST-5 dataset (Socher et al., 2013), we examine our method’s effectiveness in transforming the sentiment of movie reviews. Evaluation sets include NegToPos and PosToNeg for transforming negative to positive sentiments and vice versa. More details are provided in Appendix A.1.

heading: 3.2 Base LLMs
text: Our experiments utilize a range of base models with varying sizes and originating from AI research institutions: Phi-2 2.7B from Microsoft Research (Hughes, 2023), OPT 6.7B from Meta AI (Zhang et al., 2022), Alpaca 7B from Stanford University (Taori et al., 2023), Falcon 7B from Technology Innovation Institute (Almazrouei et al., 2023), LLaMA-2 13B from Meta AI (Touvron et al., 2023). For more details, see Appendix A.2.

heading: 3.3 Classifier Models
text: To measure the alignment of generated texts with desired attributes, we employ an embedding model, the BAAI/bge-large-en-v1.5 model (Xiao et al., 2023), augmented with an external classifier head. This classifier is fine-tuned on texts with specific attributes to enhance the evaluation of text attribute consistency.
For toxicity mitigation, the Jigsaw Toxic Comment Classification Challenge dataset (cjadams et al., 2017) was utilized to train a classifier distinguishing toxic from non-toxic content. In sentiment transformation, the IMDB dataset (Maas et al., 2011) enabled the training of a sentiment classifier to steer text generation towards the desired sentiment, aligning the emotional tone with the task. More details are provided in Appendix A.3.

heading: 3.4 Baselines
text: We compare DATG against four baselines in controlled text generation:
CONTINUATION: The normal continuation of text generation without any control. INJECTION: Injects specific prompts into the generation process to guide the model towards the desired attribute efficiently. FUDGE (Yang and Klein, 2021): Utilizes an attribute predictor to condition text generation on desired attributes. PREADD: Employs manipulation of output logits from prompts for attribute control. Additionally, we introduce two variations of our approach for comparison: DATG-L: Utilizes the Logits-Boost strategy for probability ad-
justment to guide text generation towards desired attributes. DATG-P: Applies the Prefix-Prompt strategy for adjustment, using prefixes to steer the generation process towards the desired attributes. More details are provided in Appendix A.4.

heading: 3.5 Metrics
text: To effectively evaluate the outcomes of our tasks, we utilize metrics as follows:
(1) Toxicity: For assessing the toxicity mitigation task, we measure the toxicity of generated texts using the Perspective API by Jigsaw 2. (2) Success Rate: For assessing the sentiment transformation task, success is determined by the proportion of text successfully transformed to the desired sentiment, evaluated with a RoBERTa model finetuned on SST-5. (3) Perplexity: Applied to both tasks, perplexity measures the fluency of text, using GPT-2 large for assessment. (4) Relevance: Relevance evaluates the contextual alignment between the prompt and its completion, measured by cosine similarity between their embeddings. Detailed metrics are provided in Appendix A.5.

heading: 3.6 Result Analysis
text: Toxicity Mitigation Analysis In our experiments, the DATG-L method consistently ranks among the top 2 across all tasks in the crucial metrics of toxicity and perplexity, as demonstrated in Table 1. This performance shows a significant reduction in toxicity without sacrificing text fluency, effectively validating our hypothesis.
The DATG-L approach, by adjusting the logits distribution of a few key attribute words, minimally impacts the overall performance during most decoding moments by the large model. Its outstanding performance on the ToxicTop dataset further confirms our assumption that a few attribute words play a decisive role in determining the overall sentence attribute, achieving the best toxicity scores across all models. Sentences from the ToxicTop dataset often exhibit higher toxicity due to extreme words, and adjusting a few key words is effective in reducing toxicity. In contrast, ToxicRandom texts may lack distinct attribute words, yet DATG-L effectively mitigates toxicity. Despite DATG-P’s weaker performance, it scores higher in relevance, likely due to key words in prefix-prompts acting as anchors for generating attribute-related text. Its adherence to prompts is limited by the base LLMs’
2www.perspectiveapi.com
directive-following capacity. It is noteworthy that INJECTION performs worse than simple CONTINUATION. This may stem from our use of a contextual prompt: “The following text doesn’t perpetuate negative stereotypes, isn’t threatening or sexually explicit, and does not contain profane language.” Such prompts can unintentionally act as “anchors,” prompting models to produce the very content they are instructed to avoid. This effect, akin to being told not to think of a cat and then picturing one, was similarly observed in our PREADD baseline experiments where negative prompts inadvertently prompted related content.
As shown in Table 2, DATG-L shows superior performance across models, leading in toxicity and perplexity on the ToxicTop dataset, with a 19.29% improvement in toxicity over the best baseline, and surpassing INJECTION in fluency by 41.65% over PREADD and 90.79% over FUDGE. FUDGE’s perplexity varies greatly, likely due to its classifier’s direct control disrupting LLMs’ distributions at high toxicity levels, aligning with our Air-decoding findings that too much control diminishes text quality. DATG-L also tops toxicity mitigation performance on ToxicRandom.
The DATG approach effectively reduces toxicity while preserving text fluency, validating our hypotheses about the impact of attribute words.
Sentiment Transformation Analysis In sentiment transformation tasks, our DATG approach consistently ranks in the top 2 across all tasks. However, unlike the toxicity tasks, DATG-L and DATGP show varying performances on the Neg2Pos and Pos2Neg datasets, as shown in Table 5. For Neg2Pos, DATG-L excels, achieving the best rates in perplexity and success across all models except for Phi-2 2.7B, where it slightly trails behind PREADD in success rate. Notably, its perplexity is even lower than the INJECTION method, which relies on the large model’s inherent generation capabilities. This suggests that base models may become disoriented when receiving contradictory injection directives and prompts, disrupting the natural distribution of the generated text. In the Pos2Neg task, DATG-P ranks among the top performers in all models, maintaining high fluency.
Across the five models, DATG-L stands out in the Neg2Pos dataset, surpassing the best baseline by 12.61% in success rate, while DATG-P, although slightly below FUDGE in success rate on
the Pos2Neg dataset, improves fluency by 79.70% compared to FUDGE (See Table 3). This reinforces the idea that direct control by smaller models over decoding can degrade the quality of text generated by large models, especially in sentiment transformation tasks where the prompt and generated text undergo significant changes. FUDGE’s method of directly controlling the large model’s decoding disrupts the inherent distribution during decoding.
Thus, in sentiment transformation tasks, our DATG methods effectively control sentiment while preserving text fluency, demonstrating their capability to balance successful attribute transformation with maintaining the quality of the generated text.
DATG-L vs. DATG-P DATG-L and DATG-P demonstrate varied adaptability depending on the type of base LLMs and the nature of the tasks.
Model Type Adaptability: DATG-L is ideal for white-box or grey-box environments, allowing modifications to model internals like output logits for direct control over attribute generation. It suits settings needing deep integration with the model’s functions. Conversely, DATG-P is suited for black-box scenarios, using prompt engineering to influence outputs without accessing internal mechanisms, making it versatile for various LLMs permitting only external interactions.
Task Type Suitability: The effectiveness of DATG-L and DATG-P also varies with the task objectives, particularly in relation to the LLMs’ “mainstream generation style.” This style refers to the default content generation tendency of LLMs, which is shaped by the most prevalent language patterns in their training data. Typically, LLMs are predisposed to generate non-toxic and positive content due to the predominance of such data in their training corpus. For tasks like toxicity mitigation or transforming negative sentiments to positive (NegtoPos), where the objectives align with the LLMs’ mainstream generation style, DATG-L
performs better. It fine-tunes the text attributes by subtly adjusting the generation probabilities of unwanted vocabulary, enhancing the alignment with desired attributes without drastic deviation from the model’s natural output tendencies. Conversely, for tasks that require a significant deviation from the LLMs’ mainstream style, such as converting positive to negative sentiments (PostoNeg), DATG-P is more effective. By embedding specific negative sentiment words within prompts, this method sets a new directional bias in the generation process. This “anchoring” of key words in the prompt explicitly guides the LLM away from its default positive generation tendency, facilitating the production of content that meets the task’s unique objectives.
Generation Speed Analysis As Figures 3 and 4 demonstrate, DATG-L and DATG-P significantly outperform PREADD and FUDGE by 32.67% and 40.02%, respectively. This underscores the efficiency of our methods, even with the inclusion of steps for generating contextually relevant corpora.
Using Alpaca-7B as an example, the average computation times for each stage of DATG, along with natural generation, are presented in Table 4. The minimal time required for Dynamic Attribute Graphs Construction and the primary computational load on Contextual Corpus Construction
highlight potential areas for speed enhancement through pre-constructing large attribute graphs.
For example, in toxicity tasks, we can predefine common issues such as gender discrimination, child abuse, and animal abuse. For each toxicity type, we can pre-construct contextually relevant corpora and attribute graphs. Upon receiving a specific prompt, we search the pre-constructed attribute graph for a related subgraph, perform graph ranking, and extract key attribute words.
This strategy could accelerate the generation process, potentially matching the speed of natural generation based on the computation times listed.

heading: 4 Related Work

heading: 4.1 Retrain
text: Retraining approaches in Controlled Text Generation integrate control mechanisms into model architectures, often requiring additional data or constraints. Models like CTRL (Keskar et al., 2019), POINTER (Zhang et al., 2020), Mention Flags (Wang et al., 2021), and DIRECTOR (Arora et al., 2022) demonstrate various levels of control from global themes to specific lexical choices. However, these methods are computationally intensive and constrained by the availability of annotated data, posing challenges alongside the rise of LLMs.

heading: 4.2 Fine-tuning
text: Fine-tuning has emerged as an effective strategy to adapt PLMs to specific tasks in CTG. Minimal parameter optimization approaches, such as PrefixTuning (Li and Liang, 2021) and DART (Nan et al., 2021), enhance efficiency. Techniques like Contrastive Prefixes (Qian et al., 2022) and DisCup (Zhang and Song, 2022) improve generation quality and control. Prompt-based methods, including AutoPrompt (Shin et al., 2020) and p-Tuning (Lester et al., 2021), leverage the PLMs’ latent knowledge without substantial changes. Advances in instruction-based models, such as FLAN (Wei et al., 2022) and InstructCTG (Zhou et al., 2023b), have made significant strides in zero-shot learning performance.

heading: 4.3 Decoding
text: During decoding, CTG has significantly advanced with auxiliary models and classifiers guiding LLMs. Techniques such as Plug and Play Language Models (PPLM) (Dathathri et al., 2020), FUDGE (Yang and Klein, 2021), CAIF (Sitdikov et al., 2022), and
CriticControl (Kim et al., 2023) utilize classifiers for directing generation. These classifiers modulate text direction and style, interfacing with LLMs. However, this approach may slow decoding due to sentence attribute evaluations.
Concurrently, Class-Conditioned Language Models (CCLMs) and Prefix-Conditioned Language Models (PCLMs) offer alternatives. Methods like DExperts (Liu et al., 2021), GeDi (Krause et al., 2021), CounterGeDi (Saha et al., 2022), and Air-Decoding (Zhong et al., 2023) leverage CCLMs or PCLMs for guidance.
In addition to methods that use classifiers for assistance, methods such as Self-Debiasing (Schick et al., 2021), Self-Detoxifying (Leong et al., 2023), PREADD, and RAIN (Li et al., 2024) exploit the inherent strengths of LLMs for nuanced control. Additionally, Goodtriever (Pozzobon et al., 2023) uses retrieval-augmented models for toxicity control. However, external model guidance may compromise text quality, especially under restrictive conditions, leading to attribute collapse (Zhong et al., 2023).

heading: 5 Conclusion
text: In this paper, we present Dynamic Attribute Graphs-based controlled text generation (DATG), a flexible and pluggable framework that seamlessly integrates graph models with LLMs to refine CTG. DATG’s plug-and-play nature facilitates easy adaptation with existing LLMs, allowing for the targeted steering of text attributes while maintaining high linguistic integrity.
Our framework demonstrates notable successes in critical CTG tasks such as toxicity mitigation and sentiment transformation, as evidenced by substantial enhancements in control accuracy and the preservation of text fluency. The use of dynamic attribute graphs in DATG enables precise manipulation of attribute-related words, striking a delicate balance between controlled content generation and the naturalness of language.
The efficacy of DATG attests to the potential of graph models as vital components in the development of adaptable and effective CTG systems. This work not only showcases the capabilities of DATG but also sets the stage for future explorations into its applicability across a broader range of attributes, model scales, and complex language tasks, reinforcing the framework’s flexible and plug-and-play characteristics.
Ethical Considerations
It is important to note that the algorithm designed in this study is involved in distinguishing between toxic and non-toxic comments, where toxic comments may encompass hate speech, racial discrimination, sexual harassment, and other harmful texts. Our model is trained with the sole purpose of advancing the field of Natural Language Processing (NLP) towards a healthier and toxicity-free direction.
Limitations
This work presents two main limitations. Firstly, the preprocessing required, including the generation of contextually relevant corpora, can be timeconsuming, which may impact the efficiency of time-sensitive applications. Secondly, the effectiveness of DATG heavily relies on the generative capabilities of the underlying models; insufficiently diverse or relevant content generation may reduce control over the desired attributes.
To address these issues, future work will aim to reduce preprocessing time and enhance the robustness of the framework against the variability of model outputs. One potential direction for improving speed involves pre-generating large attribute graphs of the corpus. Searching for key nodes within semantically related subgraphs could expedite this process.

heading: Acknowledgments
text: This work was supported by the National Natural Science Foundation of China (Grants No. 62072463 and 71531012), the National Social Science Foundation of China (Grant No. 18ZDA309), the Research Seed Funds of the School of Interdisciplinary Studies at Renmin University of China, and the Opening Project of the State Key Laboratory of Digital Publishing Technology at Founder Group.

heading: A Experiment Details
text: This section outlines our experimental methodology to evaluate the effectiveness of the DATG method in steering text generation towards specific attributes. Our investigation concentrates on two tasks: (1) Toxicity Mitigation and (2) Sentiment Transformation.
A.1 Tasks Toxicity Mitigation Task: Leveraging the RealToxicityPrompts dataset (Gehman et al., 2020), which includes over 100,000 prompts with toxicity scores, this task crafts two evaluation sets: RandomToxic, 1,000 prompts sampled to broadly test toxicity mitigation, and TopToxic, the 1,000 most toxic prompts to focus on critical toxicity reduction. The aim is to minimize prompt mismatch while reducing generated text toxicity, aligning outputs with initial non-toxic intents.
Sentiment Transformation Task: Utilizing the SST-5 dataset (Socher et al., 2013), which contains movie reviews across a sentiment spectrum from 1 to 5, this task prepares two sets for evaluation: NegToPos, 1,000 negative reviews (scores 1 and 2) for testing transformation to positive sentiment, and PosToNeg, 1,000 positive reviews (scores 4 and 5) for conversion to negative sentiment. The goal is to generate text that effectively shifts sentiment in the opposite direction of the initial prompt, ensuring textual coherence and relevance.
These tasks are selected to showcase the DATG method’s effectiveness in accurately guiding text generation towards desired attributes, reflecting its potential to enhance the quality and applicability of generated content. We have obtained all datasets used through official sources, and the datasets are used in a manner consistent with their intended use.
A.2 Base LLMs Our experiments utilize a diverse array of base LLMs, each developed by leading AI research institutions. The lineup includes Phi-2 2.7B by Microsoft Research, emphasizing compactness and efficiency; LLaMA-2 13B by Meta AI, optimized for dialogue and conversational contexts; Falcon 7B by Technology Innovation Institute, focusing on broad language understanding; OPT 6.7B also by Meta AI, known for its open-source accessibility; and Alpaca 7B by Stanford University, designed for instruction-following tasks. These models range from 2.7 billion to 13 billion parame-
ters, providing a solid foundation for evaluating the DATG method’s effectiveness. We have obtained all models used through official sources, and the models are used in a manner consistent with their intended use.
To ensure consistency across experiments, we employ the following generation configurations for all models:
• max_new_tokens: 32,
• do_sample: True,
• top_k: 200,
• top_p: 0.9,
• temperature: 0.7.
These settings are designed to balance creativity and coherence in generated text, enabling nuanced control over the output while facilitating the exploration of the DATG method’s capabilities in steering text generation.
A.3 Classifier Models
To improve the precision and control in text generation tasks, we integrate classifier models with our foundational generative models. At the core of our classification setup is the BAAI/bge-large-en-v1.5 model, chosen for its nuanced understanding of language and awareness of context. This model acts as the base for our task-specific classifier heads, which we fine-tune to meet the specific needs of each task.We have obtained all datasets and models used through official sources, and the datasets and models are used in a manner consistent with their intended use.
A.3.1 Toxicity Mitigation Classifier
For toxicity mitigation, we employ the Jigsaw Toxic Comment Classification Challenge dataset (cjadams et al., 2017), which includes a broad array of comments annotated for varying levels of toxicity. This dataset enables us to train a classifier that efficiently distinguishes between toxic and non-toxic content. We create a balanced dataset of 42,768 training samples to even out the distribution between toxic and non-toxic labels. This classifier reaches an accuracy of 93.39%, facilitating the generation of safer and more respectful dialogues.
A.3.2 Sentiment Transformation Classifier For sentiment transformation, we utilize the IMDB dataset (Maas et al., 2011), comprised of movie reviews annotated with binary sentiment scores. This rich dataset allows us to train a sentiment classifier that effectively directs text generation toward either positive or negative sentiments, ensuring the generated text aligns well with the intended emotional tone. We prepare a balanced training dataset of 50,000 samples to maintain equal representation of both sentiment polarities. The sentiment classifier achieves an accuracy of 95.90%.
We fine-tune the classifiers with the following hyperparameters, identical across both tasks:
• Epochs: 20
• Batch Size: 32
• Learning Rate: 1× 10−5
• Training Set Size Ratio: 90%
Fine-tuning these classifiers with carefully chosen hyperparameters and balanced datasets plays a crucial role in the DATG method’s success. It enables precise guidance of text generation towards desired attributes, ensuring both high accuracy and relevance.
A.4 Baselines
In assessing the DATG method, we benchmark against two key baselines in controlled text generation:
FUDGE conditions text generation on attributes by adjusting LLMs’ output probabilities with an attribute classifier’s scores. We select the top k = 100 tokens, adjusting logits with intensity α = 0.5, achieving nuanced control for attributes like formality and sentiment.
PREADD manipulates output logits from prompts for direct attribute control, contrasting logits between prefixed and original prompts. With α = 1.0, it modulates control strength, effectively mitigating toxicity with the prompt “The following text perpetuates negative stereotypes, is threatening or sexually explicit, and does not contain profane language.” and transforming sentiment with “The following text exhibits a very positive/negative sentiment and/or opinion.”
Moreover, we explore prompt injection techniques as an additional baseline, aligning with the PREADD’s experimental setup. This approach incorporates specific prompts into the generation process to efficiently direct the model’s output toward the desired attribute.
CONTINUATION generates text without any attribute-specific conditioning, serving as a baseline to evaluate the effect of explicit attribute control.
INJECTION uses the same prompts as PREADD, but directly integrates them into the generation process for attribute alignment. For toxicity mitigation, the prompt is “The following text doesn’t perpetuate negative stereotypes, isn’t threatening or sexually explicit, and does not contain profane language.” For sentiment transformation, the prompt is “The following text
exhibits a very positive/negative sentiment and/or opinion.” This method aims to influence the model’s output more naturally by embedding the desired attribute direction within the prompt itself.
In addition to the baseline methods, our DATG approach introduces different strategies in the context corpus construction and dynamic attribute graph phases. During the initial stage, DATG freely generates 30 sentences to build a contextually rich corpus. After constructing two dynamic attribute graphs (positive and negative), we simplify the threshold determination process by selecting 10 nodes from each graph for adjustment.
DATG-L DATG-L employs a Logits-Boost strategy, where the adjustment intensities for boosting positive nodes and avoiding negative nodes are set at α = 4.0 and β = 6.0, respectively. This method ensures a targeted manipulation of logits to enhance or mitigate specific attributes within the generated text, providing a refined control over the text generation process.
DATG-P Similarly, DATG-P applies the PrefixPrompt strategy for adjustment, using prefixes to steer the generation process towards the desired attributes. The Prefix-Prompt is “The following passage often discusses [Positive Words] but does not mention [Negative Words].”
A.5 Metrics Our evaluation framework employs specific metrics for toxicity mitigation and sentiment transformation tasks to accurately measure their outcomes:
Toxicity (For Toxicity Mitigation Task): We quantify the average toxicity level of generated text
using the Perspective API by Jigsaw. This automated tool, developed in 2017, provides a reliable measure of text toxicity, ensuring our content meets desired safety standards.
Success (For Sentiment Transformation Task): Success is defined as the proportion of generations accurately achieving the desired sentiment. This is assessed by a RoBERTa model (Liu et al., 2019) fine-tuned on the SST-5 dataset (excluding test samples), with the following parameters:
• Epochs: 20
• Batch Size: 32
• Learning Rate: 1× 10−5
• Training Set Size Ratio: 90%
Achieving a prediction accuracy of 96.78%, this model’s precision in sentiment identification allows us to calculate the success rate of our sentiment transformations effectively.
Perplexity: Applied across both tasks, perplexity is assessed by GPT-2 large, evaluating the conditional perplexity of prompt completions. This metric measures the natural flow from prompt to generated text, highlighting coherence.
Relevance: For both tasks, relevance is measured using cosine similarity between the sentence embeddings of the prompt and its completion, calculated by the BAAI/bge-large-en-v1.5 model. This ensures that generated text remains contextually aligned with the initial prompt.
These tailored metrics enable a comprehensive evaluation of the DATG method’s ability to produce text that is contextually relevant, fluent, and aligned with specific toxicity and sentiment goals.We have
obtained all datasets and models used through official sources, and the datasets and models are used in a manner consistent with their intended use.

heading: B Experiment Results
text: The appendix details experimental results omitted from the main text, including sentiment transformation across models in Table 5, average generation speeds in Table 6, and speed distribution histograms in Figure 4.

heading: C Sample Results
text: This section presents selected instances of generated text to illustrate the performance of our methods. Detailed examples across various tasks and models are provided on the following page.
Please note, the following generated text examples may contain extremely offensive or harmful content.

heading: Future Work
text: This work not only showcases the capabilities of DATG but also sets the stage for future explorations into its applicability across a broader range of attributes, model scales, and complex language tasks, reinforcing the framework’s flexible and plug-and-play characteristics. Ethical Considerations
It is important to note that the algorithm designed in this study is involved in distinguishing between toxic and non-toxic comments, where toxic comments may encompass hate speech, racial discrimination, sexual harassment, and other harmful texts. Our model is trained with the sole purpose of advancing the field of Natural Language Processing (NLP) towards a healthier and toxicity-free direction. Limitations
This work presents two main limitations. Firstly, the preprocessing required, including the generation of contextually relevant corpora, can be timeconsuming, which may impact the efficiency of time-sensitive applications. Secondly, the effectiveness of DATG heavily relies on the generative capabilities of the underlying models; insufficiently diverse or relevant content generation may reduce control over the desired attributes. To address these issues, future work will aim to reduce preprocessing time and enhance the robustness of the framework against the variability of model outputs. One potential direction for improving speed involves pre-generating large attribute graphs of the corpus. Searching for key nodes within semantically related subgraphs could expedite this process