Controlled Text Generation for Large Language Model with Dynamic Attribute Graphs (2024)

Abstract: Controlled Text Generation (CTG) aims to produce texts that exhibit specific desired attributes. In this study, we introduce a pluggable CTG framework for Large Language Models (LLMs) named Dynamic Attribute Graphs-based controlled text generation (DATG). This framework utilizes an attribute scorer to evaluate the attributes of sentences generated by LLMs and constructs dynamic attribute graphs. DATG modulates the occurrence of key attribute words and key anti-attribute words, achieving effective attribute control without compromising the original capabilities of the model. We conduct experiments across four datasets in two tasks: toxicity mitigation and sentiment transformation, employing five LLMs as foundational models. Our findings highlight a remarkable enhancement in control accuracy, achieving a peak improvement of 19.29% over baseline methods in the most favorable task across four datasets. Additionally, we observe a significant decrease in perplexity, markedly improving text fluency.

## introduction controlled text generation (ctg) focuses on generating text adhering to specific conditions or attributes, such as sentiment, non-toxicity @xcite and style @xcite . in the realm of ctg, achieving precise control over specific attributes of the generated content is a significant challenge. this instance: the novel is a masterpiece / success / disappointment / disaster of storytelling, with a complex narrative. must be accomplished without compromising the generative capabilities and text quality of llms. traditionally, ctg methods have employed small language models to influence the decoding process of larger models @xcite @xcite . though this approach provides a degree of control, it may compromise the inherent quality and variability of the output. recent studies @xcite highlight that an overemphasis on control can detrimentally affect text fluency, rendering the content less effective. this issue underscores a critical insight: excessive reliance on smaller language models to steer the outputs of llms can diminish the decoding capabilities inherent to llms. when small-scale models assume control, they effectively overshadow the original performance of llms during the inference and decoding phase. this process not only masks the vast capabilities of llms but also relegates them to a subordinate role, essentially transforming these sophisticated generative models into mere "puppets" of their smaller counterparts. in light of our exploration, we think the specific attributes of a text are predominantly determined by a limited number of words that bear close relation contextual corpus construction 1 you're the stupid one for trying it out on a production machine. original prompt (harmful): i do actually think that you're stupid one… this is the first time i've heard about it… i'm sorry, but you're not professional… you're the stupid one for thinking that… and you're the stupid one… toxicity sentiment 0.0 1.0 i do actually think you're stupid 0.9 0.3 0.4 0.7 0.7 one attribute classifier scoring 0.9 0.9 0.9 0.9 0.9 0.9 0.1 0.1 and you're the stupid one 0.7 0.3 0.3 0.3 0.3 0.7 0.7 0.7 0.1 0.1 0.1 0.1 node mapping positive links negative links dynamic attribute graphs construction 3 positive graph regeneration with dynamic boundary controlling 4 you're thinking professional good actually not stupid new generation path dynamic negative boundary sorry dynamic positive boundary graph ranking negative graph graph ranking you're professional, but we can … new output (safe): logits-boost strategy prefix-prompt strategy the following passage often discusses [positive words] but does not mention [negative words]. to those attributes @xcite . despite these key words being sparse within the text, their impact on the overall attributes is decisive. for instance, changing the word "masterpiece" to "failure" in the sentence "the novel is a masterpiece of storytelling, with a complex narrative." shifts the sentiment from positive to negative. this change alters the entire sentence's sentiment and meaning. in the conceptual framework of semantic space, these attributes can be seen as dimensions within this space. by strategically adjusting these key words, we can guide the text generated by llms to move in the desired direction within the semantic space, controlling its attributes without significant alterations to the overall content (see figure 1 ). based on these observations, we propose a pluggable ctg approach, dynamic attribute graphsbased controlled text generation (datg), which employs dynamic attribute graphs to identify key words aligned or opposed to target attribute dimensions. by modulating the occurrence of these key words, our method precisely controls text attributes without compromising the inherent capabilities of llms. this strategy allows for targeted movement within the semantic space. as described in figure 2 . our work begins with contextual corpus construction, where llms generate text sequences from specific prompts. subsequently, attribute classifier scoring assesses these texts with classifiers, such as toxicity or sentiment classifiers, to evaluate alignment with the target attribute. the core of our method, dynamic attribute graphs construction, transforms the text sequences into directed weighted graphs, informed by classifier scores. this process leads to the creation of two distinct graphs: a positive attribute graph, weighted by the consistency scores from the classifier, and a negative attribute graph, weighted by the complements of these scores. the attribute graphs represent the text's adherence to and deviation from the target attribute dimension within the semantic space. during the regeneration with dynamic boundary controlling process, the graph ranking algorithm selects key nodes that propel the generated text towards the upper boundary of the control attribute dimension in the semantic space. adjustments of the occurrence of these key nodes, facilitated by logits-boost and prefix-prompt strategies, enable the regeneration of text. the key contributions of our study are summarized as follows: ## conclusion in this paper, we present dynamic attribute graphs-based controlled text generation (datg), a flexible and pluggable framework that seamlessly integrates graph models with llms to refine ctg. datg's plug-and-play nature facilitates easy adaptation with existing llms, allowing for the targeted steering of text attributes while maintaining high linguistic integrity. our framework demonstrates notable successes in critical ctg tasks such as toxicity mitigation and sentiment transformation, as evidenced by substantial enhancements in control accuracy and the preservation of text fluency. the use of dynamic attribute graphs in datg enables precise manipulation of attribute-related words, striking a delicate balance between controlled content generation and the naturalness of language. the efficacy of datg attests to the potential of graph models as vital components in the development of adaptable and effective ctg systems. this work not only showcases the capabilities of datg but also sets the stage for future explorations into its applicability across a broader range of attributes, model scales, and complex language tasks, reinforcing the framework's flexible and plug-and-play characteristics.