FreeCtrl: Constructing Control Centers with Feedforward Layers for Learning-Free Controllable Text Generation (2024)

Abstract: Controllable text generation (CTG) seeks to craft texts adhering to specific attributes, traditionally employing learning-based techniques such as training, fine-tuning, or prefix-tuning with attribute-specific datasets. These approaches, while effective, demand extensive computational and data resources. In contrast, some proposed learning-free alternatives circumvent learning but often yield inferior results, exemplifying the fundamental machine learning trade-off between computational expense and model efficacy. To overcome these limitations, we propose FreeCtrl, a learningfree approach that dynamically adjusts the weights of selected feedforward neural network (FFN) vectors to steer the outputs of large language models (LLMs). FreeCtrl hinges on the principle that the weights of different FFN vectors influence the likelihood of different tokens appearing in the output. By identifying and adaptively adjusting the weights of attributerelated FFN vectors, FreeCtrl can control the output likelihood of attribute keywords in the generated content. Extensive experiments on singleand multi-attribute control reveal that the learning-free FreeCtrl outperforms other learning-free and learning-based methods, successfully resolving the dilemma between learning costs and model performance1.

heading: 1 Introduction
text: Controllable text generation (CTG) focuses on directing language models to produce diverse and fluent sentences that adhere to predefined single or multiple attributes such as topics and sentiment (Yang et al., 2023; Gu et al., 2023; Zhang et al., 2023; Zhong et al., 2023). Recent works on CTG can be roughly categorized into two groups based on their dependency on a learning process:
* Corresponding author 1Code is available at https://github.com/zijian678/
FreeCtrl
learning-based methods and learning-free methods (Mireshghallah et al., 2022).
Learning-based methods usually involve training (Yang and Klein, 2021; Krause et al., 2021; Lin and Riedl, 2021), fine-tuning (Ficler and Goldberg, 2017; Keskar et al., 2019; Wang et al., 2021), or prefix-tuning (Qian et al., 2022; Zhang and Song, 2022; Gu et al., 2022, 2023; Yang et al., 2023; Zhong et al., 2023) language models or discriminators based on attribute-specific data. Despite their effectiveness, these approaches come with high computational costs for training and a dependency on vast, attribute-specific datasets, posing challenges for deployment in environments with limited data or computational capacity.
Only a few existing methods are learning-free, avoiding the need for training. For instance, K2T (Pascual et al., 2021) employs attribute-focused keywords to influence token output probability during generation. Another method, Mix&Match (Mireshghallah et al., 2022), integrates diverse black-box experts as a probabilistic energy model to steer large language model (LLM) outputs. These learning-free methods, despite bypassing the
7627
training process, tend to fall short in performance compared to advanced learning-based approaches.
The analysis spotlights the classic dilemma in machine learning between the cost of learning and model performance, as shown in Figure 1. To overcome this obstacle, particularly in avoiding learning expenses for CTG while ensuring high performance, we propose FreeCtrl: Constructing Control Centers with Feedforward Layers for LearningFree Controllable Text Generation. FreeCtrl’s central idea is to manipulate FFN vectors2 to regulate the outputs of LLMs, inspired by a recent finding that the tokens generated by LLMs can be attributed to the weights of vectors in FFN layers (Geva et al., 2022).
Specifically, the key principle is that increasing a single FFN vector’s weight alters the output distribution, raising specific tokens’ output probability. This strategy enables the targeted enhancement of certain FFN vector weights to raise attribute keywords’ output probability, directing LLM generation towards preferred attributes. Our study first examines the possibility of this strategy by pinpointing three key characteristics of FFN vectors: 1) Convergence: Increasing the weight of an FFN vector can result in a stable and convergent output distribution in LLMs, thereby elevating the probabilities of specific tokens. 2) Diversity: Diverse FFN vectors can increase the output probabilities for most tokens in the LLM vocabulary, covering keywords relevant to general attributes in CTG; 3) Prompt-Invariance: the observed effects of convergence and diversity remain consistent across different input prompts. These characteristics suggest FFN vectors can enable stable, diverse controls for LLM outputs, directing sentence generation toward desired attributes.
However, we also identify a major limitation of FFN vectors: the high-maintenance challenge, where adjusting their weights for precise control proves difficult. Low weights lack the power to steer LLMs, while high weights compromise output diversity and fluency. To mitigate this, FreeCtrl initially sets up control centers using FFN vectors for various attributes, then navigates LLM output via a cycle of initialization, monitoring, adaptation, and filtering during the generation process. Continuous monitoring ensures that token generation is assessed at each step, allowing for adaptive weight
2FFN vectors refer to the value vectors in the second weight matrix of the FFN layer. More details and definitions can be found in §3.1.
adjustments of the control centers. Ultimately, a score-based filtering mechanism is employed to refine the outputs. Notably, this framework requires no training or attribute-specific data yet surpasses the efficacy of advanced learning-based models. Therefore, FreeCtrl addresses the cost-performance dilemma, situating it at the optimal upper-left corner in Figure 1, denoting learning-free but high performance. Our main contributions are summarized as follows:
1. We conduct a systematic analysis of using FFN vectors for CTG in §3, identifying three key characteristics: convergence, diversity, and prompt-invariance, alongside a notable challenge of high maintenance.
2. We propose FreeCtrl in §4, a learning-free approach that identifies and utilizes FFN vectors governing various attributes to establish control centers, thus enabling precise management of LLM outputs through initialization, monitoring, adaptation, and filtering.
3. Comprehensive experiments in §5 on both single and multi-attribute control demonstrate that FreeCtrl, without incurring any learning costs, outperforms existing learning-free baselines and cutting-edge learning-based models.

heading: 2 Related Work
text: Learning-based Methods Initial research efforts concentrate on adapting language models into attribute-conditional language models, utilizing methods like fine-tuning (Ficler and Goldberg, 2017; Keskar et al., 2019; Wang et al., 2021) and reinforcement learning (Ziegler et al., 2019; Khalifa et al., 2020; Kim et al., 2023). Weighted decoding stands out as another key strategy in CTG. These methods are primarily learning-oriented, involving updates to the model’s hidden states based on decoded logits (Dathathri et al., 2019) or training attribute discriminators to modify model output probabilities (Yang and Klein, 2021; Krause et al., 2021; Lin and Riedl, 2021). Amidst the growth of large language models like GPTX (Radford et al., 2019; OpenAI, 2023) and LLaMA2 (Touvron et al., 2023), recent techniques often preserve LLM parameters and utilize lightweight fine-tuning methods such as prefix-tuning (Li and Liang, 2021; Lester et al., 2021) followed by decoding-time control (Qian et al., 2022; Zhang and Song, 2022; Gu et al., 2022, 2023; Yang et al., 2023; Zhong et al.,
2023). These methods generally necessitate a large volume of attribute-specific data and considerable computing resources for training either prefixes or discriminators.
Learning-free Methods The realm of learningfree approaches, which eschew any training process, is sparsely populated with research. One such example is K2T (Pascual et al., 2021), which shifts the output logit distribution by calculating the semantic similarity between vocabulary words and target attribute keywords. Notably, Mix&Match (Mireshghallah et al., 2022) stands as the pioneer in introducing a “learning-free” control framework. This innovation leverages external blackbox scoring experts to evaluate the attributes of generated content, thereby regulating the model outputs. Compared to cutting-edge learning-based methods like PriorControl (Gu et al., 2023), these approaches often fall short in performance.

heading: 3 FFN Vectors for CTG
text: This section first details the theoretical basis for using FFN vectors to control LLM outputs, then examines their characteristics and challenges.

heading: 3.1 Theoretical Foundations
text: In line with previous findings (Sukhbaatar et al., 2015, 2019; Geva et al., 2021, 2022; Feng et al., 2024), the outputs from FFNs can be viewed as linear vector combinations:
FFNℓ(xℓ) = f(W ℓKx ℓ)W ℓV
=
dm∑
i=1
f(xℓ · ki)vi = dm∑
i=1
mℓivi
where f is the activation function, W ℓK and W ℓ V are the weight matrices, and xℓ is the input at layer ℓ. FFN then can be conceptualized as a neural key-value memory system, where the columns in WK represent the keys and rows in WV are the values. Given an input vector xℓ, the keys generate the coefficients mℓ = f ( W ℓKx ℓ ) ∈ Rdm , which assign weights to the values in WV . In other words, within each layer, the value vectors denoted as vi are extracted from the rows of the secondary weight matrix, WV . Taking GPT2medium (Radford et al., 2019) as an example, WV is dimensioned at 4096 × 1024. This dimensionality implies the existence of 4096 value vectors, each extending to a 1024-dimensional space within
each individual FFN layer. With 24 such layers incorporated within the GPT2-medium, the model encompasses 24 × 4096 = 98, 304 value vectors in total.
Prior research (Geva et al., 2022) has verified that the outputs generated by LLMs can be explained by examining the weights associated with the value vectors. The weights of various value vectors directly influence the probabilities of different token outputs. Building on this foundation, our study explores the identification of value vectors controlling different attributes in CTG and the feasibility of manipulating their weights to achieve attribute-specific control.

heading: 3.2 Characteristics of Value Vectors
text: Effective control via value vectors depends on three core requirements: stable impact on output distributions, the ability to manage a wide range of CTG attributes, and consistent behavior under different prompts. We highlight three key properties that affirm the value vectors’ utility in achieving trustworthy CTG. Utilizing GPT2 as an example, we iteratively select a single value vector, denoted as vi, and then incrementally increase its weight, denoted as u. The resultant model output distribution, represented as pui ∈ R|V|, is observed, where V signifies the GPT2’s vocabulary and |V| is its size.
Convergence While progressively increasing the weight u from 1 to 50, the distribution influenced by each value vector progressively attains a state of stability and constancy, as shown in Figure 2. Specifically, we treat the output distribution controlled by a weight of 503 as the ground-truth pgi and calculate Spearman’s rank correlation between distributions controlled by different weights and
3A weight of 50 is considered exceptionally large according to Geva et al. (2022).
pgi . The mean correlation across all 98,304 resultant distributions is reported in Figure 2. Spearman’s rank correlation is used because it directly compares token ranks and mitigates the impact of topk/p sampling and temperature variations. In summary, increasing the weights u establishes stable token ranking and distribution patterns. Such convergence enables the discovery of patterns related to target attributes in CTG and facilitates stable control.
Diversity The second question explores whether value vectors can sufficiently control a wide range of tokens representing various attributes in CTG. Our analysis is twofold: first, we assess the percentage of top-k controllable tokens in pui over the whole covabulary; second, we identify specific value vectors that govern general attributes. Figure 3 shows that with increasing weights, the top-1 tokens controlled by the 98,304 vectors account for up to 20% of the GPT2 vocabulary, and this figure rises to over 80% for top-50 tokens. This demonstrates the vectors’ capacity to influence most tokens in the entire vocabulary. To provide further clarity, Table 1 lists several value vectors alongside their corresponding attributes. More attributes and their associated vectors are detailed in Appendix A. These findings confirm the value vectors’ ability to control a broad spectrum of attributes in CTG.
Prompt-Invariance To ensure effective control in LLMs, it is critical that value vectors maintain their properties across various input prompts. Our
analysis involves feeding GPT2 with 35 different prompts, as provided by Dathathri et al. (2019). The results for various prompts mirror the earlier results, showcasing the characteristics of promptinvariance.

heading: 3.3 Limitation of High Maintenance
text: While value vectors show promise for CTG, a significant challenge is their high maintenance due to the difficulty in setting optimal weights. Low weights fail to effectively direct LLMs towards desired attribute-specific tokens, whereas high weights can reduce output diversity and fluency. Furthermore, the ideal weights for various value vectors can be different. As illustrated in Appendix B, a weight of 1 for politics-related vectors does not steer the model towards political content, but increasing the weight to 5 results in constrained and lower-quality outputs. Conversely, a weight of 5 is effective for sports-related attributes, producing relevant and high-quality generations.

heading: 4 Methodology
text: To maximize the benefits and mitigate the limitations of FFN value vectors, we introduce FreeCtrl: Constructing Control Centers with Feedforward Layers for Learning-Free Controllable Text Generation. FreeCtrl begins by gathering attribute keywords, subsequently constructing a control center for each attribute. It then guides the LLM to produce outputs relevant to the target attribute through a systematic pipeline of initialization, monitoring, adaptation, and filtering. The overall framework is illustrated in Figure 4.

heading: 4.1 Attribute Keyword Collection
text: For a given attribute ai in the attribute set A = {a1, · · · , an}, we begin by gathering its associated keywords to promote diverse generations. Various external knowledge bases such as WordNet (Miller, 1994), ConceptNet (Speer et al., 2017), RelatedWords4, and ChatGPT (OpenAI, 2023) can be utilized for this purpose. To reduce the noises
4https://relatedwords.org/
from these external sources, we apply a refinement function:
G(z) = r (z, ai) |A| − 1∑
aj∈A,aj ̸=ai r (z, aj) (1)
where z represents the collected keyword for attribute ai, and r(·) indicates the cosine similarity.
This function incorporates ideas from both TFIDF (Spärck Jones, 2004) and KPT (Hu et al., 2022). It operates on the premise that a suitable attribute keyword should have a relevance score for its corresponding attribute that is higher than the average relevance score for other attributes. Consequently, keywords where G(z) < 1.0 are deemed less relevant and are filtered out for refinement. As this is not the primary focus and contribution of our work, we offer only a brief introduction here. For more detailed information, please refer to KPT (Hu et al., 2022).

heading: 4.2 Control Center Construction
text: Based on attribute-specific keywords, we can identify corresponding value vectors to direct the LLM toward outputs focused on these keywords and attributes. Building on §3.2, we iteratively assign a weight of 50 to each vector and examine the output distribution to locate dominant value vectors that control the attribute keywords.
Formally, let Pumax ∈ RN×|V| represent the softmaxed output distribution across the vocabulary V , modulated by all the N value vectors
with a set weight of umax = 50. The control effect for a specific keyword z is captured by Pumax [:, dV(z)] ∈ RN , where dV(z) denotes the index of token z in the vocabulary, and N signifies the total number of value vectors. To pinpoint vectors controlling the keyword z, we choose value vectors with top-k probabilities:
cz = dvec{max k (Pumax [:, dV(z)])} (2)
where dvec(·) retrieves the index of value vectors in N . For instance, the positions (21,4020) and (12,2719) in Figure 4 of value vectors correspond to the top-2 output probabilities for the attribute keyword “champion”.
Finally, the control center for an attribute ai is established by aggregating the value vectors of the keywords related to the attribute ai:
Cai = ⋃ cz, z ∈ Z(ai) (3)
where Z(ai) denotes the set of keywords for the attribute ai.

heading: 4.3 Single-Attribute Control
text: To precisely control LLMs through control centers, we adopt a structured process that includes initialization, monitoring, adaptation, and filtering. We constantly monitor the LLM’s generation and then adaptively adjust control parameters to steer output toward the desired attribute. A final filtering step verifies the output’s compliance with the specified attribute.
Initialization For a specified attribute ai, we first locate value vectors to construct the control center Cai .
Monitoring At this stage, each token produced by the LLM in response to a prompt is evaluated for its relevance to the attribute ai. We construct current output sait by integrating the input prompt with tokens generated by the LLM at timestamp t for attribute ai. The embedding for token si in s ai t is derived as E[si] ∈ Rde , where E[·] is the LLM’s embedding matrix with dimension de. The attribute embedding E[Z(ai)] can be obtained in a similar way. The correlation between current output sait and target attribute ai can be calculated as:
ρait = 1
lt
lt∑
j=1
max{r(E[sj ], E[Z(ai)]} (4)
where lt denotes the length of the current sentence and the correlation score ρait ranges from 0 to 1. This equation initially computes the maximum cosine similarity between each token in the current output and all target attribute keywords, subsequently calculating the mean value of these correlation scores.
To enhance the correlation with the target attribute while minimizing it with other attributes, we utilize Eq.5, which resembles Eq.1 and is designed to calculate the score of the current output. By continuously tracking the sentence score µait , we are able to quickly modify the weights of Cai , ensuring a coherent and seamless output that aligns with the targeted attribute.
µait = ρ ai t |A| − 1∑ aj∈A,aj ̸=ai ρ aj t
(5)
Adaptation Utilizing µait , we can dynamically adjust the weights for timestamp t + 1, facilitating smooth control and generation. To clarify, the model is required to generate a token at each timestamp. The primary reason for this adaptation is the high maintenance of value vector weights, as discussed in §3.3. The weight for timestamp t+ 1 is determined as Eq.6. Here, µω denotes a preset hyperparameter of the sentence score, λ is a scaling parameter, and µaislt is the last token score obtained by Eqs.4 and 5. We regard µ̂ait = max(µ ai t , µ
ai slt )
as final score to ensure the fluency.
ωait+1 =
{ λ
1+exp[−(µω−µ̂ait )·lt] µω − µ̂ait > 0 0 otherwise (6)
To clarify, a value of µω − µ̂ait > 0 indicates that the score of the current sentence or the lastgenerated token is below the predefined threshold, and as a result, the weight should be determined by the difference between µω and µ̂ ai t . Additionally, the sentence length lt implies that weights at the outset of generation will be higher than those assigned later. This is grounded in the rationale that initially higher weights are necessary to guide the generation towards the desired direction. Once this direction is established, the LLM tends to continue generating tokens along this path, allowing for reduced weights in later stages to maintain fluency. Conversely, when µω − µ̂ait < 0, it signifies that the sentence or the last-generated token at timestamp t has an adequate score and aligns with the target attribute, eliminating the need for a higher weight. Setting the weight to 0 is a deliberate strategy to prevent the LLM from generating outputs associated with contrary attributes.
Filtering Through continuous monitoring and adaptation, the LLM can be steered to generate outputs focused on target attributes. However, some generations might not meet the µω threshold throughout the generation process, despite maintaining high weights. To filter out such noncompliant generations, a final screening is conducted using Eq.7. Only those sentences that achieve scores in accordance with Eq.7 are considered valid outputs.
µaiT > µω (7)
where T represents the final timestamp in the generation process, with a token produced at each timestamp.

heading: 4.4 Multi-Attribute Control
text: A significant strength of our method is its seamless adaptability to controlling multiple attributes. Specifically, in the case of multi-attributes {a1, · · · , aM}, we initially gather the respective control centers for these attributes based on Eqs.2 and 3. We then calculate sentence scores and weights for all M attributes by Eqs.4- 6. At each timestamp, we select the control center with the highest weight for control. Assuming the control
center Cam for the m-th attribute am has the maximum weight ωamt+1, then the weight for Cam is set to ωamt+1, while weights for control centers of all other attributes are set to 0. Through this process, different control centers are activated at different timestamps, ultimately yielding an output that integrates multiple attributes after undergoing a filtering process as Eq.7.

heading: 5 Experiments

heading: 5.1 Experimental Setups
text: To align with established methods for CTG and ensure fair comparisons, our experimental setups rigorously follow Discrete (Gu et al., 2022) and PriorControl(Gu et al., 2023).
Tasks Our analysis includes three CTG tasks: topic, sentiment, and detoxification, under both single- and multi-attribute control scenarios. Following PPLM (Dathathri et al., 2019), we utilize 35 neutral prompts. The GPT2-medium (Radford et al., 2019) is employed to generate sentence completions. For single-attribute control, GPT2 produces 5 completions for each attribute across all prompts, culminating in a total of 35 prompts × (2+4+1) attribute scenarios × 5 completions=1225 sentences. In the multi-attribute control, the model generates a total of 1,400 sentences, calculated as 35 prompts × (2×4×1) attribute combinations × 5 completions.
Implementation Details Implementation details and hyperparameter settings for our methods and comparative baselines are detailed in Appendix C.
Baselines We compare our FreeCtrl with both learning-based and learning-free methods. The learning-based approaches include (1) PPLM (Dathathri et al., 2019), which leverages classifiers’
gradients as bias indicators to guide the model’s outputs; (2) GeDi (Krause et al., 2021), which steers the decoding stage using compact conditional generative models; (3) Contrastive Prefix (Qian et al., 2022), incorporating contrastive learning into the prefix strategies to control the LLM generations; (4) Discrete (Gu et al., 2022), employing discrete sampling to map the distribution of attributes within latent space to guide the LLM output; and (5) PriorControl (Gu et al., 2023), which transfers complex distributions as simple Gausian distributions by using normalizing flow. For learning-free baselines, we compare with the advanced Mix&Match (Mireshghallah et al., 2022), which uses external scoring experts to assess generated content attributes.
Evaluation We conduct both automatic and human evaluation. For automatic evaluation, we leverage classifiers from prior research (Gu et al., 2022, 2023) to assess topic relevance and sentiment accuracy. Additionally, we utilize the Google Perspective API 5 to evaluate the effectiveness of detoxification. We also report the generation fluency using the mean perplexity and diversity calculated by the mean number of unique n-grams (Li et al., 2016). In human evaluation, three annotators evaluate each output based on text quality and the relevance of the specified attribute. These elements are scored on a 1 to 5 scale, where higher scores signify superior performance.

heading: 5.2 Single-Attribute Control
text: Table 2 presents the automatic evaluation results on single-attribute control. When compared to the advanced learning-free approach Mix&Match, our
5https://www.perspectiveapi.com
proposed method, FreeCtrl, demonstrates superior performance across all attributes, with average improvements of 14.9% in sentiment control, 20.9% in topic control, and 0.4% in detoxification. These results distinctly showcase FreeCtrl’s significant advancement over current state-of-the-art (SOTA) learning-free techniques in CTG. Compared to learning-based approaches, FreeCtrl demonstrates competitive or superior performance against the SOTA PriorControl. Specifically, FreeCtrl achieves an average improvement of 0.6% over PriorControl in both sentiment and topic control domains and significantly outpaces PriorControl by a notable margin of 6.6% in detoxification. The results from human evaluation, as shown in Table 4, further reveal a similar trend as automatic evaluations. It is noteworthy that FreeCtrl operates without the need for a learning/training phase or training data, yet it still secures the best results. This underlines FreeCtrl’s potential in addressing the challenge of optimizing the balance between cost and performance, as depicted in Figure 1.

heading: 5.3 Multi-Attribute Control
text: Table 3 details the results of multi-attribute control evaluations, where FreeCtrl markedly outperforms both the learning-based SOTA PriorControl and the learning-free SOTA Mix&Match by significant margins. Specifically, FreeCtrl exceeds Mix&Match’s performance by 22.2% and PriorControl’s by 7.7% in sentiment control, and by 19.8% and 2.3% in topic control, respectively. Furthermore, FreeCtrl enhances the overall average score by 13.7% over Mix&Match and by 3.5% over PriorControl. The human evaluation results presented in Table 4 further highlight the superior performance of our method. These findings underscore FreeCtrl’s efficiency in CTG, demonstrating its capability to excel without relying on a training set or undergoing a learning process.

heading: 5.4 Ablation Study
text: To elucidate the impact of each FreeCtrl component, we conduct an ablation study focusing on topic control as follows:
• Random Monitoring (Ran. Mon): We replace the monitoring score in Eq. 5 with a random score generator, which produces scores uniformly distributed between 0 and 2.
• Without Adaptation (w/o Ada): In this variant, we remove the adaptation component entirely and apply static weights (1.5 and 0.5) to examine how the system performs without the ability to adjust control weights dynamically based on monitoring feedback.
• Without Filtering (w/o Fil): This setup tests FreeCtrl’s performance without the filtering process.
Table 5 summarizes the results, demonstrating that replacing or removing monitoring, adaptation, or filtering components leads to performance drops. Specifically, using a constant weight of 0.5 reduces performance by 15.5% but increases diversity due to reduced control. Conversely, a constant weight of 1.5 significantly lowers both fluency and diversity, indicating that excessive control in LLMs is detrimental. These outcomes highlight the critical role of adaptive control in FreeCtrl for balancing performance, fluency, and diversity in text generation.

heading: 5.5 Diversity Analysis
text: The experimental results presented in Tables 2 and 3 reveal a slight decrease in the diversity of generated content. To mitigate this, we propose increasing the temperature settings of the LLMs during generation. Tables 6 and 7 display the outcomes of applying our FreeCtrl method with an increased temperature setting, compared against other prominent baselines for both single-attribute and multiattribute control tasks. The results demonstrate that increasing the temperature enables FreeCtrl to achieve the highest diversity scores while also maintaining superior control accuracy and fluency, as evidenced by Perplexity (PPL) scores.

heading: 5.6 Further Analysis
text: Further analysis is provided as follows:
• Hyperparameter analysis: We examine three hyperparameters in FreeCtrl for adjusting the control strength in Appendix D.
• Case study: For a visual illustration of control effects, output examples along with their corresponding control weights are presented in Appendix E.
• Inference speed: Given that monitoring, adaptation, and filtering could add additional time costs, we assess FreeCtrl’s inference speed and compare it with other methods in Appendix F.
• Scalability: To verify its scalability, we extend FreeCtrl to a larger language model, LLaMA27B (Touvron et al., 2023), with detailed results presented in Appendix G.

heading: 6 Conclusions
text: In this paper, we introduce FreeCtrl, a learning-free approach for controllable text generation (CTG). FreeCtrl employs FFN value vectors to establish control centers tailored to each attribute, enabling dynamic control via a structured process of initialization, monitoring, adaptation, and filtering. Comprehensive experiments demonstrate that FreeCtrl markedly outperforms both learning-based and learning-free methods.

heading: 7 Limitations
text: Our FreeCtrl approach effectively navigates the trade-off between learning expenses and model efficacy. We believe its control mechanism could be further streamlined while maintaining satisfactory outcomes. Additionally, delving deeper into the dynamics of value vectors, including their interactions, can enrich our comprehension and enhance CTG design strategies. These areas offer promising directions for future research.

heading: Acknowledgements
text: We express our sincere gratitude to the reviewers for their insightful and constructive feedback. We would like to acknowledge that this project is an outcome of the Future Resilient Systems initiative at the Singapore-ETH Centre (SEC). Additionally, we extend our thanks to the National Research Foundation, Prime Minister’s Office, Singapore, for their invaluable support through the Campus for Research Excellence and Technological Enterprise (CREATE) programme.

heading: A Attributes and Corresponding Value Vectors in GPT2
text: Table 8 details the general attribute keywords in CTG along with their associated value vectors.

heading: B Examples of High Maintenance
text: We begin by identifying value vectors linked to particular attributes and then vary their weights to assess the impact on LLM outputs, as summarized in Table 9. A weight of 1 for politics-related vectors is insufficient to direct the model’s focus towards political themes. Elevating the weight to 5 leads to diminished output diversity and quality. In contrast, applying a weight of 5 to sports-related vectors successfully generates relevant and high-quality content. These results verify the high maintenance of value vectors.
C Implementation Details
Learning-based methods typically require extensive attribute-specific datasets. In line with prior studies, we provide them with the AGNews (Zhang et al., 2015), IMDB (Maas et al., 2011), and Jigsaw Toxic datasets 6 for topics, sentiments, and detoxification, respectively. Our approach is learning-free and obviates the need for training datasets. Following KPT (Hu et al., 2022), we gather and refine topic-attribute keywords using RelatedWords7 and source sentiment-related keywords for positive and negative attributes from the AFINN (Nielsen, 2011) sentiment lexicon. Our method constructs a control center using positive versus toxic keywords from Gehman et al. (2020) for single-attribute detoxification and filters out toxic words from negative keywords to enable the generation of non-toxic, negative content for multi-attribute control. In this
6https://www.kaggle.com/c/ jigsaw-toxic-comment-classification-challenge/
7https://relatedwords.org/
way, each attribute contains 200 to 300 keywords. Our proposed FreeCtrl features three hyperparameters: the number of value vectors k for each attribute keyword in Eq.2, the sentence threshold µω, and the scaling factor λ in Eq.6. Hyperparameter configurations for single- and multi-attribute control experiments are detailed in Table 10.

heading: D Hyperparameter Analysis
text: Our proposed FreeCtrl has three hyperparameters: the number of value vectors k for each attribute keyword in Eq.2, the sentence threshold µω, and the scaling factor λ in Eq.6.
The hyperparameter k represents the number of value vectors used to regulate a single attribute keyword. Empirical evidence suggests that k = 30 for single-attribute control and k = 200 for multiattribute control yield satisfactory results. The necessity for a greater number of value vectors in multi-attribute control arises from the increased complexity and heightened competition among attributes. Figure 5 outlines the average impact of varying k on topic control. Observations reveal that when k ranges from 10 to 50, the control effects fluctuate slightly between 95.4% and 96.5%, illustrating the robustness of FreeCtrl.
The second hyperparameter, µω, defines the sentence score threshold for control, affecting the final output collection. Figure 6 shows the effect of altering µω between 1.0 and 1.2 on topic control effectiveness. As µω increases from 1.0 to 1.1, there is a gradual improvement in performance. Adjusting µω further, from 1.1 to 1.2, results in stable and satisfactory performance, ranging between 95% and 97%.
The third hyperparameter, λ, acts as a scaling factor for the control weight. Figure 7 indicates that setting λ to 0.5 yields a 94.1% effectiveness in topic control. As λ increases from 1.0 to 2.5, per-
formance improves, ranging between 95.7% and 96.5%. This suggests that a higher λ enhances the control effect. Consistently achieving over 95.7% effectiveness with λ above 1.0 underscores FreeCtrl’s efficacy and robustness.

heading: E Case Study
text: To visually demonstrate the control effects, Figure 8 displays generation results alongside their respective controlling weights. The figure uses red to denote the weights of topic keywords and blue for the weights of sentimental keywords, with the intensity of each color reflecting the magnitude of the weight.
F Inference Speed
FreeCtrl comprises four main phases: initialization, monitoring, adaptation, and filtering, potentially adding to run-time. Initialization occurs preinference, incurring no extra time. Monitoring and adaptation involve evaluating model generation at each timestamp through simple calculations, with negligible added time. Filtering, however, eliminates outputs not meeting certain criteria, leading to wasted generation efforts and additional run-time. We benchmark FreeCtrl’s inference time against the SOTA learning-free model, Mix&Match. We calculate FreeCtrl’s average inference time using total run-time in §5 for all outputs (valid and invalid) divided by the number of valid outputs (1225+1400), resulting in an average of 9.8 seconds for FreeCtrl compared to 20 seconds for Mix&Match. Thus, FreeCtrl not only significantly enhances performance but also substantially reduces inference time.

heading: G Scalability
text: We apply FreeCtrl to LLaMA2-7B, aiming to direct the model’s outputs on specified topics including politics, sports, business, and technology. Comparative analysis between the original and FreeCtrl-influenced outputs is detailed in Table 11, which demonstrates that FreeCtrl effectively guides LLaMA2’s outputs and significantly enhances attribute relevance scores.

heading: Future Work
text: We would like to acknowledge that this project is an outcome of the Future Resilient Systems initiative at the Singapore-ETH Centre (SEC). Additionally, we extend our thanks to the National Research Foundation, Prime Minister’s Office, Singapore, for their invaluable support through the Campus for Research Excellence and Technological Enterprise (CREATE) programme