Neuron Activation Modulation for Text Style Transfer: Guiding Large Language Models (2025)

Abstract: Text style transfer (TST) aims to flexibly adjust the style of text while preserving its core content. Although large language models (LLMs) excel in TST tasks, they often face unidirectional issues due to imbalanced training data and their tendency to generate safer responses. These challenges present a significant obstacle in achieving effective style transfer. To address this issue, we propose a novel method for text style transfer based on neuron activation modulation (NAM-TST). This approach identifies neurons related to style through gradient-based activation difference analysis and calculates the activation differences between the source and target styles. During text generation, we use the activation difference to align the activation values of style-related neurons with those of the target style to guide the model in performing the transfer. This strategy enables the model to generate text that satisfies specific style requirements, effectively mitigating the unidirectional issue inherent in LLMs during style transfer. Experiments on benchmark datasets demonstrate that NAM-TST significantly enhances style transfer quality while preserving content consistency.

## introduction text style transfer (tst) aims to transform text from a source style to a target style (e.g., from positive to negative) while preserving core properties, such as semantics and grammar @xcite . however, style transfer faces significant unidirectional challenges -where transfer performance is markedly better in one direction than in the reverse -due to inherent class imbalances in the training data of large language models (llms) @xcite and their tendency to generate safer and more innocuous responses @xcite . specifically, llm training sets tend to predominantly consist of texts with positive and * corresponding author: jianyi liu, liujy@bupt.edu.cn formal styles. as a result, these models often generate outputs that favor safer responses, limiting their ability to achieve the desired diversity of styles and accurate style transfer. in recent years, llms have made significant progress in tst @xcite . researchers have developed several methods to enhance the model's capabilities. some studies @xcite @xcite @xcite fine-tune llms with parallel or pseudo-data to improve style transfer performance. however, these approaches heavily rely on large amounts of high-quality data. other methods @xcite focus on guiding llms to generate text in the target style using prompts. nonetheless, prompt-based methods often fail to accurately capture a specific style accurately, and the model's sensitivity to prompt phrasing may lead to unintended content changes @xcite , potentially affecting content consistency. recent research @xcite shows that analyzing style-specific neurons can significantly enhance tst performance in llms. however, current methods primarily rely on inactivating neurons for style transfer, often resulting in partial content loss. this limitation poses a direct challenge to the effectiveness of tst, particularly in terms of maintaining content consistency during style transfer. in this paper, we propose a text style transfer method based on neuron activation modulation (nam-tst), which identifies style-related neurons and guides llms in performing style transfer by modulating their activation. specifically, our approach leverages gradient-based activation difference analysis to identify style-related neurons through a computed activation sensitivity index. recognizing intrinsic ambiguity in neuron activation patterns, we employ gradient analysis to quantitatively assess neuronal sensitivity to content variations. within the identified style-specific neuron set, those exhibiting high content sensitivity are characterized as intertwined neurons. during generation, we use the activation difference between the source and target styles to modulate the activation of the identified style neurons. this modulation aligns the style-specific neurons with the desired target style, enabling more accurate and controllable style transfer. in particular, we also introduce the parameter n grad to quantify the strength of modulation in intertwined neuron activations, ensuring that the core content of the sentence remains largely intact throughout the style transfer process. we evaluate nam-tst on four tasks: sentiment @xcite , formality @xcite , authorship @xcite and toxicity @xcite . experimental results show that our method achieves more efficient style transfer while preserving the original content and effectively mitigates the common unidirectional problem found in traditional methods. in summary, the contributions of this paper are as follows: ## related work pre-trained language models, after further finetuning, effectively alleviated the unidirectional issue in tst @xcite . however, obtaining the supervised parallel data required for training deep neural networks is both scarce and costly @xcite . consequently, most studies now rely on unsupervised methods @xcite @xcite . prompt learning @xcite has gained popularity as a way to guide models in tst without requiring additional training. @xcite prompted llms to generate a set of candidate texts in the target style and then ranked them to produce the final output. however, these models are highly sensitive to prompts, and their ability to generalize to domain-specific data or new styles not encountered during pretraining is significantly diminished. @xcite innovatively explored masking techniques and achieved promising results. however, masking techniques may be difficult to ensure highquality semantic preservation in practical applications. in contrast, our study uses a small dataset with fixed prompts, effectively mitigating the sensitivity issue and improving the generalizability of the model. studies have shown that neurons in deep neural networks can encode and represent various features that are interpretable by humans @xcite . in recent years, research using neuron activation analysis methods to guide llms for tst has made significant progress. @xcite incorporated style vectors into the activations of hidden layers during text generation, effectively influencing the style of the generated text and thus demonstrating the efficacy of activation engineering. however, this method only extracts activations at the hidden layer level and ignores the impact of individual neuron activations on tst. @xcite proposed the sneuron-tst, which identifies neurons related to the source and target styles and deactivates neurons associated with the source style, increasing the generation probability of vocabulary consistent with the target style. however, this approach overlooks the dual role that certain neurons can play in both content and style. furthermore, inhibition of neurons can result in the loss of important content information. this paper addresses these challenges by exploring the complex interactions between neurons responsible for both text style and content, introducing a more comprehensive and sophisticated activation regulation strategy to guide llms in style transfer. ## method our goal is to guide the llm in tst by identifying specific neurons and adjusting their activations. figure 1 illustrates the framework of our approach. first, we use the activation sensitivity index (asi) figure 1 : overview of the nam-tst method. the nam-tst method consists of two primary steps: first, styleand content-related neurons are identified through gradient and activation difference analysis. then, we use the activation difference between target style neurons and source style neurons to regulate the style neurons, guiding the model to perform style transfer. notably, we use the n grad parameter to handle intertwined neurons. the figure also illustrates an example of identifying style and content neurons in the authorship task. to identify style neurons. next, we design a similar task to analyze the content sensitivity of neurons and their interaction with the style. finally, we apply activation differences to regulate style neurons and introduce the n grad parameter to handle intertwined neurons. ## conclusion this paper introduces a novel text style transfer method, nam-tst, which uses the activation sensitivity index to identify style-related neurons and explores the intrinsic relationship between style and content. by precisely adjusting the activation of style neurons, nam-tst effectively guides the style transfer process. experimental results demonstrate that nam-tst outperforms existing neuron analysis methods on four distinct tasks, excelling in both style transfer and content preservation. these findings underscore the great potential of nam-tst in advancing text generation and achieving more controllable style regulation in natural language processing.