Extended Abstract: Probing-Guided Parameter-Efficient Fine-Tuning for Balancing Linguistic Adaptation and Safety in LLM -based Social Influence Systems (2025)

Abstract: Designing effective LLMs for social influence (SI) tasks demands controlling linguistic output such that it adapts to context (such as user attributes, history etc.) while upholding ethical guardrails. Standard Parameter-Efficient Fine-Tuning (PEFT) methods like LoRA struggle to manage the trade-off between adaptive linguistic expression and safety and they optimize based on overall objectives without differentiating the functional roles of internal model components. Therefore, we introduce Probing-Guided PEFT (PG-PEFT), a novel fine-tuning strategy which utilizes interpretability probes to identify LLM components associated with context-driven linguistic variations versus those linked to safety violations (e.g., toxicity, bias). This functional map then guides LoRA updates, enabling more targeted control over the model’s linguistic output. We evaluate PG-PEFT on SI tasks (persuasion, negotiation) and linguistic adaptability with safety benchmarks against standard PEFT.

## introduction dialogue systems leveraging large language models (llms) are being explored for complex social influence (si) tasks, including persuasion @xcite , negotiation @xcite , argumentation, and emotional support. a key challenge in designing these si systems leveraging llms is achieving nuanced linguistic behavior adaptation based on context-such as user personality traits, emotional state, or strategic situation (e.g., in games)-while ensuring the system operates safely and ethically @xcite . standard fine-tuning or parameter-efficient fine-tuning (peft) methods like low-rank adaptation (lora) @xcite adapt models efficiently but struggle with the inherent trade-off between adaptability and safety. optimizing for a combined objective (e.g., task success + safety score) using techniques like reinforcement learning from human feedback (rlhf) @xcite or direct preference optimization (dpo) @xcite applies updates based on overall performance, potentially sacrificing safety for adaptability or vice-versa, without understanding which internal mechanisms control these different behavioral facets. this lack of granular control hinders the development of responsible si systems. specifically, standard peft methods may inadvertently amplify unsafe tendencies while trying to achieve better adaptation. to address this, we propose probing-guided peft (pg-peft). our approach integrates interpretability insights directly into the fine-tuning process. we hypothesize that by using probing techniques @xcite we can identify llm components (e.g., attention heads, mlp layers) which are differentially responsible for generating context-adaptive linguistic variations versus those contributing to safety violations (a failure of guardrails). this allows us to guide peft (specifically lora) updates more effectively. ## related work the advent of llms offers new capabilities but also challenges, particularly in alignment @xcite and ensuring ethical behavior @xcite . peft methods like lora @xcite allow efficient adaptation but lack fine-grained control for multi-objective alignment involving safety. interpretability techniques, including probing @xcite and methods like inference-time intervention (iti) @xcite identify functionally specialized components (e.g., attention heads related to truthfulness) to understand model internals. ## methodology our proposed method involves performing evaluation post the following two stages: ## experiments & expected results setup: we have used llama-3.1 8b adapted with lora as our baseline. we focus on si tasks using datasets like persuasionforgood @xcite (persuasion, utilizes user attributes) and dealornodeal @xcite . for safety evaluation we use benchmarks covering diverse risks (alert @xcite ), implicit toxicity (toxigen @xcite ), and social bias (crows-pairs @xcite ). metrics: our evaluation compares the trade-off, measuring: expected results: we expect probing (stage 1) to successfully identify functionally relevant components for linguistic adaptation vs. safety. our central hypothesis is that pg-peft will demonstrate a superior trade-off compared to standard lora, achieving better safety for a given level of adaptive performance. we anticipate pg-peft will allow for more predictable control over generated linguistic behaviors, reducing unintentional harms during adaptation. ## conclusion and future work pg-peft introduces a novel strategy for fine-tuning llms in si systems by integrating interpretability insights into the peft process. by guiding lora updates based on the probed functional roles of internal components related to linguistic adaptation and safety, we aim to achieve a controlled balance between these critical objectives. future directions include exploring advanced probing techniques (e.g., causal probing @xcite , assessing the transferability of functional maps across models and languages and applying pg-peft to more si tasks and safety concerns (e.g., misinformation) to observe a more generalized performance.