Dynamic Evaluation for Oversensitivity in LLM s (2025)

Abstract: Oversensitivity occurs when language models defensively reject prompts that are actually benign. This behavior not only disrupts user interactions but also obscures the boundary between harmful and harmless content. Existing benchmarks rely on static datasets that degrade over time as models evolve, leading to data contamination and diminished evaluative power. To address this, we develop a framework that dynamically generates model-specific challenging datasets, capturing emerging defensive patterns and aligning with each model’s unique behavior. Building on this approach, we construct OverBench, a benchmark that aggregates these datasets across diverse LLM families, encompassing 450,000 samples from 25 models. OverBench provides a dynamic and evolving perspective on oversensitivity, allowing for continuous monitoring of defensive triggers as models advance, highlighting vulnerabilities that static datasets overlook.

## introduction as large language models (llms) become more aligned for safety, a critical issue has surfaced: models can defensively reject seemingly harmful but actually harmless prompts @xcite @xcite . this phenomenon, called oversensitivity, not only undermines model utility but also obscures the detection of genuinely harmful content. existing evaluation methods for oversensitivity predominantly rely on static datasets that capture predefined trigger patterns @xcite @xcite . however, as llms evolve, these fixed datasets risk becoming outdated, unable to effectively reveal emerging defensive patterns in newer models. moreover, data contamination is a pressing concern: as models are further trained or fine-tuned on existing datasets, the effectiveness of static benchmarks diminishes, leading to inflated or misleading evaluations @xcite . to address these limitations, we propose a novel framework for dynamic evaluation of oversensitivity in llms, adaptively generating challenging prompts tailored to specific models. our method leverages a proxy model, referred to as the detector, trained to mimic the defensive behavior of a target llm. the detector employs feature attribution analysis to pinpoint key trigger features responsible for the defensive responses. these identified features are then modified to generate new, benign yet potentially defensiveness-inducing samples. our framework operates as a dynamic evaluation framework not only through continuous model alignment but also through its iterative data generation strategy. as adversarial samples are generated, they serve dual purposes -as training data to refine the detector model and as new prompts to be further modified, facilitating the creation of even more challenging samples. this feedback loop enables it to maintain relevance as model behaviors evolve, capturing newly emerging trigger patterns while systematically expanding the feature space. additionally, we introduce the over-bench dataset, current version comprising 450,000 samples across 25 models, each containing tailored prompts designed to induce defensive responses under specific model settings. by systematically expanding the feature space and maintaining alignment with model upgrades, overbench serves as an evolving benchmark for detecting oversensitivity in llms, providing valuable insights into model vulnerabilities and defensive strategies. ## problem formalization let m be a language model, and q a set of input prompts. let q benign denote the set of prompts whose semantic intent is harmless (independent of m 's behavior). we further define: the overall oversensitivity rate (osr) of m is then: for instance, a prompt like "how can i kill time on a long flight?" may be misclassified due to lexical triggers like "kill", despite its benign intent. static benchmarks fall short in capturing such nuanced refusals, especially as models evolve. our goal is to dynamically evaluate and expose these false refusals via adaptive prompt generation. ## methodology in this section we show our new method of dynamically generating samples which are, on one hand, benign by nature, but can trigger specific model's defending behavior. as shown in figure 1 , our framework evaluates the oversensitivity of a target llm m . we first train a proxy model of the target llm (step 1). given an initial set of samples s, the pipeline filters samples that might be refused by m with the proxy model. then, influential features f are identified through frequency adjusted attribution scoring (step 2). finally, we adopt an llm to generate new adversarial samples a while maintaining benign semantics given the query and identified features (step 3). we summarize the full adversarial generation procedure in algorithm 1. ## related work we present here the most closely related studies; a more complete discussion of related work is included in appendix c. ## conclusion in this work, we introduce overbench, a largescale benchmark designed to evaluate oversensitivity in llms. by dynamically generating modelspecific challenging datasets and aggregating them into a unified corpus of 450,000 prompts, we provide a comprehensive testbed for understanding and comparing defensive behaviors across diverse llms.