Eliciting Textual Descriptions from Representations of Continuous Prompts (2025)

Abstract: Continuous prompts, or “soft prompts”, are a widely-adopted parameter-efficient tuning strategy for large language models, but are often less favorable due to their opaque nature. Prior attempts to interpret continuous prompts relied on projecting individual prompt tokens onto the vocabulary space. However, this approach is problematic as performant prompts can yield arbitrary or contradictory text, and it individually interprets each prompt token. In this work, we propose a new approach to interpret continuous prompts that elicits textual descriptions from their representations during model inference. Using a Patchscopes variant (Ghandeharioun et al., 2024) called InSPEcT over various tasks, we show our method often yields accurate task descriptions which become more faithful as task performance increases. Moreover, an elaborated version of InSPEcT reveals biased features in continuous prompts, whose presence correlates with biased model predictions. Providing an effective interpretability solution, InSPEcT can be leveraged to debug unwanted properties in continuous prompts and inform developers on ways to mitigate them.

## introduction continuous prompts, or "soft prompts", are an efficient and widely-adopted solution for priming pretrained large language models (llms) to solve various tasks @xcite . however, they are often less favorable compared to alternative parameter-efficient tuning methods, such as discrete prompt tuning, due to their opaque nature @xcite . how should continuous prompts be interpreted? prior work explored discretizing continuous prompts through projection to the model's vocabulary space @xcite . however, such approaches are problematic because it is possible to find performant continuous prompts that map to arbitrary or contradictory text ( @xcite . moreover, they assume that each prompt token has an individual interpretable meaning, which does not necessarily hold. 1 in this work, we introduce a new approach for interpreting continuous prompts that overcomes these limitations. we propose to elicit textual descriptions of the prompt from its representations, constructed by the model during inference. this is done by using the patchscopes framework @xcite ; the prompt representations are extracted during inference and "patched" into a separate inference pass that steers the model to generate a textual description of the task (see illustration in figure 1 ). concretely, we define a task-description patchscopes, called inspect (inspecting soft prompts by eliciting task descriptions), that relies on a few-shot target prompt for decoding task descriptions from patched continuous prompt tokens. unlike vocabulary projections that produce a discrete replacement for the prompt, inspect yields natural and easy-to-understand interpretations not bounded by its length. we use inspect to obtain descriptions of continuous prompts trained for 5 tasks, and find that it often yields accurate descriptions of the relevant target task (see examples in in summary, our work introduces a novel and practical approach for interpreting continuous prompts by eliciting natural descriptions from their representations. we release our code at https: //github.com/danaramati1/inspect . ## eliciting textual descriptions of continuous prompts we detail our approach for interpreting continuous prompts, which are learnable tensors concatenated with an input and optimized for a specific task on top of a frozen model @xcite . let m be a pre-trained, auto-regressive transformerbased llm @xcite with l layers, and p cont := ⟨p 1 , ..., p n ⟩ a continuous prompt optimized for classification task t . our method elicits comprehensible descriptions of continuous prompts from m 's hidden representations, unlike prior work that maps these representations directly to discrete prompts. we build on the patchscopes framework @xcite , which decodes information from a source prompt by "patching" its hidden representations into the inference pass of a carefully designed target prompt. by conditioning m 's generation on source representations through patching, these target prompts guide m to generate human-readable text reflecting the information encoded in them. we introduce inspect, a patchscopes variant for deciphering continuous prompts, which are learned disjointedly from the model's representation space. since these embeddings are optimized via an external loss function, they are not constrained to the distribution of standard language token representations. moreover, although they lie within the model's hidden representation spacethat is, their dimensionality matches that of the model's hidden states -they may reside in an opaque, compressed feature space designed to facilitate task performance. this is different from existing patchscopes (e.g., @xcite ; pal n example elicited descriptions sst-2 14 identify the sentiment of a text: positive or negative 14 categorize the tone of a text as positive, negative, or neutral 7 identify the author's intention in this text: positive, negative or neutral subj 56 "subjective opinion or objective fact? 28 subjective, objective, or both? 56 "subjective, objective, or neutral? it is a subjective, objective, or neutral text? agnews 28 identify the topic of this article: technology, business, sports, world 56 sports? technology? business? world? 28 world, technology, business, sports, and politics inspect we treat p cont as the source prompt we interpret, and design a few-shot task-description target prompt p target : denote by p ℓ i the hidden representation of p i at layer ℓ when running m on p cont . similarly, let x ℓ i be the hidden representation of the i-th placeholder token in the inference pass of m on p target . to elicit a textual description of p cont , we patch the representations p ℓ 1 . . . p ℓ n at some layer ℓ into the corresponding placeholder token representations x ℓ * 1 . . . x ℓ * n at some layer ℓ * and let m generate a sequence of tokens. if m processes p cont as a task description, we expect it will follow the structure of p target and decode p cont into a human-readable description and set of classes for t . ## debugging continuous prompts we demonstrate the utility of inspect for debugging continuous prompts trained over the snli dataset @xcite . another analysis addressing the low task accuracy on sst-5 is included in §g. ## conclusion we tackle one of the major hurdles of continuous prompts -their lack of transparency. we show that accurate task descriptions can be elicited with inspect from the model's internal representations, and task performance improves as the model's own interpretation of the prompts becomes more faithful. additionally, inspect can identify biased features in continuous prompts from the presence of prominent words in the generated outputs. overall, our work provides an effective interpretability solution that can be leveraged to debug unwanted properties in continuous prompts.