PRISM-Δ: Differential Subspace Steering for Prompt Highlighting in Large Language Models
Authors: name: Yuyao Ge, name: Shenghua Liu, name: Yiwei Wang, name: Baolong Bi, name: Lingrui Mei, name: Jiayu Yao, name: Jiafeng Guo, name: Xueqi Cheng
Organization: Institute of Computing Technology, Chinese Academy of Sciences

Prompt highlighting steers a large language model to prioritize user-specified text spans during generation. A key challenge of existing Key-editing approaches is extracting steering directions that capture the difference between relevant and irrelevant contexts, rather than shared structural patterns common to both. We propose PRISM-Δ (Projection-based Relevance-Informed Steering Method), which decomposes the difference between positive and negative cross-covariance matrices to maximize discriminative energy while eliminating shared directions. Each attention head receives a continuous softplus importance weight, letting weak-but-useful heads contribute at reduced strength. The framework extends naturally to Value representations, capturing content-channel signal that Key-only methods leave unused. Across four benchmarks and five models, our methods match or exceed the best existing method on 19 of 20 configurations, with relative gains up to +13.0%, while halving the fluency cost of steering. Our methods also scale to long-context retrieval, outperforming the best existing method by up to +5.1% relative gain. PRISM-Δ is compatible with FlashAttention and adds negligible memory overhead.