Title: Trust but Verify: Adaptive Conditioning for Reference-Based Diffusion Super-Resolution via Implicit Reference Correlation Modeling
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords: diffusion_models, deep_learning, reference-based super-resolution

Abstract: Recent works have introduced reference-based super-resolution (RefSR) to alleviate hallucinations in diffusion-based image restoration. The key challenge is constructing reliable correspondences between the low-quality (LQ) input and a reference (Ref) image, especially under severe degradations. Existing methods attempt to balance LQ and Ref contributions through weighting-based fusion strategies, but this often leads to unstable behavior—either over-reliance on references that produces unnatural details, or under-utilization of references that limits recovery quality. To address this, we adopt a “Trust but Verify” perspective: the model should leverage references when they align and suppress them otherwise. Based on this principle, we propose Ada-RefSR, a single-step reference-based diffusion framework. Its core component is the Adaptive Implicit Correlation Gating (AICG), which employs learnable summary tokens to distill dominant reference patterns and capture implicit correlations with LQ features. Integrated into the attention backbone, AICG adaptively regulates reference guidance and serves as a built-in safeguard against erroneous fusion. Experiments on multiple datasets demonstrate that Ada-RefSR achieves a strong balance of fidelity, naturalness, and efficiency, while remaining robust under varying reference alignment. Code and models will be publicly released.

TL;DR: This paper introduces a correlation-aware gating mechanism that adaptively balances fidelity and reference guidance in single-step diffusion super-resolution, improving robustness under varying reference quality.