Title: FIPER: Factorized Features for Robust Image Super-Resolution and Compression
Venue: NeurIPS 2025
Accepted: yes
Keywords: Super-resolution

Image compression
Abstract: In this work, we propose using a unified representation, termed **Factorized Features**, for low-level vision tasks, where we test on **Single Image Super-Resolution (SISR)** and **Image Compression**. Motivated by the shared principles between these tasks, they require recovering and preserving fine image details, whether by enhancing resolution for SISR or reconstructing compressed data for Image Compression. Unlike previous methods that mainly focus on network architecture, our proposed approach utilizes a basis-coefficient decomposition as well as an explicit formulation of frequencies to capture structural components and multi-scale visual features in images, which addresses the core challenges of both tasks. 
 We replace the representation of prior models from simple feature maps with Factorized Features to validate the potential for broad generalizability.
 In addition, we further optimize the compression pipeline by leveraging the mergeable-basis property of our Factorized Features, which consolidates shared structures on multi-frame compression.
 Extensive experiments show that our unified representation delivers state-of-the-art performance, achieving an average relative improvement of 204.4\% in PSNR over the baseline in Super-Resolution (SR) and 9.35\% BD-rate reduction in Image Compression compared to the previous SOTA.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\n1. The paper proposed a general feature decomposition method that can handle both image super resolution and compression. \n2. Instead of using fixed basis as fourier transform and wavelet transform, this paper is proposed to generate image specific basis, which help to improve the high-frequency repeating patterns recovery.\n3. Experiments show that the proposed method achieves SOTA performance on multiple benchmarks.\n\nSTRENGTHS AND WEAKNESSES:\n1. The experiments are solid. Toy example in Figure 2 shows the advantage of learnable basis.\n2. The pipeline is novel, which combines the advantage of both deep network and fixed basis.\n3. The description is a bit incoherent, and sometimes hard to follow. Figures (5) lack captions, which need some captions for better understanding.\n4. Experiments show that repeating patterns can be better recovered.\n\nQUESTIONS:\n1. What is z in equation (9)? And what is the R here?\n2. The learned coefficients and basis are pixel-wise, but Fourier transform is global. So how to extend equation (2) to (3) is not clear to me.\n3. In Fig. 2(b), what is the meaning of \"trainable basis settings\" here, I thought x and y are fixed.\n4. Can authors explain more about the alpha in Figure. (4)?\n5. Any visualization of the extracted high-frequency features?\n6. How to learn the k? Is it a pure scalar or the output of some conv layers.\n7. Figure 5 looks like learning the coefficients and basis in parallel. I do not fully understand why authors claimed \"our method derives the coefficient features first\"\n8. Any Coordinate Transformation tried other than Sawtooth?\n\nLIMITATIONS:\nCan authors explain more about the following limitation? \nAlthough effective, our method requires further optimization for computation-limited scenarios such as real-time decoding.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes \"Factorized Features,\" a unified representation for low-level vision tasks, specifically demonstrating its effectiveness in Single Image Super-Resolution (SISR) and Image Compression. Unlike prior work focusing on network architecture, their approach uses a learned basis-coefficient decomposition with explicit frequency formulation to capture structural and multi-scale features. This representation is designed to generalize broadly, replacing simple feature maps in existing models. The authors further optimize image compression by leveraging a mergeable-basis property for multi-frame scenarios. Extensive experiments show state-of-the-art performance, with significant PSNR gains in SR and BD-rate reduction in image compression.\n\nSTRENGTHS AND WEAKNESSES:\nStrength\n\n1. The concept of \"Factorized Features\" as a unified representation for diverse low-level vision tasks like SR and compression is a significant and novel contribution. This approach moves beyond purely architectural improvements to focus on fundamental image content representation.\n2. By replacing simple feature maps with Factorized Features, the paper successfully demonstrates the potential for broad generalizability across different low-level vision tasks and existing backbones.\n\n\nWeaknesses:\n\n1. While \"Learned Non-uniform Basis\" is introduced, the mechanism for generating the entire basis map  for better fitting and local reconstruction needs more in-depth explanation beyond just \"networks to generate.\"\n\n2. While a \"mergeable-basis property\" is mentioned for multi-image compression, the explanation (Eq. 10 and related text) is brief and could be expanded to clarify the practical benefits and challenges in this setup.\n\nQUESTIONS:\n1. For \"broad generalizability,\" the paper only evaluates on two low-level vision tasks (SISR and image compression). Expanding to other tasks like denoising, deblurring, or inpainting would further strengthen the claim.\n\n2. The Lack of ablation:\n  2.1 Could a more comprehensive ablation study be provided to quantify the individual contribution of each innovative component (e.g., presence/absence of spatially variant coefficients, learned non-uniform bases, coordinate transformation, and multi-frequency modulation) to the final performance? This would be crucial for understanding which elements are the key drivers behind the observed performance improvements.\n2.2 Could an ablation study on the choice of N (the number of frequency components) be provided? How do different values of N affect the model's performance, complexity, and its ability to capture fine image details? Is there an optimal value of N, or does it vary depending on the specific task?\n\nLIMITATIONS:\nSee Questions.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nFIPER introduces factorized features, which can be used for low-level vision tasks such as super-resolution and image compression. Unlike previous works, FIPER decomposes features into basis and coefficients while explicitly modelling frequency components. It replaces standard feature maps and demonstrates strong generalization across both tasks. Experiments show significant performance improvement for both super-resolution and image compression tasks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\nThe core contribution, leveraging a frequency-aware decomposition (basis × coefficient) across both super-resolution and image compression, is well motivated and novel.\n\nThe paper reports significant improvements over prior SOTA on multiple SR benchmarks (e.g., Set5, Urban100) and achieves a 21% BD-Rate gain on image compression compared to VTM, outperforming several recent methods like TCM and ELIC.\n\n\nWeaknesses:\n\nThe method requires pre-trained strong backbones (e.g., HAT, DAT). A comparison with lightweight SR/IC methods would improve practical relevance.\n\nThe paper presents Factorized Features as a “unified representation for low-level vision,” yet experiments cover only 4× SR and image compression; no evidence is given for de-blurring, denoising, and other low-level vision tasks.\n\nQUESTIONS:\nCould you report results (accuracy and runtime) with a lightweight CNN or Mobile-ViT backbone to demonstrate wider applicability?\n\nWhy choose N = 6 coefficient–basis pairs and alpha ∈ {1, 4, 16, 64}?  An ablation on N and frequency counts would help\n\nFigure 3 lacks clarity, as the results for the toy example appear identical across all cases. Can you shed more light on it?\n\nWhat would be the effect if, instead of applying the coordinate transformation \\gamma to x in Equation 6, we use a sine or cosine transformation?\n\nLIMITATIONS:\nPlease see weaknesses and questions", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper is interesting. The authors proposes a 100M network to estimate the best bases and coefficients for feature representation for SR and compression tasks. Experiemental results show obvious improvment in terms of repetitive textures and fine details.\n\nSTRENGTHS AND WEAKNESSES:\nA new network design for base and coefficient estimation. \nCan this network be applied to other tasks, such as image /video generations?\n\nStrengths\n1. Unified Representation for Multiple Tasks: Factorized Features generalize across SISR and Image Compression, addressing core challenges (fine detail recovery, structural preservation) with a single framework, unlike task-specific prior methods.\n2. Explicit Frequency and Structural Modeling: By combining basis-coefficient decomposition, spatially variant coefficients, learned non-uniform bases, and multi-frequency modulation, the method effectively captures both low-frequency global structures and high-frequency local details, overcoming limitations of fixed sinusoidal bases (e.g., poor flexibility) and pure network architecture optimizations (e.g., neglect of frequency patterns).\n3. Optimized Compression Pipeline: The \"mergeable-basis\" property enables consolidating shared structures in multi-frame compression, reducing redundancy and improving efficiency.\n4. Strong Empirical Performance: Outperforms SOTA methods on benchmarks:\nIn SR, achieves higher PSNR/SSIM across datasets (e.g., Set5, Urban100).\nIn compression, delivers significant BD-rate reductions (e.g., -21.09% vs. VTM for single-image, -20.97% for multi-image with M=24).\nCross-Task Prior Integration: Leverages pre-trained SR models as priors for image compression, enhancing information recovery.\n\nWeaknesses\n1. Computational Efficiency: The method requires optimization for computation-constrained scenarios (e.g., real-time decoding) due to higher model complexity (e.g., larger parameters and MACs compared to some baselines, as shown in Table 1 and 2).\n2. Lack of Semantic Information: The framework does not incorporate semantic understanding, which could limit performance in tasks requiring context-aware detail preservation.\n3. Scalability Challenges in Multi-Frame Compression: While merging bases improves efficiency, distortion increases slightly as the number of merged frames (M) grows, stretching basis capacity.\n4. Dependence on Backbone Architecture: Performance gains rely on integrating with strong backbones (e.g., HAT, DAT), which may limit applicability to lightweight models.\n\nQUESTIONS:\nFailure cases can be given in the paper.\n\nLIMITATIONS:\nCan the Factorized Features be overfitted for certain tasks? i.e. task specific.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}