BiMaCoSR: Binary One-Step Diffusion Model Leveraging Flexible Matrix Compression for Real Super-Resolution
Authors: Kai Liu
Kaicheng Yang
Zheng Chen
Zhiteng Li
Yong Guo
Wenbo Li
Linghe Kong
Yulun Zhang
Venue: ICML 2025 poster (2025)
Keywords: super-resolution
one-step diffusion
binarization
quantization
Primary area: applications->computer_vision
TL;DR: A binarized one-step diffusion model for super-resolution.

Abstract: While super-resolution (SR) methods based on diffusion models (DM) have demonstrated inspiring performance, their deployment is impeded due to the heavy request of memory and computation. Recent researchers apply two kinds of methods to compress or fasten the DM. One is to compress the DM into 1-bit, aka binarization, alleviating the storage and computation pressure. The other distills the multi-step DM into only one step, significantly speeding up inference process. Nonetheless, it remains impossible to deploy DM to resource-limited edge devices. To address this problem, we propose BiMaCoSR, which combines binarization and one-step distillation to obtain extreme compression and acceleration. To prevent the catastrophic collapse of the model caused by binarization, we proposed sparse matrix branch (SMB) and low rank matrix branch (LRM). Both auxiliary branches pass the full-precision (FP) information but in different ways. SMB absorbs the extreme values and its output is high rank, carrying abundant FP information. Whereas, the design of LRMB is inspired by LoRA and is initialized with the top r SVD components, outputting low rank representation. The computation and storage overhead of our proposed branches can be safely ignored. Comprehensive comparison experiments are conducted to exhibit BiMaCoSR outperforms current state-of-the-art binarization methods and gains competitive performance compared with FP one-step model. Moreover, we achieve excellent compression and acceleration. BiMaCoSR achieves a 23.8x compression ratio and a 27.4x speedup ratio compared to FP counterpart. Our code and model are available at https://github.com/Kai-Liu001/BiMaCoSR

BiMaCOSS: Binary One-Step Diffusion Model Leveraging Flexible Matrix Compression for Real Super-Resolution
Kai Liu
Kaicheng Yang" *1 Zheng Chen
Zhiteng Li Yulun Zhang
Yong Guo Wenbo Li Linghe Kong
Abstract
LPIPS
SSIM
While super-resolution (SR) methods based on diffusion models (DM) have demonstrated inspir- ing performance, their deployment is impeded due to the heavy request of memory and com- putation. Recent researchers apply two kinds of methods to compress or fasten the DM. One to compress the DM into 1-bit, aka binarization. alleviating the storage and computation pressure. The other distills the multi-step DM into only step, significantly speeding up the inference process. Nonetheless, it remains impossible to deploy DM to lemiteee-lmited edge devices. To address this problem, we propose BiMaCoSR, which combines binarization and one-step distil- lation obtain extreme compression and accel- eration. To prevent the catastrophic collapse of the model caused by binarization, we propose sparse matrix branch (SMB) and low rank matrix branch (LRMB). Both auxiliary branches pass the full-precision (FP) information but in differ- ent ways. SMB absorbs the extreme values and output is high rank, carrying abundant FP in- formation. Whereas, the design of LRMB in- spired by LoRA and is initialized with the top SVD components, outputting low rank repre- sentation. The computation and storage overhead of our proposed branches can be safely ignored. Comprehensive comparison experiments are con- ducted to exhibit BiMaCoSR outperforms current state-of-the-ant binarization methods and gains competitive performance compared with FP one- step model. Moreover, achieve excellent com- pression and acceleration. BiMaCoSR achieves 23.8x compression ratio and 27.4x speedup ratio compared to the FP counterpart. Our code and model are available at https://github..oo. com/auii
MUSIQ
-SNR
CLIP-IA++
Ours
ReactNet
DISTS
NIO
XNDR
BBCU
FID
BIDM
Figure I: Performance comparison between binarization
methods on the RealSR dataset. BiMaCoSR achieves con- sistently leading scores on all evaluation metrics.
1. Introduction
Single image super-resolution (SR) (Keys, 1981; Zibetti & Mayer, 2007; Lu et al., 2013; Dong et al., 2014; Yang et al., 2014; Dong et al., 2016) is a traditional yet challenging low- level vision problem. Serving as a fundamental research task, it has attracted long- long-standing and considerable atten- tion in the computer vision community. The final object of SR is to restore high-quality (HQ) image from its low- quality (LQ) observation, which suffers from various image quality degradations. The difficulty of SR mainly lies in two parts: (1) the unknown degradations (e-g., blur, downsam- pling, noise, compression, and their combinations) of LQ. (2) the multiple solutions for given LQ input image.
In recent years, numerous studies have been made to tackle this challenge, utilizing convolution neural networks (CNNs) (Dong et al., 2014; 2016; Ledig et al., 2017; Zhang et al.. 2018c), vision transformers (ViTs) (Zhang et al., 2018b; Liang et al., 2021; Wang et al., 2022; Chen et al., 2022: 2023), and their combinations. Though achieving inspiring results, these methods mostly fail real-world scenarios. This failure is attributed to assuming degradation an ideal bicubic downsampling kernel, way too different from the unknown and complex degradation in real world. Therefore, it draws researchers increasing attention to re- construct perceptually realistic HQ images in real-world scenarios. Thereafter, this challenging and meaningful task is called real-world super-resolution (Real-SR) (Gu et al.,
Equal contribution 'Shanghai Jiao Tong University HHaawei Consumer Business Group 'The Chinese University of Hong Kong. Correspondence to: Linghe Kong <linghe kong @ sytu.ccu..cn.... Yulum Zhang <yulun 100@ gomi..com>.
Proceedings of the nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

BiMaCoSR: Binary One-Step Diffusion Model Leveraging Flexible Matrix Compression for Real Super- Resolution
2019; Zhang al., 2021; Wang et al., 2021; Cai et al., 2019; Yu et al., 2024; Wu et al., 2024b; Sun al., 2024).
2. Related Work
Image Super-Resolution. Deep learning based ap- proaches have demonstrated striking power in the realm of SR (Dong et al., 2014; Luo et al.. 2022; Wang al., 2021; Lim et al., 2017: Chen et al., 2023). As a ground- breaking work, SRCNN (Dong et al., 2014) initiates the track of solving SR problem via deep learning based ap- proach. Thereafter, substantial contributions have been made to explore the best SR network architecture. For exam- ple, RCAN (Zhang et al., 2018b) leverages the residual in residual structure and deepens the network more than 400 layers. SwinIR (Liang et al., 2021) is based on vision trans- former structure and utilizes spatial window self self-attention to capture the overall structure information. CAT (Chen et al., 2022) combines the attention mechanism and the CNN structure to make the most of the local and the global information. However, most of these conventional image super-resolution methods can not handle the Real-SR task because of the complex degradation in the real world.
Recently, diffusion models (DMs) demonstrate remarkable performance in srange-generatnng tasks, particularly in per- ceptual quality. The excellence of DM comes from its vast prior knowledge in modeling real-world objects, especially generating clear textures and mitigating artifacts and distor- tions. The powerful realistic texture generation ability is inherently the same the object of Real-SR problem, lead- ing to plentiful breakthroughs (Yang et al., 2025; Yu et al., 2024; Rombach et al., 2022). However, the inference cost is still too high to run on edge devices. Therefore, it's essential to further compress DMs to accelerate the inference, reduce storage cost, and minimize degradation.
Popular model compression techniques include pruning, dis- tillation, and quantization, among which, 1-bit quantization (i.e., binarization) gains significant effectiveness. As an extreme quantization method, binarization compresses mod- els' weights from 32-bit to only 1-bit, significantly reducing memory and computational cost. However, applying naive binarization will lead to catastrophic model collapse. Hence, additional structures are required to be designed.
Diffusion Model. In recent years, the diffusion-based methods have gained remarkable performance in many computer vision tasks and SR is no exception. For in- stance, SR3 (Saharia et al., 2022) restores the LQ by trans- forming the standard normal distribution into the empiri- cal data distribution by learning a series of iterative refine- ment steps. DifBBIR (Lin et al., 2024) capitalizes on two restoration stages to seek the tradeoff of fidelity and quality. SinSR (Wang et al., 2024) effectively reduces the inference step to only one step via distillation and regularization. Fol- lowing SinSR, OSEDiff (Wu et al., 2024a) modifies the distillation paradigm, and novel losses are introduced to im- prove face restoration ability. Despite the greatly improved inference speed, the model size remains the sam,e and there is still room for further acceleration.
A popular solution is adding full-precision information, i.e., skip connection branch. However, due to UNet's fre- quent changes in resolution and dimension, skip connection faces the mismatch challenge. To address this problem, we propose two auxiliary branches, namely low rank matrix branch (LRMB) and sparse matrix branch (SMB). Inspired by LoRA, the proposed LRMB leverages low rank decom- position to achieve dimension shift. We select the top singular values in SVD and utilize its corresponding compo- nents to initialize the LRMB. As for SMB, we employ sparse matrix to absorb the top k absolute values in a full full-precision matrix. The weights of LRMB and SMB are subtracted from the binarized matrix branch (BMB) and it concentrates on restoring textures. Three branches form the BiMaCoSR and provide excellent performance shown in Fig. 1.
Binarization. As the most extreme form of quantization, binarization typically compresses the weight into only bit. In binarization, all the weights are seen as #1 and the multi- plications between weights and activations are converted to bit operations on sign bit of activation, allowing maximum compression and acceleration. Binarization is mainly about classification tasks initially (Rastegari et al., 2016; Liu et al., 2020; Qin et al., 2020; 2022: Huang et al., 2024). Recently, researchers have begun to perform binarization on image restoration tasks. Binary Latent Diffusion (Wang et al., 2023b) trains an auto-encoder with a binary latent space and mainly focuses on the Bernoulli distribution instead of acceleration. BiDM (Zheng et al., 2024) leverages timestep- friendly binary structure and space-patee patched distillation to compress the diffusion model to 1 bit. BI-DiffSR (Chen et al., 2024) designs several binary friendly modules and re- distributes the activation of different time step. However, the inference step remains the same. Therefore, it is necessary to further compress the model to one step.
To sum up, the contributions of our work are as follows:
1. We design BiMaCoSR, a new binarized one-step dif- fusion model for image super-resolution. To the best of our knowledge, BiMaCoSR is the first binarized one-step diffusion model.
2. We propose LRMB, which leverages low rank decom- position and SVD initialization to carry low frequency information and decouple the effect of BMB.
3. We propose SMB, which utilizes sparse matrix com- pression and extreme value absorption to deliver high rank features and achieve further decoupling.
4. We conduct comprehensive comparison experiments to show the state-of-ther art performance of the proposed BiMaCoSR. Besides, extensive ablation studies are conducted to prove the robustness and efficacy.
2

---

BiMaCoSR: Binary One-Step Diffusion Model Leveraging Flexible Matrix Compression for Real Super- Resolution
(a]
(b)
X LAMB
p
Compute ability
#
SMB
High rank delivery
LRMB
Low rank delivery
1p SMB
LRMB
Information delivery
SMB
Xaut
x
Information damage
bi ABM
BMB
1-bit Conv
XNOR Post Process
Compute
Sign
ability
BMB
+
Elernentwwise Addition
X Matrix Multiply
M Information Deliver
Information Generator
Strength
Weakness
Overview of our proposed BiMaCoSR which employs three different compressed matrix branches. (a) The structure of a convolution layer in BiMaCoSR after binarization. Two auxiliary branches, i.e., LRMB and SMB, support BiMaCoSR's excellent performance. The linear layer can be regarded 1 x convolution layer and is processed with the same pipeline. (b) Illustration of the initialization and how the three branches solve the weakness of the other branches.
Figure 2:
3. Methodology
and resolution make two forms of skip connection (i.e., addition and concatenation) not applicable, In addition, the distributions before and after binarized block are greatly different. Therefore, efficient module is critical to overcome the frequent changes and carry rich information.
In this section, we describe our proposed BiMaCoSR, shown in Fig. 2. First, we analyze three issues that have potential for further improvement when binarizing diffuuson models. Thereafter, we describe our proposed branches, i.e., low rank matrix branch (LRMB) and sparse matrix branch (SMB), which could serve the auxiliary branch binarized ma- trix branch (BMB). Finally, illustrate the initialization methods, shown in Fig. 3, and why our designs work.
Issue III: The inadequate usage of pre-trained FP model. The pre-trained FP model vital to quantization methods from any perspective. However, current QAT research does not make the most of the pre-trained model, simply loading the FP parameters and directly beginning the training. is popular way to leverage additional modules to enhance the binarized network. The impact of the modules could vary when different initialization methods are applied.
3.1. Analysis
Binarized blocks inevitably suffer from representation degra- dation due the extreme 1-bit compression. Previous re- search made remarkable progress (Chen et al., 2024; Zheng et al., 2024) on binarization, yet there still remains room for improvement. We conclude three issues that deserve special attention in most diffusion models.
We conclude that the above issues are attributed to the insuf- ficient ability of binarization. To be specific, binarization is just one of the matrix compression methods, inher- ently unsuitable for multifaceted challenges. Therefore, propose following branches to compress matrix in dif- ferent ways. With the combination of various compressed matrices, the issues above can be appropriately addressed.
Issue The specially severe degradation of linear layer. Experimentally, we observe that with same size of param- eters, linear layer suffers from more severe degradation in comparison convolution layer. This observation in SR is consistent with previous research (Le & Li, 2023). With the advancement of DiT and binarization, this issue is gradually becoming significant and an additional process to quantize or binarize the linear layer is in urgent need.
3.2. Low Rank Matrix Branch
To address Issue I, i.e., the specially severe degradation of linear layer, identify that the quantized linear layer is strongly limited by the bit-width. An FP linear layer usually serves two purposes, delivering the complete information of input and performing appropriate linear transformation. With limited bit-width, the quantized linear layer cannot play two roles the same time. Therefore, we decouple the roles apart. Though binarized, the weight matrix is usually high rank and contributes more to high frequency, i.e., the
Issue II: The frequent dimension changes. It's necessary to leverage skip connection network. Skip connection could carry abundant feature information and bring negligible computation and storage overhead. However, in UNet, frequent changes in dimension
auxiliary branch in binarized
3

---

BiMaCoSR: Binary One-Step Diffusion Model Leveraging Flexible Matrix Compression for Real Resper-esooltiin
detailed structure and texture in image. Hence, we design low rank matrix branch (LRMB) to serve as complemen- tary branch, transmitting low frequency information.
1:00
0:30
4.50
3.00
0.40
0.50
INPUT
W
-0.40
-010
0.33
0.20
The idea of LRMB is inspired by low rank approximation in matrix theory, which is a common strategy for matrix compression. To transmit low frequency information, matri- ces with low rank are enough and efficient. Mathematically, given an m X n matrix W, it can be approximated with two low rank matrices, i.e., W W == BA, where B and A are m X r and x n. matrices respectively, and r < min(m, n)) is hyper-parameters denoting the rank of W. Therefore, the formula of LRMB is:
1.00
0:40
00.00
3.00
Select k Values
0.10
0:40
0.30
0.40
0.50
0.00
0.00
0.02
sparse SMB
W
BMB
W
-0.40
-0.30
-0.30
0.20
0.50
0.02
0.00
0.00
0.00
0.40
-.330
Top r SVD
(L) where Xmn E IR Xm and XLRMB E RNxx are the input and output of LRMB(.) respectively, and N is the number of tokens. Usually, W is square matrix and m = n.
XLRMB = (x)) == Xin
0:09
-.229
0:13
0.30
0.05
0.07
+0:24
LRMB
W BBB
0031
.033
3275
6.40
BA
0:02
30
0.84
0.02
0.06
As for complexity, the storage overhead is: O, (m r r B = rB(m + +n) < mnB', (2) where B = 32 or 16 and B' = are the number of bits required by one element in LRMB branch and binarized branch, respectively. Meanwhile, the computation overhead is:
Binarize Online
0.00
4.02
0.00
0.05
0.04
W
4.50
2.00
4.00
0.
W, Ok
BMB
0:15
0.00
0.03
0.08
res
-1.00
1:00
3.00
0.10
O₂ = N N Nr(m n). (3) This means that even saved with 32 bits, the storage and computation overhead of LRMB is acceptable. In conclu- sion, with LRMB, Issue 1 and Issue II are partly solved.
Figure 3: Initialization of different branch. Wres represents the initial quantization error. In our method, || = 1855,,, while |Wres = 1.1275 in direct binarization.
3.4. Pretrained Friendll Initialization
3.3. Sparse Matrix Branch
We propose the following initialization method to guarantee better use of the pre-trained model, allowing better restora- tion performance and faster convergence.
LRMB is still not enough to replace the skip connection. This is because skip connection can be represented a full rank identity matrix while LRMB is low rank. To com- pensate the missed ranks, we propose sparse matrix branch (SMB), which leverages sparse matrix compression and could efficiently deliver high rank information.
The purpose of LRMB is to carry the low frequency informa- tion. Usually, compared with high frequency information, the magnitude of low frequency information is much greater. Therefore, we leverage SVD and select the top r compo- nents to initialize LRMB. To be specific, we perform SVD on W:
Sparse matrix is a matrix in which most of the elements are zero (Yan al., 2017). A common criterion of sparse matrix is that the number of non-zero elements is approxi- mated to the number of rows or columns. One way to save sparse matrix is via the coordinate format (COO), where the non-zero elements are represented by a list of triples and it component is (row, col, value).
W = U.VT = Udiag 02:011002000000 02. ) V
(5)
Then, we truncate the top r singular values and absorb the remaining singular value into U, forming matrices and B. The process can be written as: W ~ Udiag( or. or (2.,0 0) V = BA. (6)
Specifically, to form sparse matrix, we select l critical values from the weight matrix and the selection method will be described in Sec. 3.4. In the forward process, the SMB process could be equivalently expressed as
B
Thereafter, guarantee that the binarized matrix only serves to add high frequency information, the low frequency coun- terpart will be subtracted. The formula is:
(4) where W sparse is a matrix with A: non-zero elements. During training, the coordinates will be fixed while their values will be updated by gradient descent. With SMB, the high rank information could be delivered and the remaining parts of Issue I and Issue II could be solved.
XSMB = SMB := X W sparse:
W WIMB W-
where W is the matrix to be binarized after the initial- ization of LRMB. Hence, the ability of weight matrix is decoupled into two components, the low frequency part (BA) and the high frequency part (W BMB)

---

BiMaCoSR: Binary One-Step Diffusion Model Leveraging Flexible Matrix Compression for Real Super- Resolution
Thereafter, the initialization of SMB also matters. As de- seribed in Sec. 3.3, we select k values from the weight matrix to generate the sparse matrix. Due to the fixed coor- dinates of non-zero elements, random selection is apparently a good candidate. Besides, binarization often ignores the outlier values in weight matrix, which are often huge but rare (less than 0. 1).. Though rare, these outliers play a crucial role in most models. Considering their rarity and significance, we select k elements with the greatest absolute value in W BMB Subsequently, we subtract these values in W BMB to decouple the overlapping effect of BMB and SMB. The process can be formulated as:
4. Experiments
4.1. Experimental Settings
Data. We take the models on the training set of Ima- geNet (Russakovsky et al., 2015) and the LR images are generated by the same pipeline of RealESRGAN (Wang et al., 2021). We evaluate the models with three bench- mark datasets: RealSR (Cai et al., 2019), DRealSR (Wei et al., 2020), and DIV2K-Val (Agustsson & Timofte, 2017). RealSR and DRealSR are real world benchmarks while DIV2K-Val employs Bicubic interpolation to generate LR images. The upscale ratio of training set and test set is x4.
Wij: if (uly 2t t WWEMB W
Evaluation Metries. The evaluation metrics are imple- mented with Iytorht (Chen & Mo, 2022) and are twofold to assess the restoration and compression ability. Firstly, to thoroughly evaluate the restoration ability of our proposed BiMaCoSR, we employ the following quantitative metries: full-reference metrics PSNR, SSIM (Wang et al., 2004), and LPIPS (Zhang et al., 2018a) and non-reference metrics DISTS (Ding et al., 2020), FID (Heusel et al., 2017), NIQE (Zhang et al., 2015), MANIQA- (Yang et al., 2022), MUSIQ (Ke et al., 2021), and CLIPIQA+ (Wang et al., 2023a). PSNR and SSIM are distortion- metrics and are calculated on the Y channel (i.e., luminance) of the YCbCr space. The rest metrics are all perceptual metries and it is widely known that perceptual metries are more aligned with human when rating the image quality.
W. 0, otherwise
BMB
W
sparse
(8)
where Wij and wiy are the elements of W sparse and W respectively, is the k-th largest absolute value in W BMB
BMB
WBMB is the final matrix to be binarized.
3.5. Overall Structure
After adding LRMB and SMB and their initialization, the binarized matrix branch (BMB) is formed with XBMB- To be specific, the input feature and weight matrix will be binarized with Sign(-). which can be written as:
W = Sign( Dign(WamBl (WMB)) Sign(z) = +1, x 220 0,
Xn = Sign(xi);
(9)
-1, x <, 0,
where Sign(-) is performed element-wise. Thereafter, the full-precision convolution and linear transform are replaced with efficient logical XNOR and bit-counting operations. The process can be formulated as:
Secondly, to demonstrate binarization's 's extreme compres- sion and acceleration ability, we use the total parame- ters and overall operations as key metrics. Following previous work (Xia et al., 2022; Qin et al., 2023), the total parameters (Params) of the model are calculated as Params=Params' P and the overall operations (OPs) as OPs-OPS"+OPP where Params"-Parane Params//33 /32 and OPs"=OPs/ /64; the superscripts f and b denote full- precision and binarized modules, respectively. The compu- tational complexity is tested with the input size 3x64x64, 64.
XBMB bit-coont(XNOR(Xx W))), (10)
Finally, we follow the design of XNOR-Net (Rastegari et al., 2016) to compensate the precision loss, which is:
XBMB- BBB @ (A @ k),
(11)
where Aнxw is the channel-wise absolute average of input activation, k is a vector whose element is the absolute av- erage of correspond channel, C is Hadamard product, and (A & k)..i.j == A. should be calculated first.
Implementation Details. We take SinSR (Wang et al., 2024), a one-step diffusion model, as the backbone. We initialize with SinSR's s weights and use ResShift (Yue et al., 2024) as the teacher model. All the rest convolution and lin- ear layers, except the head and tail layers, are binarized into BMB to guarantee the best compression ratio and LRMB and SMB are attached with BMB. We set the rank of LRMB as r = 8 and the number of non-zero elements in SMB as k = 2 max (C1, C2), where C₁ and C2 are the numbers of input and output channels, respectively.
We binarized every convolution layer and linear layer except the first and last convolution layers due to extremely severe degradation of quantization on head and tail. Specifically, linear layer can be seen as the therefore processed in the same way as convolution layer.
convolution layer and
For one binarized layer, the output can be written as:
Xout = XBMB + XLRMB + XSMB-
(12)
In conclusion, we propose LRMB, SMB, and their corre- sponding initialization methods. With these designs, our BiMaCoSR can significantly reduce the information loss caused by binarization and improve the restoration ability. Besides, the storage and computation overhead caused by LRMB and SMB can be safely ignored.
Training Settings. We utilize Adam optimizer (Kingma & Ba, 2015) with 31 = 0.9 and B3 = 0.99, and set the learning rate as 50 The batch size is set to 8, with 100K iterations. The input LR images are center-cropped to size 64 64. All the training and testing experiments are conducted on one NVIDIA RTX A6000.
5

---

All binarization methods are implemented on SinSR (Wang et al., 2024) and trained with the same settings. As LRMB and SMB slightly increase the parameters, we addition- ally keep the first two and last two convolution layers as full-precision. We also compare BiMaCoSR with the full- precision model SinSR and ResShift (Yue et al., 2024). SinSR the distilled version of ResShift. The comparison results are exhibited in quantitative and qualitative aspects. BiMaCoSR: Binary One-Step Diffusion Model Leveraging Flexible Matrix Compression for Real
Table 1: Quantitative comparison with SOTA methods. We compare BiMaCoSR with full-precision models and current binarization methods. We employ metrics commonly used in SR and test on three benchmarks. The best and second best value are marked with red and blue respectively. In conclusion, our proposed BiMaCoSR achieves SOTA performance.
Datasets
Methods
Bits (W/A)
PSNR
SSIM f
LPIPS L
MUSIQ
MANIQA f
DISTS 4
FID 1
NIQE
CLIP-IQA+ f
SinSR
32/32
26.51
0.7380
0.3635
57.87
0,5139
0.2193
56.36
5.826
0.5736
ResShift
32/32
25.45
0.7243
0.3731
56,23
0,5005
0.2344
58.14
7.353
0.5708
XNOR
1/1
26,48
0.7434
0.3968
43.56
0,3732
0.2609
105,72
6.014
0,4380
ReActNet
1/1
26.60
0.7530
0.3834
44.18
0.3829
0.2551
109.36
6.306
0,4361
RealSR
BBCU
1/1
26.43
0.7488
0.3902
43.70
0.3792
0.2575
108.32
6.058
0.4298
ReSTE
1/1
26.26
0.7408
0.4184
41.04
0,3677
0.2719
113.86
6.174
0,4083
BiDM
1/1
25.07
0,7036
0,5042
35.60
0,3517
0.3226
115.23
6.759
0.3935
BiMaCoSR
1/1
26.84
0.7698
0.3375
49.01
0,4034
0.2183
86.09
5,856
0,4800
SinSR
32/32
27.89
0.7332
0.4499
30.81
0.4519
0.2209
16.56
5.789
0.6052
ResShift
32/32
26.64
0.7298
0.4478
31.09
0.4345
0.2337
18.12
6.959
0.5795
XNOR
1/1
29.03
0.8319
0.3712
26.19
0.3560
0.2447
29.88
6.229
0,4449
ReActNet
1/1
29.34
0.8431
0.3571
26.83
0,3618
0,2411
30.18
6.561
0,4380
DRealSR
BBCU
1/1
29.00
0.8385
0.3643
26.37
0,3594
0,2433
30.94
6.337
0.4383
ReSTE
1/1
28.91
0.8353
0.3899
25.12
0,3509
0.2641
33.64
6,459
0.4131
BiDM
1/1
27.40
0.7942
0,4849
23.38
0.3529
0.3118
37.83
6.753
0,4307
BiMaCoSR
1/1
29.33
0.8393
0.3400
29.38
0.3802
0.2278
22.31
6.150
0.4867
SinSR
32/32
27.75
0.7694
0.1903
64.62
0,5336
0.1029
6.27
4.308
0.6147
ResShift
32/32
27.18
0.7667
0.1775
65.04
0.5548
0.1016
7.54
5.121
0.6280
XNOR
1/1
26.44
0.7185
0.3727
49.10
0,3972
0,2204
55.77
5.320
0.4584
ReActNet
1/1
26.49
0,7260
0,3602
50,29
0.4078
0.2111
52.32
5,366
0.4726
DIV2K-Val
BBCU
1/1
26.39
0.7221
0,3660
50.09
0.4035
0.2148
53.22
5,263
0.4653
ReSTE
1/1
26.07
0.7125
0.3916
46.95
0.3907
0.2295
61.52
5.399
0.4328
BiDM
1/1
24.29
0.6725
0.4370
40.15
0,3747
0.2916
62.28
6.090
0,4112
BiMaCoSR
I/1
27.35
0.7547
0,2999
53.38
0,4337
0.1806
27.99
4.987
0.5176
4.2. Comparison with State-of-the-Art Methods
performance on PSNR and SSIM is on the lower side. This phenomenon means that compression on DM leads to disap- pearance of texture and details. However, their performance PSNR and SSIM only slightly affected.
We compare our proposed BiMaCoSR with recent binariza- tion methods, including XNOR (Rastegari et al., 2016).
ReActNet (Liu et al., 2020), BBCU (Xia et al., 2022).
ReSTE (Wu al., 2023), and BiDM (Zheng et al., 2024).
Efficiency Comparison. Efficiency comparison results provided in Table 2. After the distillation from multi-step to one-step, the FLOPs are significantly reduced. Furthermore, compressing the model to bit makes the model tiny and fast. In comparison with SinSR, the compression ratio is 27.45x and the speedup ratio is 23.81 x. Compared with current SOTA binarization methods, we keep almost the same parameters but take much less FLOPs one-step DM, due to the calculation efficient design. To conclude, our proposed BiMaCoSR gains remarkable compression ratio and speedup ratio and achieves outstanding performance.
Restoration Results. The quantitative comparison results are shown in Table 1. We can obtain the following observa- tions. (1) The results Table demonstrate clear advantage over competing methods in both full-reference and non- reference metrics on three benchmark datasets. (2) In some situations, such as PSNR and LPIPS on RealSR, BiMa- CoSR can even surpass SinSR and ResShift. We attribute the excellence to the I.MBB and SMB, which transmit the full-precision information in a parameter-efficient way. (3) Most binarization methods outperform the baseline model and ReActNet presents competing performance, especially on DRealSR. (4) The full-precision models, i.e., SinSR and Sesfhift, are excellent on perceptual metrics, such as DISTS and CLIP-IQA+. Whereas, the multi-step models'
Visual Results. We present visual comparison on challeng- ing cases in Fig. 4. One typical challenging case is the tiny and dense structures, such as hairs, grass, tiles, and faces. is struggling with previous binarization methods restore image details, especially in challenging cases. On the con- trary, our BiMaCoSR is able to recover results with sharper edges and richer textures. For example, in 0809, BiMaCoSR reconstructs the hairs on the nose while other methods just output rough color blocks. And in 0831, BiMaCoSR re- stores the woman's 's facial structures and expression while other methods smooth the organs the same color. In 0834, BiMaCoSR could successfully recover the tiles' texture and
6

---

BiMaCoSR: Binary One-Step Diffusion Model Leveraging Flexible Matrix Compression for Real Super-Resolution Resolution
ResShift
SinSR
ReActNet
BBCU
ReSTE
BiDM
XNOR
Ours
Inference Step
FLOPs (G)
753.45
50.23
5.83
11.60
Total Param (M)
118.59
118. 59
4,95
4.95
4.95
18.69
4.95
4.98
PSNR/LPIPS
25.45/0.3731
26.51/0.3635
26.60/0.388 3834
26.43/0.3302
26.26/0.44 4184
25.07/0.5042
26.48/0.3968
26.84/0.3375
Table 2: Efficiency comparison on RealSR. BiMaCoSR takes least FLOPs while gains the best performance.
HR
SinSR (FP)
ReSTE
BiDM
DIV2K- 0809
BBCU
XNOR
ReactNet
BiMaCoSR (ours)
SinSR (FP)
BiDM
DIV2K- Val: 0831
BBCU
XNOR
ReactNet
BiMaCoSR (ours)
HR
SinSR (F-P)
ReSTE
BiDM
DIV2K-VV Val: 0834
BBCU
XNOR
ReactNet
BiMaCoSR (ours)
Figure 4: Visual comparison for image SR. We compare our proposed BiMaCoSR with current competitive hinarization methods and the full-precision (FP) model. The visual results illustrate that BiMaCoSR gains rich details and reasonable textures.
most binarization methods fail. What's more, the difference between BiMaCoSR and the FP model (SinSR) exists but is minimal. To conclude, our proposed BiMaCoSR surpasses other methods according to visual comparison. Additionally, we provide more visual results and corresponding analysis in the supplementary material for further comparison.
Break Down Ablation. Table 3a shows results of the break down ablation. With only BMB, the model could be ex- tremely compressed but the performance is somewhat on the low side. With LRMB, though the number of parame- ters increases 1.83G, the performance also improves on metrics and the FLOPs only increase 4.98M. After adding SMB, the overhead of parameters and storage be neglected. The performance on all perceptual metrics improves while both PSNR and SSIM drop slightly.
4.3. Ablation Study
In this section, we adopt RealSR as the test set and other training settings are the same as Sec. 4. To test the ef- fectiveness and robustness, we conduct five key ablation experiments, including the break down ablation, loss func- tion, rank of low rank branch, initialization of LRMB, and initialization of SMB. What's more, detailed analysis is also provided in the following sections. These ablation studies demonstrate the robustness and efficiency of our LRMB, SMB, and their corresponding initialization methods.
Loss Function. The loss functions of SinSR play a critical role in FP model. However, these loss functions are not suitable in the binarized situation. After removing unneces- sary parts from the SinSR loss, leave only the distillation loss and the results are shown in Table 3b. With only the distillation loss, BiMaCoSR gains improvement on both distortion metrics and perceptual metrics.

---

BiMaCoSR: Binary One-Step Diffusion Model Leveraging Flexible Matrix Compression for Real Super- Resour-nsooltionn
Branch
PSNR +
SSIM 1
LPIPS L
MANIQA f
FID
CLIP-IQA+ t
FLOPs (G)
Param (M)
BMB
26.41
0.7408
0.4141
0,3704
110.15
0.4325
0.78
3.69
+LRMB
26.95
0.7718
0,3400
0,3937 0.4034
88.72
0.4663
1.83
4.98
+LRMB+SMB
26.84
0.7698
0.3375
86.09
0,4800
1.83
4.98
(a) Break down ablation.
Loss
PSNR f
SSIM 1
I.PPSS 4
CLIP-IQA++ f
Initialization
PSNR
SSIM
LPIPS J CLIP-IQA+ 1
Distill loss
26.83
0.7698
0.3375
0,4800
Zero + RRadom
26,88
0.7660 0.7698
0.3497 0.3375
0.4674 0.4800
SinSR loss
26.37
0.7466
0.4029
0,4273
SVD
26.84
(b) Ablation study on losses.
(c) Ablation study on LRMB initialization.
Rank
PSNR
SSIM
LPIPS 4 CLIP-IQA+ t FLOPs (G)
Param (M)
Initialization
PSNR
SSIM f I.PISS + CLIP-IQ++
4
26.29
0.7383
0.4197
0.4411
1.31
4.37
8
26.84
0.7698
0.3375
0.4800
1.83
4.98
Zero initial
26.72
0.7628
0,3561
0,4639
12
26.97
0.7695
0,3400
0.4766
2.32
5.60
Uni-shorteut
25.69
0.7113
0.5105
0,4008
16
26.72
0.7625
0.3442
0.4971
2.83
6.21
Sparse skip
26.84
0.7698
0.3375
0.4800
(d) Ablation study on the rank LRMB.
(e) Ablation study on SMB initialization.
Table 3: Ablation studies on branches, loss, rank of LRMB and initialization of LRMB and SMB. The experiments is tested on RealSR. The comprehensive results demonstrate the robustness and efficient performance of our proposed BiMaCoSR.
Rank of LRMB. The rank of LRMB significantly influences the complexity, while the performance is not consistent with the increase in complexity. As shown in Table 3d, the model with rank of 16 performs best only on CLIP-IQA+, while the model with rank of 8 performs best and takes acceptable parameters. This is because the LRMB with high rank may influence high frequency generated by BMB, making the training unstable and leading to worse results. Therefore, we ultimately set the rank of LRMB to 8.
1.0
Proportion
0.4
Branch Contribution
0.2
BMB
LAMB
Initialization of LRMB. Conventional LoRA's initializa- tion method is set the first matrix with random values and the second matrix with zero values. We compare this initial- ization method with our proposed SVD initialization and the result is shown in Table 3c. To conclude, our method out- performs conventional methods on SSIM, LPIPS, and CLIP- IQA+. We attribute it to the decoupling of transmitting low frequency information and generating high frequency. To conclude, our proposed SVD allows better performance.
0.0
20
Layer Layer 50
Figure 5: The proportion of high frequency information gen- erated by three branches. The high frequency information mainly comes from BMB, which obeys assumption. mation is mainly generated by BMB, accounting for around 70%. With more parameters, LRMB also provides high frequency information in some layers but overall proportion is relative low. SMB only absorbs the extreme values and in- deed carries little high frequency information. Fig. 5 shows that division of work is clear and our designs in LRMB, SMB and BMB obey our assumption.
Initialization of SMB. We search three popular ways to initialize SMB, including (1) random position and zero ini- tialization, (2) Uni-shortcut (Xu et al., 2022), and (3) our proposed sparse skip. The experiment results in Table 3e show that our sparse skip initialization consistently enjoys the best performance on all four metrics. This improvement compared with other two initialization methods strongly supports the effectiveness of decoupling and absorption.
5. Conclusion
In this paper, we propose the BiMaCoSR, a binarized SR diffusion model with only one inference step. Detailedly, we first propose LRMB and SVD initialization to decouple the effect of binarized branch and deliver low frequency in- formation. Furthermore, we propose SMB and sparse initial- ization to absorb the extreme values and provide high rank representations. Comprehensive comparison experiments demonstrate the SOTA restoration ability of the proposed BiMaCoSR. Extensive ablation studies exhibit the efficiency and robustness of both LRMB and SMB. In the future, we will focus on the combination of pruning and binarization on one-step diffusion models for further compression.
4.4. Visualization Analysis.
In Sec. 3, we assume that LRMB and SMB are in charge of transmitting low frequency information while BMB gen- erates high frequency after decoupling. To validate this assumption, we visualize the proportion of high frequency generated by three branches in first 50 Conv layers and the result is shown in Fig. 5. On average, high frequency infor-
8

---

BiMaCoSR: Binary One-Step Diffusion Model Leveraging Flexible Matrix Compression for Real Super-Resolution
Acknowledgements
Huang. W., Zheng, X., Ma, X., Qin, H., Lv, C., Chen, H., Luo, J., Qi, X., Liu, X., and Magno, M. An empirical study of Ilama3 quantization: From Ilms to milms. Visual Intelligence, 2(1):36, 2024.
This work was supported by Shanghai Municipal Science and Technology Major Project (2011SHZDZX0102), the Fundamental Research Funds for the Central Universities, and the National Natural Science Foundation of China (NSFC No. W2412089).
Ke, J.. Wang. Q., Wang. Y., Milanfar, P., and Yang, F. Musiq: Multi-scale image quality transformer. In ICCV, 2021.
Keys, R. Cubic convolution interpolation for digital image processing. IEEE Trans. Acoust., Speech, Signal Process., 1981.
Impact Statement
This paper presents work whose goal advance the field of Machine Learning. There are many potential societal consequences of our work, none of which feel must be specifically highlighted here.
Kingma, D. and Ba, J. Adam: A method for stochastic optimization. In JCLR, 2015.
Le, P.-.. C. and Li, X. Binaryvit: pushing binary vision transformers towards convolutional models. In CVPR, 2023.
References
Agustsson, E. and Timofte, R. Ntir 2017 challenge single image super-resolution: Dataset and study. In CVPRW, 2017.
Ledig, C., Theis, L.., Huszár, F,, Caballero, J., Cunningham, A., Acosta, A., Aitken, A. P., Tejani, A., Totz, J., Wang, Z. et al. Photo-realistic single image super-resolution using generative adversarial network. In CVPR, 2017.
Cai, J., Zeng, H., Yong, H., Cao, Z., and Zhang, L. To- ward real-world single image super-resolution: A new benchmark and a new model. In CVPR, 2019.
Liang, J., Cao, J., Sun, G., Zhang, K., Van Gool, L., and Timofte, R. Swinir: Image restoration using swin trans- former. In ICCVW, 2021.
Chen, C. and Mo. IQA-PyTorch: Pytorch toolbox for im- age quality assessment. [Online], Available: https:// github com/ -hwooeeggcIIIO----- PyTorch, 2022.
Lim, B., Son, S., Kim, H., Nah, S., and Lee, M. Enhanced deep residual networks for single image super-resolution. In CVPRW, 2017.
Chen. Z., Zhang, Y., Gu, J., Zhang, Y., Kong, L., and Yuan, X. Cross aggregation transformer for image restoration. In NeurlPS, 2022.
Lin, X., He, J., Chen, Z., Lyu, Z., Dai, B., Yu, F., Qiao, Y., Ouyang, W., and Dong, C. Diffbir: Toward blind image restoration with generative diffusion prior. In ECCV, 2024.
Chen, Z., Zhang, Y., Gu, J., Kong, I.., Yang, X., and Yu, Dual aggregation transformer for image super-resolution. In CVPR, 2023.
Liu, Z., Shen, Z., Savvides, M., and Cheng, K.-T. Reactnet: Towards precise binary neural network with generalized activation functions. In ECCV, 2020.
Chen, Z., Qin, H., Guo, Y., Su, X., Yuan, X., Kong, L., and Zhang, Y. Binarized diffusion model for image super- resolution. In NeurlPS, 2024.
Lu, X., Yuan, Y., and Yan, P. Image upper-rsoluuton via double spansity regularized manifold learning. TCSVT, 2013.
Ding, K., Ma, K., Wang, S., and Simoncelli, E. P. Im- age quality assessment: Unifying structure and texture similarity. TPAMI, 2020.
Dong, C., Loy, C. C., He, K., and Tang, X. Learning deep convolutional network for image super-resolution. In ECCV, 2014.
Luo, X., Qu, Y., Xie, Y., Zhang, Y., Li, C., and Fu, Y. Lattice network for lightweight image restoration. TPAMI, 2022.
Qin, H., Gong, R., Liu, X., Shen, M., Wei, Z., Yu, F., and Song, J. Forward and backward information retention for accurate binary neural networks. In CVPR, 2020.
Dong, C., Loy, C. C., He, K., and Tang, X. Image super- resolution using deep convolutional networks. TPAMI, 2016.
Qin, H., Zhang, X., Gong, R., Ding, Y., Xu, Y., and Liu, X. instribution-sensitie information retention for accurate binary neural network. IJCV, 2022.
Gu, Lu, H., Zuo, W., and Dong, C. Blind super-resolution with iterative kernel correction. CVPR, 2019.
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S. Gans trained by a two time-scale update rule converge to a local nash equilibrium. NIPS, 2017.
Qin, H., Zhang, M., Ding, Y., Li, A., Cai, Z., Liu, Z., Yu, F., and Liu, X. Bibench: Benchmarking and analyzing network binarization. In ICML, 2023.

---

BiMaCoSR: Binary One-Step Diffusion Model Leveraging Flexible Matrix Compression for Real Super- Resolution
Rastegari, M., Ordonez, V., Redmon, J., and Farhadi, A. Xnor-net: Imagenet classification using binary convolu- tional neural networks. In ECCV, 2016.
Xia, B., Zhang, Y., Wang, Y., Tian, Y., Yang, W., Timofte, R., and Van Gool, L. Basic binary convolution unit for binarized image restoration network. In ICLR, 2022.
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with latent diffusion models. In CVPR, 2022.
Xu, Y., Chen, X., and Wang, Y. Bimlp: Compact binary architectures for vision multi-layer perceptrons. NeurlPS, 2022.
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z. Karpathy, A., Khosla, A., Bern- stein, M., et al. Imagenet large scale visual recognition challenge. WCV, 2015.
Yan, D., Wu, T., Liu, Y., and Gao, Y. An efficient sparse- dense matrix multiplication on a multicore system. In ICCT, 2017.
Yang, C.-Y., Ma, C., and Yang, M.-H. Single-image super- resolution: A benchmark. ECCV, 2014.
Saharia, C., Ho, J., Chan, W. Salimans, T., Fleet, D. J., and Norouzi, M. Image super-resolution via iterative refinement. IEEE TPAMI, 2022.
Yang, S., Wu, T., Shi, S., Lao, S., Gong. Y., Cao, M., Wang, J., and Yang, Y. Maniqa: Multi-dimension attention network for no-reference image quality assessment. In CVPR, 2022.
Sun, H., Li, W., Liu, J., Chen, H., Pei, R., Zou, X., Yan, Y., and Yang. Y. Coser: Bridging image and language for CVPR, 2024.
cognitive super-resolution.
Yang. T., Wu, R., Ren, P., Xie, X., and Zhang, L. Pixel- aware stable diffusion for realistic image super-resolution and personalized stylization. In ECCV, 2025.
Wang, J., Chan, K. C., and Loy, C. C. Exploring clip for assessing the look and feel of images. In AAAI, 2023a.
Wang, X., Xie, L., Dong, C., and Shan, Y. Real-esrgan: Training real-world blind super-resolution with pure syn- thetic data. In CVPR, 2021.
Yu, F. Gu, J., Li, Z., Hu, J., Kong, X., Wang, X., He, J., Qiao, Y., and Dong, C. Scaling up to excellence: Practic- ing model scaling for photo-realistic image restoration in the wild. In CVPR, 2024.
Wang, Y., Yang, W., Chen, X., Wang, Y., Guo, L., Chau, L.-P.. Liu, Z., Qiao, Y., Kot, A. C., and Wen, B. Sinsr: diffusion-based image super-resolution in single step. In CVPR, 2024.
Yue, Z.,, Wang. J., and Loy, C. Resshift:
Efficient diffu- sion model for image super-resolution by residual shifting. NeurlPS, 2024.
Wang, Z., Bovik, A. C., Sheikh, H. R., and Simoncelli, E. P. Image quality assessment: from error visibility to structural similarity. TIP, 2004.
Zhang, K., Liang, J., Van Gool, I.., and Timofte, R. Design- ing a practical degradation model for deep blind image super-resolution. In CVPR. 2021.
Zhang, L., Zhang, I.., and Bovik, A. C. A feature-enriched completely blind image quality evaluator. TIP, 2015.
Wang, 7.,, Cun, X., Bao, J., Zhou, W., Liu, J., and Li, H. Uformer: A general u-shaped transformer for image restoration. In CVPR, 2022.
Zhang, R., Isola, P., Efros, A. A.. Shechtman, E., and Wang, O. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR, 2018a
Wang, Z., Wang, J.. Liu, Z., and Qiu, Q. Binary latent diffusion. In CVPR, 2023b.
Zhang, Y., Li, K., Li, K., Wang, L.., Zhong, B., and Fu, Y. Image super-resolution using very deep residual channel attention networks. In ECCV, 2018b.
Wei, P., Xie, Z., Lu, H., Zhan, Z., Ye, Q., Zuo, W., and Lin, L. Component divide-and-conquer for real-world image super-resolution In 2020.
Zhang, Y., Tian, Y., Kong, Y., Zhong, B., and Fu, Y. Resid- ual dense network for image super-resolution. In CVPR, 2018c.
Wu, R., Sun, L., Ma, Z., and Zhang, L. One-step effective diffuuson network for real-world image super-resole repeutresolution. arXiv, 2024a.
Zheng, X., Liu, X., Bian, Y., Ma, X., Zhang, Y., Wang, J., Guo, J., and Qin, H. Bidm: Pushing the limit of quantization for diffusion models. In NeurlPS, 2024.
Wu, R., Yang, T., Sun, I.,, Zhang, Z., Li, S., and Zhang, L. Seesr: Towards somantics-awar real-world image super-resolution. In CVPR, 2024b.
Zibetti, M. V. W. and Mayer, J. A robust and computation- ally efficient simultancous super-resolution scheme for image sequences. TCSVT, 2007.
Wu, X.MM., Zheng, D., Liu, Z., and Zheng, W.-S Estima- tor meets equilibrium perspective: A rectified straight through estimator for binary neural networks training. In ICCV, 2023.
10


Reviews:
[{"review_id": "yIq0FacRvH", "reviewer": "Reviewer_XSYS", "summary": "This paper presents BiMaCoSR, a binary one-step diffusion model for efficient real-world image super-resolution (SR), which integrates 1-bit quantization and one-step distillation to address the high computational and memory costs of conventional diffusion models. To mitigate performance degradation caused by extreme compression, a dual-branch compensation mechanism is introduced: the Low-Rank Matrix Branch (LRMB), initialized via top-r singular value decomposition (SVD) to preserve low-frequency information, and the Sparse Matrix Branch (SMB), which maintains high-rank feature representations by absorbing extreme values in feature maps. \n\nThese branches synergize with the binarized main branch (BMB) to enable decoupled learning of high- and low-frequency features. The model employs pretrained weights for initialization, with LRMB initialized through SVD decomposition and SMB via sparse value selection. \n\nEvaluations on RealSR and other benchmarks demonstrate superior performance over existing binarization methods in PSNR, SSIM, and LPIPS metrics. Ablation studies confirm the effectiveness of the dual-branch architecture and initialization strategies. This work proves that combining matrix compression techniques with one-step distillation enables efficient deployment of diffusion models on resource-constrained devices while preserving visual quality.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> Q1-1: The claims about naive binarization and skip connection lack citations.\n\nA1-1: It is a common sense that naive binarization leads to model collapse and we provide experiments in table below. Moreover, we will add citations [1], [3], and [4] for naive binarization and [3] and [4] for skip connection in the revised version.\n\n|                          | LPIPS↓ | FID↓   | CLIP-IQA+↑ | Param |\n|--------------------------|--------|--------|------------|-------|\n| BMB (Naïve Binarization) | 0.4141 | 110.15 | 0.4325     | 3.69M |\n| BiMaCoSR                 | 0.3375 | 86.09  | 0.4800     | 4.98M |\n\n\n> Q1-2: The low-rank and high-rank of LRMB and SMB don't equate to low- and high-frequency information. \n\nA1-2: Yes. We clarify that the low-rank and high-rank of LRMB and SMB are not equivalent to low- and high-frequency information. Could you please specify the corresponding line number that you are referring to?\n\n> Q1-3: The article claims to first one-step binarized diffusion, but this claim is questionable.\n\nA1-3: In current research, one-step distillation is the only successful way to achieve one-step diffusion model. Simply changing the sampling methods often lead to model collapse [5,6]. Both one-step distillation and binarization are model compression techniques. The combination of these two techniques does not exist in current research. Therefore, to our best knowledge, BiMaCoSR is the first one-step binarized diffusion model.\n\n> Q1-4: The method in the paper seems to be a general binarization approach for diffusion models, yet the rationale for validating it on the Sr task is unclear. \n\nA1-4: We are working on the binarization of one-step diffusion model on SR task to address the needs of industry. Currently, the industrial companies are in urgent need to compress the SR diffusion models, after which can they deploy these excellent SR models on mobile devices to improve the imaging process. Whereas, general diffusion models have no such application scenario and they are usually deploied on cloud. In order to solve the current practical problem in the industry, we are focusing on the SR task, i.e., binarized one-step diffusion model for SR. Yet, we provide the result of BiMaCoSR compared with BiDM, ReActNet on general diffusion models (DDPM) in the following table. This result also supports our superior performance.\n\n| Methods  | FID     | param  |\n|----------|---------|--------|\n| FP       | 16.8274 | 35.72M |\n| BiDM     | 38.5275 | 4.73M  |\n| ReActNet | 76.8448 | 1.12M  |\n| BiMaCoSR | 37.1792 | 2.02M  |\n\n> Q1-5: No actual-device running time was designed for speed-related aspects.\n\nA1-5: Currently, the hardware support to binarized model is not well-developed. Therefore, we are not able to provide the actual-device running time. Despite, we report the speedup ratio following the previous research [1,4,6] and calculate the FLOPs needed for the inference process in the same way. Therefore, it is an engineering task to reach the calculated speedup ratio. As for research, we focus more on the balance of performance and speedup ratio.  \n\n> Q1-6: Table 3 of the ablation study lacks BMB + SMB results, even though SMB's sparse matrix might be low-rank.\n\nA1-6: Thank you for your advice. We provide the result of BMB+SMB below and We will add it in the revised version.\n\n| RealSR    | PSNR    | SSIM   | LPIPS  | CLIP-IQA+ |\n|-----------|---------|--------|--------|-----------|\n| BMB + SMB | 26.3037 | 0.7466 | 0.3901 | 0.4565    |\n\n> Q1-7: ... the biggest question is why binarization is tested on SR tasks with an unclear starting point. Testing it on pure generative tasks might yield more insights.\n\nA1-7: We explain the reason and result in A1-4. Please refer to A1-4 for detailed explanation.\n\n> Q1-8: Could this binarization-designed structure be applied to pure generative tasks?\n\nA1-8: Yes, we apply BiDM and our BiMaCoSR on DDPM on CIFAR-10 and the result is provided in A1-4.\n\n\n[1] Basic binary convolution unit for binarized image restoration network. In ICLR, 2022.\n\n[2] Bibench: Benchmarking and analyzing network binarization. In ICML, 2023.\n\n[3] OneBit: Towards Extremely Low-bit Large Language Models. In NeurIPS, 2024.\n\n[4] Binarized Diffusion Model for Image Super-Resolution. In NeurIPS, 2024.\n\n[5] SinSR: Diffusion-Based Image Super-Resolution in a Single Step. In CVPR, 2024.\n\n[6] BiDM: Pushing the Limit of Quantization for Diffusion Models. In NeurIPS, 2024.", "claims_and_evidence": "- The claim that \"However, applying naive binarization will lead to catastrophic model collapse\" is made without citing relevant sources. Moreover, the claim that adding skip connections can resolve this issue also lacks citation. This absence of supporting references undermines the solidity of the motivation presented.\n\n- The low-rank and high-rank of LRMB and SMB don't equate to low- and high-frequency information. For instance, the paper mentions using skip connections (like identity matrices) to access high-frequency information, yet edge detection operators can also convey such information.\n\n- The article claims to first one-step binarized diffusion, but this claim is questionable. Many binarized diffusion works exist in related research, and one-step implementation can be achieved through various sampling methods.", "methods_and_evaluation_criteria": "The method in the paper seems to be a general binarization approach for diffusion models, yet the rationale for validating it on the Sr task is unclear. While the evaluation for the SR task is reasonable.", "theoretical_claims": "There are no relevant theoretical proofs in the main text, but rank-related parts are in the supplementary materials. I checked the logic there and found no errors", "experimental_designs_or_analyses": "The experiment includes effect comparisons, key speed tests, and ablation studies on each module. However, it has flaws: no actual - device running time was designed for speed-related aspects; Table 3 of the ablation study lacks BMB + SMB results, even though SMB's sparse matrix might be low-rank.", "relation_to_broader_scientific_literature": "No relevant papers were found.", "essential_references_not_discussed": "No relevant papers were found.", "other_strengths_and_weaknesses": "Strengths:\n+ The paper's SR performance is excellent, surpassing that of the compared methods in many dimensions.\n\nWeaknesses:\n - The paper's performance is good, but the biggest question is why binarization is tested on SR tasks with an unclear starting point. Testing it on pure generative tasks might yield more insights", "other_comments_or_suggestions": "No", "questions_for_authors": "For binarized SR tasks, its performance is excellent, but I'm puzzled by some unmentioned methods. Inconsistent sampling steps aren't a valid reason for not comparing.\nCould this binarization - designed structure be applied to pure generative tasks?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "va5P2eDMZ3", "reviewer": "Reviewer_9pC4", "summary": "This work BiMaCoSR introduces the first binarized one-step diffusion model for real-world single image super-resolution (Real-SR)​. The paper addresses the heavy memory and computation demands of diffusion-based SR by combining 1-bit model binarization with one-step diffusion distillation. The core idea is to achieve extreme model compression and acceleration without sacrificing SR performance. To counteract the severe degradation (“catastrophic collapse”) that naive weight binarization would cause, the authors propose two lightweight auxiliary branches that preserve critical full-precision information: a Low-Rank Matrix Branch (LRMB) and a Sparse Matrix Branch (SMB)​​. LRMB (inspired by LoRA) captures low-frequency/large-magnitude components via a low-rank decomposition of each weight matrix, while SMB captures a few extreme outlier weight values in a sparse form. These branches, added to the main binary weight branch (BMB), allow the network to retain important information that 1-bit weights alone would lose, with negligible overhead. The overall architecture thus has three parallel components (BMB + LRMB + SMB) whose outputs sum to produce the layer’s result​. Using a distilled one-step diffusion model (SinSR) as a full-precision baseline, BiMaCoSR demonstrates state-of-the-art SR performance among heavily compressed models. It outperforms other binarization methods on standard real-world SR benchmarks (e.g. RealSR, DRealSR) across a comprehensive suite of 9 image quality metrics​. Notably, BiMaCoSR’s results are competitive with (and sometimes even better than) its full-precision one-step counterpart on certain fidelity and perceptual measures​. In terms of efficiency, the model achieves an impressive ~27× reduction in model size and ~23× faster inference compared to the full-precision one-step model​. In summary, the paper’s key contributions are: (1) the novel integration of binarization and one-step diffusion for SR, (2) the LRMB and SMB modules (with specialized SVD and sparse initialization schemes) to preserve information, and (3) extensive experiments showing dramatic memory/computation savings (23.8× smaller, 27.4× faster) while maintaining high restoration quality", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> Q2-1：Cite LoRA and KD\n\nA2-1: Thank you for your advice. We will cite LoRA and KD in the revised version.\n\n> Q2-2：One minor claim that could use more direct evidence is the suggestion that BiMaCoSR enables diffusion SR on resource-limited edge devices.\n\nA2-2: In Table 2 in the main paper, our BiMaCoSR takes 1.83 GFlops and 4.98 M parameters, which can be safely deployed and run efficiently on mobile devices, e.g., Snapdragon 8 Gen3 based devices. We will add this example in the revised version.\n\n> Q2-3：Clarify reported speedup vs compression ratios.\n\nA2-3: Currently, the hardware support to binarized model is not well-developed. Therefore, we are not able to provide the actual-device running time. Despite, we report the speedup ratio following the previous research [1,3,4] and calculate the FLOPs needed for the inference process in the same way. Therefore, it is an engineering task to reach the calculated speedup ratio. As for research, we focus more on the balance of performance and speedup ratio.\n\n> Q2-4：Include details on k selection for SMB.\n\nA2-4: We describe the k selection for SML in Sec. 3.4. Specifically, we select the top k values with the highest absolute values of $\\mathbf{W}_{\\text{BMB}}^{\\prime}$ to form the SMB branch. Each element is save with a triple, i.e., (row index, column index, value). During inference, there are efficient algorithms to calculated sparse matrix multiplication.\n\n> Q2-5：Actual Inference Speed\n\nA2-5: Please refer to A2-3.\n\n> Q2-6：Full-Precision Layers: You chose to keep the first and last conv layers in full precision. Did you attempt to binarize those as well, and if so, how badly did it hurt performance? In other words, is this partial precision absolutely required for acceptable results? \n\nA2-6: Experimentally, quantization of the first and the last conv layers hurts the performance significantly with negligible compression effect, as shown in the table below. This quantization scheme is also leveraged in many previous works[1,2,3,4].\n\n| RealSR          | LPIPS↓ | DISTS↓ | FID↓  | CLIP-IQA+↑ | Params  |\n|-----------------|--------|--------|-------|------------|---------|\n| SinSR (FP)      | 0.3635 | 0.2193 | 56.36 | 0.5736     | 118.59M |\n| BiMaCoSR        | 0.3375 | 0.2183 | 86.09 | 0.4800     | 4.98M   |\n| Fully Quantized | 0.3524 | 0.2423 | 91.17 | 0.4617     | 4.92M   |\n\n\n\n[1] Bin Xia, Yulun Zhang, Yitong Wang, Yapeng Tian, Wenming Yang, Radu Timofte, and Luc Van Gool. Basic binary convolution unit for binarized image restoration network. In ICLR, 2022.\n\n[2] Haotong Qin, Mingyuan Zhang, Yifu Ding, Aoyu Li, Zhongang Cai, Ziwei Liu, Fisher Yu, and Xianglong Liu. Bibench: Benchmarking and analyzing network binarization. In ICML, 2023.\n\n[3] Zheng Chen, Haotong Qin, Yong Guo, Xiongfei Su, Xin Yuan, Linghe Kong, and Yulun Zhang. Binarized Diffusion Model for Image Super-Resolution. In NeurIPS, 2024.\n\n[4] BiDM: Pushing the Limit of Quantization for Diffusion Models. In NeurIPS, 2024.", "claims_and_evidence": "The paper’s main claims are generally well-supported by empirical evidence. First, the authors claim that combining binarization with one-step distillation yields extreme compression and speedup with minimal loss in SR quality. This is convincingly demonstrated: BiMaCoSR’s model size and FLOPs are indeed drastically lower than baselines (e.g. ~27× smaller than the 32-bit model​), yet it achieves consistently higher or on-par performance on multiple benchmarks​. Table 1 shows BiMaCoSR outperforming competing binarized models on all evaluated metrics; for example, on the RealSR dataset it leads in PSNR, SSIM, and perceptual scores like LPIPS​. It even surpasses the full-precision SinSR and ResShift in some cases (e.g. LPIPS on RealSR)​. These results strongly back the claim of state-of-the-art performance at unprecedented compression levels. \n\nThe authors also claim that their auxiliary branches (LRMB and SMB) effectively prevent the performance collapse normally seen with 1-bit networks. This is supported by a breakdown ablation study: with only the binarized branch (BMB) the PSNR and SSIM are much lower, but adding LRMB and SMB progressively improves all metrics​. For instance, PSNR on RealSR jumps from 26.41dB with BMB-only to 26.95dB after adding LRMB, and perceptual scores improve significantly as well​. This demonstrates that the branches successfully recover information lost due to binarization. The claim that the branches incur negligible overhead is also justified. A theoretical calculation shows the extra storage/computation for LRMB (with rank r = 8) is tiny compared to the binary weights​, and the sparse branch uses only 0.1% of weight elements​​. In practice, adding both branches only increases the model’s parameter count by a very small fraction (from ~3.7M to ~5.0M, which is still over 20× smaller than the full model)​. This evidence supports the claim that the overhead is practically negligible. Most claims are thus well-substantiated, and I did not find instances of over-claiming. One minor claim that could use more direct evidence is the suggestion that BiMaCoSR enables diffusion SR on resource-limited edge devices. The paper makes a strong case via compression and FLOPs reduction, but it does not report actual on-device inference times or memory usage. While a 27× speedup in FLOPs is promising​, real hardware speedups might be lower without specialized binary execution libraries. Thus, the deployability claim is plausible but not explicitly validated with a deployment experiment. Aside from this, all key claims (first binarized one-step diffusion SR, SOTA performance, effective information retention via LRMB/SMB) are convincingly supported by quantitative results and ablation studies.", "methods_and_evaluation_criteria": "The methodology is well-aligned with the paper’s objectives. The goal was to drastically compress a diffusion-based SR model while preserving its high-fidelity output; to that end, the authors combined two complementary strategies: model binarization for compression and one-step distillation for fast inference. This joint approach directly targets both memory and speed objectives, and the method is executed thoughtfully. In particular, the introduction of LRMB and SMB is a clever design choice to meet the quality goal – these branches explicitly mitigate the known weaknesses of binarization (loss of information from small weights and rare large weights). The method leverages known techniques (low-rank approximation, sparse outlier capture, and XNOR-Net binarization) in a novel combination tailored for diffusion SR. The evaluation setup also matches the objectives: since the task is Real-World SR, the authors test on multiple Real-SR benchmarks (RealSR​, DRealSR, and a third dataset, likely a synthetic or DIV2K-based set) covering a variety of real degradations. They report a comprehensive set of 9 evaluation metrics – including standard fidelity metrics (PSNR, SSIM), perceptual distances (LPIPS, DISTS), and no-reference image quality scores (e.g. CLIP-IQA, MANIQA, NIQE/FID). This broad evaluation criterion is appropriate, as it captures both the fidelity and perceptual quality aspects of SR, aligning with the objective of producing realistic high-quality images.", "theoretical_claims": "The paper’s theoretical claims and derivations are mostly straightforward and correct. Rather than introducing new fundamental theory, the authors apply existing theoretical constructs (low-rank matrix factorization, binary convolution via XNOR) to their problem and provide derivations to justify design choices. For example, they express each full-precision weight matrix W as a sum of a low-rank component (matrices B and A from SVD) and a residual to be binarized. This decomposition is mathematically sound and allows them to claim a separation of “low-frequency” vs “high-frequency” information between LRMB and the binarized branch. While the terms “low-frequency” and “high-frequency” are used somewhat intuitively (referring to the magnitude/content of singular values rather than literal spatial frequency), the logic is reasonable: large singular values capture dominant image structures, and retaining them in FP (LRMB) should help reconstruct smooth components, whereas the binary residual can focus on fine textures. This claim is supported by the observed reduction in initial quantization error: after subtracting the SVD-based low-rank part, the norm of the remaining weight error ∥W_res∥²_F drops to 0.1855 from 1.1275 (a substantial reduction). This quantitative evidence backs the theoretical argument that their initialization decouples the weight information effectively.\n\nThey also provide complexity analyses to support the “negligible overhead” claim. The paper derives formulas for the storage cost of LRMB: O_s = (m*r + r*n)*B (with B=32 bits for FP) versus the binarized weight cost m*n*B' (B'=1 bit). Given r ≪ m,n, they show that even if stored in 32-bit, the LRMB adds only a tiny fraction of what the full matrix would, confirming the overhead is minimal. Similar reasoning is applied to the sparse branch: only k (top 0.1%) entries of each weight matrix are kept, so the extra cost is trivial. All these derivations are mathematically correct. There are no complex new proofs in the paper; rather, the authors ensure that each design element is backed by a clear explanation or formula. I did not find any algebraic errors or logical gaps in these derivations. The use of the straight-through estimator (STE) for binarization is referenced (e.g., ReSTE [1]) and the binary convolution operation is formulated via XNOR and bit-count equations – these are standard in binary neural networks and are presented correctly. One minor observation is that the paper doesn’t formally prove why the combination of branches yields optimal retention of information (that would be a very difficult theoretical guarantee). Instead, it relies on intuitive reasoning (e.g., outlier weights are rare but crucial​, so capturing them in SMB is beneficial) which is then validated experimentally. This approach is acceptable for an applied paper. In summary, the theoretical basis of the method is solid and internally consistent.\n\n[1] Wu, Xiao-Ming, et al. \"Estimator meets equilibrium perspective: A rectified straight through estimator for binary neural networks training.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.", "experimental_designs_or_analyses": "The experimental design is comprehensive and sound, giving credibility to the results. The authors conduct evaluations on three different SR datasets, covering both real-world degradations and (likely) a standard benchmark, which strengthens the generality of their claims. For each dataset, they report a wide range of metrics (nine in total), ensuring that no single metric bias (e.g., PSNR vs perceptual quality) dominates the assessment​. The comparison includes multiple baselines: (a) the full-precision diffusion models (ResShift and its one-step distilled version SinSR), and (b) several state-of-the-art binarization approaches adapted to the same one-step model (ReActNet, BBCU, ReSTE, BiDM).\n\nThe ablation studies further bolster the experimental rigor. The paper includes a breakdown ablation (BMB only vs +LRMB vs +LRMB+SMB), loss function ablation, different ranks for LRMB, and different initialization strategies for the branches. These experiments are well-designed to answer key questions about why the method works. For example, the breakdown ablation clearly shows each component’s effect on performance and verifies that the final design (with both branches) is needed for the best balance of quality and efficiency.", "relation_to_broader_scientific_literature": "The paper does an excellent job positioning itself in the context of prior work. In the introduction and related work, the authors survey two key areas: diffusion-based super-resolution and network binarization/acceleration. They cite the foundational and latest works in diffusion models for SR, such as SR3 (first iterative SR diffusion), DiffBIR, SinSR (one-step diffusion). On the binarization side, they reference seminal works like XNOR-Net for classification, as well as more recent binarization techniques and benchmarks (ReActNet, Qin et al. 2020/2022 for improved accuracy​, etc.). Crucially, they cite very recent papers that apply quantization to diffusion models: Binary Latent Diffusion (Wang et al. 2023), BiDM (Zheng et al. 2024), and a NeurIPS 2024 work “BI-DiffSR” (Chen et al. 2024)​.", "essential_references_not_discussed": "The paper cites most of the crucial prior work, but there are a couple of references that should have been mentioned explicitly:\n1. LoRA (Low-Rank Adaptation) – Hu et al., 2021. The idea of using a low-rank decomposition to inject or preserve information is directly inspired by LoRA (as acknowledged), but the original LoRA paper does not appear in the reference list. Citing it would credit the source of the low-rank approach and contextualize LRMB within the broader use of low-rank matrices in neural network compression.\n2. Knowledge Distillation for Diffusion Models – While the authors cite SinSR and ResShift for one-step distillation, they might have also referenced the general concept of knowledge distillation (Hinton et al., 2015) or earlier works on distilling iterative generative models. However, this is a minor point since they did cite the specific SR diffusion distillation methods they used.", "other_strengths_and_weaknesses": "Strengths: Beyond the points already discussed, the paper’s notable strengths include its originality and practical significance. Combining one-step diffusion with binary networks is a non-trivial and original idea – to my knowledge, this is indeed the first attempt at this combination, addressing a clear gap for deploying diffusion models. The resulting compression (≈24×) and speedup are very impressive, pushing the boundary of what’s possible for resource-constrained SR. Another strength is the thoroughness of validation: using nine different metrics and multiple datasets demonstrates robustness. The qualitative results (described in the text and shown in figures) also strengthen the paper – e.g., the authors describe how BiMaCoSR recovers fine details like hairs, facial features, and textures better than other compressed models​, emphasizing that it’s not just about numeric scores but also visible quality gains.\n\nWeaknesses: One weakness is that the method’s complexity might make reproduction challenging. The model introduces additional branches and special initialization routines (SVD for LRMB, “sparse skip” for SMB), which require careful implementation. However, the authors mitigated this by describing them in detail and promising to release code. Another minor weakness is that some improvements come at the cost of a slight drop in certain metrics. For instance, adding the sparse branch (SMB) improved perceptual scores but caused a small drop in PSNR/SSIM. This trade-off is actually expected (perception-distortion trade-off), and the authors do report it honestly. It’s not a serious issue, but readers focused purely on distortion metrics might note that the binarized model’s PSNR is a bit lower than a full-precision model in some cases. Additionally, as mentioned earlier, the reliance on a few full-precision layers means the model isn’t completely binary; however, the impact on compression is minor since those layers are a tiny fraction of parameters. A potential weakness in significance could be argued: the paper largely engineers known techniques (binarization, LoRA, distillation) together rather than introducing fundamentally new theory. I personally find the engineering contribution significant given the difficulty of making diffusion models this compact, but some might view it as an incremental combination (albeit a well-executed one).", "other_comments_or_suggestions": "1. Clarify reported speedup vs compression ratios.\n2. Cite LoRA for completeness.\n3. Include details on k selection for SMB.", "questions_for_authors": "1. Actual Inference Speed: The paper reports a ~23× reduction in FLOPs, but have you measured wall-clock inference time or FPS on any hardware? For example, on a CPU or GPU, how much faster is BiMaCoSR compared to SinSR in practice? This would clarify if the theoretical speedup carries over given that current libraries might not be fully optimized for binary operations.\n2. Full-Precision Layers: You chose to keep the first and last conv layers in full precision​. Did you attempt to binarize those as well, and if so, how badly did it hurt performance? In other words, is this partial precision absolutely required for acceptable results? Understanding this can help gauge if future improvements (or better training techniques) might remove the need for any full-precision layers.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3fWski1OTY", "reviewer": "Reviewer_ou23", "summary": "BiMaCoSR is a method that combines binarization and one-step distillation to significantly compress and accelerate super-resolution (SR) diffusion models. It prevents model collapse from binarization using two auxiliary branches: Sparse Matrix Branch (SMB) and Low Rank Matrix Branch (LRMB). SMB captures high-rank information, while LRMB outputs low-rank representations inspired by LoRA. BiMaCoSR achieves a 23.8x compression and a 27.4x speedup over full-precision models without compromising performance. Comprehensive experiments show its superiority over existing methods.\n\n## Update after rebuttal\nI have adjusted the score for the hardware acceleration. However, the justification for the novelty concern is unconvincing.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> Q3-1：The authors claim that BMB is responsible for most of the high-frequency information. However, as demonstrated in Fig. 2 of the supplementary material, LRBM appears to play a more significant role in contributing to the high-frequency information in the MLP.\n\nA3-1: This is because the high-frequency information generated by MLP is much less than that of attention mechanism, and the function of MLP is not to generate high-frequency information. Therefore, this phenomenon is not in conflict with the claim.\n\n> Q3-2：...conduct experiments with additional diffusion models to evaluate the method's generalizability.\n\nA3-2: Please refer to Q1-4 and A1-4.\n\n> Q3-3：...provide real-time speedup results.\n\nA3-3: Please refer to Q1-5 and A1-5.\n\n> Q3-4：It is quite surprising that in Table 2, the proposed method requires fewer FLOPs than ReSTE and XNOR.\n\nA3-4: We explained this in line 308-311. To guarantee fair comparison, we leave the **first and last** conv layers in full precision in BiMaCoSr and the **first two and last two** conv layers in full precision in other binarized methods. This operation keeps the # total parameters of BiMaCoSr and other binarized methods approximately the same.\n\n> Q3-5：Why one-step diffusion models?\n\nA3-5: Please refer to Q1-4 and A1-4.\n\n> Q3-6：compare the proposed method with the following works [1,2,3].\n\nA3-6: We provide the experiment result in the following table. [2] is not open-source yet therefore we can not compare considering the limited time for rebuttal. [1] is a PTQ method. If we train [1] with QAT, it is the same with [3]. BiMaCoSR consistently performs better than other methods.\n\n||LPIPS↓|DISTS↓|CLIP-IQA+↑|FID↓|\n|-|-|-|-|-|\n|[1,3]|0.3908|0.2572|0.4347|110.36|\n|BiMaCoSR |0.3375|0.2183|0.4800|86.09|\n\n> Q3-7：The author proposes SMB, but these techniques are widely used in low-bit quantization [5, 6] and binarization [1, 4], and should have been discussed in more detail.\n\nA3-7: We clarify that the motivation of SMB is different from [1,4,5,6]. In these papers, the authors leverage sparse matrix to keep the outliers of the weight matrix in their PTQ task. As for BiMaCoSR, the purpose of SMB is to pass the information without loss in QAT scenario. To validate this, we could also quantize the SMB branch into 1-bit and result is provided in the table below. The result shows that the function of SMB is not to maintain the outliers of the weight matrix. Therefore, our SMB branch is different with [1,4,5,6]. We will add these differences in the revised version.\n\n||LPIPS↓|DISTS↓|CLIP-IQA+↑|FID↓| Params |\n|-|-|-|-|-|-|\n|BMB+1-bit SMB|0.3935|0.2562|0.4541|110.30|4.98M|\n|BMB+SMB|0.3901|0.2558|0.4565|108.95|4.98M|\n\n> Q3-8：Quantization-aware LoRA fine-tuning methods [7, 8] appear to be similar to LRBM.\n\nA3-8: The first difference is that we keep the LRMB branch during the inference as an auxiliary branch to binarized branch, which could significantly improve the performance. The second difference is that the motivation of LRMB is to pass the low-frequency information in image SR task. As for [7,8], the motivation is to minimize the precision loss of weight. Therefore, we are quite different from these two works.\n\n> Q3-9：SVD initialization [7] and magnitude-based selection [5] are also common practices, but related works in this area have not been cited.\n\nA3-9: The difference between SMB and [5] is well discussed in A3-7. [7] is not published yet so we do not need to compare our method with it. We will cite [5, 7] in the revised version.\n\n> Q3-10：It is unconvincing that the method can achieve real speedup on hardware.\n\nQ3-10: Please refer to Q1-5 and A1-5.\n\n> Q3-11：...the proposed techniques ... lack novelty\n\nQ3-11: (1) For LRMB, we clarify that our motivation is different with current main stream. The detailed difference is shown in A3-8. (2) For SMB, the motivation of our method is to pass the information without loss. As for [5], the motivation of their method is to reduce the loss of weight. We explained this in detail in A3-7. (3) The main contribution of our method lies in the exploration of the combination of one-step diffusion model and binarization domain in SR task. We have provided a successful solution, which maintains the performance and accelerates the inference. (4) Reviewer 9pC4 recognizes our novelty in the combination of one-step diffusion model and binarization.\n\n\n\n[1] Billm: Pushing the limit of post-training quantization for llms.\n\n[2] Flexible residual binarization for image super-resolution.\n\n[3] Performance guaranteed network acceleration via high-order residual quantization.\n\n[4] ARB-LLM: Alternating refined binarizations for large language models.\n\n[5] Squeezellm: Dense-and-sparse quantization.\n\n[6] SpQR: A sparse-quantized representation for near-lossless llm weight compression.\n\n[7] LQ-LoRA: Low-rank plus quantized matrix decomposition for efficient language model finetuning.\n\n[8] QLoRA: Efficient finetuning of quantized llms.", "claims_and_evidence": "The authors claim that BMB is responsible for most of the high-frequency information. However, as demonstrated in Fig. 2 of the supplementary material, LRBM appears to play a more significant role in contributing to the high-frequency information in the MLP.", "methods_and_evaluation_criteria": "1. The author applies their method solely to the SinSR model. It would be beneficial to conduct experiments with additional diffusion models to evaluate the method's generalizability.\n\n2. The author does not provide real-time speedup results, which are crucial, particularly for ultra-low bit quantization. I am curious to know whether the proposed method leads to actual computational reductions.", "theoretical_claims": "No theoretical claims and proofs.", "experimental_designs_or_analyses": "1. It is quite surprising that in Table 2, the proposed method requires fewer FLOPs than ReSTE and XNOR, which do not include any additional computational branches.\n\n3. Why does the author focus exclusively on one-step diffusion models? I haven't found any specific design or rationale for choosing this type of model.\n\n4. It is essential to compare the proposed method with the following works [1, 2, 3] to validate its contribution, as all of them also employ additional computational branches (e.g., LoRA or sparse matrix), similar to the approach proposed here.\n\n\n[1] Huang W, Liu Y, Qin H, et al. Billm: Pushing the limit of post-training quantization for llms[J]. arXiv preprint arXiv:2402.04291, 2024.\n\n[2] Zhang Y, Qin H, Zhao Z, et al. Flexible residual binarization for image super-resolution[C]//Forty-first International Conference on Machine Learning. 2024.\n\n[3] Li Z, Ni B, Zhang W, et al. Performance guaranteed network acceleration via high-order residual quantization[C]//Proceedings of the IEEE international conference on computer vision. 2017: 2584-2592.", "relation_to_broader_scientific_literature": "I believe the proposed methods can be applied to any model architecture.", "essential_references_not_discussed": "Adding new computation branches is very common in the binarization area. All of the following related studies are missing:\n\n1. The author proposes SBM, but these techniques are widely used in low-bit quantization [5, 6] and binarization [1, 4], and should have been discussed in more detail.\n\n2. Quantization-aware LoRA fine-tuning methods [7, 8] appear to be similar to LRBM, assuming they do not merge LoRA during inference.\n\n3. SVD initialization [7] and magnitude-based selection [5] are also common practices, but related works in this area have not been cited.\n\n\n[4] Li Z, Yan X, Zhang T, et al. Arb-llm: Alternating refined binarizations for large language models[J]. arXiv preprint arXiv:2410.03129, 2024.\n\n[5] Kim S, Hooper C, Gholami A, et al. Squeezellm: Dense-and-sparse quantization[J]. arXiv preprint arXiv:2306.07629, 2023\n\n[6] Dettmers T, Svirschevski R, Egiazarian V, et al. Spqr: A sparse-quantized representation for near-lossless llm weight compression[J]. arXiv preprint arXiv:2306.03078, 2023.\n\n[7] Guo H, Greengard P, Xing E P, et al. Lq-lora: Low-rank plus quantized matrix decomposition for efficient language model finetuning[J]. arXiv preprint arXiv:2311.12023, 2023.\n\n[8] Dettmers T, Pagnoni A, Holtzman A, et al. Qlora: Efficient finetuning of quantized llms[J]. Advances in neural information processing systems, 2023, 36: 10088-10115.", "other_strengths_and_weaknesses": "Strengths:\n\n1. The paper is well-written and clearly presented.\n   \n2. The proposed method achieves state-of-the-art (SOTA) performance.\n\nWeaknesses:\n\n1. It is unconvincing that the method can achieve real speedup on hardware.\n\n2. Additionally, the proposed techniques—such as LRMB, SMB, and the initialization strategy—lack novelty (see Essential References Not Discussed).", "other_comments_or_suggestions": "N/A", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The authors propose a combination of three matrix branches to address the problem of a binary one-step diffusion model for real-world image super-resolution. The three matrix branches SMB, LRMB, and BMB all exist in the literature, but their combination to address the problem is novel. This paper demonstrates promising experimental validation and state-of-the-art performance. In their rebuttals, the authors clearly answer the questions raised by the reviewers and provide detailed explanations for their motivation and additional experimental results. Two of the three reviewers recommend weak accept while the remaining reviewer recommends weak reject. Considering their clear presentation, solid experimental validation, novel combination, and promising performance, I recommend accepting this paper.
Decision: Accept (poster)