Spectral Compressive Imaging via Unmixing-driven Subspace Diffusion Refinement
Authors: Haijin Zeng
Benteng Sun
Yongyong Chen
Jingyong Su
Yong Xu
Venue: ICLR 2025 Spotlight (2025)
Keywords: Spectral compressive imaging
subspace
diffusion
fine-tune
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Spectral Compressive Imaging (SCI) reconstruction is inherently ill-posed because a single observation admits multiple plausible reconstructions. Traditional deterministic methods struggle to effectively recover high-frequency details. Although diffusion models offer promising solutions to this challenge, their application is constrained by the limited training data and high computational demands associated with multispectral images (MSIs), making direct diffusion training impractical. To address these issues, we propose a novel Predict-and-unmixing-driven-Subspace-Refine framework (PSR-SCI). This framework begins with a light-weight predictor that produces an initial, rough estimate of the MSI. Subsequently, we introduce a unmixing-driven reversible spectral embedding module that decomposes the MSI into subspace images and spectral coefficients. This compact representation facilitates the adaptation of pre-trained RGB diffusion models and focuses refinement processes on high-frequency details, thereby enabling efficient diffusion generation with minimal MSI data. Additionally, we design a high-dimensional guidance mechanism enforcing SCI consistency during sampling. The refined subspace image is then reconstructed back into an MSI using the reversible embedding, yielding the final MSI with full spectral resolution. Experimental results on the standard KAIST and zero-shot datasets NTIRE, ICVL, and Harvard show that PSR-SCI enhances overall visual quality and delivers PSNR and SSIM results competitive with state-of-the-art diffusion, transformer, and deep-unfolding baselines. This framework provides a robust alternative to traditional deterministic SCI reconstruction methods. Code and models are available at [https://github.com/SMARK2022/PSR-SCI](https://github.com/SMARK2022/PSR-SCI).

Published as a conference paper at ICLR 2025
SPECTRAL COMPRESSIVE IMAGING VIA UNMIXING- DRIVEN SUBSPACE DIFFUSION REFINEMENT
Haijin Zeng's Benteng Sun? Yongyong Chen Jingyong Su² Yong Xu²
1 Harvard University, Harbin Institute of Technology (Shenzhen) haijin zeng201 2018 8 gmail. com, SMARK201 tookkko..c
ABSTRACT
Spectral Compressive Imaging (SCI) reconstruction is inherently ill-posed be- cause a single observation admits multiple plausible reconstructions. Traditional deterministic methods struggle effectively recover high-frequency details. Al- though diffusion models offer promising solutions to this challenge, their applica- tion is constrained by the limited training data and high computational demands associated with multispectral images (MSIs), making direct diffusion training impractical. To address these issues, we propose a novel rredict-and-unmiiin Siivven-Subspaee-BBee RetfbpaceeRRfiee framework (PSR-SCI) This framework begins with a fight-weihht predictor that produces an initial, rough estimate of the MSI. Subse- quently, introduce
comixxnng-drivee reversible spectral embedding module that decomposes the MSI into subspace images and spectral coefficients. This compact representation facilitates the adaptation of pre-trained RGB diffusion models and focuses refinement processes on high-frequency details, thereby en- abling efficient diffusion generation with minimal MSI data. Additionally, we design a high-dimensional guidance mechanism enforcing SCI consistency during sampling. The refined subspace image is then reconstructed back into an MSI using the reversible embedding, yielding the final MSI with full spectral resolu- tion. Experimental results on the standard KAIST and zero-shot datasets NTIRE, ICVI., and Harvard show that PSR-SCI enhances overall visual quality and delivers PSNR and SSIM results competitive with state-of-the-ar diffusion, transformer, and deep-unfolding basclines. This framework provides robust alternative to tra- ditional deterministic SCI reconstruction methods. Code and models are available at htttt::://// httpttttttt/utta/ wwt.hh......aaaeeeeeeeeeeeeeeeee httm//MARKK22220 PSR-
INTRODUCTION
Multispectral imaging extends beyond the visible light spectrum, capturing image data across diverse wavelength ranges, such as infrared and ultraviolet spectra. This method, aided by filters specialized instruments, reveals information beyond human perception, which is limited to red, green, and blue wavelengths. Consequently, multispectral images (MSIs) find applications in diverse fields such as remote sensing Yuan et al. (2017): Zeng et al. 2200)), medical imaging Lu & Fei (2014): Meng et al. (2020b), and environmental monitoring Thenkabail al. (2014).
Despite their utility, traditional multispectral imaging suffers from prolonged acquisition times due to spatial or temporal scanning, posing a significant hurdle for many computer vision applications Arad et al. (2022). Recent advancements in snapshot compressive imaging (SCI) systems have streamlined the acquisition of two-dimensional measurements of MSIs, facilitating efficient multispectral image acquisition and processing Cao et al.) (2016); Yuan et al. 2015); Ma et al. (2021). However, SCI reconstruction poses unique challenges compared to traditional denoising or reconstruction tasks, as it must recover MSIs from compressed measurements. This process also involves coping with severe degradation caused by physical modulation, spectral compression, and unpredictable system noise.
Reconstructing MSI with full spatial-spectral resolution from single measurement presents an inherently challenging and ill-posed inverse problem. Current methods face obstacles in accurately reconstructing specific aspects due to inadequate sampling in certain areas. Insufficient sampling
"Equal contribution. Correspnnning author.

---

Published as a conference paper at ICLR 2025
GT Patch PSNR/SSIM
Transformer 32.82 / 0,949
Deep Unfolding- 30.14 0.899
SCI measurement
Deep Unfolding-2 PnP--Difusion
Our diffusion 34.10/0.955
30.55 0.889
28.49 /0.821
Figure 1: State-of-the-art methods vs. our PSR-SCI for snapshot compressive imaging. Transformer:
CST+ Cai et al. 2022b), deep unfolding-Is 1: GAP-Net Meng al. (2023). deep unfolding-2 Ma et al.
(2019). PnP-Diffusion Pan et al. (2024).
hinders the accurate recovery of detailed information. Specifically, contemporary end-to-end (E2E)
models MMng et al. (2020a); Hu et al. (2022) are commonly trained using simulated measurement-
full -patial-ppectrl image pairs through supervised learning. The prevailing approach involves minimizing L, or 122 pixel loss, optimizing for the widely-used peak signal-to-noise ratio (PSNR) metric. However, PSNR and similar distortion metrics only partially align with human perception BBlau & Michaeli (2018); Delbracio et al. (202I). sometimes resulting in visibly lower image quality in reconstructed images. To address this limitation, recent works have introduced additional loss terms Mechrez et al. (2019) aimed at enhancing image quality under metrics that more reliably represent human perception. Training networks from compressed or corrupted images to known ground truth in a supervised manner falls under the umbrella of end-to- fo-end methods Ongie et al. (2020). While these methods perform well within their distribution, they may exhibit fragility to distributional shifts changes in the image degradation or imaging process Jalal et al. (2021).
(022 demonstrated notable proficiency generating content from RGB images Zhu et al. 2023). Leveraging its generative capacity to address challenging-tt--rconssttii multispectral SCI results Ho et al. 2020); /Song et al. (2020)); Choi al. (2021); Anderson (9822)
Diffusion model Nichol Dhariwal (2021) Choi (202T):
) Kawar
segments holds promise for enhancing
Chung et al. (2022). Nonetheless, two significant challenges must be confronted: (i) Due to the broader spectrum captured by MSI, there is limited training data available for MSIs compared RGB images. (ii) The high-dimensional nature of MSIs significantly increases the computational cost for diffusion denoising, especially when considering the number of sampling steps involved. Consequently, training a diffusion model directly on MSIs presents considerable challenge.
Diffusion models pre-trained large RGB datasets hold great potential for MSI reconstruction. However, several key challenges emerge when integrating diffusion models into the MSI domain: (1) Directly inputting MSIs, which comprise dozens of spectral bands, into existing diffusion models pre-trained on 3-channel RGB images is unfeasible due to the mismatch in channel numbers. (2) MSIs exhibit a significantly different wavelength spectrum compared to RGB images, and there exists complex spectral interrelation among the bands of MSIs. (3) Diffusion models require considerable sampling time, a challenge intensified in MSIs by the increased computational cost of denoiser networks multiplied by sampling steps. This paper addresses these issues with four contributions:
(i) Our approach introduces a spectral unmixing-driven SCI) for SCI reconstruction. This method yields improved perceptual quality than deterministic methods and more efficient enhancement than typical diffusion models.
aaddict-and-subspae refine strategy (PSR-
(ii) Given the ill- ill-posedness of spectral unmixing models, introduce reversible decomposition module. The module performs hierarchical low-rank decomposition, preserving reversibility and exploiting spectral sparsity for compression.
(iii) Rather than directly enhancing the MSI, we focus the diffusion generation exclusively on the high-frequency component. This approach accelerates fine-tuning and significantly reduces the amount of required training data, thus addressing MSI data scarcity.
(iv) We introduce high-dimensional guidance with SCI imaging consistency.
2

---

Published as a conference paper at ICLR 2025
We evaluated the PSR-SCI performance simulated and real datasets. As shown in Fig. PSR-SCI preserves finer details and attains a higher PSNR than current SOTAs.
2 RELATED WORKS
The existing frameworks for SCI reconstruction predominantly consists of model-based. Plug- and-Play, End-to-end (E2E), and Deep unfolding methods. Model-based methods Wagadarikar
et al. (2008g : Kittle et al. (2010); Liu al. (2019); Wang et al. (2016); Zhang et al. (2019); Yuan (2016) : Tan et ( Figueiredo et al. (2007) depend on hand-crafted image priors such as total variation, sparsity, and low-rank structures. Although these methods offer theoretical guarantees and interpretability, they require manual parameter tuning, which slows down the reconstruction process. Additionally, they are often limited by their representation capacity and generalization ability. Plug- Hlu-andd-layy (PnP) algorithm:Chan et al. (2016); Qiao et al. (2020): Yuan et al. (202)): Meng et al. (20TT) Zheng et al. (2021b): Yuan al. (2021b) incorporate pre-traine denoising networks into traditional model-based methods for multispectral imaging (MSI) reconstruction. However, because these pre-trained networks are fixed and not re-trained, their performance is limited by the fixed
denoiser capacity and mismatch to MSI statistics.
End-to-end (E2E) algorithms Meng et al. (2020b a): Hu et al. (002)); Miao et al. (019); Yuan
et al. ( leverage convolutional neural networks (CNNs) to establish a mapping function from mcasurements to MSIs. Despite the advantages of deep learning, these methods often neglect the fundamental principles of SCI systems and are deficient in theoretical foundations, interpretability.
and adaptability due to variations in imaging models. Deep unfolding methods WWan et al. (2020)
2019); [Mng et al. (2023); Ma et al. (2019); Huann et al. (2021): Fuu et al. (2021); : /Zhang et al. (2022).
on the other hand, utilize multi-stage networks to transform measurements into MSI cubes, providing interpretability through explicit characterization of image priors and system imaging models.
In addition to the four classic frameworks mentioned above, the advancement of generative models Lin et al.] (2023); Miao et al. (2023): [H al. (2020); WWang al. (2022); Whhan al., (2022) has led to the emergence of two additional works. These works primarily aim to enhance the accuracy of SCI reconstruction by leveraging the potential of denoising diffusion models. Specifically, a model named Dif[SCI Pan et al. (2024) utilizes pre-trained denoising diffusion model for RGB images as the denoiser within the PnP framework. This approach combines structural insights from deep priors and optimization- diffusion models. Another work to use latent diffusion model to generate clean image priors for deep unfolding network, to facilitate high-quality hyperspectral reconstruction Wu et al. (2023).
methodologies with the generative capabilities of contemporary denoising
OUR PSR-SCI METHOD
3.1 PROBLEM DEFINITION AND CHALLENGES
Degradation Model of CASSI: A type of snapshot compressive imaging system is the Coded Aperture Snapshot Spectral Compressive Imaging (CASSI) system Wagadarikar et al. (2008); Meng et al.) (2020a): Gehm al. 2007) shown Fig. 2 this system, two-dimensional measurements y E IK Hx (dx(B-1) b> are modulated from a dimeessonai MSI X' "xxx where H. W. d, and B denote the MSI's height, width, shifting step, and total number of wavelengths, respectively.
To formulate the imaging process, we firstly denote the vectorized measurement as E R" with n = H(W +d( Cai eet al.] (2022d); Ma et al. (2019). vector- ized shifted MSI as x E R mask as + e R Then, the imaging process can be formulated as:
modeaaaton
dlapension
Ireaging
Figure 2: Illustration of a single disperser CASSI system.
Tx n,
where R denotes the imaging noise generated by the detector. Subsequently, is necessary to decode the measurement y to obtain x with full spatial-spectral resolution, given & Tropp & Gilbert 2200)):Donoke (2006): Dalali & Yuan (2019).
Denoising Diffusion Models for SCI? In addressing the inherently ill poseed nature of SCI recon- struction. existing approaches face various challenges in achieving accurate detail reconstruction simultancously. One promising solution to this predicament lies in the denoising diffusion model, renowned for its generative capability. Nevertheless, (i) the existing ceffusiio-bbsed methods
3

---

Published as a conference paper at ICLR 2025
(a) Initial reconsrruction & frequency separation
(b) Unmixing-drrin driven reversible spectral embedding (URSe)
Tt
22 init
Qy
Frequency Separator
Initial
Predictor
Speetnn Manaan
X Feaid
Measurement
Nirait
Spectra Embaddiiigg
High medrmma knai
High-dimersiinn gu dinne Iragingg cansiaan
400
300
bt Use wise-uinn Urovn
In PrapPe DANYE
Interpolation
convi
LonstX1
com
Towess
Afff
Fouure
Avg JOOK
Cook
A
(c) Detail reconstruction
difs
Divesioneaing
Annit
Figure 3: The overall framework of our PSR-SCI consists of three distinct yet interrelated modules, including (a) the initial predictor with frequency separator, and (b) the spectral hierarchical spectral embedding, serving as latent space decomposition method with physical significance in the context of SCI. Additionally, we (c) fine-tune the diffusion generation of high- frequency subspace images atop large-scale RGB images pre-trained models.
driven
mostly designed for RGB images in which the input and output with three channels, while the task of SCI reconstruction involves decoding a complete multi-band MSI from a single-band mea- surement. (ii) Meanwhile, limited by the inadequate datasets of MSI and the high dimension of data, the resource consumption required for retraining a powerful diffusion model from scratch on MSIs challenge. (iii) Furthermore, although many recent works have explored alternative sampling strategies that reduce the number of sampling steps Song et al.] (202T); San-Roman et al. (202T); Kong & Ping (2021;; Lee et al. (202T) for low-dimensional RGB images, the iterative diffusion process for high-dimensional MSIs with multi bands is still time-intensive.
3.2
DIFFUSION FRAMEWORK
In this section, given a measurement y H (XxxW4xx(BBI) we introduce a method for gener- ating refined approximation of full spatial-spectral resolution MSI, denoted as & "xx W through a predict-and-eee and-subspace refine framework with diffu- sion generation adjustment. The overall diagram of our PSR-SCI method is shown in Fig. Initially, we obtain a cost-effective initial estimate via a cheap predictor dy: Xiii de ()'). Then, we separate the
529 Stu
X
Referene
Inputt
2
Rederence
Figure 4: Illustration of initial low-frequency pre- diction and final high-frequency component gener- ated from diffusion, where Xly = tho" (Aligt E)) frequency components via a frequency separator Ty as depicted in Fig. (a): init, X init) = To (o preserving the PSNR-critical low-frequency structures intact, while leaving the sparse, detail-rice high-frequency texture regions to the diffusion model.
Subsequently, as shown in Fig. (c), to facilitate fast diffusion process while making full use of diffusion models pre-trained by large-scale RGB data, we decompose 2/ into low-dimensional abundance map A and spectral coefficient E using reversible spectral embedding module dh
(A E) = vhe( (e(
where the inverse of vb denoted as 1 satisfies that w (Ath within the predictor and module. Subsequently, a fine-tuned diffusion model operates on this low- dimensional abundance map: Aug diff(AAm))
mitr E) ~ X 0 denotes the weight
To ensure the diffusion sampling process aligns with the provided measurement V. we modify the diffusion model to enhance the high-frequency component of A: A diff (A This modification allows the fine-tuned RGB pretrained diffusion model to focus solely on modeling the residuals, thereby minimizing deviations from the measurement. Finally, get the reconstructed MSI by reversing the spectral embedding vp:
A 15 (Allior F.) X/hrr Alio difT((Afin))
4

---

Published as a conference paper at ICLR 2025
The initial predictor, which runs only once, effectively reduces the computational burden the subsequent diffusion model by offloading the majority of the processing tasks to itself. Our predict- and subspace refine method not only reduces the number of images required for fine-tuning the denoising diffusion process but also enables MSI generation capability through pre-trained diffusion models. Fine-tuning the RGB pre-trained denoising diffusion model with added parallel UNet encoder layers in the subspace allows for efficient diffusion sampling on nigh-dimensional MSI. Without this subspace sampling approach, the computational budget for iterative denoising of high-dimensional MSI increases significantly, as any rise in computational cost due to dimensionality amplifies with the number of sampling steps used.
3.3 UNMIXING-DRIVEN REVERSIBLE SPECTRAL EMBEDDING
The spectral unmixing theory posits that an MSI can be decomposed into an abundance map and spectral endmembers. It is inherently an ill-posed problem with numerous potential solutions. Abundance fractions denote the relative proportions of distinct pure materials, known as endmembers, present within a mixed pixel Keshava & Mustard (2002).
PSNR: 47,39IB, SSIM: 0.9928
To expedite the diffusion process and leverage pre-trained RGB denoising diffusion models efficiently, we propose decomposing the underlying MSI into reduced low- dimensional image A and spectral coefficients while ensuring an approximately reversible decomposition pro- cess. To achieve this, we introduce a Unmixing-drinn driven reversible spectral embedding module (URSe). Utilizing a hierarchical spectral subspace learning strategy, as illus- trated Fig. 3 (b), URSe ensures that the compression and reconstruction gap within each stage is minimized, The backbone of URSe comprises simple Conv N N lay- ers, focusing on compressing and decompressing spectral information. The upsampling operator utilized in URSe is "Bilinear interpolation Conv" instead of the widely used transposed convolution to reduce the checkerboard artifacts as shown in Fig. 5
X
v
(A.
V
Time: 0.00075s
Time: 0.00016s
interpolation on
Transposed C
Figure 5: Illustration of the proposed spec- tral embedding (top), the PSNR and SSIM are the averaged results of 10 scenes of the KAIST dataset, and comparison of up- sampling within URSe (hottom).
Additionally, to mitigate information loss during the re- verse process of spectral embedding, we introduce spectral attention module to generate spectral coefficient E from the embedding process. This spectral coefficient is reused during reversal to enhance reconstruction fidelity shown in Eq. equation 3 As depicted in Fig. 5 URSe trained with the CAVE dataset achieves fast spectral embedding (0.00073s) and accurate inverse reconstruc- tion (0.00016s), yielding a PSNR of 47.39dB and SSIM of 0.9928. Notably, due to its minimal parameter count, URSe can achieve effective training and decomposition even on a single image, as demonstrated in Fig. (a)(b).
3.4 UNMIXING-DRIVEN MSI DIFFUSION REFINEMENT
The proposed unmixing- mumixing-driven driven reversible spectral embedding module enables the transformation of a high-dimensional MSI into a reduced low-dimensional subspace image, with a promising inverse mapping for reversal. This facilitates the utilization of diffusion models pre-trained on large-scale RGB datasets to address MSI data absence issues, while also enabling fast diffusion process to alleviate computational budget constraints for MSI.
On the basis of Sec. 3.3 this section outlines a methodology for producing accurate subspace approximations (AA This is achieved by fine-tuning the stable diffusion model Rombach et al. (2022) pre-trained on large-scale RGB datasets, augmented with a tailored high-dimensional MSI control mechanism, atop the IRControlNet architecture Lin al. (202)) as shown in Fig. (c). As stable diffusion, all the diffusion processes of our method are performed in latent space, where an autoencoder Kingma & Welling (203)) is used to convert image into latent with encoder & and reconstructs it with decoder D.
Basic Diffusion Process. The forward process is Markov chain, where Gaussian noise with variance & E (0, 1) at time is progressively added to the latent 2 = E(x) to produce the noisy latent:
21 = Votz + CieC,
5

---

Published as a conference paper at ICLR 2025
Algorithm 1 Predict-and-subspaeeee- diffusion sampling.
Require: D: denoiser network, de: initial predictor, tha spectral subspace learning module,
Y: SCI measurement, gradient scale 8, 01:7: noise schedule, T: frequency separator.
1: Unit x ()()
Initial prediction Frequency separating
2: Aiir Kini f- Tá (Xiir)
3: Amir t- (iir
Spectral subspace embedding
4: 2T NO, Ia)
Run diffusion sampling
5: for = T, T....,1ar do
6: 6 Ia)
V1 NHEe( 21, (1 E,
7: 20 vāl
LLow-dimensionll subspace diffusion step
vāa
8: L(zn, Kimit)
1
|1((:b (D(2)), E) + Xmmit) - Xmit) + + - (v (D(2)),
( ( E) + X I1
N
9: Sample 2,-1 from q( |2, 20 - sV z. C(o,, Xinit) via Eq. equation 20
10: return Aligs D( 2n)
VAE's decoder
11: end for
12: return X = 2- 5 (Alig E) + X
> Return MSI via reversed spectral embedding
where e ~ (N, o = - 3e and āg = II as. Subsequently, for denoising step, we train a UNet denoiser Eg to predict the noise with randomly sampled t, by optimizing following loss:
C Km..,,, II|e eØ(vā + v1 vre Miirs & Kmmi t, (A
(5)
In addition, make full use of diffusion model pre-trained large-scale RGB datasets for our MSI task, we adopt Stable Diffusion 2. b as our pre-trained model, and fine-tune it with multispectral dataset CAVE Park et al. (007). In addition, as illustrated Fig. (c), we incorporate a parallel encoder alongside the original encoder of UNet, as described by [Li et al. (2023). This modification enables the diffusion model to include tuneable parameters that are specifically adapted to the small- scale MSI data. Simultaneously, it retains the foundational generative capabilities conferred by pre-traine pre-training on extensive RGB datasets.
Diffusion with high-dimensional guidance and imaging consistency. The basic diffusion gen- cration process operates within a subspace, while the final reconstruction of SCI occurs in the high-dimensional MSI space. Consequently, even if the diffusion models produce high-quality image within the subspace, it does not necessarily ensure a satisfactory final reconstruction in the MSI space. To address this issue, propose the integration of high-dimensional guidance mechanism into the conventional sampling process. This approach aims to enhance the alignment between the subspace biffusion--eneraate image and the ultimate high-dimensional MSI reconstruction. Specifi- cally, we conduct the basic diffusion process in latent space but enhance it with guidance from the original high- gigh-dimensional MSI space using our reversible spectral embedding & and its inverse di with the initial prediction Xmi and y' as reference. At time I, the denoiser first predicts the noise é of the noisy latent ž₁. Then the predicted noise 6 is removed from 2t to get the clean latent Ep:
2t VI EfC
c = co(zr. Xmir: Aliis P. E(AA( En
(6)
Var
The reverse process is updated as follows:
1
1 cal Zt. Xusi Alnor 4 E, (,(AAhi))
VI
Var
2020b): Rui et al. 2023), we formulate the ancestral sampling process as the discretization of reverse Stochastic Difterential Equations (SDE).
where 2, ~ ((0,1),t [T]. As Song et al.
Together with condition Ximi and the spectral coefficient E as conditioning variables, we reformulate the reverse SDE concerning z
dz = (2 f) - - VV log Kaiii v. E)] dt + g(f)dw
(8)

---

Published as a conference paper at ICLR 2025
Table I: Numerical evaluations between our PSR-SCI and SOTAs across 10 simulated scenes are presented. The table includes PSNR values (upper entry) and SSIM scores (lower entry) for each method. The best and second-best outcomes are emphasized in bold and underlined, respectively.
Alporilins
Caogiey
56
Desci
2019
Model
TRAMI
0.855
26.38
25,10
0,701
25.11
0.730
29.26
25,41
0.778
24.00
0.725
24.96
0.244
24.15
23.56
25.27
A-Ne Miao
2019
CNN
IOCY 2019
0.59
50.10
26.49
0.570
27.75
57.01
0.317
26.19
26.64
36.47
0,831
26.09
27.50
25.13
78.55
T5A-Ne Meng dl.
2020az
CNN
BCCV 2020
0.804
30.31
0.363
31.01
0.915
12.15
0.958
$7.05
0.564
20.47
0.902
11.05
0.850
30.02
0.586
20.32
0.909
31:14
20.15
0.561
0.805
31.35
DIP.H5 et
2021
PaP
ICCV 2021
31.32
0.855
25.89
0,009
29.91
0.539
0.526
38,69
0.796
27.45
0.524
29.53
0.700
27.46
0,802
27.69
33.45
0,863
0,733
26.10
29.75
0.903
BisKNe et al.
2024
INN
NeuriPS 2023
0.847
50.95
0.791
29.21
0.828
79.11
0.908
15.01
25 19
0.827
0.563
90.23
0.800
27.85
0.843
25.82
0.832
10.46
27.84
0.500
0.817
79.76
Hu al.
2022
Tranafurmar
CVPR 2022
54.96
0.37
35.64
0.943
0,945
35.55
0.976
41.64
32.56
0.948
34.33
0.954
0.928
33.27
32.36
0.945
0.944
34,17
0.940
32.22
0.946
34.65
MST al.
3022a
Transfomer
CVPR 2002
0.44
0.948
35.13
0.954
15.65
0.976
40.05
$1.84
0.949
0.955
3.556
0.910
19.80
0.950
53.34
0.44
34.53
0.941
31.61
34.11
2023c
Transfumaae
CVPR 2022
35.57
0.45
0.949
37.00
0.919
0.500
43.56
33.27
0.95
0:903
35.27
0.916
0.956
33.50
0.956
0.949
35.72
CST
D0EZ0
Transfome:
BCCV
33.64
0.951
37.71
0.965
0.981
41.38
0.957
32.95
0.965
35.55
0.947
34.07
0.964
ADMM NetMa
Deep Untokiing
ICCV 2019
0919
16.02
0.913
31.83
32.47
30.09
0992
DGSMP Hung
2021
Dasp Lisfokling
CVPR 2021
0.155
33.26
0.925
33.05
40.54
25.86
33.05
31,55
Ne MedMeng
Unfolling
UCV 2023
0915
33.63
33.19
0.951
33.96
39.14
0921
31.44
32.29
30:25
0.907
33.05
32,89
DAUNST- Caier
20206
Deep Lnftokiing
NeurIPS 2002
36.59
0.940
37.00
0.958
0.964
39.32
44.77
0.950
0.961
34,82
0,963
36.19
0,950
36.02
34.28
0.956
0.963
36.54
0.947
33.67
0.959
37.21
DAURST SP2 He at.l.
2024
Subipace peior Information Fusion 2024
36,73
0.956
37.76
0.963
0.970
39.57
0.958
46.31
35.00
0.966
0.969
26.15
0.960
36,65
0.966
$4.59
0.969
79.15
0.958
34.23
0.966
37.61
1XH Pan al
1116
Daflasson
CNPR 2004
0,907
34.96
34.00
0,901
0,949
29.33
0.951
42.05
35.21
0.966
0.917
33.12
0,146
36,29
03887
30.42
0931
37.27
28,09
0,521
0.916
35.25
PSR-SCI-T
Daflasion
Our
56.33
0.53
0.964
38.57
38.09
0,965
0.979
42.55
0.964
35:43
0.963
35.50
0.954
36.29
34.26
0.950
0.962
36.57
0.945
33.51
0.961
36.68
PSR-SCI-D
Exclasoon
Ouss
37.18
0.963
38.74
0,968
41.04
0.976
46.31
0.988
0.971
35.81
0.972
36.76
37. 17.38
0,965
33.55
0.955
0.972
39.09
0.950
24.10
0.967
38.14
1 where f(z, t) - ax) and 9: v 0t W is the reverse of the standard Wiener process.
The gradient =,, log (z.) is commonly referred to the score function of z,.
Then, we discretize the reverse SDE (8) using the form of ancestral sampling process
1
2t-1
(Z1
+ (1 as) V., (z Kiiir E))
(9)
Vax
1 1- at _ + v1 a: - -V 21 - (V)²()())), E) + X
Zt. V1 - 04
Va
+ y (1 (20), E) + x 1
(10)
where is gradient scale, 20- = 2t-1, 1. At time I, the sampling process can divided into two distinct components. The first component involves sampling from the parameterized distribution -(= with fixed variance of /I- at The second component adjusts the sample to maintain consistency with the initial MSI prediction constraints. Please refer to the supplementary material for detailed
explanation of the process from Eq. to Eq. 9 Based on (T). (9). and the basic framework described in Sec. 3.2
we summarize the pseudocode for the modified sampling procedure in Algorithm 1
4 EXPERIMENTS
Experiment Setup. We employ a pre-trained fast transformer model Cai et al. 2022a) and a 3-stage deep unfolding model Cai et al. (2022d) as initial predictors de for our PSR SCI-T and PSR-SCI-D, respectively. The frequency separator To is based on Gaussian filter (see supplementary materials for details). Due to the high dimensionality of spectral data and considerations for training performance, we train the URSe and VAE models individually. We initially train the URSe model on the GT and high-frequency portion of the simulation dataset, with a spatial size of 256 x 256 and 28 bands from the CAVE dataset Park et al. (2007). Subsequently, we freeze the URSe and fine-tune the VAE model. For the Diffusion model, we use the well-trained Stable Diffusion -.seb and fine-tune the ControlNet model to shift the diffusion model's 's focus from the entire image to the high-frequee
texture regions using CAVE. Similar to most existing methods Meng et al. (220a)): Hu et al. (2022); Cai et al. (2022d), we select 10 scenes with a spatial size of x56 and 28
Huang et al. (202T);
7

---

Published as a conference paper at ICLR 2025
GT Patch
ADMM-Net
CST-L+
DGSMP
DIP-HSI
GAP-Net
HDNet
Scene
A-Net
MST-L
MST++
PaP-CASSI
TSA-Nct
DiffSCI
PSK-SCI-T
GT Patch
ADMM-Nat
DGSMP
DIP-HSI
GAP-Net
HDNet
DiffSCI Figure 6: Visual comparison on the KAIST dataset. Top Scene wavelength 487. 5n.. Bottom
Scene
N-Net
MSTL
MST++
PoP-CASSI
TSA-Net
PSR-SCI-T
is Scene 2 at wavelength 575. 5nm. Additional KAIST results are shown in the supplemental material.
bands from KAISTChor al. (017)) as the simulation dataset for testing. Meanwhile, we also select 5 MSIs with spatial size of 660 660 and 28 bands, captured by the CASSI system as the real dataset[Meng et al. 2020a), and then crop the MSIs into data blocks size 256x256 for testing. To evaluate generalization performance of approach, test it several zero-shot MSI datasets, including ICVL, NTIRE, and Harvard, which were not used during training.
4.1 EVALUATION METRICS
We assessed our method using quantitative met- rics: Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM). For qualita- tive evaluations, we analyzed local patches from both SOTA methods and our PSR-SCI method against ground truth in simulated experiments. We also compared spectral density curves of reconstructed MSIs with ground truth and cal- culated their correlation coefficients. experi- ments with real data where MSI ground truth is absent, RGB images of the same scene served as a general reference, approximating the overall shape and details of scene objects. For zero-shot datasets, supplemented PSNR with the MANIQA metric thoroughly assess image fidelity and visual quality, ensuring comprehensive evaluation of our model on unseen data.
Graud Trun
Diand Truit
- oan
MST4
MSt
sun
Mrties oner
bottee 5017
P5R50 can E3773
PSP1-00 zonn 2.333
Wascclengh (nm)
Wovelength (nm)
Figure 7: Spectral Density Curves.
4.2 QUANTITATIVE RESULTS
Table. [ and 2 show quantitative results KAIST, ICVL, NTIRE and Harvard dataset. We com- pared model with the current SOTA methods: DESCI [iu et al. (2019). X-Net MMao al. (2019). TSA-NET Meng et al. (2200).
Table Comparison of PSNR, SSIM, and MANIQA metrics across several sero-shot datasets.
Mark:
HUTA.
LABE-1b9
750-510
(58.63
34.03
15:55
15.25
32335
11.35
11.34
58077
0.258
0.203
0.213
01300
8.17
6226
6.13
3114
18:35
15.11
13.35
32.79
39.80
1662
3531
MA000
11214
12210
0128
01330
1120
1130
$235
0:355
6341
Ranoel
5450
0623
0.51
0650
0.51
0773
0400
877
875
0131
DGSMP Huang al. (2021).
GAP-NET Meng al. (2023), ADMM-NeT et (2019). PnP.
CASSI Zheng et al. (2021a) lap. DIP-MSINNA Meng et al. (2021). HDNET Hu et al. (2022). MST-L Cai
al. (022a)), MST++ ++ Cai et al. (202c)), CSTL- Cai et al. (2022b). DiftSCI Pan et al. (2024).
DPU Zhang et al. (2024a). SSR Zhang et al. (024b) and LADE Wu et al. (025). On KAIST, our model achieves SOTA performance across all metrics and consistently achieves the highest PSNR or SSIM scores across all 10 scenes. Specifically, we achieve average PSNR of 36.68 66.68dB, representing an improvement of nearly 1.4dB compared to the latest SOTA method, DiffSCI, which is the current method outperforms all transformer based
leading diffusion-brat based method in SCI. Furthermore, methods in terms of PSNR across scenes except S4. These results underscore the flexibility of our framework in balancing fidelity and detail generation using generative denoising diffusion model.
8

---

Published as a conference paper at ICLR 2025
4.3 QUALITATIVE EXPERIMENTS
Results on Simulation Dataset. The detailed comparisons of local patches are presented in Fig. showcasing two scenes: the 8th band of Scene (top) and the 21 sf band of Scene 2 (bottom). Upon comparison with the ground truth, it is evident that our PSR-SCI method yields superior visual effects, featuring cleaner textures and fewer artifacts compared to other SOTA methods. For instance, in Scene 1, notable improvements nose, and mouth. In Scene 2, a challenging scenario with dark areas, only DifISCI and our method successfully reconstruct the complete structure of the cube. However, edges of the blocks, resulting in shapes and patterns closer to the ground truth. Furthermore, Fig. displays density- waaeleeggtt spectral curves, indicating that the spectral accuracy of our model, as evidenced by the high correlation with reference curves, surpasses that of competing methods.
observed in the details of facial features such the cyebrow,
PSR-SCI further refines the
Results Real Dataset. In addition, we evaluate the reconstruction performance of PSR-SCI a real dataset and compare the results with the corresponding RGB image captured from the same scene, as shown Fig. From the star depicted the figure, evident that PSR-SCI recovers a more complete and detailed shape with fewer artifacts compared to other SOTA methods. While other methods either produce blurred shape or fail to reconstruct a reasonable surface for the star,
RGB Ref.
Meas
X-Net
CST-L
DGSMP
GAP-Net
HDNet
MST-L
TSA-Net
DPU-9stg
SSR-L.
LADE-9stg
DifISCI
PSR-SCI-T
Figure Visual comparison on Scene of real dataset at wavelength 648nm
TOPEYEC
OPP-
SUMMSS 83333
15.00 83145
2LAI B251
STAID CIED
1723
SA CAM
TUSSS DEN
FRAND 033R
JLSII NOTLVE
D192
PSR-3D-B
PIURA 5272
SXSTO 0231
SIJIM LIRE
LSS7
Figure Comparison methods and generalization performance testing on additional datasets (Pscudo-RGBB
Results on Additional Datasets. To evaluate the generalization capability of our PSR-SCI model, we conducted snapshot reconstruction tasks on several zero-shot MSI spectral datasets, including ICVL, NTIRE, and Harvard. The corresponding bands were mapped, and we compared the performance against DAUHST-3stt and MST-L. As shown in Fig. 9 the PSNR (left) and MANIQA (right) metrics are presented for each method, along with pseudo-Re RGB visualizations for qualitative comparison.
In addition to PSNR, we used the MANIQA metric to assess perceptual quality, providing more comprehensive evaluation both image fidelity and visual quality. Our PSR-SCI model consistently outperformed the competing methods across various datasets in both metrics, summarized in Table 2 These results highlight the strong priors embedded in our diffusion baased pipeline, which enable superior zero-shot reconstruction.

---

Published as a conference paper at ICLR 2025
These results highlight the robustness of our diffusion- model, which leverages rich image priors to achieve superior zero-shot generation and reconstruction in MSI datasets. By consistently outperforming other methods in both traditional metrics like PSNR and perceptual quality metries, our model demonstrates its ability to deliver enhanced reconstruction fidelity and visual accuracy across diverse zero-shot scenarios.
4.4 ABLATION STUDY
Table 3: Ablation study results showing the impact of different components in PSR-SCI.
Break-down A Ablation. We performed an ablation study to evaluate the impact of each component in the PSR-SCI frame- work. As shown in T removing the diffusion model from our framework and using only the initial predictor results in a PSNR of 37.21 dB. Using the pre-trained diffusion model, trained on a large RGB dataset, alone results in a PSNR of 33.42 dB. While the pre-trained model possesses inherent generative capabilities, its mismatch with the spectral imaging domain leads to a performance drop. Fine-tuning the diffusion model on a spectral dataset improves performance by 2.83 dB. And the URSe module significantly reduces inference time from 312.43s to 13.79s, while frequency separator provides an additional 0.47 dB improvement. These results highlight the importance of fine-tuning, spectral embedding. and efficient high-frequency detail generation for optimal performance in spectral imaging. In addition, the appendix AoopendxxAIpoon provides detailed implementation steps, optimization strategies, methodological explanations, performance analysis, dataset insights, and additional experimental results.
Initial Prodictor
Frequency
URSe
Diffusion
PSNR
SSIM
LPIPS
Inference
Seporator
Time
37:21
0.959
0.05718
0.26
x
33.42
0.883
0.06423
312.43
0.05375
13.79
37.67
0.962
0.04246
193.21
38.14
0.967
0.02844
12.90
Spectral feature embedding E in the URSe. To assess the role of spectral embedding E, we replaced it with a constant value of and retrained the model. The absence of E resulted in PSNR of 45.40 dB, compared to 49.36 dB when E was included. Fig. shows the decline in reconstruction quality, confirming the importance of E' in maintaining spectral fidelity.
Tra Single 0.9927
Table 4: Inference time comparison.
will E, com 8077
E. car 8006
Inference time Inference time (200 steps)
Inference time (600 stops)
Method
Calegory
Reference
(50 steps)
603
600
Wevelergth (nm)
DinscI
Diffusion
CVPR 2024
84.54s
251.9% 55.9%
865.81s
Figure 10: Spectral Reconstruction
PSR-SCI
Diffusion
Ours
8.90x
19.10s
74.76x
Performance of several URSe variants.
Guidance scale s, time-step T and inference time. The high- dimensional guidance scale and initial time-step are critical hyperparameters our diffusion model. We optimize these jointly, achieving the highest PSNR (36.68 dB) with 0.08 and T = 50 (Fig. IT]. Table 4 shows our PSR-SCI model's efficiency, requiring only 8.9 seconds for 50 steps, significantly faster than the 85 seconds for the state-of tate-of-theear DiffSCI.
Figure 11:
Hlyper-parameten opti- mization of PSR-SCI-T
5 CONCLUSION AND FUTURE DIRECTIONS
We introduced a new framework for spectral compressive imaging reconstruction, focusing on reconstructing high-frequency details by fine-tuning a diffusion model pre-trained on large-scale RGB images in the spectral subspace of MSI. To reduce the computational burden of diffusion sampling and training a diffusion model for MSI, we have proposed four novel techniques: fast SCI diffusion framework, generation strategy, and high-dimensional guidance with imaging consistency. Our empirical results demonstrate significant improvements in detail quality and superior metrics compared to current SOTA methods. We believe that our work introduces novel direction in spectral compressive imaging reconstruction, emphasizing the importance of high- figh-frequency information, and establishes robust benchmark for future research endeavors.
senniiinn-driveee reversible spectral embedding, high-frequency diffusion
Our method demonstrates excellent generalization and detail recovery capabilities. However, our approach also has certain limitations, as detailed in Appendix Sec. [A.6.]] Exploring efficient denoising diffusion model sampling, faster schedulers, and enhancing our predictor and denoiser networks with optimized architectures promising future directions.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
This work was supported by National Natural Science Foundation of China under Grant 623507 10797, by Guangdong Basic and Applied Basic Research Foundation under Grant 100231511510000 by Shenzhen Science and Technology Innovation Committee under Grant JSGG2022083 104402004 and Guangdong Major Project of Basic and Applied Basic Research under Grant 2023B0303000010.
REFERENCES
Brian DO Anderson. Reverse-time diffusion equation models. Applications, 12(3):313-326, 1982.
Stochastic Processes and their
Boaz Arad, Radu Timofte, Rony Yahel, Nimrod Morag, Amir Bernat, Yaqi Wu, Xun Wu, Zhihao Fan, Chenjie Xia, Feng Zhang, et al. Ntire 2022 spectral demosaicing challenge and data set. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 882-896, 2022.
Yochai Blau and Tomer Michaeli. The perception-distoriinn tradeoff. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6228-6237, 2018.
Yuanhao Cai, Jing Lin, Xiaowan Hu, Haoqian Wang, Xin Yuan, Yulun Zhang, Radu Timofte, and Lue Van Gool. Mask-guider spectral-wise transformer for efficient hyperspectral image reconstruction. In CVPR, 2022a.
Yuanhao Cai, Jing Lin, Xiaowan Hu, Haoqian Wang, Xin Yuan, Yulun Zhang, Radu Timofte, and Luc Van Gool. Coarse-to- fine-fee sparse transformer for hyperspectral image reconstruction. In ECCV, pp. 686-704. Springer, 2022b.
Yuanhao Cai, Jing Lin, Zudi Lin, Haoqian Wang, Yulun Zhang, Hanspeter Plister, Radu Timofte, and Lue Van Gool. Mst++; Multi-stage spectral-wise transformer for efficient spectral reconstruction. In CVPRW, 2022c.
Yuanhao Cai, Jing Lin, Haoqian Wang. Xin Yuan, Henghui Ding, Yulun Zhang, Radu Timofte, and Luc V Gool. Degradation-aware unfolding half-shuffle transformer for spectral compressive imaging. NewrlPS, 2022d.
Yuanhao Cai, Yuxin Zheng, Jing Lin, Xin Yuan, Yulun Zhang, and Haoqian Wang. Binarized spectral compressive imaging. Advances in Neural Information Processing Systems, 36, 2024.
Xun Cao, Tao Yue, Xing Lin, Stephen Lin, Xin Yuan, Qionghai Dai, Lawrence Carin, and David J. Brady. Computational snapshot multispectral cameras: Toward dynamic capture of the spectral world. IEEE SPM, 2016.
Stanley H Chan, Xiran Wang, and Omar A Elgendy. Plug-and-plas admm for image restoration: Fixed-point convergence and applications. Transactions on Computational Imaging, 2016.
Inchang Choi, MH Kim, D Gutierrez, DS Jeon, and G Nam. High-quality hyperspectral reconstruction using a spectral prior. In Technical report, 2017.
Jooyoung Choi, Sungwon Kim, Yonghyun Jeong, Youngjune Gwon, and Sungroh Yoon. Ilvr: Condi- tioning method for denoising diffusion probabilistic models. arXiv preprint (eXXx:2210.00290 2021.
Hyungjin Chung. Jeongsol Kim, Michael T Mccann, Mare Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. arXiv preprint 2209.22209.16688 2022.
Hyungjin Chung, Jeongsol Kim, Michael Thompson Mecann, Mare Louis Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. In ICLR, 2023.
Mauricio Delbracio, Hossein Talebei, and Pevman Milanfar, Projected distribution loss for image enhancement. In 2021 IEEE International Conference on Computational Photography (ICCP), pp. 1-12. IEFF, 2021.
11

---

Published as a conference paper at ICLR 2025
Yubo Dong, Dahua Gao, Tian Qiu, Yuyan Li, Minxi Yang, and Guangming Shi. Residual degradation learning unfolding framework with mixing priors across spectral and spatial for compressive spectral imaging. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22262-22271, 2023.
David L Donoho. Compressed sensing. IEEE TIT, 2006.
Mário AT Figuciredo. Robert D Nowak, and Stephen J Wright. Gradient projection for sparse reconstruction: Application compressed sensing and other inverse problems. IEEE Journal of selected topics in signal processing, 2007.
Ying Fu, Zhiyuan Liang, and Shaodi You. Bidirectional 3d quasi-recurrent neural network for hyperspectral image super-resolution. Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021.
Michael E Gehm, Renu John, David J Brady, Rebecca M Willett, and Timothy Schulz. Single-shot compressive spectral imaging with dual-disperser architecture. Optics express, 2007.
Wei He, Zongliang Wu, Naoto Yokoya, and Xin Yuan. An interpretable and flexible fusion prior to boost hyperspectral imaging reconstruction. Information Fusion, pp. 102528, 2024.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. NeurIPS, 2020.
volume 33, pp.
Xiaowan Hu, Yuanhao Cai, Jing Lin, Haoqian Wang, Xin Yuan, Yulun Zhang, Radu Timofte, and Lue Van Gool. Hdnet: High resolutooo dual-domain learning for spectral compressive imaging. In CVPR, 2022.
Tao Huang, Weisheng Dong, Xin Yuan, Jinjian Wu, and Guangming Shi. Deep gaussian scale mixture prior for spectral compressive imaging. In CVPR, 2021.
Ajil Jalal, Marius Arvinte, Giannis Daras, Eric Price, Alexandros G Dimakis, and Jon Tamir. Robust compressed sensing mri with deep generative priors. Advances in Neural Information Processing Systems, 14998-149944 2021.
Shirin Jalali and Xin Yuan. Snapshot compressed sensing: Performance bounds and algorithms. IEEE 771, 2019.
Bahjat Kawar, Michael Elad, Stefano Ermon, and Jiaming Song. Denoising diffusion restoration models. In NeurlPS, volume 35, pp. 23593-23606, 2022.
Nirmal Keshava and John Mustard. Spectral unmixing. IEEE signal processing magazine, 19(1): 44-57, 2002.
Diederik P Kingma and Max Welling. arXiv:1312.6114, 2013.
Auto-cncoding variational bayes. arXiv preprint
David Kittle, Kerkil Choi, Ashwin Wagadarikar, and David J Brady. Multiframe image estimation for coded aperture snapshot spectral imagers. Applied optics, 2010.
Zhifeng Kong and Wei Ping. On fast sampling of diffuuson probabilistic models. In ICML Workshop on Invertible Neural Networks, Normalizing Flows, and Explicit Likelihood Models, 2021. URL
net Forum ?id=aiidd=aip 4cdofrrA
Sang-gil Lec. Heeseung Kim. Chachun Shin, Xu Tan, Chang Liu, Qi Meng, Tao Qin, Wei Chen, Sungroh Yoon, and Tie- Yan Liu. Priorgrad: Improving conditional denoising diffusion models with data-driven adaptive prior. arXiv preprint arXiv:2 2.X.v:2106.00 106.::2006.06406,, 2021.
Miaoyu Li. Ying Fu, Ji Liu, and Yulun Zhang. Pixel adaptive deep unfolding transformer for hyperspectral image reconstruction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 22959-12968. 2023.
Xinqi Lin, Jingwen He, Ziyan Chen, Zhaoyang Lyu, Ben Fei, Bo Dai, Wanli Ouyang, Yu Qiao, and Chao Dong. Diffbir: Towards blind image restoration with generative diffusion prior. arXiv preprint a008 55770, 2023.
12

---

Published as a conference paper at ICLR 2025
Yang Liu, Xin Yuan, Jinli Suo, David Brady, and Qionghai Dai. Rank minimization for snapshot compressive imaging. IEEE TPAMI, 2019.
Guolan Lu and Baowei Fei. Medical hyperspectral imaging: a review. Journal of Biomedical Optics, 2014.
Jiawei Ma, Xiao- Yang Liu, Zheng Shou, and Xin Yuan. Deep tensor admm-net for snapshot compressive imaging. In ICCV, 2019.
Xiao Ma, Xin Yuan, Chen Fu, and Gonzalo R Arce. Led-based compressive spectral-temporal imaging. Optics Express, 2021.
Roey Mechrez, Itamar Talmi, Firas Shama, and Lihi Zelnik-Manor. Maintaining natural image statistics with the contextual loss. In Computer Vision-ACCV 2018: 14th Asian Conference on Computer Vision, Perth, Australia, December 2-6, 2018, Revised Selected Papers, Part III 14, pp- 427-443. Springer, 2019.
Ziyi Meng, Jiawei Ma, and Xin Yuan. End-to-end low cost compressive spectral imaging with spatial-spectral atff-attention. In ECCV, 2020a.
Ziyi Meng. Mu Qiao, Jiawei Ma, Zhenming Yu, Kun Xu, and Xin Yuan. Snapshot multispectral endomicroscopy. Optics Letters, 2020b.
Ziyi Meng, Zhenming Yu, Kun Xu, and Xin Yuan. Self-supervised neural networks for spectral snapshot compressive imaging. ICCV, 2021.
Ziyi Meng, Xin Yuan, and Shirin Jalali. Deep unfolding for snapshot compressive imaging. IJCV, 131(11):2933-955) 2023.
Xin Miao, Xin Yuan, Yunchen Pu, and Vassilis Athitsos. I-net: Reconstruct hyperspectral images from a snapshot measurement. ICCV, 2019.
Yuchun Miao, Lefei Zhang, Liangpei Zhang, and Dacheng Tao. Dds2m: Self-supervised denoising
diffusion spatio-spectral model for hyperspectral image restoration. In ICCV, pp. 12086-12096 2023.
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusson probabilistic models. In ICMI, pp. 8162-8171. PMLR, 2021.
Gregory Ongie, Ajil Jalal, Christopher A Metzler, Richard G Baraniuk, Alexandros G Dimakis, and Rebecca Willett. Deep learning techniques for inverse problems in imaging. IEEE Journal on Selected Areas Information Theory, 1(1):39-56, 2020.
Zhenghao Pan, Haijin Zeng, Jiezhang Cao, Kai Zhang, and Yongyong Chen. Diffscci: Zero-shot snapshot compressive imaging via iterative spectral diffusion model. CVPR, 2024.
Li Pang, Xiangyong Cao, Datao Tang, Shuang Xu, Xueru Bai, Feng Zhou, and Deyu Meng. Hsigene: foundation model for hyperspectral image generation. arXiv preprint arXiv:2409. 12470, 2024a.
Li Pang, Xiangyu Rui, Long Cui, Hongzhong Wang, Deyu Meng, and Xiangyong Cao. Hir-diff: Unsupervised hyperspectral image restoration via improved diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3005-3014, 2024b.
Jong-Il Park, Moon-Hyun Lee, Michael D. Grossberg, and Shree K. Nayar. Multispectral imaging using multiplexed illumination. In ICCV, 2007.
Mu Qiao, Xuan Liu, and Xin Yuan. Snapshot spatial-tempores compressive imaging. Optics letters, 2020.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
Xiangyu Rui, Xiangyong Cao, Zeyu Zhu, Zongsheng Yue, and Deyu Meng. Unsupervised pansharp- ening via low-rank diffusion model. arXiv preprint arXiv:2305. 10925, 2023.
13

---

Published as a conference paper at ICLR 2025
Robin San-Roman, Eliya Nachmani, and Lior Wolf. Noise estimation for generative diffusion models. arXiv preprint Stti::2104.026000 2021.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint wrXiv:2010.02500) 2020a.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In Inter- national Conference on Learning Representations, 2021. URL /tppatt//eeeeeeeeeee...c
Yang Song, Jascha Sohl-Dickstein, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13355 2020b.
Jin Tan, Yanting Ma, Hoover Rueda, Dror Baron, and Gonzalo R. Arce. Compressive hyperspectral imaging via approximate message passing. IEEE Journal of Selected Topics in Signal Processing, 2016.
Prasad S Thenkabail, Murali Krishna Gumma, Pardhasaradhi Teluguntla, and AM Irshad. Hyperspec- tral remote sensing of vegetation and agricultural crops. Photogrammetric Engineering & Remote Sensing (TSI), 80(8))695-72) 2014.
Joel A Tropp and Anna C Gilbert. Signal recovery from random measurements via orthogonal matching pursuit. IEEE TIT, 2007.
Ashwin Wagadarikar, Renu John, Rebecca Willett, and David Brady. Single disperser design for coded aperture snapshot spectral imaging. Applied Optics, 2008.
Lizhi Wang, Zhiwei Xiong, Guangming Shi, Feng Wu, and Wenjun Zeng. Adaptive nonlocal sparse representation for dual-camera compressive hyperspectral imaging. IEEE TPAMI, 2016.
Lizhi Wang. Chen Sun, Ying Fu, Min H. Kim, and Hua Huang. Hyperspectral image reconstruction using a deep spatial-spectral prior. In CVPR, 2019.
Lizhi Wang, Chen Sun, Maoqing Zhang, Ying Fu, and Hua Huang. Dnu: Deep non-local unrolling for computational spectral imaging. In CVPR, 2020.
Yinhuai Wang, Jiwen Yu, and Jian Zhang. Zero-shot image restoration using denoising diffusion null-space model. arXiv preprint arXiv:22 1rXiv:2212.00490. 2022.
Jay Whang, Mauricio Delbracio, Hossein Talebi, Chitwan Saharia, Alexandros G Dimakis, and Pey- man Milanfar. Deblurring via stochastic refinement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16293-16303, 2022.
Zongliang Wu, Ruiying Lu, Ying Fu, and Xin Yuan. Latent diffusion prior enhanced deep unfolding for spectral image reconstruction. arXiv preprint arXiv:231 14280,, 2023.
Zongliang Wu, Ruiying Lu, Ying Fu, and Xin Yuan. Latent diffusion prior enhanced deep unfolding for snapshot spectral compressive imaging. In European Conference on Computer Vision, pp. 164-181 Springer, 2025.
Zhiyang Yao, Shuyang Liu, Xiaoyun Yuan, and Lu Fang. Specat: Spatial-spectral cumulative- attention transformer for high-resolution hyperspectral image reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15368-25377) 2024.
Xin Yuan. Generalized alternating projection based total variation minimization for compressive sensing. ICIP, 2016.
Xin Yuan, Tsung -sung-Han P Tsai. Ruoyu Zhu, Patrick Llull, David Brady, and Lawrence Carin, Compressive hyperspectral imaging with side information. IEEE JSTSP. 2015.
Xin Yuan, Yang Liu, Jinli Suo, and Qionghai Dai. Plug-and-play algorithms for large-scale snapshot compressive imaging. In CVPR, 2020.
14

---

Published as a conference paper at ICLR 2025
Xin Yuan, David Brady, and Aggelos K Katsaggelos. Snapshot compressive imaging: Theory, algorithms, and applications. IEEE Signal Processing Magazine, 2021a.
Xin Yuan, Yang Liu, Jinli Suo, Fredo Durand, and Qionghai Dai. Plug-and-play algorithms for video snapshot compressive imaging. IEEE TPAMI, 2021b.
Yuan Yuan, Xiangtao Zheng, and Xiaoqiang Lu. Hyperspectral image superresolution by transfer learning. IEEE JSTAEORS, 2017.
Haijin Zeng, Xiaozhen Xie, Haojje Cui, Hanping Yin, and Jifeng Ning. Hyperspectral image restoration via global 1-2 spetial- total variation regularized local low-rank tensor recovery. IEEE transactions on geoscience and remote sensing, 59(4):3309-332 2020.
Jiancheng Zhang, Haijin Zeng, Jiezhang Cao. Yongyong Chen, Dengxiu Yu, and Yin-Ping Zhao. Dual prior unfolding for snapshot compressive imaging. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2574222575,, June 2024a.
Jiancheng Zhang. Haijin Zeng, Yongyong Chen, Dengxiu Yu, and Yin-Ping Zhao. Improving spectral snapshot reconstruction with spectral-spatial rectification. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 25817-25826, June 2024b.
Shipeng Zhang, Lizhi Wang. Ying Fu, Xiaoming Zhong, and Hua Huang. Computational hyperspec- tral imaging based on dimension-disone discriminative low-rank tensor recovery. In ICCV, 2019.
Xuanyu Zhang, Yongbing Zhang, Ruiqin Xiong, Qilin Sun, and Jian Zhang. Herosnet: Hyperspectral explicable reconstruction and optimal sampling deep network for snapshot compressive imaging. In CVPR, 2022.
Siming Zheng, Yang Liu, Ziyi Meng, Mu Qiao, Zhishen Tong, Xiaoyu Yang. Shensheng Han, and Xin Yuan. Deep plug-and-play priors for spectral snapshot compressive imaging. PR, 9())B18-B22) 2021a.
Siming Zheng, Yang Liu, Ziyi Meng, Mu Qiao, Zhishen Tong, Xiaoyu Yang, Shensheng Han, and Xin Yuan. Deep plug-and-play priors for spectral snapshot compressive imaging. Photonics Research, 2021b.
Yuanzhi Zhu, Kai Zhang, Jingyun Liang, Jiezhang Cao, Bihan Wen, Radu Timofte, and Luc Van Gool. Denoising diffusion models for plug-and-play image restoration. In CVPR, pp. 1219-1229, 2023.
A APPENDIX / SUPPLEMENTAL MATERIAL
Summary. In this supplementary material, we provide the implement details of our approach in Sec. Sec. show the optimization results of typer-parameters: high-dimensional guidance scale and the start timestep T' of diffusion model by using SSIM in Sec. Sec. A.2, and provide the derivation process of the subspace diffusion with high- ligh-dimensional guidance in Sec. A.3 Section e.4ebabossas on the derivation and explanation of the Unmixing-dris driven Spectral Embedding (URSe) approach, which plays a crucial role in enhancing the spectral reconstruction process. Section AA.62 discuss how the framework, rather than solely relying on parameter scaling, contributes to its superior performance. Section A.7 contrasts the differences between the generated dataset and real collected datasets, offering further insights into the data used for training. Section presents additional experimental results on both real and simulated datasets, further validating the effectiveness of our approach across different scenarios. The Table list of this supplementary materials is listed as follows:
See.
A.
Implement Details
* Sec.
A.2
Hyper-parameters Optimization
Sec.
A.3
Subspace Diffusion with High-dimensional Guidance
Sec.
A.4
Explanation of the Unmixing-driver Spectral Embedding Approach
Sec. A.5
Framework Architectural Analysis and Performance Validation
Sec. A.6
Limitation analysis and Generalization Across Diverse Datasets
Sec.
A.7
Analysis of Training Datasets for Diffusion Models
Sec. A.8
Additional Experimental Results
15

---

Published as a conference paper at ICLR 2025
A.1 IMPLEMENT DETAILS
More Experimental Details. We enhance the multispectral image dataset (all from CAVE dataset using various augmentation techniques, in- cluding cropping (directly cropping to the target size and 1/2 by 1/2 cropping for 2 x 2 patching) and scaling (2 2 up-scaling and 2 x 2 down-scaling). Additionally, during the task of real images restoration, Gaussian noise of varying degrees is introduced to the dataset to fine-tune the ControlNet module which can simulate the sensing noise in the real data. Subsequent training is conducted on the high-frequency components (with 3 Gaussian iterations) of the spectral images output by the Initial Predictor (a pre-trained fast transformer model Cai et al. (2022a).
https://cave. wwt.cccccco........ coluubba..dduurppsiior///uaaaaaa
All experiments are conducted with data paralleling on server equipped with 4 RTX 3090 GPUs, using Python 3.9.19, 19, PyTorch 2.2.0+cu12], 121, and CUDA 12.2.
1. For the URSe training, we initialize the model randomly. Training is performed using the Adam optimizer with learning rate (1,)) of 0.02 and batchsize of for 200 epochs. During the first 30% of the epochs, an additional visual enhancement regularization term based on the mean squared error (MSE) between encoded images and pseudo-RGB images is included. The first 10 epochs employ a linear learning rate warmup strategy, and over the next 100 epochs, the 1, is gradually reduced 0.002 to achieve higher performance. The initial training of URSe takes approximately 2.5 hours.
2. For the VAE module training. we fine-tune the VAE module from the well-trained SD2.1 model siatch_size
Training is conducted with 4, = 1 x 10 batch size = 16, and using the Adam optimizer with data parallelism across four GPUs Similarly, a 10% step linear warmup is applied, with a total of 40,000 training steps, taking about 9.5 hours.
3. Next, we fine-tune the URSe output part by serially integrating both modules to avoid the cumulative errors that might arise from independent training. This fine-tuning is performed with 1, = 1 x 10 batch_size = 6 for 100 steps, taking approximately 3 hours.
4. Then, we perform a full-parameter fine-tuning of the Diffusion's 's ControlNet part within the RGB environment converted by URSe. Training uses the Adam optimizer with a batch size of 24 and is conducted in two phases: the first phase sets 4, = 2 x 10-5 training for 30,000 steps, and the second phase sets 1, training time of 18 hours.
10 training for 50,000 steps, resulting a total
Finally, during the simulated data testing experiments, we fix the random seed to 480(not fixed on real data to fully demonstrate the robust prior knowledge of the Diffusion model). This ensures that the output results of the Diffusion model are fixed and reproducible. The Diffusion model is used to complete the texture restoration task and subsequent testing. The hyper-parameter grid search is also conducted under the condition of the random seed set 480.
Method for Image High and Low-Frequency Separation. The core method for separating the high and low-frequency components of an image is through Gaussian low-pass filtering. We apply the same Gaussian low-pass filter (using a Gaussian convolution kernel to perform Gaussian blur on the image) iteratively to the original multispectral image. This iterative process ultimately yields the low-frequency component of the image, while the difference from the original image represents the high-frequency texture component. By controlling the number of Gaussian low-pass filtering iterations, we can effectively manage the scale of texture details restored by the diffusion model. For the 256 x 256 spectral task, given that the Initial Predict effectively restores the structural parts of the image, we set the number of iterations to 3 in the global task environment.
The low-frequency content retains the low-rank spectral information of the overall image, while the
component preserves the image's 's texture details (especially sharp edges). By decomposing the image into high and low-frequency components, we can casily control the scale of the texture details during the diffusion restoration steps. This approach also significantly reduces the damage to the low-rank spectral information caused by channel compression and the VAE part of the latent diffusion process Rombach et al. (2022).
Up-sampling within URSe. To upsample multispectral images from a resolution of 256 x 256 to 512 x 512, and thereby fully exploit the prior knowledge embedded in the 512 x 512-resolution-
16

---

Published as a conference paper at ICLR 2025
trained UNet model in the diffusion model, experiment with various upsampling techniques. These methods include, but are not limited to, naive interpolation methods and transposed convolutions, each presenting certain challenges, shown in Table
Specifically, although interpolation methods vary, naive interpolation techniques fail to enable the model to enhance spatial utilization through learnable parameters. This limitation impedes the compression and storage of more spectral information from different channels at the same spatial location, and additionally leading a certain degree of blurring in the upsampled images inevitably, which is detrimental to the inverse operation during the recovery phase. Conversely, direct 2x2 transposed convolutions can produce the "checkerboard artifacts" similar the Bayer Pattem (can be seen in Fig. D This artifact in the low-frequency components contradicts the original goal of achieving a "low- cew-freyyyncy controllable diffusion model" and hinders the VAE network's image representation through high-low frequency separation.
Table 5: Comparison of different up-sampling operations used in the proposed URSe.
Up-sampling method w/o upsampling
Nearest Bilinear Trans-cony
Nearest+cony Bilincar+cony
PSNR SSIM
39.58
42.12
44.25
47.16
47.24
47.39
0.9647
0.9737
0.9821
0.9958
0.9857
0.9928
After extensive experimentation, we adopt a combination of bilinear interpolation followed by a 3x3 convolution with stride and padding both of 1. This approach not only avoids the artifacts associated with convolutional upsampling but also allows the model to fully leverage the upsampled resolution through learnable parameters. Consequently, it delivers an excellent and high perf-peeformance image upsampling operation.
A.2 HYPER-PANE PAEER-PARAMETERS OPTIMIZATION USING SSIM AND VISUAL MAP
As highlighted in our paper, the high- sech-dimensionall guidance scale and starting time-step T of the diffusion model are critical apper-paramtert Initially, we optimize them jointly based on PSNR. Additionally, perform optimization based SSIM, as depicted in Fig. 12 The trends of the parameters with respect to SSIM closely resemble those with respect to PSNR, where the best values for and T are: 0.08, T 50, respectively.
SSIM Heatmap
0.933
0.802
0.502
0.934
0 0.935
0.937
0.944
0.950
0.950
0.940
0.791
High-dlnenesion gnidlncce scale:
0.00
0.009
0.935
0.830
0.533
0.034
0.934
0.537
0.942
0.052
0.959
0.527
0.734
11.94
0:07
0.930
0.030
0. 028
0.6 0.682
1935
0. 057
0. 0.000
9:7
0 856
0. 953
0.800
0.755
0.000
0:935
0.994
0. 939
9.99
0.935
0 940
919
944
0 95.
940
0.675
77400
1U50
0.030
0.925
0.032
0.043
0.574
0.930
0.920
0.812
0.528
0.033
0.037
0.940
0.945
0.945
0.923
0.067
0.739
602-
0.235
0.898
0.811
-1.95
Butz
0.920
0.921
0.913
0.909
0.525
0.930
0
0.929
0.904
0.929
0.911
0.059
0.801
0.711
601
0.913
0.898
0.905
0. 0.010
0.
909
906
0.933
808
900
0.880
0.814
0.763
0.756
1006.3
100L0
5050
7000
0000
500.5
436.0
20LD
200.0
J0b0
00.6
3001
669
Reverse process steps:
T
Figure 12: Hyper-parameters of PSR-SCI-T: guidance scale s and time-step T optimization by using SSIM.
17

---

Published as a conference paper at ICLR 2025
Stage
Stage
sibaF
sibar
gibas
siba
0.16
UbORA
sibGF
SIGGF
siboF
sibaF
siboF
siba
0.08
sita
siboF
sibaF
sibbF
SibbF
SiboF
0.04
SIGGF
SIGGF
siboF
SibaF
SibGF
SiGGF
0.02
SIGGF
sibaF
siboF
SIDGF
SAbGF
siba
0.01
1000
500
200
50
20
Steps
Figure 13: Visual comparison of hyper-parameters
Notably, regardless of whether PSNR or SSIM is used the optimization metric, found that an excessively high guidance scale (s) results in oscillations and amplification of the Guidance Loss during the guidance iterations, ultimately causing gradient overflow and image content collapse. This instability is highlighted in Fig. 13 where it is evident that Stage 2 significantly enhances high-frequency details compared to Stage However, overly high guidance scaling can interfere with the diffusion process, leading to image degradation, while very low guidance scale causes mismatches between high and oow-frequency details. We hypothesize that this phenomenon due to the excessive guidance scaling factor, which competes with the diffusion process, leading to image content degradation.
Through syper-paramett search, prove the necessity of the guidance step (0 in restoring image textures during diffusion. Without adequate guidance (either very small g. cale or insufficient gepss)), the randomness inherent the diffusion process leads to uncontrollable texture content, resulting in significantly lower restoration performance.
A.3 ADDITIONAL DETAILS OF SUBSPACE DIFFUSION WITH HIGH-DIMENSIONAL GUIDANCE
Due to space limitations, the main body of our manuscript focuses on the essential procedures the guided reverse denoising diffusion process. This section provides additional comprehensive details.
The diffusion generation process performed within lower-dimensional subspace, while the final reconstruction occurs in the nigh-dimensional multi-spectral image space. However, achieving high-quality images in the subspace does not always guarantee satisfactory reconstructions in the MSI space. To address this limitation, we have integrated a high-dimensional guidance mechanism into the sampling process. This enhancement ensures better alignment between subspace images and high-dimensional MSI reconstructions.
Specifically, our approach involves conducting the diffusion process in the latent space. Firstly, our approach leverages diffusion process that transitions through series of states, ultimately refining the latent representation. This is achieved through an iterative reverse process as detailed equation and equation 12 enabling precise sampling from the target distribution. Then, by interpreting the diffusion and reverse processes as solutions to Stochastic Differential Equations (SDEs), we align method with a rigorous mathematical framework, detailed in equation 13 to equation 15 This perspective highlights the relationship between contimuous-time diffusion and the discretized reverse sampling steps, ensuring that our approach effectively captures the underlying data distribution while facilitating high high-quality image reconstruction. Consequently, in the context of SCI reconstruction, we leverage the learned distribution of latent variables from diffusion model to reconstruct the high- frequency components of subspace images, incorporating prior knowledge for improved accuracy. By using the initial prediction, measurement matrix, observed data, and embedding operators as guidance, establish connection between the subspace and MSI space, ensuring coherence and alignment during the reconstruction process. This reformulated approach enhances the reverse process and effectively bridges the gap between latent space and high- nigh-dimensional MSI reconstruction. This customized diffusion process guided by the high-dimensionn MSI space through our reversible spectral embedding functions, s and its inverse, These functions establish connection between
18

---

Published as a conference paper at ICLR 2025
the latent space and the MSI space, ensuring information is effectively transferred during sampling. Additionally, we incorporate the initial prediction Xia and the real measurement y as references during the sampling phase, shown in equation To equation 20 These inputs provide essential guidance, aligning the latent space diffusion process with the high-dimensional reconstruction requirements.
By integrating this high-dimensional guidance mechanism, we improve the coherence between the generated subspace images and the final MSI reconstructions. This ensures that the diffusion process is informed by high-dimensional data, leading to more accurate and reliable results.
Specifically, at time I, the denoiser first predicts the noise ex of the noisy latent 2,. noise is removed from to get the clean latent En:
Then the predicted
= co(zt, Xmirs Aluas t, E (AAittt) (, 2g Zt V1 āeCt
(11)
Vor
Consequently, more precise image can be sampled at state zo through an iterative reverse process denoted as p(zo|zz) As mentioned in the main body of our paper, the reverse process is updated as follows:
2 aq (z Xinit Almis 1, ((Amii)))
+ /L - Cx
(12)
Va
where 21 ~ 11., E [T]. As Song et al. (020b): Rui et al. (2023), we formulate the ancestral
sampling process 12 as the discretization of reverse SDE.
dA ((A,)) v=. log pr:))(z;)] di + g,dw,
(13)
Recent work Song et al. 2020b) shows that as the total diffusion step "7"" goes infinity and the forward series (X,) becomes (X, [0, 1}} indexed by continuous time variable, the diffusion process
X, is actually the solution to an Itō SDE: dxX = s(X, l)dl + 9ddw, where w represents the standard
Wiener process. For example, the diffusion process with transition distribution (( 1))
N
aa)))) corresponds to the SDE as follows
1 dX
2 (-aa)dd + vI addw.
(14)
1 ait) and 9r v1- Oz. Also, the reverse process is solution to an In this case, [(X,1)
SDE:
dX [f(,t)) of Vx, x, lx,p, dt g.dw,
(15)
Viewed through the lens of SDE, sampling from z²))) can be achieved by appropriately discretizing Equation (15). Consequently, in the context of SCI reconstruction, we aim to utilize the learned dis- tribution of from diffusion model, where Ale D( This model inherently incorporates prior information of the subspace image, facilitating the reconstruction of the high-frequency component of the subspace image from the observed measurement. Then, using the initial prediction Xmnr T y, and the E (together with & -1, it is used to reverse the spectral embedding and connect the subspace with the MSI space where Vmi belongs) guidance condition, we reformulate the reverse SDD concerning as
dz (f(,,) o 2., log Pe(z, Auur, $.Y. V.Y.E)
E)) dt + g:dw,
(16)
0x) and g: v1 az w is the reverse of the standard Wiener process. The gradient vx, log (z() is commonly referred to as the score function of 2,. Using Bayes's rule, the score function can be separated into two parts
where f(z,t)
V log (z, Kiiir 4, V., (z.))(zz) + Vz, logit(Xannn E zz).
The first part can be derived under the general unconditional framework. However, the second part is intractable, since only the relation between Xmmir X' and p(zz)2)) are known. Following Chung et al. (2023): Rui et al. 2023), we approximate the second term as
- log / p(Kinir Y, (l))))((dzz)
ze log Pe (Rini V. Elz;)
2 V log Ximi, $. V, Elgu),
(17)
19

---

Published as a conference paper at ICLR 2025
where En is the expectation of zul x by Tweedie's 's formula:
60
21)
|2| _0| =r]
1
c + (1 --) V. lo)]
(18)
The term log p( El is much more available since &n can be seen as an approximation to 2, by
using computational relaxation Rui et al. (2023). it can be formulated
log P(ini: El = log P(Kniir Φ, 20)
~ sin (Vo (D(20), E) + X + - 2(0 (()))). E) + X IIF
(19)
where s is trade-off parameter. Then, we discretize the reverse SDE 16 using the form of ancestral
sampling process 1
Z (zr + (1 ac) Vz, log (Z) Kmii; V, E))
(20)
Va:
2: - 1 a co(2t,1) 1) + vI 04
2
va
v1 0r
V 2, |ninn (v/g (D(20): E) + Xmma Y ( (D(2o),
(D(o)) E) + Mma)ee,,
(21)
where & is gradient scale, En = 11 At this point, we have completed the detailed inference process to obtain the update step for the modified latent diffusion model using our high-dimensional guidance.
A.4 EXPLANATION OF THE UNMIXING-DRIVEN SPECTRAL EMBEDDING APPROACH
Motivation and Comparison with SVD-based Methods. The motivation for using the unmixing- driven spectral embedding (URSe) approach is to effectively reduce the dimensionality of spectral information channels while ensuring accurate spectral data representation and compatibility with RGB space. Unlike SVD-based decomposition methods, such as the one used in Pang et al. (2024b), which rely on static linear assumptions, our learnable URSe module captures the inherent nonlinearity of spectral data, enabling it to provide more accurate and adaptive spectral embeddings. SVD-based approaches are inherently limited in two critical aspects:
Forward- Forvard-Decomposttee Decomposition Error Amplification: SVD-baser methods cannot adaptively fine- tune the reverse process, leading to amplified errors introduced during forward decomposi- tion when reconstructing spectral images.
Misalignment with Spectral Characteristics: These methods focus on rough spectral-to-RGI mapping, failing to capture the complex nonlinear relationships between spectral bands. This results in deviations between the diffusion model's outputs and true spectral images, especially given the differences in wavelength ranges (RGB: 460-650 nm vs. spectral imaging: 750-2500 nm).
summarzzzsss the comparison of our method, URSe, with the SVD-based decomposition on the KAIST dataset. The table includes PSNR values (upper entry) and SSIM scores (lower entry) for each method across 10 scenes. Our learnable URSe module addresses these limitations by fine-tuning on spectral datasets, ensuring accurate alignment of spectral domains with pre-trained RGB diffusion models. This capability enables sask-specific spectral embeddings that are reversible, noise-robust, and optimized for high-quality SCI reconstruction. Additionally, URSe enhances both convergence and accuracy, achieving significant improvements in reconstruction speed and quality compared to traditional methods.
Robustness of URSe in the Presence of Noise. To further evaluate the robustness of our URSe spectral embedding module under noisy conditions, we conducted additional experiments with Gaussian noise perturbations at different levels. As shown in Table 7 the URSe module consistently maintained high PSNR (above 37 dB) even with increasing noise levels, demonstrating its strong resilience to noise. Specifically, even with significant noise (0.1 Gaussian perturbation), URSe's PSNR dropped only slightly from 38. 14 dB 37.04 dB, while maintaining high SSIM value.
20

---

Published as a conference paper at ICLR 2025
Table Comparison of reconstruction performance between SVD-based band selection (HIR-Diff) and our learnable URSe module on the KAIST dataset.
Method
S3
S5
57
59
S10
Avg
40.65
36.25
39.20
51.10
38.70
40.60
43.12
40.80
33.00
46.60
41.00
SVD-band-select (HIR-Ditt)
0.9810
0.9667
0.9795
0.9948
0.9909
0.9873
0.9862
0.9867
0.9623
0.9913
0,9827
50.54
54.03
49.61
57.02
49.28
49.99
47.75
49.05
50.57
50.97
50.88
URSc (Ours)
0,9982
0.9990
0.9939
0.9994
0.9984
0.9984
0.9948
0.9977 0.9967
0,9988
0.9975
Table 7: Ablation study on the robustness of spectral embedding for the diffusion model on the KAIST dataset under Gaussian noise perturbations.
Method
Noise (Gaussian) Perturbation
PSNR (dB) 1
SSIM
0
38.14
0.9670
URSe (Ours)
0.01
38. 10
0,9665
0.1
37.04
0.9567
0
36.87
0,9628
SVD-hand-seled (HIR-Diff)
0.01
36, 1.5
0,9623
0.1
26.82 82
0,9383
In contrast, the SVD-based method (HIR-Diff) showed significantly reduced performance, with sharp drop PSNR (from 36.87 dB 26.82 dB) as the noise level increased. This indicates that SVD-based methods are more sensitive to noise, unable to maintain reconstruction quality under perturbations.
These results highlight the superior noise robustness of the URSe module, which is critical for real-world SCT tasks where noise is often present. The ability of URSe to retain high-quality reconstructions in noisy environments further underscores its advantages over traditional methods.
A.5 FRAMEWORK ARCHITECTURA ANALYSIS AND PERFORMANCE VALIDATION
Importance of Initial Predictor in the Two-Stage Architecture
We removed the initial predictor from the wwo-stage architecture and directly input the shifted- back measurement results into the URSe encoder for dif- fusion. Without the initial predic- tor, the model achieved only 24.15 dB PSNR after joint fine-tuni fine-tuning, sig- nificantly impairing MSI reconstruc- tion. Incorporating the initial predic- greatly enhanced reconstruction performance (Fig. 14). highlighting its essential role in improving MSI re-
hppd
afteee PaeeePhe Hel Irbbl Prod
Figure 14: Visual comparison initial predictor ablation studies.
construction quality.
These results demonstrate that although the diffusion model pre-trained on RGB datasets has genera- tive capabilities, an initial prediction network or an iterative formulation like DiffSCI is still necessary to efficiently model the mapping between snapshot single-expe exglsurx images and multispectral data. This is something that diffusion models pre-trained on RGB data are not well-suited for.
Comparison with Refinement-Basee Frameworks. To further validate the effectiveness of our approach, we conducted detailed comparison with DAUHST-SP2 He et al. (2024), a state-of-the-art refinement-basee framework. For fair evaluation, both methods use DAUHST as the first-stage predictor. The results, summarized in Table g highlight the advantages of our PSR-SCI method across multiple evaluation metrics.
Specifically, observations indicate that PSR-SCI achieves a PSNR of 38.14 14 dB, outperforming DAUHST-SP2's 37.61 dB. In addition, our method demonstrates superior SSIM, LPIPS. MUSIQ, MANIQA, and CLIP-IQA metrics, emphasizing its capacity to improve both objective and perceptual reconstruction quality. These improvements highlight the effectiveness of our method in enhancing reconstruction quality compared to existing refinement frameworks.
21

---

Published as a conference paper at ICLR 2025
Table 8: Comparison between PSR-SCI and refinement- based frameworks. DAUHST is used as the first-stage predictor for a fair comparison.
Method
Calegory
Reference
PSNR (dB)
SSIM
LPIPS L MUSIQT MANIQA CLIP-IQA
SAU
Subspace prior Information Fusion 2024
37.61
0.966
DinSCI
Diffusion
CVPR 2024
35.28
0.916
0.07421
39.64
0,23598
0.3315
PSR-SCI-D Diffusion
Ours
38,14
0.967
0,02844
42.73
0.2527
0,3561
By including this comparison, we aim to substantiate the advantages of our approach over similar models. The consistent improvements across multiple evaluation metrics reinforce the robustness and efficacy of PSR-SCI in spectral image reconstruction.
Comparison with HIR-Diff. For comprehensive comparison, applied the HIR-Diff Pang et al. (2024b) model to the second stage of our two-stage framework while keeping the first stage unchanged. The performance results for snapshot compressive imaging tasks are presented in Table Our PSR-SCI outperforms HIR-Diff in both PSNR and SSIM, demonstrating the efficacy of our proposed approach in SCI applications.
Table 9: Comparison between HIR-Diff and PSR-SCI on Snapshot Compressive Imaging (SCT) tasks.
Method
Reference CVPR-2024
PSNR (dB) t SSIM t
HIR-Diff (+our first stage)
35.23
0.959
PSR-SCI-D (Ours)
38.14
0.967
These findings underline the benefits of our approach, where fine-tuning, spectral embedding, and efficient high-frequency detail generation significantly improve both the quality and speed of spectral image reconstruction from compressed sensing measurements.
Comparison with HIR-Diff for Snapshot Compressive Imaging (SCI). Our work differs from HIR-Diff in several important aspects, as summarized below:
Spectral Decomposition: We employ learnable spectral embedding approach, whereas HIR- Diff uses a fixed 3-band selection. Our method achieves superior reconstruction quality (50.88 dB vs. 41 dB, averaged over 10 KAIST scenes) and faster processing speed (0.0009s vs. 0.001215s). These improvements are indicative of the advantages of our trainable spectral decomposition.
Diffusion Refinement: Unlike HIR-Diff, which applies pre-trained RGB diffusion model without addressing the spectral differences between RGB and multispectral images, our framework incorporates trainable diffusion model fine-tuned on spectral data. This allows our model to achieve higher performance, with PSNR of 38.14 14 dB compared to 35.23 dB for HIR-Diff.
Task Complexity: The tasks are also distinct: HIR-Diff focuses on reconstructing multispec- tral images from N bands to N bands, whereas our work reconstructs N bands from single compressed image band to N bands), which presents additional challenges due to the low sampling rate. This makes our task more difficult and computationally demanding.
Flexibility and Performance: Our method offers more flexible and efficient framework, significantly outperforming HIR-Diff in both performance (PSNR: 38.14 dB vs. 35.23 dB) and computational efficiency. Although we incorporated the HIR-Diff approach into the second stage of our franework, the results clearly show that our approach, leveraging spectral embedding, fine-tuning, and guidance, outperforms existing methods for SCI tasks.
A.6 LIMITATION ANALYSIS AND GENERALIZATION ACROSS DIVERSE DATASETS
A.6.1 CLARIFICATION ON PSNR AND PERFORMANCE ON KAIST DATASET
For the four test datasets, our PSR-SCI achieves the best PSNR and SSIM three of them (ICVL, NTIRE, and Harvard), with the exception of the KAIST dataset. To understand this discrepancy, we carefully compared the four test datasets with the training dataset (CAVE).
Our analysis revealed that the wavelengths and scene objects in the KAIST dataset closely resemble those in the CAVE training dataset. In contrast, the ICVL,, NTIRE, and Harvard datasets are notably
22

---

Published as a conference paper at ICLR 2025
different from CAVE in terms of spectral and scene diversity. This suggests that diffusion-baseee methods, which achieve over 40dB PSNR on KAIST, perform well due to their strong fitting to the training dataset. However, this overfitting comes at cost: their generalization capability decreases significantly, leading to sharp performance drops on datasets that differ from the training data, such ICVI., NTIRE, and Harvard (as evidenced by sharp PSNR drop of over dB when applied the ICVL dataset).
In contrast, our PSR-SCI, diffusion-based method, does not prioritize overfitting the training dataset. Instead, it leverages its generative capabilities to tackle challenges that non-diffusion models struggle with, such as reconstructing fegeffreuency details across diverse scenes and performing well on real-world datasets. This difference in focus allows PSR-SCI to maintain strong performance on datasets with varied characteristics (KAIST: 38. 1 14dB, ICVL: 37+dB ), demonstrating better generalization compared to non-diffusion methods.
On the other hand, while our PSR-SCI method achieves a PSNR of 38.14 dB on the KAIST dataset, which is slightly lower than some recent end-to-end networks that exceed 39 dB (e.g.. PADUT Li ett al. (2023). RDLUF-MixS2 Dong et al. (2023), and LADE-DUN Wu et al. (2025)), () it's important to note that our method excels in other quantitative metrics. Specifically, on the KAIST dataset, our method outperforms these latest end-to-end methods terms of MUSIQ, MANIQA, and CLIP-IQA scores (Table 10).
Table 10: Comparison between PSR-SCI and related works the KAIST dataset.
Method
Category
Reference
PSNR (dR)
SSIM
TPIP8 | MUSIQ
MANIQA
CLIP-IQA
PADDU-122sg al.
Untolding
28.89
0.974
0.03553
40.55400
RDLUE MisS2 Dong
0023
Unfolding
CVPR 2023
29.57
0.974
LADE-DUN.10RE
Wa
Undolding
ECCV
40.16
0,900
0.03186
41.2968
0134466
SPECAT Yan a
Transformer
CVPR 2024
40.37
0.986
0.02785
41,72187
0 033482
01342944
MST-L
Tranformer
CVPR 2022
35.18
0,948
0.06906
37,06562
0.11199
0.31166
DAUIST 35te
Unfolding
NeurIPS 2022
27.21
0.959
0,05718
37,64531
0.18808
0.29966
DMUHST-SP2 He et 2014
Suspace prir
Information Fasion 2024
37.61
0.966
DinSCI
Diffasion
CVPR 2024
35.26
0.916
0,07421
39,64512
0.23598
0.33152
PSR-SCI-D
Diffasoo
Ours
28.14
0.967
0.02844
42,72969
0.25279
0.35602
Moreover, on the ICVL. NTIRE, and Harvard datasets, our PSR-SCI achieves the best PSNR. SSIM, and MANIQA metrics compared the latest end-to-end methods (as presented in Table 2 in the main paper, we reproduce it here in Table 11 [[]for for easy reference). This demonstrates our method's excellent generalization ability across various datasets.
Table 11: Comparison of PSNR, SSIM, and MANIQA metrics across several zero-shot datasets.
DAUHST-3ogg
MST-L.
DPU-96g
SSR-L (CVPR 2014)
LADE-1OUg (FCCV 2024)
IMNSCI (CVPR 2044
PSR-CCED
PSR-SCT -DPU
PSR-SCI
Dataset
Metrie
(NowfPS 2022)
2024)
PSNRT
34.64
34.03
36.36
36.25
35,89
33.02
37.03
$7.25
37.14
ICVL
SSIM
0,890
0.885
0.918
0.914
0.904
0.918
0.923
0.918
MANIQAT
0,200
0.209
0,200
0,209
0.210
0,207
0.217
0,216
0.213
PSNRI
34.44
33.04
36.25
33.44
33.58
32.79
36.44
36.62
33.33
NTIRE
SSIM
0.927
0.914
0.945
0.942
0.923
0.903
0.953
0.955
0.948
MANIQA
0,214
0.210
0,226
0,230
0.221
0.205
0.233
0,238
0.240
25.57
24.01
27.03
25.93
28.02
24.88
26.90
28.58
29.02
Harvard
SSIM
0.622
0.594
0.650
0,597
0.739
0.602
0.776
0.764
0.728
MANIQAT
0.187
0.204
0.197
0.195
0,239
0.247
Additionally, our method provides superior reconstruction results on real data. Our approach retains capabilities for generation and local inpainting tasks, which non-diffusion methods cannot achieve. Therefore, despite the slightly lower PSNR on the KAIST dataset, our method offers considerable advantages in perceptual quality, generalization, and additional functionalities, indicating substantial improvement over existing techniques.
A.6.2 LIMITATION ON MODEL PARAMETER COUNT
While our PSR-SCI method demonstrates significantly faster inference times compared to similar diffusion-basee methods like DiffSCI (8.9 seconds vs. 85 seconds for 50 steps), it inherits a large number of parameters from the pre- mreitrinned diffusion models. This reliance on pre- pre-trained models is crucial for leveraging their superior generative capabilities, which are essential for capturing frgg-frry details in spectral image reconstruction.
23

---

Published as a conference paper at ICLR 2025
However, this also means that model's overall parameter count is larger compared end-to-end networks like MST, as shown in Table [2] While the MST model demonstrates lower parameter count, even when scaled up. its performance and generalization ability remain limited. Specifically, increasing the MST model size from 2.018M to 39.052M parameters only yields a modest PSNR improvement of 0.58 dB, and further scaling it 480.535M parameters leads to PSNR drop to 32.98 dB due to overfitting, This underscores the limitations of non-diffusion-bert based models, where simply increasing the parameler count cannot ensure better reconstruction performance.
In contrast, our PSR-SCI method leverages large pre-trained diffusion models in a novel frame- work designed for spectral image reconstruction. This approach enables our method to capture high-frequent spectral details effectively while avoiding overfitting. Unlike MST, our method demonstrates consistent performance improvements at larger scales, achieving PSNR of 38.14 14 dB and an SSIM of 01.677. These gains are attributable to the innovative integration of trainable spectral embeddings and a fine-tuned diffusion refinement module, rather than simply the parameter count.
Table 12: Comparison of reconstruction performance for different model sizes and methods.
Model
Parameters
Test PSNR (dB) 1
Test SSIM 1
Training Time
MST-L
2.018M
35.18
0,948
MST-cxp1
39.052M
35.76
0.957
8.76h 0.921
MST-exp2
480.535M
32.98
95.21h
PSR-SCI-D (Ours)
1312M
38.14
0.967
18h
In summary, while our method's larger parameter count is a limitation, it is necessary trade-off to achieve high-quality reconstructions by utilizing the generative power of pre-trained diffusion
models. This ensures that our PSR-SCI method can deliver superior performance and faster inference times compared to other diffusion-baat methods, albeit with a higher parameter count than some end-to-end networks.
A.7 ANALYSIS OF TRAINING DATASETS FOR DIFFUSION MODELS
To address the reviewer's 's concern regarding the availability and suitability of hyperspectral image (HSI) datasets for training diffusion models, conducted a comparative study the impact of different training datasets. While the recently proposed HSIGene DanngnneePanggaaaaa et al. (2024b) offers a synthetic solution dataset limitations, experiments reveal significant differences its utility compared to real datasets.
Our results indicate that real datasets, such CAVE and Harvard, significantly improve the per- formance of diffusion models in reconstructing high-frequency details, achieving up to a +6. 18 dB PSNR improvement. In contrast, incorporating synthetic data generated by HSIGene leads slight reduction in performance, with -0.83 dB drop in PSNR and lower SSIM scores, as shown in Table 13 This demonstrates that the quality and diversity of tigh-freeueny details in real data play a critical role in effective training.
Table 13: Impact of training datasets on diffusion model performance.
Training Datasets
PSNR (dB)
SSIM f
Pretrained Diffusion Only
32.49
0.878
CAVE (Real Dataset)
14
0.967
CAVE + HSIGene (Generated)
37.31
0.959
CAVE Additional Real Dataset (Same Amount HSIGene)
38.67
0.972
While HSIGene offers an alternative alleviate dataset scarcity for simpler networks, it is less effective for diffusion models. The limitations of HSIGene stem from its reliance on restricted training dataset, resulting in generated images that often lack the diverse and high-quality high- frequency information required for advanced diffusion model training. Our findings underscore the necessity of real, high-quality datasets for optimizing the recovery of fine spectral details.
24

---

Published as a conference paper at ICLR 2025
Figure 15: Visual comparison of raw and high-freq part of datasets generated by HSIGene Pang et al. (0024a)
Figure 16: Visual comparison of raw and high-freq part of real captured datasets NTIRE2022 Arad al. (2022)
We identified that a considerable proportion of high- geegufrryyency details in the hyperspectral images produced by HSIGene (Fig. [5) exhibit misalignments or imperfections (Fig. [16), which directly contradicts the goal of leveraging diffusion models accurately restore high-frequency information. While HSIGene's ' synthetic images are beneficial for alleviating dataset limitations when training simpler networks, they prove less suitable for advanced models like diffusion. This limitation arises from HSIGene's ss reliance on a relatively restricted training dataset, which often results in generated images lacking the diversity and quality of high-frequeee details necessary for effective diffusion model training.
Experiments incorporating additional real MSI datasets, such as Harvard, ICLV, and NTIRE2022. further highlight the importance of genuine data, demonstrating significant improvements in the capacity of diffusion models to recover detailed high-frequency features. These findings underscore that, while synthetic datasets like HSIGene can serve as valuable augmentation resource for certain models, they fail to meet the stringent requirements of diffusion models designed for precise high- frequency reconstruction. Consequently, leveraging real, high-quality datasets is critical to fully exploit the capabilities of diffusion-barre based methods hyperspectral imaging.
A.8 ADDITIONAL RESULTS ON SIMULATION AND REAL DATASETS
25

---

Published as a conference paper at ICLR 2025
Patch
ADMM-Net
CST-L+
DIP-HSI
GAP-Net
HDNet
Scene
N-Net
MST-L.
MST++
PrP-CASSI
TSA-Net
DifISCI
PSR-SCI-D
GT Patch
ADMM-Ner
DGSMP
DIP-HST
GAP-Net
HDNet
Secne 4
A-Net
MSFL
MST+
PrP-CASSI
TSA-Nct
DIfISCI
PSR PSR-SCI-D
POORASGGRR
GT Patch
ADMM-Net
CST-L+
DGSMP
DIP-HSI
GAP-Net
HDNet
GGGRAGGGRA
BOGRA
MOGRANGRAA
Scene
A-Net
MST+
TSA-Net
DiffSCI
GT Patch
ADMM-Net
CST-L+
DGSMP
DIP-HSI
GAP-Net
HDNet
A-Net
MST-I.
MST+
PoP-CASSI
TSA-Net
DifiSCI
PSR-SCI-D
ADMM-Nat
Scene
A-Net
MSTL
MST++
PASSC
TSA-Nct
DitSsII
PSR-SCI-D
Patch
DGSMP
DIP-HSI
GAP-Net
HDNe
Scene
X-Net
MST-L
MST+
PaP-CASSI
TSA-Net
DiffSCI
PSR-SCI-D
Figure 17: Visual comparison from Scene 3 to Scene 8 of the KAIST dataset.
26

---

Published as a conference paper at ICLR 2025
GT Palch
ADMM-Net
CST-L+
DGSMP
DIP-TIS
GAP-Net
HDNet
N-Net
MST-L
MST+
PaP-CASSI
TSA-Nct
DifiSCI
PSR-SCI-D
HDNe
Sceme 10
A-Net
MSTL
MST++
PaP-CASSI
TSA-Net
Di#SCI
PSR-SCE-D
Figure 18: Visual comparison from Scene 9 to Scene 10 of the KAIST dataset.
RGB Ref.
Meas
CST-L
DGSMP
GAP-Net
HDNet
A-Net
MST-L
MST++
TSA-Net
DIffSCI
PSR-SCI-T
Figure 19: Visual comparison on Scene 2 of real dataset at wavelength 487nm.
RGB Ref.
Meas
CST-L
DGSMP
GAP-Net
HDNet
A-Net
MST-L
MST++
TSA-Net
DiffSCI
PSR-SCI-T
Figure 20: Visual comparison on Scene: 3 of real dataset at wavelength 575.5mm.
27

---

Published as a conference paper at ICLR 2025
RGB Ref.
Meas
CST-L
DGSMP
GAP-Net
HDNet
A-Net
MST-L
MST++
TSA-Net
DiffSCI
PSR-SCI-T
Figure 21: Visual comparison on Scene 4 of real dataset at wavelength 536.5nm.
RGB Ref.
DGSMP
GAP-Net
HDNet
X-Net
MST-L
MST+
TSA-Net
DifISCI
PSR-SCI-T
Figure 22: Visual comparison real Scene in pseudo-RGB (604nm, 536.5nm and 481.481mmmmm 1.nmmm..
RGB Ref.
CST-L
DGSMP
GAP-Net
HDNet
A-Net
Real Mcasurement
RGB PSR-SCI
MST+
TSA-Net
DifISCI
PSR-SCI-T
Figure 23: Visual comparison of SCI reconstruction models real Scene: wavelength 648.0nm.
28


Reviews:
[{"review_id": "ajqPOmW7Yn", "reviewer": "Reviewer_igVu", "summary": "The authors present a novel framework named PSR-SCI aimed at enhancing CASSI reconstruction. The framework consists of a initial predictor(MST or DAUHST), a unmixing-driven reversible spectral embedding module to decomposes the MSI into subspace images and spectral coefficients, and a pre-trained RGB latent diffusion models (SD-2.1) refinement processes. Furthermore, the authors implement a high-dimensional guidance mechanism that ensures imaging consistency. Experiments on the KAIST, NTIRE, ICVL, and Harvard dataset demonstrate that PSR-SCI enhances visual quality and achieves PSNR and SSIM metrics.\n\n---\nAfter rebuttal: I read the authors response and have raised my score.", "questions": "1. Why are the SSIM metrics omitted in Table 2?\n2. The number of abundance maps A in spectral unmixing should correspond to the number of endmembers. Why is an up-sampled 3-channel feature map used instead? Is this solely for compatibility with the RGB pre-trained model? The URSe module seems more like a data dimension reduction or latent space encoding operation, with little relevance to spectral unmixing", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The proposed method has higher performance than previous diffusion-based methods on CASSI reconstruction.\n2. The method costs less time in both training and inference time than previous diffusion-based methods.", "weaknesses": "1. The novelty of this approach is limited, as spectral decomposition and refinement using pre-trained RGB diffusion models have previously been introduced in the spectral image restoration field [1].\n2. The method relies heavily on the initial prediction network. Although the authors claim robustness for the proposed framework, it may not perform well on unseen optical setups (such as different masks, noise, and number of channels). The framework depends on initial reconstruction results from a pre-trained CASSI reconstruction network. If the pre-trained network does not perform adequately on a new system with different optical settings (a common issue in CASSI reconstruction) the framework’s effectiveness diminishes. For example, PSR-SCI-T (initial predictor: MST) and PSR-SCI-D (initial predictor: DAUHST) demonstrate this, with weaker predictors leading to inferior reconstruction results.\n3. The experiments presented are insufficient. The proposed method should be evaluated with recent state-of-the-art predictors, such as [2], DPU-9stg, and SSR-L. Additionally, comparisons with similar 'refinement' frameworks, such as [3], would further validate its performance.\n\n[1] Pang, Li, et al. \"HIR-Diff: Unsupervised Hyperspectral Image Restoration Via Improved Diffusion Models.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.\n[2] Yao, Zhiyang, et al. \"SPECAT: SPatial-spEctral Cumulative-Attention Transformer for High-Resolution Hyperspectral Image Reconstruction.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.\n[3] He, Wei, et al. \"An interpretable and flexible fusion prior to boost hyperspectral imaging reconstruction.\" Information Fusion (2024): 102528.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5xrmU8PnPm", "reviewer": "Reviewer_wQ83", "summary": "This paper proposes a new two-stage method, namely a predict-and-subspace refine (PSR-SCI) framework, using the diffusion model for snapshot compressive imaging reconstruction. In the first stage, an inexpensive predictor generates a rough MSI approximation. Then in the second stage, a spectral unmixing-driven subspace learning module is employed to reduce the dimensionality. Eventually, a diffusion model is fine-tuned to enhance the high-frequency details. There are three technical contributions:\n\n(i) Given the complexities of spectral unmixing models, a reversible decomposition network is introduced to implement spectral subspace learning while maintaining high reversibility.\n\n(ii) The diffusion generation is facilitated by low- and high-frequency decomposition.\n\n(iii) A high-dimensional guidance strategy is introduced for fine-tuning subspace diffusion, enhancing the effectiveness of guidance within the subspace.", "questions": "Some parts may need more explanation. For example,\n\nIn Algorithm1 (Line 270 - 287), the return value $\\mathcal{A}_{diff}^h$ is fed into the VAE decoder to obtain the final reconstructed hyperspectral images. However, for the latent space diffusion, the VAE is trained to decode normal RGB images. Can it directly work on hyperspectral images? Or did you fine-tune it or something like that?", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "(i) The novelty is good and interesting. The big idea of partitioning the reconstruction into two stages is cool. It seems like the latent space diffusion models. The first stage of prediction and decomposition is like the embedding process of VAE and this process is highly reversible. Besides, using the frequency domain decomposition (low- and high-frequency) to facilitate the generation process is also very fancy. It matches the nature of spectra. It is very exciting to have this work in the community of SCI reconstruction.\n\n(ii) The performance is solid. As compared in Table 1, Figures 6, 8, and 9. The proposed method PSR-SCI not only outperforms the state-of-the-art end-to-end methods by large margins but also achieves better visual results. Compared to the SOTA diffusion-based method DiffSCI, the proposed PSR-SCI is 2.86 dB higher! It is great progress of the diffusion-based method in SCI. This work even contains the results of Pseudo-RGB on the RGB-to-HSI reconstruction task. \n\n(iii) The presentation is well-dressed. The figure of the pipeline looks clear. I can easily see the workflow of the proposed framework. The table of quantitative comparisons is also neat. The style of this table is from the series work of MST, MST++, CST, DAUHST, etc. I like it very much. The writing also looks good and easy to follow.\n\n(iv) Code and the reconstruction results have been submitted,", "weaknesses": "(i) Some modifications and explanations should be added. For example, why do you want to use an inexpensive predictor to produce an initial HSI instead of directly reconstructing the HSI results from the noisy measurements? Besides, since the process looks like the Stablediffusion - latent space diffusion. So I think it is better to discuss your work with it to highlight the differences and your contributions.\n\n(ii) The experiments are insufficient now. The ablation study only has a visual study (Figure 10). This is far from satisfactory. I suggest you add a break-down ablation like MST, MST++, CST, and DAUHST series works to demonstrate the effectiveness of each technical contribution. Or your claim of your work - the three modules are not convincing since their effects are unknown.\n\n(iii) The computational cost and memory usage of different methods are not reported in Table 1. I cannot judge the efficiency of the proposed method and compare it with other algorithms. I suggest you list the number of parameters and the floating-point operations per second (FLOPS).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oRriVFAAHL", "reviewer": "Reviewer_HqgD", "summary": "The proposed Predict-and-Unmixing-driven Subspace-Refine framework (PSR-SCI) addresses the challenges in Spectral Compressive Imaging (SCI) reconstruction, particularly the difficulty of recovering high-frequency details from ill-posed inverse problems. By leveraging a cost-effective prediction module, unmixing-driven reversible spectral embedding, and pre-trained RGB diffusion models, PSR-SCI demonstrates superior visual quality and competitive performance compared to state-of-the-art diffusion, transformer, and deep unfolding approaches, providing an alternative to traditional deterministic methods.", "questions": "(1) For the reversible unmixing module, although the experiments show that it is lightweight and efficient, I am curious how it compares with SVD, PCA, or band selection-based methods.\n\n(2) It seems the hyperparameters, scale $s$and timestep $T$, shown in Fig. 12, are sensitive to the model's reconstruction accuracy. How do they influence the visual quality of the reconstructed image? The PSNR is not always a reliable metric for measuring image quality, especially for spectral images.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The problem this work aims to solve is crucial. Recovering high-frequency details from ill-posed inverse problems is always challenging for both traditional optimization-based methods and end-to-end deterministic networks. Leveraging the powerful generative capabilities of models pre-trained on large RGB datasets and making them transferable to spectral imaging is a promising approach to address this challenge.\n\nIn addition, I like the idea of using reversible spectral unmixing to bridge spectral image and RGB image. \n\nThe novelty is enough, to address the high-frequency reconstruction challenge in spectral compressive imaging, this paper solved several problems and makes 4 key contributions: (1) the introduction of a spectral unmixing-driven predict-and-subspace refine strategy, offering improved perceptual quality and efficiency; (2) the inclusion of a reversible decomposition module, tackling the ill-posed nature of spectral unmixing; (3) focusing diffusion generation on high-frequency components, reducing training data requirements and accelerating fine-tuning; and (4) employing high-dimensional guidance with SCI imaging consistency to enhance robustness.\n\nMoreover, this method shows nice results on the real SCI dataset, which is a hard case for existing methods.", "weaknesses": "In lines 296-365 of the main paper and Section A.3 of the appendix, most of the mathematical derivations are easy to follow. However, some detailed notations are unclear in the context, which reduces readability.\n\nThe motivation for designing the reversible unmixing module is not sufficiently clear. For example, why not use an SVD-based method? What advantages does the proposed unmixing module offer?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MEjHOeo204", "reviewer": "Reviewer_Z3XV", "summary": "The paper introduces a novel framework called Predict-and-Unmixing-driven Subspace Refinement (PSR-SCI) for Spectral Compressive Imaging (SCI) reconstruction. SCI is an inherently ill-posed problem where traditional methods often struggle to recover high-frequency details. The proposed PSR-SCI framework starts with a cost-effective predictor that provides an initial estimate of the multispectral image (MSI). It then introduces an unmixing-driven reversible spectral embedding module to decompose the MSI into subspace images and spectral coefficients. This decomposition allows the adaptation of pre-trained RGB diffusion models, focusing refinement on high-frequency details efficiently, even with minimal MSI data. Additionally, the authors design a high-dimensional guidance mechanism with imaging consistency to enhance the model's effectiveness. The refined subspace image is reconstructed back into the MSI using the reversible embedding, yielding the final MSI with full spectral resolution. Experimental results on standard datasets like KAIST and zero-shot datasets such as NTIRE, ICVL, and Harvard demonstrate that PSR-SCI enhances visual quality and achieves PSNR and SSIM metrics comparable to some existing diffusion, transformer, and deep unfolding techniques.", "questions": "1. Could the authors elaborate on why the unmixing-driven spectral embedding decomposition is particularly suitable for SCI reconstruction? How does it contribute to faster convergence, higher recovery accuracy, or error reduction compared to other methods?\n\n2. What is the total number of parameters in the proposed model, and how does it compare to other methods? Could the authors provide a detailed analysis of the computational complexity and inference times? How practical is the method for real-world applications given its computational demands?\n\n3. Given that recent works have successfully trained diffusion models on large-scale MSI datasets, how do the authors address the claim of MSI data scarcity? Does this affect the necessity or uniqueness of the proposed approach?\n\n4. To what extent does the high performance of PSR-SCI stem from the large number of parameters and depth of the model, as opposed to the proposed methodology itself? Have the authors conducted experiments to isolate these factors?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The proposed high-dimensional guidance mechanism with imaging consistency is a noteworthy contribution that could enhance the model's performance.\n\n2. The method is evaluated on both standard datasets and zero-shot datasets, showing competitive performance in PSNR and SSIM metrics compared to existing methods.", "weaknesses": "1. The motivation for using the unmixing-driven spectral embedding decomposition is not entirely clear. The paper does not sufficiently justify why this specific decomposition method is suitable for the SCI problem or how it leads to superior performance in terms of convergence speed, recovery accuracy, or error reduction. While the approach is innovative, its superiority over existing methods is not well-established.\n\n2. The claim that there is limited training data available for MSIs compared to RGB images is questionable. Recent works have utilized large-scale MSI datasets to train diffusion models effectively (e.g., \"HSIGene: A Foundation Model For Hyperspectral Image Generation,\" https://arxiv.org/abs/2409.12470, https://github.com/LiPang/HSIGene). This undermines the premise that data scarcity necessitates their approach.\n\n3. The proposed method appears to have a high number of parameters and computational demands. It is unclear whether the performance gains are due to the novel method or simply the result of using a much larger model compared to others. Methods like DAUHST and MST use models with millions of parameters, whereas the proposed method uses billions. This raises concerns about the practicality and efficiency of the approach, especially considering the longer inference times.\n\n4. Despite the innovative approach, the performance of PSR-SCI on the KAIST dataset is reported as 38.14dB PSNR, which is lower than some recent works that achieve PSNR values exceeding 39dB (e.g., PADUT [ICCV 2023], RDLUF-MixS2 [CVPR 2023], and \"Latent Diffusion Prior Enhanced Deep Unfolding for Snapshot Spectral Compressive Imaging\" [ECCV 2024]). This suggests that the proposed method may not offer a significant improvement over existing techniques.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Spotlight)