Unraveling Metameric Dilemma for Spectral Reconstruction: A High-Fidelity Approach via Semi-Supervised Learning
Authors: Xingxing Yang
Jie Chen
Zaifeng Yang
Venue: NeurIPS 2025 poster ()
Keywords: Semi-supervised learning
hyperspectral image reconstruction
diffusion model
Primary area: applications
TL;DR: A  semi-supervised learning approach for high-fidelity spectral reconstruction to deal with the metameric dilemma

Abstract: Spectral reconstruction from RGB images often suffers from a metameric dilemma, where distinct spectral distributions map to nearly identical RGB values, making them indistinguishable to current models and leading to unreliable reconstructions.
In this paper, we present Diff-Spectra that integrates supervised physics-aware spectral estimation and unsupervised high-fidelity spectral regularization for HSI reconstruction.
We first introduce an Adaptive illumiChroma Decoupling (AICD) module to decouple illumination and chrominance information, which learns intrinsic and distinctive feature distributions, thereby mitigating the metameric issue.
Then, we incorporate the AICD into a learnable spectral response function (SRF) guided hyperspectral initial estimation mechanism to mimic the physical image formation and thus inject physics-aware reasoning into neural networks, turning an ill-posed problem into a constrained, interpretable task. 
We also introduce a metameric spectra augmentation method to synthesize comprehensive hyperspectral data to pre-train a Spectral Diffusion Module (SDM), which internalizes the statistical properties of real-world HSI data, enforcing unsupervised high-fidelity regularization on the spectral transitions via inner-loop optimization during inference.
Extensive experimental evaluations demonstrate that our Diff-Spectra achieves SOTA performance on both Spectral reconstruction and downstream HSI classification.

Unraveling Metameric Dilemma for Spectral Reconstruction: A High-Fidelity Approach via Semi-Supervised Learning
Xingxing Yang
Jie Chen
Department of Computer Science Hong Kong Baptist University, Hong Kong csxxyangdcomp hkbu. edu.
Department Computer Science
Hong Kong Baptist University, Hong Kong peanji@@copp hkbu. edu. hk
Zaifeng Yang
Institute of High Performance Computing Agency for Science Technology and Research, Singapore Yang- Lang-zaif star. edu.sg sg
Abstract
Spectral reconstruction from RGB images often suffers from metameric dilemma, where distinct spectral distributions map to nearly identical RGB values, making them indistinguishable to current models and leading unreliable reconstructions. In this paper, we present Diff-Spectra that integrates supervised physics-aware spectral estimation and unsupervised high-fidelit spectral regularization for HSI reconstruction. We first introduce an Adaptive illumiChroma Decoupling (AICD) module to decouple illumination and chrominance information, which learns in- trinsic and distinctive feature distributions, thereby mitigating the metameric issue. Then, we incorporate the AICD into learnable spectral response function (SRF) guided hyperspectral initial estimation mechanism to mimic the physical image formation and thus inject physies-aware reasoning into neural networks, turning an ill-posed problem into a constrained, interpretable task. We also introduce a metameric spectra augmentation method synthesize comprehensive hyperspec- tral data to pre-train Spectral Diffusion Module (SDM), which internalizes the statistical properties of real-world HSI data, enforcing unsupervised sig-fiidlityy regularization on the spectral transitions via inner-loop optimization during in- ference. Extensive experimental evaluations demonstrate that our Diff-Spectra achieves competitive performance both Spectral reconstruction and downstream HSI classification.
Introduction
Hyperspectral imaging captures hundreds of spectral bands, allowing for precise identification of materials and illumination conditions that are often indistinguishable from RGB imaging. has been widely applied in remote sensing 21 medical diagnosis [3 4 and agriculture(5
Traditional hyperspectral imaging methods show limitations in time-consuming acquisition pro- cesses with limited spatial resolution. Recent advances in deep learning within the computer vision community have paved the way for hyperspectral image reconstruction from RGB inputs using data- driven methodologies 7 8). Early model-based methods, such sparse coding [9] and low-rank
Corresponding Author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Specra
Faia CESOT
Sordond
Fasl CESST
Outy
Nalaver
Mermna
Kitamer Indatigaionsy
Fece
Ourmetted
Szrdbrs
Messrerr
Netamer
Miedlnngh June
(a) HSI manifold
(b) RGB manifold
(c) Recovered HSI manifold
(d) An example of the metameric dilemma
Figure 1: Motivation. (a) shows metamer and standard HSIs. In (b), standard and metamer RGB inputs are produced by the same SRF from corresponding HSIs. In (d), Left: the dashed and solid red highlighted points are two pixels from the fake and real peppers from the CAVE dataset [[9], respectively; Right: Corresponding ground-truth spectral curves of the two pixels, and reconstructed spectral curves of CESST [8] (grey) and our method (blue). The fake and real peppers are similar colors but have different spectra. Existing methods (e.g., CESST) fail to distinguish either the two spectra from each other or from their true spectra, while our method can reconstruct faithfully.
representation [0] rely predominantly on manually crafted priors. In contrast, modern learning- based approaches 7 12 |3 leverage deep learning frameworks such as convolutional neural networks 14 15
and Vision Transformers
achieve superior reconstruction performance.
Limitations. However, we observe that most existing methods [ [2] suffer from a metameric dilemma [8), where distinct spectral distributions map to nearly identical RGB values, making them indistinguishable to current models and leading to unreliable reconstructions, as shown in [ We argue that this stems from the direct nonlinear mapping from the sRGB input space to the hyperspectral output space, leading to three key limitations in terms of data, methodologies, and pipelines: (i) Data Poverty: Existing hyperspectral datasets lack diversity in rich metameric examples, forcing existing models to memorize a limited subset of spectra rather than internalizing the full spectral manifold; (ii) Spectral Manifold Blindness: Existing methods overlook the explicit modeling of the real-world spectral distribution, leading to hallucinated outputs that minimize pixel-wise losses (e.g., MSE) but violate physical laws (e.g., unnatural spectral intensity Fig. []] (iii) Architecture Myopia: Existing methods treat spectral radiance as a monolithic entity, conflating illumination and chrominance. It amplifies metameric failures that networks cannot distinguish whether RGB variations stem from lighting or material properties.
It raises fundamental question: Can introduce auxiliary information address the above- mentioned metameric dilemma, thus achieving high-fidelity Spectral reconstruction? If so, how?
To this end, we propose Diff-Spectra, semi-supervised model for HSI reconstruction that integrates supervised physics-aware spectral estimation and unsupervised nigh-fidelity spectral regularization. Specifically, to deal with architecture myopia, first introduce Adaptive IllumiChroma Decou- pling (AICD) module by factorizing RGB inputs into independent illumination and chrominance subspace via orthogonal decoupling. Then, the AICD is incorporated into a SRF-guided HSI initial estimation (SRF-guided HIE) mechanism to estimate the target spectral signal a supervised manner. This process mimics the physical image formation and thus injects physics-aware reasoning into neural networks, turning an ill-posed problem into a constrained, interpretable task. To deal with the data poverty, introduce metameric spectra augmentation method to synthesize comprehensive HSI dataset with diverse metamer samples and spectral perturbations, transforming sparse spectral data into rich prior that can guide reconstruction beyond RGB ambiguities. To deal with the spectral manifold blindness, we introduce a spectral diffusion module (SDM) that learns to denoise corrupted spectra during pre-training on the comprehensive HSI dataset, which internalizes the statistical proper- hyperspectral data. During reconstruction. the pre-trained SDM serves as a spectral
ties of prior that regularizes the coarsely estimated HSI signal from the SRF-guided HIE mechanism with high-fidelity real-world spectral distributions via our proposed unsupervised inner loop optimization. The main contributions are given as follows:
We propose semi-supervised paradigm, Diff-Spectra, to deal with the metameric dilemma in spectral reconstruction. integrates supervised physics-aware spectral estimation and unsupervised high-fidelity spectral regularization.

---

We introduce an Adaptive IllumiChroma Decoupling (AICD) module based on orthogonal decoupling to effectively factorize illumination and chrominance information, serving as an IllumiChroma prior. It learns distinctive image features, alleviating the metameric dilemma.
We design a lightweight, learnable SRF-guided HIE mechanism to obtain an initial HSI esti- mation, which formulates the reconstruction model with physical constraints and enhances the interpretability.
We introduce metameric spectra augmentation method to synthesize a comprehensive HSI dataset to pre-train a Spectral Diffusion Module (SDM) to capture the real-world spectral distribution, serving as a spectral prior to improve spectral fidelity.
Extensive experiments on both spectral reconstruction and HSI classification demonstrate that our Diff-Spectra framework significantly outperforms SOTA methods.
2 Related Work
2.1 Hyperspectral Image Reconstruction
Early efforts [6] utilized model-based based approaches that incorporated fidelity terms and physical priors constrain the target solutions. For example, Arad et al. [9] improved this interpolation challenge using hyperspectral priors to forge a sparse dictionary of HSIs alongside their RGB counterparts. Despite their contributions, these model-based strategies depend on manually tailored priors, constricting their representational capability. Learning based methods [1] shifted the focus to data-driven approaches, learning implicit mappings from RGB to hyperspectral domains using specifically designed architectures. Notably, the HSCNN model [4] revolutionizes the field by mapping the input RGB image into hyperspectral feature space via a convolutional layer and harnessing deep residual convolutional blocks approximate the enriched HSI. Cai al. M7 proposed a sunsformer-basee approach to capture the long-range channel-wist correlations that compute the self-attention map along the channel dimension, tailored for HSI reconstruction. Wang et al. [22] introduced an intrinsic image decomposition (IID) framework to decompose input images into reflectance and shading features and then reconstruct them in the spectral domain separately. However, all existing learning-basee methods directly learn mappings between sRGB and HSI feature spaces, neglecting the intrinsic spectral distribution of HSIs and lacking physical constraints, thus encountering the challenges posed by the metameric dilemma.
12 20 21].
2.2 Diffusion-based HSI Image Reconstruction
Diffusion models 23 have witnessed an explosion of continuously growing capability and capac- ity architectures. In the context of HSI image reconstruction, Pang et al. (25) proposed HIR-Dit that leverages a powerful pre-trained diffusion prior and a product-of-expert. guidance scheme to remove degradations and recover clean hyperspectral images in an unsupervised manner, demonstrating strong generalization across scenes and noise types. Wu et al. (26) proposed a conditional denoising transformer to fuse high-resolution multi-spectral images and low-resolution hyperspectral images to generate target high-resolution HSI images. Liu et al. (27] proposed incorporating the deep generative prior of diffusion models constrain the high-resolution multi-spectral image and low-resolution hyperspectral image fusion process. While these works focus on restoration and reconstruction, diffusion's 's representational advantages have also benefited HSI classification, suggesting transferable priors for reconstruction. Chen et al. (88 proposed a spatial-spectral diffusion module to generate nigh-dimensional HSI signals for HSI classification, indicating the great potential of diffusion-basee generative models in spectral distribution modeling. Beyond natural HSI classification, Sigger et al. [29] proposed multistage unsupervised diffusion framework extract complementary high- and low-level spectral features for challenging biomedical HSI classification, underscoring diffusion's 's capacity to model fine-grat -rained spectral structure. However, applying diffusion models for practical HSI systems still faces trade-off's among cost, complexity, and acquisition speed. Moreover, directly reconstructing full hyperspectral cubes from RGB inputs with diffusion remains comparatively under- explored, presenting an opportunity combine diffusion priors with cross-modal spectral constraints and measurement-an conditioning for faithful RGB-to-HS) recovery grounded both spectral accuracy and spatial details.
3

---

(a) Siff-Spectra
Spectral Respors se Furction
SDM
intte Stage
RGB Image
AICD
Fuse
SRF guided HIE
Came
Conox: txt
(b) Adaptive IllumiChroma Decoupling (AICD)
(c) Spectral Diffusion Model (SDM)
Stage
Forword process
( 1%-1)
Roa krong
Xg
3
Mp-1
Roverse procesa
Figure 2: Overall framework of Diff-Spectra. Our model has three training stages. In Stage 1, the SRF-guided HIE and AICD pre-trained using Eq. to initialize coarse-level HSI signal while freezing the SDM. In Stage 2, the SDM is pre-trained using T3 learn the spectral distribution of HSI signals while freezing the SRF-guided HIE and AICD. In Stage
fine-tune the SRF-guided HIE and AICD using the objective function Eq. [4] which regularizes the initialized HSI signal in Stage with the spectral prior of SDM in Stage 2. loop optimization as a test-time adaptation with Eq.
During inference, further introduce inner 13 to generate refined HSI signals by following
the spectral reverse generative sequence of SDM.
3 Methodology
3.1 Problem Definition
The RGB imaging process be formulated sub-sampling process of the target HSI signal:
X YA + A cxe,
(I)
where X Hx denotes the observed RGB image, with H, W and representing height, width, and channel (c <<<<<s C)) of X, respectively. Y E ##xWCC xC and A E RCCx are the target HSI signal and the spectral response function (SRF) of the RGB sensor, respectively. is the noise residual terms that arise in the image processing pipeline. The HSI reconstruction task can be formulated as maximum posteriori problem: maxy p( Y X, A). Applying Bayes' theorem, the posterior can be further reformulated as:
X = P(X, Y)p(Y)
p(X, A)
Taking the negative logarithm and discarding irrelevant terms X, A, we obtain the objective function:
min log p X, A Y) log (Y)),
where log p(X, A Y) is the log likelihood that depicts the degradation processes of RGB sampling from the HSI signal, and is the spectral prior that will contribute to the restoration Y.
3.2 Motivation and Solution
Existing HSI reconstruction methods 7 only consider the first term in Eq, (i.e., log p(X, A Y)) and directly map RGB inputs to hyperspectral feature space, overlooking subtle variations RGB feature distributions, thereby leading to the metameric dilemma, as shown in Fig. [ and further demonstrated by the experimental results Section 4.3
In this paper, we would like to consider this challenging spectral reconstruction problem as supervised physics-aware spectral estimation and unsupervised high-fidelity spectral regularization. Thus, in terms of supervised physics-aware spectral estimation, which corresponds to to optimize the first

---

term -logp(X, A Y) in Eq. (.). we introduce an Adaptive IllumiChroma Decoupling (AICD) module by factorizing RGB imputs into independent illumination (U) and chrominance (V) subspace via orthogonal decoupling to explore intrinsic clues and distinguish image features, which effectively eliminates redundancy and enhances distinctive patterns among metameric hyperspectral data. Then, the AICD is incorporated into the SRF-guided HSI Initial Estimation (SRF-guided HIE) module to estimate the target spectral signal in a supervised manner. This process mimics the physical image formation and thus injects physics-aware reasoning into neural networks, turning an ill-posed problem into a constrained, interpretable task. Now, we express the first term log (X, A Y) as the following according to Eq. !
(r I(x, V) - YA- - -ll
Then, assume the second prior term, i.e., log pl Y), as unsupervised high-fidelitg spectral regular- ization for the estimated spectra, which further constrains the estimation via transferring distribution prior knowledge from a pre-trained model leveraging our metameric spectra dataset.
3.3 Supervised Physics-Aware Spectral Estimation
3.3.1 Adaptive IllumiChroma Decoupling
spectral radiance (A)) can be represented as Y(A) - E(A)S(A), where, X, E()), S(X) represents the spectral entity, illumination and surface chrominance. respectively. Existing deep learning methods treat ()) as monolithic entity, failing to disentangle illumination and chrominance. This conflation amplifies metameric ambiguities, as networks struggle to distinguish whether RGB variations stem from lighting changes ((A) or material properties S(A). This dilemma is the same with RGB inputs. As such, introduce AICD that aims to extract intrinsic and distinctive image features that are robust against different light variations. However, as shown in Fig. 3 we find that merely relying reflectance R and illumination L decomposition based on the Retinex theory (30 cannot distinguish between metameric examples.
Considering that HSIs capturing often exhibit low-light characteristics that lead to potential informa- tion loss, we first customize an illumination sensitivity parameter Sk. which enables image-specific
adjustment as: S = V ( L
T, where k E ++ 1 x 10-8, In specific, different from exist-
ing decomposition methods 31 32 33 that decompose images in a deterministic manner, which may not appropriately capture the diverse and complex imaging and lighting conditions that are specific to downstream tasks, we introduce trainable parameters to enable adaptive learning of intrinsic image fea- tures in an end-to-end manner, thereby enhancing compatibility with downstream tasks. Specifically, we employ convolutional layers to obtain embedded features: R = Cono(R), L = Cono(L).
Orthogonal Decoupling, As discussed, decom- posed reflectance and illumination components cannot well distinguish metamer data. Inspired by the orthogonal decoupling method in [34 33),
which effectively eliminates redundancy and en- hances distinctive patterns among visual fea- tures, we further deployed orthogonal UV trans- form to extract more distinctive features. We define the horizontal (U) and vertical (V) plane as:
Indistinct
Indistinet
Distinct
Stand/Metamer
Illumination
Reflectance
AICD
Figure 3: Difference between the decomposition
methods of Retinex and AICD, The top row is based on standard data, while the bottom row based on metamer data. AICD can transform metamer counterparts into more discriminative fea- tures than reflectance and illumination maps.
R Dv, (5) where @ denotes Hadamard production. Note that we orthogonalize the UV-planes using the two intermediate variables = cos(2.RR) ) and w 22m(2 ). Finally, the decomposed features (U, and V) will serve as the illumichroma prior for SRF-guided HIE.
U = L= Rh,
3.3.2 SRF-guided HSI Initial Estimation
Note that the SRF matrix in Eq. [ that projects the HSI to an RGB frame has a row-full rank, enabling right-inverse solution based on the matrix inversion rule. Thus, there exists transpose
5

---

matrix A of A such that the following equation obtained from Eq. holds
XA" -AA YAA"
(6)
A L is square matrix, and there exists an inverse matrix of A A such that the following equation holds:
XA (AA(AA")= - (AAA (AA")
We can rewrite Eq.
as
-
(8)
Direct inversion of RGB images into spectral space via SRFs may be suboptimal due variations in SRFs across different spectral cameras and disturbances arising from factors such as RGB format compression and noise, and can not distinguish metamers. To address these challenges, we incorporate the AICD into a spectral response function (SRF) guided HSI initial estimation (SRF-guided HIE) mechanism estimate the target spectral signal supervised manner. This process mimics the physical image formation and thus injects physics-an aware reasoning into neural networks, turning an ill-posed problem into a constrained, interpretable task. Specifically, we incorporate the decomposed illumichroma prior from the AICD module and then estimate the coarse HSI signal via the SRF-guided HIE function:
Y F(H(X, U, V)A (A
where F denotes neural network mapping function (a standard UNet- based network (7). H denotes fusion operator (via concatenation and convolution) and Y denotes the SRF-guided HIE output. Now, Eq. 4 is implemented as:
Cy Cy = Y F(H(X, U, U, V)A" (AA")
(10)
3.4 Unsupervised High-Fidelity Spectral Regularization
We introduce spectral diffusion models (SDMs) [2 36] as spectral priors to refine the initial HSI estimated by the SRF-guided HIE and achieve accurate reconstruction. However, three key challenges must be addressed: (i) SDMs struggle to capture metameric feature distributions, limiting their ability to model subtle spectral variations.
(ii) Directly modeling the spatial-spectral distribution of 3D hyperspectral data incurs a significant computational burden due to the high dimensionality of the data. (iii) A notable domain discrepancy exists between SRF 3RF-guided HIE results and real hyperspectral images (HSIs), suggesting that directly applying SDMs may result in suboptimal performance. To address these challenges, we introduce three key improvements for effectively integrating SDMs into spectral reconstruction: (1) metameric spectral augmentation, which creates comprehensive training data to enhance feature representation, (2) a lightweight spectral diffusion architecture to reduce computational complexity, and (3) gradient-basee inner loop optimization to bridge the domain gap and improve reconstruction accuracy.
Metameric spectra augmentation. Metameric augmentation leverages an orthogonal subspace decomposition (37 of a spectrum S into component that lies the sensor-responss range subspace and residual in the null space, enabling different spectra to produce the same RGB response under given spectral sensitivity [38] With spectral response matrix A, a new metameric spectrum St can be synthesized following the concept of metameric black (39
Si J + BJ1,
(11)
where we project S onto the range space as J A(ATA) "S and define the residual null space as JL = J. New metamer spectra are then synthesized by varying the scalar 3. Because Jt lies in the null space of the response, changing 3 does not alter the RGB tristimulus, so S and Si are colorimetric matches while differing spectrally. This property reflects the broader fact that rich spectra are reduced to three sensor channels in trichromatic systems, making metamers common and exploitable for augmentation.
We adopt this as a spectral-wise augmentation (rather than spatial flips or crops) to expand the diversity of training spectra while preserving color consistency seen by RGB sensors. Such augmentation can enhance robustness in spectral reconstruction pipelines, a concept established in color science as a way to explore spectra that map to the same color. Practically, we generate metamer data by sampling 3 uniformly from [0, 1); note that setting 3 recovers the original spectrum, while other

---

values yield alternative metamers that share the same RGB under A. We produce metamers at 1 ratio with standard spectra, doubling the dataset and providing both the original and augmented hyperspectral inputs for pre-training.
Lightweight spectral diffusion architecture. HSI signals exhibit significant spatial sparsity 40
suggesting that direct modeling of 3-D data cubes might be suboptimal. As such, we introduce 1-D spectral diffusion model to capture the spectral distribution HSI signals address the second term in Eq. 33. Our SDM employs two iterative processes, following the standard DDPM [2],
forward diffusion process, and a reverse denoising process, as illustrated in Fig. 2 Unlike existing
diffusion-based variations [33 28], we adopt 1-D MLP-basee UNet denoising network to ensure
that the diffusion model is compatible with I-D spectral data. While it is feasible to capture the spectral distribution of HSI signals via our proposed SDM, the key question is: how can the trained SDM be incorporated solve the second prior term log (Y) of Eq. B
In our work, we assume that any spectral vector y E R of the target HSI Y are i.... (independently identically distributed), i.e., - -Y) = Σyex log)(y), and each spectrum sample follows the spectral distribution learned by our proposed SDM, the deep generative prior, i.e., = Yo ~ ((Yo). Consequently, the optimization problem Eq. (3) can be rewritten as:
min (Y)
Σ 1.y
(12)
where À and Lai denote a balance hyperparameter and the KI. divergence, respectively. Subsequently, we can use the parameterization trick to rewrite the second term in Eq. 3 as
(13)
With the above derivation, the final objective function of the proposed Diff-Spectra can be formulated
min Lx (Y) + + Σ Lo(y,
(14)
1.y
Note that this objective function is adopted during the fine-tuning process in Stage where À is set to 0.1 empirically.
Inner loop optimization. It is impractical to optimize Eq. simultaneously for all time steps. This is because an inherent domain gap exists between the spectral distribution learned by the SDM and the spectral distribution of the coarse-level HSI learned by the SRF- SRF-guided HIE network. Simply assuming these two spectral distributions are consistent without further adaptation can lead to suboptimal results. As such, propose inner loop optimization during each time step in the inference phase that performs gradient descent K times for each t, which serves as the test-time adaptation.
4 Experiments and Analysis
4.1 Implementation Details and Datasets
We implemented Diff-Spectra using Pytorch. In Stage 1, we pre-train the SRF-guided HIE and AICD on the training dataset for 300 epochs with the Adam optimizer following [7]. Empirically, we set the learning rate 4 x 10-1 and the batch size is 20. In Stage 2, we pre-train the SDM on our generated metamer dataset for 300 epochs with the Adam optimizer. The learning rate is set to 1 x 10-2 The batch size is 1024, and the total diffusion steps T is 5000. In Stage 3, we fine-tune the SRF-guided HIE and AICD, while freezing the SDM for 100 epochs with the Adam optimizer. Empirically, we set the learning rate to 1 x 10 and the batch size is 20. During inference, we input RGB images from the testing dataset and load the pre-trained SRF-guided HIE and AICD to obtain coarse-level HSIs. We then treat the coarse-level HSIs as learnable parameters and load the pre- pre-trained SDM to update the parameters and generate refined HSIs, using a learning rate of 1 x 10-4 with diffusion steps of = 50 and inner loop = 5. To evaluate the generalization and fidelity of our method, we use two HSI reconstruction datasets (ARAD-IK (3 and ICVI (9) and two classification datasets (Indian Pines [ and Pavia University 44 Further details for implementation and dataset descriptions are provided in the Appendix A.2
7

---

km0mm
RGB Image
RGB Posh
on
AWAN
HDNet
HDNel
MST-L
MST+
BLDUT
CESST
SPECAT
DXF-Spectra Groun
Figure 4: Visual comparisons on randomly selected scene from the validation set of the ARAD-IK dataset with 4 spectral channels. The spectral curves (bottom-left) correspond to the selected blue Box of the RGB image. Please zoom in for better comparison.
Table 1: Comparison with SOTA methods ARAD-18 1K 43 and ICVI. [ datasets. The best and
second are shown in red and blue, respectively. Our method achieves the best performance on most metrics with relatively fewer parameters.
Vense
(M)
ERGAS (1)
SAM (i)
ARAD>-K
SSIM (1)
PSNR (1)
ERGAS
SAM(L)
SSIM (1)
PSNR
17.45
HDNe
40
CVPR'22
2.66
11.16
10.21
0.913
32.13
7.58
1.17
0.921
31.85
HINel 07
CVPR'21
5.21
15.16
8.18
0.916
32.51
7.93
4.19
0.928
32.01
MST-1
48
CVPR'22
2:45
12.53
7.47
0.922
33.90
5.44
3.51
0.335
32.87
MST
CVPRW"22
1.62
34.32
3.04
ICCV
SST
USWIS'24
12.74
8.39
4.67
CESST X
AAAU24
1.54
7.85
3.88
0.311
34.74
4.09
3.27
0.939
32.96
SPECAT
57
CVPR'24
0.37
8.62
6.10
0.930
33.48
4.92
3.81
0.944
32.54
Ditf Spectral (owr))
2.19
4.63
3.96
0.910
35:47
2.84
2.39
0.911
34.71
4.2 Hyperspectral Image Reconstruction
We present quantitative and qualitative comparisons with state-of-the-ard methods including AWAN [45,, HINet [47), HDNet [46], MST-L [48], MST++ [7.SST [50], PADUT 49), CESST g and SPECAT (1]
Qualitative Comparison. Visual Comparisons are given in Fig. Fadd Fig. 5 Fig. compares the reconstructed HSIs with four randomly selected spectral channels using nine SOTA methods and our Diff-Spectra on the validation set of the ARAD- IK dataset. Fig. shows the MSE error map between generated and ground-truth HSIs, calculating along the spectral dimension. is observed that existing HSI reconstruction methods struggle with spectral intensity estimation and detail recovery, particularly in regions with high-frequent details such the sky. In contrast, approach excels at restoring intricate textures and achieving superior pixel-Ievel smoothness. This improvement is attributed the novel illuniChroma prior learned by the AICD module and the spectral prior learned by the SDM. The AICD module facilitates intrinsic image decomposition, such as illuminance and color information, guiding the initial estimation of the HSI signal with perceptually pleasing spatial features, while the SDM captures spectral distributions that refine the coarse-level HSI generated by the SRF-guided HIE, ensuring spectral consistency.
Quantitative Comparison. We evaluate the performance using metrics including ERGAS, SAM, SSIM, and PSNR. The first two metrics assess spectral quality, while the latter two evaluate spatial quality. Lower ERGAS and SAM values indicate better spectral quality, while higher SSIM and PSNR values signify better spatial quality. As shown in Table I our method achieves the best performance over most metrics on both the ARAD- IK dataset and the ICVL dataset. Our approach, Diff-Spectra, outperforms state-of-the-an methods by delivering the highest PSNR and lowest ERGAS (best. spectral and spatial reconstruction quality) with significantly lower computational complexity.
4.3 Metameric Dilemma Evaluation
To demonstrate that existing methods suffer from the metameric dilemma and validate the effec- tiveness of our proposed Diff-Spectra in mitigating this issue, we generate metamer HSI data from

---

AWAI
HDN
HINK
MST-
MST++
PADUT
SST
CESST
SPECAT
Diff Spectra
Figure 5: The MSE error maps obtained from the validation subset of the ICVI dataset, which are calculated along the spectral direction, showcasing the discrepancies between the reconstructed HSIs and the corresponding ground truths.
Table Metameric dilemma Evaluation and HSI classification comparison.
Mathod
Data
ERGIAS
SAM
SSIM
PSNR 1
Class No.
AAAAN [75]
MST++
PADUT (19
MST (5]
Ours
MST++ N
meta
84.77
9.18
50.92
6.05
0,8896
0.928
27.14
34.32
2
59.36
82.71
$4.27
51.45
87.49
92.35
90.47
96.56
91.45
92.73
PADUT
19
110
80,077
7.39
42.91
3.35
0.96
27.90
34.31
80.00
73.65
95,
(41.3
91.87
62.54
89
$8.47
89.71
89.97
std
99.06
100.00
moca
70.11
52.38
CESST
mita
sid
59.82
7,85
40.74
5.88
0.85
0931
78.59
34.14
70.43
69.30 39
SPECAT
11
48.90
8,02
63.59
6.10
0.8701
00230
26.63
13.48
94.83
90.52
94.88
$2.64
89.02
98:44
Spectral (ours)
meea
24.11
(18)
0.7124
0.8349
(a) Comparison with SOTA methods standard (std) (b) Quantitative comparison of different methods in terms of the accuracy for each class.
and metamer (meta) data.
the original ARAD-IK dataset following 5 and metamer HSI data synthesize corresponding RGB images (i.e., metamer). Next, we test several pre-trained models using both standard RGB images and metamer RGB images (Note that these models pre-trat pre-trained on standard data), including MST+ 77 CESST 8 PADUT [49] SST [50], SPECAT [51], and our proposed Diff- ipeetrpectra. The quantitative results given in Table 2(a). As can be seen, all the existing methods experience catastrophic performance drops in terms of PSNR and SAM in the presence of metamers, which is also known the metameric dilemma.
4.4 Evaluation on HSI Classification
To further evaluate the fidelity and verify the reliability of the hyperspectral images generated by our method, conduct experiments the hyperspectral image classification task based on a pre-trained HSI classification model, SpectralFormer [53]. We compare our approach with existing methods on two widely used HSI classification datasets: the Indian Pines dataset and the Pavia University dataset, conducting both quantitative and qualitative analyses. For our evaluation, we reconstruct HSI images using assorted pre-trained HSI reconstruction methods. Subsequently. these synthesized HSI images are employed as the input to pre-trained HSI classification model, SpectralFormer [53] which serves benchmark for performance evaluation.
Evaluations. We evaluate the quantitative performance on the Pavia University dataset using three widely adopted metrics: Overall Accuracy (OA), Average Accuracy (AA), and the Kappa Coefficient (k), as shown in Table 2(b). As can seen, our method achieves the best OA (89.02%), AA (88.15%), and x (0.8349). It ranks first in Classes 3, and 9 and ties for first in Class while remaining competitive the remaining classes.
4.5 Ablation Study
Break-down Ablation. We perform bread-down ablation to investigate the effectiveness of each module in Table 3(a) and Figure Comparing Variant with SinDiff-Speetra (i.e., a UNet-based HSI reconstruction network, which is similar to (7]. we find that the AICD primarily contributes to the spatial details recovery of the HSI signal, which aligns with our original design intention. Comparing Variant with Variant 2, we observe that the SRF-guided HIE mechanism enhances

---

Table 3: Ablation studies of our proposed modules and inner loop mechanism.
Method
AICD
SRF
SDM
PSNR
SAM
Setting
K
PSNR
SAM
SinDiff--ppectal
32.57
7.52
Dift-S Spectrall
1
34.19
5.84
Variant
34.18
6.08
Diff- Diff-Spectral
4
35.41
4.07
Variant
34.95
5.41
Diff- Diff-Spectral
5
35.47
3.96
Variant
34.19
5.84
Dift- Diff-Spectral
7
35.25
4.31
Dift- Siff-Spectrall
35.47
3.96
Diff-Spoctral
10
35.01
4.33
(a) Break-down ablations of our proposed Diff-Spectre where de-(b) The right sub-table investigate the ner loop steps (K) in the SDM.
notes the inner loop.
both spatial and spectral performance due to the incorporation of physical constraints. Notably, by comparing Variant with Variant we find that simply incorporating SDM causes severe performance drop. Finally, comparing Variant with the full Diff-Spectry model, we find that the inner loop facilitates effective integration SDM and primarily contributes spectral recovery via the learned spectral distribution regularization.
Inner Loop Analysis. We analyze the impact of the inner loop step parameter K of the SDM in Table 3(b). The results indicate that performance improves significantly when K is greater than I as compared when K
Note that when K the performance is even worse than Variant This is because a domain gap exists between the spectral distribution learned by the SDM and the spectral distribution of the coarse-level HSI learned by the SRF-guided HIE network. Assuming these two spectral distributions are consistent without further adaptation can lead to suboptimal results. Directly assuming these two spectral distributions are consistent will introduce an inferior influence.
RGB
Variant
Variant
Variant
Diff-Spectra
Figure 6: Break- ablation study. The reconstruction MSE error map evaluation on the validation set of the ICVL dataset.
5 Conclusion
In this paper, propose Diff-Speetra, which integrates supervised physics-aware spectral estimation and unsupervised high-fidelity spectral regularization for spectral reconstruction. Especially, the supervised physics-aware spectral estimation consists of an adaptive illumichroma decoupling (AICD) and learnable SRF-guided HIE mechanism, mimicking the physical image formation, and thus in- jecting physics-aware ahysics-awwre reasoning into neural networks, turning an ill-posed problem into a constrained, interpretable task. We further introduce unsupervised pre-trained trained spectral diffusion model (SDM) to regularize the coarsely estimated HSI signal from the SRF-guided HIE mechanism with high-fidelity real-world spectral distributions. Extensive experiments on both spectral reconstruction and HSI classification demonstrate that Diff-Spectra significantly outperforms SOTA methods. Future work will focus on the investigation of the spectral distribution gap between the HSI estimated by the SRF-guided HIE and the distribution learned by the SDM, such as quantizing each distribution into dictionary and calculating their distance.
spectral regularization by incor-
porating pre-
10

---

References
[1] Farid Melgani and Lorenzo Bruzzone. Classification of hyperspectral remote sensing im- ages with support vector machines. IEEE Transactions Geoscience and Remote Sensing, 42(8):1778-17999 2004.
[2] Yuan Yuan, Xiangtao Zheng, and Xiaoqiang Lu. Hyperspectral image superresolution by transfer learning. IEFE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 1067:1963-1974, 2017.
[3] William R Johnson, Daniel Wilson, Wolfgang Fink, Mark Humayun, and Gregory H Bearman. Snapshot hyperspectral imaging in ophthalmology. Journal of biomedical optics, 12(1):014036, 2007.
[4] Guolan Lu and Baowei Fei. Medical hyperspectral imaging: a review. Journal of biomedical optics, 19(1)001900,, 2014.
[5] Telmo Adão, Jonás Hruška, Luis Pádua, José Bessa, Emanuel Peres, Raul Morais, and Joaquim Joao Sousa. Hyperspectral imaging: A review on uav-based sensors, data processing and applications for agriculture and forestry. Remote sensing, 9(11::1110, 2017.
[6] Hui Huang, Li Liu, and Michael O Ngadi. Recent developments in hyperspectral imaging for assessment of food quality and safety. Sensors, 14(4):7248-7276 2014.
[7] Yuanhao Cai, Jing Lin, Zudi Lin, Haoqian Wang, Yulun Zhang, Hanspeter Pfister, Radu Timofte, and Luc Van Gool. Mst++: Multi-stage spectral-wise transformer for efficient spectral reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 745 2022.
[8] Xingxing Yang, Jie Chen, and Zaifeng Yang. Hyperspectral image reconstruction via combina- torial embedding of cross-channel spatio-spectral clues. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 6567-6575, 2024.
[9] Boaz Arad and Ohad Ben-Shahar. Sparse recovery of hyperspectral signal from natural rgb images. In European conference on computer vision, 2016.
[10] Shipeng Zhang, Lizhi Wang, Lei Zhang, and Hua Huang. Learning tensor low-rank prior for hyperspectral image reconstruction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 22006-12055 2021.
[11] Lizhi Wang, Tao Zhang, Ying Fu, and Hua Huang. Hyperreconnet: Joint coded aperture opti- mization and image reconstruction for compressive hyperspectral imaging. IEEE Transactions on Image Processing, 2855):2277 2018
[12] Lizhi Wang, Chen Sun, Ying Fu, Min H Kim, and Hua Huang. Hyperspectral image recon- struction using deep spatial-spectral prior. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8032 8041, 2019.
[13] Xingxing Yang, Jie Chen, and Zaifeng Yang. Cooperative colorization: Exploring latent cross- domain priors for nir image spectrum translation. In Proceedings of the 3 ACM International Conference Multimedia, pages 2409-2411 2417, 2023.
[14] Zhiwei Xiong, Zhan Shi, Huiqun Li, Lizhi Wang, Dong Liu, and Feng Wu. Hsenn: Can-based hyperspectral image recovery from spectrally undersampled projections. In Proceedings of the IEEE/CVF international conference computer vision workshops, pages 518-525, 2017.
[15] Yigit Baran Can and Radu Timofte. An efficient enn for spectral reconstruction from rgb images. arXiv preprint 18040047744 2018.
[16] Antonio Robles- Kelly. Single image spectral reconstruction for multimedia applications. In Proceedings of the 23rd ACM international conference Multimedia, pages 251-260, 2015.
Kobles-Kelly,
[17] Bernhard Hill. Color capture, color management, and the problem of metamerism: does multispectral imaging offer the solution? Color Imaging: Device- Indeeee-ndeenneent Color, Color Hardcopy, and Graphic Arts V. volume 3963, pages 2-14. SPIE, 1999.

---

[18] Samuel Ortega, Martin Halicek, Himar Fabelo, Gustavo M Callico, and Baowei Fei. Hyper- spectral and multispectral imaging in digital and computational pathology: a systematic review. Biomedical Optics Express, 1 1166):11553323 2020.
[19] Fumihito Yasuma, Tomoo Mitsunaga, Daisuke Iso, and Shree K Nayar. Generalized assorted pixel camera: postcapture control of resolution, dynamic range, and spectrum. IEEE transactions on image processing, 19(9):2241- -2233, 2010.
[20] Zhan Shi, Chang Chen, Zhiwei Xiong, Dong Liu, and Feng Wu. Hsenn+: Advanced cnn-based hyperspectral recovery from rgb images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages (9999947,001
[21] Biebele Joslyn Fubara, Mohamed Sedky, and David Dyke. Rgb to spectral reconstruction via learned basis functions and weights. In Proceedings the IEEE/CVF Conference Computer Vision and Pattern Recognition Workshops, pages 480-481, 2020.
[22] Nan Wang. Shaohui Mei, Yifan Zhang, Mingyang Ma, and Xiangqing Zhang. Hyperspectral image reconstruction from rgb input through highlighting intrinsic properties. IEEE Transactions on Geoscience and Remote Sensing, 2024.
[23] Florinel-Alin Croitoru, Vlad Hondru, Radu Tudor Ionescu, and Mubarak Shah. Diffusion models in vision: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(9):10850-1088 10869, 2023.
[24] Yuchun Miao, Lefei Zhang, Liangpei Zhang, and Dacheng Tao. Dds2m: Self-supervised denoising diffusion spatio-spectral model for hyperspectral image restoration. In Proceedings the IEEE/CVF International Conference on Computer Vision, pages 1 12086-12096, 2023.
[25] Li Pang, Xiangyu Rui, Long Cui, Hongzhong Wang, Deyu Meng, and Xiangyong Cao. Hir-diff: Unsupervised hyperspectral image restoration via improved diffusion models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 3005-3014, 2024.
[26] Chanyue Wu, Dong Wang, Yunpeng Bai, Hanyu Mao, Ying Li, and Qiang Shen. Hsr-diff: Hyperspectral image super-resolution via conditional diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7083-7093, 2023.
[27] Jianjun Liu, Zebin Wu, and Liang Xiao. A spectral diffusion prior for unsupervised hyperspectral image super-resolution. JEEE Transactions on Geoscience and Remote Sensing, 2024.
[28) Ning Chen, Jun Yue, Leyuan Fang, and Shaobo Xia. Spectraldiff: A generative framework for hyperspectral image classification with diffusion models. IEEE Transactions on Geoscience and Remote Sensing, 2023.
[29] Neetu Sigger, Tuan T Nguyen, and Gianluca Tozzi. Brain tissue classification in hyperspectral images using multistage diffusion features and transformer. Journal of Microscopy, 2024.
[30] Edwin H Land. The retinex theory of color vision. Scientific american, 138(6):108-12) 1977.
[31] Yuanhao Cai, Hao Bian, Jing Lin, Haoqian Wang, Radu Timofte, and Yulun Zhang. Retinex- former: One-stage retinex-basee transformer for low-light image enhancement. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2550-12513, 2023.
[32] Zhenqi Fu, Yan Yang, Xiaotong Tu, Yue Huang, Xinghao Ding, and Kai- KuinK Ma. Learning simple low-light image enhancer from paired low-light instances. In Proceedings of the JEEE/CVF conference on computer vision and pattern recognition, pages 22252-22261, 2023.
[33] Xunpeng Yi, Han Xu, Hao Zhang, Linfeng Tang. and Jiayi Ma. Diff-retinex: Rethinking low- light image enhancement with a generative diffusion model. In Proceedings of the IEEÉ/CVF International Conference on Computer Vision, pages 12302-12311, 2023.
[34] Zhen Zhao, Zhizhong Zhang, Xin Tan, Jun Liu, Yanyun Qu, Yuan Xie, and Lizhuang Ma. Rethinking gradient projection continual learning: Stability/plasticity feature space decoupling. In Proceedings of the JEFE//VV Conference on Computer Vision and Pattern Recognition, pages 2023. 2023.
12

---

[35] Zhongze Wang, Haitao Zhao, Jingchao Peng, Lujian Yao, and Kaijie Zhao. Oder: Orthogonal decoupling contrastive regularization for unpaired image dehazing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 25479-254899 2024.
[36] Keli Deng, Yuntao Qian, Jie Nie, and Jun Zhou. Diffusion model based hyperspectral unmixing using spectral prior distribution. IEEE Transactions on Geoscience and Remote Sensing, 2024.
[37] Virginia Klema and Alan Laub. The singular value decomposition: Its computation and some applications. IEEE Transactions on automatic control, 25(2):164 176, 1980.
[38] Carlos Eduardo Rodríguez-Parde and Gaurav Sharma. Geometry of multiprimary display colors ii: Metameric control sets and gamut tiling color control functions. IEEE Access, 9:96912-96929, 2021.
[39] Ali Alsam and Reiner Lenz. Calibrating color cameras using metameric blacks. Journal of the Optical Society of America A, 24(1): 2006.
[40] Lizhi Wang, Zhiwei Xiong, Guangming Shi, Feng Wu, and Wenjun Zeng. Adaptive nonlocal sparse representation for dual-camera compressive hyperspectral imaging. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(10)2104-2111, 2016.
[41] Jize Xue, Yong-Qiang, Zhao, Yuanyang Bu, Wenzhi Liao, Jonathan Cheung-Way Wai Chan, and Wilfried Philips. Spatial-spectral structured sparse low-rank representation for hyperspectral image super-resolution. JEEE' Transactions on Image Processing, 30:3084-3099 2021.
[42] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances neural information processing systems, 33:6840-668 2020.
[43] Boaz Arad, Radu Timofte, Rony Yahel, Nimrod Morag, Amir Bernat, Yuanhao Cai, Jing Lin, Zudi Lin, Haoqian Wang, Yulun Zhang, et al. Ntire 2022 spectral recovery challenge and data set. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition Workshops, pages 863-881, 2022.
[44] Fabio Dell' Acqua, Paolo Gamba, Alessio Ferrari, Jon Aevar Palmason, Jón Atli Benediktsson, and Kolbeinn Ámason. Exploiting spectral and spatial information in hyperspectral urban data with high resolution. IEEE Geoscience and Remote Sensing Letters, 1(4):322-322 2004.
[45] Jiaojiao Li, Chaoxiong Wu, Rui Song, Yunsong Li, and Fei Liu. Adaptive weighted attention network with camera spectral sensitivity prior for spectral reconstruction from rgb images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 4634463,7 2020.
[46] Xiaowan Hu, Yuanhao Cai, Jing Lin, Haoqian Wang, Xin Yuan, Yulun Zhang, Radu Timofte, and Lue Van Gool. Hdnet: High-resolution dual-domain learning for spectral compressive imaging. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 17542-17551, 2022.
[47] Liangyu Chen, Xin Lu, Jie Zhang, Xiaojie Chu, and Chengpeng Chen. Hinet: Half instance normalization network for image restoration. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 182-192, 2021.
[48] Yuanhao Cai, Jing Lin, Xiaowan Hu, Haoqian Wang, Xin Yuan, Yulun Zhang, Radu Timofte, and Lue Van Gool. Mask -uiide spectral-wiss transformer for efficient hyperspectral image reconstruction. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 17502-17511 2022.
[49] Miaoyu Li, Ying Fu, Ji Liu, and Yulun Zhang. Pixel adaptive deep unfolding transformer for hyperspectral image reconstruction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1 2023.
[50] Zeyu Cai, Zheng Liu, Jian Yu, Ziyu Zhang, Feipeng Da, and Chengqian Jin. Reversible-prior- based spectral-spatial transformer for efficient hyperspectral image reconstruction. International Journal on Semantic Web and Information Systems (IJSWIS), 20(1):1-22, 2024.
13

---

[51] Zhiyang Yao, Shuyang Liu, Xiaoyun Yuan, and Lu Fang. Specat: Spatial- spectral cumulative- attention transformer for high-resolution hyperspectral image reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2024.
[52] Graham D Finlayson and Peter Morovic. Metamer sets. JOSA A, 22(5):810- 2005.
[53] Danfeng Hong, Zhu Han, Jing Yao, Lianru Gao, Bing Zhang, Antonio Plaza, and Jocelyn Chanussot. Spectralformer: Rethinking hyperspectral image classification with transformers.
IEEE Transactions on Geoscience and Remole Sensing, 60: 60:1-15, 2021.
[54] Boaz Arad, Radu Timofte, Ohad Ben-Shahar, Yi-Tun Lin, and Graham D Finlayson. Ntire 2020 challenge on spectral reconstruction from an rgb image. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 446 447, 2020.
[55] Magnus Magnusson, Jakob Sigurdsson, Sveinn Eirikur Armansson, Magnus O Ulfarsson, Hilda Deborah, and Johannes Sveinsson. Creating rgh images from hyperspectral images using color matching function. In IGARSS 2020-2020 IEEE International Geoscience and Remote Sensing Symposium, pages 2045-2048, 2020.
14

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction outline the key challenges addressed, contri- butions, and a summary of the experimental findings.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We include the limitation of work in Section 5
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
15

---

Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We have described the experimental details and submitted our code together with the submission. The code will be made publicly available upon acceptance.
Guidelines:
The answer NA means that the paper does include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
16

---

Answer:
[Yes]
Justification:
We used the public dataset and submitted our code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https guwdes/ddeeeeee Cooeeeoodeesssseeeeee //nips.com / ssiniioiicyp for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification:
See Section 4.1
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: Due the limited computational resources, conducted each experiment only once.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
17

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: See Section
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
We fully adhere to the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
See Appendix ??.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
18

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
The paper properly credits the creators or original owners of the assets used, such as code, data, and models. We also have correctly cited the relevant literature.
Justification:
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the Datasets.papersstt paperswi thcode. cww/ datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
19

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We have provided the source code of our proposed model.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/cooe/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
20

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs any important, original, or non-standard components.
Guidelines:
The answer NA
that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
21

---

A Implementation Details and Datasets.
A.1 Implementation Details
Three-stage Training Scheme. We implement our proposed Diff-Spectra in the PyTorch framework. Specifically, our proposed Diff-Spectra adopts three-stage training strategy:
Stage 1, we follow 7 that utilizes RGB-HSI pairs from the training dataset to pre-train the SRF-guided HIE network F(-), fuse-and-com operation H(-), and AICD module g-) jointly, optimized with the corresponding loss (i.e., Eq. 10 in the main content), while freezing the parameters of the SDM;
Stage 2, we follow the standard diffusion model DDPM 42 that utilizes HSI from the training dataset to pre-train the SDM, optimized with the loss Eq. 13, while freezing the parameters of the SRF-guided HIE;
Stage 3, to align the distribution gap between the estimated HSI from the SRF-guided HIE network and the learned spectral prior from SDM, use Eq. 14 in the main content to fine-tune the SRF-guided HIE network and AICD module for 100 epochs, while freezing the SDM. Note that the SDM is treated regularizer in this stage. Empirically, set the learning rate to 1 x 10-4 and the batch size is 20.
Test-time Adaptation. The pre-trained SRF-guided HIE is applied to RGB images from the testing dataset to generate an initial coarse- level HSI signal Y. Next, we treat Y as trainable parameters, and a spectrum y is sampled from Y. We assume each sampled spectrum satisfies the spectral distribution learned by the SDM, i.e., y Ys g(yu), where E (0, and T is the time step trained in Stage 2. Now, we can use Eq. 13 (in the main content) to refine the coarse-level HSI signal Y during the sampling process of SDM. However, it is impractical to optimize Eq. 13 for all t due to the inherent distribution difference between the HSI signal generated by SRF-guided HIE and the spectral distribution learned by SDM. Thus, we perform the gradient update K times in each time step which we named as the inner loop optimization.
A.2 Datasets.
ARAD-1K Dataset [43]. The ARAD-IK dataset includes 950 RGB-HSI pairs, with 900 for training
and 50 for validation, at 482 512 resolution across 31 spectral channels (400-700nm). This dataset not only stands as the largest collection available for HSI reconstruction tasks but also integrates content from preceding compilations, notably the NTIRE 2020 HSI dataset (54). Each HSI this collection is captured with a spatial resolution of 482 x 512, spanning 31 spectral channels ranging from 400mm 700nm.
ICVL Dataset (9]. The ICVL dataset contains 201 HSIs with a resolution of 1300 x 1392. As it lacks the provision of aligned RGB images, use the spectral sampling method proposed by Magnusson et al. [55] to generate the corresponding RGB images. Given that 18 of these images have different resolutions,
leverage the remaining 183 image pairs that maintain resolution consistency, allocating 147 pairs for training and 36 for testing.
Indian Pine Dataset [1]. The Indian Pine dataset records the landscape over area in North- Western Indiana, USA, using the Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) sensor. It comprises 145 x 145 pixels, each with ground sampling distance (GSD) of 20 meters, and encompasses 220 spectral bands that span the wavelength range from 400 nanometers to 2500 nanometers, achieving spectral resolution of 10 meters. Following the climination of 20 bands characterized by noise and water absorption, 200 spectral bands were kept, specifically bands 1-103, 109-149, and 164-219, The scene under investigation features 16 primary categories, including corn, oats, buildings, etc.
Pavia University Dataset [44]. It is collected by the Reflective Optics System Imaging Spectrometer (ROSIS) sensor, which surveyed the area surrounding Pavia University Pavia, Italy. Capable of capturing 103 spectral bands that range from 430 nanometers to 860 nanometers, the resulting image composed of 610 x 340 pixels, each with a ground sampling distance (GSD) of 1.3 meters. This particular scene encompasses 9 distinct land cover classes, including asphalt, grass, trees, etc.
22

---

Table 4: Quantitative comparison of different methods in terms of the accuracy for each class, as well as the overall performance using metrics Overall Accuracy (OA), Average Accuracy (AA), and Kappa coefficient (K) on the Indian Pines dataset. The best one shown bold.
AWAN
HDNer
(77
SPECAT
MS7-L
MST++
[19
Dijf-Spectru (Wurs)
54.16
$5.69
72.74
61.71
66.48
68.32
70.15
67.41
40.21
57.38
71.47
74.24
72.34
74.35
75.18
78.72
82.14
85.68
84.17
90.67
90.05
96.24
85,48
83.74
96.72
81.11
78.63
93.45
83.48
85,21
91.52
92.45
94.01
96.75
66.31
76.54
72,81
73,88
76.18
77.51
80.92
73.48
48.38
59.44
65.59
64,02
59.18
60.17
62.30 30
67.44
44.69
63.19
71.32
72.42
80.03
68.27
65.27
70.02
10 96,70
95.88
100.00
95.32
100,00
100,00
100,00
98 87
12
17.25
55.42
82.47
89.27
89.81
85 92
84.88
90.25
13
90.44
98.31
100.00
98.93
94,42
100.00
100,00
100,00
14
32.27
57.45
40.25
87.34
76.54
62.51
67.24
91.91
15
81.82
81.82
100.00
100.00
100.00
100,00
100,00
100,00
100.00
80.00
OA
58.26
70.41
73.02
77.32
76.36
74.15
75.85
78.14
AA (5)
63.92
76.98
82.25
85,42
86.25
$2.79
84.95
87.88
0.5161
0.59118
0.6844
0.7088
0.7304
0,7008
0,7344
0.7571
Emage
AMAN
MST
CESST
Spodir Ground truth
Figure 7: Visual comparisons randomly selected scene from the validation set of the ARAD-1K dataset with 4 spectral channels. The RGB patch corresponds to the selected red box the RGB image. The spectral curves (bottom-left) correspond to the selected blue Box of the RGB image. Please zoom for a better comparison.
B Additional Experiment Results
Hyperspectral Image Classification. We further provide more quantitative comparisons on the Indian Pines dataset in Table 4 In this table, we further add three methods for more generalized comparisons, including HDNet [46), HINet [47), and SPECAT [51]. As can be seen, our method outperforms existing methods on most classes, and enjoys the best performance over the three metries: Overall Accuracy (OA), Average Accuracy (AA), and Kappa coefficient (r)).
Hyperspectral Image Reconstruction. We provide more visual comparisons in Fig. 7 and Fig. Specifically, Fig. 7 iillustrates the false color results on four spectrum bands, 400nm, 500nm, 600nm and 700nm, respectively, which is the "ARAI)-1--090 K-0901" image chosen from the validation set of ARAD-1K dataset. In the top-left corner of Fig. 7 the RGB patch corresponds to the selected red box of the RGB image. The spectral curves (bottom left) correspond the selected blue Box of the RGB image. Please zoom in for a better comparison. Fig. 8 fillustrates the false color images on four spectrum bands, 440mm, 500nm, 600nm and 700nm, respectively, which is the Phachal-0823-11 1147" image chosen from the testing set of ICVL dataset. In the top-left corner of Fig. 8 the RGB patch corresponds to the selected red box of the RGB image. The spectral curves (bottom left) correspond to the selected blue Box of the RGB image. Please zoom in for better comparison. As can be seen, our method can recover more precise texture information and better pixel-level smoothness over other SOTA methods. In addition, both spectral intensity curves in Fig. and Fig. show that our method can recover more precise spectral values and the spectral distribution of our method closer the ground truth compared with existing methods, especially in the long-wavelengre spectrum (e.g., from
23

---

590am
703
AWAA
HDNet
HUNet
MSTLL
MSI++
PADUT
SST
CESST
SPECAT
1311 Spectra Ground truth
Figure 8: Visual comparisons on randomly selected scene from the validation set of the ICVL HSI dataset with 4 spectral channels. The RGB patch corresponds to the selected red box of the RGB image. The spectral curves (bottom left) correspond to the selected blue Box of the RGB image. Please zoom in for better comparison.
600nm - 700nm), which is benefited by both image-level prior and spectral- level prior introduced our model.
24


Reviews:
[{"review_id": "M9tLkdgdlH", "reviewer": "Reviewer_tdgT", "summary": "This paper addresses the metameric dilemma in hyperspectral image (HSI) reconstruction from RGB images—a situation where different spectral signals produce identical RGB outputs, making them indistinguishable.", "questions": "1. How the dataset for metameric hypothesis testing have been collected? \n2. It would be very interesting to see ablation of different model on metameric issue only", "limitations": "Limited amount of sensor for metameric dillema verification", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strenghts\n\n1. The idea itself to search ways to incorporate semantic knowledge to solve metameric problem\n2. Sufficient quality improvement compared to sota\n3. Extensive and systematic quality evaluation\n\nWeaknesses\n1. It is hard to have the whole picture with only mean error and without error variances\n2. Fescription of the dataset for checking metamer resolving strategy is very limited, almost absetn, so it is hard to have clear picture of this part of the result", "was_revised": true, "final_justification": "I'm not quite happy with this paper. The longer I reeding it, the better I see that author propose some approach to deal with mettamer, but provide insufissient evidence to show that their approach working. I also have worries about the fact that the authors do not ready to provide additional experimental computation to justify their result. Still I think paper might be accepted, since it provides pretty novel way to work with metamers.", "rebuttal": "# Response to Reviewer tdgT:\n\nThank you very much for your constructive comments and suggestions. We will try our best to address your concerns here.\n\n### Q1: Description of the metameric spectra augmented datasets.\n**A1:** Thanks a lot for noticing. Our metameric spectra augmentation is based on the orthogonal subspace decomposition theory defined in [Klema Virginia et al., TAC 1980] and the concept of metameric black [Ali Alsam et al., Journal of the Optical Society of America A 2006], which mathematically demonstrates that different spectra can project into the same RGB color. So we adopt metameric spectra augmentation as a spectral-wise augmentation rather than spatial-wise augmentation (e.g., spatial-wise augmentation is a classic data augmentation method used in various image processing tasks, such as flipping and cropping).\nAccording to the Eqn. 11 in our manuscript, it can be observed that we can vary the coefficient $\\beta$ (metameric black term) to generate new metamer spectra $\\mathbf{S^{\\dagger}}$ from a standard spectra $\\mathbf{S}$, ensuring both standard spectra and metamer spectra can project into the same RGB color by a specific spectral response function. Besides, there are also additional methods to generate metamer data, such as spectral uplifting [Laurent Belcour et al. Computer Graphics Forum 2023], and we only use metameric black here.\nSpecifically, we generate metamer spectra by randomly sampling the coefficient $\\beta$ from a uniform distribution in the range $[0, 1)$. Note that when $\\beta = 1$, it is the standard spectra. We generate metamer data at a ratio of $1:1$ between metamer data and standard data, so the dataset is doubled.\n\n\n### Q2: Ablation study of different models on the metameric dilemma.\n**A2:** Thanks for your question. It is very important to provide such an ablation study to demonstrate that existing methods suffer from the metameric dilemma and to validate the effectiveness of our proposed Diff-Spectra in mitigating this issue. In fact, we have conducted such an ablation study in Section 4.3 and Table 2(b) in our manuscript.\nSpecifically, we firstly pre-train our model and other compared models on standard data and then directly evaluate all pre-trained models on metamer data. The quantitative results are illustrated in Table 2(b). As can be seen, all the existing methods experience dramatic performance drops in terms of ERGAS and SAM in the presence of metamers, while our method relieves this trend. Especially, MST++, PADUT, SST, CESST, and SPECAT show 9.23, 10.81, 8.45, 7.62, and 5.67 times performance drop in terms of ERGAS, while our method only drops 4.74 times.\n\n\n### Q3: Other concerns.\n**A3:** Thanks for your concerns. We provide comprehensive evaluations using both spatial quality metrics, such as PSNR and SSIM, and spectral quality metrics, such as ERGAS and SAM. Besides, we also provide the reconstructed spectral curve comparison and visualized in Figure 1(d) and Figure 4, as well as Figure 9 and Figure 10 in the Appendix. We also appreciate that **Reviewer 5WjA** recognizes the comprehensive evaluation in the Major Strengths. We believe these quantitative and qualitative evaluations are comprehensive and convincing enough. Besides, we will include more description of our dataset in our revised paper. Thanks again for your concerns."}, {"review_id": "ATJgE1xpcX", "reviewer": "Reviewer_5WjA", "summary": "The paper proposes an approach to address the metamer dilemma - where a single RGB value can correspond to multiple possible spectras. The method includes three main stages. First, unlike many existing methods, the authors aim to disentangle the chroma and illuminance information from the RGB input; the resulting image is then fused with the original. In the second stage, the authors initialise coarse HSI values from the fused image and estimate the spectral distribution which uses a modified 1D Spectral Diffusion Model (SDM). Finally, in the third stage, the SDM is frozen, and the coarse HSI signal is refined through metameric spectra augmentation (i.e., generation of new metamers), which serves as a regulariser and is based on the concept of *metameric black*.", "questions": "see minor weaknesses section", "limitations": "It would be helpful if the authors could discuss the limitations of the proposed approach in real-world scenarios, especially in comparison to baselines", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Major Strengths:** The paper addresses a fundamental problem in spectral reconstruction—metamerism. The authors use multiple evaluation metrics and compare their method against strong state-of-the-art baselines.\n\n**Minor Strengths:** The paper is clearly written and easy to follow.\n\n**Major Weaknesses:** Based on the ablation studies, most of the performance gain seems to come from the Adaptive Illumichroma Decoupling (AICD) and the learnable Spectral Response Function (SRF) modules. The impact of modelling metamers using the Spectral Diffusion Model (SDM) and inner loop (IP) appears a bit limited.\n\n**Minor Weaknesses:** \n\n* Figure 2: If I understand correctly, SDM is frozen in both Stage 1 and Stage 3. Please correct if so.\n\n* Figure 3, column 3, row 2: please consider changing the colour label \"Indistinct\" - it's not readable", "was_revised": true, "final_justification": "My main concern about the performance gains of each module (Q1) has been addressed by the authors. I hereby change my recommendation to accept.", "rebuttal": "# Response to Reviewer 5WjA:\n\nThank you very much for your constructive comments and suggestions. We will try our best to address your concerns here.\n\n### Q1: Performance gains of each module we proposed in the ablation study.\n**A1:** Thanks a lot for noticing. The Adaptive Illumichroma Decoupling (AICD) and the learnable Spectral Response Function (SRF) modules contribute most to the initial HSI signal estimation, especially for the spatial information reconstruction, resulting in the performance gain of 34.95 dB in PSNR (i.e., Variant 2 in Table 3(a) of our manuscript). However, it also falls into the performance bottleneck of existing methods, where they fail to reconstruct high-fidelity spectral information. For example, CESST [Xingxing Yang et al., AAAI 2024] and PADUT [Miaoyu Li et al., ICCV 2023] achieve 5.88 and 5.53 in terms of SAM, respectively, which are very close to Variant 2 (5.41).\nOn top of that, the Spectral Diffusion Model (SDM) incorporated with the inner loop (IP) mechanism aims to capture the real-world spectra distribution, serving as a spectral prior to improve spectral fidelity, which further improves the performance to 35.47 dB in PSNR and 3.96 in SAM. Especially, the Spectral Diffusion Model (SDM) incorporated with the inner loop (IP) mechanism mostly contributes to the performance gain of SAM (i.e., from 5.41 to 3.96 in Table 3(b) of our manuscript), which is consistent with the module mechanism of the SDM (primarily contributes to spectral recovery via the learned spectral distribution regularization).\nAs a result, we can see that the performance of our method is significantly better than CESST and PADUT in terms of SAM (i.e., 3.96 compared with 5.88 and 5.53 in Table 1 of our manuscript).\n\n\n### Q2: Clarification for the pre-training details of the SDM.\n**A2:** Thanks a lot for your question. Yes, the SDM is frozen in both Stage 1 and Stage 3. It is only pre-trained in Stage 2. And the whole training procedure is summarized in Algorithm 1 in Appendix A.2, and a more detailed implementation is provided in Section 1.1 of the supplementary material.\nWe notice that the text description in the dashed box in the top-right region of Figure 2(a) may be confusing. We will correct it in our revised paper. \n\n\n### Q3: Improving the readability of Figure 3.\n**A3:** Thanks a lot for your suggestion. We will change the color as you mentioned.\n\n\n### Q4: Limitations of our method in real-world scenarios compared with baselines.\n**A4:** This is a good question. We provide a little discussion of the limitations of our method as future work in the conclusion of our manuscript. The main limitation is that we assume that the spectral distribution of the HSI signal estimated by the SRF-guided HIE is consistent with the spectral distribution learned by the SDM. However, the learned spectral distribution by the SDM and the estimated spectral distribution of the HSI signal reconstructed by the SRF-guided HIE are inevitably inconsistent in the real world, which results in that directly applying SDM to regularize the estimated spectral distribution of the HSI signal reconstructed by the SRF-guided HIE is suboptimal and even detrimental. As can be seen in Table 3(a) in our manuscript, directly applying SDM to Variant 2 results in performance degradation in Variant 3. To deal with this issue, we design an inner loop mechanism to bridge the distribution gap and improve reconstruction accuracy via performing gradient descent K times for each time step of the inference process of the SDM. The effectiveness of the inner loop mechanism has been validated in Table 3(a) in our manuscript by comparing the full Diff-Spectra with the Variant 3.\nHowever, it is still not enough to perfectly bridge the distribution gap between the estimated HSI and the learned spectral distribution since we treat the estimated HSI as learnable parameters in the SDM inference stage (Stage 3 in our manuscript), which is slightly different from the distribution during SDM training. We plan to focus on the optimization of the challenging distribution gap, possible solutions can be quantizing each distribution into a dictionary and calculating their distance or introducing online learning techniques."}, {"review_id": "2ONW5vXiek", "reviewer": "Reviewer_jc7N", "summary": "This paper tackles the metameric dilemma problem in spectral reconstruction by proposing a semi-supervised framework that integrates a physics-aware estimation stage with an unsupervised generative refinement stage. The core technical contribution lies in using a bespoke Adaptive IllumiChroma Decoupling (AICD) module to generate discriminative features for an initial estimation, which is then iteratively refined by a pre-trained Spectral Diffusion Model that serves as a powerful high-fidelity prior. The authors claim they achieved state-of-the-art performance on standard benchmarks and superior robustness on a dedicated metameric test set.", "questions": "1. Could you provide further performance metrics, including FLOPs, inference time, etc.?\n2. Can you provide intermediate visualization results for the AICD module to verify that the model indeed achieves the claimed illumination/chromaticity decoupling effect?\n3. Regarding the ablation study, could you report the computational load introduced by each module and the performance improvement of the core modules on the metameric dataset?", "limitations": "Please refer to the weakness.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n1. The paper commendably introduces a physical prior to address the metameric dilemma problem, which is a task-specific design. The results demonstrate that it surpasses previous methods on this specific issue.\n2. The performance, based on the data provided in the tables, appears to achieve the state-of-the-art.\n3. The SDM module processes the coarse data from the first stage in the spectral dimension using a 1D diffusion approach. This method appears to be both effective and innovative.\n\nWeaknesses\n1. The paper emphasizes its lightweight design, but it lacks critical performance metrics such as inference time and FLOPs in the benchmark.\n2. While the model claims to deconstruct illumination and chromaticity through orthogonal decoupling, the paper provides no direct evidence to verify the effectiveness of this decoupling.\n3. Related to the weakness 2, the paper's ablation study only reports the visual performance metrics for each module, but fails to validate the differences in terms of parameters, computational cost, etc.\n4. The ablation study appears to be conducted on std data. However, the paper's core innovation lies in addressing metameric data, and the current experimental results cannot adequately validate the contribution of the modules on the metameric data.", "was_revised": true, "final_justification": "Most of my concern has been addressed. I  maintain my original score.", "rebuttal": "# Response to Reviewer jc7N:\n\nThank you very much for your constructive comments and suggestions. We will try our best to address your concerns here.\n\n### Q1: Performance evaluation on the inference time.\n**A1:** Thanks a lot for your question. We have provided the parameters of each model in Table 1 of our manuscript. Here, we further provide the inference time comparison below. All evaluations are calculated on the validation set of ARAD-1k, and the image size is $512 \\times 482$, with a single NVIDIA Ampere A100 with 40G RAM. Note that considering all compared methods in our manuscript are CNN-based or Transformer-based, such as MST++, PADUT, CESST, and SPECAT, while the inference time of the diffusion-based method (like ours) is often significantly longer than these methods, we further provide a diffusion-based method (SCDM [Bowen Chen, et al., TGRS 2024]) for a fair comparison. As can be seen, although the inference speed of our method is slower than that of CNN-based and Transformer-based methods, it is significantly faster than that of the other diffusion-based method, SCDM. Meanwhile, the performance of our method in PSNR is also better than SCDM. Besides, considering the spectral reconstruction task is rarely used in real-time scenarios, we consider this inference time to be acceptable. Furthermore, considering that diffusion models dynamically adjust computations during inference, making FLOPs non-trivial to quantify, which may result in unfair comparison, we do not provide the FLOPs comparison here, following the same setting of most diffusion-based methods.\n\n| Method | MST++ | PADUT | CESST | SPECAT | Diffusion: SCDM | Diffusion: Ours |\n| :----: | :----: | :----: | :----: |:----: | :----: |:----: | \n| PSNR | 34.32 | 34.51 | 34.74 | 33.48 | 35.13 | 35.47 |\n| Time (S) | 0.471 | 1.515 | 1.476 | 0.353 | 87.482 | 19.375 |\n\n\n### Q2: Effectiveness and Visualization evaluation of the AICD module.\n**A2:** Thanks a lot for your question. The AICD module is based on Retinex theory, which decomposes images into a reflectance component (color) and an illumination component. However, we find that this hand-crafted decomposition method decomposes images in a fixed manner, which may not appropriately capture the diverse and complex imaging and lighting conditions that are specific to downstream tasks, and merely relying on such decomposition can not distinguish metameric samples. We provide the visual evidence in Figure 3 of our manuscript. The first column shows the standard data (top) and the metamer data (bottom). The second column and the third column show illumination components and reflectance components of the standard data and the metamer data, respectively. As can be seen, each corresponding component is still visually indistinguishable. \nBased on this finding, inspired by the orthogonal decoupling methods used in various vision tasks (e.g., [Zhen Zhao et al. CVPR 2023], [Zhongze Wang et al. CVPR 2024]), which effectively eliminate redundancy and enhance distinctive patterns among visual features, we further deploy orthogonal decoupling to extract more distinctive features. Note that our whole AICD is embedded with learnable parameter $S_k$ and convolution layers, ensuring this decoupling mechanism can be adapted into different scenarios. We also provide visual results of the AICD module in the last column of Figure 3. As can be seen, the visual result of standard data (top) and the visual result of metamer data (bottom) have a significant difference. The effectiveness of this module has also been validated by the breakdown ablation study in Table 3(a) of our manuscript (i.e., comparing the baseline model, SimDiff-Spectral, with Variant 1). We will include more visual results in our revised paper.\n\n\n\n### Q3: Computational cost and performance gain of each module on the metameric dataset.\n**A3:** Thanks a lot for your question. We retrained all variant models and our full Diff-Spectral model on the metameric dataset. Note that we fixed the metameric black term $\\beta = 0$ to generate one-to-one mapped metamer data from the standard data, so that the size of the metameric dataset is the same as the standard dataset. We kept other settings the same as the breakdown ablation study in Section 4.5 of our manuscript. We provide the computational cost and performance gain of each module in the table below. As can be seen, the result illustrates a similar trend to Table 3(a) in our manuscript.\n\n\n| Method | AICD | SRF | SDM | IP | Params (M) | PSNR | SAM |\n| :----:   | :----: | :----:| :----:|:----:| :----:   | :----: |:----: |\n| SimDiff-Spectral |  |  | |  | 1.36 | 31.38 | 9.74| \n| Variant 1 | &#10004; |  | |  | 1.51 | 32.70 | 8.17| \n| Variant 2 | &#10004; | &#10004; | |  | 1.73 | 33.41 | 7.35| \n| Variant 3 | &#10004; | &#10004; |  &#10004; |  | 2.49 | 32.92 | 8.01| \n| Diff-Spectral | &#10004; | &#10004; |  &#10004; | &#10004; | 2.49 | **33.73** | **5.84**|"}, {"review_id": "RsyASbYbB5", "reviewer": "Reviewer_3yrt", "summary": "To mitigate the metameric dilemma in spectral reconstruction, the authors leverage the Retinex theory to decouple material properties from illumination attributes, estimate a coarse hyperspectral image based on the spectral response function, and then refine the result using a lightweight diffusion model. Experimental results demonstrate that the method achieves state-of-the-art performance on two datasets.", "questions": "1. The authors claim that their approach is a form of semi-supervised learning. However, based on Equations (10) and (14), all constraints appear to be supervised. Where exactly is the semi-supervised aspect reflected? This is a critical point—clarifying it would greatly assist the review process.\n2. The authors state: “All existing learning-based methods directly learn mappings between sRGB and HSI feature spaces, neglecting the intrinsic spectral distribution of HSIs and lacking physical constraints.” However, based on my understanding of the field, introducing physical priors is a well-established strategy in spectral reconstruction. Many works have explored the use of spectral response functions (SRF), spectral mixing models, and other physics-informed techniques to guide the reconstruction process. The authors are advised to conduct a more comprehensive literature review and clarify this claim accordingly.\n3. The authors assert that “diffusion-based methods have yet to be explored for reconstructing HSI signals directly from RGB images.” However, diffusion models have already been applied to spectral super-resolution (SSR) tasks, which also involve reconstructing HSIs from RGB inputs. The authors should acknowledge this body of work, clearly distinguish their task if it is indeed different, and include comparisons with at least one diffusion-based SSR method.\n4. Figure 1 is confusing: For (a), what is the precise relationship between the HSI data cube and the HSI manifold? For (b), why the Standard and Metamer are separable in the RGB manifold?", "limitations": "Since the method is based on the diffusion model, I am concerned about whether it has limitations in inference speed.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Weaknesses:\n1. The review and comparison with existing methods are not comprehensive, making it difficult to properly assess the novelty of the proposed approach.\n2. The authors’ descriptions of key technical components lack sufficient detail.\n\nStrengths:\n1. The proposed method is theoretically grounded and offers strong interpretability.\n2. The topic addressed is of high practical relevance and importance in real-world applications.", "was_revised": true, "final_justification": "The author's response has largely addressed my concerns, and the quality of the manuscript has improved. The topic of the manuscript is very interesting; however, I believe that the physical interpretability of the proposed method still has room for further exploration, and the analysis of the metameric dilemma could be more in-depth. Therefore, I encourage the authors to continue refining these aspects in their future work. Accordingly, my final rating is 4: borderline accept.", "rebuttal": "# Response to Reviewer 3yrt:\n\nThank you very much for your constructive comments and suggestions. We will try our best to address your concerns here.\n\n\n### Q1: Inference speed of our method.\n**A1:** Thanks a lot for your question. We provide the inference time comparison below. All evaluations are calculated on the validation set of ARAD-1k, and the image size is $512 \\times 482$, with a single NVIDIA Ampere A100 with 40G RAM. Note that considering all compared methods in our manuscript are CNN-based or Transformer-based, such as MST++, PADUT, CESST, and SPECAT, while the inference time of the diffusion-based method (like ours) is often significantly longer than these methods, we further provide a diffusion-based method (SCDM [Bowen Chen, et al., TGRS 2024]) for a fair comparison. As can be seen, although the inference speed of our method is slower than that of CNN-based and Transformer-based methods, it is significantly faster than that of the other diffusion-based method, SCDM. Meanwhile, the performance of our method in PSNR is also better than SCDM. Besides, considering the spectral reconstruction task is rarely used in real-time scenarios, we consider this inference time to be acceptable. Furthermore, considering that diffusion models dynamically adjust computations during inference, making FLOPs non-trivial to quantify, which may result in unfair comparison, we do not provide the FLOPs comparison here, following the same setting of most diffusion-based methods.\n\n| Method | MST++ | PADUT | CESST | SPECAT | Diffusion: SCDM | Diffusion: Ours |\n| :----: | :----: | :----: | :----: |:----: | :----: |:----: | \n| PSNR | 34.32 | 34.51 | 34.74 | 33.48 | 35.13 | 35.47 |\n| Time (S) | 0.471 | 1.515 | 1.476 | 0.353 | 87.482 | 19.375 |\n\n\n### Q2: Semi-supervised learning framework clarification.\n**A2:** Thanks a lot for your question. During Stage 1 and Stage 2, as summarized in Algorithm 1 in Appendix A.2 of our manuscript, the SRF-guided HIE and the SDM are pre-trained in a supervised manner, respectively. However, we use the SDM to regularize the reconstructed HSI signal estimated by the SRF-guided HIE in Stage 3 in an unsupervised manner, since Stage 3 is actually an inference stage without any ground-truth data pairs. So we summarize the whole model as a semi-supervised model. Note that there is a mistake in the loss function of Stage 3 in Algorithm 1 in Appendix A.2 of our manuscript, where the $\\mathcal{L}_{X}(\\mathbf{Y})$ should be deleted. We will correct it in our revised paper. And thanks again for pointing out this issue.\n\n\n### Q3: Physical constraints for spectral reconstruction in the literature.\n**A3:** Thanks a lot for your concern. We would like to highlight that existing methods overlooked the effective integration of both spectral prior and physical prior, while our method proposed an effective method to integrate both physical prior (such as SRF) and spectral prior (SDM). The original description, as you mentioned, may have caused ambiguity; we will correct it in our revised paper. Thanks again for pointing out this issue.\n\n\n### Q4: Diffusion-based methods for spectral reconstruction in the literature.\n**A4:** Thanks a lot for your concern. We agree that there are several diffusion-based methods for spectral reconstruction have been proposed, such as SCDM [Bowen Chen, et al., TGRS 2024]. However, please note that our SDM is significantly different from these methods. Our SDM models the spectral distribution of HSI signal in a 1D spectral curve manner, without conditioning on any RGB images, while these methods use RGB images as conditional inputs to generate corresponding HSI images. Here, the goal of our proposed SDM is to capture the real-world spectral distribution, serving as a spectral prior to regularize the reconstructed HSI signal estimated by our proposed SRF-guided HIE to improve spectral fidelity. It is more related to regularization, not generation. The original description, as you mentioned, may have caused ambiguity; we will correct it in our revised paper. Thanks again for pointing out this issue.\n\n\n### Q5: Clarification of Figure 1.\n**A5:** Thanks a lot for your concern. Figure 1(a) is a sketch where we use the HSI manifold to represent an aggregate space of all HSI data. In Figure 1(b), standard and metamer data are very similar, but still have some differences in the real world, for example, different noise disturbance and data error introduced by different imaging systems."}]

Author rebuttal:

Meta-review: This paper proposes a semi-supervised framework that integrates physics-aware estimation with a spectral diffusion prior to address the metameric dilemma in hyperspectral reconstruction. Reviewers found the approach technically sound, with strong empirical performance and clear novelty in combining physical priors with generative modeling. Main concerns focused on dataset description, variance analysis, computational efficiency, and positioning within related work, but the rebuttal provided detailed clarifications and additional experiments that largely resolved these points. Two reviewers recommended accept, while two others raised their scores to borderline accept after rebuttal, acknowledging the paper’s strengths despite remaining minor issues. Overall, the work makes a notable and timely contribution, and the decision is to accept.
Decision: Accept (poster)