A NEW BACKBONE FOR HYPERSPECTRAL IMAGE RECONSTRUCTION
Authors: Jiamian Wang
Yulun Zhang
Xin Yuan
Yun Fu
ZHIQIANG TAO
Venue: ICLR 2022 Submitted (2022)

Abstract: As the inverse process of snapshot compressive imaging, the hyperspectral image (HSI) reconstruction takes the 2D measurement as input and posteriorly retrieves the captured 3D spatial-spectral signal. Built upon several assumptions, numerous sophisticated neural networks have come to the fore in this task. Despite their prosperity under experimental settings, it's still extremely challenging for existing networks to achieve high-fidelity reconstructive quality while maximizing the reconstructive efficiency (computational efficiency and power occupation), which prohibits their further deployment in practical applications. In this paper, we firstly conduct a retrospective analysis on aforementioned assumptions, through which we indicate the imminent aspiration for an authentically practical-oriented network in reconstructive community. By analysing the effectiveness and limitations of the widely-used reconstructive backbone U-Net, we propose a Simple Reconstruction Network, namely SRN, just based on some popular techniques, e.g., scale/spectral-invariant learning and identity connection. It turns out, under current conditions, such a pragmatic solution outperforms existing reconstructive methods by an obvious margin and maximize the reconstructive efficiency concretely. We hope the proposed SRN can further contribute to the cutting-edge reconstructive methods as a promising backbone, and also benefit the realistic tasks, i.e., real-time/high-resolution HSI reconstruction, solely as a baseline. 


Under review as a conference paper at ICLR 2022
A NEW BACKBONE FOR HyPeRSPECTRAL IMAGE RECONSTRUCTION
Anonymous authors Paper under double-blind review
ABSTRACT
As the inverse process of snapshot compressive imaging, the hyperspectral image (HSI) reconstruction takes the 2D measurement as input and posteriorly retrieves the captured 3D spatial-spectral signal. Built upon several assumptions, numer- ous sophisticated neural networks have come to the fore in this task. Despite their prosperity under experimental settings, it's still extremely challenging for exist- ing networks to achieve high-fidelity reconstructive quality while maximizing the reconstructive efficiency (computational efficiency and power occupation), which prohibits their further deployment in practical applications. In this pa- per, we firstly conduct retrospective analysis on aforementioned assumptions, through which we indicate the imminent aspiration for an authentically practical- oriented network in reconstructive community. By analysing the effectiveness and limitations of the widely-used reconstructive backbone U-Net, we propose a Sim- ple Reconstruction Network, namely SRN. just based on some popular techniques, e.g., https/peecraa-iiviiian current conditions, such a pragmatic solution outperforms existing reconstructive methods by an obvious margin and maximize the reconstructive efficiency con- cretely. We hope the proposed SRN can further contribute to the sutting-edge re- constructive methods a promising backbone, and also benefit the realistic tasks, i.e., real-time/hiee resolution HSI reconstruction, solely as a baseline.
learning and identity connection. It turns out, under
INTRODUCTION
Hyperspectral imaging (HSI) refers to multi-channel imaging where each channel stores information at specific spectral wavelength for fixed real-world scene (Plaza et al., 2009). By capturing spatial intensity in a spectral-wise manner, hyperspectral images empower richer information than traditional RGB image cubes and they have been applied in wide range of scenarios, e.g., object detection (Kim et al., 2012; Xu et al., 2015), remote sensing (Borengasser et al., 2007; Melgani & Bruzzone, 2004; Yuan et al., 2017), medical image processing (Lu & Fei, 2014; Meng et al., 2020c) etc. HSI can be captured and measured by snapshot compressive imaging (SCI) systems, which tend compress information of snapshots along the spectral axis into one single 2D measurement (Yuan et al., 2021). The coded aperture snapshot spectral imaging (CASSI) system (Wagadarikar et al., 2008; Meng al., 2020b) forms one mainstream research direction among existing SCI systems due to its passive modulation property (Llull et al., 2013; Wagadarikar et al., 2008; 2009; Yuan al., 2015).
The goal of HSI reconstruction is to transform the measurements into desired cubic hyperspectral images. As a result, dimensionallexpansioo mapping function (2D to 3D) is required, for which reason such a mapping relationship approximation is deemed to be much harder than general image regression tasks. By introducing domain expertise, previous research efforts have proposed a quite number of reconstruction algorithms (Bioucas- Dias & Figueiredo, 2007; Liu et al., 2019; Miao et al., 2019; Meng et al., 2020b; Wang et al., 2020; 2017; 2019; Yuan, 2016), among which deep neural networks (Meng al., 2020b;c; Miao et al., 2019; Wang et al., 2019; Wang et al., 2019; 2020; Zheng et al., 2021) enable an effective way to faithfully bridge between input and output compositional hierarchies (LeCun et al., 2015).

---

Under review as a conference paper at ICLR 2022
RGB Reference
Annt
TSAdee
GSMB-based
Ours
34
(1.25ML 33-45B))
(3. 13.70M, 30.28dB)
PSNR (dB)
30-
44.25M,,0024400
(62.64M, 29 29:25dB)
(38)))
28
CASSI Messarement
Our
net (ICCV19)
(51.32M, 26.8d8)
HSSP (CVPR199)
U-adt (MIOCAF15)
24
TSA (BCCV220
70
50
30
20
10
Number of Parameters (Millions)
Figure I: Left: HSI reconstructions between the proposed SRN and several state-of-the-arr methods. The RGB reference (top-left) is shown to demonstrate the color. The reconstructed details are high- lighted with yellow bounding boxes (zoom in for clear comparison). Right: Comparison between different HSI reconstruction methods in terms of PSNR (1) and model size (4).
1.1
MOTIVATION
While neural networks become increasingly prevalent in solving the HSI reconstruction problem, their success is reliant on several idcalistic hypotheses, which hardly holds in practical scenarios. This prohibits the further application and exploration of HSI.
Proper Dataset Volume. The feasibility of learning in universal sense can be defined by satisfying a VC generalization bound Abu-Mostafa et al. (2012). From this perspective, deep neural networks considered to be high potential due to their high VC bound (which can be described as the amount of free variables). Notably, the corresponding feasibility of neural networks is generally determined by their model complexity and the volume of datasets. In the recent HSI reconstruction methods, more increasingly complicated reconstructive networks have been proposed by assuming the underlying dataset is sufficiently large that the learning feasible. This neglects the fact that there's limited data accessible. Under such circumstance, simpler model might be more promising.
Ideal reconstructive rate. Practically, any reconstruetive algorithm appears with substantial for- ward model. For iapoora-inneessitve situations, the inverse model processes the measurements at any reconstructive rate Trecon > O, without considering cooperating with the forward model. However, the reconstructive rate should be higher than (or at least equal to) capture rate of the for- ward model, i.e., Trecon 2 Teap > O, for temporal-sensitive cases, i.e., real-time/highh-seee HSI reconstruction. For example, X-net proposed by Miao et al. (2019) can be applied in reality with Trecon > 30.ps. More generally, reconstructive methods are always hypothetically deemed to be efficient enough, i.e., compatible with any forward systems, which however probably suffers from a deviation from current technical support. Considering the arbitrariness of the lower bound Teap- cannot determine "ideal" reconstructive rate but only to maximize it, i.e., minimize the constructive time. For neural network-based methods, eliminating the number of parameters and computational burdens sound solution.
Unconstrained computing power. The success of the deep neural networks can largely attribute to the overparameterization (Soltanolkotabi al., 2018)-the tte parameters in neural networks are lot more than the training samples for good representation learning. Actually, just by investing more computing power P. the cornerstone which overparametrization is reliant upon, researchers have made great progress in diverse applications (Thompson et al., 2020). Latent performance boost can be further expected if more computing power is available, until one reaches the computing power bottleneck p It turns out it's difficult to solve such bottleneck in practical. For example, to conduct TSA-Net(Menng et al., 2020b) for high resolution (HR)-HSI restoration (1024x 1024), the required GPU memory would be unrestrictedly over 18Gb, which indicates dilemma encountered by all high-performanced reconstructive methods with large model size.
1.2 CONTRIBUTIONS
In this study, bearing the limitations imposed by the scale datasets, the capability of optical systems, and the computational platform, we posit the possibility of network construction with min-
2

---

Under review as a conference paper at ICLR 2022
imum inductive bias, based on observation upon the reconstructive baseline U-net. We propose simple yet quite promising CNN reconstructive network, namely Simple Recon Net (SRN), whose success is owing to revisiting and tailoring several practical techniques for HSI, i.e.. spatial/spectral invariant learning and residual learning. We summarize the contribution of this work as follows.
The proposed SRN provides new sate-of-the-ar by improving the previous one (Huang et al., 2021) > 3dB in PSNR. Moreover, our approach presents clear perceptual improve- ment across different spectral channels (Fig. Left).
We sigmificantly shrink the model size (see Fig. Right) without sacrificing the perfor- mance, ie.,, most extent, only use <1/3 parameters of GSM-baseed method (Huang et al., 2021) and to the maximum, reduce FL.OPs by > 34times as shown in Tab. 3, both of which yield a higher temporal efficiency and computational efficiency. This makes real- time/higher-resolttin images reconstruction practically executable under restricted GPU conditions.
The proposed network can either be solely used as baseline, or conjunctively referred to backbone, contributing to more complicated methods, i.c., more sophisticated E2E methods or unrolling methods.
2 RELATED WORKS
The basic idea of SCI is to modulate the high-dimensional signal using a higher frequency than the capture rate of the camera. In this manner, compressed coded frame obtained will include the information in the high-dimensional signal and a high-performance algorithm can then be employed to recover the desired data. For compression, the novel implementation of SCI, CASSI, uses coded aperture and a prism to conduct the (spatio-)spectral modulation.
Previously, hterative-hased optimization algorithms predominate the field of HSI reconstruction by approximating the image priors through diverse regularization techniques, i.e., the total variance (TV) (Kittle et al., 2010; Wang et al., 2015; Wagadarikar al., 2008), sparsity (Wang et al., 2017; 2015), non-local similarity (NLS) (Wang et al., 2016; Fu et al., 2016), Markov Random Field (MRF) (Tappen, 2007), Gaussian mixture model (Yang et al., 2015) etc, among which De- SCI (Liu et al., 2019) achieves best performance on both video and spectral compressive imaging. TwIST (Bioucas-Dic & Figueiredo, 2007) proposed a two-step Iterative shrinkage/thressooldnng al- gorithm by modeling the reconstructive problem as a linear observation model with a nonquadratic regularizer (i.e., total variation). GPSR (M. T. Figueiredo al., 2007) proposes to use gradient projection (GP) algorithms solve the inverse problem that is formulated as bound-constrained quadratic programming (BCQP) process.
Inspired by the success of deep learning in other image translation problems, researchers have started using deep learning to reconstruct hyperspectral images from CASSI measurements (Meng et al., 2020b;e; Miao et al., 2019; Wang al., 2019; Wang et 2019; 2020; Zheng et al., 2021). which can be substantially divided into two streams: end-to-end neural networks (E2E-NN) models and others. The former stream tends to directly learn complete mapping function from measurements (always packaged with masks) to estimated HSIs. Other researchers turns to introduce NN models into conventional optimization algorithms, named deep unrolling/unfolding nad plug-am auug-a methods, leading to lightweight and interpretable methods.
Proposed in Ronneberger et al. (2015), U-Net sat atop leaderboards regarding medical image track- ing and segmentation, followed by which many variants have been derived in recent years, i.e. 3D U-Net (Çiçek et al., 2016). Ternaus U-Net (Iglovikov
Shvets, 2018), MultiResUNet (Ibtehaz & Rahman, 2020) and Attention U-Net (Oktay et al., 2018) etc. Typical investigation correlates such a predominance with characteristics of medical images: 1) repeated tissue patterns and complicated gradient distribution; 2) locally anomalous region and atypical noise distribution. U-Net intelligently fits the above characteristics by firstly abstracts coarse information and then focus on localization through simultaneously expansion and skip connection.
As a classical network structure, U-Net serves as the first precast backbone in HSI reconstruction, among series of famous architectures, and has been faithfully employed in both E2E and unrolling methods. For example, the X-net (Miao et al., 2019) is a dual-stage generative model which em- ploys U-Net and residual learning strategy. The TSA-Net (Meng et al., 2020b) which combined
3

---

Under review as a conference paper at ICLR 2022
CASSI
Sinffied variante Codiel apertare
SRN
Prreference
coded aperture
dispersion
Single
Py
reconstructed #S!
Figure 2: Illustration of HSI capturing and reconstruction. An optical system, CASSI (left blue box, in which a 2D coded aperture and a disperse are displayed while other optical components omitted), compresses the hyperspectral signal F into 2D measurement Y. For reconstruction (right yellow box), we train the proposed Simple Recon Net (SRN), to learn an E2E mapping from Y (initialized by shifted variants of coded aperture) to F.
spatial-spectral self-attention with U-Net led to excellent results on both simulation and real data. As a deep unfolding method, GAP-net (Meng et al., 2020a) utilizes 15-layer U-Net trained denoiser each stage. Recent Gaussian Scale Mixture Frioe-based (GSM- bbM-db baseline (Huang et al., 2021) employing U-Net for two objects: a lightweight U-Net for approximating the regular- ization parameters, another lightweight U-Net for estimating the local-mean of GSM prior. U-Net has exemplified the rationality of uncomplicated neural network in field of HSI reconstruction, even if considering the huge difference between the medical images and hyperspectral images. Actu- ally, as reported in Tab. and Tab. 2, U-net only achieves sub-optimal performance when solely referred to as a reconstructive baseline, which reveals both the validity and the limitations of such generic solution. It turns out the performance of neural networks are sensitive to minor adjustments. Compared with the U-Net, it's possible to construct a "substitute network" that enables significant performance boost with mainstream techniques at hand.
3 METHODOLOGY
3.1 MATHEMATICAL MODEL OF CASSI
Recall that the key idea of SCI is modulate the spatio-spectral signal with different modulation patterns, and CASSI implemented this in low cost way by a fixed mask plus a disperser. Given the 3D spectral cube (Fig. 2 left), denoted by F E N.NxM, xNx, where N,, Ny. and N,, represent the spectral image's 's height, width, and total number of wavelengths, respectively, we compute signal modulation (implemented by the mask) in channel-wise where F' E N= XN, represents the modulated signals, M* E NR refers pre-defined physical mask governing on signal modulation, 7 [1, l...... ,, NN] indexes wavelengths, and @ the Hadamard product. By passing F' disperser, the cube F' becomes tilted and could be sheared along the y-axis. Let F" E N= N -N)+, be the tilted cube, and A be a reference wave- length, i.e., F' F'(:,:, nxx.] works like an anchor image that not sheared along the y-axis, we have F" E, n Fx, y+]( (iidd(((n-o))) -Xe), nx). where (u, v) locates the coordinate system on the detector plane, An denotes the rih channel, A refers to the anchored wavelength, and d((,,, - Aoa) represents spatial shift of the ny-th channel in F'. Notably, since the sensor integrates all the light within the wavelength range [Amins \max), we could model the compressed measurement at a detector v(, with the following integral: y(u, u) JAmm f" v, n2 MA), where f" is the analog (con- tinuous) representation of F" To discretize this model, we have Y = ΣNA F" nx) G. Y E (N represents a 2D measurement, which, essence, is a compressed frame corresponding 2D measurement noise.
way as F' nx) = (t. mx) O M
capturing the information, and G E X (NN+
To simplify model notations, we define M as the shifted masks and signal frames of different wavelengths as follows: M(u, v, m2) =
N and F NNNg+N 1)xNA
M' d(A,, Ae)), Fu, nx) = F(x,y + d(An nx). By using M and F, the measurement Y could be reformulated as
Y == m F(..,)) 3 M ((.in))) G.
is usually easier to use the vectorized notations. Let y = (e) and g = vec(G) E R be the vectorization of matrices Y and G, where vec(.) concatenates all the columns of matrix
4

---

Under review as a conference paper at ICLR 2022
single vector. Similarly, have f(as) vec( (f.:;, resulting in the vector f
voc(If()) ... ff (( TRNNA, where (N + N₂ 1). By defining the sensing matrix as + [D DN.1 RuxnNs, where Dn,
mx))) is diagonal matrix
expanded by vec(M[(s ny)), rewrite equation as
リ=
(2)
While obtaining the vectorized measure y is similar to compressive sensing (Donoho, 2006; Em- manuel et al., 2006), the sensing matrix & has very special structure. The recent studies (Jalali & Yuan, 2019) have shown the signal can still be recovered even when .Nx- > 1. Given the measure- ment y captured by the camera and & calibrated upon pre-design, one critical and practical problem of CASSI is to solve f used in equation 2, falling in the vein of inverse problem (Yuan et al., 2021), for which will propose simple yet promising deep convolutional neural network below.
3.2 NETWORK DESIGN
The reconstructive function generates the 3D hyperspectral estimations from measurements. There- fore, would be better if one decompose the 2D measurements into 3D data cube beforehand. A typical operation is to firstly initialize network input by conducting dot productions between the measurement and each spectral channels of shifted variants of coded aperture, namely mask, with an intention to mimic the + b- according to equation 2:
(3) where Ry-th channel of Fy E N.N.NN is computed through Hadamard product, i.e., O. and shift operation (raw output will be shifted into the desired tensor shape). Overall, the proposed model aims to learn the mapping as G(.) : Fy + F, where F denotes the reconstruction result. has been previously proved that network depth is one of the determining factors for the success of estimation (Liang Srikant, 2016; Telgarsky, 2016; Eldan & Shamir, 2016). We thereby preclude complicated modules while maintain sufficient depth. The specific network structure is quite simple and have the following function format:
Fyln nn] == shift(M,, C YY),
9o. k 0
9k =
90 E k 1,.., K,
(4)
where 9k is intermediate embedding at level k. gu is initialized by a single convo- lutional layer, ie.. 90
CONV(Fy), where 90 R. NNy.N, Note that Nº > Nx. based on which we intend to do a "learnable spectral interpolation". This provides dancy and thus enables the model to better learn the connections among different channels. For example, adjacent spectral channels tend to be more correlated to each other in general HSIs. R&((-) in equa- tion 4 is just non-linear fea- ture transformer that bridges be- tween different levels of embedding. We simply set it as Ra(x) CONV(ReLU/CONV(e)) shown in Fig. 3, the main structure of the proposed network includes K different levels of embedding. Sym- metric with the input initialization, the final output is simply computed by convolutional layer: F CONV(gy>) CONV + go). Identity connections (Bishop et al., 1995) are widely used the whole network. In Sec. we show that such simple network stands out among all existing reconstructive methods. Below we will introduce several guidelines for model establishment, followed by which the proposed network successfully learns the mapping function between cubic input-output data pairs.
spectral redun-
90
82 Rs
ge R... (.
5x
Figure 3: Illustration of the proposed Simple Recon Net (SRN), Input go = CONV(Fv). All the Rs(.), 1, .... K, are defined by CONV-RELU-CONW, To maximize efficiency, rescaling pairs are inserted right after the inter- mediate embeddings, indicated by X. For example, in SRN v2. we do downsample for 91 and then upsample for 912-
As
3.2.1 SPATIAL/SPECTRAL INVARIANCE
5

---

Under review as a conference paper at ICLR 2022
As mentioned in Sec. 2, U-net (Ronneberger et al., 2015) is barely satisfactory in this field, even being technically modified as in Miao et al. (2019). This is determined by twofold rea- sons: 1) Different characteristic between med-
4096
Receptive Fleld (RR) Lng-Dimemsion
256-
Useet Embedding Depth
Unre Eibeddinn Dupt
1024
Ratheriding dedddnggg Dey-Ds
ical images and HSIs (natural scenes) as illus- trated in Sec. 2; 2) Different utilities: medical image processing task is always more locally
16-
focused while the HSI reconstruction tends to be globally comprehensive.
Uset RF Diu Ours RE Dim
256-
turns out the receptive field (RF) is important for image tasks. As demonstrated by Luo et al. (2016), for plenty of tasks, i.e., image classifi- cation, dense prediction etc., RF should be suf- ficiently large to capture the semantic informa- tion (highest RF should be at least encompass the raw input). In object detection, RF should be cautiously determined. We argue that in low- level regression tasks, its unnecessary for the highest RF cover the whole image but is sup- posed to be large enough to capture the neigh- boring for the estimated pixel. In Fig. 4, we compare the proposed model SRN with the U-Net utilized by Miao et al. (2019) in terms of the spatial/spectral dimension. The highest RF of the U-Net is much larger than the input (256 v.s. 532), which is oversized and thus improper. The highest RF of proposed SRN, in comparison, makes more sense (256 v.s. 69). In SRN, concurrent with such linearly expanding behavior of RF, the spatial size intermediate embedding remains the same, which is thus called "Spatial Invariance".
Network Depth (Including output)
15
20 25 10
Figure 4: Receptive field (RF) dimension and net- work embedding depth comparison (U-Net (Miao al., 2019) v.s. Ours). U-Net unnecessarily ob- serves super large spatial area, at the end,i.e., x32x532, while SRN finally focuses on a suf- ficiently large neighborhood for approximation, compared with the input spatial size denoted as vertical black bar.
To approximate the 3D signal in a rank minimization manner, previous researchers attempt to reduce the spectral channels somewhere in the model, which turns out be a lossy compression operation and is avoided in our work. While in the proposed model, we simply treat the data cube as a whole and thus keeps the spectral dimension constant through the main body (spectral invariant) after expanding from NA = 28 to N, = 64.
3.2.2 EFFICIENCY MAXIMIZATION, TOWARD PRACTICAL APPLICATIONS
In practical application scenarios, performance and efficiency of the proposed method consist of paradox for all existing reconstructive methods, including our proposed network. For such a high- performanced (as demonstrated by considerable experiments in 4) and concise network, we seek to further maximize its efficiency at the expense of negligible performance descent. turns out the metric, floating point operations (FL.Ps, i.., amount of additions and multiplications), pro- vides a favourable perspective of view, which 1) is inversely proportional the reconstructive rate mentioned in 1.1, 2) is positively related to the computing power. We minimize the FLOPs via introducing rescaling manipulation-pen pairs, which are consist a downsampling operation, i.e., a convolutional layer with stride 2, and an upsampling operation (i.e., convolutional layer followed by PixelShuffle operation (Shi et al., 2016)). Spatial size will be doubled/halved by a single up- sampling/dowasampingg operation. As annotated by Fig. 3, rescaling manipulations can be flexibly inserted into model, i.e., right after the intermediate embeddings. Except for the initially proposed network SRN v1, we further try two more variants: SRN v2 contains one rescaling pair, which reduces the spatial size of go and recovers at gΣ; SRN v3, in which second rescaling pair is inserted in the after 91 and 912. Interestingly, experimental results below proves both of the variants are still competitive to state-of-the-ar performance (Tab. 1 and Tab. 2) in simulation.
EXPERIMENTS
Dataset. As in Meng al. (2020b), adopt the same 28 wavelengths distributed within the range of 450nm to 650nm obtained by spectral interpolation manipulation for the HSIs, and conduct experiments the following two data type: 1) Simulation Data and 2) Real HSI data.

---

Under review as a conference paper at ICLR 2022
Table 1: PSNR (dB) values by different algorithms 10 scenes in the simulation dataset. Our approach is denoted by vl: SRN w/o rescaling pair, v2: SRN with I rescaling pair, and v3: SRN with 2 rescaling pair. Notably, v2 and v3 are provided to further reduce the FLOPs of v1 (see Tab. 3) while still exhibiting competitive performance over seven state-of-the-a methods.
Method
Scene
Scene2 Scene3 Scene4 Scene5 Scene6 Scene7
Scene8
Scene9 Scene 10
Avg.
U-net
28.28
24.06
26.02
36.33
25.51
27.97
21.15
26.83
26.13
25.07
26.80
HSSP
31.07
26.30
29.00
38.24
27.98
29.16
24.11
27.94
29.14
26.44
28.93
A-net
30.82
26.30
29.42
37.37
27.84
30.69
24.20
28.86
29.32
27.66
29.25
TSA-Net
31.26
26.88
30.03
39.90
28.89
31.30
25.16
29.69
30.03
28.32
30.24
GSM-based
32.38
27.56
29.02
36.37
28.56
32.49
25.
31.06
29.40
30.74
30.28
PnP-DIP-HST
32.70
27.27
31.32
40.79
29.81
30.41
28.18
29.45
34.55
28.52
31.30
net
33.62
30.08
33.07
40.94
30.77
33.60
27.41
31,25
33.56
30.36
32.47
SRN (v3)
32.85
28.61
31.27
39.42
29.93
32.81
26.26
30.87
31.74
29.84
31,36
SRN (v2)
33.16
29.08
32.19
39.81
30.43
33.22
26.69
31.50
31.69
29.98
31.77
SRN (v1)
34.42
30.98
33.11
41.58
31.87
35.38
28.26
33.38
33.75
31.75
33.45
Table 2: SSIM values by different algorithms on 10 scenes in the simulation dataset, which shows the consistency with the above metric PSNR. Both of PSNR and SSIM indicate the promising per- formance of proposed SRN on syhthetic testing data.
Method
Scenel
Scene2 Scene3 Scene4 Scene5 Scene6 Scene7
Scene8
Scene9
Scene 10
Avg.
U-net
0.822
0.777
0.857
0.877
0.795
0.794
0.799
0.796
0.804
0.710
0.803
HSSP
0.852
0.798
0.875
0.926
0.827
0.823
0.851
0.831
0.822
0.740
0.834
X-net
0,880
0.846
0.916
0.962
0.866
0.886
0.875
0.880
0.902
0.843
0.886
TSA-Net
0.887
0.855
0.921
0.964
0.878
0.895
0.887
0.887
0.903
0.848
0.893
GSM-based
0.920
0.892
0.925
0.970
0.894
0.938
0.898
0.932
0.925
0.934
0.923
PnP-DIP-HSI
0.898
0.832
0.920
0.970
0.903
0.890
0.913
0.885
0.932
0.863
0.901
GAP-net
0.926
0.914
0.944
0.966
0.925
0.936
0,915
0.918
0.937
0.914
0.929
SRN (v3)
0.906
0,850
0,903
0.954
0.911
0.927
0.839
0.913
0.909
0.893
0.900
SRN (v2)
0.909
0.852
0.902
0.943
0.911
0.927
0.838
0.918
0.910
0.893
0,900
SRN (v1)
0.931
0.906
0.923
0.970
0.941
0.958
0.867
0.953
0.937
0.939
0.932
For simulation data, both CAVE (Park et al., 2007) synthetic dataset and KAIST (Choi et al., 2017) synthetic dataset are applied in our simulation experiment. Regarding the training set, we create 205 1024x x024x28 large image examplers from 30 256 216xx266222 x56x28 images from CAVE dataset by ran- domly concatenating. Operations like rotation and rescaling are both used to magnify the robustness. Training samples of size 256x256x 28 will be further randomly cropped from the examplers. We compare with other methods on the same ten different 256 256x 28 HSIs abstracted from KAIST dataset. For real HSI data, augment the above training set by additional 37 HSI images in KAIST (Choi et al., 2017) dataset, all of which will be cropped into 660x660x22 to match the real-world measurement, which are abstracted by CASSI system developed in Meng et al. (2020b).
Training and Testing Procedure. The model is trained to minimize a mean squared error (MSE) between the ground truth and output. For simulation experiment, reconstructive inputs can be com- putationally acquired by mimicking the actual CASSI reproduced in Meng et al. (2020b). Notably, the real-world captured measurements are disturbed by the noise introduced by the optical sys- tem. Therefore, we add Gaussian noise with standard deviation randomly generated from the range [0, 0.05) to mimic this scenario. For both simulation training/testing and real-wrold testing, the identical mask are employed for fair comparison.
Compared Methods We compare with seven state-of-the-art reconstruction algorithms, includ- ing U-Net (Ronneberger et al., 2015), HSSP (Wang et al., 2019), X-net (Miao et al., 2019), TSA-Net (Meng et al., 2020b), GSM-based method (Huang et al., 2021), PaP-DIP-HSI (Meng et al., 2021) and GAP-net (Meng al., 2020a), among which X-net, TSA-Net and -aseb methods yield plausible results. Peak Signal-to-Nore Racial (PSNR) and Structural SIMilarity (SSIM) (Wang et al., 2004) are used for quantitative comparison. The PSNR is computed by
7

---

Under review as a conference paper at ICLR 2022
Menurrment
Reference
Ground
A-Nei
PaP-DIP-HST
Oar
26.5
2.03
015
Figure 5: Comparison of reconstructed results for synthetic HSI on the flower pating scene. Six ofate-of-theear methods and our proposed method are included. The RGB reference is shown to demonstrate the color (top-left). Our results recover the most details contained in real scene. For the chosen regions (yellow patch) and b (red patch), density curves of our result (bottom-left) show highest correlations with the references (0.9792, 0.9975), which indicates the effectiveness of our method within corresponding waveband (yellow: 565nm~590nm; red: 625mm-~40nm)).
MAXi PSN 10 logio( USE. ), where the channel-wise PSNR value will be gathered for average computing and M 7 denotes the maximum pixel value in ground truth image
We implemented our model by PyTorch and trained it with the ADAM optimizer. There are K=16 network embeddings main body. The learning rate was initialized as 4x10- decreased by half every 50 epochs, for both simulation/real settings. We set batch size 4 for the best validation performance. The training is conducted on a NVIDIA TIATN RTX GPU on simulation data.
4.1 EXPERIMENTS ON SYNTHETIC DATA
As shown in both Tab. and 2, the proposed method (i.e., our best model without rescaling, denoted as SRN v1) has achieved the highest averaged PSNR and SSIM of 10 synthetic testing samples as 33.45 dB and 0.932, respectively, which is an considerable breakthrough. Although the other two versions of our model (i.e.. the proposed method with 1 or 2 rescaling pairs, termed as, SRN v2 and SRN v3) are mainly designed for reducing the computational cost, they also exhibit the competitive performance compared with existing methods. Validated by three variants of our model, the proposed method demonstrated great stability and effectiveness.
Fig. 5 visualizes the reconstructed results between six compared methods and our model. Our recon- structed images contain the most details among four chosen spectral channels including 462. Inm, 498nm, 575.3mm and 625. Imm. We also plot the spectral density curves corresponding two picked regions box "a" and "b" in Fig. 5). The highest correlations (i.e., 0.9792, 0.9975) and highly co- incidence between our curves and reference proves the spectral-wise effectiveness of our model. To clearly check the reconstructive ability of the proposed model (i.e., (v1)), we put RGB reconstruc- tions of eight 256x256x28 reconstructed simulation results Fig.
4.2 EXPERIMENTS ON REAL DATA
We validate the effectiveness of proposed method on hyperspectral images collected from real scenes. We show one example of legoman in Fig. 6 (a) by comparing our method with three best reconstructive methods, X-net (Miao et al., 2019), TSA-Net (Meng al., 2020b) and GSM-based method (Huang et al., 2021), on 8 spectral channels. Our recon- structive results are perceptually complete and demonstrate few artifact, which is obvious on the "face" of the legoman. We pick a small red region for spectral-wise performance verification and gets the highest correlation with the reference generated by a spectrometer.
8

---

Under review as a conference paper at ICLR 2022
457 4bm.
176.5mm
40% fnm
425 Inm
64
RGB Refocmse
Messummnt
1.5
0.5
GSS
0.2
1150
370
$50
620
Figure 6: Real reconstructed hyperspectral images comparison between X-net, TSA-Net, GSM- based method and our method from the real-captured CASSI measurement of legoman scene. The RGB reference is shown demonstrate the color. By comparison, and artifact. Within the chosen region a (green patch), our reconstructed pixels yield the highest correlation with the reference given by spectrometer, indicating the effectiveness of our method within waveband of green, i.e., 500mm~565mm.
results give little distortion
Method
Sparams (M)
FLOPs
PSNR (dB)
4.3 EFFICIENCY ANALYSIS
U-net A-net
31.32
58.99
26,80
62.64
117.98
29.25
TSA-Net GSM-based
Model size. Besides the breakthroughs in terms of performance, we further compare the model size and floating-poin operations (FLOPs) between four E2E deep-learning-bee
3.76
646.35
30.28
SRN v1
1.25
81.84
33.45
SRN
1,44
25.07
SRN v3
1.62
18.57
31.36
methods and three variants of our proposed method. As shown in Tab. 3, the proposed SRN v1 (i.e., the one without rescaling pairs) contains only 2.82% trainable parameters compared with TSA-Net, less than 1/3 params as GSM-based method. We reduces 96% and 98% trainable parameters when compared with U- net (Ronneberger et al., 2015) and X-net, respectively. The other two variants involve a little bit more trainable parameters due to the additional covolutional layers introduced by the rescaling pairs. Our models requires much less computing power and thus can be flexibly deployed, which benefits the real-time reconstruction. We visualize the reconstructed high-resolution HSI, i.c.. 1024x x024x28 in A.4.
Table Analysis efficiency of the deep learning- based algorithms, with a spa- tial size of input 256x256. The anno- tation vl: SRN w/o rescaling pair, v2: SRN with rescaling pair, and v3: SRN with 2 rescaling pair.
Floating-Poin Operations (FLOPs). Given the input of spatial shape 256x256, we count the total amount of addition and multiplication for different deep learning-based model. As shown in Tab. 3, the FLOPS of the proposed model without rescaling pairs is 81.84G, which are slightly more than that of TSA-Net. The underlying intuition is that though the proposed model is rather small, the amount of operations conducted in the model still formidable, which limits the capacity of processing large image patches. Similar problem shows GSM-based method, which contains over 30x FLOPs than SRN v3. As shown in Tab. 3, two variants (i.e., the bottom two models, SRN v2 and SRN v3) mitigate this issue successfully. Typically, reduce 69 5.44 FLOPs by using one rescaling pairs, which is less than 1/3 the FL.OPs of TSA-Net. we further add one more rescaling pair, the resulting FLOPS will be only 18.57G.
CONCLUSION AND FUTURE WORKS
We provided a simple yet promising method spesially-designedd for HSI reconstruction in this paper. The proposed method outperformed existing methods, by setting new state-of-the-ar performance with a negligible amount of model parameters. We alleviated the problem of inefficiency of neural network-based reconstructive methods and thus make step forward to practical application of HSI reconstruction. Our method can not only be solely referenced as a baseline, also conjointly served qualified backbone for future research in the HSI community.
9

---

Under review as a conference paper at ICLR 2022
REFERENCES
Yaser Abu-Mostafa, Malik Magdon-Ismail, and Hsuan- Tien Lin. Learning from data, volume 4. AMLBook New York, NY, USA:, 2012.
J.M. Bioucas-Dias and M.A.T. Figuciredo. A new TwIST: Two-step iterative shrinkagg//hrresholling algorithms for image restoration. IEEE Transactions on Image Processing, 16(12):992-33044 December 2007. 3,
Christopher M Bishop et al. Neural networks for pattern recognition. Oxford university press, 1995.
Marcus Borengasser, William S Hungate, and Russell Watkins. Hyperspectral remote sensing: principles and applications. CRC press, 2007.
Inchang Choi, MH Kim, D Gutierrez, DS Jeon, and G Nam. High-quality hyperspectral reconstruc- tion using a spectral prior. Technical report, 2017. 14
Özgün Çięek, Ahmed Abdulkadir, Soeren S Lienkamp, Thomas Brox, and Olaf Ronneberger. 3d u- net: learning dense volumetric segmentation from sparse annotation. International conference medical image computing and computer-assisted intervention, pp. 424 434-432. Springer, 2016.
D. L.. Donoho. Compressed sensing. IEEE Transactions on Information Theory, 52(4):1289-1306 April 2006.
Ronen Eldan and Ohad Shamir. The power of depth for feedforward neural networks. Conference on learning theory, pp. 907-940. PMLR, 2016.
Candes Emmanuel, Justin Romberg, and Terence Tao. Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information. IEEE Transactions on Information February 2006.
Theory, 52(2):489-509
Ying Fu, Yinqiang Zheng, Imari Sato, and Yoichi Sato. Exploiting spectral-spatial correlation for coded hyperspectral image restoration. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 3727-3736, 2016.
T. Huang, W. Dong, X. Yuan, J. Wu, and G. Shi. Deep gaussian scale mixture prior for spectral compressive imaging. In IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2021. 3,4,7 7, 8, 15, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27
Nabil Ibtchaz and M Sohel Rahman. Multiresunet: Rethinking the u-net architecture for multimodal biomedical image segmentation. Neural Networks, 121:74-87, 2020.
Vladimir Iglovikov and Alexey Shvets. Ternausnet: U-net with vgg encoder pre-trained on ima- genet for image segmentation. arXiv preprint wtXiv:1801.05746. 2018. 3
Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference machine learning, pp. 448-456. PMLR, 2015. 15
S. Jalali and X. Yuan. Snapshot compressed sensing: Performance bounds and algorithms. IEEE Transactions on Information Theory, (5112)800558024 Dec 2019. ISSN 1557-9654. doi: 10. 11 00t.TTT..201922406666
Min H Kim, Todd Alan Harvey, David S Kittle, Holly Rushmeier, Julie Dorsey, Richard Prum and David J Brady. 3d imaging spectroscopy for measuring hyperspectral patterns on solid objects. ACM Transactions on Graphics (TOG), 144 2012.
David Kittle, Kerkil Choi, Ashwin Wagadarikar, and David J Brady. Multiframe image estimation for coded aperture snapshot spectral imagers. Applied optics, 49(36):6824-6833 2010. 3
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, 521 444,
2015. 1
10

---

Under review as a conference paper at ICLR 2022
Shiyu Liang and Rayadurgam Srikant. Why deep neural networks for function approximation? arXiv preprint arXiv: 1610.04661 2016. 5
Yang Liu, Xin Yuan, Jinli Suo, David Brady, and Qionghai Dai. Rank minimization for snapshot compressive imaging. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(): 2990-3006, Dec 2019. 3,
Patrick Llull, Xuejun Liao, Xin Yuan, Jianbo Yang. David Kittle, Lawrence Carin, Guillermo Sapiro, and David J Brady. Coded aperture compressive temporal imaging. Optics Express, 21(9): 10526 10545, 2013. ISSN 1094-4087 (Electronic) 1094-4087 (Linking). doi: 10.1364/OE.21 .000026...
Guolan Lu and Baowei Fei. Medical hyperspectral imaging: review. Journal of biomedical optics, 19(1):010901, 2014. 1
Wenjie Luo, Yujia Li, Raquel Urtasun, and Richard Zemel. Understanding the effective receptive field in deep convolutional neural networks. Proceedings of the 30th International Conference on Neural Information Processing Systems, pp. 4905-4915 2016.
M. A. T. Figueiredo, R. D. Nowak, and S. J. Wright. Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems. pp. 586-597, Dec. 2007. 3
Farid Melgani and Lorenzo Bruzzone. Classification of hyperspectral remote sensing images with support vector machines. IEEE Transactions on geoscience and remote sensing, 42(8):1778- 1790, 2004. 1
Ziyi Meng, Shirin Jalali, and Xin Yuan. Gap-net for snapshot compressive imaging. arXiv preprint arXiv:2012.08364, 2020a. 7,7
Ziyi Meng, Jiawei Ma, and Xin Yuan. End-to-end low cost compressive spectral imaging with spatial-spectral self-attention. In European Conference on Computer Vision (ECCV), August 20206. 1, 2, 3, 6, 7, 8, 15, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27
Ziyi Meng. Mu Qiao, Jiawei Ma, Zhenming Yu, Kun Xu, and Xin Yuan. Snapshot multispectral endomicroscopy. Optics Letters, 45(14):3897-3900 2020c. 1,3
Ziyi Meng, Zhenming Yu, Kun Xu, and Xin Yuan. Self-supervised neural networks for spectral snapshot compressive imaging. arXiv preprint arXiv:2108, /2654, 2021.
Xin Miao, Xin Yuan, Yunchen Pu, and Vassilis Athitsos. X-net: Reconstruct hyperspectral images from snapshot measurement. In /EFF/CVF Conference on Computer Vision (ICCV), 2019. 1, 2, 3, 6, 7, 8, 15, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27
Ozan Oktay, Jo Schlemper, Loic Le Folgoc, Matthew Lee, Mattias Heinrich, Kazunari Misawa, Kensaku Mori, Steven McDonagh, Nils Y Hammerla, Bernhard Kainz, et al. Attention u-net: Learning where to look for the pancreas. arXiv preprint 18X4v:1804.03380 2018.
J. Park, M. Lee, M. D. Grossberg, and S. K. Nayar. Multispectral Imaging Using Multiplexed Illumination. In JEEE International Conference Computer Vision (ICCV), Oct 2007. 7, 14
Antonio Plaza, Jon Atli Benediktsson, Joseph W Boardman, Jason Brazile, Lorenzo Bruzzone, Gus- tavo Hamps-Valls, Jocelyn Chanussot, Mathieu Fauvel, Paolo Gamba, Anthony Gualtieri, et al. Recent advances techniques for hyperspectral image processing. Remote sensing of environ- ment, 11 11:$1 10--110--112,, 2009.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedi- cal image segmentation. In International Conference on Medical image computing and computer- assisted intervention, pp. 234-241. Springer. 2015. 3, 6, 7.9
Wenzhe Shi, Jose Caballero, Ferenc Huszár, Johannes Totz, Andrew Aitken, Rob Bishop, Daniel Rueckert, and Zehan Wang. Real-time single image and video resoutresolution using efficient sub-pixel convolutional neural network. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1874-1883, 2016.

---

Under review as a conference paper at ICLR 2022
Mahdi Soltanolkotabi, Adel Javanmard, and Jason D Lee. Theoretical insights into the optimization landscape of pver-aarameterized shallow neural networks. IEEE Transactions on Information Theory, 65(2):742-765) 2018. 2
Marshall F Tappen. Utilizing variational optimization learn markov random fields. In 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-8. IEEE, 2007. 3
Matus Telgarsky. Benefits of depth in neural networks. In Conference on learning theory, pp- 1517-1539, PMLR, 2016.
Neil C Thompson, Kristjan Greenewald, Keeheon Lee, and Gabriel F Manso. The computational limits of deep learning, arXiv preprint arXiv:2007.05558, 2020. 2
Ashwin Wagadarikar, Renu John, Rebecca Willett, and David Brady. Single disperser design for coded aperture snapshot spectral imaging. Applied Optics, 47(10):B44 B51, 2008. 1,
Ashwin Wagadarikar, Nikos P Pitsianis, Xiaobai Sun, and David Brady. Video rate spectral imaging using coded aperture snapshot spectral imager. Optics Express, 17(8))6368-6388 2009.
L. Wang. Z. Xiong, G. Shi, Wu, and W. Zeng. Adaptive nonlocal sparse representation for dual-camera compressive hyperspectral imaging. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(10):2104-211 2111, Oct 2017. ISSN 0162-8828. doi: 10.1109/TPAMI. 2016.2621051 1,3
L. Wang, C. Sun, Y. Fu, M. H. Kim, and H. Huang. Hyperspectral image reconstruction using a deep spatial-spectral prior. 2019 JEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8024-8033, June 2019. doi: 10.1109/CVPR.2219.0003
1,
Lizhi Wang. Zhiwei Xiong, Dahua Gao, Guangming Shi, Wenjun Zeng. and Feng Wu. High-speed hyperspectral video acquisition with a dual-camera architecture. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4942 4950, 2015.
Lizhi Wang. Zhiwei Xiong, Guangming Shi, Feng Wu, and Wenjun Zeng. Adaptive nonlocal sparse representation for dual-camera compressive hyperspectral imaging. IEFE transactions on pattern analysis and machine intelligence, 39(10):2104-2) 2111, 2016.
Lizhi Wang, Tao Zhang, Ying Fu, and Hua Huang. Hyperreconnet: Joint coded aperture optimization and image reconstruction for compressive hyperspectral imaging. IEEE Transactions on Image Processing, 28(5) May 2019. ISSN 1057-7149. doi: 10.1 10.1109/7 1,
Lizhi Wang, Chen Sun, Maoqing Zhang, Ying Fu, and Hua Huang. Dnu: Deep non-local unrolling for computational spectral imaging. In Proceedings of the IEEE/CVF Conference on Computer Vision and Paltern Recognition (CVPR), June 2020. 1,3
Zhou Wang, Alan C Bovik, Hamid R Sheikh, Eero P Simoncelli, al. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4): 600-612, 2004.
Yang Xu, Zebin Wu, Jun Li, Antonio Plaza, and Zhihui Wei. Anomaly detection in hyperspectral images based on low-rank and sparse representation. IEEE Transactions on Geoscience and Remote Sensing, (444):190020000 2015. 1
Jianbo Yang. Xuejun Liao, Xin Yuan, Patrick Llull, David J Brady, Guillermo Sapiro, and Lawrence Carin. Compressive sensing by learning Gaussian mixture model from measurements, 1EEE Transaction on Image Processing, 2461):106-119, January 2015. 3
X. Yuan, J. Brady, and A. K. Katsaggelos. Snapshot compressive imaging: Theory, algorithms, and applications. IEEE Signal Processing Magazine, 38(2):65-88, 2021. 1,
Xin Yuan. Generalized alternating projection based total variation minimization for compressive sensing. 2016 IEEE International Conference on Image Processing (ICIP), pp. 2539-2543
Sept 2016.
12

---

Under review as a conference paper at ICLR 2022
Xin Yuan, Tsung-Han Han Tsai, Ruoyu Zhu, Patrick Llull, David Brady, and Lawrence Carin. Com- pressive hyperspectral imaging with side information. IEEE Journal of Selected Topics in Signal Processing, 96)):6449978 September 2015.
Yuan Yuan, Xiangtao Zheng, and Xiaoqiang Lu. Hyperspectral image superresolution by transfer learning. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 10(5):1933-1974, 2017.
Siming Zheng, Yang Liu, Ziyi Meng. Mu Qiao, Zhishen Tong, Xiaoyu Yang. Shensheng Han, and Xin Yuan. Deep plug-and-play priors for spectral snapshot compressive imaging. Photon, Res.," (2):B18-B29, Feb 2021. 1,3
13

---

Under review as a conference paper at ICLR 2022
Table 4: Averaged PSNR/BB (1), SSIM (1), model size/M (1) and FLOPs/G (J) on Simplified Res- Block (without Batch Bormalization) and normal ResBlock (containing Batch Normalization) in simulation dataset. Batch size is kept as 4, learning rates initialized as 0.0004 and will be reduced by half every 50 epochs throughout the experiment. Apparently, there is no benefit for adding Batch Normalization layer.
Batch Normoliization
SRN v1
SRN v2
SRN v3
PSNR
SSIM
params
FLOPss
PSNR
SSIM
params
FLOPs
PSNR
SSIM
# params
FLOP:
0.910
0.894
0.870
18,572380
SRN w/o BN
33.45
0.932
1.251
81.839200
31.77
0.900
1.436
25,065161
31.36
0.900
1.620
18.572378
A APPENDIX
A.1 EXPERIMENTAL SETTINGS
CAVE dataset. We used CAVE dataset (Park et al., 2007) to train our model for both simulation and real data experiments. CAVE dataset contains 30 512x512 multispectral images corresponding to wavelength of 400mm~700nm. We generate 175 examplers of 1024 1024 spatial size by random concatenation. To increase the robustness of the training set, images are randomly rotated before concatenation. Besides, another 30 examplers are generated via directly rescaling the original ones. For simulation, image patches of 256x spatial size are randomly cropped and then fed into the model. For real hyperspectral images reconstruction, 660x660 images patches are randomly cropped to be the training samples of our CNN model. For high-resolution image reconstruction, x024x x024x 28 examplers are directly fed into the model as training samples. Fig. 8 (a) shows a rescaled training exampler at the wavelength of 453.3nm and (b) shows a concatenated training exampler at the wavelength of 522.7mm.
KAIST dataset. KAIST (Choi et al., 2017) dataset contains 30 high-quality hyperspectral images of spatial size 2704x3376, In simulation experiment, ten image patches of 256 256x256x28 are used as the testing data. For real HSI reconstruction, 36 image patches of spatial size 1024x 1024 are abstracted and combined with above 205 sample for training purpose. For high-resolution image reconstruction, 17 samples of spatial size 1024x 1024 are used as the testing data. Fig. 8 (c) demon- strates an example of testing image patch 998.0nm wavelength, which used for the third task.
Evaluation metric. Two widely-used validation criteria, Peak Signal-to- Ratio (PSNR) and Structural Similarity (SSIM) are used for quantitative comparison. SSIM is computed by
2Haaly + C1 11 + m2 + Ci
SSIM(z, y)
(5)
20 + +C2 02 + oo + CC2
1
Oay + C3 7 0, C3.
where Mas By are local means, dzs By are standard deviations, Jxy is cross-covariance for image and y.
A.2 PYPER-PARAMETT TUNING
In this experiment, we perform a number of trials on simulation dataset, find the optimal hyperpa- rameter setting for the proposed reconstructive mod-
learning
SRN
SRN
SRN
PSNR
5SIM
PSNR
SSIM
PSNR
SSIM
els. Specifically, ablation studies on batch normaliza-
0.0002
0.924
32.26
0.913
31.75
0.906
tion layer 4, initial learning rate of 0.0002, 0,0004, 0.0006 (see Table 5 for averaged PSNR and SSIM val- ues), and batch size of 2,4,8 (see Table 6 for aver- aged PSNR and SSIM values) are involved in three ver- sions of proposed model (SRN v1, SRN v2.SRN v3).
0.0006
0.0044
33.45
32.02
0.932
0.907
32.26
31:77
0.900
0.913
30.08
36
0.900
0.870
Table 5: Averaged PSNR (dB) and
SSIM values for different learning rate in simulation experiment, where batch size is kept as 4 and the learning rates is reduced by half every 50 epochs.
Besides, to further validate the superiority of Simplified ResBlook (RB) over normal ResBlock, we compare the
14

---

Under review as a conference paper at ICLR 2022
Table 7: PSNR (dB) and SSIM values 1024x 28 simulation testing data. Reconstruction is conducted by SRN v2 model (i.e., the model with one rescaling pair), where optimal hyperparameter settings argued in Section A.2 are leveraged in the training phase.
scene
5
11
12
14
17
Aug.
PSNR
28.78
29.97
30.40
31,74
33.51
30.96
34,56
34.46
34.49
27
34.95
33.07
31.88
33.36
29.20
29.85
30.03
31,66
SSIM
0,869
0.913
0.43
0,937
0.966
0,958
0.958
0.962
0,956
0.821
0.955
0,948
0,819
0,928
0,937
0.866
0.926
0,911
model size, FLOPs (FLoating-ponl point OPerations) and quantitative evaluations (averaged PSNR and SSIM) both settings by only adding the Batch Normalization (loffe & Szegedy, 2015) layer be- hind the first convolutional layer in each Simplified ResBlock (RB) (shown in Table 4). For compari- son, TSA-Net (Meng et al., 2020b) achieves 1.B and 0.893 respectively, with 44.25M parame- ters and 80.08G FLOPs. Obviously, Batch Normalization layers not only result in more parameters and FLOPs, also undermine the performance. As demonstrated in the three tables above, we choose the optimal hyperparameter setting according to our best model's performance.
learning rate = 0.0004,
(6)
batch size 4.
A.3 STATE-OF-THE-AR PERFORMANCE
In this section, we visualize more reconstructed HSIs produced by our best model, which is SRN v1 (i.e.,
the original model without rescaling pair). Firstly, we
butch size
PSNR
SRN v1
SSIM
PSNR
SRN vZ
SSIM
PSNR
SRN
SSIM
add one additional simulation and one additional real re-
32.24
0.509
31.44
0.894
31,02
0.658
constructed results with spectral analysis (i.e., density curves) in Fig. 9 and Fig. 10. For comparison, the same sixteen spectral channels of reconstructed HSIs gener- ated by X-net (Miao et al., 2019), TSA-Net (Meng et al., 2020b) and GSM-based method (Huang al., 2021) are also plotted, shown in Fig. 1 1~~Fig. 20. Better visualiza- tion performance be achieved by zooming in.
33.45
30.05
0.932
0.500
Table 6: Averaged PSNR (dB) and
SSIM values on different batch sizes in simulation dataset. Initial learning rate is kept as 0.0004 and reduced by half every 50 epochs.
A.4 HIGII-RESOLUTION HSI RECONSTRUCTION
As compared in Fig. 1 right, our model outperforms the TSA-Net (Meng et al., 2020b) on simula- tion data of spatial size 256 256 with only 2.82% trainable parameters. As compared Tab. ??, we outperforms GSM and concretely reduce FLOPs by > 34times, by comparison. Besides, the variants of original model, which are equipped with rescaling pairs, can still achieve promising per- formance with less than 1/3 FLOPs, as shown in Tab. 1 and Tab. 2. Both of the minimal model size and insignificant amount of computing enables our method a perfect candidate for higher-resolution hyperspectral image reconstruction. Therefore, ing previous 1024x 1024x26 28 examplers as training samples, we reconstruct high-resolution HSI accordingly through SRN v2 model (i.e. the model with one rescaling pair). The testing is composed of 17 images patches of size 1024x 1024 x28 abstracted from KAIST dataset. We also generate a 1024x 1024 noise map as mask for training and testing. As far we know, this is the first time to reconstruct HSI at such a high resolution using deep learning barning-based based method. Table 7 reports the PSNR and SSIM values on all 17 scenes. Fig. 21~Fig. 26 show the reconstructed images on 16 selected spectral channels.
setup simulation experiment by directly
15

---

Under review as a conference paper at ICLR 2022
Table 8: PSNR (dB) and SSIM values 256x256x28 simulation testing set. To create well- trained model that can easily adapt to diverse masks, We train SRN v2 model with masks gener- ated from the same distribution, during which optimal hyperparameter settings (i.e. learning rate is 0.0004, batch size is 4) argued in Section A.2 are leveraged.
scene
2
3
4
5
6
7
8
10
Avg.
PSNR
32.56
28.41
31.18
39.19
29.51
33.21
26.32
31.27
32.05
29.82
31.35
SSIM
0.901
0.841
0.899
0.948
0.898
0.927
0.837
0.918
0.908
0.893
0.897
Figure 7: RGB reconstruction of eight reconstructed simulation results.
Sat Traniig Euspply cocking
(b) Unisin Exuepla: osscaanniin
Hial--cciiie Groudd Irage
Figure 8: 1024x 1024 training/testing hyperspectral images for different tasks, shown on specific wavelengths.
Mensurement
RGB Reference Ground Trath
HSSP
A-Na
GSM based
PaP-DIP-HST
GAP-act
Ours
462 Joas
2015
D0G
201
4000
4010
ST5 Som
Figure 9: Comparison of reconstructed results for synthetic HSI (Jronman and Donald Duck scene). Six state-of-the-al methods and our method (right-column) included. The RGB reference is shown to demonstrate the color (Top-leftt. The spectral curves (bottom-left) are corresponding to the boxes denoted in measurement.
16

---

Under review as a conference paper at ICLR 2022
RGB Reference
Measurement
A-Net
TSA-Net
GSM-based
Ours
498.0nm
575.3mm
b)
625.1nm
Clainenn 2011
300
Figure 10: Reconstructed real HSI comparison between X-net, TSA-Net, GSM-based method and our proposed model v1, (i.e., with one rescaling pair) from the real CASSI measurement (top-left).
17

---

Under review as a conference paper at ICLR 2022
--net
TSA-Net
GSM-based
Ours
Figure 11:
Comparison of reconstructed results produced by X-net Miao et al. (2019), TSA- Net Meng et al. GSSM method (Huang et al., 2021) and our best model on simulation dataset scene Better visualization performance can be achieved by zooming
18

---

Under review as a conference paper at ICLR 2022
-et
TSA-Net
GSM-based
Ours
Comparison of reconstructed results produced by X-net Miao et al. (2019). TSA- Net Meng et al. (S0- method (Huang et al., 2021) and our best model on simulation dataset scene 2. Better visualization performance can be achieved by zooming in.
Figure 12:
19

---

Under review as a conference paper at ICLR 2022
7-net
TSA-Net
Gase-
Ours
Comparison of reconstructed results produced by X-net Miao et al. (2019), TSA- Net Meng et al. (2020b),GSM-base) method (Huang et al., 2021) and our best model on simulation dataset scene 3. Better visualization performance can be achieved by zooming
Figure 13:
20

---

Under review as a conference paper at ICLR 2022
A-net
TSA-Net
GSM-based
Ours
Figure 14:
Comparison of reconstructed results produced by X-net Miao et al. (2019), TSA- Net Meng et al. GSSM method (Huang et al., 2021) and our best model on simulation dataset scene 4. Better visualization performance can be achieved by zooming in.
21

---

Under review as a conference paper at ICLR 2022
7-net
TSA-Net
12
30
20
GSM-based
65
Ours
Comparison of reconstructed results produced by X-net Miao et al. (2019). TSA- Net Meng et al. GSMt method (Huang et al., 2021) and our best model on simulation dataset scene 5. Better visualization performance can be achieved by zooming in.
Figure 15:
22

---

Under review as a conference paper at ICLR 2022
e-net
POTTR
DOITER
POTEY
POT
NOT
TSA-Net
POTT
POT
GSM-based
POTEE
Ours
MIT
POT
Figure 16: Comparison of reconstructed results produced by A-net Miao et al. (2019), TSA- Net Meng al. (0200b,,GMM-basee method (Huang al., 2021) and our best model on simulation dataset scene 6. Better visualization performance can be achieved by zooming in.
23

---

Under review as a conference paper at ICLR 2022
--net
TSA-Net
GSM-based
Ours
Figure 17: Comparison of reconstructed results produced by X-net Miao et al. (2019), TSA- Net Meng et al. GSM-ae) method (Huang et al., 2021) and our best model on simulation dataset scene 7. Better visualization performance can be achieved by zooming in.
24

---

Under review as a conference paper at ICLR 2022
nene
0
TSA-Net
punca
Momos
GSM-based
O
Ours
Comparison of reconstructed results produced by X-net Miao et al. (2019), TSA- Net Meng et al. (2020b) method (Huang et al., 2021) and our best model on simulation dataset scene 8. Better visualization performance can be achieved by zooming in.
Figure 18:
25

---

Under review as a conference paper at ICLR 2022
A-net
TSA-Net
GSM-based
Ours
Figure 19: Comparison of reconstructed results produced by A-net Miao et al. (2019). TSA- Net Meng al. (20-bb), method (Huang al., 2021) and our best model on simulation dataset scene 9, Better visualization performance can be achieved by zooming in.
26

---

Under review as a conference paper at ICLR 2022
--net
TSA-Net
GSM-based
Ours
Figure 20: Comparison reconstructed results produced by X-net Miao et al. (2019), TSA- Net Meng et al. (G200),,GM-bbseee method (Huang et al., 2021) and our best model simulation dataset scene 10. Better visualization performance can be achieved by zooming in.
27

---

Under review as a conference paper at ICLR 2022
457.6 nm
466.8 nm
476.55 nm
486.9 nm
498 nm
509.9 nm
322.7 nm
336.5 nm
Ground kuit:
543.8 nm
551.4 nm
567 5 nm
684.3 nm
604.2 nm
614.4 nm
625.1 nm
648. 1 nm
Prsfiction
Figure 21: 1024 10 high-resolution HSI reconstruction (scene 4): Sixteen spectral channels of ground truth HSI (upper two rows) and corresponding reconstructive results by SRN v2 model (lower two rows). Better visualization performance by zooming in.
457.6 nm
466.8 nm
4m
486.9 nm
498 nm
509.9 nm
522.7 7 nm
536.5 nm
truth trush
Ground
543.8 nm
551.4mmm
BUY 5mm
584 14433mm mm
604 nm
614.4 nm
626, nm
648. 1 nm
Prediction
Figure 22: 1024x x024x28 high high-resolution HSI reconstruction (scene 6): Sixteen spectral channels of ground truth HSI (upper two rows) and corresponding reconstructive results by SRN v2 model (lower two rows). Better visualization performance by zooming in.
28

---

Under review as a conference paper at ICLR 2022
466.8
486.9
nm
nm
336.5
Ground trath
551.4 nm
nm
nm
614.4
648.1
Predkction
Figure 23: 1024 x x024x28 high-resolution HSI reconstruction (scene 7): Sixteen spectral channels of ground truth HSI (upper two rows) and corresponding reconstructive results by SRN v2 model (lower two rows). Better visualization performance by zooming in.
457.6 nm
466.8 nm
476.5
486.9
498
509.9
522.7 nm
536.5 nm
Ground muzs
343.8
551.4
587.5
584.3
614.4
825.1
Predection
Figure 24: 1024 1024x28 high-resolution HSI reconstruction (scene 8): Sixteen spectral channels of ground truth HSI (upper two rows) and corresponding reconstructive results by SRN v2 model (lower two rows). Better visualization performance by zooming in.
29

---

Under review as a conference paper at ICLR 2022
457.6
466.8
476.5
486.9
498
509.
522.1
536.5
Groaud trish
nm
nm
mm
Prodiction
Figure 1024 1024x28 high-resolution HSI reconstruction (scene 15): Sixteen spectral channels of ground truth HSI (upper two rows) and corresponding reconstructive results by SRN v2 model (lower two rows). Better visualization performance by zooming in.
457.6
466.8
476.5
486.9
498 nm
509.9 nm
522.7 nm
536.5 nm
Ground truit
00
$43.8
551 1.4.4
567.5 nm
584.3
604.2
614.4
8225. nm
848.1
Prediction
Figure 1024 1024x28 high-resolution HSI reconstruction (scene 17): Sixteen spectral channels of ground truth HSI (upper two rows) and corresponding reconstructive results by SRN v2 model (lower two rows). Better visualization performance by zooming in.
30


Reviews:
[{"review_id": "t6PlxjfkjK3", "reviewer": "Reviewer_gU7n", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\nThe writing of this paper is beautiful and informative.\nThe performance of the proposed method exceeds SOTA.\nDemonstration of real hardware experiments is a plus.\n\nCons\nAbout network position. In my understanding, new backbones should and could be integrated into other reconstruction networks, such as TSA-net, GSM-net, GAP-net, by replacing the U-Net in these networks. The backbone net should be plug-and-play and compatible with previous reconstruction networks. But, in this paper, the backbone is treated as a new reconstruction network, and be parallel with previous reconstruction networks. Further, the new backbone should be tested on other problems, such as video compressive imaging, image compressive sensing to demonstrate it is a backbone and not a new reconstruction method.\nAbout analysis and motivation. There should be more analysis and motivation on the design principles. It is not clear why the proposed backbone can achieve such superior performance. There is some depiction about the receptive field, but I think RF is not the key issue, since networks with small or large RFs are easy to design and not all these networks could achieve such performance. It should demonstrate why the proposed network is the specific one that others can not replace. \n\nNeed Clarity \nIn the first paragraph in Sec. 3.2, it seems other methods use the measurement y during reconstruction, but the proposed method only uses F_Y. I think y and F_Y are different for HSI reconstruction. Please clarify these differences. \nEq. (1), when shifting signal frames of different wavelengths, how to deal with the boundary? Does it have an influence on real hardware experiments?\nFigure. 6, spectral curves. I notice that the curves between Ours and References have a big deviation. Please check if something wrong. \n\n", "summary_of_the_paper": "This paper proposes a new backbone for hyperspectral image reconstruction for CASSI system. The novel thing, as claimed, is a new network that can achieve high reconstruction fidelity with high efficiency. ", "summary_of_the_review": "I have concerns about the motivation of network design, together with the position of the proposed network.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "rAPpv_F919h", "reviewer": "Reviewer_ms6v", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengthes:\n- This work proposes a backbone network working very well for snapshot compressive hyperspectral imaging and achieved SOTA performance in a number of related benchmarks over recent related works that were presented at CVPR'21, ECCV'20.\n\nWeaknesses:\n- The main idea of the proposed method is to use skip connections between blocks and groups of blocks. However, there have been similar works on using multiple, repeated skip connections for backbone networks such as\n[Huang CVPR 2017] Densely Connected Convolutional Networks\n[Zhang ECCV 2018] Image Super-Resolution Using Very Deep Residual Channel Attention Networks\nThe proposed method and the above works (and their related works - they have been cited a lot) seem to share common ingredients and this work should address it properly to clearly show the contribution of the proposed backbone. Comparing with U-Net seems to have too limited scope only for hyperspectral community.\n- This work says \"We argue that in low-level regression tasks, its unnecessary for the highest RF to cover the whole image but is supposed to be large enough to capture the neighboring for the estimated pixel.\" However, compressive sensing matrix M generates samples by summing over the whole image domain with proper weights (0 or positive values). In other words, narrowing down the RF can potentially miss the encoded information in CASSI and I believe that that's why others in HSI used U-Net instead. Thus, if this argument is true, then this part should be carefully written again with proper arguments. Even though there was no issue in the experimental results, note that the dataset was not that large, so it is still possible to have good results in limited dataset with theoretical flaw to miss some information.\n- For \"EFFICIENCY MAXIMIZATION\", this work sed a number of technics such as downsampling, upsampling and pixelshuffling. However, these technics are common in other image recovery domains such as super resolution as also indicated in this work. \n\nComments:\n- This work says \"As a result, a dimensional-expansion mapping function (2D to 3D) is required, for which reason such a mapping relationship approximation is deemed to be much harder than general image regression tasks\". I am not sure if this claim is true. There are a number of ways to define how hard the inverse problem is and there are a number of general image regression tasks with very ill-posed forward model.\n- It is not easy to agree with the motivations in section 2. For Proper Data Volume, it says \"simpler model might be more promising.\" This paragraph argued with VC dimension, but there is no analysis on VC dimension to support this motivation. For Ideal reconstruction rate, there is also possibility to simply use non-compressed samples over CASSI for real-time applications. What are the cases of using CASSI with real-time applications? Please consider revising this part with more convincing details and with removing some parts that can not be supported later by experiments.\n- It says \"we posit the possibility of network construction with minimum inductive bias\". How can we know that it is true? Any supporting data for it? Small RF seems to introduce bias by removing some measurements to use and small training / evaluation datasets seems also to be an issue for it.\n- Please comment on training time. \n", "summary_of_the_paper": "This work proposes a novel backbone network for hyperspectral image reconstruction for snapshot compressive imaging (CASSI). The key structure of the proposed method is illustrated in Figure 3, essentially using skip connections between Conv-ReLU-Conv blocks repeatedly. A number of techniques such as downsampling, upsampling, pixelshuffling were employed to maximize efficiency. The proposed method achieved SOTA performance in the given CASSI reconstruction tasks over recent works with small network size and fast computation.", "summary_of_the_review": "This work may be a nice contribution to the field of hyperspectral imaging, but it is not easy to see this work as a good contribution for ICLR; using dense skip connections is not new even though the way of connecting them is different from others, using pixelshuffling is common in super resolution field, and narrowing down the RF does not seem to be theoretically solid since it is then possible to miss some information.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "Ml3LpCFrJI", "reviewer": "Reviewer_dXDL", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper proposed an intersting points, that is to say, in lowlevel regression tasks, its unnecessary for the highest RF to cover the whole image but is supposed to be large enough to capture the neighboring for the estimated pixel. The authors also compared the proposed method to the U-Net regarding the receptive field. In fact, the authors proved in the experimental section that the proposed architecture can achieve better performance compared to Unet. I'm still confusing about the conclusion that enough RF to capture the neighboring for the estimated pixel is better for the hightest RF to cover the whole image. I suggest an analysis study should be presented about the RF and the proposed method.\nThe authors proposed three kinds of variants on the bisis of the proposed backbone. The parameters of different variants stay similar, but the flops changes significantly. Furthermore, the performance of these three variants are also of big diversity. I'm wondering what is the main factor to increase the peformance. The flops indicates the training efficiency. Why not point out the training time to show a direct efficiency of different methods?\nThe paper described that adjacent spectral channels tend to be more correlated to each other in general HSIs. What is the reference?\nHow many training samples and total epochs utilized in the training stage for simulated and real experiments?\nAs shown in Fig. 5, the proposed method introduced the artifact at the right top of the image, compared to that of GAP-net.", "summary_of_the_paper": "Different from the previous works about Unet structure for HSI reconstruction, this paper introduces a new backbone, which is efficient and lightweight. The experiments also demosntrate that the proposed model can significantly outperfom the previous works in quantitative evaluation results. From the above summary, this paper should be of significance to the application society of CASSI imaging reconstruction.", "summary_of_the_review": "The experimental results of the proposed method have largely outperformed baseline methods.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "L2_MYOyBOQ", "reviewer": "Reviewer_NYcV", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths: \n\n- This paper address an important problem in hyperspectral image processing, i.e. reconstruction from single images. The proposed method is well motivated with the drawbacks of the current methods clearly described.\n\n- The proposed method has a simple network structure, therefore, can be easily re-implemented. The network structure has well addressed the motivation of this paper.\n\n- The proposed method is very effective. It produces higher PSNR and SSIM than previous approaches. It also significantly reduces the number of parameters and flops, therefore, is efficient and affordable when facing video processing tasks.\n\nWeakness:\n\n- Hyperspectral image reconstruction has attracted more and more attention in the past several years. Many papers do not study the mechanism behind the imaging process, the property of hyperspectral images and how to make the reconstructed images really useful, instead focus on adopting some deep learning techniques to learn the mapping from single-shot measurement to multiple channel image. This paper is one of them, and is not exciting with so many papers doing the same thing. I'd rather want to see a paper that works on reconstructing images with a wider spectral range and a much higher number of bands, more complex scenes consisting of multiple objects, and addressing a real-world problem such as distinguishing the fine differences of similar materials. These are the reason that makes hyperspectral image useful and the reconstruction of hyperspectral image meaningful.\n\n- The novel contribution on network mechanism and image representation learning is low. The design of the structure seems to be empirical. Although the proposed model is simple, its rationale has not been clearly explained, especially why and how it can produce high reconstruction quality. \n\n- The presentation of the paper shall be improved. There are some typos and errors that shall be fixed, e.g. \"nad\", \"Fig. 6 (a)\" (there is no Fig. 6 (a)), and \"Tab ??\". \n\nMore comments:\n\n1) When introducing U-net, the properties of medical images are described, which is not closely relevant to this paper.\n2) Discussion on the receptive field is not convincing. There are differences between medical image and hyperspectral image, but such difference is simply on the data and can be easily solved by introducing different training samples. It is not related to the network model. The authors commented that \"medical image processing task is always more locally focused while the HSI reconstruction tends to be globally comprehensive\". However, isn't HSI reconstruction also focus on local details? This has been highlighted in the experimental results, e.g. Figure 5 that shows more details of the scene?\n3) In evaluating the quality of reconstructed spectral responses, has the authors considered the spectral angle distance between the estimated signal and the ground truth? This is more commonly used in remote sensing for quantitative evaluation.\n4) Since the depth of network is highly important, an analysis on the impact of K shall be given.\n5) \"Within the chosen region (green patch), our reconstructed pixels yield the highest a correlation with the reference given by spectrometer, indicating the effectiveness of our method within waveband of green, i.e., 500nm-565nm\". Why green patch? How is the overall quality of the reconstructed spectral responses?\n6) Section 3.1 is highly similar to the description on CASSI in some published papers. Even if the papers may be written by the same authors, such similarity shall be reduced or avoided. ", "summary_of_the_paper": "This paper introduces a hyperspectral image reconstruction method that aims to produce an image with multiple channels from a single-shot measurement with high reconstruction quality and efficiency. This is achieved through the development of a simple reconstruction network that adopts some tricks such as spatial and spectral invariant learning and identity connection. The proposed method was tested on CAVE and KAIST datasets, and was compared with several state-of-the-art methods. The results show clear advantages of the proposed method.", "summary_of_the_review": "This paper introduces a simple but effective and efficient method for hyperspectral image reconstruction. However, the novelty and significance of the contribution of the paper are not high. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a new neural network architecture for hyperspectral image reconstruction. The paper received borderline/negative reviews. Significant concerns were raised about the novelty and significance of the contribution. Unfortunately, the authors did not upload a rebuttal, preventing the reviewers from changing their opinion about the paper. There is therefore no reason to overturn their recommendation.
Decision: Reject