Untrained Neural Nets for Snapshot Compressive Imaging: Theory and Algorithms
Authors: Mengyu Zhao
Xi Chen
Xin Yuan
Shirin Jalali
Venue: NeurIPS 2024 poster (2024)
Keywords: Snapshot Compressive Imaging
Deep Image Prior
Inverse Problem
Primary area: machine_learning_for_other_sciences_and_fields

Abstract: Snapshot compressive imaging (SCI) recovers high-dimensional (3D) data cubes from a single 2D measurement, enabling diverse applications like video and hyperspectral imaging to go beyond standard techniques in terms of acquisition speed and efficiency. In this paper, we focus on SCI recovery algorithms that employ untrained neural networks (UNNs), such as deep image prior (DIP), to model source structure. Such UNN-based methods are appealing as they have the potential of avoiding the computationally intensive retraining required for different source models and different measurement scenarios. We first develop a theoretical framework for characterizing the performance of such UNN-based methods. The theoretical framework, on the one hand, enables us to optimize the parameters of data-modulating masks, and on the other hand, provides a fundamental connection between the number of data frames that can be recovered from a single measurement to the parameters of the untrained NN. We also employ the recently proposed bagged-deep-image-prior (bagged-DIP) idea to develop SCI Bagged Deep Video Prior (SCI-BDVP) algorithms that address the common challenges faced by standard UNN solutions. Our experimental results show that in video SCI our proposed solution achieves state-of-the-art among UNN methods, and in the case of noisy measurements, it even outperforms supervised solutions. Code is publicly available at [https://github.com/Computational-Imaging-RU/SCI-BDVP](https://github.com/Computational-Imaging-RU/SCI-BDVP).

Untrained Neural Nets for Snapshot Compressive Imaging: Theory and Algorithms
Mengyu Zhao', Xi Chen', Xin Yuan', Shirin Jalali ECE Department, Rutgers University, New Brunswick 2 School of Engineering, Westlake University
Abstract
Snapshot compressive imaging (SCI) recovers nigh-dimensional (3D) data cubes from a single 2D measurement, enabling diverse applications like video and hyper- spectral imaging to go beyond standard techniques in terms of acquisition speed and efficiency. In this paper, we focus on SCI recovery algorithms that employ untrained neural networks (UNNs), such as deep image prior (DIP), to model source structure. Such UNN-based methods are appealing as they have the po- tential of avoiding the computationally intensive retraining required for different source models and different measurement scenarios. We first develop a theoretical framework for characterizing the performance of such UNN-based methods. The theoretical framework, on the one hand, enables us to optimize the parameters of fata-modulating masks, and on the other hand, provides fundamental connection between the number of data frames that can be recovered from single measure- ment the parameters of the untrained NN. We also employ the recently proposed bagged-deep-iman -magee-rror (bagged-DIP) idea to develop SCI Bagged Deep Video Prior (SCI-BDVP) algorithms that address the common challenges faced by stan- dard UNN solutions. Our experimental results show that in video SCI our proposed solution achieves state-of-the-ar among UNN methods, and in the case of noisy measurements, even outperforms supervised solutions. Code is publicly available at lttttps://githut www.:///atcoomoooaaiicciacaa ioal Imaging-RU/SCI-BR RU/SCI-BDV
Introduction
Snapshot Compressive Imaging (SCI) refers imaging systems that optically encode three- dimensional (3D) data cube into a two-dimensional (2D) image and computationally recover the 3D data cube from the 2D projection. As a novel approach in computational imaging, SCI has attracted significant attention in recent years. Initially proposed for spectral imaging [1] its application has since expanded various fields, including video recording 22, depth imaging [ and coherence tomography [4] (Refer to [5] for comprehensive review).
The key advantage of SCI systems lies in significantly accelerating the data acquisition process. Traditional hyperspectral imaging methods, for example, often encounter bottlenecks due to their reliance spatial wavelength scanning, leading to time-consuming operations. In contrast, hyperspectral SCI systems capture measurements across multiple pixels and wavelengths in a single snapshot, effectively bypassing this limitation 66
The optical encoding process in SCI systems can be mathematically modeled as a linear measurement system, characterized by a sparse and structured sensing matrix, commonly referred to 'mask'. Consequently, SCI recovery algorithms aim to reconstruct high-dimensional (HD) 3D data from
*Shirin Jalali the corresponding authors <shirin,jalal @ rudg,rs.edu>.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

highly underdetermined system of linear equations. A wide range of SCI recovery methods has been proposed in the literature, which can broadly be categorized into:
Classic approaches: These methods model source structure using convex regularization functions and employ convex optimization techniques (e.g., 77 g [Q]). While robust to measurement and source distribution non-idcalities, they are typically limited to simpler structures and challenging to extend to 3D HD data cubes central to SCI applications. DNN-based methods: These approaches use deep neural networks (DNNs) to capture complex source structures, learning from training
data. They can be further categorized as: i) End-to-ena solutions (e.g., [2
14
16 ); ii)
Iterative plug-an plug-and-pe While these methods extend beyond simple structures model intricate source patterns, they require extensive training data, often struggle with generalization, and are computationally intensive.
solutions (e.g.. g. I7 18;; iii) Unrolled methods (e.g., 19
20
21
22
23 24)).
An alternative approach to SCI recovery involves using UNNs, such as deep image prior (DIP) 25 or deep decoder [6]. to model the source structure. These methods capture complex source structures without requiring any training data. Existing UNN-based SCI solutions either recover the image end-to-end in one shot [26] or employ iterative methods akin to projected gradient descent (PGD) [27]. Despite their advantages, these approaches often exhibit lower performance compared pre-trained methods and may require additional data processing steps for enhancement.
In this work, focus on leveraging UNNs address the SCI problem. We begin by establishing a theoretical framework for analyzing UNN-based methods, providing insights into optimizing the adjustable SCI masks, under both noise-free and noisy measurements. We then explore DIP-based algorithms and introduce SCI-BDVP solutions. Our results demonstrate the robustness of these solutions measurement noise and their competitive performance across diverse datasets, using consistent set of parameters.
1.1 Contributions of this Work
Theoretical: We theoretically characterize the performance DIP-based SCI recovery optimization for both noise-free and noisy measurements. Using theoretical results, establish upper bound on the number of frames that can be recovered from a single 2D measurement, as a function of the dimensions of the DIP. Furthermore, we show how the developed theoretical results enable optimize the parameters of the masks for both noisy and noise-free cases, enhancing the performance of the recovery process.
Algorithmic: Inspired by the newly proposed Bugged-DIP algorithm for the problem of coherent imaging 28), developed to address common shortcomings of DIP-based solutions for inverse prob- lems, we explore the application of bagged-DIP for SCI recovery. We conduct extensive experimental evaluations, demonstrating the following: i) Confirmation of our theoretical results on the optimized masks for both noise-free and noisy measurements. ii) The proposed SCI-BDVP solution robustly achieves state-of-the-ar performance among UNN-based solutions in the case of noise-free mea- surements. iii) In scenarios with noisy measurements, our proposed method achieves state-of-the-art performance among both end-to-end supervised and untrained methods.
1.2 Notations
Vectors are represented by bold characters like x and y. ||x||2 denotes the l norm of x. For X E X22, Vec(X) E RRn denotes the vectorized version of X, where = n2n.. This vector created by concatenating the columns of X. Given A, B E I xn2 Y= A O B denotes the Hadamard product of A and B, such that Yú A, for all i,j. Sets are represented by Calligraphic letters, like A, B. For finite set A, |A| denotes the number of elements in A. Throughout the paper, log refers to the logarithm in base 2, while in denotes the natural logarithm.
Related Work
UNNs for SCI. While the majority of SCI recovery algorithms developed for various applications fall
under classic optimization-base methods (e.g., [ 9 10) or supervised DNN-based methods (4).
recent years, there has been increasing interest in leveraging UNNs solving inverse problems. In SCI recovery, this trend has been motivated by the diversity of applications and datasets encountered

---

in various SCI applications, necessitating the availability of pre-trained denoising networks tailored to different resolutions and noise levels for various datasets. Another challenge with these traditional solutions is their robustness to various problem settings, such as measurement noise. These challenges have spurred a notable interest in developing solutions that harness the ability of DNNs to capture complex source models while not relying on training data.
While deep image priors (DIPs) have been applied various inverse problems 22
30 31, their
application to SCI recovery has been limited. The authors in 27 developed an iterative DIP- solution for hyperspectral SCI. To enhance the performance and address the challenges faced by DIP-based methods in terms of falling into local minimas, they initialize the algorithm by the solutions obtained by GAP-TV [9. In (26), the authors propose Factorized Deep Video Prior (DVP), which is a DIP-based SCI recovery algorithm for videos, which is based on separating the video into foreground and background and treating them separately. [32 develops DIP-based solution for compressed ultrafast photography (CUP), where in addition to the normal SCI 2D measurement and additional side information consisting of the integral of all the frames (referred to as the time-unsheared view in [32] is also collected. The video is reconstructed using end-to-end approach using the DIP to enforce the source model. In [33], the authors leverage the concept of video snapshot compressive imaging (SCI) reconstruction to develop algorithm for snapshot temporal compressive microscopy. They propose an iterative algorithm that utilizes UNNs to incorporate the source structure.
In the context of image recovery from underdetermined measurements corrupted by speckle noise, the authors in [28] recently proposed the idea of bagged-DIs which is based on independently training multiple DIPs operating at different frame sizes and averaging the results. In this paper, we extend the idea to videos and construct a bagged-DVP, which as we show in our experimental results robustly achieves wate-of-thhe-a performance among all UNN-based SCI video recovery methods.
Mask optimization. In various SCI applications, one can design the masks, which are typically binary-valued and used for modulating the input 3D data cube. This naturally raises the question of optimizing the masks to improve the performance. To address this problem, several empirical works have designed solutions that simultancously solve the SCI recovery problem and optimize the masks. In 34, the authors design an end-to-end autoencoder network train the reconstruction and mask simultaneously for video data and find the trained mask has some distribution such as non-zero probability around 0.4 and varies smooth spatially and temporally. Similarly, in (35), deep unfolding style networks are trained to simultancously reconstruct 3D images and also optimize the binary masks. They show that for the empirically jointly optimized masks have non-zero probability of around 0.4. The authors in 1T]] design an end-to-end VIT-based SCI video recovery solution that simultaneously learns the reconstruction signal and the mask. They consider special type of mask that constrained by their hardware design.
Due to the highly non-convex nature of the described joint optimization problem, empirically-joinie optimized solutions are likely to converge to suboptimal results. Furthermore, the optimized solution, inherently dependent on the training data, lacks theoretical guarantees. To address these limitations, framework to theoretically optimize the binary-valued masks in the case of noiseless measurements and showed that in that case the optimized probability of non-zero entries is always smaller than 0.5. Here, we theoretically characterize the performance of UNN- based SCI recovery methods and show a consistent result in the case of noise-free measurements. Interestingly, as shown in our experiments, for noisy measurements, the optimized probability can be larger than 0.5. We derive novel theoretical results explaining this phenomenon.
36 employed a compression-bast
3
DIP for SCI inverse problem
3.1 SCI inverse problem
The objective of a SCT system to reconstruct three-dimensional (3D) data cube from its two- dimensional (2I) compressed measurement. Specifically, let X E X represent the target 3D data cube. In an SCI system, X is mapped to a singular measurement frame Y E R Xn2, This mapping, particularly as implemented in hyperspectral SCI and video SCI 2, can be modeled as linear system as follows (2 37: Y
Σ C @ X; Z. Here, C E R x x represents the sensing kernel (or mask), and Z E Rd X denotes the additive noise. The terms C, = C(:,;,) and
3

---

X; =,x((,;,)) E Rmix"a correspond to the b-th sensing kernel (mask) and the associated signal frame, respectively.
To simplify the mathematical representation of the system, we vectorize each frame as x; n122- Then, vectorize the data cube X by concatenating the B Vec(X,) I with
vectorized frames into a column vector x E R"Ø as = ......... xB] T Similarly, we define y = Vee(Y) E R" and z = Vec(u) E IR". Using these definitions, the measurement process can
also be expressed as
y H
(1)
The sensing matrix H E RexaB is highly sparse matrix that is formed by the concatenation of B diagonal matrices as
DI DI...., Dnl.
where, for i I,... B, B, = diag( Vec(C,)) Rax" Using this notation, the measurement vector
can be written y = Ei= D,x; The goal of a SCI recovery algorithm is to recover the data cube from undersampled measurements y, while having access to the sensing matrix (or mask) H.
3.2 Theoretical analysis of DIP-based SCI recovery
The Deep Image Prior (DIP) 25
hypothesis provides a framework for understanding the potential of UNNs capturing the essence complex source structures without requiring training data. Define Q S R" as the class of signals of interest (e.g., class of video signals consisting of B frames.). Also, let go IPP + IR" represent a UNN parameterized by E R Informally, DIP hypothesis states that any signal in Q can be presented as the output of the DIP parameterized by parameters 0 E IR*, This can be represented more formally as follows.
This
DIP hypothesis: Assume that E Rº is sampled i.i.d. from a uniform distribution 4(00, 1). For any x E Q, the DIP hypothesis states that for any x E TR there exists E [0, 1] such that (u) x S 8, almost surely.
This hypothesis underscores the capability of UNNs to function as powerful priors, capturing intricate data structures inherent in natural images and other complex datasets, thereby bridging the gap between classical analytic methods and modern machine learning techniques.
Given SCI measurements = Hx + z. described in ① with H defined in (2), DIP represented by ge Rº + R" can be used recover x from measurements y as follows: Step 1) Randomly sample u (independent of y and H), as required by the DIP. Step 2) Solve the DIP-SCI optimization:
& arg min ly Hcll2: subject to go (u). 0 e 1]*
Before describing proposed approach to solving DIP-SCI optimization in Section 4 we the- oretically characterize the performance (3). under noise-free and noisy measurements and use our theoretical results to i) bound the number of frames that can be recovered from single 2D measurement, and ii) optimize the parameters the mask that used for modulating the data.
3.2.1 Noise-free measurements
The following theorem characterizes the performance of (3) in case where the measurements noise-free and connects its performance ([|x ()]2) to the ambient dimension n, number frames B, number of parameters of the DIP k; the distortion 5 and the Lipschitz coefficient L..
Theorem 3.1. Let Q, where Q denotes compact subset of R", such that ||x||oo S 2. for all Q. Assume that go (u) [0, 1 N R is L-Lipschitz function of f. Let Hx, where H = [D...., DRl where D: diag( Di.,,..., D, i 1,..., B, are independently generated with D Din i...d. Bern(p). Given randomly generated u, let & denote the solution of (3)
Then, if mine (=) nB cll2 < 8, we have
x112 1
Bp 1-p
2p
kB² log log
k log V
L
B
+ 1),
(4)
vp(1 p1
nB
/p(l-p
with probability larger than 1 - 2 0.5k log log n+1

---

The bound in (4) consists of multiple terms. The first term, i... /
I-p 8, accounts for the effect of the DIP representation error. For instance if x is directly selected from the output space of DIP, then 0. The goal of the following two corollaries to shed light on the interplay the three terms in and highlight their implications the performance of DIP-SCI optimization. First, Corollary 3.2 characterizes an upper bound on the number of frames B that are to be recovered from a single
2D measurement.
Corollary 3.2. Consider the same setup as in Theorem 3.4 If
B S Klog [llogn](un log
(5)
Bp then 1 8112 S v
On where C. = O(1/(logn) does not depend on p.
1-p p(1-))
Next, Corollary 3.3 states that in the case where the measurements are not corrupted by noise, the
value of p, the probability of mask entry being non-zero, that minimizes the upper bound in (4) is
always less than 0.5. This is consistent with the results established i) empirically in the literature [16 and ii) theoretically in (36) using compression-based framework.
Corollary 3.3. Consider the same setup as in Theorem 3.1 p" (0,..5).
The upper bound in 4 is minimized at
3.2.2 Noisy measurements
In many practical SCI applications, the measurements are corrupted by additive noise. This raises the following natural question: How does inclusion of noise in the model affects the optimized mask parameters? To address this question, the reconstruction error x & and Theorem 3.5 bounding the error in estimating the mean the
develop two theoretical results: Theorem 3.4 characterizing
input frames X = B E X4. As we explain later, the combination of these two results provide theoretical understanding on the performance of SCI recovery methods in the presence of noise and the corresponding optimized masks.
Theorem 3.4. Consider the same setup as in Theorem 3.1 For Q. let where z E R" denotes the additive noise and z N I for some 0g 2 Let & denote the solution of DIP-SCI optimization (3..
If B satisfies the bound in (). then
x - 812 55
Bp
30
an) +
(1 Br) no:
(6)
log
p(1
p}
p(1-p) (log n)
with a probubility larger than (2-0.5k log log n+3 + e 0.33). Here, On O Vogn 3n
of n))
and 7/ = (( do not depend on and
Theorem 3.5. Consider the same setup as in Theorem 3.4
Assuming that B satisfies the bound in + e U.33),
5 then with probability larger than - (2-0.5k log log 1+3
va E &1)ll2 S 5
1+
1 pB p
2po,
k log log (opp
+ PVB"n + L k
log nB
where Un = O(log n) and does not depend on p.
To shed light on the implications of these two theorems, the following corollary characterizes the value of optimizing each bound.
Corollary 3.6. Consider the same setting as Theorem 3.4 The upper bound Theorem 3.4 is always optimized at p² < 0.5. On the other hand, the upper bound in Theorem 3.5 is decreasing function of and is minimized at p²
J E B i... the reconstruction signal derived by repeating the average frame B X₁. Then, using the triangle inequality, we have
Let RB ix x
||x x||2 ||x xBl12 || x1|2-
5

---

Rumner
Aetiel
Crash
Kobe
Thaffic
315
27.5
40-
15
31.0
23
32
27.
23
10
30
25.0
24-
23
25
22.5
22
25
17.5
26
2.6
0.4
0.4
02
0.4
Figure 1: PSNN, shown as y-axis, of ||x 2 | S and |X x/1 2l: masks are generated as Bern(p), p shown as x-axis,. Blue, orange and green lines represent noise levels of 10 and 25, respectively. Solid black line shows ||x - Xnl12. Solid colored lines and dashed colored lines represent | 8||2 and |S Kal|2, respectively.
Figure 1 shows ||x 2, ||x X and || x||2. 2. for different video test samples. Here our key observations: 1) || x||2 is an increasing function of p. which is consistent with Corollary 3.6 2) The optimal value of p" that minimizes | x||2. is an increasing function of og for all Test videos. 2) In cases where the difference between X and x is relatively large, c.g. Traffic, the optimized p" stays smaller than 0.5, even for large values of og as predicted by Theorem 3.4 3) On the other hand, in cases where X provides high-fidelity representation of x and || x||2 relatively small (e.g., Drop), for large values of noise power, the optimal value of p" can move beyond 0.5, as predicted by Theorem 3.5 In other words, in such cases, the algorithm moves toward estimating the mean of the frames, which a good representation of the actual data frame.
SCI-BDVP: Bagged-DVP for video SCI
Recall the DIP-SCI optimization described in 33. i.e., & = arg min lly Hcll2, where go (u). 0 E [0, 11 and u generated independently and randomly according to pre-specified distribution. To solve this optimization, one straightforward approach to solve ming f()), with f(0) = lly Hgo (u)]l3. by directly applying gradient descent to the differentiable function f() However, given the highly non-linearity and non-convexity of f0), this approach is prone to readily getting trapped into local a minima and achieving considerably sub-optimal performance. Generally, better approach to is to write the DIP-SCI optimization as = arg millee(())) ||y Hcll2, where
C(u) & {c (u): 0 [0, 1]*). This alternative presentation of the problems leads to minimizing a convex cost function over a non-convex set. A classic approach to solve this optimization is projected gradient descent (PGD), which while in general is not guaranteed to converge to the global minima is more apt to recover a solution in the vicinity of the desired signal.
Remark 4.1. Theoretical feasibility of SCI recovery was first established in (38) using a compression- based framework for modeling source structure. There, the authors considered & = arg millec lly Hcllz, where C denotes a discrete set of the codewords of compression code. They theoretically proved that that case, despite the non-convexity of the problem. PGD able to converge to the vicinity of the desired signal.
The PGD applied to = arg min lly Hell proceeds as follows: Start form an initialization point Xg- For = 11,,.... T. perform the following two steps i) Gradient descent: x
X; + H (y Hx,), and ii) Projection: x = arg mineec(u) ||c S or
0c arg ming (Ige(u) - - X = 96... (u)
To solve the non-convex optimization required at the projection step, one can again employ gradient descent. However, in addition to the non-convexity of the cost function, another common known issuc with projection into the domain of DIP is overfitting [25] 6 39 40). Moreover, in PGD, idcally one needs to set the resolution of the projection step adaptively, such that during the initial steps the DIP has coarser resolution and as it proceeds it becomes finer and finer. This poses the following question: Which DIP structure should one use to optimize the final performance?
To address this question, the authors in (8 have proposed, bagged-DIPP which consists of employing multiple DIP with different structures in parallel, for the DIP projection step and averaging the outputs. They show that this approach provides robust projection module which consistently outperforms the performance achievable by each individual DIP network, and also provides, at least partially, the flexibility and adaptability required by PGD.

---

SCI-BDVP
Estimate
m 80
Ay d0
dy Minr
5D data cube x
((0.))
Cutput Block
Block
Block
H
3D patches in astimate
data cubo
Estimate k
HWW
(u,))
U ((O.))
CIP Bloox
Qutput Block
Hag
patiches estimate
Estimate K
Ug ((0.1)
DIP Block
DIP Flinc
DIP Block
Bg (u
Ya. Hgg
Nex
SD binary mask
Average N.
Fusion x, = onf +1 ain
MSE 0 argmin, 1sf dl"w olly Mi
Figure 2: The structure of SCI-BDVP. There are estimates generated, each using different patch size. The blue dot denotes averaging the K estimates, the orange dot denotes averaging x," and x with weight ax the red dot denotes the loss function used for training the DIP parameters, requiring xf, y and H. The red lines denote using 3D gradient descent result x which is used for training the parameters the k-thh DIP (red dot), and averaging with projection output x (orange dot). The green lines denote using 2D measurement y and 3D binary mask H for training parameters of DIPs in different estimate k.
Bagged-DIr essentially employs bagging idea to mitigate overfitting. As the DIP projection iterations proceeds (within each step of PGD), overfitting tends occur after certain threshold. However, due to the variance reduction facilitated by bagging, the bagged estimate can demonstrate less overfitting. In other words, the bagged estimate is less sensitive to the stopping time of the DIP training. In essence, each DIP is not required to produce the best estimate at every iteration of PGD.
Inspired by the bagged-DIP solution, here we propose the bagged-DVP for SCI (SCI-BDVP), as shown in Figure 3 SCI-BDVP, in addition to the standard gradient descent (GD) step, defined as xf_ X, HH' (y Hx,), consists of two main additional components: i) The bagged-DVE module that simultaneously projects the output of the GD step onto the domain of multiple DVP networks operating at varying patch sizes (refer to Figure 2
and and then averages their outputs, and ii) skip connection that computes a weighted average the output of the GD step and the bagged-DVi step. Next, we briefly explain the detailed construction of each component.
shows the structure of a bagged-DVE consisting of K individual DVPs, each operating at a different scale and trained separately. More specifically, for each k. 1, K, the 37) video is partitioned into non-overlapping video cubes of dimensions (hg, wx). For each video cube of dimension (h,, Wk: B). we train separate DVP. In other words, at scale k. we need to train N₂- = H//hg WW/wk separate DVPs. (The total aggregate number of DVPs that are trained is going to be Σk= NNa.) At each scale, the separately projected video cubes are concatenated to form x/_ I.k a. video frame of the same dimensions as the desired video. At scale k, let gă (-) denote DVP that generates an output video frame of dimensions hg x wk x B. To cover the whole video frame at scale k we need to train N₂ separate DVPs 9 (.), each having an independently drawn input, Ug,.. i denotes the index of partitioned video cube. To train each of these N₁ DVPs, we first extract the corresponding parts from x y and H and denote them as xG Y; and H, respectively Then, to train the corresponding DVP to form reconstruction xP 1, we minimize
SCI-BDVP. SeguueeSshemattieee
NNote that given the special structure of the sensing matrix H in SCL, given part of the input video frame of the same depth B, readily extract the corresponding mask portion and measurements.
7

---

PGD Bagged-DVP
اار Hx,) XX-1
Gradiet Descent
oip p. D.
Projection
DVP
((0,1)
argming (ss(u) x2
Figure 3: SCI-BDVP (GD): Iterative PGD-type algorithm. Each step consists of GD and BDVP
projection, with an additional skip-connection.
-9 k. (ux,)) 222 wlly; - H (u. where > 0 denotes the regularization parameter. Unlike classic DIP cost function, here we use the measurements y as an additional regularizer. After recovering x i I,..., NR. we concatenate them based on their locations to form x
|
We repeat the same process, for each lo I....,K to find x ,,, X
Finally, we use the
idea of bagging and define x K Σ x X/+1,
Skip connection. After obtaining xf+1 and x(+1, we define X as their weighted average: X
ax + (1 a)x{+1, where a E (0,1). (See Figure 3) In the experiments in Appendix C.3 we show how the addition of this skip connection consistently improves the achievable performance.
Experiments
We evaluate the performance of SCI-BDVP and compare it with existing SCI methods, for a, 0 and aN > Our experimental results are consistent with our theoretical results on mask optimization. To evaluate the performance use sasssiinna-t--oii---aaaie (PSNR) and structured similarity index metries (SSIM) 41. All the tests are performed on a single NVIDIA RTX 4090 GPU.
Datasets and baselines. We compare our method against the baselines on 6 gray-scaled benchmark videos including Kobe, Runner, Drop, Traffic, Aerial, Vehicle [8] where the spatial resolution is 256 x 256, and B = 8. We choose 5 representative baseline methods i) GAP-TV 9
the Plug-and-play (PnP) method that employs total-variation denoiser; ii) PnP-FFDnet 17 and
Dan-FattDVDnet [8] PnP methods that employ pre-trained deep denoisers, iii) PnP-DIP 27 : DIP-
based iterative method; iv) Factorized-DVP (26]: Untrained End-to-Ein (E2E) network. Baseline
setups follows that exactly stated in the respective papers. The details of proposed SCI-BDVP can be
found in Appendix
Masks for noiseless and noisy measurements. For the case of SCI without noise, we obtain the measurements from equation (I). where we randomly sample mask values from Bern(p) with 0.2, 0.3, 0.8. For the noisy setup, mero-mean Gaussian noise with variance (o²), o 10, 25 and 50, is added to the measurements. For the results reported in Tables and 2 the masks are randomly and independently generated as as Bern(0.5).
5.1 Reconstruction results for video SCI
Noiseless measurement. In Table we compare the performance of SCI-BDVP against baselines. To highlight the effectiveness of the bagged DVP idea, we also implemented two versions of our proposed method: i) SCI-BDVP (E2E), an end-to-end BDVP-based solution and 2) SCI-BDVP (GAP): an iterative algorithm that employs generalized alternative projection (GAP) update rule and BDVP projection (Refer to Appendix B.I for description of GAP and GD and our rationale for the choice of each method.). can be observed that both SCI-BDVP (E2E) and SCI-BDVP (GAP) outperform existing untrained methods. Specifically, SCI-BDVP (GAP) achieves of-te-of-the-at performance and on average improves about 1 dB PSNR compared to other methods.
Noisy measurement. Table 2 compares the performance of SCI-BDVP (E2E) and SCI-BDVP (GD) with baseline methods. As explained in Appendix B.T unlike noise-free measurements, in the case of noisy measurements, especially when noise variance grows, GAP update rule is no longer reasonable

---

Table I: Reconstruction results on Noise-free measurements. PSNR (dB) (left entry) and SSIM (right entry) of different algorithms. Best results are in bold, second-best results are underlined.
Dataset
Kobe
Traffic
Runner
Drop
Crash
Acrial
Average
GAP-TV
22.38, 0.666
19.60, 0.609
28.15, 0.884
32.49. 0.949
24.46, 0.842
25.65, 0.835
25,46, 0.798
PaP-FFD
30.39, 0.924
23,89, 0.830
32.66, 0.935
39.82, 0.986
24.18, 0.819
24.57, 0.836
25.46, 0.798
PnP FasDDDD
32.79. 0.948
27.89, 0.929
37.52, 0.967
42.35, 0.989
26.76, 0.921
27.92, 0.897
32.54, 0.942
PaP-DIP
22.52. 0.627
20.27. 0.617
29.54, 0,878
31.23, 0.908
24.33, 0.751
25.45, 0.790
25.56, 0.762
Factorized-DVD
25.54, 0,740
23.38, 0.760
30.76, 0.890
0.970
26.05, 0.850
26.84, 0. 0.860
28.21, 0.845
SCI-DVP
25.24, 0,741
18.89, 0.503
26.92, 0.852
35.00, 0.958
21.82, 0.653
21.31, 0.684
24.87, 0.732
SCI-BDVP (E:E)
27.76, 0.866
22.00, 0.741
32.86, 0.939
39.67, 0.985
23.59, 0.805
23.98, 0.809
28.31, 0.857
SCI-BDVP (GAP)
28.42. 0.886
22.84, 0.779
34.32, 0.954
40.76, 0.986
24.96, 0.851
16, 0.837
29.41, 0.882
Table 2: Reconstruction Results on Noisy Measurements. PSNR (dB) (left entry) and SSIM (right entry) of different algorithms. Best results are highlighted in bold, second-best results are underlined.
Explicit Regulatizor
Leurning based supervised methods
Learning based unsupervised methods
Dataset
GAP-TV
FFD
FasDVD (GAP)
FasDVD (PGD)
PuP DIP
SCI-BDVP (E2E)
SCI-BDVP (GD)
10
22. 16, D,580
25.68, 0,706
28.94, 0S811
22,96 0.595
22,47, 01562
25.76, 0.741
26.39, 0.805
Kobe
0.461
21.37,0,436
44 0.564
0.606
21,34,
22:34,
25.89, 0.775
60
20 0.297
16.04, 0.188
19.59, 0.341
20,82, 0.524
19,52, 0.238
16.99, 0.342
23.34, 0.640
10
20.56,
26.11. 0,855
0.20
19.95, 0.562
21.06, 0.649
22.66.
Traffic
25
19.23, 0.495
18:23, 0.524
22,77, 0.692
21.64, 0.740
19,24, 0.464
18.9,, 0,484
22.25 0.718
50
18.42, 0,585
13,00,
18.00, 01367
18.76 0.552
17,97, 0.344
15.34, 0.295
20.56, 0.611
10
27.40, 0.766
26,09, 0,739
32.21. (1845
27.92, 0.844
27.22, 0.663
2135, 0,764
31.15. 0.916
Runner
25
25.99, 0.610
22.18,
27.63, 0.630
28.33, 0.856
24,93,
21,93, 0,410
30.31, 0.895
0.398
0,270,
22.31, 0.361
21,74, 0.322
16.21,
10
30 30,75,00 0.800
29.50,
33.81,
31,54, 0.932
0.766.
31,45, 0.870
35,03, 0.962
Drop
25
28 0.614
23,36, 0.527
13, 0.646
52.50 52, 0.940
26,42, 0.842
23,46, 0,480
34,17, 0.954
50
24.09, 0,384
16.73, 0,298
23. 40, 0.350
30,48, 0.856
26, 66, 0.823
17.56, 0,233
29.86, 0.889
10
24 12, 0.728
21,83, 0.649
25.61, 0.799
24 70, 0,790
23,46, 0.647
22.54, 0,655
25,57, 0.835
Cmuh
25
23 40, 0.577
19.67, D:458
24,09, 0.609
24.54 54, 0.795
22,11, 0.492
19.06, 0,379
25.33, 0.821
15.33, 0.255
20,92, (1:342
23.35, 35, 0.706
0.294
0.178
10
25:21, 0.717
21,62, 0.641
26.51, 9.763
23.07, 0.626
24,74,
2284, 0.686
25.62. 0.817
Acrial
25
24 0,570
19,66, 0:447
24 0.586
23,99,
2A.19, 06613
20.39. 0,410
25,47, 0.796
50
22. 16, 0.575
15.18, 0.257
21,49, 01336
23.51, 0.713
21,07, 0.315
1588, 0.179
23,97, 0.638
10
25:21, 0.717
21,62, 0.641
26.51, 0.763
25.47, 0,760
24.89, 0.644
25:25, 0.727
27.73, 0.846
Average
25
24.31, 0,570
19,65, 0:447
24.58, (1586
25.61, 0.777
24.19, 0.613
21,15, 0,446
27.23. 0,827
22.16,
15.18, 0.257
21.49,
0.003
2.,17,0.389
16.27, 0.218
0.694
choice. Therefore, for noisy data, we replace the GAP update rule with GD. For completeness, for PnP-FastDVDnet [18), we report both GAP-based version (as implemented in [8] and GD-based version (newly implemented here). We observe SCI-BDVP (GD) considerably outperforms PnP-DIP Additionally, SCI-BDVP (GD) in most cases outperforms pre-trained method [8] across
27
noise levels, while showing robust performance on different datasets and noise levels.
Time/computationaal complexity. The proposed SCI-BDVP method relies on the bagging of multiple DIP projections. These DIP projections, which vary depending on the patch size, involve different levels of computational complexity. Table 3 shows the average time required to perform each DIP projection for each patch size. As observed, the time increases considerably as the patch size decreases. This is expected because the number of networks that need be trained grows significantly. (Refer to Figure for a pictorial representation.) Additional computational complexity analysis of our proposed method and its comparisons with other methods is included in Appendix B.3
5.2 Mask optimization
We consider masks that are generated independent of the data as i Bern(p). The question is what value of p optimizes the reconstruction performance? Figures 4 and BJshow the achieved recon- struction PSNR as a function of p, for the cases of noiseless and noisy measurements, respectively. For noiscless measurements, the results are shown both for SCI-BDVP (GAP) and PnP-FastDVDDet (GAP). It can be that for both methods, the optimized value of p is smaller than 0.5 (around 0.4) and
Since the code of Factorized DVP 26 is not available online, we could only compare our results with the results reported for noise-free measurements.

---

Table 3: Time complexity of our proposed SCI-BDVP was evaluated on various patch sizes (64, 128, 256) of video blocks, using a standard 1000 DVP iterations for training.
Patch size
# patches
Time (min..)
1.5
128
4
0.28
256
0.12
34 For the noisy measurements, see that p" is an increasing function of 22 consistent with our theoretical results discussed earlier in Section (Refer to Appendix C.I for further results.)
consistent with empirical observations reported in 35
3
1.0
1.00
Drop
0.05
Pall
1730
0.90
Runnaf Avenae
0.85
0.85
PHO
02 04
05
67
0:75
02
04
05 67
05 06
6.0
0.79
02
05
07
Figure 4: Reconstruction PSNR (|xx 2) and SSIM as function of using SCI-BDVP (GAP) (two leftmost figures) and FnneDaDDDDDee (GAP) (two rightmost figures). For each value of p, the masks are independently generated i.i.d.~ Bern(p).
Drop
Burner
Zetial
Crash
Kobe
Thelfic
30
25
24-
23-
20
52
21
5.2
5A
0.6
24
64
58
0.2
04
26
04
52
0.
Figure 5: Reconstruction PSNR ([[x-& [2 of SCI-BDVP (GD), y-axis, function of p, X-axis. For each value of p, the masks are independently generated i.i.d.> Bern(p).
6 Conclusion
We have studied application of UNNs SCI recovery. We propose an iterative solution with bagged DVP (multiple, separately trained DVPs with averaged outputs), achieving sate-offthhear perfor- mance among unsupervised solutions for noise-free measurements and robustly outperforming both supervised and UNN methods for noisy measurements. Additionally, we provide theoretical frame- work analyzing the performance of UNN- based methods, characterizing achievable performance and guiding hardware parameter optimization. Simulations validate our theoretical findings.
important application of SCT hyperspectral snapshot imaging (HST). Our results this paper provide theoretical foundation to understand HSI systems and optimize their hardware. Additionally, the developed theoretical framework can be used to explore aspects specific to HSI, such as masks being shifted versions of each other. We also expect our algorithm to effectively address overfitting in HSI tasks, enhancing reconstruction performance. We plan to explore these aspects further in our future research.
Several other aspects remain for future work. Theoretically, we only considered i.i.d. Bernoulli masks, while practical SCI systems typically are more constrained. Additionally, deriving information- theoretic lower bounds on SCI recovery an open problem. Experimentally, we focused on classic baseline videos; exploring richer set of samples and studying noise models beyond additive Gaussian noise are interesting directions for future research. We also plan to enhance the algorithm's efficiency by parallelizing the projections required by the bagged solution.
10

---

Acknowledgements
MZ, XC and SJ were supported by NSF grant CCF-2237538.
References
[1] M. E Gchm, R. John, D. J Brady, R. M Willett, and T. J Schulz. Single-shot compressive spectral imaging with a dual-disperser architecture. Opt. Exp., 15(21):14011-11422,,, 2007.
[2] Liull, X. Liao, X. Yuan, J. Yang, D. Kittle, L. Carin, G. Sapiro, and D. J. Brady. Coded aperture compressive temporal imaging. Opt. Exp., 2109):00266105455 2013.
[3] P. Llull, X. Yuan, L. Carin, and D. Brady. Image translation for single-shot focal tomography. Optica, 2(9): -02-82,,, 2015.
[4] M. Qiao, Y. Sun, X. Liu, X. Yuan, and P. Wilford. Snapshot optical coherence tomography. In Dig. Hol. and Vhree-Dim. Ima., pages W4B-3. Optica Publishing Group, 2019.
[5] X. Yuan, D. J. Brady, and A. K. Katsaggelos. Snapshot compressive imaging: Theory, algo- rithms, and applications. IEEE Sig. Proc. Mag., 38(2))65-888 2021.
[6] R. Heckel and Hand. Deep decoder: Concise image representations from untrained non- convolutional networks. In International Conference Learning Representations (ICLR), 2019.
[7] D. Kittle, K. Choi, A. Wagadarikar, and D. Brady. Multiframe image estimation for coded aperture snapshot spectral imagers. Applied optics, 49(36):6824 -8333, 2010.
(8] Yang, X.
Yuan, X. Liao, P. Llull, J. Brady, G. Sapiro, and I.. Carin. Video compressive sensing using gaussian mixture models. IEEE Trans. on Image Proc,, 23(1 23(11))48334888 2014.
[9] X. Yuan. Generalized alternating projection based total variation minimization for compressive sensing. In IEEE Int. Conf. on Image Proc. (ICIP), pages 2539-2543. IEEE, 2016.
[10] Y. Liu, X. Yuan, J. Suo, D. J. Brady, and Q. Dai. Rank minimization for snapshot compressive imaging. IEEE Trans. Pattern Anal. Mach. Intell., 2019.
[11] Z. Cheng, R. Lu, Z. Wang, H. Zhang, B. Chen, Z. Meng, and X. Yuan. BIRNAT: Bidirectional
recurrent neural networks with adversarial training for video snapshot compressive imaging. In Proceedings of the European Conference on Computer Vision (ECCV), August 2020.
[12) Z. Cheng, B. Chen, G. Liu, H. Zhang, R. Lu, Z. Wang, and X. Yuan. Memory-efficient network for large-scale video compressive sensing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 16246-16255, 2021.
[13] Z. Wang, H. Zhang, Z. Cheng, B. Chen, and X. Yuan. Metasci: Scalable and adaptive re- construction for video compressive sensing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2083-2092 June 2021.
[14] L. Wang, M. Cao, and X. Yuan. Efficientsci: Densely connected network with space-time factorization for large-scale video snapshot compressive imaging. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), pages 18477-18486, 2023.
[15] L. Wang, M. Cao, Zhong, and X. Yuan. Spatial-temporal transformer for video snapshot compressive imaging, IEEE Trans. Pattern Anal. Mach. Intell., 2022.
[16] P. Wang, I.. Wang, and X. Yuan. Deep optics for video snapshot compressive imaging. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 10646-10656, October 2023.
[17] X. Yuan, Y. Liu, J. Suo, and Q Dai. Plug-and-play algorithms for large-scale snapshot compressive imaging. Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.

---

[18] X. Yuan, Y. Liu, J. Suo, F. Durand, and Q. Dai. Plug-and-play algorithms for video snapshot compressive imaging. IEEE Trans. Pattern Anal. Mach. Intell., 44(10):7093-7111, 2021.
[19] J. Ma, X. Liu, Z. Shou, and X. Yuan. Deep tensor admm-net for snapshot compressive imaging. In Proceedings of the IEEE/CVF Internatinal Conference on Computer Vision (ICCV), 2019,
[20] Peng, S. Jalali, and X. Yuan. Solving inverse problems via auto-encoders. IEEE Journal on Selected Areas in Information Theory, 1(1):112-333, 2020.
[21] 7. Meng, S. Jalali, and X. Yuan. Gap-net for snapshot compressive imaging. arXiv preprint arXiv:2012.08364, 2020.
[22] Z. Wu, J. Zhang. and C. Mou. Dense deep unfolding network with 3d-enn prior for snapshot compressive imaging. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 4892-4901, 2021.
[23] C. Yang, S. Zhang, and X. Yuan. Ensemble learning priors unfolding for scalable snapshot compressive sensing. In Proceedings of the European Conference on Computer Vision (ECCV). 2022.
[24] Z. Meng, X. Yuan, and S. Jalali. Deep unfolding for snapshot compressive imaging. Interna- tional Journal of Computer Vision (IJCV), pages 1-26, 2023.
[25] D. Ulyanov, A. Vedaldi, and V. Lempitsky. Deep image prior. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), pages 9446-9454, 2018
[26] Y. C. Miao, X. L. Zhao, L.. Wang. X. Fu, and Y. Wang. Snapshot compressive imaging using domain-factorized deep video prior. IEEE Trans. on Computational Imaging, 2024.
[27] Z. Meng, Z. Yu, K. Xu, and X. Yuan. Self-supervised neural networks for spectral snapshot compressive imaging. In Proceedings of the IEEE/CVF international conference on computer vision (ICCV), pages 2622- 2231, 2021.
[28] X. Chen, Z. Hou, C. Metzler, A. Maleki, and S. Jalali, Bagged deep image prior for recovering images in the presence of speckle noise. arXiv preprint arXiv:2402. 15635, 2024.
[29] G. Jagatap and C. Hegde. Algorithmic guarantees for inverse imaging with untrained network priors. Advances in neural information processing systems (NeurIPS). 32, 2019.
[30] H. Zhang, L. Mai, N. Xu, Z. Wang, J. Collomosse, and H. Jin. An internal learning approach video inpainting. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 2720-2729, 2019.
[31] G. Mataev, P. Milanfar, and M. Elad. Deepred: Deep image prior powered by red. In Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, pages 0-0, 2019.
[32] H. Zhou, Y. Song, Z. Yao, D. Hei, Y. Li, B. Duan, Y. Liu, and L. Sheng. Unsupervised reconstruction with registered time-unsheared image constraint for compressed ultrafast photography. Opt. Exp., 32: 02116633113555 04 2024.
[33] M. Qiao, X. Liu, and X. Yuan. Snapshot temporal compressive microscopy using an iterative algorithm with untrained neural networks. Opt. Lett., 46(8):1888-189) 1891, Apr 2021.
[34] M. Iliadis, L. Spinoulas, and A. K. Katsaggelos. Deepbinarymask: Learning a binary mask for video compressive sensing. Dig. Sig. Proc., 96: 1025099,, 2020.
[35] X. Zhang,
Y. Zhang, R. Xiong, Q. Sun, and J. Zhang. Herosnet: Hyperspectral explicable reconstruction and optimal sampling deep network for snapshot compressive imaging. In Proceedings of the JEEE/CVF conference on computer vision and pattern recognition (CVPR), pages 17532-17541, June 2022.
[36] M. Zhao and S. Jalali. Theoretical analysis of binary masks in snapshot compressive imaging systems. In 2023 59th Annual Allerton Conf. on Comm., Cont., and Comp. (Allerton), pages 1-8. IEEE, 2023.
12

---

[37] A. Wagadarikar, N. Pitsianis, X. Sun, and D. Brady. Video rate spectral imaging using coded aperture snapshot spectral imager. Opt. Exp., 17(8):6368-6333 6388, Apr. 2009.
[38] Jalali and X. Yuan. Snapshot compressed sensing: Performance bounds and algorithms. IEEE Trans. Inform. Theory, 65(12):8005-8024, 2019.
[39] R. Heckel and M. Soltanolkotabi. Denoising and regularization via exploiting the structural bias of convolutional generators. In International Conference on Learning Representations (ICLR), 2020.
[40] H. Wang, T. Li, Z. Zhuang, T. Chen, H. Liang, and J. Sun. Early stopping for deep image prior. Trans, on Machine Learming Research, 2023.
[41] Z. Wang. A. Bovik, H. Sheikh, and E. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Proc, 13(4):600-61) 2004.
[42] Jalali, A. Maleki, and R. Baraniuk. Minimum complexity pursuit for universal compressed sensing. IEEE Trans. Inform. Theory, 60(4):2253-222 -268,, Apr. 2014.
[43] X. Liao, H. Li, and L. Carin. Generalized alternating projection for weighted-2,1 minimization with applications to model-based compressive sensing. SIAM Journal on Imaging Sciences, 7(2):797-22,, 2014.
13

---

A Proofs
A.1 Preliminary results and definitions
42 1,, Z2..... Zn are i..d. ((0,1) 1) random variables, then Lemma A.1 (Concentration of x² for any t 0,
n
P 2 m( + t)) Se - (t-
Definition A.1. f: + R is called an L-Lipschitz function (or L-Lipschitz continuous) if there exists a constant L such that for all x1, X2 R
If(x)) f(x2)l s E||x1 X2/12-
The constant L is called the Lipschitz constant of
A.2 Proof of Theorem 3.1
Let x = arg min (o BERA ||x c122. By assumption VnB ||x x/22 S 8. On the other hand,
since arg min c=gn DER* - Hc E. ly H&| 22 - lyy H where = Hx. Therefore,
H(x - x)112 - H(x r)]l2
(8)
Let & = 910), (u). i.e., the reconstruction corresponding to the q-bit quantized version of parameters 0. By the triangle inequality,
H x x)|12 = [ - R2 + &gg - -x)ll2 2 |H(( - 2,)112 | 8)112-
(9)
Combining
with it follows that
H(x 8 & H - x)112 + | x)112
(10)
Given our assumption about the L-Lipschitz continuity of ge as function of 0, it follows that
|& *||2 à), (u) g2(u))l2 < L1I Oll < -2
(11)
For
E BBB using Cauchy-Schward inequality, |Hu |
ΣE Dyyu <
E ンj=1 (E D2 E uzz) < B B(maxi, D2t) 1112 < Bllu 1.2. which follows since Di.j E (0,1). 1). Therefore.
H( - x)112 < B| - X||2 S BL2
(12)
For a fixed random initialization z E IRP, define the set of reconstructions derived from q-bit quantized parameters as (n) i.e.
(u) = {gm9 (u) 0 E 1].
Note that (u)l < 24k Given > 0, 62 0. and
x E R define events E and Eg
1 E, = 1 H(x x)11 <
p² Σ
x x 1| + P-P- p² xx
(13)
i=1
& - 1 11)
22
Σ
x;
c:)
p-p² ||x
c12
Bp²e2
Vc C,()). (14)
7
respectively. Then, conditioned on & n & and noting that i)
x||2 < 5 and iii) for any a, b 0, Va bb v - (x:
x;
<
x12 and
ii VnB x
Vb, from
10
12)
we have
k 1d |x - &2112 S V2 (B 1) plVE + V22++ VVL2
(15)
nB
n
14

---

Or
1 VnB
xx 8yl12 -
1 1 (B - 11 1-p
+
Ve(1- - (VEi v22) + VBL2
P
k
p(L p)n
(16)
On the other hand, by the triangle inequality, ||x --112 S |x &gl12 + ||g 8112. Therefore, combining (IT and (16). follows that
x
x112
S
1)p (B
8 +
P
(VT v22)
v nB
1-p
Vp(1 (p(1-)
k
k
VB2
p(1- - p)n + +L2-9
(17)
nB'
To finish the proof, we need to set the parameters (q. E1, E2) and bound P(g U &g)). For a fixed
RaB Hu 2 = 2== U, where U, = (E Dijuss) Note that
E|U,] = 2 Σ Dzy DijDeuuuur p² Σ 21 2 + (p- p²)
ppp Σ u
(18)
Moreover, by the Cauchy-Schwartz inequality, U, S (E D Et= u S B ((
BΣ
S
Therefore, since by assumption for all x 2 |x|1oo S f. using the Hoeffding's 's inequality, we have
2nd
P(Ef) S exp(
(19)
B
Similarly, combining the Hoeffding's inequality and the union bound, we have
2n
P(E)) < 24 exp(
(20)
B2
Finally, given free parameters 17 (0, 1), setting the parameters
ngh In
19((1 1)
BV
62 BV
and q = [log log m],
2n
2n
we have
P((E ( <
2-ig++
7gk
< 2 log log 1+
For the selected parameters, setting 1] 0.5 and using it follows that
In2
and (1) (1 + n)+) 2, from
2
Bp
2p Vp(L p)
kB² log logn
1
L
B 1).
x
2112
2
S
1+
8+
+
VnB
1- p
logn
nB
Vp(1 p)
(21)
A.3 Proof of Corollary 3.3
Let fp) denote the upper bound in Theorem 3.1 That is,
Bp
1
f(p)
1+ +
8
21 + U2
--p
vp(1 - p))
where e and eg are defined as
U1 2p hB log log
L
kB
L L
log
n
log n nB'
and do not depend on p. On the other hand, f(0) f(1) = co. Let p* denote the value of p E (0, 1) that minimizes [(), note that
Bp -1.5
B5
f')) -
1-p p (1 ,(1
(p -22)
(22)
Note that on one hand 1imp-0 f(() = -00 and on the other hand f(()) > 0. which implies that p*
where fp") belongs to (0, 1).
15

---

A.4 Proof of Theorem 3.4
Let arg min (u)::: BERK |x cll2 and
== (u)
arg min 11yv c=ge(u); BER
Hcll2. x, = 910 (u).
That is, x denotes the reconstruction corresponding to the g-bit quantized version of ô. Following
the same argument as the one used in the proof of Theorem 3.I] since y = Hx + z, it follows that
Jo ||x - X||2 S 6 and [H( - 8) + +z/l2 V
|[H(x - x) + 2/|2. On the other hand. | - 8) + 211
H +
H(x x)) + | and H x) + 2 2 H[x x) + (z, H(x x)) + |z|
Therefore,
[H( )) < - + (2, H(x x))) + H(x x>->
(23)
Moreover, using the triangle inequality,
(a)
(z, H(x - x)) ( 8,,311 + (z, H(,, - x\)) S (z. H(x
( 8,,,8 + ||//22 H 8)]12.
where (a) follows from Cauchy-Schwartz inequality. Therefore.
- 8)1 - [H( - x)]2 + (z, H(x - 8,)) + 2) H(x - x)ح> + ||/12/ 82)122 [H)l=
(24)
Note that, by the triangle inequality. |H(x 1)|| 2 H(x 2 - H(2 - x) 12. which
implies that |H(x 11 2 [x
2,)11 2H - x2) || x)||2 + H[ x) 113 2
[H( x,1 2| H( x xg) || H($; 1l Therefore, combining this inequality with
(4
follows that
H(( - 22 H( ||² + 2 H(x - 8,)) + 21 H(x - x))1
+ (l|z||2 2 H(( x,, (12) H( 121
(25)
Similar to the proof of Theorem 3.1 for fixed random initialization u E Rº, define (u)
tge (u) 0 E [0, 1]. Also, given 61, <2, > 0, and x E RRBB define events El and & as (13)
and respectively. Morcover, define event Eg as
p²
B
+ P P-P p2
Eg = H(x e) <
[x,
xxx- - 1 + Bp Ve E (u(,),, (26)
Compared to the proof of Theorem E Cy (u), define ((c)
3.1 24 involves three terms that involve Gaussian noise z. For
(<c) (z. H(( c)).
Conditioned on the mask D, (Cc) is zero-mean Gaussian random variable with
B
E[(o(c))* = o. Σ [1 Dmy
=ož1 Σ Dx c.
((c) - E - c1)1l2. Then, for any given 6z > 0,
P ((((()) 2 V2nezt(c)) Σ P (((c) 2 V2net((c) D d) P(D d)
d
(0)
2ne (c)
<
2
P(D d) exp( 20²(c)
d
< Σ
P(D = d) expl -x?)
d
<
22cxp( -ne?),
(27)
16

---

where (a) follows because for any Gaussian random variable G ~ (0, applying the Chernoff
bound, we have P[[G] - 20- 22-02 /202 Given c21, <2 0, define events Ex1, Ex2 and Ex as
E11 {1 (22 H(x x))12 SxI S2. V x)]le}
(28)
|1,, H(x e)>{ 6:2 V 2n H(( c)ll:: ve E Cg(w)).
Ex2
(29)
and
E, [|zll z no + c
(30)
respectively, From
27
noting that (u)) - 24 follows that
P (EE)) < 2exp(-nBee) BB
(31)
and
P ((E) S 20h+1 exp(-nBreg)
(32)
Define event & as & - & 31
E,2 and
h(p) - p + (B - 11))²
Conditioned on E, since by assumption
| x||2 S 6, we have
vnB
B
z
Di(x:
x:11|
S S S (B 1)p²)nB8* + =BBPei),
1=1
or
B x))3 s E2102 2 + azpezt /21,,
(z.
Σ
Di(x:
(33)
nB
where the last line follows because for any a, b 0, Va tb Va vb. Define
1
A
x -,,12.
VnB
Then, similar to
33 conditioned on E, since &2 E (u), it follows that
H(x - 2,)3 - (2
nB /2mBo2 - (B - 1)p*)|xx 2,113 nBBPes)
nB
< 2h Ag 22 2
(34)
Conditioned on E, combining
25
and it follows that
(p 1ロー p0e <h(p)o² + peer 2/22.20.(VMM)))) PVVEa
PVEa)
22222110 h(p)) +v4))
+ (0 1 E33 2v
+
(35)
n'
where we have used HH(,, 8)]12 < BLk derived in
Note that since A, p and h(p) < B, the last term in 335 that corresponds to the quantization error can be bounded as
12p.
BL2 th
(o.VItcs+22 BV +
S Cns
Vn
where
G, e (oc_/+cc+2//B//V L2s))
17

---

and does not depend on Ag. Rearranging the terms in 11 and letting
p (2) + 2V220 E22
Eo
it follows that
(p 1p-p3) - 22622 //6=2/8h(1))4 & hhpp)8 + +cI1V8h(p)00 t +
(36)
Therefore.
0x6=2/2h(p)) 2 v
h2p
(p (p-p²)
2p(1 - pp) + h(p)8 +Ga t cn. (37)
p(1 -
This implies that
A S 22(p2
(02(221(p) h(p)0² + 1.1 8h(p80 + co t +c.) (38)
+ V p(1 2pp(1 p)
p(1 pp)
Therefore, since i) |x - 8112 I 1| 2zl12 + || - 11 and ii) from [1]. ||&, - x112 - L2 o/k have
02022 2h(p) p(1 - -p)
(02c22(1))
x $1/25
h c.1 s + co 1ott0 +ca)
VnB
p(1 - p) 2p(11 - p)
+ 11-1 "
(39)
nB
Using vI+a S + 2a, for a > O, and noting that
h(p) p(1
1+ Bp/(1 - p), we have
03C22 2
Bp (I (1+
02
2
2(c. + Gn)
x 8112 S
+ 46,10x
+
VnB
p(1 - p)
p(1 - p2
h(p)02
82h(p)
+ 1
(40)
nB
Next we set the parameters by analyzing the probability event E, Applying the union bound,
P(g9) PP({) n& n E1l n &22 E23)))
2n
eepp 2ne
2nez
S <exp(
+
2
+xpp(-
B² )
+ -
B²
B²
+xexp( ( 20k+1
exp(
exp(-nBe22)),
(41)
where we have used Lemma A.l to bound P(E))). Similar to the proof of Theorem 3.1 parameters 7h 1/2 E (0, 1), let
given free
m qk In2
qk(1 + m7) m))n2 In2
6 = Bv
.2
c3 Bv
and g = [log log n]
2n
2n
Also, set
12 In
(1 + 12 )k 1n
61
62
nB and 623
nB
Following a similar argument as the one used in the proof of Theorem 3.1 it follows that
e P(E*) < 2 m + n2gk++
< 2 2 log log 2- log log n+1 -0.3n
Setting 7)1 = f/2 = 0.5, is straight forward to see that
B2
2
(a)
1.3
6, S 0²
pa=
B
p²
(log n0.25 + poz(
B2 log n)
(42)
where (a) follows from 5 Also, for the selected parameters and noting that B 2 1,
qh gh
L
k s (o.V2+4BB)) logn'
Cn S (0.V2 + 2Bp(1
(
0.25
(43)
n
logn n
n
18

---

Therefore, Cn = =0
=((ogg) loogn Moreover,
k log log nB
2h log log n nB
6,1 S
Therefore, from (39.. and using i) /E, S Ei Va, for 2 0, ii) h(p)
- B for all p E [0,
and iii) B 2 (which implies from our assumption that k(log n (log log n) < n), follows that
1 vnB
xx 1/2 _ó1
1 (B- 1)p
8 p(1-p) p) V logn + log n
30
d0; V'V p(1-p) p)
)
1-p p
+ p(1 p)
+
+Vpos B(log n) ) + von)
+ ol
(logn)
log n
or, rearranging the term in ① we have
Bp
30,
x
8112 5ó
+
VnB
1
p(1 p)
log n
)3
80,
(1
+ logn n
p(l p)
B81/1 Mogg
1 log
+ 1
(p
+ vvn)) + o
(44)
p(1 p)
(logn)=
which yields the desired result.
A.5 Proof of Theorem 3.5
Following the steps of the proof of Theorem 3.4 we derive 25 i.e., H 8,) x) 2 H(x +))+ 2 (z, H(x Hxx-xx) x))
H
[|z||| 2 H & ||2)
H((, x 2/2. We defined
events E,, &2, Ea, E21, Exe and Egg as before. Note that conditioned on Eg,
H(x 12
1
Σ
B
(x:
2
P
x - 2 Bp² p2
In the proof of Theorems positive term D (x: 1 and used 1 ||((( - &,) 112
and to apply this inequality and bound the error, we ignored the
p-p" n
xx x112 Bp²e.. Here,
instead we ignore p-p" |xx $|| and conditioned on Eg, we use
2
H
(x
x
Bp²c2-
(45)
Define 8 = & n E, Ea similar the proof Theorem 3.4 Following the steps used in deriving (36) in the proof of Theorem 3.4 and applying i) the lower bound in (45) and ii) A, S follows that, conditioned E,
x;
& n < h(p)8² + (E210 + pe,2)o, Saaoo.//hfg Sh(p) + eo + Cn.
(46)
nB
Let
1
1
X
Xq
is and
Σ
x,
B
B
B
iml
im1
and
A
xx
x
11
A
x
*1
n
n
19

---

Using these definitions,
146
can be written as
h(p)6² + (c21d (12))s + c Cn)
(47)
B²B
Noting that (pp((p) B-1
1 1
PB
pB
1 s (1 +
102
+ B²B
(E11ō pe:2)o co + cn).
(48)
pB
Using Schwarz inequality, it follows that
EE=n (x: x S Bl| x11l and combining with
X
(49)
or
1 vn
k nB
- R2 I2 I2-0V
(50)
Therefore, using the triangle inequality and (50) that
||
x1|2 ||8 - E || 8112, follows from
48
k
1
& 12
L2-9
nB + V
(1+
)02 +
(E.1d + pe.2)o + + cn
pB
F²B
k + 6
< 122 "N
+
(221 (8((p) + + +
nB
pB
PVB
(51)
where the last line follows from g p. and Va bs Va Vb, for a, 2 0. We set the parameters as in the proof of Theorem 3.5 as
0.5gk In2
1.5gk In2
e1 =B
62
E3
BV
and g [log log n.].
2n
2n
Also, set
0.5qk 2
1.5gk In
6,1
622 ====V
and 653 1.
nB
nB
Then, using the bounds in
42) and
(43
it follows from
(1
2po,
klog log h(p)
1
L
k
x x1|2 < 6
pB
+
pV
B
n
PVB
+
log
nB"
(52)
where Un O( (logn)$
and does not depend on p.
B Setup of SCI-BDVP
B.1 SCI-BDVP: Descent step
As explained in the paper, to solve the optimization described in (3), we employ the PGD algorithm, with additional skip connection. The details of the projections step using bagged-DVP and also the skip connection are described in Section 4 Here, we review the descent step, as we employ two different operators depending whether the measurements noisy noiseless.
Descent step:
For noise-free measurements, we use GAP update rule 43
x
G
x, + H (HH yy- Hx,),
Hx)),,),
(53)
20

---

For nosiy measurement, we use gradient descent (GD):
x X2 μ (y Hxx),
(54)
In both cases μl denotes the step size.
Compared to the GD, if f = 1, GAP, at each iteration, projects the current estimate x onto the y Hx hyperplane. Note that due the special structure of the sensing matrix H. HH is diagonal matrix and therefore it is straightforward to compute its inverse, as required by GAP.
In experiments, found that in the of noise-free measurements, the GAP update rule consistently showed better convergence compared to GD. Therefore, we adopted GAP update rule for the case of noise-free measurements. However, for noisy measurements, even the true signal does not lie on y Hx hyperplane, and therefore, application of GAP is no longer theoretically founded. Hence, for all experiments done for noisy measurements, we the classic GD update rule.
In summary, Algorithm| below shows the steps of SCI-BDVP.
Algorithm SCI-BDVP
Require: measurement y, mask H
1: Initial Xy = H y.
2: for t 1e.==1....5 do
3: Descent step
4: Update x." with Eq. 53 or (54).
5: Projection step
6: Generate x as the output of bagged-DVP (refer to Fig.
7: Update x, with x, ox. + xxx
8: end for
9: Output: Reconstructed signal XT.
DVP Block-1
DVP Block-2
DVP Block-3
Video Output Block
Input Nome
20
Up- Sample
2D Conv
Up- Sample
2D Conv
Up- Sample
2D Conv
Conv
ReLU
ReLU
ReLU
Sigmaid
Channel:
Figure 6: Network structure of DVP we use in SCI-BDVP.
B.2 Implementation details
In the projection step, the same structure design for bagged-DVR. for both noiseless and noisy measurements. Inspired by deep decoder structure [6], we design the neural nets, using three DVP blocks and one video output block shown in Figure Each DVP block is composed of Upsample, ReLU and Conv blocks. The output block only contains the Com block. Here, we use Conv 3 x 3 and the number of channels fixed to 128. Lastly, the input of each DVP (described in DVP function go (u)) is generated independently using a Uniform distribution, UO0 1).
Since the input video consists of B 256 256 frames, we choose three DVP structures, one with 16 64 64 B patches, one with 128 128 B patches, and one with single 256 x 256 x B frame. For each size of the patches, size of h/8, since square patch, where represent the height of the padded patch. And for each patch of each estimate, we train the separate DVP module.
perform mirror padding augment the each patch with the
The hyperparameters are set as follows: the learning rate of the DVPs set to 0.01; weight w 0.1 for measurement loss term wlly; H.g k. (u, 1 Figure For noise free case, GAP, we set the step size μ 1.0, and 0.1 for the noisy simple GD case.
21

---

Table 4: Number of inner and outer iterations for training SCI-BDVP for different datasets and different estimates.
Iterations
Kobe
Traffic
Runner
Drop
Crash
Acrial
Inner iteration. 64
2000
700
2000
2000
700
700
No moise
Iuner lierationn 128
2000
700
2000
2000
700
700
Inner iteration 256
4000
1400
4000
4000
1400
1400
Outer iteration
75
35
75
75
35
35
Inner steration 64
900
900
900
900
900
900
Noisy
Inner iterationn 128
900
900
900
900
900
900
Inner iteration 256
1800
1800
1800
1800
1800
1800
Outer iteration
35
35
35
35
35
Table 5: Time complexity over different methods one 8-frame benchmark video block.
Methods
Time (min.)
Factorized-DVP
20
15
No noise
Simple DVP(E2E)
10
SCI-RDVP
220
PmP-DIt
No Factorized-DVP
26
Simple DVP(EZE)
10
SCI-RDVP
40
B.3 Time/computational complexity
Implementing SCI-BDVP involves outer loop iterations (described in Algorithm 1 and also inner loop iterations for training DIPs. Table presents average number of inner loop iterations used for different patch sizes (64, 128, 256) of various videos, and the number of outer loop iterations. Detailed time consumption for each patch level computation is recorded Table 3 A comparison across different UNN-based methods is provided in Table 5 All comparisons are performed on a single NVIDIA RTX 4090.
important to note that traminn a bagged DIP requires training multiple separate DIPs. This process be readily parallelized. which is expected to significantly speed up the algorithm. We plan to explore this direction to optimize the algorithm's 's efficiency in future work. Lastly, making direct comparison among all methods is challenging because, for supervised methods, the main time is spent in training, whereas, for unsupervised methods, the main time is spent training the UNNs. This is expected trade-off for requiring training data and achieving robust solution.
C Additional studies
C.1 Mask optimization
In this paper, we explored binary masks that are generated i.i.d. Bern(p). Figures and 5 show the effect of probability p on the performance of SCI-BDVP (GAP) (noiseless measurements) and SCI-BDVP (GD) (noisy measurements), respectively. In this section, we perform similar investigation of the effect of p on the performance of other SCI methods, namely, PnP-FastDVD [18] and PnP-DIP [27]. Figure 7 shows the results corresponding to PnP-FastDVD. It can be observed that the trends are consistent with the performance of our proposed SCI-BDVP (GAP): i) for noiseless measurements, the optimal performance is achieved p" < 0.5, ii) for noisy measurements, p" is increasing function of o..
On the other hand, Figure 8 shows the performance achieved by PnP-DIP proposed in [27]. It can be observed that the reconstruction performance not longer a smooth function p, unlike the performance of SCI-BDVP and PnP-FastDVDD We believe that the reason lies in the the limitations of UNNs (or DIP) explained before, such as overfitting and instability. In the next section, we further explore this issue and explain how bagging can address the problem.
22

---

Also, we include the detailed PSNR and SSIM results using SCI-BDVP on different measurement noise level Table 6 We can find when o 50, the effect of the mask optimization will improve the overall result around 1 dB in PSNR, 0.1 in SSIM. Here, all the algorithm settings are kept intact, and the only variation is in the mask non-zero probability p varies between 0.5 to 0.7. This further highlights the stability of the proposed SCI-BDVP solution.
Crash
Traffi
24
22
23
0-0
00-0
12
10
23
0m
36
16-
05
1.5
2-30
14-
0.2
0.1
0.5
68
6.2
04
0.6
0.8
0.6
06
Prossiilty
Prstanilty
Pnbstiily
Runner
Kobe
Aetial
28
22
20
22-
18
13
0w
14
Prozeoilty
Probwallty
Probati
Figure 7: PSNR of ||x - &|] under different mask generated from Bern(p) of different measurement noise level using baseline method ChnP-astDVD[13 with GAP gradient descent algorithm).
Drop
Crash
Traffic
54
32-
23-
24
22-
20
22-
50
50
041
Prossslity
Pritwallty
Prbtatilty
Runner
Kobe
Aetial
32
25
74-
23
22-
22
0=12
5
22
02
0.8
0.2
0.4
0.6
03
62
0.6
Prosssilty
Prbsttiity
Figure 8: PSNR of ||x ||| under different mask generated from Bern(p) of different measurement
noise level using baseline method (PnP-DIP 27] with ADMM gradient descent algorithm).
C.2 Effect of bagging
We discussed how bagging can help address the DIP (or DVP) overfitting issue and provide a robust projection module, which can robustly capture the source structure, without any training data. Figurepi shows the impact of bagging on the performance of SCI-BDVP in Section by comparing it with the performances of different SCI-DVP solutions, where instead of bagged version, we used simple DVP for projection.
Figure 10 shows the qualitative reconstruction results of our proposed SCI-BDVP in comparision with the nongba version, SCI-DVP. An expected, the results show that using bagging improves the reconstruction quality, in both noise-free and noisy cases.
23

---

Table 6: Detailed mask optimization effect reconstruction with SCI-BDVP. PSNR (dB) (left entry) and SSIM (right entry) of the reconstruction results on different videos. (Reg.) represent reconstruction on using fixed regular binary mask, Dij ~ (orn((.5).. (OPT.) represents model tested on fixed optimized mask.
Noise Level
Mask Choice
Kobe
Traffic
Runner
Deup
Crash
Actial
Average
B-DVP (GAP) Reg.
28.42, 0.386
22,84,
34,32,0.955 0.954
40.76, 0,966
24.96, 0.851
25.16, 0.837
29.41, 0.841,08882
B-DVP (GAP)
0.893
23.47, 0.791
35,00, 0.958
25.66, 0,500
2955, 0.888
B-DVP (PGD) Reg.
26.39, 0,805
22.66, 0.740
31.15, 31,1,,0916 0.96
35 0.962
25.57, 0.835 0,835
25.62, 0.817
27.73, 01846
Opl.
R-DVP (PGD) Reg.
25.89.0.775
22.23,
90.31, 0.800
25.33,0.821
27.23.
B-DVP.PPGD Opt.
25.89, 0.775
0.718
30.31,
34.1 14,17,
25.33, 0.821
27.23, 0.827
(PGD) Reg.
25.11,
29:
24.21,
B-DVP (PGD) Opt.
23.71, 0.085
21.02, 0.649
26.52, 0.812
31.85, 0.900
24.26, 0.772
24.10, 0.735
25.24, 0.764
Txfftc
Kobb
28
12
22
12
13
2.
30
02-
016-122
50 254-336
50-0VP-254
1.
Numbbr
Figure 9: Effect of bagging. Reconstruction PSNR SCI-BDVP vs. SCI-DVP, where in each SCI-DVP separate DVP is employed (noise-free measurements).
Finally, further highlight the impact of the bagging operation, here explore the performance of the proposed bagged DVP solution for the classic inverse problem of denoising from additive Gaussian noise. Figure shows the denoising performance of the proposed bagged-DVP solution (refer to Figure 2b in denoising x from measurements y = + z, where z is generated i.i.d. N (O, o?). We compare the performance of bagged-DVP with the three DVP structures that are used as the building components of our bagged-DVP. As explained earlier, each of these DVPs operates at different patch size. can be observed that BDVP consistently outperforms the individual DVPs and shows a much more smooth convergence behaviour. Given that BDVP only averages the outputs of the three individual DVPs, observed gain suggests the independence of the estimates (at least partially), which leads to the observed gain.
C.3 Effect of averaging coefficient
We explore the effect of coefficient a used in combining the outputs from gradient descent and projection steps at time f,
X = ox. + (1-o) o)x
In Figure TT left panel, it can observed that without the skip connection, the performance drops by around 4 dB. However, for E 10.1, 0.7] the performance is stable and does not vary considerably as a changes. On the other extreme case when 1, when there no projection, as expected, the performance severely degrades.
24

---

a==
== 10
25
50
SCI-BDVP
Kobe #9
SCI-DVP
SCI-BDVP
Kobe #18
SCI-DVP
SCI-BDVP
Drap #6
SCI-DVP
SCI-BDVP
Traffic #36
SCI-DVP
C.4 Effect of reconstruction loss coefficient
We explore the effect of coefficient in reconstruction measurement loss, the second term the loss function in Figure 2
w|yy (1)
In the middle and right panel of Figure 11 we can find that for noise-free case the reconstruction results is not sensitive the change of w. However, the noisy case, when we increase the the reconstruction will drop and if not including the measurement loss term, 0, the results will also
25

---

10
02 25
50
SCI-BDVP
Runner #5
SCI-DVP
SCI-BDVP
Crash #27
SCI-DVP
SCI-BDVP
Aerial #22
SCI-DVP
Figure 10: Reconstruction results of SCI-BDVP vs. SCI-DVP. (leftmost images are clean frames).
drop. This is due in noisy case, the measurement y is longer the actual measurement, but still need some information from corrupted to boost the reconstruction results.
26

---

Stp connaction confiiiett nosseless
resonstruciion 055 cseffiiint seseless
reconstruciion 055 coefficien: 10 (5gma-25)
P5NR 1S51
PSNA (1E5
25.0
0.5
Figure 11: (Left) Effect of skip connection coefficient (noiseless measurements). (Middle) Effect of reconstruction loss coefficient (noiseless measurements). (Right) Effect of reconstruction loss coefficient effect (noisy measurements) (ar 25).
0.58
CHaviL
CrMPh-25
Osm/4)
134
140 vel
201
12.4
171
1808 200
500
2008
Nurter
1800
2000
3008
1300 Soide
1000
Scitco far
Isbert)
10ee13)
30.0-
212-
280
170
100
120
220
300
2000
V128
280
3000
Figure 12: Unsupervised video denoising Effect of bagging. Reconstruction PSNR corresponding to denoising using BDVP versus different DVP structures. (8 frames videos with additive Gaussian noise level of 25)
27

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: In the following we list the claims made in the abstract and introduction and references on how they are accurately addressed in the paper. (Section I.1 lists the contributions of the paper)
(theoretic) Theoretical analysis of cetrainnd--eural-nete
(UNN-based) SCI re- covery methods: Sections 3.2 employs DIP hypothesis and theoretically analyzes such bJsed SCI methods, for both noisy and noiseless measurements. (Refer Theorems 3.1, 3.4 and 3.5 for the theoretical characterization of UNN-based SCI recovery.)
(thcoretic) Theoretical statements the optimization of binary-valued masks under both noisy and noiseless mcasurements: Corollary 3.3 and Corollary provide the implications of our theoretical results mask optimization.
(theoretic) Theoretic upper bound the number of frames that can be recovered single encoded 2D measurement frame as a function of the parameters on the UNN: Corollary 3.2 presents this result.
(algorithmic) Introduction of novel unsupervised SCI recovery algorithm: The pro-
posed method, SCI-BDVP, is introduced in Section 4
(experimental) We have implemented the proposed method (SCI-BDVP) and compared its performance with both unsupervised and supervised SCI solutions, under both noisy and noiseless measurements. (Refer to Section
(experimental) We have claimed that the proposed method achieves state-of-the-ar performance among existing unsupervised methods. This is confirmed in the reported results in Table and Table
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: The second paragraph Section 6 reviews the theoretical and experimental limitations of our results.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: Our theorems are self-contained, including all necessary assumptions and settings. In the following section, we list the theoretical contributions of our paper and provide references for their proofs.
Theorem 3.1
Proved in Section
A.2
Corollary
3.2
Straightforward result following Theorem 3.1
Corollary
3.3
Proved in Scction
A.3
Theorem 3 4
Proved in Section A.4
Thcorem
3.5
Proved in Section A.5
Unfortunately we didn't have enough space in the paper to provide highlights of the proofs
in the main body of the paper.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
28

---

Justification:
We have released codes.
Our algorithm clearly explained the paper. We have also given a detailed descrip- tion of the UNNs used in the our proposed bagged-DVP solution.
We have clearly stated our choice of hyperparameters in the paper.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We have provided our code which includes all the parameter settings and implementations. For data, we have used benchmark vidcos including Kobe, Runner, Drop, Traffic, Aerial, Vehicle [8]
6. Experimental Setting/Details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification:
Our proposed SCI-BDVP algorithm and the structure main component, i.e., bagged DVP, are presented in Section 4
The descent step used by our proposed SCI-BDVP algorithm is explained in Section B.1 of the supplementary materials.
The hyperparameter settings are explained in Section B.2 supplementary materials.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: We have not provided error bars due to computational complexity of solving the required NN training. We want to highlight points: Since method unsupervised, its performance on each data cube independent of any training data. 2) None of the cited prior works SCI recovery include error bars.
8. Experiments Compute Resources
Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification: The details of our computation resources presented in Section B.2
Sec-
tion B.3 5
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
29

---

Justification: Our proposed solution is an unsupervised method for SCI recovery, which can enable recovery without any training data. Moreover, we provide theoretical under- standing of the problem, which enable hardware optimization, without requiring extensive computationallly-nteenn
empirical optimizations. All the positive aspects are discussed in the paper. The work does not have any negative societal impact.
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have cited the source of the benchmark videos and also the codes of different algorithms used in our implementations.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We have submitted an anonymized version of our code that implements our proposed SCI-BDVP algorithm.
We will later upload the code to GitHub and make public.
14. Crowdsoureing and Research with Human Subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
30


Reviews:
[{"review_id": "PXsgEZSVFT", "reviewer": "Reviewer_pMi8", "summary": "This paper leverages untrained neural networks UNN (deep image priors DIP or deep decoder DD) as a prior to solve Snapshot Compressive Imaging (SCI), a technique used in ($n_1$ x $n_2$ x B)-dimensional 3-D imaging where the captured measurements lie in a 2-D plane ($n_1$ x $n_2$). The application of UNNs in the context of SCI itself is not novel; this paper's main contributions are:\n1) Theoretical recovery guarantees for SCI (i.e. existence of a minimizer to the reconstruction problem) denoting number of 2-D frames B that can be recovered using 1 2-D measurement as a function of UNN model complexity (assuming original signal is close to range of UNN by $\\delta$) under noise-free and additive gaussian noise settings.\n2) Use bagged-DIP as algorithm for signal recovery, called SCI Bagged Deep Video Prior (BDVP).\n3) Optimize binary-valued masks used in the measurement process and show empirical analysis.", "questions": "Theoretical claims are summarized in main paper; however I was not able to find longer version of paper in supplementary zip folder to validate the proofs - this would help consolidate the contents.", "limitations": "Limitations have been discussed; they should additionally discuss time/computational complexity of the proposed BDVP algorithm.", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "The main contribution of this paper are theoretical reconstruction guarantees for SCI using UNNs under both noisy and noiseless measurements. They derive a bound on reconstruction error in terms of signal parameter B, Bernoulli sampling pattern mask p, measurement and signal parameter n, and $\\sigma_z$ noise. \n\nAuthors further validate their reconstruction error bound and its dependence on sampling parameter p, on empirical datasets on 6 videos: Drop, Runner, Aerial, Crash, Kobe, Traffic.\n\nThey propose SCI Bagged Deep Video Prior method as an algorithmic framework for solving SCI. On the empirical data, they show improved SNR and SSIM against baseline untrained methods.", "weaknesses": "Main comments:\nComputational complexity: Deep Video Prior/Untrained Network Prior setups have high computational complexity; on top of this a bagging approach requires solving K such problems simultaneously. This requires a discussion on how computational complexity compares to baselines.\n\nTheoretical claims are summarized in main paper; however I was not able to find longer version of paper in supplementary zip folder to validate the proofs.\n\nTheorem 3.1: Are parameters $p$ in $u [0,1]^p$ and Bern($p$) for $D_{i,j}$ the same? \n\nMinor comments:\nline 47-48: \"Existing UNN-based SCI solutions either recover the image 48 end-to-end in one shot or employ iterative methods akin to projected gradient descent (PGD).\" (relevant citations missing)\n\nline 88-97: Literature review of DIP + UNN: Add relevant citation to \"Qiao, M., Liu, X., & Yuan, X. (2021). Snapshot temporal compressive microscopy using an iterative algorithm with untrained neural networks. Optics Letters, 46(8), 1888-1891.\"\n\nTheorem 3.1. reconstruction error bound - $\\rho$ not defined.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for carefully reading our paper and providing constructive feedback.\n\n&nbsp;\n\n`Q-1:` Discussion about Computational complexity.\n\n`A-1:` We have made the following changes to the paper: In the main body of the paper, in Section 5.1, we have added the following explanation on the computational complexity of our proposed method. _\"The proposed SCI-BDVP method relies on the bagging of multiple DIP projections. These DIP projections, which vary depending on the patch size, involve different levels of computational complexity. Table 2 shows the average time required to perform each DIP projection for each patch size. As observed, the time increases considerably as the patch size decreases. This is expected because the number of networks that need to be trained grows significantly. (Refer to Figure 1 for a pictorial representation.)  Additional  computational complexity analysis  of our proposed method and its comparisons with other methods is included in Appendix B.3.\"_ \n\nFurthermore, we have added Section B.3 in the Appendix, which provides detailed information about the computational and time complexity of our proposed method. It also includes a comparison between our method and other UNN-based approaches. For completeness, we have copied the newly added section here as well.\n\n_\"Implementing SCI-BDVP involves outer loop iterations (described in Algorithm 1 in appendix B.1) and also inner loop iterations for training DIPs. Table 3 presents average number of inner loop iterations used for different patch sizes (64, 128, 256) of various videos, and the number of outer loop iterations. Detailed time consumption for each patch level computation is recorded in Table 2. A comparison across different UNN-based methods is provided in Table 1. All comparisons are performed on a single NVIDIA RTX 4090. It is important to note that training a bagged DIP requires training multiple separate DIPs. This process can be readily parallelized, which is expected to significantly speed up the algorithm. We plan to explore this direction to optimize the algorithm's efficiency in future work. Lastly, making a direct comparison among all methods is challenging because, for supervised methods, the main time is spent in training, whereas, for unsupervised methods, the main time is spent on training the UNNs. This is an expected trade-off for requiring no training data and achieving a robust solution.\"_\n\n|| **Methods**| **Time (min.)** |\n|-|-|-|\n| **No noise**| PnP-DIP| 18|\n|| Factorized-DVP| 15|\n|| Simple-DVP (E2E)| 10|\n|| SCI-BDVP| 35 or 220|\n| **With noise**| PnP-DIP| 18|\n|| Factorized-DVP| $-$|\n|| Simple-DVP (E2E)| 10|\n|| SCI-BDVP|40|\n\n**Table 1**: Time complexity of different methods on one 8-frame benchmark video.\n\n| **Patch size** | # of patches | Time (min.) |\n|-|-|-|\n| 64| 16| 1.2|\n| 128| 4| 0.28|\n| 256| 1| 0.15|\n\n**Table 2:** Time complexity of our proposed SCI-BDVP was evaluated on various patch sizes (64, 128, 256) of video blocks, using a standard 1000 DVP iterations for training.\n\n&nbsp;\n\n`Q-2:` Proofs of the theoretical results.\n\n`A-2:` The main pdf file (29 pages) includes detailed proofs of all the results. More specifically, in Appendix A, we have provided detailed proofs in the following order: proof of Theorem 3.1 in A.2, proof for Corollary 3.3 in A.3, proof for Theorem 3.4 in A.4 and finally proof for Theorem 3.5 in A.5.\n\n&nbsp;\n\n`Q-3:` Theorem 3.1: Are parameters $p$ in $u[0,1]^p$ and Bern($p$) for $D_{i,j}$ the same?\n\n`A-3:` We had inadvertently overloaded the symbol $p$. We will substitute $p$ in $u[0,1]^p$ with $N$.\n\n&nbsp;\n\n`Q-4:` Theorem 3.1. reconstruction error bound - $\\rho$ not defined.\n\n`A-4:` Thanks for pointing this out. $\\rho$ denotes an upper bound on the $\\ell$-infinity norm of the signals in $\\mathcal{Q}$. We  added the description in the revised version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DQxJYojKrz", "reviewer": "Reviewer_tX5M", "summary": "The focus of this paper is on developing recovery algorithms of snapshot compressive imaging (SCI) using untrained neural networks (UNNs). Besides, the paper introduces the concept of bagged-deep-image-prior (bagged-DIP) to create SCI Bagged Deep Video Prior (SCI-BDVP) algorithms, which are designed to address common challenges faced by standard UNN solutions in SCI recovery. Extensive experiments demonstrate the effectiveness of the proposed method in video SCI recovery. In scenarios with noisy measurements, this untrained network even outperforms supervised methods.", "questions": "- SCI is also widely used in hyperspectral image, can your method be used in hyperspectral reconstruction task? If so, what's the prominent advantage of your method?\n\n- How does the method handle different noise models beyond additive Gaussian noise?", "limitations": "- Exploring the application on additional datasets from different domains or varing conditions in the future may increase persuasiveness, e.g. hyperspectral data. And exploring the performance of the proposed method under different noise models would provide a more comprehensive evaluation of its robustness and general applicability.", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "- This paper is well structured and has clear logic.\n\n- The experimental results on the performance are convincing.\n\n- The proposed method is based on untrained neural network, making it more flexible to be applied in various scenes.", "weaknesses": "- For untrained methods, runtime may be a crucial metric [1,2]. But this paper doesn't provide analysis in comparison to existing methods.\n\n- To improve clarity, consider adding clear notations for the symbols used. For example, what do different colored arrows represent in Figure 2, and the color of \"Fusion\" and \"MSE\" are too similar to be distinguished.\n\n[1] Rui, Xiangyu, et al. \"Unsupervised hyperspectral pansharpening via low-rank diffusion model.\" Information Fusion 107 (2024): 102325.  \n[2] Pang, Li, et al. \"HIR-Diff: Unsupervised Hyperspectral Image Restoration Via Improved Diffusion Models.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for carefully reading our paper and providing constructive feedback. \n\n&nbsp;\n\n`Q-1:` Discussion about computational complexity and runtime.\n\n`A-1:` We have made the following changes to the paper: In the main body of the paper, in Section 5.1, we have added the following explanation on the computational complexity of our proposed method. _\"The proposed SCI-BDVP method relies on the bagging of multiple DIP projections. These DIP projections, which vary depending on the patch size, involve different levels of computational complexity. Table 2 shows the average time required to perform each DIP projection for each patch size. As observed, the time increases considerably as the patch size decreases. This is expected because the number of networks that need to be trained grows significantly. (Refer to Figure 1 for a pictorial representation.)  Additional  computational complexity analysis  of our proposed method and its comparisons with other methods is included in Appendix B.3.\"_ \n\nFurthermore, we have added Section B.3 in the Appendix, which provides detailed information about the computational and time complexity of our proposed method. It also includes a comparison between our method and other UNN-based approaches. For completeness, we have copied the newly added section here as well.\n\n_\"Implementing SCI-BDVP involves outer loop iterations (described in Algorithm 1 in appendix B.1) and also inner loop iterations for training DIPs. Table 3 presents average number of inner loop iterations used for different patch sizes (64, 128, 256) of various videos, and the number of outer loop iterations. Detailed time consumption for each patch level computation is recorded in Table 2. A comparison across different UNN-based methods is provided in Table 1. All comparisons are performed on a single NVIDIA RTX 4090. It is important to note that training a bagged DIP requires training multiple separate DIPs. This process can be readily parallelized, which is expected to significantly speed up the algorithm. We plan to explore this direction to optimize the algorithm's efficiency in future work. Lastly, making a direct comparison among all methods is challenging because, for supervised methods, the main time is spent in training, whereas, for unsupervised methods, the main time is spent on training the UNNs. This is an expected trade-off for requiring no training data and achieving a robust solution.\"_\n\n\n||**Methods**|**Time (min.)**|\n|-|-|-|\n|**No noise**|PnP-DIP| 18|\n||Factorized-DVP|15|\n||Simple-DVP (E2E)|10|\n||SCI-BDVP|35 or 220|\n| **With noise**|PnP-DIP| 18|\n||Factorized-DVP|$-$|\n||Simple-DVP (E2E)|10|\n||SCI-BDVP|40|\n\n**Table 1**: Time complexity of different methods on one 8-frame benchmark video.\n\n|**Patch size**|# of patches|Time (min.)|\n|-|-|-|\n|64|16|1.2|\n|128|4|0.28|\n|256|1|0.15|\n\n**Table 2:** Time complexity of our proposed SCI-BDVP was evaluated on various patch sizes (64, 128, 256) of video blocks, using a standard 1000 DVP iterations for training.\n\n&nbsp;\n\n`Q-2:` Clarity of notations and figure presentation. \n\n`A-2:` In the revised version, we modify Figure 2 as follows: i) increase the font size; ii)  add a detailed caption; iii)  separate the 2D measurement $\\bf y$ and the 3D binary mask $H$; iv)  change some of the solid lines to dashed lines; v)  improve the layout of the figure to make it easier to follow. The updated Figure 2 is included in the one-page PDF response under the general rebuttal.\n\nAlso, we have reviewed the paper to ensure that all notations are clearly defined and consistently used.\n\n&nbsp;\n\n`Q-3:` Potential application of  our proposed method in hyperspectral imaging.\n\n`A-3:` We expect our method to be applicable to hyperspectral imaging as well. In fact, one of the papers we have cited and used in our simulations for comparison (reference [27] in the paper) utilizes DIP for hyperspectral imaging (HI). Here are two key advantages of our proposed framework:\n\n1. Our proposed theoretical framework is applicable to hyperspectral imaging and provides a solid theoretical foundation to analytically explore other aspects of HI, such as the effect of shifted masks.\n\n2. A well-known challenge in using UNNs in reconstruction algorithms is their tendency to suffer from overfitting. However, our proposed method, based on bagging, is expected to perform well in HI and overcome these issues.\n\nTo highlight this direction and its potential, we add this paragraph in the conclusion section of the revised version: \"An important application of SCI is hyperspectral snapshot imaging (HSI). Our results in this paper provide a theoretical foundation to understand HSI systems and optimize their hardware. Additionally, the developed theoretical framework can be used to explore aspects specific to HSI, such as masks being shifted versions of each other. We also expect our algorithm to effectively address overfitting in HSI tasks, enhancing reconstruction performance. We plan to explore these aspects further in our future research.\"\n\n&nbsp;\n\n`Q-4:` Different noise models beyond additive Gaussian noise?\n\n`A-4:` The exploration of noise models beyond additive white Gaussian  (AWGN) noise is a valuable and important   direction for future work. While AWGN is a commonly-adopted model in many imaging solutions, including most SCI systems, there are situations, such as some newly-proposed coherence imaging SCI methods, that other types of noise such as speckle noise are dominant. We believe that  our UNN-based approach to model the source has the potential to address these alternative noise scenarios. To accommodate non-AWGN noise models, the loss function $||\\bf y-\\bf H\\bf c||_2$ would require modification depending on the noise model. We consider this an important avenue for future investigation.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eD1KAv2RIT", "reviewer": "Reviewer_E4xU", "summary": "This work gives some theoretical analysis for mask optimization and DIP-based SCI recovery methods. The work claims that the proposed SCI-BDVP achieves SOTA performance among UNN methods.", "questions": "How can the UNNs used in this work be guaranteed to satisfy the DIP hypothesis and Lipschitz's condition?", "limitations": "The authors provide some limitations, which is their future research.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. This work provides some theoretical analysis, compared with the conventional work is rare.\n2. Bagged DIP is introduced to develop SCI iterative methods and the proposed method claims to achieve SOTA performance among UNN methods.", "weaknesses": "1. While this work does some theoretical analysis by introducing DIP's theory, it cannot be directly generalized to all untrained networks. On the one hand, it seems impossible that all untrained networks satisfy the DIP hypothesis in the paper, and on the other hand, how can the UNNs used in this work be guaranteed to satisfy the DIP hypothesis and Lipschitz's condition? As far as I know, networks that satisfy Lipschitz's condition require a specific design, and this work does not seem to give some related introduction.\n2. The core of this work is to prove theoretical results for untrained networks, but the actual contribution is less than stated. When the properties of certain untrained networks are demonstrated by DIP, the work does not propose any new hypotheses in snapshot compressive imaging but simply adopts the original one, which decreases the contribution of the paper. \n3. When the DIP hypothesis is used to simplify the UNNs as a minimization operator, the work is merely proving a boundary based on the minimization operator. Similar theoretical results based on the minimization operator can also be found in the theoretical analysis section of GAP-Net[1].\n4. The DIP hypothesis demonstrated in this paper seems to ignore some conditions in the original DIP paper and doesn't capture the unique characteristics of untrained networks, what if the same is true of pre-trained networks? \n5. The comparison algorithm needs to be perfected. The current comparison algorithms seem to be some traditional optimization algorithms and your methods, and we even can't distinguish which of them is previous UNN works. So what does it mean that your method achieves SOTA performance in UNNs？\n6. Considering this work as a deep network-based approach, such comparisons are unfair. Why not add some recent self-supervised methods to demonstrate the effectiveness of the algorithm? In addition, despite claiming to be compared with the supervised algorithms in noisy scenarios, some recent algorithms, such as SCI3D[2] and EfficientSCI[3] have not been included.\n\n[1]Deep Unfolding for Snapshot Compressive Imaging, in IJCV2023.\n\n[2]Dense Deep Unfolding Network with 3D-CNN Prior for Snapshot CompressiveImaging, in ICCV2021.\n\n[3]EfficientSCI: Densely Connected Network with Space-time Factorization for Large-scale Video Snapshot Compressive Imaging, in CVPR2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for carefully reading our paper and providing constructive feedback. \n\n`Q-1:` Application of DIP hypothesis to all untrained networks and the relevance of the Lipschitz condition.\n\n`A-1:` As mentioned by the reviewer, not all untrained neural nets (UNNs) satisfy the DIP hypothesis and this  is not the assumption in our paper. The assumption is that for any class of signals, there exist UNN structures that satisfy the DIP hypothesis. More specifically,  consider a class of signals denoted by $\\mathcal{Q}\\subset\\mathbb{R}^n$. A UNN, modeled as $g_{\\theta}(\\bf u)$, satisfies the DIP-hypothesis if for any randomly generated $\\bf u\\in\\mathbb{R}^N$ (generated according to some preset distribution), and any $\\bf x\\in\\mathcal{Q}$, $\\min_{\\theta} ||g_{\\theta}(\\bf u) - \\bf x||_2 \\leq \\delta,\n$, almost surely. Existence of such UNN structures for different class of signals $\\mathcal{Q}$ is shown empirically. For example, deep image prior (DIP) and deep decoder (DD) are well-known UNN structures  satisfying this property for natural images.  \n\nRegarding Lipschitz continuity, note that our UNN architecture comprises multiple hidden layers, each employing a linear convolution operator followed by a ReLU nonlinearity. The final layer consists of a linear operation and a sigmoid activation function. Given the compositional nature of these layers, it is evident that our UNN satisfies the Lipschitz condition. Notably, our theorem is applicable to any Lipschitz constant $L$.\n\n&nbsp;\n\n`Q-2:` Questions about the contributions of the paper.\n\n`A-2:` The focus of this work is on the _application_ of UNNs in solving SCI problem and developing i) a theoretical  understanding of the problem, and ii)  a robust UNN-based SCI solution. Specifically, here are our key contributions:\n\n1. Theoretically characterizing the performance of SCI solutions that employ a generic UNN for recovery. The key implication of our theoretical characterizations is that it enables us to optimize the parameters of SCI systems' hardware, under both noise-free and noisy setups.  \n\n2. Proposing a novel unsupervised SCI solution for video SCI. We establish the effectiveness of our proposed solution through  extensive simulations. \n\n\n&nbsp;\n\n`Q-3:` Comparison between this work and the GAP-Net paper [1] and the potential overlap between the two papers.\n\n`A-3:` In this paper, we theoretically analyze the performance of DIP-SCI optimization defined as\n\n\\begin{align}\n\\hat{\\bf x}=\\arg\\min ||\\bf y-\\bf H\\bf c||_2,\n\\end{align}\n\nsubject to $\\bf c=g_{\\theta}(\\bf u),\\theta\\in[0,1]^k$\n\n[1] _does not_ include any theoretical analysis of this optimization or any of its variants. Instead, [1] focuses on the convergence behavior of projected gradient descent and its variant, GAP, in solving a related compression-based optimization. Also note that in [1], the  entries of the masks are assumed to be i.i.d.~ Gaussian, which is inconsistent with masks used in practice. Instead, in our work, we consider an i.i.d.~Bern$(p)$ distribution for masks.\n\n&nbsp;\n\n`Q-4:` Conditions of DIP hypothesis compared to the original DIP paper and connections to pre-trained networks.\n\nPre-trained networks and UNNs differ significantly in their operational basis. While pre-trained networks leverage extensive datasets to acquire knowledge, UNNs operate without requiring any training data. This fundamental distinction grants UNNs a clear advantage in scenarios with limited data, making them more adaptable to various applications. Consequently, although pre-trained networks can be integrated into our framework, the unique strength of UNNs in data-scarce environments underscores their important role in our methodology.\n\nFinally, we would like to ask the reviewer to explicitly identify which DIP conditions they believe are overlooked in our paper. This helps us better address the concerns. \n\n&nbsp;\n\n`Q-5:` Comparison algorithms included in the paper.\n\n`A-5:` In Section 5, we compare the performance of our proposed method against several other  methods, listed in the section labeled `Datasets and baselines'. To the best of our knowledge, we cover all existing UNN-based video SCI solutions in our comparisons. We also extended one UNN-based method, originally proposed for hyperspectral SCI, and compared its performance against our method. \n\nTo better clarify and distinguish the methods compared to our proposed method, we will update Table 1 in our main paper by organizing the comparisons into four categories:\n\n1. GAP-TV [9], a traditional optimization-based algorithm.\n\n2. PnP-FFDnet [17] and PnP-FastDVDnet [18], pre-trained deep denoisers + PnP algorithm.\n\n3. Existing UNN-based solutions for video SCI [30], [31].\n\n4. Our main proposed method (SCI-BDVP) and two simpler variants for comparison: 1) SCI-DVP, simple E2E DVP, 2) SCI-BDVP,  bagged E2E DVP, and 3) SCI-BDVP, our main proposed method. \n\nThe last two categories are UNN-based methods, we consider our model to be state-of-the-art among this category.\n\n&nbsp;\n\n`Q-6:` Comparison with some mentioned recent papers.\n \n`A-6:` Methods such as SCI3D and EfficientSCI are supervised approaches that require extensive training data. A key limitation of these supervised methods is that they fix the sensing matrix during training. This makes it challenging to infer the impact of masks on performance and to separate this effect from the optimization performance. Additionally, applying the trained network to different sensing matrices and simulation settings typically results in performance degradation.\n\nIn this paper, one of our main goals is to develop _unsupervised_ solutions that do not require training data or rely on prior knowledge of the sensing matrix and also to provide a fundamental understanding of the role of masks in performance. Therefore, we have not included supervised, mask-dependent methods in our comparisons.\n\n__References can be found in our main paper.__", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "w6w2dwADJo", "reviewer": "Reviewer_ytyT", "summary": "Please see Strengths and Weaknesses.", "questions": "NA", "limitations": "NA", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Rigorous theoretical analysis of the proposed formulation\n2. Detailed empirical evaluation.", "weaknesses": "1. Formatting in Figure 2 can be improved. The text near smaller cubes is not readable.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for carefully reading our paper and providing constructive feedback. \n\n&nbsp;\n\n`Q-1:` Formatting in Figure 2 can be improved. The text near smaller cubes is not readable.\n\n`A-1:` We thanks the reviewer for the  suggestion.  In the revised version, we modify Figure 2 as follows: i) We increase the font size to make the figure more readable; ii) We add a detailed caption to explain the various components of the figure, including the meaning of different colored lines and dots; iii) We separate the 2D measurement $\\bf y$ and the 3D binary mask $H$ to avoid confusion; iv) We change some of the solid lines to dashed lines to make the figure easier to interpret; v) We improve the layout of the figure to make it easier to follow. The updated Figure 2 is included in the one-page PDF response under the general rebuttal.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank all reviewers for their valuable feedback and thoughtful comments. We have carefully considered their suggestions and revised the paper accordingly. In the following, we address the main comments/questions from each reviewer..

The attached pdf file includes an improved version of Figure 2 in our original manuscript.

Meta-review: The paper introduces a new approach for solving the snapshot compressive imaging (SCI) problem, where the goal is to reconstruct 3D data cubes from 2D measurements. Applications of this problem include video and hyperspectral imaging. The approach is based on a 3D extension of bagged deep image priors. The authors provide both theoretical bounds on the reconstruction error (under the so-called deep image prior hypothesis), as well as simulation results on benchmark video datasets.

The paper is well written, the method is sensible, and the supporting results are comprehensive. During the response phase the authors were able to adequately address most of the reviewers' comments.
Decision: Accept (poster)