Scalable Deep Compressive Sensing
Authors: Zhonghao Zhang
Yipeng Liu
Xingyu Cao
Fei Wen
Ce Zhu
Venue: Accepted by TMLR ()

Abstract: Deep learning has been used to image compressive sensing (CS) for enhanced reconstruction performance. However, most existing deep learning methods train different models for different subsampling ratios, which brings an additional hardware burden. In this paper, we develop a general framework named scalable deep compressive sensing (SDCS) for the scalable sampling and reconstruction (SSR) of all existing end-to-end-trained models. In the proposed way, images are measured and initialized linearly. Two sampling matrix masks are introduced to flexibly control the subsampling ratios used in sampling and reconstruction, respectively. To achieve a reconstruction model with flexible subsampling ratios, a training strategy dubbed scalable training is developed. In scalable training, the model is trained with the sampling matrix and the initialization matrix at various subsampling ratios by integrating different sampling matrix masks. Experimental results show that models with SDCS can achieve SSR without changing their structure while maintaining good performance, and SDCS outperforms other SSR methods.

shonghaozhang@@yaa..cc Published in Transactions on Machine Learning Research (04/2023)
Scalable Deep Compressive Sensing
Zhonghao Zhang
School of Information and Communication Engineering, University of Electronic Science and Technology of China (UESTC), Chengdu, China
Yipeng Liu
yipengliu vipengliu@uesttc Bueste. edu.cn on School of Information and Communication Engineering, University of Electronic Science and Technology of China (UESTC), Chengdu, China
Xingyu Cao Alibaba DAMO Academy
shutong.cry wrwphenccccoobbbb--beeeeeee
Fei Wen
wenfei@sjtu.edu.cn
Department of Electronic Engineering, Shanghai Jiao Tong University, Shanghai, China
Ce Zhu
ecaha ocehttuestc..c edu.cn School of Information and Communication Engineering, University of Electronic Science and Technology of China (UESTC), Chengdu, China
Reviewed on OpenReview: https: //
// openrevieww net/ forum? id= 10JdgrzN00
Abstract
Deep learning has been used to image compressive sensing (CS) for enhanced reconstruc- lion performance. However, most existing deep learning methods train different models for different subsampling ratios, which brings an additional hardware burden. In this paper, we develop a general framework named scalable deep compressive sensing (SDCS) for the scalable sampling and reconstruction (SSR) of all existing end- and-to-end-trained models. In the proposed way, images are measured and initialized linearly. Two sampling matrix masks are introduced to flexibly control the subsampling ratios used in sampling and reconstruction, respectively. To achieve reconstruction model with flexible subsampling ratios, training strategy dubbed scalable training is developed. In scalable training, the model is trained with the sampling matrix and the initialization matrix at various subsampling ratios by inte- grating different sampling matrix masks. Experimental results show that models with SDCS can achieve SSR without changing their structure while maintaining good performance, and SDCS out performs other SSR methods.
1 Introduction
Compressive sensing (CS) is technique that simultancously samples and compressos signals. And the signal is sampled and reconstructed at ratio that can be much lower than the Nyquist rate. The sampling process of CS can be expressed y Ax where E NR is the original signal, y E R denotes the measurement, A E F M is the sampling matrix with M < N and M/N is the CS ratio. The signal recovery from is under-determined, and it is usually carriod out by solving an optimization problem as follows:
min R(x), S. Ax,
(1)
where R(x) is the regularization term. In this paper, we mainly focus on the visual image CS Lohit et al.
2018a) which has been applied in single-pixel imaging (SPI) (oohit et al. 2018a Duarte el. al. 2008) and
*Corresponding author

---

Published in Transactions on Machine Learning Research (04/2023)
vec(-)
x()
x()
vec ())
MOA
MOB
Scalable Sampling
Scalable Initialization
Figure Scalable sampling and scalable initialization of an image block.
wireless broadcast. YYin et al.
2016
Li et al. 2013
Guo et al.) 2020 And since block-by-block sampling and
reconstruction (Dong et al.)
2014 Dinh & Jeon 2017 Lohit et al.
2018 Zhang & Ghanem 2018 Zhang
et al. 2021) would bring less burden to the hardware, we mainly focus on the block-based visual image CS
problem.
To solve the problem 1
model-basod mothods (Dong et al., 2014 Li ct al. 2020 introduce various hand-
crafted regularizers (Elad 2010 Liu et al., 2019) and apply non-linear iterative algorithms (Beck & Teboulle,
2009 Donoho et al., 2009 to recover images. These methods usually have theoretical guarantees and work
well using sampling matrices with different CS ratios. However, their performance needs to be further improved.
In recent years, deep learning has achieved great success in computer vision (Rick Chang et al. 2017 Dong ct al. 2018 Yan et al. (2200a Tu et al.
2021 Among them, models for visual image CS can be cast into two categories: traditional deep learning models and deep unfolding models. Traditional
2022 Hu al.
deep learning models (Mousavi et al. 2015 Lohit al. 2018a Shi et al.) 2019a; 2020) are usually stacked by non-linear computational layers. These models map the measurement to the output without considering the prior information of images. Although they can reconstruct high-quality images at high speed, there is no good interpretability and theoretical guarantee (Huang et. al.] 2018). Deep unfolding models denote a series of models constructed by mapping iterative algorithms with unfixed numbers of steps onto deep neural networks with fixed numbers of steps GGregor & LeCun 2010 Zhang & Ghanem 2018 Metzler et al., 2017]
Zhang et al., 2021 Dong al. 2018 Sun et al. 2016 Ma al. 2019). By combining the interpretability of model-based methods and the trainable characteristics of traditional deep learning models, they make good balance between reconstruction performance and interpretability.
Csually, the above two kinds of deep-learning based models are trained end-to-end using some well- well-known backpropagation algorithms [Kingma & Ba 2015). However, a common shortage of most existing end-to- end-trained models is that diffcrent models have to be trained for differont CS ratios. In some applications, sampling and reconstructing images at. different CS ratios may be required (Yin et al., 2016 Li et al. 2013). However, storing more than one model with the same structure would bring additional burdens the hardware. Thus, sampling and reconstructing images at different CS ratios with only model is needed.
At present. there exist few methods (Xu et al. 2020 Su & Lian 2020 Shi et al. 2019b Zhang et. al. 2020:
Lohit et al. 2018b Li et al. 2020) which reconstruct images at different CS ratios using only one model, and they can be roughly cast into two categories. The first kind (Xu et al. 2020 Su & Lian 2020 trains
a single model to adapt to a set of sampling matrices with different CS ratios. The second kind (Shi et al. 2019b; Lohit et al.] 2018b) applics only one sampling matrix and integrates rows to achieve sampling and reconstruction different CS ratios, and we call such a strategy as scalable sampling and reconstruction (SSR) in this paper. This paper focuses on SSR. methods, because they are more practical in existing applications such as SP1 (Lohit et al. 2018a), wireless broadcasting (Yin et al. 2016), and MR1 (Sun et al. 2016). In detail, in image CS for wireless broadcast (Yin et al. 2016 Li ct al. 2013). images are reconstructed with different quality according to different channel conditions using one sampling matrix at the receiving end. And in SPI (Lohit et al.) 22118)), different CS ratios can be applied for different image quality and sampling time by combining rows of the sampling matrix. However, existing SSR methods cannot be applied universally (Shi et al.) 2019bb or a more appropriate sampling matrix is needed (Zhang et al., 2020 Lohit et al. 2018b). Therefore, a general and more effective SSR method is expected.
2

---

Published in Transactions on Machine Learning Research (04/2023)
Generating sampling matrix masks
Scalable Reconstruction m (:))
MM.M,,.. M
Reconstructed Image Blocks
Original Image Blocks
MOA
A M OA
OB
or
Scalable Sampligg
Scalable
Bunt 65, .)
Ini Initialization
X HxL
X, = H.L.
y, e E
X e NER" HaT
Figure 2: proppottroooogttion of the scalable training.
In this paper, propose general framework dubbed scalable deep compressive sensing (SDCS) to achieve sampling and reconstructing images at all CS ratios in certain range. In detail, two binary sampling matrix masks are developed to activate rows of the sampling matrix and the initialization matrix to control the CS ratios for SSR. And we develop a novel training strategy named scalable training, which integrates the multi-ratio information into the training stage by randomly generating sampling matrix masks for different CS ratios. We emphasize that SDCS can bring to the model the ability of SSR, while maintaining the characteristics of its structure. Furthermore, experimental results show that the model with SDCS obtain more effective combination of sampling matrix and model than existing SSR methods.
Our paper has three contributions:
We propose a framework named SDCS that jointly trains the sampling matrix and the model to achieve sampling and reconstruction at all CS ratios certain range.
With SDCS, a deep learning model can achieve SSR without changing its original structure, while maintaining good performance.
Technically, SDCS can be used for end-to-end -rraineedd deep learning models.
2 SDCS
In this section, we introduce the proposed framework SDCS which is simple but powerful. SDCS is composed of four parts: scalable sampling, scalable initialization, scalable reconstruction, and scalable training.
2.1 Scalable Sampling
Assume that the largest CS ratio is RM, then the sampling matrix can be expressed as E R R[RaN] It can be noticed that the CS ratio is determined by the row number of A. Therefore, to achiove scalable sampling, we design a sampling matrix mask Ms E TR [RxN control the activities of the rows of A. Ms is a zero-one matrix which satisfies Ms(1 [Rs) 1 and Ms([ denotes the CS ratio for sampling. In such case, where denoles the element-wise product. Since the NR + 1-th row to the -Rg row of As are all filled with 0. we say that the first row to the RR row of A are activated. In detail, if the original image block is X E x satisfying N = HL, then the scalable sampling at the CS ratio of Rs can be expressed as:
1 [ .:) 0, where Rs generate new sampling matrix As Mş A,
As vec(X),
(2)
3

---

Published in Transactions on Machine Learning Research (04/2023)
where v-c(-) is an operator which transforms a matrix to vector and y E R [RxN is the measurement. It can be noticed that y( Rg + 11 [RxNN] = 0 and y(1 [R is the valid measurement for reconstruction. In this way, we can achieve a unified learning mode with different compression rates.
2.2 Scalable Initialization
For deep learning methods, the initialized image is important in the following reconstruction. In SDCS, we use a linear operation to initialize the image block.
Based on some deep-learning based models (Shi et al. 2019a Zhang & Ghanem 2018 Zhang et al. 2021 an initialization matrix B V is developed. Similar to (2). sampling matrix mask MR R Rxa X
is proposed to control the activities of the columns of B, where (1 N .:) 1 and Mn Rn N7 +1 [ = 0. RR denotes the CS ratio for initialization and reconstruction, which satisfies AR S Rs. In such a case, we can aclivate the first column to the [R column of B to generate a new initialization matrix as BR = MR 2 B. The detailed scalable initialization at the CS ratio of Rg can be expressed as:
x = vec (BRy),
where X E R denotes the initialized image block and vec vec (-) is the operator which transforms vector to matrix.
In some cases, RR can be lower than Rg. For example, in wireless broadcasting Yin al. 2016 images are transferred at a high CS ratio and are received low CS ratio due the poor channel condition. Fig. illustrates the scalable sampling and scalable initialization of an image.
2.3 Scalable Reconstruction
In this subsection, we describe the scalable reconstruction of two different kinds of deep learning models: traditional deep learning models and deep unfolding models.
The generalized reconstruction process of traditional deep learning models can be expressed as:
X = 3trr O),
(4)
where X HR is the reconstructed image block and O contains trainable parameters of the model. Tn SDCS, O is trained with A and B to make sure that (:al: ²) perform well at all CS ratios.
The reconstruction model of deep unfolding model is usually composed of K reconstruction modules with the same structure. In each module, the sampling matrix also participates in the image reconstruction. In detail, the generalized roconstruction process of a deep unfolding model can be expressed as:
x = 3mn((XX y; A, +) 3K (X K-1 ,K),
(5)
X Bt (X
y; A, O*
(6)
where Bunt(.,:; A, ②) is the entire deep unfolding model, of which e contains its trainable parameters. ( A, hk) is the k-th reconstruction module and @ contains its trainable parameters. the inputs of Bunf(:;;; A, θ) and & ; @*) usually contain the image block X and the measurement y. Since plays an important role each reconstruction module, the scalable reconstruction of the deep unfolding model is achieved by applying an activated sampling matrix. In detail, the scalable reconstruction of the k-th reconstruction model can be expressed as:
x = Shi x y: O A, O*
(7)
Similar the traditional deep learning models, c (0', A .... @k) trained with A and B. Since the sampling matrix A usually appears in the image sampling and reconstruction of deep unfolding models, deep unfolding models have great potential to achieve SSR.
4

---

Published in Transactions on Machine Learning Research (04/2023)
Algorithm Scalable training of one epoch.
Input: training set I,, batch size B, loss function L, max CS ratio RM, sampling matrix A, initialization matrix B, reconstruction model Bura:: () or Buat(; A, O).
Output: trained parameters.
1: T +-0
repeat
Select 8= X2, XB) ET.
T' + US.
Generate (R); R2, .RR randomly, where R; E RM].
6: Generate [M1, M2. MB). where M,(1: [R .:) = and (. + +1: [RMN = 0.
7: Generate Ag = (Asi. As2, ASB), , An = {AR.. An2, ... , and Bp = (BIi, Buz, ,BRB)
where As; = M, C A, Ari = M, @ and BR: = M B.
for 1 B do
Y; Asi vec
10:
X vec (BRiY:)
11:
X; = Bura (XP: +) or X, (un, ARt. ))
Compute loss L using (X1,X2,- X2. {8} and S.
12:
Update A, B and + 14: until T \ T = 0
13:
15: return A, B, O
2.4 Scalable Training
As shown in 12 and 7 and are important in effective SSR. How obtain an appropriate combination of A, B and the roconstruction model is the main issuc. To this end, develop novel training strategy dubbed scalable training train A, B with parameters of the reconstruction model jointly.
In scalable training, it is assumed that all parameters are trained using algorithms like Adam (Kingına & Ba 2015). If the batch size for training is B and the loss function is L, the training process of A, B and @ of one epoch can be expressed as Algorithm| [ And Fig. 2 illlustrates the proparinnoon of the scalable training. The gradients of A and B can be computed as follows:
B V Σ
M
M,
V.MALL
(8)
1 B
V =
M, T
M. l...
(9)
B
It can be noticed that the closer to the top of A or the left of B, the more gradient information for updating is obtained, which makes using Mş and Mn. for effective SSR possible. It emphasized that the form of L is limited by SDCS, but related to the combined reconstruction model.
Furthermore, to validate the trained model, CS ratio validation group (RVG) is applied. Each RVG contains G validation CS ratios as {R1, R2,... Ro). At the end of each epoch, for each ratio R; the average PSNR on the validation set can be obtained. And the model with the best average PSNR on RVG is regarded as the model for test.
We emphasize that SDCS has no restriction on the structure of deep learning models, which means it can be combined with any and-to-end-trained model for SSR. However, the final performance determined by the structure of the reconstruction model.
5

---

Published in Transactions on Machine Learning Research (04/2023)
3
Related Works
In this section, we first introduce some deep-learning-basee methods for image CS, then some SSR methods are compared with SDCS.
3.1 Deep Learning Models for Image CS
For traditional deep learning models, Mousavi et al. (Mousavi et al. 2015 first designed fully-comnected- layer-based stacked denoising autoencoder (SDA) for visual image CS. Lohit et al. (Lohit et al. 2018a first proposed a six-layers CNN-based model named ReconNet to reconstruct image blocks from measurements. Shi ct al. (Shi ct al.) 2019a proposed a deeper CNN model named CSNet which has trainable deblocking operations and integrated residual connection (He et al. 2016) for better performance. Furthermore, there are some other models (Du et al., 2019 Yao et al. [019 Bora et al., 2017 Sun et al. (020) for visual image CS, and all these models have thing in common the models for reconstruction are trained end-to-end.
Deep unfolding models are first developed for the sparse coding problem (Gregor & LeCun 2010 Chen
et al. 2018 Borgerding et al., 2017). And inspired by these models, Zhang et al. (Zhang & Ghanem 2018)
developed deep unfolding model named ISTA-Net for image CS problem by unfolding iterative shrinkage- thresholding algorithm (ISTA) and learning sparse transformation functions. Metzler et al. (Metzler et al. 2017 and Zhang et al. (Zhang et al. 2021 established deep unfolding models named LDAMP and AMP- Net respectively based on approximate message passing (AMP) algorithm, where LDAMP samples and reconstructs the entire image, and AMP-Net mcasures and recovers image block-by-blod with general trainable deblocking modules. Dong al. (Dong et al., 2018) designed model named DPDNN inspired by the half-quadratic splitting (HQS) algorithm for image inverse problems which can be applied to visual image CS. These deep unfolding models apply the sampling matrix for reconstruction and they can also be trained end-to-end.
Some of the above methods discuss the sampling matrix training strategies, including in traditional deep learning models (oousav et al. 2015 Shi et al. 2019aa or in the deep unfolding model (Zhang et al.) 2021], and they all train their initialization matrices. Although the trained sampling matrices can improve the reconstruction performance, they are designed for the single CS ratio and the performance would decrease scriously when the CS ratio changes for SSR. However, using SDCS, the model and the trained sampling matrix can perform well in all CS ratios in a certain range.
3.2
SSR Methods
As far as we know, there exist several SSR methods {Sh et al. 2019b Lohit et al. 2018b Zhang et al. 2020). We introduce and compare them with SDCS in the following paragraphs.
Shi al. (Shi et al. 2019b) proposed model dubbed SCSNet. SCSNet trains the sampling matrix with the reconstruction model which is composed of seven independent sub-models with the same structure. Each sub-model adapts with sub-range of CS ratios to make sure that the whole model can achieve SSR at CS ratios from 1% to 50%. And greedy algorithm is applied to rearrange the rows of the sampling matrix for better reconstruction. However, SCSNet has two weaknesses: 1) The number of parameters is very large due to the existence of multiple sub-models. 2) Based on SCSNet, the existing deep learning models have to change their structure to achieve scalable reconstruction which would bring more burden to the hardware. However, SDCS nceds only one model to achieve SSR and it can be applied all aod-to-end-traindd models without changing their structures.
Zhang et al. (Zhang et al. 2020 propose framework named CRA which applies two reconstruction models, of which the first one is for initializing and completing the measurement, and the second one is for further reconstruction. Compared with SDCS, CRA do not train the sampling mairix, and two reconstruction models would introduce more parameters. Furthermore, we emphasize that CRA is essentially a pluggable method, which can be combined with other SSR methods by applying non-linear model for initialization and measurement completion. Therefore, the experimental comparison between SDCS and CRA not the focus of our paper.

---

Published in Transactions on Machine Learning Research (04/2023)
Lohit et al. (Lohit et al, 2018b) designed general framework like SDCS named Rate-Adaptive CS (RACS) which does not need to change the structure of the model, and it has three training stages. In stage 1, the model is trained with the sampling matrix at a single CS ratio of RM- And all parameters of the model are frozen after stage 1. In stage 2, The first N rows of the sampling matrix are optimized, where Rx < Rx- In stage 3, the following rows of the sampling malrix are trained one by one. It can be noticed that RACS has an obvious weakness: the model is learned for specific sampling matrix with CS ratio RM in stage 1, which means the performance of the model at lower CS ratios can be further improved. Different from RACS, with SDCS, the learned model adapts to sampling matrix that can change its CS ratios from 1% to RM using a sampling matrix mask. Our strategy brings the model the potential that performs better for SSR.
4 Experimental Results
4.1 Experimental settings
In this paper, the model combined with SDCS is named as model-SDCS. To evaluate the performance of SDCS, six models are combined with SDCS, namely SDA (Mousavi et al. 2015). RoconNet (Loiit et al. 2018a), CSNet* + (Shi et al. (001a)), ISTA-Net+ (Zhang & Ghanem 2018). DPDNN (Dong et al. 2018) and AMP-Net (Zhang et al. (001), which sample and reconstruct images block- block-by-bil by-block with the block size of 33 x 33 that makes N- 1089. SDA, ReconNet, CSNet are traditional doep learning models. ISTA-Net*, DPDNN and AMP-Net are deep unfolding models with 9, 6 and 6 reconstruction modules respectively. In this paper, the activation functions of SDA are changed to the Rectified Linear Unit (ReLU) (Lohit et al. 2018a) for better performance. It is worth noting that in CSNet* and AMP-Net, trainable deblocking operations are applied. SDA, ReconNet, ISTA-Net* and DPDNN do not train the sampling matrix in their original matrix, and CSNet and AMP-Net have the trainable sampling matrix. Furthermore, the above six models have the same initialization matrix in equation 3 of this paper. Furthermore, since SCSNet (Shi let al. 2019b and RACS (Lohit al. 2018b) can achieve SSR like model-SDGS, they are compared with SDCS to show the effectiveness of our framework.
All of our experiments are performed on two datasets: BSDS500 (Arbelacz et al. 2010) and Sct11 Lohit et al. 2018a). BSDS500 contains 500 colorful visual images and is composed of a training set (200 images), a validation set (100 images) and a test set (200 images). Set1l (Lohit et al. 2018a) contains 11 grey-scale images. In this paper, BSDS500 is used for training, validation and testing. And Set is used for testing, We generate two training sets for models with and without trainable deblocking operations. (a) Training set contains 89600 sub-images sized of 99 x 99 which are randomly extracted from the luminance components of images in the training set of BSDS500 (Shi et al. [019aa.. (b) Training set 2 contains 195200 sub-images sized of 33 x 33 which are randomly extracted from the luminance components of images in the training set of BSDS500 (Zhang& & Ghanem 2018). In this paper, CSNett, AMP-Net, CSNeL+-SDCS AMP-Nel-SDCS and SCSNet are trained on training set 1 due to the existence of trainable deblocking operations. SDA, NeconNet, 1ST,-Net*, DPDNN, SDA-SDCS, ReconNet-SDCC.
-STA-N and DPDNN-SDCS are trained on training set 2. The way to combine these models with SDCS is described in Algorithm I And they are trained the conditions in their original papers. Moreover, we use the validation set of BSDS500 for model choosing and the test set of BSDS550 for testing. In this paper, all sampling matrices are initialized randomly in Gaussian distribution. RM is 50% and RVG is (1%, 4%, 10%, 25%, 30%, 40%, 50%). All experiments are performed on computer with an AMD Ryzen7 2700X CPU and an RIX2080Ti GPU.
4.2 Comparison with original deep learning methods
In this subsection, we compare SDA, ReconNet, CSNet + ISTA-NeL+, DPDNN and AMP-Net with SDA- SDCS, ReconNet-SDDCS, CSNet -SDCS, ISTA-Net DAAA-NNe-SSDCS... -SDCS, DPDNN-SDCS and NeM--Net-SDCSAAMEE
SDCS* represents the experimental results that sampling matrix A and initialization matrix B are not involved in training. Table I and Table show the average PSNR and SSIM of 12 models tested on Setll and the testing set of BSDS500 at different. CS ratios respectively. We emphasize that there are seven
7

---

Published in Transactions on Machine Learning Research (04/2023)
different models for seven different test CS ratios for the method without SDCS, and single model is tested at different CS ratios for model-SDCS.
Table 1: The results of twelve models tested on Set at different CS ratios, where the best is marked in bold.
50%
40%
30%
25%
10%
4%
1%
Method
PSNR. (SB)
SDA
46.48/0.8007
25.14/0.7357
24.77/0.7191
24.77/0.7234
23.66/0.6794
21.05/0.5720
17.69/0.4376
SDA-SDCS
30.63/0.9000 (0.9009
30.63/0
29.43/0..0 /0.893
60..6/0..866
25.58/0. 76.7660
22.77 22.77/0.6455 /0.6458
19.87/0.4829 /0.4899
19.87
ReconNet
$2.12/0.9137
30.39/00 89228
28.72/0.8517
28.04/ /0.8303
/0.00400.8003
24.07/0. 60.6588
21.00/0.5817
17.54/0.4426
RecooNet-SDCS
34.29/0.9532
33.81/0.9244
32.42/ 82.42/0..311
31.42/ 70.977
26.90/0. 00.825
23.57 /0.6931
20.02/ 00.0220.5071
CSNett
38.19/0.97.3
36.15 36.15/0.9000 00.9655
33.90/0.94449 /0.9449
33.90
32.76 00.9322
27. 27.76/0.8511 00.513
24.24/0. 22.2122
20.09/0.5338
CSNert-SDCS
36.65/0.964
35,48/0 /0/8//00000 9568
33.58/0.9911
32. 32.44/0.9299 4,95
27.85/1 /0.8-493
23.92/0.7333
00.32/0.5380
ISTA-Net+
38.08/0..80 00.960
35.93/0 00.9537
33.66/0.9338
02.22/009910
25,93/0. 20.00/077788 78410
21.1//0.5917
17. 17.48/0.4100
ISTA-Net -SSCSS -DCCS
36.51 00.9693
34.922 99.77798777
32.855/0.94400
31 65.6550000 0.9256
26.99 26.99/0.8333 /0.8334
23.57 /0.7073
20.13/ 13/0.5144
DPDNN
05.85/0.9532
34.30/0.33 9411
32.06/ 00.91449155
30 63/.8344
24.53/0.770 7392
21.11/0. 6029
17. 19/,4/0
SDCS
00.50/00.9775
37.61/0 0...110..960 /0.966
35.38/ /0.9543
34.12/00.453
29.07 /0.8708
25.08 (0.7622
20.55, /0.5423
Net.Net
40.27 00.9844
38. 33..23/0.9713
/55_90/0999774
34.59/ 01,477
29.45 29.45/0.8788
25.16/0.7 76..660
20.57 /0.5699
AMP-Net-SDCSS
34.57/0 /0.947
32.89/1
022.8//0.9224 /0.9449
30.12 22228222
29:32/0086688
24.99/0.775 7201
21.21 /0.5649
18.97 45.4561
AMP-Ne-SDDS
39.67/0.9781
37.96/0.9700
35.89/0.9577
34.67 90.977
29.59/U /0..9/
25.43 43/00777770
20.47 /0.5629
Table 2: The results of twelve models tested on the test set of BSDS500 at different CS ratios, where the best is marked in bold.
50%
40%
30%
25%
10%
4%
1%
Method
PSNR (dIB)/SSIM
SDA
26.16/0.8048
21.97/0. 7392
24.58/0.7127
21.58/0.7107
23.77/0.6-199
21.75/0.5531
19.05/0.4522
ADCSS
30.17/0.9026
19..9/0..997
28.77 28.77/0.8704 /0.8704
18.13/0.8510
25.43//07333 7338
23.38 /0.6145
21.08 4..00.4865
NetonNet.
50.85/00.8944
29.47 /0. 8647
27.95, 00.8190
27.20/0.79114
23.98 /0.6472
21.69 69/0.5557
18.96/0 40.41
ReeonNetSSDCS
33.27 44.9448
32.52/ 0.9355
31.04/ 31.04/0.9107
30.13/ 0.8921
26.46 70.77733
23.99/ 23.99/0.6502 /0.6502
21. 21.20/ 21.20/0.5063
CSNet
35.89/0. 00.9677
33.96 0.9513
31.94/ 00.2511
30.91 00.9067
27.01 /0.77..95
24.41/ 24.41/0.6741 /0.6747
21. 42.42// 42..5661
CSNec* SDCS
34.91/01 /0.9588
33.59 /0. 9442
31.80/0.922
30.82 20.82//1 82/0.9043
26.97 /0. 7906
24.21 00.221/...00 /0.6692
21. .48 /0.5288
ISTA-Nor Nett
34.92/0.9511
32.87 /0. 9264
30.7T/0..0 8098001
29.64 80.8638
25. /0. 7124
21.82/0.550
18.92/0 18.99/00.4299
ISTA-Nert-SDOS
31.85/0.9622 0.9622
33.26 0...0....
31.38/ 00.9199
30.36 0.9003
26.56 /0. 7811
24.00/ 05.6555
21. 24/ 00.5096
DPDNN
33.56/0.9373
32.05/00 9961
29.98/0.8777 /0.8759
28.87 85.87/8 00.8491
24.37 /0. 6863
11.80/0..710
18.97 45.4
DPDNN-SDCS
/0.84/
34.91 0.9560
32.85/ 00.9323
31.74/0.99 000
27.58 /0.8069
24.78/ /0.6858
21.72 0.,3319
AMP-Net
37.48/ 37.48/0.9744
06.33400.9390
33.17 0,9358
92.01/0.9188
27.82/0.35 81333
24.95/0
1055/06.9911 69-19
21.90/0.3500
Set-Net-SDCS
37.04/0.99720
18/18/0995880
38.14/009955
32.04/0.9187
27.84 27.84/0.8136
25.03 /0.6967
21 27.87/11 /0/55499
From Table 1 and Table 2 it can be found that compared with models without trained sampling matrices, although model-SDCS has only one model for reconstruction, obtains better performance in terms of PSNR and SSIM at most test CS ratios. And compared with models that also apply trained sampling matrices (CSNet+ and AMP-Net), model-SDCS can still obtain competitive performance at all test. CS ratios with only a single model. Such a result implies the great potential of deep learning techniques and the sampling matrix training strategy. Therefore, we conclude that model-SDCS can effectively achieve SSR without changing the structure of the model.
Furthermore, can be noticed from Table and Table 2 that the above models combined with SDCS can have good performance of SSR, which verifies the universality of SDCS. It is worth emphasizing that the purpose of the universality of SDCS is not to combine SDCS with all existing models, but to bring reconstruction models the effective SSR performance. This means that researchers can design reconstruction models at a single CS ratio. To achieve SSR, they only need to combine these models with SDCS.
In addition, Table I and Table verify the benefits of the sampling matrix after scalable training, which can be summarized into two points: 1) The trained sampling matrix can improve the performance of the reconstruction model, comparing models with SDCS and without SDCS. 2) The trained sampling matrix can adapt the reconstructed model to different. CS ratios.
8

---

Published in Transactions on Machine Learning Research (04/2023)
4.3
Comparison with SSR methods
In this subsection, compare SDCS with two SSR methods: SCSNet (Shi et al. 2019b and RACS (Lohit
et al. 2018b).
Table 3: Parameter number the reconstruction model of seven models.
Parameter Number
SDA-SDOS
RecomNet-SDDSS
CSNet -SCCS -DDCS ISTA-Net'-SD
DPDNN SDCS AMP-Net-SDCS
SCSNet 1110823
6534
22914
370560
336978
1363712
229254
First, SCSNet compared with SDA-SDCS, ReconNet-SDCS, CSNet -DDCS ISTA-Net*-SDCS DPDNN- SDCS and AMP-Net-SDCS. Table 3 shows the parameter number of the seven modcls. Fig. g plots the average PSNR and SSIM of the seven models tested the test set of BSDS500 at CS ratios from 1% to 50%. It can be noticed that except for DPDNN-SDCS, other models have fewer parameters than SCSNet and achieve SSR. And DPDNN-SDCS and AMP-Net-SDCS even outperform SCSNet, which shows the great potential of SDCS. Furthermore, deep unfolding models have better SSR performance than traditional deep learning models. For examples, AMP-Net-SDCS and DPDNN-SDCS outperform SDA-SDCS, ReconNet- SDCS and CSNet+-SDOS, and SAA-Nett-SDDCE outperform SDA-SDCS and ReconNet-SDDCS, we conclude that deep unfolding models are more suitable for SSR to certain degree due to the important role of the sampling matrix in the image reconstruction process.
35
PSNR (dB)
30
AMP-Net-IDD'S DPDNN-SDCS
SSIM
AMP-Net-SDCS
DPDNN-SDD DONNNSSDCS
CSNet -SDCS
CSNet -DCS
"DNCS
25
ISTA-Net -SCCS RecOnNet-SDCS
0.6
ISTA-Net SDCS
ReconNet-SDCC.
SDA-SDCS
SDA-SDCS
SCSNet
0.5
SCSNet
20
10
20
30
40
50
40
50
CS ratio (%)
CS ratio (%)
Figure Comparison between SCSNet and six models with SDCS at different CS ratios.
40
35
0.8
PSNR (dB)
30
AMP-Net-SDCS
SMD-Net-DDCS
AMP-Nat-RACS-19,
AMP-NatRRACS-15
25
IMP--
SSIM
AMP-Net-ACCS-10%
20
AMP 25P--
AMP-NetRACS-25%,
CSNet' -DDCS
CSNet -DDCS
15
CSNet* RRCS-1%
CSNet*-RACS..
CSNNet' -RACS-10%
0.2
CSNet' CSNeI'RRACS RRCC--00%
10
CSNet* CSNer* RRACS-25%
CSNet* RRACS255%
5
10
20
30
40
50
10
20
30
40
50
CS ratio (%)
CS ratio (%)
Figure 4: Comparison between SDCS and RACS at different CS ratios.
Second, SDCS compared with RACS. Since with SDCS, AMP-Net outperforms other deep unfolding models and C outperforms other traditional deep learning models, AMP-Net. and CSNet as examples

---

Published in Transactions on Machine Learning Research (04/2023)
to compare SDCS and RACS. In this subsection, the values of RK of RACS mentioned in 3.2 1%, 10% USNet=-SDCC AMP-Net-RACS- Rk and 25%. Fig. 4 plots average PSNR and SSTM of AMP-Net-SDCS,, 1% to 50%, where model-RACS- and CSNet - -RACt--RACS-RER Rk on the test set of BSDS500 at CS ratios from denotes the model combined with RACS with the hyperparameter Rg. It can be noticed that when the CS ralio lower than RK, model-RACS- has bad performance. For AMP-Net, AMP-Net-SDCS outperforms all compared AMP-Net-RACS--Re Rys when the CS ratio lower than 30%. And for CSNet CSNet*-SDCS all CS ratios. Such a result implies that SDCS can generate more appropriate combination of sampling matrix and model than RACS.
has better performance than all compared SSNNet-RACSS
Fig. shows the Parrots images in Sctl1 reconstructed by different SSR models at different CS ratios. Fig. is quite revealing in several ways. 1) AMP-Nei-SDCS generates better results than SCSNet while main- taining fewer parameters which shows the great potential SDCS. 2) Model-RACS-
can not inherit the characteristics of original models well. For example, AMP-Net and CSNet both have trainable deblocking operations, but HAI--Net-RACS-E and -RANSt-RACS- 11% generates images with obvious blocking arti- lacts CS ratios of 1%, 4% and 10%. However, AMP-Nel-SDDS and CSNet CSNett-SDCC -DDCS generate smooth images without blocking artifacts. Therefore, conclude that models with SDCS can get good SSR performance. In particular, they can inherit the characteristies of original models.
Ground Truth
10%
30%
40%
SCSNet
PSNR (dBYSSIM 21.26/0.7108
23.0990.82887
27.91/0.8896
32.00/0.9359
33.71/099557
35.89/0.9600
2.20900.770
AMP-Net- RACS-1%
PSNR (dB)SSIM 21.55/0.6899 6899
24.4 24.45/0.7994
27.76/0.8758
33.22/0.9449
34. 8070.9575
37.60/0.9733
39.88/0 9819
CSNet' RACS-1%
PSNR (dB)SSIM
21.11.0.6689
23.41N0.7564
25.93/0.2286
30.008L8988
31.00/0.91.3
32.91/0.9336
35.29/009611
AMP-Net-
SDCS
PSNR (dBYSSIM
21.61/0.7161
25.19/0.8316
28.91/0.9033
34.45/0.9546
33.75/0.9633
37.83/0.9735
39.60/0.990
CSNet* SDCS
PSNR 21.52/0.7038
23.85/0.5031
27.53/0.8796
32.51/0.9416
35.93/0.9627
Figure The Parrots images Sct11 Reconstructed by different SSR methods different CS ratios.
To further prove the offectiveness of SDCS, we compare AMP-Net-SDCS and CSNet+-SDCS with AMP-Net and CSNet which train their sampling matrices for one single CS ratio and apply the greedy algorithm in SCSNet (Shi et al. 2019b). In this subsection, the sampling matrices of AMP-Net and CSNett are trained for the CS ratio of 50%. And their rows are rearranged using the greedy algorithm in SCSNet (Shi et al. 2019bb for better SSR. Fig. 6 plots the average PSNR and SSIM of four models tested on the test set of BSDS500 at CS ratios from 1% to 50%. It can be noticed that at the CS ratio of 50, the specially trained
10

---

Published in Transactions on Machine Learning Research (04/2023)
Table 4: The results of different models on the test set of BSDS500
with different SNRs.
30%
10%
4%
SNR
Method
PSNN {dB)/SSM
CSNet
31.89/0.9212
27.00/0.7942
24.61/0.6767
CSNet+-SDCS
31.76/0.9219
27.03/0.7952
24.41/0.6700
40dB
AMP-Net
32.93/0,9314
27.73/0.8058
24.94/0.6998
AMP-Net-SDCS
32.83/0,9300
27.71 37.71/0.8083
24.95/0.6916
CSNet
31.58/0.912
26.93/0.7863
24. 23.003/66711
*SNett-SDCS
31.62/0. 91.67/0.9178
26.95/0.7900
24.30/0.6720
30dB
AMP-Net
$1.40/0.9002
26.97 /0.7611
24.46/0. 16.46/0.666 6689
AMP-Net-SDOS
31.61/0.903
27.12/0.7818
24.57/0.670
CSNett
30.87 30.87/0.8963
/0.8963
26.46/0.7660
65.05000
+SNet
00.92/0.9023
26.66/0.7775
24.03/0. 6624
25dB
AMP-Net
29.83/0,
26.07/0.7328
23.82/0. 6350
AMP-Net-SDOS
29.90/0.8500
26.15/0.7349
23.78/0. 6254
CSNet
26.84/0.7611
23.85/0.63377
21.69/ 21.69/0.6228 60.6000
CSNet+-SDCC
26.49/0. 70.49/0.799
24.15/0.6434
22.04/0.5417
15dB
AMP-Net
26.43/0..807 00.8077
23.21 13.66/0
21.66/0.5324
AMP- -CCSSS
25.20 04.29/0.6036
22.64/0.5446
20.52/0.4433
models can obtain better results than the models with SDCS, but such models have a bad performance at other CS ratios. However, models combined with SDCS perform well all CS ratios. Therefore, we conclude that model-SDCS outperforms the model with trained sampling matrix for single CS ratio, and SDCS provides more obvious improvement than the greedy algorithm of SCSNet under the condition of only one reconstruction model.
35
0.8
30
PSNR (dB)
25
SSTM
0.6
0.4
AMP-Net-SDCS
AMP-Nct-SDCS
15
CSNet -SDCS
-SDCS
CSNet -DDCS
AMP-Net-G
AMP-Net-G
10
CSNet -G
CSNet -G
10
20
30
40
50
10
20
30
40
50
CS ratio (%)
CS ratio (%)
Figure 6: Comparison between two models with and without SDCS at different CS ratios. model-G is the model combined with the greedy algorithm in (Shi al.] 2019b).
4.4 Simulating the actual imaging conditions
In some practical conditions, noises may be introduced to the measurement, y. this end, we validate the anti-noise performance of SDCS this subsoction simulate the actual CS imaging conditions. detail, additive Gaussian white noises [Lepskii 1991) are added to y of all datasets train and test models in the subsection. And the signal-to-noist ratios (SNRs) are 40dB, 30dB, 25dB and 15dB, All results are obtained by testing 5 times on the test set and averaging.
Since with SDCS, AMP-Net outperforms the other deep unfolding models, and CSNet* + oulperforms other traditional deep learning models, we use AMP-Net and CSNet as examples to validate the anti-noist noise performance of SDCS. Table A shows the average PSNR and SSIM by different models on the test set of BSDS500 with different SNRs at different CS ratios of 30, 10%, and 4%. It can be noticed that in most cases, the original model and the model with SDCS have similar performance, which demonstrates that
11

---

Published in Transactions on Machine Learning Research (04/2023)
35
30
CSNet SDCS (40dB)
CSNet'-SDCS (40dB)
PSNR (dB)
25
CSNet CSDC (30dB)
CSNe" (30dB)
CSNet -DDCS (25dB)
SSIM
CSNet "SNet"-SDCS (25dB)
20
CSNet CSNet-SDDC -SDCS (15dB)
"SNCS' (15dB3)
15
CSNet -RACS-1% (40dB)
0.4
CSNet CSNet'-RACS -RACS (40dB)
-RACS-1% (30dB)
CSNet -RACS (30dB)
10
CSNet -RACS-1% (25dB)
0.2
CSNet CSNet"-RACS (25dB)
CSNet RACS-1% (15dB)
CSNet MANNt'-RASS (15dB)
5
10
20
30
40
50
10
20
30
40
50
CS ratio (%)
CS ratio (%)
35
PSNR (dB)
AMP-Net-SDCS (4MIB
AMP-Nat-DDCS (40dB)
AMP-Net-DDCS (30dB)
SSIM
AMP-Net-SDCS (30dB)
AMP-Net-SDCS (25dB)
AMP-Net-DDCS (25dB)
AMP-Ne+-SDCS (15dB)
AMP-Net-SDCS (15dB)
AMP-Net-RACS (40dB)
AMP-Net-ACCS11% (40dB)
20
AMP-Net-RACS-I (30dB)
AMP-Nat-RACS-11% (30dB)
AMP-Nc:-ACS--P (25dB)
AMP-Nat-RACS-1% (25dB)
AMP-Ne: 1% (15dB)
AMP-Not-RACS-1% (15dB)
15
10
20
30
40
50
10
20
30
40
50
CS ratio (%)
CS ratio (%)
Figure 7: Comparison between SDCS and RACS with different SNRs.
SDCS will not weaken the anti-noisc ability of the original model to certain extent. The only exception is that AMP-Net outperforms AMP-Net-SDCS with a low SNR of 15dB, which means that the anti-noise performance model may decline with a low SNR when it combined with SDCS. Furthermore, combined with SDCS, a single model can be used to achieve sampling and reconstruction at multiple CS ratios, which further illustrates the advantage of SDCS.
Since ModeI-RACS-1% has the most similar performance to model-SDCS in subsection 4.3 compare model-RACS-1% with model-SDCS to further validate the anti-noise performance of SDCS. Fig. 7 shows the average PSNR and SSIM of CSNet -SDCS, CSNet "SNeet-RACS-I -RACS-1% AMP-Net-SDCSS, with different SNRs at CS ratios from 1% to 50%. In Fig. 4 when there is no noise, the maximum difference the PSNR and the SSIM of mdoel-SDCS and model-RACS-1% are 1dB and 0.04, respectively. It can be noticed from Fig. 7 that as the SNR decreases, the performance of each model decreases, and particular, the
CeP--Ne-RRAC-LL
the performance difference between mdoel-SDCS and model-RACS-11% also increases. performance of CSNet* -SDCS with an SNR of 15dB is even better than model-RACS-1%% with an SNR of 30dB, and for CSNett-RACS-19 -RCS-1% the PSNR is even lower than 10dB and the SSIM is lower than 0.1, which may be due to that the traditional deep model combined with RACS cannot be well adapted to the condition of low SNR. Therefore, it can be concluded that SDCS has better anti-noise performance than RACS and is more suitable for imaging under actual conditions.
5 Conclusion
In this paper, for the visual image CS problem, we propose a general framework named SDCS to achieve SSR of doep-learning-base models. Besides of the initialization matrix and two sampling matrix masks, SDCS does change the structure of the model. The proposed scalable training can generate an appropriate combination of the sampling matrix and the reconstruction model for efficient SSR. Experimental results show
12

---

Published in Transactions on Machine Learning Research (04/2023)
that SDCS outperforms other SSR methods. Specifically, models with SDCS can inherit the characteristics of the original models, e.g. the deblocking ability. In addition, it is shown that SDCS can work well with additive noises.
However, SDCS has one shortcoming: R3 being sampled uniformly during training makes the different training times of rows of the sampling matrix, which may affect the performance of SDCS. In the future, we will try to find a better way to generate R2 and try some bigger datasets like ImageNet (He et al. 2016) to improve the power of SDCS. Furthermore, will extend SDCS to high-dimensional CS problems which demand SSR. For example, snapshot compressive imaging (SCI) (Liu et al. 2018 Ma et al. 2019 is promising to use a single model to reconstruct hyperspectral images in different frequency bands, and some applications like transient imaging (Sun et al., 2018) and magnetic resonance imaging (MRI) (Liu et al. 2017 2020) can obtain images at different ratios using one model with a binary sampling matrix. As different CS applications have different sampling and reconstruction strategies, which makes the current SDCS has to be updated to adapt to them.
6 Acknowledgements
This research is supported by National Natural Science Foundation of China (NSFC, No. 62171088, U19A2052, 62020106011), dleiic-Engineering Cooperation Funds from University of Electronic Science and Technology of China (No. ZYGX2021YGL1215, ZYGX2022CRRH065)
References
Pablo Arbelaez, Michael Maire, Charless Fowlkes, and Jitendra Malik. Contour detection and hierarchical image sogmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(5):898-99) 2010.
Amir Beck and Marc Teboulle. A fast iterative shrisoigg algorithm for linear inverse problems. SIAM Journal on Imaging Sciences, 2(1):183-200) 2009.
Ashish Bora, Ajil Jalal, Eric Price, and Alexandros G. Dimakis. Compressed sensing using generative models. Tn Doina Precup and Yee Whye Teh (eds.), Proceedings of the 94th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pp. 537 546, International Convention Centre, Sydney, Australia, 06 11 Aug 2017. PMLR.
Mark Borgerding, Philip Schniter, and Sundeep Rangan. AMP-inspired deep networks for sparse linear inverse problems. IEEE Transactions on Signal Processing, (5(16))4493-4438 2017.
Xiaohan Chen, Jialin Liu, Zhangyang Wang, and Wotao Yin. Thcoretical lincar convergence of unfolded ISTA and its practical weights and thresholds. In Advances in Newral Information Processing Syslems, pp. 0061-9071, 2018.
Khanh Quoe Dinh and Byeungwoo Jeon. Iterative weighted recovery for block- compressive sensing of image/ video at a low subrate. TEEE Transactions on Circuits and Systems for Video Technology, 27(11) 2294 2308, 2017. doi: D0.119//TTSV
soage/videoo
Weisheng Dong, Guangming Shi, Xin Li, Yi Ma, and Feng Huang. Compressive sensing via nonlocal low-rank regularization. IEEE Transactions on Image Processing, 23(8):3618-300) 2014.
Weisheng Dong, Peiyao Wang, Wotao Yin, Guangming Shi, Fangfang Wu, and Xiaotong Lu. Denoising prior driven deep neural network for image restoration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(10):2305 2218, 2018.
David L Donoho, Arian Maleki, and Andrea Montanari. Message-passing algorithms for compressed sensing. Proceedings of the National Academy of Sciences, 106(45):18914-1890) 2009.
Jiang Du, Xuemei Xie, Chenye Wang, Guangming Shi, Xun Xu, and Yuxiang Wang. Fully convolutional measurement network for compressive sensing image reconstruction. Neurocomputing, 328:105-11 112, 2019.
13

---

Published in Transactions on Machine Learning Research (04/2023)
Marco F Duarte, Mark A Davenport, Dharmpal Takhar, Jason N Laska, Ting Sun, Kevin F Kelly, and Richard G Baraniuk. Single-pixel imaging via compressive sampling. IEEE Signal Processing Magazine, 25(2):83 91, 2008.
Michael Elad. Sparse and redundant representations: from theory to applications signal and image pro- cessing. Springer Science Business Media, 2010.
Karol Gregor and Yann LeCun. Learning fast approximations of sparse coding. In The 27th International Conference on International Conference on Machine Learning, pp. 399-99999990 Omnipress, 2010.
Jiajia Guo, Chao-Kai Wen, Shi Jin, and Geoffrey Ye Li. Convolutional neural network-based multiple-rate compressive sensing for massive mimo esi feedback: Design, simulation, and analysis. IEFF Transactions on Wireless Communications,, 19(4):2827 2840, 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. The IEEE Conference on Computer Vision and Pattern Recogrition, pp. 770 778, 2016.
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, and Weizhu Chen. 2021. URL httph:
Lora: Low-rank adaptation of large language models. CoRR,
arg/abs/2106 09685
Yixing Huang, Tobias Würfl, Katharina Breininger, Ling Liu, Günter Lauritsch, and Andreas Maier. Some investigations on robustness of deep learning in limited angle tomography. In International Conference on Medical Image Compuling and Compuler-Assisted Intervention, pp. 145 153. Springer, 2018.
Diederik P Kingma and Jimmy Ba. ADAM: A method for stochastic optimization. International Conference on Learning Representations, 2015.
OV Lepskii. On a problem of adaptive estimation in gaussian white noise. Theory of Probability & Its Applications, 35(3):454-4666 1991.
Chengbo Li, Hong Jiang, Paul Wilford, Yin Zhang, and Mike Scheutzow. A new compressive video sensing framework for mobile broadcast. IEEE Transactions on Broadcasting, 59(1):197-205, 2013.
Yong Li, Wenrui Dai, Junni Zhou, Hongkai Xiong, and Yuan F. Zheng. Scalable structured compressive video sampling with hierarchical subspace learning. IEEE Transactions Circuits and Systems for Video Techmology, 30(10):35283355 3543,, 2020. doi: 10. 20000/CCSV..200.2992902
Yang Liu, Xin Yuan, Jinli Suo, David J Brady, and Qionghai Dai. Rank minimization for snapshot com- pressive imaging. IEEE Transactions on Pattern Analysis and Machine Intelligence, (111)):29003300 2018.
Yipeng Liu, Shan WU, Xiaolin Huang, Bing Chen, and Ce Zhu. Hybrid CS-DMRI: Periodic time-variant subsampling and innidirectional total variation based reconstruction. IEEF Transactions on Medical Imaging, 36(10):2148 2159, 2017.
Yipeng Liu, Zhen Long, Huyan Huang, and Ce Zhu. Low CP rank and Tucker rank tensor completion for estimating missing components in image data. IEEE Transactions Circuits and Systems for Video Techmology, 2019.
Yipeng Liu, Tengleng Liu, Jiani Liu, and Ce Zhu. Smooth robust lensor principal component analysis for compressod sensing of dynamic mri. Pattern Recognition, 102:107252, 2020.
Suhas Lohit, Kuldeep Kulkarni, Ronan Kerviche, Pavan Turaga, and Amit Ashok. Convolutional neural networks for noniterative reconstruction of compressively sensed images. IEEE Transactions on Compu- lational Imaging, 4(3):326 340, 2018a.
Suhas Lohit, Rajhans Singh, Kuldeep Kulkarni, and Pavan Turaga. Rate-adaptive neural networks for spatial multiplexers. arXiv preprint arxiv: 1009. 209.02850, 2018b.
14

---

Published in Transactions on Machine Learning Research (04/2023)
Jiawei Ma, Xiao- Yang Liu, Zheng Shou, and Xin Yuan. Deep tensor ADMM-Net for snapshot compressive imaging. In Proceedings of the IEFE International Conference on Computer Vision, pp. 2019.
Chris Metzler, Ali Mousavi, and Richard Baraniuk. Learned d-amp: Principled neural network basod compressive inage recovery. In Advances in Neural Information Processing Systems, pp. 1772 1783, 2017.
Ali Mousavi, Ankit B Patel, and Richard G Baraniuk. A deep learning approach to structured signal recovery. Tn The 53rd Annual Allerton Conference on Communication, Control, and Compuling, pp. 1336 1343. IEEE, 2015.
JH Rick Chang, Chun-Liang Li, Barnabas Poczos, BVK Vijaya Kumar, and Aswin C Samkaranarayanan. One network to solve them all soll-solving linear inverse problems using deep projection models. In Proceedings of the IEEE International Conference on Computer Vision, pp.
2017.
Wuzhen Shi, Feng Jiang, Shaohui Liu, and Debin Zhao, Image compressed sensing using convolutional neural network. IEEE Transactions on Image Processing, 29:375-388, 2019a.
Wuzhen Shi, Feng Jiang, Shaohui Liu, and Debin Zhao. Scalable convolutional neural network for image compressed sensing. In The IEEE Conference Computer Vision and Pattern Recognition, pp. 12290- 12299, 2019b.
Wuzhen Shi, Shaohui Liu, Feng Jiang, and Debin Zhao. Video compressed sensing using a convolutional neural network. IEEE Transactions on Circuits and Systems for Video Technology, pp. 1-1, 2020. doi: TC.1100/TCSVT..2202225
Yueming Su and Qiusheng Lian. ipiano-net: Nonconvex optimization inspired multi-scale reconstruction network for compressed sensing. Signal Processing: Image Communication, 89:115989, 2020.
Jian Sun, Huibin Li, Zongben Xu, et al. Deep ADMM-Net for compressive sensing MRI. In Advances in
Newral Information Processing Systems, pp. 10 10-18, 2016.
Qilin Sun, Xiong Dun, Yifan Peng, and Wolfgang Heidrich. Depth and transient imaging with compressive spad array cameras. In Proceedings of the IEFE Conference on Computer Vision and Pattern Recognition, pp. 273 282, 2018.
Yubao Sun, Jiwei Chen, Qingshan Liu, and Guangean Liu. Learning image compressed sensing with sub-pixel convolutional generative adversarial network. Pallern Recognilion, 98: 98:107051, 2020.
Zhengzhong Tu, Hossein Talebi, Han Zhang, Feng Yang, Peyman Milanfar, Alan Bovik, and Yinxiao Li. Maxvit: Multi-axis vision transformer. In Computer Vision-ECCV 2022: 17th European Conference, Tel Avic, Isruel, October 23 27, 2022, Proceedings, Part XXIV, pp. 459 479. Springer, 2022.
Yibo Xu, Weidi Liu, and Kevin F Kelly. Compressed domain image classification using a dynamic-rate neural network. IEEE Access, 3:217711-21772, 2020.
Chenggang Yan, Biao Gong, Yuxuan Wei, and Yue Gao. Deep multi-view enhancement hashing for image retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020a.
Chenggang Yan, Zhisheng Li, Yongbing Zhang, Yutao Liu, Xiangyang Ji, and Yongdong Zhang, Depth image denoising using nuclear norm and learning graph model. ACM Transactions on Mullimedia Compuling, Communications, and Applications (TOMM), 16(4):1-17, 2020b.
Chenggang Yan, Biyao Shao, Hao Zhao, Ruixin Ning, Yongdong Zhang, and Feng Xu. 3d room layout estimation from a single rgb image. IEEE Transactions on Multimedia, 22(11):3014 3024, 2020c.
Hantao Yao, Feng Dai, Shiliang Zhang, Yongdong Zhang, Qi Tian, and Changsheng Xu. Dr2-net: Deep residual reconstruction network for image compressive sensing. Newrocomputing, 359:483 -493, 2019.
15

---

Published in Transactions on Machine Learning Research (04/2023)
Wenbin Yin, Xiaopeng Fan, Yunhui Shi, Ruiqin Xiong, and Debin Zhao. Compressive sensing based soft video broadcast. using spatial and temporal sparsity. Mobile Networks and Applicalions, (6(6):0002-10 1012, 2016.
Jian Zhang and Bernard Ghanem. ISTA-Net Interpretable optimization-inspreed deep network for image compressive sensing. In The IEEE Conference Computer Vision and Pattern Recognition, pp. 1828- 1837, 2018.
Zhikang Zhang, Kai Xu, and Fengbo Ren. CRA: generic compression ratio adapter for end-to- data- driven image compressive sensing reconstruction frameworks. In ICASSP 2020-2020 IEEE International Conference Acoustics, Speech and Signal Processing (ICASSP), pp- 1439-1443. IEEE, 2020.
Zhonghao Zhang, Yipeng Liu, Jiani Liu, Fei Wen, and Ce Zhu. Amp-net: Denoising- deep unfolding for compressive image sensing. IEEE Transactions on Image Processing, 30:1487-1500, 2021. doi: 10. 1109/TTP.2020.33444
16


Reviews:
[{"review_id": "CGxmThjIg9", "reviewer": "Reviewer_417w", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n* The problem that the authors study is relevant.\n* The paper builds upon a large line of prior work.\n* The proposed solution is intuitive.\n* The results show that the method outperforms both single model and previously proposed SSR solutions.\n\n\nWeaknesses:\n* The paper ignores a body of work based on generative networks for solving inverse problems. A different approach to the supervised learning route is to use a generative model as a prior for the domain and then find a solution from the network that explains the measurements. This approach has found a lot of empirical success (especially recently with the rise of powerful generative networks) and doesn't have the problem of different ratios which is the topic of investigation of the current paper. This line of work should be at least cited (if not compared to). For example, the Bora et. al. (2017) paper, is incorrectly cited by the authors as a supervised learning approach for solving inverse problems. \n* The paper's presentation could be improved (see Requested Changes for a list of typos, suggested improvements).\n* Experiments are limited. The biggest training set contains only 200 images. Also, since the testing on Set11 isn't it considered out-of-distribution testing since the training images were colorful? What is the point of the experiments on this test set? Is this a claim of better out-of-distribution performance?\n* From the paper, I gathered that the forward matrix is free to be chosen at will (which seems to be one of the main innovations of the paper). It would be cool to see what the learned forward matrix is, in the sense of how the images look visually after the degradation (sampling) at different rations.\n* Also, in many problems, the forward matrix is fixed, i.e. in physical systems. It would be cool to evaluate performance on this setting, e.g. maybe in the setting where A is a random inpainting matrix (with different probabilities of masking based on the corresponding subsampling ratio).\n* One of the main arguments of the paper against having separate models based on the subsampling ratio is the storage of the trained models. But these models will not be doing something entirely different so it is possible that some method for efficient parameter tuning would be efficient. For example, the [LoRA](https://arxiv.org/abs/2106.09685) that was proposed for efficient tuning of LLMs, has been lately applied to Computer Vision and Stable Diffusion. It would be nice to hear the author's perspective on this.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes a framework for solving (with a single model) Compressed Sensing problems in which different subsampling ratios might be encountered. The paper considers the supervised setting for solving inverse problems, i.e. a model is trained to reconstruct an image from a corrupted observation. The empirical finding of prior work is that the best performance is obtained when different models are trained for (significantly) different subsampling rations. The current paper considers this a prior work limitation mainly because of hardware (storage?) and training constraints. The paper proposes to solve this problem by using a trainable sampling (forward) matrix, A. During training, the matrix is masked at different ratios to reconstruct images, with solution initializations arriving from masking (at different ratios) of an initialization matrix B.\n\n", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Critical for acceptance:\n* I would recommend citing, discussion and potential comparison to the unsupervised approach for solving inverse problems, i.e. the approach of using pre-trained generative networks as priors.\n* Experimental section should be strengthened. I think it would be best to include some results on classic datasets, such as CIFAR-10 or ImageNet. I understand that the computational requirements might be an issue, but I think at least CIFAR-10 should be doable in the described hardware and it will be a convincing experiment that this framework scales.\n* I think it would be valuable to the paper to evaluate also performance when the forward (sampling) matrix is given. This will also help getting a better sense of how this method performs in standard benchmarks and compared to recent state-of-the-art approaches for solving specific inverse problems. For example, a valuable comparison would be with MaxVIT (MAXIM).\n\nNot so critical for acceptance:\n* Fix several typos and improve the delivery of the paper. Examples: \n     * \"bring the model the ability\" -> \"bring to the model the ability\".\n     * in the related work section, there are repetitions in the citations, such as Lohit et. al Lohit et. al (2018b)\n     * The first paragraph of 3.1. feels a bit disconnected with the paper since it just lists architectural innovations, which is orthogonal to the proposed framework. \n* I think some parts could be better explained for the unfamiliar reader. E.g. why is it better to have one sampling matrix (that gets masked every time?).  Also, it would help define what the block-based visual image CS problem is.\n* It would be nice to explain why the initial value for the problem to be solved is chosen to be linear matrix times the measurements. I could imagine \"rough\" non-linear reconstruction approximations that would later be improved.", "broader_impact_concerns": "I do not have any ethical concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "Hnt2tlzKh0", "reviewer": "Reviewer_s8cQ", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. SDCS seems to show results across multiple subsampling ratios that are superior to/as effective as the original techniques tuned to each subsampling ratio.\n2. SDCS is modular and can be combined with any network architecture.\n\nWeaknesses:\n1. While the experiments demonstrate the efficacy of SDCS, the authors do not indicate whether this improvement can be attributed to the fact that SDCS trained networks simply see a lot more data ($B$ times the data that a single network sees). Especially at lower subsampling ratios, the number of samples that are used to train the models and sensing matrices could be a lot more than just $B$ times the data seen by a single network since higher subsampling ratios also influence the first few rows/columns of the sensing/reconstruction matrices. A clarification of this point, or an experiment that allows baselines to see more data will be helpful in pointing out the efficacy of the proposed method.\n\n2. The authors do not perform any ablation studies that demonstrate the efficacy of different components. It would be interesting to know if the improvements are primarily due to learning the sampling/reconstruction matrices or due to the fact that the single network is trained on all the data. Implementing a version of SDCS that does not update the $\\mathbf{A,B}$ matrices and only updates the parameter matrices $\\Theta$ will answer this question. \n\n3. It is unclear whether this reconstruction technique can generalize to sampling matrices that are not learned by the model. There may be scenarios where the measurements are generated by a randomized sampling technique and one may want to reconstruct the image from these measurements without access to the exact measurement matrix. How does SDCS perform in this scenario? \n\nPresentation Improvements:\n1. The paper could be written in a more accessible manner so that it can reach a wider audience than just the research community in deep learning for compressive sensing. A short primer explaining the scaling problem with deep networks for compressive sensing would help motivate the problem for the reader who is unfamiliar with the problems in this sub-field. In fact moving a version of Table 3 to the first few pages of the paper could give us a sense of the contributions of the paper before we dive into the algorithm.\n\n2. Citations could use citep to put them in parantheses rather than just cite. There are a few places where the author names are repeated (Shi et al. Shi et al.; Lohit et al. Lohit et al.)\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a technique that adapts deep networks for compressive sensing to multiple subsampling ratios. The proposed method involves learning common sampling and reconstruction matrices and using the corresponding rows/columns of the matrices for multiple subsampling ratios. This approach is modular and can be combined with any deep network architecture that performs compressive sensing and reconstruction of images/image patches. The authors perform experiments using different deep network architectures to demonstrate the efficacy at individual subsampling ratios. The authors also compare their methods to other proposals for robust and scalable compressive sensing and show their techniques are more parameter/compute efficient as well as achieving superior PSNR.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Ablation experiment on learning $\\mathbf{A,B}$ vs just training at different subsampling ratios without learning $\\mathbf{A,B}$\n2. Normalizing the amount of training data seen by SDCS vs baselines. If this is already the case, it should be mentioned/emphasized in the paper.", "broader_impact_concerns": "None beyond the usual caveats and concerns that accompany models that are learned from data. The paper could perhaps emphasize that it does not address bias and fairness issues that could arise from using datasets that are not representative of certain populations.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "seI1uMRsFu", "reviewer": "Reviewer_S97u", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper is well written, easy to read, and most claims are supported well with empirical evidence. \n\nThe idea is simple, and straightforward, and I appreciate the authors providing extensive empirical evidence that the idea works across different models and dataset. However, I've a few questions/comments about the set-up:\n\n1. The idea of training A and B together with the network probably exists in the literature already. I might have missed this while the reading the paper, but can the authors provide more context around related work and confirm which elements (a) A matrix, (b) B matrix, (c) and the masking strategy is introduced in this paper vs what exists in the literature? \n2. The results in Table 1 and 2 are very impressive, but it seems peculiar that a model optimized for a single compression ratio does not even come close to the performance of a model trained for multiple compression ratios. Can authors provide some insight on why this maybe the case? Are all experimental parameters the same (was same amount of effort spent on hyperparameter tuning)?\n3. From algorithm 1, it seems like R_i s are sampled uniformly. Does this sampling strategy affect performance? For example, I might imagine that training with smaller R_i s in the initial epochs and expanding R_is as epoch increases will help get better performance for the top rows of A_i. \n4. In Section 4.4 is Gaussian a realistic noise model? If yes, can authors kindly provide references and place it in context? Further, I assume that the models are not trained in the presence of Gaussian noise - but only evaluated - can the authors confirm this/make this clear in the paper?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces a new training scheme to support compressed sensing with different compression ratios. The method learns a sampling matrix, a reconstruction initialization matrix, and the network parameters together. Authors propose a masking scheme to ensure that the same sampling matrix and reconstruction initiation matrix easily work across different compression ratios. Authors perform extensively experimentation with 6 different network architectures for reconstruction, and shows that when combined with the proposed method, these architectures outperform their counterpart trained with a vanilla scheme achieving state-of-the-art performance across multiple compression ratios. Furthermore, they show that training with the proposed scheme provides some robustness to addition to Gaussian noise. ", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "See weakness above. \n\nMinor nitpik: it might make sense to combine table 3 with tables 1 and 2 so that it is easier to read the results. ", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The main contribution of this paper is the proposal of a flexible deep compressive sensing approach that can accommodate multiple subsampling ratios. Extensive experiments support the performance of the proposed approach.  the paper received mixed recommendations, but the majority is leaning toward acceptance. Together with the fact that better performance is offered by the proposed approach in many scenarios and the claims are supported by accurate and clear evidence, I recommend acceptance.
Decision: Accept (poster)