S ELECTIVE S ENSING : A D ATA - DRIVEN N ONUNIFORM S UBSAMPLING A PPROACH FOR C OMPUTATION - FREE O N -S ENSOR D ATA D IMENSIONALITY R EDUCTION
Anonymous authors
Paper under double-blind review
A BSTRACT
Designing an on-sensor data dimensionality reduction scheme for efﬁcient signal sensing has always been a challenging task.
Compressive sensing is a state-of-the-art sensing technique used for on-sensor data dimensionality reduction.
However, the undesired computational complexity involved in the sensing stage of compressive sensing limits its practical application in resourceconstrained sensor devices or high-data-rate sensor devices dealing with highdimensional signals.
In this paper, we propose a selective sensing framework that adopts the novel concept of data-driven nonuniform subsampling to reduce the dimensionality of acquired signals while retaining the information of interest in a computation-free fashion.
Selective sensing adopts a co-optimization methodology to co-train a selective sensing operator with a subsequent information decoding neural network.
We take image as the sensing modality and reconstruction as the information decoding task to demonstrate the 1st proof-ofconcept of selective sensing.
The experiment results on CIFAR10, Set5 and Set datasets show that selective sensing can achieve an average reconstruction accuracy improvement in terms of PSNR/SSIM by 3.73dB/0.07 and 9.43dB/0. over compressive sensing and uniform subsampling counterparts across the dimensionality reduction ratios of 4-32x, respectively. Source code is available at https://figshare.com/s/519a923fae8f386d7f5b .
I NTRODUCTION
In the era of Internet-of-things (IoT) data explosion (Biookaghazadeh et al., 2018), efﬁcient information acquisition and on-sensor data dimensionality reduction techniques are in great need.
Compressive sensing is the state-of-the-art signal sensing technique that is applicable to on-sensor data dimensionality reduction.
Compressive sensing performs a linear mapping of signals onto a lowerdimensional space.
Due to the high computational complexity of the linear transformation ( O ( n 2 ) when original signals are n dimensional and a sensing matrix is a standard Gaussian random matrix), its implementation on sensor devices requires either a high cost (implemented in the analog domain, such as single pixel camera (Duarte et al., 2008) ) or large computing resources (implemented in the digital domain).
To mitigate this problem, several approaches have been proposed to reduce the computational complexity of compressive sensing by constraining the sensing matrices to be sparse, binary, or ternary (Wang et al., 2016;
Nguyen et al., 2017; Zhao et al., 2018; Hong et al., 2019).
These approaches can reduce the computational complexity by a constant factor ( O ( cn 2 ) , where c can be as low as 10 − 2 ).
Nonetheless, the reduced computational complexity can be still too high thus hardly affordable for resource-constrained sensor devices, e.g. , low-cost IoT sensors (Djelouat et al., 2018), or high-data-rate sensor devices dealing with high-dimensional signals, e.g. , LiDAR and depth map (Chodosh et al., 2019).
In this paper, we propose a selective sensing framework that fundamentally addresses the abovementioned problem.
Selective sensing adopts the novel concept of data-driven nonuniform subsampling to reduce the dimensionality of acquired signals while retaining the information of interest in a computation-free fashion.
Speciﬁcally, the data dimensionality reduction in selective sensing is a nonuniform subsampling (or selection) process that simply selects the most informative entries of a signal vector based on an optimized, stationary selection index vector informed by training data.
Since no computation is involved for any form of data encoding, the computational complexity of the selective sensing operator is simply O (1) , leading to the computation-free data dimensionality reduction during the selective sensing process.
1 Selective sensing adopts a co-optimization methodology to co-train a selective sensing operator with a subsequent information decoding neural network.
As the trainable parameters of the sensing operator (the selection index) and the information decoding neural network are discrete- and continuous-valued, respectively, the co-optimization problem in selective sensing is a mixed discrete-continuous optimization problem that is inherently difﬁcult to solve.
We propose a feasible solution to solve it by transforming the mixed discretecontinuous optimization problem into two continuous optimization subproblems through interpolation and domain extension techniques.
Both of the subproblems can be then efﬁciently solved using gradient-descent-based algorithms.
We take image as the sensing modality and reconstruction as the information decoding task to demonstrate the 1st proof-of-concept of selective sensing.
The experiments on CIFAR10, Set5 and Set14 datasets show that the selective sensing framework can achieve an average reconstruction accuracy improvement in terms of PSNR/SSIM by 3.73dB/0. and 9.43dB/0.16 over compressive sensing and uniform subsampling counterparts across the dimensionality reduction ratios of 4-32x, respectively.
The contributions of this paper are summarized as follows:
We propose a new on-sensor data dimensionality reduction method called selective sensing.
Selective sensing efﬁciently reduces the dimensionality of acquired signals in a computation-free fashion while retaining information of interest.
The computation-free nature of selective sensing makes it a highly suitable solution for performing on-sensor data dimensionality reduction on resourceconstrained sensor devices or high-data-rate sensor devices dealing with high-dimensional signals.
We propose and apply the novel concept of data-driven nonuniform subsampling.
Speciﬁcally, we ﬁrst formulate the problem of co-optimizing a selective sensing operator with a subsequent information decoding neural network as a mixed discrete-continuous optimization problem.
Furthermore, we propose a viable solution that transforms the problem into two continuous optimization subproblems that can be efﬁciently solved by gradient-descent-based algorithms, which makes the co-training feasible.
We empirically show that data-driven nonuniform subsampling can well preserve signal information under the presence of a co-trained information decoding network.
R ELATED W ORK 2. N ONUNIFORM S UBSAMPLING
Model-based nonuniform subsampling has been proposed in Chepuri et al.
(2016) in the name of sparse sensing.
Sparse sensing requires a hand-crafted sparsity model of a signal as prior knowledge.
Differently, selective sensing requires no prior knowledge about the sparsity model of a signal, as all the necessary information needed for reconstruction can be learned from data through the training process.
Therefore, selective sensing has a much broader range of applications, especially in IoT, than sparse sensing, considering a vast majority of IoT signals are not well studied nor understood yet, but huge amounts of IoT data are already available for training and learning.
Dadkhahi & Duarte (2014) proposes to generate an image mask that can preserve the manifold structure presented in image data. Differently, we focus on the task of single image sensing and reconstruction in this paper. Weiss et al. ; Huijben et al.
(2019) proposes to perform image nonuniform subsampling in k-space (frequency domain).
Speciﬁcally, the image sensing and reconstruction tasks in Bahadir et al.
(2020) are both completed in the frequency domain. Consequently, the complex computation of Fourier transformation is required, which makes this method hardly applicable to on-sensor data dimensionality reduction. Differently, selective sensing works in the spatial domain and is computation-free. Therefore, selective sensing is extremely friendly to resource-constrained or high-data-rate sensors.
2. S ENSING M ATRIX S IMPLIFICATION M ETHODS
The computational complexity of the linear transformation in compressive sensing is O ( n 2 ) .
Zhao et al.
; Hong et al.
(2019) proposes model-based methods to construct sparse sensing matrices.
Wang et al.
; Nguyen et al.
(2017) propose data-driven methods to build binary or ternary sensing matrices.
However, all these approaches could only reduce the computational complexity by constant factors, i.e. O ( cn 2 ) , where c can be as low as 10 − 2 ).
A key differentiator of selective sensing is that by adopting the novel concept of data-driven non-uniform subsampling, it is computation-free and has a computational complexity of O (1) .
2. D ATA - DRIVEN C OMPRESSIVE S ENSING
Kulkarni et al.
; Mousavi & Baraniuk (2017);
Yao et al.
(2019) propose to directly learn the inverse mapping of compressive sensing through the training of reconstruction neural network models.
In addition, Mousavi et al.
; Wu et al.
(2018) propose to co-train a customized sensing scheme with a reconstruction neural network to improve the reconstruction accuracy.
These approaches inspire us to develop a framework that co-trains a selective sensing operator and a subsequent information decoding network.
M ETHODOLOGY
In this section, we ﬁrst formulate the co-optimization of a selective sensing operator and a subsequent information decoding network as a mixed discrete-continuous optimization problem.
Then, by applying continuous interpolation and domain extension on the integer variables, we reformulate the mixed discrete-continuous optimization problem into two continuous optimization problems, both of which can be solved by conventional gradient-descent-based algorithms.
Based on the new formulation, we extend the conventional backpropagation(BP) algorithm to derive a general co-training algorithm to co-optimize a selective sensing operator and a subsequent information decoding network.
At last, by taking images as the sensing modality and using reconstruction as the information decoding task, we propose a practical approach, referred to as SS+Net, to compose a selective sensing framework for image selective sensing and reconstruction.
In this paper, a lowercase letter denotes a scalar or a scalar-valued function, and a uppercase letter denotes a vector, a matrix, a tensor, or a vector-valued function.
We use brackets to index the element of a vector, a matrix, or a tensor.
For example, assume X denotes a n -dimensional vector
X =
[ x 0 , ..., x n − 1 ] , then X
[ i ] = x i for i = 0 , · · · , n − 1 .
3. P ROBLEM F ORMULATION
Consider the original signal X is an n -dimensional vector, the subsampling rate is mn , and the subsampled measurement Y is a m -dimensional vector.
The selective sensing of X is a nonuniform subsampling or a selection process that can be formulated as
[Equation]
where S ( X, I ) is a function that stands for the selective sensing operator.
I is a m -dimensional vector denoting the selection set, which contains the indices (integer values between 0 and n − 1 ) of the elements to be selected.
Consider N ( Y, Θ) is a subsequent information decoding network and Θ is the trainable parameters.
o is a differentiable objective function that measures the information loss throughout the entire selective sensing process with respect to a information acquisition task.
For instance, in a signal reconstruction task, the objective function can be deﬁned as a loss function which measures the difference between the reconstructed signal and the original signal.
The cooptimization problem of the sensing operator S and the information decoding network N can be formulated as
[Equation]
subject to i 0 , . . , i m − 1 are integers within interval [0 , n [Equation]
[Equation]
Given the entries of Θ are continuous variables, and the entries of I are constrained to be integer variables within [0 , n − 1] , the problem in (2) is a mixed discrete-continuous optimization problem that can not be directly solved with conventional gradient-descent-based algorithms.
This is because the gradient of o with respect to I does not exist.
3. R EFORMULATION BY C ONTINUOUS I NTERPOLATION AND D OMAIN E XTENSION
By applying the continuous interpolation on S with respect to I and the extension on the domain of
S , we can reformulate the problem in (2) into two subproblems.
For simplicity, we adopt a linear interpolation in our method.
However, nonlinear interpolation methods can be also applied.
We deﬁne a piece-wise linear function f
( X, i ) as
[Equation]
where i u = floor ( i )
+ 1 , i d = floor ( i ) , r u = i u mod n and r d = i
d mod n.
In (3), i is a real-valued scalar, floor () is the ﬂooring function returning the closest integer that is less than or equal to the input, and mod is the modulo operation.
f ( X, i ) essentially interpolates the value of X over a continuous index i in a piece-wise linear fashion and extends the range of i to ( −∞ , ∞ ) .
Given a X , f
( X, i ) is periodic over every n -length interval of i .
At integer values of i , we have f
( X, i ) =
X [ i mod n ] , which returns the original value of the [ i mod n ] -th element of X .
Speciﬁcally, when i is an integer in interval [0 , n − 1] , we have f ( X, i ) =
X
[ i ] .
Due to the continuous interpolation and domain extension, f ( X, i ) is almost everywhere differentiable over i except for all the integer points.
The choice of the gradient value at integer points turn out to be insensitive to the algorithm performance.
For simplicity, we deﬁne the derivatives of f ( X, i ) at integer values of i as zero.
As such, we have the gradient value of f with respect to i in the whole space which can be expressed as
[Equation]
Based on (4), we deﬁne a continuous selective sensing operator function S (cid:48) as
[Equation]
Leveraging (3) and (5), we reformulate the mixed discrete-continuous optimization problem in (2) into two subproblems deﬁned as
[Equation]
and
I opt =
[ round ( i ) mod n for each entry i in I R ] ,
[Equation]
where round () is an even rounding function that returns the closest integer value of the input, and the initial values of Θ in (7) is Θ R .
Note that both the subproblems in (6) and (7) are unconstrained and the gradient of o with respect to I and Θ can be calculated over the whole space in (6).
Therefore, we can solve the subproblems in (6) and (7) sequentially using gradient-descent based algorithms.
For the brevity of illustration, we refer to the process of solving the subproblem in (6) and (7) as the initial-training and the ﬁne-tuning step, respectively, in the rest of this paper.
3. E XTENSION OF THE B ACKPROPAGATION A LGORITHM
Generally, neural network models are trained over multiple training samples and the gradients of trainable parameters are calculated using the BP algorithm.
We extend the BP algorithm and derive the gradient calculation (with respect to I ) over a batch of training samples as follows.
Given a batch of b samples X 1 , · · · , X b of the signal X for training, the forward pass of the BP algorithm in the initial-training step can be derived as
[Equation]
where i = 1 , · · · , b , Z i is the representation of the information decoded by the network and o batch is the loss function that measures the average information loss throughout the selective sensing process.
One can also choose to use the total information loss here.
In the backward pass of the BP algorithm, the gradient calculation with respect to Θ is the same as in regular neural network training.
The gradient calculation with respect to I can be derived using the chain rule of derivative.
Speciﬁcally, the gradient calculation of o batch with respect to Y i can be derived as
[Equation]
Subsequently, the gradient calculation of o batch with respect to I over a batch of training samples can be derived as
[Equation]
Leveraging the gradient calculations in (9) and (10), the subproblem in (6) can be therefore solved by using gradient-descent-based algorithms.
The outputs from the initial-training step include the optimized selection set I R and the corresponding reconstruction network parameters Θ R .
As the entries of I R are continuous over interval ( −∞ , ∞ ) , one needs further convert I R to an integer selection set I opt as shown in (7).
To compensate for the accuracy loss due to rounding, the reconstruction network shall be further ﬁne tuned in the ﬁne-tuning step while keeping I opt ﬁxed as shown in (7).
The entire algorithm of co-training a information decoding network N and a selective sensing operator S is summarized in Algorithm 1 in the appendix.
3. I MAGE S ELECTIVE S ENSING AND R ECONSTRUCTION
In the rest of the paper, we take image as the sensing modality and reconstruction as the information decoding task to demonstrate the ﬁrst proof-of-concept of selective sensing.
The prior work discussed in subsection 2.3 shows that neural network models can be trained to directly approximate the inverse mapping of compressive sensing to perform the reconstruction.
Therefore, we hypothesize that there exists a direct mapping from the selective sensed (nonuniformly subsampled) domain to the original image domain, and such a mapping can be well approximated by a neural network co-trained with the selective sensing operator.
Furthermore, we hypothesize that the existing image compressive sensing reconstruction networks can be also used for image selective sensing reconstruction.
Based on our hypotheses, we use the loss function l ( ˆ X, X ) as the objective function in (2), where l is a function that measures the distance between ˆ X and X , e.g. a mean-square-error function, and ˆ X is the output ( Z in (8)) of the information decoding network N .
As such, N is trained to directly reconstruct the original image from the selective sensing measurement as
[Equation]
We refer to the image selective sensing and reconstruction frameworks composed in such way as
SS+Net.
To evaluate the performance of SS+Net, we compare it against compressive sensing and uniform subsampling counterparts referred to as CS+Net and US+Net, respectively.
CS+Net and US+Net use same reconstruction networks but replace the selective sensing operator in SS+Net with a Gaussian random sensing matrix 2 and a uniform subsampling operator 3 , respectively. Additionally, we set DCNet ReconNet
PS NR ( d B )
SS I M 0.06 0.12 0.18 0.24 0.06 0.12 0.18 0. SS(Ours)CS US DCNet
ReconNet 0.06 0.12 0.18 0.24 0.06 0.12 0.18 0.
R e c on s t r u c t i on A cc u r a cy
Measurement Rate
Network:
CIFAR Set5 and Set14 Dataset: 5. all the hyper-parameters in SS+Net, CS+Net, and US+Net to be the same during the training for a fair comparison. The purpose of using CS+Net and US+Net as the reference methods is to reveal the true impact of selective sensing on compressive information acquisition in comparison to the compressive sensing and uniform subsampling counterparts.
E XPERIMENTS 4. E XPERIMENT S ETUP
We conduct experiments on two datasets with two different reconstruction networks at the measurement/subsampling rates ranging from 0.03125 to 0.25 (corresponding to the dimensionality reduction ratios of 32-4x).
The ﬁrst dataset is CIFAR10 (Krizhevsky et al., 2009).
The second dataset is composed in the same way as illustrated in Xu et al.
(2018), which has 228,688 training samples and 867 testing samples.
The testing samples are the non-overlapped image patches from Set (Bevilacqua et al., 2012) and Set14 (Zeyde et al., 2010).
All the samples are of size 64x64 with three color channels(RGB).
The sensing is performed channel-wise in all the experiments, i.e., for each framework of SS-Net, CS-Net and US-Net, each sample is ﬁrst reshaped to three 4096-dimensional vectors corresponding to the three color channels. Subsequently, three sensing operators corresponding to the three color channels are used to sense the three vectors of each sample, respectively. The sensed measurements from the three color channels are grouped together and then fed into the reconstruction network.
Prior to the training, 5% of the training samples are randomly selected and separated out as the validation set.
The two reconstruction networks we experimented with are DCNet and ReconNet (Kulkarni et al., 2016).
DCNet has the same network structure as the generator network of DCGAN (Radford et al., 2015).
We made some minor but necessary modiﬁcations to the structure of DCNet and ReconNet in order to perform image reconstruction and speed up the training.
The modiﬁcation details are summarized in subsection A.2 in the Appendix.
In the beginning of the training process of SS+Net, the selection index I is randomly initialized with real values from the uniform distribution U (0 , n ) . We co-train the selective sensing operator
DimensionalityReductionRatio
Uniform Subsampling
Selective Sensing(Ours) Compressive Sensing (20.49,0.52) (17.72,0.28) (19.22,0.43) (18.40,0.33) (PSNR,SSIM) (22.77,0.69) (21.11,0.59) (19.77,0.46) (18.79,0.36)
Original (20.82,0.56) (16.33,0.23) (17.84,0.29) (19.16,0.41) and the reconstruction network for 300 iterations, of which the ﬁrst 150 iterations are used for the initial-training step and the rest are used for the ﬁne-tuning step.
We use two different optimizers to optimize different components of SS+Net:
An Adam optimizer with a learning rate of 0.001 is used to optimize reconstruction networks and a SGD optimizer with a learning rate of 100000 is used to optimize sensing operators.
Using a high learning rate for training the selective sensing operators is because the gradient values with respect to the selection index I (calculated with (4)) turns out to be orders of magnitude smaller (because adjacent pixels in natural images mostly have very close pixel values) than the the rest of the gradient values and the learning rate of 100000 performs well in the experiments.
For the training of the CS+Net and US+Net counterparts, except that there is no optimizer for sensing operators, all the other experiment setups remain the same with SS+Net.
4. R ESULTS AND A NALYSIS
The reconstruction accuracy is measured as average reconstruction PSNR and SSIM.
The experiment results of PSNR and SSIM are plotted in Figure 1.
As shown in Figure 1, selective sensing achieves up to 44.92dB/0.9952 reconstruction PSNR/SSIM at the measurement rate of 0.25 (dimensionality reduction ratio of 4).
Even at the low measurement rate of 0.03125 (dimensionality reduction ratio of 32), selective sensing still achieves at least 23.35dB/0.6975 reconstruction PSNR/SSIM.
The experiment results validate our hypothesis that the direct mapping from the selective sensed domain to the original image domain can be well approximated by existing reconstruction neural networks co-trained with the selective sensing operator, and data-driven nonuniform subsampling can well preserve signal information under the presence of the co-trained information decoding neural network.
Furthermore, the experiment results show that selective sensing consistently outperforms compressive sensing and uniform subsampling, especially at higher dimensionality reduction ratios.
The average PSNR/SSIM improvement of selective sensing over compressive sensing and uniform subsampling across all the experiments is 3.73dB/0.07 and 9.43dB/0.16, respectively.
As the only difference between SS+Net, CS+Net, and US+Net is the sensing operator used, the experiment results imply that selective sensing better preserves signal information than compressive sensing and uniform subsampling as a result of the co-optimization of the sensing and reconstruction stages.
The detailed numerical PSNR and SSIM results are presented in Table 1, Table 2, Table 3 and Table 4 in the Appendix.
Compared with the images in CIFAR10, the images in Set5 and Set14 have more details, which makes the reconstruction inherently more difﬁcult.
We take one image from Set14 as an example to illustrate the visual reconstruction quality comparison.
As shown in Figure 2, selective sensing reconstructs the image with ﬁner and sharper details, such as the beard and the textures of the skin, than compressive sensing and uniform subsampling across all dimensionality reduction ratios.
More visual reconstruction quality comparison showing the same results are illustrated in Figure 3, Figure 4 and Figure 5 in the Appendix.
These visual reconstruction quality comparisons show strong evidence that selective sensing better preserves signal information than compressive sensing and uniform subsampling. 4. C OMPARISON A GAINST R ANDOM INDICES
To understand the effectiveness of the trained selection indices in selective sensing over randomly generated selection indices, we design a trivial random subsampling approach called RS+Net and compared it with SS+Net.
The experimental setup for RS+Net is consistent with US+Net except that the selection indices of RS+Net are integers randomly generated from the uniform distribution
U (0 , n ) . The experiments are performed on CIFAR10 at the dimensionality reduction ratios ofand 8 (measurement rates of 0.03125 and 0.125) with ReconNet. At each measurement rate, we randomly generate eight different sets of selection indices, based on which we train eight reconstruction networks from scratch, respectively, for RS+Net. We observe that the ﬁnal reconstruction accuracy of RS+Net on average is signiﬁcantly lower than
SS+Net counterparts.
The average PSNR of RS+Net is 22.78 dB and 31.02 dB at measurement rates of 0.03125 and 0.125, which is 5.73 dB and 9.49 dB lower than the SS+Net counterparts, respectively.
In addition, we observe a large variance in the ﬁnal reconstruction accuracy of RS+Net across the eight sets of selection indices.
Speciﬁcally, at the measurement rate of 0.03125 and 0.125, the gap between the maximum and the minimum PSNRs is 0.59 dB and 0.54dB, and the standard deviation of PSNRs is 0.19 dB and 0.20 dB, respectively.
The detailed results are summarized in
Table 5 in the Appendix.
Furthermore, to illustrate the superiority of the co-optimized selection indices in selective sensing over random selection indices, we also train the reconstruction networks in
RS+Net from scratch based on the pre-trained selection indices in SS+Net.
The ﬁnal reconstruction accuracy is 28.49 dB and 40.47 dB at the measurement of 0.03125 and 0.0125, which is 5.71 dB and 9.45 dB higher than the original RS+Net counterparts using random selection indices, respectively.
These results indicate that random selection indices are insufﬁcient for retaining information of interest with respect to the subsequent reconstruction network.
Differently, learned indices from co-training a selective sensing operator with a subsequent reconstruction network can signiﬁcantly improve the selective sensing performance in terms of retaining the information of interest for highaccuracy reconstruction.
The visualization and comparison of learned (co-trained) and random selection masks are shown in Figure 6, 7, and 8 in subsection A.3 of the Appendix.
In this paper, we propose a selective sensing framework that adopts the novel concept of data-driven nonuniform subsampling to perform on-sensor data dimensionality reduction in a computation-free fashion.
Selective sensing adopts a co-optimization methodology to co-train a selective sensing operator with a subsequent information decoding neural network.
The co-training of selective sensing is ﬁrst formulated as a mixed-discrete-continuous optimization problem.
By applying continuous interpolation and domain extension to the sensing index domain with quantization and ﬁne-tuning techniques, we reformulate the problem into two continuous optimization subproblems that can be solved by gradient-descent-based algorithms.
This is the key to enabling the co-training of the selective sensing operator with the subsequent information decoding neural network.
The experiments on CIFAR10, Set5, and Set14 datasets show that the proposed selective sensing framework can achieve an average reconstruction accuracy improvement in terms of PSNR/SSIM of 3.73dB/0.
and 9.43dB/0.16 over compressive sensing and uniform subsampling counterparts across the dimensionality reduction ratios of 4-32x, respectively.
The computation-free nature of selective sensing makes it a highly suitable solution for performing compressive information acquisition on resourceconstrained sensor devices or high-data-rate sensor devices dealing with high-dimensional signals.
A. M ORE EXPERIMENTS DETAILS
The code, datasets and pretrained models can be downloaded from: https://figshare.com/s/519a923fae8f386d7f5b .
The experiments are conducted in parallel on four RTX 2080 Ti GPU cards.
One training process(300 iterations) runs on one GPU card takes around 80 minutes.
We made following necessary modiﬁcations to the structure of DCNet and ReconNet in order to perform image reconstruction and speed up the training process.
We modify the ﬁrst layer of DCNet by increasing the number of input channels to be equal to the number of measurements in order to feed measurements into DCNet.
As ReconNet is designed to reconstruct grayscale images, we made some modiﬁcations to the network structure of ReconNet in order to reconstruct color images. Speciﬁcally, we replace the ﬁrst layer of ReconNet, which is a fully-connected layer, with three fullyconnected layers in parallel to generate three feature map corresponding to three color channels. In addition, we modify the third and the sixth convolutional layers to have three output channels in order to maintain the dimensionality of the original image. The batch normalization layers are also added right behind each convolutional layer (except for the last convolutional layer which is the output layer) to accelerate the training.
A. M ORE EXPERIMENT RESULTS
Figure 6,Figure 7 and Figure 8 shows that sampling points in learned masks are more evenly distributed on the whole image and less clustered at speciﬁc locations across all measurement rates than random masks.
Consequently, the measurements from selective sensing are less noisy but more informative than random sensing.
For instance, the outlines of objects in the measurements sensed by selective sensing are more visually comprehensible than random sensing, which provides a possible visual explanation on the effectiveness of selective sensing over random subsampling.
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table] [Table] [Table] [Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table] Measurement [Table] [Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
Dimensionality Reduction Ratio    (36.74,0.95) (33.45,0.91) (30.88,0.86) (28.70,0.80) (33.04,0.89) (26.71,0.72) (30.63,0.83) (28.52,0.77)
Compressive
Sensing
Selective
Sensing(Ours) (PSNR,SSIM)
Original (30.69,0.86) (19.43,0.59) (22.54,0.65) (26.36,0.75) Uniform Subsampling
Dimensionality
Reduction
Ratio    (30.89,0.95) (26.64,0.89) (22.75,0.81) (19.48,0.69) (25.72,0.83) (16.36,0.49) (22.53,0.76) (19.20,0.63)
Compressive
Sensing
Selective
Sensing(Ours) (PSNR,SSIM)
Original (22.19,0.82) (13.39,0.38) (14.63,0.46) (17.16,0.62) Uniform Subsampling
DimensionalityReduction
Ratio    (33.10,
[Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table]
Compressive
Sensing
Selective
Sensing(Ours) (PSNR,SSIM)
Original (23.34,0.77) (14.96,0.35) (16.
[Table]
[Table] [Table]
[Figure]