Selective Sensing: A Data-driven Nonuniform Subsampling Approach for Computation-free On-Sensor Data Dimensionality Reduction

1 INTRODUCTION . In the era of Internet-of-things ( IoT ) data explosion ( Biookaghazadeh et al. , 2018 ) , efficient information acquisition and on-sensor data dimensionality reduction techniques are in great need . Compressive sensing is the state-of-the-art signal sensing technique that is applicable to on-sensor data dimensionality reduction . However , directly performing compressive sensing in the digital domain as a linear transformation of signals can be computationally costly , especially when the signal dimension n is high and/or a data-driven sensing matrixMousavi et al . ( 2017 ; 2018 ) ; Lohit et al . ( 2018 ) ; Wu et al . ( 2018 ) is used . To mitigate this problem , several approaches have been proposed to reduce the computational complexity of compressive sensing by constraining the sensing matrices to be sparse , binary , or ternary ( Wang et al. , 2016 ; Nguyen et al. , 2017 ; Zhao et al. , 2018 ; Hong et al. , 2019 ) . While these approaches can reduce the computational complexity by a constant factor ( O ( cn2 ) , where c can be as low as 10−2 ) , such reduced computational complexity can be still too high to be affordable for resource-constrained sensor devices , e.g. , low-cost IoT sensors ( Djelouat et al. , 2018 ) , or high-data-rate sensor devices dealing with high-dimensional signals , e.g. , LiDAR and depth map ( Chodosh et al. , 2019 ) . Other approaches ( Duarte et al. , 2008 ; Robucci et al. , 2010 ) propose to implement compressive sensing in the analog domain instead , eliminating or reducing the computation cost of compressive sensing through custom hardware implementation . However , such custom hardware implementation inevitably increases the cost of the sensor and is often specific to the sensor design , thereby can not be generally applied to other sensors or applications . In this paper , we propose a selective sensing framework to address the above-mentioned problem by adopting the novel concept of data-driven nonuniform subsampling to reduce the dimensionality of acquired signals while retaining the information of interest in a computation-free fashion . Specifically , the data dimensionality reduction in selective sensing is a nonuniform subsampling ( or selection ) process that simply selects the most informative entries of a signal vector based on an optimized , stationary selection index vector informed by training data . Since no computation is involved for any form of data encoding , the computational complexity of the selective sensing operator is simply O ( 1 ) , leading to the computation-free data dimensionality reduction during the selective sensing process.1 Selective sensing adopts a co-optimization methodology to co-train a selective sensing operator with a subsequent information decoding neural network . As the trainable parameters of the sensing operator ( the selection index ) and the information decoding neural network are discrete- and continuous-valued , respectively , the co-optimization problem in selective sensing is a mixed discrete-continuous optimization problem that is inherently difficult to solve . We propose a feasible solution to solve it by transforming the mixed discrete-continuous optimization problem into two continuous optimization subproblems through interpolation and domain extension techniques . Both of the subproblems can then be efficiently solved using gradient-descent-based algorithms . We take images as the sensing modality and reconstruction as the information decoding task to demonstrate the 1st proof-of-concept of selective sensing . The experiments on CIFAR10 , Set5 and Set14 datasets show that the selective sensing framework can achieve an average reconstruction accuracy improvement in terms of PSNR/SSIM by 3.73dB/0.07 and 9.43dB/0.16 over compressive sensing and uniform subsampling counterparts across the dimensionality reduction ratios of 4-32x , respectively . The contributions of this paper are summarized as follows : 1 . We propose a new on-sensor data dimensionality reduction method called selective sensing . Selective sensing efficiently reduces the dimensionality of acquired signals in a computation-free fashion while retaining information of interest . The computation-free nature of selective sensing makes it a highly suitable solution for performing on-sensor data dimensionality reduction on resourceconstrained sensor devices or high-data-rate sensor devices dealing with high-dimensional signals . 2 . We propose and apply the novel concept of data-driven nonuniform subsampling . Specifically , we first formulate the problem of co-optimizing a selective sensing operator with a subsequent information decoding neural network as a mixed discrete-continuous optimization problem . Furthermore , we propose a viable solution that transforms the problem into two continuous optimization subproblems that can be efficiently solved by gradient-descent-based algorithms , which makes the co-training feasible . 3 . We empirically show that data-driven nonuniform subsampling can well preserve signal information under the presence of a co-trained information decoding network . 2 RELATED WORK . 2.1 NONUNIFORM SUBSAMPLING . Model-based nonuniform subsampling has been proposed in Chepuri et al . ( 2016 ) in the name of sparse sensing . Sparse sensing requires a hand-crafted sparsity model of a signal as prior knowledge . Differently , selective sensing requires no prior knowledge about the sparsity model of a signal , as all the necessary information needed for reconstruction can be learned from data through the training process . Therefore , selective sensing has a much broader range of applications , especially in IoT , than sparse sensing , considering a vast majority of IoT signals are not well studied nor understood yet , but huge amounts of IoT data are already available for training and learning . Dadkhahi & Duarte ( 2014 ) proposes to generate an image mask that can preserve the manifold structure presented in image data . Differently , we focus on the task of single image sensing and reconstruction in this paper . Baldassarre et al . ( 2016 ) ; Weiss et al . ( 2019 ) ; Gözcü et al . ( 2018 ) ; Bahadir et al . ( 2019 ; 2020 ) propose to perform MRI image nonuniform subsampling in k-space ( frequency domain ) . As many spatial-domain signals are much sparser in the frequency domain , e.g. , natural images and MRI images , the existing nonuniform subsampling approaches performed in k-space are insufficient 1For temporal signals , the selection operation can be simply implemented in the digital domain with a counter and a mux that already exists in the control logic of most sensors . We consider such operations as control rather than data computation as no data is computed during the selective sensing process . For spatial signals such as images , the selective sensing operator can also be implemented as a low-cost masked sensor array with no computation involved . In addition , Mayberry et al . ( 2014 ) ; Centeye ( 2020 ) present image sensor architectures for embedded systems that can provide pixel-level control of image sensors . for dealing with dense signals directly in the spatial domain . In addition , the complex computation of or the custom hardware ( Macfaden et al. , 2017 ) for implementing Fourier transformation required in these methods is a deal-breaker for resource-constrained sensor devices and/or high-data-rate sensor devices dealing with high-dimensional signals . Differently , selective sensing works directly in the spatial domain and the selective sensing operators require no computation upon the sensor data at all . Huijben et al . ( 2019 ) propose to co-optimize a probabilistic subsampling mask and a subsequent task-specific neural network in an end-to-end fashion . The sensing mask is dynamically generated from a random distribution with respect to each signal . The computation of generating such masks is hardly affordable for resource-constrained and/or high-data-rate sensor devices . Differently , selective sensing uses a static sensing mask learnt through the co-training algorithm . Once a sensing mask is depolyed to sensor devices , no computation is needed to update the existing mask . Therefore , selective sensing is extremely friendly to resource-constrained or high-data-rate sensors . 2.2 SENSING MATRIX SIMPLIFICATION METHODS . The computational complexity of the linear transformation in compressive sensing is O ( n2 ) . Zhao et al . ( 2018 ) ; Hong et al . ( 2019 ) proposes model-based methods to construct sparse sensing matrices . Wang et al . ( 2016 ) ; Nguyen et al . ( 2017 ) propose data-driven methods to build binary or ternary sensing matrices . However , all these approaches could only reduce the computational complexity by constant factors , i.e . O ( cn2 ) , where c can be as low as 10−2 ) . A key differentiator of selective sensing is that by adopting the novel concept of data-driven non-uniform subsampling , it is computation-free and has a computational complexity of O ( 1 ) . 2.3 DATA-DRIVEN COMPRESSIVE SENSING . Kulkarni et al . ( 2016 ) ; Mousavi & Baraniuk ( 2017 ) ; Yao et al . ( 2019 ) propose to directly learn the inverse mapping of compressive sensing through the training of reconstruction neural network models . In addition , Mousavi et al . ( 2017 ; 2018 ) ; Lohit et al . ( 2018 ) ; Wu et al . ( 2018 ) propose to co-train a customized sensing scheme with a reconstruction neural network to improve the reconstruction accuracy . It should be noted that such co-training algorithms are specific to the reconstruction network proposed in the corresponding literatures . To the best of our knowledge , there is no general co-training algorithm that can be applied to various reconstruction networks in the domain of datadriven compressive sensing . These approaches inspire us to develop a framework that co-trains a selective sensing operator and a subsequent information decoding network . Co-trained signal sensing and reconstruction frameworks can be viewed as a specific type of autoencoders ( Goodfellow et al. , 2016 ) . The main difference between such frameworks and a general autoencoder model is that the sensing ( encoder ) part of such frameworks must be implemented on sensors for on-sensor data dimensionality reduction . Therefore , the computation complexity of the encoder has to be extremely low in order to be affordable for sensor devices . 2.4 IMAGE SUPER-RESOLUTION . The problem of neural-network-based image super-resolution has been studied in recent years ( Ledig et al. , 2017 ; Dong et al. , 2015 ; Yang et al. , 2019 ) . The image super-resolution task is fundamentally different from the image reconstruction task of selective sensing in following two aspects . First , images in super-resolution tasks are uniformly subsampled in the training phase , while images in selective sensing are nonuniformly subsampled . Therefore , the existing network structures for image super-resolution can not be directly applied to perform the image reconstruction task in selective sensing . Second , the downsizing factor of images in super-resolution tasks is only up to 4x to the best of our knowledge in the existing literature . Differently , in selective sensing and reconstruction tasks , the nonuniformly subsampling factor ( compression ratio ) of images can have a much larger range ( 4-32x in this paper ) . 3 METHODOLOGY . In this section , we first formulate the co-optimization of a selective sensing operator and a subsequent information decoding network as a mixed discrete-continuous optimization problem . Then , by applying continuous interpolation and domain extension on the integer variables , we reformulate the mixed discrete-continuous optimization problem into two continuous optimization problems , both of which can be solved by conventional gradient-descent-based algorithms . Based on the new formulation , we extend the conventional backpropagation ( BP ) algorithm to derive a general co-training algorithm to co-optimize a selective sensing operator and a subsequent information decoding network . At last , by taking images as the sensing modality and using reconstruction as the information decoding task , we propose a practical approach , referred to as SS+Net , to compose a selective sensing framework for image selective sensing and reconstruction . In this paper , a lowercase letter denotes a scalar or a scalar-valued function , and a uppercase letter denotes a vector , a matrix , a tensor , or a vector-valued function . We use brackets to index the element of a vector , a matrix , or a tensor . For example , assume X denotes a n-dimensional vector X = [ x0 , ... , xn−1 ] , then X [ i ] = xi for i = 0 , · · · , n− 1 .

Summary: The paper proposes a nonuniform sampling design scheme chosen using training data to reduce the computation of compressed sensing acquisition. The use of learning methods such as back propagation for the nonuniform sampling design problem is interesting. However, compressive sensing approaches in practice do not perform computation to obtain the measurement vector; instead, the sensors rely on custom hardware (such as the single pixel camera or the modulated wideband converter) to have the hardware act on the discretized signal according to the matrix design. Thus, considerations of the "sensing complexity" are moot in those cases, and the approach provided in this paper is only relevant in cases where custom sampling schemes can be designed (e.g., when the referred imaging sensors are used).