Selective Sensing: A Data-driven Nonuniform Subsampling Approach for Computation-free On-Sensor Data Dimensionality Reduction

1 INTRODUCTION . In the era of Internet-of-things ( IoT ) data explosion ( Biookaghazadeh et al. , 2018 ) , efficient information acquisition and on-sensor data dimensionality reduction techniques are in great need . Compressive sensing is the state-of-the-art signal sensing technique that is applicable to on-sensor data dimensionality reduction . However , directly performing compressive sensing in the digital domain as a linear transformation of signals can be computationally costly , especially when the signal dimension n is high and/or a data-driven sensing matrixMousavi et al . ( 2017 ; 2018 ) ; Lohit et al . ( 2018 ) ; Wu et al . ( 2018 ) is used . To mitigate this problem , several approaches have been proposed to reduce the computational complexity of compressive sensing by constraining the sensing matrices to be sparse , binary , or ternary ( Wang et al. , 2016 ; Nguyen et al. , 2017 ; Zhao et al. , 2018 ; Hong et al. , 2019 ) . While these approaches can reduce the computational complexity by a constant factor ( O ( cn2 ) , where c can be as low as 10−2 ) , such reduced computational complexity can be still too high to be affordable for resource-constrained sensor devices , e.g. , low-cost IoT sensors ( Djelouat et al. , 2018 ) , or high-data-rate sensor devices dealing with high-dimensional signals , e.g. , LiDAR and depth map ( Chodosh et al. , 2019 ) . Other approaches ( Duarte et al. , 2008 ; Robucci et al. , 2010 ) propose to implement compressive sensing in the analog domain instead , eliminating or reducing the computation cost of compressive sensing through custom hardware implementation . However , such custom hardware implementation inevitably increases the cost of the sensor and is often specific to the sensor design , thereby can not be generally applied to other sensors or applications . In this paper , we propose a selective sensing framework to address the above-mentioned problem by adopting the novel concept of data-driven nonuniform subsampling to reduce the dimensionality of acquired signals while retaining the information of interest in a computation-free fashion . Specifically , the data dimensionality reduction in selective sensing is a nonuniform subsampling ( or selection ) process that simply selects the most informative entries of a signal vector based on an optimized , stationary selection index vector informed by training data . Since no computation is involved for any form of data encoding , the computational complexity of the selective sensing operator is simply O ( 1 ) , leading to the computation-free data dimensionality reduction during the selective sensing process.1 Selective sensing adopts a co-optimization methodology to co-train a selective sensing operator with a subsequent information decoding neural network . As the trainable parameters of the sensing operator ( the selection index ) and the information decoding neural network are discrete- and continuous-valued , respectively , the co-optimization problem in selective sensing is a mixed discrete-continuous optimization problem that is inherently difficult to solve . We propose a feasible solution to solve it by transforming the mixed discrete-continuous optimization problem into two continuous optimization subproblems through interpolation and domain extension techniques . Both of the subproblems can then be efficiently solved using gradient-descent-based algorithms . We take images as the sensing modality and reconstruction as the information decoding task to demonstrate the 1st proof-of-concept of selective sensing . The experiments on CIFAR10 , Set5 and Set14 datasets show that the selective sensing framework can achieve an average reconstruction accuracy improvement in terms of PSNR/SSIM by 3.73dB/0.07 and 9.43dB/0.16 over compressive sensing and uniform subsampling counterparts across the dimensionality reduction ratios of 4-32x , respectively . The contributions of this paper are summarized as follows : 1 . We propose a new on-sensor data dimensionality reduction method called selective sensing . Selective sensing efficiently reduces the dimensionality of acquired signals in a computation-free fashion while retaining information of interest . The computation-free nature of selective sensing makes it a highly suitable solution for performing on-sensor data dimensionality reduction on resourceconstrained sensor devices or high-data-rate sensor devices dealing with high-dimensional signals . 2 . We propose and apply the novel concept of data-driven nonuniform subsampling . Specifically , we first formulate the problem of co-optimizing a selective sensing operator with a subsequent information decoding neural network as a mixed discrete-continuous optimization problem . Furthermore , we propose a viable solution that transforms the problem into two continuous optimization subproblems that can be efficiently solved by gradient-descent-based algorithms , which makes the co-training feasible . 3 . We empirically show that data-driven nonuniform subsampling can well preserve signal information under the presence of a co-trained information decoding network . 2 RELATED WORK . 2.1 NONUNIFORM SUBSAMPLING . Model-based nonuniform subsampling has been proposed in Chepuri et al . ( 2016 ) in the name of sparse sensing . Sparse sensing requires a hand-crafted sparsity model of a signal as prior knowledge . Differently , selective sensing requires no prior knowledge about the sparsity model of a signal , as all the necessary information needed for reconstruction can be learned from data through the training process . Therefore , selective sensing has a much broader range of applications , especially in IoT , than sparse sensing , considering a vast majority of IoT signals are not well studied nor understood yet , but huge amounts of IoT data are already available for training and learning . Dadkhahi & Duarte ( 2014 ) proposes to generate an image mask that can preserve the manifold structure presented in image data . Differently , we focus on the task of single image sensing and reconstruction in this paper . Baldassarre et al . ( 2016 ) ; Weiss et al . ( 2019 ) ; Gözcü et al . ( 2018 ) ; Bahadir et al . ( 2019 ; 2020 ) propose to perform MRI image nonuniform subsampling in k-space ( frequency domain ) . As many spatial-domain signals are much sparser in the frequency domain , e.g. , natural images and MRI images , the existing nonuniform subsampling approaches performed in k-space are insufficient 1For temporal signals , the selection operation can be simply implemented in the digital domain with a counter and a mux that already exists in the control logic of most sensors . We consider such operations as control rather than data computation as no data is computed during the selective sensing process . For spatial signals such as images , the selective sensing operator can also be implemented as a low-cost masked sensor array with no computation involved . In addition , Mayberry et al . ( 2014 ) ; Centeye ( 2020 ) present image sensor architectures for embedded systems that can provide pixel-level control of image sensors . for dealing with dense signals directly in the spatial domain . In addition , the complex computation of or the custom hardware ( Macfaden et al. , 2017 ) for implementing Fourier transformation required in these methods is a deal-breaker for resource-constrained sensor devices and/or high-data-rate sensor devices dealing with high-dimensional signals . Differently , selective sensing works directly in the spatial domain and the selective sensing operators require no computation upon the sensor data at all . Huijben et al . ( 2019 ) propose to co-optimize a probabilistic subsampling mask and a subsequent task-specific neural network in an end-to-end fashion . The sensing mask is dynamically generated from a random distribution with respect to each signal . The computation of generating such masks is hardly affordable for resource-constrained and/or high-data-rate sensor devices . Differently , selective sensing uses a static sensing mask learnt through the co-training algorithm . Once a sensing mask is depolyed to sensor devices , no computation is needed to update the existing mask . Therefore , selective sensing is extremely friendly to resource-constrained or high-data-rate sensors . 2.2 SENSING MATRIX SIMPLIFICATION METHODS . The computational complexity of the linear transformation in compressive sensing is O ( n2 ) . Zhao et al . ( 2018 ) ; Hong et al . ( 2019 ) proposes model-based methods to construct sparse sensing matrices . Wang et al . ( 2016 ) ; Nguyen et al . ( 2017 ) propose data-driven methods to build binary or ternary sensing matrices . However , all these approaches could only reduce the computational complexity by constant factors , i.e . O ( cn2 ) , where c can be as low as 10−2 ) . A key differentiator of selective sensing is that by adopting the novel concept of data-driven non-uniform subsampling , it is computation-free and has a computational complexity of O ( 1 ) . 2.3 DATA-DRIVEN COMPRESSIVE SENSING . Kulkarni et al . ( 2016 ) ; Mousavi & Baraniuk ( 2017 ) ; Yao et al . ( 2019 ) propose to directly learn the inverse mapping of compressive sensing through the training of reconstruction neural network models . In addition , Mousavi et al . ( 2017 ; 2018 ) ; Lohit et al . ( 2018 ) ; Wu et al . ( 2018 ) propose to co-train a customized sensing scheme with a reconstruction neural network to improve the reconstruction accuracy . It should be noted that such co-training algorithms are specific to the reconstruction network proposed in the corresponding literatures . To the best of our knowledge , there is no general co-training algorithm that can be applied to various reconstruction networks in the domain of datadriven compressive sensing . These approaches inspire us to develop a framework that co-trains a selective sensing operator and a subsequent information decoding network . Co-trained signal sensing and reconstruction frameworks can be viewed as a specific type of autoencoders ( Goodfellow et al. , 2016 ) . The main difference between such frameworks and a general autoencoder model is that the sensing ( encoder ) part of such frameworks must be implemented on sensors for on-sensor data dimensionality reduction . Therefore , the computation complexity of the encoder has to be extremely low in order to be affordable for sensor devices . 2.4 IMAGE SUPER-RESOLUTION . The problem of neural-network-based image super-resolution has been studied in recent years ( Ledig et al. , 2017 ; Dong et al. , 2015 ; Yang et al. , 2019 ) . The image super-resolution task is fundamentally different from the image reconstruction task of selective sensing in following two aspects . First , images in super-resolution tasks are uniformly subsampled in the training phase , while images in selective sensing are nonuniformly subsampled . Therefore , the existing network structures for image super-resolution can not be directly applied to perform the image reconstruction task in selective sensing . Second , the downsizing factor of images in super-resolution tasks is only up to 4x to the best of our knowledge in the existing literature . Differently , in selective sensing and reconstruction tasks , the nonuniformly subsampling factor ( compression ratio ) of images can have a much larger range ( 4-32x in this paper ) . 3 METHODOLOGY . In this section , we first formulate the co-optimization of a selective sensing operator and a subsequent information decoding network as a mixed discrete-continuous optimization problem . Then , by applying continuous interpolation and domain extension on the integer variables , we reformulate the mixed discrete-continuous optimization problem into two continuous optimization problems , both of which can be solved by conventional gradient-descent-based algorithms . Based on the new formulation , we extend the conventional backpropagation ( BP ) algorithm to derive a general co-training algorithm to co-optimize a selective sensing operator and a subsequent information decoding network . At last , by taking images as the sensing modality and using reconstruction as the information decoding task , we propose a practical approach , referred to as SS+Net , to compose a selective sensing framework for image selective sensing and reconstruction . In this paper , a lowercase letter denotes a scalar or a scalar-valued function , and a uppercase letter denotes a vector , a matrix , a tensor , or a vector-valued function . We use brackets to index the element of a vector , a matrix , or a tensor . For example , assume X denotes a n-dimensional vector X = [ x0 , ... , xn−1 ] , then X [ i ] = xi for i = 0 , · · · , n− 1 .

Summary: The paper proposes a framework for jointly optimizing a selective sensing operator and a neural network for reconstruction. The motivation is to alleviate the quadratic (in dimensions) complexity associated with standard compressive sensing by using a dimension-free selective sensing approach, which can be jointly optimized with the decoder while guaranteeing that the relevant information is preserved. As this formulation yields a mixed discrete-continuous optimization, the authors propose a standard relaxation of the discrete constraints using interpolation.