Paper:
Compressive Sensing of Signals from a GMM with Sparse Precision Matrices
This paper is concerned with compressive sensing of signals drawn from a Gaussian mixture model (GMM) with sparse precision matrices. Previous work has shown: (i) a signal drawn from a given GMM can be perfectly reconstructed from r noise-free measurements if the (dominant) rank of each covariance matrix is less than r; (ii) a sparse Gaussian graphical model can be efficiently estimated from fully-observed training signals using graphical lasso. This paper addresses a problem more challenging than both (i) and (ii), by assuming that the GMM is unknown and each signal is only observed through incomplete linear measurements. Under these challenging assumptions, we develop a hierarchical Bayesian method to simultaneously estimate the GMM and recover the signals using solely the incomplete measurements and a Bayesian shrinkage prior that promotes sparsity of the Gaussian precision matrices. In addition, we provide theoretical performance bounds to relate the reconstruction error to the number of signals for which measurements are available, the sparsity level of precision matrices, and the "incompleteness" of measurements. The proposed method is demonstrated extensively on compressive sensing of imagery and video, and the results with simulated and hardware-acquired real measurements show significant performance improvement over state-of-the-art methods.

Review:
This paper proposes a method to recover signals from compressive measurements. The method consists of jointly estimating the signal and a Gaussian Mixture Model (GMM) capable of representing it succinctly. The main contribution of the paper is the idea of imposing a sparse structure on the GMM adapted to the case when the signal of interest corresponds to image patches. This is further exploited by a more structured prior that promotes an appropriate group-sparsity pattern (essentially interactions between adjoining pixels are not penalized by the sparsity-inducing penalty). Some theoretical results are provided for the case in which there is just one Gaussian in the mixture, but their implication or the conditions under which they hold are not explained clearly. Numerical simulations show that the method yields good results for very small images and for individual image patches. However the question of how to apply the method to normal-sized images is not addressed. This paper presents an interesting and potentially useful method for the compressed sensing of image patches and provides some hopeful numerical results. Its main weaknesses are:-The theoretical results are very confusing.  It is very difficult to derive any conclusions from the statement of the theorems or the additional explanations, which are very vague. The conditions for the theorems, which seem quite strong and ad hoc, are not explained. For example, a bound delta_max on the error seems to be assumed in Theorem 3. Also, the results in Theorem 2 are asymptotic and in high probability but this is not mentioned anywhere (except in the proof within the supplementary material). Finally, no attempt is made to clarify for what kind of sensing matrices the results would hold. -The extension of the theory to the case of several Gaussians, which does not seem trivial and is the method that is actually applied in practice, is not discussed. -Not a lot of visual results are shown either in the paper or in the supplementary material (this would be a good sanity check and I encourage the authors to include them).-It is not discussed how to combine the patches obtained from the method to produce whole images. Reference [3] states that this is not trivial, so the authors should at least discuss it.Quality:The paper introduces a novel approach to signal estimation using GMMs by promoting sparsity and group sparsity in the covariance matrices. The authors provide some numerical evidence for very small images and image patches, although they do not tackle the reconstruction of whole images. The theoretical results are not very conclusive.Clarity:The paper is relatively clearly written, except for the more technical parts which are confusing and difficult to follow. This is apparent in Section 3, where the authors barely discuss the conditions and the results of the theorems. Theorem 3 for example can only hold with high probability and asymptotically as it is “a combination of Theorems 1 and 2”, but this is not even mentioned!Some of the other mathematical statements are not very rigorous. For example, in equation (15) it seems that the authors consider x hat and x tilde random variables but they do not clarify in what sense the equality holds. The equality “follows from [21]” if delta_i is independent of x_i, but it is not clear to me that this is true. If it is, the authors should explain in what sense (in expectation, with high probability, etc.). Also, the authors refer to other works: “Following [21]” (line 280), “Similar to [21]” (line 284) … in a very vague way without specifying what part of the cited works they mean, which makes it difficult for the reader to follow. Originality:The idea of promoting a sparse or group-sparse structure of the covariance matrices in a GMM while performing signal estimation is novel to the best of my knowledge.Significance:I don’t think that the contribution of the paper is very significant because of the limited scope of the theory and because the numerical results do not include reconstructions of whole images.After reading the authors' rebuttal, my views on the paper are basically unchanged, although I'm increasing the score to 6. In particular, even though it might be straightforward to combine the patches, this does not mean that they should produce an accurate estimate of features in the image that span several patches. However, it is perhaps understandable that the authors do not address this given the space constraints.