Paper:
Learning convolution filters for inverse covariance estimation of neural network connectivity
We consider the problem of inferring direct neural network connections from Calcium imaging time series. Inverse covariance estimation has proven to be a fast and accurate method for learning macro-and micro-scale network connectivity in the brain and in a recent Kaggle Connectomics competition inverse covariance was the main component of several top ten solutions, including our own and the winning team's algorithm. However, the accuracy of inverse covariance estimation is highly sensitive to signal preprocessing of the Calcium fluorescence time series. Furthermore, brute force optimization methods such as grid search and coordinate ascent over signal processing parameters is a time intensive process, where learning may take several days and parameters that optimize one network may not generalize to networks with different size and parameters. In this paper we show how inverse covariance estimation can be dramatically improved using a simple convolution filter prior to applying sample covariance. Furthermore, these signal processing parameters can be learned quickly using a supervised optimization algorithm. In particular, we maximize a binomial log-likelihood loss function with respect to a convolution filter of the time series and the inverse covariance regularization parameter. Our proposed algorithm is relatively fast on networks the size of those in the competition (1000 neurons), producing AUC scores with similar accuracy to the winning solution in training time under 2 hours on a cpu. Prediction on new networks of the same size is carried out in less than 15 minutes, the time it takes to read in the data and write out the solution.

Review:
This paper addresses the reconstruction of network topology from calcium imaging data, using inverse covariance matrix estimation. It is shown empirically that a simple convolution filter (to be applied to the calcium traces) can be learned (once and for all) that substantially improves the reconstruction performance, and the time it takes to infer connectivity on new datasets.----QUALITYOverall, what is done seems solid. There are a couple of wrinkles that I would like the authors to clarify.First: the authors restricted the length of the convolution filter to 10 time steps. The learned filter peaks at time step 8, which left me to wonder whether there was enough room for the filter to "converge" (in "time lag space"). For example, had it been restricted to 5 time steps, the authors would have missed this presumably crucial peak. Was that a computational restriction? If not, is there a biophysical (or empirical?) reason why correlations would fall off anyway after 10 time steps? The authors should probably discuss this.Second: although I understand that the study is focused on improving the inverse covariance method specifically, I would have liked to hear more of a discussion regarding the inherent limitations of the method. For example, the fact that only an undirected graph can be extracted seems like a big restriction as far as neural circuits are concerned.Third, how crucial is it to get the value of $\chi$ in Eq.7 in the right ballpark? In particular, how does reconstruction generalize to datasets of very different sparsity than the one assumed for training?CLARITYThe paper is well-written and well structured.It'd be great if the Kaggle dataset (e.g. the fact that the ground truth connectivity is known) could be described upfront (at the beginning of the methods) in 1/2 sentences.  I would also like the authors to clarify the timescales for the non-experts (e.g. what does "one time step" in the convolution filter mean? what does "time (20 ms)" mean exactly in the x-axis labels? that 20ms are shown in total, or 1000*20ms = 20 sec?). It would also help the readers to appreciate the difficulty of the task.Also, AUC is nowhere defined (not even spelled out -- Area Under the (ROC) Curve?).ORIGINALITY I'm not an expert in this specific calcium imaging literature, but it seems surprising that nobody had tried (even heuristic) convolution filters on top of calcium data prior to covariance estimation before... Anyhow, the work presented here is original, and clearly improves on the current leaderboard for Kaggle.SIGNIFICANCEEstimating network topology has very important implications for the neurosciences, especially with the advent of whole-network imaging techniques. There definitely is a need for statistical methods. While this study provides state-of-the-art performance and speed, I believe it remains essentially a simple (though important) addition to a known algorithm (L2-regularized inverse covariance estimation) which unfortunately does not address its most inherent limitations (e.g.  underlying, implicit Gaussian assumption, undirected graph recovery, ...).