Paper:
OnACID: Online Analysis of Calcium Imaging Data in Real Time
Optical imaging methods using calcium indicators are critical for monitoring the activity of large neuronal populations in vivo. Imaging experiments typically generate a large amount of data that needs to be processed to extract the activity of the imaged neuronal sources. While deriving such processing algorithms is an active area of research, most existing methods require the processing of large amounts of data at a time, rendering them vulnerable to the volume of the recorded data, and preventing real-time experimental interrogation. Here we introduce OnACID, an Online framework for the Analysis of streaming Calcium Imaging Data, including i) motion artifact correction, ii) neuronal source extraction, and iii) activity denoising and deconvolution. Our approach combines and extends previous work on online dictionary learning and calcium imaging data analysis, to deliver an automated pipeline that can discover and track the activity of hundreds of cells in real time, thereby enabling new types of closed-loop experiments. We apply our algorithm on two large scale experimental datasets, benchmark its performance on manually annotated data, and show that it outperforms a popular offline approach.

Review:
This paper proposes an online framework for analyzing calcium imaging data. This framework is built upon the popular and now widely used constrained non-negative matrix factorization (CNMF) method for cell segmentation and calcium time-series analysis (Pnevmatikakis, et al., 2016). While the existing CNMF approach is now being used by many labs across the country, I’ve talked to many neuroscientists that complain that this method cannot be applied to large datasets and thus its application has been limited. This work extends this method to a real-time decoding setting, making it an extremely useful contribution for the neuroscience community.
The paper is well written and the results are compelling. My only concern is that the paper appears to combine multiple existing methods to achieve their result. Nonetheless, I think the performance evaluation is solid and an online extension of CNMF for calcium image data analysis and will likely be useful to the neuroscience community.
Major comments:
- There are many steps in the described method that rely on specific thresholds or parameters. It would be useful to understand the sensitivity of the method to these different hyperparameters. In particular, how does the threshold used to include new components affect performance?
- Lines 40-50: This paragraph seems to wander from the point that you nicely set up before this paragraph (and continue afterwards). Not sure why you’re bringing up closed loop, except for saying that by doing this in real time you can do closed-loop experiments. I’m not sure the idea that you want to communicate here. 
- The performance evaluations do not really address the use of isotonic regression for deconvolution. In Figure 3C, the traces appear to be demixed calcium after segmentation and not the deconvolved spike trains. Many of the other comparisons focus on cell segmentation. Please comment on the use of deconvolution and how it possibly might help in cell segmentation.
- The results show that the online method outperforms the original CNMF method. Can the authors comment on where the two differ? 
Minor comments:
Figure 1: The contours cannot be seen in B. Perhaps a white background in B (and later in the results) can help to see the components and contours. C is hard to see and interpret initially as the traces overlap significantly.