# Computational Methods for Neuron Segmentation in Two-Photon Calcium Imaging Data: A Survey
## 7 Summary of Performance on Public Benchmarks, Applications and Recommendations
### Conclusions and Future Work  
In this article, we provide a comprehensive overview of the computational methods researchers use for neuron segmentation in calcium imaging. We introduce the problem in the context of spatio-temporal segmentation and then divide the approaches in three main categories: signal/image based approaches, matrix factorization-based approaches and machine learning-based approaches. We describe the core methods shortly and also provide performance figures (if reported by the authors). We also provide a comprehensive list of all the benchmarked approaches and links to their source codes if available. Finally, we also include recommendations for new researchers which are based on the problem definition and the approaches we studied. Based on the literature survey we conducted, we have identified following areas which can be further improved. * Neuron segmentation would benefit greatly if more annotated datasets are made available apart from the Neurofinder benchmark. \begin{table}
\begin{tabular}{l l l l} \hline \hline
**Category** & **Strengths** & & **Weaknesses** \\ \hline \multirow{4}{*}{Signal/Image processing-based methods} & \multirow{4}{*}{*} & \multirow{4}{*}{Simple and explainable Computational complexity Fewer parameters to tune Easily implementable} & \multirow{4}{*}{\begin{tabular}{l} Have to compromise either on temporal or spatial information \\ \end{tabular} } \\  & & & \(\bullet\) & Difficulties with separating overlapping neurons \\  & & & \(\bullet\) & Vulnerable to noise \\  & & & \(\bullet\) & Not transferable \\  & & & \(\bullet\) & Domain knowledge needed for parameters tuning \\ \hline \multirow{4}{*}{Matrix decomposition-based methods} & \multirow{4}{*}{*} & Better at capturing spatio-temporal context Explainable & \multirow{4}{*}{\begin{tabular}{l} Computationally expensive \\ Unwanted artefacts \\ \end{tabular} } \\  & & Better performance on in vivo datasets than signal/image processing-based methods & \multirow{4}{*}{\
\begin{tabular}{l} More problem dependent \\ Most of these methods cannot be scaled linearly \\ \end{tabular} } \\ \hline \multirow{4}{*}{ML-based methods} & \multirow{4}{*}{*} & \multirow{4}{*}{Can be taught to process calcium imaging stacks of different resolutions and frequencies State-of-the-art results on most of the datasets Knowledge from one dataset can be transferred to other datasets. End-to-end learning} & \multirow{4}{*}{
\begin{tabular}{l} Higher computational power needed in many cases Require lots of training data Manual labelling required in most approaches Mostly black box models. Although researchers have proposed empirical methods for explaining the behaviour of DL models, an analytical explanation is still out of the question. \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Comparison of the main strong and weak points of each algorithm category. * In their current form, the annotated datasets of Neurofinder have one two-dimensional mask for a calcium imaging video where only the location of every neuron is identified. It would greatly help if the datasets were annotated in temporal domain as well by adding information about when those neurons were active. * Machine learning-based approaches can specifically benefit from synthetic data. * Another interesting avenue to explore is the use of reinforcement learning. We know the firing patterns of neurons and we know they follow the dynamics of calcium indicators. This information can be used to to train reinforcement learning methods to identify neurons. * Another interesting avenue to explore is generative adversarial networks to learn the calcium imaging distributions and then use the learned distribution for neuron segmentation. * Since calcium imaging datasets are acquired by different devices and under different conditions, they might not be similar. Moreover, there are other methods as well which capture the same neural activity but in different form (functional magnetic resonance imaging, for example). Therefore, domain alignment and domain adaptation can also help.