# Computational Methods for Neuron Segmentation in Two-Photon Calcium Imaging Data: A Survey
## 6 Machine Learning-Based Methods
### ML Methods Applied to 3D Calcium Imaging Stacks  
In this sub-section, we cover all those ML/DL based approaches which apply an ML/DL algorithm directly to the spatio-temporal stacks. These methods can either be end-to-end or might need an additional post-processing step. [PERSON] et al. proposed a CNN architecture with a graph regularization term which can utilize unlabelled data along with labelled data for neuron segmentation [80]. First, they over-segment a neuron image into superpixels [81]. The superpixels are then fed to a CNN which minimizes a composite loss function over the labelled and unlabelled superpixels. The CNN is composed of two convolutional layers with 6 and 50 filters each. The loss function over labelled superpixels is represented by a softmax loss while the loss function over unlabelled superpixels is represented by a graph-regularized loss. The graph regularized loss function is formulated on the assumption that neighbouring elements in a graph will most likely share the same label. Therefore, they perform label propagation from labelled superpixels onto unlabelled superpixels by Gaussian field harmonic functions [82]. They repeat the segmentation process for all the images in a volume. Finally, they perform neuron segmentation in 3D volume by global association method [83]. They have validated their approach on a private one-photon data set. They manually annotated 2000 neuron regions spread over 1000 images (which form a 3D volume when stacked up), with half of them used for training and the remaining used for testing. Along with 2000 manually annotated regions, they include 70,000 more unlabelled regions (superpixels) which include 4000 neurons. They have reported an F1 score of 0.96 when they use as few as 40 images for training and an F1 score of 0.99 when they use 1000 images for training. [PERSON] et al. [84] proposed a 3D (referred to as (2 + 1)D) convolutional network based neuron segmentation approach for calcium imaging data. It accepts an input image stack containing T time slices. There are four \(10\times 10\) convolutional layers, a max pooling over all time slices and two \(1\times 1\) bottleneck fully connected layers. The output layer is configured to yield two 2D greyscale images as output, which together represent the softmax probability of each pixel being inside an ROI centroid. The dimensions of the input image stack are \(37\times 37\times T\), therefore to cover the whole spatial domain, the window is made to slide in two dimensions over the input image stack to produce an output pixel for every location of the window fully contained within the image bounds. The network was trained on two-photon calcium imaging data gathered from both the primary visual cortex (V1) and medial entorhinal cortex (MEC) from awake-behaving mice. size. Human experts annotated ROIs using the ImageJ Cell Magic Wand Tool [85], which automatically generates a ROI based on a single mouse click. The human experts found 4006 neurons in the V1 dataset with an average of 148 neurons per image series and 538 neurons in the MEC dataset with an average of 54 neurons per image series. They have used F1 score to evaluate the network performance and reported F1 = 71%. [PERSON] et al. [86] proposed a 3D-UNet [21] based neuron segmentation approach on the calcium imaging data acquired from the nematode \"Caenorhabditis elegans\". The UNet accepts a \(160\times 160\times 16\) image stack and produces a binary mask of the same dimension, with every voxel either representing a neuron or background. The 3D UNet consists of three encoding layers, one middle layer and three decoding layers. The first encoding layer consists of two convolutional operations with 8 and 16 filters, second and third also consists of two convolutional operations with 16, 32 and 32, 64 filters. Each encoding layer is followed by \(2\times 2\) downsampling. The middle layer has 2 convolutional operations with 64 filters each. The output of each encoding layer is concatenated with the input of decoding layer at similar level.