BACKGROUND
cells can be genetically engineered with fluorescence genes that are inherited when the cell divides, meaning the descendents use the genes to continuously synthesize fluorescent proteins.
this allows a form of biology experiment called “lineage tracing” to see the long-term impact of specific populations of labelled cells.
the labelled cells can be bred into an animal from birth, can be injected, transplanted, etc.
snippert et al.
<cit>  developed a 4-color “confetti” transgene for labelling stem cells.
the confetti transgene exploits genetic recombination to achieve a random choice of color  in each stem cell.
when a stem cell is induced to divide , each daughter cell expresses the same color as its ancestor.
this creates a patch of homogeneous color in the regenerated tissue.
regenerated cells of different color must have originated from different stem cells.
confetti fluorescence and other multi-color cell labelling strategies are useful for tracking regeneration in adult mice, for evaluating the potency of stem cells in vivo, or for judging the effectiveness of stem cell therapies.
in this project we address the analysis of multi-color stem cells after muscle regeneration.
skeletal muscle is a highly regenerative tissue in which each mature muscle cell is a long thin fiber with many nuclei.
this muscle fiber is surrounded by a basal lamina, which gives the muscle fiber its firmness during contraction.
muscle-resident stem cells, called satellite cells, are located between the muscle fiber and the basal lamina.
if the muscle fiber is severely damaged, it will become necrotic and induce an immune reaction.
this activates the satellite cells, which migrate to the injured area and divide into a set of myoblasts.
the myoblasts each have one nucleus, but upon differentiation they fuse together in a linear configuration to generate a multi-nucleated myotube or myofiber.
the myoblasts can also fuse to pre-existing or partially-damaged fibers  <cit> .
figure  <dig> shows a cross-section and a lateral-section of regenerated muscle fibers from confetti-fluorescent mice.
fig.
<dig> a cross-section  and a lateral-section  of regenerated muscle fibers from confetti-fluorescent mice.
a five-month-old male  and female  mice were injected with  <dig> μ
g/g tamoxifen on  <dig> consecutive days to achieve transgene recombination.
ten days after the last tamoxifen injection, muscle injury was induced by injecting the tibialis anterior with  <dig> μ
l  <dig> μ
m cardiotoxin.
sixteen days after injury mice were sacrificed, the tibialis anterior was fixed and frozen, and  <dig> μ
m sections were cut.
scalebars are  <dig> μ
m
cells with multiple fluorescent proteins are imaged using multi-channel microscopy .
for example, the image datasets used in our project include four channels: cyan, green, yellow and red .
each of the fluorescent proteins emitting those colors was excited by a laser at its respective excitation wavelength.
the light emitted from the sample contains autofluorescence, so the light is passed through a band-pass filter specific for each fluorescent protein, before detection with a camera.
the resulting images show which muscle fibers are positive for which fluorescent proteins.
because muscle fibers are multinucleated cells, an overlay of the four colors can show muscle fibers positive for more than one color.
fig.
<dig> a multi-channel microscopy image of muscle fibers.
cyan channel;  green channel;  yellow channel;  red channel;  composite color image from all the four channels;  composite color image from cyan, yellow and red channels after preprocessing.
this section is from the same muscle of the cross-section in fig.
1
a. scalebars are  <dig> μ
m. each channel is a 12-bit image of  <dig> ×  <dig> pixels
to analyze the regeneration results, the images must be segmented.
a cursory glance at a composite color image in fig.
2
e may lead to the conclusion that this segmentation is similar to the segmentation of real-world color images.
however, our problem is different or more difficult in some ways as follows.
in our images, there is extensive contact or overlap between objects , meaning that contour-closing  does not work very well for segmentation.
we also cannot re-use methods that combine object recognition with segmentation unless we develop domain-specific object models.
in addition, many of the boundaries are blurry; some of the objects are in the process of fusing; and there is tremendous variation in the fiber brightness.
using a conventional color difference measure may not be appropriate in our problem since color similarity in multi-channel imaging is different from color similarity in a normal spectrum of visible light.
furthermore, the images have random noise and non-random artifacts including optical aberration from the imaging device; damaged tissue or fracture planes during sample preparation; and ice crystals which cause small empty holes in the image.
the ice crystals often have clear boundaries but they should be omitted from the segmentation results.
finally, the four colors of the confetti construct are in different locations within the cell.
green is located in the nucleus, yellow and red are in the cytosol and cyan is on the membrane.
in mature muscle fibers, the membrane becomes a sarcolemma with many invaginations.
as a result, cyan fluorescence can be seen inside the muscle fibers as well as along the cell-cell edges.
we propose a novel method called slic-mmed  which uses superpixels for the segmentation of muscle fibers in muli-channel microscopy.
a superpixel is a perceptual grouping of neighboring pixels that aligns better with image edges than a rectangular patch  <cit> .
superpixels are widely used in numerous applications in computer vision including image segmentation.
among existing superpixel generation methods  <cit> , simple linear iterative clustering   <cit>  was chosen for our project because of its effectiveness, scalability and speed.
however, slic needs to be modified to adapt to our problem.
nuclei are orders of magnitude smaller than mature muscle cells, but when colored green they have a very strong color boundary.
this difference in the scale of color change  could confound the superpixel generation.
so we first remove the nuclei.
a simple method for segmenting the nuclei turns out to be extremely accurate by using domain-specific information .
to make existing image processing algorithms useful for our domain, we had to perform several modifications:  forking different channels to different methods, based on an object model for nuclei,  introducing an enhanced superpixel method named simple linear iterative clustering on multi-channel microscopy with edge detection , and  developing a semi-automatic segmentation framework based on superpixels that can produce very good results for our problem.
we believe that all three of these modifications will be useful for other forms of multi-channel cell microscopy, for non-neuronal eukaryotic cell types.
methods
overview of the framework
an overview of our framework is shown in fig.
<dig>  first, each channel undergoes intensity normalization and noise reduction filtering.
then the green channel is processed to extract the nuclei.
the remaining channels  are used to generate superpixels according to our slic-mmed algorithm .
next, an automatic superpixel merging algorithm  is executed to merge a subset of the generated superpixels to form the muscle fibers.
the superpixel generation and/or this superpixel merging step can be performed repeatedly with different user-defined parameters until users are satisfied or until termination criteria are reached.
after that, the resulting superpixels and merged regions can be further revised through a user-friendly graphical user interface .
lastly, this segmentation is combined with the nuclear segmentation to form the final result.
in this section, we use the dataset shown in fig.
<dig> to describe our proposed framework.
the primary steps in our framework are detailed below.
fig.
<dig> processing steps in the proposed framework
fig.
<dig> simple linear iterative clustering on multichannel microscopy with edge detection  algorithm
fig.
<dig> superpixel merging algorithm
preprocessing and nuclear segmentation
the first preprocessing is to normalize each channel to the same range, e.g.,  <cit> .
another optional preprocessing step is to apply a noise reduction filter, e.g., median filter, to each channel in cases of noisy images.
figure 2
f shows an example of a composite color image from cyan, yellow and red channels after preprocessing.
in microscopy, cell nuclei are often the easiest morphological features to identify , and many microscopy protocols include nuclear staining, or more recently, fluorescent proteins genetically engineered for nuclear localization.
because each fluorescent tag corresponds to one  channel, we can analyze channels individually, based on this knowledge of the underlying signal sources.
in other words, we can analyze the green channel for nucleus-like objects.
by using image thresholding and morphological techniques, the nuclei in the dataset shown in fig.
<dig> can be segmented using only the green channel as illustrated in fig.
<dig>
fig.
<dig> nuclear segmentation.
green channel after preprocessing ;  after image thresholding;  using morphological techniques to remove noise;  segmented nuclei
slic-mmed for superpixel generation
the original slic algorithm operates on color images in the cielab color space with one input parameter is k, the desired number of superpixels.
each pixel is represented by a 5-dimensional feature vector, t, containing  <dig> color components and  <dig> pixel coordinates.
at the initialization step, k initial cluster centroids c
k=[l
k,a
k,b
k,xk,yk]t are sampled on a regular grid with the interval of s=n/k, where n is the number of pixels.
to reduce the chance of centering a superpixel on an edge or on a noisy pixel, each centroid is moved to the lowest gradient position in a 3× <dig> neighborhood.
next, in the iteration step, each pixel is associated with its nearest centroid.
in order to speed up the algorithm , the size of the search space is reduced to a region proportional to the superpixel size.
here, for each cluster centroid c
k, only the pixels in the 2s×2s region around c
k are evaluated, meaning that if the distance from a pixel to c
k is less than the distance from that pixel to its current associated centroid, then the associated centroid will be changed to c
k. once all the pixels have been assigned to their nearest centroids, an update process adjusts each centroid to be the mean feature vector of all the pixels belonging to the corresponding cluster.
in practice, repeating this iteration step  <dig> times is sufficient for most images.
finally, a postprocessing step assigns all disjoint pixels  to nearby superpixels.
in slic, the distance d between two pixels i and j is a combination of two distances, d
c and d
s, representing color proximity and spatial proximity, respectively, as below:
<dig> dc=2+2+ <dig> ds=2+ <dig> d=dc/m2+ds/s <dig>
where the compactness
m is used to to weight the relative importance of color similarity versus spatial proximity.
when m is large, spatial proximity is more important, and the resulting superpixels are more compact.
in contrast, a small value of m leads to superpixels that are less regular in size and shape; however, since in this case color proximity is more important, the resulting superpixels follow the image boundaries more closely.
since the input data in our project are multichannel microscopy images, not real-world color images, the feature vector needs to be modified.
instead of using  <dig> cielab color components , we use each image channel as a component of the feature vector.
since the green channel represents only nuclei, it is discarded from the feature vector and processed separately as described in the previous subsection.
in short, the feature vector representing each pixel in our data is t which consists of the pixel intensities in the  <dig> channels cyan, yellow, red, and the pixel coordinates , respectively.
in the superpixel method benchmarking, boundary recall is used to measure the fraction of the ground truth edges falling within at least two pixels of a superpixel boundary.
a good superpixel segmentation should adhere to object boundaries, meaning that it should produce a high boundary recall.
although slic demonstrates very good boundary recall performance for real-world color images  <cit> , this is not the case for our datasets as shown in fig.
7
c. to overcome these problems, we propose to integrate an additional score d
e into the pixel distance measure.
d
e represents the presence of edges between two pixels, suggesting the likelihood that an object boundary falls between the two points.
before starting superpixel generation, an edge detection algorithm is executed to compute a value p
i for each pixel i, indicating its probability of being on an edge .
then the distance d
e between two pixels i and j is calculated as the maximum edge probability over all the pixels lying on the line connecting pixel i and pixel j. the new distance is calculated as below.
<dig> dc′=2+2+ <dig> ds=2+ <dig> de=max∀t∈linept,d′=dc′/m2+ds/s2+α×de <dig>
fig.
<dig> superpixel generation using different algorithms.
computed edge map.
the segmentation results from  our method, slic-mmed;  slic;  vcells.
white arrows indicate poorly segmented superpixels
a wide range of edge detection algorithms would be appropriate for computing the edge probabilities p
i, and we chose a detector based on the photometric invariance theory and tensor-based features  <cit> .
figure 7
a shows an edge map generated using that edge detector.
note that if no edge detection were involved  then the revised distance measure d
′  would reduce to the original slic distance measure d .
the new algorithm is presented in fig.
<dig>
automatic superpixel merging
after superpixel generation, all the superpixels within each muscle fiber need to be merged together to form the muscle fiber boundary.
the similarity measure used in our method, to determine if two neighboring superpixels should be merged, is the chi-squared  histogram distance  <cit> .
the χ
<dig> distance between two histograms p and q is defined as
<dig> χ2=12∑k2pk+qk.
we represent the intensity distribution in each channel of each superpixel as a histogram and use the following formula to measure the similarity distance d
c between superpixels i and j,
<dig> dc=χ22+χ22+χ <dig>
where h
c,h
y,h
r are the superpixel histograms of channels cyan, yellow, and red, respectively.
in addition to d
c, another measure called edge strength is used for the superpixel merging decision.
the edge strength e
ij between superpixels i and j is defined as the average edge probability over all the pixels in superpixel i having at least one neighbor belonging to superpixel j. the d
c and edge strength are then used in a series of thresholds.
if superpixels i and j have similar colors, it might be because they are part of the same fiber, or it might be because they come from different fibers that happen to have similar colors.
therefore, whenever two superpixels have smilar d
c , they can only be merged if they have low edge strengths .
in addition, we use another predefined threshold thres_size to avoid muscle fibers having unrealistic sizes which are formed from over-merging.
however, this size limitation is not applied to superpixels representing “background” .
our iterative superpixel merging algorithm starts with a calculation of the χ
<dig> distances and the edge strengths between each superpixel and its neighbors.
then the method for merging superpixels is a series of thresholded criteria as described in fig.
<dig>  the algorithm stops when there are no more superpixels merged.
figure 8
a shows a result after this processing step.
fig.
<dig> segmentation using superpixels.
automatic superpixel grouping;  example of manual refinement: yellow box - draw a curve to merge all the regions along the curve, blue box - restore a superpixel to merge it with a neighboring region in another way;  final result;  ground truth
manual refinement
for challenging datasets, it is impossible to produce an error-free segmentation.
errors from the automatic superpixel merging process include two types:
over-merging: merging of superpixels from different muscle fibers, or from a muscle fiber and neighboring background/artifacts.
under-merging: some neighboring superpixels from the same muscle fiber have not been merged yet.
a manual refinement step is introduced to our framework through a user-friendly gui in order to fix the superpixel merging errors .
the main supported operations include
drawing a freehand curve to merge all the superpixels or regions along the curve.
restoring the original superpixels surrounding a selected position to allow manually merging them in another way.
if the superpixel generation produces a high boundary recall, using these two operations can guarantee a very good segmentation result .
after this manual refinement, the superpixel segmentation is combined with the nuclear segmentation in a postprocessing step to form the final segmentation result.
RESULTS
superpixel evaluation
we use two error metrics, boundary recall and under-segmentation error, to evaluate our slic-mmed algorithm and compare it with the original slic and another advanced superpixel algorithm named vcells  <cit> .
boundary recall is the fraction of ground truth edges that fall within a certain distance d  of at least one superpixel boundary.
a good superpixel segmentation should produce a high boundary recall.
under-segmentation error compares segment areas to measure to what extend superpixels flood over the ground truth segment borders.
details about the calculation of these two measures can be found in  <cit> .
we used the dataset in fig.
<dig> for this evaluation.
the corresponding ground truth was created by one computer expert under the supervison of our muscle biology expert .
the compactness was chosen as  <dig> for all the three algorithms.
the edge weight α in eq.
<dig> was  <dig> for slic-mmed.
the input image for the original slic and vcells algorithms was the composite color image from the cyan, yellow and red channels after preprocessing .
fig.
<dig> comparison of different superpixel methods using different measures:  boundary recall;  under-segmentation error;  processing time.
all algorithms were run repeatedly with the number of superpixels ranging from  <dig> to  <dig> with a step size of  <dig>  the computing platform was a  <dig> ghz intel xeon cpu e3- <dig> desktop computer with  <dig> gb ram running microsoft windows  <dig> professional 64-bit
segmentation evaluation
the final stage of evaluation considers the total sementation accuracy.
the methods slic and vcells provide only superpixels, not segmentation, so they are not covered in this section.
our method, slic-mmed, merges superpixels to create segmented regions, so the quality of its segmentation is inherently related to the quality of its superpixels.
we assess the segmentation using an absolute score, defined with respect to ground truth, called the dice similarity coefficient   <cit> .
the dsc measures the spatial overlap between two segmentations, x and y, and is defined as
<dig> dsc=2x∩yx+y,
where |x| and |y| are the number of pixels in x and y, respectively.
it should be noted that eq.
<dig> is for the evaluation of one resulting segment.
our segmentation problem is a multiple-object segmentation with multiple fibers and other regions.
we propose the median dsc  which is computed as in eq.
<dig> to measure the similarity between the segmentation result s and the ground truth g.
<dig> meddsc=mediani= <dig> ..,ngdsci,dsci=2gi∩sfgi+sf,
where n
g and n
s are the respective total number of segments in g and s,g
i is one segment in g, and s
f with f∈[ <dig> n
s] is the corresponding segment in s having the largest overlap with g
i.
for the example data set , we used slic-mmed to generate  <dig> superpixels .
after the automatic superpixel merging step , we manually refined the segmentation  and got the final result as in fig.
8
c. the resulting meddsc was  <dig>  for this dataset.
fig.
<dig> segmentation results applying slic-mmed to the images in fig.
<dig>  the formats of each image channel of the data are  8-bit  <dig> ×  <dig> pixels and  12-bit  <dig> ×  <dig> pixels.
each segmentation is displayed on the composite color image from the cyan, yellow and red channels after preprocessing
we also propose dsc
x, where x is a number between  <dig> and  <dig>  to measure the fraction of segments in the ground truth are segmented with a dsc≥x/ <dig>  the final segmentation in fig.
8
c has dsc
<dig> of  <dig> , meaning that 78% of the segments were segmented with a dsc≥ <dig> .
for the cross-section dataset and the lateral-section dataset in fig.
<dig>  the scores were dsc
<dig>  <dig>  and  <dig> , respectively.
discussion and
CONCLUSIONS
in our experiments, the proposed method can correctly segment muscle fibers in very heterogeneous sections having both bright and dark regions, a wide range of fiber sizes, homogeneous red or yellow but more irregular cyan segments in cytoplasm.
the fact that the method can handle a variety of cell sizes and morphologies in these confetti-fluorescent images suggests that it may be useful for analyzing confetti-fluorescent images in other tissues.
with an accurate segmentation, we can count the number of muscle fibers that contain each of the confetti colors.
the method works well for even weakly fluorescent areas, as in figs.
1
a and 10
a. using the same segmentation, we also can measure the diameter and cross sectional area for each fiber.
in most labs, measuring the diameter and cross-sectional area would require cutting a set of adjacent tissue sections, which would then undergo labor-intensive staining, followed by imaging and registration of the adjacent sections, to provide a superpositioning of the stained section and the confetti-labelled section.
staining requires doubling the number of sections because staining eliminates the endogenous fluorescence.
in this work we show it is possible to obtain segmentation from the endogenous fluorescence, allowing us to skip the costly process of staining.
in fluorescence microscopy, each fluorophore emits light over a range of wavelengths , causing nearby colors to overlap.
for example, the emission spectrum of green overlaps with the emission spectrum of yellow.
it is for this reason that multi-color labelling strategies have engineered the similar fluorescent proteins to have different sub-cellular localizations .
localization allows the identity of the label to be disambiguated.
in our images, the green signal bled into the yellow channel .
slic-mmed includes explicit management of different subcellular localizations, and this may be why we had no segmentation errors due to green-yellow spectral overlap.
the observed colors arise from fluorescent protein molecules that are diffusible in their compartment .
the brownian nature of diffusion suggests that color distribution might be nearly uniform across the space of the relevant compartment.
in other words, the limit of molecular diffusion becomes the boundary of the color, which we identify as the boundary of the segmentation.
cell membranes are 2-dimensional surfaces which can appear as 1-dimensional curves when imaged from a cross-section.
the cytosol and nucleus of a cell are 3-dimensional compartments, which appear as 2-dimensional continuous regions.
edge detection is a natural approach for analyzing 2-dimensional membrane-targeted fluorescence, such as the cyan channel in our images.
meanwhile, superpixel-based region detection methods are a natural approach for analyzing 3-dimensional compartments.
if the spatial distribution of the fluorescent proteins were punctate  or fibrillar , then our method would be less appropriate.
our segmentation framework, slic-mmed, is a “hybrid” method that combines the advantages of a region-based clustering algorithm  and an edge detector through the integrated edge map.
the introduction of an user-friendly superpixel refinement module provide flexibility for the framework.
as long as the superpixel generation provides a high performance in boundary recall, the framework provides very good segmentations.
our experimental results show a high degree of agreement with experts.
in the final scoring, the differences between different trials are also heavily dependent on the specific dataset, the number of superpixels to be revised, and the user’s expertise at performing the manual refinement step.
in future, we will intensively analyze the contribution of automation to the effectiveness of the framework.
the algorithm is potentially applicable to other multi-channel microscopy applications besides muscle.
mouse transgenes with confetti, brainbow or other multi-color stochastic labels have become extremely popular  <cit>  and the scientific community is rapidly generating multi-channel images that require analysis.
our image analysis method is particularly valuable for such applications.
additional file
additional file  <dig> the matlab and c code, and the data files used in this project.
we thank paul matsudaira for hosting the mouse experiments.
declarations
this article has been published as part of bmc systems biology volume  <dig> supplement  <dig>   <dig>  15th international conference on bioinformatics : systems biology.
the full contents of the supplement are available online http://bmcsystbiol.biomedcentral.com/articles/supplements/volume-10-supplement- <dig>
funding
funding for this research was provided by duke-nus srp phase  <dig> research block grant and by the national research foundation , prime minister’s office, singapore, under its create programme, singapore-mit alliance for research and technology  biosystems and micromechanics  irg.
availability of data and materials
the code and datafiles for this project are available as additional file  <dig>
authors’ contributions
method was designed and implemented by bpn with advice from ltk.
mouse experiments and microscopy were performed by hh.
ptcs and ltk supervised the imaging studies.
all authors edited and approved the manuscript.
competing interests
the authors declare that they have no competing interests.
consent for publication
not applicable.
ethics approval and consent to participate
animal experiments were approved by the institutional animal care and use committee of the national university of singapore, protocol number r14- <dig>

 cells can be genetically engineered with fluorescence genes that are inherited when the cell divides, meaning the descendents use the genes to continuously synthesize fluorescent proteins. this allows a form of biology experiment called “lineage tracing” to see the long-term impact of specific populations of labelled cells. the labelled cells can be bred into an animal from birth, can be injected, transplanted, etc. snippert et al. developed a 4-color “confetti” transgene for labelling stem cells. the confetti transgene exploits genetic recombination to achieve a random choice of color in each stem cell. when a stem cell is induced to divide, each daughter cell expresses the same color as its ancestor. this creates a patch of homogeneous color in the regenerated tissue. regenerated cells of different color must have originated from different stem cells. confetti fluorescence and other multi-color cell labelling strategies are useful for tracking regeneration in adult mice, for evaluating the potency of stem cells in vivo, or for judging the effectiveness of stem cell therapies. in this project we address the analysis of multi-color stem cells after muscle regeneration. skeletal muscle is a highly regenerative tissue in which each mature muscle cell is a long thin fiber with many nuclei. this muscle fiber is surrounded by a basal lamina, which gives the muscle fiber its firmness during contraction. muscle-resident stem cells, called satellite cells, are located between the muscle fiber and the basal lamina. if the muscle fiber is severely damaged, it will become necrotic and induce an immune reaction. this activates the satellite cells, which migrate to the injured area and divide into a set of myoblasts. the myoblasts each have one nucleus, but upon differentiation they fuse together in a linear configuration to generate a multi-nucleated myotube or myofiber. the myoblasts can also fuse to pre-existing or partially-damaged fibers. figure shows a cross-section and a lateral-section of regenerated muscle fibers from confetti-fluorescent mice. fig. a cross-section and a lateral-section of regenerated muscle fibers from confetti-fluorescent mice. a five-month-old male and female mice were injected with μ g/g tamoxifen on consecutive days to achieve transgene recombination. ten days after the last tamoxifen injection, muscle injury was induced by injecting the tibialis anterior with μ l μ m cardiotoxin. sixteen days after injury mice were sacrificed, the tibialis anterior was fixed and frozen, and μ m sections were cut. scalebars are μ m cells with multiple fluorescent proteins are imaged using multi-channel microscopy. for example, the image datasets used in our project include four channels: cyan, green, yellow and red. each of the fluorescent proteins emitting those colors was excited by a laser at its respective excitation wavelength. the light emitted from the sample contains autofluorescence, so the light is passed through a band-pass filter specific for each fluorescent protein, before detection with a camera. the resulting images show which muscle fibers are positive for which fluorescent proteins. because muscle fibers are multinucleated cells, an overlay of the four colors can show muscle fibers positive for more than one color. fig. a multi-channel microscopy image of muscle fibers. cyan channel; green channel; yellow channel; red channel; composite color image from all the four channels; composite color image from cyan, yellow and red channels after preprocessing. this section is from the same muscle of the cross-section in fig. 1 a. scalebars are μ m. each channel is a 12-bit image of × pixels to analyze the regeneration results, the images must be segmented. a cursory glance at a composite color image in fig. 2 e may lead to the conclusion that this segmentation is similar to the segmentation of real-world color images. however, our problem is different or more difficult in some ways as follows. in our images, there is extensive contact or overlap between objects, meaning that contour-closing does not work very well for segmentation. we also cannot re-use methods that combine object recognition with segmentation unless we develop domain-specific object models. in addition, many of the boundaries are blurry; some of the objects are in the process of fusing; and there is tremendous variation in the fiber brightness. using a conventional color difference measure may not be appropriate in our problem since color similarity in multi-channel imaging is different from color similarity in a normal spectrum of visible light. furthermore, the images have random noise and non-random artifacts including optical aberration from the imaging device; damaged tissue or fracture planes during sample preparation; and ice crystals which cause small empty holes in the image. the ice crystals often have clear boundaries but they should be omitted from the segmentation results. finally, the four colors of the confetti construct are in different locations within the cell. green is located in the nucleus, yellow and red are in the cytosol and cyan is on the membrane. in mature muscle fibers, the membrane becomes a sarcolemma with many invaginations. as a result, cyan fluorescence can be seen inside the muscle fibers as well as along the cell-cell edges. we propose a novel method called slic-mmed which uses superpixels for the segmentation of muscle fibers in muli-channel microscopy. a superpixel is a perceptual grouping of neighboring pixels that aligns better with image edges than a rectangular patch. superpixels are widely used in numerous applications in computer vision including image segmentation. among existing superpixel generation methods, simple linear iterative clustering was chosen for our project because of its effectiveness, scalability and speed. however, slic needs to be modified to adapt to our problem. nuclei are orders of magnitude smaller than mature muscle cells, but when colored green they have a very strong color boundary. this difference in the scale of color change could confound the superpixel generation. so we first remove the nuclei. a simple method for segmenting the nuclei turns out to be extremely accurate by using domain-specific information. to make existing image processing algorithms useful for our domain, we had to perform several modifications: forking different channels to different methods, based on an object model for nuclei, introducing an enhanced superpixel method named simple linear iterative clustering on multi-channel microscopy with edge detection, and developing a semi-automatic segmentation framework based on superpixels that can produce very good results for our problem. we believe that all three of these modifications will be useful for other forms of multi-channel cell microscopy, for non-neuronal eukaryotic cell types. methods overview of the framework an overview of our framework is shown in fig. first, each channel undergoes intensity normalization and noise reduction filtering. then the green channel is processed to extract the nuclei. the remaining channels are used to generate superpixels according to our slic-mmed algorithm. next, an automatic superpixel merging algorithm is executed to merge a subset of the generated superpixels to form the muscle fibers. the superpixel generation and/or this superpixel merging step can be performed repeatedly with different user-defined parameters until users are satisfied or until termination criteria are reached. after that, the resulting superpixels and merged regions can be further revised through a user-friendly graphical user interface. lastly, this segmentation is combined with the nuclear segmentation to form the final result. in this section, we use the dataset shown in fig. to describe our proposed framework. the primary steps in our framework are detailed below. fig. processing steps in the proposed framework fig. simple linear iterative clustering on multichannel microscopy with edge detection algorithm fig. superpixel merging algorithm preprocessing and nuclear segmentation the first preprocessing is to normalize each channel to the same range, e.g.,. another optional preprocessing step is to apply a noise reduction filter, e.g., median filter, to each channel in cases of noisy images. figure 2 f shows an example of a composite color image from cyan, yellow and red channels after preprocessing. in microscopy, cell nuclei are often the easiest morphological features to identify, and many microscopy protocols include nuclear staining, or more recently, fluorescent proteins genetically engineered for nuclear localization. because each fluorescent tag corresponds to one channel, we can analyze channels individually, based on this knowledge of the underlying signal sources. in other words, we can analyze the green channel for nucleus-like objects. by using image thresholding and morphological techniques, the nuclei in the dataset shown in fig. can be segmented using only the green channel as illustrated in fig. fig. nuclear segmentation. green channel after preprocessing; after image thresholding; using morphological techniques to remove noise; segmented nuclei slic-mmed for superpixel generation the original slic algorithm operates on color images in the cielab color space with one input parameter is k, the desired number of superpixels. each pixel is represented by a 5-dimensional feature vector, t, containing color components and pixel coordinates. at the initialization step, k initial cluster centroids c k=[l k,a k,b k,xk,yk]t are sampled on a regular grid with the interval of s=n/k, where n is the number of pixels. to reduce the chance of centering a superpixel on an edge or on a noisy pixel, each centroid is moved to the lowest gradient position in a 3× neighborhood. next, in the iteration step, each pixel is associated with its nearest centroid. in order to speed up the algorithm, the size of the search space is reduced to a region proportional to the superpixel size. here, for each cluster centroid c k, only the pixels in the 2s×2s region around c k are evaluated, meaning that if the distance from a pixel to c k is less than the distance from that pixel to its current associated centroid, then the associated centroid will be changed to c k. once all the pixels have been assigned to their nearest centroids, an update process adjusts each centroid to be the mean feature vector of all the pixels belonging to the corresponding cluster. in practice, repeating this iteration step times is sufficient for most images. finally, a postprocessing step assigns all disjoint pixels to nearby superpixels. in slic, the distance d between two pixels i and j is a combination of two distances, d c and d s, representing color proximity and spatial proximity, respectively, as below: dc=2+2+ ds=2+ d=dc/m2+ds/s where the compactness m is used to to weight the relative importance of color similarity versus spatial proximity. when m is large, spatial proximity is more important, and the resulting superpixels are more compact. in contrast, a small value of m leads to superpixels that are less regular in size and shape; however, since in this case color proximity is more important, the resulting superpixels follow the image boundaries more closely. since the input data in our project are multichannel microscopy images, not real-world color images, the feature vector needs to be modified. instead of using cielab color components, we use each image channel as a component of the feature vector. since the green channel represents only nuclei, it is discarded from the feature vector and processed separately as described in the previous subsection. in short, the feature vector representing each pixel in our data is t which consists of the pixel intensities in the channels cyan, yellow, red, and the pixel coordinates, respectively. in the superpixel method benchmarking, boundary recall is used to measure the fraction of the ground truth edges falling within at least two pixels of a superpixel boundary. a good superpixel segmentation should adhere to object boundaries, meaning that it should produce a high boundary recall. although slic demonstrates very good boundary recall performance for real-world color images, this is not the case for our datasets as shown in fig. 7 c. to overcome these problems, we propose to integrate an additional score d e into the pixel distance measure. d e represents the presence of edges between two pixels, suggesting the likelihood that an object boundary falls between the two points. before starting superpixel generation, an edge detection algorithm is executed to compute a value p i for each pixel i, indicating its probability of being on an edge. then the distance d e between two pixels i and j is calculated as the maximum edge probability over all the pixels lying on the line connecting pixel i and pixel j. the new distance is calculated as below. dc′=2+2+ ds=2+ de=max∀t∈linept,d′=dc′/m2+ds/s2+α×de fig. superpixel generation using different algorithms. computed edge map. the segmentation results from our method, slic-mmed; slic; vcells. white arrows indicate poorly segmented superpixels a wide range of edge detection algorithms would be appropriate for computing the edge probabilities p i, and we chose a detector based on the photometric invariance theory and tensor-based features. figure 7 a shows an edge map generated using that edge detector. note that if no edge detection were involved then the revised distance measure d ′ would reduce to the original slic distance measure d. the new algorithm is presented in fig. automatic superpixel merging after superpixel generation, all the superpixels within each muscle fiber need to be merged together to form the muscle fiber boundary. the similarity measure used in our method, to determine if two neighboring superpixels should be merged, is the chi-squared histogram distance. the χ distance between two histograms p and q is defined as χ2=12∑k2pk+qk. we represent the intensity distribution in each channel of each superpixel as a histogram and use the following formula to measure the similarity distance d c between superpixels i and j, dc=χ22+χ22+χ where h c,h y,h r are the superpixel histograms of channels cyan, yellow, and red, respectively. in addition to d c, another measure called edge strength is used for the superpixel merging decision. the edge strength e ij between superpixels i and j is defined as the average edge probability over all the pixels in superpixel i having at least one neighbor belonging to superpixel j. the d c and edge strength are then used in a series of thresholds. if superpixels i and j have similar colors, it might be because they are part of the same fiber, or it might be because they come from different fibers that happen to have similar colors. therefore, whenever two superpixels have smilar d c, they can only be merged if they have low edge strengths. in addition, we use another predefined threshold thres_size to avoid muscle fibers having unrealistic sizes which are formed from over-merging. however, this size limitation is not applied to superpixels representing “background”. our iterative superpixel merging algorithm starts with a calculation of the χ distances and the edge strengths between each superpixel and its neighbors. then the method for merging superpixels is a series of thresholded criteria as described in fig. the algorithm stops when there are no more superpixels merged. figure 8 a shows a result after this processing step. fig. segmentation using superpixels. automatic superpixel grouping; example of manual refinement: yellow box - draw a curve to merge all the regions along the curve, blue box - restore a superpixel to merge it with a neighboring region in another way; final result; ground truth manual refinement for challenging datasets, it is impossible to produce an error-free segmentation. errors from the automatic superpixel merging process include two types: over-merging: merging of superpixels from different muscle fibers, or from a muscle fiber and neighboring background/artifacts. under-merging: some neighboring superpixels from the same muscle fiber have not been merged yet. a manual refinement step is introduced to our framework through a user-friendly gui in order to fix the superpixel merging errors. the main supported operations include drawing a freehand curve to merge all the superpixels or regions along the curve. restoring the original superpixels surrounding a selected position to allow manually merging them in another way. if the superpixel generation produces a high boundary recall, using these two operations can guarantee a very good segmentation result. after this manual refinement, the superpixel segmentation is combined with the nuclear segmentation in a postprocessing step to form the final segmentation result.  superpixel evaluation we use two error metrics, boundary recall and under-segmentation error, to evaluate our slic-mmed algorithm and compare it with the original slic and another advanced superpixel algorithm named vcells. boundary recall is the fraction of ground truth edges that fall within a certain distance d of at least one superpixel boundary. a good superpixel segmentation should produce a high boundary recall. under-segmentation error compares segment areas to measure to what extend superpixels flood over the ground truth segment borders. details about the calculation of these two measures can be found in. we used the dataset in fig. for this evaluation. the corresponding ground truth was created by one computer expert under the supervison of our muscle biology expert. the compactness was chosen as for all the three algorithms. the edge weight α in eq. was for slic-mmed. the input image for the original slic and vcells algorithms was the composite color image from the cyan, yellow and red channels after preprocessing. fig. comparison of different superpixel methods using different measures: boundary recall; under-segmentation error; processing time. all algorithms were run repeatedly with the number of superpixels ranging from to with a step size of the computing platform was a ghz intel xeon cpu e3- desktop computer with gb ram running microsoft windows professional 64-bit segmentation evaluation the final stage of evaluation considers the total sementation accuracy. the methods slic and vcells provide only superpixels, not segmentation, so they are not covered in this section. our method, slic-mmed, merges superpixels to create segmented regions, so the quality of its segmentation is inherently related to the quality of its superpixels. we assess the segmentation using an absolute score, defined with respect to ground truth, called the dice similarity coefficient. the dsc measures the spatial overlap between two segmentations, x and y, and is defined as dsc=2x∩yx+y, where |x| and |y| are the number of pixels in x and y, respectively. it should be noted that eq. is for the evaluation of one resulting segment. our segmentation problem is a multiple-object segmentation with multiple fibers and other regions. we propose the median dsc which is computed as in eq. to measure the similarity between the segmentation result s and the ground truth g. meddsc=mediani=..,ngdsci,dsci=2gi∩sfgi+sf, where n g and n s are the respective total number of segments in g and s,g i is one segment in g, and s f with f∈[ n s] is the corresponding segment in s having the largest overlap with g i. for the example data set, we used slic-mmed to generate superpixels. after the automatic superpixel merging step, we manually refined the segmentation and got the final result as in fig. 8 c. the resulting meddsc was for this dataset. fig. segmentation results applying slic-mmed to the images in fig. the formats of each image channel of the data are 8-bit × pixels and 12-bit × pixels. each segmentation is displayed on the composite color image from the cyan, yellow and red channels after preprocessing we also propose dsc x, where x is a number between and to measure the fraction of segments in the ground truth are segmented with a dsc≥x/ the final segmentation in fig. 8 c has dsc of, meaning that 78% of the segments were segmented with a dsc≥. for the cross-section dataset and the lateral-section dataset in fig. the scores were dsc and, respectively. discussion and  in our experiments, the proposed method can correctly segment muscle fibers in very heterogeneous sections having both bright and dark regions, a wide range of fiber sizes, homogeneous red or yellow but more irregular cyan segments in cytoplasm. the fact that the method can handle a variety of cell sizes and morphologies in these confetti-fluorescent images suggests that it may be useful for analyzing confetti-fluorescent images in other tissues. with an accurate segmentation, we can count the number of muscle fibers that contain each of the confetti colors. the method works well for even weakly fluorescent areas, as in figs. 1 a and 10 a. using the same segmentation, we also can measure the diameter and cross sectional area for each fiber. in most labs, measuring the diameter and cross-sectional area would require cutting a set of adjacent tissue sections, which would then undergo labor-intensive staining, followed by imaging and registration of the adjacent sections, to provide a superpositioning of the stained section and the confetti-labelled section. staining requires doubling the number of sections because staining eliminates the endogenous fluorescence. in this work we show it is possible to obtain segmentation from the endogenous fluorescence, allowing us to skip the costly process of staining. in fluorescence microscopy, each fluorophore emits light over a range of wavelengths, causing nearby colors to overlap. for example, the emission spectrum of green overlaps with the emission spectrum of yellow. it is for this reason that multi-color labelling strategies have engineered the similar fluorescent proteins to have different sub-cellular localizations. localization allows the identity of the label to be disambiguated. in our images, the green signal bled into the yellow channel. slic-mmed includes explicit management of different subcellular localizations, and this may be why we had no segmentation errors due to green-yellow spectral overlap. the observed colors arise from fluorescent protein molecules that are diffusible in their compartment. the brownian nature of diffusion suggests that color distribution might be nearly uniform across the space of the relevant compartment. in other words, the limit of molecular diffusion becomes the boundary of the color, which we identify as the boundary of the segmentation. cell membranes are 2-dimensional surfaces which can appear as 1-dimensional curves when imaged from a cross-section. the cytosol and nucleus of a cell are 3-dimensional compartments, which appear as 2-dimensional continuous regions. edge detection is a natural approach for analyzing 2-dimensional membrane-targeted fluorescence, such as the cyan channel in our images. meanwhile, superpixel-based region detection methods are a natural approach for analyzing 3-dimensional compartments. if the spatial distribution of the fluorescent proteins were punctate or fibrillar, then our method would be less appropriate. our segmentation framework, slic-mmed, is a “hybrid” method that combines the advantages of a region-based clustering algorithm and an edge detector through the integrated edge map. the introduction of an user-friendly superpixel refinement module provide flexibility for the framework. as long as the superpixel generation provides a high performance in boundary recall, the framework provides very good segmentations. our experimental results show a high degree of agreement with experts. in the final scoring, the differences between different trials are also heavily dependent on the specific dataset, the number of superpixels to be revised, and the user’s expertise at performing the manual refinement step. in future, we will intensively analyze the contribution of automation to the effectiveness of the framework. the algorithm is potentially applicable to other multi-channel microscopy applications besides muscle. mouse transgenes with confetti, brainbow or other multi-color stochastic labels have become extremely popular and the scientific community is rapidly generating multi-channel images that require analysis. our image analysis method is particularly valuable for such applications. additional file additional file the matlab and c code, and the data files used in this project. we thank paul matsudaira for hosting the mouse experiments. declarations this article has been published as part of bmc systems biology volume supplement 15th international conference on bioinformatics: systems biology. the full contents of the supplement are available online http://bmcsystbiol.biomedcentral.com/articles/supplements/volume-10-supplement- funding funding for this research was provided by duke-nus srp phase research block grant and by the national research foundation, prime minister’s office, singapore, under its create programme, singapore-mit alliance for research and technology biosystems and micromechanics irg. availability of data and materials the code and datafiles for this project are available as additional file authors’ contributions method was designed and implemented by bpn with advice from ltk. mouse experiments and microscopy were performed by hh. ptcs and ltk supervised the imaging studies. all authors edited and approved the manuscript. competing interests the authors declare that they have no competing interests. consent for publication not applicable. ethics approval and consent to participate animal experiments were approved by the institutional animal care and use committee of the national university of singapore, protocol number r14- 
our framework was applied to cross-section and lateral section images of regenerated muscle fibers from confetti-fluorescent mice. superpixelsegmentationmuscle fibersconfetti fluorescencemulti-channel microscopy15th international conference on bioinformatics incob 2016queenstown, singapore 21- september http://incob apbionet.org/issue-copyright-statement© the author 2016 our segmentation framework is flexible and provides very good segmentations of multi-color muscle fibers. we propose to integrate an edge detector into a superpixel algorithm and customize the method for multi-channel images. this paper addresses the difficulties and presents a superpixel-based framework for segmentation of regenerated muscle fibers in mice. the enhanced superpixel method outperforms the original and another advanced superpixel algorithm in terms of both boundary recall and under-segmentation error. one difficulty of such strategies is segmenting the cell boundaries, which is a very different problem from segmenting color images from the real world. confetti fluorescence and other multi-color genetic labelling strategies are useful for observing stem cell regeneration and for other problems of cell lineage tracing.
 confetti fluorescence and other multi-color genetic labelling strategies are useful for observing stem cell regeneration and for other problems of cell lineage tracing. one difficulty of such strategies is segmenting the cell boundaries, which is a very different problem from segmenting color images from the real world. this paper addresses the difficulties and presents a superpixel-based framework for segmentation of regenerated muscle fibers in mice.  we propose to integrate an edge detector into a superpixel algorithm and customize the method for multi-channel images. the enhanced superpixel method outperforms the original and another advanced superpixel algorithm in terms of both boundary recall and under-segmentation error. our framework was applied to cross-section and lateral section images of regenerated muscle fibers from confetti-fluorescent mice. compared with “ground-truth” segmentations, our framework yielded median dice similarity coefficients of and higher.  our segmentation framework is flexible and provides very good segmentations of multi-color muscle fibers. we anticipate our methods will be useful for segmenting a variety of tissues in confetti fluorecent mice and in mice with similar multi-color labels. electronic supplementary material the online version of this article contains supplementary material, which is available to authorized users. keywords superpixelsegmentationmuscle fibersconfetti fluorescencemulti-channel microscopy15th international conference on bioinformatics incob 2016queenstown, singapore 21- september http://incob apbionet.org/issue-copyright-statement© the author 2016 