BACKGROUND
the widespread use of automated florescent confocal microscopy has resulted in a significant role for image analysis in modern quantitative biology.
quantitative features such as the number of cells or fluorescent intensity of subcellular organelles have become crucial for the elucidation of many biological and pharmaceutical hypotheses ranging from cell biology to anticancer drug development in various organisms such as caenorhabditis elegans  <cit> , drosophila melanogaster  <cit>  and even rodent models  <cit> .
with the advent of three-dimensional  optical sectioning of confocal microscopes and green fluorescent protein  as an expression marker  <cit> , spatial distribution of cellular organelles can be studied.
histone tagged with fluorescent protein   <cit>  allow the observation of dna distribution in living cells.
recent innovations in light sheet microscopy enabled the study of the spatiotemporal organisation of nuclei in whole zebrafish and drosophila embryos  <cit> .
a vast amount of visual information is acquired in automated microscopy.
some of the extracted features are less sensitive to the precision of segmentation, such as the number of objects and their location based on centroid coordinates.
other, biologically important features such as shape or volume require a more precise segmentation.
mechanisms of cell cycle regulation can be elucidated by live cell imaging and subsequent automated quantification of nuclei in intact organisms  <cit> .
the living drosophila embryo provides an attractive experimental system for the study of mitosis, where nuclei can be observed in situ  <cit> .
neurodegeneration is another biological phenomenon of intense interest that has been subjected to extensive study in drosophila models, but for which there are few quantitative cell biological readouts.
generalized brain neurodegeneration has been studied in blue cheese  drosophila mutants  <cit> , where ubiquitinated protein accumulation and a failure of degradative trafficking pathways have been implicated  <cit> .
kumarasamy et al.
<cit>  determined by quantitative automated multivariate analysis of wide field fluorescence images that the degenerative phenotype was accompanied by changes in the size and distribution of lysosomal compartments within neuronal termini.
loss of motor neurons has been documented in the third instar larval nervous system of bchs mutants, as well as superfical observations of smaller ventral ganglion size by confocal microscopy  <cit> .
image segmentation is an important step in the image processing workflow that is extensively applied in fluorescence microscopy.
during segmentation foreground pixels are separated from background pixels.
the use of machine segmentation  in automatic image cytometry enables the measurement of cellular features in a high throughput fashion.
however an automated imaging workflow cannot fully supplant the expertise of a trained biologist to detect and evaluate phenotypes.
in a previous report, poorly-segmented cells were identified by eye in the framework of a high-content screening imaging pipeline  <cit> .
each of the authors of the report independently reviewed an equal fraction of the test image set, classifying into well-segmented and poorly-segmented qualitative groups using subjective criteria.
segmentation performance evaluation is still not common in cell-based high-content screening.
subjective descriptive terms such as "reasonably conformed perimeter" can serve well to train classifiers evaluating segmentations qualitatively and find features resistant  or prone  to imprecise cellular segmentation  <cit> .
besides "good" and "poor" segmentation, a quantitative evaluation can answer questions such as "how good" or "how poor" a machine segmentation algorithm is.
the first step towards achieving such a quantitative evaluation is building a segmentation dataset that contains only well-segmented objects.
performance evaluation methods can be divided into analytical and empirical groups, where the former investigates the algorithm directly and the latter judges it through the quality of the image segmentation.
the empirical discrepancy method uses an ideal or expected segmentation result to objectively quantify the performance of an algorithm  <cit> .
this concept takes into account the difference between an automatically segmented and a reference image and is generally used for practical image processing performance evaluation in real applications, where the accuracy of segmentation is the primary concern  <cit> .
this approach produces easily interpretable results and is useful to perform a quantitative comparison of different segmentation algorithms.
however, such a reference, or ground truth  dataset creation is generally considered as a labour-intensive step  <cit> , where human intuition or judgment makes an expected objective evaluation to be influenced by subjective factors.
the gt is defined as a reference data set that acts as the gold standard in segmentation evaluation.
in the context of image segmentation, the gt can be represented in various formats that are created manually or semi-automatically by human experts.
contours represent regions of interest, while labelled 2d and 3d images are the most comprehensive format that include all pixels and voxels of detected image objects.
in specialised applications such as nuclear segmentation, the gt is encoded in the form of centroids  <cit> .
a designer of an image analysis system has an array of often task-specific machine segmentation choices, where the gt is the "correctly" segmented image, which is needed for objective numerical evaluation of those algorithms.
since there is no unique universal segmentation ground truth, against which machine segmentation results can be compared, a human expert must create a perceptually consistent gt.
currently, there is no dedicated software tool to evaluate segmentation quality.
a number of programs have been developed to segment and visualise 3d optical image stacks automatically or interactively.
the potential software packages ranges from the commercially available imaris , definiens developer xd  and matlab  to open source alternatives as cellprofiler  <cit> , the segmentation editor plugin under imagej  <cit> /fiji  <cit>  and itk-snap  <cit> .
however, the use of these applications are limited in 3d segmentation performance metrics  <cit> .
here, we present a software for manual 3d segmentation and segmentation performance evaluation: gebiss .
gebiss was developed as an imagej  <cit>  plugin.
we used gebiss to assess the neurodegenerative state of drosophila mutants by measuring brain volumes.
such a phenotypic readout would permit us to test the effects of pharmacologic or genetic interventions that may affect the severity of the phenotype.
gebiss  <cit>  was applied to the analysis of over  <dig> single images from different 3d image datasets.
implementation
gebiss was developed as an imagej plugin to help the biologist to generate a ground truth.
the source code availability, platform independence and wide developer base made nih imagej  <cit>   an optimal environment for a ground truth creation and benchmarking application.
gebiss leverages imagej's core capabilities such as easy installation, opening and saving in a wide range of image formats, such as image cytometry standard ics  <cit> , uncompressed and zip-compressed tiff stack or zeiss lsm by the plugin ome loci bio-formats  <cit> .
the graphical user interface of gebiss  was developed using the swing toolkit for java and its simple design guides the user along the workflow.
formerly an unsettled issue, memory allocation is not limited anymore.
both the current versions of java and imagej are able to handle 64-bit platforms and >  <dig> gb ram.
besides imagej, gebiss requires a java3d installation for spatial visualisation.
this can be done easily by following the steps of benjamin schmid's guide.
an alternative to the standard imagej is fiji  <cit>  which contains java3d as part of a package.
the gebiss installation itself consists of downloading gebiss_.jar and biiimagej3dviewer.jar from http://imaging.bii.a-star.edu.sg/projects/gebiss/ and copying those into the "plugins" folder of imagej or fiji, after which a "gebiss" submenu appears automatically in the plugins menu.
gebiss uses jarek sacha ij-plugins toolkit that can be freely downloaded from http://ij-plugins.sourceforge.net/ or version  <dig> .
<dig> can be found at the project web folder that must be copied into the "plugins" folder of imagej or fiji.
gebiss is run after an  <dig>  or 16-bit greyscale microscopic image stack is opened.
the spatial voxel dimensions can be imported either automatically from an ics, tiff, or lsm stack header, or otherwise can be set manually in imagej.
gebiss provides a function to set and save voxel depth enabling the storage and repeated retrieval of a value over several imaging sessions.
the methodology is illustrated with examples of live, wild type drosophila melanogaster embryonic nuclei monitored with gfp histone h2a variant  in various phases of the cell cycle.
the confocal microscopic image stacks of the anterior part of the embryo were captured by a zeiss lsm  <dig> duo microscope.
gebiss workflow
the generation of a labelled ground truth image stack in gebiss can be achieved through: i) segmenting each 2d roi of a spatial object, ii) 3d segmentation of a spatial object or segmenting a group of spatial objects .
the labelled gt stack can be saved and used to benchmark an arbitrary machine segmentation.
each original microscopic image stack is smoothened by applying a  <dig> pixel radius  two dimensional median filter.
2d slice-based gt creation workflow
in the course of a slice-based segmentation , each optical slice of a spatial object can be segmented individually using different threshold values if necessary.
the resulting rois can effectively exclude otherwise merged segmentation artefacts.
inspired by michael m. miller's segmentingassistant plugin and using imagej's core wand tool class  <cit> , each roi is defined by four parameters: startx, starty, a minimum  and a maximum intensity  threshold value.
while segmentingassistant makes 2d local segmentation much easier and precise compared with a manual "freehand selection", there was room for improvement.
for example, sliders are not perfect for startx, starty selection in order to assign several points in an image.
a slider is not optimal to set θmin, when the typical step is too small to be set by dragging the slider.
also, the parameter load and save functions were missing.
earlier the binary mask and labelled stack creation was complicated, as it required the combined use of roi manager and the 3d object counter  <cit>  plugin.
several features were built in gebiss to improve productivity.
startx and starty coordinates can be assigned by a single mouse click on the original image and θmin can be changed by rolling the mouse middle scroll wheel in intensity steps of  <dig> and  <dig> when combined with shift or ctrl key respectively.
the parameter θmax is set to the brightest intensity value of the image by default, and requires adjustment in rare cases.
the algorithm searches in the right direction until a pixel is found with an intensity falling into the range of .
following that, it traces the contour of the 2d object in a clockwise direction.
however, the resulting roi is calculated by using the median filtered intensity values, which are invisible to the user.
superimposed on the original image, users are guided with continuous visual feedback as the contours are updated.
using the gui of the slice-based gt specification tab , the optimal startx, starty, θmin and θmax values are stored in a parameter database for each object, where they can be updated, deleted or invoked to create a binary and a labelled stack.
the 2d slice-based gt creation workflow is flexible enough to allow the use of multiple threshold values per 3d object if needed.
at the same time, the reuse of the stored parameters of an object in the present optical slice enables faster processing of the consecutive slice.
the latter approach assumes that the startx, starty and θmin parameters of the object roi in the previous slice gives a correct contour.
if not, the values can be adjusted to fit on the given contour.
3d object-based gt creation workflow
gebiss is able to visualise 3d-thresholded foreground voxels as an isosurface superimposed on background voxels with original intensity.
this double rendering feature can be switched on by checking in the add isosurface submenu in the view menu of gebiss, which appears when the tab is activated.
after an image stack is opened with imagej and gebiss is started, 3d connected objects can be segmented interactively  by using the tab "object-based gt specification" .
each 3d object is segmented individually by a seeded 3d region growing algorithm  <cit>  originally implemented as ij-plugins toolkit version  <dig>  by jarek sacha.
the connected threshold region growing segmentation algorithm requires five parameters specified by the user: the x, y, z coordinates of a seed pixel as well as a minimum  and a maximum intensity threshold value.
following the right side section of the flowchart in figure  <dig>  the seed pixel coordinates are defined on the image stack window by a single ctrl plus left mouse key click on a bright fluorescent region  of a given object.
the θmin is defined immediately after the seed selection.
to provide the user with 3d visual information to find the optimal θmin, a customized version of benjamin schmid's imagej 3d viewer plugin  <cit>  was implemented into gebiss.
in order to simultanously visualise background and foreground, gebiss superimposes two 3d renderings.
background voxels are visualised by texture-based volume rendering using brightness-corrected fluorescent intensities.
foreground voxels are added forming a semi-transparent, red coloured isosurface mesh that allows the user to observe simultanously both the interior and the exterior of the object.
the default maximum intensity threshold value is  <dig> for images with a dark background and a bright foreground.
the user selects an optimal θmin value in the "adjust threshold..." window .
in the case of drosophila embryos such a θmin value is low enough to include all heterochromatin regions and chromosome arms but high enough to exclude free histone and lipid droplets.
by dragging the threshold slider, a 10- <dig> slice thick 3d mesh of a typical nucleus is rendered.
the image can be freely rotated in 3d and zoomed using the mouse and the middle wheel.
the whole rotating 3d virtual environment can be recorded as an animated movie using imagej's avi writer plugin.
an automated light attenuation compensation in gt creation is offered by a dual thresholding function implemented under the tab "object subset gt specification".
there are cases where the signal-to-noise ratio would allow the segmentation of a stack with a global threshold, though the light attenuation requires the use of a higher threshold value for deeply located objects.
in this module two different thresholds are applied to objects located in shallower and deeper axial depth.
the user is prompted for a data file containing the x, y, z seed coordinates of each object, a threshold value for the shallower and a second threshold value for the deeper objects as well as the demarcation z slice number that separates the shallow and deep slices.
the x, y, z seed data file is generated using the 3d object counter  <cit>  plugin.
even if its global thresholding segmentation produces imprecise contours in such cases, the derived object centroids are saved in a data file that serves as an x, y, z seed input file.
this gebiss module applies a 3d region growing segmentation using shallower or deeper θmin to the objects according to their z centroid value.
in practice it is done as follows.
a separation slice is set up at a certain depth.
all nuclei are segmented by a 3d region growing algorithm.
those nuclei whose centroid's z parameter  is above the separation slice are segmented by a user-determined threshold value.
those nuclei whose centroid's z parameter is below the separation slice are segmented by a different threshold value.
gt contour visualisation
the requirement of human supervision for gt creation in any system may require an optional high level check.
fast and precise visualisation is achieved by the superimposition of all gt roi contours on the original images.
an imagej macro was created which uses a binary gt and original microscopic greyscale stacks as input files .
the binary stack must be inverted , and the original greyscale stack must be converted to rgb format.
the macro automatically draws all segmented object contours on each of the original slices, visualising both the foreground and background pixels , therefore any missing 2d roi or 3d object can be detected at the object level.
at pixel level, false negatives such as unsegmented chromosome arms and false positives such as attached free histones or lipid droplets can be recognised easily.
any further gt roi adjustment can be done by the respective gebiss steps.
the isosurface 3d rendering of a labelled stack can reveal falsely merged gt objects.
benchmarking workflow
for segmentation performance evaluation, the most useful measures are precision, recall  and f-measure.
all of those measures need the quantification of true positive , false positive  and false negative  class labels defined by the four outcomes of the relation between the predicted class and the actual class.
the value of precision is calculated as , thus it depends on the number of false positives.
fewer false positives result in a precision value closer to  <dig>  similarly, the number of false negatives affect the recall value, that is calculated as , which is closer to  <dig> when the number of false negatives is low.
the f-measure is expressed as , and it evaluates the performance of a machine segmentation in a single value as the harmonic mean of precision and recall.
the closer the f-value is to  <dig>  the better the given ms is.
a ground truth segmentation contains neither any false positive nor any false negative pixel, therefore pgt =  <dig>  rgt =  <dig> and fgt =  <dig>  in the context of segmentation, the performance evaluation can be quantified at two levels: object level and pixel level.
at the object level, the object number of ms and its deviation from that of the gt is calculated.
the "object" term refers to either a 2d foreground region of interest  composed of  <dig> or 8-connected pixels or a 3d body composed of  <dig> or 26-connected foreground voxels.
ideally, each gt object matches an ms object resulting in a one-to-one correspondence.
the machine segmentation may result in a false positive object that does not occur in gt or a false negative, missing one that does occur in gt.
an ms object may be split  if more than one ms object matches a gt object or merged  if more than one gt object matches an ms object.
at pixel level, the correspondence between the foreground and the background region is quantified.
each segmented pixel or volumetric pixel  can be classified as true positive, false positive, true negative  or false negative.
the number of tp, fp, tn and fn pixels are counted, and precision, recall and f-measure are calculated for a 3d stack or a 2d slice.
the benchmarking module requires a labelled ms and a labelled gt stack as input files.
it has a numerical output  and a visual output that is shown in additional files  <dig> and  <dig> .
the numerical output displays benchmark measures for the whole stack as well as each individual slice.
for the whole stack, the number of gt and ms objects are indicated at object level.
several well-established performance metrics  <cit>  are in use , that are derived from the confusion matrix.
the precision, recall and f-measure are calculated at the pixel level.
for each individual slice, the module calculates the number of gt and ms objects as well as the number and label of merged and split objects .
this feature enables the precise identification of the slice where roi merge occurs.
the number and label of occurrent false positive ms and false negative gt objects are also calculated.
RESULTS
gebiss is a tool for semi-automated 3d image segmentation which can be used either as a ground truth in performance evaluation or in image quantification.
we applied it to different biological datasets.
a good example is the process of nuclear division in the early drosophila embryo, which has been extensively described.
precise nuclear segmentation is critical in quantifying this aspect of embryonic development.
contour finding in nuclear segmentation can be challenging as lipid droplets surround and attach to nuclei.
the condensed chromosomes are often surrounded by a cloud of free histone that impedes the segmentation of a specific surface.
this problem also applies to all deeply located nuclei in a stack where laser attenuation leads to poor signal-to-noise ratio.
additionally, the contrast of a whole image stack may be low owing to the uniform intensity of foreground and background pixels.
gebiss offers the biologist diverse ways to overcome these obstacles.
case study 1: nucleus segmentation in embryogenesis
2d segmentation of embryonic nuclei
one of the objectives of this study is measuring the volume of nuclei.
to measure the precise nuclear volume, segmentation must correctly separate an attached lipid droplet from a discrete one that locates in the close proximity of a nucleus.
gebiss can be first used to measure the volume of nuclei marked by h2av-gfp.
to quantify nuclear features like size, shape, number, etc., it is necessary to separate the two classes of nuclear histone and cytoplasmic histone.
a lipid droplet-specific marker, such as nile red dye  <cit> , effectively allows lipid droplet segmentation and separation from the nuclei, but using the less organelle-specific h2av-gfp it is still possible to create a precise ground truth segmentation.
the top row of figure  <dig> illustrates the 2d slice segmentation of a nucleus optical section from the original image  through a contour resulting from an incorrect  and a correct  θmin value.
figure 3d shows the final segmented image of the nucleus.
decreasing the θmin value for a whole 3d object is usually not helpful to detach a lipid droplet artefact because 3d shrinking leads to the loss of the correct spherical shape of an interphase nucleus.
consequently, we disconnected a lipid droplet by the θmin reduction of only certain 2d nuclear slices while we preserved the correct nuclear shape.
the h2av-gfp intensity gradient is often low between a distinct lipid droplet and a nucleus before the histone assembles into the chromatin .
this complicates the 3d gradient-based segmentation.
the intensity variation inside a nucleus can exceed that of the variation between the nucleus and a distinct lipid droplet nearby .
the repeated use of a certain parameter set in multiple roi slices allows accelerated gt creation, however it points out the intrinsic limitations of the method when used for segmentation of a spatial object.
certain rois can overlap each other and can be omitted inadvertently as a result of parameter reuse.
since only one slice is shown at a time and the thresholding parameter can be adjusted arbitrarily, the resulting contour can be flickery over slices of low contrast stacks.
finding the optimal thresholds of the top and bottom slices and the contours can be difficult in such cases.
3d segmentation alleviates those shortcomings.
to determine whether machine segmentation artefacts cause remarkable changes in measured volumes, dividing nuclei were traced throughout mitosis.
the high h2av-gfp intensity gradient inside a prophase nucleus led to unfilled holes in segmenting a syncytial blastoderm nucleus with ms , resulting in different volume measurement between gt and ms segmentation.
3d nucleus segmentation to detach free histone
the individual and manual specification of several thousand roi contours of a segmented image stack is a flexible, but slow and laborious method, prone to human error.
3d object-based segmentation speeds up the process and the user can semi-automatically segment a whole object by using a single threshold.
a typical nucleus spreads over ~ <dig> to ~ <dig> slices, thus by choosing a 3d object-level θmin many rois can be segmented simultaneously.
selecting an optimal θmin is the main user task after the x, y, z coordinates of a seed pixel are chosen.
the middle row image series of figure  guides the viewer through the process via the 3d segmentation of a metaphase nucleus.
the volume-rendered 3d view  shows a heterochromatin region surrounded by free histone from below that has been spuriously attached because of an incorrectly low θmin value .
the optimal θmin removes the free histone and at the same time preserves the shape of the mitotic nucleus , resulting in the correctly segmented optical slices .
as a result, nuclei are segmented in interphase and various mitotic phases such, as prophase, metaphase, anaphase a, anaphase b and telophase, respectively .
this module also enables the user to distinguish between axial and lateral 3d object fusions .
segmentation of a low contrast embryonic image stack
the brightly stained central yolk mass of the drosophila embryo is still overlapping with the periplasm during nuclear cycle  <dig> in the syncytial blastoderm, which lasts until the the depth of yolk-free periplasm increases dramatically at the expense of the central yolk region in nuclear cycle  <dig>  <cit> .
as a result, the contrast of a whole image stack becomes low owing to the uniform intensity of foreground and background pixels .
global thresholding  gives visually unacceptable segmentation results.
the individual 3d object-based gt specification is a segmentation method that was able to remove the noise around the nuclei  and create a labelled image stack .
case study 2: 3d segmentation of drosphila brain
in order to provide a quantitative measurement of the strength of phenotypes resulting from bchs gain-of-function or loss-of-function mutations, lim and kraut first made use of the percentages of larval rp <dig> motor neuron loss through the immunostaining of gfp-expressing rp <dig> motor neurons  <cit> .
however, we sought to make the phenotypic measurement faster and more quantitative.
therefore in this study, we have measured the brain volumes of different late third instar bchs mutant larvae  by labelling the dissected whole larval brain with rhodamine-conjugated phalloidin, which recognizes f-actin.
the bchs <dig> allele is a nonsense mutation that encodes for a truncated protein, while dfclot <dig> is a deficiency on the left arm of chromosome  <dig>  as shown in table  <dig>  both genotypes of bchs58/dfclot <dig> and homozygous bchs <dig> have a 15% to 17% reduction in the brain volumes of the third instar larvae, when compared with the brains of the larvae from the genetic background of the bchs <dig> allele mutants .
using gebiss we show that both bchs null mutant combinations of bchs58/dfclot <dig> and homozygous bchs <dig> have a reduction in the brain volumes of third instar larvae, when compared with the background genotype yw brains as a control.
athe difference in brain volume of genotype  <dig> in comparison with genotype  <dig> as the reference control.
na: non-applicable.
this reduction in bchs mutant brain volume is statistically significant  as compared with the yw control and it is in agreement with our previous phenotypic quantification method of rp <dig> motor neuron loss  <cit> .
in addition, the smaller reduction in bchs mutant brain volume at the third instar larval stage in comparison with an estimated 40% reduction in adult bchs mutant brain volume  <cit>  may be explained by the longer duration of the adult stage which allows progressive neurodegeneration to occur.
from this brain volumetric analysis, there is no significant difference between the two different bchs null alleles, bchs58/dfclot <dig> and homozygous bchs <dig>  indicating that the bchs <dig> allele is, as expected from earlier studies  <cit> , acting as a null allele with respect to bchs' effect on brain volume.
the reduction in brain volume of bchs null larvae is consistent with the earlier observation made in adult animals  <cit>  and suggests that the overall level of degeneration in the larval brain can be assessed using the gebiss method of volumetric analysis.
conversely, over-expression of the bchs using the ep <dig> line in conjunction with the c <dig> gal <dig> driver in the central nervous system resulted in an increase of  <dig> % brain volume when compared with the control, c155/yw.
together with the observed shrinkage in brain volume of loss of function in bchs mutants, these results suggest that bchs may have some role in determining cellular volume and/or proliferation in the brain.
segmentation performance evaluation
to test the performance of 3d segmentation methods, we applied six different automated ms algorithms to segment drosophila brain stacks: yen's method  <cit> , rényi entropy  <cit> , li's minimum cross entropy  <cit> , huang's fuzzy thresholding  <cit> , ridler and calvard's iterative isodata method  <cit>  and otsu's thresholding  <cit>  .
the segmentation result image stacks can be found in additional file 6: "visual segmentation performance evaluation".
all algorithms are implemented under imagej by gabriel landini, and compute a global threshold from the stack image to segment the stack, with the limitation that only 8-bit image stacks are supported.
the advantage is that no manual parameter entry is required.
the computed threshold is lower, equal to or higher than that of ground truth.
therefore, the segmented brain stacks resulted in greater, equal to or smaller volumes, respectively.
ms volumes greater than gt lack any false negative pixels, thus leading to r =  <dig> recall in the case of li and huang segmentation.
conversely, ms volumes smaller than gt lack any false positive pixels, thus resulting in p =  <dig> precision at yen, rényi, isodata and otsu methods.
the 8-bit image stacks had histograms shifted to darker voxel intensities.
we used the global threshold value  <dig> to create gt .
however, global threshold values between  <dig> and  <dig> gave f-measures  <dig>  and above, indicating that the brain contour was not sharp.
huang's fuzzy thresholding belongs to the algorithms that uses shannon's entropy function and segments based on attribute similarity.
measuring the similarity between the original and the binary image stack, it determined the threshold by minimising the measure of fuzziness.
including average foreground and background intensities, the algorithm computed a low ms threshold value in our dark stacks, resulting in numerous false positive voxels and the poorest mean f-measure, which is still above  <dig> .
although widely used popular methods in imaging, the performance of the clustering-based segmentation algorithms of isodata and otsu performed less well than the others.
both methods assume that a stack has two grey intensity maxima and partition the histogram of an image stack into two classes, based on intra-class variance minimisation and inter-class variance maximisation.
the otsu method searches for the optimal threshold globally, whereas the isodata does this locally, thus these methods result in almost identical thresholds.
these algorithms work optimally with a bimodal histogram, where the number of voxels are similar in both the foreground and the background class.
one reason for the weaker segmentation performance is that the number of black background voxels affect the threshold computed by those two algorithms.
also the background and foreground intensity modes are not separated sharply in the histograms of our brain stacks.
among the tested ms algorithms, the three entropy-based segmentation methods performed the best.
li's method is based on cross-entropic thresholding.
it has a significantly reduced computational requirement compared with the exhaustive search method but it tends to calculate a lower threshold value than that of gt.
we found that yen's method and rényi's entropy performed the best among the algorithms that were tested.
the two algorithms are similar and performed similarly well.
both belong to shannon-entropy maximisation-based segmentation methods  <cit> , originating from one-dimensional entropic thresholding introduced by pun in  <dig>  <cit> .
CONCLUSIONS
in this paper, we present gebiss, a new software for quantitative 3d segmentation performance evaluation.
gebiss was designed to be a productive and user friendly tool for ground truth creation, and it includes a benchmarking module that objectively evaluates a 3d segmentation.
the package was developed as a plugin for imagej, is platform independent and can be freely downloaded from http://imaging.bii.a-star.edu.sg/projects/gebiss/.
gebiss was successfully used in various biological tasks.
availability and requirements
project name: gebiss
project home page: http://imaging.bii.a-star.edu.sg/projects/gebiss/
operating system: platform independent
programming language: java
other requirements: imagej  <dig>  m or higher, java  <dig> .0_ <dig> or higher
license: gnu gplv3
authors' contributions
jkv carried out the design and implementation of the software, analysed the images and wrote the manuscript.
nwt participated in the implementation of the software, clp coordinated the implementation of the software, kcy participated in the implementation of the software, jspl carried out the drosphila brain experiment, rk conceived and coordinated the drosphila brain experiment, mw conceived and coordinated the study.
all authors read and approved the final manuscript.
supplementary material
additional file 1
imagej macro for gt contour visualization.
imagej macro file in unix text format.
before running, it requires two input image stacks to be opened.
an original image stack, converted into rgb format and renamed as "orig" as well as its matching binary stack renamed as "bin" with black foreground and white background pixels.
for the macro operation see the text.
click here for file
additional file 2
3d gt contour visualization.
compressed, spatially calibrated rgb image stack that can be opened by standard imagej.
gt segmentation contours  are superimposed on the original, low contrast image slices.
by simultaneously visualising both the foreground and background pixels, the user can easily check the ground truth segmentation.
the stack's xyz dimensions are  <dig>  μm ×  <dig>  μm ×  <dig>  μm.
click here for file
additional file 3
numerical output of the gebiss benchmarking module.
quantitative machine segmentation benchmark results demonstrated on a low contrast image stack of a drosophila embryo in syncytial blastoderm developmental stage, containing  <dig> slices.
data in columns represents the evaluation of each optical slices.
the slice numbers are indicated in the table header, followed by measures of each individual slices respectively: number of gt and ms objects , number and labels of merged and split objects , number and labels of false positive and false negative ms objects .
click here for file
additional file 4
visual output of the gebiss benchmarking module.
compressed, spatially calibrated image stack.
it can be opened using imagej.
it horizontally combines the ms  and the gt  32-bit labelled stacks in an easily comparable manner.
demonstrated on a low contrast image stack containing  <dig> slices.
click here for file
additional file 5
nuclei in various phases.
six animated gif movies in one compressed file.
after uncompressing, those can be opened by imagej or an internet browser.
the movies show nuclei of a drosophila embryo in postcellular blastoderm developmental stage.
interphase volume  =  <dig>  μm <dig>   prophase v =  <dig>  μm <dig>   metaphase v =  <dig>  μm <dig>   anaphase a v =  <dig>  μm <dig>   anaphase b v =  <dig>  μm <dig>   telophase v =  <dig>  μm <dig>
click here for file
additional file 6
visual segmentation performance evaluation.
zip compressed, spatially calibrated, segmented binary image stacks, that can be opened by standard imagej.
the images represent the 3d segmentation results discussed in the section "segmentation performance evaluation".
file names indicate the machine segmentation that resulted the given stack.
click here for file
acknowledgements
the authors thank wee choo puah for the drosophila embryo image acquisition, chinta rambabu for his constructive comments, derek smith and daniel bosch ibáñez for their critical revision of the manuscript, wayne rasband for imagej and all the developers of the imagej plugins that were used in this paper.

 the widespread use of automated florescent confocal microscopy has resulted in a significant role for image analysis in modern quantitative biology. quantitative features such as the number of cells or fluorescent intensity of subcellular organelles have become crucial for the elucidation of many biological and pharmaceutical hypotheses ranging from cell biology to anticancer drug development in various organisms such as caenorhabditis elegans, drosophila melanogaster and even rodent models. with the advent of three-dimensional optical sectioning of confocal microscopes and green fluorescent protein as an expression marker, spatial distribution of cellular organelles can be studied. histone tagged with fluorescent protein allow the observation of dna distribution in living cells. recent innovations in light sheet microscopy enabled the study of the spatiotemporal organisation of nuclei in whole zebrafish and drosophila embryos. a vast amount of visual information is acquired in automated microscopy. some of the extracted features are less sensitive to the precision of segmentation, such as the number of objects and their location based on centroid coordinates. other, biologically important features such as shape or volume require a more precise segmentation. mechanisms of cell cycle regulation can be elucidated by live cell imaging and subsequent automated quantification of nuclei in intact organisms. the living drosophila embryo provides an attractive experimental system for the study of mitosis, where nuclei can be observed in situ. neurodegeneration is another biological phenomenon of intense interest that has been subjected to extensive study in drosophila models, but for which there are few quantitative cell biological readouts. generalized brain neurodegeneration has been studied in blue cheese drosophila mutants, where ubiquitinated protein accumulation and a failure of degradative trafficking pathways have been implicated. kumarasamy et al. determined by quantitative automated multivariate analysis of wide field fluorescence images that the degenerative phenotype was accompanied by changes in the size and distribution of lysosomal compartments within neuronal termini. loss of motor neurons has been documented in the third instar larval nervous system of bchs mutants, as well as superfical observations of smaller ventral ganglion size by confocal microscopy. image segmentation is an important step in the image processing workflow that is extensively applied in fluorescence microscopy. during segmentation foreground pixels are separated from background pixels. the use of machine segmentation in automatic image cytometry enables the measurement of cellular features in a high throughput fashion. however an automated imaging workflow cannot fully supplant the expertise of a trained biologist to detect and evaluate phenotypes. in a previous report, poorly-segmented cells were identified by eye in the framework of a high-content screening imaging pipeline. each of the authors of the report independently reviewed an equal fraction of the test image set, classifying into well-segmented and poorly-segmented qualitative groups using subjective criteria. segmentation performance evaluation is still not common in cell-based high-content screening. subjective descriptive terms such as "reasonably conformed perimeter" can serve well to train classifiers evaluating segmentations qualitatively and find features resistant or prone to imprecise cellular segmentation. besides "good" and "poor" segmentation, a quantitative evaluation can answer questions such as "how good" or "how poor" a machine segmentation algorithm is. the first step towards achieving such a quantitative evaluation is building a segmentation dataset that contains only well-segmented objects. performance evaluation methods can be divided into analytical and empirical groups, where the former investigates the algorithm directly and the latter judges it through the quality of the image segmentation. the empirical discrepancy method uses an ideal or expected segmentation result to objectively quantify the performance of an algorithm. this concept takes into account the difference between an automatically segmented and a reference image and is generally used for practical image processing performance evaluation in real applications, where the accuracy of segmentation is the primary concern. this approach produces easily interpretable results and is useful to perform a quantitative comparison of different segmentation algorithms. however, such a reference, or ground truth dataset creation is generally considered as a labour-intensive step, where human intuition or judgment makes an expected objective evaluation to be influenced by subjective factors. the gt is defined as a reference data set that acts as the gold standard in segmentation evaluation. in the context of image segmentation, the gt can be represented in various formats that are created manually or semi-automatically by human experts. contours represent regions of interest, while labelled 2d and 3d images are the most comprehensive format that include all pixels and voxels of detected image objects. in specialised applications such as nuclear segmentation, the gt is encoded in the form of centroids. a designer of an image analysis system has an array of often task-specific machine segmentation choices, where the gt is the "correctly" segmented image, which is needed for objective numerical evaluation of those algorithms. since there is no unique universal segmentation ground truth, against which machine segmentation results can be compared, a human expert must create a perceptually consistent gt. currently, there is no dedicated software tool to evaluate segmentation quality. a number of programs have been developed to segment and visualise 3d optical image stacks automatically or interactively. the potential software packages ranges from the commercially available imaris, definiens developer xd and matlab to open source alternatives as cellprofiler, the segmentation editor plugin under imagej /fiji and itk-snap. however, the use of these applications are limited in 3d segmentation performance metrics. here, we present a software for manual 3d segmentation and segmentation performance evaluation: gebiss. gebiss was developed as an imagej plugin. we used gebiss to assess the neurodegenerative state of drosophila mutants by measuring brain volumes. such a phenotypic readout would permit us to test the effects of pharmacologic or genetic interventions that may affect the severity of the phenotype. gebiss was applied to the analysis of over single images from different 3d image datasets. implementation gebiss was developed as an imagej plugin to help the biologist to generate a ground truth. the source code availability, platform independence and wide developer base made nih imagej an optimal environment for a ground truth creation and benchmarking application. gebiss leverages imagej's core capabilities such as easy installation, opening and saving in a wide range of image formats, such as image cytometry standard ics, uncompressed and zip-compressed tiff stack or zeiss lsm by the plugin ome loci bio-formats. the graphical user interface of gebiss was developed using the swing toolkit for java and its simple design guides the user along the workflow. formerly an unsettled issue, memory allocation is not limited anymore. both the current versions of java and imagej are able to handle 64-bit platforms and > gb ram. besides imagej, gebiss requires a java3d installation for spatial visualisation. this can be done easily by following the steps of benjamin schmid's guide. an alternative to the standard imagej is fiji which contains java3d as part of a package. the gebiss installation itself consists of downloading gebiss_.jar and biiimagej3dviewer.jar from http://imaging.bii.a-star.edu.sg/projects/gebiss/ and copying those into the "plugins" folder of imagej or fiji, after which a "gebiss" submenu appears automatically in the plugins menu. gebiss uses jarek sacha ij-plugins toolkit that can be freely downloaded from http://ij-plugins.sourceforge.net/ or version. can be found at the project web folder that must be copied into the "plugins" folder of imagej or fiji. gebiss is run after an or 16-bit greyscale microscopic image stack is opened. the spatial voxel dimensions can be imported either automatically from an ics, tiff, or lsm stack header, or otherwise can be set manually in imagej. gebiss provides a function to set and save voxel depth enabling the storage and repeated retrieval of a value over several imaging sessions. the methodology is illustrated with examples of live, wild type drosophila melanogaster embryonic nuclei monitored with gfp histone h2a variant in various phases of the cell cycle. the confocal microscopic image stacks of the anterior part of the embryo were captured by a zeiss lsm duo microscope. gebiss workflow the generation of a labelled ground truth image stack in gebiss can be achieved through: i) segmenting each 2d roi of a spatial object, ii) 3d segmentation of a spatial object or segmenting a group of spatial objects. the labelled gt stack can be saved and used to benchmark an arbitrary machine segmentation. each original microscopic image stack is smoothened by applying a pixel radius two dimensional median filter. 2d slice-based gt creation workflow in the course of a slice-based segmentation, each optical slice of a spatial object can be segmented individually using different threshold values if necessary. the resulting rois can effectively exclude otherwise merged segmentation artefacts. inspired by michael m. miller's segmentingassistant plugin and using imagej's core wand tool class, each roi is defined by four parameters: startx, starty, a minimum and a maximum intensity threshold value. while segmentingassistant makes 2d local segmentation much easier and precise compared with a manual "freehand selection", there was room for improvement. for example, sliders are not perfect for startx, starty selection in order to assign several points in an image. a slider is not optimal to set θmin, when the typical step is too small to be set by dragging the slider. also, the parameter load and save functions were missing. earlier the binary mask and labelled stack creation was complicated, as it required the combined use of roi manager and the 3d object counter plugin. several features were built in gebiss to improve productivity. startx and starty coordinates can be assigned by a single mouse click on the original image and θmin can be changed by rolling the mouse middle scroll wheel in intensity steps of and when combined with shift or ctrl key respectively. the parameter θmax is set to the brightest intensity value of the image by default, and requires adjustment in rare cases. the algorithm searches in the right direction until a pixel is found with an intensity falling into the range of. following that, it traces the contour of the 2d object in a clockwise direction. however, the resulting roi is calculated by using the median filtered intensity values, which are invisible to the user. superimposed on the original image, users are guided with continuous visual feedback as the contours are updated. using the gui of the slice-based gt specification tab, the optimal startx, starty, θmin and θmax values are stored in a parameter database for each object, where they can be updated, deleted or invoked to create a binary and a labelled stack. the 2d slice-based gt creation workflow is flexible enough to allow the use of multiple threshold values per 3d object if needed. at the same time, the reuse of the stored parameters of an object in the present optical slice enables faster processing of the consecutive slice. the latter approach assumes that the startx, starty and θmin parameters of the object roi in the previous slice gives a correct contour. if not, the values can be adjusted to fit on the given contour. 3d object-based gt creation workflow gebiss is able to visualise 3d-thresholded foreground voxels as an isosurface superimposed on background voxels with original intensity. this double rendering feature can be switched on by checking in the add isosurface submenu in the view menu of gebiss, which appears when the tab is activated. after an image stack is opened with imagej and gebiss is started, 3d connected objects can be segmented interactively by using the tab "object-based gt specification". each 3d object is segmented individually by a seeded 3d region growing algorithm originally implemented as ij-plugins toolkit version by jarek sacha. the connected threshold region growing segmentation algorithm requires five parameters specified by the user: the x, y, z coordinates of a seed pixel as well as a minimum and a maximum intensity threshold value. following the right side section of the flowchart in figure the seed pixel coordinates are defined on the image stack window by a single ctrl plus left mouse key click on a bright fluorescent region of a given object. the θmin is defined immediately after the seed selection. to provide the user with 3d visual information to find the optimal θmin, a customized version of benjamin schmid's imagej 3d viewer plugin was implemented into gebiss. in order to simultanously visualise background and foreground, gebiss superimposes two 3d renderings. background voxels are visualised by texture-based volume rendering using brightness-corrected fluorescent intensities. foreground voxels are added forming a semi-transparent, red coloured isosurface mesh that allows the user to observe simultanously both the interior and the exterior of the object. the default maximum intensity threshold value is for images with a dark background and a bright foreground. the user selects an optimal θmin value in the "adjust threshold..." window. in the case of drosophila embryos such a θmin value is low enough to include all heterochromatin regions and chromosome arms but high enough to exclude free histone and lipid droplets. by dragging the threshold slider, a 10- slice thick 3d mesh of a typical nucleus is rendered. the image can be freely rotated in 3d and zoomed using the mouse and the middle wheel. the whole rotating 3d virtual environment can be recorded as an animated movie using imagej's avi writer plugin. an automated light attenuation compensation in gt creation is offered by a dual thresholding function implemented under the tab "object subset gt specification". there are cases where the signal-to-noise ratio would allow the segmentation of a stack with a global threshold, though the light attenuation requires the use of a higher threshold value for deeply located objects. in this module two different thresholds are applied to objects located in shallower and deeper axial depth. the user is prompted for a data file containing the x, y, z seed coordinates of each object, a threshold value for the shallower and a second threshold value for the deeper objects as well as the demarcation z slice number that separates the shallow and deep slices. the x, y, z seed data file is generated using the 3d object counter plugin. even if its global thresholding segmentation produces imprecise contours in such cases, the derived object centroids are saved in a data file that serves as an x, y, z seed input file. this gebiss module applies a 3d region growing segmentation using shallower or deeper θmin to the objects according to their z centroid value. in practice it is done as follows. a separation slice is set up at a certain depth. all nuclei are segmented by a 3d region growing algorithm. those nuclei whose centroid's z parameter is above the separation slice are segmented by a user-determined threshold value. those nuclei whose centroid's z parameter is below the separation slice are segmented by a different threshold value. gt contour visualisation the requirement of human supervision for gt creation in any system may require an optional high level check. fast and precise visualisation is achieved by the superimposition of all gt roi contours on the original images. an imagej macro was created which uses a binary gt and original microscopic greyscale stacks as input files. the binary stack must be inverted, and the original greyscale stack must be converted to rgb format. the macro automatically draws all segmented object contours on each of the original slices, visualising both the foreground and background pixels, therefore any missing 2d roi or 3d object can be detected at the object level. at pixel level, false negatives such as unsegmented chromosome arms and false positives such as attached free histones or lipid droplets can be recognised easily. any further gt roi adjustment can be done by the respective gebiss steps. the isosurface 3d rendering of a labelled stack can reveal falsely merged gt objects. benchmarking workflow for segmentation performance evaluation, the most useful measures are precision, recall and f-measure. all of those measures need the quantification of true positive, false positive and false negative class labels defined by the four outcomes of the relation between the predicted class and the actual class. the value of precision is calculated as, thus it depends on the number of false positives. fewer false positives result in a precision value closer to similarly, the number of false negatives affect the recall value, that is calculated as, which is closer to when the number of false negatives is low. the f-measure is expressed as, and it evaluates the performance of a machine segmentation in a single value as the harmonic mean of precision and recall. the closer the f-value is to the better the given ms is. a ground truth segmentation contains neither any false positive nor any false negative pixel, therefore pgt = rgt = and fgt = in the context of segmentation, the performance evaluation can be quantified at two levels: object level and pixel level. at the object level, the object number of ms and its deviation from that of the gt is calculated. the "object" term refers to either a 2d foreground region of interest composed of or 8-connected pixels or a 3d body composed of or 26-connected foreground voxels. ideally, each gt object matches an ms object resulting in a one-to-one correspondence. the machine segmentation may result in a false positive object that does not occur in gt or a false negative, missing one that does occur in gt. an ms object may be split if more than one ms object matches a gt object or merged if more than one gt object matches an ms object. at pixel level, the correspondence between the foreground and the background region is quantified. each segmented pixel or volumetric pixel can be classified as true positive, false positive, true negative or false negative. the number of tp, fp, tn and fn pixels are counted, and precision, recall and f-measure are calculated for a 3d stack or a 2d slice. the benchmarking module requires a labelled ms and a labelled gt stack as input files. it has a numerical output and a visual output that is shown in additional files and. the numerical output displays benchmark measures for the whole stack as well as each individual slice. for the whole stack, the number of gt and ms objects are indicated at object level. several well-established performance metrics are in use, that are derived from the confusion matrix. the precision, recall and f-measure are calculated at the pixel level. for each individual slice, the module calculates the number of gt and ms objects as well as the number and label of merged and split objects. this feature enables the precise identification of the slice where roi merge occurs. the number and label of occurrent false positive ms and false negative gt objects are also calculated.  gebiss is a tool for semi-automated 3d image segmentation which can be used either as a ground truth in performance evaluation or in image quantification. we applied it to different biological datasets. a good example is the process of nuclear division in the early drosophila embryo, which has been extensively described. precise nuclear segmentation is critical in quantifying this aspect of embryonic development. contour finding in nuclear segmentation can be challenging as lipid droplets surround and attach to nuclei. the condensed chromosomes are often surrounded by a cloud of free histone that impedes the segmentation of a specific surface. this problem also applies to all deeply located nuclei in a stack where laser attenuation leads to poor signal-to-noise ratio. additionally, the contrast of a whole image stack may be low owing to the uniform intensity of foreground and background pixels. gebiss offers the biologist diverse ways to overcome these obstacles. case study 1: nucleus segmentation in embryogenesis 2d segmentation of embryonic nuclei one of the objectives of this study is measuring the volume of nuclei. to measure the precise nuclear volume, segmentation must correctly separate an attached lipid droplet from a discrete one that locates in the close proximity of a nucleus. gebiss can be first used to measure the volume of nuclei marked by h2av-gfp. to quantify nuclear features like size, shape, number, etc., it is necessary to separate the two classes of nuclear histone and cytoplasmic histone. a lipid droplet-specific marker, such as nile red dye, effectively allows lipid droplet segmentation and separation from the nuclei, but using the less organelle-specific h2av-gfp it is still possible to create a precise ground truth segmentation. the top row of figure illustrates the 2d slice segmentation of a nucleus optical section from the original image through a contour resulting from an incorrect and a correct θmin value. figure 3d shows the final segmented image of the nucleus. decreasing the θmin value for a whole 3d object is usually not helpful to detach a lipid droplet artefact because 3d shrinking leads to the loss of the correct spherical shape of an interphase nucleus. consequently, we disconnected a lipid droplet by the θmin reduction of only certain 2d nuclear slices while we preserved the correct nuclear shape. the h2av-gfp intensity gradient is often low between a distinct lipid droplet and a nucleus before the histone assembles into the chromatin. this complicates the 3d gradient-based segmentation. the intensity variation inside a nucleus can exceed that of the variation between the nucleus and a distinct lipid droplet nearby. the repeated use of a certain parameter set in multiple roi slices allows accelerated gt creation, however it points out the intrinsic limitations of the method when used for segmentation of a spatial object. certain rois can overlap each other and can be omitted inadvertently as a result of parameter reuse. since only one slice is shown at a time and the thresholding parameter can be adjusted arbitrarily, the resulting contour can be flickery over slices of low contrast stacks. finding the optimal thresholds of the top and bottom slices and the contours can be difficult in such cases. 3d segmentation alleviates those shortcomings. to determine whether machine segmentation artefacts cause remarkable changes in measured volumes, dividing nuclei were traced throughout mitosis. the high h2av-gfp intensity gradient inside a prophase nucleus led to unfilled holes in segmenting a syncytial blastoderm nucleus with ms, resulting in different volume measurement between gt and ms segmentation. 3d nucleus segmentation to detach free histone the individual and manual specification of several thousand roi contours of a segmented image stack is a flexible, but slow and laborious method, prone to human error. 3d object-based segmentation speeds up the process and the user can semi-automatically segment a whole object by using a single threshold. a typical nucleus spreads over ~ to ~ slices, thus by choosing a 3d object-level θmin many rois can be segmented simultaneously. selecting an optimal θmin is the main user task after the x, y, z coordinates of a seed pixel are chosen. the middle row image series of figure guides the viewer through the process via the 3d segmentation of a metaphase nucleus. the volume-rendered 3d view shows a heterochromatin region surrounded by free histone from below that has been spuriously attached because of an incorrectly low θmin value. the optimal θmin removes the free histone and at the same time preserves the shape of the mitotic nucleus, resulting in the correctly segmented optical slices. as a result, nuclei are segmented in interphase and various mitotic phases such, as prophase, metaphase, anaphase a, anaphase b and telophase, respectively. this module also enables the user to distinguish between axial and lateral 3d object fusions. segmentation of a low contrast embryonic image stack the brightly stained central yolk mass of the drosophila embryo is still overlapping with the periplasm during nuclear cycle in the syncytial blastoderm, which lasts until the the depth of yolk-free periplasm increases dramatically at the expense of the central yolk region in nuclear cycle. as a result, the contrast of a whole image stack becomes low owing to the uniform intensity of foreground and background pixels. global thresholding gives visually unacceptable segmentation results. the individual 3d object-based gt specification is a segmentation method that was able to remove the noise around the nuclei and create a labelled image stack. case study 2: 3d segmentation of drosphila brain in order to provide a quantitative measurement of the strength of phenotypes resulting from bchs gain-of-function or loss-of-function mutations, lim and kraut first made use of the percentages of larval rp motor neuron loss through the immunostaining of gfp-expressing rp motor neurons. however, we sought to make the phenotypic measurement faster and more quantitative. therefore in this study, we have measured the brain volumes of different late third instar bchs mutant larvae by labelling the dissected whole larval brain with rhodamine-conjugated phalloidin, which recognizes f-actin. the bchs allele is a nonsense mutation that encodes for a truncated protein, while dfclot is a deficiency on the left arm of chromosome as shown in table both genotypes of bchs58/dfclot and homozygous bchs have a 15% to 17% reduction in the brain volumes of the third instar larvae, when compared with the brains of the larvae from the genetic background of the bchs allele mutants. using gebiss we show that both bchs null mutant combinations of bchs58/dfclot and homozygous bchs have a reduction in the brain volumes of third instar larvae, when compared with the background genotype yw brains as a control. athe difference in brain volume of genotype in comparison with genotype as the reference control. na: non-applicable. this reduction in bchs mutant brain volume is statistically significant as compared with the yw control and it is in agreement with our previous phenotypic quantification method of rp motor neuron loss. in addition, the smaller reduction in bchs mutant brain volume at the third instar larval stage in comparison with an estimated 40% reduction in adult bchs mutant brain volume may be explained by the longer duration of the adult stage which allows progressive neurodegeneration to occur. from this brain volumetric analysis, there is no significant difference between the two different bchs null alleles, bchs58/dfclot and homozygous bchs indicating that the bchs allele is, as expected from earlier studies, acting as a null allele with respect to bchs' effect on brain volume. the reduction in brain volume of bchs null larvae is consistent with the earlier observation made in adult animals and suggests that the overall level of degeneration in the larval brain can be assessed using the gebiss method of volumetric analysis. conversely, over-expression of the bchs using the ep line in conjunction with the c gal driver in the central nervous system resulted in an increase of % brain volume when compared with the control, c155/yw. together with the observed shrinkage in brain volume of loss of function in bchs mutants, these results suggest that bchs may have some role in determining cellular volume and/or proliferation in the brain. segmentation performance evaluation to test the performance of 3d segmentation methods, we applied six different automated ms algorithms to segment drosophila brain stacks: yen's method, rényi entropy, li's minimum cross entropy, huang's fuzzy thresholding, ridler and calvard's iterative isodata method and otsu's thresholding. the segmentation result image stacks can be found in additional file 6: "visual segmentation performance evaluation". all algorithms are implemented under imagej by gabriel landini, and compute a global threshold from the stack image to segment the stack, with the limitation that only 8-bit image stacks are supported. the advantage is that no manual parameter entry is required. the computed threshold is lower, equal to or higher than that of ground truth. therefore, the segmented brain stacks resulted in greater, equal to or smaller volumes, respectively. ms volumes greater than gt lack any false negative pixels, thus leading to r = recall in the case of li and huang segmentation. conversely, ms volumes smaller than gt lack any false positive pixels, thus resulting in p = precision at yen, rényi, isodata and otsu methods. the 8-bit image stacks had histograms shifted to darker voxel intensities. we used the global threshold value to create gt. however, global threshold values between and gave f-measures and above, indicating that the brain contour was not sharp. huang's fuzzy thresholding belongs to the algorithms that uses shannon's entropy function and segments based on attribute similarity. measuring the similarity between the original and the binary image stack, it determined the threshold by minimising the measure of fuzziness. including average foreground and background intensities, the algorithm computed a low ms threshold value in our dark stacks, resulting in numerous false positive voxels and the poorest mean f-measure, which is still above. although widely used popular methods in imaging, the performance of the clustering-based segmentation algorithms of isodata and otsu performed less well than the others. both methods assume that a stack has two grey intensity maxima and partition the histogram of an image stack into two classes, based on intra-class variance minimisation and inter-class variance maximisation. the otsu method searches for the optimal threshold globally, whereas the isodata does this locally, thus these methods result in almost identical thresholds. these algorithms work optimally with a bimodal histogram, where the number of voxels are similar in both the foreground and the background class. one reason for the weaker segmentation performance is that the number of black background voxels affect the threshold computed by those two algorithms. also the background and foreground intensity modes are not separated sharply in the histograms of our brain stacks. among the tested ms algorithms, the three entropy-based segmentation methods performed the best. li's method is based on cross-entropic thresholding. it has a significantly reduced computational requirement compared with the exhaustive search method but it tends to calculate a lower threshold value than that of gt. we found that yen's method and rényi's entropy performed the best among the algorithms that were tested. the two algorithms are similar and performed similarly well. both belong to shannon-entropy maximisation-based segmentation methods, originating from one-dimensional entropic thresholding introduced by pun in.  in this paper, we present gebiss, a new software for quantitative 3d segmentation performance evaluation. gebiss was designed to be a productive and user friendly tool for ground truth creation, and it includes a benchmarking module that objectively evaluates a 3d segmentation. the package was developed as a plugin for imagej, is platform independent and can be freely downloaded from http://imaging.bii.a-star.edu.sg/projects/gebiss/. gebiss was successfully used in various biological tasks. availability and requirements project name: gebiss project home page: http://imaging.bii.a-star.edu.sg/projects/gebiss/ operating system: platform independent programming language: java other requirements: imagej m or higher, java.0_ or higher license: gnu gplv3 authors' contributions jkv carried out the design and implementation of the software, analysed the images and wrote the manuscript. nwt participated in the implementation of the software, clp coordinated the implementation of the software, kcy participated in the implementation of the software, jspl carried out the drosphila brain experiment, rk conceived and coordinated the drosphila brain experiment, mw conceived and coordinated the study. all authors read and approved the final manuscript. supplementary material additional file 1 imagej macro for gt contour visualization. imagej macro file in unix text format. before running, it requires two input image stacks to be opened. an original image stack, converted into rgb format and renamed as "orig" as well as its matching binary stack renamed as "bin" with black foreground and white background pixels. for the macro operation see the text. click here for file additional file 2 3d gt contour visualization. compressed, spatially calibrated rgb image stack that can be opened by standard imagej. gt segmentation contours are superimposed on the original, low contrast image slices. by simultaneously visualising both the foreground and background pixels, the user can easily check the ground truth segmentation. the stack's xyz dimensions are μm × μm × μm. click here for file additional file 3 numerical output of the gebiss benchmarking module. quantitative machine segmentation benchmark results demonstrated on a low contrast image stack of a drosophila embryo in syncytial blastoderm developmental stage, containing slices. data in columns represents the evaluation of each optical slices. the slice numbers are indicated in the table header, followed by measures of each individual slices respectively: number of gt and ms objects, number and labels of merged and split objects, number and labels of false positive and false negative ms objects. click here for file additional file 4 visual output of the gebiss benchmarking module. compressed, spatially calibrated image stack. it can be opened using imagej. it horizontally combines the ms and the gt 32-bit labelled stacks in an easily comparable manner. demonstrated on a low contrast image stack containing slices. click here for file additional file 5 nuclei in various phases. six animated gif movies in one compressed file. after uncompressing, those can be opened by imagej or an internet browser. the movies show nuclei of a drosophila embryo in postcellular blastoderm developmental stage. interphase volume = μm prophase v = μm metaphase v = μm anaphase a v = μm anaphase b v = μm telophase v = μm click here for file additional file 6 visual segmentation performance evaluation. zip compressed, spatially calibrated, segmented binary image stacks, that can be opened by standard imagej. the images represent the 3d segmentation results discussed in the section "segmentation performance evaluation". file names indicate the machine segmentation that resulted the given stack. click here for file acknowledgements the authors thank wee choo puah for the drosophila embryo image acquisition, chinta rambabu for his constructive comments, derek smith and daniel bosch ibáñez for their critical revision of the manuscript, wayne rasband for imagej and all the developers of the imagej plugins that were used in this paper.
scenarios include the quantification of 3d images with poor signal-to-noise ratios or the generation of so-called ground truth segmentations that are used to evaluate the accuracy of automated segmentation methods. depending on the degree of user interactions, segmentation methods can be divided into manual, automated or semi-automated approaches. image segmentation is a crucial step in quantitative microscopy that helps to define regions of tissues, cells or subcellular compartments. we have developed gebiss; an imagej plugin for the interactive segmentation, visualisation and quantification of 3d microscopic image stacks. gebiss was developed as a cross-platform imagej plugin and is freely available on the web at http://imaging.bii.a-star.edu.sg/projects/gebiss/.
 image segmentation is a crucial step in quantitative microscopy that helps to define regions of tissues, cells or subcellular compartments. depending on the degree of user interactions, segmentation methods can be divided into manual, automated or semi-automated approaches. 3d image stacks usually require automated methods due to their large number of optical sections. however, certain applications benefit from manual or semi-automated approaches. scenarios include the quantification of 3d images with poor signal-to-noise ratios or the generation of so-called ground truth segmentations that are used to evaluate the accuracy of automated segmentation methods.  we have developed gebiss; an imagej plugin for the interactive segmentation, visualisation and quantification of 3d microscopic image stacks. we integrated a variety of existing plugins for threshold-based segmentation and volume visualisation.  we demonstrate the application of gebiss to the segmentation of nuclei in live drosophila embryos and the quantification of neurodegeneration in drosophila larval brains. gebiss was developed as a cross-platform imagej plugin and is freely available on the web at http://imaging.bii.a-star.edu.sg/projects/gebiss/. 