BACKGROUND
microscopic image analysis is becoming an enabling technology for modern systems-biology research, and cell nucleus segmentation is often the first step in the pipeline.
despite recent advances, the segmentation performance remains unsatisfactory in many cases.
for example, on the popular public databases  <cit> , the state-of-the-art segmentation accuracies are just around 85%.
the challenges of automated cell nucleus segmentation mainly arise from two imaging artifacts, as shown in figure  <dig>  first, the cell nuclei regions are inhomogeneous – the pixels of a cell nucleus exhibit non-uniform intensities and different cell nuclei also display varying patterns.
second, the background is also inhomogeneous and certain regions might have very similar appearance to the cell nuclei.
these problems imply that:  precise delineation of boundaries between cell nuclei and the background is difficult;  some background areas could be mistaken as cell nuclei; and  certain cell nuclei could be missed.
the segmentation problem can be characterized as a localization issue that includes both object detection and pixel-wise segmentation.
related work
numerous works have been conducted on segmenting various structures in cell images  <cit> , and unsupervised approaches appear to dominate.
for example, the morphological methods based on thresholding, k-means clustering or watershed  <cit>  can be quite effective, as long as the objects exhibit good contrast with the background.
watershed methods are also effective in separating touching cells, although the results might deviate from the actual contours slightly.
a more popular trend of unsupervised segmentation is the energy-based deformable models, based on active contours  <cit>  or level sets  <cit> .
compared with modeling contours explicitly, level sets have the advantage of being non-parametric and free from topology constraints.
it is also relatively easy to incorporate continuous object-level regularization into level sets, such as shape priors.
another type of energy-based model is based on graph search  <cit> , graph cuts  <cit>  or normalized cuts  <cit> .
such methods attempt to derive the segmentation with global constraints, using well-defined graphical structures to represent the spatial relationships between regions.
many of these methods require good initial seeds or contours.
however, the usual initialization techniques, such as thresholding and watershed, would not handle images with high inhomogeneities well, hence causing extra or missing detection of cell regions.
such detection errors during initialization could propagate into the final segmentation outputs.
it has been shown that intensity inhomogeneities can be tackled by integrating convex bayesian functional with the chan-vese model  <cit> , and discrete region-competition  <cit>  based on the piecewise-smooth mumford-shah model [> <cit> .
however, without performing cell detection explicitly, the deformable models might become very complicated in order to filter background regions with cell-like features while keeping cell regions with background-like features.
to detect cells from inhomogeneous background, one way is to reconstruct the ideal image  <cit> , which however, requires specific imaging modeling.
reconstruction can also be built into active contours with constrained iterative deconvolution without explicitly computing the inverse problem  <cit> ; however, it requires the point-spread function of a microscope, which is measured or modeled.
another way is to enhance the objects using h-dome transformation  <cit> ; however, it might have difficulties with inhomogeneous foreground.
the inhomogeneity can also be reduced with reference-based intensity normalization  <cit> ; however, the image-level normalization would not well handle the intra-image variation.
in addition, shape-based nucleus detection has been proposed, with laplacian of gaussian   <cit>  or sliding band filter   <cit> .
while the latter method is less sensitive to low contrast and better representative of irregular shapes, the detection accuracy partially relies on validation from the corresponding cytoplasm image, which is not always available.
different from the unsupervised approaches, classification-based methods have also been proposed to incorporate prior information from labeled images.
these classifiers include bayesian  <cit> , k-nearest neighbor   <cit> , support vector machine   <cit> , and atlas-based approaches  <cit> .
since the apparent difference between the foreground and background is their intensities, simple intensity-based features, such as histograms  <cit> , have been widely used.
on the other hand, the effectiveness of classification-based methods depends highly on the separation of feature spaces between foreground and background.
therefore, approaches based on a bag of local classifiers  <cit> , and more complex features such as the local fourier transform   <cit> , spatial information  <cit> , and combination of appearance, shape and context features  <cit> , have been proposed.
while most such supervised approaches describe the pixel- or region-level features, there are methods that tackle intensity inhomogeneity by explicitly modeling the inter-cell variations as more structural features.
one way is to perform color standardization within pixelwise classification  <cit>  to account for the inter-image intensity variations.
to also address the intra-image variations, contrast information between an image region and the global foreground and local interest regions is computed  <cit> .
a similar approach is to estimate foreground probabilities based on intensity distributions derived from global images and local detection outputs  <cit> .
while both approaches introduce cell-adaptive features, the methods for global feature representation and local region detection might not work well with large feature overlapping between the foreground and the background.
an additional false positive reduction step has also been proposed to remove bright background regions that are misidentified as cell nuclei  <cit> .
however, this approach requires a learned classifier, whose performance could be affected by inter-image feature variation.
more differently, registration-based approach has also been studied, by creating a template set from training images and segmenting the testing image based on best matches  <cit> ; such templates however, might have difficulties capturing large varieties of object shapes and textures.
our contribution
the contribution of our work is to localize the cell nuclei in images with high intensity inhomogeneity with various data-adaptive modeling techniques in a progressive manner.
specifically, we design a three-stage cell nucleus localization method that:  salient regions representing cell nuclei and cell clusters are extracted with image-adaptive contrast enhancement;  the clusters are further processed to identify true cell nuclei based on feature-distance profiles of reference regions with cluster-adaptive probability estimates; and  the contours of detected cell nuclei are refined in a graphical model with region-adaptive contrast information.
figure  <dig> gives an overview of the proposed method.
we also design distinctive data-adaptive priors that can be categorized by the level of generalization:  global-level features modeled as support vectors from training images;  image-level features representing the distribution of varying appearances of the nearby cell nuclei; and  region-level features computed at all three stages of the method for interest region detection, candidate validation and contour refinement.
being adaptive to the specific image or interest region, the image- and region-level features are especially effective in accommodating the intensity inhomogeneities.
compared to localization methods based on global criteria , our approach is more capable of accommodating  intensity variations between cell nuclei  and  feature overlapping between cell nuclei and background areas.
compared to the energy-based techniques that target pixel-wise segmentation , our method has a stronger focus on cell nuclei detection with explicit modeling of cell-specific characteristics, to effectively filter cell-like background regions and identify obscure cell nuclei.
we suggest that the proposed region-based progressive localization  method can be potentially extended to other localization problems, if the objects of interest can be modeled as regions with distinct features from the surrounding background.
a similar three-stage approach would be used, and the application-specific modifications would mainly focus on the feature design.
one example could be tumor localization in functional images.
methods
initial segmentation
while cell nuclei might appear similar to the background, there is always some degree of contrast between them.
such an observation motivates us to localize the cell nuclei by extracting salient regions.
during initial segmentation, we do not have strict requirements about the extracted regions.
in particular, if multiple cell nuclei are tightly connected, or cell nuclei are surrounded by high-intensity background and difficult to differentiate, identifying them as a single region is acceptable.
we design a contrast-enhanced salient region detector for initial segmentation.
specifically, an iterative approach is developed based on the maximally stable extremal region  method  <cit> .
since mser does not require any initial contour and the region stability is constrained by local regional information, it is easy to use and able to accommodate large intra-image variations.
however, the effectiveness of mser depends highly on the intensity contrast between the foreground and background.
if the contrast is low, some regions would not be detected .
it is intuitive to add contrast enhancement.
however, basic approaches such as intensity stretching would not work due to large intensity span.
instead, we design an iterative approach by alternating between the following two steps.
first, interest regions {r} are detected using mser, as shown in figure 3c.
second, based on the detection result, the image is enhanced  by:
i:=i <dig> ·c
where {r}0 and {r}2 denote the minimum and mean intensities of the detected interest regions in i, and c is a scaling constant.
the normalization factor  <dig>  is chosen based on:  it should normally be smaller than c so that all pixels in i are scaled up with contrast between pixels increased proportionally; and  it should not be so small that the image becomes distorted from the original patterns with intensities capped at  <dig> for grayscale images.
the iteration stops when the number of regions created does not change any further.
with such a contrast-enhanced approach, better region detection output can be seen in figure 3e.
the resultant regions are either single-level, or form a hierarchy of lower- and upper-level regions.
it is also observed that during each iteration, the parameter maxvariation in mser  needs to vary for individual images to better accommodate the inter-image variations.
therefore, the parameter value is determined at runtime by first setting maxvariation to v1 then gradually reducing it by a certain step Δv until it reaches v2 or the number of region levels is larger than one.
furthermore, while the resultant single- and upper-level regions are mostly cell nuclei, occasionally under-segmentation happens.
in other words, a single cell nucleus could be divided into two nested regions and the upper-level region would become a under-segmented portion of the cell nucleus.
to reduce such under-segmentation, we find that if the combined area of two nested regions is roughly elliptical with a suitable size, they can be merged as a single region.
the shape and size criteria are determined using a linear-kernel binary svm obtained from the training data.
the overall process of initial segmentation is listed in algorithm  <dig>
algorithm 1: initial segmentation
decluster processing
as seen in figure 3e, the detected single- and upper-level regions usually represent the cell nuclei, and lower-level regions usually represent the background with elevated intensities.
however, the upper-level regions could contain false positives caused by bright background, and lower-level regions could also include undetected cell nuclei.
it is also observed that such incorrect detections are mainly present among the two-level nested regions , while the single-level regions are normally true cell nuclei.
therefore, in the second stage, we focus on further processing on the detected clusters, with two objectives.
first, we expect to identify any cell nucleus that has not been detected after the initial segmentation.
such cell nuclei typically exhibit similar intensities to the surrounding background, and hence would not be highlighted as salient regions.
second, we need to filter out high-intensity background regions, which usually have rounded or irregular shapes, and could be easily confused as cell nuclei.
a two-step approach is designed, using candidate identification then candidate validation.
an example output is shown in figure 3f.
formally, let u = {ui : i =  <dig> ...,nu} be a detected cluster, with nu pixels ui.
define the set of labels {f,b} representing the foreground  and background respectively, and a foreground region as a connected component gx ⊂ u with ∀ui ∈ gx : li=f.
the problem is to label each pixel ui∈u as li = {f,b}, with the object-level constraint that any detected foreground region gx should have suitable characteristics as a cell nucleus.
candidate identification
in the first step, we try to identify a set of non-overlapping candidate foreground regions {gx} from each cluster u by labeling each pixel ui as foreground or background.
we specify that any upper-level region enclosed in a cluster u is a candidate region gx.
to identify more candidates from the cluster u itself, it is observed that to differentiate between f and b pixels, the texture feature in a local patch is more discriminative than pixel intensities.
for example, compared with cell nuclei, the background usually has more homogeneous texture that might be dark or bright.
in this work, we choose to use the scale-invariant feature transform  descriptor  <cit> , which describes the gradient distribution within a local patch and is invariant to scale, translation and rotation.
sift feature of each pixel ui is computed, and then labeled using a binary svm.
the svm kernel is polynomial, with other default settings in libsvm  <cit> .
a connected component of f pixels is identified as a candidate region gx.
while most of the candidate regions are true cell nuclei, some are actually bright background areas with round shapes .
to filter out the false detections, we pass them to the candidate validation stage.
candidate validation
in the second step, we validate if the identified candidate region gx in image i is a cell nucleus.
there are two reasons that motivate this step.
first, there might be misclassification during candidate identification due to inter-image intensity variations .
the labeling performance could be improved based on reference information gathered from the testing image itself.
second, pixel-level labeling based on sift features has limited spatial information and often does not represent the overall region gx.
we design a probability estimation via distance profile method to derive the probability q of gx being a valid cell nucleus based on the feature-distance profiles of other reference cell nuclei, as detailed below.
probability inference although cell nuclei in an image could have varying characteristics, we expect that gx, if representing a true cell nucleus, should have similar features to the other cell nuclei in the same image, especially those spatially adjacent to gx, as can be seen from the examples in figure  <dig>  therefore, if we have a set of determined cell nuclei in i, we can use them as references to validate gx.
to cope with inter-image variations, we would only select references from the image i in which gx resides.
this means we could not use the ground truths for reference construction.
instead, we use the single- and upper-level regions that are detected during initial segmentation as references.
we use these references by first creating a distance profile per reference, and computing the probability of gx being a cell nucleus based on its feature distance to each reference.
specifically, assume within an area near gx, there are k reference regions g={gk:k= <dig> ...,k}.
here near is defined as both gx and gk being in the same quadrant of image i. let fx describe the region-level feature of gx, and the feature distance between gx and gk as δ .
intuitively, the more similar gx and gk are, the more likely gx is a cell nucleus.
however, since gk may be a false positive detection, decision based on direct feature distance δ might be error prone.
therefore, we devise an alternative hypothesis that, if δ is comparable with {k′= <dig> ...,k,k′≠k:δ}, then gx is likely a cell nucleus.
to measure if δ is comparable with {∀k′:δ}, we use the non-parametric kernel density estimation :
p^0=1k-1∑k′≠k1hkk
where δk,x is short for δ, k is the gaussian kernel and hk is the bandwidth approximation following normal distribution of all data samples {∀k′:δ}.
the density value p^ <dig> is then normalized by the maximum density of the distribution to obtain the comparability measure in terms of probability p^∈ <cit> :
p^=p^0/maxk′{p^0}
with this model, p^ is larger when δk,x approaches the gaussian mean of the samples, which means that gx is more likely a cell nucleus if the distance between gx and gk is similar to how the other references {gk′} vary from gk.
next, by combining the estimates p^ from all references {gk}, the final probability of gx being a cell nucleus is derived:
q=1k∑kp^
the averaging operation helps to ensure that a single reference gk with very different features from gx would not affect the overall probability q significantly.
then, based on q, we define a thresholding rule to determine if gx is a valid cell nucleus:
l=f, ifq>α1maxx′q,q>α <dig>  forgx′=∅
where gx′ denotes other candidate regions that are within the same cluster u as gx and x′ ≠ x; α1 and α2 are predefined thresholds.
examples of the density computation and probability derivation are shown in figure  <dig>  and the overall process of candidate validation is listed in algorithm  <dig>
algorithm 2: candidate validation
appearance feature we observe that a region tends to comprise patches of similar textures and repetitive patterns.
therefore, we choose to represent gx with bag-of-features.
first, the image i that contains gx is divided into a grid of patches {p}.
then for each patch, we represent its texture feature by its minimum, maximum, mean intensity, standard deviation, and a histogram of intensity differences between each pair of pixels.
each patch-wise feature is then assigned a feature word.
a histogram summarizing the occurrence frequencies of such feature words in gx is defined as fx.
here each feature vector is normalized by the size of gx, to represent the percentages of various intensities and feature words in gx.
note that if gx is a newly identified candidate during decluster processing, gx might only represent a small under-segmented portion of the actual cell nucleus due to the pixel-level labeling.
therefore, to have a good summary of the actual candidate feature, we first estimate an elliptical region gxo that is a minimum volume ellipsoid covering gx <cit> .
to avoid including many background pixels into gxo, we ensure gxo is part of the cluster u in which gx is detected: gxo=gxo∩u.
gxo is then used in place of gx as the detected candidate, from which fx is computed.
the refined elliptical regions are shown in figure 4c.
appearance distance to compute the distances δ between two histogram features, the diffusion distance  <cit>  is used.
the diffusion distance models the distance between histogram-based descriptors as heat diffusion process on a temperature field.
compared to the bin-to-bin histogram distances, such as euclidean distance, the diffusion distance is able to measure cross-bin distances, avoiding explicit computation of histogram alignment.
while the earth mover’s distance   <cit>  has similar advantages, the computation of diffusion distance is much faster, with o complexity only, where h is the number of histogram bins.
contour refinement
at this stage, a detected region could contain a single or multiple cell nuclei, which could be under-segmented or include extra background.
we thus expect to achieve better contour delineation of cell nuclei.
our idea is that, while the foreground and background are often inhomogeneous, there is always relatively good contrast between them in a local area.
therefore, by performing contour refinement for each detected cell region g individually, the foreground and background can be better differentiated by analyzing the localized contrast information.
we employ a regional contrast-based graphical model for the contour refinement.
specifically, a conditional random field   <cit>  with the following energy function is designed:
e=∑iη+η+ <dig> {∑iφ+∑i,i′ϕ}
where g¯ denotes the detected region g  plus its surrounding area of a fixed width  , and l denotes the labeling vector of all pixels in g¯.
then, the model attempts to refine the contour of g by relabeling each pixel ui∈g¯ as li = {f,b}.
here η is the unary contrast-based intensity term, η combined with φ is the contrast-based detection term with lg representing the detected region g, and ϕ is the spatial term associating neighboring pixels ui and ui′.
the constant  <dig>  is set to obtain equal contributions from the unary costs +η) and the combined pairwise costs +∑i,i′ϕ).
graph cut  <cit>  method is used to derive the most probable labeling l that minimizes the energy function, to produce the final segmentation of cell nuclei from g¯.
here our customized definition of the intensity term and inclusion of the detection term are the main distinctions from the other crf constructs  <cit> .
the contrast-based intensity term η describes the unary costs of pixel ui labeled as li ∈ {f,b}.
basically, the costs of li = f and li = b represent the inverse probabilities, and the probability pr of li = f is computed by:
pr=))- <dig>
fi=ii/ig
λg=fg-γg
where ig denotes the mean intensity of g, and pr follows a sigmoid probability distribution based on the contrast feature fi.
we expect pixels with fi > λg to more likely represent the foreground.
λg is computed based on fg and ⌞fg, which are the mean and minimum of all feature values {fi : ui ∈ g}, and is adjusted by γg for a balance of foreground and background partitioning in g. the parameter γg is calculated at runtime, by gradually increasing it from γ1 to γ2 with a step value Δγ, and choosing the smallest γg ∈  that does not cause the entire g to be labeled as b. with pr = 1 - pr, the cost values for both labels are:   η=1-pr
note that since λg would be closer to fg in most cases with small γg, it would cause portions of g to have pr <  <dig>  , resulting in possible under-segmentation.
it is however not advisable to lower λg, due to considerable overlap between low-intensity areas in g and the background.
therefore, we introduce a second contrast-based detection term to encourage labeling of li = f. an auxiliary node lg is first included to the graph with the following unary costs:
η=0iflg=fng¯otherwise
where ng¯ is the number of pixels in g¯, and such a large cost of lg = b ensures lg is assigned  <dig>  then for each pixel ui, a pairwise cost φ is computed based on the contrast ν between ii and the mean intensity of g:
φ=δ·ν
with δ =  <dig> if li ≠ lg and  <dig> otherwise, and ν =  <dig> if ii > ig, or:
ν=exp
where 〈 · 〉 denotes the average euclidean distances of all such pairwise distances in g¯.
in this way, pixels with pr ≈  <dig>  could be better labeled with the additional cost factor; and obvious background pixels would still obtain the correct b label, with φ much lower than η.
the spatial term ϕ then further enhances the delineation by encouraging spatial labeling consistencies between neighboring pixels ui and ui′.
a pairwise cost for li≠li′ is thus defined as:
ϕ=δ·ν
where δ and ν follow eq.
.
such a cost function implies that pixels with more similar intensities would be more penalized if they take different labels.
materials and evaluation methods
three different datasets that are publicly available with segmentation ground truth are used in this study.
their main properties are summarized in table  <dig>  the images in the first two datasets were acquired with nuclear markers whereas the third dataset also includes the cytoplasm.
detailed information can be found in  <cit> .
among the three, dataset  <dig> has higher contrast between cell nuclei and background.
datasets  <dig> and  <dig> have large intensity inhomogeneity and considerable degree of intensity overlapping between the cell nuclei and the background.
the inclusion of cytoplasm in dataset  <dig> poses more challenges.
the images in dataset  <dig> are preprocessed to remove the pink areas and converted to grayscale.
figure  <dig> shows an example image after the preprocessing.
most parameters used in this study are set to the same values for all three datasets:  in initial segmentation, v1 =  <dig> , v2 =  <dig>  and Δv =  <dig> ;  in probability inference, α1 =  <dig>  and α2 =  <dig> ;  in appearance feature, the number of histogram bins is  <dig>  and the number of feature words is 12; and  in contour refinement, γ1 =  <dig> , γ2 =  <dig> and Δγ =  <dig> .
while these settings are chosen empirically, using a common setting for all three datasets suggests that the method is robust to different image acquisition and manual tuning of parameters can be minimal.
there are only two dataset-specific parameters.
one is the patch size in appearance feature, which is 8× <dig> pixels for datasets  <dig> and  <dig>  and 4× <dig> pixels for dataset  <dig>  the smaller size for dataset  <dig> is chosen due to its smaller cell nuclei compared to datasets  <dig> and  <dig>  the other parameter is c in initial segmentation, which is set to  <dig> for datasets  <dig> and  <dig>  and  <dig> for dataset  <dig>  this ensures the contrast enhanced images in dataset  <dig> would not become too bright to cause distortion.
for dataset  <dig>  four representative images are selected to train two svm classifiers, for the cell-cluster differentiation and candidate identification.
while testing is performed on all images to make the results directly comparable with the state-of-the-art  <cit> , we note that the testing results are not sensitive to the selection of training data, with very similar testing results observed based on different training sets.
similar procedures are performed for dataset  <dig>  for dataset  <dig>  in order to have comparable performance evaluation with  <cit> , half of the images are used for training  and the rest for testing.
we evaluate the localization of cell nuclei by two measures.
first, performance of object-level detection is evaluated by recall , precision , and accuracy :
r=tp/
p=tp/
a=tp/
where tp, fn, and fp are the numbers of true positive, false negative and false positive detections of cell nuclei.
given a detected region od and the ground truth mask ogt, if the overlap ratio r is at least  <dig> :
r=|od∩ogt|/|od∪ogt|
then the detection is considered tp  <cit> ; and correspondingly fn and fp are determined.
second, the segmentation performance is evaluated by both region- and contour-based measures, including dice, normalized sum of distances  and hausdorff distance :
dice=2|f∩m|/
nsd=∑ui∈d/∑ui∈d
hd=maxui∈∂fd
here f represents the foreground pixels identified, m is the ground truth mask, and d is the minimal euclidean distance of pixel ui to ∂m of the corresponding reference nuclei, with ∂ indicating the contour.
we have compared with popular cell imaging segmentation techniques, including otsu thresholding, k-means clustering and watershed  <cit> .
furthermore, in view of the popularity of level set for cell imaging and our design on tackling the intensity inhomogeneities, we have experimented with a level set method that has a similar focus, using the authors’ released code  <cit> , with initial contours generated using watershed method.
for all methods, post-processing is conducted to remove isolated segments that are smaller than 1/ <dig> of the average size of foreground regions detected in the image.
in addition, we report direct performance comparisons with the state-of-the-art results reported on the same datasets  <cit> , by including the same performance measures as used in these works.
RESULTS
cell detection
we report the object-level detection results in table  <dig>  comparing the results at various stages of the methodology, the improvement is larger on dataset  <dig> than dataset  <dig>  e.g.
<dig> % increase in detection accuracy on dataset  <dig> vs  <dig> % increase on dataset  <dig>  this is because inhomogeneity is more prominent on dataset  <dig> while dataset  <dig> exhibits clearer contrast between the cell nuclei and the background in most images.
in our evaluation, a detection is only considered as tp if the overlap ratio in eq.
is at least  <dig> .
therefore, a largely over- or under-segmented object would be counted as fn for the second stage, and corrected after the contour refinement.
this explains why although cell nuclei are detected after the decluster processing, the recall results only improve significantly after the third stage.
on dataset  <dig>  the presence of cytoplasm causes many cell nuclei to clutter into one region during the initial segmentation; this leads to fn.
the third stage better differentiates the cell nuclei and cytoplasm, and the improvement is significant with  <dig> % and  <dig> % increase in detection recall and precision.
figure 7a gives a better overview of the overlap ratios obtained from the final localization outputs.
while most cell nuclei exhibit ratios not less than  <dig> , less optimal results are observed on dataset  <dig> again due to the influence from the cytoplasm.
after initial segmentation , decluster processing , and contour refinement .
the performance improvement introduced by the iterative process of interleaving interest region extraction and image enhancement are shown in figure 8a.
the higher recall  suggests that such an approach is especially useful for identifying foreground regions that originally display low contrast from the background.
the benefits of having candidate validation are shown in figure 8b.
by filtering out interest regions that are very different from the reference regions, the detection precision thus improves by on average 2%.
the recall improves by on average  <dig> % only, mainly because of the same constraints imposed by the overlap ratio.
to evaluate the effect of the default threshold setting α1 for candidate validation, the receiver operating characteristics  curves are plotted by varying the threshold value.
the probability estimates q/maxx′q from all candidate regions are included in the plot, and candidate regions with at least  <dig>  overlapping ratio with the ground truth are marked as foreground class and the rest as the background class.
as shown in figure  <dig>  the  <dig>  threshold setting provides a good balance between the tp and fp detections, with close to maximum tp rates.
note that the numbers of true negatives here are small , hence the fp rates appear relatively high.
nucleus segmentation
table  <dig> summarizes the region- and contour-based segmentation results.
on datasets  <dig> and  <dig>  the decluster processing improves the dice measure by about 3% and 4%, due to better object-level labeling of candidate regions.
the contour-based measures, however, are mainly enhanced at the third stage of the methodology, with on average more than half reduction in nsd and hd.
this is attributed to better contour delineations based on the detection results from the first two stages.
besides the mean values listed in the table, the distributions of hausdorff distances on the final localization results are also shown in figure 7b.
after initial segmentation , decluster processing , and contour refinement .
to further evaluate the design of the graphical model for contour refinement, the foreground probabilities for all pixels of interest are computed with the intensity term, as summarized in figure  <dig>  while many pixels exhibit suitable probabilities, some background pixels, especially those in datasets  <dig> and  <dig>  have larger foreground probabilities and would lead to misclassification.
the pixel-level classification is improved by introducing the contrast-based detection and spatial terms, as shown in figure  <dig>
performance comparison
the localization results using the standard approaches are listed in table  <dig>  with example outputs shown in figure  <dig>  compared to our proposed method, the level set and watershed techniques produce the second best results for dataset  <dig>  especially with good contour-based measures.
however, without explicitly handling high-intensity background regions, both methods result in about 3% lower detection precision.
on dataset  <dig>  our proposed method demonstrates stronger advantages, with  <dig> % increase in detection accuracy, <dig> % increase in dice coefficient and  <dig>  decrease in hd over the second best approach .
both the level set and watershed approaches face the following challenges:  difficulty separating cell nuclei from surrounding background areas with low contrast, and  incapability of classifying background regions that resemble cell nuclei.
on dataset  <dig>  the intensity inhomogeneities within the cell nuclei and the cytoplasm make it particularly difficult to achieve good segmentation.
as a result, the watershed method tends to largely over-segment the cell nuclei, generating many clusters and cause low detection recall and more errors in contour delineation.
the level set method based on localized energy optimization is quite effective in splitting the clusters, but is less optimal for areas with high similarity between the cell nuclei and cytoplasm.
the thresholding method does not perform as well as the level set or watershed approaches, but it does outperform the clustering-based approach.
compared to level set, our method achieves 7% increase in detection accuracy,  <dig> % increase in dice coefficient and  <dig>  decrease in hd.
tables  <dig>   <dig> and  <dig> show that our proposed method delivers better localization even using only the initial segmentation step.
higher performance margins are obtained with decluster processing and contour refinement, especially on datasets  <dig> and  <dig>
comparison between our proposed region-based progressive localization method , otsu thresholding , k-means clustering , watershed  and level set  <cit>  .
a comparison with the state-of-the-art results reported for the same datasets is summarized in table  <dig>  our method achieves better results in most measures, as bold-faced in the table.
on dataset  <dig>   <dig>  more fp cell nuclei are detected compared to the level set method  <cit> .
it is possible that such false detections are caused by accidental highlighting of background regions during the iterative image enhancement for the initial segmentation stage.
however, our method exhibits overall much better detection performance with minimal numbers of fns  and only  <dig>  fps.
the accuracy of pixel-level segmentation on dataset  <dig> improves significantly, as indicated by the 5% increase in dice and rand indices over  <cit> .
<dig> % performance improvement of object-level accuracy over  <cit>  on dataset  <dig> is also obtained.
these observations suggest that our method is indeed quite effective in handling the intensity inhomogeneity issue that is the major cause hindering satisfactory segmentation on datasets  <dig> and  <dig>  the improvement on the contour-based measures, i.e.
on average  <dig>  nsd decrease and  <dig>  hd decrease over  <cit> , also demonstrate the suitability of boundary delineation using region-based designs, i.e.
the salient region extraction and graphical model-based contour refinement.
‘–’ means not available.
our method is currently implemented in matlab, running on a standard pc with a  <dig> -ghz dual core cpu and  <dig>  gb ram.
the computational time is related to the number of cells and the size of cells in an image.
on a  <dig> × <dig> pixel image with about  <dig> cell nuclei, an average  <dig> s is needed for the entire localization process.
this is faster than applying the level set method  <cit> , which requires about  <dig> s with  <dig> iterations.
CONCLUSIONS
a fully automatic localization method for cell nuclei in microscopic images is presented in this paper.
intensity inhomogeneities in cell nuclei and the background often cause unsatisfactory localization performance.
not many works have been reported to address this problem in a robust manner.
we propose a method that exploits various scales of data-adaptive information to tackle the intensity inhomogeneity.
first, the regions of interest, i.e.
cell nuclei or clusters, are extracted as salient regions with iterative contrast enhancement.
then with feature-based classification and reference-based probability inference, the clusters are further processed to detect more cell nuclei and filter out spurious regions.
lastly, based on regional contrast information encoded in a graphical model, the pixel-level segmentation is enhanced to create the final contours.
this region-based progressive localization  method has been successfully applied to three publicly available datasets, showing good object-level detection and region- and contour-based segmentation results.
compared to popular approaches in this problem domain such as level sets, our method achieved consistently better performance, with on average  <dig> % increase in dice coefficient and 6% increase in object-level detection accuracy.
our method also outperformed the state-of-the-art with on average  <dig> % and 7% improvement of region- and contour-based segmentation measures.
we also suggest that the proposed method is general in nature and can be applied to other localization problems, as long as the objects of interest can be modeled as salient regions with measurable contrast from the background.
as a future study, we will investigate improving the graphical model for better contour delineation.
a potential approach is to incorporate an additional term as the cost of difference between the model image and the measured image, as inspired by  <cit> .
the model image could be derived as a convolution of a point-spread function of the microscope with an object intensity function defined based on the pixel labels.
we will also investigate replacing the pixel-wise labeling with region-level processing for computational efficiency while maintaining the segmentation accuracy.
other future work could explore the applicability of the proposed method on other types of images.
images with nuclear membrane marker and different nuclear markers such as the green fluorescent protein , and those with higher resolution or dimension, are of particular interest.
to accommodate the specific characteristics of these images, possible changes to the method are to design more comprehensive intensity and texture features to differentiate among cell structures and background, and to enhance the contour refinement with boundary constraints.
competing interests
the authors declare that they have no competing interests.
authors’ contributions
ys designed and carried out research and drafted the manuscript.
wc discussed and helped to design the methodology and draft the manuscript.
hh, yw and mc helped with the draft manuscript.
df coordinated the designed research.
all authors read and approved the final manuscript.

 microscopic image analysis is becoming an enabling technology for modern systems-biology research, and cell nucleus segmentation is often the first step in the pipeline. despite recent advances, the segmentation performance remains unsatisfactory in many cases. for example, on the popular public databases, the state-of-the-art segmentation accuracies are just around 85%. the challenges of automated cell nucleus segmentation mainly arise from two imaging artifacts, as shown in figure  first, the cell nuclei regions are inhomogeneous – the pixels of a cell nucleus exhibit non-uniform intensities and different cell nuclei also display varying patterns. second, the background is also inhomogeneous and certain regions might have very similar appearance to the cell nuclei. these problems imply that: precise delineation of boundaries between cell nuclei and the background is difficult; some background areas could be mistaken as cell nuclei; and certain cell nuclei could be missed. the segmentation problem can be characterized as a localization issue that includes both object detection and pixel-wise segmentation. related work numerous works have been conducted on segmenting various structures in cell images, and unsupervised approaches appear to dominate. for example, the morphological methods based on thresholding, k-means clustering or watershed can be quite effective, as long as the objects exhibit good contrast with the background. watershed methods are also effective in separating touching cells, although the results might deviate from the actual contours slightly. a more popular trend of unsupervised segmentation is the energy-based deformable models, based on active contours or level sets. compared with modeling contours explicitly, level sets have the advantage of being non-parametric and free from topology constraints. it is also relatively easy to incorporate continuous object-level regularization into level sets, such as shape priors. another type of energy-based model is based on graph search, graph cuts or normalized cuts. such methods attempt to derive the segmentation with global constraints, using well-defined graphical structures to represent the spatial relationships between regions. many of these methods require good initial seeds or contours. however, the usual initialization techniques, such as thresholding and watershed, would not handle images with high inhomogeneities well, hence causing extra or missing detection of cell regions. such detection errors during initialization could propagate into the final segmentation outputs. it has been shown that intensity inhomogeneities can be tackled by integrating convex bayesian functional with the chan-vese model, and discrete region-competition based on the piecewise-smooth mumford-shah model [>. however, without performing cell detection explicitly, the deformable models might become very complicated in order to filter background regions with cell-like features while keeping cell regions with background-like features. to detect cells from inhomogeneous background, one way is to reconstruct the ideal image, which however, requires specific imaging modeling. reconstruction can also be built into active contours with constrained iterative deconvolution without explicitly computing the inverse problem; however, it requires the point-spread function of a microscope, which is measured or modeled. another way is to enhance the objects using h-dome transformation; however, it might have difficulties with inhomogeneous foreground. the inhomogeneity can also be reduced with reference-based intensity normalization; however, the image-level normalization would not well handle the intra-image variation. in addition, shape-based nucleus detection has been proposed, with laplacian of gaussian or sliding band filter. while the latter method is less sensitive to low contrast and better representative of irregular shapes, the detection accuracy partially relies on validation from the corresponding cytoplasm image, which is not always available. different from the unsupervised approaches, classification-based methods have also been proposed to incorporate prior information from labeled images. these classifiers include bayesian, k-nearest neighbor, support vector machine, and atlas-based approaches. since the apparent difference between the foreground and background is their intensities, simple intensity-based features, such as histograms, have been widely used. on the other hand, the effectiveness of classification-based methods depends highly on the separation of feature spaces between foreground and background. therefore, approaches based on a bag of local classifiers, and more complex features such as the local fourier transform, spatial information, and combination of appearance, shape and context features, have been proposed. while most such supervised approaches describe the pixel- or region-level features, there are methods that tackle intensity inhomogeneity by explicitly modeling the inter-cell variations as more structural features. one way is to perform color standardization within pixelwise classification to account for the inter-image intensity variations. to also address the intra-image variations, contrast information between an image region and the global foreground and local interest regions is computed. a similar approach is to estimate foreground probabilities based on intensity distributions derived from global images and local detection outputs. while both approaches introduce cell-adaptive features, the methods for global feature representation and local region detection might not work well with large feature overlapping between the foreground and the background. an additional false positive reduction step has also been proposed to remove bright background regions that are misidentified as cell nuclei. however, this approach requires a learned classifier, whose performance could be affected by inter-image feature variation. more differently, registration-based approach has also been studied, by creating a template set from training images and segmenting the testing image based on best matches; such templates however, might have difficulties capturing large varieties of object shapes and textures. our contribution the contribution of our work is to localize the cell nuclei in images with high intensity inhomogeneity with various data-adaptive modeling techniques in a progressive manner. specifically, we design a three-stage cell nucleus localization method that: salient regions representing cell nuclei and cell clusters are extracted with image-adaptive contrast enhancement; the clusters are further processed to identify true cell nuclei based on feature-distance profiles of reference regions with cluster-adaptive probability estimates; and the contours of detected cell nuclei are refined in a graphical model with region-adaptive contrast information. figure  gives an overview of the proposed method. we also design distinctive data-adaptive priors that can be categorized by the level of generalization: global-level features modeled as support vectors from training images; image-level features representing the distribution of varying appearances of the nearby cell nuclei; and region-level features computed at all three stages of the method for interest region detection, candidate validation and contour refinement. being adaptive to the specific image or interest region, the image- and region-level features are especially effective in accommodating the intensity inhomogeneities. compared to localization methods based on global criteria, our approach is more capable of accommodating intensity variations between cell nuclei and feature overlapping between cell nuclei and background areas. compared to the energy-based techniques that target pixel-wise segmentation, our method has a stronger focus on cell nuclei detection with explicit modeling of cell-specific characteristics, to effectively filter cell-like background regions and identify obscure cell nuclei. we suggest that the proposed region-based progressive localization method can be potentially extended to other localization problems, if the objects of interest can be modeled as regions with distinct features from the surrounding background. a similar three-stage approach would be used, and the application-specific modifications would mainly focus on the feature design. one example could be tumor localization in functional images. methods initial segmentation while cell nuclei might appear similar to the background, there is always some degree of contrast between them. such an observation motivates us to localize the cell nuclei by extracting salient regions. during initial segmentation, we do not have strict requirements about the extracted regions. in particular, if multiple cell nuclei are tightly connected, or cell nuclei are surrounded by high-intensity background and difficult to differentiate, identifying them as a single region is acceptable. we design a contrast-enhanced salient region detector for initial segmentation. specifically, an iterative approach is developed based on the maximally stable extremal region method. since mser does not require any initial contour and the region stability is constrained by local regional information, it is easy to use and able to accommodate large intra-image variations. however, the effectiveness of mser depends highly on the intensity contrast between the foreground and background. if the contrast is low, some regions would not be detected. it is intuitive to add contrast enhancement. however, basic approaches such as intensity stretching would not work due to large intensity span. instead, we design an iterative approach by alternating between the following two steps. first, interest regions {r} are detected using mser, as shown in figure 3c. second, based on the detection result, the image is enhanced by: i:=i ·c where {r}0 and {r}2 denote the minimum and mean intensities of the detected interest regions in i, and c is a scaling constant. the normalization factor is chosen based on: it should normally be smaller than c so that all pixels in i are scaled up with contrast between pixels increased proportionally; and it should not be so small that the image becomes distorted from the original patterns with intensities capped at for grayscale images. the iteration stops when the number of regions created does not change any further. with such a contrast-enhanced approach, better region detection output can be seen in figure 3e. the resultant regions are either single-level, or form a hierarchy of lower- and upper-level regions. it is also observed that during each iteration, the parameter maxvariation in mser needs to vary for individual images to better accommodate the inter-image variations. therefore, the parameter value is determined at runtime by first setting maxvariation to v1 then gradually reducing it by a certain step Δv until it reaches v2 or the number of region levels is larger than one. furthermore, while the resultant single- and upper-level regions are mostly cell nuclei, occasionally under-segmentation happens. in other words, a single cell nucleus could be divided into two nested regions and the upper-level region would become a under-segmented portion of the cell nucleus. to reduce such under-segmentation, we find that if the combined area of two nested regions is roughly elliptical with a suitable size, they can be merged as a single region. the shape and size criteria are determined using a linear-kernel binary svm obtained from the training data. the overall process of initial segmentation is listed in algorithm algorithm 1: initial segmentation decluster processing as seen in figure 3e, the detected single- and upper-level regions usually represent the cell nuclei, and lower-level regions usually represent the background with elevated intensities. however, the upper-level regions could contain false positives caused by bright background, and lower-level regions could also include undetected cell nuclei. it is also observed that such incorrect detections are mainly present among the two-level nested regions, while the single-level regions are normally true cell nuclei. therefore, in the second stage, we focus on further processing on the detected clusters, with two objectives. first, we expect to identify any cell nucleus that has not been detected after the initial segmentation. such cell nuclei typically exhibit similar intensities to the surrounding background, and hence would not be highlighted as salient regions. second, we need to filter out high-intensity background regions, which usually have rounded or irregular shapes, and could be easily confused as cell nuclei. a two-step approach is designed, using candidate identification then candidate validation. an example output is shown in figure 3f. formally, let u = {ui: i = ...,nu} be a detected cluster, with nu pixels ui. define the set of labels {f,b} representing the foreground and background respectively, and a foreground region as a connected component gx ⊂ u with ∀ui ∈ gx: li=f. the problem is to label each pixel ui∈u as li = {f,b}, with the object-level constraint that any detected foreground region gx should have suitable characteristics as a cell nucleus. candidate identification in the first step, we try to identify a set of non-overlapping candidate foreground regions {gx} from each cluster u by labeling each pixel ui as foreground or background. we specify that any upper-level region enclosed in a cluster u is a candidate region gx. to identify more candidates from the cluster u itself, it is observed that to differentiate between f and b pixels, the texture feature in a local patch is more discriminative than pixel intensities. for example, compared with cell nuclei, the background usually has more homogeneous texture that might be dark or bright. in this work, we choose to use the scale-invariant feature transform descriptor, which describes the gradient distribution within a local patch and is invariant to scale, translation and rotation. sift feature of each pixel ui is computed, and then labeled using a binary svm. the svm kernel is polynomial, with other default settings in libsvm. a connected component of f pixels is identified as a candidate region gx. while most of the candidate regions are true cell nuclei, some are actually bright background areas with round shapes. to filter out the false detections, we pass them to the candidate validation stage. candidate validation in the second step, we validate if the identified candidate region gx in image i is a cell nucleus. there are two reasons that motivate this step. first, there might be misclassification during candidate identification due to inter-image intensity variations. the labeling performance could be improved based on reference information gathered from the testing image itself. second, pixel-level labeling based on sift features has limited spatial information and often does not represent the overall region gx. we design a probability estimation via distance profile method to derive the probability q of gx being a valid cell nucleus based on the feature-distance profiles of other reference cell nuclei, as detailed below. probability inference although cell nuclei in an image could have varying characteristics, we expect that gx, if representing a true cell nucleus, should have similar features to the other cell nuclei in the same image, especially those spatially adjacent to gx, as can be seen from the examples in figure  therefore, if we have a set of determined cell nuclei in i, we can use them as references to validate gx. to cope with inter-image variations, we would only select references from the image i in which gx resides. this means we could not use the ground truths for reference construction. instead, we use the single- and upper-level regions that are detected during initial segmentation as references. we use these references by first creating a distance profile per reference, and computing the probability of gx being a cell nucleus based on its feature distance to each reference. specifically, assume within an area near gx, there are k reference regions g={gk:k=...,k}. here near is defined as both gx and gk being in the same quadrant of image i. let fx describe the region-level feature of gx, and the feature distance between gx and gk as δ. intuitively, the more similar gx and gk are, the more likely gx is a cell nucleus. however, since gk may be a false positive detection, decision based on direct feature distance δ might be error prone. therefore, we devise an alternative hypothesis that, if δ is comparable with {k′=...,k,k′≠k:δ}, then gx is likely a cell nucleus. to measure if δ is comparable with {∀k′:δ}, we use the non-parametric kernel density estimation: p^0=1k-1∑k′≠k1hkk where δk,x is short for δ, k is the gaussian kernel and hk is the bandwidth approximation following normal distribution of all data samples {∀k′:δ}. the density value p^ is then normalized by the maximum density of the distribution to obtain the comparability measure in terms of probability p^∈: p^=p^0/maxk′{p^0} with this model, p^ is larger when δk,x approaches the gaussian mean of the samples, which means that gx is more likely a cell nucleus if the distance between gx and gk is similar to how the other references {gk′} vary from gk. next, by combining the estimates p^ from all references {gk}, the final probability of gx being a cell nucleus is derived: q=1k∑kp^ the averaging operation helps to ensure that a single reference gk with very different features from gx would not affect the overall probability q significantly. then, based on q, we define a thresholding rule to determine if gx is a valid cell nucleus: l=f, ifq>α1maxx′q,q>α forgx′=∅ where gx′ denotes other candidate regions that are within the same cluster u as gx and x′ ≠ x; α1 and α2 are predefined thresholds. examples of the density computation and probability derivation are shown in figure  and the overall process of candidate validation is listed in algorithm algorithm 2: candidate validation appearance feature we observe that a region tends to comprise patches of similar textures and repetitive patterns. therefore, we choose to represent gx with bag-of-features. first, the image i that contains gx is divided into a grid of patches {p}. then for each patch, we represent its texture feature by its minimum, maximum, mean intensity, standard deviation, and a histogram of intensity differences between each pair of pixels. each patch-wise feature is then assigned a feature word. a histogram summarizing the occurrence frequencies of such feature words in gx is defined as fx. here each feature vector is normalized by the size of gx, to represent the percentages of various intensities and feature words in gx. note that if gx is a newly identified candidate during decluster processing, gx might only represent a small under-segmented portion of the actual cell nucleus due to the pixel-level labeling. therefore, to have a good summary of the actual candidate feature, we first estimate an elliptical region gxo that is a minimum volume ellipsoid covering gx. to avoid including many background pixels into gxo, we ensure gxo is part of the cluster u in which gx is detected: gxo=gxo∩u. gxo is then used in place of gx as the detected candidate, from which fx is computed. the refined elliptical regions are shown in figure 4c. appearance distance to compute the distances δ between two histogram features, the diffusion distance is used. the diffusion distance models the distance between histogram-based descriptors as heat diffusion process on a temperature field. compared to the bin-to-bin histogram distances, such as euclidean distance, the diffusion distance is able to measure cross-bin distances, avoiding explicit computation of histogram alignment. while the earth mover’s distance has similar advantages, the computation of diffusion distance is much faster, with o complexity only, where h is the number of histogram bins. contour refinement at this stage, a detected region could contain a single or multiple cell nuclei, which could be under-segmented or include extra background. we thus expect to achieve better contour delineation of cell nuclei. our idea is that, while the foreground and background are often inhomogeneous, there is always relatively good contrast between them in a local area. therefore, by performing contour refinement for each detected cell region g individually, the foreground and background can be better differentiated by analyzing the localized contrast information. we employ a regional contrast-based graphical model for the contour refinement. specifically, a conditional random field with the following energy function is designed: e=∑iη+η+ {∑iφ+∑i,i′ϕ} where g¯ denotes the detected region g  plus its surrounding area of a fixed width, and l denotes the labeling vector of all pixels in g¯. then, the model attempts to refine the contour of g by relabeling each pixel ui∈g¯ as li = {f,b}. here η is the unary contrast-based intensity term, η combined with φ is the contrast-based detection term with lg representing the detected region g, and ϕ is the spatial term associating neighboring pixels ui and ui′. the constant is set to obtain equal contributions from the unary costs +η) and the combined pairwise costs +∑i,i′ϕ). graph cut method is used to derive the most probable labeling l that minimizes the energy function, to produce the final segmentation of cell nuclei from g¯. here our customized definition of the intensity term and inclusion of the detection term are the main distinctions from the other crf constructs. the contrast-based intensity term η describes the unary costs of pixel ui labeled as li ∈ {f,b}. basically, the costs of li = f and li = b represent the inverse probabilities, and the probability pr of li = f is computed by: pr=))- fi=ii/ig λg=fg-γg where ig denotes the mean intensity of g, and pr follows a sigmoid probability distribution based on the contrast feature fi. we expect pixels with fi > λg to more likely represent the foreground. λg is computed based on fg and ⌞fg, which are the mean and minimum of all feature values {fi: ui ∈ g}, and is adjusted by γg for a balance of foreground and background partitioning in g. the parameter γg is calculated at runtime, by gradually increasing it from γ1 to γ2 with a step value Δγ, and choosing the smallest γg ∈  that does not cause the entire g to be labeled as b. with pr = 1 - pr, the cost values for both labels are: η=1-pr note that since λg would be closer to fg in most cases with small γg, it would cause portions of g to have pr < , resulting in possible under-segmentation. it is however not advisable to lower λg, due to considerable overlap between low-intensity areas in g and the background. therefore, we introduce a second contrast-based detection term to encourage labeling of li = f. an auxiliary node lg is first included to the graph with the following unary costs: η=0iflg=fng¯otherwise where ng¯ is the number of pixels in g¯, and such a large cost of lg = b ensures lg is assigned then for each pixel ui, a pairwise cost φ is computed based on the contrast ν between ii and the mean intensity of g: φ=δ·ν with δ =  if li ≠ lg and otherwise, and ν =  if ii > ig, or: ν=exp where 〈 · 〉 denotes the average euclidean distances of all such pairwise distances in g¯. in this way, pixels with pr ≈  could be better labeled with the additional cost factor; and obvious background pixels would still obtain the correct b label, with φ much lower than η. the spatial term ϕ then further enhances the delineation by encouraging spatial labeling consistencies between neighboring pixels ui and ui′. a pairwise cost for li≠li′ is thus defined as: ϕ=δ·ν where δ and ν follow eq.. such a cost function implies that pixels with more similar intensities would be more penalized if they take different labels. materials and evaluation methods three different datasets that are publicly available with segmentation ground truth are used in this study. their main properties are summarized in table  the images in the first two datasets were acquired with nuclear markers whereas the third dataset also includes the cytoplasm. detailed information can be found in. among the three, dataset has higher contrast between cell nuclei and background. datasets and have large intensity inhomogeneity and considerable degree of intensity overlapping between the cell nuclei and the background. the inclusion of cytoplasm in dataset poses more challenges. the images in dataset are preprocessed to remove the pink areas and converted to grayscale. figure  shows an example image after the preprocessing. most parameters used in this study are set to the same values for all three datasets: in initial segmentation, v1 = , v2 =  and Δv = ; in probability inference, α1 =  and α2 = ; in appearance feature, the number of histogram bins is and the number of feature words is 12; and in contour refinement, γ1 = , γ2 =  and Δγ = . while these settings are chosen empirically, using a common setting for all three datasets suggests that the method is robust to different image acquisition and manual tuning of parameters can be minimal. there are only two dataset-specific parameters. one is the patch size in appearance feature, which is 8× pixels for datasets and and 4× pixels for dataset the smaller size for dataset is chosen due to its smaller cell nuclei compared to datasets and the other parameter is c in initial segmentation, which is set to for datasets and and for dataset this ensures the contrast enhanced images in dataset would not become too bright to cause distortion. for dataset four representative images are selected to train two svm classifiers, for the cell-cluster differentiation and candidate identification. while testing is performed on all images to make the results directly comparable with the state-of-the-art, we note that the testing results are not sensitive to the selection of training data, with very similar testing results observed based on different training sets. similar procedures are performed for dataset for dataset in order to have comparable performance evaluation with, half of the images are used for training and the rest for testing. we evaluate the localization of cell nuclei by two measures. first, performance of object-level detection is evaluated by recall, precision, and accuracy: r=tp/ p=tp/ a=tp/ where tp, fn, and fp are the numbers of true positive, false negative and false positive detections of cell nuclei. given a detected region od and the ground truth mask ogt, if the overlap ratio r is at least: r=|od∩ogt|/|od∪ogt| then the detection is considered tp; and correspondingly fn and fp are determined. second, the segmentation performance is evaluated by both region- and contour-based measures, including dice, normalized sum of distances and hausdorff distance: dice=2|f∩m|/ nsd=∑ui∈d/∑ui∈d hd=maxui∈∂fd here f represents the foreground pixels identified, m is the ground truth mask, and d is the minimal euclidean distance of pixel ui to ∂m of the corresponding reference nuclei, with ∂ indicating the contour. we have compared with popular cell imaging segmentation techniques, including otsu thresholding, k-means clustering and watershed. furthermore, in view of the popularity of level set for cell imaging and our design on tackling the intensity inhomogeneities, we have experimented with a level set method that has a similar focus, using the authors’ released code, with initial contours generated using watershed method. for all methods, post-processing is conducted to remove isolated segments that are smaller than 1/ of the average size of foreground regions detected in the image. in addition, we report direct performance comparisons with the state-of-the-art results reported on the same datasets, by including the same performance measures as used in these works.  cell detection we report the object-level detection results in table  comparing the results at various stages of the methodology, the improvement is larger on dataset than dataset e.g. % increase in detection accuracy on dataset vs % increase on dataset this is because inhomogeneity is more prominent on dataset while dataset exhibits clearer contrast between the cell nuclei and the background in most images. in our evaluation, a detection is only considered as tp if the overlap ratio in eq. is at least. therefore, a largely over- or under-segmented object would be counted as fn for the second stage, and corrected after the contour refinement. this explains why although cell nuclei are detected after the decluster processing, the recall results only improve significantly after the third stage. on dataset the presence of cytoplasm causes many cell nuclei to clutter into one region during the initial segmentation; this leads to fn. the third stage better differentiates the cell nuclei and cytoplasm, and the improvement is significant with % and % increase in detection recall and precision. figure 7a gives a better overview of the overlap ratios obtained from the final localization outputs. while most cell nuclei exhibit ratios not less than, less optimal results are observed on dataset again due to the influence from the cytoplasm. after initial segmentation, decluster processing, and contour refinement. the performance improvement introduced by the iterative process of interleaving interest region extraction and image enhancement are shown in figure 8a. the higher recall suggests that such an approach is especially useful for identifying foreground regions that originally display low contrast from the background. the benefits of having candidate validation are shown in figure 8b. by filtering out interest regions that are very different from the reference regions, the detection precision thus improves by on average 2%. the recall improves by on average % only, mainly because of the same constraints imposed by the overlap ratio. to evaluate the effect of the default threshold setting α1 for candidate validation, the receiver operating characteristics curves are plotted by varying the threshold value. the probability estimates q/maxx′q from all candidate regions are included in the plot, and candidate regions with at least overlapping ratio with the ground truth are marked as foreground class and the rest as the background class. as shown in figure  the threshold setting provides a good balance between the tp and fp detections, with close to maximum tp rates. note that the numbers of true negatives here are small, hence the fp rates appear relatively high. nucleus segmentation table  summarizes the region- and contour-based segmentation results. on datasets and the decluster processing improves the dice measure by about 3% and 4%, due to better object-level labeling of candidate regions. the contour-based measures, however, are mainly enhanced at the third stage of the methodology, with on average more than half reduction in nsd and hd. this is attributed to better contour delineations based on the detection results from the first two stages. besides the mean values listed in the table, the distributions of hausdorff distances on the final localization results are also shown in figure 7b. after initial segmentation, decluster processing, and contour refinement. to further evaluate the design of the graphical model for contour refinement, the foreground probabilities for all pixels of interest are computed with the intensity term, as summarized in figure  while many pixels exhibit suitable probabilities, some background pixels, especially those in datasets and have larger foreground probabilities and would lead to misclassification. the pixel-level classification is improved by introducing the contrast-based detection and spatial terms, as shown in figure  performance comparison the localization results using the standard approaches are listed in table  with example outputs shown in figure  compared to our proposed method, the level set and watershed techniques produce the second best results for dataset especially with good contour-based measures. however, without explicitly handling high-intensity background regions, both methods result in about 3% lower detection precision. on dataset our proposed method demonstrates stronger advantages, with % increase in detection accuracy, % increase in dice coefficient and decrease in hd over the second best approach. both the level set and watershed approaches face the following challenges: difficulty separating cell nuclei from surrounding background areas with low contrast, and incapability of classifying background regions that resemble cell nuclei. on dataset the intensity inhomogeneities within the cell nuclei and the cytoplasm make it particularly difficult to achieve good segmentation. as a result, the watershed method tends to largely over-segment the cell nuclei, generating many clusters and cause low detection recall and more errors in contour delineation. the level set method based on localized energy optimization is quite effective in splitting the clusters, but is less optimal for areas with high similarity between the cell nuclei and cytoplasm. the thresholding method does not perform as well as the level set or watershed approaches, but it does outperform the clustering-based approach. compared to level set, our method achieves 7% increase in detection accuracy, % increase in dice coefficient and decrease in hd. tables  and show that our proposed method delivers better localization even using only the initial segmentation step. higher performance margins are obtained with decluster processing and contour refinement, especially on datasets and comparison between our proposed region-based progressive localization method, otsu thresholding, k-means clustering, watershed and level set. a comparison with the state-of-the-art results reported for the same datasets is summarized in table  our method achieves better results in most measures, as bold-faced in the table. on dataset more fp cell nuclei are detected compared to the level set method. it is possible that such false detections are caused by accidental highlighting of background regions during the iterative image enhancement for the initial segmentation stage. however, our method exhibits overall much better detection performance with minimal numbers of fns and only fps. the accuracy of pixel-level segmentation on dataset improves significantly, as indicated by the 5% increase in dice and rand indices over. % performance improvement of object-level accuracy over on dataset is also obtained. these observations suggest that our method is indeed quite effective in handling the intensity inhomogeneity issue that is the major cause hindering satisfactory segmentation on datasets and the improvement on the contour-based measures, i.e. on average nsd decrease and hd decrease over, also demonstrate the suitability of boundary delineation using region-based designs, i.e. the salient region extraction and graphical model-based contour refinement. ‘–’ means not available. our method is currently implemented in matlab, running on a standard pc with a -ghz dual core cpu and gb ram. the computational time is related to the number of cells and the size of cells in an image. on a × pixel image with about cell nuclei, an average s is needed for the entire localization process. this is faster than applying the level set method, which requires about s with iterations.  a fully automatic localization method for cell nuclei in microscopic images is presented in this paper. intensity inhomogeneities in cell nuclei and the background often cause unsatisfactory localization performance. not many works have been reported to address this problem in a robust manner. we propose a method that exploits various scales of data-adaptive information to tackle the intensity inhomogeneity. first, the regions of interest, i.e. cell nuclei or clusters, are extracted as salient regions with iterative contrast enhancement. then with feature-based classification and reference-based probability inference, the clusters are further processed to detect more cell nuclei and filter out spurious regions. lastly, based on regional contrast information encoded in a graphical model, the pixel-level segmentation is enhanced to create the final contours. this region-based progressive localization method has been successfully applied to three publicly available datasets, showing good object-level detection and region- and contour-based segmentation results. compared to popular approaches in this problem domain such as level sets, our method achieved consistently better performance, with on average % increase in dice coefficient and 6% increase in object-level detection accuracy. our method also outperformed the state-of-the-art with on average % and 7% improvement of region- and contour-based segmentation measures. we also suggest that the proposed method is general in nature and can be applied to other localization problems, as long as the objects of interest can be modeled as salient regions with measurable contrast from the background. as a future study, we will investigate improving the graphical model for better contour delineation. a potential approach is to incorporate an additional term as the cost of difference between the model image and the measured image, as inspired by. the model image could be derived as a convolution of a point-spread function of the microscope with an object intensity function defined based on the pixel labels. we will also investigate replacing the pixel-wise labeling with region-level processing for computational efficiency while maintaining the segmentation accuracy. other future work could explore the applicability of the proposed method on other types of images. images with nuclear membrane marker and different nuclear markers such as the green fluorescent protein, and those with higher resolution or dimension, are of particular interest. to accommodate the specific characteristics of these images, possible changes to the method are to design more comprehensive intensity and texture features to differentiate among cell structures and background, and to enhance the contour refinement with boundary constraints. competing interests the authors declare that they have no competing interests. authors’ contributions ys designed and carried out research and drafted the manuscript. wc discussed and helped to design the methodology and draft the manuscript. hh, yw and mc helped with the draft manuscript. df coordinated the designed research. all authors read and approved the final manuscript. 
we perform localization in a three-stage approach: first identify all interest regions with contrast-enhanced salient region detection, then process the clusters to identify true cell nuclei with probability estimation via feature-distance profiles of reference regions, and finally refine the contours of detected regions with regional contrast-based graphical model. to tackle the intensity inhomogeneities in cell nuclei and background, a region-based progressive localization method is proposed for cell nuclei localization in fluorescence microscopy images. for example, compared to the second best approach, on the first dataset, our method achieves and reduction in hausdorff distance and false negatives; on the second dataset that has larger intensity inhomogeneity, our method achieves 5% increase in dice coefficient and rand index; on the third dataset, our method achieves 4% increase in object-level accuracy. the proposed region-based progressive localization method is evaluated on three different datasets, with the first two containing grayscale images, and the third one comprising of color images with cytoplasm in addition to cell nuclei. segmenting cell nuclei in microscopic images has become one of the most important routines in modern biological applications. detection and segmentation, of cell nuclei is highly desirable compared to time-consuming manual processes. however, automated segmentation is challenging due to large intensity inhomogeneities in the cell nuclei and the background. we present a new method for automated progressive localization of cell nuclei using data-adaptive models that can better handle the inhomogeneity problem. the rpl method is demonstrated highly effective on three different public datasets, with on average % and 7% improvement of region- and contour-based segmentation performance over the state-of-the-art. we demonstrate performance improvement over the state-of-the-art.
 segmenting cell nuclei in microscopic images has become one of the most important routines in modern biological applications. with the vast amount of data, automatic localization, i.e. detection and segmentation, of cell nuclei is highly desirable compared to time-consuming manual processes. however, automated segmentation is challenging due to large intensity inhomogeneities in the cell nuclei and the background.  we present a new method for automated progressive localization of cell nuclei using data-adaptive models that can better handle the inhomogeneity problem. we perform localization in a three-stage approach: first identify all interest regions with contrast-enhanced salient region detection, then process the clusters to identify true cell nuclei with probability estimation via feature-distance profiles of reference regions, and finally refine the contours of detected regions with regional contrast-based graphical model. the proposed region-based progressive localization method is evaluated on three different datasets, with the first two containing grayscale images, and the third one comprising of color images with cytoplasm in addition to cell nuclei. we demonstrate performance improvement over the state-of-the-art. for example, compared to the second best approach, on the first dataset, our method achieves and reduction in hausdorff distance and false negatives; on the second dataset that has larger intensity inhomogeneity, our method achieves 5% increase in dice coefficient and rand index; on the third dataset, our method achieves 4% increase in object-level accuracy.  to tackle the intensity inhomogeneities in cell nuclei and background, a region-based progressive localization method is proposed for cell nuclei localization in fluorescence microscopy images. the rpl method is demonstrated highly effective on three different public datasets, with on average % and 7% improvement of region- and contour-based segmentation performance over the state-of-the-art. 