the principal types of cells present in the blood are red blood cells ( rbc ) , wbcs , and platelets .
the percentage of wbc subtypes observed in human blood normally ranges between the following values : neutrophils 50 - 70% , eosinophils 1 - 5% , basophils 0 - 1% , monocytes 2 - 10% , lymphocytes 20 - 45% .
the observation of whole blood smears under a microscope provides important qualitative and quantitative information regarding the diagnosis of various diseases including leukemia .
wbc identification and classification into these subtypes is frequently manually performed by experienced technicians or pathologists and involves two main elements .
the first is the qualitative study of the morphology of the wbcs which gives information about the adequacy of the smear among other parameters including morphologic features of the cells .
the second is quantitative and it consists of differential counting of at least 100 wbcs .
the accuracy of cell classification and counting is markedly affected by individual operator 's abilities and experience .
in addition , the identification and differential count of blood cells is a time - consuming , repetitive task .
substituting automatic detection of wbcs for the manual process of locating , identifying , and counting different classes of cells is an important challenge in the domain of clinical diagnostic laboratories .
examples of wbc subtypes distribution of wbc subtypes while other technologies such as automated blood analyzers can perform a rapid and inexpensive five part differential , there is no morphologic correlate with these methods which is frequently an important factor in diagnosis .
even samples which have normal differential counts may have morphologic clues to underlying conditions such as myelodysplasia , infection , or b12 or folate deficiency .
our proposed method is best viewed as a complement to the traditional laboratory analysis ( such as a cbc ) rather than a replacement .
other technologies such as flow cytometry can also perform accurate differential cell counts , but the cost using this technology for a differential count far outweighs the benefit unless phenotypic analysis is also needed .
amnis image stream offers a rather unique technology which merges flow cytometery and imaging technologies .
however , in addition to the costs associated with standard flow cytometry it is also uncertain how cellular and subcellular details in this setting which uses fluorescent antibody labeled cells would compare to those of wright staining .
however , our technique is the first known implementation of these techniques on whole slide images .
the advantages of using whole slide imaging for this study include leveraging existing hardware , enhancing the software to facilitate development of other applications , and annotating the wsi so that the cells may be classified and identified in the context of the original image .
our proposed method has been applied successfully to a large dataset and achieves good classification accuracy for all of the wbc subtypes considered .
the software is in the development stage and our initial data only evaluated normal wbcs from manually selected regions of the wsi .
our method has been implemented using matlab 2009a , a high level technical computing language on a system with intel dual - core t4200 processor , 2ghz , 4 gb ram .
the functions from the image processing toolbox and statistics toolbox have been used to develop our algorithm .
the segmentation step is crucial because the accuracy of the subsequent feature extraction and classification step depends on correct segmentation of wbcs .
it is also a difficult and challenging problem due to the complex appearance of these cells , uncertainty and inconsistencies in the microscopic image with variations in illumination .
many automatic segmentation methods have been proposed , most of them based on local image information such as histogram of regions , pixel intensity , discontinuity , and clustering techniques .
many of the segmentation algorithms introduced are based on the edge information present in images .
wbcs were segmented using active contour models ( snakes and balloons ) which were initialized using morphological operators .
this method works well only if the wbcs have dark cytoplasm and are distinctly separate from adjacent rbcs .
kumar defined a new edge operator , the teager energy operator , to highlight the nucleus boundary which is very effective for segmenting the nuclei in cell images .
they also used a simple morphological method to segment the cytoplasm from the background and the rbcs .
the cytoplasm segmentation works well when the rbcs and wbcs are not close to each other .
jiang and colleagues , introduced a novel wbc segmentation scheme by combining scale - space filtering and watershed clustering .
watershed clustering in 3-d hsv ( hue , saturation , value ) histogram is processed to extract the cytoplasm .
this method may not be sufficient in the case of high density of cells , where the rbcs are clustered close to the wbcs . to overcome the difficulty associated with the high density of cells , dorini introduced the use of some simple morphological operators and explored the scale - space properties of a toggle operator to improve the segmentation accuracy in the wbcs . to avoid leaking , a common problem in cell images due to the low contrast between nucleus , cytoplasm and background , they used a scale - space toggle operator for contour regularization . in this method
, cytoplasm segmentation presents a few limitations : the rbcs touching the wbc are also detected as part of the cytoplasm . in our method
edge detection methods do not work very well when not all cell details are sharp .
but these methods work well if the contrast between the background and the gray internal membrane of the cell is stretched using a contrast stretching filter as stated by piuri and scotti .
sadeghian 's method used zack 's simple thresholding method for cytoplasm segmentation , which is based on the fact that the color intensity of rbc in a blood image is quite different from that of cytoplasm .
a few approaches that were introduced recently dealt only with the segmentation of the nucleus of the wbc .
for instance , hamghalam and ayatollahi used histogram analysis and measurement of distance among nuclei .
rezatofighi introduced a novel method based on orthogonality theory and gram - schmidt process for segmenting the wbc nuclei .
to apply gram - schmidt orthogonalization for the segmentation of nucleus of wbc , a 3d feature vector was defined for each pixel using the rgb components of the images .
then , according to the gram - schmidt method , a weighting vector w was calculated for amplifying the desired color vectors and weakening the undesired color vectors .
. the successful segmentation of the cytoplasm along with the nucleus segmentation aids in the automatic classification of the wbcs .
pathologists traditionally report normal wbcs as classified into five classes , i.e. , monocytes , lymphocytes , neutrophils , eosinophils , and basophils .
since the chosen features affect the classifier performance , deciding which features must be used in a specific data classification problem is as important as the classifier itself .
hematology experts examine the cell shape , size , color , and texture in combination with the nuclear features .
it is important to reflect the rules and heuristics used by the hematology experts in selection of the features .
as proposed by ongun , several types of features such as intensity- and color - based features , texture - based features , and shape - based features are utilized for a robust representation of wbcs .
classification methods used in this work include k - nearest neighbors , learning vector quantization , multilayer perceptron , and support vector machine .
scotti and piuri evaluated the binary images of the cytoplasm and nucleus to characterize the feature set .
the standard set of features like area , perimeter , convex areas , solidity , major axis length , orientation , filled area , eccentricity were separately evaluated for the nucleus and the cytoplasm . in addition features like the ratio between the nucleus and the cell areas , the nuclear  rectangularity 
( ratio between the perimeter of the tightest bounding rectangle and the nuclear perimeter ) , the cell  circularity 
( ratio between the perimeter of the tightest bounding circle and the cell perimeter ) , number of lobes in nucleus , area , and mean gray - level intensity of the cytoplasm were computed .
the performance was compared using different classifiers like nearest neighbor classifiers , feed - forward neural network , radial basis function neural network , parallel classifier built with feed - forward neural network . in our method , we suggest a preliminary classification of the wbc based on the number of lobes ( single or multi - lobed ) in the nucleus along with the feature set to achieve better classification rate for each of the wbc subtypes .
our approach aims at achieving a method to identify wbcs from manually prepared , wright - stained wsi .
the manually prepared wright - stained peripheral smear is easily and routinely performed at low cost .
a manual differential count is routinely performed in many laboratories when a standard cbc has an abnormal value or when requested by a clinician .
the personnel time associated with performing a 100 cell count can range from 1 - 2 minutes per slide .
however , considering the volume of peripheral smears reviewed in a day , the time dedicated to cell counting can be considerable .
in addition , the technologist time per slide can be considerably greater when the white cell count is very low . by utilizing our existing whole slide imaging technology
, we sought to investigate the feasibility of automating the identification of wbcs from digital images .
this is a preliminary investigation in which we sought to determine if these techniques were flexible enough to use without performing additional stains or cell preparations , adding additional steps , or interrupting the current procedures or workflow .
we propose a simple segmentation scheme based on the difference in color channels and morphological operations to segment the wbcs .
two - step classification with the aid of a comprehensive feature set is helpful in realizing better accuracy rates in the classification of wbcs . in this initial work
we implemented the segmentation algorithm proposed by dorini and the features selected for classification were based on the features implemented by scotti and piuri .
the segmentation step is crucial because the accuracy of the subsequent feature extraction and classification step depends on correct segmentation of wbcs .
it is also a difficult and challenging problem due to the complex appearance of these cells , uncertainty and inconsistencies in the microscopic image with variations in illumination .
many automatic segmentation methods have been proposed , most of them based on local image information such as histogram of regions , pixel intensity , discontinuity , and clustering techniques .
many of the segmentation algorithms introduced are based on the edge information present in images .
as proposed by ongun , wbcs were segmented using active contour models ( snakes and balloons ) which were initialized using morphological operators .
this method works well only if the wbcs have dark cytoplasm and are distinctly separate from adjacent rbcs .
kumar defined a new edge operator , the teager energy operator , to highlight the nucleus boundary which is very effective for segmenting the nuclei in cell images .
they also used a simple morphological method to segment the cytoplasm from the background and the rbcs .
the cytoplasm segmentation works well when the rbcs and wbcs are not close to each other .
jiang and colleagues , introduced a novel wbc segmentation scheme by combining scale - space filtering and watershed clustering .
watershed clustering in 3-d hsv ( hue , saturation , value ) histogram is processed to extract the cytoplasm .
this method may not be sufficient in the case of high density of cells , where the rbcs are clustered close to the wbcs . to overcome the difficulty associated with the high density of cells , dorini introduced the use of some simple morphological operators and explored the scale - space properties of a toggle operator to improve the segmentation accuracy in the wbcs . to avoid leaking , a common problem in cell images due to the low contrast between nucleus , cytoplasm and background , they used a scale - space toggle operator for contour regularization . in this method
, cytoplasm segmentation presents a few limitations : the rbcs touching the wbc are also detected as part of the cytoplasm . in our method , we eliminate the rbcs before detecting the wbcs .
edge detection methods do not work very well when not all cell details are sharp .
but these methods work well if the contrast between the background and the gray internal membrane of the cell is stretched using a contrast stretching filter as stated by piuri and scotti .
sadeghian 's method used zack 's simple thresholding method for cytoplasm segmentation , which is based on the fact that the color intensity of rbc in a blood image is quite different from that of cytoplasm .
a few approaches that were introduced recently dealt only with the segmentation of the nucleus of the wbc . for instance , hamghalam and ayatollahi used histogram analysis and measurement of distance among nuclei .
rezatofighi introduced a novel method based on orthogonality theory and gram - schmidt process for segmenting the wbc nuclei . to apply gram - schmidt orthogonalization for the segmentation of nucleus of wbc ,
a 3d feature vector was defined for each pixel using the rgb components of the images .
then , according to the gram - schmidt method , a weighting vector w was calculated for amplifying the desired color vectors and weakening the undesired color vectors .
the successful segmentation of the cytoplasm along with the nucleus segmentation aids in the automatic classification of the wbcs .
pathologists traditionally report normal wbcs as classified into five classes , i.e. , monocytes , lymphocytes , neutrophils , eosinophils , and basophils .
since the chosen features affect the classifier performance , deciding which features must be used in a specific data classification problem is as important as the classifier itself .
hematology experts examine the cell shape , size , color , and texture in combination with the nuclear features .
it is important to reflect the rules and heuristics used by the hematology experts in selection of the features .
as proposed by ongun , several types of features such as intensity- and color - based features , texture - based features , and shape - based features are utilized for a robust representation of wbcs .
classification methods used in this work include k - nearest neighbors , learning vector quantization , multilayer perceptron , and support vector machine .
scotti and piuri evaluated the binary images of the cytoplasm and nucleus to characterize the feature set .
the standard set of features like area , perimeter , convex areas , solidity , major axis length , orientation , filled area , eccentricity were separately evaluated for the nucleus and the cytoplasm . in addition features like the ratio between the nucleus and the cell areas , the nuclear  rectangularity 
( ratio between the perimeter of the tightest bounding rectangle and the nuclear perimeter ) , the cell  circularity 
( ratio between the perimeter of the tightest bounding circle and the cell perimeter ) , number of lobes in nucleus , area , and mean gray - level intensity of the cytoplasm were computed .
the performance was compared using different classifiers like nearest neighbor classifiers , feed - forward neural network , radial basis function neural network , parallel classifier built with feed - forward neural network . in our method , we suggest a preliminary classification of the wbc based on the number of lobes ( single or multi - lobed ) in the nucleus along with the feature set to achieve better classification rate for each of the wbc subtypes .
our approach aims at achieving a method to identify wbcs from manually prepared , wright - stained wsi .
the manually prepared wright - stained peripheral smear is easily and routinely performed at low cost .
a manual differential count is routinely performed in many laboratories when a standard cbc has an abnormal value or when requested by a clinician .
the personnel time associated with performing a 100 cell count can range from 1 - 2 minutes per slide .
however , considering the volume of peripheral smears reviewed in a day , the time dedicated to cell counting can be considerable .
in addition , the technologist time per slide can be considerably greater when the white cell count is very low . by utilizing our existing whole slide imaging technology
, we sought to investigate the feasibility of automating the identification of wbcs from digital images .
this is a preliminary investigation in which we sought to determine if these techniques were flexible enough to use without performing additional stains or cell preparations , adding additional steps , or interrupting the current procedures or workflow .
we propose a simple segmentation scheme based on the difference in color channels and morphological operations to segment the wbcs .
two - step classification with the aid of a comprehensive feature set is helpful in realizing better accuracy rates in the classification of wbcs . in this initial work
we implemented the segmentation algorithm proposed by dorini and the features selected for classification were based on the features implemented by scotti and piuri .
the segmentation step is crucial because the accuracy of the subsequent feature extraction and classification step depends on correct segmentation of wbcs .
it is also a difficult and challenging problem due to the complex appearance of these cells , uncertainty and inconsistencies in the microscopic image with variations in illumination .
many automatic segmentation methods have been proposed , most of them based on local image information such as histogram of regions , pixel intensity , discontinuity , and clustering techniques .
many of the segmentation algorithms introduced are based on the edge information present in images .
as proposed by ongun , wbcs were segmented using active contour models ( snakes and balloons ) which were initialized using morphological operators .
this method works well only if the wbcs have dark cytoplasm and are distinctly separate from adjacent rbcs .
kumar defined a new edge operator , the teager energy operator , to highlight the nucleus boundary which is very effective for segmenting the nuclei in cell images .
they also used a simple morphological method to segment the cytoplasm from the background and the rbcs .
the cytoplasm segmentation works well when the rbcs and wbcs are not close to each other .
jiang and colleagues , introduced a novel wbc segmentation scheme by combining scale - space filtering and watershed clustering .
watershed clustering in 3-d hsv ( hue , saturation , value ) histogram is processed to extract the cytoplasm .
this method may not be sufficient in the case of high density of cells , where the rbcs are clustered close to the wbcs . to overcome the difficulty associated with the high density of cells , dorini introduced the use of some simple morphological operators and explored the scale - space properties of a toggle operator to improve the segmentation accuracy in the wbcs . to avoid leaking , a common problem in cell images due to the low contrast between nucleus , cytoplasm and background , they used a scale - space toggle operator for contour regularization . in this method
, cytoplasm segmentation presents a few limitations : the rbcs touching the wbc are also detected as part of the cytoplasm . in our method , we eliminate the rbcs before detecting the wbcs .
edge detection methods do not work very well when not all cell details are sharp .
but these methods work well if the contrast between the background and the gray internal membrane of the cell is stretched using a contrast stretching filter as stated by piuri and scotti .
sadeghian 's method used zack 's simple thresholding method for cytoplasm segmentation , which is based on the fact that the color intensity of rbc in a blood image is quite different from that of cytoplasm .
a few approaches that were introduced recently dealt only with the segmentation of the nucleus of the wbc . for instance , hamghalam and ayatollahi used histogram analysis and measurement of distance among nuclei .
rezatofighi introduced a novel method based on orthogonality theory and gram - schmidt process for segmenting the wbc nuclei . to apply gram - schmidt orthogonalization for the segmentation of nucleus of wbc ,
a 3d feature vector was defined for each pixel using the rgb components of the images .
then , according to the gram - schmidt method , a weighting vector w was calculated for amplifying the desired color vectors and weakening the undesired color vectors .
the successful segmentation of the cytoplasm along with the nucleus segmentation aids in the automatic classification of the wbcs .
pathologists traditionally report normal wbcs as classified into five classes , i.e. , monocytes , lymphocytes , neutrophils , eosinophils , and basophils .
since the chosen features affect the classifier performance , deciding which features must be used in a specific data classification problem is as important as the classifier itself .
hematology experts examine the cell shape , size , color , and texture in combination with the nuclear features .
it is important to reflect the rules and heuristics used by the hematology experts in selection of the features .
several types of features such as intensity- and color - based features , texture - based features , and shape - based features are utilized for a robust representation of wbcs .
classification methods used in this work include k - nearest neighbors , learning vector quantization , multilayer perceptron , and support vector machine .
scotti and piuri evaluated the binary images of the cytoplasm and nucleus to characterize the feature set .
the standard set of features like area , perimeter , convex areas , solidity , major axis length , orientation , filled area , eccentricity were separately evaluated for the nucleus and the cytoplasm . in addition features like the ratio between the nucleus and the cell areas , the nuclear  rectangularity 
( ratio between the perimeter of the tightest bounding rectangle and the nuclear perimeter ) , the cell  circularity 
( ratio between the perimeter of the tightest bounding circle and the cell perimeter ) , number of lobes in nucleus , area , and mean gray - level intensity of the cytoplasm were computed .
the performance was compared using different classifiers like nearest neighbor classifiers , feed - forward neural network , radial basis function neural network , parallel classifier built with feed - forward neural network . in our method , we suggest a preliminary classification of the wbc based on the number of lobes ( single or multi - lobed ) in the nucleus along with the feature set to achieve better classification rate for each of the wbc subtypes .
our approach aims at achieving a method to identify wbcs from manually prepared , wright - stained wsi .
the manually prepared wright - stained peripheral smear is easily and routinely performed at low cost .
a manual differential count is routinely performed in many laboratories when a standard cbc has an abnormal value or when requested by a clinician .
the personnel time associated with performing a 100 cell count can range from 1 - 2 minutes per slide .
however , considering the volume of peripheral smears reviewed in a day , the time dedicated to cell counting can be considerable .
in addition , the technologist time per slide can be considerably greater when the white cell count is very low . by utilizing our existing whole slide imaging technology
, we sought to investigate the feasibility of automating the identification of wbcs from digital images .
this is a preliminary investigation in which we sought to determine if these techniques were flexible enough to use without performing additional stains or cell preparations , adding additional steps , or interrupting the current procedures or workflow .
we propose a simple segmentation scheme based on the difference in color channels and morphological operations to segment the wbcs .
two - step classification with the aid of a comprehensive feature set is helpful in realizing better accuracy rates in the classification of wbcs . in this initial work
we implemented the segmentation algorithm proposed by dorini and the features selected for classification were based on the features implemented by scotti and piuri .
an aperio cs scanner ( aperio technologies , vista , ca ) outfitted with an olympus uplansapo 20 , 0.75 numerical aperture objective lens and basler l301kc trilinear - array ccd line scan camera was used to create the whole slide images .
ten sequential , wright - stained peripheral blood slides submitted for hematopathologist review were scanned at 20. slides were prepared manually according to the lab protocol and our current workflow .
a drop of blood was pulled between two slides at roughly 45 angle to create a blood film slide which was allowed to air dry .
slides were then cover slipped prior to scanning at 20. the use 20 vs. 40 is frequently an issue of discussion in digital pathology . while 20 provides faster scan times and smaller files , the image quality is better with a 40 scan .
we chose 20 as a starting point to demonstrate proof of concept with a secondary goal of comparing the performance of the algorithms on 40. scan times ranged from 5 - 10 minutes per slide .
whole slide imaging captured a technician selected area of the blood film ( ranging from 17 mm - 22 mm in height by 15 mm - 25 mm in length ) using automated focus .
images were compressed at a quality setting of 70% during the capture process with resulting file sizes averaging between 20 and 60 megabytes .
similarly , we manually selected optimal regions adjacent to feathered edge and stored the images corresponding to the thin sections of the blood smear .
while regions were manually selected in this preliminary study , additional work is being done to automate this process .
each of these images has 2 - 15 wbcs . for easy identification of the wbcs ,
the whole blood slides are stained with wright 's stain , which produces a dark intensity in nuclei of wbcs .
we convert the images from rgb ( red , green , blue ) to hsv ( hue , saturation , value ) space .
first , using the saturation image , we threshold it to approximately identify the nuclei .
a threshold value of 0.55 is used in our experiment ( saturation range = 0 - 1 ) .
the advantage of using the saturation image for thresholding is to eliminate variations in illumination that occur .
finally , we crop the part around the nucleus making sure that the whole wbc is captured .
if the distance between two nuclei is less than 5.52 m , then they belong to the same wbc .
the advantage of using only the cropped image is that the regions containing mostly rbcs ( which lack nuclei ) are eliminated .
the advantage of this method is that we do not have any false negatives in the detection of the wbcs .
we do capture some redundant cells ( false positives ) which have been stained , but we eliminate them in further processing by manually assigning them to a noise class . figure 3 depicts the flowchart for wbc detection .
flowchart for wbc detection we have implemented a novel , simple , and robust segmentation scheme for segmenting wbcs .
rbcs are mostly pink / red in color and the wbcs have a dark stained nucleus .
first we threshold ( threshold value = 0.9 255 ) the given subimage based on intensity and eliminate small regions to obtain a binary image .
thresholding and smoothing the difference in the red and blue channels helps in identifying the rbcs ( threshold value = 0.1 255 ) .
subtracting the rbcs and the background from the original image results mostly in an image of wbcs .
hence , we are left with wbcs which may have small parts of rbcs attached to them .
we fill in details lost due to our thresholding operation using a standard hole filling algorithm . using a disk - shaped structuring element we get rid of the thin parts ( morphological opening ) of the rbcs which are attached to the wbc .
once the wbc boundary has been extracted we concentrate on separating the wbc region into cytoplasm and nucleus .
this thresholding ( threshold value = 0.55 ) operation yields a binary image of the nucleus . by taking the difference between the wbc image and the nucleus image
hence this simple , extremely fast method can be used to segment the blood image to get the desired results .
flowchart for wbc segmentation wbcs can be broadly classified into cells with segmented nuclei and cells with nonsegmented nuclei as shown in figure 6 .
the nucleus shape is the key factor in deciding the class to which the wbc belongs .
we have found that deciding whether the nucleus is segmented or nonsegmented first gives a significant advantage to the final classification process .
ambiguities associated with connected nuclear lobes are resolved by detecting local maximum curvature points , detecting concavities , and partitioning them through geometric rules .
the flowchart in figure 7 gives the steps for the classification of wbc into segmented and nonsegmented wbcs .
the process of classification of the wbc into these two broad classes can be described in the following steps : examples of wbc with ( a ) segmented and ( b ) nonsegmented nucleus step 1 of classification steps representing segmented vs. nonsegmented nucleus classification , ( a ) white blood cell image ; ( b ) nucleus boundary superimposed on the white blood cell image ; ( c ) maximum curvature points ; ( d ) triangulation of maximum curvature points ; ( e ) removal of background edges ; ( f ) retaining edges if the tangents are in opposite directions ; ( g ) retaining edges if the edge vector and tangent vector are perpendicular ; ( h ) retains edges if the end points have curvatures in the same direction detection of maximum curvature points : the goal of this step is to identify local curvature maxima corresponding to sharp bends ( corners ) .
after contour extraction we compute the curvature using equation and retain the local curvature maxima points .
where x(s ) and y(s ) represent the x - coordinates and y - coordinates of the boundary points respectively . x and
y represent the first derivatives with respect to s. x  and y  represent the second derivatives with respect to s. elimination of low curvature maxima is done by calculating an adaptive threshold according to the mean curvature within a region of interest .
the region of interest ( roi ) of a maximum curvature point is defined as the segment of the contour between the two nearest curvature minima points surrounding it denoted by l1 and l2 .
the roi of each maximum curvature point is used to calculate a local threshold adaptively where p is the position of the maximum curvature point on the contour , and r is a coefficient : where   is the mean curvature of the roi and i is the index of the point on the nucleus boundary . the absolute value of the curvature is used to distinguish between low curvature maxima points against high curvature maxima points . a round corner ( low curvature maxima ) tends to have absolute curvature smaller than t(p ) , while a sharp corner ( high curvature maxima ) tends to have an absolute curvature larger than t(p ) .
the reasoning for choosing an appropriate value of r can be found in reference 12 .
delaunay triangulation of points of maximum curvature : the goal of this step is to construct the set of all potential edges that might correspond to the boundaries between the different lobes of a segmented nucleus .
the high curvature maxima found in the previous step are used as the candidate vertices for these edges .
a triangle s from t satisfies the delaunay criterion if the interior of the circumcircle through the vertices of s does not contain any points .
if all triangles satisfy the delaunay criterion , then the triangulation t is called delaunay triangulation .
we apply the delaunay triangulation ( dt ) to all points of maximum curvature to find candidate edges which potentially separate different segments of the nuclei .
a set of conditions based on observations are first checked to see if a wbc has segmented nuclei .
we eliminate all edges if the nucleus has a roundness ratio > 1 and classify the cell as nonsegmented nucleus type .
the roundness ratio is calculated as the ratio of the area and the square of the perimeter of the contour as shown in equation ( 3 ) .
a circle has the value  1  and for structures with increasing irregularity , the value tends to  0 .
 eliminate all edges if the ratio of the original area of nucleus to the convex hull of nucleus > 1 and classify the cell as of nonsegmented nucleus type . a set a is said to be convex if the straight line segment joining any two points in a lies entirely within a. the convex hull h of an arbitrary set >  is the smallest convex set containing s. find the ratio of the area of the red / pink regions to the area of the entire of wbc .
the red / pink regions are found by taking the difference between the red and blue image ( red - blue ) and thresholding it to obtain a binary image . if the ratio is > 3 , then the nucleus is classified directly without further processing as a segmented nucleus .
if there are two or more nuclei present in one cell , the nucleus is directly classified as a segmented nucleus ; if the above reasons are not satisfied , a series of geometric rules helps us in separating the cells with segmented nucleus from the cells with nonsegmented nucleus .
let sij be the edge connecting two points of maximum curvature maxima pi and pj .
ti and tj are the unit vectors representing the tangent directions at pi and sij .
the following set of rules is used : we need to retain edges that are inside the nucleus only .
hence , we eliminate edges that pass through the background and edges that intersect the boundary [ figure 8e ] . for a valid edge separating two lobes ,
the edge sij is eliminated if ti 
 tj 
 ti > th1 ; observe here that the dot product of these two tangents will be negative always if these tangents are oppositely directed [ figure 8f ] .
we eliminate edges if   if the edges and the tangents are orthogonal , the dot product of vectors along these vectors will be approximately 0 [ figure 8 g ] .
we need to ensure that the endpoints of the edges have curvatures in the same direction as in figure 9a . to check this condition
, we calculate the product of the curvature values at the endpoints of the edge .
the product is negative if the points have curvatures in different directions as shown in figure 9b .
this condition eliminates edges if the end points of the edge have opposite curvatures [ figure 8h ] .
examples to illustrate the curvature of convex and concave regions we used th2 = 0 and th2 = 0.4 .
we check the above conditions for each edge in the delaunay triangulation . after the above conditioning
, we label as segmented nucleus if the number of detected edges is larger than zero .
figure 8d shows the triangulation of the points of maximum curvature to find the candidate edges . in figure
figure 8f retains edges if the angle between tangents is close to  radians . figure
8 g retains edges if the edge vector and tangent vector are close to perpendicular .
finally , three edges are present in the end indicating that the nucleus has three segments that are interconnected . to classify the wbc to its respective subtype , we use features that describe the characteristics of the cytoplasm and the nucleus .
we choose 19 features such as area , perimeter , convex area , solidity , orientation , eccentricity , separately evaluated for the nucleus and the wbc .
the result obtained from the previous step gives us information about the broad nucleus type ( segmented or nonsegmented ) .
in addition features like  circularity  ( ratio between the perimeter of the tightest bounding circle and the nuclear perimeter ) of the nucleus , nucleus to cytoplasm ratio , ratio of nucleus area to area of wbc , entropy of the cytoplasm , and mean gray - level intensity of the cytoplasm ( all three color channels ) are computed .
we use a linear discriminant in this six - dimensional space to classify the data to their respective type .
linear discriminant analysis ( lda ) is used to find a linear combination of the features which characterizes or separates these five classes of wbcs .
the classifier is biased using the number of samples in each class . the system is evaluated using 10-fold cross - validation .
cross - validation is a technique for assessing how the results of a statistical analysis will generalize to an independent dataset .
it is mainly used in settings where the goal is prediction , and one wants to estimate how accurately a predictive model will perform in practice .
one round of cross - validation involves partitioning a sample of data into complementary subsets , performing the analysis on one subset ( called the training set ) , and validating the analysis on the other subset ( called the validation set or testing set ) . to reduce variability , multiple rounds of cross - validation
are performed using different partitions , and the validation results are averaged over the rounds .
the functions from the statistics toolbox in matlab have been used to analyze the data .
an aperio cs scanner ( aperio technologies , vista , ca ) outfitted with an olympus uplansapo 20 , 0.75 numerical aperture objective lens and basler l301kc trilinear - array ccd line scan camera was used to create the whole slide images .
ten sequential , wright - stained peripheral blood slides submitted for hematopathologist review were scanned at 20. slides were prepared manually according to the lab protocol and our current workflow .
a drop of blood was pulled between two slides at roughly 45 angle to create a blood film slide which was allowed to air dry .
slides were then cover slipped prior to scanning at 20. the use 20 vs. 40 is frequently an issue of discussion in digital pathology . while 20 provides faster scan times and smaller files , the image quality is better with a 40 scan .
we chose 20 as a starting point to demonstrate proof of concept with a secondary goal of comparing the performance of the algorithms on 40. scan times ranged from 5 - 10 minutes per slide .
whole slide imaging captured a technician selected area of the blood film ( ranging from 17 mm - 22 mm in height by 15 mm - 25 mm in length ) using automated focus .
images were compressed at a quality setting of 70% during the capture process with resulting file sizes averaging between 20 and 60 megabytes .
similarly , we manually selected optimal regions adjacent to feathered edge and stored the images corresponding to the thin sections of the blood smear .
while regions were manually selected in this preliminary study , additional work is being done to automate this process .
each of these images has 2 - 15 wbcs . for easy identification of the wbcs ,
the whole blood slides are stained with wright 's stain , which produces a dark intensity in nuclei of wbcs .
we convert the images from rgb ( red , green , blue ) to hsv ( hue , saturation , value ) space .
first , using the saturation image , we threshold it to approximately identify the nuclei .
a threshold value of 0.55 is used in our experiment ( saturation range = 0 - 1 ) .
the advantage of using the saturation image for thresholding is to eliminate variations in illumination that occur .
finally , we crop the part around the nucleus making sure that the whole wbc is captured .
if the distance between two nuclei is less than 5.52 m , then they belong to the same wbc .
the advantage of using only the cropped image is that the regions containing mostly rbcs ( which lack nuclei ) are eliminated .
the advantage of this method is that we do not have any false negatives in the detection of the wbcs .
we do capture some redundant cells ( false positives ) which have been stained , but we eliminate them in further processing by manually assigning them to a noise class . figure 3 depicts the flowchart for wbc detection .
flowchart for wbc detection we have implemented a novel , simple , and robust segmentation scheme for segmenting wbcs .
rbcs are mostly pink / red in color and the wbcs have a dark stained nucleus .
first we threshold ( threshold value = 0.9 255 ) the given subimage based on intensity and eliminate small regions to obtain a binary image .
thresholding and smoothing the difference in the red and blue channels helps in identifying the rbcs ( threshold value = 0.1 255 ) .
subtracting the rbcs and the background from the original image results mostly in an image of wbcs .
hence , we are left with wbcs which may have small parts of rbcs attached to them .
we fill in details lost due to our thresholding operation using a standard hole filling algorithm . using a disk - shaped structuring element we get rid of the thin parts ( morphological opening ) of the rbcs which are attached to the wbc .
once the wbc boundary has been extracted we concentrate on separating the wbc region into cytoplasm and nucleus .
this thresholding ( threshold value = 0.55 ) operation yields a binary image of the nucleus . by taking the difference between the wbc image and the nucleus image
hence this simple , extremely fast method can be used to segment the blood image to get the desired results .
wbcs can be broadly classified into cells with segmented nuclei and cells with nonsegmented nuclei as shown in figure 6 .
the nucleus shape is the key factor in deciding the class to which the wbc belongs .
we have found that deciding whether the nucleus is segmented or nonsegmented first gives a significant advantage to the final classification process .
ambiguities associated with connected nuclear lobes are resolved by detecting local maximum curvature points , detecting concavities , and partitioning them through geometric rules .
the flowchart in figure 7 gives the steps for the classification of wbc into segmented and nonsegmented wbcs .
the process of classification of the wbc into these two broad classes can be described in the following steps : examples of wbc with ( a ) segmented and ( b ) nonsegmented nucleus step 1 of classification steps representing segmented vs. nonsegmented nucleus classification , ( a ) white blood cell image ; ( b ) nucleus boundary superimposed on the white blood cell image ; ( c ) maximum curvature points ; ( d ) triangulation of maximum curvature points ; ( e ) removal of background edges ; ( f ) retaining edges if the tangents are in opposite directions ; ( g ) retaining edges if the edge vector and tangent vector are perpendicular ; ( h ) retains edges if the end points have curvatures in the same direction detection of maximum curvature points : the goal of this step is to identify local curvature maxima corresponding to sharp bends ( corners ) .
we utilize global and local curvature properties in extracting the maximum curvature points . after contour extraction
we compute the curvature using equation and retain the local curvature maxima points . where x(s ) and y(s ) represent the x - coordinates and y - coordinates of the boundary points respectively . x and
y represent the first derivatives with respect to s. x  and y  represent the second derivatives with respect to s. elimination of low curvature maxima is done by calculating an adaptive threshold according to the mean curvature within a region of interest .
the region of interest ( roi ) of a maximum curvature point is defined as the segment of the contour between the two nearest curvature minima points surrounding it denoted by l1 and l2 .
the roi of each maximum curvature point is used to calculate a local threshold adaptively where p is the position of the maximum curvature point on the contour , and r is a coefficient : where   is the mean curvature of the roi and i is the index of the point on the nucleus boundary . the absolute value of the curvature is used to distinguish between low curvature maxima points against high curvature maxima points . a round corner ( low curvature maxima ) tends to have absolute curvature smaller than t(p ) , while a sharp corner ( high curvature maxima ) tends to have an absolute curvature larger than t(p ) .
the reasoning for choosing an appropriate value of r can be found in reference 12 .
delaunay triangulation of points of maximum curvature : the goal of this step is to construct the set of all potential edges that might correspond to the boundaries between the different lobes of a segmented nucleus
. the high curvature maxima found in the previous step are used as the candidate vertices for these edges .
a triangle s from t satisfies the delaunay criterion if the interior of the circumcircle through the vertices of s does not contain any points .
if all triangles satisfy the delaunay criterion , then the triangulation t is called delaunay triangulation .
we apply the delaunay triangulation ( dt ) to all points of maximum curvature to find candidate edges which potentially separate different segments of the nuclei .
a set of conditions based on observations are first checked to see if a wbc has segmented nuclei .
we eliminate all edges if the nucleus has a roundness ratio > 1 and classify the cell as nonsegmented nucleus type .
the roundness ratio is calculated as the ratio of the area and the square of the perimeter of the contour as shown in equation ( 3 ) .
a circle has the value  1  and for structures with increasing irregularity , the value tends to  0 .
 eliminate all edges if the ratio of the original area of nucleus to the convex hull of nucleus > 1 and classify the cell as of nonsegmented nucleus type . a set
a is said to be convex if the straight line segment joining any two points in a lies entirely within a. the convex hull h of an arbitrary set >  is the smallest convex set containing s. find the ratio of the area of the red / pink regions to the area of the entire of wbc .
the red / pink regions are found by taking the difference between the red and blue image ( red - blue ) and thresholding it to obtain a binary image . if the ratio is > 3 , then the nucleus is classified directly without further processing as a segmented nucleus .
if there are two or more nuclei present in one cell , the nucleus is directly classified as a segmented nucleus ; if the above reasons are not satisfied , a series of geometric rules helps us in separating the cells with segmented nucleus from the cells with nonsegmented nucleus .
let sij be the edge connecting two points of maximum curvature maxima pi and pj .
ti and tj are the unit vectors representing the tangent directions at pi and sij .
the following set of rules is used : we need to retain edges that are inside the nucleus only .
hence , we eliminate edges that pass through the background and edges that intersect the boundary [ figure 8e ] . for a valid edge separating two lobes ,
the edge sij is eliminated if ti 
 tj 
 ti > th1 ; observe here that the dot product of these two tangents will be negative always if these tangents are oppositely directed [ figure 8f ] .
we eliminate edges if   if the edges and the tangents are orthogonal , the dot product of vectors along these vectors will be approximately 0 [ figure 8 g ] .
we need to ensure that the endpoints of the edges have curvatures in the same direction as in figure 9a . to check this condition
, we calculate the product of the curvature values at the endpoints of the edge .
the product is negative if the points have curvatures in different directions as shown in figure 9b .
this condition eliminates edges if the end points of the edge have opposite curvatures [ figure 8h ] .
examples to illustrate the curvature of convex and concave regions we used th2 = 0 and th2 = 0.4 .
we check the above conditions for each edge in the delaunay triangulation . after the above conditioning
, we label as segmented nucleus if the number of detected edges is larger than zero .
figure 8d shows the triangulation of the points of maximum curvature to find the candidate edges . in figure
figure 8 g retains edges if the edge vector and tangent vector are close to perpendicular .
finally , three edges are present in the end indicating that the nucleus has three segments that are interconnected .
a set of conditions based on observations are first checked to see if a wbc has segmented nuclei .
we eliminate all edges if the nucleus has a roundness ratio > 1 and classify the cell as nonsegmented nucleus type .
the roundness ratio is calculated as the ratio of the area and the square of the perimeter of the contour as shown in equation ( 3 ) .
a circle has the value  1  and for structures with increasing irregularity , the value tends to  0 .
 eliminate all edges if the ratio of the original area of nucleus to the convex hull of nucleus > 1 and classify the cell as of nonsegmented nucleus type . a set
a is said to be convex if the straight line segment joining any two points in a lies entirely within a. the convex hull h of an arbitrary set >  is the smallest convex set containing s. find the ratio of the area of the red / pink regions to the area of the entire of wbc .
the red / pink regions are found by taking the difference between the red and blue image ( red - blue ) and thresholding it to obtain a binary image . if the ratio is > 3 , then the nucleus is classified directly without further processing as a segmented nucleus .
if there are two or more nuclei present in one cell , the nucleus is directly classified as a segmented nucleus ;
if the above reasons are not satisfied , a series of geometric rules helps us in separating the cells with segmented nucleus from the cells with nonsegmented nucleus .
let sij be the edge connecting two points of maximum curvature maxima pi and pj .
ti and tj are the unit vectors representing the tangent directions at pi and sij .
the following set of rules is used : we need to retain edges that are inside the nucleus only .
hence , we eliminate edges that pass through the background and edges that intersect the boundary [ figure 8e ] . for a valid edge separating two lobes , the tangent vectors at the endpoints are in opposite directions .
the edge sij is eliminated if ti 
 tj 
 ti > th1 ; observe here that the dot product of these two tangents will be negative always if these tangents are oppositely directed [ figure 8f ] .
we eliminate edges if   if the edges and the tangents are orthogonal , the dot product of vectors along these vectors will be approximately 0 [ figure 8 g ] .
we need to ensure that the endpoints of the edges have curvatures in the same direction as in figure 9a . to check this condition
, we calculate the product of the curvature values at the endpoints of the edge .
the product is negative if the points have curvatures in different directions as shown in figure 9b
. this condition eliminates edges if the end points of the edge have opposite curvatures [ figure 8h ] .
examples to illustrate the curvature of convex and concave regions we used th2 = 0 and th2 = 0.4 .
we check the above conditions for each edge in the delaunay triangulation . after the above conditioning
, we label as segmented nucleus if the number of detected edges is larger than zero .
figure 8d shows the triangulation of the points of maximum curvature to find the candidate edges . in figure
figure 8f retains edges if the angle between tangents is close to  radians . figure
8 g retains edges if the edge vector and tangent vector are close to perpendicular .
finally , three edges are present in the end indicating that the nucleus has three segments that are interconnected .
to classify the wbc to its respective subtype , we use features that describe the characteristics of the cytoplasm and the nucleus . we choose 19 features such as area , perimeter , convex area , solidity , orientation , eccentricity , separately evaluated for the nucleus and the wbc .
the result obtained from the previous step gives us information about the broad nucleus type ( segmented or nonsegmented ) .
this result is a novel binary feature added to our classifier . in addition features like
( ratio between the perimeter of the tightest bounding circle and the nuclear perimeter ) of the nucleus , nucleus to cytoplasm ratio , ratio of nucleus area to area of wbc , entropy of the cytoplasm , and mean gray - level intensity of the cytoplasm ( all three color channels ) are computed .
we use a linear discriminant in this six - dimensional space to classify the data to their respective type
. linear discriminant analysis ( lda ) is used to find a linear combination of the features which characterizes or separates these five classes of wbcs .
the classifier is biased using the number of samples in each class . the system is evaluated using 10-fold cross - validation .
cross - validation is a technique for assessing how the results of a statistical analysis will generalize to an independent dataset .
it is mainly used in settings where the goal is prediction , and one wants to estimate how accurately a predictive model will perform in practice .
one round of cross - validation involves partitioning a sample of data into complementary subsets , performing the analysis on one subset ( called the training set ) , and validating the analysis on the other subset ( called the validation set or testing set ) . to reduce variability , multiple rounds of cross - validation
are performed using different partitions , and the validation results are averaged over the rounds .
the functions from the statistics toolbox in matlab have been used to analyze the data .
this system has been tested using images obtained from the arup laboratory , university of utah .
our dataset consists of 320 images at 20 magnification that contain 1938 expert labeled wbcs .
the false positives consist of regions of artifact or debris that have stain and color characteristics which fall within the thresholding parameters .
the performance of the segmentation algorithm was evaluated by comparing our proposed method with a hematologist 's visual segmentation .
the segmentation algorithm was applied to 1938 subimages of wbcs ; 1804 of them were accurately segmented .
some of the examples present segmentation results for more complex images , with a complicated background and containing several rbcs close to the wbc .
the resulting dataset ( 1938(number of cells)19 ( features ) ) has been used to determine the subtype of the wbc segmented using linear discriminants as described in the methods section .
we observed that there is a lot of misclassification between various classes as seen in table 1 .
the rows in the confusion matrix represent the real subtype and the columns represented the detected subtype .
an initial classification of the wbcs into cells with a segmented nucleus ( neutrophils , eosinophils , basophils ) and cells without nuclear segmentation ( lymphocyte , monocyte ) improves the performance of the classification .
confusion matrix for the comparison method ( five subtypes ) rows represent the correct classification by experts . for each row ,
diagonal entries indicate correct classification and are indicated in bold confusion matrix using our novel two - step classification ( five subtypes ) .
diagonal entries indicate correct classification and are indicated in bold this five subtype classification is generally performed by pathologists .
we obtain very good classification results for the given dataset as seen in table 2 .
experiments show that the two - step classification implemented achieves a 93.9% overall accuracy in the five subtype classification .
results indicate that the morphological analysis of white blood cells is achievable and it offers good classification accuracy .
a higher segmentation accuracy would result in lowering the error rate in the classification process .
this system has been tested using images obtained from the arup laboratory , university of utah .
our dataset consists of 320 images at 20 magnification that contain 1938 expert labeled wbcs .
the false positives consist of regions of artifact or debris that have stain and color characteristics which fall within the thresholding parameters .
the performance of the segmentation algorithm was evaluated by comparing our proposed method with a hematologist 's visual segmentation .
the segmentation algorithm was applied to 1938 subimages of wbcs ; 1804 of them were accurately segmented .
some of the examples present segmentation results for more complex images , with a complicated background and containing several rbcs close to the wbc .
the resulting dataset ( 1938(number of cells)19 ( features ) ) has been used to determine the subtype of the wbc segmented using linear discriminants as described in the methods section .
we observed that there is a lot of misclassification between various classes as seen in table 1 .
the rows in the confusion matrix represent the real subtype and the columns represented the detected subtype .
an initial classification of the wbcs into cells with a segmented nucleus ( neutrophils , eosinophils , basophils ) and cells without nuclear segmentation ( lymphocyte , monocyte ) improves the performance of the classification .
confusion matrix for the comparison method ( five subtypes ) rows represent the correct classification by experts . for each row , the columns represent the classification made by the approach used in the comparison method .
diagonal entries indicate correct classification and are indicated in bold confusion matrix using our novel two - step classification ( five subtypes ) .
diagonal entries indicate correct classification and are indicated in bold this five subtype classification is generally performed by pathologists .
we obtain very good classification results for the given dataset as seen in table 2 .
experiments show that the two - step classification implemented achieves a 93.9% overall accuracy in the five subtype classification .
results indicate that the morphological analysis of white blood cells is achievable and it offers good classification accuracy .
a higher segmentation accuracy would result in lowering the error rate in the classification process .
there are currently wbc classification systems based on morphology in clinical use such as the cellavision system .
however , the classification performed by cellavision utilizes small snap shot images obtained directly from a glass slide .
in contrast the techniques described in this paper allow identification of wbcs from a whole slide image .
the use of wsi technology allows some advantages over the cellavision system including leveraging of capital equipment , expansion of software to include other applications involving wsi , and identifying and localizing the wbcs in the context of the original wsi . while the localization functions were not part of the current study , further work is currently being pursued to implement some of these benefits . viewing the classified cells in their original context on wright stained wsi allows the pathologist to also evaluate red blood cells , platelets , and smear adequacy data in addition to the wbcs .
this technique may facilitate work flow by replacing tedious and monotonous work with automation while maintaining all of the digitized data in the original smear .
even though the time to scan and analyze the slide may is longer than a manual differential count , the time associated with pathologist review may actually be decreased by completing the automated analysis prior to the pathologist spending any time on the case .
in addition , scanning and classifying the cells is mostly machine driven and requires very little hands on time , thus allowing personnel to perform other duties .
this is a report of a developmental step to assess the viability of a larger scale automated solution .
the methodology achieves a semiautomated system for the detection and classification of normal wbcs from scanned wsi .
experiments show that the two - step classification implemented achieves good overall accuracy in the five subtype classification .
results indicate that the morphological analysis of white blood cells is achievable and it offers good classification accuracy .
further studies will be focused on detecting and segmenting abnormal wbcs , comparison of 20 and 40 data , and expanding the applications for bone marrow aspirates . in the future
, we will also perform the elimination of false positives in wbc detection in an automated manner using linear discriminants on the same feature set used for classification .
 introduction : an automated system for differential white blood cell ( wbc ) counting based on morphology can make manual differential leukocyte counts faster and less tedious for pathologists and laboratory professionals . 
 we present an automated system for isolation and classification of wbcs in manually prepared , wright stained , peripheral blood smears from whole slide images ( wsi).methods : a simple , classification scheme using color information and morphology is proposed . 
 the performance of the algorithm was evaluated by comparing our proposed method with a hematopathologist 's visual classification . 
 the isolation algorithm was applied to 1938 subimages of wbcs , 1804 of them were accurately isolated . 
 then , as the first step of a two - step classification process , wbcs were broadly classified into cells with segmented nuclei and cells with nonsegmented nuclei . 
 the nucleus shape is one of the key factors in deciding how to classify wbcs . 
 ambiguities associated with connected nuclear lobes are resolved by detecting maximum curvature points and partitioning them using geometric rules . 
 the second step is to define a set of features using the information from the cytoplasm and nuclear regions to classify wbcs using linear discriminant analysis . 
 this two - step classification approach stratifies normal wbc types accurately from a whole slide image.results:system evaluation is performed using a 10-fold cross - validation technique . 
 confusion matrix of the classifier is presented to evaluate the accuracy for each type of wbc detection . 
 experiments show that the two - step classification implemented achieves a 93.9% overall accuracy in the five subtype classification.conclusion:our methodology achieves a semiautomated system for the detection and classification of normal wbcs from scanned wsi . 
 further studies will be focused on detecting and segmenting abnormal wbcs , comparison of 20 and 40 data , and expanding the applications for bone marrow aspirates . 