# Fast Background Elimination in Fluorescence Microbiology Images: Comparison of Four Algorithms

Shan Gong, Antonio Artés-Rodríguez

► To cite this version:

Shan Gong, Antonio Artés-Rodríguez. Fast Background Elimination in Fluorescence Microbiology Images: Comparison of Four Algorithms. 12th Engineering Applications of Neural Networks (EANN 2011) and 7th Artificial Intelligence Applications and Innovations (AIAI), Sep 2011, Corfu, Greece. pp.285-290, 10.1007/978-3-642-23960-1_34. hal-01571482

HAL Id: hal-01571482

https://inria.hal.science/hal-01571482

Submitted on 2 Aug 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Fast Background Elimination in Fluorescence Microbiology Images: Comparison of Four Algorithms

Shan Gong and Antonio Artés-Rodríguez *

Department of Signal Theory and Communications
Universidad de Carlos III de Madrid
{concha,antonio}@tsc.uc3m.es

**Abstract.** In this work, we investigate a fast background elimination front-end of an automatic bacilli detection system. This background eliminating system consists of a feature descriptor followed by a linear-SVMs classifier. Four state-of-the-art feature extraction algorithms are analyzed and modified. Extensive experiments have been made on real sputum fluorescence images and the results reveal that 96.92% of the background content can be correctly removed from one image with an acceptable computational complexity.

**Key words:** Tuberculosis bacilli, background remove, object detection, fluorescence image

## 1 Introduction

Tuberculosis is a contagious disease that causes thousands of death every year around the whole word. Sputum specimens are firstly stained with fluorescent dye and fluorescence images can be obtained by scanning these specimens with a fluorescence microscope. Since conventional diagnosis require amount of human efforts, detecting Tuberculosis bacilli automatically has drove many attentions [1], [2], [3]. All of these systems use a specific segmentation method to highlight the bacilli regions from the original image.

However, all these methods are based on statistical analysis of the profile and color information of bacilli, so they have disadvantages: *Poor generalization ability*. Stained M. tuberculosis bacilli display in images with varying appearances. Thus in [1], [2], [3], both color and contour features have been considered. But, statistical knowledge about the shape and color of the labeled bacilli is limited to dataset which is sensitive to experimental conditions such as luminance and camera types. *Poor ability dealing with outliers*. The contrast between bacilli and the background is regularly manifest given that stained bacilli fluoresce in the range between green and yellow up to white. In this case, color segmentation might be able to produce desired results. Nevertheless, when one bacteria

* This work has been partly supported by Ministerio de Educación of Spain (projects 'DEIPRO', id. TEC2009-14504-C02-01, and 'COMONSENS', id. CSD2008-00010).
is overlapped by other debris objects such as cells, or when images have been
taken under inapplicable luminance conditions, similar segmentation attempts
may be proved futile.

<figure><img src="image_2.png" /><figcaption>**Fig. 1.** The input original image passes through a front-end quick background subtraction system and drops most of background content. The filtered image (i.e., retained potential bacteria region) will be reanalyzed and reclassified with the purpose of improving the specificity of the whole system.</figcaption></figure>

Instead of getting involved in intricate segmentation algorithm investigation,
we propose a quick background subtraction system as a front-end of the M. tu-
berculosis bacillus detection system shown in Fig.1. This background elimination
system is termed as a coarse feature descriptor followed by a binary classification
(i.e., bacillus or background). The essential idea of designing such a front-end is
to reduce the computational complexity while maintaining elevated sensibility
and compensatory specificity. As the majority of the original image has been
disposed by the front-end processor, steps of locating bacilli are only required
to take on the retained possible regions which will dramatically accelerate the
whole detection process. In section 2, four feature extraction algorithms: SIFT,
color SIFT, Haralick Features and Histograms of Colors are modified to suit our
M. bacillus recognition task. Section 3 gives experimental results based on real
fluorescence images.

## 2 Feature Extraction Algorithms

Patches are extracted by sliding a local window across the image and classified
as positive (i.e., presence of object) or negative (i.e, absence of object). Unlike
articulated objects (e.g., pedestrians), bacilli do not contain a lot of structural
or intra-class variations, specially scale variations, thus, it is reasonable to use
single fixed-size sliding window. Characters based on small sliding window are
more discriminative, but computational expensive. On the contrary, big sliding
window avoids expensive computations by allowing more flexible moving steps,
however, as more background pixels have been included, the results are less
reliable. Given that bacillus generally is of size up to 20 × 20, in our experiment the size of patches has been chosen as 36 × 36.

Patch classifier is chosen as SVMs with linear kernel [4], [5].

## 2.1 SIFT and Color Based SIFT

The original SIFT [6] descriptor initials with a keypoint detector followed by a local image descriptor. Different with the general image matching problem, in our case, neither multiscale invariability nor distinctive points localization should be necessarily required. Smooth the *i*-th color component $I(x, y, i)$ of one image with a variable-scale Gaussian mask $G(x, y, \sigma)$,

$$L(x, y, i, \sigma) = G(x, y, \sigma) * I(x, y, i), \quad 1 \le i \le c. \qquad (1)$$

$$G(x, y, \sigma) = \frac{1}{2\pi\sigma^2} e^{-(x^2+y^2)/2\sigma^2},$$

where * is a convolution operator.

Then the gradient of the Gaussian mask smoothed image can be written as

$$\begin{aligned} J(x, y, i) &= \nabla[L(x, y, i, \sigma)] = \nabla[G(x, y, \sigma) * I(x, y, i)] \\ &= [\nabla G(x, y, \sigma)] * I(x, y, i), \quad 1 \le i \le c. \end{aligned} \qquad (2)$$

where $\nabla$ is the derivative operator, $\sigma$ indicates the width of the Gaussian and determines the quantity of smoothing (i.e., the gradient can be obtained by convolving the image with the horizontal and vertical derivatives of the Gaussian kernel function).

SIFT features for the *i*-th color channel of the patch $p(u, v; c)$ are formed by computing the gradient $J(u, v, i)$ at each pixel of the patch. The gradient orientation is then weighted by its magnitude. This patch is then split into 4 × 4 grid, in each tile a gradient orientation histogram is formed by adding the weighted gradient value to one of the 8 orientation histogram bins.

An extension is CI-SIFT which embeds color information to conventional SIFT descriptor by applying the five color invariant sets presented in Table 3 of [7].

## 2.2 Histogram of Color

Color signatures are extracted by using the histograms of color method proposed in [8].

## 2.3 Haralick Features

Based on co-occurrence matrix, several statistics were proposed in [9], known as *Haralick features*.

Co-occurrence matrix of a *n* bits gray-level image is of size $2^n \times 2^n$. One way to reduce the dimension of this matrix is to quantitate the image. The quantization for images of multidimensional color space can be done with *K*-means clustering algorithm. In the case of images of unit color component, uniform quantization can be employed.
# 3 Patches Classification Experiments

## 3.1 Datasets Description

Images are required from a 20x fluorescence Nikon microscope with a prior au-
tomatic slide loader and a Retiga 2000R camera with acquisition time 0.6 sec,
gain five and zero offset. For each patient, 200 RGB 1200 × 1600 (8 bits per
color) images have been taken and stored. 4,987 bacilli of 952 images of 45 in-
fected patients have been identified and labeled manually by experts. Patches
centered at each bacteria haven been extracted. Furthermore, bacilli have been
rotated 90, 180 and 270 grades so as to enrich the dataset. Finally, a positive
dataset of 19,890 patches has been generated. Regarding the negative dataset,
100,031 negative patch samples haven been created by arbitrarily moving the
local windows in images of a couple of healthy patients.

In the following experiments, 3,000 positive as well as negative randomly
selected patches are used for training step and all the rest for test. Results are
obtained by averaging 100 runs of a linear SVM. Due to space limitation, full
detailed results can be found in technical report [10].

## 3.2 Experiments

**Experiment E01: Without any feature descriptor.** For the purpose of com-
parison, in this experiment none feature extraction processes have been taken.
Patches are considered to be of size 20 × 20. PCA has been used to reduce
the feature dimension. And, only the *BLUE* component has been analyzed. As
long as the patch is small enough, even without feature extraction, one simple
linear-SVM classifier is capable to achieve a sensibility up to 93.6%.

**Experiment E02: SIFT.** Both the sensibility and specificity are compara-
ble regardless of the color spaces. For white color bacilli surrounded by yellow
tissues, *BLUE* is the only discriminative in the *RGB* color space, which may
explain why *BLUE* component obtains the optimal results.

**Experiment E03: CI-SIFT.** In [7], the author stated that the degree of
invariance of the five color invariant sets have the following relation: $E_{\omega} < W_{\omega} <$
$C_{\omega} < N_{\omega} < H_{\omega}$. $H_{\omega}$ which is the most robust to changes in imaging condition
(see Table 1 in [7]) had the lowest discriminative power while $E_{\omega}$ which is the
most sensitive to any changes achieved the best recognition precision.

**Experiment E04: HF-without Quantization** It is always better not to
normalize features sets obtained from the four adjacently measurements (see Fig.
2 of [10]) regardless of the implemented color spaces.

**Experiment E05: HF-with Quantization.** Quantization (see section 2.3)
is firstly employed for each patch. 1,000 positive as well as negative patches have
been picked randomly from the entire data set from which codebooks of colors are
constructed with *K*-means clustering method. Regarding unit dimensional color
spaces, uniform quantization method also has been studied. Excluding *RGB*,
sensibility as well as specificity obtained in all the unit dimensional color spaces
in this experiment are comparable to or even higher than those obtained in the
previous experiment which reveals that quantizations are highly recommended. Furthermore, uniform quantization is much more reliable than *K*-means quantization.

## 4 Conclusion and Future Line

<figure><img src="image_3.png" /><figcaption>**Fig. 2.** The left figure is a fluorescence image with one bacteria. The right figure is the filtered image in which the white regions are detected potential bacilli locations, with the real bacteria bounded by the red rectangular for purpose of reference. Parameters used here are: SIFT descriptor on green color channel with step of the moving window equal to 1. Most of the pixels of the image have been classified as background (i.e., the black regions).</figcaption></figure>

In this paper, we have analyzed the performance of several feature extraction algorithms. In Table 1 are the optimal cases for each algorithm. The *BLUE* scale state-of-the-art SIFT-like descriptor outperforms the others. The specificity achieved by Histograms of colors method is more than 15% lower than the average level. *CI-SIFT* as well as *HF* take use both the geometrical and color signatures and are capable to obtain appealing results. Small patch may contain more discriminative signatures, but we aim to reduce the background as fast as possible which means that with the same accuracy bigger patches are always recommended. Since the computational complexity of most of attributes of the Haralick feature sets are *O*(number of bits of the image²), the quantization step considerably accelerate the algorithm. Another conclusion we can make is that the *BLUE* channel is the most robust compared with other color representations. By applying this background elimination process, approximated 96.92% background pixels can be subtracted from the original image. One example is given in Fig. 2.

One possible way to improve sensibility without degrading specificity is to use a different evaluation function (such as F-score, AUC) of the linear SVM classifier or add other simple implemented characters into the feature set.
<table><caption>**Table 1.** Comparison of all the algorithms</caption><thead><tr><th rowspan="2">method</th><th rowspan="2">parameters</th><th rowspan="2">sensibility</th><th rowspan="2">specificity</th><th colspan="3">computational complexity (approximated number of executions per patch)</th></tr><tr><th>+</th><th>×</th><th>others</th></tr></thead><tbody><tr><td>SIFT</td><td>Blue</td><td>95.199(0.187)</td><td>97.234(0.151)</td><td>11l<sup>2</sup></td><td>12l<sup>2</sup></td><td>2l<sup>2</sup></td></tr><tr><td rowspan="2">HoColor</td><td>RGB, NumBin = 8</td><td>93.418(0.357)</td><td>79.672(0.464)</td><td>0</td><td>0</td><td>3l<sup>2</sup></td></tr><tr><td>BLUE, NumBin = 8</td><td>91.097(0.484)</td><td>75.931(0.525)</td><td>0</td><td>0</td><td>l<sup>2</sup></td></tr><tr><td>CI-SIFT</td><td>E<sub>ω</sub></td><td>94.000(0.272)</td><td>96.949(0.167)</td><td>30l<sup>2</sup></td><td>31l<sup>2</sup></td><td>2l<sup>2</sup></td></tr><tr><td rowspan="2">HF</td><td>RGB, no normalization</td><td>95.260(0.312)</td><td>93.704(0.256)</td><td>75k<sup>2</sup></td><td>66k<sup>2</sup></td><td>21k<sup>2</sup></td></tr><tr><td>BLUE, no normalization</td><td>93.792(0.232)</td><td>93.409(0.218)</td><td>25k<sup>2</sup></td><td>22k<sup>2</sup></td><td>7k<sup>2</sup></td></tr><tr><td>HF</td><td>BLUE, codebook = 8, uniform Quantization, no normalization</td><td>94.845(0.256)</td><td>92.577(0.272)</td><td>l<sup>2</sup> + 25k<sub>Q</sub><sup>2</sup></td><td>l<sup>2</sup> + 22k<sub>Q</sub><sup>2</sup></td><td>6k<sub>Q</sub><sup>2</sup></td></tr><tr><td>None feature extraction</td><td>BLUE, patch size = 20×20, feature vector is of size 1 × 400</td><td>93.617(0.307)</td><td>97.046(0.189)</td><td>0</td><td>0</td><td>0</td></tr></tbody><tfoot><tr><td colspan="7">where l = 36, k = 2<sup>8</sup>, k<sub>Q</sub> = 2<sup>3</sup>. And for HF the complexity is considered for one single adjacency measurement</td></tr></tfoot></table>

## References

1. Osman, M.K., Mashor, M.Y., Saad, Z., Jaafar, H.: Colour image segmentation of Tuberculosis bacillus in Ziehl-Neelsen-Stained tissue Images using moving K-Mean clustering procedure. In: 4th Asia International Conference on Mathematical/Analytical Modelling and Computer Simulation, pp. 215–220. IEEE Press, Washington, DC (2010)

2. Foreo, M.G., Gristóbal, G., Alvarez-Borrego, J.: Automatic identification techniques of tuberculosis bacteria. In: Proceedings of the Applications of Digital Image Processing XXVI, vol. 5203, pp. 71–81. San Diego, CA (2003)

3. Foreo, M.G., Sroukeb, F., Cristóbal, G.: Identification of tuberculosis bacteria based on shape and color. J. Real-Time Imaging, 10(4):251–262. London (2004)

4. C. Cortes and V. Vapnik. Support-vector network. Machine Learning, vol. 20, pp. 273–297 (1995)

5. Chih-Chung and Chilh-Jen Lin. LIBSVM: A library for Support Vector Machines. Available at http://www.csie.ntu.edu.tw/~cjlin/papers/libsvm.pdf

6. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. of Computer Vision, vol. 60, pp. 91–110, (2004)

7. Geusebroek, J.M., Boomgaard R.V.D., Smeulders, A.W.M., Geerts, H.: Color invariance. J. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, No. 12 (2001)

8. Swain, M.J., Ballard, D.H.: Color indexing. Int. J. of Computer Vision, vol. 7, No. 1, pp. 11–32 (1991)

9. Haralick, R.M., K. Shanmugan, and I. Dinstein: Textural features for image classification. IEEE Transactions on Systems, Man, and Cybernetics, vol. SMC-3, pp. 610–62 (1973)

10. Shan GONG, and A. Artés-Rodrigués, Science report. Available at www.tsc.uc3m.es/~concha/TecReportBacilos.pdf