<figure><img src="image_1.png" /></figure>

HAL
open science

# Color Image Segmentation in RGB Color Space Based on Color Saliency

Chen Zhang, Wenzhu Yang, Zhaohai Liu, Daoliang Li, Yingyi Chen, Zhenbo Li

► To cite this version:

Chen Zhang, Wenzhu Yang, Zhaohai Liu, Daoliang Li, Yingyi Chen, et al.. Color Image Segmentation in RGB Color Space Based on Color Saliency. 7th International Conference on Computer and Computing Technologies in Agriculture (CCTA), Sep 2013, Beijing, China. pp.348-357, 10.1007/978-3-642-54344-9_41. hal-01220937

HAL Id: hal-01220937

https://inria.hal.science/hal-01220937

Submitted on 27 Oct 2015

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Color image segmentation in RGB color space
based on color saliency

Chen Zhang¹, Wenzhu Yang¹,*, Zhaohai Liu¹, Daoliang Li², Yingyi Chen², Zhenbo Li²

¹College of Mathematics and Computer Science, Hebei University, Baoding 071002, PR China

² College of Information and Electrical Engineering, China Agriculture University, Beijing
100083, PR China

*Corresponding author Email: wenzhuyang@163.com

**Abstract.** It is difficult to separate the foreign fiber objects in a captured color image from the background accurately. To solve this problem, this paper presents a new approach for color image segmentation in RGB color space based on color saliency. Firstly, the captured RGB color image was separated to R, G and B color channels, and the color information for each channel was calculated. Secondly, the R saliency for each pixel in the R channel, the G saliency for each pixel in the G channel, and the B saliency for each pixel in the B channel was calculated respectively. Then comprehensive saliency map was obtained by the weighted R, G and B saliency. The weights for the R, G and B saliency were determined by the corresponding color information of each color channel. At last, the foreign fiber targets were separated out from the comprehensive saliency map using a threshold method. The results indicate that the proposed method can segment out the foreign fiber objects from the color image accurately.

**Keywords.** Color image segmentation; RGB color space; RGB color image; Color saliency; Color information; Foreign fiber targets; Comprehensive saliency map; Comprehensive saliency; Threshold method; Foreign fiber objects

# 1. Introduction

Foreign fibers in cotton, such as hairs, plastic, polypropylene twines, affect the quality of the final cotton textile products seriously[1-4]. So they should be removed before spinning. Automated Visual Inspection is the state-of-the-art technology for picking out the foreign fibers from cotton lint. Image segmentation is one of the most important stage for foreign fiber detection. But it is very difficult to separate the foreign fiber objects in a captured color image from the background accurately. Nowadays, many methods for gray image segmentation are studied and widely used in many applications. Even the color image, will be transformed into gray image firstly[5-6], and then segmented using these methods developed for gray image segmentation. However, these methods fail to make full use of the color information to perform accurate segmentation[7-8]. So color image segmentation is one of the hot spot in image processing.

Image segmentation is one of the key techniques in image processing and machine
vision system, and it is the first step in image analysis and pattern recognition. The aim of image segmentation is to partition an image into meaningful connected components. Nowadays, various image segmentation methods are available and some of them are used for foreign fiber recognition.

It is very difficult to separate the foreign fiber targets from the background using traditional image segmentation methods duo to the inhomogeneous background brightness and variety of foreign fibers in different colors and shapes. So it is high time for us to solve this problem to improve the image segmentation accuracy. Visual attention based image processing method is one of the state-of-the-art methods and has been successfully used for object detection.

Scientists took considerable time and efforts to study the human visual attention system[9] in order to segment the targets automatically. Saliency map was firstly introduced by Koch and Ullman. For human vision, color information is important in a image, and the sensitive degree is different from one image to another. Itti proposed a color image segmentation method based on visual attention and saliency map[10], but the segmentation efficiency is not high. Hou introduced a new construction method of saliency map based on spectral residual[11]. Due to its simplicity and high efficiency, it has been widely used in many applications[12-14].

This paper presents a color image segmentation method based on RGB comprehensive saliency. Color information in a color image is vital to the human vision, so we consider red, green and blue value as the color information. We can obtain R, G, B information saliency map separately from the correspond channel. According to the different color information, we give different weights to different saliency map, and then, the comprehensive saliency map is created. We used a threshold method to realize the separation of target and background. The aim of this paper is to develop a method which can be employed in the AVI system to detect the foreign fibers precisely and quickly. The paper is organized as follows. We present the materials and methods in Section 2. We describe the experiments carried out to validate the performance of the proposed method for the task of foreign fiber detection in Section 3. Finally, we conclude this work in Section 4.

## 2. Materials and methods

We chose feather, hair, plastic and polypropylene twine as typical foreign fibers in this research. An AVI system in our lab, as shown in Fig. 1, is used to capture the live color images the foreign fibers.
<figure><img src="image_3.png" /><figcaption>Fig. 1 Outline of the AVI system</figcaption></figure>

The AVI system mainly consists of three sub-systems: a cotton lint pre-opening system, a visual inspection system and a cotton lint collection system. The compact cotton lint with foreign fibers was firstly pre-opened in the pre-opening system to make it loose. Then the loosed cotton lint was transferred to the visual inspection system. In the vision system, color images of foreign fibers were captured by line-scan camera, and analyzed in the computer. After the inspection finished, the cotton lint was collected and stored in the collection system.

## 2.1 Image acquisition

Live color images of foreign fibers were captured by a line-scan camera in the vision system. The camera we used is a Piranha line scan color camera with 4096 resolution. It was set to internal exposure mode working in 10 KHz frequency. Live scanned lines were collected by an image acquisition board (X64-CL Express, CORECO, Canada) and stored in an image buffer. The frame size was set to 128 lines, so the image resolution in this application was 128×4096 pixels. Some sample images with different type of foreign fibers were shown as Fig. 2.
<figure><img src="image_4.png" /><figcaption>(c) Red polypropylene twine
**Fig. 2** Foreign fiber images</figcaption></figure>

In the first image, a piece of red paper was in it. In the second image, we can seize black feather as the foreign fiber. In the third image, a piece of red polypropylene twine was found.

## 2.2 The frame of the method

Foreign fiber images we obtained were color images, and different images have different color information. We can divide a RGB color image into R channel, G channel and B channel. Because of R information, G information and B information varying from one image to another, the visual feeling about different channel is different. If the amount of R information is greater than the other two channels' information, the human is more sensitive to the R information. If the amount of G information is the largest, the human is more sensitive to the G information. It is the same for B information. According to the amount of R information, G information and B information, we can determine the weight of R saliency, G saliency and B saliency. Thus, we can form a comprehensive saliency map, and we use a threshold method in order to do the segmentation between the target and the background. The frame of the proposed method was shown in Fig. 3.

<figure><img src="image_5.png" /><figcaption>**Fig. 3** The frame of the proposed method</figcaption></figure>

## 2.3 Channel information calculation

The input color image *I* is a RGB color image whose resolution is *M*\**N*\*3 pixels. *M* and *N* are the number of rows and columns. Let *R*(*i*, *j*), *G*(*i*, *j*) and *B*(*i*, *j*) represent each channel information respectively, where *i* represents the abscissa of a pixel, *j* represents the ordinate of a pixel. Let *S*<sub>*r*</sub>, *S*<sub>*g*</sub>, *S*<sub>*b*</sub> be the sum of the *R*, *G*, *B* information respectively, and let *m*<sub>*r*</sub>, *m*<sub>*g*</sub>, *m*<sub>*b*</sub> be the mean of the *R*, *G*, *B* information respectively.
$$S_r = \sum_{i=1}^{M} \sum_{j=1}^{N} R(i, j) \qquad (1)$$

$$S_g = \sum_{i=1}^{M} \sum_{j=1}^{N} G(i, j) \qquad (2)$$

$$S_b = \sum_{i=1}^{M} \sum_{j=1}^{N} B(i, j) \qquad (3)$$

$$M_r = S_r / (M * N) \qquad (4)$$

$$M_g = S_g / (M * N) \qquad (5)$$

$$M_b = S_b / (M * N) \qquad (6)$$

## 2.4 Color saliency calculation

The saliency of one channel information can be obtained by computing the absolute difference of the color feature with the mean color feature value. Let $L_r$, $L_g$, $L_b$ present the corresponding color information saliency, and the formulae are as below.

$$L_r(i, j) = abs(R(i, j) - M_r) \qquad (7)$$

$$L_g(i, j) = abs(G(i, j) - M_g) \qquad (8)$$

$$L_b(i, j) = abs(B(i, j) - M_b) \qquad (9)$$

## 2.5 Weights determination

The weight for each color saliency depends on the amount of R information, G information, and B information. A larger amount of R information indicates that the R characteristics is more important for human visual perception, and a larger amount of G information presents that the G characteristics is more important for human visual perception. It is the same for B information. Let $W_1$, $W_2$, $W_3$ represent the weight of each channel saliency. We can calculate them by the following formula.

$$W_1 = M_r / (M_r + M_g + M_b) \qquad (10)$$
$$W_2 = M_g / (M_r + M_g + M_b) \qquad (11)$$

$$W_3 = M_b / (M_r + M_g + M_b) \qquad (12)$$

## 2.6 The comprehensive saliency map calculation

After working out the weights, we can obtain the comprehensive saliency map by using the formula below.

$$S(i, j) = W_1 * L_r(i, j) + W_2 * L_g(i, j) + W_3 * L_b(i, j) \qquad (13)$$

## 2.7 Foreign fiber objects detection and fusion

We can use a threshold method to distinguish the foreign fiber objects from the background. Let $T$ be the threshold. When $S(i, j)$ is greater than $T$, we regard $pix(i, j)$ as a pixel in the foreign fiber object, while $S(i, j)$ is smaller than $T$, we think $pix(i, j)$ as a pixel in the background.

$$O(i, j) = \begin{cases} 0, & S(i, j) < T \\ 1, & S(i, j) \ge T \end{cases} \qquad (14)$$

## 2.8 Algorithm implementation

Step1: calculate the color feature $R$, $G$ and $B$ respectively.

Step2: calculate the channel information $M_r$, $M_g$ and $M_b$ respectively.

Step3: calculate the weight for each channel saliency based on $M_r$, $M_g$ and $M_b$.

Step4: calculate the channel saliency map $L_r$, $L_g$ and $L_b$ respectively.

Step5: Obtain the comprehensive saliency map $S$ according to the weights and the channel saliency maps.

Step6: Segment the targets by using a threshold method to get the final object $O$.
<figure><img src="image_6.png" /><figcaption>Fig. 4 The process of weights calculation</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Fig. 5 The process of the Image segmentation method</figcaption></figure>

**3. Results and discussion**

The images we used in the experiment are live images captured by the AVI system. The algorithm was implemented in Matlab 7.0, and the operation system is Window XP.

A large amount of color images of foreign fibers were used to test the proposed method. The processing results of polypropylene twine were shown below. In this experiment, the human visual attention is associated with the whole image's color information. The red color is dominant in the whole image, so we think red is the most important color for human visual feeling in this image of polypropylene twine. Thus, we raise the weight of red color saliency map in our experiment.

<figure><img src="image_8.png" /></figure>

The original RGB color image

<figure><img src="image_9.png" /></figure>

The R channel

<figure><img src="image_10.png" /></figure>

The G channel
<figure><img src="image_11.png" /><figcaption>**Fig. 5** The results of the segmentation of polypropylene twine</figcaption></figure>

The processing results of black hair obtained by the proposed method were shown in Fig. 6. In this test, the human visual attention is associated with the whole image's color. $M_r$ and $M_g$ larger than $M_b$, so red and green are regarded as the both important color for human visual feeling. Thus, we raise the weight of red and green color saliency map.

<figure><img src="image_12.png" /><figcaption>**Fig. 6** The results of the segmentation of black feather</figcaption></figure>

Compared with the spectral residual method, our method can get better flexibility. We use the color information in a color image to determine the saliency map dynamically. We use a threshold method to separate out the target. In our experiment, $T$ was set to 0.115. The spectral residual method can not segment all the selected images in our experiment, while our method can get accurate segmentation, as shown in (A3), (B3), (C3).

In our experiments, we use empirical value as the threshold, which can make the method simple and fast. The Otsu's algorithm has the problem in segmentation in our image, because of the low contrast ratio between the foreign fiber objects and the background. So we use the empirical value as the threshold in order to promote the segmentation accuracy.
<table><caption>**Table 1** The comparison of the segmentation method</caption><thead><tr><th></th><th></th></tr></thead><tbody><tr><td>Original<br>RGB<br>image</td><td>(A1) black feather<br>(B1) plastic film<br>(C1) polypropylene twine</td></tr><tr><td>Spectral<br>residual</td><td>(A2) the result of black feather<br>(B2) the result of plastic film<br>(C2) the result of polypropylene twine</td></tr><tr><td>our method</td><td>(A3) the result of black feather<br>(B3) the result of plastic film<br>(C3) the result of polypropylene twine</td></tr></tbody></table>
## 4. Conclusion

A new approach for color image segmentation in RGB color space based on color saliency was proposed. The amount of R, G and B color information was firstly calculated, and the weight of each channel saliency map was determined according to the corresponding color information. And then, the comprehensive saliency map was obtained. Finally, the foreign fiber targets were segmented out from the comprehensive saliency map. The results indicate that the proposed approach is consistent with the human visual perception characteristics and can get accurate foreign fiber objects from the captured color images.

The threshold for comprehensive saliency map segmentation is empirical value. How to segment the foreign fiber images automatically will be researched in our future work.

## Acknowledgements

The authors thank The Ministry of Science and Technology of the People's Republic of China (2013DFA11320, 2013BAK07B04), National Natural Science Foundation of China (31228016), Hebei University (2010-207), Key Laboratory of Modern Precision Agriculture System Integration Research Centre of the Education Ministry in China Agriculture university (X11-01), Hebei Education Department (Q2012063), and Hebei Science and Technology Department (12210133), for their financial support, and China Agriculture University for providing the testing environment.

## References

<footnote n="1">¹W.Z Yang, D.L Li, X.H Wei, Y.G Kang, F.T Li: 'An automated visual inspection system for foreign fiber detection in lint', IEEE GCIS, 2009, 364-368.</footnote>

<footnote n="2">²Wen zhu Yang, Sukui Lu, Sile Wang, Daoliang Li: 'Fast recognition of foreign fibers in cotton lint using machine vision', Mathematical and Computer Modelling, 2011, 54, 877-882.</footnote>

<footnote n="3">³Yang Wen-zhu, Li Dao-liang, Zhu Liang, et al: 'A New Approach for Image Processing in Foreign Fiber Detection', Computers and Electronics in Agriculture, 2009, 68, (1), 68-77.</footnote>

<footnote n="4">⁴Wenzhu Yang, Daoliang Li, Sile Wang, Suiku Lu, Jingwei Yang: 'Saliency map basee color image segmentation for foreign fiber detection', Mathematics and Computer modelling, 2013, 58, 846-852.</footnote>

<footnote n="5">⁵Ye qi-xiang, Gao wen, Wang wei-qiang, et al: 'A Color Image Segmentation Algorithm by Using Color and Spatial Information', Journal of software, 2004, 15, (4), 522-530.</footnote>

<footnote n="6">⁶Bao Quan-lei: 'Color Image Segmentation Based on HSV Color Space', Software Guide, 2010, 9, (7), 171-172.</footnote>

<footnote n="7">⁷Zhao jing-xiu, Wang jing, Zhao zhao: 'Color Image Edge Detection Based on Subdivision of RGB Space', Optoelectronic technology, 2009, 29, (3), 171-173.</footnote>

<footnote n="8">⁸Han xiao-wei, Yang zhe, Li yang-ping, et al: 'A Method for Color Image Segmentation Based on Color Similarity Coefficient', Journal of shenyang university, 2004, 26, (6), 14-17.</footnote>
<footnote n="9">⁹ Zhang Hua-wei, Zheng Ya-feng, Zhang qiao-rong, 'Color Image Segmentation Based on Visual Attention Mechanism', Computer Engineer and Application, 2011, 47, (10), 154-157.</footnote>

<footnote n="10">¹⁰ Laurent Itti, Christof Koch, Ernst Niebur, 'A model of saliency-based visual attention for rapid scene analysis', IEEE transactions on pattern analysis and machine intelligence, 1998, 20, (11), 1254-1259.</footnote>

<footnote n="11">¹¹ Hou Xiao-di, Zhang Li-qing. Saliency Detection: A Spectral Residual Approach. Proceedings of The IEEE Conference on Computer Vision and Pattern Recognition, 2007, 1-8.</footnote>

<footnote n="12">¹² Goferman, Stas;Zelnik-Manor, Lihi;Tal, Ayellet: 'Context-Aware Saliency Detection', Pattern Analysis and Machine Intelligence, IEEE Transactions, 2012, 34, (10): 1915-1926.</footnote>

<footnote n="13">¹³ H. Shi, Y. Yang: 'A Computational Model of Visual Attention Based on Saliency Maps', Applied Mathematics and Computation, 2007, 188:1671-1677.</footnote>

<footnote n="14">¹⁴ Hang Shi, Yu Yang: 'A computational model of visual attention based on saliency maps', Applied Mathematics and Computation, 2007, 188, (2),1671–1677.</footnote>