# Application of Skin Color Model in Image Segmentation

Wei Wei, Tianyong Li, Jinfu Wei, Detian Zen, Weimin Ning

► To cite this version:

Wei Wei, Tianyong Li, Jinfu Wei, Detian Zen, Weimin Ning. Application of Skin Color Model in Image Segmentation. 10th International Conference on Intelligent Information Processing (IIP), Oct 2018, Nanning, China. pp.394-401, 10.1007/978-3-030-00828-4_40. hal-02197777

HAL Id: hal-02197777

https://inria.hal.science/hal-02197777

Submitted on 30 Jul 2019

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Application of skin color model in image segmentation

Wei Wei¹,²,³, Tian yong Li¹*, Jin fu Wei¹, De tian Zen⁴, Wei min Ning¹

¹Guangxi University, College of Computer and Electronics Information, Nanning, China

²Guangxi University, Guangxi Key Laboratory of Multimedia Communications and Network Technology, Nanning, China

³Guangxi University, Guangxi Colleges and Universities Key Laboratory of Multimedia Communications and Information Processing, Nanning, China

⁴Central South University, School of Information Science and Engineering, Changsha, China

weiweigxu@qq.com
tianyonglee@163.com
2053126817@qq.com
zengdetian@csu.edu.cn

**ABSTRACT.** Skin color detection refers to the process of selecting pixels corresponding to human skin in the image. With the popularity of video image acquisition equipment, the research of color region segmentation in images has become more and more active in recent years. In this paper, a skin color detection algorithm for color images with complex background is proposed. This paper mainly studies how to use skin color model to detect human body parts from complex background. The first step is to select YCbCr color space as the color space of this paper. The second step is to select the elliptical model based on color plane. This paper finally decides to use the fixed threshold elliptic model based on color plane to detect skin color.

**KEYWORDS:** Skin color model, Image segmentation, Human image detection, Color space

# 1 Introduction

Object detection and tracking based on vision is a basic problem in the field of computer vision. It involves many subjects, such as image processing, pattern recognition, artificial intelligence, machine learning and so on. With the development of computer technology, visual object detection technology has been widely used in the fields of handicraft detection, intelligent robot, human-computer interaction, and military reconnaissance [1,2]. Human-Computer interaction (HCI) is a technology used to study the interaction between human beings and computers, including the ways, methods, devices and interfaces of interaction. The purpose of this study is to improve the nature and efficiency of human-computer communication by using possible information channels.

Skin color detection refers to the process of selecting pixels corresponding to human skin in the image. With the popularity of video image acquisition equipment,
the research of color region segmentation in images has become more and more active
in recent years. Skin color is an effective feature in face detection and tracking,
although the difference may be significant for respective human skin color, but it is
mainly in brightness, rather than color. In color space, the skin tone is only a narrow
part of the frequency band, so it is possible to detect whether the pixel point has a skin
tone characteristic according to the color information. This paper mainly studies how
to use skin color model to detect human body from complex backgrounds.

## 2 Image Segmentation and skin Color Model

Human image segmentation is a process of separating human image from complex
background, especially part of human body. It is a basic foundation in human body
recognition and behavior understanding system. At present, there are mainly five
methods of human image segmentation.

### 2.1 Motion based segmentation method

This method uses foreground moving target detection method to segment. The main
methods are inter-frame difference method and background difference method [3].
The method of inter-frame difference uses the difference between adjacent images to
judge whether there are moving objects in the foreground. The background difference
method firstly models the background image and segments the foreground by
comparing the background image with the image containing human image. Many
studies have found that changes of the shadows produced by motion, as well as the
dynamic changes in the background, will have an impact on the segmentation results.

### 2.2 Segmentation method based on special color markers

The method requires that a special color mark different from the environmental color
is worn on the human body to be recognized [4]. The method is simple, the obtained
segmentation result is more accurate, and the influence of the environment is not easy.
However, due to the need to wear special logo, the user experience in the process of
human-computer interaction is reduced.

### 2.3 Contour based Segmentation method

There are also typical model-based segmentation methods [5], but there are two main
problems in this method: firstly, the initial contour is difficult to obtain due to the
rotation or bending of human limbs; The second is that the shape of the body
movement itself has four depth trapping areas, and the contour is often unable to
converge to such a region. Although this problem can be solved by the improved
model, it is difficult to be used in real-time systems because of the increased number
of iterations and the cost of computation.
## 2.4 Segmentation method based on Infrared camera

This method uses infrared camera to obtain infrared image to obtain accurate foreground region [6]. It can effectively avoid the influence of ambient light and skin-like-region in background in limb motion segmentation, and has a high robustness. However, due to the need to use infrared camera, it increases hardware cost.

## 2.5 Segmentation method based on skin Color Detection

Skin color detection is a process of selecting pixels corresponding to human skin in an image. This technique is widely used in face recognition, gesture recognition and image content filtering. Over the years, skin color detection methods, including color, texture, and multi-information fusion, have been proposed [7,8]. Among them, the color-based method is widely used in various systems because of its simple algorithm and high real-time performance, so it has strong robustness to the change of target shape and angle of view. However, the color-based skin detection method is easy to be influenced by the area looks like skin color due to the environment light, which causes the leakage detection and the false detection of the skin color region. How to minimize the influence of the environment in the skin color detection is important.

It is found that skin color usually has good cohesion and non-color separability in a specific color space. At the same time, brightness has a great influence on the color change of skin color, but the influence on tone and saturation of skin color is relatively small. Therefore, researchers usually transform skin color from normal color space to a special color space that is separated by brightness.

Based on the above problems, this paper describes the common color space, and recognizes that the YCbCr color space is more cohesive and better than other color space. The YCbCr color space is chosen as the color space for color detection. After comparing various skin color detection models, it is found that the elliptical model based on color plane has better detection effect [9]. On this basis, the actual segmentation effects of dynamic threshold method based on elliptic model and static threshold method are compared [10]. At the same time, the dynamic threshold method also increases the difficulty of parameter determination to some extent and reduces the efficiency of algorithm execution [11]. Based on the above analysis, a fixed threshold elliptic model based on color plane is used to detect the skin color.

# 3 Method

The process of model based skin color detection is usually divided into three steps. Firstly, the color space is determined; secondly, the skin color model is determined; finally, the corresponding skin color pixels in the image are selected under the color space and skin color model.

1) YCbCr color space is widely used in digital image, it is derived from the YUV model. The luminance information is represented by a separate component Y, and the color information is stored with two difference components, Cb and Cr. Component Cb is the difference between the blue component and the reference value, and the
component Cr is the difference between the red component and the reference value [12].

$$ \begin{bmatrix} Y \\ Cb \\ Cr \end{bmatrix} = \begin{bmatrix} 16 \\ 128 \\ 128 \end{bmatrix} + \begin{bmatrix} 65.481 & 128.553 & 24.966 \\ -37.797 & -74.203 & 112.000 \\ 112.000 & -93.786 & -18.214 \end{bmatrix} \begin{bmatrix} R \\ G \\ B \end{bmatrix} \quad (1) $$

YCbCr color space can be obtained directly by linear transformation of RGB color space, so its luminance component Y is not completely independent of chromaticity information. Therefore, the color clustering region is also a trend of non-linear change with the difference of Y. These characteristics of YCbCr space accord with the condition of the establishment of the skin color model in this paper.

2) Skin color elliptic model. If skin information is mapped to YCbCr space, these skin pixels are approximately an elliptical distribution in Cr-Cb two-dimensional space. For a coordinate (Cr, Cb) we only need to determine whether it is in the ellipse, including the boundary. If it is, we can judge it as a skin, otherwise it is a non-skin pixel. The results of the model are as follows:

$$ \frac{(x - ec_x)^2}{a^2} + \frac{(y - ec_y)^2}{b^2} = 1 \quad (2) $$

$$ \begin{bmatrix} x \\ y \end{bmatrix} = \begin{bmatrix} \cos \theta & \sin \theta \\ -\sin \theta & \cos \theta \end{bmatrix} \begin{bmatrix} C'_{b} - c_{x} \\ C'_{r} - c_{y} \end{bmatrix} \quad (3) $$

3) Skin color segmentation. Skin color is one of the most prominent features in color face images [13]. It is shown that, although human skin color varies widely, skin color has good clustering characteristics in chrominance space [14]. Since the image captured by the camera we use is described in YCbCr color space, it separates brightness from chroma, which can better limit the skin color area [15]. After the experiment, we use Cr red component directly to segment, which is more in line with the requirements of the difference between non-skin color and skin color. The formula for the division of skin color is as follows:

$$ Y(x, y) = \begin{cases} 255 & \mathrm{Cr} \in \mathrm{D} \\ 0 & \mathrm{Cr} \notin \mathrm{D} \end{cases} \quad (4) $$

Where Y(x,y) represents the gray value of the x row y column, Cr is a red component, D is the range of the red component of the skin. The resulting binary image distinguishes a face from a non-human face.
# 4 Experimentation

The experimental image set has a total of 200 pairs, and is represented by experiments in both face and gesture. After detecting the NTSC image, one of the images is taken as the luminance frame. The image results are shown in Figure 1 and Figure 2.

<figure><img src="image_2.png" /><figcaption>Fig. 2. NTSC brightness image(gesture)</figcaption></figure>

The luminance image (that is, the gray image) is obtained, and we begin to binarize the gray image and convert it into black and white image. In order to segment the human body better, the threshold value must be selected before binarization. In this experiment, the threshold is selected according to the experimental experience, the selection range is 0.03 to 0.16 for 1, but not for this range is 0. The resulting binarization effect is shown in Figure 3.
<figure><img src="image_3.png" /><figcaption>Fig. 3. Binary image</figcaption></figure>

In order to reduce the influence of noise on the image, image differential processing and filtering. The result is as shown in 4:

<figure><img src="image_4.png" /><figcaption>Fig. 4. Filter effect diagram</figcaption></figure>

As can be seen from the above figures, the skin color model presented in this paper has a good effect in image segmentation. This method is compared with the classical segmentation method proposed by Liu et al [16]. The results are compared in Table 1. It can be seen that the detection accuracy of this model has been greatly improved and the missing detection rate of human body image has been reduced.

<table><caption>Table 1. Contrast result</caption><thead><tr><th>Detection method</th><th>Correct rate</th><th>Missed detection rate</th><th>False detection rate</th></tr></thead><tbody><tr><td>Literature 16</td><td>90.3</td><td>12.4</td><td>3.0</td></tr><tr><td>This article</td><td>96.6</td><td>7.3</td><td>3.1</td></tr></tbody></table>
## 5 Analysis and summary

In order to separate areas of human body from non-human regions, a reliable skin color model suitable for different skin colors and different illumination conditions is needed. Because of the change of ambient illumination brightness may make the detection of human body more complicated and unreliable in the process of skin segmentation. In order to make use of the clustering of skin color in chrominance space, we need to separate the chrominance information from the luminance information in the color expression. In the YCbCr color space, the Y component gives all the luminance information, while Cr and Cb components are not affected by the luminance, so the luminance component is effectively separated out. Compared with other color space formats such as HSV, the calculation process and space coordinate representation of YCbCr color space format is relatively simple. The color components Y, Cr, and Cb can be obtained by linear transformation of the three primary colors R, G, and B, so it has a high computational efficiency, and the color clustering characteristics in the YCbCr color space are better. From the experimental results we can see that there is still a lot of interference in the black and white image (non-human region) so a series of filtering is needed to detect the binary image after the complete filtering.

## 6 Acknowledgement

This research was financially supported by Student's Platform for Innovation and Entrepreneurship Training Program.

## 7 References

1. Rein-Lien Hsu, Abdel-Mottaleb M., Jain A.K: "Face Detection in color images." *IEEE Trans. on Pattern Analysis and Machine Intelligence*, 2002, 24(5): 696-706

2. Faux, Francis, and F. Luthon. "Theory for evidence to detect." *Traitement Du Signal* 28.5(2011):515-545.

3. Chen Juan, Ying Jun, Wang Jian, Zeng Shuang: "A Moving Target Segmentation Method Based on Interframe Difference and Image Segmentation." *Journal of Shanghai Normal University (Natural Science)*, 2017, 46(02): 242-246

4. Zong Gaigai: "Research on image colorization algorithm based on local color diffusion." *Shanghai University of Applied Sciences*, 2016

5. Liu Xiao: "Research on geometrical active contour model for local segmentation of gray uneven images." Shandong University, 2017

6. Shang Lei: "Research and implementation of key technologies of infrared imaging system." Xidian University, 2013

7. Wei Wei, et al: "Pupil Center Location Based on Skin Color Segmentation and Radial Symmetry Transform." *International Conference on Computer, Mechatronics, Control and Electronic Engineering 2015*

8. Zuo, Haiqiang, et al. "Combining Convolutional and Recurrent Neural Networks for Human Skin Detection." *IEEE Signal Processing Letters* 24.3(2017):289-293.
9. Bianco, Simone, F. Gasparini, and R. Schettini. "Computational strategies for skin detection." *International Conference on Computational Color Imaging* Springer-Verlag, 2013:199-211.

10. Sanchez-Cuevas, et al. "A Comparison of Color Models for Color Face Segmentation." 2013:134-141.

11. Kawulok, Michal, et al. "Self-adaptive algorithm for segmenting skin regions." *Eurasip Journal on Advances in Signal Processing* 2014.1(2014):170.

12. Rafael C. Gonzalez, Richard E. Wood, Steven L. Eddins: "Implementation of Digital Image Processing." Beijing: Tsinghua University Press, 2013, 213-214

13. Huang Yaqin: "Research and implementation of eye-controlled mouse based on gaze tracking technology." Chengdu, Sichuan Province: Xihua University, 2011

14. Zhang Min, Tao Liang: "Detection and orientation of human eyes in face images." *Optoelectronic Engineering*, 2006, 33(8): 33-36

15. SanMiguel, Juan C., and José M. Martínez. "Shadow detection in video surveillance by maximizing agreement between independent detectors." *Image Processing (ICIP), 2009 16th IEEE International Conference on*. IEEE, 2009.

16. Liu, Xin-sheng, Ming-qiang An, and Yang Gao. "An upper bound for the adjacent vertex distinguishing acyclic edge chromatic number of a graph." *Acta Mathematicae Applicatae Sinica, English Series* 25.1 (2009): 137-140.