# Candidate Smoke Region Segmentation of Fire Video Based on Rough Set Theory

Authors: Yaqin Zhao
Journal: Journal of Electrical and Computer Engineering
 (2015)

---

## Abstract

Candidate smoke region segmentation is the key link of smoke video detection; an effective and prompt method of candidate smoke region segmentation plays a significant role in a smoke recognition system. However, the interference of heavy fog and smoke-color moving objects greatly degrades the recognition accuracy. In this paper, a novel method of candidate smoke region segmentation based on rough set theory is presented. First, Kalman filtering is used to update video background in order to exclude the interference of static smoke-color objects, such as blue sky. Second, in RGB color space smoke regions are segmented by defining the upper approximation, lower approximation, and roughness of smoke-color distribution. Finally, in HSV color space small smoke regions are merged by the definition of equivalence relation so as to distinguish smoke images from heavy fog images in terms ofV component value variety from center to edge of smoke region. The experimental results on smoke region segmentation demonstrated the effectiveness and usefulness of the proposed scheme.


---

## Body

## 1. Introduction

During one early fire, smoke is the main visual phenomenon of video surveillance image. Therefore, the research of smoke video recognition is particularly important to detect early fire. Most of smoke detection algorithms proceed as in the following steps: candidate smoke region segmentation, feature extraction, and smoke recognition. In this paper, we study the first step (i.e., candidate smoke region segmentation). At present, candidate smoke region segmentation methods generally can be divided into two categories: one is based on color features, such as RGB color space [1], HSV color space [2], and the combination of RGB color space and HSV color space [3–5]. Another is based on color features and moving features. Kim et al. [6] got the counter of smoke region with two-dimensional wavelet transform. Qin and Ma [7] extracted smoke regions of interest by static saliency image and moving saliency image. Wu et al. [8] established a background model using mixed Gaussian model in RGB space and then suspected that smoke regions are detected by comparing the current frame with this reference model. However, the first kind of method cannot detect smoke of small area and low concentration because moving characteristics are not considered. And the methods cannot distinguish smoke from heavy fog and blue sky.Rough set can realize the representation and inference of the uncertain knowledge [9]. The theories of approximation representation and equivalence relation were widely used for image processing and pattern recognition [10–12]. This paper presents a novel candidate smoke region segmentation method based on rough set theory. Figure 1 shows the basic process of the proposed scheme. Above all, video backgrounds are automatically updated based on three-frame difference approach to avoid the interference of static smoke-color objects such as blue sky. Afterwards, the upper approximation, lower approximation, and roughness of smoke-color distribution are represented using rough set theory in order to exclude moving regions of non-smoke-color objects, such as leaves and flowers. Finally, smoke regions are initially segmented by gray wave brand of R, G, B basic color features. However, superfine partition can lead to small regions since excessive wave peaks are generated by initial segmentation. Thus these similar regions are merged based on equivalence relation of rough set theory.Figure 1
The basic process of the proposed scheme.


## 2. Moving Pixels Detection

Kalman filtering is used to update video background [13]. Let B(i,j,k) and B(i,j,k+1) denote the value of the pixel (i,j) of kth background image and (k+1)th background image, respectively, and let I(i,j,k) denote the value of the pixel (i,j) of kth frame; then the recursive formula is as follows:(1)B(i,j,k+1)=B(i,j,k)k=0B(i,j,k)+g(i,j,k)·[I(i,j,k)-B(i,j,k)]k≠0,where g(i,j,k) denotes gain factor and is computed using the following formula:(2)g(i,j,k)=β1-Mi,j,k+αMi,j,kM(i,j,k)=1ifIi,j,k-Bi,j,k>Th10otherwise,where the parameters α and β are the moving factor and the background factor, respectively. The pixel (i,j) of the kth frame is marked as moving pixel if Ii,j,k-Bi,j,k>Th1, and Th1 is the threshold. The method based on Kalman filtering is a common method for moving pixel detection. The selection method of the parameters α, β, and Th1 is described in [13]. In this paper, the moving factor α is set to 0.001<α<0.01, and the background factor β is set to 0.01<β<0.1 [13].


## 3. Candidate Smoke Regions Segmentation

There are a mass of non-smoke-color moving objects such as trees and flowers in a surveillance video. In this section roughness measure approach is used to distinguish smoke-color moving regions from non-smoke-color moving regions.


### 3.1. Suspected Smoke Pixels Detection

Because fire smoke color meetsR≈G≈B in RGB color space and 80≤V≤220 in HSV color space, the intensity value V=(R+G+B)/3 meets 80≤V≈B≤220. Therefore, suspected smoke pixels are detected by the following two rules.Rule  1: 80≤B≤220.Rule  2: Cmax⁡-Cmin⁡≤Th2,where the maximum value Cmax⁡=max⁡(R,G,B) and the minimum value Cmin⁡=min⁡(R,G,B). The threshold Th2 is set to 5≤Th2≤20 since fire smoke color meets R≈G≈B. Thus the pixels that meet Rules  1 and 2 will be detected as suspected smoke pixels.


### 3.2. Approximation Representation of Color Distribution

In this section, we deal with pixels generated in suspected smoke pixels detection. That is, only suspected smoke pixels are checked in terms of rough set theory. Let the imageF denote an RGB image of the size M×N and g denote gray level; then approximation representations of the image F can be defined based on rough set theory. The lower approximation of color distribution is defined as [11, 14](3)H_n(g)=∑i=1M∑j=1NδIi,j,n-g,where 80≤g≤220 and n∈{R,G,B}. I(i,j,n) is the gray value of the pixel (i,j) on R, G, B component and δ(·) is impulse function.The pixels that meet smoke color feature will be used to computeH_n(g). According to the definitions, the lower approximation H_n(g) represents the number of smoke-color pixels whose gray level is g; thus H_n(g) is the certainty measure of gray distribution on R, G, B component.LetP×Q be the neighborhood of the pixel (i,j) and the pixel (p,q)∈P×Q; then the similarity of the pixels (i,j) and (p,q) is defined as [10, 14](4)dP×Q(i,j)=∑p∈P∑q∈QdIi,j,Ip,q,where d(I(i,j),I(p,q)) is the Euclidean distance of I(i,j,n) and I(p,q,n), n∈{R,G,B}. The homogeneity degree function H(i,j) is defined as follows:(5)H(i,j)=1dP×Q(i,j)<Th30otherwise.The upper approximation of color distribution is defined as(6)H¯n(g)=∑i=1M∑j=1N1+Hi,jδIi,j,n-g,where 80≤g≤220 and n∈{R,G,B}, and H¯n(g) represents the uncertainty measure of gray distribution of smoke-color pixels whose gray level is g on R, G, B component. Obviously, the approximations H_n(g) and H¯n(g) satisfy the properties of rough set theory. Therefore, the roughness of smoke-color distribution is defined as [11, 12](7)ρng=1-H_ngH¯ng,where 80≤g≤220 and n∈{R,G,B}. If the roughness ρn(g) is larger, the pixels of the gray level g locate in the regions of less pixel variety, that is, central region. Otherwise, the pixels of the gray level g locate in the edge of region. Figure 2 shows the roughness representation of one smoke video frame.Figure 2
Roughness representation of color distribution.
(a)
Original image
(b)
Roughness representation


### 3.3. Candidate Smoke Region Segmentation of Fire Video

The smoke region can be initially segmented by roughness of smoke-color distribution defined in Section2. Then similarity of two small regions is defined on the basis of equivalence relation theory in order to merge similar regions.


#### 3.3.1. Initial Segmentation of Candidate Smoke Region

Smoke regions can be segmented by gray wave brands that are formed by wave peaks and valleys. Therefore, the selection of peak and valley is important to increase the performance of smoke region extraction. LetPl be one peak of gray level g(80≤g≤120); then the peak Pl will be reserved if it meets the condition Pl>μm-σm, where μm and σm are mean value and variance, respectively. Thus one new peak sequence {Pl1,Pl2,…,Plk} is generated, where li denotes ith gray level. Then peak values of gray wave brand are selected as follows [12, 14]:(8)Plg=max⁡(Pli,Pli+1)ifli-li+1≥PWPliotherwise,where PW=10 is the threshold of peak width. The peak location can be gained on the basis of Plg. Wave valley location can be easily gained by computing the maximum between two adjacent peak values. Wave peaks and wave valleys come into being gray wave brands that are used to segment smoke region. The results of initial segmentation are shown in Figure 3. As seen from Figure 3, one smoke region is divided into some small clusters of smoke pixels. Smoke pixels of each cluster have similar roughness of smoke-color distribution.Figure 3
The results of initial segmentation ((a) and (b) show original image and the results of initial segmentation, resp.).
(a)
Original image
(b)
Results of initial segmentation


#### 3.3.2. Merging Small Smoke Region

Although smoke region can be segmented using the proposed method, the method often results in superfine partition. Furthermore, the method cannot distinguish smoke images from heavy fog images. Fortunately, smoke has one characteristic that is different from heavy fog. Smoke concentration gradually decreases from centre to edge of smoke region, which results in the fact thatV component value of smoke region gradually reduces in HSV color space. Therefore, in this section small smoke regions are merged in HSV color space. Initial equivalence relation Ri of one small smoke region Ci is defined as(9)Ri=Ci,U-Ci,where U is discourse domain and [Ci]={Ci∣S(Ci,Cj)≥β1}, i,j=1,2,…,K. β1 is the threshold of smoke region similarity. The threshold β1 is set to 0.7≤β1≤0.8. S(Ci,Cj) is computed as follows:(10)SCi,Cj=∑i,j∈Ci∑p,q∈CjdVi,j,Vp,q.Let Ri and Rj be two initial equivalence relations, and then generalized equivalence relation is defined as(11)Ri′=Pi,U-Pi,where Pi=∪1≤j≤K{Cj∣S(Ri,Rj)≥β2},  i,j=1,2,…,K. β2 is the threshold of equivalence relation similarity. The selection of the threshold β2 is discussed in Section 4.2.1. S(Ri,Rj) is computed as follows:(12)SRi,Rj=Ci∩CjCi∪Cj,where Ci∩Cj denotes the number of the objects in the intersection [Ci]∩[Cj], and Ci∪Cj denotes the number of the objects in the union set [Ci]∪[Cj]. Namely, the similarity of two smoke regions generated in Section 3.3.1 can be computed by the formula (12). If the similarity (Ri,Rj)≥β2, two small smoke regions Ci and Cj are merged into one larger smoke region. Figure 4 shows the results after merging small smoke region. As seen from Figure 4, some small clusters of smoke pixels are merged. Five small regions of the image at the top right of Figure 3(b) are merged into three clusters. Three small regions of the image at the bottom right of Figure 3(b) are merged into two clusters. We can exclude many smoke-color objects according to the characteristic that V component value of smoke pixels in HSV color space reduces from center to edge of smoke region. If one candidate smoke region is divided into 2~3 clusters and the V component value of the smoke region decreases from center to edge, it is decided as smoke region. Otherwise, it is detected as non-smoke regions.Figure 4
The results after merging small smoke region.
(a)
(b)


## 3.1. Suspected Smoke Pixels Detection

Because fire smoke color meetsR≈G≈B in RGB color space and 80≤V≤220 in HSV color space, the intensity value V=(R+G+B)/3 meets 80≤V≈B≤220. Therefore, suspected smoke pixels are detected by the following two rules.Rule  1: 80≤B≤220.Rule  2: Cmax⁡-Cmin⁡≤Th2,where the maximum value Cmax⁡=max⁡(R,G,B) and the minimum value Cmin⁡=min⁡(R,G,B). The threshold Th2 is set to 5≤Th2≤20 since fire smoke color meets R≈G≈B. Thus the pixels that meet Rules  1 and 2 will be detected as suspected smoke pixels.


## 3.2. Approximation Representation of Color Distribution

In this section, we deal with pixels generated in suspected smoke pixels detection. That is, only suspected smoke pixels are checked in terms of rough set theory. Let the imageF denote an RGB image of the size M×N and g denote gray level; then approximation representations of the image F can be defined based on rough set theory. The lower approximation of color distribution is defined as [11, 14](3)H_n(g)=∑i=1M∑j=1NδIi,j,n-g,where 80≤g≤220 and n∈{R,G,B}. I(i,j,n) is the gray value of the pixel (i,j) on R, G, B component and δ(·) is impulse function.The pixels that meet smoke color feature will be used to computeH_n(g). According to the definitions, the lower approximation H_n(g) represents the number of smoke-color pixels whose gray level is g; thus H_n(g) is the certainty measure of gray distribution on R, G, B component.LetP×Q be the neighborhood of the pixel (i,j) and the pixel (p,q)∈P×Q; then the similarity of the pixels (i,j) and (p,q) is defined as [10, 14](4)dP×Q(i,j)=∑p∈P∑q∈QdIi,j,Ip,q,where d(I(i,j),I(p,q)) is the Euclidean distance of I(i,j,n) and I(p,q,n), n∈{R,G,B}. The homogeneity degree function H(i,j) is defined as follows:(5)H(i,j)=1dP×Q(i,j)<Th30otherwise.The upper approximation of color distribution is defined as(6)H¯n(g)=∑i=1M∑j=1N1+Hi,jδIi,j,n-g,where 80≤g≤220 and n∈{R,G,B}, and H¯n(g) represents the uncertainty measure of gray distribution of smoke-color pixels whose gray level is g on R, G, B component. Obviously, the approximations H_n(g) and H¯n(g) satisfy the properties of rough set theory. Therefore, the roughness of smoke-color distribution is defined as [11, 12](7)ρng=1-H_ngH¯ng,where 80≤g≤220 and n∈{R,G,B}. If the roughness ρn(g) is larger, the pixels of the gray level g locate in the regions of less pixel variety, that is, central region. Otherwise, the pixels of the gray level g locate in the edge of region. Figure 2 shows the roughness representation of one smoke video frame.Figure 2
Roughness representation of color distribution.
(a)
Original image
(b)
Roughness representation


## 3.3. Candidate Smoke Region Segmentation of Fire Video

The smoke region can be initially segmented by roughness of smoke-color distribution defined in Section2. Then similarity of two small regions is defined on the basis of equivalence relation theory in order to merge similar regions.


### 3.3.1. Initial Segmentation of Candidate Smoke Region

Smoke regions can be segmented by gray wave brands that are formed by wave peaks and valleys. Therefore, the selection of peak and valley is important to increase the performance of smoke region extraction. LetPl be one peak of gray level g(80≤g≤120); then the peak Pl will be reserved if it meets the condition Pl>μm-σm, where μm and σm are mean value and variance, respectively. Thus one new peak sequence {Pl1,Pl2,…,Plk} is generated, where li denotes ith gray level. Then peak values of gray wave brand are selected as follows [12, 14]:(8)Plg=max⁡(Pli,Pli+1)ifli-li+1≥PWPliotherwise,where PW=10 is the threshold of peak width. The peak location can be gained on the basis of Plg. Wave valley location can be easily gained by computing the maximum between two adjacent peak values. Wave peaks and wave valleys come into being gray wave brands that are used to segment smoke region. The results of initial segmentation are shown in Figure 3. As seen from Figure 3, one smoke region is divided into some small clusters of smoke pixels. Smoke pixels of each cluster have similar roughness of smoke-color distribution.Figure 3
The results of initial segmentation ((a) and (b) show original image and the results of initial segmentation, resp.).
(a)
Original image
(b)
Results of initial segmentation


### 3.3.2. Merging Small Smoke Region

Although smoke region can be segmented using the proposed method, the method often results in superfine partition. Furthermore, the method cannot distinguish smoke images from heavy fog images. Fortunately, smoke has one characteristic that is different from heavy fog. Smoke concentration gradually decreases from centre to edge of smoke region, which results in the fact thatV component value of smoke region gradually reduces in HSV color space. Therefore, in this section small smoke regions are merged in HSV color space. Initial equivalence relation Ri of one small smoke region Ci is defined as(9)Ri=Ci,U-Ci,where U is discourse domain and [Ci]={Ci∣S(Ci,Cj)≥β1}, i,j=1,2,…,K. β1 is the threshold of smoke region similarity. The threshold β1 is set to 0.7≤β1≤0.8. S(Ci,Cj) is computed as follows:(10)SCi,Cj=∑i,j∈Ci∑p,q∈CjdVi,j,Vp,q.Let Ri and Rj be two initial equivalence relations, and then generalized equivalence relation is defined as(11)Ri′=Pi,U-Pi,where Pi=∪1≤j≤K{Cj∣S(Ri,Rj)≥β2},  i,j=1,2,…,K. β2 is the threshold of equivalence relation similarity. The selection of the threshold β2 is discussed in Section 4.2.1. S(Ri,Rj) is computed as follows:(12)SRi,Rj=Ci∩CjCi∪Cj,where Ci∩Cj denotes the number of the objects in the intersection [Ci]∩[Cj], and Ci∪Cj denotes the number of the objects in the union set [Ci]∪[Cj]. Namely, the similarity of two smoke regions generated in Section 3.3.1 can be computed by the formula (12). If the similarity (Ri,Rj)≥β2, two small smoke regions Ci and Cj are merged into one larger smoke region. Figure 4 shows the results after merging small smoke region. As seen from Figure 4, some small clusters of smoke pixels are merged. Five small regions of the image at the top right of Figure 3(b) are merged into three clusters. Three small regions of the image at the bottom right of Figure 3(b) are merged into two clusters. We can exclude many smoke-color objects according to the characteristic that V component value of smoke pixels in HSV color space reduces from center to edge of smoke region. If one candidate smoke region is divided into 2~3 clusters and the V component value of the smoke region decreases from center to edge, it is decided as smoke region. Otherwise, it is detected as non-smoke regions.Figure 4
The results after merging small smoke region.
(a)
(b)


## 3.3.1. Initial Segmentation of Candidate Smoke Region

Smoke regions can be segmented by gray wave brands that are formed by wave peaks and valleys. Therefore, the selection of peak and valley is important to increase the performance of smoke region extraction. LetPl be one peak of gray level g(80≤g≤120); then the peak Pl will be reserved if it meets the condition Pl>μm-σm, where μm and σm are mean value and variance, respectively. Thus one new peak sequence {Pl1,Pl2,…,Plk} is generated, where li denotes ith gray level. Then peak values of gray wave brand are selected as follows [12, 14]:(8)Plg=max⁡(Pli,Pli+1)ifli-li+1≥PWPliotherwise,where PW=10 is the threshold of peak width. The peak location can be gained on the basis of Plg. Wave valley location can be easily gained by computing the maximum between two adjacent peak values. Wave peaks and wave valleys come into being gray wave brands that are used to segment smoke region. The results of initial segmentation are shown in Figure 3. As seen from Figure 3, one smoke region is divided into some small clusters of smoke pixels. Smoke pixels of each cluster have similar roughness of smoke-color distribution.Figure 3
The results of initial segmentation ((a) and (b) show original image and the results of initial segmentation, resp.).
(a)
Original image
(b)
Results of initial segmentation


## 3.3.2. Merging Small Smoke Region

Although smoke region can be segmented using the proposed method, the method often results in superfine partition. Furthermore, the method cannot distinguish smoke images from heavy fog images. Fortunately, smoke has one characteristic that is different from heavy fog. Smoke concentration gradually decreases from centre to edge of smoke region, which results in the fact thatV component value of smoke region gradually reduces in HSV color space. Therefore, in this section small smoke regions are merged in HSV color space. Initial equivalence relation Ri of one small smoke region Ci is defined as(9)Ri=Ci,U-Ci,where U is discourse domain and [Ci]={Ci∣S(Ci,Cj)≥β1}, i,j=1,2,…,K. β1 is the threshold of smoke region similarity. The threshold β1 is set to 0.7≤β1≤0.8. S(Ci,Cj) is computed as follows:(10)SCi,Cj=∑i,j∈Ci∑p,q∈CjdVi,j,Vp,q.Let Ri and Rj be two initial equivalence relations, and then generalized equivalence relation is defined as(11)Ri′=Pi,U-Pi,where Pi=∪1≤j≤K{Cj∣S(Ri,Rj)≥β2},  i,j=1,2,…,K. β2 is the threshold of equivalence relation similarity. The selection of the threshold β2 is discussed in Section 4.2.1. S(Ri,Rj) is computed as follows:(12)SRi,Rj=Ci∩CjCi∪Cj,where Ci∩Cj denotes the number of the objects in the intersection [Ci]∩[Cj], and Ci∪Cj denotes the number of the objects in the union set [Ci]∪[Cj]. Namely, the similarity of two smoke regions generated in Section 3.3.1 can be computed by the formula (12). If the similarity (Ri,Rj)≥β2, two small smoke regions Ci and Cj are merged into one larger smoke region. Figure 4 shows the results after merging small smoke region. As seen from Figure 4, some small clusters of smoke pixels are merged. Five small regions of the image at the top right of Figure 3(b) are merged into three clusters. Three small regions of the image at the bottom right of Figure 3(b) are merged into two clusters. We can exclude many smoke-color objects according to the characteristic that V component value of smoke pixels in HSV color space reduces from center to edge of smoke region. If one candidate smoke region is divided into 2~3 clusters and the V component value of the smoke region decreases from center to edge, it is decided as smoke region. Otherwise, it is detected as non-smoke regions.Figure 4
The results after merging small smoke region.
(a)
(b)


## 4. Experimental Result and Analysis

The proposed method of candidate smoke region segmentation is performed with an image size of320×240. In the experiments, a total of 2,304 video frames from 16 fire video clips are used to verify the performance of the proposed method, including 326 positive video frames and 129 negative video frames. This section compares our method with other two methods: smoke regions detected based on RGB and HSV color space proposed in [3] (RGB-HSV for short) and smoke regions obtained using regions of interest extraction method based on vision saliency proposed in [7] (ROI for short).


### 4.1. Initial Smoke Region Segmentation

#### 4.1.1. Low Concentration Smoke Region Detection

For one smoke image of low concentration, smoke region is difficultly segmented not only since it is very small, but also since the color difference between it and background is not obvious. For example, RGB-HSV method cannot detect the small smoke region of low concentration. Figure5 shows the results of candidate smoke region extraction using our method and ROI method. As shown in Figure 5, our method and ROI method can effectively segment smoke region of low concentration.Figure 5
Smoke region extraction of low concentration.


#### 4.1.2. Detecting Smoke Region of Images including Interference Objects

Color features of some interference objects in smoke images are similar to smoke region; thus these objects may be falsely detected as smoke and result in the fact that smoke regions cannot be segmented. Figure6 shows experimental results of candidate smoke region extraction using our method, RGB-HSV method, and ROI method. As shown in Figure 6, our method and ROI method can truly segment smoke region of images including interference objects such as blue sky. RGB-HSV method falsely regards sky region as smoke.Figure 6
Smoke region extraction of images including interference objects.


### 4.2. Small Smoke Region Merging

#### 4.2.1. Selection of the Thresholdβ2

The thresholdβ2 is directly related to the performance of candidate smoke region detection. 60 positive images and 50 negative images are randomly selected from experimental dataset in order to select the threshold β2. Figure 7 shows average precision and recall with variety of the threshold β2. Precision and recall rates are computed as follows:(13)Presicion=NCorrectNCorrent+NError,Rate=NCorrectNCorrent+NMiss.As shown in Figure 7, when 0.75≤β2≤0.85, both precision and recall are the highest. Small regions cannot be merged when the threshold β2 is too large. All the regions can be merged into one smoke cluster when the threshold β2 is too small. Therefore, too low threshold or too high threshold does not facilitate the segmentation of smoke regions using the characteristic that smoke concentration gradually decreases from centre to edge in HSV color space.Figure 7
Precision and recall with variety of the thresholdβ2.


#### 4.2.2. Excluding Heavy Fog Region

Color features of dense fog are very similar to those of smoke, and fog pixels are moving pixels. Therefore, it is a difficult problem distinguishing smoke from fog. Figure8 shows V component value variety of smoke region and heavy fog. As shown in Figure 8, the intensity of smoke region decreases from center to edge, and the intensity of fog pixels in one image is nearly the same. Therefore, we distinguish smoke from heavy fog using V component variety of smoke region.Figure 8
V component value variety of smoke region and heavy fog.


### 4.3. Performance of Smoke Region Extraction

Figure9 shows average precision and recall of smoke region extraction using the three methods. It can be clearly seen from this figure that the proposed smoke region extraction using rough set theory outperforms the other two approaches in both precision and recall. Precision and recall of RGB-HSV method are the lowest since RGB-HSV method cannot detect some smoke images of low concentration including interference objects. Recall of ROI method is lower since ROI method falsely considers fog as smoke.Figure 9
Average precision and average recall of three methods.


## 4.1. Initial Smoke Region Segmentation

### 4.1.1. Low Concentration Smoke Region Detection

For one smoke image of low concentration, smoke region is difficultly segmented not only since it is very small, but also since the color difference between it and background is not obvious. For example, RGB-HSV method cannot detect the small smoke region of low concentration. Figure5 shows the results of candidate smoke region extraction using our method and ROI method. As shown in Figure 5, our method and ROI method can effectively segment smoke region of low concentration.Figure 5
Smoke region extraction of low concentration.


### 4.1.2. Detecting Smoke Region of Images including Interference Objects

Color features of some interference objects in smoke images are similar to smoke region; thus these objects may be falsely detected as smoke and result in the fact that smoke regions cannot be segmented. Figure6 shows experimental results of candidate smoke region extraction using our method, RGB-HSV method, and ROI method. As shown in Figure 6, our method and ROI method can truly segment smoke region of images including interference objects such as blue sky. RGB-HSV method falsely regards sky region as smoke.Figure 6
Smoke region extraction of images including interference objects.


## 4.1.1. Low Concentration Smoke Region Detection

For one smoke image of low concentration, smoke region is difficultly segmented not only since it is very small, but also since the color difference between it and background is not obvious. For example, RGB-HSV method cannot detect the small smoke region of low concentration. Figure5 shows the results of candidate smoke region extraction using our method and ROI method. As shown in Figure 5, our method and ROI method can effectively segment smoke region of low concentration.Figure 5
Smoke region extraction of low concentration.


## 4.1.2. Detecting Smoke Region of Images including Interference Objects

Color features of some interference objects in smoke images are similar to smoke region; thus these objects may be falsely detected as smoke and result in the fact that smoke regions cannot be segmented. Figure6 shows experimental results of candidate smoke region extraction using our method, RGB-HSV method, and ROI method. As shown in Figure 6, our method and ROI method can truly segment smoke region of images including interference objects such as blue sky. RGB-HSV method falsely regards sky region as smoke.Figure 6
Smoke region extraction of images including interference objects.


## 4.2. Small Smoke Region Merging

### 4.2.1. Selection of the Thresholdβ2

The thresholdβ2 is directly related to the performance of candidate smoke region detection. 60 positive images and 50 negative images are randomly selected from experimental dataset in order to select the threshold β2. Figure 7 shows average precision and recall with variety of the threshold β2. Precision and recall rates are computed as follows:(13)Presicion=NCorrectNCorrent+NError,Rate=NCorrectNCorrent+NMiss.As shown in Figure 7, when 0.75≤β2≤0.85, both precision and recall are the highest. Small regions cannot be merged when the threshold β2 is too large. All the regions can be merged into one smoke cluster when the threshold β2 is too small. Therefore, too low threshold or too high threshold does not facilitate the segmentation of smoke regions using the characteristic that smoke concentration gradually decreases from centre to edge in HSV color space.Figure 7
Precision and recall with variety of the thresholdβ2.


### 4.2.2. Excluding Heavy Fog Region

Color features of dense fog are very similar to those of smoke, and fog pixels are moving pixels. Therefore, it is a difficult problem distinguishing smoke from fog. Figure8 shows V component value variety of smoke region and heavy fog. As shown in Figure 8, the intensity of smoke region decreases from center to edge, and the intensity of fog pixels in one image is nearly the same. Therefore, we distinguish smoke from heavy fog using V component variety of smoke region.Figure 8
V component value variety of smoke region and heavy fog.


## 4.2.1. Selection of the Thresholdβ2

The thresholdβ2 is directly related to the performance of candidate smoke region detection. 60 positive images and 50 negative images are randomly selected from experimental dataset in order to select the threshold β2. Figure 7 shows average precision and recall with variety of the threshold β2. Precision and recall rates are computed as follows:(13)Presicion=NCorrectNCorrent+NError,Rate=NCorrectNCorrent+NMiss.As shown in Figure 7, when 0.75≤β2≤0.85, both precision and recall are the highest. Small regions cannot be merged when the threshold β2 is too large. All the regions can be merged into one smoke cluster when the threshold β2 is too small. Therefore, too low threshold or too high threshold does not facilitate the segmentation of smoke regions using the characteristic that smoke concentration gradually decreases from centre to edge in HSV color space.Figure 7
Precision and recall with variety of the thresholdβ2.


## 4.2.2. Excluding Heavy Fog Region

Color features of dense fog are very similar to those of smoke, and fog pixels are moving pixels. Therefore, it is a difficult problem distinguishing smoke from fog. Figure8 shows V component value variety of smoke region and heavy fog. As shown in Figure 8, the intensity of smoke region decreases from center to edge, and the intensity of fog pixels in one image is nearly the same. Therefore, we distinguish smoke from heavy fog using V component variety of smoke region.Figure 8
V component value variety of smoke region and heavy fog.


## 4.3. Performance of Smoke Region Extraction

Figure9 shows average precision and recall of smoke region extraction using the three methods. It can be clearly seen from this figure that the proposed smoke region extraction using rough set theory outperforms the other two approaches in both precision and recall. Precision and recall of RGB-HSV method are the lowest since RGB-HSV method cannot detect some smoke images of low concentration including interference objects. Recall of ROI method is lower since ROI method falsely considers fog as smoke.Figure 9
Average precision and average recall of three methods.


## 5. Conclusions

This paper presents a novel candidate smoke region extraction scheme of fire video, which adaptively updates video background and detects smoke regions in terms of roughness definition and equivalence relation definition of small smoke region using rough set theory. Experimental results show that the interference of static and moving smoke-color objects is excluded, and the proposed scheme outperforms the other methods in both precision and recall.It is worth noting that although the performance of the proposed methods has been significantly improved, there are several potential works for future development. One is to amend the roughness of color distribution to effectively eliminate noise. Another is to adaptively select the threshold which contributes to merging small smoke region.



---

*Source: 280415-2015-03-26.xml*