# Plane Detection Using Appearance-Based Pre-Processing and Convolutional Neural Network
## 2 Proposed Method
### Image Pre-Processing  
In this study, HSV color transformation was performed with the first image pre-processing. The HSV color model can handle color features in a similar way to how humans perceive colors, so it can be used to identify colors of objects in many applications, in addition to image pre-processing. These properties of HSV color models make them ideal tools for developing image processing algorithms based on color sensing properties [21; 22]. In HSV color models, hue represents the distribution of colors based on the longest wavelength of red, and saturation represents the degree to which pure colors contain white light. Value is also used to measure the intensity of light. The value can be independent of a single component to control the range, thus creating algorithms that are robust to lighting changes. \[ROI_{HSV(x,y)}=\left\{ \begin{array}{rl}&(20<H(x,y)<40)\ and\\ 1,&(50<S(x,y)<255)\ and\\ &(50<V(x,y)<255)\\ &0,\qquad otherwise\\ \end{array} \right. \tag{1}\]  
In Equation (1), the pixel values 1 for \(H\), \(S\), and \(V\), respectively refer to the regions corresponding to the color space where flames could exist at the image location, and those in that range were extracted as the \(ROI\) of the first image pre-processing. A pixel value of 0 means a pixel is classified as a non-flame area. Figure 2 shows the HSV color conversion: Figure 2a is the original flame image, and Figure 2b is the resultant image from applying the HSV color conversion. However, even after HSV color conversion, results from items other than flames or objects containing light-yellow remained, such as leaves. Therefore, to filter this additionally, a Harris corner detector was used for the second image preprocessing. Figure 1: The flame detection algorithms proposed in this paper. First, when there is a reference point \((x,y)\) in the image, when the amount of change is moved by \((u,v)\) from the reference point, it can be expressed as Equation (2). \(I\) is the brightness, and \((x_{i},\ y_{i})\) are the points inside the Gaussian window \(W\). \[E(x,y)=\sum_{W}[I(x_{i}+u,\ y_{i}+v)]^{2} \tag{2}\]  
The Taylor series allows the area that has moved as much as \((u,\ v)\) to be organized as in the following Equation (3). \[\text{I}(x_{i}+u,\ y_{i}+v)\approx I(x_{i},\ y_{i})+\big{[}I_{x}(x_{i},\ y_{i})I_{y}(x_{i},\ y_{i})\big{]}\bigg{[} \begin{array} []{c}u\\ v\end{array} \bigg{]} \tag{3}\]  
The first-order derivative in the \(x\) and \(y\) directions, \(I_{x}\) and \(I_{y}\), could be obtained via convolution arithmetic using \(S_{x}\), the Sobel \(x\) kernel, and \(S_{y}\), the Sobel \(y\) kernel, as shown in Figure 3.  
If Equation (3) is substituted for Equation (2), it can be expressed as Equation (4). \[E(u,v)=[u\ v]\Bigg{[} \begin{array}{c}\sum_{W}(I_{x}(x_{i},\ y_{i}))^{2}\ \sum_{W}I_{x}(x_{i},\ y_{i})I_{y}(x_{i},\ y_{i})\\ \sum_{W}I_{x}(x_{i},\ y_{i})I_{y}(x_{i},\ y_{i})\ \sum_{W}(I_{y}(x_{i},\ y_{i}))^{2} \end{array} \Bigg{]}\bigg{[} \begin{array}{c}u\\ v\end{array} \bigg{]}=[u\ v]M\bigg{[} \begin{array}{c}u\\ v\end{array} \bigg{]} \tag{4}\]  
Figure 3: Sobel x kernel and y kernel. Figure 2: Conversion of flame image using the HSV color model. (**a**) Original image before color conversion; and (**b**) HSV color-converted image within the set range. If \(M\) is defined as \(M=\left[ \begin{array}{cc}A&C\\ C&B\end{array} \right]\), properties such as Equations (5) and (6) are satisfied. Finally, Equation (7) allows us to determine the edge, corner, and flat. \(K\) is an empirical constant, and a value of 0.04 was used in this paper. \[det(M)=AB-C=\lambda_{1}\lambda_{2}, \tag{5}\]  
\[trace(M)=A+B=\lambda_{1}+\lambda_{2} \tag{6}\]  
\[R(x,\ y)=det(M)-k(trace(M))^{2} \tag{7}\]  
Each pixel's location will have a different value, and the final calculated \(R(x,y)\) will be compared to the following conditions to distinguish between the edge, corner, and flat [23; 24; 25; 26].