# Learning-Based Colorization of Grayscale Aerial Images Using Random Forest Regression
## 2 Materials and Methods
### Method
#### 2.3.1 Preprocessing  
As mentioned above, the proposed preprocessing step is divided into color-space conversion and extraction of training pixels. First, in this study, a Lab color space is selected since its underlying metric has been designed to express color coherency. Furthermore, Lab has three coordinates: L is the luminance, or lightness, which consequently represents the grayscale axis, whereas a and b represent the two-color axes [16]. In other words, the L component can be known in advance through the  
Figure 5: The flowchart of the proposed method. RGB: Red-Green-Blue color space, Lab: CIE L“a”b color space, L component: grayscale axis, a and b components: two-color axes, RF: random forest. input grayscale image, and only the remaining 2D color information, a and b, can be predicted [23]. Thus, the color space of the reference color image is converted from RGB to Lab, and, in the colorization step, only the color relationships for a and b are established through regression. Then, in order to extract a meaningful set of training data, change detection between the input grayscale image and the L component of the reference color image is performed. The change detection method used here comprises two steps for accurate extraction. The first step is a pixel-based method that uses principal component analysis (PCA) [40], and the threshold for distinguishing between changed and unchanged pixels is selected using [PERSON]'s method. However, this process will result in fragmentation and incomplete expression of the change [41]. Therefore, the second step applies the object-based method, which consists of four sub-steps: (1) the morphological closing operation, (2) gap filling, (3) the morphological opening operation, and (4) elimination of small patches. The morphological closing operation is the combination of dilation followed by erosion, which is used to remove holes in the image [42]. Thus, the closing operation is applied to the image to fill the spaces. Then, gaps within the changed regions that are not filled by the closing operation are additionally filled, which makes the changed information more complete [41]. The morphological opening is then applied, in which erosion is conducted on the image, and it is followed by a dilation operation. The aim of the opening is to remove unnecessary portions. For the structure elements used in the morphological operation, the closing and the opening are set to 3 \(\times\) 3 and 5 \(\times\) 5, respectively, as selected in [PERSON] et al. [41]. Small, insignificant patches persist following the opening processing, which can be removed by applying an area threshold. The area threshold is set based on the minimum object size and is acquired through a zero-parameter version of simple linear iterative clustering (SLICO). The SLICO is a spatially localized version of the k-means [43]. To initialize it, the k-cluster centers, which are located on a regular grid and spaced S pixels apart, are sampled [44]. Then, an iterative procedure assigns each pixel to a cluster center using the distance measure D, as defined in Equation (2), which combines the distance of color proximity (Equation (3)) and the distance of spatial proximity (Equation (4)) [45]:  
\[\mathrm{D}=\ \sqrt{\left(\frac{d_{c}}{m}\right)^{2}+\ \left(\frac{d_{s}}{S} \right)^{2}} \tag{2}\]  
\[d_{c}=\ \sqrt{\sum_{s_{i}\in S}\left(I(x_{1},y_{1},\ s_{i})-I(x_{2},y_{2},s_{i}) \right)^{2}} \tag{3}\]  
\[d_{s}=\ \sqrt{\left(x_{1}-x_{2}\right)^{2}+\ \left(y_{1}-y_{2}\right)^{2}} \tag{4}\]  
where \(d_{c}\) and \(d_{s}\) represent the color and spatial distance between pixels \(I(x_{1},y_{1},\ s_{i})\) and \(I(x_{2},y_{2},s_{i})\) in the spectral band \(S_{i}\), respectively; and \(m\) controls the compactness of the superpixels, which is adaptively chosen for each superpixel. In this study, the number of iterations is set to ten, which is sufficient for error convergence [46], and the initial size of the superpixels is set to 10 \(\times\) 10, which represents optimal accuracy and computational time [45].