# Learning-Based Colorization of Grayscale Aerial Images Using Random Forest Regression
## 1 Introduction  
Image colorization can be described as the process of assigning colors to the pixels of a grayscale image in order to increase the image's visual appeal [1]. This application is often utilized in the image processing community to colorize old grayscale images or movies [2]. Particularly in aerial and satellite imagery, the problem with colorization occurs in a multitude of scenarios that seek to replace topographic maps with vivid, photorealistic renderings of terrain models [3]. There are three main reasons for colorizing aerial and satellite images: (1) grayscale satellite images are available at higher spatial resolutions than their color counterparts are; (2) there are many old grayscale aerial and satellite images that should be represented by color images, typically for monitoring purposes; and (3) grayscale aerial-satellite images can be obtained for approximately one-tenth the cost of color images of the same resolution [3]. In the case of satellite images, modern systems acquire a panchromatic (grayscale) image, with high spatial and low spectral resolutions, and a multispectral (color) image that has complementary properties [4]. In other words, grayscale and color images with different resolutions over the same time period are provided. In order to perform colorization through the color information of the multispectral image while maintaining the high resolution of the panchromatic image, the two components are fused, which iscalled pansharpening [5]. The fused images provide increased interpretation capabilities and more reliable results [6]. However, this colorization method is confined to satellite images that provide panchromatic and multispectral images of the same time periods, making this type of colorization of aerial images impossible. On the other hand, grayscale aerial image colorization is a poorly posed problem with more than one solution [7]. As mentioned above, the satellite images fuse the grayscale and color images from the same time period to perform colorization, whereas aerial images do not usually have the two types of imagery available. Furthermore, in contrast to the natural images, this colorization solution depends on the sun elevation angle, season, sensor parameters, etc. It is also problematic that the same intensity may represent different colors, so there is no exact solution [8]. In general, existing colorization methods can be divided into three main categories, all of which have limitations: user-scribbled methods, example-based methods, and those that employ a large number of training images [9]. User-scribbled techniques [10; 11; 12; 13] require a user to manually add colored marks to a grayscale image [13]. The colors from these marks are then smoothly propagated across the entire image, based on an optimization framework. A key weakness is that such methods require users to provide a considerable number of scribbles on the grayscale image, which is time-consuming and requires expertise [14]. Moreover, it is almost impossible to add such markings to large volumes (gigabytes) of aerial imagery. In the case of the example-based method [14; 15; 16; 9; 1], it typically transfers the color information from a similar reference image to the input grayscale image rather than obtaining chromatic values from the user, thereby reducing the burden on users. However, as feature matching is critical to the quality of the results, satisfactory results cannot be obtained if feature matching is not performed correctly [15]. Moreover, the procedure is very sensitive to image brightness and contrast, whereas real aerial images always include large areas of shadow and low contrast, due to relief, vignetting, and so on. An alternative approach is to employ a large number of training images [17; 18; 19], which is a recent example of deep learning. These methods use multiple color images to automatically transfer the color information to the grayscale image, and deep neural networks are used to solve the colorization problem.