# CNN-Based Illumination Estimation with Semantic Information
## 3 Experimental Results and Evaluations  
A feasibility study uses two benchmark standard datasets: the reprocessed [15] Color Checker Dataset [15] and the NUS 8-Camera Dataset [8]. These datasets consist of 568 and 1736 raw images, respectively. The \(768\times 384\) input image in Figure 2 is resized to \(512\times 512\) pixels and then cropped into overlapping \(224\times 224\) image patches. There is a trade-off between patch coverage (and accuracy) and efficiency. With more patches, the CNN performs higher coverage and accuracy, but gets lower efficiency. Through additional pooling, the proposed network combines patch-based estimates to obtain a global illuminant. The proposed network is trained in an end-to-end fashion with back-propagation. For the proposed network, Adam [29] is used to optimize parameter setting for all layers, which reduces overfitting and improves performance. The experiment with the proposed network is performed to compare total training losses at four different learning rates with 10,000 iterations (or epochs) using a server with Titan XP GPU and taking 1.5 days. Figure 3 illustrates the comparative experimental results and finds that \(4\times 10^{-4}\) is the optimized base learning rate. The symbol \"1.00E-03\" represents a learning rate of \(1\times 10^{-3}\). Likewise, parameters are optimized, including a dropout probability of 0.5 for the \(6\times 6\times 64\) convolution layer in Figure 2, a batch size of 16, a weight decay of \(5\times 10^{-5}\), and so forth. Figure 2: Proposed deep convolutional neural network (DCNN) architecture with pre-activation, atrous convolution, and batch-normalization. Figure 4 compares median angular errors with and without semantic information, recording the errors every 20 iterations (or epochs). As a result, the errors sharply drop with semantic information. From an illuminant estimation point of view, the choice of semantic information has the effect of improving computational color constancy significantly. Figure 5 shows [PERSON]'s re-processed dataset [30] and their resulting images from implementing the proposed network in Tensorflow [31]. The proposed DCNN architecture is focused on increasing the accuracy of estimating the local semantic information to improve performance. As a result, in Figure 5e, the greenish blue illuminant of the original image is efficiently removed, and the true colors of objects are well represented without color distortion compared with the original image in Figure 5a. Figure 4: Comparison of median angular errors with and without semantic information (SI). Figure 3: Total training loss comparison in the logarithm space of four different learning rates to optimize the base learning rate. The proposed network is compared with 27 different state-of-the-art methods which include both unitary and combinational methods. The 27 different methods are benchmarked from several sources. Specifically, AlexNet-FCand SqueezeNet-FC are benchmarked from [22]; and except for DS-Net, the other 22 methods are from [32; 33; 34; 35; 36; 37; 38; 39]. DS-Net is cited from [40]. In this comparative study, the source codes of AlexNet-FC and SqueezeNet-FC are downloaded from GitHub website [41] and DS-Net is downloaded from GitHub as well [42]. The source codes of CCATI [23] and [PERSON] et al. [24] are implemented by MATLAB and Tensorflow, with parameters fixed as suggested by those articles. For quantitative comparison purposes only, Table 1 compares the proposed method with previous mainstream algorithms in terms of the illuminant estimation accuracy. It illustrates several standard metrics: mean, median, trimean, mean of the best quarter (best 25%), and mean of the worst quarter (worst 25%) of angular error (Equation (1)). This comparative study uses the well-known dataset, the [PERSON]'s dataset [15], which contains 568 images of people, places, and objects in indoor and outdoor scenes, where the Macbeth color checker chart is placed in a known location of every scene.