# CNN-Based Illumination Estimation with Semantic Information
## 2 Technical Approach
### The Proposed DCNN Architecture  
A deep convolutional neural network (DCNN) is a major breakthrough in image classification. The DCNN naturally incorporates low, mid, and high-level image features and classifiers in an  
Figure 1: Block diagram of the proposed method. end-to-end multi-layer form. Its depth, or the number of stacked layers, can enrich the \"level\" of image features. However, some deeper networks like AlexNet and VGG-16 have a degradation problem. Increasing depth of the network accelerates accuracy degradation as well as accuracy saturation. That is, an increasing number of layers cause some deeper network models to make more training errors. This is why the proposed method has adopted the well-known residual block to solve the issues with Alexnet and VGG-16 such as expensive computational cost, gradient degradation, and vanishing gradient, which are common issues in handling deep convolutional neural networks. In the proposed approach, a residual network is comprised of multiple convolutional layers. With the input of \(y_{i-1}\), the output of the ith block is recursively defined as follows:  
\[y_{i}\equiv f_{i}(y_{i-1})+y_{i-1} \tag{8}\]  
Let each layer take the sequential steps of convolution \(f_{i}(x)\), batch normalization, and rectified linear unit (ReLU) as nonlinearities, and \(f_{i}(x)\) is defined as follows:  
\[f_{i}(x)\equiv W_{i}\cdot\sigma\Big{(}B\Big{(}W_{i}^{\prime}\cdot\sigma(B(x)) \Big{)}\Big{)} \tag{9}\]  
where \(W_{i}\) and \(W_{i}^{\prime}\) are weight matrices and \(\cdot\) denotes convolution, \(B(x)\) is batch normalization, and \(\sigma(x)\equiv\max(x,\,0)\). The proposed ResNet architecture shows that the resolution of feature maps drop down to a fourth of input resolution after passing through the first three layers. This allows the architecture to aggregate contexts and train faster. However, smaller feature maps constrain the architecture from learning high-resolution features which is useful and required at later stages. To support the learning of the high-resolution feature, the proposed network has an additional convolution layer with a \(3\times 3\) kernel before the first convolution layer. This enables the network to learn high-resolution features, without increasing the inference time by much. Furthermore, down-sampling principally reduces the resolution of feature maps. Although deconvolution layers are able to up-sample low-resolution feature maps, they cannot recover all the details completely. In addition, this procedure requires higher computational cost as well as intensive memory. To address such problems, the proposed method uses atrous convolution, also called dilated convolution [26]. Atrous convolution widens the kernel and simulates a larger field of perception. For a 1D input signal \(x[i]\) with a filter \(w[k]\) of \(K\) in length, the atrous convolution is described as follows:  
\[y[i]=\sum_{k=1}^{k}x[i+r\times k]w[k] \tag{10}\]  
The rate \(r\) refers to a stride with regard to sampling of the input signal. For instance, a rate of 2 represents a convolution on a \(2\times 2\) pooled feature map. The proposed network has changed the stride of the last convolution from 2 to 1 and set the others at \(r=2\). In this way, the smallest resolution is 16 times down-sampled, not 32 times, but still preserves the higher resolution details, as well as aggregates the usual number of contexts. Every object in a scene potentially varies in size, distance, and position. DCNN filters usually do not fit in this multiscale appearance. This has motivated researchers to investigate how DCNN [27; 28] learns the multiscale feature. Their finding is that DCNN is given multi-resolution input images, which thus incur a higher computational cost. To reduce expensive computational cost and increase estimation efficiency, the proposed method gets the ResNet blocks made up of several different scale atrous convolutions with \(r>1\). In this way, the network is enabled to learn multiscale features in every block.