# Infrared Image Enhancement Using Convolutional Neural Networks for Auto-Driving
## 3 Methodology  
In this section, we use convolutional neural networks consisting of feature extraction and enhancement modules to enhance infrared images. The input image is first processed by the feature extraction module, and then, goes to the image enhancement module and finally outputs the enhanced image. The function of the feature extraction module is to extract the features of the input IR image and to concatenate and fuse these features to obtain a pre-fused image. The enhancement module is utilized to enhance the pre-fused image and to obtain a result similar to the target image. In the proposed method, infrared image enhancement is considered a supervised learning problem, and low- and high-contrast images are considered input and output data, respectively. Figure 1 shows the structure of our model. We define the low-contrast image as the input \(X\), and the corresponding high-contrast image as the output \(Y\). Assuming that\(f_{1}\) and\(f_{2}\) denote the function of the feature extraction and image enhancement modules, respectively, our model can be written as a combination of two functions:  
\[Y=f(X)=f_{2}(f_{1}(X)) \tag{1}\]  
The feature extraction module consists of three branches, a concatenation layer, and a fusion layer. The three branches are connected in parallel to extract the first, second, and third feature images from the input infrared image. The first branch includes a convolutional layer and a ReLU activation layer. \[X_{01}=\max(0,X*W_{01}+b_{01}) \tag{2}\]  
Here, \(X_{01}\) denotes the output of the first branch, and * denotes the convolution operation. \(W_{01}\) and \(b_{01}\) denote the convolution kernel of the convolution layer and offset, respectively. max corresponds to ReLU operation. The second and third branches both include two convolutional layers and a ReLU activation layer:  
\[X_{0i1}=\max(0,X*W_{0i1}+b_{0i1}) \tag{3}\]  
\[X_{0i}=\max(0,X_{0i1}*W_{0i2}+b_{02})(\ i=2,3) \tag{4}\]  
where \(X_{0i}\) denotes the output of the \(i\) branch, and \(X_{0i1}\) denotes the output of the first convolutional layer and ReLU activation layer of the first branch. \(W_{0i1}\) and \(b_{0i1}\) are the convolutional kernel and offset for the first convolutional layer in branch \(i\), respectively. \(W_{0i2}\) and \(b_{0i2}\) are the convolutional kernel and offset for the second convolutional layer in branch \(i\), respectively. Figure 1: Structure of the proposed convolutional neural network for infrared image enhancement. The concatenation layer connects the feature image outputs from the three branches by channel. The input of the fusion layer is the output of the concatenation layer, and outputs the pre-fusion image, including a convolutional layer and a ReLU activation layer. \[X_{04}=[X_{01},X_{02},X_{03}] \tag{5}\]  
\[X_{1}=\max(0,X_{04}*W_{04}+b_{04}) \tag{6}\]  
Here, \(X_{04}\) denotes the output of the splicing layer. \(W_{04}\) and \(b_{04}\) denote the convolution kernel of the convolution layer in the fusion layer and the offset, respectively. The training images in the dataset usually have low luminance, so an image enhancement module is proposed following the convolutional difference strategy. The input of the image enhancement module is \(X_{1}\), which generates an output \(X_{2}\) with the same width and height. \[X_{11}=\max(0,X_{1}*W_{11}+b_{11}) \tag{7}\]  
\[X_{1i}=\max\Bigl{(}0,X_{1(i-1)}*W_{1i}+b_{1i}\Bigr{)}(i=2,3,\ldots,8) \tag{8}\]  
Here, \(X_{11}\) denotes the output of the first convolution. \(W_{11}\) and \(b_{11}\) denote the convolution kernel and offset of the first convolution, respectively. \(W_{1i}\) and \(b_{1i}\) denote the convolution kernel and offset of the \(i\) convolution, respectively. The images after each convolution are then joined by the channel and convoluted again as follows:  
\[X_{19}=[X_{11},X_{12},\ldots,X_{18}] \tag{9}\]  
\[X_{01}=\max(0,X*W_{01}+b_{01}) \tag{2}\]0  
where \(X_{19}\) denotes the output after concatenation, and \(X_{2}\) denotes the output after convolution.