# Attention-Gate-Based Model with Inception-like Block for Single-Image Dehazing
## 3 Proposed Method
### Attention Gate  
The original U-Net feature fusion involves directly concatenating the output of the transposed convolution and the output of the encoder with the same resolution, while [PERSON] et al. [27] proposed Attention U-Net to replace the direct concatenation part with the attention gate; that is, the same spatial dimension of the feature maps needs to be multiplied by the same attention coefficient between 0 and 1, as in Equation (6). This attention mechanism is beneficial for automatically highlighting salient regions for semantic segmentation.  
\[\hat{x}_{i,c}^{l}=\alpha_{i}^{l}\times x_{i,c}^{l},\ 1\geq\alpha_{i}^{l}\geq 0, \tag{6}\]  
where \(i\) is the spatial dimension, \(c\) is the channel dimension, \(l\) is the layer, \(x_{i,c}^{l}\) is the input, \(\alpha_{i}^{l}\) is the attention coefficient, and \(\hat{x}_{i,c}^{l}\) is the output.  
Due to different task orientations, to pay more attention to the unclear region of the hazy image, the values of the same spatial dimension are multiplied by different attention coefficients, as follows:  
\[\hat{x}_{i,c}^{l}=\alpha_{i,c}^{l}\times x_{i,c}^{l},\ \ \alpha_{i,c}^{l}\geq 0, \tag{7}\]  
where \(\alpha_{i,c}^{l}\) is positive, which differs from Attention U-Net [27].  
The details of the attention gate are shown in Figure 4, where the attention coefficients are used to pay more attention to the unclear region of the hazy image during backpropagation. The difference between the original attention coefficient and the proposed attention coefficient is shown in Figure 5, where the first half was proposed by Attention U-net [27] for semantic segmentation. The second half is proposed in this study for single-image dehazing. The attention coefficient is determined by the gate signal and the output of a certain layer of the encoder. The gate signal is the result of the transposed convolution of the decoder. For example, in Figure 1, the output of inception-like block 1 is the input of the attention gate, and inception-like block 4, after the transposed convolution, is the gate signal.  
Figure 4: An illustration of an attention gate.  
Figure 5: Schematic diagram of an attention gate with single and multiple attention coefficients.  
The attention coefficient is calculated using Equation (8). The activation function adopts the ReLU function to ensure that the attention coefficient is greater than 0.  
\[\alpha_{i,r}^{l}=\sigma\Big{(}\psi^{T}\Big{(}\sigma\Big{(}W_{x}^{T}x_{i}^{l}+W_{g }^{T}g_{i}^{l}+b_{g}\Big{)}\Big{)}+b_{\psi}\Big{)}, \tag{8}\]  
where \(x_{i}^{l}\) is the input, \(\sigma\) is the ReLU function, \(W_{x}\in\mathbb{R}^{C_{i}\times C_{int}}\), \(W_{g}\in\mathbb{R}^{C_{g}\times C_{int}}\), \(C_{l}\) is the number of channels in the input feature map, \(C_{g}\) is the number of channels in output feature map, \(C_{int}\) is the number of channels in the intermediate feature map, \(\psi\in\mathbb{R}^{C_{int}\times C_{l}}\), and \(b_{g}\) and \(b_{\psi}\) are biases.  
Figure 6 shows that the attention gate calls attention to objects of distant buildings, ground textures, trees, and pedestrians that are blurred due to haze. The feature map after the attention gate changes the gain by the attention coefficient to achieve the effect of focusing on objects with complex structures and suppressing objects with relatively simple structures, such as focusing on the characteristics of pedestrians and suppressing the characteristics of the sky.