# Frequency-Based Haze and Rain Removal Network (FHRR-Net) with Deep Convolutional Encoder-Decoder
## 2 Proposed Frequency-Based Haze and Rain Removal Network (FHRR-Net) Framework
### Frequency-Based Haze and Rain Removal Network (FHRR-Net) Architecture  
Restoring degraded images by haze and rain using the models is equivalent to solving the inverse problem of the models. Deep neural network is used to address the lack of information problem that is characteristic of a single degraded image. In this section, details of the structure of proposed encoder-decoder networks is introduced. The details of the overall network architecture are shown in Table 1.  
Figure 4: Applying guided filter on the hazy and rainy input image. (**a**) Input image. (**b**) Low-frequency of the image. (**c**) High-frequency of the image.  
Inspired by the residual network [27] and the encoder-decoder style network [28], the proposed FHRR-Net consists of two encoder-decoder models: one is for training the high-frequency layer of the input image, and one is for training the low-frequency layer of the image. Dilated structures and symmetric skip connections are applied to increase the image restoration performance of the network as illustrated in Figure 5. It also shows the linking of the encoder and decoders through symmetric skip connections. After the image is decomposed into low and high frequency domain using guided filter, it goes through high-frequency network or low-frequency network. Both network architectures are symmetric, which is efficient for estimating frequency domain into image domain.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline \multicolumn{5}{c}{**High-Frequency Encoder-Decoder Network**} \\ \hline
**Layer** & **Kernel Size** & **Dilated Factor** & **Output Size** & **Skip Connections** \\ \hline Input & & & 256 \(\times\) 256 & \\ Encoder\_conv 0 & 3 & 1 & \(16\times 256\times 256\) & \\ Encoder\_conv 1\_1 & 3 & 2 & \(32\times 128\times 128\) & \\ Encoder\_conv 1\_2 & 3 & 2 & \(32\times 128\times 128\) & Decoder\_deconv 2 \\ Encoder\_conv 2\_1 & 3 & 2 & \(64\times 64\times 64\) & \\ Encoder\_conv 2\_2 & 3 & 2 & \(64\times 64\times 64\) & Decoder\_deconv 1 \\ Encoder\_conv 3\_1 & 3 & 2 & \(128\times 32\times 32\) & \\ Encoder\_conv 3\_2 & 3 & 2 & \(128\times 32\times 32\) & \\ \hline Decoder\_deconv 1 & 2 & 2 & \(64\times 64\times 64\) & Encoder\_conv 2\_2 \\ Decoder\_conv 1\_1 & 3 & 2 & \(64\times 64\times 64\) & \\ Decoder\_conv 1\_2 & 3 & 2 & \(64\times 64\times 64\) & \\ Decoder\_deconv 2 & 2 & 2 & \(32\times 128\times 128\) & Encoder\_conv 1\_2 \\ Decoder\_conv 2\_1 & 3 & 2 & \(32\times 128\times 128\) & \\ Decoder\_conv 2\_2 & 3 & 2 & \(32\times 128\times 128\) & \\ Decoder\_deconv 3\_2 & 2 & 2 & \(32\times 256\times 256\) & \\ Decoder\_conv 3\_1 & 3 & 2 & \(16\times 256\times 256\) & \\ Decoder\_conv 3\_2 & 3 & 2 & \(16\times 256\times 256\) & \\ \hline Output & 3 & 2 & \(3\times 256\times 256\) & \\ \hline \multicolumn{5}{c}{**Low-frequency Encoder-decoder network**} \\ \hline Layer & Kernel size & Dilated factor & Output size & Skip connections \\ \hline Input & & & 256 \(\times\) 256 & \\ Encoder\_conv 0 & 3 & 1 & \(16\times 256\times 256\) & \\ Encoder\_conv 1\_1 & 3 & 2 & \(32\times 128\times 128\) & \\ Encoder\_conv 1\_2 & 3 & 2 & \(32\times 128\times 128\) & Decoder\_deconv 2 \\ Encoder\_conv 2\_1 & 3 & 2 & \(64\times 64\times 64\) & \\ Encoder\_conv 2\_2 & 3 & 2 & \(64\times 64\times 64\) & Decoder\_deconv 1 \\ Encoder\_conv 3\_1 & 3 & 2 & \(128\times 32\times 32\) & \\ \hline Decoder\_deconv 1 & 2 & 2 & \(64\times 64\times 64\) & Encoder\_conv 2\_2 \\ Decoder\_conv 1\_1 & 3 & 2 & \(64\times 64\times 64\) & \\ Decoder\_conv 1\_2 & 3 & 2 & \(64\times 64\times 64\) & \\ Decoder\_deconv 2 & 2 & 2 & \(32\times 128\times 128\) & \\ Decoder\_conv 2\_1 & 3 & 2 & \(32\times 128\times 128\) & \\ Decoder\_conv 2\_2 & 3 & 2 & \(32\times 128\times 128\) & \\ Decoder\_deconv 3 & 2 & 2 & \(32\times 256\times 256\) & \\ Decoder\_conv 3\_1 & 3 & 2 & \(16\times 256\times 256\) & \\ Decoder\_conv 3\_2 & 3 & 2 & \(16\times 256\times 256\) & \\ \hline Output & 3 & 2 & \(1\times 256\times 256\) & \\ \hline \hline \end{tabular}
\end{table}
Table 1: FHRR-Net architecture details.