## 3 Materials and Methods
### Architecture
#### 3.2.2 Rain Accumulation Removal  
Rain streaks and rain accumulation impose different visual distortion effects on captured images. Distant rain streaks accumulate and impose a distortion similar to fog, which also causes visibility degradation and leads to further hiding of image and present objects' features, thus should also be removed. Since its distortion effect caused by rain accumulation is similar to fog (Equation (1)), then removal of rain accumulation may follow same algorithm (e.g., [13; 58; 64]).  
The rain accumulation removal network is based on a transmission-guided lightweight neural network (TGL-Net) [65]. In this network, instead of producing simulated transmission maps from both depth data and clear images, guided transmission maps are automatically computed using a filter-refined dark-channel-prior (F-DCP) method which facilitates network training on both synthetic and natural images.  
The architecture is based on the very deep residual encoder-decoder network (REDNet) [66] which owns a symmetric architecture and employs skip connections; however, TGL-Net has fewer layers and parameters and is very lightweight. TGL-Net is composed of three stages: (i) downsampling, (ii) encoder-decoder, and (iii) upsampling, as shown in Figure 4[65].  
Figure 4: TGL-Net’s Symmetric architecture, comprised of three stages: downsampling, encoder–decoder, and upsampling. Dotted lines are skip connections [65].  
Figure 3: Progressive network architecture.  
The downsampling phase includes a convolutional layer (\(3\times 3\)) then a max pooling layer. The convolutional layer extracts images features, and then the max-pooling layer reduces the total amount by 25 times by using a stride of 5. This leads to improving the computational efficiency of the network.  
In the encoder-decoder phase, three convolutional and three deconvolutional layers are connected through a condensed encoder-decoder connection. This stage performs feature extraction and transmission estimation. The convolutional layers extract further image features and also clear noise. On the other hand, the deconvolutional layers recover the details of transmission maps. In the encoder, the sizes of the convolutional kernel are \(3\times 3\), \(5\times 5\), and \(5\times 5\) sequentially, which are equal to the corresponding deconvolutional kernels in the decoder, but in reverse.  
The symmetric structure of the convolutional and deconvolutional links retains the benefits from the residual network characteristics [43], including rapid convergence and high accuracy. Using skip connections that directly propagate signals to subsequent layers reduces the common problem of vanishing gradients which commonly happens in deep neural networks (DNNs) [67]. Moreover, using skip connections helps maintain useful image features and details.  
After the encoder-decoder layers, another convolutional layer is used, it combines a three-channel feature map into a single-channel transmission map. Additionally, a non-linear sigmoid activation function has been applied, it reaches a saturation region when the input is either too small or too big (i.e., larger than 5 or smaller than \(-5\)), which increases the risk of vanishing gradient when adding more layers. However, the employment of ELU units and skip connections in the encoder-decoder layers prevent the vanishing gradient problem. The non-linear sigmoid function outputs a preliminary estimation of a reduced transmission map which is fifth the size of the input image.  
In order to enlarge the transmission map to the same size as the input image, the upsampling stage is applied. The upsampling stage is composed of two steps. First, image expansion is performed using bilinear interpolation to maintain the computational efficiency of the network. Second, a convolutional layer is used to further refine the output of the transmission map.