# Single Image Reflection Removal Based on Residual Attention Mechanism
## 3 Proposed Method
### Network Architecture
If the time series data is an image, adding a convolution operation to LSTM can improve the capacity to capture the spatial characteristics of the image, which will be more effective than using LSTM only. As shown in Figure 4, the ConvLSTM module has four gates, including input gate \(i\), forgetting gate \(f\), output gate \(o\), and unit state \(c\). Input gate \(i\) controls how much information from the current computation is added to the cell state. Forgetting gate \(f\) determines how much of the information passed over from the previous iteration will be forgotten. Output gate \(o\) controls how much information from the current state will be passed to the next iteration. Unit state \(c\) controls the state after passing through the input gate and the forgetting gate, which determines the current state of a unit. In each iteration, the information from the previous step is saved and provided to the next step in the ConvLSTM, which can be used to solve the gradient elimination problem in recursive convolutional neural networks. The key equation of ConvLSTM is shown in Equation  
Figure 3: Our discriminator: the output is a two-dimensional matrix in which each element represents a local region of the input image, and the value of the matrix element is 1 if the local region is real and 0 otherwise. (1), where \(*\) denotes the convolution operation, \(\circ\) denotes the Hadamard product, and \(b\) represents the activation value of the cell. \[ \begin{split} i_{t}=\sigma(W_{xi}*x_{t}+W_{hi}*h_{t-1}+W_{ci}\circ c _{t-1}+b_{i})\\ f_{t}=\sigma\Big{(}W_{xf}*x_{t}+W_{hf}*h_{t-1}+W_{cf}\circ c_{t-1}+b_{ f}\Big{)}\\ c_{t}=f_{t}\circ c_{t-1}+i_{t}\circ tanh(W_{xc}*x_{t}+W_{hc}*h_{t-1}+b_{ c})\\ o_{t}=\sigma(W_{xo}*x_{t}+W_{ho}*h_{t-1}+W_{co}\circ c_{t}+b_{o})\\ h_{t}=o_{t}\circ tanh(c_{t})\end{split}  \tag{1}\]  
As shown in Figure 1, the reflection removal network progressively refines the prediction results for the transmission and reflection layers through iterations. At the beginning of training, the I and T are initialized as the original reflection image I, and R is set to be a tensor of the same size as I, and the value of each pixel is set to 0.1 for the three RGB channels. In addition, the input to the network is a cascade of T, I, and R. The predicted values of the new transmission and reflection layers, \(\hat{\text{T}}\) and \(\hat{\text{R}}\), are then obtained by convolving two sub-networks of the LSTM utilized as inputs for the following time step, while information is memorized in the LSTM network for the prediction of the next time step. The predicted values of the transmission and reflection are reconstructed by the hybrid image formation model for the hybrid image \(\hat{\text{I}}\). The reconstructed hybrid image is used to compare with the real input hybrid image to guide the output of the network; there are corresponding loss functions between the predicted transmission layer image and the real transmission layer image as well as between the predicted reflection image and the real reflection image. Finally, the result of the prediction network for the transmission layer at the final step n is the transmission layer's final prediction result.