# Monocular Depth Estimation Using Res-UNet with an Attention Model
## 3 Proposed Method
### Proposed Model
#### 3.2.5 Model Training  
In the first step, the NYU-depth-v2 dataset was rearranged, and all the bad images were removed. The images along with depth maps were resized to 128 \(\times\) 128 to input to the network. The size of input is not fixed and can be resized to any square size such as 128 \(\times\) 128 and 256 \(\times\) 256. In our case, we used a small input size due to memory limitations.  
The images and depth maps were inputted into the first residual block. Inside a residual block, the images were convolved with a convolutional operation. Then, the output from the first conv-layer was passed through the second convolutional layer with batch normalization. After the second convolution, the output and the original input layers were added and passed through a ReLU activation function. This completed a whole residual block (Figure 7).  
The output from the first residual block was passed through a 2D max-pooling layer which decreased its size. Our Res-UNet comprised four residual blocks, each followed by the max-pooling layer. Similar convolution and pooling were performed for proceeding layers until the bottleneck layer was reached. The bottleneck layer connected the encoder to the decoder. The decoder took the input from the last encoder layer concatenated with the attention model weights. The additive nature of attention made the model very slow, but it was capable of outstanding results. At the last decoder layer, the depth map was generated. The output from this layer was a single channel depth image.  
\begin{table}
\begin{tabular}{c c c c} \hline
**Layer No.** & **Input Layer** & **Stride and Filters** & **Output** \\ \hline \multicolumn{4}{c}{Encoder} \\ \hline Input & 128 \(\times\) 128 \(\times\) 3 & (1, 1, 64) & 128 \(\times\) 128 \(\times\) 64 \\ \hline L1 & 128 \(\times\) 128 \(\times\) 64 & (2, 2, 128) down-sampling & 64 \(\times\) 64 \(\times\) 128 \\ \hline L2 & 64 \(\times\) 64 \(\times\) 128 & (2, 2, 256) down-sampling & 32 \(\times\) 32 \(\times\) 256 \\ \hline L3 & 32 \(\times\) 32 \(\times\) 256 & (2, 2, 512) down-sampling & 16 \(\times\) 16 \(\times\) 512 \\ \hline L4 & 16 \(\times\) 16 \(\times\) 512 & (2, 2, 1024) down-sampling & 8 \(\times\) 8 \(\times\) 1024 \\ \hline L5 & \multicolumn{4}{c}{Bottleneck (8 \(\times\)8 \(\times\) 1024)} \\ \hline \multicolumn{4}{c}{Decoder} \\ \hline att \#1 & Attention (L5, L4) (16\(\times\) 16 \(\times\) 512) & 16 \(\times\) 16 \(\times\) 512 \\ \hline up \#1 & L5\_up-sampling & (2, 2, 512) & 16 \(\times\) 16 \(\times\) 512 \\ \hline L6 & concatenation (att \#1, up \#1) + Res-Conv & 16 \(\times\) 16 \(\times\) 512 \\ \hline att \#2 & Attention (L6, L3) (32 \(\times\) 32 \(\times\) 256) & 32 \(\times\) 32 \(\times\) 256 \\ \hline up \#2 & L6\_up-sampling & (2, 2, 256) & 32 \(\times\) 32 \(\times\) 256 \\ \hline L7 & concatenation (att \#2, up \#2) + Res-Conv & 32 \(\times\) 32 \(\times\) 256 \\ \hline att \#3 & Attention (L7, L2) (64 \(\times\) 64 \(\times\) 128) & 64 \(\times\) 64 \(\times\) 128 \\ \hline up \#3 & L7\_up-sampling & (2, 2, 128) & 64 \(\times\) 64 \(\times\) 128 \\ \hline L8 & concatenation (att \#3, up \#3) + Res-Conv & 64 \(\times\) 64 \(\times\) 128 \\ \hline att \#4 & Attention (L8, L1) (128 \(\times\) 128 \(\times\) 64) & 128 \(\times\) 128 \(\times\) 64 \\ \hline up \#4 & L8\_re-sampling & (2, 2, 64) & 128 \(\times\) 128 \(\times\) 64 \\ \hline L9 & concatenation (att \#4, up \#4) + Res-Conv & 128 \(\times\) 128 \(\times\) 64 \\ \hline Output & L9\_re-sampling & (1, 1, 1) & 128 \(\times\) 128 \(\times\) 1 \\ \hline \end{tabular}
\end{table}
Table 1: Summary of our proposed model.