# Birds Eye View Look-Up Table Estimation with Semantic Segmentation
## 4 Proposed Deep Learning Network  
The network (Figure 2) consists of an encoder, a decoder, an LUT generator, and a pose regressor. As loss functions, segmentation loss in the encoder, LUT loss in the LUT generator, and pose loss in the pose regressor were used. The input of the whole network is a three-channel RGB image, and the output is the predicted pose and LUT of the BEV. The predicted pose is output through the encoder and pose regressor, and the LUT is output through the encoder, decoder, and LUT generator. The pose regressor is attached in the form of an add-on, and by connecting it, the overall direction of the network is given and the performance is improved. The predicted pose helps the user to intuitively monitor the learning progress during the learning process. The image output in the LUT reduces the post-processing time because it enables the BEV to be produced immediately after performing only the memory copy process and simple interpolation operation without additional operation. In Tables 1-5, C denotes the customizable channel, and H and W denote the height and width of the input image (or input feature map), respectively. \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & & & **(a)** & \\ \hline
**Name** & **In Channels** & **Out Channels** & **Layer** & **Remarks** \\ \hline EL00 & 3 & C & \multicolumn{2}{c}{Semantic segmentation} \\ \hline EL01 & C & 16 & \multicolumn{2}{c}{2D convolution (kernel: 3; stride: 1; padding: 1)} \\ \hline EL02 & 3 & 16 & \multicolumn{2}{c}{2D convolution (kernel: 3; stride: 1; padding: 1)} \\ \hline EL03 & 16 & 64 & \multicolumn{2}{c}{Encoder layer} \\ \hline EL04 & 64 & 256 & \multicolumn{2}{c}{Compress block} \\ \hline EL05 & 256 & 1024 & \multicolumn{2}{c}{(b)} \\ \hline
**Name** & **Channel** & **Height** & **Width** & **Remarks** \\ \hline EF00 & 3 & H & W & \\ \hline EF01 & C & H & W & \\ \hline EF02 & 16 & H & W & \\ \hline EF03 & 64 & H/2 & W/2 & Encoder feature \\ \hline EF05 & 256 & H/4 & W/4 & \\ \hline EF06 & 1024 & H/8 & W/8 & \\ \hline \hline \end{tabular}
\end{table} 
Table 1: (a). Encoder layer detail; (b). Encoder feature map detail. Figure 2: Entire network structure. \begin{table}
\begin{tabular}{c c c c c} \hline \hline MF00 & & & & \\ \hline MF01 & 4096 & H/16 & W/16 & \\ \hline MF02 & 64 & H/2 & W/2 & Middle feature \\ \hline MF03 & 256 & H/4 & W/4 & \\ \hline MF04 & 1024 & H/8 & W/8 & \\ \hline MF05 & 4096 & H/16 & W/16 & \\ \hline \hline \end{tabular}
\end{table} 
Table 1: _Cont._ \begin{table}
\begin{tabular}{c c c c} \hline \hline \multicolumn{4}{c}{**(a)**} \\ \hline
**Name** & **In Channels** & **Out Channels** & **Layer** & **Remarks** \\ \hline CL00 & & & & \\ \hline CL01 & & & & \\ \hline CL02 & 4C & 8C & Concatenate & Compress layer \\ \hline CL03 & 8C & 4C & 2D convolution (kernel: 3; stride: 1; padding: 1) & \\ \hline \hline
**Name** & **Channel** & **Height** & **Width** & **Remarks** \\ \hline CF00 & & & & \\ \hline CF01 & & & & \\ \hline CF02 & & & & \\ \hline CF03 & 4C & H/2 & W/2 & \\ \hline CF04 & 8C & H/2 & W/2 & \\ \hline CF05 & & & & \\ \hline CF06 & 4C & H/2 & W/2 & \\ \hline CF07 & & & & \\ \hline \hline
**Name** & **In Channels** & **Out Channels** & **Layer** & **Remarks** \\ \hline RL00 & C & 2C & 2D convolution (kernel: 3; stride: 2; padding: 1) & \\ \hline RL01 & 2C & 4C & 2D convolution (kernel: 3; stride: 1; padding: 1) & \\ \hline RL02 & C & 4C & 2D convolution (kernel: 3; stride: 2; padding: 1) & Residual layer \\ \hline RL03 & 4C & 4C & Element Add & \\ \hline \hline
**Name** & **Channel** & **Height** & **Width** & **Remarks** \\ \hline RF00 & C & H & W & \\ \hline RF01 & 2C & H/2 & W/2 & \\ \hline RF02 & 4C & H/2 & W/2 & Residual feature \\ \hline RF03 & 4C & H/2 & W/2 & \\ \hline RF04 & 4C & H/2 & W/2 & \\ \hline \hline \end{tabular}
\end{table} 
Table 2: **(a). ** Compress Block Layer Detail; **(b). ** Compress block feature map detail. **(c). Custom residual block layer detail. **(d). Custom residual block feature map detail.