## 3 Methodology and Materials
### Our Multimodal Deep Learning Structure
\begin{table}
\begin{tabular}{l l l l l} \hline \hline  & **Name** & **Type** & **Activation** & **Learnables** \\ \hline \multirow{2}{*}{1} & Scene Images & Image Input & 200 \(\times\) 200 \(\times\) 3 & - \\  & 200 \(\times\) 200 \(\times\) 3 & & & \\ \hline \multirow{2}{*}{2} & Conv\_1 & \multirow{2}{*}{Convolution} & 200 \(\times\) 200 \(\times\) 128 & Weights 3 \(\times\) 3 \(\times\) 3 \(\times\) 128 \\  & 1283 \(\times\) 3 \(\times\) 3 with stride [1 1] and padding ’same’ & \multirow{2}{*}{Max Pooling} & 200 \(\times\) 200 \(\times\) 128 & - \\ \hline \multirow{2}{*}{3} & Maxpool\_1 & \multirow{2}{*}{ReLU} & 200 \(\times\) 200 \(\times\) 128 & - \\  & 2 \(\times\) 2 with padding ’same’ & & & \\ \hline \multirow{2}{*}{4} & Relu\_1 & \multirow{2}{*}{ReLU} & 200 \(\times\) 200 \(\times\) 128 & - \\  & ReLU & & & \\ \hline \multirow{2}{*}{5} & Conv\_3 & \multirow{2}{*}{Convolution} & 200 \(\times\) 200 \(\times\) 64 & Weights 3 \(\times\) 3 \(\times\) 128 \(\times\) 64 \\  &'same’ & & & Bias 1 \(\times\) 1\(\times\) 64 \\ \hline \multirow{2}{*}{6} & Maxpool\_3 & \multirow{2}{*}{Max Pooling} & 200 \(\times\) 200 \(\times\) 64 & - \\  & 2 \(\times\) 2 with padding ’same’ & & & \\ \hline \multirow{2}{*}{7} & Relu\_3 & \multirow{2}{*}{ReLU} & 200 \(\times\) 200 \(\times\) 64 & - \\  & ReLU & & & \\ \hline \multirow{2}{*}{8} & Conv\_5 & \multirow{2}{*}{Convolution} & 200 \(\times\) 200 \(\times\) 32 & Weights 3 \(\times\) 3 \(\times\) 64 \(\times\) 32 \\  & 323 \(\times\) 3 \(\times\) 64 with stride [1 1] and padding ’same’ & \multirow{2}{*}{Convolution} & 200 \(\times\) 200 \(\times\) 32 & Bias 1 \(\times\) 1 \(\times\) 32 \\ \hline \multirow{2}{*}{9} & Maxpool\_5 & \multirow{2}{*}{Max Pooling} & 200 \(\times\) 200 \(\times\) 32 & - \\  & 2 \(\times\) 2 with padding ’same’ & & & \\ \hline \multirow{2}{*}{10} & Relu\_5 & \multirow{2}{*}{ReLU} & 200 \(\times\) 200 \(\times\) 32 & - \\  & ReLU & & & \\ \hline \multirow{2}{*}{11} & Fc\_1 & \multirow{2}{*}{Fully Connected} & 1 \(\times\) 1 \(\times\) 1024 & Weights 1024 \(\times\) 1,280,000 \\  & 1024 & & & Bias 1024 \(\times\) 1 \\ \hline \multirow{2}{*}{12} & Conv\_2 & \multirow{2}{*}{Convolution} & 200 \(\times\) 200 \(\times\) 128 & Weights 3 \(\times\) 3 \(\times\) 3 \(\times\) 128 \\  & 1283 \(\times\) 3\(\times\)3 with stride [1 1] and padding ’same’ & \multirow{2}{*}{Convolution} & 200 \(\times\) 200 \(\times\) 128 & Bias 1 \(\times\) 1 \(\times\) 128 \\ \hline \multirow{2}{*}{13} & Maxpool\_2 & \multirow{2}{*}{Max Pooling} & 200 \(\times\) 200 \(\times\) 128 & - \\  & 2 \(\times\) 2 with padding ’same’ & & & \\ \hline \multirow{2}{*}{14} & Relu\_2 & \multirow{2}{*}{ReLU} & 200 \(\times\) 200 \(\times\) 128 & - \\  & ReLU & & & \\ \hline \multirow{2}{*}{15} & Conv\_4 & \multirow{2}{*}{Convolution} & 200 \(\times\) 200 \(\times\) 64 & Weights 3 \(\times\) 3 \(\times\) 128 \(\times\) 64 \\  & 643 \(\times\) 3 \(\times\) 128 with stride [1 1] and padding ’same’ & \multirow{2}{*}{Convolution} & 200 \(\times\) 200 \(\times\) 64 & Bias 1 \(\times\) 1 \(\times\) 64 \\ \hline \multirow{2}{*}{16} & Maxpool\_4 & \multirow{2}{*}{Max Pooling} & 200 \(\times\) 200 \(\times\) 64 & - \\  & 2 \(\times\) 2 with padding ’same’ & & & \\ \hline \multirow{2}{*}{17} & Relu\_4 & \multirow{2}{*}{ReLU} & 200 \(\times\) 200 \(\times\) 64 & - \\  & ReLU & & & \\ \hline \multirow{2}{*}{18} & Conv\_6 & \multirow{2}{*}{Convolution} & 200 \(\times\) 200 \(\times\) 32 & Weights 3 \(\times\) 3 \(\times\) 64 \(\times\) 32 \\  & 323 \(\times\) 3 \(\times\) 64 with stride [1 1] and padding ’same’ & \multirow{2}{*}{Convolution} & 200 \(\times\) 200 \(\times\) 32 & Bias 1 \(\times\) 1 \(\times\) 32 \\ \hline \multirow{2}{*}{19} & Maxpool\_6 & \multirow{2}{*}{Max Pooling} & 200 \(\times\) 200 \(\times\) 32 & - \\  & 2 \(\times\) 2 with padding ’same’ & & & \\ \hline \multirow{2}{*}{20} & Relu\_6 & \multirow{2}{*}{ReLU} & 200 \(\times\) 200 \(\times\) 32 & - \\  & ReLU & & & \\ \hline \multirow{2}{*}{21} & Fc\_2 & \multirow{2}{*}{Fully Connected} & 1 \(\times\) 1 \(\times\) 1024 & Weights 1024 \(\times\) 1,280,000 \\  & 1024 & & & Bias 1024 \(\times\) 1 \\ \hline \multirow{2}{*}{22} & Addition & \multirow{2}{*}{Addition} & 1 \(\times\) 1 \(\times\) 1024 & - \\ \cline{2-2}  & Eliminate-wise addition of 2 input & & & \\ \hline \hline \end{tabular}
\end{table}