# Vehicle Speed Estimation Based on 3D ConvNets and Non-Local Blocks
## 3 Model
### Non-Local Blocks  
The Non-local block expands the receptive field of the model to capture long-range dependency. We first present the mathematical expression for the non-local operation. Then, we detail how to add non-local blocks to our model. A non-local operation computes the response in one position as a weighted sum of the features in all positions. According to the non-local methods [31; 33], the generic non-local operation in neural networks is presented as:  
\[y_{i}=\frac{1}{\mathcal{C}\left(x\right)}\sum_{\forall j}f\left(x_{i},x_{j} \right)g\left(x_{j}\right). \tag{2}\]  
where \(x_{i}\) is the input signal, which could be in the form of images, sequences, and video features, while \(y_{i}\) is the output signal of position \(i\). \(f\) is a pairwise function that computes the correlation between \(x_{i}\) and all \(x_{j}\). The function \(g\) conducts a transformation on \(x_{j}\). The summation of \(f\) and \(g\) is then normalized by a factor \(\mathcal{C}\left(x\right)\). From Equation (2), we know that the non-local operation considers all position (\(\forall j\)) of inputs. In contrast, a standard 1D convolution sums up the local neighborhood weight (e.g., \(i-1\leq j\leq i+1\) \begin{table}
\begin{tabular}{c c c} \hline \hline
**Name** & **2D ResNet50** & **R3D50** & **R2+1D50** \\ \hline Conv1 & \(7\times 7,64\) & \(7\times 7\times 7,64\) & \(5\times 7\times 7,64\) \\ Pooling & \(3\times 3\) & \(3\times 3\times 3\) & \(1\times 3\times 3\) \\ Conv2\_x & \(\left[\begin{array}{c}1\times 1,64\\ 3\times 3,64\\ 1\times 1,256\end{array}\right]\times 3\) & \(\left[\begin{array}{c}1\times 1\times 1,64\\ 3\times 3\times 3,64\\ 1\times 1\times 1,256\end{array}\right]\times 3\) & \(\left[\begin{array}{c}3\times 1\times 1,64\\ 1\times 3\times 3,64\\ 1\times 1\times 1,256\end{array}\right]\times 3\) & \(\left[\begin{array}{c}3\times 1\times 1,64\\ 1\times 3\times 3,64\\ 1\times 1\times 1,256\end{array}\right]\times 3\) \\ Conv3\_x & \(\left[\begin{array}{c}1\times 1,128\\ 3\times 3,128\\ 1\times 1,512\end{array}\right]\times 4\) & \(\left[\begin{array}{c}1\times 1\times 1,128\\ 3\times 3,128\\ 1\times 1\times 1,512\end{array}\right]\times 4\) & \(\left[\begin{array}{c}3\times 1\times 1,128\\ 1\times 3\times 3,128\\ 1\times 1\times 1,512\end{array}\right]\times 1\) & \(\left[\begin{array}{c}1\times 1\times 1,128\\ 1\times 3\times 3,128\\ 1\times 1\times 1,512\end{array}\right]\times 2\) \\ Conv4\_x & \(\left[\begin{array}{c}1\times 1,256\\ 3\times 3,256\\ 1\times 1,1024\end{array}\right]\times 3\) & \(\left[\begin{array}{c}1\times 1\times 1,256\\ 1\times 3\times 3,256\\ 1\times 1\times 1,1024\end{array}\right]\times 6\) & \(\left[\begin{array}{c}3\times 1\times 1,256\\ 1\times 3\times 3,256\\ 1\times 1\times 1,1024\end{array}\right]\times 3\) \\ Conv5\_x & \(\left[\begin{array}{c}1\times 1,512\\ 3\times 3,512\\ 1\times 1,2048\end{array}\right]\times 3\) & \(\left[\begin{array}{c}1\times 1\times 1,512\\ 3\times 3,512\\ 1\times 1\times 1,2048\end{array}\right]\times 3\) & \(\left[\begin{array}{c}1\times 1\times 1,512\\ 1\times 3\times 3,512\\ 1\times 1\times 1,2048\end{array}\right]\times 3\) & \(\left[\begin{array}{c}1\times 1\times 1,512\\ 1\times 3\times 3,512\\ 1\times 1\times 1,2048\end{array}\right]\times 1\times 1,2048\) \\ Avgpooling & \(7\times 7\) & \(1\times 7\times 7\) & \(1\times 7\times 7\) \\ \hline \hline \end{tabular}
\end{table} 
Table 1: 2D ResNet50 vs R3D50 vs R(2+1)D50. Figure 2: 3D Conv VS (2+1)D Conv. in the case of kernel size equal to 3), and the recurrent operation at time \(i\) is often based only on the current and the preceding time steps (e.g., \(j=i\) or \(i-1\)). There are several candidates for the pairwise function \(f\), such as the Dot product, the Gaussian, and the Embedded Gaussian. According to the experiments in [33], the non-local models are not sensitive to the choice of \(f\), so we randomly choose the Embedded Gaussian function for \(f\).