## 3 Methods
### Estimation of the Initial 3D Box
#### 3.1.4. Keypoint Estimation  
If we directly perform 3D parameters estimation using the fused feature, that is directly perform 3D object detection in 3D space, the search range is too large and the cost is too high. Considering that the projection point of the 3D box is relatively close to the center point of the 2D box, as shown in Figure 4, the red point is the center point of the 2D box and the green point is the projection point of the 3D box. Therefore, we estimate the projection point of the 3D box based on the center point of the 2D box; that is to say we regard the projection point as a key point for estimation and then back-project the projection point to 3D space to obtain the center point coordinates of the 3D box. The relationship between the projection point of the 3D box \(b_{pro}(x_{pro},y_{pro})\) and the center point of the 2D box \(\mathfrak{b}_{2d}(x_{2d},y_{2d})\) is as follows. \[b_{pro}(x_{pro},y_{pro})=b_{2d}(x_{2d},y_{2d})+(\triangle x,\triangle y) \tag{1}\]  
In the experiment, it was found that when estimating the offsets between the projection point of the 3D box and the center point of the 2D box, the experimental performance of carrying out twice offsets estimations better than the experimental performance of carrying out once offsets estimation. That is because we perform the offsets loss calculation twice, which increases the constraints and make the keypoint estimation more accurately. Thus, our keypoint estimation is designed according to the following formula:  
\[b_{pro}(x_{pro},y_{pro})=b_{2d}(x_{2d},y_{2d})+(\triangle x_{1},\triangle y_{1} )+(\triangle x_{2},\triangle y_{2}) \tag{2}\]  
An important part of the keypoint estimation is the generation of keypoint labels \(b_{pro}^{gt}(x_{pro}^{gt},y_{pro}^{gt})\). For each object, we project the ground truth of the center point of the 3D box to the image plane through the known camera internal parameter matrix K, and then obtain the label of the key point. The formula is as follows:  
\[z_{3d}^{gt}b_{pro}^{gt}(x_{pro}^{gt},y_{pro}^{gt})=KB_{3d}^{gt}(x_{3d}^{gt},y_{ 3d^{\prime}}^{gt},z_{3d}^{gt})\]  
\[ \begin{bmatrix}z_{3d}^{gt}\cdot x_{pro}^{gt}\\ z_{3d}^{gt}\cdot y_{pro}^{gt}\\ z_{3d}^{gt}\end{bmatrix} =K\cdot \begin{bmatrix}z_{3d}^{gt}\\ y_{3d}^{gt}\\ z_{3d}^{gt}\end{bmatrix}  \tag{3}\]  
Knowing the camera internal parameter matrix K, the projection formula for the center point of the 3D box \(B_{3d}(x,y,z)\) projected onto  
\[ \begin{bmatrix}z\cdot x_{pro}\\ z\cdot y_{pro}\\ z\end{bmatrix} =K\cdot \begin{bmatrix}x\\ y\\ z\end{bmatrix} ,K= \begin{bmatrix}f_{x}&0&c_{x}\\ 0&f_{y}&c_{y}\\ 0&0&1\end{bmatrix}  \tag{4}\]  
where \(f_{x}\) and \(f_{y}\) are the focal lengths of the x-axis and y-axis directions and \(c_{x}\) and \(c_{y}\) are the projections of the camera's optical center on the image plane. According to the projection Formula (3), when the projection point \(b_{pro}(x_{pro},y_{pro})\) is back-projected to the 3D space to obtain the center point coordinate of the 3D box, the back-projection formula is as follows:  
\[B_{3d}(x,y,z)=K^{-1}\cdot z\cdot b_{pro}(x_{pro},y_{pro})\]  
\[ \begin{bmatrix}x\\ y\\ z\end{bmatrix} =K^{-1}\cdot\mathbf{z}\cdot \begin{bmatrix}x_{pro}\\ y_{pro}\\ 1\end{bmatrix} =K^{-1}\cdot\mathbf{z}\cdot \begin{bmatrix}x_{2d}+\triangle x_{1}+ \triangle x_{2}\\ y_{2d}+\triangle y_{1}+\triangle y_{2}\\ 1\end{bmatrix}  \tag{5}\]  
It can be seen from Formula (4) that in order to obtain the center point coordinates of the 3D box, we need to estimate not only the position of the projection point but also the depth value of the center point of the 3D box. As shown in Figure 5, at the end of the keypoint estimation network, there are three fully connected layers, after which is the detection head corresponding to the 3D box's parameters.