## 1 Introduction  
Three-dimensional object detection is an essential and important environmental perception task in autonomous driving which can provide the precise location of the surroundings within a certain distance for a moving car. In recent years, 3D object detection algorithms based on lidar [1; 2; 3] or stereo cameras [4; 5; 6] have achieved an excellent detection performance by virtue of their direct or indirect depth information, however, due to the inherent high cost or low robustness to adverse weather conditions of lidar-based detectors and cumbersome online calibration problems of stereo cameras, the application and development of these two detection methods have been limited.  
On the contrary, the research of monocular 3D object detection algorithms [7; 8; 9; 10; 11; 12; 13] has become a promising research direction by virtue of its advantages of a low cost and easy operation. Monocular 3D object detection means to classify and locate each object within a certain distance in a surrounding environment by only a single camera; the goal of monocular 3D object detection is to estimate a minimal 3D bounding box tightly wrapping the object. In KITTI [14], the 3D bounding box can be parameterized as follows: (i) the object's position (x, y, z), meaning the position of the center point of the 3D box in the camera coordinate system (unit: meters); (ii) the object's dimensions (l, w, h), representing the length, width, and height of the 3D bounding box (in meters); and (iii) the object's angle, which is the orientation angle of the 3D box. The biggest challenge in improving the performance of a 3D object detection algorithm is how to accurately estimate the object's position (x, y, z), among which the estimation of the depth value is the most important. Therefore, how to obtain accurate depth information is an import role of monocular 3D object detection. Previous methods [7; 15; 16] perform 3D object detection by virtue of geometric constraints between 2D images and 3D objects, but their experimental performances are worse on account of the lack of direct or indirect depth information. For example, Deep3D box [16] performs 3D object detection by right of strong geometric constraint information from the 2D bounding box, and the estimated 3D bounding box closely fits the 2D bounding box. The performance of such algorithms largely depends on the precision of 2D object detection because of the strong dependency between 3D object detection and 2D detection. Other methods [17; 18; 19] generate pseudo-lidar by introducing extra depth information and utilize existing mature lidar-based detection framework to perform 3D object detection, whereas such algorithms are computationally inefficient because it has to deal with a huge number of pseudo-lidar points.  
The paper proposes a monocular 3D object detection algorithm based on pseudo-multimodal information extraction and keypoint estimation. First, we utilize mature 2D detection framework to accomplish 2D object detection. Second, we fuse the two different modalities: the 2D region of interest and the region pseudo-lidar to perform the keypoint-based initial 3D object detection. Finally, we utilize the bird's-eye view features generated by the pseudo-lidar to refine the initial 3D box, consequently achieving the purpose of extracting three different modal information of the original image for 3D object detection.  
In summary, our main contributions are as follows:  
1. We propose a monocular 3D object detection algorithm based on pseudo-multimodal information extraction and keypoint estimation.  
2. It is the first time that multi-modal information has been extracted from an original image for 3D object detection, which makes the extracted information more comprehensive and richer.  
3. Our method achieves a state-of-the-art experimental performance on the KITTI dataset.