###### Abstract  
A digital twin city maps a virtual three-dimensional (3D) city model to the geographic information system, constructs a virtual world, and integrates real sensor data to achieve the purpose of virtual-real fusion. Focusing on the accuracy problem of vision sensor registration in the virtual digital twin city scene, this study proposes a 2D-3D registration method under inverse depth and structural semantic constraints. First, perspective and inverse depth images of the virtual scene were obtained by using perspective view and inverse-depth nascent technology, and then the structural semantic features were extracted by the two-line minimal solution set method. A simultaneous matching and pose estimation method under inverse depth and structural semantic constraints was proposed to achieve the 2D-3D registration of real images and virtual scenes. The experimental results show that the proposed method can effectively optimize the initial vision sensor pose and achieve high-precision registration in the digital twin scene, and the Z-coordinate error is reduced by 45%. An application experiment of monocular image multi-object spatial positioning was designed, which proved the practicability of this method, and the influence of model data error on registration accuracy was analyzed. digital twin city; 2D-3D registration; virtual-real fusion; inverse depth; structural semantics +
Footnote †: journal: Applied Sciences  
objects. It is significantly affected by the sensor accuracy and outdoor environment, and the accuracy has difficulty meeting the requirements of 2D-3D registration in the outdoor environment. Vision-based registration technology uses the image obtained by the vision sensor to optimize the camera's 6-DOF pose and then achieves the fusion of the image and virtual scene. Because of seasonal and weather changes, in digital twin applications, sensor image and virtual scene appearances often vary significantly, which makes matching the virtual and real images difficult. Additionally, because a city scene image often contains a sky background and has a large depth range, its feature points are unevenly distributed in the vertical direction, resulting in inaccurate positioning. In addition, urban outdoor scenes have typical Manhattan-world [6] characteristics, containing rich structural semantic features. For more accurate and robust camera pose estimation, semantic features can be used as constraints. However, only a few works have applied structural features to the outdoor environment. This study focused on the accuracy problem of vision sensor registration in the virtual digital twin city scene. Regarding the characteristics of the city scene with structural semantic information, we propose a 2D-3D registration method for the real image and virtual scene under inverse depth and structural semantic constraints. First, perspective view and inverse-depth nascent technology (PDNT) is used to obtain the perspective and inverse depth images of digital twin scenes, and the plumb line, which contains structural semantic information, is extracted from the vision sensor image. Based on the general vision-based framework, we also propose a simultaneous feature matching and pose estimation method utilizing inverse depth coordinate and structural semantic constraints (MP-IDSSC) to optimize the position and orientation of the vision sensor, which effectively solves the pose error caused by the uneven distribution of feature points. Finally, we use the ray intersection method to obtain the location of any object in the real image and achieve the registration of the vision sensor. The main contributions of this study are as follows:  
* We developed a 2D-3D registration method with constraints of structural semantics and inverse depth coordinates, which effectively solves the problem of the large error caused by the uneven distribution of feature points in the vertical direction and achieves high-accuracy registration of monocular images in the digital twin scene.