## 1 Introduction  
With the development of display technology and the popularity of 2K screens, 4K screens and even 8K screens, people's pursuit of visual sense has become higher. Only clearer picture quality experience and fuller color enjoyment can fully satisfy people's exploration of the objective world scene, and the three-dimensional (3D) expression of the scene can better interpret the world we know. The rise of 3D film, VR and driverless technology represents great progress in the exploration of new ways of presentation and shows the infinite potential of 3D display technology [1]. Three-dimensional display is very attractive to researchers because of its features of dense viewpoints and realistic views, but the problem of how to effectively obtain high-quality dense viewpoints has not been solved, which makes the popularization of 3D display technology limited. In the early new view synthesis methods [2; 3; 4], geometric object information between multi-view images was always used. There were also some researchers who used hardware camera array for dense views acquisition, but this was limited by the high cost, the large size of the cameras and the complexity of the hardware system and so on, so the dense camera array model cannot be realized. Many classical virtual viewpoint rendering techniques [5; 6; 7] are faced with problems of requiring high-complexity models or small baselines. The multi-plane image (MPI) [8; 9] improves the perspective, but it has more restrictions on the input images and is easy to produce artifacts on the inclined surfaces. With the rapid development of computer hardware and the generation of massive available training data, deep neural networks have made great progress in the field of computer vision. In many early studies on virtual view using deep learning methods [10; 11; 12], since only the convolutional neural network (CNN) was used for simple data synthesis,the edges of objects in the synthesized images were very fuzzy. In the method combining geometric structure and deep learning [13], the input multi-view image was used to estimate the geometric structure between views. However, in many cases, we cannot obtain a large number of effective multi-view images, and the use of monocular images as the input of the network is more extensive in practical applications. The existing synthesis methods for virtual view through monocular images [14; 15] can usually only generate images of single specific viewpoints. Recently, many researchers used the layered representation of images [16; 17] to predict virtual views. They combined the depth estimation of images [18; 19] with the layered representation, and then used a generative adversarial network (GAN) for inpainting of the occluded areas in virtual views, so as to realize the synthesis of new views. In this paper, the method similar to [18; 19] is also adopted for virtual view estimation of monocular images with complex depth information. Monocular depth estimation: the CNN [20; 21; 22] has been used to obtain high-quality depth images of monocular images, which improves the accuracy of depth images. However, these methods mainly improve the learning ability by increasing the number of network layers, which limits the accuracy of the depth images obtained. Therefore, researchers began to combine CNN architecture based on monocular depth estimation with other geometric constraint information, such as the conditional random field (CRF) model [23; 24] and regression forest [25; 26; 27]. In recent work, semantic information has been used to further improve the quality of depth estimation [28; 29]. The method we adopted was to improve the depth estimation ability by using semantic segmentation information [30]. The extracted feature information was integrated by semantic segmentation-depth estimation subnets and the multi-scale feature fusion module, and the network was trained by the generable dataset of [31] so as to reduce the limit on the type range of the input images.