# NTPP-MVSNet: Multi-View Stereo Network Based on Neighboring Tangent Plane Propagation
## 1 Introduction  
Multi-view stereo was initiated by [PERSON] [1] in 1979 as a segment of a theoretical framework for computer vision. According to [PERSON], humans interpret objects by reconstructing a three-dimensional representation of their contours, which can be accomplished through computational approaches. A number of mathematical models have been proposed over the years to tackle the problem, for instance, Patchmatch Stereo [2] and Colmap [3]. Nevertheless, due to insufficient computational resources, the accuracy and completeness of these models are questionable. With the advent of convolutional neural networks in computer vision [4; 5; 6; 7], the domain of deep learning-based multi-view stereo has witnessed significant strides [8; 9; 10; 11]. Deep learning-based multi-view stereo algorithms encompass several modular components, which include feature extraction, depth sampling, cost volume aggregation [12], and depth map optimization regression. However, it must be noted that accurate depth hypotheses serve as the foundation for regressing high-precision depth maps. Traditional depth sampling methodologies [13] often resort to dense sampling within a fixed range to acquire sampling points that are proximal to the depth truth values, employing a brute-force enumeration methodology that is not only time-consuming but also computationally inefficient. We combined the normal vector estimation of the scene and used the homography feature of the plane to propose a more accurate depth sampling method, which can improve the accuracy of scene estimation by applying it to the multi-view stereo matching network. A small number of researchers have proposed some algorithm modules for high-efficiency deep sampling. One such solution involves processing the image's depth information at multiple scales to progressively narrow the depth range of each pixel basedon low-scale depth prediction maps [14; 15; 16]. While this technique can effectively reduce the computational cost of dense sampling in high-resolution images by minimizing the depth hypothesis range, it remains an enumeration method that may be computationally demanding. An alternative group of researchers proposed utilizing neighborhood information in the image to provide depth values for three-dimensional points in the image's four-neighborhood/eight-neighborhood [17; 18; 19]. Since the pixel point distribution in the depth map can be treated as a Markov random field, a correlation exists between adjacent points in the image. Therefore, they upscale the low-scale depth prediction outcomes and employ the neighborhood-depth value to generate reliable depth assumptions for each pixel [16; 20]. However, the success of this depth sampling technique hinges on the assumption of equal depth values of each pixel in the image, implying that the current object's surface needs to be parallel to the imaging plane. Nevertheless, real-world scenes may not fulfill this condition and this sampling method may consequently produce poorer depth assumptions on heavily inclined surfaces. The above methods either still rely on uniform sampling in a fixed range under different resolution images, which has low depth estimation efficiency, or there is a certain deviation between the premise of depth sampling and the actual scene, which is prone to producing wrong depth assumptions. In this paper, we present a multi-view stereo network that leverages neighborhood normal vector information to obtain efficient and robust depth sampling values under realistic conditions, while concurrently enhancing the computational efficiency and depth map regression accuracy of the network model. Our proposed algorithmic framework utilizes the neighborhood geometric structure of the current scene as a starting point for the depth sampling module. Through the employment of a homography mapping between the neighborhood's tangent plane information and the image plane, the depth hypothesis of the current pixel in the neighborhood's tangent plane assumption can be calculated.