# NTPP-MVSNet: Multi-View Stereo Network Based on Neighboring Tangent Plane Propagation
## 3 Methods
### Depth Hypothesis  
The brute force enumeration approach demonstrated in Figure 3 is typically used by the traditional deep sampling method, which establishes a pre-determined range and  
Figure 1: Overview of our proposed method. Multi-scale feature information is acquired by employing the target view image as well as the source view images as the input into the FPN network. The MVS estimation model undergoes three stages of iterative optimization, leading to the ultimate acquisition of high-resolution depth map and normal map information. Figure 2: In the proposed network model, initial proceedings involve performing a differentiable homography transformation on the features of various perspectives, simultaneous to the acquisition of several depth hypotheses via depth sampling. After mapping to the target view, the cost volume is calculated and further regularization is carried out via 3D-UNet. Eventually, the prediction of the depth map and normal information can be accomplished through channel-specific cost volume. conducts uniform sampling accordingly. Whilst dense sampling density will facilitate the acquisition of depth values close to the assumed true value, this technique is heavily flawed owing to its inefficiency. Low-density depth sampling renders the outcome susceptible to missing the nearby true value points, whereas the high-density alternative carries the potential for memory overload. As demonstrated in Figure 4, depth sampling confers to each pixel various depth hypotheses with high reliability and leverages neighborhood results acquired at lower scales [17; 18; 19; 28]. With the same or fewer depth sampling numbers, disparities between the depth sampling results and actual depth values are more minute. Nevertheless, this method's establishment requires an ideal premise wherein the surface is imaged in parallel to the camera imaging plane. Significant tilting of the surface from the camera imaging plane or even a perpendicular approach will result in significant errors relative to the current point's depth true value. To address the aforementioned issues, we proposed a novel depth sampling method that operates on neighborhood tangent plane information, encapsulating the assumption that the 3D representation of the current pixel point is precisely located on the tangent plane of the neighborhood. As presented in Figure 5, homography transformation was employed to deduct the normal vector and spatial position of the current pixel point based on known coarse depth values and neighborhood normals, consequently modeling the neighborhood's tangent plane. The incentive step entails estimating normal vector information of the image, ultimately leading to the acquisition of relatively continuous neighborhood locations of the surface and resulting in the computation of depth assumptions through homography transformation. Figure 3: Uniform sampling: This approach’s efficacy is comparatively poor as it demands a substantial density of sampling to procure dependable depth hypotheses, which in turn incurs a time-consuming process. \(\{Z_{i}|i=1,...,4\}\) represent four hypothetical depth values after uniform sampling, and the error between the depth sampling value \(Z_{2}\) closest to point A and the true depth value of point A is \(E_{2}\). Therefore, the error between assumed and actual values is the largest underlying this methodology. Figure 4: Sampling based on neighborhood depth information propagation: Enhancement of sampling efficiency is feasible to a certain extent. Ideally, consistency between the current point and the depth value of neighboring points should be achieved, signifying a parallel formation of the local section to the camera plane.