Original: CURVATURE-GUIDED DYNAMIC SCALE NETWORKS FOR MULTI-VIEW STEREO
Revised: CURVATURE-GUIDED DYNAMIC SCALE NETWORKS FOR MULTI-VIEW STEREO

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Multi-view stereo (MVS) is a crucial task for precise 3D reconstruction.
Revised: Multi-view stereo (MVS) is a crucial task for precise 3D reconstruction.

Original: Most recent studies tried to improve the performance of matching cost volume in MVS by introducing a skilled design to cost formulation or cost  regularization.
Revised: Most recent studies tried to improve the performance of matching cost volume in MVS by designing aggregrated 3D cost volumes and their regularization.

Original: This paper focuses on learning a robust feature extraction network to enhance the per formance of matching costs without heavy computation in the other steps.
Revised: This paper focuses on learning a robust feature extraction network to enhance the performance of matching costs without heavy computation in the other steps.

Original: In par - ticular, we present a dynamic scale feature extraction network, namely, CDSFNet
Revised: In particular, we present a dynamic scale feature extraction network, namely, CDSFNet.

Original: It is composed of multiple novel convolution layers each of which can select a proper patch scale for each pixel guided by the normal curvature of the image surface.
Revised: It is composed of multiple novel convolution layers, each of which can select a proper patch scale for each pixel guided by the normal curvature of the image surface.

Original: As a result CDFSNet can estimate the optimal patch scales to learn discriminative features for accurate matching computation between reference and source images.
Revised: As a result, CDFSNet can estimate the optimal patch scales to learn discriminative features for accurate matching computation between reference and source images.

Original: By combining the robust extracted features with an appropriate cost formulation strategy, our ﬁnal MVS architecture can estimate depth maps more precisely.
Revised: By combining the robust extracted features with an appropriate cost formulation strategy, our resulting MVS architecture can estimate depth maps more precisely.

Original: Extensive experiments showed that the proposed method outperforms other state-of-the-art methods on complex outdoor scenes.
Revised: Extensive experiments showed that the proposed method outperforms other state-of-the-art methods on complex outdoor scenes.

Original: It signiﬁcantly improves the completeness of reconstructed models.
Revised: It signiﬁcantly improves the completeness of reconstructed models.

Original: Moreover, the method can process the high resolution with faster run-time and lower memory than other MVS methods.
Revised: As a result, the method can process higher resolution inputs within faster run-time and lower memory than other MVS methods.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: A challenging problem in multi-view stereo (MVS) is accurately estimating dense correspondences across a collection of high-resolution images.
Revised: A challenging problem in multi-view stereo (MVS) is accurately estimating dense correspondences across a collection of high-resolution images.

Original: Many MVS studies have tried to solve ill-posed
Revised: Many MVS studies have tried to solve ill-posed

Original: MVS problems such as matching ambiguity and high computational complexity.
Revised: MVS problems such as matching ambiguity and high computational complexity.

Original: In such problems, learning-based MVS methods (Chen et al., 2019; Luo et al., 2019; Chen et al., 2020a; Xu & Tao, 2020a; Xue et al., 2019) usually outperform traditional methods (Galliani et al., 2015; Schonberger & Frahm, 2016).
Revised: In such problems, learning-based MVS methods (Chen et al., 2019; Luo et al., 2019; Chen et al., 2020a; Xu & Tao, 2020a; Xue et al., 2019) usually outperform traditional methods (Galliani et al., 2015; Schonberger & Frahm, 2016).

Original: The learning-based methods are generally composed of three steps, including feature extraction, cost volume formulation, and cost volume regularization.
Revised: The learning-based methods are generally composed of three steps, including feature extraction, cost volume formulation, and cost volume regularization.

Original: Most studies made an effort to improve the performance of cost volume formulation (Luo et al., 2019; Zhang et al., 2020;
Revised: Most studies made an effort to improve the performance of cost volume formulation (Luo et al., 2019; Zhang et al., 2020;

Original: Xu & Tao, 2020b; Yi et al., 2020) or cost volume regularization (Luo et al., 2019; 2020; Wang et al., 2021).
Revised: Xu & Tao, 2020c; Yi et al., 2020) or cost volume regularization (Luo et al., 2019; 2020; Wang et al., 2021).

Original: The visibility information (Zhang et al., 2020; Xu & Tao, 2020b) or attention techniques (Yi et al., 2020; Luo et al., 2020) are considered for cost aggregation and feature matching, respectively.
Revised: The visibility information (Zhang et al., 2020; Xu & Tao, 2020c) or attention techniques (Yi et al., 2020; Luo et al., 2020) are considered for cost aggregation and feature matching, respectively.

Original: Furthermore, some studies employed a hybrid 3D UNet structure (Luo et al., 2019; Sormann et al., 2020) for cost volume regularization.
Revised: Furthermore, some studies employed a hybrid 3D UNet structure (Luo et al., 2019; Sormann et al., 2020) for cost volume regularization.

Original: While these approaches signiﬁcantly improve MVS performances, their feature extraction networks have some drawbacks that degrade the quality of 3D reconstruction.
Revised: While these approaches signiﬁcantly improve MVS performances, their feature extraction networks have some drawbacks that degrade the quality of 3D reconstruction.

Original: First, they observed a restricted size of receptive ﬁelds from the ﬁxed size of convolutional kernels.
Revised: First, they observed a restricted size of receptive ﬁelds from the ﬁxed size of convolutional kernels.

Original: This leads to difﬁculty in learning a robust pixel-level representation when an object’s scale varies extremely in images.
Revised: This leads to difﬁculty in learning a robust pixel-level representation when an object’s scale varies extremely in images.

Original: As a result, the extracted features cause the low quality of matching costs, especially when the difference of camera poses between reference and source images is large.
Revised: As a result, the extracted features cause the low quality of matching costs, especially when the difference of camera poses between reference and source images is large.

Original: Second, MVS networks are often trained on low-resolution images because of limited memory and restricted computation time.
Revised: Second, MVS networks are often trained on low-resolution images because of limited memory and restricted computation time.

Original: Therefore, the existing feature extraction methods utilizing ﬁxed-scale feature representation on deep networks could not generalize to high-resolution images in prediction; the MVS performance is downgraded on a high-resolution image set.
Revised: Therefore, the existing feature extraction methods utilizing ﬁxed-scale feature representation on deep networks could not generalize to high-resolution images in prediction; the MVS performance is downgraded on a high-resolution image set.

Original: To address these issues, we propose a new feature extraction network that can be adapted to various object scales and image resolutions.
Revised: To address these issues, we propose a new feature extraction network that can be adapted to various object scales and image resolutions.

Original: We refer the proposed network as curvature-guided dynamic scale feature network (CDSFNet).
Revised: We refer the proposed network as curvature-guided dynamic scale feature network (CDSFNet).

Original: The proposed network is composed of novel convolution layers, curvature-guided dynamic scale convolution (CDSConv), which select a suitable patch scale for each
Revised: The proposed network is composed of novel convolution layers, curvature-guided dynamic scale convolution (CDSConv), which select a suitable patch scale for each

Original: [Figure]
Revised: [Figure]

Original: pixel to learn robust representation.
Revised: pixel to learn robust representation.

Original: This selection is achieved by computing the normal curvature of image surface at several candidate scales and analyzing the computed outputs via a classiﬁcation network.
Revised: This selection is achieved by computing the normal curvature of image surface at several candidate scales and analyzing the computed outputs via a classiﬁcation network.

Original: The pixel-level patch scales estimated from proposed network are dynamic with respect to textures, scale of objects, and epipolar geometry.
Revised: The pixel-level patch scales estimated from proposed network are dynamic with respect to textures, scale of objects, and epipolar geometry.

Original: Therefore, it trains more discriminative features than existing networks (Gu et al., 2020; Cheng et al., 2020; Yang et al., 2020; Zhang et al., 2020;
Revised: Therefore, it trains more discriminative features than existing networks (Gu et al., 2020; Cheng et al., 2020; Yang et al., 2020; Zhang et al., 2020;

Original: Luo et al., 2020) for accurate matching cost computation.
Revised: Luo et al., 2020) for accurate matching cost computation.

Original: Fig.
Revised: Fig.

Original: 1a illustrates the dynamic pixel-level patch scales estimated from CDSConv where the window size is ranged from 7 × 7 to 17 × 17 .
Revised: 1a illustrates the dynamic pixel-level patch scales estimated from CDSConv where the window size is ranged from 7 × 7 to 17 × 17 .

Original: As shown in the ﬁgure, CDSConv predicts adaptive patch scales where the scale is small in thin structure or texture-rich regions and large in the texture-less regions.
Revised: As shown in the ﬁgure, CDSConv predicts adaptive patch scales where the scale is small in thin structure or texture-rich regions and large in the texture-less regions.

Original: Moreover, the scale is dynamic according to image resolution.
Revised: Moreover, the scale is dynamic according to image resolution.

Original: CDSConv generally produces small scale values in a low-resolution image and large scale values in a high-resolution image.
Revised: CDSConv generally produces small scale values in a low-resolution image and large scale values in a high-resolution image.

Original: We also formulate a new MVS framework, CDS-MVSNet, which aims at improving the quality of
Revised: We also formulate a new MVS framework, CDS-MVSNet, which aims at improving the quality of

Original: MVS reconstruction while decreasing computation time and memory consumption.
Revised: MVS reconstruction while decreasing computation time and memory consumption.

Original: Similar to previous works (Gu et al., 2020; Cheng et al., 2020; Yang et al., 2020; Wang et al., 2021), CDS-MVSNet is composed of a cascade network structure to estimate high-resolution depth maps in a coarse-toﬁne manner.
Revised: Similar to previous works (Gu et al., 2020; Cheng et al., 2020; Yang et al., 2020; Wang et al., 2021), CDS-MVSNet is composed of a cascade network structure to estimate high-resolution depth maps in a coarse-toﬁne manner.

Original: For each cascade stage, it formulates a 3D cost volume based on the output features of
Revised: For each cascade stage, it formulates a 3D cost volume based on the output features of

Original: CDSFNet, which reduces the matching ambiguity by using proper pixel’s scale.
Revised: CDSFNet, which reduces the matching ambiguity by using proper pixel’s scale.

Original: For instance, consider a 3D point on the scene that needs to be reconstructed, CDSFNet can capture the same context information in both reference and source views for this point to extract the matchable features.
Revised: For instance, consider a 3D point on the scene that needs to be reconstructed, CDSFNet can capture the same context information in both reference and source views for this point to extract the matchable features.

Original: Furthermore, CDS-MVSNet applies visibility-based cost aggregation to improve the performance of stereo matching.
Revised: Furthermore, CDS-MVSNet applies visibility-based cost aggregation to improve the performance of stereo matching.

Original: The pixel-wise visibility is estimated from the normal curvature information that encodes the matching ability of extracted features implicitly.
Revised: The pixel-wise visibility is estimated from the normal curvature information that encodes the matching ability of extracted features implicitly.

Original: Therefore, CDS-MVSNet effectively performs accurate stereo matching and cost aggregation even on low-resolution image patches.
Revised: Therefore, CDS-MVSNet effectively performs accurate stereo matching and cost aggregation even on low-resolution image patches.

Original: Finally, the proposed network can generate high quality 3D models only by processing half-resolution images.
Revised: Finally, the proposed network can generate high quality 3D models only by processing half-resolution images.

Original: This approach can signiﬁcantly reduce the computation time and memory consumption of
Revised: This approach can signiﬁcantly reduce the computation time and memory consumption of

Original: MVS reconstruction.
Revised: MVS reconstruction.

Original: The contribution of this paper is summarized as follows
Revised: The contribution of this paper is summarized as follows

Original: : • We present the CDSConv that learns dynamic scale features guided by the normal curvature of image surface.
Revised: : • We present the CDSConv that learns dynamic scale features guided by the normal curvature of image surface.

Original: This operation is implemented by approximating surface normal curvatures in several candidate scales and choosing a proper scale via a lightweight classiﬁcation network.
Revised: This operation is implemented by approximating surface normal curvatures in several candidate scales and choosing a proper scale via a lightweight classiﬁcation network.

Original: • We propose a new feature extraction network, CDSFNet, which is composed of multiple CDSConv layers for learning robust representation at pixel level.
Revised: • We propose a new feature extraction network, CDSFNet, which is composed of multiple CDSConv layers for learning robust representation at pixel level.

Original: CDFSNet estimates the optimal pixel’s scale to learn features.
Revised: CDFSNet estimates the optimal pixel’s scale to learn features.

Original: The scale is selected adaptively with respect to structures, textures, and epipolar constraints.
Revised: The scale is selected adaptively with respect to structures, textures, and epipolar constraints.

Original: • We present CDS-MVSNet for MVS depth estimation.
Revised: • We present CDS-MVSNet for MVS depth estimation.

Original: CDS-MVSNet performs accurate stereo matching and cost aggregation by handling the ambiguity and visibility in image matching process.
Revised: CDS-MVSNet performs accurate stereo matching and cost aggregation by handling the ambiguity and visibility in image matching process.

Original: It also signiﬁcantly reduces the run-time and memory consumption by processing the half-resolution images while maintaining the reconstruction quality.
Revised: It also signiﬁcantly reduces the run-time and memory consumption by processing the half-resolution images while maintaining the reconstruction quality.

Original: • We verify the effectiveness of our method on two benchmark datasets; DTU (Aanæs et al., 2016) and Tanks & Temples (Knapitsch et al., 2017).
Revised: • We verify the effectiveness of our method on two benchmark datasets; DTU (Aanæs et al., 2016) and Tanks & Temples (Knapitsch et al., 2017).

Original: The results demonstrate that the proposed feature learning method boosts the performance of MVS reconstruction.
Revised: The results demonstrate that the proposed feature learning method boosts the performance of MVS reconstruction.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Multi-view Stereo.
Revised: Multi-view Stereo.

Original: Traditional MVS studies can be divided into three categories: volumetric, point cloud-based, and depth map-based methods (Furukawa & Hern´andez, 2015).
Revised: Traditional MVS studies can be divided into three categories: volumetric, point cloud-based, and depth map-based methods (Furukawa & Hern´andez, 2015).

Original: Comparatively, depth map-based methods are more concise and ﬂexible, they estimate depth maps for all reference images and then perform depth fusion to achieving a 3D model (Galliani et al., 2015; Sch¨onberger et al., 2016; Xu & Tao, 2019).
Revised: Comparatively, depth map-based methods are more concise and ﬂexible, they estimate depth maps for all reference images and then perform depth fusion to achieving a 3D model (Galliani et al., 2015; Sch¨onberger et al., 2016; Xu & Tao, 2019).

Original: The popular MVS methods in this category are usually based on a PatchMatch Stereo algorithm (Barnes et al., 2009).
Revised: The popular MVS methods in this category are usually based on a PatchMatch Stereo algorithm (Barnes et al., 2009).

Original: Recently, many studies have applied a learning-based approach, which has achieved signiﬁcant performance improvements over traditional methods.
Revised: Recently, many studies have applied a learning-based approach, which has achieved signiﬁcant performance improvements over traditional methods.

Original: Several volumetric methods (Kar et al., 2017;
Revised: Several volumetric methods (Kar et al., 2017;

Original: Ji et al., 2017) employed 3D CNNs to explicitly predict global 3D surfaces.
Revised: Ji et al., 2017) employed 3D CNNs to explicitly predict global 3D surfaces.

Original: Yao et al.
Revised: Yao et al.

Original: (2018) proposed a depth-map-based method, MVSNet, which is an end-to-end trainable architecture with three steps.
Revised: (2018) proposed a depth-map-based method, MVSNet, which is an end-to-end trainable architecture with three steps.

Original: However, these methods still confronted the matching ambiguity problem and could not process high-resolution images due to limited memory.
Revised: However, these methods still confronted the matching ambiguity problem and could not process high-resolution images due to limited memory.

Original: Many methods based on the three-step architecture of MVSNet are proposed to address these issues.
Revised: Many methods based on the three-step architecture of MVSNet are proposed to address these issues.

Original: Several studies (Gu et al., 2020; Cheng et al., 2020; Yang et al., 2020; Wang et al., 2021;
Revised: Several studies (Gu et al., 2020; Cheng et al., 2020; Yang et al., 2020; Wang et al., 2021;

Original: Zhang et al., 2020) applied a coarse-to-ﬁne framework which estimates the depths through multiple stages to reduce the computational complexity.
Revised: Zhang et al., 2020) applied a coarse-to-ﬁne framework which estimates the depths through multiple stages to reduce the computational complexity.

Original: They used the estimated depth of each stage to adjust the depth hypothesis range for cost formulation in next stage.
Revised: They used the estimated depth of each stage to adjust the depth hypothesis range for cost formulation in next stage.

Original: To enhance the matching cost volume, other methods introduced a novel strategy for cost volume formulation.
Revised: To enhance the matching cost volume, other methods introduced a novel strategy for cost volume formulation.

Original: Luo et al.
Revised: Luo et al.

Original: (2019) proposed a patch-wise feature matching instead of pixel-wise matching to compute the cost.
Revised: (2019) proposed a patch-wise feature matching instead of pixel-wise matching to compute the cost.

Original: Yi et al.
Revised: Yi et al.

Original: (2020) and
Revised: (2020) and

Original: Luo et al.
Revised: Luo et al.

Original: (2020) applied an attention technique to improve feature matching.
Revised: (2020) applied an attention technique to improve feature matching.

Original: Zhang et al.
Revised: Zhang et al.

Original: Xu & Tao (2020b) estimated visibility information to score the view weights for cost aggregation.
Revised: Xu & Tao (2020c) estimated visibility information to score the view weights for cost aggregation.

Original: All these methods (Gu et al., 2020; Cheng et al., 2020; Yang et al., 2020; Wang et al., 2021; Zhang et al., 2020; Luo et al., 2020; Yu & Gao, 2020) had a trade-off between computation and performance; the methods with high performance usually require expensive computations.
Revised: All these methods (Gu et al., 2020; Cheng et al., 2020; Yang et al., 2020; Wang et al., 2021; Zhang et al., 2020; Luo et al., 2020; Yu & Gao, 2020) had a trade-off between computation and performance; the methods with high performance usually require expensive computations.

Original: Also, they only applied standard CNNs with static scale representation for feature extraction.
Revised: Also, they only applied standard CNNs with static scale representation for feature extraction.

Original: In this work, we aim to learn dynamic scale representation for more accurate feature matching.
Revised: In this work, we aim to learn dynamic scale representation for more accurate feature matching.

Original: We also estimate pixel-wise visibility information to remove noises and wrong matching pixels in cost aggregation.
Revised: We also estimate pixel-wise visibility information to remove noises and wrong matching pixels in cost aggregation.

Original: These goals are achieved by analyzing the normal curvature of the image surface.
Revised: These goals are achieved by analyzing the normal curvature of the image surface.

Original: Recently, Xu et al.
Revised: Recently, Xu et al.

Original: (2020) proposed an MVS method that considers the normal curvature information for feature extraction.
Revised: (2020) proposed an MVS method that considers the normal curvature information for feature extraction.

Original: This method handles ambiguity in the patch-match stereo method (Barnes et al., 2009).
Revised: This method handles ambiguity in the patch-match stereo method (Barnes et al., 2009).

Original: It selects a patch scale for each pixel by thresholding the normal curvatures computed in several candidate scales.
Revised: It selects a patch scale for each pixel by thresholding the normal curvatures computed in several candidate scales.

Original: Although the method effectively improves the quality of patch matching, the use of its handcrafting features limits the overall MVS performance.
Revised: Although the method effectively improves the quality of patch matching, the use of its handcrafted features limits the overall MVS performance.

Original: In contrast, our method does not require extracting handcrafting features and determining the threshold for scale selection.
Revised: In contrast, our method does not require extracting handcrafted features and determining the threshold for scale selection.

Original: Furthermore, CDSConv operates not only on the color information of images but also on the high dimensional features.
Revised: Furthermore, CDSConv operates not only on the color information of images but also on the high dimensional features.

Original: Multiple CDSConv operations can be stacked to form a very deep architecture for robust feature extraction in MVS.
Revised: Multiple CDSConv operations can be stacked to form a very deep architecture for robust feature extraction in MVS.

Original: Multi-scale features.
Revised: Multi-scale features.

Original: Due to large-scale variations of objects and textures in complex scenes, multiscale representations are adapted for accurate and effective dense pixel-level prediction in many computer vision tasks.
Revised: Due to large-scale variations of objects and textures in complex scenes, multiscale representations are adapted for accurate and effective dense pixel-level prediction in many computer vision tasks.

Original: For the MVS task, Feature Pyramid Network (FPN) (Lin et al., 2017; Gu et al., 2020; Cheng et al., 2020; Zhang et al., 2020; Luo et al., 2020) is a common approach for multi-scale feature extraction.
Revised: For the MVS task, Feature Pyramid Network (FPN) (Lin et al., 2017; Gu et al., 2020; Cheng et al., 2020; Zhang et al., 2020; Luo et al., 2020) is a common approach for multi-scale feature extraction.

Original: It is designed as the UNet-liked architecture (Ronneberger et al., 2015) to fuse features from different scales into a single one.
Revised: It is designed as the UNet-liked architecture (Ronneberger et al., 2015) to fuse features from different scales into a single one.

Original: However, the multi-scale of FPN is achieved at an image level through down-upsampling operations.
Revised: However, the multi-scale of FPN is achieved at an image level through down-upsampling operations.

Original: It cannot capture the large-scale variation of objects at a pixel level.
Revised: It cannot capture the large-scale variation of objects at a pixel level.

Original: Several studies in image segmentation can learn pixel-level multi-scale features by using multiple kernels with different sizes (He et al., 2019; Liu et al., 2016;
Revised: Several studies in image segmentation can learn pixel-level multi-scale features by using multiple kernels with different sizes (He et al., 2019; Liu et al., 2016;

Original: Chen et al., 2017; Zhao et al., 2017).
Revised: Chen et al., 2017; Zhao et al., 2017).

Original: However, these methods prefer a large receptive ﬁeld for segmentation tasks and usually ignore the details of objects essential for MVS tasks.
Revised: However, these methods prefer a large receptive ﬁeld for segmentation tasks and usually ignore the details of objects essential for MVS tasks.

Original: In contrast to the aforementioned methods (Gu et al., 2020; Cheng et al., 2020; Zhang et al., 2020; Luo et al., 2020; He et al., 2019), our proposed CDSConv can select an optimal scale for each pixel to produce a robust representation and enhance the matching uncertainty.
Revised: In contrast to the aforementioned methods (Gu et al., 2020; Cheng et al., 2020; Zhang et al., 2020; Luo et al., 2020; He et al., 2019), our proposed CDSConv can select an optimal scale for each pixel to produce a robust representation and enhance the matching uncertainty.

Original: Moreover, our feature extraction
Revised: Moreover, our feature extraction

Original: CDSFNet can cover denser scales by utilizing the power of a deep CNN.
Revised: CDSFNet can cover denser scales by utilizing the power of a deep CNN.

Original: Dynamic ﬁltering.
Revised: Dynamic ﬁltering.

Original: Instead of observing static receptive ﬁelds, several studies modiﬁed standard convolution to choose the adaptive receptive ﬁeld for each pixel (Wu et al., 2018; Jia et al., 2016;
Revised: Instead of observing static receptive ﬁelds, several studies modiﬁed standard convolution to choose the adaptive receptive ﬁeld for each pixel (Wu et al., 2018; Jia et al., 2016;

Original: Han et al., 2018).
Revised: Han et al., 2018).

Original: SAC (Zhang et al., 2017) was proposed to modify the ﬁxed-size receptive ﬁeld by learning position-adaptive scale coefﬁcients.
Revised: SAC (Zhang et al., 2017) was proposed to modify the ﬁxed-size receptive ﬁeld by learning position-adaptive scale coefﬁcients.

Original: Deformable ConvNets (Dai et al., 2017; Zhu et al., 2019) learned offsets for each pixel in a regular sampling grid of standard convolution to enlarge the sampling ﬁeld with an arbitrary form that can discover geometric-invariant features.
Revised: Deformable ConvNets (Dai et al., 2017; Zhu et al., 2019) learned offsets for each pixel in a regular sampling grid of standard convolution to enlarge the sampling ﬁeld with an arbitrary form that can discover geometric-invariant features.

Original: Recently,
Revised: Recently,

Original: Chen et al.
Revised: Chen et al.

Original: (2020b) proposed a dynamic convolution composed of multiple kernels with the same size to increases model complexity without increasing the network depth or width.
Revised: (2020b) proposed a dynamic convolution composed of multiple kernels with the same size to increases model complexity without increasing the network depth or width.

Original: All these existing works learn dynamic ﬁlters directly from the input images.
Revised: All these existing works learn dynamic ﬁlters directly from the input images.

Original: They are suitable for segmentation or recognition tasks.
Revised: They are suitable for segmentation or recognition tasks.

Original: For MVS task, they ignore the epipolar constraint between reference and source images.
Revised: For MVS task, they ignore the epipolar constraint between reference and source images.

Original: In contrast, our dynamic ﬁltering is guided by normal curvature computed in the direction of epipolar line, which can measure the matching capacity between reference and source image patches.
Revised: In contrast, our dynamic ﬁltering is guided by normal curvature computed in the direction of epipolar line, which can measure the matching capacity between reference and source image patches.

Original: To the best of our knowledge, this is the ﬁrst work that introduces dynamic ﬁlters to MVS.
Revised: To the best of our knowledge, this is the ﬁrst work that introduces dynamic ﬁlters to MVS.

Original:  PROPOSED FEATURE EXTRACTION NETWORK 3. N ORMAL C URVATURE
Revised:  PROPOSED FEATURE EXTRACTION NETWORK 3. N ORMAL C URVATURE

Original: Normal curvature is used to estimate how much a surface is curved at a speciﬁc point along a particular direction (Do Carmo, 2016).
Revised: Normal curvature is used to estimate how much a surface is curved at a speciﬁc point along a particular direction (Do Carmo, 2016).

Original: Let p ( X, σ ) be a patch centered at pixel X and having a scale σ which is proportional to the window size.
Revised: Let p ( X, σ ) be a patch centered at pixel X and having a scale σ which is proportional to the window size.

Original: L ( X, I src ) is denoted as the epipolar line on the reference image I ref , which passes through X and is related to a source image I src .
Revised: L ( X, I src ) is denoted as the epipolar line on the reference image I ref , which passes through X and is related to a source image I src .

Original: To reduce the matching ambiguity between p in I ref and the correct patch p (cid:48) in I src , a proper scale σ should be chosen when the corresponding patch p ( X, σ ) is less linear correlated to its surrounding patches along the epipolar line L .
Revised: To reduce the matching ambiguity between p in I ref and the correct patch p (cid:48) in I src , a proper scale σ should be chosen when the corresponding patch p ( X, σ ) is less linearly correlated to its surrounding patches along the epipolar line L .

Original: From the scale space theory (Lindeberg, 1994), the patch p ( X, σ ) can be considered as a point on the image surface represented at the scale σ .
Revised: From the scale space theory (Lindeberg, 1994), the patch p ( X, σ ) can be considered as a point on the image surface represented at the scale σ .

Original: Therefore, the scale σ is optimal for stereo matching if the image surface represented in that scale is curved signiﬁcantly at the point X along the epipolar line direction (Xu et al., 2020).
Revised: Therefore, the scale σ is optimal for stereo matching if the image surface represented in that scale is curved signiﬁcantly at the point X along the epipolar line direction (Xu et al., 2020).

Original: This requires to compute the normal curvature for the patch p ( X, σ ) along the direction ω
Revised: This requires to compute the normal curvature for the patch p ( X, σ ) along the direction ω

Original: =
Revised: =

Original: [ u, v ] T of the epipolar line L .
Revised: [ u, v ] T of the epipolar line L .

Original: The formula can be expressed in terms of the values of the ﬁrst and second fundamental forms of the image surface:
Revised: The formula can be expressed in terms of the values of the ﬁrst and second fundamental forms of the image surface:

Original: [Equation]
Revised: [Equation]

Revised: where I x , I y , I xx , I xy ,

Revised: I yy are the ﬁrst-order and second-order derivatives of the image I along the x -axis

Revised: and y -axis , respectively.

Original: 3. CURVATURE-GUIDED DYNAMIC SCALE CONVOLUTION
Revised: 3. CURVATURE-GUIDED DYNAMIC SCALE CONVOLUTION

Original: This section describes the novel convolutional module CDSConv to extract dynamic scale features.
Revised: This section describes the novel convolutional module CDSConv to extract dynamic scale features.

Original: Given a set of K convolutional kernels with different size { C 1 , C 2 , ..., C K } corresponding to K candidate scales { σ 1 , σ 2 , ..., σ K } , CDSConv aims to select a proper scale for each pixel X .
Revised: Given a set of K convolutional kernels with different size { C 1 , C 2 , ..., C K } corresponding to K candidate scales { σ 1 , σ 2 , ..., σ K } , CDSConv aims to select a proper scale for each pixel X .

Original: Hence, it can produce a robust output feature F out from the input F in based on the selected scale.
Revised: Hence, it can produce a robust output feature F out from the input F in based on the selected scale.

Original: Fig.
Revised: Fig.

Original: 1b shows our overall pipeline of CDSConv including two steps.
Revised: 1b shows our overall pipeline of CDSConv including two steps.

Original: First, CDSConv estimates approximately normal curvatures at K candidate scales.
Revised: First, CDSConv estimates approximately normal curvatures at K candidate scales.

Original: Second, it performs a scale selection step to output the optimal scale from K estimated curvatures.
Revised: Second, it performs a scale selection step to output the optimal scale from K estimated curvatures.

Original: This selection is implemented by a classiﬁcation network composed of two convolutional blocks and a Softmax activation for the output.
Revised: This selection is implemented by a classiﬁcation network composed of two convolutional blocks and a Softmax activation for the output.

Original: Learnable normal curvature.
Revised: Learnable normal curvature.

Original: The formula of normal curvature of a patch centered at X along the direction of epipolar line ω
Revised: The formula of normal curvature of a patch centered at X along the direction of epipolar line ω

Original: =
Revised: =

Original: [ u, v ] T in Eq.
Revised: [ u, v ] T in Eq.

Original: 1 can be re-written as follows
Revised: 1 can be re-written as follows

Original: [Equation]
Revised: [Equation]

Original: where I ( X, σ ) = I ( X ) ∗ G ( X, σ ) is the image intensity of pixel X in the image scale σ ; it is determined by convolving I with a Gaussian kernel G ( X, σ ) with the window size/scale σ (Lindeberg, 1994).
Revised: where I ( X, σ ) = I ( X ) ∗ G ( X, σ ) is the image intensity of pixel X in the image scale σ ; it is determined by convolving I with a Gaussian kernel G ( X, σ ) with the window size/scale σ (Lindeberg, 1994).

Original: The derivatives I x , I y , I xx , I xy , and I xy can be computed by convolution between the original image I and the derivatives of Gaussian kernel G ( X, σ )
Revised: The derivatives I x , I y , I xx , I xy , and I yy can be computed by convolution between the original image I and the derivatives of Gaussian kernel G ( X, σ )

Original: [Equation]
Revised: [Equation]

Original: where ∗ is the convolution operator.
Revised: where ∗ is the convolution operator.

Original: There are two main drawbacks when embedding the normal curvature in Eq.
Revised: There are two main drawbacks when embedding the normal curvature in Eq.

Original: 2 into a deep neural network.
Revised: 2 into a deep neural network.

Original: First, the computation is heavy because of ﬁve convolution operations for computing the derivatives.
Revised: First, the computation is heavy because of ﬁve convolution operations for computing the derivatives I x , I y , I xx , I xy , and I yy .

Original: Second, Eq. 2 is intractable when the pixel X is a latent feature F in ( X ) instead of the image intensity I ( X ) .
Revised: Second, using Eq.to compute curvature is infeasible when the pixel X is a latent feature F in ( X ) instead of the image intensity I ( X ) .

Original: For these reasons, we derive an approximate form of the normal curvature, which reduces the computational cost and supports estimation for high-dimensional data.
Revised: For these reasons, we derive an approximate form of the normal curvature, which reduces the computational cost and can handle the high-dimensional feature input.

Original: It is acceptable to assume that G ( X, σ ) (cid:28) 1 ; this is achieved by multiplying G ( X, σ ) with a small constant in practice.
Revised: To address the heavy computation issue, we notice that the curvature curv σ and derivatives I x i y j are proportional to Gaussian kernel G , following Eq.

Original: Following Eq. 
Revised: 2 and Eq. Moreover, we use a classiﬁcation network to select the patch scale automatically from the curvature inputs. Therefore, we can perform normalization for the curvatures by rescaling the kernel G . We restrict the gaussian kernel in a small range, i.e., G ( X, σ ) (cid:28) 1 .

Original: 3, the derivatives I x , I y , I xx , I xy , and I yy should be much smaller than 1.
Revised: As a result, the derivatives I x , I y , I xx , I xy , and I yy are also much smaller than 1.

Original: Therefore, we can approximate the denominator in Eq.
Revised: We can approximate the denominator in Eq.

Original: 2 by 1.
Revised: 2 by 1.

Original: Eq.
Revised: Eq.

Original: 2 can be re-deﬁned as
Revised: 2 can be re-deﬁned as

Original: [Equation]
Revised: [Equation]

Original: where curv σ ( X, ω ) , I xx , I xy and I
Revised: where curv σ ( X, ω ) , I xx , I xy and I

Original: yy is much smaller than 1, which implies that their values should be restricted in a small range.
Revised: yy is much smaller than 1, which implies that their values should be restricted in a small range.

Original: Now, the computation is reduced by haft compared to Eq.
Revised: Now, the computation is reduced by haft compared to Eq.

Original: On the other hand, to make Eq.
Revised: To make Eq.

Original: 4 work with the high-dimensional image feature F in ( X, σ ) , we propose to use learnable kernels instead of using ﬁxed derivatives of Gaussian kernel.
Revised: 4 work with the high-dimensional image feature F in ( X, σ ) , we propose to use learnable kernels instead of using ﬁxed derivatives of Gaussian kernel.

Original: In particular, for each scale σ , we introduce three learnable convolution kernels K xxσ , K xyσ , K yyσ to replace G xx , G xy , and G yy respectively.
Revised: In particular, for each scale σ , we introduce three learnable convolution kernels K xxσ , K xyσ , K yyσ to replace G xx , G xy , and G yy respectively.

Original: These kernels adapt to the input features to approximately compute the second-order derivatives of the image surface.
Revised: These kernels adapt to the input features to approximately compute the second-order derivatives of the image surface.

Original: They are trained implicitly by backpropagation when training the end-to-end networks.
Revised: They are trained implicitly by backpropagation when training the end-to-end networks.

Original: Weights of these kernels need to be restricted to a small value K ( . )
Revised: Weights of these kernels need to be restricted to a small value K ( . )

Original: σ (cid:28) 1 because of the assumption about the Gaussian kernel.
Revised: σ (cid:28)because of the assumption about the Gaussian kernel.

Original: We enforce this constraint by adding a regularization term to the loss in Section 4.2.
Revised: We enforce this constraint by adding a regularization term to the loss in Section 4.2.

Original: In summary, we propose learnable normal curvature having the formula as follows (written in the matrix form)
Revised: In summary, we propose learnable normal curvature having the formula as follows (written in the matrix form)

Original: [Equation]
Revised: [Equation]

Original: where K ( . )
Revised: where K ( . )

Original: σ s are the learnable kernels and (cid:107) K ( . )
Revised: σ s are the learnable kernels and (cid:107) K ( . )

Original: σ (cid:107) → 0 , F in is the input feature.
Revised: σ (cid:107) → 0 , F in is the input feature.

Original: Scale selection.
Revised: Scale selection.

Original: After obtaining K normal curvatures { curv σ 1 , curv σ 2 , ..., curv σ K } in K candidate scales, this step selects a proper scale σ ( . ) for each pixel X
Revised: After obtaining K normal curvatures { curv σ 1 , curv σ 2 , ..., curv σ K } in K candidate scales, this step selects a proper scale σ ( . ) for each pixel X

Original: and then outputs the feature F out ( X ) from the corresponding convolution kernel C ( . ) .
Revised: and then outputs the feature F out ( X ) from the corresponding convolution kernel C ( . ) .

Original: The proper scale is selected by analyzing the curvatures estimated in K candidate scales.
Revised: The proper scale is selected by analyzing the curvatures estimated in K candidate scales.

Original: It is observed that a small scale cannot capture enough context information to learn discriminative feature, especially in low-textured regions.
Revised: It is observed that a small scale cannot capture enough context information to learn discriminative feature, especially in low-textured regions.

Original: Meanwhile, a large scale smooths local structure in rich texture regions.
Revised: Meanwhile, a large scale smooths local structure in rich texture regions.

Original: MARMVS (Xu et al., 2020) chose the proper scale by searching the normal curvature with a threshold T = 0 .
Revised: MARMVS (Xu et al., 2020) chose the proper scale by searching the normal curvature with a threshold T = 0 .

Original: However, using of ﬁxed threshold T prevents generalization on various scenes and structures.
Revised: However, using of ﬁxed threshold T prevents generalization on various scenes and structures.

Original: Instead, we propose a classiﬁcation strategy that automatically selects the proper scale from K curvature inputs.
Revised: Instead, we propose a classiﬁcation strategy that automatically selects the proper scale from K curvature inputs.

Original: This is achieved by using a lightweight CNN with two convolutional blocks.
Revised: This is achieved by using a lightweight CNN with two convolutional blocks.

Original: For each pixel X , the CNN outputs one-hot vector [ w 1 , w 2 , ..., w K ] for scale selection by applying a Softmax with small temperature parameter, which is a differentiable approximation to argmax operator (Jang et al., 2017).
Revised: For each pixel X , the CNN outputs one-hot vector [ w 1 , w 2 , ..., w K ] for scale selection by applying a Softmax with small temperature parameter, which is a differentiable approximation to argmax operator (Jang et al., 2017).

Original: Finally, the feature output F out is produced from the feature input F in with K candidate kernels { C 1 , C 2 , ..., C K } by using weighted sum
Revised: Finally, the feature output F out is produced from the feature input F in with K candidate kernels { C 1 , C 2 , ..., C K } by using weighted sum

Original: [Equation]
Revised: [Equation]

Original: where ∗ is the convolution operator.
Revised: where ∗ is the convolution operator.

Original: Also, we can extract the normal curvature corresponding to the selected scale by
Revised: Also, we can extract the normal curvature corresponding to the selected scale by

Original: [Equation]
Revised: [Equation]

Original: 3. CURVATURE-GUIDED DYNAMIC SCALE FEATURE NETWORK
Revised: 3. CURVATURE-GUIDED DYNAMIC SCALE FEATURE NETWORK

Original: This section introduces the Curvature-guided dynamic scale feature network, CDSFNet, which is used as feature extraction step for our MVS framework.
Revised: This section introduces the Curvature-guided dynamic scale feature network, CDSFNet, which is used as feature extraction step for our MVS framework.

Original: CDSFNet is composed of multiple CDSConv layers instead of standard convolution layers.
Revised: CDSFNet is composed of multiple CDSConv layers instead of standard convolution layers.

Original: By expanding the searching scale-space,
Revised: By expanding the searching scale-space,

Original:  CDSFNet can select the optimal scale for each pixel to learn robust representation that reduces matching ambiguity.
Revised: Figure b shows CDSFNet can select the optimal scale for each pixel to learn robust representation that reduces matching ambiguity.

Original: Architecture.
Revised: Architecture.

Original: CDSFNet is an Unet-liked architecture (Ronneberger et al., 2015).
Revised: CDSFNet is an Unet-liked architecture (Ronneberger et al., 2015).

Original: It is organized into three levels of spatial resolution l ∈ { 0 , 1 , 2 } to adapt the coarse-to-ﬁne MVS framework.
Revised: It is organized into three levels of spatial resolution l ∈ { 0 , 1 , 2 } to adapt the coarse-to-ﬁne MVS framework.

Original: Givenb the inputs including image I and its estimated epipole e , the network outputs three features for three levels { F (0) , F (1) , F (2) } and three estimated normal curvatures { NC est, (0) , NC est, (1) , NC est, (2) } .
Revised: Given the inputs including image I and its estimated epipole e , the network outputs three features for three levels { F (0) , F (1) , F (2) } and three estimated normal curvatures { NC est, (0) , NC est, (1) , NC est, (2) } .

Original: Understanding CDSFNet.
Revised: Understanding CDSFNet.

Original: The main goal of CDSFNet is to select pixel-level scale for robust feature extraction.
Revised: The main goal of CDSFNet is to select pixel-level scale for robust feature extraction.

Original: The scale changes dynamically, depending on the detail of image objects and the epipolar constraint, i.e. the relative camera pose between reference and source images.
Revised: The scale changes dynamically, depending on the detail of image objects and the epipolar constraint, i.e. the relative camera pose between reference and source images.

Original: Therefore, our method is more appropriate for MVS than the other dynamic scale networks (He et al., 2019;
Revised: Therefore, our method is more appropriate for MVS than the other dynamic scale networks (He et al., 2019;

Original: Wu et al., 2018; Jia et al., 2016) proposed in image segmentation task.
Revised: Wu et al., 2018; Jia et al., 2016) proposed in image segmentation task.

Original: To have a low computational complexity, the number of candidate scales in CDSConv layer should be small, we only use 2 or 3 candidates.
Revised: To have a low computational complexity, the number of candidate scales in CDSConv layer should be small, we only use 2 or 3 candidates.

Original: However, the searching scale-space is expanded profoundly when multiple CDSConv layers are stacked in CDSFNet.
Revised: However, the searching scale-space is expanded profoundly when multiple CDSConv layers are stacked in CDSFNet.

Original: Fig.
Revised: Fig.

Original: 2 show the patch scale or window size for each pixel estimated roughly from the ﬁrst two layers of CDSFNet.
Revised: 2 show the patch scale or window size for each pixel estimated roughly from the ﬁrst two layers of CDSFNet.

Original: For each reference and source views, we draw the reference and source scale maps respectively.
Revised: For each reference and source views, we draw the reference and source scale maps respectively.

Original: There are two main advantages of CDSFNet shown in Fig. 2.
Revised: There are two main advantages of CDSFNet shown in Fig. 2.

Original: First, the estimated scales in rich texture, thin structure, and near-edge regions are often small while those in untextured regions are large.
Revised: First, the estimated scales in rich texture, thin structure, and near-edge regions are often small while those in untextured regions are large.

Original: Second, the scales are changed with respect to viewpoints (Fig. 2a) and relative camera pose between reference and source views (Fig. 2b).
Revised: Second, the scales are changed with respect to viewpoints (Fig. 2a) and relative camera pose between reference and source views (Fig. 2b).

Original: In Fig.
Revised: In Fig.

Original: 2a, the closer viewpoint is, the larger scales are estimated.
Revised: 2a, the closer viewpoint is, the larger scales are estimated.

Original: In
Revised: In

Original: Fig.
Revised: Fig.

Original: 2b, the reference and source scale maps are similar when the difference of camera poses is not large.
Revised: 2b, the reference and source scale maps are similar when the difference of camera poses is not large.

Original: Otherwise, when the difference is large, the scale map of reference view is changed to adapt the source view, which was marked in the red circle.
Revised: Otherwise, when the difference is large, the scale map of reference view is changed to adapt the source view, which was marked in the red circle.

Original: This is a superiority of CDSFNet because it estimates scale based on epipolar constraint between reference and source views.
Revised: This is a superiority of CDSFNet because it estimates scale based on epipolar constraint between reference and source views.

Original:  CDS-MVSN ET
Revised:  CDS-MVSN ET

Original: In this section, we describe the proposed MVS network referred to as CDS-MVSNet, which predicts a depth map D ref from the reference I 0 and N source images { I i } Ni =1 .
Revised: In this section, we describe the proposed MVS network referred to as CDS-MVSNet, which predicts a depth map D ref from the reference I 0 and N source images { I i } Ni =1 .

Original: We adopted the cascade structure of CasMVSNet (Gu et al., 2020) as the baseline structure of CDS-MVSNet.
Revised: We adopted the cascade structure of CasMVSNet (Gu et al., 2020) as the baseline structure of CDS-MVSNet.

Original: The network is composed of multiple cascade stages to predict the depth maps in a coarse-to-ﬁne manner.
Revised: The network is composed of multiple cascade stages to predict the depth maps in a coarse-to-ﬁne manner.

Original: Each stage estimates a depth through three steps: feature extraction, cost volume formulation, cost volume regularization & depth regression.
Revised: Each stage estimates a depth through three steps: feature extraction, cost volume formulation, cost volume regularization & depth regression.

Original: For each cascade stage, CDS-MVSNet upsamples the predicted depth map and uses it to narrow the range of depth hypothesis for cost volume formulation in the next stage.

Original: This cascade structure can estimate high-resolution depths with relatively low memory consumption and fast computation time.

Original: CDS-MVSNet formulates a 3D cost volume based on the output features of CDSFNet.
Revised: CDS-MVSNet formulates a 3D cost volume based on the output features of CDSFNet.

Original: The features of CDSFNet effectively reduce the matching ambiguity by considering the proper pixel scales.
Revised: The features of CDSFNet effectively reduce the matching ambiguity by considering the proper pixel scales.

Original: CDSMVSNet also performs visibility-based cost aggregation, which reduces matching errors induced by invisible pixels.

Original: The visibility information is estimated from the normal curvatures computed by

Original: CDSFNet in Eq. CDS-MVSNet produces more accurate depths in coarser stages compared to other cascade MVS methods (Gu et al., 2020; Cheng et al., 2020; Yang et al., 2020; Wang et al., 2021) based on the robustness of our feature extraction and cost formulation.
Revised: Due to the robustness of our feature extraction and cost formulation, CDS-MVSNet can produce more accurate depths in coarser stages compared to other cascade MVS methods (Gu et al., 2020; Cheng et al., 2020; Yang et al., 2020; Wang et al., 2021).

Original: There is even little difference in the accuracy of the depths estimated at half resolution and full resolution.
Revised: There is even little difference in the accuracy of the depths estimated at half resolution and full resolution.

Original: Therefore, differently from CasMVSNet, CDS-MVSNet only computes depths of half-resolution images using three-step MVS computation and then upsamples the depths to the original resolution in the ﬁnal stage.
Revised: Therefore, differently from CasMVSNet, CDS-MVSNet only computes depths of half-resolution images using three-step MVS computation and then upsamples the depths to the original resolution in the ﬁnal stage.

Original: The upsampled depths at the original resolution can be reﬁned by a 2D CNN to obtain the ﬁnal depths.
Revised: The upsampled depths at the original resolution can be reﬁned by a 2D CNN to obtain the ﬁnal depths.

Original: This approach can drastically reduce the computation time and GPU memory requirements.
Revised: This approach can drastically reduce the computation time and GPU memory requirements.

Original: 4. DETAILS OF ARCHITECTURE
Revised: 4. DETAILS OF ARCHITECTURE

Original: Robust Feature Extraction.
Revised: Robust Feature Extraction.

Original: Given the reference image I 0 and N source images { I 1 , I 2 , ..., I N } ,
Revised: Given the reference image I 0 and N source images { I 1 , I 2 , ..., I N } ,

Original: CDSFNet extracts a robust feature pair ( F 0 ,i , F i ) i =1 ...N for each image pair ( I 0 , I i ) i =1 ...N .
Revised: CDSFNet extracts a robust feature pair ( F 0 ,i , F i ) i =1 ...N for each image pair ( I 0 , I i ) i =1 ...N .

Original: Note that the feature of the reference image F 0 ,i varies according to each source image i due to the guidance of epipolar geometry in normal curvature estimation.
Revised: Note that the feature of the reference image F 0 ,i varies according to each source image i due to the guidance of epipolar geometry in normal curvature estimation.

Original: This is our superiority compared to the state-of-the-art MVS networks which learn an identical feature for the reference image.
Revised: This is our superiority compared to the state-of-the-art MVS networks which learn an identical feature for the reference image.

Original: Cost Volume Formulation.
Revised: Cost Volume Formulation.

Original: We ﬁrst compute two-view matching costs and then aggregate them into a single cost.
Revised: We ﬁrst compute two-view matching costs and then aggregate them into a single cost.

Original: The two-view matching cost is computed per sampled depth hypothesis to form cost volume (Yao et al., 2018; Gu et al., 2020; Wang et al., 2021).
Revised: The two-view matching cost is computed per sampled depth hypothesis to form cost volume (Yao et al., 2018; Gu et al., 2020; Wang et al., 2021).

Original: Given a sampled depth hypothesis d j ( j = 1 ...D ) along with all camera parameters, we warp the feature maps of source image { F i } Ni =1 into this depth plane.
Revised: Given a sampled depth hypothesis d j ( j = 1 ...D ) along with all camera parameters, we warp the feature maps of source image { F i } Ni =1 into this depth plane.

Original: Let F wi ( d j ) be the warped feature map of source image i at the depth hypothesis d j .
Revised: Let F wi ( d j ) be the warped feature map of source image i at the depth hypothesis d j .

Original: The two-view matching cost V i ( d j ) between the reference image I 0 and source image
Revised: The two-view matching cost V i ( d j ) between the reference image I 0 and source image

Original: I i at the depth hypothesis d j is determined as V i ( d j )
Revised: I i at the depth hypothesis d j is determined as V i ( d j )

Original: = (cid:104) F 0 ,i , F wi ( d j ) (cid:105) , where (cid:104) .
Revised: = (cid:104) F 0 ,i , F wi ( d j ) (cid:105) , where (cid:104) .

Original: (cid:105) is the pixel-wise inner product for measuring the similarity of two feature maps.
Revised: (cid:105) is the pixel-wise inner product for measuring the similarity of two feature maps.

Original: Next, we perform cost aggregation over the computed two-view cost volumes.
Revised: Next, we perform cost aggregation over the computed two-view cost volumes.

Original: Inspired from visibility-awareness for cost aggregation (Zhang et al., 2020; Xu & Tao, 2020b), we employ pixelwise view weight prediction.
Revised: Inspired from visibility-awareness for cost aggregation (Zhang et al., 2020; Xu & Tao, 2020c), we employ pixelwise view weight prediction.

Original: However, instead of predicting directly from two-view cost volume (Zhang et al., 2020; Xu & Tao, 2020b; Yi et al., 2020), we additionally utilize the normal curvature maps, NC est , estimated by CDSFNet in Eq.
Revised: However, instead of predicting directly from two-view cost volume (Zhang et al., 2020; Xu & Tao, 2020c; Yi et al., 2020), we additionally utilize the normal curvature maps, NC est , estimated by CDSFNet in Eq.

Original: The normal curvature can implicitly provide information about level details of surface.
Revised: The normal curvature can implicitly provide information about level details of surface.

Original: For instance, a pixel with small normal curvature indicates that it belongs to a large untextured region.
Revised: For instance, a pixel with small normal curvature indicates that it belongs to a large untextured region.

Original: In this case, its feature is unmatchable and needs to be eliminated from cost aggregation to reduce noises for cost regularization step.
Revised: In this case, its feature is unmatchable and needs to be eliminated from cost aggregation to reduce noises for cost regularization step.

Original: In summary, the estimated normal curvature encodes implicitly the matching ability of learned feature.
Revised: In summary, the estimated normal curvature encodes implicitly the matching ability of learned feature.

Original: Let NC esti be the estimated normal curvature of the reference view related to source view i , and H i be the entropy computed from two-view matching cost volume V i .
Revised: Let NC esti be the estimated normal curvature of the reference view related to source view i , and H i be the entropy computed from two-view matching cost volume V i .

Original: We use a simple 2D CNN denoted as V is ( . ) to predict a view weight map W i from two inputs NC esti and H i .
Revised: We use a simple 2D CNN denoted as V is ( . ) to predict a view weight map W i from two inputs NC esti and H i .

Original: Finally, the aggregated cost V is computed by weighted mean.
Revised: Finally, the aggregated cost V is computed by weighted mean.

Original: [Equation]
Revised: [Equation]

Original: Cost volume regularization & Depth regression.
Revised: Cost volume regularization & Depth regression.

Original: Following most recent learning-based MVS methods, a 3D CNN is applied to obtain a depth probability volume P from the aggregated matching cost volume V .
Revised: Following most recent learning-based MVS methods, a 3D CNN is applied to obtain a depth probability volume P from the aggregated matching cost volume V .

Original: The depth is then regressed from P by D est
Revised: The depth is then regressed from P by D est

Original: = (cid:80) Dj =1 d j P ( d j )
Revised: = (cid:80) Dj =1 d j P ( d j )

Original: 4. LOSS FUNCTION
Revised: 4. LOSS FUNCTION

Original: We propose a feature loss for training CDSFNet effectively.
Revised: We propose a feature loss for training CDSFNet effectively.

Original: Given a ground truth depth D gt of the reference image, we warp the feature of source images into this depth map and then compute the matching cost V ( D gt ) .
Revised: Given a ground truth depth D gt of the reference image, we warp the feature of source images into this depth map and then compute the matching cost V ( D gt ) .

Original: We label this map as positive, hence deﬁnes its classiﬁcation label as c = 1 .
Revised: We label this map as positive, hence deﬁnes its classiﬁcation label as c = 1 .

Original: To generate the matching costs with negative labels c = 0 , we randomly sample N d neighboring depths D neg around the ground truth D gt and then compute the matching costs
Revised: To generate the matching costs with negative labels c = 0 , we randomly sample N d neighboring depths D neg around the ground truth D gt and then compute the matching costs

Original: V ( D neg ) for these depths.
Revised: V ( D neg ) for these depths.

Original: Finally, binary cross entropy is used to deﬁne the feature loss.
Revised: Finally, binary cross entropy is used to deﬁne the feature loss.

Original: However, we need to add regularization terms to restrict the convolutional weights of CDSFNet, w CDSF , and estimated normal curvature NC est to a small range as mentioned in Section 3.2.
Revised: However, we need to add regularization terms to restrict the convolutional weights of CDSFNet, w CDSF , and estimated normal curvature NC est to a small range as mentioned in Section 3.2.

Original: Therefore, the ﬁnal feature loss is deﬁned as:
Revised: Therefore, the ﬁnal feature loss is deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: where sig ( . ) is the sigmoid function, M is total number of pixels X , λ 1 = 0 .
Revised: where sig ( . ) is the sigmoid function, M is total number of pixels X , λ 1 = 0 .

Original: 01 and λ 2 = 0 .
Revised: 01 and λ 2 = 0 .

Original: 1 are the regularized hyperparameters.
Revised: 1 are the regularized hyperparameters.

Original: Similar to most learning-based MVS methods (Yao et al., 2018; Gu et al., 2020), the depth loss
Revised: Similar to most learning-based MVS methods (Yao et al., 2018; Gu et al., 2020), the depth loss

Original: L depth is deﬁned by L 1 loss between the predicted depth and the ground truth depth.
Revised: L depth is deﬁned by L 1 loss between the predicted depth and the ground truth depth.

Original: Because our method includes 4 stages, there are four depth loss {L ( l ) depth }
Revised: Because our method includes 4 stages, there are four depth loss {L ( l ) depth }

Original: 3 l =0 .
Revised: 3 l =0 .

Original: Finally, the total loss L total is deﬁned as a sum of two mentioned losses L total = 5 L feat + (cid:80) 3 l =

Revised: Finally, the total loss L total is deﬁned as a sum of two mentioned losses L total = 5 L feat + (cid:80) 3 l =

Original: L ( l ) depth
Revised: L ( l ) depth

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: [Table]

Original:  EXPERIMENTAL RESULTS 5. DATASETS
Revised:  EXPERIMENTAL RESULTS 5. DATASETS

Original: The datasets used in our evaluation are DTU (Aanæs et al., 2016), BlendedMVS (Yao et al., 2020), and Tanks & Temples (Knapitsch et al., 2017).
Revised: The datasets used in our evaluation are DTU (Aanæs et al., 2016), BlendedMVS (Yao et al., 2020), and Tanks & Temples (Knapitsch et al., 2017).

Original: Due to the simple camera trajectory of all scenes in
Revised: Due to the simple camera trajectory of all scenes in

Original: DTU, we additionally utilize the BlendedMVS dataset with diverse camera trajectories for training.
Revised: DTU, we additionally utilize the BlendedMVS dataset with diverse camera trajectories for training.

Original: Tanks & Temples is provided as a set of video sequences in realistic environments.
Revised: Tanks & Temples is provided as a set of video sequences in realistic environments.

Original: We train the model on the training sets of DTU and BlendedMVS, then evaluate the testing set of DTU and intermediate set of Tanks & Temples.
Revised: We train the model on the training sets of DTU and BlendedMVS, then evaluate the testing set of DTU and intermediate set of Tanks & Temples.

Original: The details of implementation setup are described in the supplemental materials.
Revised: The details of implementation setup are described in the supplemental materials.

Original: 5. BENCHMARK PERFORMANCE
Revised: 5. BENCHMARK PERFORMANCE

Original: Results for DTU dataset.
Revised: Results for DTU dataset.

Original: We predict the depth at the highest resolution 1536 × 1152 .
Revised: We predict the depth at the highest resolution 1536 × 1152 .

Original: The performance of the proposed method was compared with state-of-the-art MVS methods, including conventional and learning-based methods, using the DTU dataset.
Revised: The performance of the proposed method was compared with state-of-the-art MVS methods, including conventional and learning-based methods, using the DTU dataset.

Original: However, we especially notice the learning-based methods including CasMVSNet, UCSNet, CVP-MVSNet, and Vis-MVSNet, which adopted the cascade structure for MVS depth estimation.
Revised: However, we especially notice the learning-based methods including CasMVSNet, UCSNet, CVP-MVSNet, and Vis-MVSNet, which adopted the cascade structure for MVS depth estimation.

Original: Similar to us, these methods used the same 3D CNN cost regularization.
Revised: Similar to us, these methods used the same 3D CNN cost regularization.

Original: Therefore, we can validate our feature extraction and cost formulation by comparing with them.
Revised: Therefore, we can validate our feature extraction and cost formulation by comparing with them.

Original: Table 1 presents the quantitative results achieved by the methods on the DTU evaluation dataset.
Revised: Table 1 presents the quantitative results achieved by the methods on the DTU evaluation dataset.

Original: We calculated the three standard error metrics (Aanæs et al., 2016) given on the ofﬁcial site of DTU: accuracy, completeness, and overall error.
Revised: We calculated the three standard error metrics (Aanæs et al., 2016) given on the ofﬁcial site of DTU: accuracy, completeness, and overall error.

Original: The learning-based methods exhibited better overall performance than the conventional methods.
Revised: Following the same setups with most of the baselines, we trained our model on the DTU training set and then used this model for evaluation. Moreover, we also provided the results which produced by training on both DTU and BlendedMVS simultaneously.

Original: Our method exhibited the best performance in terms of completeness and overall error.
Revised: In both cases, our method exhibited the best performance in terms of completeness and overall error.

Original: In particular, the overall performance of our method at 0 .

Original: 315 had a signiﬁcant margin over to the second-best performer at, 0 . 344 .

Original: We  achieved good results for both accuracy and completeness, while the other methods often produced a large trade-off between these two metrics.
Revised: We also achieved good results for both accuracy and completeness, while the other methods often produced a large trade-off between these two metrics.

Original: For example, Gipuma had a high accuracy at 0 .
Revised: For example, Gipuma had a high accuracy at 0 .

Original: but it produced very low completeness, 0 . 873 .
Revised: but it produced very low completeness, 0 . 873 .

Original: In contrast, PatchMatchNet was with a high completeness (0 . 277) and very low accuracy (0 . 427) .
Revised: In contrast, PatchMatchNet was with a high completeness (0 . 277) and very low accuracy (0 . 427) .

Original: Moreover, our method outperformed all similar methods using the cascade structure.
Revised: Moreover, our method outperformed all similar methods using the cascade structure.

Original: This indicates that our feature representation and cost aggregation guided by normal curvature can remarkably improve MVS depth prediction remarkably.
Revised: This indicates that our feature representation and cost aggregation guided by normal curvature can remarkably improve MVS depth prediction remarkably.

Original: To validate the dynamic scale property of our CDSFNet, we measure depth precision on different image resolutions.
Revised: To validate the dynamic scale property of our CDSFNet, we measure depth precision on different image resolutions.

Original: Depth precision is the average percentage of depths with errors lower than a deﬁned threshold (Gu et al., 2020).
Revised: Depth precision is the average percentage of depths with errors lower than a deﬁned threshold (Gu et al., 2020).

Original: Here we computed precision with the thresholds of 2mm and 4mm.
Revised: Here we computed precision with the thresholds of 2mm and 4mm.

Original: We compared to CasMVSNet, PatchMatchNet, UCSNet, and CVP-MVSNet, which applied the cascade structure to predict high-resolution depths.
Revised: We compared to CasMVSNet, PatchMatchNet, UCSNet, and CVP-MVSNet, which applied the cascade structure to predict high-resolution depths.

Original: Note that all methods are trained on the datasets with image-resolution of 640 × 512 .
Revised: Note that all methods are trained on the datasets with image-resolution of 640 × 512 .

Original: Table 2 presents the quantitative results for the depth estimated on ﬁve image resolutions.
Revised: Table 2 presents the quantitative results for the depth estimated on ﬁve image resolutions.

Original: The baseline methods degraded the performance when the image-resolution was increased while our method could maintain a stable performance at any image resolution.
Revised: The baseline methods degraded the performance when the image-resolution was increased while our method could maintain a stable performance at any image resolution.

Original: The reason is that our feature extraction CDSFNet can utilize dynamic scales for learning features.
Revised: The reason is that our feature extraction CDSFNet can utilize dynamic scales for learning features.

Original: The dynamic scale adapts to various image-resolutions and object scales in image.
Revised: The dynamic scale adapts to various image-resolutions and object scales in image.

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: 49.06 CVP-MVSNet(Yangetal.,2020) 54.

Original: [Figure] [Figure] [Figure] [Figure]
Revised: 47.54 Ours(ﬁne-tuningonBlendedMVS) 60.

Original: [Figure] [Figure]
Revised: 50.58 Ours(DTU+BlendedMVS)

Original: Generalization on Tanks & Temples Dataset.
Revised: Generalization on Tanks & Temples Dataset.

Original: To verify the generalization capability of our CDSMVSNet, we used the  modeltrained on DTU and BlendedMVS to evaluate on the Tanks & Temples benchmark.
Revised: To verify the generalization capability of our CDSMVSNet, we also provided two evaluations similar to the evaluation of DTU. First, we ﬁne-tuned the pre-trained model on DTU by using BlendedMS dataset.

Revised: Second, we used directly the model trained on both datasets for evaluation.

Original: The reconstructed point clouds were submitted to the benchmark website of Tanks & Temples (Knapitsch et al., 2017) to receive the F-score, which is a combination of precision and recall of the reconstructed 3D model.
Revised: The reconstructed point clouds were submitted to the benchmark website of Tanks & Temples (Knapitsch et al., 2017) to receive the F-score, which is a combination of precision and recall of the reconstructed 3D model.

Original: Table 3 lists the quantitative results of our method and other state-of-the-art methods.
Revised: Table 3 lists the quantitative results of our method and other state-of-the-art methods.

Original: As shown in Table 3, our method achieved the best mean Fscore, 61 . 58 . For individual results, we achieved the ﬁrst rank for 4 scenes (Family, Francis, Horse, and M60) and the second rank for 2 scenes (Panther, Playground).
Revised: As shown in Table 3, our method achieved the best mean Fscores for both evaluations, 60 . 82 and 61 . 58 , respectively. Our method outperformed both the state-of-the-art traditional methods such as ACMM, ACMP and the learning-based methods including AttMVS and Vis-MVSNet.

Original: The qualitative results of all scenes are shown in detail in the supplemental materials.
Revised: The qualitative results of all scenes are shown in detail in the supplemental materials.

Original: Compared to CasMVSNet and UCSNet, which are most similar to our method except the feature extraction and cost formulation, we obtained better mean F-score with high margins.
Revised: Compared to CasMVSNet and UCSNet, which are most similar to our method except the feature extraction and cost formulation, we obtained better mean F-score with high margins.

Original: This demonstrates the effectiveness of our feature extraction CDSFNet and cost aggregation strategy on complex outdoor scenes.
Revised: This demonstrates the effectiveness of our feature extraction CDSFNet and cost aggregation strategy on complex outdoor scenes.

Original: Run-time and Memory.
Revised: Run-time and Memory.

Original: This section evaluates the memory consumption and run-time compared to several state-of-the-art learning-based methods that achieve competing performance and employ the cascade structure similar to us: CasMVSNet, UCS-Net, CVP-MVSNet, and FastMVSNet.
Revised: This section evaluates the memory consumption and run-time compared to several state-of-the-art learning-based methods that achieve competing performance and employ the cascade structure similar to us: CasMVSNet, UCS-Net, CVP-MVSNet, and FastMVSNet.

Original: Fig.shows the memory and run-time of all methods measured on DTU evaluation set with different image resolutions.
Revised: Fig.shows the memory and run-time of all methods measured on DTU evaluation set with different image resolutions.

Original: We observe that the run-time and memory of all methods are more required when the image-resolution increases higher.
Revised: We observe that the run-time and memory of all methods are more required when the image-resolution increases higher.

Original: However, we notice that the memory consumption of our method was dropped from the resolution of 80% while our run-time was slightly increased.
Revised: However, we notice that the memory consumption of our method was dropped from the resolution of 80% while our run-time was slightly increased.

Original: This indicates that our method can release a signiﬁcant amount of GPU memory by only sacriﬁcing a small amount of run-time.
Revised: This indicates that our method can release a signiﬁcant amount of GPU memory by only sacriﬁcing a small amount of run-time.

Original: Our method achieves signiﬁcantly faster run-time and lower required memory at the very high image-resolution compared to the other baselines.
Revised: Our method achieves signiﬁcantly faster run-time and lower required memory at the very high image-resolution compared to the other baselines.

Original: For example, at a resolution of 1536 × 1152 (92.2%), the memory consumption is reduced by 54.0% , 34.1%, and 54.0% compared to
Revised: For example, at a resolution of 1536 × 1152 (92.2%), the memory consumption is reduced by 54.0% , 34.1%, and 54.0% compared to

Original: CasMVSNet, UCSNet, and CVP-MVSNet respectively.
Revised: CasMVSNet, UCSNet, and CVP-MVSNet respectively.

Original: The corresponding comparison results of run-time are 43.2%, 37.8%, and 74.9% .
Revised: The corresponding comparison results of run-time are 43.2%, 37.8%, and 74.9% .

Original: Finally, we conclude that our method is more efﬁcient in memory consumption and run-time than most state-of-the-art learning-based methods.
Revised: Finally, we conclude that our method is more efﬁcient in memory consumption and run-time than most state-of-the-art learning-based methods.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: We propose a dynamic scale network CDSFNet for MVS task.
Revised: We propose a dynamic scale network CDSFNet for MVS task.

Original: CDSFNet can extract the discriminative features to improve the matching ability by analyzing the normal curvature of the image surface.
Revised: CDSFNet can extract the discriminative features to improve the matching ability by analyzing the normal curvature of the image surface.

Original: We then design an efﬁcient MVS architecture, CDS-MVSNet, which achieves state-of-the-art performance on various outdoor scenes
Revised: We then design an efﬁcient MVS architecture, CDS-MVSNet, which achieves state-of-the-art performance on various outdoor scenes

Original: A A PPENDIX A. D ERIVED FORMULAS OF NORMAL CURVATURE
Revised: A A PPENDIX A. D ERIVED FORMULAS OF NORMAL CURVATURE

Original: First, we re-derive Eq. 1 proposed by Xu et al.
Revised: First, we re-derive Eq. 1 proposed by Xu et al.

Original: The image surface can be represented by s ( x, y ) =
Revised: The image surface can be represented by s ( x, y ) =

Original: [ x, y, I ( x, y )] ,
Revised: [ x, y, I ( x, y )] ,

Original: where X = [ x, y ] is a pixel coordinate and I ( x, y ) is the color information of image.
Revised: where X = [ x, y ] is a pixel coordinate and I ( x y ) is the color information of image.

Original: The unit normal vector of surface s at point X is determined asn =
Revised: The unit normal vector of surface s at point X is determined as

Original: [Equation]
Revised: [Equation]

Original: y (cid:107) s x

Original: [Equation]

Original: s y (cid:107) =

Original: [-(cid:113) 
-]
Revised: [-(cid:113) 
-]

Original: + I 2 x + I 2 y [ − I x , − I y , 1] , where s x and s y are the ﬁrst partial derviatives with respect to x − axis and y − axis , ∧ is the cross-product operator.
Revised: where s x and s y are the ﬁrst partial derviatives with respect to x -axis and y -axis , ∧ is the crossproduct operator.

Original: Then, the ﬁrst and second fundamental form of surface s at point X can be written by
Revised: Then, the ﬁrst and second fundamental form of surface s at point X can be written by

Original: [Equation]
Revised: [Equation]

Original: Note that ” · ” is the dot-product operation between two vectors.
Revised: Note that ” · ” is the dot-product operation between two vectors.

Original: Finally, the normal curvature of s at point X along the direction ω
Revised: Finally, the normal curvature of s at point X along the direction ω

Original: =
Revised: =

Original: [ u, v ] T of epipolar line is deﬁned as
Revised: [ u, v ] T of epipolar line is deﬁned as

Original: [Equation]
Revised: [Equation]

Original: Because ω =
Revised: Because ω =

Original: [ u, v ] T is an unit vector, u 2 + v 2 = 1 .
Revised: [ u, v ] T is an unit vector, u 2 + v 2 = 1 .

Original: Therefore,
Revised: Therefore,

Original: [Equation]
Revised: [Equation]

Original: This formula can be applied to an image patch p with a scale σ centered at X .
Revised: This formula can be applied to an image patch p with a scale σ centered at X .

Original: Following the scale space theory (Lindeberg, 1994), the patch p can be regarded as a pixel when the image I is represented at the scale σ .
Revised: Following the scale space theory (Lindeberg, 1994), the patch p can be regarded as a pixel when the image I is represented at the scale σ .

Original: Let I ( X, σ ) be the image intensity at the scale σ .
Revised: Let I ( X, σ ) be the image intensity at the scale σ .

Original: I ( X, σ ) can be computed from I ( X ) by using a Gaussian kernel with size σ , I ( X, σ )
Revised: I ( X, σ ) can be computed from I ( X ) by using a Gaussian kernel with size σ , I ( X, σ )

Original: =
Revised: =

Original: I ( X ) ∗ G ( X, σ )
Revised: I ( X ) ∗ G ( X, σ )

Original: Therefore, we can ﬁnally write the normal curvature computed for X at the scale σ as
Revised: Therefore, we can ﬁnally write the normal curvature computed for X at the scale σ as

Original: [Equation]
Revised: [Equation]

Original: which was Eq.
Revised: which was Eq.

Original: 2 mentioned in Section 3.
Revised: 2 mentioned in Section 3.

Original: A. D ETAIL OF MVS ARCHITECTURE
Revised: A. D ETAIL OF MVS ARCHITECTURE

Original: Fig.
Revised: Fig.

Original: 4 depicts the multi-stage architecture of CDS-MVSNet.
Revised: 4 depicts the multi-stage architecture of CDS-MVSNet.

Original: First, CDS-MVSNet uses the feature extraction CDSFNet to extract outputs at three resolutions { 18 , 14 , 12 } corresponding to the ﬁrst three stages { 0 , 1 , 2 } .
Revised: First, CDS-MVSNet uses the feature extraction CDSFNet to extract outputs at three resolutions { 18 , 14 , 12 } corresponding to the ﬁrst three stages { 0 , 1 , 2 } .

Original: For each of these stages l ∈ { 0 , 1 , 2 } , CDS-MVSNet predicts the depth at the corresponding image-resolution W 2 3 − l × H 2 3 − l by using three-step MVS as described in section 4.1. 
Revised: For each of these stages l ∈ { 0 , 1 , 2 } , CDS-MVSNet predicts the depth at the corresponding image-resolution W 2 3 − l × H 2 3 − l by using three-step MVS as described in section 4.1. CDS-MVSNet also uses an independent 3D CNN for each stage to regularize 3D cost volume.

Original: The estimated depth of current stage is upsampled and used to reduce the depth hypothesis range of 3D cost volume in the next stage.
Revised: The estimated depth of current stage is upsampled and used to reduce the depth hypothesis range of 3D cost volume in the next stage.

Original: For the last stage l = 3 , CDS-MVSNet outputs the full-resolution depths byperforming depth reﬁnement from the upsampled depth and the color image.
Revised: For the last stage l = 3 , CDS-MVSNet outputs the full-resolution depths by

Original: CDSFNet Given a reference image I 0 and N source images { I 1 , I 2 , ..., I N } with corresponding camera parameters { Q 0 , ..., Q N } , we can compute an epipole pair { ( e 0 ,i , e i ) } Ni =1 for each reference-source image pair { ( I 0 , I i ) } Ni =1 based on the epipolar geometry.
Revised: CDSFNet Given a reference image I 0 and N source images { I 1 , I 2 , ..., I N } with corresponding camera parameters { Q 0 , ..., Q N } , we can compute an epipole pair { ( e 0 ,i , e i ) } Ni =1 for each reference-source image pair { ( I 0 , I i ) } Ni =1 based on the epipolar geometry.

Original: We use the epipole to determine the epipolar line for each pixel.
Revised: We use the epipole to determine the epipolar line for each pixel.

Original: Therefore, given the inputs including image I and epipole e ,
Revised: Therefore, given the inputs including image I and epipole e ,

Original: CDSFNet extracts three feature maps corresponding to the three levels { 0 , 1 , 2 } of CDS-MVSNet.
Revised: CDSFNet extracts three feature maps corresponding to the three levels { 0 , 1 , 2 } of CDS-MVSNet.

Original: Fig.
Revised: Fig.

Original: 5 describes the Unet-liked architecture of CDSFNet.
Revised: 5 describes the Unet-liked architecture of CDSFNet.

Original: Fig.
Revised: Fig.

Original: 6 and 7 show the results of dynamic scale estimation and normal curvature estimation respectively.
Revised: 6 and 7 show the results of dynamic scale estimation and normal curvature estimation respectively.

Original: Depth Reﬁnement.
Revised: Depth Reﬁnement.

Original: Instead of using three-step MVS depth estimation as described for the ﬁnest resolution level (stage 3), we directly up-sample the depth estimated in previous stage (from resolution W 2 × H 2 to W × H )
Revised: Instead of using three-step MVS depth estimation as described for the ﬁnest resolution level (stage 3), we directly up-sample the depth estimated in previous stage (from resolution W 2 × H 2 to W × H )

Original: and reﬁne the upsampled depth with the RGB image.
Revised: and reﬁne the upsampled depth with the RGB image.

Original: Following MSG-Net (Hui et al., 2016) and PatchMatchNet (Wang et al., 2021), we implement a depth residual network to perform reﬁnement.
Revised: Following MSG-Net (Hui et al., 2016) and PatchMatchNet (Wang et al., 2021), we implement a depth residual network to perform reﬁnement.

Original: The network use a 2D CNN to output a residual from the upsampled depth.
Revised: The network use a 2D CNN to output a residual from the upsampled depth.

Original: This residual depth is then added to the up-sampled depth, to get the reﬁned depth map D ref .
Revised: This residual depth is then added to the up-sampled depth, to get the reﬁned depth map D ref .

Original: A. I MPLEMENTATION D ETAILS
Revised: A. I MPLEMENTATION D ETAILS

Original: Training.
Revised: Training.

Original: We applied the same image-resolution 640 × 512 in both DTU and BlendedMVS datasets.
Revised: We applied the same image-resolution 640 × 512 in both DTU and BlendedMVS datasets.

Original: The preprocessing of these datasets for view selection strategy and ground-truth depth maps was provided by Yao et al.
Revised: The preprocessing of these datasets for view selection strategy and ground-truth depth maps was provided by Yao et al.

Original: For CDS-MVSNet, we set the number of input views to 3, the number of depth hypothesis planes to { 48 , 32 , 8 } with the corresponding depth interval scales { 4 , 2 , 1 } .
Revised: For CDS-MVSNet, we set the number of input views to 3, the number of depth hypothesis planes to { 48 , 32 , 8 } with the corresponding depth interval scales { 4 , 2 , 1 } .

Original: To train CDFSNet effectively we used the feature loss as mentioned in Section 4.2.
Revised: To train CDFSNet effectively, we used the feature loss as mentioned in Section 4.2.

Original: Besides, we implemented the scale selection step of CDSConv by ﬁrst initializing the Softmax temperature by1 .
Revised: Besides, we implemented the scale selection step of CDSConv by ﬁrst initializing the Softmax temperature by

Original: This hyperparameter was then decreased over training time until it reached 0 .
Revised: This hyperparameter was then decreased over training time until it reached 0 .

Original: 01 to produce the sparse output for scale selection.
Revised: 01 to produce the sparse output for scale selection.

Original: The entire network CDS-MVSNet was trained with 30 epochs, using the SGD optimizer with an initial learning rate of 0 .  and on two NVIDIA Titan V GPU with a batch size of 6 . 
Revised: The entire network CDS-MVSNet was trained with 30 epochs, using the SGD optimizer with an initial learning rate of 0 . 01 , and on two NVIDIA Titan V GPU with a batch size of 6 . We provided two experimental scenarios.

Original: Note that we trained thesingle model using both DTUand Blended datasets instead of training individually on each dataset as in previous methods (Zhang et al., 2020; Luo et al., 2020; Sormann et al., 2020).
Revised: First, we trained the model on DTU training set and then evaluated on DTU test set. This pre-trained model is then ﬁne-tuned on BlendedMVS dataset for 15 epochs with a learning rate of 10 − 4 to evaluate on Tanks & Temples.

Original:  This procedure prevented the model from biasing to each dataset in the evaluation step; it guaranteed the generalized property of model.
Revised: Second, we trained a single model on both DTU and BlendedMVS datasets and use this model for evaluation. This training strategy can improve the performance as indicated in Table 1 and

Original: Point cloud generation.
Revised: Point cloud generation.

Original: To reconstruct the ﬁnal point cloud, we ﬁltered out the low conﬁdence depths based on the probability maps estimated from the depth probability volumes of all stages (Zhang et al., 2020).
Revised: To reconstruct the ﬁnal point cloud, we ﬁltered out the low conﬁdence depths based on the probability maps estimated from the depth probability volumes of all stages (Zhang et al., 2020).

Original: Similar to (Gu et al., 2020; Zhang et al., 2020), we then integrated all the depth maps based on the fusion method proposed in (Galliani et al., 2015; Yao et al., 2018). 
Revised: Similar to (Gu et al., 2020; Zhang et al., 2020), we then integrated all the depth maps based on the fusion method proposed in (Galliani et al., 2015; Yao et al., 2018). A. A BLATION STUDY

Revised: Effectiveness of learnable curvature.

Revised: We demonstrate the advantage of our proposed learnable curvature in comparison to the original curvature (Xu et al., 2020).

Revised: We replace the curvature estimation in Eq.5 with Eq.1 and then re-train the MVS network on the DTU dataset with identical setups.

Original: [Figure]
Revised: Table 4 presents four model variants denoted as A, B, C, and D. Each variant implements a speciﬁc version of curvature (original or learnable) and then uses the visibility aggragation with or without the curvature prior. We consider a comparison of the two groups, models A and B versus models C and D. To evaluate the depth maps, we measure the percentage depth precision with the thresholds 2 , 4 mm and Mean Absolute Error (MAE).

Revised: To evaluate the reconstructed models, we use the accuracy, completeness, and overall metrics.

Revised: We also provide the runtime and memory consumption to evaluate the computational efﬁciency.

Revised: Table 4 indicates that the models with learnable curvature (C and D) achieve a better performance than the original curvature (A and B) for all evaluation metrics.

Revised: The learnable curvature has a low computation cost and boosts the quality of depth and reconstruction signiﬁcantly.

Revised: The original

Revised: [Table]

Revised: curvature degrades the performance because it cannot handle the high-dimensional features.

Revised: The original method treats each feature channel equally and averages all feature channels when computing the derivatives by Gaussian ﬁlters.

Revised: Therefore, it may produce a high curvature estimation even though the neighboring feature vectors are visually similar to each other, as shown in Fig. 8.

Revised: Fig.

Revised: 8 shows the normal curvatures estimated by the original method (Xu et al., 2020) and our learnable method.

Revised: The curvature maps are extracted from the ﬁrst four CDSConv layers of CDSFNet.

Revised: As shown in Fig.

Revised: 8, the original method provides a ground truth estimation only in the ﬁrst layer when the input is a color image (i.e., the estimated curvatures are small in untextured regions while large in near-edge or rich-textured regions).

Revised: However, it produces noisy estimation when the inputs are the feature maps in the subsequent layers, especially in the third and fourth layers where the curvatures do not reﬂect the edge or texture information.

Revised: On the other hand, although the learnable curvature approximates the ground truth (the original curvature in the ﬁrst layer), it consistently preserves the properties of normal curvature concerning the untextured and rich-textured regions in every layer.

Revised: We further notice that our learning-based estimation takes advantage of the attention mechanism, as shown in the red circles.

Revised: It focuses on the rich texture region to improve the completeness of reconstruction, which is illustrated in Fig.

Revised: 9 in Appendix A.5.

Revised: Fig.

Revised: 8 also implies that the estimated curvature tends to be higher at the deeper layers because it encodes a larger patch scale.

Revised: Curvature-guided visibility aggregation.

Revised: We proposed a pixel-wise visibility prediction method for cost aggregation in Section 4.1.

Revised: Previous works generally predicted the visibility weights only from the two-view cost volume (Zhang et al., 2020; Xu & Tao, 2020c).

Revised: Compared to these works, our method additionally considers the curvature prior estimated by CDSFNet.

Revised: To evaluate the effectiveness of curvature prior to visibility prediction, we compared the performance of our method with the baseline method Zhang et al.

Revised: To implement the baseline method, we removed the curvature input from our visibility prediction network and only used the entropy information of two view cost volumes.

Revised: We compared the models using curvature prior (Models B and D) with those without using it (Models A and C).

Revised: Table 4 shows the experimental results.

Revised: As shown in Table 4,

Revised: Model B had the worst performance in terms of overall.

Revised: Because Model B produced the noisy curvature, this noisy estimation degraded the performance of Model B compared to Model A. Model

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: D remarkably improves the depth estimation performances remarkably and achieves the similar performance of reconstruction compared to Model C without increasing the runtime and memory consumption.

Revised: These results demonstrate the effectiveness of the proposed learnable curvature and curvature-guided visibility prediction in the MVS process

Revised: [Table]

Revised: Analysis of CDSFNet and the cascade structure.

Revised: This section ﬁrst analyzes the effectiveness of proposed CDSFNet when increasing the number of candidate scales.

Revised: To measure explicitly the matchability of features extracted by CDSFNet, we remove the cost-volume regularization step in the MVS architecture and then regress the depth directly from the aggregated 3D cost volume.

Revised: Table 5 presents the results when increasing the number candidate scales N c in each CDSConv layer.

Revised: When N c is equal to 1, CDSFNet is considered as Feature Pyramid Network (FPN), which is applied to most of the state-of-the-art learning-based methods (Gu et al., 2020; Zhang et al., 2020).

Revised: As shown in Table 5, CDSFNet achieved better performance when using more candidate scales.

Revised: However, it suffered from heavy computation.

Revised: Therefore, we chose the design with two or three scales for each CDSConv (CDSFNet architecture as shown in Fig. 5), to guarantee both the performance of reconstruction and computational efﬁciency.

Revised: Next, we investigate the improvement of depth estimation over the cascade stages.

Revised: Table 6 provides the accuracy of depth estimation over all stages of our CDS-MVSNet.

Revised: The ﬁrst three stages mainly contribute to the ﬁnal quality of estimated depth.

Revised: The last stage l = 3 is used for depth upsampling; it outputs the full-resolution depth while maintaining the same depth accuracy compared to the previous stage.

Revised: Suppose this stage is implemented by the three-step MVS depth estimation pipeline mentioned in Section 4.1, the entire MVS network suffers from a considerable computation and cannot ﬁt into a limited GPU memory.

Revised: Therefore, we only apply the depth reﬁnement with a 2D

Revised: CNN to the last stage for efﬁciency in time and memory.

Revised: [Figure]

Revised: [Figure]

Original: Q UALITATIVE RESULTS ON DTU AND T ANKS & T EMPLES
Revised: Q UALITATIVE RESULTS ON DTU AND T ANKS & T EMPLES