Original: Revisiting Non-Parametric Matching Cost Volumes for Robust and Generalizable Stereo Matching
Revised: Revisiting Non-Parametric Matching Cost Volumes for Robust and Generalizable Stereo Matching

Original: Abstract
Revised: Abstract

Original: Stereo matching is a classic challenging problem in computer vision, which has recently witnessed remarkable progress by Deep Neural Networks (DNNs).
Revised: Stereo matching is a classic challenging problem in computer vision, which has recently witnessed remarkable progress by Deep Neural Networks (DNNs).

Original: This paradigm shift leads to two interesting and entangled questions that have not been addressed well.
Revised: This paradigm shift leads to two interesting and entangled questions that have not been addressed well.

Original: First , it is unclear whether stereo matching DNNs that are trained from scratch really learn to perform matching well.
Revised: First , it is unclear whether stereo matching DNNs that are trained from scratch really learn to perform matching well.

Original: This paper studies this problem from the lens of white-box adversarial attacks.
Revised: This paper studies this problem from the lens of white-box adversarial attacks.

Original: It presents a method of learning stereo-constrained photometrically-consistent attacks, which by design are weaker adversarial attacks, and yet can cause catastrophic performance drop for those DNNs.
Revised: It presents a method of learning stereo-constrained photometrically-consistent attacks, which by design are weaker adversarial attacks, and yet can cause catastrophic performance drop for those DNNs.

Original: This observation suggests that they may not actually learn to perform matching well in the sense that they should otherwise achieve potentially even better after stereo-constrained perturbations are introduced.
Revised: This observation suggests that they may not actually learn to perform matching well in the sense that they should otherwise achieve potentially even better after stereo-constrained perturbations are introduced.

Original: Second , stereo matching
Revised: Second , stereo matching

Original: DNNs are typically trained under the simulation-to-real (Sim2Real) pipeline due to the data hungriness of DNNs.
Revised: DNNs are typically trained under the simulation-to-real (Sim2Real) pipeline due to the data hungriness of DNNs.

Original: Thus, alleviating the impacts of the Sim2Real photometric gap in stereo matching DNNs becomes a pressing need.
Revised: Thus, alleviating the impacts of the Sim2Real photometric gap in stereo matching DNNs becomes a pressing need.

Original: Towards joint adversarially robust and domain generalizable stereo matching, this paper proposes to learn DNN-contextualized binary-pattern-driven non-parametric costvolumes .
Revised: Towards joint adversarially robust and domain generalizable stereo matching, this paper proposes to learn DNN-contextualized binary-pattern-driven non-parametric costvolumes .

Original: It leverages the perspective of learning the cost aggregation via DNNs, and presents a simple yet expressive design that is fully end-to-end trainable, without resorting to specific aggregation inductive biases.
Revised: It leverages the perspective of learning the cost aggregation via DNNs, and presents a simple yet expressive design that is fully end-to-end trainable, without resorting to speciﬁc aggregation inductive biases.

Original: In experiments, the proposed method is tested in the SceneFlow dataset, the KITTI2015 dataset, and the Middlebury dataset.
Revised: In experiments, the proposed method is tested in the SceneFlow dataset, the KITTI2015 dataset, and the Middlebury dataset.

Original: It significantly improves the adversarial robustness, while retaining accuracy performance comparable to state-of-the-art methods.
Revised: It signiﬁcantly improves the adversarial robustness, while retaining accuracy performance comparable to state-of-the-art methods.

Original: It also shows a better Sim2Real generalizability.
Revised: It also shows a better Sim2Real generalizability.

Original: Our code and pretrained models are released at this Github Repo.
Revised: Our code and pretrained models are released at this Github Repo.

Original:  Introduction
Revised:  Introduction

Original: Stereo matching remains a long-standing problem in computer vision that has been studied for several decades.
Revised: Stereo matching remains a long-standing problem in computer vision that has been studied for several decades.

Original: As shown in the left of Fig.
Revised: As shown in the left of Fig.

Original: 1, given a pair of (rectified) stereo images, the reference left image I L and the right image I R , for a pixel I L ( x, y ) , its corresponding pixel in I R is constrained to be at I R ( x − D ( x, y ) , y ) , where D ( x, y ) is called the disparity of the pixel ( x, y ) in the reference left image.
Revised: 1, given a pair of (rectiﬁed) stereo images, the reference left image I L and the right image I R , for a pixel I L ( x, y ) , its corresponding pixel in I R is constrained to be at I R ( x  D ( x, y ) , y ) , where D ( x, y ) is called the disparity of the pixel ( x, y ) in the reference left image.

Original: The disparity is inversely proportional to the depth Z ( x, y ) , D ( x, y )
Revised: The disparity is inversely proportional to the depth Z ( x, y ) , D ( x, y )

Original: = f · b Z ( x,y ) , where f is the focal length of the left camera and b is the baseline (i.e., the distance between the two cameras).
Revised: = f · b Z ( x,y ) , where f is the focal length of the left camera and b is the baseline (i.e., the distance between the two cameras).

Original: High-performing stereo matching will enable highly reliable and cost effective depth estimation from stereo images without leveraging expensive Lidar sensors, which has great potential in a wide range of applications such as autonomous driving and robot autonomy.
Revised: High-performing stereo matching will enable highly reliable and cost effective depth estimation from stereo images without leveraging expensive Lidar sensors, which has great potential in a wide range of applications such as autonomous driving and robot autonomy.

Original: To infer the disparity, the basic approach is to first compute the matching costs between I L ( x, y ) and I R ( x − d, y ) for different values of the disparity d and then find the best match that corresponds to the minimum cost.
Revised: To infer the disparity, the basic approach is to ﬁrst compute the matching costs between I L ( x, y ) and I R ( x  d, y ) for different values of the disparity d and then ﬁnd the best match that corresponds to the minimum cost.

Original: Due to occlusions and textureless regions, the pixel correspondences can not be solved individually and the disparity map needs to be inferred in its entirety via prior-constrained (e.g., local disparity 
Revised: Due to occlusions and textureless regions, the pixel correspondences can not be solved individually and the disparity map needs to be inferred in its entirety via prior-constrained (e.g., local disparity 36th Conference on

Revised: Information Processing Systems (NeurIPS 2022).

Original: [Figure]
Revised: [Figure]

Original: smoothness) global matching cost optimization, which remains a challenging problem.
Revised: smoothness) global matching cost optimization, which remains a challenging problem.

Original: Stereo matching has recently witnessed remarkable progress using Deep Neural Networks (DDNs).
Revised: Stereo matching has recently witnessed remarkable progress using Deep Neural Networks (DDNs).

Original: As shown in the right-bottom of Fig.
Revised: As shown in the right-bottom of Fig.

Original: 1, DNN-based stereo matching methods amortize the need of inducing inductive biases and of designing global matching cost optimization algorithms, which often consist of four components (see Appendix.
Revised: 1, DNN-based stereo matching methods amortize the need of inducing inductive biases and of designing global matching cost optimization algorithms, which often consist of four components (see Appendix.

Original: A)
Revised: A)

Original: For the end-to-end fully differentiable learning of stereo matching in the prior art, two interesting questions arise: First , how well do DNNs that are trained from scratch learn to match ?
Revised: For the end-to-end fully differentiable learning of stereo matching in the prior art, two interesting questions arise: First , how well do DNNs that are trained from scratch learn to match ?

Original: We can do some deductive reasoning: If they indeed learn to perform matching, their performance should potentially increase after we add the same perturbations at I L ( x, y ) and I R ( x − D ( x, y ) , y ) using the ground-truth disparity D ( x, y )
Revised: We can do some deductive reasoning: If they indeed learn to perform matching, their performance should potentially increase after we add the same perturbations at I L ( x, y ) and I R ( x  D ( x, y ) , y ) using the ground-truth disparity D ( x, y )

Original: (i.e., the stereo-constrained photometric consistency), regardless of what the perturbations are (e.g., adversarial attacks).
Revised: (i.e., the stereo-constrained photometric consistency), regardless of what the perturbations are (e.g., adversarial attacks).

Original: As we shall show, DNNs do not learn to match well and state-of-the-art stereo matching DNNs are vulnerable even when the photometric consistency is preserved against adversarial attacks (Section 2).
Revised: As we shall show, DNNs do not learn to match well and state-of-the-art stereo matching DNNs are vulnerable even when the photometric consistency is preserved against adversarial attacks (Section 2).

Original: Second , parallel to the adversarial vulnerability, cross-domain generalizabilty also is an important problem in stereo matching: DNN-based stereo matching is typically pre-trained under the so-called simulation to reality (Sim2Real) pipeline due to the high cost of collecting ground-truth matching results in practice and the data-hungry aspect of
Revised: Second , parallel to the adversarial vulnerability, cross-domain generalizabilty also is an important problem in stereo matching: DNN-based stereo matching is typically pre-trained under the so-called simulation to reality (Sim2Real) pipeline due to the high cost of collecting ground-truth matching results in practice and the data-hungry aspect of

Original: DNNs.
Revised: DNNs.

Original: It has been shown that DNNs may learn shortcut solutions that are strongly biased by the training dataset [2].
Revised: It has been shown that DNNs may learn shortcut solutions that are strongly biased by the training dataset [2].

Original: At the same time, it is desirable to have stereo matching systems that are more directly transferable from the simulation (training) to the reality (testing).
Revised: At the same time, it is desirable to have stereo matching systems that are more directly transferable from the simulation (training) to the reality (testing).

Original: Since stereo matching methods are widely used in autonomous driving, adversarial vulnerabilities in these models can lead to catastrophic consequences, and boosting the Sim2Real generalizability can significantly improve the applicability in diverse driving scenarios.
Revised: Since stereo matching methods are widely used in autonomous driving, adversarial vulnerabilities in these models can lead to catastrophic consequences, and boosting the Sim2Real generalizability can signiﬁcantly improve the applicability in diverse driving scenarios.

Original: Thus, jointly addressing adversarial vulnerabilities and the Sim2Real generalizability has become a pressing need in DNN-based stereo matching, as well as many other deep learning applications.
Revised: Thus, jointly addressing adversarial vulnerabilities and the Sim2Real generalizability has become a pressing need in DNN-based stereo matching, as well as many other deep learning applications.

Original: To address the two questions jointly in stereo matching, one key is to enable learning to really perform matching based on features that are robust and generalizable between the simulated data (training distributions) and real data (testing distributions).
Revised: To address the two questions jointly in stereo matching, one key is to enable learning to really perform matching based on features that are robust and generalizable between the simulated data (training distributions) and real data (testing distributions).

Original: To that end, this paper proposes DNNcontextualized binary-pattern-driven non-parametric cost-volumes .
Revised: To that end, this paper proposes DNNcontextualized binary-pattern-driven non-parametric cost-volumes .

Original: On top of that, it revisits the perspective of learning the cost aggregation via DNNs for stereo matching, and presents a simple yet expressive design that is fully end-to-end trainable, without resorting to specific aggregation inductive biases.
Revised: On top of that, it revisits the perspective of learning the cost aggregation via DNNs for stereo matching, and presents a simple yet expressive design that is fully end-to-end trainable, without resorting to speciﬁc aggregation inductive biases.

Original: We briefly summarize the proposed methods as follows.
Revised: We brieﬂy summarize the proposed methods as follows.

Original: To defend against adversarial attacks, most methods rely on adversarial training [3], often at the expense of decreasing performance on clean images, long training time, and potential over-fitting to specific attacks and datasets ( e .
Revised: To defend against adversarial attacks, most methods rely on adversarial training [3], often at the expense of decreasing performance on clean images, long training time, and potential over-ﬁtting to speciﬁc attacks and datasets ( e .

Original: g .
Revised: g .

Original: cannot transfer the robustness to other datasets as shown in the later sections).
Revised: cannot transfer the robustness to other datasets as shown in the later sections).

Original: In contrast, we propose to utilize domain-specific knowledge to facilitate the built-in robustness of the neural networks.
Revised: In contrast, we propose to utilize domain-speciﬁc knowledge to facilitate the built-in robustness of the neural networks.

Original: Because of the strong photometric consistency between stereo images , stereo matching provides an ideal case to defend against adversarial attacks through the design of the neural network (see the right-top of Fig. 1, and Fig.
Revised: Because of the strong photometric consistency between stereo images , stereo matching provides an ideal case to defend against adversarial attacks through the design of the neural network (see the right-top of Fig. 1, and Fig.

Original: 5 and Table 7 in the Appendix).
Revised: 5 and Table 7 in the Appendix).

Original: For non-occluded regions in stereo images, the corresponding pixels of the same physical point have similar colors.
Revised: For non-occluded regions in stereo images, the corresponding pixels of the same physical point have similar colors.

Original: We observe that by using DNN features for matching, attacks will increase the matching costs for features that belong to the same physical point (see the appendix for details).
Revised: We observe that by using DNN features for matching, attacks will increase the matching costs for features that belong to the same physical point (see the appendix for details).

Original: Therefore, we propose to remove DNN features for matching and use hand-crafted features that will preserve the color differences to construct the cost volume, then the DNN is used to aggregate/optimize a non-parametric cost volume.
Revised: Therefore, we propose to remove DNN features for matching and use hand-crafted features that will preserve the color differences to construct the cost volume, then the DNN is used to aggregate/optimize a non-parametric cost volume.

Original: To further make the cost as hard to alter as possible, we use local binary patterns that compare each pixel intensity to their neighbors (i.e., Census Transform [1, 4]) as the feature descriptor.
Revised: To further make the cost as hard to alter as possible, we use local binary patterns that compare each pixel intensity to their neighbors (i.e., Census Transform [1, 4]) as the feature descriptor.

Original: For occluded regions, the photometric consistency does not hold.
Revised: For occluded regions, the photometric consistency does not hold.

Original: For textureless non-occluded regions , the local photometric features are weak for matching, even though the underlying consistency is retained.
Revised: For textureless non-occluded regions , the local photometric features are weak for matching, even though the underlying consistency is retained.

Original: Thus, high-level semantic information learned by
Revised: Thus, high-level semantic information learned by

Original: DNN features will be entailed.
Revised: DNN features will be entailed.

Original: We use a DNN feature backbone for the reference (left) image only to contextualize the feature volume, which, albeit being fully differentiable, turns out not to hurt the adversarial robustness (see Sec. 4.2).
Revised: We use a DNN feature backbone for the reference (left) image only to contextualize the feature volume, which, albeit being fully differentiable, turns out not to hurt the adversarial robustness (see Sec. 4.2).

Original: This shows that the majority of the vulnerability actually comes from the matching part rather than the end-to-end trainable contextual information.
Revised: This shows that the majority of the vulnerability actually comes from the matching part rather than the end-to-end trainable contextual information.

Original: The non-parametric feature volume and the contextual DNN features will be fed through a head sub-network playing the role of a learnable optimizer that seeks the best matching result (Fig. 1).
Revised: The non-parametric feature volume and the contextual DNN features will be fed through a head sub-network playing the role of a learnable optimizer that seeks the best matching result (Fig. 1).

Original: In essence, we cast stereo matching as a cost aggregation/optimization problem over a non-parametric cost volume with parametric contextual features.
Revised: In essence, we cast stereo matching as a cost aggregation/optimization problem over a non-parametric cost volume with parametric contextual features.

Original: In experiments, we show that this more transparent approach improves adversarial robustness significantly while maintaining high accuracy.
Revised: In experiments, we show that this more transparent approach improves adversarial robustness signiﬁcantly while maintaining high accuracy.

Original: In regards of boosting the Sim2Real generalizability, removing the DNN feature backbone for matching and utilizing the binary patterns of CT will induce the DNN to be a more general cost volume optimizer, thus alleviating the opportunity of shortcut learning in the feature space and resulting in better performance in cross-domain deployment, especially when no fine-tuning is used.
Revised: In regards of boosting the Sim2Real generalizability, removing the DNN feature backbone for matching and utilizing the binary patterns of CT will induce the DNN to be a more general cost volume optimizer, thus alleviating the opportunity of shortcut learning in the feature space and resulting in better performance in cross-domain deployment, especially when no ﬁne-tuning is used.

Original: These are verified in our experiments from the SceneFlow dataset [5] to the KITTI benchmark [6] and the MiddleBury dataset [7] when no fine-tuning is used.
Revised: These are veriﬁed in our experiments from the SceneFlow dataset [5] to the KITTI benchmark [6] and the MiddleBury dataset [7] when no ﬁne-tuning is used.

Original: With a comprehensive study of the proposed method (Section 4), we have five observations as follows : (1) By removing feature backbones (i.e., (ii) in the right-bottom of Fig. 1) that are trained on certain datasets and using non-parametric cost volumes as inputs, we can improve the cross-domain genearlizability through training the DNN as a general optimizer.
Revised: With a comprehensive study of the proposed method (Section 4), we have ﬁve observations as follows : (1) By removing feature backbones (i.e., (ii) in the right-bottom of Fig. 1) that are trained on certain datasets and using non-parametric cost volumes as inputs, we can improve the cross-domain genearlizability through training the DNN as a general optimizer.

Original: (2) Even without contextual information (i.e., the component shown in dashed arrows and boxes in the right-top of Fig. 1), DNNs can do reasonably well on aggregating/optimizing the raw cost volume, which is nontrivial due to the loss of image information.
Revised: (2) Even without contextual information (i.e., the component shown in dashed arrows and boxes in the right-top of Fig. 1), DNNs can do reasonably well on aggregating/optimizing the raw cost volume, which is nontrivial due to the loss of image information.

Original: The vulnerability mostly comes from the matching part instead of the learned contextual information.
Revised: The vulnerability mostly comes from the matching part instead of the learned contextual information.

Original: (4) Since the vanilla CT can block the gradients in learning attacks, the obfuscated gradient problem [8] of CT is studied using a differentiable approximation.
Revised: (4) Since the vanilla CT can block the gradients in learning attacks, the obfuscated gradient problem [8] of CT is studied using a differentiable approximation.

Original: With this approximation, the overall DNN output can still be altered, showing the internal vulnerability of
Revised: With this approximation, the overall DNN output can still be altered, showing the internal vulnerability of

Original: DNNs.
Revised: DNNs.

Original: The robustness of adversarial training in stereo matching does not transfer well to other datasets or attacks, thus inducing the built-in robustness is very important, as done by the CT based non-parameteric cost volume in our proposed method (Section 3.2).
Revised: The robustness of adversarial training in stereo matching does not transfer well to other datasets or attacks, thus inducing the built-in robustness is very important, as done by the CT based non-parameteric cost volume in our proposed method (Section 3.2).

Original: Our Contributions.
Revised: Our Contributions.

Original: This paper makes two main contributions to the field of stereo matching: (i) It proposes a novel design for stereo matching by utilizing DNNs to aggregate/optimize non-parametric cost volumes with parametric contextual features, which shows significantly better adversarial robustness and improved cross-domain (Sim2Real) generalizability when no-fine tuning is used.
Revised: This paper makes two main contributions to the ﬁeld of stereo matching: (i) It proposes a novel design for stereo matching by utilizing DNNs to aggregate/optimize non-parametric cost volumes with parametric contextual features, which shows signiﬁcantly better adversarial robustness and improved cross-domain (Sim2Real) generalizability when no-ﬁne tuning is used.

Original: (ii) It presents the stereo-constrained projected gradient descent (PGD) attack method, which by design preserves photometric consistency to show the more serious vulnerabilities of state-of-the-art
Revised: (ii) It presents the stereo-constrained projected gradient descent (PGD) attack method, which by design preserves photometric consistency to show the more serious vulnerabilities of state-of-the-art

Original: DNN-based stereo matching methods.
Revised: DNN-based stereo matching methods

Original:  Attacking Stereo Matching Deep Neural Networks 2.1 The Proposed Stereo-Constrained PGD Attacks
Revised:  Attacking Stereo Matching Deep Neural Networks 2.1 The Proposed Stereo-Constrained PGD Attacks

Original: To study the brittleness of DNN based stereo matching models, we intentionally develop a physically realizable attacking method based on the widely-used white-box PGD method [3], which retains the underlying photometric consistency in stereo matching by changing the pixel values of the same physical point in both images.
Revised: To study the brittleness of DNN based stereo matching models, we intentionally develop a physically realizable attacking method based on the widely-used white-box PGD method [3], which retains the underlying photometric consistency in stereo matching by changing the pixel values of the same physical point in both images.

Original: Given a perturbation map P ( x, y ) , ( x, y ) ∈ Λ ( Λ denotes the image[Table] +adv. 0. 6.07 Oursw/octx.+adv. 0.
Revised: Given a perturbation map P ( x, y ) , ( x, y ) 2 ⇤ ( ⇤ denotes the image

Original: 7.51 Ours+adv.

Original: 0.41 ( ↑ 0.

Original: [Table]

Original: [Table]

Original: ( ↑ 0.04)

Original: [Table]
Revised: [Table]

Original: .69 ( ↑

Original: [Table][Table]
Revised: [Table]

Original: 88 ( ↑ 0.17)

Original: [Table] [Table] [Table] [Table]
Revised: [Figure]

Original: [Table]

Original: [Figure] [Table] [Figure] lattice), the distorted pixel values for each pixel location ( x, y ) are computed as:
Revised: lattice), the distorted pixel values for each pixel location ( x, y ) are computed as:

Original: [Equation]
Revised: [Equation]

Original: where D ( x, y ) is the ground-truth disparity map, and occluded areas will not be modified.
Revised: where D ( x, y ) is the ground-truth disparity map, and occluded areas will not be modiﬁed.

Original: Since the left image is the reference image for computing the disparity loss, we disallow to attack and evaluate occluded regions of the reference image.
Revised: Since the left image is the reference image for computing the disparity loss, we disallow to attack and evaluate occluded regions of the reference image.

Original: We note that by excluding occluded pixels, our goal is to show that the matching component is still vulnerable even with this constraint, and thus seeking more robust matching components is a pressing need in robust stereo matching.
Revised: We note that by excluding occluded pixels, our goal is to show that the matching component is still vulnerable even with this constraint, and thus seeking more robust matching components is a pressing need in robust stereo matching.

Original: We also provide experiments on unconstrained attacks for comparisons.
Revised: We also provide experiments on unconstrained attacks for comparisons.

Original: Intuitively, this is a weaker attack method by design compared with the unconstrained counterpart, since it actually leaks the ground-truth disparity information to the algorithm using the pair ( P ( x, y ) , P ( x − D ( x, y ) , y )) which may potentially help stereo matching, instead of attacking it.
Revised: Intuitively, this is a weaker attack method by design compared with the unconstrained counterpart, since it actually leaks the ground-truth disparity information to the algorithm using the pair ( P ( x, y ) , P ( x  D ( x, y ) , y )) which may potentially help stereo matching, instead of attacking it.

Original: Consider two corresponding patches with constant disparity on the left and the right images containing the same physical points, the absolute sum of difference between these two patches will remain the same after the attack.
Revised: Consider two corresponding patches with constant disparity on the left and the right images containing the same physical points, the absolute sum of difference between these two patches will remain the same after the attack.

Original: The proposed stereo-constrained attacks uses the ground-truth disparity map in generating the perturbations, which is designed (i) to understand better the underlying vulnerability in stereo matching methods, and (ii) to simulate the physical scenarios, as the adversarial stereo-constrained patch attack experiments shown in Section 2.2.
Revised: The proposed stereo-constrained attacks uses the ground-truth disparity map in generating the perturbations, which is designed (i) to understand better the underlying vulnerability in stereo matching methods, and (ii) to simulate the physical scenarios, as the adversarial stereo-constrained patch attack experiments shown in Section 2.2.

Original: We use the L ∞ norm to measure similarities between images.
Revised: We use the L 1 norm to measure similarities between images.

Original: Two images will appear visually identical under a certain threshold.
Revised: Two images will appear visually identical under a certain threshold.

Original: To learn a L ∞ bounded adversarial perturbation P adv , the iterative
Revised: To learn a L 1 bounded adversarial perturbation P adv , the iterative

Original: PGD method is used,
Revised: PGD method is used,

Original: [Equation]
Revised: [Equation]

Original: where t = 0 , 1 , 2 , · · · , T and P advstarts with all zeros.
Revised: where t = 0 , 1 , 2 , · · · , T and P advstarts with all zeros.

Original: ℓ ( · ) denotes the mean absolute error for a perturbation P advt between the predicted disparity map for the perturbed images (Eqn. 1) and the ground-truth disparity map.
Revised: ` ( · ) denotes the mean absolute error for a perturbation P advt between the predicted disparity map for the perturbed images (Eqn. 1) and the ground-truth disparity map.

Original: And, clip ϵP clips the perturbation to be within the ϵ -ball of the corresponding zero-plane and the maximum color range.
Revised: And, clip ✏ P clips the perturbation to be within the ✏ -ball of the corresponding zero-plane and the maximum color range.

Original: In our experiments, we set ϵ = 0 .
Revised: In our experiments, we set ✏ = 0 .

Original: 06 or 0 .(the larger it is, the stronger the attack is), α = 0 .
Revised: 06 or 0 .(the larger it is, the stronger the attack is), ↵ = 0 .

Original: 01 and T = 20 .
Revised: 01 and T = 20 .

Original: Appendix C.1 shows a toy example.
Revised: Appendix C.1 shows a toy example.

Original: 2.2 Attacking Results in KITTI 
Revised: 2.2 Attacking Results in KITTI

Original: We test the proposed stereo-constrained PGD attack method and the unconstrained counterpart using three state-of-the-art methods in the KITTI2015 benchmark dataset.
Revised: We test the proposed stereo-constrained PGD attack method and the unconstrained counterpart using three state-of-the-art methods in the KITTI2015 benchmark dataset.

Original: Table 1 (Row 1 & 3) shows the results.
Revised: Table 1 (Row 1 & 3) shows the results.

Original: Since different methods use different training and validation splits when finetuning in the KITTI benchmark, the performance in Table 1 are computed on the entire dataset (200 images in total).
Revised: Since different methods use different training and validation splits when ﬁnetuning in the KITTI benchmark, the performance in Table 1 are computed on the entire dataset (200 images in total).

Original: Both the stereo-constrained attacks and the unconstrained ones can lead to catastrophic drop of performance.
Revised: Both the stereo-constrained attacks and the unconstrained ones can lead to catastrophic drop of performance.

Original: Such vulnerability may raise serious concerns for the deployment of DNNs in safety critical applications.
Revised: Such vulnerability may raise serious concerns for the deployment of DNNs in safety critical applications.

Original: The three methods cover DNN architectures by both sophisticated manual design and differentiable neural architecture search.
Revised: The three methods cover DNN architectures by both sophisticated manual design and differentiable neural architecture search.

Original: These strong attack performance clearly call for alternative designs of stereo matching networks that can exploit the nature of stereo matching for better robustness.
Revised: These strong attack performance clearly call for alternative designs of stereo matching networks that can exploit the nature of stereo matching for better robustness.

Original: We can see that the stereo-constrained attacks are indeed relatively weaker attacks.
Revised: We can see that the stereo-constrained attacks are indeed relatively weaker attacks.

Original: The Physical Realizability of the Proposed Stereo-Constrained Attack.
Revised: The Physical Realizability of the Proposed Stereo-Constrained Attack.

Original: To test if the adversarial vulnerability can be intentionally exploited in a more realistic setting, such as autonomous driving, we constructed the patch attack experiment to demonstrate the possibility of such attempts.
Revised: To test if the adversarial vulnerability can be intentionally exploited in a more realistic setting, such as autonomous driving, we constructed the patch attack experiment to demonstrate the possibility of such attempts.

Original: We select 10 scenarios where 40 × 40 adversarial patches can be put on more flat surfaces (Fig. 2, details and all images are provided in Appendix C.2).
Revised: We select 10 scenarios where 40 ⇥ 40 adversarial patches can be put on more ﬂat surfaces (Fig. 2, details and all images are provided in Appendix C.2).

Original: To preserve the depth of the scene, the ground truth disparities of the patches are the same as the corresponding part of the original image.
Revised: To preserve the depth of the scene, the ground truth disparities of the patches are the same as the corresponding part of the original image.

Original: For each image pair, we apply stereo-constrained PGD attacks with ϵ = 0 .
Revised: For each image pair, we apply stereo-constrained PGD attacks with ✏ = 0 .

Original: 03 and T = 100 .
Revised: 03 and T = 100 .

Original: Table 2 shows the results supporting the physical realizability [Figure]
Revised: Table 2 shows the results supporting the physical realizability.

Original: 2.3 Adversarial Training
Revised: 2.3 Adversarial Training

Original: [Figure]
Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Original: [Table] 
Revised: [Table] [Figure] [Table] [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Original: To further support the pressing need of exploring alternative networks towards robust stereo matching, we show that the prior art reinforced by adversarial training still do not perform sufficiently well against the attacks.
Revised: [Figure] [Table] [Figure] [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: To further support the pressing need of exploring alternative networks towards robust stereo matching, we show that the prior art reinforced by adversarial training still do not perform sufﬁciently well against the attacks.

Original: To compare with adversarial training, we fine-tune each method on the KITTI2015 training images perturbed by 3-step unconstrained PGD attacks for 20 epochs 1 , denoted as +adv.
Revised: To compare with adversarial training, we ﬁne-tune each method on the KITTI2015 training images perturbed by 3-step unconstrained PGD attacks for 20 epochs 1 , denoted as +adv.

Original: in tables.
Revised: in tables.

Original: The bottom of Table 1 and the bottom of Table 2 show the results of attacks at the image level and the patch level respectively.
Revised: The bottom of Table 1 and the bottom of Table 2 show the results of attacks at the image level and the patch level respectively.

Original: While being relatively more robust after adversarial training, the performance drops significantly on clean images.
Revised: While being relatively more robust after adversarial training, the performance drops signiﬁcantly on clean images.

Original: The proposed robust stereo matching method will significantly improve this.
Revised: The proposed robust stereo matching method will signiﬁcantly improve this.

Original:  The Proposed Stereo Matching Network
Revised:  The Proposed Stereo Matching Network

Original: Strong attacking results on state-of-the-art stereo matching methods and the lack of sufficient correction via adversarial training in Section 2 motivate us to explore alternative design of stereo matching networks that enjoy built-in robustness.
Revised: Strong attacking results on state-of-the-art stereo matching methods and the lack of sufﬁcient correction via adversarial training in Section 2 motivate us to explore alternative design of stereo matching networks that enjoy built-in robustness.

Original: In this section, we present the details of the proposed method.
Revised: In this section, we present the details of the proposed method.

Original: 3.1 Harnessing the Best of Classic Designs and DNNs
Revised: 3.1 Harnessing the Best of Classic Designs and DNNs

Original: Fig.
Revised: Fig.

Original: 1 illustrates the proposed workflow consisting of four main components:
Revised: 1 illustrates the proposed workﬂow consisting of four main components:

Original: Extracting Features and Computing the Feature Volume Using Multi-Scale Census Transform.
Revised: Extracting Features and Computing the Feature Volume Using Multi-Scale Census Transform.

Original: Most current stereo matching methods use DNN-based features to form the 4D feature volume.
Revised: Most current stereo matching methods use DNN-based features to form the 4D feature volume.

Original: In[-
-]
Revised: In[-
-]

Original: [Figure]
Revised: [Figure]

Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: terms of matching, DNNs can increase the uniqueness of the feature for each pixel, but they also suffer from the inherent adversarial vulnerability.
Revised: terms of matching, DNNs can increase the uniqueness of the feature for each pixel, but they also suffer from the inherent adversarial vulnerability.

Original: In contrast, traditional methods often use simple window-based similarity functions to initialize the costs, then rely on the optimization or cost aggregation stage to integrate all local cost information [12].
Revised: In contrast, traditional methods often use simple window-based similarity functions to initialize the costs, then rely on the optimization or cost aggregation stage to integrate all local cost information [12].

Original: Following the same philosophy, we propose to use hand-crafted feature descriptors and similarity functions that are less sensitive to adversarial perturbations to initialize the costs, then rely on DNNs to integrate the local cost information.
Revised: Following the same philosophy, we propose to use hand-crafted feature descriptors and similarity functions that are less sensitive to adversarial perturbations to initialize the costs, then rely on DNNs to integrate the local cost information.

Original: Specifically, we want the feature descriptor to change as little as possible when local intensities are perturbed.
Revised: Speciﬁcally, we want the feature descriptor to change as little as possible when local intensities are perturbed.

Original: This specific requirement lead us to the Census Transform (CT), a traditional feature descriptor that is developed to eliminate the issue of radiometric differences caused by different exposure timing or non-Lambertian surfaces.
Revised: This speciﬁc requirement lead us to the Census Transform (CT), a traditional feature descriptor that is developed to eliminate the issue of radiometric differences caused by different exposure timing or non-Lambertian surfaces.

Original: Previous studies find that CT is the most robust and well-rounded cost function with global or semi-global methods [1, 4].
Revised: Previous studies ﬁnd that CT is the most robust and well-rounded cost function with global or semi-global methods [1, 4].

Original: We use grey-scale raw intensity values in computing the census transform.
Revised: We use grey-scale raw intensity values in computing the census transform.

Original: Given a local window patch R centered at a pixel u ∈ Λ (Fig. 3), the census transform computes the local binary pattern by comparing each neighboring pixel v ∈ R with u such that it equals 1 if I ( v ) >
Revised: Given a local window patch R centered at a pixel u 2 ⇤ (Fig. 3), the census transform computes the local binary pattern by comparing each neighboring pixel v 2 R with u such that it equals 1 if I ( v ) >

Original: = I ( u ) and 0 otherwise.
Revised: = I ( u ) and 0 otherwise.

Original: Hamming Distance (i.e. the number of different values in two bit strings) is used to compute the cost between two patches.
Revised: Hamming Distance (i.e. the number of different values in two bit strings) is used to compute the cost between two patches.

Original: Unlike in traditional semi-global or global methods in which the cost of each pair can only be a scalar, we utilize multi-scale CTs to incorporate the context at different scales.
Revised: Unlike in traditional semi-global or global methods in which the cost of each pair can only be a scalar, we utilize multi-scale CTs to incorporate the context at different scales.

Original: We use local windows with sizes from k 1 to k 2 (e.g. k 1 = 3 , k 2 = 11 in our experiments) so there are K = k 2 − k 1 +costs associated with each matching candidate pairs, resulting in an initial 4D cost volume of the dimensions, K × D × H × W .
Revised: We use local windows with sizes from k 1 to k 2 (e.g. k 1 = 3 , k 2 = 11 in our experiments) so there are K = k 2  k 1 +costs associated with each matching candidate pairs, resulting in an initial 4D cost volume of the dimensions, K ⇥ D ⇥ H ⇥ W .

Original: To normalize the cost at each scale, we divide the Hamming Distance by the number of pixels of each local window.
Revised: To normalize the cost at each scale, we divide the Hamming Distance by the number of pixels of each local window.

Original: To reduce the overall computational cost and the memory footprint, we use 3D convolutions to down-scale the cost volume to 1 / 3 (i.e., s = 3 in
Revised: To reduce the overall computational cost and the memory footprint, we use 3D convolutions to down-scale the cost volume to 1 / 3 (i.e., s = 3 in

Original: Fig. 1) of its spatial size and the maximum disparity level, resulting the down-scaled cost volume of the dimensions, C × D 1 × H 1 × W 1 .
Revised: Fig. 1) of its spatial size and the maximum disparity level, resulting the down-scaled cost volume of the dimensions, C ⇥ D 1 ⇥ H 1 ⇥ W 1 .

Original: Details of the architecture are included in the Appendix B.
Revised: Details of the architecture are included in the Appendix B.

Original: Boosting the Sim2Real generalizability.
Revised: Boosting the Sim2Real generalizability.

Original: Due to its binary nature, the cost curves will be less affected by the different color distributions of different datasets, thus improving the Sim2Real generalizability of the proposed stereo matching network, especially when no finetuning is used.
Revised: Due to its binary nature, the cost curves will be less affected by the different color distributions of different datasets, thus improving the Sim2Real generalizability of the proposed stereo matching network, especially when no ﬁnetuning is used.

Original: Contextualizing the Feature Volume and Computing and Aggregating the Cost Volume.
Revised: Contextualizing the Feature Volume and Computing and Aggregating the Cost Volume.

Original: Although being robust to adversarial attacks, the CT-based feature volume alone is not sufficiently powerful to handle occlusion, textureless (flat) regions, and more challenging semantic information, such as transparent objects and specular reflections.
Revised: Although being robust to adversarial attacks, the CT-based feature volume alone is not sufﬁciently powerful to handle occlusion, textureless (ﬂat) regions, and more challenging semantic information, such as transparent objects and specular reﬂections.

Original: We introduce a 2-stack Hourglass [13] module with 2D convolutions to extract context information from the left reference image, resulting in a feature map of the same spatial size as the down-scaled CT cost volume, i.e., C × H 1 × W 1 .
Revised: We introduce a 2-stack Hourglass [13] module with 2D convolutions to extract context information from the left reference image, resulting in a feature map of the same spatial size as the down-scaled CT cost volume, i.e., C ⇥ H 1 ⇥ W 1 .

Original: The contextual feature map is unsqueezed and broadcasted along the second dimension to account for the down-scaled disparity levels, i.e., C × D 1 [-× H 
-]
Revised: The contextual feature map is unsqueezed and broadcasted along the second dimension to account for the down-scaled disparity levels, i.e., C ⇥ D 1 [-× H 
-]⇥

Original: × W 1 .
Revised: H 1 ⇥ W 1 .

Original: The two are then concatenated along the first channel dimension (see Fig. 1).
Revised: The two are then concatenated along the ﬁrst channel dimension (see Fig. 1).

Original: The contextualized cost volume will be fed into an encoder-decoder sub-network for the cost volume computation and aggregation stage, C D × H × W , which is realized by a 3-stack Hourglass [13] module with 3D convolutions.
Revised: The contextualized cost volume will be fed into an encoder-decoder sub-network for the cost volume computation and aggregation stage, C D ⇥ H ⇥ W , which is realized by a 3-stack Hourglass [13] module with 3D convolutions.

Original: If the DNN feature backbone is not used to extract context information from the left image (Fig. 1), our model will be denoted as “ours w/o ctx."
Revised: If the DNN feature backbone is not used to extract context information from the left image (Fig. 1), our model will be denoted as “ours w/o ctx."

Original: in tables and figures.
Revised: in tables and ﬁgures.

Original: Disparity Map Prediction.
Revised: Disparity Map Prediction.

Original: To predict the final disparity map D ( x, y ) , ∀ ( x, y ) ∈ Λ , the output of each module in the decoder (i.e., a stack in the Hourglass module) of the cost aggregation is first up-sampled to the original size D × H × W , denoted as D m ( d, x, y ) where m is the stack layer index.
Revised: To predict the ﬁnal disparity map D ( x, y ) , 8 ( x, y ) 2 ⇤ , the output of each module in the decoder (i.e., a stack in the Hourglass module) of the cost aggregation is ﬁrst up-sampled to the original size D ⇥ H ⇥ W , denoted as D m ( d, x, y ) where m is the stack layer index.

Original: Then, similar to the method used in [14], the predicted disparity map
Revised: Then, similar to the method used in [14], the predicted disparity map

Original: D m ( x, y ) is computed by,
Revised: D m ( x, y ) is computed by,

Original: [Equation]
Revised: [Equation]

Original: where Softmax is applied along the first disparity level dimension in D m .
Revised: where Softmax is applied along the ﬁrst disparity level dimension in D m .

Original: The Loss Function.
Revised: The Loss Function.

Original: In training, we use the smooth L 1 loss, ℓ L 1 ( z ) =
Revised: In training, we use the smooth L 1 loss, ` L 1 ( z ) =

Original: z 2 2 , ( if z < 1); | z | − 0 .
Revised: z 2 2 , ( if z < 1); | z |  0 .

Original: 5 , ( otherwise ) due to its robustness at disparity discontinuities and low sensitivity to outliers [9, 10].
Revised: 5 , ( otherwise ) due to its robustness at disparity discontinuities and low sensitivity to outliers [9, 10].

Original: Given the ground-truth disparity map D ∗ ( u ) , the loss is defined by,
Revised: Given the ground-truth disparity map D ⇤ ( u ) , the loss is deﬁned by,

Original: [Equation]
Revised: [Equation]

Original: where Θ collects all parameters in our model, β m represents the weight for the output from a stack layer m (e.g., 0.5, 0.7, and 1 are used for the 3-stack Hourglass module in our experiments).
Revised: where ⇥ collects all parameters in our model,  m represents the weight for the output from a stack layer m (e.g., 0.5, 0.7, and 1 are used for the 3-stack Hourglass module in our experiments).

Original: 3.2 Attacking Census Transform and Its Built-in Robustness
Revised: 3.2 Attacking Census Transform and Its Built-in Robustness

Original: Since Census Transform contains the non-differentiable comparison operator, the gradients from the constructed cost volume are blocked to the input images thus leading to an illusion of safety, i .
Revised: Since Census Transform contains the non-differentiable comparison operator, the gradients from the constructed cost volume are blocked to the input images thus leading to an illusion of safety, i .

Original: e .
Revised: e .

Original: the obfuscated gradient problem [8].
Revised: the obfuscated gradient problem [8].

Original: For fair comparisons with differentiable methods, we combine subtraction and the sigmoid function as a differentiable approximation of the comparison operator.
Revised: For fair comparisons with differentiable methods, we combine subtraction and the sigmoid function as a differentiable approximation of the comparison operator.

Original: [Equation]
Revised: [Equation]

Original: As suggested in [8], we should perform the usual forward pass through the comparison operator, but compute the gradients through the differentiable combination of subtraction and the sigmoid function on the backward pass.
Revised: As suggested in [8], we should perform the usual forward pass through the comparison operator, but compute the gradients through the differentiable combination of subtraction and the sigmoid function on the backward pass.

Original: However, as the derivative of the sigmoid function is (1 − x ) x , a boolean value (0 or 1) will make it zero, and thus blocking the gradient flow.
Revised: However, as the derivative of the sigmoid function is (1  x ) x , a boolean value (0 or 1) will make it zero, and thus blocking the gradient ﬂow.

Original: Therefore, we use the sigmoid
Revised: Therefore, we use the sigmoid

Original: [Table]
Revised: [Table]

Original: in the forward pass as well and multiply the inputs by a large constant (e.g. C = 10 5 ) such that the output of the sigmoid function is close to either zero or one, while maintaining the gradient flow.
Revised: in the forward pass as well and multiply the inputs by a large constant (e.g. C = 10 5 ) such that the output of the sigmoid function is close to either zero or one, while maintaining the gradient ﬂow.

Original: Built-in Robustness of Census Transform.
Revised: Built-in Robustness of Census Transform.

Original: Without using this differentiable approximation, our method without the contextual feature backbone will be unattackable since the gradient flows are completely blocked.
Revised: Without using this differentiable approximation, our method without the contextual feature backbone will be unattackable since the gradient ﬂows are completely blocked.

Original: From the perspective of attacks, the binary patterns generated by Census
Revised: From the perspective of attacks, the binary patterns generated by Census

Original: Transform is more difficult to alter due to the comparison operator.
Revised: Transform is more difﬁcult to alter due to the comparison operator.

Original: Given a threshold of maximum pixel difference in perturbation ϵ , neighbors will not be altered if their difference with the center is larger than 2 ϵ .
Revised: Given a threshold of maximum pixel difference in perturbation ✏ , neighbors will not be altered if their difference with the center is larger than 2 ✏ .

Original: Let CT ( u, v ) denote the local binary value at u with the neighbor v , I be the image, and CT ′ be the binary pattern after the perturbation, we have:
Revised: Let CT ( u, v ) denote the local binary value at u with the neighbor v , I be the image, and CT 0 be the binary pattern after the perturbation, we have:

Original: [Equation]
Revised: [Equation]

Original: If the attack does not violate photometric consistency, it will be even harder to alter the cost between binary patches of corresponding pairs.
Revised: If the attack does not violate photometric consistency, it will be even harder to alter the cost between binary patches of corresponding pairs.

Original: Specifically, if a neighboring pixel appears in both the left and the right binary patches, its relative magnitude relationship with the center pixel will be the same for both patches, no matter how its intensities change.
Revised: Speciﬁcally, if a neighboring pixel appears in both the left and the right binary patches, its relative magnitude relationship with the center pixel will be the same for both patches, no matter how its intensities change.

Original: It is our interest to test if this highly non-linear operator can defend the DNNs against attacks.
Revised: It is our interest to test if this highly non-linear operator can defend the DNNs against attacks.

Original:  Experiments
Revised:  Experiments

Original: We first present the results on the Sim2Real cross-domain generalizability, followed by showing results on the adversarial robustness.
Revised: We ﬁrst present the results on the Sim2Real cross-domain generalizability, followed by showing results on the adversarial robustness.

Original: Implementation details are provided in the Appendix B.
Revised: Implementation details are provided in the Appendix B.

Original: Data.
Revised: Data.

Original: We evaluate our method on the SceneFlow [5] and KITTI2015 [6] datasets.
Revised: We evaluate our method on the SceneFlow [5] and KITTI2015 [6] datasets.

Original: We also test pretrained models on the KITTI2012 [15] and the Middlebury [7] dataset at quarter resolution.
Revised: We also test pretrained models on the KITTI2012 [15] and the Middlebury [7] dataset at quarter resolution.

Original: The
Revised: The

Original: SceneFlow dataset is a large-scale synthetic dataset that contains 35 , 454 training images and 4 ,test images at the resolution of 540 × 960 .
Revised: SceneFlow dataset is a large-scale synthetic dataset that contains 35 , 454 training images and 4 ,test images at the resolution of 540 ⇥ 960 .

Original: The KITTI2015 dataset is a real-world dataset of driving scenes, which contains 200 training images and 200 test images at the resolution of 375 × 1242 .
Revised: The KITTI2015 dataset is a real-world dataset of driving scenes, which contains 200 training images and 200 test images at the resolution of 375 ⇥ 1242 .

Original: Since the depth of each scene is obtained through LiDAR, the ground truth is not dense.
Revised: Since the depth of each scene is obtained through LiDAR, the ground truth is not dense.

Original: Evaluation Metrics.
Revised: Evaluation Metrics.

Original: We adopt the provided protocols in the two datasets.
Revised: We adopt the provided protocols in the two datasets.

Original: There are three metrics:
Revised: There are three metrics:

Original: EPE [px] which measures the end-point error in pixels, Bad 1.0 [%] and Bad 3.0 [%] which represents the error rate of errors larger than 1 pixel and 3 pixels respectively.
Revised: EPE [px] which measures the end-point error in pixels, Bad 1.0 [%] and Bad 3.0 [%] which represents the error rate of errors larger than 1 pixel and 3 pixels respectively.

Original: Baseline Methods.
Revised: Baseline Methods.

Original: We compare with state-of-the-art deep stereo matching methods: the PSMNet [9], the GANet [10], and the LEAStereo [11].
Revised: We compare with state-of-the-art deep stereo matching methods: the PSMNet [9], the GANet [10], and the LEAStereo [11].

Original: We use their publicly released codes and trained model checkpoints in comparisons.
Revised: We use their publicly released codes and trained model checkpoints in comparisons.

Original: 4.1 The Sim2Real Cross-Domain Generalizability
Revised: 4.1 The Sim2Real Cross-Domain Generalizability

Original: To verify the conjecture that cross-domain generalizability in stereo matching can be induced by removing the dependency between the cost volume computation and the dataset-dependent feature backbone, we evaluate all models pretrained on SceneFlow directly on the KITTI training datasets and the Middlebury training dataset [7].
Revised: To verify the conjecture that cross-domain generalizability in stereo matching can be induced by removing the dependency between the cost volume computation and the dataset-dependent feature backbone, we evaluate all models pretrained on SceneFlow directly on the KITTI training datasets and the Middlebury training dataset [7].

Original: As shown in Table 3, our method outperforms prior art by a large margin .
Revised: As shown in Table 3, our method outperforms prior art by a large margin .

Original: This result shows that our proposed design of combining a non-parametric cost volume formed by the multi-scale census transform and a generalized cost aggregation/optimization
Revised: This result shows that our proposed design of combining a non-parametric cost volume formed by the multi-scale census transform and a generalized cost aggregation/optimization

Original: DNN is indeed more cross-domain consistent.
Revised: DNN is indeed more cross-domain consistent.

Original: It also shows that the head sub-network DNN indeed learns to play the role of a domain-independent optimizer over a given cost volume.
Revised: It also shows that the head sub-network DNN indeed learns to play the role of a domain-independent optimizer over a given cost volume.

Original: 4.2 The Adversarial Robustness
Revised: 4.2 The Adversarial Robustness

Original: Attacks at the image level.
Revised: Attacks at the image level.

Original: To evaluate the adversarial robustness in KITTI2015, we directly test the trained models on the entire training dataset (200 images).
Revised: To evaluate the adversarial robustness in KITTI2015, we directly test the trained models on the entire training dataset (200 images).

Original: Due to the GPU memory limitation, we only use the 240 × 384 center part of each image.
Revised: Due to the GPU memory limitation, we only use the 240 ⇥ 384 center part of each image.

Original: Because of cropping, we also ignore those pixels where their correspondences are outside of the cropped images.
Revised: Because of cropping, we also ignore those pixels where their correspondences are outside of the cropped images.

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: Table 1 (Row 2 & 4) shows the results.
Revised: Table 1 (Row 2 & 4) shows the results.

Original: Compared with the prior art, our method shows significantly better robustness on both stereo-constrained and unconstrained attacks.
Revised: Compared with the prior art, our method shows signiﬁcantly better robustness on both stereo-constrained and unconstrained attacks.

Original: We note that although the approximation used in attacking the CT is highly nonlinear (Eqn. 5), adversarial attacks can still find ways to perturb the input images, which further demonstrate the vulnerability of the DNNs.
Revised: We note that although the approximation used in attacking the CT is highly nonlinear (Eqn. 5), adversarial attacks can still ﬁnd ways to perturb the input images, which further demonstrate the vulnerability of the DNNs.

Original: As aforementioned, our method with the contextualized feature volume is more robust than its counterpart, showing that the majority of the vulnerability actually comes from the matching part rather than the contextual information.
Revised: As aforementioned, our method with the contextualized feature volume is more robust than its counterpart, showing that the majority of the vulnerability actually comes from the matching part rather than the contextual information.

Original: We note that although the context extractor is fully differentiable and thus vulnerable to attacks, to attack the
Revised: We note that although the context extractor is fully differentiable and thus vulnerable to attacks, to attack the

Original: CT-based matching component and the context extractor at the same time, an attaching method (e.g., the PGD) must find ways to alter both the cost volume and the context cues from the reference image.
Revised: CT-based matching component and the context extractor at the same time, an attaching method (e.g., the PGD) must ﬁnd ways to alter both the cost volume and the context cues from the reference image.

Original: The fact that the robustness is not affected significantly shows that the alternation of the context cues either contradicts the alternation of the matching component, or being too weak compared to the matching component.
Revised: The fact that the robustness is not affected signiﬁcantly shows that the alternation of the context cues either contradicts the alternation of the matching component, or being too weak compared to the matching component.

Original: In other words, the gradients from the context extractor have different directions or much smaller magnitudes compared to the gradients from the matching component.
Revised: In other words, the gradients from the context extractor have different directions or much smaller magnitudes compared to the gradients from the matching component.

Original: This finding shows a way to improve accuracy without sacrificing robustness in stereo matching.
Revised: This ﬁnding shows a way to improve accuracy without sacriﬁcing robustness in stereo matching.

Original: Those been said, the context extractor may still introduce the training domain dependence.
Revised: Those been said, the context extractor may still introduce the training domain dependence.

Original: Attacks at the patch level.
Revised: Attacks at the patch level.

Original: Table 2 shows the comparisons.
Revised: Table 2 shows the comparisons.

Original: Fig.
Revised: Fig.

Original: 4 shows an example.
Revised: 4 shows an example.

Original: Our method is very robust against adversarial patches.
Revised: Our method is very robust against adversarial patches.

Original: In contrast, other methods perform poorly, even with adversarial training.
Revised: In contrast, other methods perform poorly, even with adversarial training.

Original: 4.3 Improvement via Adversarial Training
Revised: 4.3 Improvement via Adversarial Training

Original: As shown in Table 1, our method without adversarial training shows comparable adversarial robustness with ϵ = 0 .
Revised: As shown in Table 1, our method without adversarial training shows comparable adversarial robustness with ✏ = 0 .

Original: 03 , 0 . 06 , especially for EPE and Bad 1 . 0 .
Revised: 03 , 0 . 06 , especially for EPE and Bad 1 . 0 .

Original: In comparisons, for the prior art (Row 1 & 3), the difference between with and without adversarial training are significantly larger.
Revised: In comparisons, for the prior art (Row 1 & 3), the difference between with and without adversarial training are signiﬁcantly larger.

Original: For the patch attack experiment, our method w/o adversarial training is already much more robust than the prior art with adversarial training (see Tabel 2).
Revised: For the patch attack experiment, our method w/o adversarial training is already much more robust than the prior art with adversarial training (see Tabel 2).

Original: With adversarial training, our method has a stronger robustness than all other methods, showing that our approach is orthogonal to adversarial training and they could be jointly used to further improved robustness.
Revised: With adversarial training, our method has a stronger robustness than all other methods, showing that our approach is orthogonal to adversarial training and they could be jointly used to further improved robustness.

Original: Transferability of the reinforcement.
Revised: Transferability of the reinforcement.

Original: To show the importance of built-in robustness, we also test on
Revised: To show the importance of built-in robustness, we also test on

Original: KITTI2012 using models trained on KITTI2015 to see how the adversarial robustness generalize on unseen data.
Revised: KITTI2012 using models trained on KITTI2015 to see how the adversarial robustness generalize on unseen data.

Original: In Table.
Revised: In Table.

Original: 4, our method shows a stronger cross-domain adversarial robustness than other adversarially trained methods.
Revised: 4, our method shows a stronger cross-domain adversarial robustness than other adversarially trained methods.

Original: Similarly, our method with adversarial training is the most robust over all methods.
Revised: Similarly, our method with adversarial training is the most robust over all methods.

Original: 4.4 Ablation Study
Revised: 4.4 Ablation Study

Original: How important is the non-parametric cost volume?
Revised: How important is the non-parametric cost volume?

Original: To check whether the importance of the non-parametric cost volume can be justified, we modify our model by replacing the CT component with the concatenation of normal DNN features from the left and right images, while keeping other components unchanged (denoted by “ours w/ feat." ).
Revised: To check whether the importance of the non-parametric cost volume can be justiﬁed, we modify our model by replacing the CT component with the concatenation of normal DNN features from the left and right images, while keeping other components unchanged (denoted by “ours w/ feat." ).

Original: We compare with traditional Sum of Absolute
Revised: We compare with traditional Sum of Absolute

Original: Difference (SAD) and CT with a single scale.
Revised: Difference (SAD) and CT with a single scale.

Original: Table 5 shows that the ones with non-parametric cost volumes generalize much better than the one with DNN features.
Revised: Table 5 shows that the ones with non-parametric cost volumes generalize much better than the one with DNN features.

Original: The contextualized CT feature volume is consistently better than the DNN feature volume.
Revised: The contextualized CT feature volume is consistently better than the DNN feature volume.

Original: Why using the CT?
Revised: Why using the CT?

Original: It is chosen due to its non-differentiability and the fact that it is a well-rounded choice in the literature.
Revised: It is chosen due to its non-differentiability and the fact that it is a well-rounded choice in the literature.

Original: We use multi-scale representations to respect the common recognition of its expressivity, and to alleviate choosing window size as a dataset-dependent hyper-parameter.
Revised: We use multi-scale representations to respect the common recognition of its expressivity, and to alleviate choosing window size as a dataset-dependent hyper-parameter.

Original: From
Revised: From

Original: Table 5, we can see that CT is indeed much more robust than SAD due to its non-differentiability.
Revised: Table 5, we can see that CT is indeed much more robust than SAD due to its non-differentiability.

Original: CT with multiple scales has a stronger robustness than the single-scale version, while having a slightly better accuracy due to its flexibility.
Revised: CT with multiple scales has a stronger robustness than the single-scale version, while having a slightly better accuracy due to its ﬂexibility

Original: [Table]
Revised: [Table]

Original: [Table] 
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: 4.5 KITTI Leaderboard Comparisons
Revised: 4.5 KITTI Leaderboard Comparisons

Original: Table 6 shows the comparisons.
Revised: Table 6 shows the comparisons.

Original: Our method is competitive compared to other state-of-the-art methods, while having much stronger adversarial robustness and cross-domain generalizability.
Revised: Our method is competitive compared to other state-of-the-art methods, while having much stronger adversarial robustness and cross-domain generalizability.

Original:  Related Work
Revised:  Related Work

Original: Deep Stereo Matching.
Revised: Deep Stereo Matching.

Original: After the first deep learning approach for stereo matching was proposed in [16], the first end-to-end trainable DNN-based method (DispNet) was developed by [5], together with the synthetic SceneFlow dataset.
Revised: After the ﬁrst deep learning approach for stereo matching was proposed in [16], the ﬁrst end-to-end trainable DNN-based method (DispNet) was developed by [5], together with the synthetic SceneFlow dataset.

Original: The GCNet [14] further extends the end-to-end approach by concatenating features in the cost volume stage, using 3D convolutional layers for cost aggregation, and introducing the soft arg min operator to compute the expected disparity.
Revised: The GCNet [14] further extends the end-to-end approach by concatenating features in the cost volume stage, using 3D convolutional layers for cost aggregation, and introducing the soft arg min operator to compute the expected disparity.

Original: Most subsequent approaches followed these design choices and use the SceneFlow dataset in pretraining [6].
Revised: Most subsequent approaches followed these design choices and use the SceneFlow dataset in pretraining [6].

Original: The cost aggregation stage was further studied in [9, 17, 18] using the Spatial Pyramid Pooling module for feature extraction and the stacked Hourglass structures [13].
Revised: The cost aggregation stage was further studied in [9, 17, 18] using the Spatial Pyramid Pooling module for feature extraction and the stacked Hourglass structures [13].

Original: In [19], fast stereo matching is studied by by building the cost volume purely using highly optimized hand-crafted features (e.g. Census
Revised: In [19], fast stereo matching is studied by by building the cost volume purely using highly optimized hand-crafted features (e.g. Census

Original: Transform and Sum of Absolute Differences) at the expense of accuracy performance.
Revised: Transform and Sum of Absolute Differences) at the expense of accuracy performance.

Original: They did not study adversarial attack and defense and the Sim2Real generalizability, which are the focuses of this paper.
Revised: They did not study adversarial attack and defense and the Sim2Real generalizability, which are the focuses of this paper.

Original: In [20], multiple hand-crafted features including CT are used for improving the Sim2Real generalizability, but did not study the contextualized settings as done in the proposed method.
Revised: In [20], multiple hand-crafted features including CT are used for improving the Sim2Real generalizability, but did not study the contextualized settings as done in the proposed method.

Original: Spatial cost propagation layers are studied in [10, 21] to reduce the number of 3D convolutional layers.
Revised: Spatial cost propagation layers are studied in [10, 21] to reduce the number of 3D convolutional layers.

Original: Neural Architecture Search techniques are used in [11] to automatically find optimal architectures for each stage and further improve the performance, which are the current state-of-the-art in the KITTI 2015 benchmark [6].
Revised: Neural Architecture Search techniques are used in [11] to automatically ﬁnd optimal architectures for each stage and further improve the performance, which are the current state-of-the-art in the KITTI 2015 benchmark [6].

Original: However, they significantly suffer from the proposed adversarial attacks.
Revised: However, they signiﬁcantly suffer from the proposed adversarial attacks.

Original: The SGM-Nets [22] provide an elegant and well-designed integration between exploiting the SGM for more robust cost aggregation (handling occlusion) and leveraging DNNs for learnable hyperparameter optimization (P1 and P2 in SGM).
Revised: The SGM-Nets [22] provide an elegant and well-designed integration between exploiting the SGM for more robust cost aggregation (handling occlusion) and leveraging DNNs for learnable hyperparameter optimization (P1 and P2 in SGM).

Original: In terms of the integration strategy, our method shares similar spirits and motivations with SGM-Nets.
Revised: In terms of the integration strategy, our method shares similar spirits and motivations with SGM-Nets.

Original: The main differences lie in (i) the generic fullytrainable cost aggregation component in our method versus the SGM inductive bias, and (ii) with versus without contextualized cost volumes.
Revised: The main differences lie in (i) the generic fullytrainable cost aggregation component in our method versus the SGM inductive bias, and (ii) with versus without contextualized cost volumes.

Original: In [23], domain-invariant stereo matching networks were proposed using the “domain normalization” approach that regularizes the distribution of learned representations to allow them to be invariant to domain differences.
Revised: In [23], domain-invariant stereo matching networks were proposed using the “domain normalization” approach that regularizes the distribution of learned representations to allow them to be invariant to domain differences.

Original: In [24], the generalizability of stereo matching is addressed using multi-level cost volume and multi-scale feature constancy.
Revised: In [24], the generalizability of stereo matching is addressed using multi-level cost volume and multi-scale feature constancy.

Original: In [25], cascade and fused cost volumes are used for robust stereo matching.
Revised: In [25], cascade and fused cost volumes are used for robust stereo matching.

Original: While these works improve cross-domain generalizability, our approach is orthogonal to theirs (i.e., different normalization techniques and cost volume refinement strategies) and we believe they can be used together to further improve the cross-domain accuracy.
Revised: While these works improve cross-domain generalizability, our approach is orthogonal to theirs (i.e., different normalization techniques and cost volume reﬁnement strategies) and we believe they can be used together to further improve the cross-domain accuracy.

Original: From the perspective of adversarial attacks, the first two approaches focus on the cross-domain representation power of DNN features, which will potentially make them more vulnerable against attacks.
Revised: From the perspective of adversarial attacks, the ﬁrst two approaches focus on the cross-domain representation power of DNN features, which will potentially make them more vulnerable against attacks.

Original: Adversarial Attacks and Defense.
Revised: Adversarial Attacks and Defense.

Original: Assuming full access to DNNs pretrained with clean images, white-box targeted attacks are powerful ways of investigating the brittleness of DNNs.
Revised: Assuming full access to DNNs pretrained with clean images, white-box targeted attacks are powerful ways of investigating the brittleness of DNNs.

Original: In autonomous driving, although physically realizable attacks are investigated in many tasks [26, 27, 28, 29, 30], attacking stereo matching has not been well studied.
Revised: In autonomous driving, although physically realizable attacks are investigated in many tasks [26, 27, 28, 29, 30], attacking stereo matching has not been well studied.

Original: [31] show that DNN-based stereo matching methods are vulnerable against unconstrained adversarial attacks on both images separately.
Revised: [31] show that DNN-based stereo matching methods are vulnerable against unconstrained adversarial attacks on both images separately.

Original: Without enforcing photometric consistency, these attacks will violate the underlying physical properties of binocular vision and thus are not realizable in practice.
Revised: Without enforcing photometric consistency, these attacks will violate the underlying physical properties of binocular vision and thus are not realizable in practice.

Original: For example, unconstrained attacks cannot compute adversarial patches to fool stereo systems.
Revised: For example, unconstrained attacks cannot compute adversarial patches to fool stereo systems.

Original: They also only focused on the relatively simpler FGSM attacks [32].
Revised: They also only focused on the relatively simpler FGSM attacks [32].

Original: To find out whether stereo matching methods are indeed vulnerable in a physically realizable setting, we propose the stereo-constrained projected gradient descent (PGD) attack [3] and show that state-of-the-art methods are vulnerable even when the color differences between corresponding pixels are preserved.
Revised: To ﬁnd out whether stereo matching methods are indeed vulnerable in a physically realizable setting, we propose the stereo-constrained projected gradient descent (PGD) attack [3] and show that state-of-the-art methods are vulnerable even when the color differences between corresponding pixels are preserved.

Original: [27] studied adversarial patch based attacks in optical flow, which is inherently different to attacking stereo matching due to the underlyingly different matching nature of the problems.
Revised: [27] studied adversarial patch based attacks in optical ﬂow, which is inherently different to attacking stereo matching due to the underlyingly different matching nature of the problems.

Original: They did not study adversarial defense for optical flow methods.
Revised: They did not study adversarial defense for optical ﬂow methods.

Original: The neural matching paths [33] leverage neural paths in DNNs trained for classification for matching, which exploit low, middle and high level features jointly and thus are robust with respect to the style transfer experiments. Consider that DNNs trained for classification are also significantly vulnerable to adversarial attacks, we suspect that we could still learn to “fool" those DNNs in neural-path-based matching using the white-box PGD method.
Revised: The neural matching paths [33] leverage neural paths in DNNs trained for classiﬁcation for matching, which exploit low, middle and high level features jointly and thus are robust with respect to the style transfer experiments. Consider that DNNs trained for classiﬁcation are also signiﬁcantly vulnerable to adversarial attacks, we suspect that we could still learn to “fool" those DNNs in neural-path-based matching using the white-box PGD method.

Original: Towards defense, adversarial training is the most widely used method to improve adversarial robustness [3, 34].
Revised: Towards defense, adversarial training is the most widely used method to improve adversarial robustness [3, 34].

Original: However, it also suffers from the disadvantages of dropping accuracy, long training time, and over-fitting to specific attacks and datasets.
Revised: However, it also suffers from the disadvantages of dropping accuracy, long training time, and over-ﬁtting to speciﬁc attacks and datasets.

Original: While adversarial training is universal to all kinds of DNNs, our method increases the built-in robustness by utilizing the photometric consistency of stereo matching, thus avoiding the mentioned disadvantages.
Revised: While adversarial training is universal to all kinds of DNNs, our method increases the built-in robustness by utilizing the photometric consistency of stereo matching, thus avoiding the mentioned disadvantages.

Original: It can also be combined with adversarial training to further improve robustness.
Revised: It can also be combined with adversarial training to further improve robustness.

Original:  Limitations and Potential Negative Impacts of the Proposed Work
Revised:  Limitations and Potential Negative Impacts of the Proposed Work

Original: One main limitation is that while being more robust and Sim2Real generalizable, the performance of the proposed method on clean images has some room to improve.
Revised: One main limitation is that while being more robust and Sim2Real generalizable, the performance of the proposed method on clean images has some room to improve.

Original: One direction is to leverage neural architecture search to find more suitable DNN aggregation component.
Revised: One direction is to leverage neural architecture search to ﬁnd more suitable DNN aggregation component.

Original: One potential negative impact of the proposed attack method is that since it is easy to implement the proposed physically realizable attacks they could be used in some unintended way to computer vision systems relying on the conventional DNN-based stereo matching.
Revised: One potential negative impact of the proposed attack method is that since it is easy to implement the proposed physically realizable attacks they could be used in some unintended way to computer vision systems relying on the conventional DNN-based stereo matching.

Original: Another limitation is that handcrafted CT features used in the proposed method may cause accuracy performance issues in multi-view stereo matching, where there can be strong viewpoint and illumination changes as the images can be taken at different points in time and by different cameras.
Revised: Another limitation is that handcrafted CT features used in the proposed method may cause accuracy performance issues in multi-view stereo matching, where there can be strong viewpoint and illumination changes as the images can be taken at different points in time and by different cameras.

Original: It is not clear whether the proposed contextualized non-parametric cost volume and the generic Hourglass stack based cost aggregation could potentially address them, which we leave for future work.
Revised: It is not clear whether the proposed contextualized non-parametric cost volume and the generic Hourglass stack based cost aggregation could potentially address them, which we leave for future work.

Original:  Conclusions
Revised:  Conclusions

Original: This paper presents a novel design for stereo matching, which utilizes DNNs to aggregate/optimize non-parametric cost volumes with parametric contextual features.
Revised: This paper presents a novel design for stereo matching, which utilizes DNNs to aggregate/optimize non-parametric cost volumes with parametric contextual features.

Original: It harnesses the best of classic features (multi-scale census transform) and end-to-end trainable DNNs for adversarially-robust and cross-domain generalizable stereo matching.
Revised: It harnesses the best of classic features (multi-scale census transform) and end-to-end trainable DNNs for adversarially-robust and cross-domain generalizable stereo matching.

Original: The proposed method is motivated by the observation that DNN-based stereo matching methods can be deceived by a type of physically realizable attacks that entail stereo constraints in learning the perturbation.
Revised: The proposed method is motivated by the observation that DNN-based stereo matching methods can be deceived by a type of physically realizable attacks that entail stereo constraints in learning the perturbation.

Original: In experiments, the proposed method is tested in SceneFlow and KITTI2015 datasets with significantly better adversarial robustness and
Revised: In experiments, the proposed method is tested in SceneFlow and KITTI2015 datasets with signiﬁcantly better adversarial robustness and

Original: Sim2Real cross-domain generalizability achieved.
Revised: Sim2Real cross-domain generalizability achieved.

Original: Acknowledgment
Revised: Acknowledgment

Original: K. Cheng and T. Wu were supported by NSF IIS-1909644, ARO Grant W911NF1810295, NSF
Revised: K. Cheng and T. Wu were supported by NSF IIS-1909644, ARO Grant W911NF1810295, NSF

Original: CMMI-2024688, NSF IUSE-2013451 and DHHS-ACL Grant 90IFDV0017-01-00.
Revised: CMMI-2024688, NSF IUSE-2013451 and DHHS-ACL Grant 90IFDV0017-01-00.

Original: T. Wu was also supported by a research gift fund from the Innopeak Technology, Inc. (an affiliate of OPPO).
Revised: T. Wu was also supported by a research gift fund from the Innopeak Technology, Inc. (an afﬁliate of OPPO).

Original: We thank the reviewers and area chairs for their constructive comments.
Revised: We thank the reviewers and area chairs for their constructive comments.

Original: T. Wu also thanks Drs.
Revised: T. Wu also thanks Drs.

Original: Zhebin
Revised: Zhebin

Original: Zhang and Hongyu Sun at the OPPO research, Seattle for their helpful discussions improving the writing.
Revised: Zhang and Hongyu Sun at the OPPO research, Seattle for their helpful discussions improving the writing.

Original: The views presented in this paper are those of the authors and should not be interpreted as representing any funding agencies.
Revised: The views presented in this paper are those of the authors and should not be interpreted as representing any funding agencies.

Original: Checklist
Revised: Checklist

Original: The checklist follows the references.
Revised: The checklist follows the references.

Original: Please read the checklist guidelines carefully for information on how to answer these questions.
Revised: Please read the checklist guidelines carefully for information on how to answer these questions.

Original: For each question, change the default [TODO] to [Yes] , [No] , or [N/A] .
Revised: For each question, change the default [TODO] to [Yes] , [No] , or [N/A] .

Original: You are strongly encouraged to include a justification to your answer , either by referencing the appropriate section of your paper or providing a brief inline description.
Revised: You are strongly encouraged to include a justiﬁcation to your answer , either by referencing the appropriate section of your paper or providing a brief inline description.

Original: For example:
Revised: For example:

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [Yes] See the license file in the code folder in the supplementary material.
Revised: [Yes] See the license ﬁle in the code folder in the supplementary material.

Original: Please do not modify the questions and only use the provided macros for your answers.
Revised: Please do not modify the questions and only use the provided macros for your answers.

Original: Note that the
Revised: Note that the

Original: Checklist section does not count towards the page limit.
Revised: Checklist section does not count towards the page limit.

Original: In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answers below.
Revised: In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answers below.

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
Revised: For all authors (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] See Section 6.
Revised: [Yes] See Section 6.

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [Yes] See Section 6.
Revised: [Yes] See Section 6.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 2.
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A] (b) Did you include complete proofs of all theoretical results?
Revised: [N/A] (b) Did you include complete proofs of all theoretical results?

Original: [N/A] 3.
Revised: [N/A] 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes] See the code in the supplementary material.
Revised: [Yes] See the code in the supplementary material.

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
Revised: [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [No] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
Revised: [No] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] See the Appendix D 4.
Revised: [Yes] See the Appendix D 4.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] (b) Did you mention the license of the assets?
Revised: [Yes] (b) Did you mention the license of the assets?

Original: [Yes] (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: [Yes] (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [Yes] Code is provided.
Revised: [Yes] Code is provided.

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?

Original: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
Revised: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?

Original: [N/A] 5.
Revised: [N/A] 5.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?[N/A] A Background on Stereo Matching The left of Fig.
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: 1 illustrates the stereo matching setup.

Original: The rectified left image is used as the reference image to infer the disparity map.

Original: For each pixel in the left image, the goal of stereo matching is to find the target pixel on the rectified epipolar line in the right image.

Original: The search range (i.e., disparity levels) often is predefined and fixed to a sufficiently large value in the cost volume computation stage.

Original: The matching is based on minimizing the cost between features centered at the source pixel and the target pixel respectively.

Original: The challenge of stereo matching is to seek the globally optimal matching for all pixels in the left reference image and to handle many uncertainties such as in the appearance features (textureless regions and specularities), the cost function, and unknown repeated patterns and occlusion situations.

Original: Let Λ be an image lattice (e.g., 540 × 960 ) on which the rectified left and right images are defined, denoted by I L and I R respectively.

Original: Denote by D ( x, y ) be the disparity map for the reference image

Original: I L .

Original: In traditional methods, stereo matching is formulated as an energy/cost minimization problem,

Original: [Equation]

Original: (7) where the first term is the data energy/cost, E d ( I L , I R , D )

Original: = (cid:80) ( x

Original: ,y ) ∈ Λ Cost ( F I L ( x, y ) , F I R ( x −

Original: D ( x, y ) , y )) capturing the matching cost between a source pixel ( x, y ) in the left reference image and the target pixel ( x − D ( x, y ) , y ) on the rectified epipolar line (i.e., the same row) in the right image.

Original: The cost is measured based on features F I L and F I R extracted for the source and target pixel respectively.

Original: The second term represents the prior/regularity of a disparity map such as the pairwise smoothness assumption, E s ( D ) =

Original: (cid:80) ( u,v ) ∈N S ( D ( u ) , D ( v )) where u, v ∈ Λ and N the set of neighboring pixels (e.g., the 4-connected neighborhood).

Original: The challenges in the traditional formulation are in two-fold: what are the good features and the cost functions in the data term?

Original: And, what is the good prior that are sufficiently expressive to capture the disparity structures while facilitating efficient optimization (e.g., by the dynamic programming algorithm or semi-global method [35])?

Original: Deep learning approaches mitigate the aforementioned challenges by exploiting the highly-expressive representational power and the end-to-end learning capability of DNNs.

Original: As shown in the rightbottom of Fig. 1, DNN-based stereo matching methods amortize the need of inducing proper priors (inductive biases) and of designing global matching cost optimization algorithms, which often consist of four components: (i)

Original: Extracting DNN features, F L and F R for matching with the spatial downsampling rate, s , (ii)

Original: Computing the 4D feature volume F by concatenating features F L ( u, v ) and F R ( u − d, v ) w.r.t.

Original: each disparity level d ∈ [0 , Ds ] where D is the predefined maximum disparity level, (iii) Computing and aggregating the matching cost volume C for each disparity level at the input resolution, which is typically realized via 3D convolution under an U-Net type of encoderdecoder architecture, which represents the solution space with respect to the data term in Eqn. 7, and (iv) Estimating the final disparity map D .

Original: The prior/regularity term of a disparity map is made implicitly by the supervised loss function.

Original: The optimization algorithms (e.g., the traditional global or semi-global methods [35]) are also implicitly realized by a head sub-network (e.g., the stacked

Original: Hourglass sub-network in Fig.

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: B Network Architecture and Training Details

Original: We show the detailed configuration of the proposed architecture in Table 7 with the workflow reproduced in Fig.

Original: Besides how layers are wired, one key difference is the down-sample scale of the cost volume.

Original: While our method uses the same 1 / 3 scale as GANet, PSMNet uses 1 / 4 and

Original: LEAStereo uses 1 / 6 .

Original: Intuitively, a smaller down-sample size should lead to a stronger robustness, because more perturbations are averaged out in the cost volume.

Original: Therefore, our method is not taking advantage of the dowm-sampled size.

Original: In fact, it is much more robust than PSMNet and LEAStereo with a finer resolution.

Original: Implementation Details.

Original: Our method is implemented in PyTorch and trained end-to-end using the Adam optimizer with β 1 = 0 .

Original: 9 and β 2 = 0 .

Original: All images are preprocessed with color normalization.

Original: During training, we use a batch size of 8 on four GPUs (Tesla V100) using 240 ×random crops.

Original: The maximum disparity level is set to 192 and any values larger than this threshold will be ignored during training.

Original: For SceneFlow, we train our model from random initialization forepochs with a constant learning rate of 0 .

Original: For KITTI2015, we split the 200 training images into a training set of 140 images and a validation set of 60 images.

Original: We fine-tune our model pretrained on the SceneFlow with 600 epochs and use the validation set to select the best model.

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Figure]

Original: C More Results C.1 A Toy Experiment

Original: We conduct a toy experiment to show that state-of-the-art stereo matching methods can be easily attacked even by the simplest form of stereo-constrained attack, i

Original: [Figure]

Original: e .

Original: shifted patches (Fig. 6).

Original: We create five synthetic toy stereo image pairs.

Original: In a stereo pair, the left reference image is composed by superposing a white-noise patch onto a constant background.

Original: The right image is created using the same patch and the same background in which the patch is horizontally shifted with respect to a given disparity level (such as ℓ = 20 ).

Original: So, the ground-truth disparity for the entire patch will be the specified ℓ .

Original: The background is excluded from the evaluation.

Original: As shown in Table 8, for the clean synthetic images, state-of-the-art stereo matching methods work very well using the SceneFlow trained model checkpoints.

Original: Our model shows better performance for all disparity levels.

Original: After applying the proposed stereo-constrained PGD attack only to the

Original: [Figure]

Original: patches (similar in spirit to the adversarial patches [36]), state-of-the-art methods’ performance drop significantly except for the LEAStereo method [11] at two disparity levels (60 and 100).

Original: Fig.

Original: 7 further shows the effects of applying the same adversarial patch at different disparity levels, showing that this simplest form of attack has a certain transferability with different depth levels.

Original: Through this toy experiment, we can observe: (i)

Original: Although not trained with this synthetic setting, state-of-the-art stereo matching methods are capable of recovering matching results when no attacks are applied.

Original: (ii)

Original: However, the matching capabilities are not stable even with respect to the much weaker stereo-constrained attacks.

Original: This may indicate that state-of-the-art methods could learn shortcut solutions in computing the cost volume, while our methods that directly utilize local rank information in computing the cost volume are more robust, either with or without the feature backbone in computing the cost volume.

Original: C.2 Examples of Adversarial Patch Attacks

Original: We show all of the 10 scenarios for adversarial patch attack in figures 8, 9, 10, 11, and 12.

Original: All scenarios are selected where the adversarial patches can be put on more flat surfaces, but they are not necessarily horizontal to the image plane.

Original: Note that the ground truth disparities of the patches are the same as the corresponding part of the original image.

Original: The first row shows the attacked image pairs for our method.

Original: Other methods will have patches on the same location but the texture will be different.

Original: PSMNet [9], LEAStereo [11], and GANet [10], and our method are shown on the second, third, fourth, and fifth row respectively.

Original: The first and the third columns are the after attack disparity maps, and the second and the fourth columns are the after attack error maps from the ground truth.

Original: From the results, we can see that our method is significantly more robust than others in this physically realizable attack setting.

Original: [Figure]

Original: C.3 Ablation Study on Adversarial Training

Original: Here we modify our model by using two feature backbones while keeping other components fixed and use this setting to study the effects of different PGD iteration steps and training epochs on adversarial training in Table 9.

Original: Our method is not suitable for justifying these hyper-parameters since it has significantly stronger robustness.

Original: In this experiment

Original: [Figure]

Original: we test the modified counterpart under FGSM attack, the unconstrained PGD attacks with different iterations ( 3 and 9 ) and training epochs ( 20 and 50 ).

Original: In order for adversarial training to be effective, the attack should be as strong as possible [3].

Original: Table 9 shows that all the model trained with PGD attacks are indeed more robust than the one with FGSM attack.

Original: However, it does not make much difference for using 9 iterations or 50 epochs, showing that 3 iterations withepochs are sufficient for the adversarial training.

Original: Besides justifying the hyper-parameters, this result also shows that our method is indeed more robust than its counterpart as it uses the proposed multi-scale Census Transform for the matching.

Original: [Figure]

Original: C.4 Supplementary Results on the Experiments

Original: Here we provide the EPE and Bad 1.0 for the adversarial patch attack and the transferability of adversarial robustness experiments.

Original: From Tables 10 and 11, our method shows significantly stronger robustness also in EPE and Bad 1.0, especially in Bad 1.0.

Original: In addition, we show two different versions of the unconstrained attacks in KITTI2015, where one use the ground truth to attack, and the other use the original neural network prediction.

Original: Results are shown in Table 12.

Original: It shows that using prediction is weaker than using the ground truth for unconstrained attacks.

Original: C.5 Occluded Regions

Original: For the proposed stereo-constrained PGD attack, we disallow to attack and evaluate occluded regions of the reference image, which prevents the perturbation to attack the regions where the estimation

Original: [Figure]
Revised: [N/A]

Original: KITTI2012 training dataset using adversarially trained neural networks on KITTI2015.

Original: does not rely on matching.

Original: Nonetheless, it is still possible to make perturbation on the occluded regions of the right image to hinder the matching, e.g. by creating false positive correspondence.

Original: We also consider this situation and experiment with an even weaker attack such that the occluded regions of both the left and right images will not be attacked.

Original: The results are shown in Table 13 Table 15, which are consistent with those reported in the submission.

Original: Our methods are the best in all metrics except for EPE in ScenFlow with ϵ = 0 . 03 .

Original: C.6 Results in SceneFlow

Original: We first compare the adversarial robustness.

Original: Table 14 and Table 15 shows the comparisons: our models are much more robust than the prior art .

Original: As the original SceneFlow dataset does not have ground truth occlusion, we use a subset provided by the same authors [5] with occlusions.

Original: We randomly select 1 , 000 images from the test data.

Original: Our methods show significantly better robustness against attacks.

Original: In fact, our methods are the best in all metrics except for EPE in ScenFlow with ϵ = 0 .

Original: 03 in Table 15.

Original: We note that if we do not allow to use the proposed differentiable approximation

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: of Census Transform

Original: , our method shows much better robustness, thanks to the non-differentiable cost volume computation.

Original: We also compare the results using the entire image, instead of only non-occlusion regions in evaluating attack performance.

Original: Table 16 shows the comparisons.

Original: Our method obtains competitive performance against state-of-the-art methods.

Original: Recent work suggests there exists an inherent conflict between accuracy and robustness [37, 38].

Original: From this perspective, the comparable performance on clean images and the significantly better robustness of our method show that the proposed design for stereo matching is effective.

Original: D Environment

Original: All the experiments were done on a Nvidia DGX server running Ubuntu 18 . 04 .

Original: 5 , which equips

Original: Tesla V-100 GPUs, each has 32 gigabytes of memory.

Original: For Pytorch, we use version 1 . 8 . 0 with CUDA 11 . 1 .

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]