SMD-Nets: Stereo Mixture Density Networks
Fabio Tosi1∗
Yiyi Liao2
Carolin Schmitt2
Andreas Geiger2
1
Department of Computer Science and Engineering (DISI), University of Bologna
2
Autonomous Vision Group, MPI-IS / University of Tübingen1{fabio.tosi5}@unibo.it 2 {firstname.lastname}@tue.mpg.de

Abstract
Despite stereo matching accuracy has greatly improvedby deep learning in the last few years, recovering sharpboundaries and high-resolution outputs efficiently remains
challenging. In this paper, we propose Stereo Mixture Density Networks (SMD-Nets), a simple yet effective learning
framework compatible with a wide class of 2D and 3Darchitectures which ameliorates both issues. Specifically,
we exploit bimodal mixture densities as output representation and show that this allows for sharp and precise disparity estimates near discontinuities while explicitly modeling the aleatoric uncertainty inherent in the observations.
Moreover, we formulate disparity estimation as a continuous problem in the image domain, allowing our model toquery disparities at arbitrary spatial precision. We carryout comprehensive experiments on a new high-resolution
and highly realistic synthetic stereo dataset, consisting ofstereo pairs at 8Mpx resolution, as well as on real-worldstereo datasets. Our experiments demonstrate increased
depth accuracy near object boundaries and prediction of ultra high-resolution disparity maps on standard GPUs. We
demonstrate the flexibility of our technique by improving theperformance of a variety of stereo backbones.

(a) PSM [4](b) PSM [4] + Ours

Figure 1: Point Cloud Comparison between the stereo network PSM [4] and our Stereo Mixture Density Network(SMD-Net) on the UnrealStereo4K dataset. Notice how
SMD-Net notably alleviates bleeding artifacts near object
boundaries, resulting in more accurate 3D reconstructions.

1. Introduction
Stereo matching is a long standing and active research
topic in computer vision. It aims at recovering dense correspondences between image pairs by estimating the disparity between matching pixels, required to infer depth throughtriangulation. It also plays a crucial role in many areas like
3D mapping, scene understanding and robotics.
Traditional stereo matching algorithms apply handcrafted matching costs and engineered regularization strategies. More recently, learning methods based on Convolutional Neural Networks (CNNs) have proven to be superior, given the increasing availability of large stereo datasets∗ Work done as intern at MPI-IS.

[11, 10, 52]. Although such methods produce compelling
results, two major issues remain unsolved: predicting accurate depth boundaries and generating high-resolution outputs with limited memory and computation.
The first issue is shown in Fig. 1a: As neural networks
are smooth function approximators, they often poorly reconstruct object boundaries, causing “bleeding” artifacts(i.e., flying pixels) when converted to point clouds. These
artifacts can be detrimental to subsequent applications suchas 3D reconstruction or 3D object detection. Thus, while
being ignored by most commonly employed disparity metrics, accurate 3D reconstruction of contours is a desirable
property for any stereo matching algorithm.

Furthermore, existing methods are limited to discrete
predictions at pixel locations of a fixed resolution image
grid, while geometry is a piecewise continuous quantity
where object boundaries may not align with pixel centers.
Increasing the output resolution by adding extra upsamplinglayers partially addresses this problem as this leads to a significant increase in memory and computation.
In this work, we address both issues. Our key contribution is to learn a representation that is precise at object
boundaries and scales to high output resolutions. In particular, we formulate the task as a continuous estimation problem and exploit bimodal mixture densities [2] as output representation. Our simple formulation lets us (1) avoid bleeding artifacts at depth discontinuities, (2) regress disparity
values at arbitrary spatial resolution with constant memoryand (3) provides a measure for aleatoric uncertainty.
We illustrate the boundary bleeding problem and our solution to it in Fig. 2. While classical deep networks forstereo regression suffer from smoothness bias and are incapable of representing sharp disparity discontinuities, theproposed Stereo Mixture Density Networks (SMD-Nets) effectively address this issue. The key idea is to alter the
output representation adopting a mixture distribution suchthat sharp discontinuities can be regressed despite the factthat the underlying neural networks are only able to makesmooth predictions (note that all curves in Fig. 2b are indeedsmooth while the predicted disparity is discontinuous).
Furthermore, the proposed model is capable of regressing disparity values at arbitrary continuous locations in theimage, effectively solving a stereo super-resolution task. In
combination with the proposed representation, this allows
for regressing sharp discontinuities at sub-pixel resolution
while keeping memory requirements constant.
In summary, we present: (i) A novel learning frameworkfor stereo matching that exploits compactly parameterized
bimodal mixture densities as output representation and canbe trained using a simple likelihood-based loss function. (ii)
A continuous function formulation aimed at estimating disparities at arbitrary spatial resolution with constant memory footprint. (iii) A new large-scale synthetic binocularstereo dataset with ground truth disparities at 3840 × 2160
resolution, comprising photo-realistic renderings of indoor
and outdoor environments. (iv) Extensive experiments on
several datasets demonstrating improved accuracy at depth
discontinuities for various backbones on binocular stereo,
monocular and active depth estimation tasks.
Our source code and dataset are available at https://github.com/fabiotosi92/SMD-Nets.

2. Related Work
Deep Stereo Matching: Stereo has a long history in computer vision [41]. With the rise of deep learning, CNN based
methods for stereo were pioneered in [56] with the aim ofreplacing the traditional matching cost computation.
More recent works attempt to solve the stereo matching task without hand-crafted post processing steps. They
can be categorized into 2D architectures and 3D architectures. In the first category, [22, 32, 15, 53, 54, 46, 1]
extend the seminal DispNet [24], an end-to-end networkfor disparity regression. The second class, instead, consists of architectures that explicitly construct 3D feature
cost volumes by means of concatenation/feature difference[4, 18, 47, 48, 8, 57, 49, 55, 29, 3, 7, 45, 50, 21] and groupwise correlation [14]. A thorough review of these works canbe found in [36]. We stress once again how such networks,
although achieving state-of-the-art results on most stereo
benchmarks, suffer from severe over-smoothing at object
discontinuities which are not captured by commonly employed disparity metrics, but which matter for many downstream applications. Therefore, the ideas proposed in thiswork to address this issue are orthogonal to the aforementioned networks and can be advantageously combined withnearly any stereo backbone.
Disparity Output Representation: Standard stereo networks directly regress a scalar disparity at every pixel. This
output representation suffers from over-smoothing and does
not expose the underlying aleatoric uncertainty. The latterproblem can be addressed by modeling the disparity using
a parametric distribution, e.g., a Gaussian or Laplacian distribution [16, 25] while the over-smoothing issue remains
unsolved. A key result of our work is to demonstrate that replacing the unimodal output representation with a bimodal
one is sufficient to significantly alleviate this problem.
Another line of methods estimate a non-parametric distribution over a set of discrete disparity values. However,this approach leads to inaccurate results when the estimateddistribution is multi-modal [17]. Some works tackle theproblem by enforcing a unimodal constraint during training [5, 58]. In contrast, we explicitly model the bimodal
nature of the distribution at object boundaries by adopting asimple and effective bimodal representation. In concurrent
work, [9] also predicts multi-modal distributions supervised
by a heuristically designed multi-modal ground truth overa set of depth values. In contrast to them, our bimodal approach can be learned by maximizing the likelihood without
requiring direct supervision on the distribution itself.
Continuous Function Representation: Existing deep
stereo networks use fully convolutional neural networks andmake predictions at discrete pixel locations. Recently, continuous function representations have gained attention in
many areas, including 3D reconstruction [27, 33, 6, 39, 44,
30, 35], texture estimation [31], image synthesis [28, 42]
and semantic segmentation [20]. To the best of our knowledge, we are the first to adopt a continuous function representation for disparity estimation, allowing us to predict a

Disparity
Discontinuity

Background

Foreground

Foreground

True Disparity
Predicted Disparity

First Mode WeightFirst Mode Mean
Second Mode Mean
Predicted Disparity(a) Stereo Regression Network

Background(b) Stereo Mixture Density Network

Figure 2: Overcoming the Smoothness Bias with Mixture Density Networks. For clarity, we visualize the disparity d only
for a single image row. (a) Classical deep networks for stereo regression suffer from smoothness bias and hence continuouslyinterpolate object boundaries. In addition, disparity values are typically predicted at discrete spatial locations. (b) In thiswork, we propose to use a bimodal Laplacian mixture distribution (illustrated in gray) with weight π as output representation
which can be queried at any continuous spatial location x. This allows our model to accurately capture uncertainty close
to depth discontinuities while at inference time recovering sharp edges by selecting the mode with the highest probability
density. In this example, the first mode (µ1 , b1 ) models the background and the second mode (µ2 , b2 ) models the foregrounddisparity close to the discontinuity. When the probability density of the foreground mode becomes larger than the probability
density of the background mode, the most likely disparity sharply transitions from the background to the foreground value.
disparity value at any continuous pixel location. In contrast
to works that allow for high-resolution stereo matching bydesigning memory efficient architectures [51, 13], our simple output representation is able to exploit ground truth disparity maps at a higher resolution than the input stereo pair,thus effectively learning stereo super-resolution.

lecting the mode with the highest density value. Thus, ourmodel is able to transition from one disparity to another ina discontinuous fashion while at the same time relying only
on the regression of functions (π, µ1 , b1 , µ2 , b2 ) which aresmooth with respect to the image domain and which therefore can easily be represented using neural networks.

3. Method

3.2. Stereo Mixture Density Networks

Fig. 3 illustrates our model. We first encode a stereo pair
into a feature map using a convolutional backbone (left).
Next, we estimate parameters of a mixture density distribution at any continuous 2D location via a multi-layer perceptron head, taking the bilinearly interpolated feature vector
as input (middle). From this, we obtain a disparity as wellas uncertainty map (right). We now explain our model, loss
function and training protocol in detail.

3.1. Problem Statement
Let I ∈ RW ×H×6 denote an RGB stereo pair for which
we aim to predict a disparity map D at arbitrary resolution.
As shown in Fig. 2, classical stereo regression networks suffer from over-smoothing due to the smoothness bias of neural networks. In this work, we exploit a mixture distribution
as output representation [2] to overcome this limitation.
More specifically, we propose to use a bimodal Laplacian mixture distribution with weight π and two modes(µ1 , b1 ), (µ2 , b2 ) to model the continuous probability distribution over disparities at a particular pixel. Using twomodes allows our model to capture both the foreground as
well as the background disparity at object boundaries. At
inference time, we recover sharp object boundaries by se-

We now formally describe our model. LetΨθ : RW ×H×6 → RW ×H×D

(1)

denote a stereo backbone network with parameters θ asshown in Fig. 3 (left). Ψθ takes as input the stereo pair I and
outputs a D-dimensional feature map, represented in the domain of the reference image (e.g. the left image of a stereopair). Examples for such networks are standard 2D convolutional networks, or networks which perform 3D convolutions. For the 2D networks, the stereo pair can be concatenated as input or processed by means of siamese towers withshared weights as typically done for 3D architectures. Similarly, this generic formulation also applies to the structured
light setting (e.g., Kinect setting where I ∈ RW ×H ) and themonocular depth estimation problem (I ∈ RW ×H×3 ).
As geometry is a piecewise continuous quantity, we apply a deterministic transformation to obtain feature points
for any continuous location in RW ×H . More specifically,for every continuous 2D location x ∈ R2 , we bilinearly interpolate the features from its four nearest pixel locationsin the feature map RW ×H×D . More formally, we describe
this transformation as:ψ : R2 × RW ×H×D → RD

(2)

Stereo Backbone

SMD Head

Figure 3: Method Overview. We assume a 2D or 3D stereo backbone network Ψθ which takes as input a stereo pair I (either
concatenated or processed by siamese towers), and outputs a D-dimensional feature map in the domain of the reference
image. Given any continuous 2D location x, we query its feature from the feature map via bilinear interpolation as denotedby ψ. The interpolated feature vector is then fed into a multi-layer perceptron fθ to estimate a five-dimensional vector(π, µ1 , b1 , µ2 , b2 ) which represents the parameters of a bimodal distribution. N denotes the number of points randomly
sampled at continuous 2D locations during training and the number of pixels during inference. On the right we show mapsˆof µ1 , µ2 , π, 1 − π, uncertainty h and predicted disparity d.
Finally, we employ a multi-layer perceptron to map thisabstract feature representation to a five-dimensional vector(π, µ1 , b1 , µ2 , b2 ) which represents the parameters of a univariate bimodal mixture distribution:fθ : RD → R5

(3)

Note that we have re-used the parameter symbol θ to simplify notation. In the following, we use θ to denote allparameters of our model. We refer to fθ (ψ(·, ·)) as SMD
Head, see Fig. 3 for an illustration.
To robustly model a distribution over disparities which
can express two modes close to disparity discontinuities, wechoose a bimodal Laplacian mixture as output representation:1|2|
1 − π − |d−µ
π − |d−µe b1 +e b2(4)p(d) =
2 b1
2 b2
In summary, our model can be compactly expressed as:p(d|x, I, θ) = p(d|fθ (ψ(x, Ψθ (I))))(5)

At inference time, we determine the final disparity dˆ bychoosing the mode with the highest density value:dˆ = argmax p(d)(6)

d∈{µ1 ,µ2 }Note that our formulation allows to query the disparitydˆ ∈ R at any continuous 2D pixel location, enabling ultra
high-resolution predictions with sharply delineated object
boundaries. This is illustrated in Fig. 4.
Our model also allows for capturing the aleatoric uncertainty of the predicted disparity by evaluating the differential entropy of the continuous mixture distribution as:Z
h = − p(d) log p(d) dd(7)
In practice, we use numerical quadrature to obtain an approximation of the integral.

3.3. Loss Function
We consider the supervised setting and train our modelby minimizing the negative log-likelihood loss:
LN LL (θ) = − Ed,x,I log p(d|x, I, θ)(8)where the input I is randomly sampled from the dataset,
x is a random pixel location in the continuous image domain Ω = [0, W − 1] × [0, H − 1], sampled as described inSec. 3.4, and d is the ground truth disparity at location x.

3.4. Training Protocol
Sampling Strategy: While a naı̈ve strategy samples pixel
locations x randomly and uniformly from the image domain Ω, our framework also allows for exploiting custom
sampling strategies to focus on depth discontinuities during training. We adopt a Depth Discontinuity Aware (DDA)
sampling approach during training that explicitly favors
points located near object boundaries while at the same time
maintaining a uniform coverage on the entire image space.
More specifically, given a ground truth disparity map attraining time, we first compute an object boundary maskin which a pixel is considered to be part of the boundary if
its (4-connected) neighbors have a disparity that differs bymore than 1 from its own disparity. This mask is then dilated using a ρ × ρ kernel to enlarge the boundary region.
We report an analysis using different ρ values in the experimental section. Given the total number of training points N ,we randomly and uniformly select N/2 points from the domain of all pixels belonging to depth discontinuity regions
and N/2 points uniformly from the continuous domain of
all remaining pixels. At inference time, we leverage ourmodel to predict disparity values at each location of an (arbitrary resolution) grid.
Stereo Super-Resolution: Our continuous formulation al-

128Mpx
0.5MpxFigure 4: Ultra High-resolution Estimation. Comparisonof our model using the PSM backbone at 128Mpx resolution(top) to the original PSM at 0.5Mpx resolution (bottom),both taking stereo pairs at 0.5Mpx resolution as input. Eachcolumn shows a different zoom-level. Note how our method
leads to sharper boundaries and high resolution outputs.
lows us to exploit ground truth at higher resolution than the
input I, which we refer to as stereo super-resolution. In
contrast, classical stereo methods cannot realize arbitrary
super-resolution without changing their architecture.

4. Experimental Results
In this section, we first describe the datasets used forevaluation and implementation details. We then present an
extensive evaluation that demonstrates the benefits of the
proposed SMD Head in combination with different stereo
backbones on several distinct tasks.

4.1. Datasets
UnrealStereo4K: Motivated by the lack of large-scale, realistic and high-resolution stereo datasets, we introduce a
new photo-realistic binocular stereo dataset at 3840 × 2160
resolution with pixel-accurate ground truth. We create this
synthetic dataset using the popular game engine Unreal Engine combined with the open-source plugin UnrealCV [37].
We additionally create a synthetic active monocular dataset(mimicking the Kinect setup) at 4112 × 3008 resolution by
warping a gray-scale reference dot pattern to each image,following [38]. We split the dataset into 7720 training pairs,
80 validation pairs and 200 in-domain test pairs. To evaluate the generalization ability of our method, we also create
an out-of-domain test set by rendering 200 stereo pairs froman unseen scene. Similarly, the active dataset contains 3856
training images, 40 validation images, 100 test images.
RealActive4K: We further collect a small real-world active dataset of an indoor room with a Kinect-like stereo sensor, including 2570 images at a resolution 4112 × 3008 pixels from which we use 2500 for training, 20 for validationand 50 for testing. We perform Block Matching with leftright consistency check to use as co-supervision for training
models jointly on synthetic (UnrealStereo4K) and real data.

KITTI 2015 [26]: The KITTI dataset is a collection ofreal-world stereo images depicting driving scenarios. It
contains 200 training pairs with sparse ground truth depth
maps collected by a LiDAR and 200 testing pairs. We divide the KITTI training set into 160 training stereo pairs and40 validation stereo pairs, following [45].
Middlebury v3 [40]: Middlebury v3 is a small highresolution stereo dataset depicting indoor scenes under controlled lighting conditions containing 10 training pairs and
10 testing pairs with dense ground truth disparities.

4.2. Implementation Details
Architecture: In principle, our SMD Head is compatible with any stereo backbone ψθ from the literature. In our
implementation, we build on top of two state-of-the-art 3D
stereo architectures: Pyramid Stereo Matching (PSM) network [4] and Hierarchical Stereo Matching (HSM) network[51]. PSM is a well-known and popular stereo network
while HSM represents a method with good trade-off between accuracy and computation. Moreover, we also adopt
a naı̈ve U-Net structure [12] that takes as input concatenatedimages of a stereo pair in order to show the effectiveness of
our model on 2D architectures. For the aforementioned networks, we follow the official code provided by the authors.
Our SMD Head fθ is implemented as a multi-layer perceptron (MLP) following [39]. More specifically, the number of neurons is (D, 1024, 512, 256, 128, 5). We use sine
activations [44] except for the last layer that uses a sigmoid
activation for regressing the five-parameter output. For the
3D backbone, we select the matching probabilities from thecost volume in combination with features of Ψθ at different
resolutions as input to our SMD Head. For the 2D backbone
case, instead, we select features from different layers of thedecoder. We refer the reader to the supplementary material
for details.
Training: We implement our approach in PyTorch [34]and use Adam with β1 = 0.9 and β2 = 0.999 as optimizer [19]. We train all models from scratch using a single NVIDIA V100 GPU. During training, we use random
crops from I as input to the stereo backbone and sample
N = 50, 000 training points from each crop. We scale theground truth disparity to [0, 1] for each dataset for numerical stability. Moreover, for RGB inputs we perform chromatic augmentations on the fly, including random brightness, gamma and color shifts sampled from uniform distributions. We further apply horizontal and vertical random
flipping while adapting the ground truth disparities accordingly. Please see the supplementary material for details regarding the training procedure for each dataset.
Evaluation Metrics: Following [5], we evaluate the Soft
Edge Error (SEEk ) metric on pixels belonging to object

4.3. Ablation Study
We first examine the impact of different components
and training choices of the proposed SMD-Nets on the indomain UnrealStereo4K test set. Unless specified otherwise, we use 960 × 540 as resolution for the binocular input
I and 3840 × 2160 for the corresponding ground truth, used
for both supervision and testing purposes. The active input
images consist of random dot patterns where the dots become indistinguishable at low resolution (e.g., 960 × 540).
Therefore we use 2056 × 1504 as active input size while
keeping the ground truth dimension at 4112 × 3008.
Output Representation: In Tab. 1, we evaluate the effectiveness of our mixture density output representation acrossboth, 2D and 3D stereo backbones on multiple tasks including binocular stereo, monocular depth and active depth. We
adopt U-Net and PSM on the binocular stereo dataset as
representatives of 2D and 3D backbones and report resultsof HSM in the supplementary for the sake of space. We also
use the same U-Net backbone for a monocular depth estimation task by replacing the input with only the reference image of a binocular stereo pair to show the advantage of our
method on various tasks. For the active setup, we choose
HSM as it represents a network designed specifically forhigh-resolution inputs which takes as input the monocular
active image and the fixed reference dot pattern.
We compare our bimodal distribution to two other output representations, standard disparity regression and a unimodal Laplacian distribution [16]. For fairness, we implement these baselines by replacing the last layer of our SMD
Head to predict the disparity d or the unimodal parameters(µ, b), respectively, where the former is trained with a standard L1 loss while the latter with a negative log-likelihood
loss. For all cases we use the proposed bilinear feature interpolation and the naı̈ve random sampling strategy.
Tab. 1 shows that the proposed method effectively addresses the over-smoothing problem at object boundaries,
achieving the lowest SEE for all backbones on all tasks,
compared to both the standard disparity regression and theunimodal representation. Moreover, we observe that the

Mono. Binocular Stereo
Activeboundaries, defined as the minimum absolute error between
the predicted disparity and the corresponding local groundtruth patch of size k × k (k = {3, 5} in our experiments).
Intuitively, SEE penalizes over-smoothing artifacts stronger
compared to small misalignments in a local window, where
the former is more harmful to subsequent applications.
While not our main focus, we also report the End Point
Error (EPE) as the standard error metric obtained by averaging the absolute difference between predictions andground truth disparity values to evaluate the overall performance. For both SEE and EPE, we compute the average(Avg) and σ(∆) metrics, with the latter one representing
the percentage of pixels having errors greater than ∆.

Ψθ

Dim.

U-Net
(2D )
[12]
PSM
(3D)
[4]
U-Net
(2D)
[12]
HSM
(3D)
[51]

1
2
5
1
2
5
1
2
5
1
2
5

Avg
2.15
2.38
1.57
1.98
2.50
1.52
3.29
4.01
2.92
3.40
4.93
2.69

SEE3
σ(1)
41.69
42.28
30.06
36.32
39.40
26.98
60.18
61.06
51.32
47.87
57.05
41.84

σ(2)
24.16
25.74
14.77
20.35
23.63
12.68
41.37
43.19
32.33
24.80
33.44
17.35

Avg
2.03
2.26
1.45
1.85
2.37
1.38
3.25
3.86
2.78
3.18
4.69
2.43

SEE5
σ(1)
39.65
40.42
28.05
34.42
37.57
24.93
58.49
59.40
49.54
46.14
55.47
39.83

σ(2)
22.98
24.57
16.57
19.21
22.51
11.49
40.08
41.90
31.06
23.76
32.41
16.17

EPE
Avg σ(3)1.48 8.18
1.97 10.44
1.28 5.94
1.10 5.52
1.88 7.73
1.11 4.80
4.21 35.92
5.49 41.88
4.06 30.59
1.29 5.84
2.83 10.70
1.42 5.48

Table 1: Output Representation analysis on the UnrealStereo4K test set. “Dim.” refers to the output dimensionof the SMD Head where 1 indicates the point estimate d,
2 the unimodal output representation (µ, b) [16] and 5 ourbimodal formulation (π, µ1 , b1 , µ2 , b2 ).
Samplingρ

Random
DDA
DDA
DDA

0
10
20

Avg
1.52
1.34
1.13
1.30

SEE3
σ(1)
26.98
21.62
18.64
20.42

σ(2)
12.68
9.77
8.69
9.88

Avg
1.38
1.19
0.98
1.15

SEE5
σ(1)
24.93
19.58
16.67
18.40

σ(2)
11.49
8.59
7.55
8.71

EPE
Avg σ(3)1.11 4.80
1.08 4.44
0.92 3.88
1.11 4.44

Table 2: Sampling Strategy analysis on the UnrealStereo4K test set using the PSM backbone.
Eval. GT

Training GT

960 × 540
960 × 540
3840 × 2160
3840 × 2160

960 × 540
3840 × 2160
960 × 540
3840 × 2160

Avg
1.19
0.98
1.33
1.13

SEE3
σ(1)
20.36
15.42
23.35
18.64

σ(2)
9.16
7.05
10.82
8.69

Avg
0.93
0.78
1.19
0.98

SEE5
σ(1)
16.55
12.44
21.34
16.67

σ(2)
7.08
5.54
9.63
7.55

EPE
Avg σ(3)1.02 4.30
0.89 3.81
1.03 4.30
0.92 3.88

Table 3: Ground Truth Resolution analysis on the UnrealStereo4K test set using the PSM backbone.
unimodal representation sacrifices EPE for capturing theuncertainty, while our method is on par with the standard L1
regression. On the stereo dataset, the 3D backbone (PSM)
consistently outperforms the 2D backbone (U-Net), therefore we use PSM for the following ablation experiments.
Sampling Strategy: In Tab. 2, we show the impact of thesampling strategy adopted during training. More specifically, we compare the naı̈ve uniform sampling strategy andthe proposed DDA approach using different dilation kernel
sizes ρ × ρ. As can be observed, DDA enables SMD-Nets tofocus on depth discontinuities, resulting in better SEE compared to random point selection. Moreover, we observe thatsampling exactly at depth boundaries (i.e., ρ = 0) leads toslightly degraded EPE and is less effective on SEE which
penalizes small misalignment in a local window. Instead,setting ρ = 10 allows the network to focus on larger regions near edges and results in the best performance, while
increasing ρ does not improve performance further. Finally,
it is worth to notice that this strategy also allows our model
to improve the overall performance, achieving lower EPE
metrics. In the following experiments, we thus adopt the
DDA strategy using ρ = 10 for our SMD-Nets.

Method
PSM [4]
PSM [4] + BF [23]
PSM [4] + SM [5]
PSM [4] + CE + SM [5]
PSM [4] + Ours
HSM [51]
HSM [51] + BF [23]
HSM [51] + SM [5]
HSM [51] + CE + SM [5]
HSM [51] + Ours

Avg
1.73
1.65
1.50
1.33
1.13
2.01
1.88
1.83
2.00
1.31

SEE3
σ(1)
33.06
30.93
29.22
27.31
18.64
41.63
39.68
40.52
45.71
24.31

σ(2)
16.57
15.26
12.71
10.14
8.69
23.81
21.70
22.30
25.99
10.81

Avg
1.61
1.52
1.37
1.19
0.98
1.89
1.77
1.70
1.87
1.17

In-domain
SEE5
σ(1)
σ(2)
31.11 15.44
28.92 14.10
27.16 11.54
25.25 8.99
16.67 7.55
39.69 22.62
37.67 20.49
38.53 21.07
43.72 24.71
22.30 9.67

Avg
1.09
1.10
1.10
0.86
0.92
1.16
1.19
1.17
1.17
1.00

EPE
σ(1) σ(2)11.88 6.94
11.81 6.95
11.65 6.69
10.40 4.93
8.24 5.06
14.81 8.20
14.78 8.21
14.73 8.11
16.17 8.12
11.40 6.09

σ(3)
5.19
5.23
4.97
3.50
3.88
5.84
5.88
5.74
5.46
4.34

Avg
2.19
2.16
2.03
1.84
1.59
2.43
2.39
2.31
2.61
2.03

SEE3
σ(1)
36.94
35.64
33.91
29.87
24.58
44.49
43.60
43.76
48.27
34.82

σ(3)
20.07
19.16
16.74
13.30
12.54
26.17
24.14
25.16
28.84
17.75

Avg
1.99
1.95
1.82
1.62
1.38
2.24
2.19
2.11
2.41
1.82

Out-of-domain
SEE5
σ(1)
σ(3)
34.09 18.25
32.76 17.32
30.92 14.82
26.84 11.46
21.63 10.73
41.74 24.33
40.82 23.28
40.97 23.29
45.56 26.98
31.88 15.83

Avg
1.53
1.56
1.54
1.37
1.27
1.75
1.80
1.76
1.91
1.66

EPE
σ(1)
σ(2)
16.92 10.25
18.89 10.28
16.43 9.73
13.29 7.84
12.11 7.69
22.03 12.73
22.05 12.79
21.88 12.54
26.12 14.40
19.16 10.72

σ(3)
7.83
7.89
7.36
6.03
6.06
9.23
9.33
9.03
10.14
7.77

Table 4: Comparison on UnrealStereo4K. All methods evaluated on ground truth at 3840×2160 given input size 960×540.

(c) PSM + Ours(d) GT, Input
Figure 5: Qualitative Results on UnrealStereo4K. The first row shows the predicted disparity maps while the second row
depicts the corresponding error maps. We zoom-in a patch in all images to better perceive details near depth boundaries.
(a) PSM [4](b) PSM [4] + CE + SM [5]Ground Truth Resolution: Tab. 3 shows the results of ourmodel trained and tested on the stereo data using groundtruth maps at different resolutions, while maintaining the
input size at 960 × 540. Towards this goal, we train ourmodel adopting ground truth disparities 1) resized to thesame resolution as the input using nearest interpolation and2) at the original resolution (i.e. 3840 × 2160). We noticethat sampling points from higher resolution disparity maps
always leads to better results compared to using low resolution ground truth. We remark that the proposed model effectively leverages high resolution ground truth thanks to its
continuous formulation, without requiring additional memory compared to standard stereo networks based on CNNs.

4.4. Comparison to Existing Baselines
We now compare to several baselines [23, 5] which aim
to address the over-smoothing problem. Bilateral medianfiltering (BF) is often adopted to sharpen disparity predictions [23, 43]. Chen et al. [5] address the over-smoothingproblem of 3D stereo backbones using 1) a post-processing
step to extract a single-modal (SM) distribution from thefull discrete distribution; 2) a cross-entropy (CE) loss to enforce a unimodal distribution during training. We reimplement [5] as no official code is available. As [5] has been
proposed for 3D backbones only, we use PSM [4] and HSM[51] as the stereo backbones in the following experiments.
UnrealStereo4K: Tab. 4 collects results obtained from dif-ferent models on both in-domain and out-of-domain test
splits of the binocular UnrealStereo4K dataset. We use
the same input resolution of 960 × 540 for all methods.
While our baseline methods can only use supervision withthe same size as the input, we leverage our continuous
formulation to supervise SMD-Nets using ground truth at
3840 × 2160, on which we also evaluate all methods. For
our competitors, we upsample their outcome using nearest
neighbor interpolation during testing. Both original PSM
and HSM follow the same training setting of our SMD-Nets.
Tab. 4 suggests that BF [23] and SM [5] slightly improve
SEE on both backbones while leading to degraded performance on EPE metrics. Using the CE loss combined withSM [5] leads to effective improvement on both SEE andEPE on the PSM backbone. Interestingly, we notice that
adopting the same CE + SM strategy leads to worse performance on HSM. A possible explanation is that the CE loss
requires to trilinearly interpolate a matching cost probability distribution to the full resolution W × H × Dmax (withDmax denoting the maximum disparity), where HSM predicts a less fine-grained cost distribution compared to PSM,
thus making the cross-entropy loss less effective. Moreover,
we remark that the CE loss is more expensive to compute,compared to our simple continuous likelihood-based formulation and CE + SM can only be applied on 3D backbones.
In contrast, our approach based on the bimodal output representation notably outperforms our competitors on SEE onboth the in-domain and out-of-domain test sets, showing

Method
PSM [4]
PSM [4] + CE + SM [5]
PSM [4] + Ours

Avg
1.10
1.02
0.90

SEE3
σ(1)
20.57
16.12
13.09

σ(2)
9.74
7.53
6.66

Avg
0.99
0.90
0.79

SEE5
σ(1)
17.83
13.80
10.93

σ(2)
9.02
6.94
6.01

EPE
Avg σ(3)0.73 2.49
0.66 2.09
0.59 1.95

Table 5: Comparison on KITTI 2015 Validation Set using boundaries extracted from instance segmentation masks
to evaluate on depth discontinuity regions.
MethodGANet-deep [57]
HD3 -Stereo [54]GwcNet-g [14]PSM [4]
PSM [4] + CE + SM [5]
PSM [4] + Ours

Bg
1.48
1.70
1.74
1.86
1.54
1.69

All Areas
Fg
3.46
3.63
3.93
4.62
4.33
4.01

All
1.81
2.02
2.11
2.31
2.14
2.08

Bg
1.34
1.56
1.61
1.71
1.70
1.54

Non Occluded
Fg
All
3.11
1.63
3.43
1.87
3.49
1.92
4.31
2.14
3.90
1.93
3.70
1.89

Table 6: Comparison on KITTI 2015 Test Set, evaluatedon the official online benchmark. All the reported numbers
represent official submissions from the authors.
MethodPSM [4]
PSM [4] + CE + SM [5]
PSM [4] + Ours

Avg
3.35
2.62
2.61

SEE3
σ(1) σ(2)46.50 29.40
34.80 19.02
34.26 19.83

Avg
2.61
1.83
1.88

SEE5
σ(1) σ(2)41.04 24.87
28.92 14.11
28.71 15.32

EPE
Avg σ(3)4.12 17.43
2.80 12.12
3.03 13.60

Table 7: Generalization on Middlebury v3. All models
are trained on UnrealStereo4K and evaluated on the training
set of Middlebury v3 dataset.
how our strategy predicts better disparities near boundaries.
Moreover, we highlight that we achieve consistently better
estimates on standard EPE metrics compared to the original backbone while performing comparably to the CE +SM baseline. Fig. 5 shows our gains at object boundaries.
KITTI 2015: We fine-tune all methods trained using UnrealStereo4K on the KITTI 2015 training set. Since the provided ground truth disparities are sparse, we rely on thenaı̈ve random sampling strategy to train our model. On
the validation set, we evaluate SEE on boundaries of instance segmentation maps from the KITTI dataset, following the evaluation procedure described in [5]. Furthermore,
we predict disparities on the test set using the same finetuned model and submit to the online benchmark. Tab. 5
and Tab. 6 show our results using PSM as backbone (weprovide additional results on the validation set adopting
HSM in the supplement). Note that our SMD-Net not only
achieves superior performance on both SEE and EPE metrics on the validation set compared to the original PSM and[5] (Tab. 5), but also outperforms both on the test set andis on par with state-of-the-art stereo networks on standard
metrics of the KITTI benchmark (Tab. 6).

4.5. Synthetic-to-Real Generalization
Lastly, we demonstrate how models trained on the synthetic dataset generalize to the real-world domain for bothbinocular stereo and active depth estimation.

(a) Disparity Regression (L1)(b) SMD Head (Bimodal)

Figure 6: Generalization on RealActive4K using the HSM
backbone. The point clouds of standard disparity regression using L1 loss (a) show bleeding artifacts whereas ourbimodal distribution (b) leads to clean reconstructions.
Middlebury v3: Tab. 7 reports the performance of supervised methods trained on the UnrealStereo4K and tested
without fine-tuning on the training set of the Middlebury v3
dataset. We evaluate them using the high-resolution groundtruth. Compared to the original PSM baseline, our SMDNet achieves much better generalization on both SEE andEPE metrics while performing on par with [5].
RealActive4K: Moreover, we fine-tune our active depth
models jointly on active UnrealStereo4K and RealActive4K
with pseudo-ground truth from Block Matching. Fig. 6
shows that this allows for estimating sharp disparity edges
for real captures even though Block Matching does not provide supervision in these areas. In contrast, standard disparity regression fails to predict clean object boundaries.

5. Conclusion
In this paper, we propose SMD-Nets, a novel stereo
matching framework aimed at improving depth accuracy
near object boundaries and suited for disparity superresolution. By exploiting bimodal mixture densities as
output representation combined with a continuous function
formulation, our method is capable of predicting sharp
and precise disparity values at arbitrary spatial resolution,
notably alleviating the common over-smoothing problem inlearning-based stereo networks. Our model is compatible
with a broad spectrum of 2D and 3D stereo backbones.
Our extensive experiments demonstrate the advantagesof our strategy on a new high-resolution synthetic stereo
dataset and on real-world stereo pairs. We plan to extend
our bimodal output representation to other regression tasks
such as optical flow and self-supervised depth estimation.
Acknowledgements. This work was supported by the Intel
Network on Intelligent Systems, the BMBF through the Tüebingen
AI Center (FKZ: 01IS18039A), the ERC Starting Grant LEGO3D (850533) and the DFG EXC number 2064/1 - project number 390727645. We thank the IMPRS-IS for supporting Carolin
Schmitt. We acknowledge Stefano Mattoccia, Matteo Poggi and
Gernot Riegler for their helpful feedback.

References[1] Filippo Aleotti, Matteo Poggi, Fabio Tosi, and Stefano Mattoccia. Learning end-to-end scene flow by distilling single
tasks knowledge. Proc. of the Conf. on Artificial Intelligence(AAAI), 2020. 2[2] Christopher M. Bishop. Mixture density networks. 1994. 2,
3[3] Rohan Chabra, Julian Straub, Christopher Sweeney, Richard
Newcombe, and Henry Fuchs. Stereodrnet: Dilated residual stereonet. In Proc. IEEE Conf. on Computer Vision and
Pattern Recognition (CVPR), 2019. 2[4] Jia-Ren Chang and Yong-Sheng Chen. Pyramid stereo
matching network. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2018. 1, 2, 5, 6, 7,
8[5] Chuangrong Chen, Xiaozhi Chen, and Hui Cheng. On theover-smoothing problem of cnn based disparity estimation.
In Proc. of the IEEE International Conf. on Computer Vision(ICCV), 2019. 2, 5, 7, 8
[6] Zhiqin Chen and Hao Zhang. Learning implicit fields forgenerative shape modeling. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2019. 2[7] Pier Luigi Dovesi, Matteo Poggi, Lorenzo Andraghetti,
Miquel Martı́, Hedvig Kjellström, Alessandro Pieropan, andStefano Mattoccia. Real-time semantic stereo matching. In
Proc. IEEE International Conf. on Robotics and Automation(ICRA), 2020. 2[8] Shivam Duggal, Shenlong Wang, Wei-Chiu Ma, Rui Hu,
and Raquel Urtasun. Deeppruner: Learning efficient stereo
matching via differentiable patchmatch. In Proc. of the IEEE
International Conf. on Computer Vision (ICCV), 2019. 2[9] Divyansh Garg, Yan Wang, B. Hariharan, M. Campbell, K.
Weinberger, and Wei-Lun Chao. Wasserstein distances forstereo disparity estimation. arXiv.org, 2007.03085, 2020. 2[10] Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel
Urtasun. Vision meets robotics: The kitti dataset. International Journal of Robotics Research (IJRR), 32(11):1231–
1237, 2013. 1[11] Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we
ready for autonomous driving? the kitti vision benchmark
suite. In Proc. IEEE Conf. on Computer Vision and Pattern
Recognition (CVPR), 2012. 1[12] Clément Godard, Oisin Mac Aodha, Michael Firman, andGabriel J. Brostow. Digging into self-supervised monoculardepth prediction. In Proc. of the IEEE International Conf.
on Computer Vision (ICCV), 2019. 5, 6
[13] Xiaodong Gu, Zhiwen Fan, Siyu Zhu, Zuozhuo Dai, Feitong
Tan, and Ping Tan. Cascade cost volume for high-resolution
multi-view stereo and stereo matching. In Proc. IEEE Conf.
on Computer Vision and Pattern Recognition (CVPR), 2020.
3[14] Xiaoyang Guo, Kai Yang, Wukui Yang, Xiaogang Wang, andHongsheng Li. Group-wise correlation stereo network. In
Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2019. 2, 8
[15] Eddy Ilg, Tonmoy Saikia, Margret Keuper, and Thomas
Brox. Occlusions, motion and depth boundaries with ageneric network for disparity, optical flow or scene flow estimation. In Proc. of the European Conf. on Computer Vision(ECCV), 2018. 2[16] Alex Kendall and Yarin Gal. What uncertainties do we needin bayesian deep learning for computer vision? In Advancesin Neural Information Processing Systems (NeurIPS), 2017.
2, 6[17] Alex Kendall, Hayk Martirosyan, Saumitro Dasgupta, Peter
Henry, Ryan Kennedy, Abraham Bachrach, and Adam Bry.
End-to-end learning of geometry and context for deep stereo
regression. In Proc. of the IEEE International Conf. on Computer Vision (ICCV), 2017. 2[18] Sameh Khamis, Sean Fanello, Christoph Rhemann, Adarsh
Kowdle, Julien Valentin, and Shahram Izadi. Stereonet:
Guided hierarchical refinement for real-time edge-aware
depth prediction. In Proc. of the European Conf. on Computer Vision (ECCV), 2018. 2[19] Diederik P. Kingma and Jimmy Ba. Adam: A method forstochastic optimization. In Proc. of the International Conf.
on Learning Representations (ICLR), 2015. 5[20] Alexander Kirillov, Yuxin Wu, Kaiming He, and Ross B.
Girshick. Pointrend: Image segmentation as rendering. Proc.
IEEE Conf. on Computer Vision and Pattern Recognition(CVPR), 2020. 2[21] Uday Kusupati, Shuo Cheng, Rui Chen, and Hao Su. Normal assisted stereo depth estimation. In Proc. IEEE Conf. on
Computer Vision and Pattern Recognition (CVPR), 2020. 2[22] Zhengfa Liang, Yiliu Feng, Yulan Guo, Hengzhu Liu, Wei
Chen, Linbo Qiao, Li Zhou, and Jianfeng Zhang. Learning for disparity estimation through feature constancy. In
Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2018. 2[23] Ziyang Ma, Kaiming He, Yichen Wei, Jian Sun, and Enhua Wu. Constant time weighted median filtering for stereo
matching and beyond. In Proc. of the IEEE International
Conf. on Computer Vision (ICCV), December 2013. 7[24] Nikolaus Mayer, Eddy Ilg, Philip Hausser, Philipp Fischer,
Daniel Cremers, Alexey Dosovitskiy, and Thomas Brox. A
large dataset to train convolutional networks for disparity,
optical flow, and scene flow estimation. In Proc. IEEE Conf.
on Computer Vision and Pattern Recognition (CVPR), 2016.
2[25] Max Mehltretter and Christian Heipke. Cnn-based cost volume analysis as confidence measure for dense matching. In
Proc. of the IEEE International Conf. on Computer Vision(ICCV) Workshops, 2019. 2[26] Moritz Menze and Andreas Geiger. Object scene flow for autonomous vehicles. In Proc. IEEE Conf. on Computer Vision
and Pattern Recognition (CVPR), 2015. 5[27] Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. Occupancy networks:
Learning 3d reconstruction in function space. In Proc. IEEE
Conf. on Computer Vision and Pattern Recognition (CVPR),
2019. 2[28] Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik,
Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. NeRF:
Representing scenes as neural radiance fields for view syn-thesis. In Proc. of the European Conf. on Computer Vision(ECCV), 2020. 2[29] Guang-Yu Nie, Ming-Ming Cheng, Yun Liu, Zhengfa Liang,
Deng-Ping Fan, Yue Liu, and Yongtian Wang. Multi-level
context ultra-aggregation for stereo matching. In Proc. IEEE
Conf. on Computer Vision and Pattern Recognition (CVPR),
2019. 2[30] Michael Niemeyer, Lars Mescheder, Michael Oechsle, andAndreas Geiger. Differentiable volumetric rendering: Learning implicit 3d representations without 3d supervision. In
Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2020. 2[31] Michael Oechsle, Lars Mescheder, Michael Niemeyer, Thilo
Strauss, and Andreas Geiger. Texture fields: Learning texture representations in function space. In Proc. of the IEEE
International Conf. on Computer Vision (ICCV), 2019. 2[32] Jiahao Pang, Wenxiu Sun, Jimmy SJ Ren, Chengxi Yang, andQiong Yan. Cascade residual learning: A two-stage convolutional neural network for stereo matching. In Proc. of theIEEE International Conf. on Computer Vision (ICCV) Workshops, 2017. 2[33] Jeong Joon Park, Peter Florence, Julian Straub, Richard A.
Newcombe, and Steven Lovegrove. Deepsdf: Learning continuous signed distance functions for shape representation.
In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2019. 2[34] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer,
James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch:An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems (NeurIPS), 2019. 5[35] Songyou Peng, Michael Niemeyer, Lars Mescheder, Marc
Pollefeys, and Andreas Geiger. Convolutional occupancy
networks. In Proc. of the European Conf. on Computer Vision (ECCV), 2020. 2[36] Matteo Poggi, Fabio Tosi, Konstantinos Batsos, Philippos
Mordohai, and Stefano Mattoccia. On the synergies between machine learning and stereo: a survey. arXiv.org,
2004.08566, 2020. 2[37] Weichao Qiu, Fangwei Zhong, Yi Zhang, Siyuan Qiao, Zihao Xiao, Tae Soo Kim, Yizhou Wang, and Alan Yuille. Unrealcv: Virtual worlds for computer vision. ACM Multimedia
Open Source Software Competition, 2017. 5[38] Gernot Riegler, Yiyi Liao, Simon Donne, Vladlen Koltun,
and Andreas Geiger. Connecting the dots: Learning representations for active monocular depth estimation. In Proc.
IEEE Conf. on Computer Vision and Pattern Recognition(CVPR), 2019. 5[39] Shunsuke Saito, Zeng Huang, Ryota Natsume, Shigeo Morishima, Angjoo Kanazawa, and Hao Li. Pifu: Pixel-aligned
implicit function for high-resolution clothed human digitization. In Proc. of the IEEE International Conf. on Computer
Vision (ICCV), 2019. 2, 5[40] Daniel Scharstein, Heiko Hirschmüller, York Kitajima,
Greg Krathwohl, Nera Nešić, Xi Wang, and Porter Westling. High-resolution stereo datasets with subpixel-accurate

ground truth. In Proc. IEEE Conf. on Computer Vision and
Pattern Recognition (CVPR), 2014. 5[41] Daniel Scharstein and Richard Szeliski. A taxonomy andevaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision (IJCV),
47(1-3):7–42, 2002. 2[42] Katja Schwarz, Yiyi Liao, Michael Niemeyer, and Andreas
Geiger. GRAF: generative radiance fields for 3d-aware image synthesis. Advances in Neural Information Processing
Systems (NIPS), 2020. 2[43] Meng-Li Shih, Shih-Yang Su, Johannes Kopf, and Jia-Bin
Huang. 3d photography using context-aware layered depth
inpainting. In Proc. IEEE Conf. on Computer Vision and
Pattern Recognition (CVPR), 2020. 7[44] Vincent Sitzmann, Julien N. P. Martel, Alexander W.
Bergman, David B. Lindell, and Gordon Wetzstein. Implicit
neural representations with periodic activation functions. Advances in Neural Information Processing Systems (NIPS),
2020. 2, 5[45] Xiao Song, Xu Zhao, Liangji Fang, Hanwen Hu, and Yizhou
Yu. Edgestereo: An effective multi-task learning network forstereo matching and edge detection. International Journal ofComputer Vision (IJCV), pages 1–21, 2020. 2, 5
[46] Alessio Tonioni, Fabio Tosi, Matteo Poggi, Stefano Mattoccia, and Luigi Di Stefano. Real-time self-adaptive deep
stereo. In Proc. IEEE Conf. on Computer Vision and Pattern
Recognition (CVPR), 2019. 2[47] Stepan Tulyakov, Anton Ivanov, and Francois Fleuret. Practical deep stereo (pds): Toward applications-friendly deep
stereo matching. In Advances in Neural Information Processing Systems (NeurIPS), 2018. 2[48] Yan Wang, Zihang Lai, Gao Huang, Brian H Wang, Laurens
Van Der Maaten, Mark Campbell, and Kilian Q Weinberger.
Anytime stereo image depth estimation on mobile devices. In
Proc. IEEE International Conf. on Robotics and Automation(ICRA), 2019. 2[49] Zhenyao Wu, Xinyi Wu, Xiaoping Zhang, Song Wang, andLili Ju. Semantic stereo matching with pyramid cost volumes. In Proc. of the IEEE International Conf. on Computer
Vision (ICCV), 2019. 2[50] Haofei Xu and Juyong Zhang. Aanet: Adaptive aggregation
network for efficient stereo matching. In Proc. IEEE Conf.
on Computer Vision and Pattern Recognition (CVPR), 2020.
2[51] Gengshan Yang, Joshua Manela, Michael Happold, andDeva Ramanan. Hierarchical deep stereo matching on highresolution images. In Proc. IEEE Conf. on Computer Vision
and Pattern Recognition (CVPR), 2019. 3, 5, 6, 7
[52] Guorun Yang, Xiao Song, Chaoqin Huang, Zhidong Deng,
Jianping Shi, and Bolei Zhou. Drivingstereo: A large-scale
dataset for stereo matching in autonomous driving scenarios.
In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2019. 1[53] Guorun Yang, Hengshuang Zhao, Jianping Shi, ZhidongDeng, and Jiaya Jia. Segstereo: Exploiting semantic information for disparity estimation. In Proc. of the European
Conf. on Computer Vision (ECCV), 2018. 2

[54] Zhichao Yin, Trevor Darrell, and Fisher Yu. Hierarchicaldiscrete distribution decomposition for match density estimation. In Proc. IEEE Conf. on Computer Vision and Pattern
Recognition (CVPR), 2019. 2, 8[55] Yurong You, Yan Wang, Wei-Lun Chao, Divyansh Garg, Geoff Pleiss, Bharath Hariharan, Mark Campbell, and Kilian Q
Weinberger. Pseudo-lidar++: Accurate depth for 3d object
detection in autonomous driving. Proc. of the International
Conf. on Learning Representations (ICLR), 2020. 2[56] Jure Žbontar and Yann LeCun. Stereo matching by training a convolutional neural network to compare image
patches. Journal of Machine Learning Research (JMLR),
17(1):2287–2318, 2016. 2[57] Feihu Zhang, Victor Prisacariu, Ruigang Yang, andPhilip HS Torr. Ga-net: Guided aggregation net for endto-end stereo matching. In Proc. IEEE Conf. on Computer
Vision and Pattern Recognition (CVPR), 2019. 2, 8
[58] Youmin Zhang, Yimin Chen, Xiao Bai, Suihanjin Yu, Kun
Yu, Zhiwei Li, and Kuiyuan Yang. Adaptive unimodal cost
volume filtering for deep stereo matching. Proc. of the Conf.
on Artificial Intelligence (AAAI), 2020. 2