Supplementary Material:Displets: Resolving Stereo Ambiguities using Object Knowledge
Fatma Güney
MPI Tübingen

Andreas Geiger
MPI Tübingenfatma.guney@tue.mpg.deandreas.geiger@tue.mpg.deAbstract
In this supplementary document, we first show how disparities relate to the 3D planes used in our representation.
Next, we provide an illustration of the sampled displets for a single test image and visualize the influence of the displetswith respect to the associated superpixels. For reproducibility, we also list the learned parameter settings which weused throughout all our experiments and show the change in performance with respect to these parameters. Finally, weshow additional qualitative results on KITTI validation set. On our project website1 we also provide a supplementary
video demonstrating the semi-convex hull optimization process which we use in order to simplify the CAD models.

1. Plane Representation
In our formulation, each superpixel is represented as a random variable ni ∈ R3 describing a plane in 3D (nTi x = 1for x ∈ R3 on the plane). We denote the disparity of plane ni at pixel p = (u, v)T by ω(ni , p). In the following, weshow how ω(ni , p) can be derived for a (rectified) pinhole stereo camera with known intrinsics and extrinsics:
u =v=d =fxz+ cu , x = (u − cu )zfz
fy+ cv , y = (v − cv )
z
f
fLz

1
11z

=

xnx + yny + znzzz= (u − cu ) nx + (v − cv ) ny + znz
ffnx
ny= (u − cu )+ (v − cv )+ nz
ff

d =(u − cu )Lnx + (v − cv )Lny + f Lnz

d =Lnx u + Lny v + f Lnz − cu Lnx − cv Lny(1)d =au + bv + c(2)Here, f, cu , cv denote the intrinsic camera calibration parameters and L is the baseline.

2. Illustration of DispletsFig. 1 visualizes a subset of the sampled displets for a single test image which serve as input to our CRF. Wrongdisplet proposals are eliminated during inference if they don’t agree with the observations or overlap with over displets.

1 http://www.cvlibs.net/projects/displets/1

Figure 1: Illustration of Displets. This figure shows a subset of sampled displets for an image by overlaying thecorresponding disparity maps on top of the image regions they have been proposed for.

3. Displet Parameters
The influence of a displet on its associated superpixels is determined by a penalty function λki . We take λki as aweighted sigmoid function of the distance transformλki = λθ

11 + exp(λa − λb DTki )where DTki denotes the Euclidean distance transform of superpixel i with respect to the boundary of displet k. Themodel parameters λθ , λa and λb are learned from training data as described in the next section. In Fig. 2, we visualize
λki for a couple of displets and the associated superpixels. Less transparency indicates a higher penalty, e.g., we allow
more deviation at the displet boundaries. In contrast, λki takes a very large value (λθ ) in the center of the displet (seeSection 4), effectively approximating a hard constraint (λki = ∞). Furthermore, we take the fitness score κk as thedisplet log likelihood (−EΩ̂ ) in order to increase the plausibility of high quality displets. In practice, we subtract thelargest fitness score of all displets originating from the same object proposal region O in order to calibrate for thescores of different regions.

4. Parameter Settings
Table 1 lists the parameter values we used throughout our experiments. The parameters in the left table are obtainedusing block coordinate descent on 50 randomly selected images from the KITTI training set as described in the paper
submission. The values of the parameters in the right table have been determined empirically. Furthermore, Fig. 3
shows the sensitivity when varying the main parameters in our model for both CNN and SGM features. For thisexperiment, we vary one parameter at a time while setting all other parameters to their optimal values listed in Table 1.
We observe that our method is not very sensitive with respect to small parameter changes for most of the parameters.
The performance in reflective regions is highly dependent on the displet parameters as the induced sigmoid function
defines the contribution of the inferred displets to the individual superpixels.
2

Figure 2: Illustration of Displet Influence λki . We penalize superpixels which do not agree (ni 6= n̂k,zi ) with activedisplets (dk = 1). Less transparency indicates a higher penalty, e.g., we allow more deviation at the displet boundaries.
Parameter
Value
Unary Threshold (τ1 )6.98
Pairwise Boundary Threshold (τ2 )3.40
Pairwise Normal Threshold (τ3 )0.06
Pairwise Boundary Weight (θ1 )1.50
Pairwise Normal Weight (θ2 )
586.87
Displet Weight (θ3 )1.53
Occlusion Weight
1.00
Displet Consistency Weight (λθ )6 × 104
Displet Consistency Sigmoid (λa , λb ) [7.90 0.82]

Parameter
Value
Number of Particles
30
Number of Superpixels
1000
Number of Iterations
40
TRWS - Number of Iterations
200
Particle Standard Deviations [0.5 0.5 5]

Table 1: Model Parameters. Here, we show the values of the learned parameters in our model.

3

0.48

3.08

5.68

8.28

0.40

10.88 13.48

CNN−Out−All
CNN−Out−Noc
SGM−Out−All
SGM−Out−Noc

15

Error (%)

2020

Error (%)

2.80

100.000.40

Error (%)

100.40

3.40

6.40

9.40

0.00

0.80

1.20

1.60

0.03

200.10Error (%)

CNN−Out−All
CNN−Out−Noc
SGM−Out−All
SGM−Out−Noc

15100.12

100.04

0.01

0.64

1.24

1.84

2.44

0.60

1.21

1.80

2.40

3.00

Pairwise Boundary Weight20

CNN−Out−All
CNN−Out−Noc
SGM−Out−All
SGM−Out−Noc

15100.03

3.04

0.27

0.51

0.75

0.99

1.23

Displets Consistency Weight (x104)

Displet Weight20

100.0312.40 15.40

0.07

CNN−Out−All
CNN−Out−Noc
SGM−Out−All
SGM−Out−Noc

152.00CNN−Out−All
CNN−Out−Noc
SGM−Out−All
SGM−Out−Noc

150.35

0.67

0.99

1.31

CNN−Out−All
CNN−Out−Noc
SGM−Out−All
SGM−Out−Noc

151051.63

10 15 20 25 30 35 40 45 50

Number of Particles

Displet Consistency Sigmoid (b)Displet Consistency Sigmoid (a)

0.05

20

Pairwise Normal ThresholdOcclusion Weight2015106.40

10

Pairwise Normal Weight
CNN−Out−All
CNN−Out−Noc
SGM−Out−All
SGM−Out−Noc

5.20

CNN−Out−All
CNN−Out−Noc
SGM−Out−All
SGM−Out−Noc

1586.88 286.88 486.88 686.88 886.881086.88

204.0015

Pairwise Boundary Threshold

Unary Threshold

Error (%)

1.60

Error (%)

10CNN−Out−All
CNN−Out−Noc
SGM−Out−All
SGM−Out−Noc

Error (%)

1015

Error (%)

1520
CNN−Out−All
CNN−Out−Noc
SGM−Out−All
SGM−Out−Noc

Error (%)

Error (%)

CNN−Out−All
CNN−Out−Noc
SGM−Out−All
SGM−Out−Noc

Error (%)

2020(a) Reflective Regions. See text for details.

433
3.08

5.68

8.28

0.40

10.88 13.48

544.00

5.20

6.40

CNN−Out−All
CNN−Out−Noc
SGM−Out−All
SGM−Out−Noc

6Error (%)

Error (%)

2.80

7
CNN−Out−All
CNN−Out−Noc
SGM−Out−All
SGM−Out−Noc

60.00

540.400.80

541.20

1.60

0.40

3.40

6.40

9.40

12.40 15.40

Displet Consistency Sigmoid (a)50.120.01

0.60

1.21

1.80

2.40

3.00

Pairwise Boundary Weight47

CNN−Out−All
CNN−Out−Noc
SGM−Out−All
SGM−Out−Noc

65430.04

0.64

1.24

1.84

2.44

3.04

Displet Weight

0.03

0.27

0.51

0.75

0.99

1.23

Displets Consistency Weight (x104)

54

CNN−Out−All
CNN−Out−Noc
SGM−Out−All
SGM−Out−Noc

6543330.10

CNN−Out−All
CNN−Out−Noc
SGM−Out−All
SGM−Out−Noc

62.00

CNN−Out−All
CNN−Out−Noc
SGM−Out−All
SGM−Out−Noc

6Error (%)

60.07

77
CNN−Out−All
CNN−Out−Noc
SGM−Out−AllSGM−Out−Noc

0.05

7Occlusion Weight

Pairwise Normal Weight
70.03

30.00

86.88 286.88 486.88 686.88 886.88 1086.88

4

Pairwise Normal Threshold

33Error (%)

1.60

53

Pairwise Boundary Threshold

Unary Threshold
743

Error (%)

0.48

5

CNN−Out−All
CNN−Out−Noc
SGM−Out−All
SGM−Out−Noc

6Error (%)

457Error (%)

5CNN−Out−All
CNN−Out−Noc
SGM−Out−All
SGM−Out−Noc

6Error (%)

Error (%)

67
CNN−Out−All
CNN−Out−Noc
SGM−Out−All
SGM−Out−Noc

6Error (%)

CNN−Out−All
CNN−Out−Noc
SGM−Out−All
SGM−Out−NocError (%)

77

0.03

0.35

0.67

0.99

1.31

51.63

10 15 20 25 30 35 40 45 50

Number of Particles

Displet Consistency Sigmoid (b)(b) All Regions. See text for details.

Figure 3: Sensitivity to Parameters. This figure shows the change in performance while varying one parameter at atime and keeping all others fixed at their optimal values. Errors are with respect to an error threshold of 3 pixels.

45. Results with Degraded Input Disparity Maps
As our sampling procedure is stochastic, it can handle degraded input disparity maps to some extend. Fig. 4
illustrates this fact on very challenging scenes for which the initial disparity maps contain large number of outliers.

Figure 4: Effect of Input Maps on Results. Each subfigure shows: The input image, the superpixels and the semanticsegments (first row), the input disparity maps, and our result by using these as input (second row).

5

6. Additional Qualitative Results
In this section, we show additional qualitative results on randomly sampled images from KITTI validation set. For
each scene, we show the input image, the superpixels and semantic segments which our method takes as input as wellas our inference results without and with displets and the corresponding error images, encoded from black (≤ 1 pixelerror) to white (≥ 5 pixels error).

Figure 5: Qualitative Results. Each subfigure shows from top-to-bottom: The input image, the superpixels and thesemantic segments which our method takes as input (first row), our inference results without and with displets (secondrow), and the corresponding error maps from ≤ 1 pixel error in black to ≥ 5 pixels error in white (third row).

6

Figure 6: Qualitative Results. Each subfigure shows from top-to-bottom: The input image, the superpixels and thesemantic segments which our method takes as input (first row), our inference results without and with displets (secondrow), and the corresponding error maps from ≤ 1 pixel error in black to ≥ 5 pixels error in white (third row).
7

Figure 7: Qualitative Results. Each subfigure shows from top-to-bottom: The input image, the superpixels and thesemantic segments which our method takes as input (first row), our inference results without and with displets (secondrow), and the corresponding error maps from ≤ 1 pixel error in black to ≥ 5 pixels error in white (third row).
8

Figure 8: Qualitative Results. Each subfigure shows from top-to-bottom: The input image, the superpixels and thesemantic segments which our method takes as input (first row), our inference results without and with displets (secondrow), and the corresponding error maps from ≤ 1 pixel error in black to ≥ 5 pixels error in white (third row).
9

Figure 9: Qualitative Results. Each subfigure shows from top-to-bottom: The input image, the superpixels and thesemantic segments which our method takes as input (first row), our inference results without and with displets (secondrow), and the corresponding error maps from ≤ 1 pixel error in black to ≥ 5 pixels error in white (third row).
10

Figure 10: Qualitative Results. Each subfigure shows from top-to-bottom: The input image, the superpixels and thesemantic segments which our method takes as input (first row), our inference results without and with displets (secondrow), and the corresponding error maps from ≤ 1 pixel error in black to ≥ 5 pixels error in white (third row).
11

Figure 11: Qualitative Results. Each subfigure shows from top-to-bottom: The input image, the superpixels and thesemantic segments which our method takes as input (first row), our inference results without and with displets (secondrow), and the corresponding error maps from ≤ 1 pixel error in black to ≥ 5 pixels error in white (third row).
12

Figure 12: Qualitative Results. Each subfigure shows from top-to-bottom: The input image, the superpixels and thesemantic segments which our method takes as input (first row), our inference results without and with displets (secondrow), and the corresponding error maps from ≤ 1 pixel error in black to ≥ 5 pixels error in white (third row).
13

Figure 13: Qualitative Results. Each subfigure shows from top-to-bottom: The input image, the superpixels and thesemantic segments which our method takes as input (first row), our inference results without and with displets (secondrow), and the corresponding error maps from ≤ 1 pixel error in black to ≥ 5 pixels error in white (third row).
14

Figure 14: Qualitative Results. Each subfigure shows from top-to-bottom: The input image, the superpixels and thesemantic segments which our method takes as input (first row), our inference results without and with displets (secondrow), and the corresponding error maps from ≤ 1 pixel error in black to ≥ 5 pixels error in white (third row).
15