Supplementary Material for
SphereNet: Learning Spherical Representations for
Detection and Classification in Omnidirectional Images
Benjamin Coors1,3 , Alexandru Paul Condurache2,3 , and Andreas Geiger1
1

Autonomous Vision Group, MPI for Intelligent Systems and University of Tübingen2
Institute for Signal Processing, University of Lübeck
3
Robert Bosch GmbH

Abstract. In this supplementary document, we present a visualization of thesampling error made by regular convolutional filters on omnidirectional images.
Next, we provide details on the Spherical Transformer Network, which is able
to transform inputs to a canonical orientation in support of a recognition model.
Furthermore, we present additional experiments on the impact of varying object
scale, object elevation and choice of interpolation on the Omni-MNIST classification task. Finally, we give a qualitative comparison of the CNN and SphereNet
transfer learning models on the omnidirectional parked cars (OmPaCa) datasets.

1Sampling Error

When applying a regular convolutional neural network to omnidirectional images, thenormalized average absolute geodesic error is relatively small close to the equator region but grows large towards the poles when compared to the optimal sampling locations used by SphereNet (see Fig. 1).

1.0
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2

latitude φ

1.57

0.00

−1.57
−3.142−1.571

0.000

1.571

3.142

longitude θFig. 1: Average Geodesic Error of the sampling locations of a regular convolution
kernel applied to the equirectangular image representation. Note how the error increases
towards the poles φ ∈ {− π2 , + π2 }.

2Benjamin Coors, Alexandru Paul Condurache, Andreas Geiger

WarpTransformation Encoder

Fig. 2: Spherical Transformer Network. A transformation encoder network outputs
the rotation parameters r specifying the axis v and angle β of rotation. The input Iin iswarped to the output Iout via bilinear interpolation according to r.

2Spherical Transformer Network

As a baseline for the classification and detection tasks, we investigate the use of a Spherical Transformer Network (SphereTN), which aims to undistort objects by performing
a global rotation of the spherical image conditioned on the input image. Compared toSphereNet, the Spherical Transformer needs to learn how to undistort an object. Additionally, it only performs a single global transformation of the input and may therefore
be unable to undistort multiple objects at once.
Similar to a Spatial Transformer Network (STN) [2], the Spherical Transformer
Network uses a localization network to predict a set of transformation parameters which
are used to resample the input image. However, unlike a Spatial Transformer which
typically outputs an affine transformation, the Spherical Transformer Network predicts
and applies a 3D rotation of the spherical image representation.
In order to avoid the gimbal lock problem, we do not represent the axial rotations
using Euler angles but instead leverage the axis-angle representation. More formally,
we rotate 3D points y ∈ R3 located on the surface of the unit sphere S by applying arotation y0 = Ry where the rotation matrix R ∈ SO(3) is given as follows:
R = I + (sin β)[v]× + (1 − cos β)[v]2×(1)

Here, v ∈ R3 is a unit vector defining the rotation axis and β denotes the rotation angle.
[v]× denotes the cross-product matrix and is an element of the Lie algebra so(3).
As v is a unit vector with two degrees of freedom, we are able to encode the rotational component β as its length. More specifically, we parameterize the rotation axisv and rotation angle β with a 3-dimensional vector r = β · v which is the output of
the localization network conditioned on the input image. Note that after prediction, r
can be easily decomposed into β = krk2 and v = r/β. The predicted transformation
is applied to the omnidirectional image via differentiable image sampling with bilinear
interpolation [2]. See Fig. 2 for an illustration.

Supplementary Material for SphereNet3

Table 1: Digit Scale Evaluation on Omni-MNIST. Performance comparison on theomnidirectional MNIST dataset for varying digit sizes.
MethodLarge Medium Small

GCNN [3]17.21
S2CNN [1]11.86
CubeMapCNN
10.03
EquirectCNN
9.61
EquirectCNN+SphereTN 8.22

20.35 28.54
19.90 38.80
11.37 24.46
9.10 8.60
8.69 8.46

SphereNet (Uniform)
SphereNet (NN)
SphereNet (BI)6.91 8.77
6.32 7.51
5.03 5.89

7.16
7.03
5.59

Table 2: Digit Elevation Evaluation on Omni-MNIST. Performance comparison on
the omnidirectional MNIST dataset for varying digit elevation.
|φd | ∈ [0, π8 ] |φd | ∈ [ π8 , π4 ] |φd | ∈ [ π4 , 3π] |φd | ∈ [ 3π, π2 ]88Method
GCNN [3]S2CNN [1]CubeMapCNN
EquirectCNN
EquirectCNN+SphereTN

17.48
11.63
8.70
7.02
6.46

19.48
11.45
8.70
8.64
7.76

18.39
11.41
9.10
9.17
7.74

20.63
11.49
13.80
11.34
11.08

SphereNet (Uniform)
SphereNet (NN)
SphereNet (BI)

6.25
6.15
4.50

6.87
6.36
4.87

7.18
6.23
4.89

9.14
8.67
6.84

Table 3: Interpolation Evaluation for SphereNet on Omni-MNIST. Performancecomparison of SphereNet when varying the location of the bilinear interpolation (BI)layers.
Bi-layers none conv1 conv2 pool1 pool2 conv1,2 pool1,2 all
Test error 7.03

3

6.31

6.25 6.47 6.25

6.18

6.17 5.59

Omni-MNIST Analysis

In order to analyze the image classification results on the Omni-MNIST dataset moreclosely, we conduct further studies. First, we test the effect of varying the digit scaleon the performance of the different models. To do so, we generate the Omni-MNIST

4Benjamin Coors, Alexandru Paul Condurache, Andreas Geiger

dataset at three different scales (small, medium, large). We then train and evaluate eachmodel on each of the three variants. The results are shown in Table 1 and demonstratethat the performance of the nearest neighbor (NN) and bilinear interpolation (BI) variants of SphereNet are not significantly impacted by changes in digit scale. On the otherhand, the uniformly sampled variant (Uniform) of SphereNet drops in performance forsmaller digits, as in this case important information is lost at a smaller object scale. In
contract, the EquirectCNN baseline performs slightly better for small digit scales, assmaller digits minimize the amount of object distortion in the equirectangular image.
The CubeMapCNN, S2CNN and GCNN baselines all show significantly degraded performance for smaller digit sizes, with the S2CNN model particularly struggling withthe classification of digits of smaller size.
Second, we evaluate the performance of all models for different ranges of digit
elevation φd (see Table 2). While CubeMapCNN and EquirectCNN models perform
gradually worse with increasing elevation and object distortion, the SphereNet variants
offer near constant performance for elevations |φd | ≤ 3π
8 . When further investigatingπ
the decrease in SphereNet’s performance for elevations |φd | ∈ [ 3π8 , 2 ], we find it to becaused by a sudden drop in performance to nearly 20% test error at the poles (|φd | = π2 ).
The reason for SphereNet’s loss in performance at this elevation is that its assumptionof an upright object orientation no longer holds at the poles. Unlike SphereNet, S2CNN
encodes full rotation equivariance and therefore is the only baseline with near-constantperformance over all digits elevations.
Finally, we evaluate which network layers in the SphereNet model benefit mostfrom the use of bilinear interpolation (BI) over a nearest neighbor interpolation (NN).
Table 3 indicates that all layers benefit from bilinear interpolation with the benefit beingslightly larger in the second layers and increasing further when both convolutional or
pooling layers utilize bilinear interpolation.

4OmPaCa Detection Comparison

In order to visualize the performance benefit of a SphereNet model compared to theEquirectCNN baseline on the omnidirectional parked cars (OmPaCa) detection task,
we provide a qualitative comparison of the detection results in Fig. 3.

References
1. Cohen, T.S., Geiger, M., Köhler, J., Welling, M.: Spherical CNNs. In: International Conference on Learning Representations (2018) 3
2. Jaderberg, M., Simonyan, K., Zisserman, A., Kavukcuoglu, K.: Spatial transformer networks.
In: Advances in Neural Information Processing Systems (NIPS) (2015) 2
3. Khasanova, R., Frossard, P.: Graph-based classification of omnidirectional images. In: Proc.
of the IEEE International Conf. on Computer Vision (ICCV) Workshops (2017) 3

Supplementary Material for SphereNet(a) EquirectCNN5(b) SphereNet (NN)

Fig. 3: Qualitative Performance Comparison between EquirectCNN and SphereNet(NN) model on the OmPaCa dataset. The ground truth is shown in green, detections areshown in red. Unlike SphereNet, the baseline EquirectCNN model struggles to detectobjects in the polar regions of omnidirectional images (row 1 − 3) and, in general,
outputs less tight bounding boxes (row 4 − 5).