Deep Orientation Uncertainty Learning based on a Bingham Loss
Authors: Igor Gilitschenski
Roshni Sahoo
Wilko Schwarting
Alexander Amini
Sertac Karaman
Daniela Rus
Venue:  (2020)
Keywords: Orientation Estimation
Directional Statistics
Bingham Distribution

Abstract: Reasoning about uncertain orientations is one of the core problems in many perception tasks such as object pose estimation or motion estimation. In these scenarios, poor illumination conditions, sensor limitations, or appearance invariance may result in highly uncertain estimates. In this work, we propose a novel learning-based representation for orientation uncertainty. By characterizing uncertainty over unit quaternions with the Bingham distribution, we formulate a loss that naturally captures the antipodal symmetry of the representation. We discuss the interpretability of the learned distribution parameters and demonstrate the feasibility of our approach on several challenging real-world pose estimation tasks involving uncertain orientations.

Published as a conference paper at ICLR 2020
DEEP ORIENTATION UNCERTAINTY LEARNING BASED ON A BINGHAM Loss
Igor Gilitschenski! Roshni Sahoo', Wilko Schwarting's Alexander Sertac Karaman", Daniela Rus"
Alexarre Amini',
Computer Science and Artificial Intelligence Lab, MIT
2 Laboratory for Information and Decision Systems, MIT (igilitschenski, rsahco, wilkos, amini, sertac, rusf@mit edu
ABSTRACT
Reasoning about uncertain orientations is one of the core problems in many per- ception tasks such as object pose estimation or motion estimation. In these scenar- ios, poor illumination conditions, sensor limitations, appearance invariance may result highly uncertain estimates. In this work, we propose a novel learning- based representation for orientation uncertainty. By characterizing uncertainty over unit quaternions with the Bingham distribution, we formulate a loss that nat- urally captures the antipodal symmetry of the representation. We discuss the inter- pretability of the learned distribution parameters and demonstrate the feasibility of our approach several challenging real-world pose estimation tasks involving uncertain orientations.
INTRODUCTION
Reasoning about uncertain poses and orientations, specifically 3-dimensional (3d) positions and 3-axes orientations, is one of the main inference tasks in computer vision (Satteer et al., 2019) robotics (Glover et al. 2011). aerospace (Crassidis & Markley 2003), and other fields. Proper representation and estimation of uncertainty is im- portant, e.g. when dealing with structural ambiguities in object pose estimation coping with sensor corruption.
In vision and robotics tasks, high levels of pose uncer- tainty may occur due to potentially adversarial conditions that arise in real-world scenarios. A principled approach to uncertainty quantification allows for better execution of planning and Iltuation-awarenee tasks such as grasping, tracking, and motion estimation.
When representing uncertainties over poses, the position be modeled using Gaussian distribution. This ap- proach is well-motivated by the Central Limit Theorem and widely used in probabilistic deep learning models. However, this paradigm cannot be as easily applied to modeling periodic quantities, such the orientation of an object. Therefore, Gaussian models become unsuit- able particularly in learning regimes involving high un- certainties where one cannot assume local linearity of the underlying space. this work, we set out to develop a principled probabilistic deep learning approach capable of coping with uncertain orientations.
Objects from the T-LESS dataset and the corresponding orienta- tion uncertainty predicted by the model trained the newly proposed Bingham loss, which is capable of capturing rota- tional symmetries.
Figure I:
Currently, most deep learning approaches that predict poses or rigid-body motions suffer from at least one of three drawbacks: 1) they do not model the uncertainty at all and merely focus on the accuracy of the predicted pose, 2) they make simplifying assumptions not taking into account that the orientation is defined periodic manifold, making the approach

---

Published as a conference paper at ICLR 2020
only suitable in low-noise regimes, or 3) even when trying to account for periodicity, dependency is assumed between the orientation axes and usually an Euler angle- mngle-based representation is required. To this point, there are probabilistic deep learning models for uncertainty of orientations that take the geometry of the underlying domain into account.
In this work, we close this research gap by proposing a probabilistic deep learning model inspired by Directional Statistics (Mardia & Jupp 1999). We present a loss based on the Bingham distri- bution (Binghami 1974), an antipodally symmetric distribution on the sphere. With this loss, we represent uncertain orientations by modeling uncertainty over unit quaternions. Our contributions involve Bingham parameter learning using backpropagation through sure orthonormalization, efficient approximate evaluation of the normalization constant of the Bing- ham distribution from a lookup table, and backpropagating through an interpolation scheme during learning. We also discuss interpretabilityy of the Bingham distribution parameters and establish the feasibility of the approach through extensive evaluations.
Crram-Schmidt method to en-
In summary, this work makes the following contributions: 1) We propose the Bingham loss, novel loss function for deep learning-based predictions of orientations and their uncertainty. 2) We provide methodology for making the newly proposed loss and its normalization constant computationally tractable in deep learning pipeline. 3) We demonstrate multi-modal orientation prediction using a Bingham variant of Mixture Density Networks. 4) We demonstrate how our approach outperforms the state-of-thee -ofthe--- on challenging pose and orientation estimation tasks"
2 BACKGROUND: BINGHAM DISTRIBUTION FOR UNCERTAIN ORIENTATIONS
Unit quaternions are a widely used representation for object orientation in 3d space. They are more compact than rotation matrices, and unlike Euler angles, do not suffer from degeneracies such as Gimbal lock. Additionally, quaternions provide a convenient mathematical notation where the quaternion product, qI 92. of two unit quaternions qı, 92 H, results in concatenation of the rotations represented by each of the quaternions individually. A full introduction to this representation by given in Kuipers (1999) and notational aspects are discussed by /Sommer et al. (2018). In this work, a quaternion 91 + Ø2 + q3k + 91 will be interpreted as vector E IR. It is important to note that the definition of unit quaternions is equivalent the vector q being of unit length ||و|| = 1. Furthermore, the quaternions q and -q represent the same orientation. Therefore, representing uncertain orientations using quaternions requires a probability distribution on the 4d hypersphere that exhibits antipodal symmetry, i.e. for the density function f()) of this distribution f(q) = ((-q) has to hold.
A probability distribution exhibiting these properties was proposed by Bingham (974)). It arises by conditioning a zero mean Gaussian to unit length. The Bingham distribution is given in terms of its p.d.f. as p(x; M, Z) (MZM ))- exp(x MZM x) ,weee
R with ||x|| 1, N(MZM is a normalization constant, M E R orthogonal, and Z = diag(æ1, 22, 23, 0) e 44x4 diagonal, with diagonal entries 2; <= 0 and the last entry being zero. We use the notation Bingham (M, Z). The restriction on the range of the diagonal entries in Z has numerical and rep- resentational convenience reasons. It can be shown that Bingham(M. (M, Z)- Bingham Z + cl) for all e E R with I E R4x4 denoting the identity matrix. Similarly, changing the order of diagonal entries in Z has no effect on the distribution as long as the columns in M are permuted accordingly.
In the definition above. the parameters M and Z bear some similarity to the mean and variance of a Gaussian. The density obtains its maxima at +M,, (the fourth column of M) which can be thought of as a mean orientation respecting the manifold structure. The diagonal entries of Z can be interpreted as dispersion parameters, and the first three columns of M can be interpreted as the directions of the dispersion (the Gaussian analog is the orientation of the covariance ellipsoid). Bingham distributions allow for representation of uniform priors over individual axes or even the entire space, making them superior to Gaussians in any of the usual orientation representations.
'Code available httpt:
wtchhesskiiiiittiiiiocok /daennsskkddeee.e /deep_binghahn
2

---

Published as a conference paper at ICLR 2020
10
06
0.5
0.2
-35
0.0
-10
1.0
0.
0.5
0.0
0.5
25
0.5
1.0
-1.0
1.0
S
S
(c) g
Figure 2: Densities the Bingham distribution represented for different dimensionality. For the circular case (a), the density is shown as a function of unit vectors on the plane. For the spherical case (b), it is shown as a heatmap on a 3d unit sphere. For the 4d case (c), which of our particular interest, visualize the mode of the Bingham in terms of the coordinate system orientation repre- sented by the corresponding quaternion. Then, we draw samples from the distribution and visualize each sample a potential coordinate arrow endpoint for each axis (i.e. each sample drawn from the Bingham distribution is represented by three points in the plot). This representation allows us to simultaneously represent the orientation and the corresponding uncertainty.
One of the main challenges of using the Bingham distribution is the computation of its normalization constant
(MZMT) = f / exp(q MZM q) dq.
which is Hypergeometric function of matrix argument Herz 1955)
Evaluating these functions
imposes high computational burden and is still an area of active research (Koev & Edelman 2006 Kume al., 2013 Koyama al., 2014: Kume & Sei 2018). Using the transformation theorem and the fact that M orthogonal, the normalization constant can be simplified as N N(MZM = N N()), 3) and motivating the of pre-
making it merely function of the three parameters 2ạ computed lookup tables in practice.
Furthermore, make the uncertainty of a Bingham Distribution more interpretable in practice, propose the use of Expected Absolute Angular Deviation (EAAD) which is defined as
EAAD(Z)
0(q c) p(q: I, Z2 dq.
91|=1
where p(-) is the Bingham(L, Z) density, I is the identity matrix, e 00,0,0,1] is the vector corresponding to the unit quaternion representing the identity and 0(q, e) 2 arccos(] e)() denotes the angular distance between q and The EAAD describes the expected angular deviation from the "mean" orientation. It can be loosely thought of as the orientation counterpart to the standard deviation in Fuclidean space. For the same reason as the normalization constant, EAAD computation does not involve the parameter M.
DEEP ORIENTATION UNCERTAINTY LEARNING
The Bingham distribution is the main component of the proposed probabilistic framework for rep- resenting deep learned uncertain orientations. Drawing inspiration from Mixture Density Net- works (Bishop 1994), we propose using the Bingham distribution's negative log-likelihood loss function
L(y, M, Z) = - log M, Z) = -y" TMM y + log N(Z),
with M, as defined above and y being the orientation label given in the training data. We use a neural network to learn M and Z, end-to-end, directly from the input data (e.g. RGB images). From
3

---

Published as a conference paper at ICLR 2020
15
Bingham Loss
Gram-Schmikl
M
Orthonomalization
A/y ML 23
2022
(00000
Convolutiona Encoder
Constrained
Inte poterrplltto
diggonalization
Input
Lookup Table
Figure 3: The proposed orientation uncertainty estimation pipeline predicts the parameters Bingham distribution for representing uncertain unit quaternions. Backpropagation through in- terpolator and use of a lookup table allows for avoiding evaluations the computationally expensive Bingham normalization constant.
this prediction, the point estimate of is obtained as ý = M.,1 the highest diagonal entry of Z and thus represents one of the modes of the distribution (the other being- y due to antipodal symmetry).
1 as the last column corresponds to
No costly evaluation of the normalization constant required and no major computational chal- lenges arise in the special case where the dispersion parameter Z is known and not predicted by a neural network. However, as our goal is the modeling of uncertainty, we propose methods for modeling M and Z as well as backpropagating through N(Z).
3.1 MODELING OF DISTRIBUTION PARAMETERS
In order to obtain predictions M and 2, we require 19 dimensional output (o E R 19 of the pre- dictor network (3 outputs for Z, 16 outputs for M). On its own, these outputs do not satisfy the above-mentioned constraints on the Bingham distribution parameters. Thus, we define the differen- tiable transforms TM 16 + p4x4 and Tz R R 4x4 that transform these outputs such that the constraints are satisfied.
The transform Tz is obtained as (2(01, 02 02, 0z) diag($1, 22 23. 0) with é, exp(o:). For computing M, we first subdivide 01,-.., 019 into four vectors v, Ra 1,..., 4). Then, apply the Gram-Schmidt orthonormalization method to these vectors according m Normalize(v; Σi=1 (m,, vz) my) with i E 21,2,3,4 3) and Normalize(x) x/ ||x||- Fi- nally, the prediction M is obtained as (M = _m..... mi). and is orthogonal by construction.
3.2
BACKPROPAGATION THROUGH THE BINGHAM NORMALIZATION CONSTANT
As mentioned carlier, computation of the Bingham normalization constant is numerically burden- some. This also true for its derivatives which can be shown be proportional to the normalization constant of Bingham distributions of higher dimension (Kume & Wood 2007). A forward-backward pass for one single data point requires evaluations of hypergeometric functions matrix argument.
We avoid this by precomputing a lookup table for (ZZ at L different locations t, (with Z, (it, O). This table is then used to build interpolator (z) Σ w;o([lz t;ll) with 3 and denoting radial basis function. The weights w; can also precomputed during generation of the interpolator. Thus, we can approximate N (Z) z fN (z) and V (Z) ~ (zf To the best our knowledge, this the first time that lookup table based interpolation mechanism has been included in the computation graph of neural network.
3.3 MULTI-MODAL PREDICTION
Bingham variant of Mixture Density Networks can be used to obtain multi modal predictions. However, MDNs are hard to train even in the Gaussian case. Following the discussion in Makansi
4

---

Published as a conference paper at ICLR 2020
al. 20199) separate the training in two stages. In the first stage, only learn to predict M and assume the dispersion to be fixed with = diag(-a, -a, -a, 0). In practice a E R can usually be set to merely scales the cost term. In the second stage, train to predict M and Z jointly. Our evaluation will show that in high uncertainty regimes, this training method is also helpful for the unimodal case.
EXPERIMENTS
In this section we evaluate the proposed Bingham loss on its ability learn calibrated uncertainty estimates for orientations. This goes beyond comparing point estimates of orientations; we evaluate how well the estimated distribution of orientations can explain the data. We will also show that the Bingham distribution representation is capable of capturing ambiguity and uncertainty in SO(3) better than state-of of-e------httrrr-- approaches.
We investigate characteristics and behaviors by training neural networks two head- pood datasets, IDIAP (Odobez) 2003) and UPNA (Ariz et al. 2016), as well as the object pose dataset T- LESS (Hodañ al.) 2017p. We show the capability of calibrated uncertainty estimation by applying artificial label-noise to IDIAP and UPNA and observing that the Bingham parametrization allows for accurate prediction of uncertainty. In addition calibrated uncertainty estimation, we demon- strate advanced capabilities in the face of object orientation ambiguity on the T-LESS dataset by visualizing the predicted distributions for different orientation ambiguous objects, e.g. symmetric, and comparing to objects with clear orientation.
4.1 ARCHITECTURE AND EXPERIMENTAL SETUP
We seek to estimate the Bingham distribution parameters directly from image data. Our pipeline is shown in Figure and begins by passing image input convolutional encoder, in case standard ResNet- 18 network followed by fully connected layer, populating the entries of 01, O2, and U1 U2. U3 44. Subsequently, Gram-Schmidt orthonommaiizaiinn of U1 12, uate the Bingham loss, the normalizer (Z)) needs be queried from the RBF lookup table, Section 3.2] Differentiation of the interpolator via finite differences enables to back-propagen through the entire pipeline. All models were implemented in PyTorch and optimized with the Adam optimizer.
is computed by constrained diagonalization yields M, as described in Section 3.1 To eval-
01 02, 03. and
We create the lookup table by numerical integration. More precisely, we use Scipy's tplquad method to compute triple integral for each Z in the table. We set the relative error tolerance to le-3 and the absolute error tolerance to le-7. The actual computed integral is
2m [/
S exp (t(01, o2 03) Z Zt(0i, 02,, Ø2))))) sin(á1)² 2 sin(()2 déi dø² dØs
(Z)
with
sin Φ1) sin((2) sin sin(()) sin( cos (33) sin( (os(
cos(
to account for a transformation of coordinates from unit quaternions to 4d spherical coordinates. Because the Bingham log likelihood as our optimization objective, we compute the logarithm before the interpolation to avoid failure at locations where the interpolator wrongly outputs negative values.
4.2 BASELINES
We compare our work with the approach proposed by Prokudin et al. (2018). It also uses loss based on directional statistics, specifically the Von Mises distribution. The Von Mises distribution be thought of as circular analog of the Normal distribution. In order apply this approach to our setting, orientations are modeled with Euler angles. The loss then consists of the sum of log-likelssoods for each angle. While this approach properly account for periodicity of the
5

---

Published as a conference paper at ICLR 2020
underlying data, expect it to fail in cases where the underlying uncertainty is not axis aligned because it does not account for dependencies between uncertain rotation axes.
Furthermore, also evaluate several different representations of the parameter matrix M. We consider the classical Gram-Schmidt (CGS), modified Stram-Schmide (MGS), and the matrix rep- also include two
resentation of the quaternion (QM) used by Birdal et al. (2018). Finally, probabilistic orientation prediction baselines. The first one is based on a Mean Square Error (MSE) between the predicted and ground truth quaternion. The second one is based on a cosine loss applied to each angle's biternion as discussed by Prokudin et al. (2018).
4.3 EVALUATION METRICS
To evaluate error metrics over predicted orientations, it is unsuitable to compute the RMSE over angles, since does not sufficiently consider the spherical nature of the underlying data. Instead, we make of the Mean Absolute Angular Deviation (MAAD) which has also been used by Prokudin ict al. (018). is based on the angular distance between two angles defined above. We also com- pute the EAAD assess the quality of the results. Additionally, the difference between EAAD and MAAD serves as an indicator of the quality of the predicted uncertainty. The acceptable differ- ence in practice is application dependent. For the cases of the Von Mises distribution parameters, EAAD computation is carried out in similar way as for the Bingham defined above. EAAD is calculated over the learned dispersion parameters for each example and averaged. The quality of the respective model is measured in terms of log-likellng leglikklihood to indicate the goodness of an individual fit. For MDNs, we additionally report Mean Minimum Absolute Angular Deviation (MMAAD), which uses the component closest to ground-truth for absolute angular deviation computation. The MAAD and EAAD for MDNs are computed in a per-component fashion and then weighted using the predicted mixture weights.
4.4 CALIBRATED UNCERTAINTY ESTIMATION
We evaluate the distribution fit on the head pose datasets UPNA and IDIAP, which consist of head images from a video of several people inside a room. Each image is annotated with head orientation given by pan, tilt and roll angles. We use these datasets as they provide accurate labels and allow for carrying out experiments involving artificial label noise.
The results on the raw dataset are shown in Table [ They demonstrate that the general performance for point esti- mates, indicated by MAAD, of the Bingham distribution remains on a similar level
UPNA
IDIAP
EAAD
MAAD
LL
EAAD
MAAD
LL
BD-CGS
0.10
0.11
4.70
0.10
0.09
4.49
BD-MGS
0.10
0.13
3.87
0.10
0.10
4.58
BD-QM
0.10
0.16
0.31
0.10
0.09
4.74
VM
0.13
0.11
3.69
0.12
0.09
2.08
the Von Mises distribu- tion and the non-pro poobbbbbaii approaches. In this setting, most motions of the subjects' heads are aligned with the grav- ity axis allowing both distri- butions to successfully capture the noise. However, the Bingham still attains higher log-likelihood and smaller gap between MAAD and EAAD. Similarly, the parametrization of the concentration matrix M has a relatively small impact on the estimation performance. Although MGS has stronger robustness guarantees than CGS (the latter has a quadratic dependency on the condition number of the input matrix, see Giraud et al. (2005) for discussion of both), the condition of the input is not poor enough to impact performance. While the quaternion matrix approach is easier to train, it also loses some of the ex- pressiveness of the Bingham distribution because the underlying mapping (from quaternions to the space of orthogonal matrices) is not surjective.
MSE
0.12
0.10
Cosine
0.12
0.10
Table 1: Bingham (BD), Von Mises (VM), Mean Square Fr-
ror (MSH), and cosine based loss prediction performance on raw UPNA and IDIAP datasets.
To estimate how well the predicted uncertainties calibrated, we add artificial noise by drawing random perturbations from the Bingham distribution with varying 21 22, and 23 parameters and applying them to the quaternion labels before training. Both UPNA and IDIAP contain negligible

---

Published as a conference paper at ICLR 2020
BAAD
-22
EAAD MAAD
1E -22
EAAD MAAD
EAAD MAAD
-22 -2g
EAAD
MAAD
Tabel noise
Na noion
0
70
30
20
0.52
250 150
75
0.23
500
100
300
0.13
UPNA
197 497 197 0.10
19 19
19
0.51
186 105
63
0.3
130
114
74
0.25
305
300
293
0.13
404
104
20.5
0.89
-7 4:30
1:15
0.29
#35
110
114
0.26
116
1:16
=17
0.20
IDIAP
499 499 499 0.10
18
0.55
167
93
K7
76
0.25
300
704
0.13
#05
+ 0.5
+0.3
0.59
=17
+:0
#3
0.29
48
28
#7
0.28
24
225
335
0.20
Table 2: Testing accuracy of uncertainty calibration. Prior to training, we perturb the labels with
noise sampled from the Bingham distribution with M equal to the identity and varying 21, 22, 23- The figures represent the different noise distributions.
amounts of noise, the dispersion of the noise distribution should be captured by the learned Z to high accuracy. An evaluation of uncertainty and label noise is shown in Table [ For the case of no noise, the Bingham uncertainty parameters approximate the highest certainty levels represented in the lookup table. Thus, the maximum and minimum values in the lookup table automatically become the bounds of what certainty levels can be represented by the proposed loss. When noise is applied to the training labels, the learned uncertainty parameters closely match the dispersion of label noise, the predicted EAAD accurately captures the EAAD corresponding to the dispersion of the label noise distribution. We note that the MAAD is slightly higher than the true and estimated EAAD values. This overconfidence effect is typical in probabilistic deep learning and also arises when predicting the parameters of a Gaussian (Amini et al. 2019). In addition, we evaluated scenario where the noise newly sampled and applied to the true labels in each iteration (rather than corrupting the labels with the sampled noise prior to training). In this scenario, the EAAD computed from the learned dispersion parameters, the true EAAD, and the MAAD are approximately equal in value. While this scenario is less realistic in practice (and thus not visualized), it provides further evidence for representational consistency of the loss.
4.5 HANDLING AMBIGUOUS DATA
We use the T-LESS dataset for evaluating the proposed model using ambiguous data. It contains images of 30 different textureless objects taken from different cameras. We use the Kinect RGB single-object images all of which are split into training, test, and validation sets. At a coarse scale most of the objects in the dataset exhibit rotational or other symmetries. At a finer scale some of these ambiguities disappear due to smaller structures. On the one hand, we expect those to be more challenging to learn. On the other hand, capturing these structures allows for very precise orientation estimation. To be able to disregard these structures, create variant of T-LESS where we add blur to each image using a uniform 10px x 10px kernel.
We carry out sets of experiments. In the first set of experiments, we train orientation estimation models for 5 epochs using the Bingham loss (BD-5) and the Von Mises loss (VM-5) on the blurred and original set of images. This allows to investigate the uncertainty estimation properties before the network captures the finer grained structures. In the second set of experiments, we use the original set of images to evaluate multi-modal orientation prediction using the two-stage training approach for models with 1 (BD-MDN-I), 2 (BD-MDN-2), and 4 (BD-MDN-4) mixture components. Each stage is carried out for 30 epochs. The comparison methods use Von Mises (VM), Mean Square Error (MSE), and Cosine losses with an overall training duration of 60 epochs (or until convergence that is earlier).
Method
Log-likellhhod
MAAD
EAAD
VM-5
-0.12
0.48
0.33
BD-5
2.82
1.57
1.58
VM-5 blur
-0.03
0.56
0.44
BD-5 w. blur
2.71
1.59
1.58
Table 3: Results on the T-LESS dataset in the high uncertainty regime.
The results for the first set of experiments are visualized in Table 3 As expected, both approaches are on average far off in terms of the true orientation. While Von Mises performs better on the MAAD, we observe that there is larger difference between the MAAD and EAAD values for the
7

---

Published as a conference paper at ICLR 2020
Von Mises distribution than the Bingham distribution. This indicates that the uncertainty estimates of the Von Mises distribution may be overconfident. On the other hand the Bingham distribution better captures the uncertainty over individual axes. One interesting insight is that allowing for uniform distributions over individual non-aligned periodic axes can make it hard for the learning method to pick up on the proper pose and thus may require pre-training the pure pose estimation task in such regimes.
In the second set of experiments, as visu- alized in Table we use this training strat- egy for all Bingham MDN models result- ing in robust convergence behavior. How- ever, the unimodal Bingham (BD-MDN- 1) converges slower than Von Mises (VM) thus achieving higher MAAD, which is adequately captured by the Bingham's EAAD. For multiple mixture components, obtain very low MAAD and can ob- serve again the phenomenon of the lookup table limitations in the EAAD. Thus, the
Method
Log likelihood
MAAD
MMAAD EAAD
VM
3.73
0.10
0.17
BD-MDN-1
5,00
0.20
0.21
BD-MDN-2
6.17
0.07
0.06
0.12
BD-MDN-4
6.19
0.06
0.05
0.10
MSE
0.22
Cosine
0.10
Table 4: Results on the T-LESS dataset involving multi modal prediction.
MAAD achieved during the first training stage can not only be used for inspecting the network's accuracy but also for determining the minimum Z parameter values stored the lookup table. An- other interesting phenomenon can be observed in the EAAD and MAAD of the VM loss. As the representation required by Von Mises assumes that each axis is independent, EAAD computed per rotation axis. This results in an overapproximation of the uncertainty overall. For the non- probabilistic losses, the cosine loss achieves better performance which probably due better consideration of the underlying geometry. In summary, while the proposed Bingham loss shares the general challenges of training Mixture Density Networks, it better captures the underlying noise structure by explicitly modeling dependencies between rotation axes.
5 DISCUSSION AND RELATED WORK
Quantifying and representing uncertainty by and in neural networks has been subject of extensive research initially focused on modeling probability distribution parameters (Nix & Weigend, 1994) and mixture distributions (Bishop, 1994) as neural network outputs. More recent approaches fo- cus on improving understanding of the underlying uncertainties (Kendall & Gal 2017), providing scalable techniques for estimating predictive uncertainty @Lakshminarayanal et al. 2017). and sta-
bilizing training to avoid mode collapse (Makansi et al. 2019). The present work is orthogonal to these approaches in the sense that it focuses on proper modeling of the underlying geometric domain and coping with computationally demanding normalization constant.
Handling of poses and orientations has been extensively studied in the context of Bayesian filtering for applications such spacecraft attitude estimation (Crassidis & Markley 2003) and ego-motion estimation (Bloesch et al., 2015). where one can often assume the underlying uncertainties to be small. This allows for leveraging local-linearity and using the Gaussian distribution. Recently, methods based on directional statistics enabled modeling of high uncertainty levels for inferring orientations (Gilitschenski et al. 2016) and full poses (Glover et al.) 201T Glover Kaelbling 2014 Srivatsan et al. 2016) by using the Bingham distribution. Drawing inspiration from these results, this work extends the applicability of these approaches to probabilistic deep learning models.
Particularly in computer vision, deep learning has been applied to spherical regression and pose estimation problems (Liao et al.
2019 Huang et al. 2018). These applications involve inferring
object (Brachmann et al. 2014 Hodañ et al. [2088] Li et al. 2018b|a Manhardt et al.) 2019) (Sun- dermeyer et al. 2018 Tekin et al. 2018 Wang et al. 2019b a), body (Yang et al. 2019). and camera poses (Clark et al. 2017 Sattler et al. 2019 Wang al. 2017 2018). In all of these scenarios there is a multitude of sources for potentially high uncertainties such as the use of low-resolution data (e.g. tracking pose of distant pedestrians), absence of textures (e.g. when operating depth data), or motion blur (e.g. due to high speeds in ego-motion estimation). However, most of the existing approaches merely focus inferring the pose but do account for the underlying uncertainty.
8

---

Published as a conference paper at ICLR 2020
The representation proposed in our work closes this gap by allowing for neural networks to output well-calibrated orientation uncertainty estimates.
Only few approaches consider modeling the uncertainty of orientations for deep learning based pose estimation. PoseRBPF by Deng et al. (2019) discretizes the orientation space into over 190000 bins and learns codebook to allow for tractable inference. In contrast to that approach, we do not require an a priori discretization and can directly obtain interpretable estimates. Similarly to us, Prokudin et al. (2018) propose loss based directional statistics. By making use of the Von Mises distribution, their work can properly account for periodicity of circular data. However, as we have shown in our evaluations, this approach cannot properly account for dependencies between different axes and thus, struggles when the underlying uncertainty is not axis aligned.
6 CONCLUSION
In this work, we introduced the Bingham loss, a loss function based on the Bingham distribution that enables neural networks to predict uncertainty over unit quaternions and thus uncertain orienta- tions. This allows for using Dolaaton-)symmeetre objects and ambiguous sensor data in the context of pose and orientation estimation. addition, we demonstrate how cope with intractable likeli- hoods in deep learning pipelines by using non-linear interpolation and lookup tables as part of the computation graph.
The presented approach is directly usable in existing probabilistic deep learning techniques. More- over, we demonstrate its applicability for mixture density models. The choice of parametrization remains one of the main design decisions in pose and orientation estimation pipelines. Our work supports the case for using quaternions over other parametrizations for deep learning. It also mo- tivates further research how to properly model dependencies between uncertain periodic and non-periodic quantities.
ACKNOWLEDGMENTY
This work was supported part by NSF Grant 1723943, the Office Naval Research (ONR) Grant N00014-18 1-0014188-11 11114--8---2888 -2-300, and Toyota Research Institute (TRI). This article solely reflects the opinions and conclusions of its authors and not TRI, Toyota, or any other Toyota entity. Their support is gratefully acknowledged.
REFERENCES
Alexander Amini, Wilko Schwarting, Ava Soleimany, and Daniela Rus. Deep Evidential Regression. arXiv 1eppiit:191100220 2019.
Mikel Ariz, José J. Bengoechea, Arantxa Villamueva, and Rafael Cabeza. A novel 2D/3D database with automatic face annotation for head tracking and pose estimation. Computer Vision and Image Understanding, 148:201-210, 2016.
Christopher Bingham. An Antipodally Symmetric Distribution on the Sphere. The Annals of Statis- fics, 120::12011225, 1974.
Tolga Birdal, Umut Simsekli, Mustafa Onur Eken, and Slobodan Ili.. Bayesian Pose Graph Op- timization via Bingham Distributions and Tempered Geodesic MCMC. In Advances in Neural Information Processing Systems (NeurIPS), 2018.
Christopher M. Bishop. Mixture Density Networks. Technical report, Neural Computing Research Group, Aston University, 1994.
Michael Bloesch, Sammy Omari, Marco Hutter, and Roland Siegwart. Robust visual inertial odom- etry using a direct baF-d approach. In Proceedings of the International Conference on Intel- ligent Robots and Systems (IROS), 2015.
Eric Brachmann, Alexander Krull, Frank Michel, Stefan Gumhold, Jamie Shotton, and Carsten Rother. Learning 6D Object Pose Estimation Using 3D Object Coordinates. In Proceedings of the European Conference on Computer Vision (ECCV), 2014.

---

Published as a conference paper at ICLR 2020
Ronald Clark, Sen Wang, Hongkai Wen, Andrew Markham, and Niki Trigoni. VINet: Visual-Inens Odometry as a Sequence-ter -oeee-ttt--eeeeeeeeee Learning Problem. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), 2017.
John L. Crassidis and F. Landis Markley. Unscented Filtering for Spacecraft Attitude Estimation. Journal of Guidance, Control, and Dynamics, 26(4):536-542, 2003.
Xinke Deng, Arsalan Mousavian, Yu Xiang, Fei Xia, Timothy Bretl, and Dieter Fox. PoseRBPF:
A RaaoBBlakkwellized Particle Filter for 6D Object Pose Estimation. In Proceedings of Robolics: Science and Systems (RSS), 2019.
Igor Gilitschenski, Gerhard Kurz, Simon J. Julier, and Uwe D. Hanebeck. Unscented Orientation Estimation Based the Bingham Distribution. Transactions Automatic Control, 61(1), 2016.
Luc Giraud, Julien Langou, Miroslav Rozložník, and Jasper van den Eshof. Rounding error analysis of the classical Sraa--chmiidt orthogonalization process. Numerische Mathematik, 101(1), 2005.
Jared Glover and Leslie Kaelbling- Tracking the spin on ping pong ball with the quaternion Bingham filter. In Proceedings of the International Conference on Robotics and Automation (ICRA), 2014.
Jared Glover, Radu Rusu, and Gary Bradski. Monte Carlo Pose Estimation with Quaternion Kernels and the Bingham Distribution. In Proceedings Robotics: Science and Systems (RSS), 2011.
Carl S. Herz. Bessel Functions of Matrix Argument. The Annals of Mathematics, 61(3):474, 1955.
Tomáš Hodañ, Pavel Haluza, Stěpán Obdržálek, Jiñ Matas, Manolis Lourakis, and Xenophon Zabu- lis. T-LESS: An RGB-D dataset for 6D pose estimation of texture-less objects. Winter Conference Applications of Computer Vision (WACV), 2017.
Tomáš Hodañ, Frank Michel, Eric Brachmann, Wadim Kehl, Anders Glent Buch, Dirk Kraft, Bertram Drost, Joel Vidal, Stephan Ihrke, Xenophon Zabulis, Caner Sahin, Fabian Manhardt, Federico Tombari, Tae-Kyun Kim, Jiff Matas, and Carsten Rother. BOP: Benchmark for 6D Ob- ject Pose Estimation. In Proceedings of the European Conference on Computer Vision (ECCV), 2018.
Siyuan Huang, Siyuan Qi, Yinxue Xiao, Yixin Zhu, Ying Nian Wu, and Song-Chun Zhu. Cooper- ative holistic scene understanding: Unifying 3d object, layout, and camera pose estimation. In Advances in Neural Information Processing Systems (NeurIPS), pp. 206-217, 2018.
Alex Kendall and Yarin Gal. What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? In Advances in Neural Information Processing Systems (NeurIPS), 2017.
Plamen Koev and Alan Edelman. The efficient evaluation of the hypergeometric function of matrix argument. Mathematics of Computation, 75(254):833-$4) 2006.
Tamio Koyama, Hiromasa Nakayama, Kenta Nishiyama, and Nobuki Takayama. Holonomic gra- dient descent for the FisherBingham distribution on the d-dimensional sphere. Computational Statistics, 29(3-4):661-668 2014.
Jack B. Kuipers. Quaternions and rotation sequences primer with applications orbits, aerospace, and virtual reality. Princeton University Press, 1999.
Alfred Kume and Tomonari Sei. On the exact maximum likelihood inference of FisherBingham distributions using an adjusted holonomic gradient method. Statistics and Computing, 28(4): 835-847, 2018.
Alfred Kume and Andrew T.A. Wood. On the derivatives of the normalising constant of the Bingham distribution. Statistics & Probability Letters, 77(8):832-88) 2007.
Alfred Kume, Simon P. Preston, and Andrew T. A. Wood. Saddlepoint approximations for the nor- malizing constant of Fisher-Binn Bithher-Biggammm distributions on products of spheres and Stiefel manifolds. Biometrika, 100(4):971-988 2013.
10

---

Published as a conference paper at ICLR 2020
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles. In Advances in Neural Information Processing Systems (NeurIPS), 2017.
Chi Jin Bai, and Gregory D Hager. A Unified Framework for Multi-view Multi-class Object Pose Estimation. In Proceedings of the European Conference on Computer Vision (ECCV), 2018a.
Yi Li, Gu Wang, Xiangyang Ji, Yu Xiang, and Dieter Fox. DeepIM: Deep Iterative Matching for 6D Pose Estimation. In Proceedings of the European Conference on Computer Vision (ECCV), 2018b.
Shuai Liao, Efstratios Gavves, and Cees G M Snoek. Spherical Regression: Learning Viewpoints, Surface Normals and 3D Rotations on N-Spheres. In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
Osama Makansi, Eddy 1lg, Ozgun Cicek, and Thomas Brox. Overcoming Limitations of Mixture Density Networks: A Sampling and Fitting Framework for Multimodal Future Prediction. In Proceedings of the Conference on Computer Vision and Puttern Recognition (CVPR), 2019.
Fabian Manhardt, Wadim Kehl, and Adrien Gaidon. ROI-10D: Monocular Lifting of 2D Detection to 6D Pose and Metric Shape. In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
Kanti V. Mardia and Peter E. Jupp. Directional Statisties. Wiley, 1999.
David A. Nix and Andreas S. Weigend. Estimating the Mean and Variance of the Target Probability Distribution. In Proceedings of the International Conference on Neural Networks (ICNN), 1994.
Jean Marc Odobez. Idiap Head Pose Database. headpose 2003.
Sergey Prokudin, Peter Gehler, and Sebastian Nowozin. Deep Directional Statistics: Pose Estima- tion with Uncertainty Quantification. In Proceedings of the European Conference on Computer Vision (ECCV), 2018.
Torsten Sattler, Qunjie Zhou, Marc Pollefeys, and Laura Leal-Taixe. Understanding the Limitations of CNN-Based Absolute Camera Pose Regression. In Proceedings the Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
Hannes Sommer, Igor Gilitschenski, Michael Bloesch, Stephan Weiss, Roland Siegwart, and Juan Nieto. Why and How Avoid the Flipped Quaternion Multiplication. Aerospace, 5(3):22, 2018.
Rangaprasad A. Srivatsan, Gillian Rosen, D. Feroze Naina Mohamed, and Howie Choset. Esti- mating SE(3) Elements Using a Dual Quaternion Based Linear Kalman Filter. In Proceedings of Robotics Science and Systems (RSS), 2016.
Martin Sundermeyer, Zoltan-Csabo Marton, Maximilian Durner, Manuel Brucker, and Rudolph Triebel. Implicit 3D Orientation Learning for 6D Object Detection from RGB Images. In Pro- ceedings of the European Conference on Computer Vision (ECCV), 2018.
Bugra Tekin, Sudipta N Sinha, and Pascal Fua. Real-Time Seamless Single Shot 6D Object Pose Pre- diction. In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
Chen Wang, Danfei Xu, Yuke Zhu, Roberto Martin-Martin, Cewu Lu, Li Fei-Fei, and Silvio Savarese. DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion. In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), 2019a.
He Wang, Srinath Sridhar, Jingwei Huang, Julien Valentin, Shuran Song, and Leonidas J Guibas. Normalized Object Coordinate Space for Category-Leevel 6D Object Pose and Size Estimation. In Proceedings of the Conference Computer Vision and Pattern Recognition (CVPR), 2019b.
Sen Wang, Ronald Clark, Hongkai Wen, and Niki Trigoni. DeepVO: Towards end-to-end visual odometry with deep Recurrent Convolutional Neural Networks. In Proceedings of the Interna- tional Conference on Robotics and Automation (ICRA), 2017.
11

---

Published as a conference paper at ICLR 2020
Sen Wang. Ronald Clark, Hongkai Wen, and Niki Trigoni. End-to-end, sequence-te to -eeu-ence prob- abilistic visual odometry through deep neural networks. The International Journal of Robotics Research, 2018.
Tsun-Yi Yang, Yi-Ting Chen, Yu Lin, and Yung- Yu Chuang. FSA-Net: Learning Fine-Grained Structure Aggregation for Head Pose Estimation From a Single Image. In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
12


Reviews:
[{"review_id": "ryeBokZ4qB", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes a Brigham loss (based on the Brigham distribution) to model the uncertainty of orientations (an important factor for pose estimation and other tasks). This distribution has the necessary characteristics required to represent orientation uncertainty using quaternions (one way to represent object orientation in 3D) such as antipodal symmetry. The authors propose various additions such as using precomputed lookup tables to represent a simplified version of the normalization constant (to make it computationally tractable), and the use of Expected Absolute Angular Deviation (EAAD) to make the uncertainty of the Bingham distribution more interpretable. \n\n+Uncertainty quantification of neural networks is an important problem that I believe should gain more attention so I am happy to see papers such as this one. \n+Various experiments on multiple datasets show the efficacy of the method as well as out performing or showing comparable results to state-of-the-art\n\n-In the caption for Table 1 the author’s write: “the high likelihood and lower difference between EAAD and MAAD indicate that the Bingham loss better captures the underlying noise.” How much difference between EAAD and MAAD is considered significant and why?\n\n-In section 4.5 they write “While Von Mises performs better on the MAAD, we observe that there is a larger difference between the MAAD and EAAD values for the Von Mises distribution than the Bingham distribution. This indicates that the uncertainty estimates of the Von Mises distribution may be overconfident.” Same question as above. What amount of difference between MAAD and EAAD is considered significant and why?\n", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "B1lYHtmY9H", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper focuses on the problem of reasoning about uncertain poses and orientations. To address the limitations of current deep learning-based approaches, the authors propose a probabilistic deep learning model with a novel loss function, Bingham loss, to predict uncertain orientations. The experimental results demonstrate the effectiveness of the proposed approach.\n\nThis paper is well-motivated and the proposed method addresses important problems in uncertain orientation prediction. The paper is well-supported by theoretical analysis, however, the empirical analysis is a little weak and the model does not consider multimodal cases. For the above reasons, I tend to accept this paper but wouldn't mind rejecting it. \n\nQuestions: \n1. You only compared with one baseline. How does the model compare with a loss that is not based on directional statistics or Gaussian models?\n2. How can you improve the model for multimodal cases?", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper considers the problem of reasoning about uncertain poses of objects in images. The reviewers agree that this is an interesting direction, and that the paper has interesting technical merit. 
Decision: Accept (Poster)