Deep Orientation Uncertainty Learning based on a Bingham Loss

1 INTRODUCTION . Reasoning about uncertain poses and orientations , specifically 3-dimensional ( 3d ) positions and 3-axes orientations , is one of the main inference tasks in computer vision ( Sattler et al. , 2019 ) , robotics ( Glover et al. , 2011 ) , aerospace ( Crassidis & Markley , 2003 ) , and other fields . Proper representation and estimation of uncertainty is important , e.g . when dealing with structural ambiguities in object pose estimation or coping with sensor corruption . In vision and robotics tasks , high levels of pose uncertainty may occur due to potentially adversarial conditions that arise in real-world scenarios . A principled approach to uncertainty quantification allows for better execution of planning and situation-awareness tasks such as grasping , tracking , and motion estimation . When representing uncertainties over poses , the position can be modeled using a Gaussian distribution . This approach is well-motivated by the Central Limit Theorem and widely used in probabilistic deep learning models . However , this paradigm can not be as easily applied to modeling periodic quantities , such as the orientation of an object . Therefore , Gaussian models become unsuitable particularly in learning regimes involving high uncertainties where one can not assume local linearity of the underlying space . In this work , we set out to develop a principled probabilistic deep learning approach capable of coping with uncertain orientations . Currently , most deep learning approaches that predict poses or rigid-body motions suffer from at least one of three drawbacks : 1 ) they do not model the uncertainty at all and merely focus on the accuracy of the predicted pose , 2 ) they make simplifying assumptions not taking into account that the orientation is defined on a periodic manifold , making the approach only suitable in low-noise regimes , or 3 ) even when trying to account for periodicity , no dependency is assumed between the orientation axes and usually an Euler angle-based representation is required . To this point , there are no probabilistic deep learning models for uncertainty of orientations that take the geometry of the underlying domain into account . In this work , we close this research gap by proposing a probabilistic deep learning model inspired by Directional Statistics ( Mardia & Jupp , 1999 ) . We present a loss based on the Bingham distribution ( Bingham , 1974 ) , an antipodally symmetric distribution on the sphere . With this loss , we represent uncertain orientations by modeling uncertainty over unit quaternions . Our contributions involve Bingham parameter learning using backpropagation through a Gram-Schmidt method to ensure orthonormalization , efficient approximate evaluation of the normalization constant of the Bingham distribution from a lookup table , and backpropagating through an interpolation scheme during learning . We also discuss interpretability of the Bingham distribution parameters and establish the feasibility of the approach through extensive evaluations . In summary , this work makes the following contributions : 1 ) We propose the Bingham loss , a novel loss function for deep learning-based predictions of orientations and their uncertainty . 2 ) We provide a methodology for making the newly proposed loss and its normalization constant computationally tractable in a deep learning pipeline . 3 ) We demonstrate multi-modal orientation prediction using a Bingham variant of Mixture Density Networks . 4 ) We demonstrate how our approach outperforms the state-of-the-art on challenging pose and orientation estimation tasks1 . 2 BACKGROUND : BINGHAM DISTRIBUTION FOR UNCERTAIN ORIENTATIONS . Unit quaternions are a widely used representation for object orientation in 3d space . They are more compact than rotation matrices , and unlike Euler angles , do not suffer from degeneracies such as Gimbal lock . Additionally , quaternions provide a convenient mathematical notation where the quaternion product , q1 q2 , of two unit quaternions q1 , q2 ∈ H1 results in a concatenation of the rotations represented by each of the quaternions individually . A full introduction to this representation by given in Kuipers ( 1999 ) and notational aspects are discussed by Sommer et al . ( 2018 ) . In this work , a quaternion q1i + q2j + q3k + q4 will be interpreted as a vector q ∈ R4 . It is important to note that the definition of unit quaternions is equivalent to the vector q being of unit length ||q|| = 1 . Furthermore , the quaternions q and −q represent the same orientation . Therefore , representing uncertain orientations using quaternions requires a probability distribution on the 4d hypersphere that exhibits antipodal symmetry , i.e . for the density function f ( · ) of this distribution f ( q ) = f ( −q ) has to hold . A probability distribution exhibiting these properties was proposed by Bingham ( 1974 ) . It arises by conditioning a zero mean Gaussian to unit length . The Bingham distribution is given in terms of its p.d.f . as p ( x ; M , Z ) = N ( MZM > ) −1 exp ( x > MZM > x ) , where x ∈ R4 with ||x|| = 1 , N ( MZM > ) is a normalization constant , M ∈ R4×4 orthogonal , and Z = diag ( z1 , z2 , z3 , 0 ) ∈ R4×4 diagonal , with diagonal entries zi < = 0 and the last entry being zero . We use the notation Bingham ( M , Z ) . The restriction on the range of the diagonal entries in Z has numerical and representational convenience reasons . It can be shown that Bingham ( M , Z ) = Bingham ( M , Z+ cI ) for all c ∈ R with I ∈ R4×4 denoting the identity matrix . Similarly , changing the order of diagonal entries in Z has no effect on the distribution as long as the columns in M are permuted accordingly . In the definition above , the parameters M and Z bear some similarity to the mean and variance of a Gaussian . The density obtains its maxima at ±M : ,4 ( the fourth column of M ) which can be thought of as a mean orientation respecting the manifold structure . The diagonal entries of Z can be interpreted as dispersion parameters , and the first three columns of M can be interpreted as the directions of the dispersion ( the Gaussian analog is the orientation of the covariance ellipsoid ) . Bingham distributions allow for representation of uniform priors over individual axes or even the entire space , making them superior to Gaussians in any of the usual orientation representations . 1Code available at https : //github.com/igilitschenski/deep_bingham One of the main challenges of using the Bingham distribution is the computation of its normalization constant N ( MZM > ) = ∫ ||q||=1 exp ( q > MZM > q ) dq , which is a Hypergeometric function of matrix argument ( Herz , 1955 ) . Evaluating these functions imposes a high computational burden and is still an area of active research ( Koev & Edelman , 2006 ; Kume et al. , 2013 ; Koyama et al. , 2014 ; Kume & Sei , 2018 ) . Using the transformation theorem and the fact that M is orthogonal , the normalization constant can be simplified asN ( MZM > ) = N ( Z ) , making it merely a function of the three parameters zi ( i = 1 , 2 , 3 ) and motivating the use of precomputed lookup tables in practice . Furthermore , to make the uncertainty of a Bingham Distribution more interpretable in practice , we propose the use of Expected Absolute Angular Deviation ( EAAD ) which is defined as EAAD ( Z ) = ∫ ||q||=1 θ ( q , e ) · p ( q ; I , Z ) dq , where p ( · ) is the Bingham ( I , Z ) density , I is the identity matrix , e = [ 0 , 0 , 0 , 1 ] is the vector corresponding to the unit quaternion representing the identity and θ ( q , e ) = 2 · arccos ( |〈q , e〉| ) denotes the angular distance between q and e. The EAAD describes the expected angular deviation from the “ mean ” orientation . It can be loosely thought of as the orientation counterpart to the standard deviation in Euclidean space . For the same reason as in the normalization constant , the EAAD computation does not involve the parameter M . 3 DEEP ORIENTATION UNCERTAINTY LEARNING . The Bingham distribution is the main component of the proposed probabilistic framework for representing deep learned uncertain orientations . Drawing inspiration from Mixture Density Networks ( Bishop , 1994 ) , we propose using the Bingham distribution ’ s negative log-likelihood as a loss function L ( y , M , Z ) = − log p ( y ; M , Z ) = −y > MZM > y + logN ( Z ) , with M , Z as defined above and y being the orientation label given in the training data . We use a neural network to learn M and Z , end-to-end , directly from the input data ( e.g . RGB images ) . From this prediction , the point estimate of y is obtained as ŷ = M : ,4 as the last column corresponds to the highest diagonal entry of Z and thus represents one of the modes of the distribution ( the other being −ŷ due to antipodal symmetry ) . No costly evaluation of the normalization constant is required and no major computational challenges arise in the special case where the dispersion parameter Z is known and not predicted by a neural network . However , as our goal is the modeling of uncertainty , we propose methods for modeling M and Z as well as backpropagating through N ( Z ) . 3.1 MODELING OF DISTRIBUTION PARAMETERS . In order to obtain predictions M̂ and Ẑ , we require a 19 dimensional output ( o ∈ R19 ) of the predictor network ( 3 outputs for Z , 16 outputs for M ) . On its own , these outputs do not satisfy the above-mentioned constraints on the Bingham distribution parameters . Thus , we define the differentiable transforms TM : R16 → R4×4 and TZ : R3 → R4×4 that transform these outputs such that the constraints are satisfied . The transform TZ is obtained as TZ ( o1 , o2 , o3 ) = diag ( ẑ1 , ẑ2 , ẑ3 , 0 ) with ẑi = − exp ( oi ) . For computing M̂ , we first subdivide o4 , . . . , o19 into four vectors vi ∈ R4 ( i = 1 , . . . , 4 ) . Then , we apply the Gram-Schmidt orthonormalization method to these vectors according to m̂i = Normalize ( vi − ∑i−1 k=1〈m̂k , vi〉 · m̂k ) with i ∈ { 1 , 2 , 3 , 4 } and Normalize ( x ) = x/ ||x|| . Finally , the prediction M̂ is obtained as TM ( o3 , . . . , o19 ) = [ m̂1 , . . . , m̂4 ] , and M̂ is orthogonal by construction . 3.2 BACKPROPAGATION THROUGH THE BINGHAM NORMALIZATION CONSTANT . As mentioned earlier , computation of the Bingham normalization constant is numerically burdensome . This is also true for its derivatives which can be shown to be proportional to the normalization constant of Bingham distributions of higher dimension ( Kume & Wood , 2007 ) . A forward-backward pass for one single data point requires 4 evaluations of hypergeometric functions of matrix argument . We avoid this by precomputing a lookup table for N ( Z ) at L different locations ti ( with Zi = diag ( [ t > i , 0 ] ) . This table is then used to build an interpolator fN ( z ) = ∑L i−1 wiφ ( ||z− ti|| ) with z ∈ R3 and φ denoting a radial basis function . The weights wi can also be precomputed during generation of the interpolator . Thus , we can approximateN ( Z ) ≈ fN ( z ) and∇zN ( Z ) ≈ ∇zfN ( z ) . To the best of our knowledge , this is the first time that a lookup table based interpolation mechanism has been included in the computation graph of a neural network .

Summary: The paper proposes a Brigham loss (based on the Brigham distribution) to model the uncertainty of orientations (an important factor for pose estimation and other tasks). This distribution has the necessary characteristics required to represent orientation uncertainty using quaternions (one way to represent object orientation in 3D) such as antipodal symmetry. The authors propose various additions such as using precomputed lookup tables to represent a simplified version of the normalization constant (to make it computationally tractable), and the use of Expected Absolute Angular Deviation (EAAD) to make the uncertainty of the Bingham distribution more interpretable.