Learning Irreducible Representations of Noncommutative Lie Groups

1 INTRODUCTION . Many tasks in machine learning exactly or approximately obey a continuous symmetry such as 2D rotations . An ML model is said to be equivariant to such a symmetry if the model respects it automatically ( without training ) . Equivariant models have been applied to tasks ranging from computer vision to molecular chemistry , leading to a generalization of equivariance techniques beyond 2D rotations to other symmetries such as 3D rotations . This is enabled by known mathematical results about each new set of symmetries . Specifically , explicit group representation matrices for each new symmetry group are required . For many important symmetries , formulae are readily available to produce these representations . For other symmetries we are not so lucky , and the representations may be difficult to find explicitly . In the worst cases , the classification of the group representations is an open problem in mathematics . For example , in the important case of the homogeneous Galilean group , which we define in section 2 , the classification of the finite dimensional representations is a so-called “ wild algebraic problem ” for which we have only partial solutions ( De Montigny et al. , 2006 ; Niederle & Nikitin , 2006 ; Levy-Leblond , 1971 ) . To construct an equivariant network without prior knowledge of the group representations , novel approaches are needed . In this work , we propose an algorithm LearnRep that finds the representation matrices with high precision . We validate that LearnRep succeeds for the Poincaré group , a set of symmetries governing phenomena from particle physics to object tracking . We further validate LearnRep on two additional sets of symmetries where formulae are known . We apply the Poincaré group representations obtained by LearnRep to construct SpacetimeNet , a Poincaré-equivariant object-tracking model . As far as we are aware , LearnRep is the first automated solver which can find explicit representation matrices for sets of symmetries which form noncompact , noncommutative Lie groups Further , SpacetimeNet is the first object-tracking model with a rigorous guarantee of Poincaré group equivariance . 1.1 GROUP REPRESENTATIONS AND EQUIVARIANT MACHINE LEARNING . Group theory provides the mathematical framework for describing symmetries and building equivariant ML models . Informally , a symmetry group G is a set of invertible transformations α , β ∈ G which can be composed together using a product operation αβ . We are interested in continuous symmetries for which G is a Lie group . In prior constructions of Lie group-equivariant models , group representations are required . For a group G , an n−dimensional ( real ) group representation ρ : G→ Rn×n is a mapping from each element α ∈ G to an n× n-dimensional matrix ρ ( α ) , such that for any two elements α , β ∈ G , we have ρ ( α ) ρ ( β ) = ρ ( αβ ) . Two parallel techniques have been developed for implementing Lie group equivariant neural networks . The first approach was described in general by Cohen et al . ( 2019 ) . For the latter approach taken by Thomas et al . ( 2018 ) ; Anderson et al . ( 2019 ) ; Bogatskiy et al . ( 2020 ) , convolutions and nonlinearities are performed directly on the irreducible representations of the group , which we define in section 2.4 . A common thread in these works has been to utilize existing formulas derived for the matrix elements of these irreducible representations . However , these formulas are only available for specific Lie groups where the representation theory is well-understood . A more convenient approach for extending equivariance to novel Lie groups would utilize an automated computational technique to obtain the required representations . The primary contribution of this work is such a technique . 1.2 CONTRIBUTIONS . In this work , we automate the generation of explicit group representation matrices of Lie groups using an algorithm called LearnRep . LearnRep poses an optimization problem defined by the Lie algebra associated with a Lie group , whose solutions are the representations of the algebra . A penalty term is used to prevent the formation of trivial representations . Gradient descent of the resulting loss function produces nontrivial representations upon convergence . We apply LearnRep to three noncommutative Lie groups for which the finite-dimensional representations are well-understood , allowing us to verify that the representations produced are irreducible by computing their Clebsch-Gordan coefficients and applying Schur ’ s Lemma . One of the Lie groups where LearnRep performs well is the Lorentz group of special relativity . Prior work has applied Lorentz-equivariant models to particle physics . In this work we explain that the Lorentz group along with the larger Poincaré group also governs everyday object-tracking tasks . We construct a Poincaré-equivariant neural network architecture called SpacetimeNet and demonstrate that it can learn to solve a 3D object-tracking task subject to “ motion equivariance , ” where the inputs are a time series of points in space . In summary , our contributions are : • LearnRep , an algorithm which can find irreducible representations of a noncompact and noncommutative Lie group . • SpacetimeNet , a Poincaré group-equivariant neural network applied to object-tracking tasks . Our work contributes towards a general framework and toolset for building neural networks equivariant to novel Lie groups , and motivates further study of Lorentz equivariance for object tracking . 1.3 ORGANIZATION . We summarize all necessary background and terminology in section 2 . We describe the LearnRep algorithm in section 3.1 and SpacetimeNet in section 3.2 . We summarize related work in section 4 . We present our experimental results in section 5 : our experiments in learning irreducible Lie group representations with LearnRep in section 5.1 and the performance of our Poincaré-equivariant SpacetimeNet model on a 3D object tracking task in section 5.2 . 2 TECHNICAL BACKGROUND . We explain the most crucial concepts here and defer to Appendix A.1 for a derivation of the representation theory of the Lorentz group . 2.1 SYMMETRY GROUPS SO ( n ) AND SO ( m , n ) A 3D rotation may be defined as a matrix A : ∈ R3×3 which satisfies the following properties , in which 〈~u , ~v〉 = ∑3 i=1 uivi : ( i ) detA = 1 ( ii ) ∀~u , ~v ∈ R3 , 〈A~u , A~v〉 = 〈~u , ~v〉 ; these imply the set of 3D rotations forms a group under matrix multiplication and this group is denoted SO ( 3 ) . This definition directly generalizes to the n−dimensional rotation group SO ( n ) . For n ≥ 3 , the group SO ( n ) is noncommutative , meaning there are elements A , B ∈ SO ( n ) such that AB 6= BA . Allowing for rotations and translations of n dimensional space gives the n−dimensional special Euclidean group SE ( n ) . SO ( n ) is generalized by a family of groups denoted SO ( m , n ) , with SO ( n ) = SO ( n , 0 ) . For integers m , n ≥ 0 , we define 〈~u , ~v〉m , n = ∑m i=1 uivi − ∑m+n i=m+1 uivi . The group SO ( m , n ) is the set of matrices A ∈ R ( m+n ) × ( m+n ) satisfying ( i-ii ) below : ( i ) detA = 1 ( ii ) ∀~u , ~v ∈ Rm+n , 〈A~u , A~v〉m , n = 〈~u , ~v〉m , n ; these imply that SO ( m , n ) is also a group under matrix multiplication . While the matrices in SO ( n ) can be seen to form a compact manifold for any n , the elements of SO ( m , n ) form a noncompact manifold whenever n , m ≥ 1 . For this reason SO ( n ) and SO ( m , n ) are called compact and noncompact Lie groups respectively . The representations of compact Lie groups are fairly well understood , see Bump ( 2004 ) ; Cartan ( 1930 ) . 2.2 ACTION OF SO ( m , n ) ON SPACETIME We now explain the physical relevance of the groups SO ( m , n ) by reviewing spacetime . We refer to Feynman et al . ( 2011 ) ( ch . 15 ) for a pedagogical overview . Two observers who are moving at different velocities will in general disagree on the coordinates { ( ti , ~ui ) } ⊂ R4 of some events in spacetime . Newton and Galileo proposed that they could reconcile their coordinates by applying a spatial rotation and translation ( i.e. , an element of SE ( 3 ) ) , a temporal translation ( synchronizing their clocks ) , and finally applying a transformation of the following form : ti 7→ ti ~ui 7→ ~ui + ~vti , ( 1 ) in which ~v is the relative velocity of the observers . The transformation equation 1 is called a Galilean boost . The set of all Galilean boosts along with 3D rotations forms the homogeneous Galilean group denoted HG ( 1 , 3 ) . Einstein argued that equation 1 must be corrected by adding terms dependent on ||~v||2/c , in which c is the speed of light and ||~v||2 is the ` 2 norm of ~v . The resulting coordinate transformation is called a Lorentz boost , and an example of its effect is shown in figure 1 . The set of 3D rotations along with Lorentz boosts is exactly the group SO ( 3 , 1 ) . In the case of 2 spatial dimensions , the group is SO ( 2 , 1 ) . Including spacetime translations along with the Lorentz group SO ( n , 1 ) gives the larger Poincaré group Pn with n spatial dimensions . The Poincaré group P3 is the group of coordinate transformations between different observers in special relativity . Consider an object tracking task with input data consisting of a spacetime point cloud with n dimensions of space and 1 of time , and corresponding outputs consisting of object class along with location and velocity vectors . A perfectly accurate object tracking model must respect the action of Pn on the input . That is , given the spacetime points in any observer ’ s coordinate system , the perfect model must give the correct outputs in that coordinate system . Therefore the model should be Pn-equivariant . For low velocities the symmetries of the homogeneous Galilean groups HG ( n , 1 ) provide a good approximation to SO ( n , 1 ) symmetries , so Galilean-equivariance may be sufficient for some tasks . Unfortunately the representations of HG ( n , 1 ) are not entirely understood De Montigny et al . ( 2006 ) ; Niederle & Nikitin ( 2006 ) ; Levy-Leblond ( 1971 ) . 2.3 LIE GROUPS AND LIE ALGEBRAS . Here we give an intuitive summary of Lie groups and Lie algebras , deferring to Bump ( 2004 ) for a rigorous technical background . A Lie group G gives rise to a Lie algebra A as its tangent space at the identity . This is a vector space V along with a bilinear product called the Lie bracket : [ a , b ] which must behave like1 the commutator for an associative ring R with multiplication operation ×R : [ a , b ] = a×R b− b×R a The Lie algebra for SO ( 3 ) , denoted so ( 3 ) , has a basis { J1 , J2 , J3 } satisfying [ Ji , Jj ] = ijkJk , ( 2 ) in which ijk ∈ { ±1 , 0 } is the totally antisymmetric Levi-Civita symbol.2 Intuitively , the Lie bracket shows how group elements near the identity fail to commute . For example , the matrices Rx , Ry , Rz 1Specifically , the Lie bracket must satisfy the Jacobi identity and [ a , a ] = 0 . 2The symbol ijk simply expresses in equation 2 that [ J1 , J2 ] = J3 , [ J2 , J3 ] = J1 , [ J3 , J1 ] = J2 . for rotations about the x and y axes by a small angle θ satisfy RxRy −RyRx = Rz +O ( θ2 ) ; more generally the Lie bracket of equation 2 is satisfied to first order in θ . The Lia algebra so ( 3 , 1 ) of the Lorentz Group SO ( 3 , 1 ) also satisfies equation 2 for the generators J1 , J2 , J3 of its subalgebra isomorphic to so ( 3 ) . It has 3 additional generators denoted K1 , K2 , K3 , which satisfy : [ Ji , Kj ] = ijkKk [ Ki , Kj ] = − ijkJk ( 3 ) These Ki correspond to the Lorentz boosts in the same way that the Ji correspond to the rotations . In general , if A is a t-dimensional Lie algebra with generators T1 , ... , Tt such that [ Ti , Tj ] = t∑ k=1 AijkTk , ( 4 ) we call the tensor Aijk the structure constants of A . For connected matrix Lie groups such as SO ( m , n ) , the structure constants Aijk are easily obtained . For example , one may apply the matrix logarithm to several elements of the group to obtain elements of the algebra , then find a complete basis for the algebra and write the commutator of all basis elements in this basis .

Summary: The paper proposes the algorithm LearnRep that uses gradient descent methods to learn Lie algebras from structure constants, before obtaining the corresponding group representation through the exponential map. The algorithm is tested on SO(3), SO(2, 1), and SO(3, 1). In addition to this, the paper proposes SpaceTimeNet, a Poincaré-equivariant neural network architecture, and applies this architecture to an object-tracking task involving MNIST digits moving uniformly through space.