Exploiting Redundancy: Separable Group Convolutional Networks on Lie Groups

1 INTRODUCTION . Minsky & Papert ( 1988 ) suggest that the power of the perceptron comes from its ability to learn to discard irrelevant information . In other words ; information that does not bear significance to the current task does not influence representations built by the network . According to Minsky & Papert ( 1988 ) , this leads to a definition of perceptrons in terms of the symmetry groups their learned representations are invariant to . Progress in geometric deep learning has shown the power of pro-actively equipping models with such geometric structure as inductive bias , reducing model complexity and improving generalisation and performance ( Bronstein et al. , 2017 ) . An early example of such geometric inductive bias at work can be seen in the convolutional layer in a CNN ( LeCun et al. , 1998 ) . CNNs have been instrumental in conquering computer vision tasks , and much of their success has been attributed to their use of the convolution operator , which commutes with the action of the translation group . This property , known as equivariance to translation , comes about as a result of the application of the same convolution kernel throughout an input signal , enabling the CNN to learn to detect the same features at any location in the input signal , directly exploiting translational symmetries that naturally occur in many tasks . Although invariance to object-identity preserving transformations has long been recognised as a desirable model characteristic in machine learning literature ( Kondor , 2008 ; Cohen , 2013 ; Sifre & Mallat , 2014 ) , only recently Cohen & Welling ( 2016a ) introduced the Group Equivariant CNN ( G-CNN ) as a natural extension of the CNN ( LeCun et al. , 1998 ) , generalising its equivariance properties to group actions beyond translation . The layers of a G-CNN are explicitly designed to be equivariant to such transformations , hence the model is no longer burdened with learning invariance to transformations that leave object identity intact . It has since been shown that equivariant deep learning approaches may serve as a solution in fields that as of yet remain inaccessible to machine learning due to scarce availability of labelled data , or when compact model design due to limited computational power is required ( Winkels & Cohen , 2018 ; Linmans et al. , 2018 ; Bekkers , 2019 ) . Complexity and redundancy issues impeding regular group convolutions A growing body of work shows applications of G-CNNs consistently and decisively outperforming classical CNNs ( Worrall et al. , 2017 ; Weiler et al. , 2018a ; Bekkers et al. , 2018 ; Esteves et al. , 2018 ; Bekkers , 2019 ; Worrall & Welling , 2019 ; Sosnovik et al. , 2021b ) . However , a practical challenge impeding application to larger groups is the computational complexity of regular group convolutions , which scales exponentially with the dimensionality of the group . Furthermore , Lengyel & van Gemert ( 2021 ) show that group convolution filters in the original formulation of the G-CNN by Cohen & Welling ( 2016a ) exhibit considerable redundancies along the group axis for the p4m and Z2 groups . Similar observations motivated depthwise separable convolutions ( Chollet , 2017 ) , which not only increased parameter efficiency but also model performance ; observed correlations between weights are explicitly enforced with further parameter sharing through the use of kernels separable along spatial and channel dimensions . We address the observations of redundancy along with the scalability issues of regular G-CNNs in their current form . Our paper contains the following contributions : • We introduce separable group convolutions for affine Lie groups Rn oH , sharing the kernels for translation elements x ∈ Rn along subgroup elements h ∈ H . See Fig . 1 for an overview . • We propose the use of a SIREN ( Sitzmann et al. , 2020 ) as kernel parameterisation in the Lie algebra - imposing a fixed number of parameters per convolution kernel , regardless of the resolution at which this kernel is sampled , and ensuring smoothness over the Lie group . • Separable group convolutions allow us to build Sim ( 2 ) -CNNs , which we thoroughly experiment with . We show equivariance to Sim ( 2 ) increases accuracy over a range of vision benchmarks . • To achieve equivariance to continuous affine Lie groups , we propose a random sampling method over subgroups H for approximating the group convolution operation . First , we position this work within the area of equivariant deep learning by giving an overview of related works , and explaining which current issues we are addressing with this work . We derive separable group convolutions , and show how they may be applied to continuous groups . Lastly , we apply these ideas by experimenting with implementations for roto-translations in 2D ( SE ( 2 ) ) , dilation and translation in 2D ( R2 oR+ ) and dilation , rotation and translation in 2D ( Sim ( 2 ) ) . 2 RELATED WORK . Group equivariant convolutional neural networks Broadly speaking , research on G-CNNs can be divided into two approaches . First , Regular G-CNNs use the left-regular representation of the group of interest to learn representations of scalar functions over the group manifold , or a quotient space of the group . The left-regular representation acts on the convolution kernels , yielding an orbit of the kernel under the group action . Convolving the input using these transformed filters , a feature map defined over the group is obtained at each layer . This approach most naturally extends the conventional CNN , where convolution kernels are transformed under elements of the translation group . Regular G-CNNs have been implemented for discrete groups ( Cohen & Welling , 2016a ; Winkels & Cohen , 2018 ; Worrall & Brostow , 2018 ) , compact continuous groups ( Marcos et al. , 2017 ; Bekkers et al. , 2018 ) and arbitrary non-compact continuous Lie groups ( Bekkers , 2019 ; Finzi et al. , 2020 ; Romero et al. , 2020 ) . However , practical implementations for continuous groups often require some form of discretisation of the group , possibly introducing discretisation artefacts , and requiring a choice of resolution over the group . For the second class , steerable G-CNNs , representation theory is used to compute a basis of equivariant functions for a given group , which are subsequently used to parameterise convolution kernels ( Cohen & Welling , 2016b ; Weiler et al. , 2018a ; b ; Sosnovik et al. , 2019 ; 2021a ) . Although steerable G-CNNs decouple the cardinality of the group from the dimensionality of the feature maps , this approach is only compatible with compact groups . The current paper may , in approach , be compared to Bekkers ( 2019 ) and Finzi et al . ( 2020 ) , who define convolution kernels on the Lie algebra of continuous groups to enable convolutions on their manifold . Similarly , we make use of the Lie algebra and exponential map to obtain convolution kernels on the group , but separate the kernels by subgroups . Bekkers ( 2019 ) defines a set of basis vectors in the Lie algebra , which , when combined with the exponential map , allow for the identification of group elements by a vector in Rn . Subsequently , a set of B-splines is defined on the algebra , which form a basis to expand convolution kernels in . A linear combination of these bases creates a locally continuous function on the Lie algebra defining a convolution kernel and its behaviour under transformations of the group . Although this method allows for direct control over kernel smoothness , the learned convolution filters are limited in their expressivity by their basis functions . Finzi et al . ( 2020 ) instead use an MLP to learn convolution kernels on the Lie algebra , which in addition allows them to handle point cloud data . The MLP is constructed to learn kernel values at ( arbitrary ) relative offsets in the Lie algebra . In contrast , we propose to use SIRENs ( Sitzmann et al. , 2020 ) to parameterise convolution kernels , as they have been shown to outperform other forms of MLPs in parameterising convolution kernels ( Romero et al. , 2021 ) , and offer more explicit control over kernel smoothness : a desirable property for addressing discretisation artefacts that occur when modelling features on continuous groups ( see App . A.6 ) . Separable filters in machine learning In image processing , spatially separable filters have long been known to increase parameter- and computational efficiency , and learning such constrained filters may even increase model performance ( Rigamonti et al. , 2013 ) . In Sifre & Mallat ( 2014 ) , authors investigate SE ( 2 ) -invariant feature learning through scattering convolutions , and propose separating the group convolution operation for affine groups into a cascade of two filterings , the first along the spatial dimensions Rn , and the second along subgroup dimension H . From this , authors derive a separable approach to the convolution operation with learnable filters as used in CNNs . This formulation has since been named the depthwise-separable convolution ( Chollet , 2017 ) , a special case of the Network-In-Network principle ( Lin et al. , 2013 ) which forms the basis for the success of the Inception architectures ( Szegedy et al. , 2015 ) . In depthwise separable convolutions , each input channel first gets convolved using a ( set of ) kernel ( s ) with limited spatial support . Afterwards , a 1x1 convolution is used to project the feature set detected in the input channels to the output space . Chollet ( 2017 ) speculates that the Inception architectures are successful due to the explicit separation of spatial and channel mapping functions , whereas in conventional CNNs , kernels are tasked with simultaneously mapping inter-channel and spatial correlations . Haase & Amthor ( 2020 ) argue that the original formulation of the depthwise-separable convolution reinforces inter-kernel correlations , but does not in fact leverage intra-kernel correlations . Subsequently , they propose an inverse ordering of the operations given in depthwise-separable convolutions , sharing the same spatial kernel along the input channels , and show convincing results . Extending this investigation of learned convolution filters to the original G-CNN ( Cohen & Welling , 2016a ) , Lengyel & van Gemert ( 2021 ) remark on the high degree of correlation found among filters along the rotation axis , and propose to share the same spatial kernel for every rotation feature map . We attempt to generalise this approach , proposing separable convolutions on arbitrary affine Lie groups . 3 BACKGROUND . In the following section we give the theoretical background for implementing group convolutions for arbitrary affine Lie groups . We assume familiarity with the basics of group theory and provide the relevant concepts in Appx . A.1 and Appx . A.4 . For simplicity of notation , we initially assume that our input signal/feature map f has a single channel . Lifting convolutions To preserve information on the pose of features in the input , an equivariant convolution operation is achieved by lifting a function from the input space to ( a homogeneous space of ) the group . As we are interested in real signals , specifically image data living on R2 , we assume the Lie group of interest H is taken in semidirect product with the domain of our data ; G=R2 oH . In group convolutions , a given kernel is left-acted by all transformations in G , thereby generating a signal on the higher dimensional space G instead of R2 . Hence , the output feature maps disentangle poses through a domain expansion , e.g . positions plus rotations or scales . For a given group element h ∈ H , kernel k , and location ~x in the input domain R2 , the lifting convolution is given by : ( f ∗lifting k ) ( g ) = ∫ R2 f ( x̃ ) kh ( x̃− x ) dx̃ . ( 1 ) where g= ( x , h ) and kh= 1| deth|Lh [ k ] is the kernel k : R 2 → R transformed via the action of group element h via Lh [ k ] ( x ) : = k ( h−1x ) , and with deth the determinant of the matrix representation of h that acts on Rd . The output of lifting convolutions yields a G-feature map with the original two spatial input dimensions ( R2 ) , and an additional group dimension ( H ) . See Fig . 10 . Group convolutions Now that the data is lifted to the domain of the group , we continue with group convolutions in subsequent layers . Given a kernel k ( now a function on G ) , Haar measures dg̃ and dh̃ on the group G and sub-group H respectively , group convolutions are given by : ( f ∗group k ) ( g ) = ∫ G f ( g̃ ) k ( g−1 · g̃ ) dg̃ = ∫ G f ( g̃ ) Lgk ( g̃ ) dg̃ = ∫ R2 ∫ H f ( x̃ , h̃ ) LxLhk ( x̃ , h̃ ) 1 |h| dx̃ dh̃ = ∫ R2 ∫ H f ( x̃ , h̃ ) k ( h−1 ( x̃− x ) , h−1 · h̃ ) 1 |h| dx̃ dh̃ . ( 2 ) Evaluating this convolution for every group element g ∈ G , we again obtain a function defined on G. As we know G=R2 oH , we can factor this operation into a transformation of a kernel k by a group element h ∈ H , kh=Lh ( k ) , followed by a convolution at every spatial location in f . See Fig . 11 . Achieving invariance Using lifting and group convolution operations , we can construct convolutional layers that co-vary with the action of the group and explicitly preserve pose information throughout the representations of the network . In most cases , we ultimately want a representation that is invariant to transformations of the input in order to achieve invariance to these identity-preserving transformations . This is achieved by aggregating the information at all group elements in a feature map with an operation invariant to the group action , e.g. , max- , mean- or sum-projection . In practice , this is done after the last group convolution , and is followed by one or more fully connected layers .

Summary: This paper discusses considerable parameter redundancy in regular group convolution networks and then proposes separable convolution kernels to share the weights over the subgroups. Besides, the authors explored the equivariance of three different groups and presented a continuous parameterization scheme. Evaluations on several datasets show that their proposed method gains improved performance and computational efficiency.