E XPLOITING R EDUNDANCY : S EPARABLE G ROUP C ONVOLUTIONAL N ETWORKS ON L IE G ROUPS
Anonymous authors
Paper under double-blind review
A BSTRACT
Group convolutional neural networks (G-CNNs) have been shown to increase parameter efﬁciency and model accuracy by incorporating geometric inductive biases.
In this work, we investigate the properties of representations learned by regular G-CNNs, and show considerable parameter redundancy in group convolution kernels.
This ﬁnding motivates further weight-tying by sharing convolution kernels over subgroups.
To this end, we introduce convolution kernels that are separable over the subgroup and channel dimensions.
In order to obtain equivariance to arbitrary afﬁne Lie groups we provide a continuous parameterisation of separable convolution kernels.
We evaluate our approach across several vision datasets, and show that our weight sharing leads to improved performance and computational efﬁciency.
In many settings, separable G-CNNs outperform their non-separable counterpart, while only using a fraction of their training time.
In addition, thanks to the increase in computational efﬁciency, we are able to implement G-CNNs equivariant to the Sim(2) group; the group of dilations, rotations and translations of the plane.
Sim(2) -equivariance further improves performance on all tasks considered.
I NTRODUCTION
Minsky & Papert (1988) suggest that the power of the perceptron comes from its ability to learn to discard information irrelevant to the problem it is tasked with solving.
In other words; information that is not relevant to the task does not inﬂuence the representation built by the network.
According to Minsky & Papert (1988) this leads to a deﬁnition of perceptrons in terms of the symmetry groups their learned representations are invariant to.
Progress in geometric deep learning has shown the power of pro-actively equipping models with such geometric structure as inductive bias, reducing model complexity and improving generalisation and performance (Bronstein et al., 2017).
An early example of such geometric inductive bias at work can be seen in the convolutional layer in a CNN (LeCun et al., 1998).
CNNs have been instrumental in conquering computer vision tasks, and much of their success has been attributed to their use of the convolution operator, which commutes with the action of the translation group.
This property, known as equivariance to translation, comes about as a result of the application of the same convolution kernel throughout an input signal, enabling the CNN to learn to detect the same features at any location in the input signal, directly exploiting translational symmetries that naturally occur in many tasks.
Although invariance to object-identity preserving transformations has long been recognised as a desirable model characteristic in machine learning literature (Kondor, 2008; Cohen, 2013; Sifre & Mallat, 2014), only recently Cohen & Welling (2016a) introduced the Group Equivariant CNN (G-CNN) as a natural extension of the CNN (LeCun et al., 1998), generalising its equivariance properties to group actions beyond translation.
It has since been shown that equivariant deep learning approaches may serve as a solution in ﬁelds that as of yet remain inaccessible to machine learning due to scarce availability of labelled data (Winkels & Cohen, 2018;
Linmans et al., 2018; Bekkers, 2019), or when compact model design due to limited computational power is required.
The layers of a G-CNN are explicitly designed to be equivariant to these transformations, hence the model is no longer burdened with learning invariance to transformations that leave object identity intact.
Complexity and redundancy issues impeding regular group convolutions A growing body of work shows applications of G-CNNs consistently and decisively outperforming classical CNNs (Worrall et al., 2017; Weiler et al., 2018a;
Bekkers et al., 2018; Esteves et al., 2018; Bekkers,
) is convolved with a ﬁlter k shown in Fig.
b) of size | G | × s × s , where s is the kernel size and | G | the size of the group.
We propose factorising this convolution into a convolution over the group with a kernel k H shown in Fig.
c) with size | G |
1 , followed by a convolution over the with a shown in
) with ; Worrall & Welling, 2019; Sosnovik et al., 2021b).
However, a practical challenge impeding application to larger groups is the computational complexity of regular group convolutions, which scales exponentially with the dimensionality of the group.
Furthermore, Lengyel & van Gemert (2021) show that group convolution ﬁlters in the original formulation of the G-CNN by Cohen &
Welling (2016a) exhibit considerable redundancies along the group axis for the p 4 m and Z 2 groups.
Similar observations motivated depthwise separable convolutions (Chollet, 2017), which not only increased parameter efﬁciency but also model performance; observed correlations between weights are explicitly enforced with further parameter sharing through the use of kernels separable along spatial and channel dimensions.
We address the observations of redundancy along with the scalability issues of regular G-CNNs in their current form.
Our paper contains the following contributions:
• We introduce separable group convolutions for afﬁne Lie groups R n (cid:111) H , sharing the kernels for translation elements x ∈ R n along subgroup elements h ∈ H .
See Fig. 1 for an overview.
• We propose the use of a SIREN (Sitzmann et al., 2020) as kernel parameterisation in the Lie algebra - imposing a ﬁxed number of parameters per convolution kernel, regardless of the resolution at which this kernel is sampled, and ensuring smoothness over the Lie group.
• Separable group convolutions allow us to build Sim(2)
-equivariant G-CNNs, which we thoroughly experiment with.
We show equivariance to this group increases model performance over a range of vision benchmarks.
• To achieve equivariance to continuous afﬁne Lie groups, we propose a random sampling method over H for approximating the group convolution operation.
First, we position this work within the area of equivariant deep learning by giving an overview of related works, and explaining which current issues we are addressing with this work.
We derive separable group convolutions, and show how they may be applied to continuous groups.
Lastly, we apply these ideas by experimenting with implementations for roto-translations in 2D ( SE(2) ), dilation and translation in 2D ( R 2 (cid:111) R + ) and dilation, rotation and translation in 2D ( Sim(2) ).
R ELATED W ORK
Group Equivariant Convolutional Neural Networks Broadly speaking, research on G-CNNs can be divided into two approaches.
The ﬁrst, the regular G-CNN, uses the left-regular representation of the group of interest to learn representations of scalar functions over the group manifold, or a quotient space of the group.
We let the left-regular representation act on the convolution kernels, yielding an orbit of the kernel under the group action.
Convolving the input using these transformed ﬁlters, we obtain a feature map deﬁned over the group at each layer.
This approach most naturally extends the conventional CNN, in which each convolution kernel is transformed under elements of the translation group.
Regular G-CNNs have been implemented for both discrete groups (Cohen & Welling, 2016a;
Winkels & Cohen, 2018; Worrall & Brostow, 2018), compact continuous groups (Marcos et al., 2017;
Bekkers et al., 2018) and arbitrary non-compact continuous Lie groups (Bekkers, 2019; Finzi et al., 2020; Romero et al., 2020).
However, practical implementations for continuous groups often require some form of discretisation of the group, possibly introducing discretisation artefacts, and requiring a choice of resolution over the group.
In the second approach, that of steerable G-CNNs, representation theory is used to compute a basis of equivariant functions for a given group, which are subsequently used to parameterise convolution kernels in (Cohen & Welling, 2016b; Weiler et al., 2018a;b;
Sosnovik et al., 2019; 2021a).
Although steerable G-CNNs decouple the cardinality of the group from the dimensionality of the feature maps, this approach is only compatible with compact groups.
The current paper may, in approach, be compared to recent works by Bekkers (2019) and Finzi et al.
(2020), who deﬁne convolution kernels on the algebra corresponding to a Lie group of interest, to enable convolutions on the manifold of continuous, non-compact groups.
We similarly make use of the Lie algebra and exponential map to obtain convolution kernels on the group, but separate the kernels by subgroup.
Bekkers (2019) deﬁnes a set of basis vectors in the Lie algebra, which, when combined with the exponential map, allow for the identiﬁcation of group elements by a vector in R n .
Subsequently, a set of B-splines is deﬁned on the algebra, which form a basis to expand convolution kernels in.
A linear combination of these bases creates a locally continuous function on the Lie algebra deﬁning a convolution kernel and its behaviour under transformations of the group.
Although this method allows for direct control over kernel smoothness, the learned convolution ﬁlters are limited in their expressivity by their basis functions.
Finzi et al.
(2020) instead use an MLP to learn this same continuous function of convolution kernels on the Lie algebra, while extending the group convolution operator to handle point cloud data, whereas the convolutional layer is generally deﬁned for data lying on a regular grid.
The MLP is constructed to learn kernel values at (arbitrary) relative offsets in the Lie algebra.
We propose using SIRENs (Sitzmann et al., 2020) to parameterise convolution kernels, as they have been shown to outperform other forms of MLPs in parameterising convolution kernels (Romero et al., 2021), and offer more explicit control over kernel smoothness, a desirable property for addressing discretisation artefacts that occur when modelling features on continuous groups (see App. A.4).
Separable ﬁlters in machine learning In image processing, spatially separable ﬁlters have long been known to increase parameter- and computational efﬁciency, and learning such constrained ﬁlters may even increase model performance (Rigamonti et al., 2013).
In Sifre & Mallat (2014), authors investigate SE(2)-invariant feature learning through scattering convolutions, and propose separating the group convolution operation for afﬁne groups into a cascade of two ﬁlterings, the ﬁrst along the spatial dimensions R n , and the second along subgroup dimension H . From this, authors derive a separable approach to the convolution operation with learnable ﬁlters as used in CNNs.
This formulation has since been named the depthwise-separable convolution (Chollet, 2017), a special case of the Network-In-Network principle (Lin et al., 2013) which forms the basis for the success of the Inception architectures (Szegedy et al., 2015).
In depthwise separable convolutions, each input channel ﬁrst gets convolved using a (set of) kernel(s) with limited spatial support.
Afterwards, a 1x1 convolution is used to project the feature set detected in the input channels to the output space.
Chollet (2017) speculates that the Inception architectures are successful due to the explicit separation of spatial and channel mapping functions, whereas in conventional CNNs, kernels are tasked with simultaneously mapping inter-channel and spatial correlations.
Haase & Amthor (2020) argue that the original formulation of the depthwise-separable convolution reinforces inter-kernel correlations, but does not in fact leverage intra-kernel correlations.
Subsequently, they propose an inverse ordering of the operations given in depthwise-separable convolutions, sharing the same spatial kernel along the input channels, and show convincing results.
Extending this investigation of learned convolution ﬁlters to the original G-CNN proposed by Cohen & Welling (2016a),
Lengyel & van Gemert (2021) remark on the high degree of correlation found among ﬁlters along the rotation axis, and propose to share the same spatial kernel for every rotation feature map.
We attempt to generalise this approach, proposing separable convolutions on arbitrary afﬁne Lie groups.
B ACKGROUND
In the following section we give the theoretical background for implementing group convolutions for arbitrary afﬁne Lie groups (Bekkers, 2019).
We assume familiarity with the basics of group theory.
For those not familiar, see App. A.1 for an introduction on relevant concepts, and App D for a group-theoretic treatment of regular CNNs.
For simplicity of notation, we initially assume that our input signal/feature map f is deﬁned over a single channel.
Lifting Convolutions To preserve information on the pose of features in the input, an equivariant convolution operation is achieved by lifting a function from the input space to (a homogeneous space of) the group.
As we are interested in signal data, and speciﬁcally image data living on R 2 , we assume the Lie group of interest H is taken in semidirect product with the domain of our data; G = R 2 (cid:111) H .
In group convolutions, a given kernel is left-translated by all transformations in G , thereby generating a signal on the higher dimensional space G instead of R 2 .
The output feature maps thus disentangle poses (e.g. positions plus rotations or scales) through a domain expansion.
For a given group element h ∈ H , kernel k , and location (cid:126)x in the input domain R 2 , the lifting convolution is given by:
[Equation]
where g =
( x , h ) and k h
= 1 | det h | L h ( k ) is the kernel k : R 2 → R transformed via the action of group element h via [ L h k ]( x ) :
=
k ( h − 1 x ) , and with det h the determinant of the matrix representation of h that acts on R d .
The output of lifting convolutions yields a G -feature map with the original two spatial input dimensions ( R 2 ), and an additional group dimension ( H ). See Fig.
Group convolutions Now that our data is lifted to the domain of the group, we continue with group convolutions in subsequent layers.
Given a kernel k (now a function on G ), Haar measures d˜ g and d˜ h on the group G and sub-group H respectively, group convolutions are given by
[Equation]
Evaluating this convolution for every group element g ∈ G , we again obtain a function deﬁned on G .
As we know G = R 2 (cid:111)
H , we can factor this operation into a transformation of a kernel k by a group element h ∈ H , k h =
L h ( k ) , followed by a convolution at every spatial location in f . See Fig.
Achieving invariance Using the lifting and group convolution operations, we are able to construct convolutional layers that co-vary with the action of the group, explicitly preserving pose information throughout the representations created in the network.
In most cases, we would ultimately like to create representations that are invariant to transformations in the input, as we want our model to be invariant to these identity-preserving transformations.
This may be achieved by aggregating the information at all group elements in a feature map using an operation invariant to the group action, such as max- or mean-projection or simply integrating over the group.
In practice, this is often done after the last group convolution operation, and followed by one or more fully connected layers.
S EPARABLE C ONTINUOUS K ERNEL G ROUP C ONVOLUTIONS
Redundancies in group convolution ﬁlters Similar to Haase & Amthor (2020); Lengyel & van
Gemert (2021) we investigate parameter efﬁciency of learned convolution kernels to motivate separable ﬁlters.
We train an SE(2) − equivariant CNN on the Galaxy10 dataset (see 5 for experimental details) and analyse the resulting group convolution kernels.
We apply PCA by treating the values of the group convolution kernel for each subgroup element h ∈ H as distinct spatial kernels with k × k features.
The ratio of variance explained by the ﬁrst principle component gives an indication of the variability of the group convolution kernel along the subgroup axis.
If the ratio of explained variance is high, the distinct spatial convolution kernels along the subgroup axis are well-characterised by a single shared kernel.
In Fig. 2, results are shown for the group convolution layers in our SE(2) -CNN before and after training.
We ﬁnd that during the training process, redundancy along the subgroup axis increases considerably.
This motivates sharing a single spatial kernel along the subgroup elements, which can be achieved by separating the group convolution operation into a convolution over the subgroup H , followed by a convolution over the spatial domain R 2 .
Separable group convolutions To separate the group convolution operation over G given in Eq. 2 into separate convolutions over R 2 and H , let us assume that convolution kernel k : G → R is separable k ( g )
=
k R 2 ( x ) k H ( h ) .
We can derive the factorised separable group convolution as (see App. A.
[Equation]
Here k H is a convolution kernel over the group H , and k R 2 is a convolution kernel over the spatial domain R 2 .
We set k θ = k θH k θ R 2 , with k θH constant along R 2 and k θ R 2 constant along H .
This can be thought of as parameterising a convolution kernel k over the group G by sharing same spatial kernel k θ R 2 weighed by a value k θH ( h ) at every input group element h ∈ H , see Fig. 21.
Importantly, this factorisation greatly increases the efﬁciency of the group convolution operation.
Since we can precompute the inner integral in Eq. 3, convolving over a single channel of a feature map f of size | H | × x × y goes from O ( | H | 2 × x × y × k 2 ) to O ( | H | × x × y × ( | H | + k 2 )) . See Fig. 22 for a visual intuition of the separable group convolution.
Deﬁning convolution kernels on Lie algebras
In order to perform lifting- and group convolutions given in Eq. 1 and Eq. 2, we are required to evaluate our convolution kernels k at relative offsets g (cid:48) on the group.
As in Bekkers et al.
; Weiler et al.
(2018b)
; Bekkers (2019); Finzi et al.
(2020), we express our group convolution kernel k in analytical form, as a function of relative group elements yielding kernel values.
We use a Sinusoidal Representation Network (SIREN) (Sitzmann et al., 2020) as parameterisation, motivated by ﬁndings in Romero et al.
(2021), which showed great performance improvements when using Sine activations over ReLU , LeakyReLU and Swish when parameterising convolution kernels using MLPs (we replicate this comparison in App.
C.4). Furthermore, the SIREN offers explicit control of kernel smoothness through a frequency multiplier parameter ω 0 , an important property we discuss in App. A.4.
Since the afﬁne Lie group of interest can be non-euclidean, and neural networks are functions generally deﬁned over euclidean spaces, we resort to deﬁning the kernel function on the Lie algebra of our group of interest as in Finzi et al.
The kernel function k θ maps points in the Lie algebra (which may be associated with the relative offsets on the group we are convolving over by the exponential map) to kernel values, k θ : g → R .
For a given element g , we have: k θ ( g )=
SIREN θ (log(g)) .
See Fig. 23. Approximating equivariance for compact continuous and non-compact continuous groups In the case of small discrete subgroups H, such as the p 4 m group of rotations by 90 degrees, we are able to perform lifting- and group convolution operations which are exactly equivariant to the action of the group, as the integral given in Eq. 2 is tractable.
In the current paper we consider image data in particular, which is deﬁned over a discrete grid Z 2 , a subgroup of R 2 . This makes both integrals over the group discrete sums:
[Equation]
with ∆˜ x and ∆˜ h denoting the volume elements corresponding to the grid points.
In the case of continuous groups, it is possible to either make a discretisation of the subgroup H , or to approximate the group convolution integral by means of random sampling.
In order to obtain a volumetrically uniform sampling grid over the group, we sample a set of n equidistant points in the Lie algebra as in (Bekkers, 2019), and map those to the group using the exponential map to obtain a grid H :=
[ h e , ..., h n ] , see Fig.
For noncompact H such as the dilation group R + , we localize the support in the Lie algebra.
In the case of compact continuous groups such as SO(2) we approximate the integral by convolving over a uniformly spaced grid H which is perturbed by left-multiplication with a uniformly randomly sampled group element h (cid:15) ∼ d˜ h , i.e., h (cid:15) (cid:55)→ H (cid:15) :
= h (cid:15) H .
By uniform sampling of H we obtain left-invariant ∆˜ h which only scales the overall convolution result, allowing us to omit it from 4.
We further let ∆˜ x = 1 and obtain the discrete separable group convolution for continuous groups:
[Equation]
During training and inference, randomly sampling over the rotation group makes the network approximately equivariant to the continuous group SO(2) , whereas discretisation makes the network exactly equivariant only to a discretised subgroup of SO(2) .
In our experiments we show that randomly sampling outperforms discretisation when modelling equivariance to SO(2) .
Depthwise separability In CNNs (and G-CNNs), a feature map f l at layer l generally has multiple channels C ; f : R 2 → R C l , where C l is the number of channels at l .
Factorising the group convolution operation into distinct subgroup and spatial dimensions presents us with a choice: we can either (1) deﬁne both k θH and k θ R 2 over the input channels, or choose to (2) reduce the support of our spatial kernel to a single channel, and share a reweighing of it along the input channels in an inverse depth-wise separable manner, similar to Haase & Amthor (2020).
An ablation study detailed in App. C.2 shows that on a ﬁxed parameter budget, additional depthwise separation (2) consistently outperforms (1) deﬁning both kernels over the input channels.
As such, we will keep to the additional depthwise separation and refer to this implementation as the separable group convolution.
Kernel expressivity of separable group convolutions Separable convolution kernels are strictly less expressive than their non-separable counterpart, but to what extent would this limit the expressivity of their learned representations?
In separable group convolutions, we are sharing a weighted version of a single spatial kernel along the input group axis.
In contrast, we could view non-separable group convolutions as having the ability to learn distinct spatial conﬁgurations of features along the group axis.
For a visual example of this distinction for the SE(2) group, see App.
A.3.
Although this reduction in expressivity could theoretically prove limiting in the application of G-CNNs on vision problems, in the next section we experimentally show that separable group convolutions in fact often outperform their non-separable counterpart.
E XPERIMENTS
We empirically motivate the use of seperable group convolutions by studying the performance of three G-CNNs that incorporate equivariance to three distinct groups.
As our goal is to isolate the effect of separating the group convolution, we use the same shallow ResNet architecture throughout all experiments, only varying the sampling resolution over the group, see App. B.1 for details.
After the last convolution block, we apply max-projection over the remaining group dimensions to achieve an invariant representation.
We experiment with three different groups: the roto-translation group in 2 dimensions ( SE(2) ), the translation-dilation group in two dimensions ( R 2 (cid:111) R + ) and the group of rotations, dilations and translations in two dimensions ( Sim(2) ).
For SE(2) , we ﬁrst evaluate the inﬂuence of random sampling versus discretisation of SO(2) .
Next, we assess the difference in performance of separable and non-separable group convolutions on transformed MNIST variants; MNIST-rot, MNIST-scale for SE(2) and R 2 (cid:111) R + respectively.
Due to the reduction in computational complexity separable group convolutions bring, we are able to model equivariance to higher-dimensional groups: we implement two variations of a Sim(2) -CNN and evaluate it on
MNIST-rot-scale.
Lastly, to investigate the advantages equivariance brings in more complex problem settings, we experiment with all setups on three vision benchmark datasets; CIFAR10, CIFAR
Krizhevsky et al.
(2009) and Galaxy10 Leung & Bovy (2019).
See App. B.2 for more details on the datasets and training regimes used in these experiments, and App. C.1 for an additional experiment empirically validating the equivariance properties of our models.
Discretisation versus random sampling over compact subgroups We investigate random sampling to approximate the group convolution.
We trained SE(2) -CNNs on rotated MNIST for different resolutions over SO(2) , using random sampling and discretisation.
Results are shown in Fig.
We can clearly see the advantage of approximating the group convolution integral through random sampling, likely attributable to the fact that through random sampling we obtain an unbiased estimator for the convolution, whereas discretisation equates to a biased sampling of the rotation group.
Separable and non-separable G-CNN performance on MNIST variants Following Cohen &
Welling (2016a)
; Weiler et al.
(2018b); Sosnovik et al.
; Finzi et al.
we conduct experiments on MNIST variants with separable and non-separable implementations of equivariant models for each of the three groups.
We discuss results per dataset in-depth.
Rotated MNIST The SE(2) − CNNs are evaluated on rotated MNIST, the standard benchmark for rotation equivariant models.
To assess the inﬂuence of sampling resolution over the group, we vary the number of sampled group elements from 1 to 20.
The group convolution is approximated through random sampling.
Results are shown in Fig.
We see the inﬂuence of increasing the resolution over the rotation group from 1 to 4 elements, after which the non-separable model saturates, and performance starts to drop when further increasing the number of sampled rotations.
This is in line with ﬁndings by Bekkers et al.
(2018), who note that as the group resolution increases, so does the possibility for overﬁtting on speciﬁc spatial conﬁgurations.
The separable implementation saturates around 7 elements, but performance does not drop signiﬁcantly beyond this point.
This may imply that the reduction in kernel expressivity also has a regularising effect that beneﬁts generalisation.
In this experiment, separable group convolutions decisively outperform the non-separable variant.
SOTA on rotated MNIST To compare separable G-CNNs with related work, we ﬁnetune our models to competitive performance.
With minimal adjustments to model and training regime, listed in App.
C.6, we are able to achieve state of the art performance on MNIST-rot, see Tab. 1.
Scaled MNIST
The R 2 (cid:111) R + -CNNs are evaluated on MNIST scale.
The group convolution integral is approximated through discretisation.
In this experiment, we retain the same resolution over the dilation group in all experiments, but instead vary the value at which the group is truncated, ranging from 1.0 at a grid of 1 element in the Lie algebra to 3.0 at a grid of 20 group elements.
As in Sosnovik et al.
, we found that the inter-scale interactions in the group convolution operation reduced performance, likely because of increase in equivariance error due to truncation of the group.
Therefore, we limited support of the group convolution kernel on R + to two neighbouring elements for all experiments in this paper.
Results are shown in Fig.
Here, non-separable group convolutions outperform the separable variant, suggesting that the ability of non-separable group convolutions to model spatial feature patterns over different scales is beneﬁcial for the scaled MNIST dataset.
Scaled rotated MNIST Lastly, the Sim(2) − CNN is benchmarked on MNIST-rot-scale.
We limit the sampling resolution at each layer of the G-CNN to 2,4,6 and 8 elements for each subgroup.
We test two implementations: (1) separable, where the group convolution is factorised into two convolutions, one over R + (cid:111) SO(2) and one over R 2 , and (2) H-separable, which factorises [Figure] [Figure] [Table]
the convolution into three steps: a convolution over R + , one over SO(2) and one over R 2 .
As can be seen in Fig.
6 the separable implementation turned out to be unstable in the training process for low SO(2) resolutions. We suspect that an approximation of SO(2) of only 2 rotation elements, paired with possible aliasing effects and equivariance error occurring over the dilation group, impedes the model in learning robust representations.
For higher resolution over
SO(2) , this effect seems to decrease.
The results for the H-separable implementation, shown in Fig. 7, highlight a much clearer increase in performance as the resolution over both groups increases, which suggests decoupling these information mappings stabilises the learning process.
Application to computer vision benchmarks To evaluate the value of equivariance to different transformations groups in natural image classifcation, we apply the SE(2) − , R 2 (cid:111) R + − and
Sim(2) − CNNs to three vision datasets.
For SE(2) and R 2 (cid:111) R + we use resolutions of 4 andgroup elements, and truncate the scale group at a value of √ 3 , as we found this to work well for our experiments.
For Sim(2) , we use a resolution of 4 elements over the SO(2) group and 4 elements over the R 2 (cid:111) R + subgroups and again truncate R + at √ 3 .
On CIFAR10, as GPU memory allows, we also experiment with using resolutions of 8 group elements for both subgroups.
As baseline, we implement the same architecture of our non-separable G-CNN, but restrict it to equivariance to translations.
Results are summarised in table 2 and exhibit the following general pattern:
For most groups, separable group convolutions outperform non-separable ones, and in all experiments best performance is achieved by a Sim(2) -CNN.
Next, we discuss results per dataset in-depth.
CIFAR10 First, we evaluate our models on the CIFAR10 dataset (Krizhevsky et al., 2009).
Results in table 2 show that on CIFAR10, the non-separable variant of our SE(2) -CNN is outperformed by the translation equivariant baseline.
With separable group convolutions and a group resolution of 8 elements, SE(2) -CNN show improved performance.
Interestingly, R 2 (cid:111) R + -
CNNs hugely outperform both baseline and SE(2) -CNNs, so it seems scale invariance is a better inductive bias than rotation invariance in CIFAR10.
Combining scale and rotation invariance ultimately leads to the best performance, with separable Sim(2) − CNNs and a resolution of 8 scale and 8 rotation group elements achieving 89 .
38% test accuracy.
The H-separable
Sim(2) -CNN, which models no interactions between scale and rotation, performs slightly worse.
CIFAR100 To assess the impact of separating spatial and group information in more complex problem settings, we experiment with CIFAR100.
Notably, the non-separable R 2 (cid:111) R + -CNNs perform well [Table] under baseline.
We discuss this particular ﬁnding in Sec. 6.
For SE(2) -CNNs, non-separable convolutions outperform separable ones, suggesting the model beneﬁts from modelling distinct spatial conﬁgurations at different poses.
Again, best performance is achieved by the Sim(2) -CNN.
Galaxy10 Lastly, we apply our models in the domain of astro-photography, on the Galaxy10 dataset
Leung & Bovy (2019).
Rotation and scaling symmetries exist naturally in this dataset, making it an interesting application for our invariant models.
Here, results show little difference between performance of non-separable and separable approaches.
Rotation invariance seems to be the stronger inductive bias in this dataset, but again, combining rotation and scale invariance leads to the best performance, with the H-separable Sim(2) -CNN achieving 87 .
45% test accuracy.
The higher separable Sim(2) test error seems to indicate that interactions between scale and rotation features for this dataset impede model performance, again, possibly due to overﬁtting.
Comparison in training time between separable and non-separable group convolutions As one of the main motivators for the separable group convolution is its theoretical increase in computational efﬁciency, a comparison is made in training time between separable and non-separable G-CNNs.
We tracked processing times per epoch, which consists of a train- and inference procedure, for the rotated MNIST experiments, and show them in Fig.
Because of the overhead of two sequential convolution operations offset by the fact that convolution operations on GPUs are heavily optimised, the theoretical decrease in computational complexity, described in Sec. 4, is not seen in practice.
Nonetheless, separable group convolutions present a remarkable decrease in inference time.
D ISCUSSION & F UTURE W ORK
On implicit kernel representations Some of the non-separable conﬁgurations generated results below baseline, which is unexpected given that G-CNNs usually improve upon regular CNNs.
We conjecture that these results are caused by the SIREN parameterisation, which take as input a coordinate vector of different units (mixing both spatial and sub-group h coordinates).
This could limit kernel expressivity, as sharing of a frequency multiplier ω 0 sets kernel smoothness to be identical along spatial and subgroup dimensions.
The fact that for the separable variant we use two distinct SIRENs would lift this restriction.
Despite useful advantages of MLP parametrisations for the kernels, such as their ﬂexibility and ease of implementation for arbitrary Lie groups, it remains an open problem to have full control over their smoothness; ideally one band-limits the kernel MLPs to the discretised resolution on each axis, which would be an important direction for future work.
Conclusion Motivated by observed redundancies in learned group convolution ﬁlters, we introduced separable group convolutions, a computationally efﬁcient implementation of regular group convolutions.
We showed that separable group convolutions not only drastically increase computational efﬁciency, but in many settings also outperform their non-separable counterpart.
Furthermore, we demonstrated the value of separable group convolutions as a solution for modelling equivariance to larger groups; Sim(2) -CNNs decivisely outperform models only equivariant to subgroups of Sim(2) , clearly reinforcing equivariance as means of model generalisation.
A T HEORY AND BACKGROUND A. G ROUP THEORETIC PRELIMINARIES
In this section group theoretical prerequisites used throughout the paper are brieﬂy refreshed.
This is by no means intended as an exhaustive exposition of the fundamentals of group theory we only introduce those concepts relevant to the current work.
Group.
A group is deﬁned by a set G of group elements , along with a binary operator · : G × G → G , called the group product .
The group product deﬁnes a way to combine each pair of elements g 1 , g 2 ∈ G .
For the binary operator · to be considered a group product, it needs to satisfy four constraints:
Closure.
G is closed under · ; for all g 1 , g 2 ∈ G we have g 1 · g 2 ∈ G . 2.
Identity.
There exists an identity element e s.t.
for each g ∈ G , we have e · g = g · e = g . 3.
Inverse.
For every element g ∈ G we have an element g − 1 ∈ G ,
s.t. g
· g − 1 = e . 4.
Associativity.
For every set of elements g 1 , g 2 , g 3 ∈ G , we have ( g 1 · g 2 ) · g 3 = g 1 · ( g 2 · g 3 ) .
Lie groups.
A Lie group is a group of which the elements form a smooth manifold.
Since the group itself is not necessarily a vector space, combination of elements through addition or subtraction is not deﬁned.
However, to each Lie group G , an algebra g may be associated, given by the tangent space of the Lie group at the identity T e ( G ) .
The Lie algebra may be interpreted as a vector space of inﬁnitesimal generators of the group, a set of elements from which we can obtain the group G , by repeated application.
Exponential and logarithmic map.
The exponential map exp : g → G is a function mapping elements from the Lie algebra to the group.
For many transformation Lie groups of interest this map is surjective, and it is possible to deﬁne an inverse mapping; the logarithmic map which maps from the group to the algebra.
Semi-direct product groups.
In practice, we are often only interested in data deﬁned over R d , and hence in this paper only consider afﬁne Lie groups of the form G = R d (cid:111) H , where R d is the translation group in d dimensions and H is a transformation Lie group of interest.
Group action.
A group G may have an action on a given space X .
Given a group element g ∈ G and a set X , the group action T g deﬁnes what happens to any element of x ∈ X when we apply the transformation given by element g to it.
This action is given by:
[Equation]
such that for any two elements g, h ∈ G , we can combine their actions into a single action; T g,h =
T g ◦ T h .
To avoid clutter, we write the action T g ( x ) as g · x .
Note that the action of a group G on domain X also extends to functions deﬁned on this domain.
The induced group action T g of an element g ∈ G maps a square integrable function f to a transformed function f (cid:48) under the action of group G .
For a given element of the transformed domain x (cid:48) ∈ X (cid:48) , the value of the transformed function f (cid:48) is obtained by applying the inverse transformation of g to the domain of the function f :
T g : f → f (cid:48) and for x (cid:48) [Equation]
Representations In practice, the object on which the group operates is often deﬁned as a vector space R.
We operate on 2d images, which can be seen as functions on R 2 .
This makes the object our group operates on a vector space, meaning the group actions (or elements) need to be constructed as linear transformations from this vector space onto itself.
Simply put, every group element g ∈ G needs to be represented in this vector space by a matrix.
We say that linear transformations L G → L 2 ( X ) g [Equation] 2 → Lon some space X are a representation of a group G when they satisfy the group structure, meaning for g 1 , g 2 ∈ G, x ∈ X :
[Equation]
A representation may be seen as the action of a group on a space in matrix form.
Equivariance .
An operator is equivariant with respect to a group, if it commutes with the action of the group.
For an operator Φ : L 2 ( X ) →
L 2 ( Y ) :
[Equation]
A. D ERIVING SEPARABLE GROUP CONVOLUTIONS
If we set the kernel k to be separable, meaning we parameterise it by multiplying a kernel k H which is constant along the spatial domain and a kernel k R 2 which is constant along the group domain:
[Equation]
We can derive separable group convolutions as:
[Equation]
A. E XPRESSIVITY LIMITATIONS OF SEPARABLE GROUP CONVOLUTIONS
As discussed, separable group convolution kernels are strictly less expressive than their non-separable counterpart.
For example, in the case of the roto-translation group SE(2) , this would enable nonseparable group convolution kernels to learn to represent features that are built up of different spatial conﬁgurations at different orientations. We draw up a simpliﬁed example illustrated in Fig. 9.
Assume we have an elementary feature type e , and a spatial kernel k with which we can recognise this feature type in its canonical pose θ 0 . In our input f in , we have three instances of the feature e , one under the canonical pose θ 0 , and two under a 90 ◦ rotation.
Applying the lifting convolution using kernel k for the group G = R 2 (cid:111) C 4 of translations and 90 ◦ rotations yields a feature map f out deﬁned over G , with spatial feature maps for θ 0 , ..., θ 270 . The spatial feature map f θ 0 out contains a response at a single spatial position.
In contrast, the feature map f θ 90 out contains a response at two spatial positions.
The spatial conﬁgurations for the feature maps along H are different.
A single conventional group convolution kernel could learn to recognise these distinct spatial conﬁgurations along the subgroup axis, whereas a separable group convolution kernel could not, since it simply repeats (a weighted version of) the same spatial kernel k R 2 along the group axis.
Although this reduction in expressivity could theoretically prove limiting in the application of GCNNs on vision tasks, our experiments show that in practice this rarely seems a problem, and may even help prevent overﬁtting.
A. K ERNEL SMOOTHNESS FOR GROUP CONVOLUTIONS EQUIVARIANT TO CONTINUOUS GROUPS
As mentioned, using SIRENs we are able to explicitly control kernel smoothness. We brieﬂy elaborate on the importance of kernel smoothness in G-CNNs.
In conventional CNNs, weights at distinct spatial locations are generally initialised independently. Because the kernels are only transformed using discrete translation operations x ∈ Z 2 , translation equivariance is ensured by virtue of using the exact same weights values throughout all spatial locations.
In G-CNNs for continuous groups, the kernel is transformed under actions of a continuous transformation group of interest H to obtain equivariance.
However, in our convolution operation we are still using a discretised kernel; we are required to sample kernel values at different grid points for different elements h ∈ H . We are no longer able to simply reuse the same weight values throughout the group as with regular CNNs.
To this end, we deﬁne our kernels in an analytical form, which we can trivially evaluate at arbitrary grid points. Because our grid has a ﬁxed resolution, the kernels sampled from this analytical form are susceptible to aliasing effects; the analytical kernel function may exhibit higher frequencies than can be captured in the discretisation of the kernel. We visualise the effects of aliasing in Fig.
In short, for continuous groups, the group action transforms a signal smoothly as the group is traversed. To prevent discretisation artefacts, we want our kernel to exhibit the same smoothly transforming behaviour, hence we use SIRENs, as they offer explicit control over the smoothness of kernels in their analytical form.
B A DDITIONAL EXPERIMENTAL DETAILS B. A RCHITECTURAL DETAILS
Model architecture As architecture for our experiments we use a simple ResNet model (He et al., 2016).
We use a single lifting convolution with 32 output channels, followed by two residual blocks of 2 group convolutions. The ﬁrst block has 32 output channels, the second has 64 output channels.
After the ﬁrst residual block we apply max-pooling with kernel size 2 over the spatial dimensions of the feature map.
After the last residual block, we apply max pooling over remaining spatial and subgroup dimensions, followed by two linear layers with batchnorm and ReLU in between.
An overview is given in Fig.
SIREN architecture All our kernels are parameterised by a SIREN (Sitzmann et al., 2020).
In a SIREN, output y l for a layer l and input x l − 1 is deﬁned by:
[Equation]
In this equation, ω 0 acts as a multiplier for low dimensional frequencies found in the input domain (the grid of relative offsets on the group), which explicitly introduces higher frequencies, allowing the neural net to learn high-frequency functions (such as kernels).
We found a value for ω 0 of 10 to work well in all our experiments.
For the SIREN, we used an architecture of two hidden layers of 64 units. [Table]
In non-separable G-CNNs we have a single SIREN with a ﬁnal layer mapping to a vector R c in × c out .
In separable G-CNNs, we use two SIRENs, the ﬁrst mapping to a function the Lie algebra of the subgroup H ; R c in × c out , and the second mapping to R c out . Lastly, for H-separable Sim(2) -CNNs we use three SIRENs, one mapping to SO(2) .
In Tab. 3, we report the number of trainable parameters for each model conﬁguration.
Throughout our experiments, we kept the number of channels in all our conﬁgurations constant, to fairly compare the expressivity of the learned representations in non-separable and separable group convolutions.
This has as effect that the number of parameters in the separable implementations is larger than for non-separable implementations, due to our use of separate SIRENs to parameterise kernels over the different subgroups.
To ensure that the difference in number of trainable parameters does not inﬂuence this comparison, we explicitly chose to over-parameterise our SIREN architecture, as shown in an additional ablation in App.
C.3. B. E XPERIMENTAL DETAILS
Here, we list training regimes for all experiments.
We keep these as consistent as possible, as we are only interested in the effect of separating the group convolution operation, and in isolating the effect of incorporating equivariance into our models. Optimizer All architectures are trained with Adam
optimisation (Kingma & Ba and 1 e
−
weight decay. All models are trained on a single Titan V.
Datasets and dataset-speciﬁc conﬁgurations We list information on all datasets, and any datasetspeciﬁc model conﬁguration details we use in our experiments here. Rotated MNIST The 62.000 MNIST images (LeCun et al., 1998) are split into a training, validation and test set of 10.000, 2.000 and 50.000 images respectively, and randomly rotated to orientations between [0 , 2 π ) .
Note that in (Weiler et al.,
; Weiler & Cesa, 2019), the rotated MNIST dataset is augmented during training by transforming images with random continuous rotations.
We only use train-time augmentation for the state-of-the-art results obtained in Tab. 1, as detailed in App.
C.6.
All models trained on rotated MNIST, except for the state-of-the-art runs detailed in C.6, are trained for 200 epochs with a batch size of 128 and a learning rate of 1 · 10 − 4 .
Scaled MNIST
The 62.000 MNIST images are again split into a training, validation and test set of 10.000, 2.000 and 50.000 images respectively, but now randomly scaled by a factor [0.3, 1] and padded with zeros to retain the original resolution.
No data-augmentations of any kind are used in the experiments on scaled MNIST.
All models trained on scaled MNIST are trained for 200 epochs with a batch size of 128 and a learning rate of 1 · 10 − 4 .
Scaled rotated MNIST The 62.000 MNIST images are again split into a training, validation and test set of 10.000, 2.000 and 50.000 images respectively, but now randomly scaled by a factor [0.3, 1], padded with zeros to retain the original resolution and randomly rotated by orientations between [0 , 2 π ) .
For the experiments on scaled rotated MNIST we do not use data-augmentation of any kind.
All models trained on rotated scaled MNIST are trained for 200 epochs with a batch size of 128 and a learning rate of 1 · 10 − 4 .
CIFAR10 We evaluate our models on the CIFAR10 dataset, containing 62.000 32 × 32 color images in 10 balanced classes (Krizhevsky et al., 2009).
All models trained on CIFAR10 are trained forepochs with a batch size of 128 and a learning rate of 1 · 10 − 4 .
CIFAR100 We evaluate our models on the CIFAR100 dataset, containing 62.000 32 × 32 color images in 100 balanced classes (Krizhevsky et al., 2009).
All models trained on CIFAR10 are trained forepochs with a batch size of 128 and a learning rate of 1 · 10 − 4 .
{+Galaxy
+}
This dataset contains 21785 69x69 color images of galaxies divided into 10 unbalanced classes.
For Galaxy10, we limit batch size to 32 images, and scale learning rate accordingly to 2 .
5 · 10 − 5 , as suggested by Goyal et al.
B. G ROUP CONVOLUTION BLOCKS AND RANDOM SAMPLING ON THE GROUP
In our residual blocks (He et al., 2016), we subsequently convolve the input to the block x in by two group convolutions, gconv 1 and gconv 2 , yielding x out and apply elementwise addition with the input x in , followed by ReLU activation.
When approximating the group convolution through random sampling, we must take care to deﬁne the input and output of the two group convolution layers on the same grid over the group as the skip-connection to ensure a well-deﬁned equivariant group convolution block.
This may be done by adding a group shortcut layer which maps from the set of input elements on the group to the set of output elements.
We implement this as a group convolution with a 1 × 1 spatial extent.
The group shortcut layer thus simultaneously serves as a channelwise projection from the input space of gconv 1 to the output space of gconv 2 and maps from the input grid on H of the ﬁrst group convolution to the output grid on H of the second group convolution. C A DDITIONAL EXPERIMENTS C. V ALIDATING MODEL INVARIANCE TO TRANSFORMATION GROUPS Following Weiler et al.
(2018b), we empirically assess the level of model-invariance to rotation transformations, by training on upright MNIST, and subsequently assessing performance on a test set of MNIST images that have been transformed by a subgroup element h ∈ H .
Each model is trained on the MNIST training set containing 60,000 images, and tested on 10, transformed images.
Performance is tested for rotations over a range between [0 , 2 π ) , in 100 steps.
The group convolution is approximated through random sampling.
Results are shown in ﬁgures 12 and 13 for 1, 2 and 8 group elements for the separable and non-separable variants.
We can see that equivariance error is very similar for separable and non-separable variants, and greatly reduces with increased resolution over the group.
We do note that for 8 elements, the non-separable group convolution achieves slightly better performance.
C. S UPPORT OF THE SEPARABLE CONVOLUTION KERNELS OVER THE CHANNEL DIMENSIONS
As explained in Sec. 4, we can choose to either deﬁne both k H and k R 2 over the input channel dimension, or incorporate depthwise separability as well; additionally sharing the same spatial kernel k R 2 over the input channels as well as the group input elements.
In an additional ablation study we ﬁnd that at any resolution above | H | = 1 , additional depthwise separability outperforms deﬁning both kernels over the input channel dimensions on a ﬁxed parameter budget.
For this experiment on rotated MNIST, we use the same depthwise group separable SE(2) − CNN as in Sec. 5, with the architecture described in App.
B.1.
For the non-depthwise group separable
SE(2) − CNN, we use a single residual group convolution block of 48 channels instead of two of 32 and 64 channels respectively to arrive at approximately the same number of parameters (815k for the implementation without depthwise separability, 803k for the implementation with depthwise separability).
Results, shown in Fig.
14 highlight that additional depthwise separability outperforms non-depthwise group separable convolutions, which is why we stick with additional depthwise separation of the spatial kernel, and name this approach the separable group convolution.
C. H OW DOES SIREN ARCHITECTURE INFLUENCE PERFORMANCE ?
As explained, we keep the architecture of our SIRENs constant over all experiments.
Because we use multiple SIRENs in separable architectures, and we want to isolate the effect of separability of the group convolution operation, we deliberately choose to over-parameterise the SIREN architectures, so as not to advantage the separable implementations by their increased parameter numbers.
To this end, we look for the largest size of hidden layer that does not negatively impact performance in the non-separable implementation.
We assess the impact of the size of the hidden layers in our SIREN on performance on rotated
MNIST for separable and non-separable SE(2) -CNN implementations.
We sample at a resolution over SO(2) of 8 elements.
Interestingly, results in ﬁgure 15 show that even with very small SIRENsa hidden size of 4 or 8 - performance of the non-separable implementations is remarkable.
Separable implementations clearly underperform for small SIREN sizes, leading us to hypothesize that the restriction in kernel expressivity due separability paired with under-parameterisation is detrimental to performance in these conﬁgurations.
For non-separable implementations, performance barely increases from 16 hidden units on, and starts to decrease after a size of 64 hidden units.
For this reason, we choose a size of 64 hidden units in all
SIRENs in our experiments.
[Table]
C. C OMPARING ACTIVATION FUNCTIONS FOR CONTINUOUS CONVOLUTION KERNELS
To assess the impact of our use of SIRENs on the performance of our separable G-CNNs, we perform an additional ablation on rotated MNIST in which we compare with other activation functions.
We train separable SE(2) -CNNs with a resolution 8 group elements over SO(2) , and only vary activation functions in the MLP parameterising the convolution kernels.
The convolution on SO(2) is approximated through random sampling.
We look at other activation functions used in parameterising convolution kernels; ReLU (Wu et al., 2019), LeakyReLU
[Table]
and SiLU (Finzi et al., 2020), see Tab. 4.
These results conclusively show the power of sine as activation function, one of the reasons we chose to use SIRENs in our work.
C. R ANDOM SAMPLING OVER NON - COMPACT GROUPS
As explained in section 4, for non-compact groups we approximate the group convolution through a discretisation of the transformation group of interest.
Motivation for this decision is the fact that in non-compact groups, we have to deal with boundary effects of truncating the group, introducing equivariance error into the convolution operation.
For dilation, further motivation for approximation through discretisation is the loss of information that occurs in for example natural images, as a result of downscaling a signal on a ﬁxed resolution grid.
Random sampling over the dilation group would have as effect that the representation built by a group convolution layer contains different spatial resolutions of information at every sampling step.
This results in subsequent layers receiving noisy information at each iteration, which would impede model performance, with training being highly unstable for low resolutions over R + .
To empirically reinforce this hypothesis, we performed the same experiment with our R 2 (cid:111) R + -CNN on MNIST-scale described in section 5 with random sampling over the dilation group.
Results, shown in ﬁgure 16, highlight that indeed, random sampling over the dilation group leads to signiﬁcantly worse performance.
We also noticed great instability during the training process, seen in the variance of the results achieved with random sampling.
C. A CHIEVING SOTA ON ROTATED MNIST
To compare performance of our separable regular G-CNNs with previous work, we apply our implementation of SE(2) and Sim(2) -CNNs to achieve state of the art results on the rotated MNIST dataset, discussed in Sec. 5.
We make some slight adjustment to our architecture and training regime: we reduce the SIREN to a single hidden layer of 64 units.
The convolution over SO(2) is approximated using random sampling, while the convolution over R + is approximated using a discretisation with a truncation of the scale group at s = √ 3 .
We use a batch size of 64, and a learning rate of 1 · 10 − 4 and train for 300 epochs.
To fairly assess the impact of equivariance, we train a number of models without continuous rotation augmentations.
of redundancy non separable convolution kernels variability the redundancy decreases
Separable convolutions
To compare with previous SOTA, achieved by Weiler & Cesa (2019), we also train models with continuous rotation augmentations.
Weiler & Cesa (2019) does not specify performance results for models without train-time augmentations, therefore we also compare with the prior SOTA by Weiler et al.
(2018b), which trains models both with and without augmentations.
We show results in Tab. 1.
With SE(2) -CNNs we achieve best performance with a resolution of 20 elements over SO(2) .
For the Sim(2) -CNNs, we achieve best performance with a resolution of 12 elements over SO(2) and 5 elements over R + .
These results show that without train-time augmentation Sim(2) -CNNs outperform the previous SOTA by Weiler & Cesa (2019), which was trained with continuous rotation augmentations.
This improvement further increases when we add train-time augmentation by continuous rotations, to a test error of 0.59%.
C. D O SEPARABLE GROUP CONVOLUTIONS REDUCE REDUNDANCY IN G-CNN S ?
As explained, convolving with separable group convolution kernels is analogous to a group convolution with a kernel which shares a reweighting of a single spatial kernel along the group axis.
This approach was motivated by redundancy observations in regular group convolution kernels along the group axis, seen in Fig.
To assess whether the separable group convolution solves these redundancy issues in G-CNNs, we may want to perform a similar test of the variability of our introduced separable implementation along the group axis.
Note that, if we were to reconstruct the full group convolution kernel k from k H and k 2 R and apply the same PCA variance test as used to obtain Fig.
2, we would ﬁnd that each group convolution could be fully explained by the ﬁrst principal component (corresponding to the reshared spatial kernel).
Instead, we measure the variability of our separable kernels by assessing the variance of k H , which lies along this axis.
Results are shown in ﬁgure 17 for the separable
SE(2) -CNN trained on Galaxy10 dataset with a resolution of 8 elements on SO(2) .
Here, we observe the variance increasing in all layers over the training process.
This indicates that indeed, separable group convolutions reduce parameter redundancy in G-CNNs.
D V ISUALISATIONS OF OPERATIONS IN ( SEPARABLE ) G-CNN S
To ease the reading experience of this paper, in this section we provide some additional group theoretic perspective on regular CNNs, and additional visualisations and for a number of operations used in (separable) group convolutions.
CNNs from a group theoretic perspective We give a brief treatment of ordinary CNNs The ordinary convolution operation used in neural networks requires a deﬁnition of a convolution kernel k on R 2 , as we are modulating a signal f which itself lives on R 2 .
The kernel k is applied to f on every location in the input space R 2 to again yield a function over R 2 .
Intuitively, this is the same as saying (1) we transform the convolution kernel k under the action of every group element x ∈ R 2 , to obtain a set of kernels K =
{L x ( k ) | x ∈ R 2 } , and (2) apply the convolution operation on f using this set of transformed kernels K .
By tying the kernel weights used throughout the translation group, learned features are automatically generalised over spatial positions.
This intuition is visualised in Fig.
Lifting convolution In Fig.
19 we show a visualisation of the lifting convolution for the SE(2) group.
Group convolution In Fig.
20 we show a visualisation of the group convolution for the SE(2) group.
Separable group convolution kernel In Fig.
21 we show a visualisation of a separable group convolution kernel.
Separable group convolution In Fig.
22 we show a visualisation of the separable group convolution operation.
Deﬁning convolution kernels on Lie groups In Fig.
24 we show how we obtain a grid on the Lie group SO(3) / SO(2) by mapping from its algebra.
In Fig.
23 we show how we subsequently obtain a kernel on this group by deﬁning a SIREN on its algebra