The Selective $G$-Bispectrum and its Inversion: Applications to $G$-Invariant Networks
Authors: Simon Mataigne
Johan Mathe
Sophia Sanborn
Christopher Hillar
Nina Miolane
Venue: NeurIPS 2024 poster (2024)
Keywords: CNN
group invariance
bispectrum
Neural Network
AI
Primary area: deep_learning_architectures
TL;DR: We propose a new layer in the architecture of Group-Equivariant Neural Networks to achieve invariance to group action on the input.

Abstract: An important problem in signal processing and deep learning is to achieve *invariance* to nuisance factors not relevant for the task. Since many of these factors are describable as the action of a group $G$ (e.g. rotations, translations, scalings), we want methods to be $G$-invariant. The $G$-Bispectrum extracts every characteristic of a given signal up to group action: for example, the shape of an object in an image, but not its orientation. Consequently, the $G$-Bispectrum has been incorporated into deep neural network architectures as a computational primitive for $G$-invariance\textemdash akin to a pooling mechanism, but with greater selectivity and robustness. However, the computational cost of the $G$-Bispectrum ($\mathcal{O}(|G|^2)$, with $|G|$ the size of the group) has limited its widespread adoption. Here, we show that the $G$-Bispectrum computation contains redundancies that can be reduced into a *selective $G$-Bispectrum* with $\mathcal{O}(|G|)$ complexity. We prove desirable mathematical properties of the selective $G$-Bispectrum and demonstrate how its integration in neural networks enhances accuracy and robustness compared to traditional approaches, while enjoying considerable speeds-up compared to the full $G$-Bispectrum.

The Selective G-Bispectrum and its Inversion: Applications to G-Invariant Networks
Simon Mataigne"
Johan Mathe Atmo
ICTEAM, UCLouvain Nouvain-la-Neure Belgium s simon. matai mnttaiigeeuuclouuviin
San Francisco, CA johan@atmo.co ai
Sophia Sanborn
Christopher Hillar Algebraic San Francisco, CA hill arrmat@@gmaaal.commmmmm com
Nina Miolane UC Santa Barbara Santa Barbara, CA ninami walaunaaabb......... edu
Science San Francisco, CA sophhassaboorn@maal.cc com
Abstract
An important problem in signal processing and deep learning is to achieve invari- ance nuisance factors relevant for the task. Since many of these factors are describable as the action of a group G (e.g., rotations, translations, scalings), we want methods to G-invariant. The G-Bispectrum extracts every character- istic of a given signal up to group action: for example, the shape of an object an image, but not its orientation. Consequently, the G-Bispectrum has been incorporated into deep neural network architectures as a computational primitive pooling mechanism, but with greater selectivity and
for G-invariance aain robustness. However, the computational cost of the G-Bispectrum (O(G))), with G| the size of the group) has limited widespread adoption. Here, we show that the G-Bispectrum computation contains redundancies that can be reduced into a selective G- G-Bispectrum with ([G)) complexity. We prove desirable mathematical properties of the selective G-Bispectrum and demonstrate how its integration in neu- networks enhances accuracy and robustness compared traditional approaches, while enjoying considerable speeds-up compared to the full G-Bispectrum.
Introduction
The visual world is rich with symmetries. For example, the identity of an object is invariant to its position in the visual field; vision has translational symmetry. Group theory is the mathematics used to describe transformations, their actions on objects, and the object's 's symmetry. As such, group theory has penetrated the fields of signal processing and deep learning alike. For example, the Fourier transform, pillar of signal processing, has been adapted to the G-Fourier transform, with its spectrum decomposing signal defined over group into several frequencies. More recently, researchers have become interested in the properties of higher-order spectra such as the Bispectrum, and its generalization to signals over groups via the G-Bispectrum.
G-Bispectrum
The G-Bispectrum is the Fourier transform of the G-Triple Correlation (G-TC). Historically, higher-order spectra found initial applications in the context classical signal processing generalizations of the two-point autocorrelation [31, 25). The work of Kakarala [15] illuminated
Simon Mataigne a Research Fellow the Fonds de la Recherche Scientifique FNRS.
Nina Miolane acknowledges funding from the NSF grant 2313150.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

the relevance of the G-Bispectrum for invariant theory, as it is the lowest-degree spectral invariant that is complete. Since then, has appeared in diverse settings such as vision science [34], machine learning [19, 20], and 3D modeling [18].
Limitations of the G-Bispectrum for Deep Learning The computational complexity of the G-Bispectrum has severely limited the reach of its applications. The most salient example of this limitation is in machine learning and deep learning. Convolutional Neural Network (CNN) [22. 24] reflect and exploit the translational symmetry of the visual world. CNNs (G-CNNs) [6. 21] do just this, with more general group-equivariant convolutions to exploit symmetries like rotational symmetries. In both cases, one typically wants to preserve transformations throughout the layers of a network (i.e., to be group-equivariant) and remove them only at the end when "canonicalizing" an image for classification (i.e., to be group-invariant) While the theory of equívariant layers has been thoroughly developed [7, 33]. less attention has been paid to the theory of invariant layers [12]. This is where the G-Bispectrum enters the picture, and where its computational cost has strongly limited its integration into deep learning.
Commonly, invariance in G-CNNs is achieved by simply taking an average or maximum over the transformation group (Average or Max G-Pooling, respectively). However, as noted by Sanborn & Miolane [28], this is a highly lossy operation removing information about the structure of the signal. While the max operation is indeed invariant (the max of an image is the same as the max of an image rotated by 90 degrees). it is excessively invariant: one could permute all of the pixels in the image and without changing the maximum, but with none of the same structure (see Figure 8). To address this, Sanborn & Miolane [28] used the G-TC as G-invariant layer that is complete that is, it removes group transformations with no loss of signal structure. This approach achieves demonstrable gains in accuracy and robustness [28], but it is computationally expensive.
Indeed, the space complexity of the G-TC, i., its number of coefficients, scales as O(G where |G] is the size of the group. As each coefficient demands for O(GG)) operations, the computational cost or the time complexity of the G-TC is ((G))). An alternative would be to use the G-Bispectrum as the pooling layer. However, both its space and time complexities are ([GG By comparison, the Max G--pollig layer features a O(G)) computational cost and returns a scalar output. This raises the question of whether one can achieve complete invariance and adversarial robustness without sacrificing too much in terms of computational efficiency.
Contributions. In this work, we prove for the first time that we can significantly reduce the computational complexity of the G-Bispectrum. This result has important implications for signal processing and deep learning on groups, for which the G-Bispectrum a foundational computational primitive. Our contributions are:
We provide a general algorithm that reduces the computational complexity of the G-Bispectrum from O(GG)2 to O(GG)) in space complexity and from O(G))² to O[G log |G') in time com- plexity if an FFT is available on G. We term it the selective G-Bispectrum. The algorithm can be applied to any finite group.
We prove that the selective G-Bispectrum is complete for the most important finite groups used in practice, i.e., all discrete commutative groups, the dihedral groups of any order, the octahedral and full octahedral group. This significantly extends the work of [10, 15, 27], who first showed this for some finite, commutative groups. where it was demonstrated that the G-Bispectrum can be computed with only |G| space complexity.
We use the selective G-Bispectrum to propose a new G-invariant layer that strikes a balance between robustness and efficiency. In particular, it is more expensive than the Max G-pooling, but cheaper than the G-TC pooling.It is also cheaper than the full G-bispectral pooling of O(GG?² time and ([GGF) space complexity. The selective G-Bispectrum is more robust than the max G-pooling, and almost as robust as the G-TC.
We run extensive experiments on the MNIST [23] and EMNIST [5] datasets to evaluate how each invariant layer (Max G-pooling, G-TC, selective G- G-Bispectrum) impacts accuracy and speed on classification tasks. We achieve the expected results: Our layer is faster than the G-TC and full G- G-Bispectrum and more accurate than Max G-pooling.
We present several findings important to the design of invariant layers to guide further advances in the field of geometric deep learning. In particular, we show that the accuracy and speed advantages

---

of the selective G-Bispectrum is most striking for G-CNNs with low number of convolutional filters. Conversely, increasing the number of filters in the G-Convolutions allows the Max G-Pooling to catch up on the accuracy. This demonstrates that the G-bispectral pooling will be particularly interesting for neural networks operating under smaller parameter budget.
We hope that the proposed reduction of the G-Bispectrum complexity will further open areas of research in signal processing on groups, that were previously prohibited due to the high complexity of the operation.
2 Background: G-Triple Correlation and G-Bispectrum
The proposed selective G-Bispectrum operation is closely related to two other foundational operations on signals defined on groups: the G-Triple Correlation and the full G- G-Bispectrum, which we introduce here. The background on group theory, including the definitions of groups, group actions, equivariance and invariance, is presented in Appendix A.
The G-Triple Correlation Given a real signal defined on finite group θ G + R. the G-Triple Correlation (G-TC) [15] is the lowest order polynomial that is complete, i.c., that conserves all of the information of the signal O up group action by G. Definition 2.1. The G-Triple Correlation real signal @ + R is given by
T(O) .92 = Σ ((g) 9))0(g 92) for all g1, E G.
(1)
SEG
The original triple-correlation was introduced for the classical framework of translations of one- dimensional signal, i.e., where X = E and (G, .) (Z, +). The G-triple correlation from Definition 2.1 extends the original definition any finite group (G, .). In our setting, the signal G + IR will be obtained after the G-convolution of a function f X + IRS representing continuous image with channels, with filter bhannel-y--channel. Importantly, the G-TC layer has computational complexity O(G))) and outputs ((G))) coefficients.
& X + R', and the G-TC will be applied
The G-Bispectrum The G-TC operation has a Fourier equivalent: the G-Bispectrum. Indeed, the definition of the Discrete Fourier Transform (DFT) can be extended to any finite group (see, e.g., [9]), as recalled below.
Definition 2.2. Given set of unitary representatives (Def. A.2) of the equivalence classes of irreps p. G + GL(V)) (Def. A.6), the G-Fourier Transform on a finite group G of signal θ G - R is defined as
F(O),. Σ (()))((o)t
(2)
9EG
where pt(g) refers to the conjugate transpose of the matrix pi(g) (or simply transpose if pt is real-valued).
The G-Bispectrum (()) is defined as F(T(O)), with F evaluated over the group G x G. Kakarala [15] proposed a closed-form expression for the G-Bispectrum ((O) directly in terms of F(O). We recall it in Theorem 2.3.
Theorem 2.3. [17] The G-Bispectrum of signal θ G + R, (())) is given by:
((0) [F(0)ps p())pz) Cpr.p
A F(O)}
C Cha.er
Lp Opp
where Opr.p2 is a unitary matrix called the Clebsch- Gordan matrix, whose definition is recalled in Appendix A. For each pair Pt, P2, the matrix 3(0) is called G-bispectral coefficient.
For commutative groups, Theorem 2.3 simplifies to a more compact expression, recalled in Theo- rem A.12. However, both the space and time complexity of the G-Bispectrum remain O(
To the authors' best knowledge, there is no generic analytical formula for computing Cpt-p2 for an arbitrary group G. However, there exist formulas for specific classes of groups (see Appendix E.2). Additionally, there exist packages for computing these for many groups using packages such as escnn [3].
3

---

Secondary NN
G-pooling
1/x max, Ox(s)
Input
@1(0)
(s1 * f)
Output
Geonvvluttin layer
@2(0)
(02 * f)
G-TC layer
(19
(x(s)
(øx f15
Bispectrum
layer
Figure I: Illustration of the different proposed G-CNN modules [6. 28]. The input f is first processed through the G-convolutional layer composed of K filters { Then, invariant layer is chosen (Max G-pooling, G-TC, or the selective/full G-Bispectrum layer). Finally, the "pooled" output is fed neural network designed for the machine learning task at hand.
Complete G-Invariants The G-TC and the G-Bispectrum are desirable computational primitives for signal processing and deep learning because they are complete G-invariants (for generic data e, θ). Indeed, this completeness property make them very interesting for building invariant layers in G-CNNs, as they are selectively invariant. We define complete G-invariance next.
Theorem 2.4. [16, Thm.3.2) The G-TC and the G-Bispectrum are complete G-invariants, i.e., for O, Ô: G + R with F(O), nonsingular for all irreps p. T(O) T(θ), respectively ((O) = B()),
if and only if there exists E G such that ((g) axhh, (g)) for all g E
The G-CNN architecture, first proposed [6], is illustrated in Figure 1. The imput signal f + R. typically image, is processed through a G-Convolution layer using filters {Ør K 1==1 The output is feature maps {0x]==] that form a set of real-valued signals with domain G. This G-Convolution layer is traditionally followed by a G-invariant layer. The most common the Max G-Pooling layer. More recent works have proposed two alternatives based on the G-TC and the full G-Bispectrum: the G-TC Pooling [28] and the (full) G- G-Bispectrum [29] respectively, where the latter requires the computations of the Fourier transforms of the feature maps, preferably computed using Fast Fourier Transform (FFT) algorithm on G [9]. When testing the impact of the choice of G-invariant layer, the output of the invariant layer is typically fed to Secondary Neural Network (NN) to perform the desired task, e.g., image classification. The Secondary NN often takes the form of a Multi-Layer Perceptron (MLP).
Application: G-invariant layers
Experimental results have demonstrated the superior accuracy and adversarial robustness of the G-CNN equipped with a G-TC and G-Bispectrum invariant layer [29, 28). However, both methods inherit the high space and time complexity of their respective operations. This raises the question of whether we can reduce this computational complexity.
Method: The Selective G-Bispectrum and its Inversion
The Selective G-Bispectrum We introduce a novel tool for signal processing on groups: the selective G-Bispectrum. A selective G-Bispectrum Aeel is any O(GG) subset of all coefficients of the G-Bispectrum 3 (Definition 2.3), only conserving well-chosen pairs of irreps (P1, P2). Which pairs of irreps to select depends on the group of interest. This is possible due to redundancies and symmetries in the full object. Below, we provide an algorithmie procedure compute the selective G-Bispectrum for any finite group G that features at most [Irreps (< [G') coefficients. The procedure summarized in Algorithm 1. We have the following proposition.
Proposition 3.1. The selective G-Bispectrum 3sel from Algorithum has at most |G] coefficients.
Proof. By construction of Algorithm 1, |pl < [Irrep|]. Since [Ireps] is at most the number of conjugacy classes G (see, e.g., Steinberg [30, Corollary 4.3.10J), we have Irreps] < |G].
In Algorithm 1, we note that the choice of Pi is important, since some # will not allow the user to recover all of the irreps and therefore not ensure the completeness of the selective. We illustrate the computation of the selective G- G-Bisspecrum in Figure where we choose タ1 P6.

---

Algorithm Selective G-Bispectrum on any finite group G
1: Input: Signal θ G + R with [G] Empty list of coefficients Lp. Empty list of irreps Lp. Kronecker
Table of G.
2: Add (())a to Le where po is the trivial irreps (po(g) 1 for all g G).
3: Add to Tip-
4: Choose PI such that #r @ pr is (DEER))
C/ generates at least one irres p yet in Lp- Add (( and (..) to Lss add #I to Lpp
6: Add every p that appears in f1 @ Pr to Ltp-
7: while T.p keeps changing: do
8: Find p',p" in Lp such that p' @ p" generates least irreps already Lp.
Add p("))'.e" to La.
10: Add p that appears in p¹ @ p" to Lp.
11: end while
12: Return (e) = Lg: the selected G-bispectral coefficients.
Computation of the Selective Bispectrum for the full octahedral group
Super-imposed on top of its Kronecker Table
CC
(Pg)
P0
0000000000
000C10000
P1
0113000001
DI1 ((0000
MAAGGNAANA
DOKK
00000OOT
0000001 10U
P3
0010000000
0111100000
111000000
0440000000
0100000000
0000000100
0CP0001111
0000011110
0000001000
G1 011000000
0110000000
1001100000
0001000000
0000000010
0000001 100
0000001 1.00
0000010011
COOORCOU IC
1010000110
C00003000
1000000000
10000099
10000000000
00000001000
COOOO TICO
09
0,0%0
10000
000000000
0000000010
0000060011
00000
888999-488
989888
8898668139
CTI
(10000000
0111100000
01100000000
0
D'0300000
0003000010
000000110M
0000001
0005010011
0000000010
000 (00000))
0113000000
DO00000
1001100000
1000000
4P
0X0XXX661
000XO (OXR)
3003000000
TARORORIOR
Selective Bispectrum 3 9 Porper @ Pure 8 Purt 9 Preas Pp.m
Figure 2: Computation of the selective G-Bispectrum for the Full Octahedral Group. The gradient of color represents the order in which the G-bispectral coefficients are computed. The Kronecker Table represents which irreps emerge from the decomposition into irreps of the tensor product P; Py- We observe that the selective G-Bispectrum has only 6 coefficients, compared to 100 coefficients for the full G--Bispectrum.
Inverting the Selective G-Bispectrum for completeness The inversion of the selective G- Bispectrum (e) is reconstructing a signal F(() from the G-Bispectrum coefficients in the list L,g such that θ = aggg θ) for some G (The G- G-Bispectrum is G-invariant, hence, O can only be recovered at best up to group action). Once F(θ) is known, Ô can be obtained using the Inverse Fourier Transform. If the selective G-Bispectrum can be inverted, then, by definition, it is complete in the sense of Theorem 2.4.
Theory: Completeness of the Selective G-Bispectrum
Our main theoretical claim is that the selective G-Bispectrum can be inverted and is a complete G-invariant that drastically reduces the complexity of the G-Bispectrum. We prove this claim for many finite groups G of interest in signal processing and deep learning in sequence of theorems presented in this section.
Known Theorems Previous authors had looked into the G-Bispectrum inversion problem. It is well known that |G| = coefficients are enough for the cyclic group (C,,-) (Z/Z,,, mod n).
Theorem 4.1. [17] For cyclic groups C.. n E No, the Cn-Bisppctrrm can be inverted using |G] = n coefficients if F())p for all irreps p of Cn.
Similarly for product of such groups, have the following theorem.
Theorem 4.2. [10] For a product of cyclic groups Cn x Cmn n, m E No, the G-Bispectrum can be inverted using [G] coefficients if F(O), # 0 for all p Cn x Cm:
5

---

Full and Selective Dispectra for the group D₃ Super-impoed on top of Dj's Kronceker Table
Full and Selective Bispectra the group Super-imposed top of Oa's Kronecker Table
Ao
@
Po
P1
P2
P3
Po
10000
01000
00100
00010
00001
Po
10000
01000
00100
00010
00001
P1 01000
10000
00010
00100
00001
P1
01000
11110
01111
01100
00100
P2 00100
00010
10000
01000
00001
P2
00100
01111
11110
01100
01000
Ps
00010
00100
01000
10000
00001
Ps
00010
01100
01100
10011
00010
P4
00001
00001
00001
00001
11110
P4
00001
00100
01000
00010
10000
Full Bispectrum
Full Bispectrum
(Bouo 1-00...
Selective Bispectrum
(3 pupup Pracor Ppape
Selective Bispectrum
(Bmons y Prreoo 8 Prsps Preeos
Comparison of full and selective G-Bispectra for the dihedral group Da (left) and the octahedral group On (right). The Kronecker tables of both groups show which irreps emerge from the decomposition into irreps of the tensor product of irreps Ps @ Pj. The colored boxes highlight the bispectral coefficients chosen for the full and selective Bispectra. Our proposed selective Bispectrum captures the same information the full Bispectrum but with significantly fewer coefficients.
Figure 3:
New Theorems From now on, we assume that the Fourier transform F(O) only features non-zero elements, or invertible matrices in the case of non-scalar Fourier coefficients. This assumption is supported by the zero probability of encountering this corner case (an arbitrarily small perturbation of any signal makes this assumption true).
We first extend the above results to all commutative groups. The proof relies on the fact that every finite commutative group is the direct sum of finitely many cyclic groups
Theorem 4.3. For finite commutative groups G, the G-Bispectrum can be inverted using G coeffi- cients if F(O), # 0 for all p E G.
See Appendix D for the proof and derivation of the inversion for the specific case of commutative groups We note that our approach to inversion is symbolic, in that solution can be expressed explicitly as a formula terms of the input. Other approaches are also possible to determine an inverse, such as using least squares [11] or more recent spectral methods [4].
We now extend the result to dihedral groups. Dihedral groups are ubiquitous in signal processing and deep learning because they represent the group of rotations and reflections.
Theorem 4.4. For any dihedral group Dn (symmetries of the n-gon), n E No, we need at most 2 + bispectral matrix coefficients for inversion if det(F(())p) + 0 for all irreps p of D,, This corresponds 1 4++++0 16
1=1
4 Dr,) scalar values.
The proof is provided in Appendix E. We now extend the result to octahedral and full octahedral groups, that are related to the symmetries of the octahedron. These groups are very important in signal processing and deep learning of 3D images.
Theorem 4.5. For the octahedral group O which has |G] = 24 group elements and 5 irreps, we need only G-Bispectral coefficients in the selective G-Bispectrum. this corresponds to 172 scalars. For the full octahedral group FO which has G 48 elements, we only need G- --sspecctrll coefficients in the selective G-Bispectrum to perform inversion. This corresponds to 334 scalars.
A sketch of proof is provided in Appendix F given the redundancy of the procedure. We see that the selective G-Bispectrum uses only 4 coefficients, compared to 25 coefficients needed for the full G-Bispectrum of the octahedral group. For the full octahedral group, it requires only 6 coefficients compared the 100 coefficients of the full G-Bispectrum. In Figure 3, we compare the full and selective G- G-Bispectra of the dihedral group DA (symmetries of the square) and the octahedral group.
5 Experimental results
Implementation and architecture Our implementation of the selective G-bispectrum layer is based on the gtc-invarien innce repository, implementing the G-CNN with G-convolution and G-TC layer [28] and relying itself on the escnn library [3, 32). The implementations related to this section can be found at the g- g-invariance repository.

---

Invariance layer
Computational Complexity
Ouput size
Complete G-invariant
G-TC
K
Full G- G-Bispectrum
G
KO
Select. G- G-Bispectrum
[O[ log G or GI
KO(GG)
Max G-pooling
K KOGG)
KO(1)
Avg G-pooling
KO
KO(1)
X
Table 1: G-CNN invariant layers and their computational cost and output size. K is the number of filters. The selective G- G-Bispectrum that propose is the complete G-invariant layer with the lowest time and space complexity. reduces significantly the cost compared to the G-TC layer while preserving its completeness.
We propose an experimental assessment of the newly proposed selective G-Bispectrum layer by comparing it with the Avg G-pooling, the Max G-pooling, the G-TC as invariance operations after the G-convolution of a G-CNN on the classification problems of the MNIST dataset of handwritten digits [23], the EMNIST dataset of handwritten letters [5] with standard train-test division. These datasets count 10 and 26 classes, respectively. We obtain transformed versions of the datasets
G-MNIST/EMNIST by applying a random action g G on each image in the original dataset.
The objective our experiments is to isolate the speed-up the G-Bispectrum layer. Hence, we consider architectures that only differ by the invariant layer in the classification task, following the experimental set up by [28]. The neural network architecture is composed a G-convolution, G-invariant layer, and finally connected layers with ReLU nonlinearity. Finally, a fully connected linear layer added to perform classification. The MLP's widths tuned to match the number of parameters across each neural network model. The details are given in Appendix G. We highlight here that the pursued objective is to compare the differences in performances of the G-invariant layers, not provide the fate-of-the-ane accuracy on the datasets involved. Henceforth, we do not optimize the architectures to reach the highest possible accuracy. We set simple architectures providing interpretable results for analysis. The experiments a performed using 8 cores of a NVIDIA A30 GPU.
SaaltiLaayer-Pereeptte
(MLP), itself composed of three fully
Training speed performance Table recalls the theoretical complexities of the different layers. The computational cost of computing the selective E-Biiseectumm is [ log [G]) if an FFT algorithm is available on G [9], and O( (? with classical DFT. in Figure 4, we report the average training times on (0(2)/((2) (2))))NNTSTT for 10 runs as the discretization C./ / D, of (0(2)/O(2) varies. In the first case, we use the FFT and observe that the Max G-pooling and G- G-Bispectrum training time scale linearly whereas it scales quadratically for the G-TC. For O(2), we perform a classic DFT on D, so that the G- G-Bispectrum scales worth. However, an FFT could be implemented to speed-up the process.
SO(3) MNIST
0(2) MNIST
1400
G-pooling
pooolnnng
Gepollng
G-pooling
1200
Selezt, G-hispectrue
Selort, (Hiispertaam TC
Training time
Trsining time
1000
800
900
600
600
300
4
6
Log,
log,n
Figure 4: Evolution of the average training times for the different invariant layers. The parameter n is the size of the groups C, and D,.- The average and standard deviations are obtained over 10 runs. For all runs, the number of parameters of the complete neural network (filters and MLP) is set to 50000 and 150000 for SO(2) and O(2) respectively. Standard deviations are reported by vertical intervals. When a FFT is available, our selective G-Bispectrum significantly outperforms other complete G-invariant pooling layers in terms of speed. Specifically, when working with Cz training on dataset of 60000 images takes only 247 seconds, whereas the G-TC requires 1465 seconds.
7

---

Dataset
Group
G
Pooling
K filters
Avg
Std. dev.
Param. count
Avg G-pooling
24
0.74
0.01
50247
Max G-pooling
24
0.96
0.01
50247
S0(2)
Cs
Select. G-Bispectrum
24
0,95
< 0.01
49116
G-TC
24
0.96
< 0.01
48385
MNIST
Avg G-pooling
4
0.60
< 0.01
147675
Max G-pooling
4
0.78
0.01
147675
((2)
Ds
Select. G-Bispectrum
0.93
0.01
143029
G-TC
4
0.96
< 0.01
142220
Avg G-pooling
24
0.40
< 0.01
50195
Max G-pooling
24
0.76
0.01
50195
SO(2)
Cs
Select. G-Bispectrum
24
0.77
< 0.01
49254
G-TC
24
0.80
< 0.01
48494
EMNIST
Avg G-pooling
20
0.38
< 0.01
48832
Max G-pooling
20
0.71
< 0.01
48832
(())
Ds
Select. G-Bispectrum
20
0.74
0.01
47320
G-TC
20
0.79
< 0.01
46954
Table 2: Results of numerical experiments averaged over rums with Avg G-pooling, Max G-pooling. our selective G--ispeecrrum and G-TC. The experiments are performed on
and (2)-NSSTTT.. The table shows the number of filters, the average classification accuracy. standard deviation and parameter count. This table shows that the selective G- G--ispectrumm conserves the accuracy of the G-TC at an equivalent number of parameters.
Classification Performance We compare the performances of the G-Bispectrum layer with respect the G-TC, the Max G-pooling and the Avg G-pooling models, trained on the (0)))0 MNIST/EMNIST datasets and we assess the accuracy by averaging the validation accuracy over 10 runs. The classification accuracy is provided in Table For the experiments in Table 2, the following pattern holds: at equivalent number of parameters, the more computationally expensive the pooling layer, the better the accuracy. However, the use of the G-TC becomes prohibitive when G increases. In the next section, we discuss the settings where each invariant layer should be preferred, and highlight each invariant layer's strengths and weaknesses.
Discussion on the choice of invariant layer The first observation from Table 2 is though the selective G-Bispectrum complete, the model obtains slightly lower accuracy than G-TC. This observation might be surprising first, since we prove mathematically in Section that the selective G-Bispectrum is complete, just as the full version. An explanation to this lies the paradoxes of the Universal Approximation Theorem [13]. Just because arbitrarily large MLP theoretically fit any function, this does not imply that it will happen for a practical, limited MLP. In practice, we hypothesize that the redundancy of the G-TC allows the MLP to distinguish inputs more casily. If the size the model allows it, the G-TC or the full G-Bispectrum will provide better accuracy. However, when the size of the group is big, their use is often out of reach while the selective G-Bispectrum scalable. Table 2, we also notice that the Max G-pooling performs well compared to the others though it is not complete. This is because classification. Indeed, assume f, Φu are Black-and-white images with N pixels. In consequence, max, O& (0, .... N) for k 1, 2... K. The Max G-pooling allows maximum separation of (N 1)K classes. In practice, this value is not reached, but it explains why Max G-pooling performs well. Figure 5 highlights this dependency the Max G-pooling on the number of filters since the accuracy drops less than 60% with filters. In comparison, the G-TC and the selective G-Bispectrum, which are complete, keep an accuracy above 85% with filters.
have many filters that allow for refined
To conclude our numerical experiments, study the robustness of the selective G-Bispectrum to adversarial attacks, following the analysis in Sanborn & Miolane [28, Figure 2). Given an image f X + Re and filter Φ X + R they numerically verified the robustness (=completeness) of the G-TC by showing that
Completeness
f E arg min f:X [| * f) T( - f
+ -
f for some g E G.
(3)
Indeed, Sanborn Miolane [28, Figure 2] shows that only images that are identical up to rota- tion/reflection can yield the same C./ Cn.D,. DCC. That is, the G-CNN with G-TC can not be "fooled"

---

MONS)-MINSST
012)-
Average accuracy
0.97
Average accuracy
0.8
0.8
0.7
0.7
G-pooling G-pooling
G-pooling
0.88
0.6
Crapooling
Select, G Bispectrum TC
Select. Dispectrum
G CC
0.5
0.5
10
15
20
5
Number K of filters MO(S)
Number K of filters
0(3)-MN1S1'
0000
Average accuraty
Average accuracy
0.9
0.8
0.8
0.7
0.7
G-poeling
C-poolinn
0.8
Max Erppooling Seleet. Disportrum
0.6
Seleet. & Disperraum TC
O TC
0.5
0.5
125
130
135
140
145
150
156
140
160
180
200
220
Training time [s]
Training time
Figure 5: At the top: Evolution of the average classification accuracy with rotated MNIST (SO(2)- MNIST) and rotated-reflected MNIST (0(2)-MNIST) over 10 runs when the number of filters varies from to 20 for the Avg G-pooling, the Max G-pooling, the selective G-Bispectrum and the G-TC. The number of parameters of each model is maintained equal for fair comparison. The standard deviations are represented using vertical intervals. With the selective G-Bispectrum layer, we can reduce the number of convolutional filters needed for given accuracy. For example, with only K filters, we achieve 96% accuracy, compared 63% with the Max G-pooling layer. Our approach allows G-CNNs maintain competitive accuracy while using smaller neural networks. At the bottom, the same results are displayed with time instead of number filters on the æ-axis. The dotted lines reproduce the evolution of K from the figures at the top. We can observe that the selective G-Bispectrum faster than the G-TC when a FFT is available, thus here in the case of Recall that FFT can be implemented for many groups [9]
since only input the same orbit yield the same output. We perform similar experiment Figure Moreover, it is well-known that the G-convolution f is G-equivariant. Hence, an equivalent experiment is to show that
e* E arg min e:G-+R
[T()) - T(θ) e = alg, θ) for some g E G.
In Figure 7, we show that the selective G-Bispectrum Bees is robust to adversarial attacks by solving
θe E arg min e:GrR
(9) Seel (
(4)
The signals are indeed recovered up to translation, i.c., a group action of C3n Moreover, despite (4) only optimizes using the selective G-Bispectrum, the full G-Bispectrum correctly recovered. This is an additional evidence of the completeness of the selective G-Bispectrum.
6 Conclusion and Future works
In this paper, we introduced new type of complete invariant layer for G-invariant CNNs called selective G-Bispectrum layer with the objective of increasing the accuracy and robustness of G-CNNs compared to those implemented with the initially proposed Max G-pooling. The G-TC p layer also achieves this goal, but at an output cost of 2 coefficients and [[G)(² flops that prevents application to large groups, while the selective G-Bispectrum layer only outputs ([G)) coefficients. Building the result of Kakarala [15] for cyclic groups, we have shown that the completeness of the selective G-Bispectrum layer holds for all commutative groups, all dihedral
groups, the octahedral and full octahedral groups In a suite of experiments, provided global

---

Trained solert. G-Blspeerum
Traned G-Mas nooling
Thained solert. Z-Bpseerran
Trained G-Mas maoling
Thainel solest. G-Baspestrun
Traied C-Max poeling
Original Image
Initiel Inaage
Original Image
Initiel Intage
Original Image
Initial Image
S YOU
93
7
1
3
/
9
Figure 6: Adversarial attacks experiments with G C4. Images optimized output, respectively, a target selective G-bispectrum and target G-Max pooling, obtained from an original image. The initial image is the initialization of the optimization process (3). After training, only for the selective G-Bispectrum (in blue), the recovered image a copy of the original image up to group action (rotation). This is numerical illustration of the robustness of the selective G-Bispectrum to adversarial attacks: one can not obtain the same output with an input that is not in the same class, On the other hand, G-Max pooling (in red) outputs noisy image because it is complete.
Original fall G-Dapestram
Recreored full G-Diepectum
Orisinal full G-Bapertrun
Recowrrd full G-Rapertrun
Origiial fall G-Eilepetrrum
Recyoted full G-Blepectrmm ti
Oriannal sagual
Recoerel signal
Original sigual
Recevsped signal
Original stgunl
Recreered signal
Morry ww oahor yahn
mow Ma Nanon mnrtha
vamul Wom
ryway Mmm
myw whan Wingh rymow
Figure 7: Numerical experiment of signal recovering from original signals (6. where θ; G + TR by solving (4) with G = C20- We use the gradient method with Armijo line search to solve (4). The recovered solutions {O ] are represented and correspond translations the original signals. The moduli of the full G-Bispectra are also represented and are identical. This experiment corroborates the completeness of the selective G-Bispectrum since we are able to recover unknown signal only from the knowledge of its selective G-Bispectrum.
picture of the strength and weaknesses of each invariant layer. We studied the performance in terms of training speed, classification accuracy and robustness to adversarial attacks. As result, the selective G-Bispectrum paves the way for the development of complete invariant pooling layers that can accommodate larger group sizes and, hence, a larger set of symmetries.
10

---

References
[1] Bhatia, R. Matrix Analysis, volume 169. Springer, 1997. ISBN 0387948465.
[2] Brilliiger, D. R. Some history of higeer-order statistics and spectra. Statistica Sinica, 1(2):465- 476, 1991. ISSN 10170405, 19968507. URL http://www. /tt.atpp////ww.sssorroro/////// http://wwww..jooooooo/o//abaebba wttps/orr///////0/0000/0/
[3] Cesa, G., Lang, I.., and Weiler, M. A program to build F(n)-equivariant steerable CNNs. In International Conference on Learning Representations, 2022. URL https: net/forum? 1d=WEAge9xInQw.
[4] Chen, H., Zehni, M., and Zhao, 7. spectral method for stable bispectrum inversion with application multireference alignment. IEEE Signal Processing Letters, 25(7):911-915, 2018.
[5] Cohen, G., Afshar. S., Tapson, J., and van Schaik, A. EMNIST: Extending MNIST handwrit- ten letters. In 2017 International Joint Conference on Neural Networks (IJCNN), pp. 2921-2926. 2017. doi: 10. (ttp:/JCNN.2011..9909
[6] Cohen, T. and Welling, M. Group equivariant convolutional networks. In Balcan, M. F. and Weinberger, K. Q. (eds.), Proceedings of The 33rd International Conference Machine Learming, volume 48 of Proceedings of Machine Learming Research, pp. 2990-2999, New York, New York, USA, 20 22 Jun 2016. PMLR. URL https: cohenc16 html.
http::::/t:::eeeeeeeee http///rreeeed..././e// mosiinggsml. mlr S.l...reessvv..
[7] Cohen, et al. Equivariant convolutional networks. PhD thesis, Taco Cohen, 2021.
[8] Deng, I.. The MNIST database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 29(6)1141-142, 2012.
[9] Diaconis, P. and Rockmore, D. N. Efficient computation the Fourier transform on finite groups. Journal of the American Mathematical Society, 3:297-332, 1990. URL https: / //api www.seeemammsoll..m.
Arg/CorpusID:: 120893890.
[10] Giannakis, G. B. Signal reconstruction from multiple correlations: frequency- and time-domain approaches. J. Opt. Soc. Am. A, 6(5):682-697, May 1989. doi: 10. 1064//0OAA 6.00082.. URL https://opg optica. org/ josaa/ aatraaaaastttctt...m cfn?URI=josaa 6--aaa---66688- 582.2.
[11) Haniff, C. A. Least-squares Fourier phase estimation from the modulo bispectrum phase.
Journal of the Optical Society of America A, 8(1):134-140, January 1991. doi: 10.1364JJOSAA. 8.000134.
[12) Higgins, I., Amos, D., Pfau, D., Racaniere, S., Matthey, I.., Rezende, D., and Lerchner, A. Towards definition of disentangled representations, 2018. URL https: org. abs/ 1812. 1812.02230.
[13] Hornik, K., Stinchcombe, M., and White. H. Multilayer feedforward networks are universal approximators. Neural Networks, 2(5):359-333 3369-366, 1989. ISSN 0893-6080. URL https: //doi. f/doi..or//0... 1000.001608999 6080 90020-8.
[14] Joyal, A. and Street, R. An introduction to Tannaka duality and quantum groups In Carboni, A., Pedicchio, M. C., and Rosolini, G. (eds.), Category Theory, pp. 413-492, Berlin, Heidelberg. 1991. Springer Berlin Heidelberg. ISBN 978-3-540-46435..
[15] Kakarala, R. Triple correlation on groups PhD thesis, UC Irvine, 1992.
[16] Kakarala, R. Completeness of bispectrum on compact groups. 2009. URL https://api. semanticscholar- www.oorpgg/D:D:i...oo.oo :ttp//oorpu.IDI:142222222 18425284.
[17] Kakarala, R. Bispectrum on finite groups. In 2009 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 3293-3296. 2009. doi: 10. 1LLLL///AAASS20009999963333
[18] Kakarala, R. The bispectrum as source of mhase-sensitive invariants for fourier descriptors: troup-theoretic approach. Journal of Mathematical Imaging and Vision, 44:341-353, 2012.

---

[19] Kondor, R. A novel set of rotationally and translationally invariant features for images based on the non-commutative bispectrum, 2007. URL https: https: //arxiv.// org/ abs/cs/0701127
[20] Kondor, R. Group theoretical methods in machine learning. PhD thesis, Columbia University, 2008.
[21] Kondor, R. and Trivedi, S. On the generalization of equivariance and convolution in neural networks to the action of compact groups. In Dy, J. and Krause, A. (eds.), Proceedings of the 35th International Conference Machine Learning, volume 80 of Proceedings of Machine Learning Research, pp. 2747-2755, 2018.
[22] Lecun, Y. and Bengio, Y. Convolutional Networks for Images, Speech and Time Series, pp. 255-258. The MIT Press, 1995.
[23] LeCun, and Cortes, C. MNIST handwritten digit database. 2010. URL http::/yannn
lecun. /ec/n.com/eae www.coo///c///eeee boom/exdbmmiis///
[24] LeCun, Y., Bottou. L., Bengio. Y., and Haffner, P. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2333 1998. doi: 10.1109/5.726791.
[25] Nikias, C. L. and Mendel, J. M. Signal processing with higher-order spectra. IEEE Signal processing magazine, 10(3):10-37, 1993.
[26] Norman, C. Finitely Generated Abelian Groups and Similarity of Matrices over a Field. Springer London, 2012. ISBN 9781447127307. URL http: //xx doi. .tg/10.10077 978- 178-1-4471-273--
[27] Sadler, Shift and rotation invariant object reconstruction using the bispectrum. Workshop on Higher-Order Spectral Analysis, pp. 106-111, 1989. doi: 10.119/HHSSA1199..75522
[28] Sanborn, S. and Miolane. N. A general framework for robust g-invariance in g-cquivariant networks. In Advances in Neural Information Processing Systems, volume 36, pp. 67 67103-67122 124. Curran Associates, Inc., 2023. URL https://proceeddng.. Comper-cc pdf.
[29] Sanborn, S., Shewmake, C., Olshausen, B., and Hillar, C. Bispectral neural networks. In International Conference Learning Representations (ICLR), 2023.
[30] Steinberg. Representation Theory of Finite Groups: An Introductory Approach. Universitext. Springer New York, 2011. ISBN 9781461407751. URL https://booke: /ooggs/ com/ bookss7.d-ugggggAAAAAA
[31] Tukey, J. The spectral representation and transformation properties of the higher moments of stationary time series. Reprinted in The Collected Works of John W. Tukey, 1:165-184, 1953.
[32] Weiler, M. and Cesa, G. General E(2)-equivariant steerable CNNs. In Proceedings of the 33rd International Conference on Neural Information Processing Systems, Red Hook, NY, USA, 2019. Curran Associates Inc.
[33] Weiler, M., Forré, P., Verlinde, E., and Welling, M. Equivariant and Coordinate Independent Convolutional Networks. 2023. URL
weiler gitllb.......................... io/cnn_bockk pdf.
[34] Zetzsche, C. and Krieger, G. Nonlinear mechanisms and higher-order statistics in biological vision and electronic image processing: review and perspectives. Journal of Electronic Imaging, 10(1):56- 99, 2001. URL https: http://doo..ooooo//// www/oo../............ 1117/1. 1333056.
12

---

A Background on groups
We introduce the fundamentals of group theory, which provide the foundation for the theory of G-CNNs. These notions can be found in [30].
Definition A.I. A group is a pair (G, -) where G is a set and G x G G an associative multiplication such that there is an identity element E G (i.e., for all g E G, e.e.g - E) and, for all g E G, there is an inverse g E G' such that g g1 g = g 1----
A group is thus a set G combined with a product preserving the characteristics of G. Here, the established term "product" can be misleading. It denotes any operation which makes Definition A.1 true given the set G. For instance, (IR, +) is a group. Another example is GL,(R"), the n x n real invertible matrices, associated to the usual matrix product. This group is said to be non-commutative since A B B + B A in general for A, B E GL(RR". An important group for us is the set {0, n - 13 associated with addition modulo n. It is usually written Z/nZ and called the cyclic group C.. A single group can arise in different contexts under seemingly distinct forms. For instance, Z/4Z and the rotations leaving the square unchanged in R are fundamentally the same object. This observation gives rise to representation theory, branch of group theory studying how the same abstract idea of group can emerge under different forms.
Definition A.2. A representation of group (G, .) is pair (p. V) where V is a vector space and p: G + GL(V) is a group homomorphism, i.e., for all g, h E G, p(g h) = p(g) If V is equipped with an inner product and if for all g E G and all u, E V, (p(g)v, p(g)w) (u, v), is unitary.
Remark A.3. Throughout this paper, we use the shorthand G' to refer to the group (G,-) and p to refer to a representation (p, V).
To illustrate Definition A.2, a representation of C, is given by the complex roots of unity, p(k) exp (22))), on the complex one-dimensional vector space V = C. Every group also admit the trivial representation: (o(g)= 1 for all g G. There is specific subset of these representations called the irreducible representations, irreps for short, being those that can not be expressed in a more compact form. The irreps fundamental objects of group theory since they allow us define an invertible Fourier transform on finite groups The irreps are therefore needed to define the G- G--ispectrum ie., the Fourier transform of the G-TC. The notion of irreps is derived from that of G-invariant subspace, which we recall in Definition A.4.
Definition A.4. Given a representation (p, V), a subspace W C V is G-invariant if p(g) W for all E G, w E W.
The formal definition of the irreps is then stated the representations with no non-trivial invariant subspace.
Definition A.S. A non-zero representation (p. V) of a group G is irreducible if the only G-invariant subspaces of V are {0) and V itself.
A single group acting on different spaces will have different representations. However, one can reveal the similarity between these representations by the mean of an equivalence relation.
Definition A.6. Two representations (p. V) and (yp, W) are equivalent if there exists isomorphism T V + W such that for all g E G = TTp(g)-
For the interested reader, the invertibility property of the Fourier transform is a consequence of the concepts of Pontryagin duality (commutative groups) and Tannaka- Krein duality non-commutativv groups); see, e.g., [14]. Our proofs will also rely on the notion of generating set of G. which we introduce here.
Definition A.7. A generating set S of group (G. is subset S C G such that every g E G can be expressed as a finite combination of the elements in S and their inverses under the group action :,
Remark A.8. It can be shown that every group G of size |G| has a generating set of size at most log2 G).
Group Actions A group (G, .) represents a set of transformations such as rotations that can act on data such as images. We define formally how groups can indeed transform datasets through the concept of group action.
13

---

(A) Complete Invariance of G- Bispectrum
Image dataset
Power Spectrum
Bispectrum
excessive invariance
complete invoriance
Figure 8: Illustration of the concepts of excessive and complete invariance to group action. With the excessive invariance, samples from different classes can be mapped to the same output.
Definition A.9. Given a group (G. -), group action a G x X + X is function satisfying i) Identity: a(e, x) z, ii) Compatibility: a(h, alg, x)) a(h g, g. for any x X and h, E G and where is the identity of G.
Processing operations and neural networks can be designed so that they respect group actions: specifically, group acting the input (e.g., rotating input image) should yield group action on the output (e.g., a rotation of the output feature map). This the notion of G-equivariance.
Definition A.10. function v + Y is G-equivariant if (((1(9.x)) = a2(g, (x)) for all X and all g E G, where 0₁ and ag group actions X and Y. respectively.
For example, the G-convolution layer is G-equivariant by design [6]. An important problem in signal processing and deep learning to achieve invariance nuisance factors not relevant for the task. Many of these factors are describable as group actions (e.g. rotations, translations, scaling). Thus, we want processing methods and machine learning models be G-invariant:
Definition A.11. function & X + Y is G-invariant if (b((g, x)) (x) for all and all E G.
For example, the Max G-pooling (maxgeG O(g)) traditionally follows G-convolutional layer to remove the equivariance of the convolution and achieve G-invariance. A G-CNN is neural network that consists of G-convolutional layers and a pooling/invariacee operation. The main applications of our proposed selective G-Bispectrum operation is to act as a G-invariant pooling layer, that can conveniently replace the classical Max G-Pooling layer of G-CNNs, as shown the rest of the paper.
Clebsh-Jordan matrices Given a group (G, and family of unitary irreps {pi) Clebsh-Jordan matrix is analytically defined for each pair Pt, P2
[Ireps]-1 the
(or s (2) Cpr.pa
+ p(g) Ct - PER
(5)
G-Bispectrum for Commutative Finite Groups The computation of the G-Bispectrum simplifies for commutative groups compared to Theorem 2.3, as recalled below.
Theorem A.12. [17] (G, .) commulative group and θ G IR, the G-Bispectrum can be computed as
B(O) =()),, Sps
(6)
For commutative groups, the G-bispectral coefficients are complex scalars [30].
B Indeterminacy of G-Bispectrum inversion problem
It is important to state precisely which information we can possibly retrieve from the G-Bispectrum.
consequence of the G-invariance of ((() that the G-Bispectrum inversion problem is ill posee...d.
Recall that G-invariance means that for all h G, we have 8(a(h, @)) -3())p p2 Given

---

function θ G + I., a possible definition for the group action on θ is given by a(h, O = O(h-1 g) for all h E G (see, e.g., [6]). Therefore. for all G. have
F(a(h, 0))p
Σ ((9,)())))()))
gEG
p(hh
Σ DEG
C 9)p())))())
= =(h))F(Opp=
which shows that the G-Fourier transform F(O) is G-equivariant. In consequence, recovering F(O) from 3()) can best be done up to an unknown factor p(h). Moreover, as explained in [15, 20], the indeterminacy is not limited to p(h). Take for instance C,. An indeterminacy factor ps(hh) exp (2mthk) corresponds to translation of h E C. of the signal, ((g) Og + h). [15] showed that h is not restricted to Cn Z/nž: it may take any value in [0, ra]. The Bispectrum is
not only invariant to a discrete set of rotations, but to the continuous group of rotations SO(2). The
factor can thus be written exp(ipk) where yp E [0, 2TT).
C Selective G-Bispectrum inversion: known results
From now on, we assume that the Fourier transform F(O) only features non-zero elements, or invertible matrices in the case of non-scalar Fourier transform. This assumption is supported by the zero probability of encountering this corner case.
Cyclic groups C., We start with (G, - = (Z/n2, + mod n) =: Cn, Recall that the irreps are given by (g(g) = exp (wag) where wis == for l Z./nZ (see, e.g., [30]).
Theorem C.1. [17] For cyclic groups Cn, the Cn-Bispectrum can be inverted using |G]
coefficients if F(O), for all p On-
Proof. The Fourier coefficient associated to the trivial representation F(O) po: is uniquely determined and can be recovered from Theorem A.12 by identifying phase and modulus:
B()) F(()) pull 1 exp (i arg(F(()).))
We proceed using Pontryagin duality: the irreps {p=) form a group & themselves, with G G. In the case of the cyclic group C.,, notice that for all j, E 2/nZ., Pj & Ps: Pjtk- Leveraging (6), we can use 3(O) to recover F(0),)
[F(O) 12- 2= 3(0) po.01
(8)
F((), pm
Equation (8) leaves an indeterminacy the phase of F(O),, This corresponds to the indeterminacy factor exp(is),, E [o, 2T) of Appendix B. inherited from the G-invariance of (()) (it is not injective, hence you cannot distinguish inputs that have the same G- G-Bispectrum). For now, let arg(F(()p.) = 0. The key to recover all the other Fourier coefficients to notice that S = {1} is generating set of G L/nZ. Therefore, computing sequentially
3(0) PL
F(O) 11 F(O),,, F(O),,
for k = 1,2, before is not a valid shift. valid phase shift for F(O)pr is such that the shift w.r.t the original signal has the form exp 2mih for h E N. This valid phase shift is easy to find. It is the unique P E [0, 2x such that, if we define F((),z exp((kk)F(()) for all k E Z/nZZ, then we have n (F(() R." (i.e., with no imaginary part). Note that this is explicit method to recover valid phase while [17] relies on existence without explicit method to find it. The method is summarized in Algorithm 2 and illustrated in Figure 9. In consequence only the following G- Gibispett coefficients are needed for completeness: B(O)po.po 3()) po.p. and ((O)p.. pL. for 1, ..., n 2. This makes total of |G| coefficients. We summarize this result in Theorem C.I.
recovers completely F(O). We done yet because the phase fixed
15

---

Algorithm/ 2 Bispectrum inversion on 2/nž [17]
B Ppp.po- - -poppt
Pprip P
I: Input: 8(0) p0.00 (())po.p and 3()) pI for k
1, 1,2,
2: Compute [F(@) =
(8(0) po. and
F
arg(F(()po) (TTg(B(()))..uu po.P))
--> is followed by
gives
3: Compute |F(H)pp
[(0))po01 x(())o
and set
arg(F(()))
4: for h 1,
do Compute F(0) F(()g F(B)oz
Figure 9: Illustration of Algorithm 2.
((0)
The Bispectrum coefficients allow re- cover the Fourier transform sequentially, up to group action. First, Spo.Po gives F por which, combined with $ pp.e gives F etc.
6: end for
7: Find E [0, 2 such that F (F(()) E R" where
F(O)ps exp(ok)F(e))
8: Return F()) (up group action).
D Selective G-Bispectrum inversion: commutative groups
Here, we prove the theorem stated in the main text and recalled below:
Theorem D.1. For finite commutative groups G, the G-Bispectrum can be inverted using |G] coefficients if F(O), 0 for all p G.
Specifically, we extend Algorithm 2 to all commutative groups, based on the Theorem D.2. That is, design method for the direct sum of finitely many eyclic groups.
Theorem D.2. (see, e.g- (26]) Every finite commutative group G is isomorphic to finite direct sum of cyclic groups: G ~ θ 2/nz where L E N and E N for I = 1, L.
For all G (k is integer-valued vector of length L), the irreps Pk : + are given by nt.
())(g) = I exp( 2miks gt). The number of irreps is |G| = I nt- We detail and prove in Theorem D.3 the procedure to invert the G-Bispectrum on commutative groups. The procedure is summarized in Algorithm 3 where
the two following notations.
1. e denotes the basis vector in Z such that et = if k: = 1 and o = 0 otherwise.
2. K2 = [tc' + k 1,2... 1 and k E K(--1 for = 1,2, L.
The sets Kt are recursively constructed such that KL G. For G = (Z/32)3, the sets K1, K2, K3 represented in Figure 10.
Ki
(2/32))
K.
Ka
2.0
Figure 10: Representation of the sets K1, K2 and K3 for G = (2/32)3.
Theorem D.3. For finite commutative groups G. the G-Bispectrum can be inverted using G coefficients if F(O), + for all G.
Proof. Notice that for the commutative groups, we keep the property Pte @ Pk. = Pk+ where (k + k'); = ki + k/ mod for | 1,2, ..., L. The first step is to obtain generating set S of the
16

---

irreps is of size L. It is given by the usual basis vectors S = fe' E zz for = 1,... L} where
1ifk= c 10 otherwise.
By Theorem A. it is sufficient to have the Fourier coefficients associated to each generating element in S to recover all the Fourier coefficients. Indeed, knowing F(() and F(O),,, allows us compute F(0)pk By definition of the generating set S, can thus recover F(@) pu. for all G. The moduli of the coefficients F(O) for = 1,,2,-. .., can be computed follows:
( F(())o pa
(3(() po:02
PF())
(10)
where F(O) is the Fourier coefficient of the trivial representation (computed as (7)). We claim that the phase be fixed independently for each label thus L times. This is because only one factor h2 remains among the L independent factors in
2wihi
F(a(h, 2))p2 = Pel (b)F(()) = exp
F(0)pg)
for all / = 1,2,, L. Therefore, fixing an arbitrary phase in (10) only fixes h4. Again, the indetermi- nacy factor h4 is not restricted to Z/n,Z but can belong to [0, ny]. We will have solve this issue further. For now, we set the phase of F(O),.. to zero. Once the Fourier coefficients are known for all the generators of the group of the irreps (Pet 1 remains to combine them to obtain all the elements in the groups and, consequently, all the associated Fourier coefficients.
At this point, it helps to consider the problem geometrically. Each irreps Pk can be associated to its integer coordinate k inside a hyper-rectangle in R whose length of edges is n; for | 1, 2... We combine the coordinates e' to obtain all the possible integer coordinates inside the hyper-rectangle. First, we can obtain the L orthogonal edges of the hyper-rectangle. For 1,2, L, Pei 3 Pe' gives Pzet- Pet 2 P2e' gives Pset. etc. This is in fact the procedure of Algorithm 1. Now, we combine the edges to generate the inside of the hyper-rectangle. We proceed iteratively. For = 1,2. I., we define K = {to + k f 1,2, .., n2 1 and k E K(-1) and T == {0}. This construction such that K = 1 Z/n, and KL G. We generate the missing Fourier coefficients by combining the ones associated to the generating set of G. For I 2, .... I,, compute
( 3(9) 3(0) PLa F(O))).)(()).)
F(O)
(11)
fort = 1.2.... - for all k E Kt-1, Intuitively for L = 3, we obtain first an edge, then face and finally the full parallelepiped. To conclude, we reproduce the procedure from Algorithm find a valid phase shift for in each basis direction e!. The last step is then to compute, for all k E G,
F(O), = F(O),, II (xp()))))
The procedure is summarized in Algorithm 3 and illustrated in Figure 11. shows that the bispectral
coefficients needed for completeness are: 3(9) p(O))o 8(0) for | = 1, ..., L, t = 1,, 2,... 2 and all k K!-1, We recover exactly Fourier coefficient per G-bispectrum coefficient. This makes thus a total of GG bispectral coefficients precisely and proves the following theorem.
E Selective G-Bispectrum inversion: dihedral groups
Dihedral group D, The dihedral group D,, the group of all symmetries of the n-gon. Mathe- matically, it is defined as
Dn = (2, alal a" x² vax a -'),
(12)
17

---

Algorithm 3 Bispectrum inversion for finite commuta-
tive groups (G θ 1=1 2/m(2).
Input: [G] bispectral coeffs.
2: Compute [ =
(8(O)) and ( )arr(
3: for
do
(p(9)nuipa and set 5(0)ro
4: Compute |F(()
arg ((())aa) =
3ponn Apoin
Pee IP
5: for f = =1 - do
6: if 1 then
7:
F(O)pra!
F(O)pa F(
F
Pd
F Paol
is followed
8:
end
for E K' (0)
9(0), 1.0k F(o), 110)
10: Compute F(O)
Figure 11: Illustration of Algorithm 3. The Bispectrum coefficients allow re- cover the Fourier transform sequentially, up group action.
end 12: end for
13: end for
14: for = 1,2, 1,2,...11 I..,I.o do
15: Find şi E 10, 2 s.t. 1ol.F-(FF (F(O)) mo. "R
where F(O)prat = exp(btk)FF(O)
16: end for
fork g do
18: F(() F(O) I exp(
19: end for
20: Return F(θ) (up group action).
where a is the rotation and æ is the reflection, and they form a generating set of D... We will only consider the case n >> 2 since the cases n and 2 commutative groups covered by the previous subsection, while n > 2 gives non-commutative groups The 2D irreps of D, are given by
(w)(()))) sin (wrk) cos(wikk)
sin(wrk)
[
0
(a(aaa=") =
(13)
where w/ for l 1,2,... J.
There are also or 1D irreps if odd or even, respectively. We denote these 1D irreps by Pa, P01, Puz, and Pos (see Appendix E.1). The two last ones only exist for n even.
Theorem E.1. For the family of dihedral groups D we need at most | matrix coefficients for inversion if det dct()(())p) + 0 for all irreps p of D,, This corresponds to 1+4 16 1=1
+ bispectral
D,,, scalar values.
Proof. In view of section C, we wish to show that there is an irrep P1 that generates all the irreps of D,. As in the cyclic and commutative cases, first deduce F(O)) po from 3(0) po. Now, have
(()), ;(0) (00))o.P1
(14)
F((),,
The novelty for this non commuuuttaevee group is that F(O),, is 2 x 2 matrix. After computing the eigenvalue decomposition ((0) VAVI, we can choose
F(0)mo
F(O),, p. VAiviu.
(15)
for all unitary (i.e., uut UtU /) to solve (14). In the case Z./nZ the indeterminacy belonged to SO(2), the continuous set of 2d rotations. For D it belongs to O(2), the continuous set of 2d
18

---

Algorithm 4 Bispectrum inversion on the dihedral group D.. 1, 2, .. 1=
1: Input: ((0) po-Po 8(0)po-P1 and (())er
2: Compute (F(0) po) = (3(()) ) and arg(F(())) pr
3: Compute VAVt
8(0)
5(0)p
4: Set F(O)p1 VA V with valid U (existence of U ensured but not computed easily).
for ..... -
do
6: Compute
+ pEP1 S
F(O), = (Cpr.px 1 [For @ Fp CPLPn--)
7: end for 8:
Return F(O) (up group action).
rotations and reflections. For finite groups, Kakarala [15] ensures that the only choices for U such that F(O) a Fourier transform on G(= Dn) are such that identical to the original signal up to some group action g D,, For k = 2, .... n== J, can then obtain
F(O), (ct, .p pF))), @ F(O)p= ((O) C CLL....)
(16)
PEP1 08
is shown in Appendix E.1.1 that appears in the tensor decomposition (5) of @ Pk-1 so that the for-loop can be applied. Moreover, know from Appendix E. that pmi appears in the decomposition of #1 @ P1 and, for even, P02. P03 in pi @ 12--1- Thus the iteration recovers the complete DFT F(O). B(+)
a scalar, ((+) po.p1 is a x matrix and ((O)
PI.P (()))..p * 0, is
4 x matrix. Hence the total number of scalars that is required + 4 + 16 _6
The procedure is summarized in Algorithm 4.
E.1 The 1D irreps of D,
We recall the definition of the dihedral group D,, given in (12). The ID irreps of D, can be found, e.g., in [30]. They are given by:
po(g) for all g E Dn.
1 if g E (a),
(g)
1-1 otherwise.
1 if (a², x), otherwise.
If n even, Po)(g)
1 if (a?, az),
If n even, poa
- otherwise.
To exemplify the ID irreps, we give their values for DA in Table 3.
Q
a
a.
a=x
a
Pu (g)
1
1
1
1
(g)
-1
-1
-1
(02(g
-1
Po3(g)
-1
-1
Table 3: ID irreps of D4.
E.1.1 Generation the coefficients of D,,
Theorem E.I makes two assertions that we verify explicitly in this appendix. First, it said that Pk in the tensor decomposition (5) (TD) of pı & Ph-1 for k .,.. n-1 that the iteration of
19

---

Algorithm 4 recovers all the Fourier coefficients associated to the 2D irreps. The second assertion to verify is that Pai is in the TD of p1 @ P1 and, for even, Po2. Poa the TD of p1 Pr -1 so that the Fourier coefficients associated to the ID irreps are also recovered, asserting the validity of the inversion procedure. We provide an analytical proof of these two assertions. The proof is based on the theory of character functionn..
Definition E.2. [30] Given group G and a representation p. the character of is the function X G - R + Tr(p(g)). Xo is said to be an irreducible character if is an irreducible representation.
The character function Xp class function on G, i.e., Xp is constant on conjugacy class of G. The space of class functions on a finite group G, written Sg, can be equipped with an inner product (: Sg x Se + C such that for u, E Sg, we have
(u, u)a ==
u(g)v(g).
(17)
SEIGI
The irreducible characters form an orthonormal basis w.r.t (,, de for So [30], i.e.
1 if Pa pb.
(Xp. Xp.)a =
(18)
0 otherwise.
Therefore, for Pa. Pb two irreps of G. Pe is in the TD of Pa @ Po if and only if (Xp, Xp. * 0. Let us apply this to the 2D irreps of D, (n 2). Let Pt, Pj. Pk be three irreps defined as (13).
Notice that we have Xp(a'xx 0. This yields
: Xpx) = 2n
Xnss, (a') (a') (a')
1=1
Xp (a xxp, (ax) (a))
2n
1
8 cos( cas (wrr) cos(ur)))
2n
cos + +++ + cos(wili + k))
+ cos((a (a)) + k)) + cos(wili - i- k))-
(0 #0
Recall that Ei= cus(umm) generality, (Kuop, Kpx) # if and only if
Therefore, if we assume 0 I i j without loss of
n if m
h = i or k = j-i- i.
Therefore, based on Definition 2.3, by utilizing Pp..py. F(O)p. and F(O),, p,, we can compute F(O), pa for k E (i J. j - i). By iterating, if F(O)), is known, SpI.0r .n gives F(O) Then, 3pi.0o can be leveraged to obtain F(O)pp Continuing the procedure provides F(O),a for 1: 2, ..., 1"- by using P We have thus obtained the Fourier coefficients associated to all the 2D irrpps.
It remains to show that the iteration also recovered the Fourier coefficients associated the ID irreps. This is because Poi is in the TD of @ P1 and, for even, Po2, P03 are in the TD of Pl & P3-11 Indeed,
(Kpreprl Xpai
Xprop ( (a')
4 cos(w))2
2n 1=1
+ cosooosuu (2w)) = 1.
20

---

Moreover, for n even and p E (Pu2:Pos) we have
(2 0p Xp) xp)=
1
22
X Snopp- (4))p/-
2n
Wi 2 -2)) (-1)
cos
m
1.
In conclusion, the procedure of Algorithm recovers all the Fourier coefficients and the selective G-Bispectrum.
E.2 The Clebsch-Gordan matrices on DT,
The matrix algebra properties that we use in this subsection can be found, e.g., in [1]. Recall from Theorem A.12 the (implicit) definition of the Clebsch- matrices:
(pl @ (g) Cpr.p
D(T)) - p(g) pER
Ct PL.
(19)
where Ct Pp = I. We only consider the case of Pt P2 both 2d irreps of Dn since otherwise, the Clebsch-Gordan matrix is the scalar 1. First notice that (pı @ (g), is orthonormal matrix. Indeed, using the properties of the Kronecker product, we obtain ("(g)" omitted for clarity):
(p1 @ p2) @ p2) = (p) & p2) @ p!)
(pIPl) @ (p2ph)
=181=1
For Q real orthogonal matrix (QTQ= 1). and VSVT with VTV = J, real Schur decomposition of Q, known that is block diagonal with blocks of size themselves orthogonal matrices. Therefore, the real Schur decomposition is the decomposition in (19) up to permutations. In order for S to represent exactly the irreps from (13), the non-zero sub-diagonal elements should all be positive. If not, the symmetric element is positive and permutation P must be added to exchange their positions: Q = (VP) (V)PP))) P permutes the two columns of V associated with the permuted 2 block of S.
2 x 2. These blocks are
Algorithm 5 Compute Clebsch-Gordan matrices on Dn
1: Input: Pl, P2, two 2d irreps of D,.-
2: Pick any g Dn, e.g., a.
3: Compute (p1 @ p2)
(PP) SP)(VP) valid real Schur form.
4: Set Cpr.e VP.
5: Set + DpER p(g) SP
6: Return: Cpr.pe ppER
F Bispectrum inversion for octahedral and full octahedral groups
We provide sketch of the procedure retrieve F(O) given ((O) for the octahedral group and the full octahedral group. These two groups are available in the escmn library. These groups are easier to deal with than the cyclic and dihedral groups presented in the paper, given that they do not come from a family of groups. Indeed, our proofs for the cyclic (resp. dihedral) groups needed to work for all cyclic groups Cx, and for all dihedral groups Dx. for all N. The octahedral and full octahedral groups are only two groups.
21

---

Po
P1
P2
P3
P4
Po
10000
01000
00100
00010
00001
P1
01.00
11110
01111
01100
00100
P2
00100
01111
11110
01100
01000
P3
00010
01100
01100
10011
00010
P4
00001
00100
01000
00010
10000
Table 4: Kronecker table of the octahedral group using escnn. For the binary word at position i. in the table, the kth 'letter' if Pk E ps @ py. 0 otherwise.
F.1 Octahedral group
The octahedral group has 24 elements and 5 irreps. We can compute its Kronecker table, either manually using characters X or using a Python package such as escnn. We give its Kronecker table below (Table 4), where each column/row represents one irre,, labelled Pos P1....: P4-
We apply the procedure from Algorithms 2, 3 and 4 to Table 4. This procedure relies on the use of Theorem 2.3. We first select the bispectral coefficient 3 3roupo (we omit "(0)" for clarity) to get the component Foo where Pa is the trivial representation. Next, we choose 3 Spo.p: and use Fo, to obtain Fo, (we know from Table that P1 E po @ p1) up to indeterminacy which is transformation in ((3) and corresponds to the indeterminacy factor from Appendix B. Then, we select BpT.Pt to get the Fourier components Fpz: Lastly, we select 3pI.p2 get the missing Fourier component F
In summary, we only need 4 bispectral coefficients (3po Pos Spi.no 3o, PI1: Ppr.02) instead of 5 = 25 in order get the five Fourier components, i.e., the full Fourier transform of the signal. In total, this involves 1 + 9 + 81 + 81 172 scalar coefficients.
F22 Full octahedral group
1000030000
D100000500
0010000000
0C0100300)
0030000000
00000
000000
00000001000
C000050010
0000000001
TEL DM02200
O1TL.0000
0000002000
001000000000
1200011100
0000000 11
BET 1000200
10010000000
(100000000
000000001
(0030/11 10
DICCROT DEM)
(0000
D0000000
0100000000
1801100000
000 100000
500050001C
0O0000I
0000000
00001000000
000 00000000
0000000000
0501000000
1000000000
00000000001
0000001000
C000000000
0900010000
PG
P8
P7
1O00000000
100000
1001
20
05000300001
Table 5: Kronecker table of full octahedral group using escnn. For the binary word at position i, in the table, the kth 'letter' is 1 if Pk p; Pj. 0 otherwise.
The full octahedral group has 48 elements and 10 irreps. Again, we can compute its Kronecker table using Python package such as escnn. We give its Kronecker table below (Table 5), where each column/row represents one irrep, labelled Pu. P1.
We apply the procedure from Algorithms 2, 3 and 4 to Table 5. Again, this procedure relies on the use of Theorem 2.3. Spo.po (we omit "(@)" for elarity) allows to compute F,p, directly, such in Algorithm 4. Then, from 3 po..po we obtain Fp,. up an unknown group action in O(3). Then, using Bpn.po and Fpc. we obtain Fp; Fpr and F Next, leveraging 3p, Fp, and Fp we obtain Fp.- Using 3 Fpr and Fps. we obtain F F Fp..Foo. Finally, with Bp. Fp, and Fpr. we obtain the last coefficient Fpg. Hence we have recovered all the Fourier coefficients using only Bpo.po 8po-pe Bre.po Bpr.pa: Bpp.0o 3pr.07. thus a total of 6 bispectral coefficients instead of 100. In terms of scalar values, this involves
+ 81 + 81 81 334 coefficients.
G Training of the G-CNN architecture
The -NIIS/EINI//MMNIIST datasets are obtained after applying random planar rotations on each image the datasets MNIST [23], EMNIST [5] respectively. In the case of O())-NNIST/EMMIIT, in addition planar rotation, reflection is applied with probability 7. The original size of each image conserved. The size of the training sets are 60 000 and 800 for MNIST and EMNIST, respectively.
22

---

We conserve the architecture of [28]. For all invariant layers, being the G-TC, the selective G- Bispectrum and the Avg/Max G-pooling, the architecture is composed Cz/Dg convollooolll block with K filters (see Table 2). Then, the invariant layer is applied before feeding the output to a MLP. The MI.P is composed of 3 fully-connected layers with ReLU non-linearity. A final fully-connected linear layer is applied for classification. The vector of the output sizes of these layers given by [ol, o2, o3, ol] respectively. o2 o3 = 64 and is equal to the number of classes of the dataset. ol is tuned to reach the parameter count from Table 2.
23

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The claims that we make in the abstract, i.e., theoretical and practical guaran- tees of the G-Bispectrum are respectively proven and studied in Sections 4 and 5.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: In Section 5, we discuss the pros and cons of the selective G-Bispectrum compared to the other possible invariant layers.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
24

---

Answer: [Yes]
Justification: The previously known theorems and results are referenced and each new theorem accompanied by a proof, usually proposed in the appendices.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: We provide all the details about the architectures of the G-CNNs that use in Section 5 and Appendix G. The code will be made available upon publication, with precise guidance reproduce the results.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
25

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
The code will only be made available after publication preserve the anony- mous component of the review process. This code will include precise guidance to reproduce the results.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines huttps://nips.. cc/ lttii//gguidee/ddeeeuuubbe wttpuuuuiissiiioossoiioiieiaae onPolicy) for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
/ //n https/cccppblccciccccceeeeo httpc/ggguuieeeeeeeeebbieeeeeeee (CodeSubmisse wwwssnnnnplcyyl.... for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide in Section the details about the datasets and the architectures of the models implemented for experiments. The optimization details and pypee-pprameters choice will be made available with the code upon publication.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: All the experiments. i.c., Table and Figures 4 and 5 represent the standard deviations of the experiments performed.
Guidelines:
The answer NA means that the paper does not include experiments.
26

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results are out range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We give in section 5 the rumning times of the algorithms and the hardware used for training.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: / /ttp///eerrpp.ee/ee//eee//e/eeee peppblic/Ethissuuidddee
Answer: [Yes]
Justification: This paper is not involved with specific ethical issues, except that it contributes to the field of image processing. The latter field is related to the ethical issue of face- recognition and automatic surveillance.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
27

---

This work is research work to propose new layer in the architecture of G-CNNS. It has no direct societal impacts beyond the fact that it contributes image- processing, which can be used for malicious purposes.
Justification:
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g.. pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification:
This paper does not leverage models with high risk of misuse.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We use the datasets MNIST [8] and EMNIST [5], which we properly mention in the main text. The URLs are displayed with hyperereferences.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
28

---

The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g.. website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswi wwwodee com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's ' creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We do not use new assets and rely on existing datasets.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the datasct/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: No participants were required during the experiments.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: No human subjects necessary prepare this manuscript.
Guidelines:
29

---

The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
30


Reviews:
[{"review_id": "iHisEubDCU", "reviewer": "Reviewer_P7yz", "summary": "The paper proposes a new invariant layer for group convolutional neural networks (G-CNN). The proposed layer computes the spectral coefficients of the input data, that is based on higher-order spectral analysis: the bispectrum, which is the Fourier transform of the triple correlation (G-TC). While there can be up to |Irrep|^2 different coefficients in the bispectrum, the main contribution of the work is an algorithm that finds a subset of at most |G| of these coefficients. For specific groups (general abelian, dihedral, (full) octahedral), the authors proved that there exists a subset of size at most |G| that can reconstruct the input data (signal) perfectly, and that their algorithm, with the right choice of \\rho_1, can find this set. Experiments are conducted on MNIST and EMNIST, comparing the newly proposed layer with max/sum pooling and G-TC layer.", "questions": "See weakness section.", "limitations": "The authors have acknowledged some of the limitations in their methods and definitions (detailed in Weakness section). There are no potential negative societal impact that are specific to this work and are worth highlighting.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- Novelty. The use of higher-order spectral analysis is relatively new in designing deep learning architecture over group and can be powerful since they are originally used to capture nonlinear/autocorrelational structure of the input signal. Capturing nonlinearity is nontrivial in designing G-CNNs as pointwise nonlinearity is known to not be the most appropriate choice of nonlinearity.\n- Theoretical contributions, especially to nonabelian groups such as Dn and O are interesting and has substantial savings in the number of necessary coefficients to be efficient in practice. I did not go through the proofs line by line but believe that the proof idea is sound.\n- Self-contained. The paper, together with the appendix is relatively self-contained in introducing group-theoretic and representation-theoretic concepts. The overall exposition/English is also easy to follow.\n- Clear experiments and diagrams. The paper demonstrate a regime of small number of filters where their methods work much better than simple group pooling, which is a clear and instructive takeaway. Diagrams to demonstrate algorithms are also clear and make for a quicker read.", "weaknesses": "The main issues I have with the current manuscript is its motivation and some ill-defined notion.\n\nIssues with ill-defined notions:\n- The term \"selective G-bispectrum\" is not well-defined, despite being the central notion of the whole paper. The authors defined this notion by giving an algorithm that depends crucially on a hyperparameter \\rho_1. The authors themselves have noted that the choice of \\rho_1 will make or break the resulting set of coefficient (being a complete system). Thus, completeness is a property of the algorithm + choice of \\rho_1 and it does not make sense to claim, for e.g., that the selective G-bispectrum is complete without specifying this choice of irrep. \n- Algorithm 1 depends crucially on computing the Clebsch–Gordan matrices, which may be very costly in general for nonabelian groups. The authors have also acknowledged this point.\n\nIssues with motivations:\n- The authors motivate the paper by arguing that a complete (Definition in Theorem 2.4) invariance layer is more desirable than simple G-pooling layer (such as max or sum), which is an intuitive point. However, in the experiments, average-pooling, which is arguably more complete than max-pooling (or, to quote, less \"excessively invariant\") performs consistently worse than anything else, and to a large extent. Is there a good explanation for this? \n- The proofs of all completeness theorems essentially boils down to being able to reconstruct the group FT coefficients of the input signal from the subset of bispectrum coefficient, which begs the questions of what happens if you simply input these |Irrep| numbers of group FT coefficients as an invariant layer, instead of going to the bispectrum and hoping that it can reconstruct group FT coefficients? The answer to this probably lies in the reference \"The work of Kakarala (1992) illuminated the relevance of the G-Bispectrum for invariant theory, as it is the lowest-degree spectral invariant that is complete (Sturmfels, 2008)\" but scheming through Sturmfels' book did not lead me to any such conclusion. Can the authors provide a more specific reference, and/or discuss this point (why bispectrum but not just FT coefficients, in light of completeness) a bit more? This would be an interesting experiments to look at too. \n- Lastly, why spectral? Usually, signal processing literature makes use of spectral domain because of sparsity in that domain. The authors have also shown that it is faster to compute G-bispectrum coefficients than G-TC. These motivations (and perhaps others, if presence) to using spectral domain should be written up more clearly in the beginning of the paper. \n\nThere are also various typos (e.g. line 26, R not defined in line 493, ...) that I did not carefully keep track of in the interest of time.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer P7yz, we wish to thank you again for your time. In what follows, we address the points you made in your review.\n\n# Issues with ill-defined notions:\n\n## The term \"Selective G-Bispectrum\"\n\nYou are right to mention that the \"Selective G-bispectrum\" admits degrees of freedom, that is, it is not uniquely defined and there are several \"Selective G-bispectra\". As you correctly pointed out, each choice of $\\rho_1$ will yield a different \"Selective G-bispectrum\". \n\nIn the revised version of the paper, we propose to clarify this point as follows. For the most common groups appearing in our theorems, we call \"Selective G-bispectrum\" _the_ bispectrum proposed in their respective proofs. Indeed, each theorem provides a precise set of G-bispectral coefficients. The Selective G-bispectrum will refer to that set. For other groups, we call Selective G-bispectra the G-bispectra obtained via our algorithmic procedure that yield the maximum number of G-Fourier coefficients.\n\nWe thank you for this important remark on the main term of our paper. We will make sure to clarify this in the final version of our work.\n\n## Computing the Clebsch-Gordan matrices\n\nYou are correct mentioning that obtaining the Clebsch-Gordan (CG) matrices is not an easy task. However, this work needs to be done only once before the training. This is because these matrices are not data-dependent and can be stored. In consequence, obtaining the CG matrices is not a bottleneck.\n\nAdditionally, we point out that - for the octahedral and the full octahedral groups - we only use the CG matrices in the proofs of our theorems. Specifically, we only use them to find out the minimal set of G-bispectral coefficients that is needed. In practice, when we implement the selective G-bispectra for these two groups, we do not even use the CG matrices. This is because the G-bispectral coefficients can be written in the following alternative form:\n\n$$\\beta(\\Theta)\\_{\\rho_1,\\rho_2} = \\sum\\_{g_1 \\in G} \\sum_\\{g_2 \\in G} \\sum\\_{g \\in G} \\Theta^*(g)  \\Theta\\left(g g_1\\right) \\Theta(g g_2)[ \\rho_1(g_1)^{\\dagger} \\otimes \\rho_2(g_2)^{\\dagger}].$$\n\nThis expression comes from the definition of the G-bispectrum as the G-Fourier transform of the G-triple correlation.\n\nSince computational gain is an important part of our contribution, we feel that it is important that we discuss and clarify this important point in the final version of the paper. We thank you for this feedback.\n  \n# Issues with motivations:\n\n## Completeness of avg pooling versus max pooling\n\nWe respectfully think there might me a confusion with the notion of completeness. You are right to notice that average pooling uses all data to be computed, however so does the max pooling since all data must be compared to retrieve the maximum. In consequence, the average of a list is not more complete than the maximum, one does not give more information about the distribution of a list than the other.\n\n## Distinction between the G-bispectrum and the G-Fourier transform\n\nWe are glad to precise the interest of the G-bispectrum w.r.t the Fourier transform (FT). The advantage of the G-bispectrum is its invariance to group action while the FT is not invariant: the FT is only equivariant. To achieve group-invariance of a neural network, the FT is thus not a interesting quantity, at least it does not provide specific improvement towards this goal. \n\nKakarala (1992) proved the invariance of the $G$-Triple Correlation (G-TC) for any compact group $G$, and by extension, the invariance of the $G$-bispectrum. This line of research also proves that the G-TC is the unique, lowest-degree polynomial invariant map that is also complete. Consequently, to achieve completeness, the simplest approach is the use of the G-TC or of its Fourier transform: the G-bispectrum. This comment raises the follow-up question: why using the G-Bispectrum over the G-TC? This question is linked to your question \"why a spectral approach?\", which we answer below.\n\n## Why a spectral approach?\n\nThis is a very interesting question. Our motivation for proposing, and using, the \"Selective G-bispectrum\" is motivated by the $O(n\\log(n))$ complexity that it allows to obtain. By comparison, the complexity of the G-TC is $O(n^3)$ and we were not able to design a \"selective G-TC\" to reduce it. Consequently, the selective G-bispectrum is the unique, lowest degree polynomial invariant map that is also complete and that is the most computationally efficient. We will insist more on this point in the final version of the paper.\n\n# Conclusion\n\nAgain, we thank you for the attention given to our work and for your thorough review. We have seeked to address your concern and comments with this rebuttal. If the elements provided above enhance your appreciation for our work, may we kindly ask whether you would consider defending our paper and increasing your score?", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dfe6uLyAaC", "reviewer": "Reviewer_UtSC", "summary": "The work focuses on the problem of achieving invariance to extraneous nuisance variables in signal processing and deep learning by utilizing group actions such as rotations and translations.\n\nThe focus is on the G-Bispectrum, which is a tool used to extract signal characteristics that are invariant to such actions. This tool provides advantages similar to pooling techniques, but with more selectivity and robustness.\n\nThe G-Bispectrum has been limited in its application because to its high computational cost, which is O(|G|2). However, this study presents a strategy to lower this complexity to O(|G|) by proposing selective G-Bispectrum.\n\nThe novel methodology preserves the mathematical characteristics of the G-Bispectrum and enhances the precision and robustness of neural networks, while simultaneously delivering substantial speed ups.", "questions": "The proposed approach is currently limited to discrete groupings. Various contemporary equivariant networks have been utilized for continuous groups like $SO(3)$ and $SE(3)$. Could you provide an analysis of the constraints associated with applying your approach to these continuous categories? Do you anticipate any theoretical or practical obstacles, and do you view this as a promising avenue for future investigation?\n\nRecommendation: It is advisable to evaluate the suggested technique on image datasets of greater magnitude in order to demonstrate its efficacy. To better showcase the benefits of your technique, it is advisable to conduct trials using established benchmarks or more intricate datasets.\n\nQuestion: What is the difference between the suggested selective G-Bispectrum and contemporary equivariant networks tailored for continuous groups? Are there any particular benefits or drawbacks that might be emphasized in the context of ongoing group actions?", "limitations": "NA", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The research presents a clear motivation, which is to decrease the computational complexity of the G-Bispectrum from O(|G|2) to O(|G|) without compromising its effectiveness as a computational primitive for G-invariance. The emphasis on enhancing computing efficiency while maintaining performance is clearly expressed and pertinent.\n\nThe proposed method is technically valid. The assumptions employed are rational, and the research establishes a robust theoretical basis for the selected G-Bispectrum, effectively showcasing its mathematical qualities such as G-invariance, completeness, and uniqueness.\n\n The paper is very well-written and easy to follow. Figure 1 enhances the clarity of the presentation by providing a comprehensive overview of the motivation for the proposed strategy and successfully conveying the essential themes.\n\nThe research presents extensive empirical evidence showing that the selective G-Bispectrum achieves significant improvements in speed compared to the whole G-Bispectrum in real-world scenarios. The enhancement in efficiency renders the selected G-Bispectrum a feasible and efficient substitute for attaining G-invariance in deep learning models.", "weaknesses": "The proposed approach is currently limited to discrete groups. Several contemporary equivariant networks have been applied in domains that involve continuous groups like $SO(3)$ and $SE(3)$. However, the current technique does not readily adapt to these situations, and the paper does not investigate or experiment with such extensions in these domains. Could the authors comment on this limitation and discuss whether it could be a potential direction for future research?\n\nFurthermore, the tests conducted in this research are restricted to small-scale datasets such as MNIST. There is the possibility of analyzing larger and more intricate datasets in order to more effectively showcase the advantages of the suggested approach. I strongly urge the authors to contemplate using their methodology on larger-scale image datasets in order to demonstrate its efficacy in a more thorough manner.\n\n**Minor**\n\nIn line 183, novel theorems, Although the coefficients may not be precisely zero, they could be in close proximity to zero. When performing numerical computations, this can result in problems related to stability and precision.\n\nThe formula presented in Theorem 4.4, which states that the number of bispectral matrix coefficients is equal to ⌊n-1/2⌋ + 2, may benefit from a more comprehensive explanation for readers who are less acquainted with dihedral groups.\n\nThe sentence in Theorem 4.4, \"This corresponds to 1 + 4 + 16 · ⌊n-1/2⌋ ≈ 4|Dn| scalar values,\" may be clarified by providing a concise explanation of the derivation of this approximation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer UtSC,\n\nWe thank you for your review and insightful comments and suggestions. We address your points below.\n\n## Continuous Groups\n\nWe thank you for asking the very interesting question of how to generalize to continuous groups.\n\nThe formulas defining the full and selective G-Bispectra (G-Bsp) only depend on the G-Fourier transform. Since the G-Fourier transform can be defined for continuous groups, so can the G-Bsp. However, practical difficulties arise upon implementation. \n\nIndeed, the full G-Bsp has one G-bispectral coefficient per irreps of the group G. While discrete groups have a discrete number of irreps, continuous groups can have an infinite number of them. For example, SO(3) has an infinite number of irreps. Accordingly, because of memory constraints, we cannot compute the _full_ G-Bsp for SO(3), nor for other continuous groups with an infinite number of irreps. \n\nInterestingly, the _selective_ G-Bsp that we propose does not require all of the G-bispectral coefficients. Thus, it does not require to use all the irreps of the group G. As a result, it is possible that we could compute the _selective_ G-Bsp for continuous groups.\n\nLet us, therefore, assume that the selective G-Bsponly requires a finite number of irreps for a given continuous group G. In this case, we face another obstacle. This obstacle appears when we want to use this G-Bsp as the pooling mechanism of a G-convolutional neural net - as done in our classification experiments. Indeed, the G-Convolution that preceeds the (selective) G-Bsp is a so-called regular G-convolution (Group Equivariant Convolutional Networks. Cohen & Welling, 2016), which uses a discretization of the group G. Thus, the output $\\Theta$ of the G-convolution would be a signal over a discrete/discretized group. Consequently, we would not be able to use the continuous selective G-Bsp there.\n\nTo overcome this obstacle, one could think of using a steerable G-Convolution, instead of a regular one, since steerable G-convolutions apply to continuous groups (Steerable CNNs. Cohen & Welling, 2017). In our paper, we chose to use regular G-Convolutions because: (i) we are in fact interested in discrete groups and, (ii) a regular G-convolution outputs a signal $\\Theta$ defined over a group - much needed to apply the definition of the G-Bispectrum - whereas a steerable G-Convolution outputs a more complicated mathematical object for which the G-Bispectrum needs to be generalized. We discuss this aspect below.\n\nConsider a group $G$ that is the semi-direct product $G=\\mathbb{Z}_2 \\ltimes H$ of the group of translations $\\mathbb{Z}_2$ and a group $H$ of transformations that fixes the origin $0 \\in \\mathbb{Z}_2$. Consider the output of the steerable G-Convolution, which is a map $\\Theta: \\mathbb{Z}_2 \\rightarrow \\mathbb{R}^K$, where $K$ is the number of filters in the convolution. The signal $\\Theta$ has the following mathematical property: $\\Theta$ is a $K$-dimensional field over the grid $\\mathbb{Z}_2$ that transforms according to a specific representation $\\pi$ of the group $G$. This representation is induced by a representation $\\rho$ of its subgroup $H$ on $R^K$ (Steerable CNNs. Cohen \\& Welling, 2017), such that it is common to write $\\pi=\\mathrm{Ind}\\_H^G \\rho$. Consequently, the interaction between the signal $\\Theta$ and the group $G$ is very different for the steerable case, compared to the regular case.\n\n- For a signal $\\Theta$, the action of an element $(t, r) \\in G = \\mathbb{Z}^2 \\ltimes H$ is:\n$[\\pi\\_{t, r} \\Theta] (x) = \\rho(r)\\Theta((t, r)^{-1} x) \\in \\mathbb{R}^K.$\n\n- By contrast, when we use regular G-convolution, the action of an element $g \\in G$ on each coordinate of the signal $\\Theta$ is:\n$[g \\ast \\Theta] (g') = \\Theta((g^{-1} g) \\in \\mathbb{R}.$\n\nThe interaction between the signal $\\Theta$ and the group $G$ is central to the definition of a G-Bsp. In fact, there is currently no definition of a G-bispectrum for the general steerable case. However, there is a definition of G-Bsp for signals defined over the homogeneous space of a group $G$ by Kakarala, which corresponds to a special steerable case for which $\\rho$ is the trivial representation: $\\rho(r) = r$ for all $r \\in H$.\n\nTherefore, generalizing to continuous groups involves i) determining whether the selective G-Bsp only requires a finite number of coefficients, and for the classification experiments: ii) using a steerable G-convolution with trivial $\\rho$ together with the G-Bsp for homogeneous spaces.\n\nThese ideas could provide fruitful paths for exploration in future work. We believe, nonetheless, that presenting the theory for discrete groups is an important first step and should remain the focus on our work. Relatedly, in geometric deep learning, the paper Group-Equivariant Convolution Networks (Cohen & Welling, 2016) introduced the G-CNN for discrete groups. Only later did the paper Steerable CNNs (Cohen & Welling 2017) present a theory that applies to continuous groups.\n\nWe will gladly add this discussion to the final version of the paper, thank you for this suggestion.\n\n## Larger datasets\n\nThank you for raising this point. We discuss it in the global answer to all reviewers.\n\n# Minor\n- You are right to mention that numerical zeros are a recurrent issue in the field of numerical computations, especially when used for division. However, the G-Bsp relies on additions and multiplications. Hence, we did not encounter such problems in our experiments.\n- Thank you for raising this point about the number of scalar coefficients. In the revised paper, we add a paragraph clarifying the number of bispectral matrices and the related number of scalar coefficients: see our answer to reviewer Wppf.\n  \nSpecifically, the approximation you mention comes from the fact that $|D_n |=2n$. Hence, $16\\frac{n-1}{2} \\approx 4|D_n|$.\n# Conclusion\n\nAgain, we thank you for your time and attention. We hope that we have addressed your concerns. We look forward to further discussing these topics.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rfLDKNQQpU", "reviewer": "Reviewer_ZVp5", "summary": "The paper addresses the problem of achieving invariance to nuisance factors in signal processing and deep learning, particularly those describable by group actions (e.g., rotations, translations). The authors propose the selective G-Bispectrum, a computationally efficient variant of the G-Bispectrum, which reduces complexity from $O(|G|^2)$ to $O(|G|)$ in space and $O(|G| \\log |G|)$ in time when an FFT is available on the group $G$.", "questions": "**Question:** It is mentioned in the paper that the selective G-Bispectrum offers greater selectivity and robustness. Could you provide experimental evidence to validate these claims?  \n**Suggestion:** Include experiments and analyses specifically designed to demonstrate the greater selectivity and robustness of the selective G-Bispectrum. This could involve:\n- **Selectivity:** Experiments that show how well the method distinguishes between different classes or handles variations within the same class.\n- **Robustness:** Tests under various noise conditions, transformations, or adversarial attacks to demonstrate the method's stability and reliability.\n\n\n**Question:** Why were only MNIST and EMNIST datasets used for the evaluation? Would the selective G-Bispectrum perform similarly on more complex datasets?  \n**Suggestion:** Expand the experimental section to include more diverse and challenging datasets, such as CIFAR-10, ModelNet10. This will provide a more comprehensive assessment of the method's effectiveness.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The paper presents a novel approach to reducing the computational complexity of the G-Bispectrum, a key operation in achieving G-invariance in signal processing and deep learning. This reduction is significant, lowering the complexity from \\(O(|G|^2)\\) to \\(O(|G|)\\) in space and from \\(O(|G|^2)\\) to \\(O(|G|\\log|G|)\\) in time, assuming the availability of an FFT on \\(G\\). The selective G-Bispectrum is a creative combination of existing ideas in group theory and signal processing, applied in a new way to enhance deep learning architectures. This work stands out by addressing computational limitations that have hindered the widespread adoption of the G-Bispectrum.\n\nThe paper is well-structured and thorough in its theoretical contributions. The authors provide rigorous proofs of the mathematical properties of the selective G-Bispectrum.\n\nThe significance of this work lies in its potential to advance the field of geometric deep learning and signal processing on groups. By reducing the computational complexity of the G-Bispectrum, the authors make this tool more practical for real-world applications.", "weaknesses": "The paper primarily evaluates the proposed selective G-Bispectrum on the MNIST and EMNIST datasets. While these datasets are standard benchmarks in machine learning, they are relatively simple and may not fully stress-test the proposed method’s effectiveness in more complex and varied scenarios, especially larger group such as 3D group.\n\nAlthough the paper claims that the selective G-Bispectrum has higher computational efficiency, it does not explicitly compare the computational costs and performance together in the experiments. This omission undermines the argument for computational efficiency, particularly given that the performance of the selective G-Bispectrum is not as strong as the G-TC method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer ZVp5,\n\nWe thank you for your time and thoughtful review. We address your comments and suggestions below.\n\n### Experiments on larger datasets\n\nThank you for raising this point. We comment on it in our global response to all reviewers. \n\n### Showing accuracy versus computational cost\n\nWe agree that showing accuracy versus computational cost would make our points much stronger. We have added a scatter plot quantifying this trade-off for the different pooling methods. See global response to all reviewers.\n\n### Showing \"selectivity\"\n\nIndeed, we mention that the G-Bispectrum offers greater selectivity in our sentence: \n \"the $G$-Bispectrum has been incorporated into deep neural network architectures as a computational primitive for $G$-invariance - akin to a pooling mechanism, but with greater selectivity and robustness.\" \n \nIn this sentence, we use the word \"selectivity\" to refer to the completeness property of the G-Bispectrum. The G-Bispectrum of a signal is complete in the sense that it removes only the variation due to the action of a group G on the signal, while preserving all information about the signal's structure: it _selectively_ removes the group action without destroying the representation. We have provided empirical evidence to validate this claim in our results presented in Figure 6 and the new experiment from the PDF attached.\n\nIn contrast, we believe that you understood the word \"selectivity\" as it is commonly used in machine learning, where it refers to \"specificity\", the true negative rate (TNR) of a classification task. We apologize for our poor wording that led to this confusion. \n\nTo avoid misunderstanding, we propose to rephrase our original sentence as follows:\n\"the $G$-Bispectrum has been incorporated into deep neural network architectures as a computational primitive for $G$-invariance - akin to a pooling mechanism, but with greater selectivity and robustness.\"\n\nSince we do not claim greater \"specificity\" in the classification sense, we have not provided results on this front. However, we can show the confusion matrices of our classification results in supplementary materials. We hope that we have addressed this concern. Let us know if we should provide additional details here.\n\n### Showing robustness\n\nThank you very much for this suggestion. We have added an experiment in our global answer to all reviewers accordingly.\n\n### Conclusion\n\nAgain, we thank you for your time and attention. We hope that we have addressed your concerns and we look forward to discussing these topics further.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MxKfstGpvL", "reviewer": "Reviewer_Wppf", "summary": "The paper proposes a novel invariant layer for group-convolution networks (GCNNs) based on the \"selective bi-spectrum\".\nWith respect to the expensive bispectrum which has $O(|G|^2)$ coefficients, the selective bispectrum only contains a subset of $O(|G|)$ coefficients which still ensure the completeness of the invariant features and can be computed in $O(|G| log |G|)$ when a FFT algorithm is available.\nSome preliminary experiments show the proposed method has comparable runtime and performance to the typically used max-pooling operator.", "questions": "Fig. 6: this seems just a qualitative results useful to demonstrate the robustness of the selective bispectrum, but a proper quantitative validation of the robustness is still missing. A more comprehensive and thorough comparison between selective bispectrum, normal bispectrum and, maybe, max-pooling seems necessary. Why not reporting some statistics aggregated over a much larger dataset of random signals rather than visualising just 6 examples? It would be even more convincing to do so using the deep features extracted by a GCNN from a dataset of natural images, to prove the completeness is relevant beyond simple synthetic settings.\n\n\nTheorem 4.5: Is it correct that the (4) G-Bispectral coefficients are matrices? I find the choice of counting the matrices instead of individual scalars to be a bit misleading, as these matrices have very different shapes, making this number not very informative. Why not reporting only the number of scalar coefficients? This is a quantity which is meaningful across different groups and that can be directly compared with the group size |G| (the Fourier transform over the group G always has |G| coefficients, regardless of which finite group we consider).\n\n\nTheorem 2.4: alpha is used but never defined\n\nTable 1: why is K not included inside the big-O notation?\n\n\nLine 242: \"... so that the G-Bispectrum scales worth\". This sentence is a bit unclear.", "limitations": "The authors could discuss practical challenges in an efficient or parallel implementation of the selective bispectrum operator due to the sparsity and irregular shapes of the matrices involved.", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "The manuscript is clearly written and the proposed method is original and interesting.\nThe paper also includes some novel theoretical results, which could be useful for a wider audience.\nThe empirical validation is in its very early stage, but the preliminary results are encouraging.", "weaknesses": "I feel like the empirical evaluation of the method is still too weak for acceptance.\n\nFirst, the authors only experiment with variations of the MNIST dataset: while this is a good dataset for some preliminary experiments to validate an idea is a simple and easily reproducible environment, I don't think it is sufficient to draw significant conclusions, especially about properties such as expressiveness of an operator or completeness of a representation.\nThe authors should consider experimenting with more realistic datasets: for example, previous works typically used medical datasets, which still show rotational symmetries but provide higher resolution images and more challenging classification tasks.\nBecause the contributions of this paper include the theoretical description of the selective bispectrum for some 3D groups, the authors should include some experiments on 3D datasets to validate that too; these datasets could also provide more convincing evidence of the value of the proposed method.\n\nSecond, even if larger datasets were used, the current experimental setting doesn't provide conclusive answers on when one should use max-pooling or bi-spectrum.\nIndeed, the comparison in Fig. 5 is a bit misleading as the accuracy is plotted as a function of the number of filters K, but the computational cost of the two methods is different when varying K.\nInstead, a plot showing the accuracy vs (some proxy for) the computational cost seems a more suitable choice, drawing a more complete picture and quantifying the trade-offs discussed between lines 259-263.\n\n\nOverall, while I am very positive about the proposed idea, I think that deserves a much more thorough empirical validation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer Wppf,\n\nWe thank you very much for your review and for your time. We address your comments and questions below.\n\n### Experiments on larger datasets\n\nWe address this concern in our global response to all reviewers.\n\n### Accuracy versus computational cost\n\nWe agree that showing accuracy versus computational cost is necessary to make our main point much stronger. We have added a scatter plot quantifying this trade-off for the different pooling methods. We describe it in our global reponse to all reviewers. We thank you for the great suggestion.\n\n### Question on evaluation of robustness/completeness\n\nThank you for raising this important point. We appreciate your question, as it helped us realize that the purpose of Figure 6 was not clearly presented. We clarify it here, and also refer you to the additional experiments on robustness/completeness that we discuss in our global response to all reviewers.\n\nFigure 6 is _not_ meant to be a _validation_ of the completeness of the G-Bispectrum, but rather an _illustration_. This is why, as you correctly noted, Figure 6 is qualitative and not quantitative.\n\nOur point here is that Theorem 4.1 (completeness of the selective G-bispectrum on $C_n$) is known to be true since it is proved: all signals that have the same selective $G$-bispectrum are known to be identical up to group action. Hence, numerical experiments can only validate it. In consequence, Fig. 6 is not a statistical validation but an illustrative example to convince the reader. This is why we visualize examples and do not report statistics aggregated over a much larger dataset of random signals.\n\nBesides, to provide a more comprehensive picture of the robustness of the selective $G$-bispectrum, we have added in the \"Answer to all reviewer\" and the PDF attached an experiment where, this time, input image are recovered.\n\nWe hope this re-framing of Figure 6 clarifies its purpose. We will integrate this wording into our revised manuscript.\n\nAdditionally, we provide novel experiments illustrating robustness and completeness in our global response to all reviewers.\n\n### Question on memory output: report the number of scalar coefficients\n\nYes, the G-bispectral (Bsp) coefficients are matrices in general, although they are scalars for commutative groups. We originally chose to report the number of G-bispectral coefficients to report a single value across groups, and also because the size of these matrices does not change the O complexity. Yet, you are absolutely right that it would be very useful (and potentially less misleading) to report the number of scalar coefficients as well. We do so below.\n\nThe number of scalar coefficients in the G-TC is $K * |G| * \\frac{|G| + 1}{2}$. This formula applies to all the groups discussed on our paper: $C_n$, $D_n$ the octahedral group $O_h$ and the full octahdral group $FO_h$.\n\nThe number of scalar coefficients in the Max G-pool is $K$, also the same across the groups $C_n$, $D_n$, $O_h$ and $FO_h$.\n\nThe number of scalar coefficients in the selective G-Bsp varies depending on the group.\n  - $C_n$: $K * |G|$ scalar coefficients,\n  - $D_n$: $\\approx K * 4|G|$ scalar coefficients,\n  - $O_h$: $K * 172$ scalar coefficients ($\\approx K* 7 |G|$),\n  - $FO_h$: $K*334$ scalar coefficients ($\\approx K *7 |G|$).\n\nThese numbers are computed by considering which the G-Bsp coefficients are used in the selective G-Bsp (which is given in our theorems and proofs), and by summing the scalar coefficients of their respective matrices. \n\n### Discussion on practical challenges in parallel implementation\n\nThank you for raising this point. Our main contribution is about computational gains, made possible thanks to new mathematical theorems on group theory. Thus, we agree that it is important to discuss other ways to obtain computational gains. Parallel computations is one of them. Following your suggestion, we discuss this point below. (Notation: Bsp = Bispectrum)\n\nTo further gain in computational efficiency, we could compute the G-Bsp coefficients in parallel. This can be done both for the full G-Bsp and for the selective G-Bsp. \n\nIndeed, for both, we know _which_ G-Bsp coefficients are needed: we need all of them for the full G-Bsp (or, half of them, given the symmetry rho_i, rho_j <> rho_j, rho_i), and a subset of them for the selective G-Bsp where the subset is given by our theorems. Consequently, each G-Bsp coefficient can be computed with a separate process. One would need to account for the fact that each G-Bsp coefficient has a different memory footprint. Indeed, each G-Bsp coefficient is a matrix with a size depending on the nature of the coefficient. \n\nSince this parallelization would speed up both the full G-Bsp and the selective G-Bsp, the selective G-Bsp would still be faster. Lastly, since the matrices defining the G-Bsp coefficients are generally non-sparse, no further computational gain that can be obtained there.\n\n### Notations\n\nThank you for reporting the typos. We have corrected these.\n\n### Conclusion\n\nAgain, we thank you for the thorough review and the very interesting questions. Your feedback has been beneficial for us, and we believe that the additional experiments and discussions significantly strenghten our paper. If you have any remaining questions or comments, we would be happy to discuss further!\n\nWe hope that we have addressed your concerns and we look forward to discussing these topics further.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank the four reviewers for their time and attention. All reviewers found our manuscript very well-written and easy to follow. We find this very encouraging especially given its substantial theoretical components in group theory. The reviewers also appreciated the novelty and depth of our theoretical results, the rigor of our proofs, and their importance for signal processing as a whole. We thank all reviewers for recognizing this contribution.

We have provided responses to each reviewer for unique points made in the review. Here, we highlight a few points that appear across reviews, to avoid redundancy.

### Use of Simple, Controlled Datasets in our experiments

Reviewers ZVp5 and Wppf asked for the rationale behind using simple datasets for our experiments, and asked for experiments on more complex datasets.

We agree that the datasets we use here are relatively simple. We chose to use simple datasets in our analyses because they allow us to precisely control their mathematical structure. We generate our synthetic datasets by applying specific groups' actions to these data exemplars. This approach ensures that we know exactly which groups structure the data, enabling us to formulate theoretical expectations for the application of the selective G-bispectrum layer. 

In contrast, the CIFAR image dataset (mentioned by the reviewers) contains many variations not attributable to group actions, such as lighting changes and general differences in appearance across exemplars within the same category. Therefore, we theoretically expect less benefit from using group-invariant and group-equivariant structures with this dataset. Our goal in this work was to empirically demonstrate the theoretically expected properties of our proposed layer, which we believe we have achieved using (i) strong theoretical contributions with several new theorems, and (ii) simple, controlled datasets.

### Robustness to adversarial attacks

Reviewers ZVp5 and Wppf asked for additional experiments to illustrate the robustness of the bispectrum. We first explain the link between completeness and robustness. Then, we provide new experiments and present their results here.

First, we emphasize that the completeness of the the triple correlation (TC), and its Fourier transform, the bispectrum,  is well-documented in signal processing literature (see Yellott & Iverson (1992) for translation-invariant case, and Kakarala (2009) for compact groups). The selective G-Bispectrum contains the same information as the G-bispectrum and is therefore also complete for the groups we consider. This completeness implies that it is inherently robust against invariance-based adversarial attacks. In such attacks, one finds non-identical inputs (up to group action) that produce identical output representations. Completeness prevents from this event, thereby ensuring robustness to these attacks. 

We provide empirical illustration of this property with a new set of experiments. Specifically, we perform invariance-based adversarial attacks on the models trained in our experiments. Our results are presented in Figure 1 of the PDF attached. They show that the model with selective G-bispectrum is complete and robust to these attacks, whereas the Max G-Pool model is not. Specifically, when we optimize input images to match a target selective G-bispectrum, obtained form a target image, every optimized input image is identical to the target image up to group action. By contrast, an adversarial agent can find several input images that would yield a target output in the context of the Max G-pool model because it is not complete.

This analysis, initially established by Sanborn et al. (ICLR 2023), and later used in Sanborn & Miolane (2023) for the TC, empirically demonstrates the theoretically-expected completeness and robustness property in the trained model.

### Cost-Accuracy Trade-offs

Reviewers ZVp5 and Wppf raised the concern that the trade-off between computational cost and accuracy was not explicit enough, since computational cost and accuracy were on two separate figures in the manuscript. We agree and address this point here. We now plot computational cost and accuracy on the same figure to draw a complete, quantified picture of their trade-offs.

In the PDF attached, we provide a scatter-plot that shows the relation between computational cost (x-axis) and accuracy error (y-axis, log scale). Each dot is colored according to the pooling method, and we additionally vary the number K of filters, with K going from 2 to 20. Since the accuracy is monotone with K, one can additionally read the figure in K by going from top to bottom.

This scatter-plot provides additional evidence of our claim that the selective G-Bispectrum is faster than the G-TC while being more accurate than the Max G-Pooling and the Avg G-pooling. In terms of speed, we see that the selective G-Bispectrum provides an average gain of around 15 seconds over the G-TC, which corresponds to around 10% improvement. In terms of accuracy, we see that the selective G-Bispectrum provides an average gain of 0.02 accuracy over the Max G-pooling, and 0.26 over the Avg G-pooling.

We will include this plot in the main paper, and add corresponding plots for O2 and SO2/O2-EMNIST in the supplementary materials. We believe that they do, indeed, better quantify our claims. We thank the reviewers for the suggestion.

## Conclusion on experiments

We believe that the additional plots and proposed additional experiments on robutness strengthen our manuscript, and we thank the reviewers for their important feedback. Yet, we would like to emphasize that our work also makes several _theoretical_ contributions, and thus should not be reduced to its experiments. We believe that the impact of our reduction of the group-bispectrum is profound, with consequences for applied mathematics, signal processing and machine learning that go beyond classification results on benchmarks.

Meta-review: The paper presents a new invariant layer for discrete group-convolutional networks based on the notion of the selective bispectrum. It tracks an earlier line of work in signal processing involving the notable works of Kakarala and Kondor, followed by the use of these ideas in the context of designing invariant layers in deep learning by Sanborn and Miolane. The advantage of using the triple correlation, and the bispectrum is that it is complete. But an issue with the use of the bispectrum is that it is expensive, involving $O(|G|^2)$ coefficients. The proposed selective bispectrum consists of only a subset i.e. $O(|G|)$ coefficients while still ensuring completeness. Since FFTs for groups of interest are available (or can be designed), this translates to a computation cost of $O(|G| log |G|)$. The paper supports the use of the selective bispectrum for designing invariant layers by some basic experiments which show that the proposed layer is faster than the bispectrum and TC layers (while trading off some robustness), and generally more accurate than a max-pooling layer.

As echoed by the reviewers, the paper is well-written and clearly motivated. More importantly, it does a good job at being self-contained, covering the necessary background on the triple correlation and the bispectrum. It also does a good job at covering existing work. The theoretical results basically involve a simple use of the Fourier transform and as such are straightforward, but are nevertheless novel and could also possibly be of interest to researchers in signal processing. Some of the reviewers were underwhelmed by the experiments. This stems from a valid concern since the method is aiming to be a replacement layer in deep neural networks. It is possible that the benefit from the proposed layer is most prominent in low data regimes. The authors have responded to the reviewers with some additional experiments on CIFAR-10, and providing computational tradeoffs. On considering these factors, I think the paper could still be a valuable contribution at NeurIPS. The paper's potential impact could be improved if it could also include results on a medical dataset (such as those used by Cohen and Winkels). The responses made to the reviewers should be incorporated in the paper for camera ready, inclduing more analysis such as the computation tradeoff. It is important to include computation metrics since it is presented as a selling point. It would also be useful to add a discussion about the use of the selective bispectrum in continuous groups. Note that in some networks on continuous groups, such as the Clebsch-Gordan Networks approach of Kondor, Lin, and Trivedi, an invariant layer is obtained by a projection. No explicit pooling is done. I wonder if the selective bispectrum still has an advantage in such situations. Finally, it would also be useful to spend some space to define the term "selective bispectrum" rather than leaving it implicit.
Decision: Accept (poster)