Title: The Selective $G$-Bispectrum and its Inversion: Applications to $G$-Invariant Networks
Venue: NeurIPS 2024
Accepted: yes
Keywords: CNN

group invariance

bispectrum

Neural Network

AI
Abstract: An important problem in signal processing and deep learning is to achieve *invariance* to nuisance factors not relevant for the task. Since many of these factors are describable as the action of a group $G$ (e.g. rotations, translations, scalings), we want methods to be $G$-invariant. The $G$-Bispectrum extracts every characteristic of a given signal up to group action: for example, the shape of an object in an image, but not its orientation. Consequently, the $G$-Bispectrum has been incorporated into deep neural network architectures as a computational primitive for $G$-invariance\textemdash akin to a pooling mechanism, but with greater selectivity and robustness. However, the computational cost of the $G$-Bispectrum ($\mathcal{O}(|G|^2)$, with $|G|$ the size of the group) has limited its widespread adoption. Here, we show that the $G$-Bispectrum computation contains redundancies that can be reduced into a *selective $G$-Bispectrum* with $\mathcal{O}(|G|)$ complexity. We prove desirable mathematical properties of the selective $G$-Bispectrum and demonstrate how its integration in neural networks enhances accuracy and robustness compared to traditional approaches, while enjoying considerable speeds-up compared to the full $G$-Bispectrum.

The Selective G -Bispectrum and its Inversion: Applications to G -Invariant Networks

Abstract

An important problem in signal processing and deep learning is to achieve invariance to nuisance factors not relevant for the task. Since many of these factors are describable as the action of a group G (e.g., rotations, translations, scalings), we want methods to be G -invariant. The G -Bispectrum extracts every characteristic of a given signal up to group action: for example, the shape of an object in an image, but not its orientation. Consequently, the G -Bispectrum has been incorporated into deep neural network architectures as a computational primitive for G -invariance-akin to a pooling mechanism, but with greater selectivity and robustness. However, the computational cost of the G -Bispectrum ( O ( | G | 2 ) , with | G | the size of the group) has limited its widespread adoption. Here, we show that the G -Bispectrum computation contains redundancies that can be reduced into a selective G -Bispectrum with O ( | G | ) complexity. We prove desirable mathematical properties of the selective G -Bispectrum and demonstrate how its integration in neural networks enhances accuracy and robustness compared to traditional approaches, while enjoying considerable speeds-up compared to the full G -Bispectrum.

1 Introduction

The visual world is rich with symmetries. For example, the identity of an object is invariant to its position in the visual field; vision has translational symmetry . Group theory is the mathematics used to describe transformations, their actions on objects, and the object's symmetry. As such, group theory has penetrated the fields of signal processing and deep learning alike. For example, the Fourier transform, pillar of signal processing, has been adapted to the G -Fourier transform, with its spectrum decomposing a signal defined over a group into several frequencies. More recently, researchers have become interested in the properties of higher-order spectra such as the Bispectrum , and its generalization to signals over groups via the G -Bispectrum.

G -Bispectrum The G -Bispectrum is the Fourier transform of the G -Triple Correlation ( G -TC). Historically, higher-order spectra found initial applications in the context of classical signal processing as generalizations of the two-point autocorrelation [31, 2, 25]. The work of Kakarala [15] illuminated

Johan Mathe

the relevance of the G -Bispectrum for invariant theory, as it is the lowest-degree spectral invariant that is complete. Since then, it has appeared in diverse settings such as vision science [34], machine learning [19, 20], and 3D modeling [18].

Limitations of the G -Bispectrum for Deep Learning The computational complexity of the G -Bispectrum has severely limited the reach of its applications. The most salient example of this limitation is in machine learning and deep learning. Convolutional Neural Network (CNN) [22, 24] reflect and exploit the translational symmetry of the visual world. Group-Equivariant CNNs ( G -CNNs) [6, 21] do just this, with more general group-equivariant convolutions to exploit symmetries like rotational symmetries. In both cases, one typically wants to preserve transformations throughout the layers of a network (i.e., to be groupequivariant ), and remove them only at the end when 'canonicalizing' an image for classification (i.e., to be groupinvariant ). While the theory of equivariant layers has been thoroughly developed [7, 33], less attention has been paid to the theory of invariant layers [12]. This is where the G -Bispectrum enters the picture, and where its computational cost has strongly limited its integration into deep learning.

Commonly, invariance in G -CNNs is achieved by simply taking an average or maximum over the transformation group (Average or Max G -Pooling, respectively). However, as noted by Sanborn & Miolane [28], this is a highly lossy operation removing information about the structure of the signal. While the max operation is indeed invariant (the max of an image is the same as the max of an image rotated by 90 degrees), it is excessively invariant: one could permute all of the pixels in the image and without changing the maximum, but with none of the same structure (see Figure 8). To address this, Sanborn & Miolane [28] used the G -TC as a G -invariant layer that is complete -that is, it removes group transformations with no loss of signal structure. This approach achieves demonstrable gains in accuracy and robustness [28], but it is computationally expensive.

Indeed, the space complexity of the G -TC, i.e, its number of coefficients, scales as O ( | G | 2 ) , where | G | is the size of the group. As each coefficient demands for O ( | G | ) operations, the computational cost or the time complexity of the G -TC is O ( | G | 3 ) . An alternative would be to use the G -Bispectrum as the pooling layer. However, both its space and time complexities are O ( | G | 2 ) . By comparison, the Max G -pooling layer features a O ( | G | ) computational cost and returns a scalar output. This raises the question of whether one can achieve complete invariance and adversarial robustness without sacrificing too much in terms of computational efficiency.

Contributions. In this work, we prove for the first time that we can significantly reduce the computational complexity of the G -Bispectrum. This result has important implications for signal processing and deep learning on groups, for which the G -Bispectrum is a foundational computational primitive. Our contributions are:

We provide a general algorithm that reduces the computational complexity of the G -Bispectrum from O ( | G | 2 ) to O ( | G | ) in space complexity and from O ( | G | 2 ) to O ( | G | log | G | ) in time complexity if an FFT is available on G . We term it the selective G -Bispectrum . The algorithm can be applied to any finite group.

We prove that the selective G -Bispectrum is complete for the most important finite groups used in practice, i.e., all discrete commutative groups, the dihedral groups of any order, the octahedral and full octahedral group. This significantly extends the work of [10, 15, 27], who first showed this for some finite, commutative groups, where it was demonstrated that the G -Bispectrum can be computed with only | G | space complexity.

We use the selective G -Bispectrum to propose a new G -invariant layer that strikes a balance between robustness and efficiency. In particular, it is more expensive than the Max G -pooling, but cheaper than the G -TC pooling.It is also cheaper than the full G -bispectral pooling of O ( | G | 2 ) time and O ( | G | 2 ) space complexity. The selective G -Bispectrum is more robust than the max G -pooling, and almost as robust as the G -TC.

We run extensive experiments on the MNIST [23] and EMNIST [5] datasets to evaluate how each invariant layer (Max G -pooling, G -TC, selective G -Bispectrum) impacts accuracy and speed on classification tasks. We achieve the expected results: Our layer is faster than the G -TC and full G -Bispectrum and more accurate than Max G -pooling.

We present several findings important to the design of invariant layers to guide further advances in the field of geometric deep learning. In particular, we show that the accuracy and speed advantages

of the selective G -Bispectrum is most striking for G -CNNs with low number of convolutional filters. Conversely, increasing the number of filters in the G -Convolutions allows the Max G -Pooling to catch up on the accuracy.This demonstrates that the G -bispectral pooling will be particularly interesting for neural networks operating under a smaller parameter budget.

We hope that the proposed reduction of the G -Bispectrum complexity will further open areas of research in signal processing on groups, that were previously prohibited due to the high complexity of the operation.

2 Background: G -Triple Correlation and G -Bispectrum

The proposed selective G -Bispectrum operation is closely related to two other foundational operations on signals defined on groups: the G -Triple Correlation and the full G -Bispectrum, which we introduce here. The background on group theory, including the definitions of groups, group actions, equivariance and invariance, is presented in Appendix A.

The G -Triple Correlation Given a real signal defined on a finite group Θ : G ↦→ R , the G -Triple Correlation ( G -TC) [15] is the lowest order polynomial that is complete, i.e., that conserves all of the information of the signal Θ , up to group action by G .

Definition 2.1. The G -Triple Correlation of a real signal Θ : G ↦→ R is given by

The original triple-correlation was introduced for the classical framework of translations of a onedimensional signal, i.e., where X = Z and ( G, · ) = ( Z , +) . The G -triple correlation from Definition 2.1 extends the original definition to any finite group ( G, · ) . In our setting, the signal Θ : G ↦→ R will be obtained after the G -convolution of a function f : X ↦→ R c , representing a continuous image with c channels, with a filter ϕ : X ↦→ R c , and the G -TC will be applied channel-by-channel. Importantly, the G -TC layer has computational complexity O ( | G | 3 ) and outputs O ( | G | 2 ) coefficients.

The G -Bispectrum The G -TC operation has a Fourier equivalent: the G -Bispectrum. Indeed, the definition of the Discrete Fourier Transform (DFT) can be extended to any finite group (see, e.g., [9]), as recalled below.

Definition 2.2. Given a set of unitary representatives (Def. A.2) of the equivalence classes of irreps ρ i : G ↦→ GL( V i ) (Def. A.6), the G -Fourier Transform on a finite group G of a signal Θ : G ↦→ R is defined as where ρ i ( g ) † refers to the conjugate transpose of the matrix ρ i ( g ) (or simply transpose if ρ i is real-valued).

The G -Bispectrum β (Θ) is defined as F ( T (Θ)) , with F evaluated over the group G × G . Kakarala [15] proposed a closed-form expression for the G -Bispectrum β (Θ) directly in terms of F (Θ) . We recall it in Theorem 2.3.

Theorem 2.3. [17] The G -Bispectrum of a signal Θ : G ↦→ R , β (Θ) , is given by:

where C ρ 1 ,ρ 2 is a unitary matrix called the Clebsch-Gordan matrix, whose definition is recalled in Appendix A. For each pair ρ 1 , ρ 2 , the matrix β (Θ) ρ 1 ,ρ 2 is called a G -bispectral coefficient.

For commutative groups, Theorem 2.3 simplifies to a more compact expression, recalled in Theorem A.12. However, both the space and time complexity of the G -Bispectrum remain O ( | G | 2 ) .

To the authors' best knowledge, there is no generic analytical formula for computing C ρ 1 ,ρ 2 for an arbitrary group G . However, there exist formulas for specific classes of groups (see Appendix E.2). Additionally, there exist packages for computing these for many groups using packages such as escnn [3].

Figure 1: Illustration of the different proposed G -CNN modules [6, 28]. The input f is first processed through the G -convolutional layer composed of K filters { ϕ k } K k =1 . Then, an invariant layer is chosen (Max G -pooling, G -TC, or the selective/full G -Bispectrum layer). Finally, the 'pooled' output is fed to a neural network designed for the machine learning task at hand.

Complete G -Invariants The G -TC and the G -Bispectrum are desirable computational primitives for signal processing and deep learning because they are complete G -invariants (for generic data Θ , ˜ Θ ). Indeed, this completeness property make them very interesting for building invariant layers in G -CNNs, as they are selectively invariant. We define complete G -invariance next.

Application: G -invariant layers The G -CNN architecture, first proposed in [6], is illustrated in Figure 1. The input signal f : X ↦→ R , typically an image, is processed through a G -Convolution layer using filters { ϕ k } K k =1 . The output is feature maps { Θ k } K k =1 that form a set of K real-valued signals with domain G . This G -Convolution layer is traditionally followed by a G -invariant layer. The most common is the Max G -Pooling layer. More recent works have proposed two alternatives based on the G -TC and the full G -Bispectrum: the G -TC Pooling [28] and the (full) G -Bispectrum [29] respectively, where the latter requires the computations of the Fourier transforms of the feature maps, preferably computed using a Fast Fourier Transform (FFT) algorithm on G [9]. When testing the impact of the choice of G -invariant layer, the output of the invariant layer is typically fed to a Secondary Neural Network (NN) to perform the desired task, e.g., image classification. The Secondary NN often takes the form of a Multi-Layer Perceptron (MLP).

Theorem 2.4. [16, Thm.3.2] The G -TC and the G -Bispectrum are complete G -invariants, i.e., for Θ , ˜ Θ : G ↦→ R with F (Θ) ρ nonsingular for all irreps ρ , T (Θ) = T ( ˜ Θ) , respectively β (Θ) = β ( ˜ Θ) , if and only if there exists h ∈ G such that Θ( g ) = α ( h, ˜ Θ( g )) for all g ∈ G .

Experimental results have demonstrated the superior accuracy and adversarial robustness of the G -CNN equipped with a G -TC and G -Bispectrum invariant layer [29, 28]. However, both methods inherit the high space and time complexity of their respective operations. This raises the question of whether we can reduce this computational complexity.

3 Method: The Selective G -Bispectrum and its Inversion

The Selective G -Bispectrum We introduce a novel tool for signal processing on groups: the selective G -Bispectrum. A selective G -Bispectrum β sel is any O ( | G | ) subset of all coefficients of the G -Bispectrum β (Definition 2.3), only conserving well-chosen pairs of irreps ( ρ 1 , ρ 2 ) . Which pairs of irreps to select depends on the group of interest. This is possible due to redundancies and symmetries in the full object. Below, we provide an algorithmic procedure to compute the selective G -Bispectrum for any finite group G that features at most | Irreps | ( ≤ | G | ) coefficients. The procedure is summarized in Algorithm 1. We have the following proposition.

Proposition 3.1. The selective G -Bispectrum β sel from Algorithm 1 has at most | G | coefficients.

Proof. By construction of Algorithm 1, | L ρ | ≤ | Irreps | . Since | Irreps | is at most the number of conjugacy classes of G (see, e.g., Steinberg [30, Corollary 4.3.10]), we have | Irreps | ≤ | G | .

In Algorithm 1, we note that the choice of ρ 1 is important, since some ρ 1 will not allow the user to recover all of the irreps and therefore not ensure the completeness of the selective. We illustrate the computation of the selective G -Bispectrum in Figure 2, where we choose ˜ ρ 1 = ρ 6 .

Algorithm 1 Selective G -Bispectrum on any finite group G

1: Input : Signal Θ : G ↦→ R with n = | G | . Empty list of coefficients L β . Empty list of irreps L ρ . Kronecker Table of G .

2: Add β (Θ) ρ 0 ,ρ 0 to L β where ρ 0 is the trivial irreps ( ρ 0 ( g ) = 1 for all g ∈ G ).

3: Add ρ 0 to L ρ .

4: Choose ˜ ρ 1 such that ˜ ρ 1 ⊗ ˜ ρ 1 = C ˜ ρ 1 , ˜ ρ 1 ( ⊕ ρ ∈R ρ ) C † ˜ ρ 1 , ˜ ρ 1 generates at least one irreps ρ not yet in L ρ .

6: Add every ρ that appears in ˜ ρ 1 ⊗ ˜ ρ 1 to L ρ .

5: Add β (Θ) ρ 0 , ˜ ρ 1 and β (Θ) ˜ ρ 1 , ˜ ρ 1 to L β , add ˜ ρ 1 to L ρ .

7: while L ρ keeps changing: do

8: Find ρ ′ , ρ ′′ in L ρ such that ρ ′ ⊗ ρ ′′ generates at least one irreps not already in L ρ .

9: Add β (Θ) ρ ′ ,ρ ′′ to L β .

10: Add ρ that appears in ρ ′ ⊗ ρ ′′ to L ρ .

11: end while

12: Return β sel (Θ) := L β : the selected G -bispectral coefficients.

Computation of the Selective Bispectrum for the full octahedral group

Super-imposed on top of its Kronecker Table

Po

P1

P2

P3

P4

P5

P6

P7

P8

p9

PO

1000000000

0100000000

0010000000

0001000000

0000100000

0000010000

0001000000

0000000100

0000000010

I000000000

P1

0100000000

1111000000

0000010

0110000000

0010000000

0000001000)

00

0000001100

0000000100

P2

0010000000

0111100000

1111000000

0110000000

0100000000

0000000100

0000001111

0000011110

0000001100

0000001000

P3

0001000000

0110000000

0110000000

1001100000

0001000000

0000000010

0000001100

0000001100

0000010011

0000000010

P4

0000100000

0010000000

0000000010

0001000000

1000000000

I000000000

0000000100

0000001000

0000000010

0000010000

P5

0000010000

0000001000

0000000100

0000000010

I000000000

1000000000

0000000000

0010000000

0001000000

0000100000

P6

00000100

000001-1110

0000001-111

0000001100

0000000100

0000000T00

0100000000

111000000

000001110

0110000000

0010000000

P7

0000000100

I11000000

0111100000

0000001100

0000001000

0010000000

0111100000

1111000000

0110000000

0100000000

P8

0000000010

0000001100

0000001100

100100000

0000000010

0001000000

0110000000

0110000000

1001100000

0001000000

6d

I000000000

0000000100

0000001000

0000000010

0000010000

0000100000

0010000000

0100000000

0001000000

1000000000

Selective Bispectrum {βpo,po, βp6,Po βp6,Ps βp1,p2, βp1,P6s βp1,p7}

Inverting the Selective G -Bispectrum for completeness The inversion of the selective G -Bispectrum β sel (Θ) is reconstructing a signal F ( ˜ Θ) from the G -Bispectrum coefficients in the list L β such that ˜ Θ = α ( g, Θ) for some g ∈ G (The G -Bispectrum is G -invariant, hence, Θ can only be recovered at best up to group action). Once F ( ˜ Θ) is known, ˜ Θ can be obtained using the Inverse Fourier Transform. If the selective G -Bispectrum can be inverted, then, by definition, it is complete in the sense of Theorem 2.4.

4 Theory: Completeness of the Selective G -Bispectrum

Our main theoretical claim is that the selective G -Bispectrum can be inverted and is a complete G -invariant that drastically reduces the complexity of the G -Bispectrum. We prove this claim for many finite groups G of interest in signal processing and deep learning in a sequence of theorems presented in this section.

Known Theorems Previous authors had looked into the G -Bispectrum inversion problem. It is well known that | G | = n coefficients are enough for the cyclic group ( C n , · ) = ( Z / Z n , + mod n ) .

̸

Theorem 4.1. [17] For cyclic groups C n , n ∈ N 0 , the C n -Bispectrum can be inverted using | G | = n coefficients if F (Θ) ρ = 0 for all irreps ρ of C n .

Similarly for a product of two such groups, we have the following theorem.

Theorem 4.2. [10] For a product of cyclic groups C n × C m , n, m ∈ N 0 , the G -Bispectrum can be inverted using | G | = nm coefficients if F (Θ) ρ = 0 for all ρ ∈ C n × C m .

̸

Full and Selective Bispectra for the group D4 Super-imposed on top of D4's Kronecker Table

Full and Selective Bispectra for the group Oh Super-imposed on top of On's Kronecker Table

Po

P1

P2

P3

P4

Po

P1

P2

P3

P4

Po

10000

01000

00100

00010

00001

Po

10000

01000

00100

00010

10000

P1

01000

10000

00010

00100

00001

P1

01000

11110

01111

01100

00100

P2

00100

00010

10000

01000

00001

P2

00100

01111

11110

01100

01000

P3

00010

00100

01000

10000

00001

P3

00010

01100

01100

10011

00010

P4

00001

00001

00001

00001

11110

P4

00001

00100

01000

00010

10000

Full Bispectrum

{βpi,p;}i,j=0,,4

Full Bispectrum

{βpo,po, βps,po Pps,p4}

{βpo,po βppo, βpp βpP2 }

New Theorems From now on, we assume that the Fourier transform F (Θ) only features non-zero elements , or invertible matrices in the case of non-scalar Fourier coefficients. This assumption is supported by the zero probability of encountering this corner case (an arbitrarily small perturbation of any signal makes this assumption true).

We first extend the above results to all commutative groups. The proof relies on the fact that every finite commutative group is the direct sum of finitely many cyclic groups.

Theorem 4.3. For finite commutative groups G , the G -Bispectrum can be inverted using | G | coefficients if F (Θ) ρ = 0 for all ρ ∈ G .

̸

See Appendix D for the proof and derivation of the inversion for the specific case of commutative groups. We note that our approach to inversion is symbolic, in that a solution can be expressed explicitly as a formula in terms of the input. Other approaches are also possible to determine an inverse, such as using least squares [11] or more recent spectral methods [4].

We now extend the result to dihedral groups. Dihedral groups are ubiquitous in signal processing and deep learning because they represent the group of rotations and reflections.

̸

The proof is provided in Appendix E. We now extend the result to octahedral and full octahedral groups, that are related to the symmetries of the octahedron. These groups are very important in signal processing and deep learning of 3D images.

Theorem 4.4. For any dihedral group D n (symmetries of the n -gon), n ∈ N 0 , we need at most ⌊ n -1 2 ⌋ +2 bispectral matrix coefficients for inversion if det( F (Θ) ρ ) = 0 for all irreps ρ of D n . This corresponds to 1 + 4 + 16 · ⌊ n -1 2 ⌋ ≈ 4 | D n | scalar values.

Theorem 4.5. For the octahedral group O which has | G | = 24 group elements and 5 irreps, we need only 4 G -Bispectral coefficients in the selective G -Bispectrum. this corresponds to 172 scalars. For the full octahedral group FO which has | G | = 48 elements, we only need 6 G -Bispectral coefficients in the selective G -Bispectrum to perform inversion. This corresponds to 334 scalars.

A sketch of proof is provided in Appendix F given the redundancy of the procedure. We see that the selective G -Bispectrum uses only 4 coefficients, compared to 25 coefficients needed for the full G -Bispectrum of the octahedral group. For the full octahedral group, it requires only 6 coefficients compared to the 100 coefficients of the full G -Bispectrum. In Figure 3, we compare the full and selective G -Bispectra of the dihedral group D 4 (symmetries of the square) and the octahedral group.

5 Experimental results

Implementation and architecture Our implementation of the selective G -bispectrum layer is based on the gtc-invariance repository, implementing the G -CNN with G -convolution and G -TC layer [28] and relying itself on the escnn library [3, 32]. The implementations related to this section can be found at the g-invariance repository.

O

|

|

O

We propose an experimental assessment of the newly proposed selective G -Bispectrum layer by comparing it with the Avg G -pooling, the Max G -pooling, the G -TC as invariance operations after the G -convolution of a G -CNN on the classification problems of the MNIST dataset of handwritten digits [23], the EMNIST dataset of handwritten letters [5] with standard train-test division. These datasets count 10 and 26 classes, respectively. We obtain transformed versions of the datasets G -MNIST/EMNIST - by applying a random action g ∈ G on each image in the original dataset.

The objective of our experiments is to isolate the speed-up of the G -Bispectrum layer. Hence, we consider architectures that only differ by the invariant layer in the classification task, following the experimental set up by [28]. The neural network architecture is composed of a G -convolution, a G -invariant layer, and finally a Multi-Layer-Perceptron (MLP), itself composed of three fully connected layers with ReLU nonlinearity. Finally, a fully connected linear layer is added to perform classification. The MLP's widths are tuned to match the number of parameters across each neural network model. The details are given in Appendix G. We highlight here that the pursued objective is to compare the differences in performances of the G -invariant layers, not to provide the state-of-the-art accuracy on the datasets involved. Henceforth, we do not optimize the architectures to reach the highest possible accuracy. We set simple architectures providing interpretable results for analysis. The experiments a performed using 8 cores of a NVIDIA A30 GPU.

Training speed performance Table 1 recalls the theoretical complexities of the different layers. The computational cost of computing the selective G -Bispectrum is O ( | G | log | G | ) if an FFT algorithm is available on G [9], and O ( | G | 2 ) with classical DFT. in Figure 4, we report the average training times on SO(2) / O(2) -MNIST for 10 runs as the discretization C n /D n of SO(2) / O(2) varies. In the first case, we use the FFT and observe that the Max G -pooling and G -Bispectrum training time scale linearly whereas it scales quadratically for the G -TC. For O(2) , we perform a classic DFT on D n so that the G -Bispectrum scales worth. However, an FFT could be implemented to speed-up the process.

Classification Performance We compare the performances of the G -Bispectrum layer with respect to the G -TC, the Max G -pooling and the Avg G -pooling models, trained on the SO(2) / O(2) -MNIST/EMNIST datasets and we assess the accuracy by averaging the validation accuracy over 10 runs. The classification accuracy is provided in Table 2. For the experiments in Table 2, the following pattern holds: at equivalent number of parameters, the more computationally expensive the pooling layer, the better the accuracy. However, the use of the G -TC becomes prohibitive when | G | increases. In the next section, we discuss the settings where each invariant layer should be preferred, and highlight each invariant layer's strengths and weaknesses.

Discussion on the choice of invariant layer The first observation from Table 2 is though the selective G -Bispectrum is complete, the model obtains slightly lower accuracy than G -TC. This observation might be surprising at first, since we prove mathematically in Section 4 that the selective G -Bispectrum is complete, just as the full version. An explanation to this lies in the paradoxes of the Universal Approximation Theorem [13]. Just because an arbitrarily large MLP can theoretically fit any function, this does not imply that it will happen for a practical, limited MLP. In practice, we hypothesize that the redundancy of the G -TC allows the MLP to distinguish inputs more easily. If the size of the model allows it, the G -TC or the full G -Bispectrum will provide better accuracy. However, when the size of the group is big, their use is often out of reach while the selective G -Bispectrum is scalable. In Table 2, we also notice that the Max G -pooling performs well compared to the others even though it is not complete. This is because we have many filters that allow for refined classification. Indeed, assume f, ϕ k are black-and-white images with N pixels. In consequence, max g Θ k ( g ) ∈ { 0 , 1 , ..., N } for k = 1 , 2 , ..., K . The Max G -pooling allows a maximum separation of ( N + 1) K classes. In practice, this value is not reached, but it explains why Max G -pooling performs well. Figure 5 highlights this dependency of the Max G -pooling on the number of filters since the accuracy drops to less than 60% with 2 filters. In comparison, the G -TC and the selective G -Bispectrum, which are complete , keep an accuracy above 85% with 2 filters.

Completeness To conclude our numerical experiments, we study the robustness of the selective G -Bispectrum to adversarial attacks, following the analysis in Sanborn & Miolane [28, Figure 2]. Given an image ̂ f : X ↦→ R c and a filter ϕ : X ↦→ R c , they numerically verified the robustness (=completeness) of the G -TC by showing that

Indeed, Sanborn & Miolane [28, Figure 2] shows that only images that are identical up to rotation/reflection can yield the same C n /D n -TC. That is, the G -CNN with G -TC can not be 'fooled'

since only input in the same orbit yield the same output. We perform a similar experiment in Figure 6. Moreover, it is well-known that the G -convolution ϕ ∗ f is G -equivariant. Hence, an equivalent experiment is to show that

In Figure 7, we show that the selective G -Bispectrum β sel is robust to adversarial attacks by solving

The signals are indeed recovered up to a translation, i.e., a group action of C 30 . Moreover, despite (4) only optimizes using the selective G -Bispectrum, the full G -Bispectrum is correctly recovered. This is an additional evidence of the completeness of the selective G -Bispectrum.

6 Conclusion and Future works

In this paper, we introduced a new type of complete invariant layer for G -invariant CNNs - called selective G -Bispectrum layer - with the objective of increasing the accuracy and robustness of G -CNNs compared to those implemented with the initially proposed Max G -pooling. The G -TC layer also achieves this goal, but at an output cost of O ( | G | 2 ) coefficients and O ( | G | 3 ) flops that prevents its application to large groups, while the selective G -Bispectrum layer only outputs O ( | G | ) coefficients. Building on the result of Kakarala [15] for cyclic groups, we have shown that the completeness of the selective G -Bispectrum layer holds for all commutative groups, all dihedral groups, the octahedral and full octahedral groups. In a suite of experiments, we provided a global

Original

Image

Initial

Image

Trained select.

G

-Bispectrum

Trained

G

-Max

pooling

Original

Image

Initial

Image

Trained select.

G

-Bispectrum

Trained

G

-Max

pooling

Original

Image

Initial

Image

Trained select.

G

-Bispectrum

Trained

G

-Max

pooling

536

Original

signal

Recovered

signal

Original full

G

-Bispectrum

Recovered full

G

-Bispectrum

Original

signal

Recovered

signal

Original full

G

-Bispectrum

Recovered full

G

-Bispectrum

Original

signal

Recovered

signal

Original full

G

-Bispectrum

Recovered full

G

-Bispectrum

1

1

picture of the strength and weaknesses of each invariant layer. We studied the performance in terms of training speed, classification accuracy and robustness to adversarial attacks. As a result, the selective G -Bispectrum paves the way for the development of complete invariant pooling layers that can accommodate larger group sizes and, hence, a larger set of symmetries.

References

Bhatia, R. Matrix Analysis , volume 169. Springer, 1997. ISBN 0387948465.

Brillinger, D. R. Some history of higher-order statistics and spectra. Statistica Sinica , 1(2):465476, 1991. ISSN 10170405, 19968507. URL http://www.jstor.org/stable/24304021 .

Cesa, G., Lang, L., and Weiler, M. A program to build E(n)-equivariant steerable CNNs. In International Conference on Learning Representations , 2022. URL https://openreview. net/forum?id=WE4qe9xlnQw .

Chen, H., Zehni, M., and Zhao, Z. A spectral method for stable bispectrum inversion with application to multireference alignment. IEEE Signal Processing Letters , 25(7):911-915, 2018.

Cohen, G., Afshar, S., Tapson, J., and van Schaik, A. EMNIST: Extending MNIST to handwritten letters. In 2017 International Joint Conference on Neural Networks (IJCNN) , pp. 2921-2926, 2017. doi: 10.1109/IJCNN.2017.7966217.

Cohen, T. and Welling, M. Group equivariant convolutional networks. In Balcan, M. F. and Weinberger, K. Q. (eds.), Proceedings of The 33rd International Conference on Machine Learning , volume 48 of Proceedings of Machine Learning Research , pp. 2990-2999, New York, New York, USA, 20-22 Jun 2016. PMLR. URL https://proceedings.mlr.press/v48/ cohenc16.html .

Cohen, T. et al. Equivariant convolutional networks . PhD thesis, Taco Cohen, 2021.

Deng, L. The MNIST database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine , 29(6):141-142, 2012.

Diaconis, P. and Rockmore, D. N. Efficient computation of the Fourier transform on finite groups. Journal of the American Mathematical Society , 3:297-332, 1990. URL https: //api.semanticscholar.org/CorpusID:120893890 .

Giannakis, G. B. Signal reconstruction from multiple correlations: frequency- and time-domain approaches. J. Opt. Soc. Am. A , 6(5):682-697, May 1989. doi: 10.1364/JOSAA.6.000682. URL https://opg.optica.org/josaa/abstract.cfm?URI=josaa-6-5-682 .

Haniff, C. A. Least-squares Fourier phase estimation from the modulo 2 π bispectrum phase. Journal of the Optical Society of America A , 8(1):134-140, January 1991. doi: 10.1364/JOSAA. 8.000134.

Higgins, I., Amos, D., Pfau, D., Racaniere, S., Matthey, L., Rezende, D., and Lerchner, A. Towards a definition of disentangled representations, 2018. URL https://arxiv.org/abs/ 1812.02230 .

Hornik, K., Stinchcombe, M., and White, H. Multilayer feedforward networks are universal approximators. Neural Networks , 2(5):359-366, 1989. ISSN 0893-6080. URL https: //doi.org/10.1016/0893-6080(89)90020-8 .

Joyal, A. and Street, R. An introduction to Tannaka duality and quantum groups. In Carboni, A., Pedicchio, M. C., and Rosolini, G. (eds.), Category Theory , pp. 413-492, Berlin, Heidelberg, 1991. Springer Berlin Heidelberg. ISBN 978-3-540-46435-8.

Kakarala, R. Triple correlation on groups . PhD thesis, UC Irvine, 1992.

Kakarala, R. Completeness of bispectrum on compact groups. 2009. URL https://api. semanticscholar.org/CorpusID:18425284 .

Kakarala, R. Bispectrum on finite groups. In 2009 IEEE International Conference on Acoustics, Speech and Signal Processing , pp. 3293-3296, 2009. doi: 10.1109/ICASSP.2009.4960328.

Kakarala, R. The bispectrum as a source of phase-sensitive invariants for fourier descriptors: a group-theoretic approach. Journal of Mathematical Imaging and Vision , 44:341-353, 2012.

Kondor, R. A novel set of rotationally and translationally invariant features for images based on the non-commutative bispectrum, 2007. URL https://arxiv.org/abs/cs/0701127 .

Kondor, R. Group theoretical methods in machine learning . PhD thesis, Columbia University, 2008.

Kondor, R. and Trivedi, S. On the generalization of equivariance and convolution in neural networks to the action of compact groups. In Dy, J. and Krause, A. (eds.), Proceedings of the 35th International Conference on Machine Learning , volume 80 of Proceedings of Machine Learning Research , pp. 2747-2755, 2018.

Lecun, Y. and Bengio, Y. Convolutional Networks for Images, Speech and Time Series , pp. 255-258. The MIT Press, 1995.

LeCun, Y. and Cortes, C. MNIST handwritten digit database. 2010. URL http://yann. lecun.com/exdb/mnist/ .

LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. Gradient-based learning applied to document recognition. Proceedings of the IEEE , 86(11):2278-2324, 1998. doi: 10.1109/5.726791.

Nikias, C. L. and Mendel, J. M. Signal processing with higher-order spectra. IEEE Signal processing magazine , 10(3):10-37, 1993.

Norman, C. Finitely Generated Abelian Groups and Similarity of Matrices over a Field . Springer London, 2012. ISBN 9781447127307. URL http://dx.doi.org/10.1007/ 978-1-4471-2730-7 .

Sadler, B. Shift and rotation invariant object reconstruction using the bispectrum. In Workshop on Higher-Order Spectral Analysis , pp. 106-111, 1989. doi: 10.1109/HOSA.1989.735279.

Sanborn, S. and Miolane, N. A general framework for robust g-invariance in g-equivariant networks. In Advances in Neural Information Processing Systems , volume 36, pp. 67103-67124. Curran Associates, Inc., 2023. URL https://proceedings.neurips.cc/paper_files/ paper/2023/file/d42523d621194ba54dda098669645f91-Paper-Conference.pdf .

Sanborn, S., Shewmake, C., Olshausen, B., and Hillar, C. Bispectral neural networks. In International Conference on Learning Representations (ICLR) , 2023.

Steinberg, B. Representation Theory of Finite Groups: An Introductory Approach . Universitext. Springer New York, 2011. ISBN 9781461407751. URL https://books.google.com/ books?id=uwggkgEACAAJ .

Tukey, J. The spectral representation and transformation properties of the higher moments of stationary time series. Reprinted in The Collected Works of John W. Tukey , 1:165-184, 1953.

Weiler, M. and Cesa, G. General E(2)-equivariant steerable CNNs. In Proceedings of the 33rd International Conference on Neural Information Processing Systems , Red Hook, NY, USA, 2019. Curran Associates Inc.

Weiler, M., Forré, P., Verlinde, E., and Welling, M. Equivariant and Coordinate Independent Convolutional Networks . 2023. URL https://maurice-weiler.gitlab.io/cnn_book/ EquivariantAndCoordinateIndependentCNNs.pdf .

Zetzsche, C. and Krieger, G. Nonlinear mechanisms and higher-order statistics in biological vision and electronic image processing: review and perspectives. Journal of Electronic Imaging , 10(1):56 - 99, 2001. URL https://doi.org/10.1117/1.1333056 .

A Background on groups

We introduce the fundamentals of group theory, which provide the foundation for the theory of G -CNNs. These notions can be found in [30].

Definition A.1. A group is a pair ( G, · ) where G is a set and · : G × G ↦→ G is an associative multiplication such that there is an identity element e ∈ G (i.e., for all g ∈ G , e · g = g · e = e ) and, for all g ∈ G , there is an inverse g -1 ∈ G such that g -1 · g = g · g -1 = e .

̸

A group is thus a set G combined with a product · preserving the characteristics of G . Here, the established term 'product' can be misleading. It denotes any operation which makes Definition A.1 true given the set G . For instance, ( R , +) is a group. Another example is GL( R n ) , the n × n real invertible matrices, associated to the usual matrix product. This group is said to be non-commutative since A · B = B · A in general for A,B ∈ GL( R n ) . An important group for us is the set { 0 , 1 , ..., n -1 } associated with addition modulo n . It is usually written Z /n Z and called the cyclic group C n . A single group can arise in different contexts under seemingly distinct forms. For instance, Z / 4 Z and the rotations leaving the square unchanged in R 2 are fundamentally the same object. This observation gives rise to representation theory , a branch of group theory studying how the same abstract idea of a group can emerge under different forms.

Definition A.2. A representation of a group ( G, · ) is a pair ( ρ, V ) where V is a vector space and ρ : G ↦→ GL( V ) is a group homomorphism, i.e., for all g, h ∈ G , ρ ( g · h ) = ρ ( g ) ρ ( h ) . If V is equipped with an inner product and if for all g ∈ G and all u, v ∈ V , ⟨ ρ ( g ) v, ρ ( g ) w ⟩ = ⟨ u, v ⟩ , ρ is unitary .

Remark A.3 . Throughout this paper, we use the shorthand G to refer to the group ( G, · ) and ρ to refer to a representation ( ρ, V ) .

To illustrate Definition A.2, a representation of C n is given by the complex roots of unity, ρ ( k ) = exp ( 2 πi n k ) , on the complex one-dimensional vector space V = C . Every group also admit the trivial representation: ρ 0 ( g ) = 1 for all g ∈ G . There is a specific subset of these representations called the irreducible representations, irreps for short, being those that can not be expressed in a more compact form. The irreps are fundamental objects of group theory since they allow us to define an invertible Fourier transform on finite groups. The irreps are therefore needed to define the G -Bispectrum i.e., the Fourier transform of the G -TC. The notion of irreps is derived from that of a G -invariant subspace, which we recall in Definition A.4.

Definition A.4. Given a representation ( ρ, V ) , a subspace W ⊆ V is G -invariant if ρ ( g ) w ∈ W for all g ∈ G, w ∈ W .

The formal definition of the irreps is then stated as the representations with no non-trivial invariant subspace.

Definition A.5. A non-zero representation ( ρ, V ) of a group G is irreducible if the only G -invariant subspaces of V are { 0 } and V itself.

A single group acting on different spaces will have different representations. However, one can reveal the similarity between these representations by the mean of an equivalence relation.

Definition A.6. Two representations ( ρ, V ) and ( φ, W ) are equivalent if there exists an isomorphism T : V ↦→ W such that for all g ∈ G , ρ ( g ) T = Tφ ( g ) .

For the interested reader, the invertibility property of the Fourier transform is a consequence of the concepts of Pontryagin duality (commutative groups) and Tannaka-Krein duality (non-commutative groups); see, e.g., [14]. Our proofs will also rely on the notion of generating set of G , which we introduce here.

Definition A.7. A generating set S of a group ( G, · ) is a subset S ⊂ G such that every g ∈ G can be expressed as a finite combination of the elements in S and their inverses under the group action · .

Remark A.8 . It can be shown that every group G of size | G | has a generating set of size at most log 2 | G | .

Group Actions A group ( G, · ) represents a set of transformations such as rotations that can act on data such as images.We define formally how groups can indeed transform datasets through the concept of group action.

(A) Complete Invariance of G-Bispectrum

Imagedataset

Bispectrum

AW

4.000

4.015

Image Orbits

48F-086-0.86-0.56

-0.03

-0.00

4.01

excessive

complete

invariance

invariance

Definition A.9. Given a group ( G, · ) , a group action α : G × X ↦→ X is a function satisfying i) Identity: α ( e, x ) = x , ii) Compatibility: α ( h, α ( g, x )) = α ( h · g, x ) for any x ∈ X and h, g ∈ G and where e is the identity of G .

Processing operations and neural networks can be designed so that they respect group actions: specifically, a group acting on the input (e.g., rotating an input image) should yield a group action on the output (e.g., a rotation of the output feature map). This is the notion of G -equivariance .

Definition A.10. A function ψ : X ↦→ Y is G -equivariant if ψ ( α 1 ( g, x )) = α 2 ( g, ψ ( x )) for all x ∈ X and all g ∈ G , where α 1 and α 2 are group actions on X and Y , respectively.

For example, the G -convolution layer is G -equivariant by design [6]. An important problem in signal processing and deep learning is to achieve invariance to nuisance factors not relevant for the task. Many of these factors are describable as group actions (e.g. rotations, translations, scaling). Thus, we want processing methods and machine learning models to be G -invariant:

Definition A.11. A function ψ : X ↦→ Y is G -invariant if ψ ( α ( g, x )) = ψ ( x ) for all x ∈ X and all g ∈ G .

For example, the Max G -pooling ( max g ∈ G Θ( g ) ) traditionally follows a G -convolutional layer to remove the equivariance of the convolution and achieve G -invariance. A G -CNN is a neural network that consists of G -convolutional layers and a pooling/invariance operation. The main applications of our proposed selective G -Bispectrum operation is to act as a G -invariant pooling layer, that can conveniently replace the classical Max G -Pooling layer of G -CNNs, as shown in the rest of the paper.

Clebsh-Jordan matrices Given a group ( G, · ) and a family of unitary irreps { ρ i } | Irreps |-1 i =0 , the Clebsh-Jordan matrix is analytically defined for each pair ρ 1 , ρ 2 as:

G -Bispectrum for Commutative Finite Groups The computation of the G -Bispectrum simplifies for commutative groups compared to Theorem 2.3, as recalled below.

Theorem A.12. [17] If ( G, · ) is a commutative group and Θ : G → R , the G -Bispectrum can be computed as

For commutative groups, the G -bispectral coefficients are complex scalars [30].

B Indeterminacy of G -Bispectrum inversion problem

It is important to state precisely which information we can possibly retrieve from the G -Bispectrum. A consequence of the G -invariance of β (Θ) is that the G -Bispectrum inversion problem is ill-posed. Recall that G -invariance means that for all h ∈ G , we have β ( α ( h, Θ)) ρ 1 ,ρ 2 = β (Θ) ρ 1 ,ρ 2 . Given

a function Θ : G ↦→ R , a possible definition for the group action on Θ is given by α ( h, Θ( g )) = Θ( h -1 · g ) for all h ∈ G (see, e.g., [6]). Therefore, for all h ∈ G , we have which shows that the G -Fourier transform F (Θ) is G -equivariant. In consequence, recovering F (Θ) from β (Θ) can at best be done up to an unknown factor ρ ( h ) . Moreover, as explained in [15, 20], the indeterminacy is not limited to ρ ( h ) . Take for instance C n . An indeterminacy factor ρ k ( h ) = exp ( 2 πihk n ) corresponds to a translation of h ∈ C n of the signal, ˜ Θ( g ) = Θ( g + h ) . [15] showed that h is not restricted to C n = Z /n Z : it may take any value in [0 , n ] . The Bispectrum is not only invariant to a discrete set of rotations, but to the continuous group of rotations SO(2) . The factor can thus be written exp( iφk ) where φ ∈ [0 , 2 π ) .

C Selective G -Bispectrum inversion: known results

From now on, we assume that the Fourier transform F (Θ) only features non-zero elements , or invertible matrices in the case of non-scalar Fourier transform. This assumption is supported by the zero probability of encountering this corner case.

Cyclic groups C n We start with ( G, · ) = ( Z /n Z , + mod n ) =: C n . Recall that the irreps are given by ρ k ( g ) = exp ( ω k g ) where ω k := 2 πik n for k ∈ Z /n Z (see, e.g., [30]).

̸

Theorem C.1. [17] For cyclic groups C n , the C n -Bispectrum can be inverted using | G | = n coefficients if F (Θ) ρ = 0 for all ρ ∈ C n .

Proof. The Fourier coefficient associated to the trivial representation F (Θ) ρ 0 , is uniquely determined and can be recovered from Theorem A.12 by identifying phase and modulus:

We proceed using Pontryagin duality: the irreps { ρ k } n k =1 , form a group ̂ G themselves, with ̂ G = G . In the case of the cyclic group C n , notice that for all j, k ∈ Z /n Z , ρ j ⊗ ρ k = ρ j + k . Leveraging (6), we can use β (Θ) ρ 0 ,ρ 1 to recover F (Θ) ρ 1 :

Equation (8) leaves an indeterminacy on the phase of F (Θ) ρ 1 . This corresponds to the indeterminacy factor exp( iφ ) , ϕ ∈ [0 , 2 π ) of Appendix B. It is inherited from the G -invariance of β (Θ) (it is not injective, hence you cannot distinguish inputs that have the same G -Bispectrum). For now, let arg( F (Θ) ρ 1 ) = 0 . The key to recover all the other Fourier coefficients is to notice that S = { 1 } is a generating set of ̂ G = Z /n Z . Therefore, computing sequentially for k = 1 , 2 , ..., n -2 recovers completely F (Θ) . We are not done yet because the phase we fixed before is not a valid shift. A valid phase shift for F (Θ) ρ 1 is such that the shift w.r.t the original signal has the form exp ( 2 πih n ) for h ∈ N . This valid phase shift is easy to find. It is the unique φ ∈ [0 , 2 π n ) such that, if we define F ( ˜ Θ) ρ k = exp( φk ) F (Θ) ρ k for all k ∈ Z /n Z , then we have F -1 ( F ( ˜ Θ)) ∈ R n (i.e., with no imaginary part). Note that this is an explicit method to recover a valid phase while [17] relies on its existence without explicit method to find it. The method is summarized in Algorithm 2 and illustrated in Figure 9. In consequence only the following G -bispectral coefficients are needed for completeness: β (Θ) ρ 0 ,ρ 0 , β (Θ) ρ 0 ,ρ 1 and β (Θ) ρ 1 ,ρ k for k = 1 , 2 , ..., n -2 . This makes a total of n = | G | coefficients. We summarize this result in Theorem C.1.

Algorithm 2 Bispectrum inversion on Z /n Z [17]

1: Input : β (Θ) ρ 0 ,ρ 0 , β (Θ) ρ 0 ,ρ 1 and β (Θ) ρ 1 ,ρ k for k = 1 , 2 , ..., n -2 .

3: Compute |F (Θ) ρ 1 | = ( β (Θ) ρ 0 ,ρ 1 F (Θ) ρ 0 ) 1 2 and set arg( F (Θ) ρ 1 ) = 0 .

2: Compute |F (Θ) ρ 0 | = ( β (Θ) ρ 0 ,ρ 0 ) 1 3 and arg( F (Θ) ρ 0 ) = arg( β (Θ) ρ 0 ,ρ 0 ) .

4: for k = 1 , 2 , ..., n -2 do

6: end for

5: Compute F (Θ) ρ k +1 = ( β (Θ) ρ 1 ,ρ k F (Θ) ρ 1 F (Θ) ρ k ) † .

7: Find φ ∈ [0 , 2 π n ) such that F -1 ( F ( ˜ Θ)) ∈ R n where F ( ˜ Θ) ρ k = exp( ϕk ) F (Θ) ρ k .

8: Return F ( ˜ Θ) (up to group action).

β

ρ

0

,ρ

0

is followed by

gives

F

ρ

0

β

ρ

0

,ρ

1

F

ρ

1

β

ρ

1

,ρ

1

,

2

,...,n

-

2

F

ρ

2

,...,n

-

1

D Selective G -Bispectrum inversion: commutative groups

Here, we prove the theorem stated in the main text and recalled below:

Theorem D.1. For finite commutative groups G , the G -Bispectrum can be inverted using | G | coefficients if F (Θ) ρ = 0 for all ρ ∈ G .

̸

Specifically, we extend Algorithm 2 to all commutative groups, based on the Theorem D.2. That is, we design a method for the direct sum of finitely many cyclic groups.

Theorem D.2. (see, e.g., [26]) Every finite commutative group G is isomorphic to a finite direct sum of cyclic groups: G ∼ = ⊕ L l =1 Z /n l Z where L ∈ N and n l ∈ N for l = 1 , 2 , ..., L .

For all k ∈ G ( k is integer-valued vector of length L ), the irreps ρ k : G ↦→ C are given by ρ k ( g ) = ∏ L l =1 exp( 2 πi k l n l g l ) . The number of irreps is | G | = ∏ L l =1 n l . We detail and prove in Theorem D.3 the procedure to invert the G -Bispectrum on commutative groups. The procedure is summarized in Algorithm 3 where we use the two following notations.

e l denotes the basis vector in Z L such that e l k = 1 if k = l and e l k = 0 otherwise.

K l := { t e l + k | t = 1 , 2 ..., n l -1 and k ∈ K l -1 } for l = 1 , 2 , ..., L .

The sets K l are a recursively constructed such that K L ∼ = G . For G = ( Z / 3 Z ) 3 , the sets K 1 , K 2 , K 3 are represented in Figure 10.

0

1

2 0

1

2

0

1

2

K

1

K

2

K

3

(

Z

/

3

Z

)

3

e

1

e

2

e

3

Theorem D.3. For finite commutative groups G , the G -Bispectrum can be inverted using | G | coefficients if F (Θ) ρ = 0 for all ρ ∈ G .

̸

Proof. Notice that for the commutative groups, we keep the property ρ k ⊗ ρ k ′ = ρ k + k ′ where ( k + k ′ ) l := k l + k ′ l mod n l for l = 1 , 2 , ..., L . The first step is to obtain a generating set S of the

irreps is of size L . It is given by the usual basis vectors S = { e l ∈ Z L for l = 1 , ..., L } where

By Theorem A.12, it is sufficient to have the Fourier coefficients associated to each generating element in S to recover all the Fourier coefficients. Indeed, knowing F (Θ) ρ k 1 and F (Θ) ρ k 2 allows us to compute F (Θ) ρ k 1 + k 2 . By definition of the generating set S , we can thus recover F (Θ) ρ k for all k ∈ G . The moduli of the coefficients F (Θ) ρ e l for l = 1 , 2 , ..., L can be computed as follows:

where F (Θ) ρ 0 is the Fourier coefficient of the trivial representation (computed as in (7)). We claim that the phase can be fixed independently for each label l , thus L times. This is because only one factor h l remains among the L independent factors in h :

for all l = 1 , 2 , ., L . Therefore, fixing an arbitrary phase in (10) only fixes h l . Again, the indeterminacy factor h l is not restricted to Z /n l Z but can belong to [0 , n l ] . We will have to solve this issue further. For now, we set the phase of F (Θ) ρ e l to zero. Once the Fourier coefficients are known for all the generators of the group of the irreps { ρ e l } L l =1 , it remains to combine them to obtain all the elements in the groups and, consequently, all the associated Fourier coefficients.

At this point, it helps to consider the problem geometrically. Each irreps ρ k can be associated to its integer coordinate k inside a hyper-rectangle in R L , whose length of edges is n l for l = 1 , 2 , ..., L . We combine the coordinates e l to obtain all the possible integer coordinates inside the hyper-rectangle. First, we can obtain the L orthogonal edges of the hyper-rectangle. For l = 1 , 2 , ..., L , ρ e l ⊗ ρ e l gives ρ 2 e l , ρ e l ⊗ ρ 2 e l gives ρ 3 e l , etc. This is in fact the procedure of Algorithm 1. Now, we combine the edges to generate the inside of the hyper-rectangle. We proceed iteratively. For l = 1 , 2 , ..., L , we define K l := { t e l + k | t = 1 , 2 , .., n l -1 and k ∈ K l -1 } and K 0 := { 0 } . This construction is such that K l ∼ = ⊕ l j =1 Z /n j Z and K L = G . We generate the missing Fourier coefficients by combining the ones associated to the generating set of G . For l = 2 , ..., L , compute

for t = 1 , 2 , ..., n l -1 , for all k ∈ K l -1 . Intuitively for L = 3 , we obtain first an edge, then a face and finally the full parallelepiped. To conclude, we reproduce the procedure from Algorithm 2 to find a valid phase shift φ l in each basis direction e l . The last step is then to compute, for all k ∈ G ,

The procedure is summarized in Algorithm 3 and illustrated in Figure 11. It shows that the bispectral coefficients needed for completeness are: β (Θ) ρ 0 ,ρ 0 , β (Θ) ρ 0 ,tρ l e , β (Θ) tρ l e ,ρ k for l = 1 , 2 , ..., L , t = 1 , 2 , ..., n l -2 and all k ∈ K l -1 . We recover exactly one Fourier coefficient per G -bispectrum coefficient. This makes thus a total of | G | bispectral coefficients precisely and proves the following theorem.

E Selective G -Bispectrum inversion: dihedral groups

Dihedral group D n The dihedral group D n is the group of all symmetries of the n -gon. Mathematically, it is defined as

Algorithm 3 Bispectrum inversion for finite commutative groups ( G = ⊕ L l =1 Z /n l Z ). 1: Input : | G | bispectral coeffs. 2: Compute |F (Θ) ρ 0 | = ( β (Θ) ρ 0 ,ρ 0 ) 1 3 and arg( F (Θ) ρ 0 ) = arg( β (Θ) ρ 0 ,ρ 0 ) . 3: for l = 1 , ..., L do 4: Compute |F (Θ) ρ e l | = ( β (Θ) ρ 0 ,ρ e l F (Θ) ρ 0 ) 1 2 and set arg ( F (Θ) ρ e l ) = 0 . 5: for t = 1 , ..., n l -1 do 6: if t > 1 then 7: F (Θ) ρ t e l = ( β (Θ) ρ e l ,ρ ( t -1) e l F (Θ) ρ e l F (Θ) ρ ( t -1) e l ) † . 8: end if 9: for k ∈ K l -1 \ { 0 } do 10: Compute F (Θ) ρ t e l + k = ( β (Θ) ρ t e l ,ρ k F (Θ) ρ t e l F (Θ) ρ k ) † . 11: end for 12: end for 13: end for 14: for l = 1 , 2 , ..., L do 15: Find φ l ∈ [0 , 2 π n ) s.t. F -1 ( F ( ˜ Θ) ρ (1 ,..,n l ) e l ) ∈ R n l where F ( ˜ Θ) ρ k e l = exp( ϕ l k ) F (Θ) ρ k e l . 16: end for 17: for k ∈ G do 18: F ( ˜ Θ) ρ k = F (Θ) ρ k ∏ L l =1 exp( ϕ l k l ) . 19: end for 20: Return F ( ˜ Θ) (up to group action).

β

ρ

0

,ρ

0

l

= 1

is followed by

gives

F

ρ

0

β

ρ

0

,ρ

e

l

k

∈

K l

t

= 1

,

2

, ..., n l

-

1

l

= 2

,

3

, ..., L

F

ρ

e

l

β

ρ

t

e

l

,ρ

k

F

ρ

t

e

l

+

k

where a is the rotation and x is the reflection , and they form a generating set of D n . We will only consider the case n > 2 since the cases n = 1 and n = 2 are commutative groups covered by the previous subsection, while n > 2 gives non-commutative groups. The 2D irreps of D n are given by

where ω l = 2 πl n for k = 1 , 2 , ..., ⌊ n -1 2 ⌋ . There are also 2 or 4 1D irreps if n is odd or even, respectively. We denote these 1D irreps by ρ 0 , ρ 01 , ρ 02 , and ρ 03 (see Appendix E.1). The two last ones only exist for n even.

̸

Proof. In view of section C, we wish to show that there is an irrep ρ 1 that generates all the irreps of D n . As in the cyclic and commutative cases, we can first deduce F (Θ) ρ 0 from β (Θ) ρ 0 ,ρ 0 . Now, we have

Theorem E.1. For the family of dihedral groups D n , we need at most ⌊ n -1 2 ⌋ + 2 bispectral matrix coefficients for inversion if det( F (Θ) ρ ) = 0 for all irreps ρ of D n . This corresponds to 1 + 4 + 16 · ⌊ n -1 2 ⌋ ≈ 4 | D n | scalar values.

The novelty for this non-commutative group is that F (Θ) ρ 1 is a 2 × 2 matrix. After computing the eigenvalue decomposition β (Θ) ρ 0 ,ρ 1 F (Θ) ρ 0 = V Λ V † , we can choose

for all unitary U (i.e., UU † = U † U = I ) to solve (14). In the case Z /n Z the indeterminacy belonged to SO(2) , the continuous set of 2d rotations. For D n , it belongs to O(2) , the continuous set of 2d

Algorithm 4 Bispectrum inversion on the dihedral group D n .

1: Input : β (Θ) ρ 0 ,ρ 0 , β (Θ) ρ 0 ,ρ 1 and β (Θ) ρ 1 ,ρ k for k = 1 , 2 , ..., ⌊ n -1 2 ⌋ .

3: Compute V Λ V † = β (Θ) ρ 0 ,ρ 1 F (Θ) ρ 0 .

2: Compute |F (Θ) ρ 0 | = ( β (Θ) ρ 0 ,ρ 0 ) 1 3 and arg( F (Θ) ρ 0 ) = arg( β (Θ) ρ 0 ,ρ 0 ) .

4: Set F (Θ) ρ 1 = V Λ 1 2 V † U with valid U (existence of U ensured but not computed easily).

6: Compute

5: for k = 2 , ..., ⌊ n -1 2 ⌋ do

7: end for

8: Return F (Θ) (up to group action).

rotations and reflections. For finite groups, Kakarala [15] ensures that the only choices for U such that F (Θ) is a Fourier transform on G (= D n ) are such that Θ is identical to the original signal up to some group action g ∈ D n . For k = 2 , ..., ⌊ n -1 2 ⌋ , we can then obtain

̸

It is shown in Appendix E.1.1 that ρ k appears in the tensor decomposition (5) of ρ 1 ⊗ ρ k -1 so that the for-loop can be applied. Moreover, we know from Appendix E.1.1 that ρ 01 appears in the decomposition of ρ 1 ⊗ ρ 1 and, for n even, ρ 02 , ρ 03 in ρ 1 ⊗ ρ n 2 -1 . Thus the iteration recovers the complete DFT F (Θ) . β (Θ) ρ 0 ,ρ 0 is a scalar, β (Θ) ρ 0 ,ρ 1 is a 2 × 2 matrix and β (Θ) ρ 1 ,ρ k , k = 0 , is a 4 × 4 matrix. Hence the total number of scalars that is required is 1 + 4 + 16 ⌊ n -1 2 ⌋ .

The procedure is summarized in Algorithm 4.

E.1 The 1D irreps of D n

We recall the definition of the dihedral group D n given in (12). The 1D irreps of D n can be found, e.g., in [30]. They are given by:

ρ 0 ( g ) = 1 for all g ∈ D n .

If n even, ρ 02 ( g ) = { 1 if g ∈ ⟨ a 2 , x ⟩ , -1 otherwise.

ρ 01 ( g ) = { 1 if g ∈ ⟨ a ⟩ , -1 otherwise.

If n even, ρ 03 ( g ) = { 1 if g ∈ ⟨ a 2 , ax ⟩ , -1 otherwise.

To exemplify the 1D irreps, we give their values for D 4 in Table 3.

E.1.1 Generation of the coefficients of D n

Theorem E.1 makes two assertions that we verify explicitly in this appendix. First, it is said that ρ k is in the tensor decomposition (5) (TD) of ρ 1 ⊗ ρ k -1 for k = 2 , ..., ⌊ n -1 2 ⌋ so that the iteration of

Algorithm 4 recovers all the Fourier coefficients associated to the 2D irreps. The second assertion to verify is that ρ 01 is in the TD of ρ 1 ⊗ ρ 1 and, for n even, ρ 02 , ρ 03 in the TD of ρ 1 ⊗ ρ n 2 -1 so that the Fourier coefficients associated to the 1D irreps are also recovered, asserting the validity of the inversion procedure. We provide an analytical proof of these two assertions. The proof is based on the theory of character functions .

Definition E.2. [30] Given a group G and a representation ρ , the character of ρ is the function χ ρ : G → R : g ↦→ Tr( ρ ( g )) . χ ρ is said to be an irreducible character if ρ is an irreducible representation.

The character function χ ρ is a class function on G , i.e., χ ρ is constant on a conjugacy class of G . The space of class functions on a finite group G , written S G , can be equipped with an inner product ⟨· , ·⟩ G : S G ×S G ↦→ C such that for u, v ∈ S G , we have

The irreducible characters form an orthonormal basis w.r.t ⟨· , ·⟩ G for S G [30], i.e.,

Therefore, for ρ a , ρ b two irreps of G , ρ c is in the TD of ρ a ⊗ ρ b if and only if ⟨ χ ρ a ⊗ ρ b , χ ρ c ⟩ G = 0 . Let us apply this to the 2D irreps of D n ( n > 2 ). Let ρ i , ρ j , ρ k be three irreps defined as in (13). Notice that we have X ρ ( a l x ) = 0 . This yields

̸

̸

Recall that ∑ n l =1 cos( ω l m ) = { 0 if m = 0 n if m = 0 . Therefore, if we assume 0 ≤ i ≤ j without loss of generality, ⟨X ρ i ⊗ ρ j , X ρ k ⟩ ̸ = 0 if and only if

.

Therefore, based on Definition 2.3, by utilizing β ρ i ,ρ j , F (Θ) ρ i and F (Θ) ρ j , we can compute F (Θ) ρ k for k ∈ { i + j, j -i } . By iterating, if F (Θ) ρ 1 is known, β ρ 1 ,ρ 1 gives F (Θ) ρ 2 . Then, β ρ 1 ,ρ 2 can be leveraged to obtain F (Θ) ρ 3 . Continuing the procedure provides F (Θ) ρ k for k = 2 , ..., ⌊ n -1 2 ⌋ by using β ρ 1 ,ρ k -1 . We have thus obtained the Fourier coefficients associated to all the 2D irreps.

It remains to show that the iteration also recovered the Fourier coefficients associated to the 1D irreps. This is because ρ 01 is in the TD of ρ 1 ⊗ ρ 1 and, for n even, ρ 02 , ρ 03 are in the TD of ρ 1 ⊗ ρ n 2 -1 . Indeed,

Moreover, for n even and ρ ∈ { ρ 02 , ρ 03 } , we have

In conclusion, the procedure of Algorithm 4 recovers all the Fourier coefficients and the selective G -Bispectrum.

E.2 The Clebsch-Gordan matrices on D n

The matrix algebra properties that we use in this subsection can be found, e.g., in [1]. Recall from Theorem A.12 the (implicit) definition of the Clebsch-Gordan matrices:

where C † ρ 1 ,ρ 2 C ρ 1 ,ρ 2 = I . We only consider the case of ρ 1 , ρ 2 both 2d irreps of D n since otherwise, the Clebsch-Gordan matrix is the scalar 1 . First notice that ( ρ 1 ⊗ ρ 2 )( g ) , is an orthonormal matrix. Indeed, using the properties of the Kronecker product, we obtain (' ( g ) ' omitted for clarity):

For Q a real orthogonal matrix ( Q T Q = I ), and V SV T with V T V = I , a real Schur decomposition of Q , it is known that S is block diagonal with blocks of size 1 × 1 or 2 × 2 . These blocks are themselves orthogonal matrices. Therefore, the real Schur decomposition is the decomposition in (19) up to permutations. In order for S to represent exactly the irreps from (13), the non-zero sub-diagonal elements should all be positive. If not, the symmetric element is positive and a permutation P must be added to exchange their positions: Q = ( V P )( P T SP )( V P ) T . P permutes the two columns of V associated with the permuted 2 × 2 block of S .

Algorithm 5 Compute Clebsch-Gordan matrices on D n

1: Input: ρ 1 , ρ 2 , two 2d irreps of D n .

2: Pick any g ∈ D n , e.g., g = a .

4: Set C ρ 1 ,ρ 2 = V P .

3: Compute ( ρ 1 ⊗ ρ 2 )( g ) = ( V P )( P T SP )( V P ) T , a valid real Schur form.

6: Return: C ρ 1 ,ρ 2 , ⊕ ρ ∈R ρ ( g ) .

5: Set ⊕ ρ ∈R ρ ( g ) = P T SP

F Bispectrum inversion for octahedral and full octahedral groups

We provide a sketch of the procedure to retrieve F (Θ) given β (Θ) for the octahedral group and the full octahedral group. These two groups are available in the escnn library. These groups are easier to deal with than the cyclic and dihedral groups presented in the paper, given that they do not come from a family of groups. Indeed, our proofs for the cyclic (resp. dihedral) groups needed to work for all cyclic groups C N , and for all dihedral groups D N , for all N . The octahedral and full octahedral groups are only two groups.

F.1 Octahedral group

The octahedral group has 24 elements and 5 irreps. We can compute its Kronecker table, either manually using characters χ or using a Python package such as escnn . We give its Kronecker table below (Table 4), where each column/row represents one irrep, labelled ρ 0 , ρ 1 , ..., ρ 4 .

We apply the procedure from Algorithms 2, 3 and 4 to Table 4. This procedure relies on the use of Theorem 2.3. We first select the bispectral coefficient β ρ 0 ,ρ 0 (we omit ' (Θ) ' for clarity) to get the component F ρ 0 where ρ 0 is the trivial representation. Next, we choose β ρ 0 ,ρ 1 and use F ρ 0 to obtain F ρ 1 (we know from Table 4 that ρ 1 ∈ ρ 0 ⊗ ρ 1 ) up to an indeterminacy which is a transformation in O(3) and corresponds to the indeterminacy factor from Appendix B. Then, we select β ρ 1 ,ρ 1 to get the Fourier components F ρ 2 , F ρ 3 . Lastly, we select β ρ 1 ,ρ 2 to get the missing Fourier component F ρ 4 .

In summary, we only need 4 bispectral coefficients ( β ρ 0 ,ρ 0 , β ρ 1 ,ρ 0 , β ρ 1 ,ρ 1 , β ρ 1 ,ρ 2 ) instead of 5 2 = 25 in order to get the five Fourier components, i.e., the full Fourier transform of the signal. In total, this involves 1 + 9 + 81 + 81 = 172 scalar coefficients.

F.2 Full octahedral group

The full octahedral group has 48 elements and 10 irreps. Again, we can compute its Kronecker table using a Python package such as escnn . We give its Kronecker table below (Table 5), where each column/row represents one irrep, labelled ρ 0 , ρ 1 , ..., ρ 9 .

We apply the procedure from Algorithms 2, 3 and 4 to Table 5. Again, this procedure relies on the use of Theorem 2.3. β ρ 0 ,ρ 0 (we omit ' (Θ) ' for clarity) allows to compute F ρ 0 directly, such as in Algorithm 4. Then, from β ρ 0 ,ρ 6 , we obtain F ρ 6 up to an unknown group action in O(3) . Then, using β ρ 6 ,ρ 6 and F ρ 6 , we obtain F ρ 1 , F ρ 2 and F ρ 3 . Next, leveraging β ρ 1 ,ρ 2 , F ρ 1 and F ρ 2 , we obtain F ρ 4 . Using β ρ 1 ,ρ 6 , F ρ 1 and F ρ 6 , we obtain F ρ 5 , F ρ 7 , F ρ 8 . Finally, with β ρ 1 ,ρ 7 , F ρ 1 and F ρ 7 , we obtain the last coefficient F ρ 9 . Hence we have recovered all the Fourier coefficients using only β ρ 0 ,ρ 0 , β ρ 0 ,ρ 6 , β ρ 6 ,ρ 6 , β ρ 1 ,ρ 2 , β ρ 1 ,ρ 6 , β ρ 1 ,ρ 7 , thus a total of 6 bispectral coefficients instead of 100 . In terms of scalar values, this involves 1 + 9 + 81 + 81 + 81 + 81 = 334 coefficients.

G Training of the G -CNN architecture

The SO(2) -MNIST/EMNIST datasets are obtained after applying random planar rotations on each image of the datasets MNIST [23], EMNIST [5] respectively. In the case of O(2) -MNIST/EMNIST, in addition to a planar rotation, a reflection is applied with probability 1 2 . The original size of each image is conserved. The size of the training sets are 60 000 and 88 800 for MNIST and EMNIST, respectively.

We conserve the architecture of [28]. For all invariant layers, being the G -TC, the selective G -Bispectrum and the Avg/Max G -pooling, the architecture is composed of a C 8 /D 8 -convolutional block with K filters (see Table 2). Then, the invariant layer is applied before feeding the output to a MLP. The MLP is composed of 3 fully-connected layers with ReLU non-linearity. A final fully-connected linear layer is applied for classification. The vector of the output sizes of these layers is given by [ o 1 , o 2 , o 3 , ol ] respectively. o 2 = o 3 = 64 and ol is equal to the number of classes of the dataset. o 1 is tuned to reach the parameter count from Table 2.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The claims that we make in the abstract, i.e., theoretical and practical guarantees of the G -Bispectrum are respectively proven and studied in Sections 4 and 5.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: In Section 5, we discuss the pros and cons of the selective G -Bispectrum compared to the other possible invariant layers.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: The previously known theorems and results are referenced and each new theorem is accompanied by a proof, usually proposed in the appendices.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We provide all the details about the architectures of the G -CNNs that we use in Section 5 and Appendix G. The code will be made available upon publication, with precise guidance to reproduce the results.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: The code will only be made available after publication to preserve the anonymous component of the review process. This code will include precise guidance to reproduce the results.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We provide in Section 5 the details about the datasets and the architectures of the models implemented for experiments. The optimization details and hyper-parameters choice will be made available with the code upon publication.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: All the experiments, i.e., Table 2 and Figures 4 and 5 represent the standard deviations of the experiments performed.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We give in section 5 the running times of the algorithms and the hardware used for training.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: This paper is not involved with specific ethical issues, except that it contributes to the field of image processing. The latter field is related to the ethical issue of facerecognition and automatic surveillance.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: This work is a research work to propose a new layer in the architecture of G -CNNS. It has no direct societal impacts beyond the fact that it contributes to imageprocessing, which can be used for malicious purposes.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: This paper does not leverage models with high risk of misuse.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We use the datasets MNIST [8] and EMNIST [5], which we properly mention in the main text. The URLs are displayed with hyper-references.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: We do not use new assets and rely on existing datasets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: No participants were required during the experiments.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: No human subjects were necessary to prepare this manuscript.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a novel invariant layer for group-convolution networks (GCNNs) based on the \"selective bi-spectrum\".\nWith respect to the expensive bispectrum which has $O(|G|^2)$ coefficients, the selective bispectrum only contains a subset of $O(|G|)$ coefficients which still ensure the completeness of the invariant features and can be computed in $O(|G| log |G|)$ when a FFT algorithm is available.\nSome preliminary experiments show the proposed method has comparable runtime and performance to the typically used max-pooling operator.\n\nSTRENGTHS:\nThe manuscript is clearly written and the proposed method is original and interesting.\nThe paper also includes some novel theoretical results, which could be useful for a wider audience.\nThe empirical validation is in its very early stage, but the preliminary results are encouraging.\n\nWEAKNESSES:\nI feel like the empirical evaluation of the method is still too weak for acceptance.\n\nFirst, the authors only experiment with variations of the MNIST dataset: while this is a good dataset for some preliminary experiments to validate an idea is a simple and easily reproducible environment, I don't think it is sufficient to draw significant conclusions, especially about properties such as expressiveness of an operator or completeness of a representation.\nThe authors should consider experimenting with more realistic datasets: for example, previous works typically used medical datasets, which still show rotational symmetries but provide higher resolution images and more challenging classification tasks.\nBecause the contributions of this paper include the theoretical description of the selective bispectrum for some 3D groups, the authors should include some experiments on 3D datasets to validate that too; these datasets could also provide more convincing evidence of the value of the proposed method.\n\nSecond, even if larger datasets were used, the current experimental setting doesn't provide conclusive answers on when one should use max-pooling or bi-spectrum.\nIndeed, the comparison in Fig. 5 is a bit misleading as the accuracy is plotted as a function of the number of filters K, but the computational cost of the two methods is different when varying K.\nInstead, a plot showing the accuracy vs (some proxy for) the computational cost seems a more suitable choice, drawing a more complete picture and quantifying the trade-offs discussed between lines 259-263.\n\n\nOverall, while I am very positive about the proposed idea, I think that deserves a much more thorough empirical validation.\n\nQUESTIONS:\nFig. 6: this seems just a qualitative results useful to demonstrate the robustness of the selective bispectrum, but a proper quantitative validation of the robustness is still missing. A more comprehensive and thorough comparison between selective bispectrum, normal bispectrum and, maybe, max-pooling seems necessary. Why not reporting some statistics aggregated over a much larger dataset of random signals rather than visualising just 6 examples? It would be even more convincing to do so using the deep features extracted by a GCNN from a dataset of natural images, to prove the completeness is relevant beyond simple synthetic settings.\n\n\nTheorem 4.5: Is it correct that the (4) G-Bispectral coefficients are matrices? I find the choice of counting the matrices instead of individual scalars to be a bit misleading, as these matrices have very different shapes, making this number not very informative. Why not reporting only the number of scalar coefficients? This is a quantity which is meaningful across different groups and that can be directly compared with the group size |G| (the Fourier transform over the group G always has |G| coefficients, regardless of which finite group we consider).\n\n\nTheorem 2.4: alpha is used but never defined\n\nTable 1: why is K not included inside the big-O notation?\n\n\nLine 242: \"... so that the G-Bispectrum scales worth\". This sentence is a bit unclear.\n\nLIMITATIONS:\nThe authors could discuss practical challenges in an efficient or parallel implementation of the selective bispectrum operator due to the sparsity and irregular shapes of the matrices involved.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-15"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addresses the problem of achieving invariance to nuisance factors in signal processing and deep learning, particularly those describable by group actions (e.g., rotations, translations). The authors propose the selective G-Bispectrum, a computationally efficient variant of the G-Bispectrum, which reduces complexity from $O(|G|^2)$ to $O(|G|)$ in space and $O(|G| \\log |G|)$ in time when an FFT is available on the group $G$.\n\nSTRENGTHS:\nThe paper presents a novel approach to reducing the computational complexity of the G-Bispectrum, a key operation in achieving G-invariance in signal processing and deep learning. This reduction is significant, lowering the complexity from \\(O(|G|^2)\\) to \\(O(|G|)\\) in space and from \\(O(|G|^2)\\) to \\(O(|G|\\log|G|)\\) in time, assuming the availability of an FFT on \\(G\\). The selective G-Bispectrum is a creative combination of existing ideas in group theory and signal processing, applied in a new way to enhance deep learning architectures. This work stands out by addressing computational limitations that have hindered the widespread adoption of the G-Bispectrum.\n\nThe paper is well-structured and thorough in its theoretical contributions. The authors provide rigorous proofs of the mathematical properties of the selective G-Bispectrum.\n\nThe significance of this work lies in its potential to advance the field of geometric deep learning and signal processing on groups. By reducing the computational complexity of the G-Bispectrum, the authors make this tool more practical for real-world applications.\n\nWEAKNESSES:\nThe paper primarily evaluates the proposed selective G-Bispectrum on the MNIST and EMNIST datasets. While these datasets are standard benchmarks in machine learning, they are relatively simple and may not fully stress-test the proposed method’s effectiveness in more complex and varied scenarios, especially larger group such as 3D group.\n\nAlthough the paper claims that the selective G-Bispectrum has higher computational efficiency, it does not explicitly compare the computational costs and performance together in the experiments. This omission undermines the argument for computational efficiency, particularly given that the performance of the selective G-Bispectrum is not as strong as the G-TC method.\n\nQUESTIONS:\n**Question:** It is mentioned in the paper that the selective G-Bispectrum offers greater selectivity and robustness. Could you provide experimental evidence to validate these claims?  \n**Suggestion:** Include experiments and analyses specifically designed to demonstrate the greater selectivity and robustness of the selective G-Bispectrum. This could involve:\n- **Selectivity:** Experiments that show how well the method distinguishes between different classes or handles variations within the same class.\n- **Robustness:** Tests under various noise conditions, transformations, or adversarial attacks to demonstrate the method's stability and reliability.\n\n\n**Question:** Why were only MNIST and EMNIST datasets used for the evaluation? Would the selective G-Bispectrum perform similarly on more complex datasets?  \n**Suggestion:** Expand the experimental section to include more diverse and challenging datasets, such as CIFAR-10, ModelNet10. This will provide a more comprehensive assessment of the method's effectiveness.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-16"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe work focuses on the problem of achieving invariance to extraneous nuisance variables in signal processing and deep learning by utilizing group actions such as rotations and translations.\n\nThe focus is on the G-Bispectrum, which is a tool used to extract signal characteristics that are invariant to such actions. This tool provides advantages similar to pooling techniques, but with more selectivity and robustness.\n\nThe G-Bispectrum has been limited in its application because to its high computational cost, which is O(|G|2). However, this study presents a strategy to lower this complexity to O(|G|) by proposing selective G-Bispectrum.\n\nThe novel methodology preserves the mathematical characteristics of the G-Bispectrum and enhances the precision and robustness of neural networks, while simultaneously delivering substantial speed ups.\n\nSTRENGTHS:\nThe research presents a clear motivation, which is to decrease the computational complexity of the G-Bispectrum from O(|G|2) to O(|G|) without compromising its effectiveness as a computational primitive for G-invariance. The emphasis on enhancing computing efficiency while maintaining performance is clearly expressed and pertinent.\n\nThe proposed method is technically valid. The assumptions employed are rational, and the research establishes a robust theoretical basis for the selected G-Bispectrum, effectively showcasing its mathematical qualities such as G-invariance, completeness, and uniqueness.\n\n The paper is very well-written and easy to follow. Figure 1 enhances the clarity of the presentation by providing a comprehensive overview of the motivation for the proposed strategy and successfully conveying the essential themes.\n\nThe research presents extensive empirical evidence showing that the selective G-Bispectrum achieves significant improvements in speed compared to the whole G-Bispectrum in real-world scenarios. The enhancement in efficiency renders the selected G-Bispectrum a feasible and efficient substitute for attaining G-invariance in deep learning models.\n\nWEAKNESSES:\nThe proposed approach is currently limited to discrete groups. Several contemporary equivariant networks have been applied in domains that involve continuous groups like $SO(3)$ and $SE(3)$. However, the current technique does not readily adapt to these situations, and the paper does not investigate or experiment with such extensions in these domains. Could the authors comment on this limitation and discuss whether it could be a potential direction for future research?\n\nFurthermore, the tests conducted in this research are restricted to small-scale datasets such as MNIST. There is the possibility of analyzing larger and more intricate datasets in order to more effectively showcase the advantages of the suggested approach. I strongly urge the authors to contemplate using their methodology on larger-scale image datasets in order to demonstrate its efficacy in a more thorough manner.\n\n**Minor**\n\nIn line 183, novel theorems, Although the coefficients may not be precisely zero, they could be in close proximity to zero. When performing numerical computations, this can result in problems related to stability and precision.\n\nThe formula presented in Theorem 4.4, which states that the number of bispectral matrix coefficients is equal to ⌊n-1/2⌋ + 2, may benefit from a more comprehensive explanation for readers who are less acquainted with dihedral groups.\n\nThe sentence in Theorem 4.4, \"This corresponds to 1 + 4 + 16 · ⌊n-1/2⌋ ≈ 4|Dn| scalar values,\" may be clarified by providing a concise explanation of the derivation of this approximation.\n\nQUESTIONS:\nThe proposed approach is currently limited to discrete groupings. Various contemporary equivariant networks have been utilized for continuous groups like $SO(3)$ and $SE(3)$. Could you provide an analysis of the constraints associated with applying your approach to these continuous categories? Do you anticipate any theoretical or practical obstacles, and do you view this as a promising avenue for future investigation?\n\nRecommendation: It is advisable to evaluate the suggested technique on image datasets of greater magnitude in order to demonstrate its efficacy. To better showcase the benefits of your technique, it is advisable to conduct trials using established benchmarks or more intricate datasets.\n\nQuestion: What is the difference between the suggested selective G-Bispectrum and contemporary equivariant networks tailored for continuous groups? Are there any particular benefits or drawbacks that might be emphasized in the context of ongoing group actions?\n\nLIMITATIONS:\nNA", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-17"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a new invariant layer for group convolutional neural networks (G-CNN). The proposed layer computes the spectral coefficients of the input data, that is based on higher-order spectral analysis: the bispectrum, which is the Fourier transform of the triple correlation (G-TC). While there can be up to |Irrep|^2 different coefficients in the bispectrum, the main contribution of the work is an algorithm that finds a subset of at most |G| of these coefficients. For specific groups (general abelian, dihedral, (full) octahedral), the authors proved that there exists a subset of size at most |G| that can reconstruct the input data (signal) perfectly, and that their algorithm, with the right choice of \\rho_1, can find this set. Experiments are conducted on MNIST and EMNIST, comparing the newly proposed layer with max/sum pooling and G-TC layer.\n\nSTRENGTHS:\n- Novelty. The use of higher-order spectral analysis is relatively new in designing deep learning architecture over group and can be powerful since they are originally used to capture nonlinear/autocorrelational structure of the input signal. Capturing nonlinearity is nontrivial in designing G-CNNs as pointwise nonlinearity is known to not be the most appropriate choice of nonlinearity.\n- Theoretical contributions, especially to nonabelian groups such as Dn and O are interesting and has substantial savings in the number of necessary coefficients to be efficient in practice. I did not go through the proofs line by line but believe that the proof idea is sound.\n- Self-contained. The paper, together with the appendix is relatively self-contained in introducing group-theoretic and representation-theoretic concepts. The overall exposition/English is also easy to follow.\n- Clear experiments and diagrams. The paper demonstrate a regime of small number of filters where their methods work much better than simple group pooling, which is a clear and instructive takeaway. Diagrams to demonstrate algorithms are also clear and make for a quicker read.\n\nWEAKNESSES:\nThe main issues I have with the current manuscript is its motivation and some ill-defined notion.\n\nIssues with ill-defined notions:\n- The term \"selective G-bispectrum\" is not well-defined, despite being the central notion of the whole paper. The authors defined this notion by giving an algorithm that depends crucially on a hyperparameter \\rho_1. The authors themselves have noted that the choice of \\rho_1 will make or break the resulting set of coefficient (being a complete system). Thus, completeness is a property of the algorithm + choice of \\rho_1 and it does not make sense to claim, for e.g., that the selective G-bispectrum is complete without specifying this choice of irrep. \n- Algorithm 1 depends crucially on computing the Clebsch–Gordan matrices, which may be very costly in general for nonabelian groups. The authors have also acknowledged this point.\n\nIssues with motivations:\n- The authors motivate the paper by arguing that a complete (Definition in Theorem 2.4) invariance layer is more desirable than simple G-pooling layer (such as max or sum), which is an intuitive point. However, in the experiments, average-pooling, which is arguably more complete than max-pooling (or, to quote, less \"excessively invariant\") performs consistently worse than anything else, and to a large extent. Is there a good explanation for this? \n- The proofs of all completeness theorems essentially boils down to being able to reconstruct the group FT coefficients of the input signal from the subset of bispectrum coefficient, which begs the questions of what happens if you simply input these |Irrep| numbers of group FT coefficients as an invariant layer, instead of going to the bispectrum and hoping that it can reconstruct group FT coefficients? The answer to this probably lies in the reference \"The work of Kakarala (1992) illuminated the relevance of the G-Bispectrum for invariant theory, as it is the lowest-degree spectral invariant that is complete (Sturmfels, 2008)\" but scheming through Sturmfels' book did not lead me to any such conclusion. Can the authors provide a more specific reference, and/or discuss this point (why bispectrum but not just FT coefficients, in light of completeness) a bit more? This would be an interesting experiments to look at too. \n- Lastly, why spectral? Usually, signal processing literature makes use of spectral domain because of sparsity in that domain. The authors have also shown that it is faster to compute G-bispectrum coefficients than G-TC. These motivations (and perhaps others, if presence) to using spectral domain should be written up more clearly in the beginning of the paper. \n\nThere are also various typos (e.g. line 26, R not defined in line 493, ...) that I did not carefully keep track of in the interest of time.\n\nQUESTIONS:\nSee weakness section.\n\nLIMITATIONS:\nThe authors have acknowledged some of the limitations in their methods and definitions (detailed in Weakness section). There are no potential negative societal impact that are specific to this work and are worth highlighting.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-22"}