Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper considered the approximation property of GCNNs on the frame of the ridgelet theory. By representing a typical GCNN into a nonlinear mapping between group representations, corresponding formulations on $(G,T)-convolution$ and $(G,T)-equivariance$ are established for the convenience of further demonstration under ridgelet-frame. The main results showed the cc-universality of GCNNs in a constructive and unified manner. 
strengths_and_weaknesses: *Strengths: The ridgelet transform induced group convolution can cover typical class of neural network convolutions such as deep sets and E(n)-equivariant convolutions. This study derives the ridgelet transform for a general class of GCNNs. The reconstructed formula is given which discretized version is then applied to prove the universality of (G,T)-CNNs. The cc-universality of GCNNs for a general class of group equivalent continuous vector-valued functions is shown as an example. Examples of GCNNs are given.
*Weaknesses: It is best the numerical experiments are shown to test the theory developed.

questions: Is graph equivariant NNs (such as SE(3)-transformer) an example of ridgelet transform-based CNNs?
limitations: This is theoretical study. There is no negative societal impact.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper uses the ridgelet theory technique to show the universality of group convolutional neural networks for various groups and feature spaces. Their results are new in the way that they apply ridgelet theory to group CNNs. Prior works have used ridgelet transform to show the universality of fully-connected neural networks, and literature exists on the universality of GCNN for limited groups. 
strengths_and_weaknesses: The universality of GCNN is of interest to the community. The paper provides a proper amount of details and explanations. However, I found the paper a bit hard to follow. Many terms are defined without additional explanation/elaboration. For example, Section 5 provides several examples without any description. Moreover, I recommend covering related works in more depth.

The originality and significance of the work are unclear: 1) ridgelet transform as a technique is previously used for the universality of fully-connected networks, and 2) prior works have shown the universality of GCNN under certain conditions.
questions: Answering the following questions is to provide insight into the significance of the method and its novelty. Please explain

1. How the method differs from others?
2. How difficult and unique the analysis are compared to the one using ridgelet for fully-connected networks? Is it a trivial extension?
3. How do their universality results for GCNN differ from already existing literature on GCNN? The authors have briefly touched on this, but it needs more clarification.

limitations: N/A
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: [updated score from 6->7 based on technical soundness and contribution to DL theory]

The paper describes a mathematical analysis of group convolutional neural networks, showing that they are universal approximators. To show this, the authors derived the ridgelet transform (a mathematical analysis tool) for G-CNNs. This is a result in itself, which is further used in this paper to provide a constructive proof of universality.
strengths_and_weaknesses: **Strengths**
* The paper is technically sound and rigorous
* The proposed proof constructive and does not rely on indirect proofing methods

**Weaknesses**
* The paper is very technical, and could perhaps better fit a mathematical functional analysis venue, rather than an ML conference. The theme (universality of G-CNNs) falls within the conference scope, but the content otherwise is purely mathematical
* Related is my concern about the impact of the paper in the field. What is the relevance of a new universality proof? I see the beauty/elegance of the paper, but that is on math merits, not on ML principles per se.
* The paper could overall benefit from a more intuitive exposition. Why is it important to derive the ridgelet transform? How should I interpret this transform? Why can we show universality once it is derived?

I believe the paper could be a great contribution to NeurIPs if it were to be made more accessible to a broader audience (like me, I like to learn from these papers, but this one was actually far out my comfort zone). E.g. up and until page 3.2 I could clearly follow everything, but what comes after I had to read several times to understand, and still don't fully understand the significance of e.g., 3.3. These are things a mathematician may get straight away though..

**Minor details**

line 54: Could you provide some intuition on the role of $\gamma$ in equation 2. It somehow weights the different parameter sets (kernel a, bias b) and can be used to rewrite into (1), but a bit more intuition may be helpful.

line 64: What does $C(G)$ denote? Only later are such spaces defined, but not yet this early..

Line 69: I like the analysis - synthesis viewpoint of R and S, in relation to the Fourier transform. This was helpful.

Line 87: "As an application, we show ..." Are there any other applications of the ridgelet transform? I find it hard to assess the impact of the result of deriving it.

Line 126: Why do you refer to it as a *generalized* form of the group convolution? I have seen this form several times with $\mathcal{H}=\mathbb{L}_2(G/H)$. Does generalized refer to the fact that $\mathcal{H}$ can be different then the usual function spaces?

Line 151: This sentence misses a part!

Line 158: Why does f now map to $\mathbb{C}^G$ and not $C(G)$?

Line 184: Fourier transform on $\mathcal{H}$, this is way more abstract then I have ever seen the FT (most general to me is FT based on group irreps). Is it possible to give interpretation to this? Why do we still call it a Fourier transform?

Line 221: What does notation $f(\cdot,e)$ mean? Is it the same as $f(\cdot)(e)$?

Line 263: This is another example of where I understand the explicit forms (of the ridgelet trafo and reconstruction formula), however, I do not understand why it is relevant for me to read this. One the one hand it is nice to see explicit examples, on the other hand side I do not understand what to do with it? E.g., what are the implications to the universality proof?
questions: Important:
1. Why is it important to derive the ridgelet transform? 
2. How should I interpret this transform? 
3. Why can we show universality once it is derived?

Less important:

4. Line 87: "As an application, we show ..." Are there any other applications of the ridgelet transform? I find it hard to assess the impact of the result of deriving it.

5. Line 184: Fourier transform on $\mathcal{H}$, this is way more abstract then I have ever seen the FT (most general to me is FT based on group irreps). Is it possible to give interpretation to this? Why do we still call it a Fourier transform?
limitations: I see no issues here.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The authors prove the (cc-) universality property of group convolutional neural networks (GCNNs) with one hidden layer using the tools from ridglet theory.
To do so, the authors derive the first closed form solution for the ridglet transform of GCNNs, covering a wide spectrum of existing equivariant model designs.

strengths_and_weaknesses: The manuscript is well written and the presentation is clear.
The contributions are clear and novel and the theoretical analysis seem rigorous (although I am not an expert on ridglet analysis and universality proofs).

While the universality of GCNNs is a well known fact, existing proofs were only indirect or existential, while the proposed proof is constructive.
However, I feel like the manuscript does not motivate sufficiently well why this difference is important; see "Questions".

Finally, while the work seems quite rigorous, the authors do not comment much on their final result.
Indeed, the paper is missing a proper "conclusion" section, stating what the main take-away points should be.

questions: The authors claim that existing universality proofs for GCNNs are indirect and rely on the universality of either fully-connected networks or invariant polynomials, while the proposed method is direct and constructive.
Unfortunately, I am not familiar with the literature on the universality property; could the authors elaborate further about the limitations of these existing methods and the benefit of the proposed one?

Is this proof considered constructive because of the existence of the reconstruction formula for the ridglet transform?

What kind of conclusions can be drawn from this theoretical work?
Does this proof provide any new intuition about GCNNs with respect to previous methods?


Line 189: why is this an advantage for geometric understanding of CNNs? Could you elaborate further on this?

limitations: While the work seems rigorous, I think it misses some motivation and an analysis of the theoretical results.
However, I am not familiar with the related literature on universality properties of neural networks.
review_title: null

Meta-review: metareview: The paper studies the approximation properties of group convolutional neural networks. It establishes the “cc-universality” of group CNNs, i.e. that such networks can approximate any continuous function over any compact set, using a new constructive approach which is based on a generalization of the ridgelet transform. The proof is constructive, in the sense that approximating networks are given in closed form by discretizing the transform. This approach may have applications beyond the scope of the paper — most immediately, to identifying classes of functions for which neural network approximations are accurate in a quantitative sense; this can be tied to the decay properties of the ridgelet transform. Reviewers found the paper to be clearly written and of high technical quality, albeit somewhat mathematically dense in its presentation. Universal approximation theorems provide an important piece of theoretical background, as well as a “sanity check” for new network architectures; having a unified, constructive approach to derive them could stimulate further work. 
confidence: Less certain
Decision: Accept