Paper: Fast, Expressive $\mathrm{SE}(n)$ Equivariant Networks through Weight-Sharing in Position-Orientation Space (ICLR-2024)

Reviewer comment (Novelty):
There are so many contents from previous works that I can hardly tell the novelty. From my point of view, only Theorem 1 is new, but it is still quite easy to prove.

Author response:
The first mentioned weakness is that it is hard to recognize the novelty, given the many previous works. This is a valuable comment and agree that the issues arises from our drive to unify related works through our approach. We do indeed build upon the so-called shoulders of giants and as such thoroughly discuss related work. Our new contribution can however be described on several levels:
1. **Conceptual**: the formal notion of weight sharing in the equivariance setting provides a simple recipe that so far did not exist in literature. 
2. **Practical**: we provide a simple yet effective architecture. Many of previous works either adopt the steerable approach, or an intricate wiring of message passing schemes based on invariants. Our method greatly simplifies the latter approach. Our architecture in itself follows the standard convnet paradigm is predominatly used in computer vision, and make it effectively applicable to point-cloud data, whilst being fully equivariant. 
3. **Theoretical**: even though theorem 1 might be simple to proof, we think it is an important contribution because by formally deriving identifiers for equivalence classes we have proven that the obtained attributes are in fact all you need to build fully expressive NNs. Based on your other comment, *we now also include a proof of our approach being a universal equivariant approximator*. 

The contributions in the intro is therefore slightly updated. Thanks again for your remark, we think it did improve the presentation.

Reviewer comment (Presentation):
Though this work is titled with $SE(n)$, it only discusses $n=3$ case.

Author response:
Though the work is titled with $SE(n)$, it only discusses the $n=3$ case. It is true that we focussed almost exculisivey on $n=3$. We decided to use $n$ instead of only $n=3$ because this is where a unifying view was still missing. With that we mean, $SE(2)$ as a space is equivalent to the position space $\mathbb{R}^2 \times S^1$. Since literature already extensively covers group convolutional methods for $SE(2)$ equivariance, there is not yet much to add to this. Our method still adds conceptually to the field via the viewpoint of weightsharing, and non-linear convolutions. Notably, our method when applied to 2D data would boil down to regular $SE(2)$ convolutions, which in Weiler and Cesa 2019 have been proven to be more effective than steerable convolutions. This motives the use of working with invariants over the use of tensor field neworks (based on irreducible representations). *We deepen the discussion on the impact of regular group convolutions with a newly added appendix A "The Fiber Bundle vs Point Cloud Viewpoint"*.

Reviewer comment (Theory):
Does this work have provable expressivity? For example, can it universally approximate continuous equivariant function?

Author response:
Finally, you had a question on expressivity. As also responded to one of the other reviewers, *we now formally state that our networks are equivariant universal approximators*, that can represent any $SE(n)$ equivariant function. Since we consider equivariance to the continuous group $SE(n)$ this implies continuous functions as well. 

*We deeply thank you for this suggestion as we think this addition might further improve the impact of our work.*

Reviewer comment (Novelty):
So I think it could make the work stand out if we can find some applications that require to directly deal with signals on $\mathbb{R}^3\times S^2$.

Author response:
The second bullet indicates that the work could stand out with more applications that directly require working with signals on $\mathbb{R}^3 \times S^2$. As concrete applications we now explicitly mention its applicability to diffusion weighted MRI. In this modality MRI scanners directly obtain measurements related to the diffusivity of water molecules in the brain, as measured along different directions at each voxel location. As diffusivity of water is constraint by the presence and direction of neurons in the brain, this type of data is often used to perform neuron tractography. Other applications are curvature penalized geodesic tracking of bloodvessels (or neurons) in medical data. Both the DWI and tracking applications are discussed in the orientation-score section of the related work section. *Here we now make explicit mention of the diffusion weighted MRI application.* 

Another application could be the processing of meshes as point clouds. A mesh surface element could be represent by its centroid and it's normal direction, and thus it could be treated as a point cloud in position-orientaiton space. Another application is that of processing of molecules by doing message passing over the edges (each edge has a starting position and a direction towards its neightbor). Dimenet/Gemnet are examples of this. *In Appendix A, we further discuss the connection to the equivariant learning field and now explicitly mention the mesh and molecular edge cloud application.*

Reviewer comment (Writing):
Also just a minor personal idea: I think the word "weight sharing" makes me think of constructing linear transformation invariant/equivariant to certain symmetry. While here we are actually seeking for the full invariants describing the equivalence class.

Author response:
The third comment is on the terminology of "weight sharing", which often is discussed in terms of linear transformations that are equivariant or invariant to a certain symmetry. Thank you for this insightful remark! We agree that the term might evoke different associations with different audiences and hence decided to formally define it in Definition 3.3. 

The following may be familiar to you, but for the sake of completeness we would like to further clarify what we think the connection you have in might may refer to. 

The notion of symmetries in linear maps in fact motivated our work and is the underlying principle behind the "convolution is all you need" theorems cited in the paper. In context of signal analysis (as an example here we take feature maps over $\mathbb{R}^n$), a linear layer is an integral transform
$f^{out}(y) = \int_{\mathbb{R}^n} k(y,x) f(x) {\rm d}x$, parametrized by a two-argument kernel $k(y,x)$. Note that this is like the continuous equivalent of a matrix-vector multiplication $\mathbf{y} = \mathbf{K} \mathbf{x}$ between two finite dimensional vectors, here we consider infitinite dimensional vectors (functions). Now, when constraining this linear map to be translation equivariant, the transform is parametrized by a one-argument kernel via
$$f^{out}(y) = \int_{\mathbb{R}^n} k(x-y)f(x) {\rm d}x .$$
When the signal is one-dimensional and discretized on a grid, the corresponding matrix $\mathbf{K}$ is a circulant matrix (convolution can be implemented as a matrix-vector multiplication!). 

What we do in section 2.4 "Conditional Message Passing as Generalized Convolution" we build on this view point and say that any NN based on pair-wise interactions that are conditioned on invariants (like $x-y$) could be considered *non-linear* generalizations of convolutions.

Final remark, our construction is different to methods like for example Finzi 2021, which are about finding linear maps that are equivariant to a large class of groups. I.e., for vector spaces (as in MLPs) they derive the matrices that satisfy the equivariance constraint.

Finzi, M., Welling, M. and Wilson, A.G.. (2021). A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groups. *Proceedings of the 38th International Conference on Machine Learning*, in *Proceedings of Machine Learning Research* 139:3318-3328