Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper develops the theory of equivariant partial differential operators -- a steerable PDO which is equivariant under any given symmetry. Interestingly, this work reveals the relation between convolution and PDO by unifying them into a single framework. The work also provides the rigorous theoretical analysis and proof. The paper also provides experimental results to validate its theoretical analysis. 
main_review: The paper provides a very solid theoretical analysis for equivariant PDO. Different from other works on equivariant PDO, this paper enforces more properties of PDO such as divergence and gradient. Also some theoretical findings are useful to me. For example, the locality of PDO is not helpful. The PDO and convolution can be unified into a single framework.

However, I still have several questions and concerns which I will detail below:

- While theoretical findings are interesting, from my point of view, it’s not surprising and novel. For example, PDO is a specific form of convolution. I think this has been revealed by previous works in particular in the context of the diffusion process.

- Regarding the proposed equivariant PDO, I appreciate the deep theoretical analysis. However, I am not very attracted. First, I believe the PDO-eConv should have already revealed these analyses more or less. Second, the equivariant PDO is worse than other works experimentally. In other words, I am not very sure of the significance of the proposed PDO to our community given the limited performance and novelty.

- Experimental analysis is also very limited. What are the advantages of the proposed PDO compared with previous extensive steerable convolution? I recommend having more results. 

- Do you have any insight why the performance of the proposed PDO is worse than convolution? Is that potentially because of the isotropic nature of PDO? If so, I recommend the analysis in this direction. 

summary_of_the_review: The paper theoretically develops the steerable PDO which enlarges the family of equivariant operators. However, the proposed method is quite limited in terms of performance. Also, the theoretical findings are not very surprising to me given the previous literature. Thus, I doubt the significance of this work to our community. But in case of any misunderstanding, I would like to hear more from the authors during rebuttal. 

> Post-discussion: the authors addressed my concerns. So I'm very happy to improve my rating.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper introduces equivariant Partial Differential Operators as a drop in replacement for equivariant kernels in $G$-steerable networks. The paper provides a thorough treatment on the subject matter and gives the necessary equivariant constraints for PDO's in a similar flavor to $G$-steerable kernels. In addition, explicit solutions to these constraints are provided for subgroups of $O(2)$, $O(3)$ and $SO(3)$. Finally, the paper also empirically validates equivariant PDO's on rotation MNIST and STL-10 highlighting important discussion points on the discretization error.
main_review: This paper is exceptionally well written and covers an impressive body of work across multiple different domains. Personally, I was extremely impressed with the production quality and thoroughness of the Appendix as well as the ambitious research goal of introducing equivariant PDOs in the main paper. The main technical results are both novel and illuminating and gracefully extends prior work on $E(2)$-CNNs and their theory.

I do not have many strong criticisms regarding the work but rather a few minor questions and comments. The largest of which is that while $G$-steerable equivariant networks guarantee equivariance between layers there exists another body of work that achieves equivariance in a slightly different manner. Specifically, prior works also achieve equivariance in expectation using a Monte-Carlo estimation of the group convolution integral (see [1]). These approaches differ in that they do not treat data as associated vector bundles and also equivariance is achieved over functions on the group $G$ itself necessitating a lifting operation. While this school of thought is different than the $G$-steerable variety as presented in this paper it would be great to harmonize both directions with a discussion on their differences especially when it comes to practical applications. Finally, the paper could be strengthened a bit more if the discussion on the discretization error for the various discretization methods was elevated from the appendix to the main paper. It would be good here to get some numerical quantification of the errors for the datasets and architectures considered as this is an important insight which informs practice.

***Minor Comments***
- I had a bit of a hard time understanding how a group element $g$ acts on a polynomial $x$ in the main paper. Could the authors provide a bit more exposition---if it doesn't already exist---in an appendix?
- The notation in section 2.1 was a bit confusing. Specifically, $c_{\alpha}: \mathbb{R}^d \to \mathbb{R}$ but right after proposition 1. $c_{\alpha} \in \mathbb{R}$. Is this an artifact of picking constants due to enforcing translational equivariance?


[1] Finzi, Marc, et al. "Generalizing convolutional neural networks for equivariance to lie groups on arbitrary continuous data." International Conference on Machine Learning. PMLR, 2020.
summary_of_the_review: The paper studies equivariant Partial Differential Operators towards building equivariant networks. The work is both novel, interesting, and brings new perspectives to the equivariant networks literature while maintaining an exceptional level of polish in both writing and execution.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The authors developed the general theory of equivariant partial differential operators (PDOs) between feature fields on the Euclidean space. Given an arbitrary group G, the G-steerability constraint is derived that fully characterized when a PDO is equivariant between vector fields for given representations. Experiments on the rotated-MNIST and STL-10 datasets have been conducted to compare the performance of equivariant-PDOs and equivariant steerable kernels.
main_review: Strength:
+ The paper is very rigorously and well-written. The layout of the paper is very clean.
+ The paper tries to bridges the idea between deep learning and physics, drawing attention in particular to convolution between features and PDOs between vector fields.
+ Even though the derivation of the PDO G-steerablility constraint follows similarly from that of kernel steerability constraint, the theory developed is interesting. In particular, it is interesting to see a special example in section 2 that equivariant PDOs are combinations of well-known operators such as Laplacian and divergence.
+ Connections between convolutions and PDOs have also been made through Schwartz distributions in the appendix, which I have to admit that I have not read in full.

Weakness:
- It seems that in Table 1 and 2, equivariant PDOs typically underperform steerable CNNs. The authors did explain the reason being the locality of PDOs, but such underwhelming results do in some sense limit the utility of the developed equivariant PDOs.
- It would be great to see examples where PDOs have a significant advantage over steerable CNNs.
summary_of_the_review: The theoretical contribution of the paper is significant. Even though experimental results are not as satisfactory to demonstrate the utility of equivariant PDOs, I am leaning towards accepting the paper.
review_title: null

Meta-review: The paper develops steerable partial differential operator and show how it can be used to build equivariant network. Experimentation on rotated MNIST and STL10 show the merits of the proposed method. Reviewers agreed on the significance of the work and that it brings new perspective on equivariance that would be interesting to the ICLR community. Accept
Decision: Accept (Poster)