Title: Projective Equivariant Networks via Second-order Fundamental Differential Invariants
Venue: NeurIPS 2025
Accepted: yes
Keywords: equivariant networks

differential invariants

projective group
Abstract: Equivariant networks enhance model efficiency and generalization by embedding symmetry priors into their architectures. However, most existing methods, primarily based on group convolutions and steerable convolutions, face significant limitations when dealing with complex transformation groups, particularly the projective group, which plays a crucial role in vision. In this work, we tackle the challenge by constructing projective equivariant networks based on differential invariants. Using the moving frame method with a carefully selected cross section tailored for multi-dimensional functions, we derive a complete and concise set of second-order fundamental differential invariants of the projective group. We provide a rigorous analysis of the properties and transformation relationships of their underlying components, yielding a further simplified and unified set of fundamental differential invariants, which facilitates both theoretical analysis and practical applications. Building on this foundation, we develop PDINet, the first framework for deep projective equivariant networks, achieving full projective equivariance without discretizing or sampling the group. Empirical results on the projectively transformed STL-10 and Imagenette datasets show that PDINet achieves improvements of 11.39\% and 5.66\% in accuracy over the respective standard baselines under out-of-distribution settings, demonstrating its strong generalization to complex geometric transformations.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a novel architecture equivariant under projective transformations (which are defined as the action of SL(3, R) on the plane via fractional linear transformations) based on differential features that are provably invariant. The paper focuses on the theoretical exposition of the recovery of the proposed family of differential invariants, though empirical evaluations are included. The appendix focuses on practical considerations regarding the implementation.\n\nSTRENGTHS AND WEAKNESSES:\nOverall, the authors method is relatively original, an extensive theoretical investigation is provided, and a simple neural module is proposed which serves as a building block and is empirically shown to be robust performance under the desired perturbations. On these criterion I feel this paper meets the bar for acceptance. \n\nA few key weaknesses are as follows:\n\nAs stated, it is not clear the authors proposed method generalizes to other projective groups, the most obvious similar case being the action of SL(2, C) on the (Riemann) sphere. (This action has been previously studied in https://arxiv.org/pdf/2201.12212, which is closely related to the authors work -- appearing even to define a class of SL(2, C) moving frames to facilitate equivariance, see section 5.1 -- but is not discussed in the related work section and should be). \n\nMost importantly, one of the authors main claims is that the proposed method is more efficient than previous approaches and is able to scale to higher resolution. This obviously appears to be true, and is demonstrated explicitly in experiments. However, the key practical benefit of this is that this should allow the model to actually be applied to real-world data. Unfortunately, such an experiment is lacking and in fact there is no evidence that projective equivariance is useful in practical problems other than being theoretically interesting. To be fair, such a motivation is not included in the work of MacLaughlan et al. nor any other paper I know of which seeks to take more challenging forms of equivariance. However, it begs the question of whether the authors work is necessary or useful from a practical perspective because the practical usefulness of projective equivariance is not justified in the literature.  \n\nSpecifically, the efficiency of the method means it can actually be applied to real-world data, and showing it can be effective in those cases, and outperform existing methods, is extremely important and what this paper is missing. If the authors can show something convincing along these lines, I will increase my rating. Essentially, all the ingredients for success are here, but it's up to the authors to be more ambitious and creative in their applications. Simply saying that the network holds promise for vision applications in the conclusion is not enough and is potentially misleading.\n\nQUESTIONS:\nSee above.\n\nLIMITATIONS:\nSee above -- in particular, whether the generalization to other projective groups (such as SL(2, C) ) is not discussed. \n\nThis concern has now been satisfactorily addressed in the authors' rebuttal.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper demonstrates a novel method to design a projective equivariant neural network. It is based on finding the projective differential invariants explicitly and use these features to perform learning. The experimental result demonstrates its effectiveness and improvements.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. Projective equivariance is an important type of equivariance, and there have not been many works addressing this. Having a projective equivariant design can be important for general vision problems. The significance and originality of this paper are good.\n2. This paper is theoretically strong, and the reviewer believes the derivation of differential invariants is correct.\n3. The experimental results demonstrate the initial performance and improvement of this method.\n\nWeaknesses:\n\n1. This paper is not formulated in its best. The derivation is important but could be moved to the appendix. The reviewer suggests that the authors move Figure 1 earlier in the paper.\n2. The paper demonstrates a promising strategy to enable projective equivariance, but the claim is not fully verified. The authors should find a way to visualize and verify that the map is equivariant. It is hard to fully understand what causes the improvement.\n3. The OOM issue might be resolved either by using a larger device or by reducing the sample size (if the reviewer understands the OOM error correctly).\n4. The experimental study and baseline comparison are relatively weak.\n\nQUESTIONS:\n1. What is the computational complexity/cost of these operations? \n2. Would the authors extend experiment/baseline study? How to verify the exact equivariance experimentally? \n3. What type of changes would happen when we act a random projective group action to an image? Would the authors be able to show a few example visualizations in the main paper (or the appendix)?  \n4. Could the authors be sure to include more equivariant network literature?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a deep neural network equivariant to projective transformations. The core contribution is the theoretical derivation of a complete and concise set of second-order fundamental differential invariants for the projective group. These invariants are derived using moving frames with a novel cross-section for multi-dimensional functions (e.g., multi-channel feature maps), which allows the use of second-order derivatives, unlike previous approaches for scalar functions that required more complex higher-order derivatives. The resulting neural network is sampling-free and thus does not have the scalability problem as previous work.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nThe paper could be the first work that tackles the problem of developing neural networks with projective equivariance, which is common in camera geometry, in a scalable way. The theoretical framework based on differential invariants is novel and solid. The framework leads to a scalable network implementation, which is impressive. \n\nThe paper is well-structured and well-written. The paper requires a substantial mathematical background to follow, but the authors did a good job explaining the background and the derivations. \n\nWeaknesses:\n\nLimited Experimental Comparisons: The primary baseline is a standard ResNet-18, which is not designed for geometric robustness. While the OOM issue with homConv is a valid point about scalability, the lack of any direct numerical comparison to it is a weakness. It would strengthen the paper to include a comparison on a shallower network where homConv could run. Furthermore, a comparison to a strong data augmentation baseline (i.e., ResNet-18 trained on projectively augmented data) is missing. Such a comparison helps disentangle the benefits of the equivariant architecture from the benefits of simply exposing the model to the test distribution's transformations during training. \n\nComputational Cost Analysis: The paper argues that its invariants are more concise and computationally efficient than alternatives. However, it lacks a practical analysis of the computational overhead. How does it compare with the original ResNet-18 in terms of training/inference time, memory consumption, and FLOPs?\n\nQUESTIONS:\nPlease address the mentioned weaknesses. \n\nBesides, how are first and second-order partial derivatives computed in practice within the network? Does the pixel discretization affect the accuracy of the computed invariants and the overall performance of the network?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper intrdocues PDINet, a network architecture that is equivariant to homographies. The key idea is to construct differential invariants of the projective group using the method of moving frames. Specifically, the authors derive a set of second-order differential invariants for multi-channel input functions, thus supporting RGB images and intermediate CNN features. These invariants are combined with small MLPs to produce learnable equivariant layers, which can be used as drop-in replacements for standard convolutional layers.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**  \n(+) It's really refreshing to see a apper tacking equivariance of projective transformations. I think it's a great research topic with potentially valuable outcomes in practice\n(+) Thue usage of second-order differential invariants derived from the moving frame method is elegant and novel. Removing the need to sample/discretize the group \n(+) As far as i could tell, the paper is mathematically sound\n(+) improvement over non equivariant baseline is significant\n\n**Weaknesses**  \n(-) the evaluation design is lacking. Usually, it is common to show the vanilla (non equivariant network) performance with augmentatinos (see works like vector neurons, or G-CNNs). Also, reporting the network results on a non transformed data would give a clue regarding the results themselves -- is 51% on STL-10 considered strong?\n(-) i'm also missing some analysis --  what are failure cases? do they make sense?\n(-) the baseline homConv is only showed with OOM -- is there a setup where its performance could be evaluated and conpared? \n(-) i'm missing a discussion of computational cost, stability is also of interest given the usage of second order derivatives \n(-) classificaiton is generally not a problem where homographies are of interest. It is more suitable for point correspondences -- if the authors would have showen this it would really strenghten the paper in my view\n\nQUESTIONS:\n* Can the authors report accuracy on the unwarped test?\n* Can the authors add a baseline where ResNet-18 is trained with random projective data augmentations?\n* can the authors discuss runtime, memory and training?\n* what would happen if the network is applied to a near projection mapping? say, in a multiview setting?\n\nLIMITATIONS:\nthe paper should put more emphasis on scalability and computational overhead", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}