Title: Reducing Symmetry Increase in Equivariant Neural Networks
Primary area: learning on graphs and other geometries & topologies
Keywords: Equivariant Neural Networks, Symmetry Increase, Compact Group, Isotropy Subgroup, Orbit Type, Curie’s Principle

Abstract: Equivariant Neural Networks (ENNs) have enpowerred numerous applications in scienctific fields. 
Despite their remarkable capacity for representing geometric structures, ENNs suffer from degraded expressivity when processing symmetric inputs: the output representations are invariant to transformations that extend beyond the input's symmetries. The mathematical essence of this phenomenon is that a symmetric input, after being processed by an equivariant map, experiences an increase in symmetry. While prior research has documented symmetric increase in specific cases, a rigorous understanding of its underlying causes and general mitigation strategies remains lacking. In this paper, we provide a detailed and indepth characterization of symmetry increase: (i) For any given feature space and input symmetry group, we prove that the increased symmetry admits an infimum determined by the structure of the feature space; (ii) Building on this foundation, we develop a computable algorithm to derive this infimum by calculating the orbit types within the feature space. (iii) Under standard regularity assumptions, such as the manifold hypothesis, we demonstrate that for most equivariant maps (or parameterized maps with sufficient approximation capabilities), our algorithm can effectively mitigate symmetry increase.
To complement our theoretical findings, we provide visualizations and experimental results to illustrate the proposed concepts and validate our theoretical contributions.