ATLAS: Automatic Local Symmetry Discovery

Authors: Manu Bhat, Jonghyun Park, Jianke Yang, Nima Dehmamy, Robin Walters, Rose Yu
Primary area: learning on graphs and other geometries & topologies
Keywords: local symmetry discovery, symmetry discovery, equivariance, gauge equivariant neural network, Lie theory

Abstract:
Existing symmetry discovery methods predominantly focus on global transformations across the entire system or space, but they fail to consider the symmetries in local neighborhoods. This may result in the reported symmetry group being a misrepresentation of the true symmetry. In this paper, we formalize the notion of local symmetry as atlas equivariance. Our proposed pipeline, automatic local symmetry discovery (ATLAS), recovers the local symmetries of a function by training local predictor networks and then learning a Lie group basis to which the predictors are equivariant. We demonstrate ATLAS is capable of discovering local symmetry groups with multiple connected components in top-quark tagging and partial differential equation experiments. The discovered local symmetry is shown to be a useful inductive bias that improves the performance of downstream tasks in climate segmentation and vision tasks.

Decision: reject
Reviewer scores: 6, 6, 3, 5, 6

Reviews and discussion:
<thread>
creation date:
1730240048496

thread_type:
official_review

<thread_object>
creator: reviewer_51d4

<content>
summary:
Formalizing the notion of local symmetry as atlas equivariance, the authors proposed an automatic local symmetry discovery approach (ATLAS) to recover the local symmetries of a function using local predictor networks. They learned a Lie group basis to ensure that the predictors are equivariant, resulting in the discovery of local symmetry groups with multiple connected components. This approach was tested in experiments on top-quark tagging, synthetic partial differential equations, classification, and climate segmentation.

strengths:
**S1.** The paper is very well-written and presented, with a well-defined motivation for studying local symmetries. 

**S2.** The development of a method to discover atlas equivariance in a given data distribution in the form of a Lie group is unique. 

**S3.** The experiments covered diverse datasets, including tasks like classification and segmentation, demonstrating the model's applicability across various domains.

weaknesses:
**W1.** The assumption of limiting the target group to a finite number of connected components requires further justification and additional experimentation, if possible. Lie groups can have an infinite number of components in some of the extensively researched areas like diffeomorphisms. I assume that this approach does not hold for such cases, which could limit the generalizability of the method to certain pre-defined knowledge. I would encourage the authors to shed more light on this limitation and to experimentally validate why the model might fail to converge, as they have suggested.

**W2.** As the downstream test results show very minor differences between the baselines and the proposed approach, what are the advantages of using this approach? Also, I would suggest the authors to experiment with some real-world image datasets.

**W3.** While the authors extensively focused on the inability of existing SOTA models to capture local neighborhoods, it would be great if the authors could experimentally compare them and validate this, specifically with models such as (Moskalev et al., 2022), (Dehmamy et al., 2021), LieGAN, and similar works that capture global symmetries. Comparing these models would effectively validate the authors' assumptions.

**W4.** I believe the incorporation of the symmetries discovered by ATLAS leading to parameter efficiency has not been validated. I request the authors to discuss how they ensure effective parameter efficiency with their proposed approach in the rebuttal.

**W5.** How can we randomly sample a localized function $\phi_c$ from $\phi$? Do we need to consider the random bias that might be introduced through the random sampling of $\phi_c$? Besides, is there any potential adversarial impact of the randomly sampled coefficient vector $\eta$ in terms of deriving the group element $g$?

**Minor**
- How the parameter $\epsilon$ is being selected? An ablation would be great to analyze the robustness of this parameter over the model's performance.


While there are some experimental concerns and methodological confusion, the current rating reflects the accumulated efforts in problem definition, motivation, presentation, and diverse experimentation. I strongly suggest that the authors consider all of the findings in their rebuttal. Good luck.

questions:
I tried to cover most of my concerns and questions in the *Weaknesses* section. I kindly request the authors to review that section.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730574967309

thread_type:
official_review

<thread_object>
creator: reviewer_jszk

<content>
summary:
In this paper, the authors proposed a new method called atlas to capture symmetries in local neighborhoods. Local symmetries of a function is recovered by training local predictor networks and learning a Lie group basis to ensure the learned predictors are equivariant. To illustrate the efficacy of the proposed method, the authors experimented on a few well-designed tasks.

strengths:
1. The problem of local symmetry may have some specific applications as the authors presented in the experiment section. The proposed method might be a good complement to the global sysmmetry discovery methods.

weaknesses:
1. Motivation of this work is not well clarified. Since existing equivariant models can achieve global symmetries, then local symmetries are also guaranteed at the same time. The authors are encouraged to show important problems or areas where we do not care about global symmetries, but insteand local symmetries are more important.

2. Theoretically I am not seeing advantages of the proposed method over existing methods. The authors are encouraged to clarify novelty of this work and present more analysis on why the proposed method is superior to existing methods.

questions:
1. Since Gauge equivariant neural networks could enforce local symmetries, what are the major differences between the proposed atlas and the Gauge equivariant neural networks? Does theorem 1 mean they are equivalent under certain conditions?   

2. In Algo.1 the authors train a predictor network for each chart. What are the scales of the chart? Talking about local symmetries, I assume the locality can be referred at different scales. It also depends on the problem/data themselves. 

3. What are the trade-offs when we optimize local symmetries using atlas? Do we lose global symmetries? How could we balance these two objectives?

4. Fig. 9, can the authors explain a little bit more on why the proposed method is superior to the LieGAN method? Visually the differences are subtle to me.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730648874019

thread_type:
official_review

<thread_object>
creator: reviewer_sqyp

<content>
summary:
This paper presents ATLAS, a method for discovering local symmetry. It defines atlas locality and atlas equivariance to formalize local symmetry. ATLAS is based on trained predictors, sequentially treating Lie algebra bases and coset representatives as learnable parameters, with the optimization objective being the equivariant loss $\mathcal{L}(\Phi_c(g \cdot x), g \cdot \Phi_c(x))$. It then compares the relationship between gauge equivariance and atlas equivariance. Experimental results demonstrate that this method can correctly identify local symmetry in the task.

strengths:
This paper focuses on the unique problem of local symmetry discovery and formally defines atlas equivariance to distinguish between global symmetry and local symmetry. In addition to Lie algebra bases, ATLAS can learn coset representatives, thereby discovering discrete symmetry.

weaknesses:
- I disagree with the core argument of this paper: previous methods for symmetry discovery could only find global symmetry rather than local symmetry. In Algorithm 1, the atlas $\mathcal{A}$ of the manifold $\mathcal{M}$ needs to be specified manually. However, after completing this step, we can treat the symmetry discovery for each chart $(U_c, \varphi_c)$ as separate subproblems, with their common symmetry being the atlas symmetry. These subproblems do not differ from global symmetry discovery, as mentioned in Definition 2, where each mapping $\Phi_c$ is globally equivariant. Therefore, previous methods for symmetry discovery can be naturally extended to the case of local symmetry. For example, we can apply LieGG [1] on each trained predictor $\Phi_c$, or apply LieGAN [2] on the data of each chart $\mathcal{D_c} = \\{(X_i: U_c \rightarrow \mathbb{R}^{d_{in}}, Y_i: U_c \rightarrow \mathbb{R}^{d_{out}})\\}_{i=1}^{n_c}$.

- ATLAS simply treats group generators as learnable parameters and optimizes the equivariant loss $\mathcal{L}(\Phi_c(g \cdot x), g \cdot \Phi_c(x))$. I think that this approach has limited contributions to the problem of symmetry discovery. It does not show significant advantages over previous methods and has some drawbacks. For example, compared with LieGG [1], which mathematically solves Lie algebra bases, ATLAS has poorer interpretability; compared with LieGAN [2], which directly discovers symmetries from datasets, ATLAS is highly dependent on the accuracy of trained predictors. Furthermore, in Appendix B, it is mentioned that the number of Lie algebra bases $k$ is determined by incrementally increasing from a minimum value. Will this lead to significant computational overhead when the true $k$ is large? If the norm of the weakest generator varies continuously, will changes in the set threshold also affect the final result for $k$?  

- The reason why the method can work lacks rigorous explanation. See the first two questions for details.

- Suggestion on presentation: "atlas" refers to a collection of local regions or charts that cover a manifold, while "ATLAS" stands for automatic local symmetry discovery. The use of the same word, differing only in capitalization, may confuse readers.

**References**  

[1] Moskalev, Artem, Anna Sepliarskaia, Ivan Sosnovik, and Arnold Smeulders. "Liegg: Studying learned lie group generators." Advances in Neural Information Processing Systems 35 (2022): 25212-25223.  

[2] Yang, Jianke, Robin Walters, Nima Dehmamy, and Rose Yu. "Generative adversarial symmetry discovery." In International Conference on Machine Learning, pp. 39488-39508. PMLR, 2023.

questions:
- Does Equation (3) in Section 4.2.1 (calculating the absolute value first and then the cosine similarity) have any practical significance mathematically? I am skeptical about whether it works in all cases. Can you provide a theoretical proof that when $\mathcal{L}_{str}(B)$ is minimized, the number of non-zero elements shared among different basis vectors is also minimized?

- In Section 4.2.2, when learning coset representatives, even though the number of random matrices $K$ is much larger than the actual number of cosets, will all these matrices converge to the strong symmetry embodied by the predictor $\Phi_c$? Formally, if for $\forall C \neq C_1, C_2$, we have $\mathcal{L}(\Phi_c(C_1 \cdot x), C_1 \cdot \Phi_c(x)) < \mathcal{L}(\Phi_c(C_2 \cdot x), C_2 \cdot \Phi_c(x)) \ll \mathcal{L}(\Phi_c(C \cdot x), C \cdot \Phi_c(x))$ (where we omit the normalization notation for brevity). In this case, it is possible that all matrices converge to $C_1$ while missing the symmetry corresponding to $C_2$.

- In Section 5.2, does the $\mathrm{SO}(2)$ transformation refer to rotating the entire field around the origin of the coordinate system, or does it refer to rotating each chart separately around its center point? In Figure 6b, which charts correspond to local predictors $\Phi_c$ that exhibit global symmetry?

- In the assumptions of Theorem 1, it is already mentioned that $M$ is $G$ gauge equivariant. Is it necessary to further elaborate that $M$ is $G$ equivariant in the third paragraph of the proof? When the manifold is Euclidean space, are gauge equivariance and equivariance naturally equivalent?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730696829110

thread_type:
official_review

<thread_object>
creator: reviewer_cuzv

<content>
summary:
This work proposed ATLAS to recover local symmetries (both continuous and discrete) from a dataset. By incorporating the symmetries discovered by ATLAS in a gauge equivariant CNN can lead to better performance and parameter efficiency.

strengths:
1. The figures provided in the paper are of high quality, easy to understand

2. The algorithm proposed is straightforward and concise.

weaknesses:
1. The motivation for discovering local symmetries through ATLAS is not clearly articulated. The current description is somewhat abstract, which may make it challenging for general readers to understand. It would be helpful to clarify what the symmetries represent in the context of the three experiments. Are these symmetries intended primarily to enhance performance in gauge-equivariant CNNs, or do they have a broader purpose?

2. While the three experiments in Section 5 provide concrete examples, the connections between group actions, cosets, and the datasets are not immediately apparent. It would be beneficial for the authors to explicitly state these connections within each experimental subsection to enhance clarity.

3. The objective of the experimental section lacks clarity. If the primary goal is benchmarking, the comparisons with other methods appear limited. On the other hand, if the goal is to illustrate the advantages of local symmetries over global symmetries, it remains unclear how these local symmetries contribute to downstream tasks. Providing more specific insights into the role and utility of local symmetries in these tasks would improve the section’s effectiveness.

questions:
See weakness
</content>
</thread_object>

</thread>
<thread>
creation date:
1730712835604

thread_type:
official_review

<thread_object>
creator: reviewer_h7l3

<content>
summary:
The paper presents ATLAS (Automatic Local Symmetry Discovery), an approach for discovering local symmetries in various data forms.  The authors provide formal descriptions of the locality and equivariance of ATLAS, along with algorithms for discovering infinitesimal generators and discrete symmetries. Results from these experiments show that integrating the discovered symmetries into downstream models can enhance performance. Through experiments involving top-quark tagging, synthetic PDEs, MNIST classification, and climate segmentation, the authors demonstrate the effectiveness of ATLAS in discovering local symmetries and integrating them into gauge equivariant networks, leading to improved performance metrics.

strengths:
1. The paper is well-motivated, presenting a new conceptual framework. The authors validate their approach across multiple tasks and demonstrate its effectiveness.

weaknesses:
1. The definitions of Atlas Locality and Atlas Equivariance both contain the term "Atlas," an abbreviation for the authors' method, i.e., Automatic Local Symmetry Discovery. Defining a mathematical object using the abbreviation of a specific method is uncommon and could lead to confusion. Additionally, the authors do not clearly differentiate atlas locality from general locality, nor do they clarify how atlas equivariance differs from equivariance to conventional local transformations.
2. While the authors emphasize the importance of local equivariance, many deep learning tasks, such as natural language processing and video understanding, exhibit long-range dependencies, highlighting the coupling of local and global equivariance. Is the proposed framework applicable in these scenarios?
3. The authors report only the final results of their complete method in each experiment, lacking ablation studies and fine-grained visualizations of features to provide a more credible explanation of their approach.
4. The descriptions of experimental details are somewhat lacking, particularly concerning the discovery of symmetry and its integration with neural networks on specific datasets.
5. The paper conducts many experiments, and for the MNIST classification task, it projects the data onto a spherical space. What are the physical meanings or practical applications of this projection? 
6. Although the paper provides the steps of the ATLAS algorithm, there is limited analysis on the time and space complexity of the algorithm. It is recommended to include this section to assess its feasibility for applications on large-scale datasets.

questions:
see Weaknesses in the previous section
</content>
</thread_object>

</thread>
<thread>
creation date:
1732129027760

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Response to the Reviewers

comment:
We appreciate the feedback and detailed comments left by the reviewers. We are encouraged that reviewers find local symmetry discovery to be a novel problem (R3) with practical applications (R4). We are also pleased that our paper is considered well-motivated (R1,5) with proper presentation (R2,5) and demonstrates effective results across various domains (R1,5). 


We have revised the paper to address some of the questions posed by reviewers. The major changes include an analysis of the time and space complexity of ATLAS, a comparison to LieGG, additional experiments as well as ablations, and a proof regarding the minimization of the standard basis loss. These changes can primarily be found in the Appendix and are highlighted in red.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper presents an automatic local symmetry discovery method. Experiments on several tasks demonstrate the effectiveness of the proposed method. After the rebuttal, it receives mixed ratings, including three borderline accept with relatively low confidence, one borderline reject with high confidence, and one reject with high confidence. The advantages, including the clear motivation, straightforward and concise algorithm, and good results,  are recognized by the reviewers. However, there are still several concerns that need to be addressed, including the insufficient contribution, unconvincing analysis about the improvement and theoretical advantages, and so on. I think the current manuscript does not meet the requirements of this top conference. I suggest the authors carefully revise the paper and submit it to another relevant venue.