Axis-level Reflectional Symmetry Detection with Group-Equivariant Representation

Authors: Won-gyun Yu, Ahyun Seo, Minsu Cho
Primary area: applications to computer vision, audio, language, and other modalities
Keywords: Symmetry detection, Equivariant learning, Group equivariance

Abstract:
Reflectional symmetry detection remains a challenging task in machine perception, particularly in complex real-world scenarios involving noise, occlusions, and distortions. We introduce a novel equivariant approach to axis-level reflectional symmetry detection that effectively leverages dihedral group-equivariant representation to detect symmetry axes as line segments. We propose orientational anchor expansion for fine-grained rotation-equivariant analysis of diverse symmetry patterns across multiple orientations. Additionally, we develop reflectional matching with multi-scale kernels to extract effective cues of reflectional correlations, allowing for robust symmetry detection across different receptive fields. Our approach unifies axis-level detection with reflectional matching while preserving dihedral group equivariance throughout the process. Extensive experiments demonstrate the efficacy of our method while providing more accurate axis-level predictions than existing pixel-level methods in challenging scenarios.

Decision: reject
Reviewer scores: 5, 6, 5

Reviews and discussion:
<thread>
creation date:
1730596573048

thread_type:
official_review

<thread_object>
creator: reviewer_ygvv

<content>
summary:
The paper proposes a group-equivariant neural network for axis-level reflectional symmetry detection。The authors introduce orientational anchor expansion for fine-grained rotational equivariant analysis of different symmetry patterns across multiple orientations. Additionally, the paper develops reflectional matching with multi-scale kernels, enabling robust symmetry detection across various receptive fields. Experimental results demonstrate the effectiveness of the proposed method.

strengths:
1.	The idea is interesting. Compared to existing methods, which primarily treat reflectional symmetry detection as a pixel-level heatmap prediction problem, this paper classifies the presence of a mid-point of a reflectional symmetry axis for each pixel position and also regress the angle and length of the axis , directly performing axis-level prediction. 

2.	Extensive experiments validate the effectiveness of the proposed method, providing more accurate axis-level predictions than existing pixel-level methods.

3.	The paper is well-organized, making it easy and quick to follow.

weaknesses:
1.	The literature review is incomplete. The cited references are almost entirely from 2022 and earlier (with only one paper from 2023 and none from 2024), raising questions about the novelty of the work.

2.	The proposed multi-scale expansion has already been widely explored and proven effective in tasks such as object detection and segmentation.

questions:
1.	In Line 521, you claim that “In Fig. 4, F1-scores for all three methods are plotted across different distance thresholds,” but Fig. 4 only shows two methods, missing the presentation of PMCNet.

2.	The proposed method adapts a line detection network and applies it to the reflectional symmetry detection task. Can the adaptation strategy be effective on other line detection networks?

3.	Could you provide comparison results with existing methods on other datasets (such as SDRW[1] and LDRS[2]) to fully demonstrate the superiority of the proposed method? 

4.	What are the application scenarios, research value, and significance of this study?

[1] Liu, Jingchen, et al. "Symmetry detection from realworld images competition 2013: Summary and results."
[2] Seo, Ahyun, Woohyeon Shim, and Minsu Cho. "Learning to discover reflection symmetry via polar matching convolution."
Flag For Ethics Review: No ethics review needed.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730643669234

thread_type:
official_review

<thread_object>
creator: reviewer_72w3

<content>
summary:
This paper presents a novel axis-level reflectional symmetry detection network that leverages dihedral group-equivariant representations to improve the detection of symmetry axes in images. The authors introduce an orientational anchor expansion method for fine-grained, rotation-equivariant analysis across multiple orientations, enhancing the model's ability to detect diverse symmetry patterns. They also develop a reflectional matching module using multi-scale kernels to capture reflectional correlations across different receptive fields, improving robustness. Extensive experiments demonstrate that the proposed method outperforms existing pixel-level approaches in challenging scenarios, establishing a new benchmark in reflectional symmetry detection. The work offers a fresh perspective and significant contributions to the field of symmetry detection.

strengths:
* The paper introduces an innovative axis-level reflectional symmetry detection method based on dihedral group-equivariant representations.
* The proposed orientational anchor expansion and reflectional matching modules effectively enhance the model's detection capabilities across various orientations and scales.
* The method demonstrates strong robustness and generalization in complex real-world scenarios.
* The paper provides clear explanations of complex concepts and methodologies, aiding reader comprehension.

weaknesses:
* The implementation details for orientational anchors could be expanded to clarify their integration within the broader architecture and their impact on computational efficiency.
* While multi-scale reflectional matching is beneficial, further analysis on the trade-off between accuracy and computational overhead would improve the study.
* The model’s applicability to continuous symmetries, such as ellipses or curved patterns, is limited, which may constrain its use in certain symmetry-dense applications.
* The dependency on pre-defined kernels in multi-scale matching might limit adaptability to unknown scales or orientations in real-time applications.
* The paper lacks evaluation of the method's generalization performance on different datasets, which could limit its applicability to other scenarios.

questions:
* The implementation details for orientational anchors could be expanded to clarify their integration within the broader architecture and their impact on computational efficiency.
* The model’s applicability to continuous symmetries, such as ellipses or curved patterns, is limited, which may constrain its use in certain symmetry-dense applications.
* The paper lacks evaluation of the method's generalization performance on different datasets, which could limit its applicability to other scenarios.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730939646285

thread_type:
official_review

<thread_object>
creator: reviewer_qqr5

<content>
summary:
This paper presents a reflection symmetry detection system with matched, multiscale kernels rotationally equivariant network.  The work uses equivalent networks to allow symmetries to be detected at rotations.   The authors also use a fiber based approach to directly find the symmetry rather than first predicting a heatmap for the symmetry.

strengths:
The paper is using rotational equivariant networks to improve symmetry detection.  It’s an interesting approach (though needs to be sufficiently distinguished from others in the field). 

This paper is clearly written and I can follow the logic on what they are trying to do.

I appreciate the approach with fibers and it seems interesting to not use a dense approach.    I like the difference in the approach and would like to see more of that with different backbones since I think it would work even better.

weaknesses:
Major:
For group representation and a longer background of symmetry detection and needs to be cited here is Computational symmetry in computer vision and computer graphics by Yanxi Liu et al.  2010

The evaluation only compares against a recent method and doesn’t go back to any of the previous methods (check out Funk et al 2017 for a list of methods where most are freely available online).  They are used in the papers the authors compared with: Seo, Ahyun, Woohyeon Shim, and Minsu Cho. "Learning to discover reflection symmetry via polar matching convolution." Proceedings of the IEEE/CVF international conference on computer vision. 2021.  In addition, why have you deviated from the standard precision recall with F1 marked curve like the previously mentioned papers?  That is really useful to understand how your metric compares to others.  

This paper (which cited thoughout the paper), also uses equivariant networks for both rotation and reflection symmetry detection.   Seo 2021 and 2022 both use equivariant kernels with Seo 2022 uses rotation group equivalents.  The main difference between the group equivalence of the other papers, at least that I understand, is that the group d8 is used rather than a special Euclidean symmetry group out of the 17.  I'm only referring to the difference in the equivariant and not other differences in the approach.  


Missing citations
Gens, R. and Domingos, P. Deep Symmetry Networks. NeurIPs, 2014.  proposed equivariant convolutional arch that needs to be cited and compared with.  
Rotationally-invariant CCNs - Dieleman, Sander, Kyle W. Willett, and Joni Dambre. "Rotation-invariant convolutional neural networks for galaxy morphology prediction." Monthly notices of the royal astronomical society 450, no. 2 (2015): 1441-1459.

Authors should mention that this is equivariant NN for just 2D data or other papers such as “Equivariant Multi-View Networks. Carlos Esteves et al.  ICCV 2019” should be cited.  

A figure to help understand Sections 3 and 4 would be helpful for understanding what you are getting at here visually.  There is a lot of text and equations and I think a figure to get at the expansion of fibers and how the author sare using symmetry groups would be a big help.


Minor:
First paragraph needs citations.  You can’t just state facts in a paper without a citation on symmetry being a fundamental concept.  You can go back to Gestalt theory or how symmetry detection is prevalent in the animal kingdom but cite it.  

In Figure 1, (a) (b)... needs to be labeled in the image.  This is hard to follow

questions:
The authors are basing this on Cohen et al.’s work for equivariant cnns and I’m not sure how is this different?  Those filters are already rotational equivariant based on the symmetry groups they represent.  

“Lenc & Vedaldi (2015) show that the AlexNet CNN (Krizhevsky et al., 2012) trained on imagenet spontaneously learns representations that are equivariant to flips, scaling and rotation.” from Group Equivariant Convolutional Network.  Why would this approach be necessary for rotational symmetry invariance for reflection symmetry detection?

How large is the kernel size?  If using too small a size, how can it be 8-fold symmetric?

How well does this objective function work on non-changed neural networks?  What about modern networks like Convnext?

Line 236: why D8 and not some other amount of rotations?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732163828403

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Response to All Reviewers

comment:
We sincerely thank the reviewers for taking the time to provide such thoughtful and constructive feedback on our manuscript. We have carefully addressed the comments in our responses, and incorporated additional details into both the main text and supplementary materials, marked in red for easy reference. We would greatly appreciate if the reviewers could verify these modifications.

The key modifications are as follows:
- Added Supplementary Material C.1 about evaluation on SDRW and LDRS datasets (including PR curves)
- Added Supplementary Material C.2 about comparison with previous methods (including PR curves)
- Added Supplementary Material C.3 about ablation studies with different backbones
- Added Supplementary Material C.4 about ablation studies on line segment detection
- Added Supplementary Material C.5 about ablation studies on computational overhead
- Added Supplementary Material D about architecture details and additional figure (Figure 7)
- Added citations (L28, L29, L30, L31, L48, L49, L50, L105, L106, L107, L194)
- Updated Figure 1 with alphabetical notations and revised visual elements, supported by color-coded indications for better clarity
- Added implementation details of frozen backbone (L404)
- Updated terminology from "multi-scale expansion" to "multi-scale matching" to better reflect our method (L342, L426, Figure 1, Table 1)
- Clarify our method is for 2D reflection symmetry detection (L52, 58, 196)

We hope these revisions have helped to clarify our work in response to the reviewers' valuable feedback.
</content>
</thread_object>

</thread>
<thread>
creation date:
1733289914969

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Final Comment by Authors

comment:
Dear Reviewers, Area Chairs, Senior Area Chairs, and Program Chairs.
We sincerely thank all reviewers for their thorough evaluation and constructive feedback throughout the review process, and all the support of Area Chairs, Senior Area Chairs, and Program Chairs for managing the review process. We are pleased that we were able to address most of the reviewers' concerns through our detailed responses:

- In response to Reviewer qQr5, we provided extensive comparison results including PR curves and comprehensive comparisons with previous methods([1, 2, 3, 4, 5, 6]) on multiple datasets (DENDI[1], LDRS[2], and SDRW[7]). We incorporated fundamental symmetry detection works including Gestalt theory and computational symmetry studies([8, 9, 10, 11, 12]), and demonstrated that our method outperforms modern backbones such as ConvNeXt[13] through detailed ablation studies. We clarified our method's contributions from an equivariance perspective through detailed comparisons and analysis, and enhanced the understanding of our method by improving existing figures and adding a new figure illustrating the fiber expansion and orientational anchor mechanisms.

- In response to Reviewer 72W3, we provided a comprehensive analysis of computational efficiency and architecture details, including a new figure demonstrating the integration of orientational anchors within the broader architecture. We also conducted evaluations across multiple datasets (DENDI[1], LDRS[2], and SDRW[7]), with a detailed study on performance characteristics such as precision-recall trade-offs and F1 scores at various distance thresholds.

- In response to Reviewer YGvV, we expanded our evaluation to additional benchmarks (LDRS[2] and SDRW[7]), demonstrated our method's effectiveness on the Wireframe dataset[14], clarified terms about multi-scale matching, and provided examples of practical applications across medical imaging[15], robotics([16, 17]), and computer vision domains([18, 19]).

We believe our paper presents valuable contributions to the field of reflectional symmetry detection, as evidenced by our experimental results and the positive feedback received during the review process.

Thank you for your time and consideration.

Best regards,

Authors


**References**
1. Seo, Ahyun, et al. "Reflection and rotation symmetry detection via equivariant learning." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022
2. Seo, Ahyun, et al. "Learning to discover reflection symmetry via polar matching convolution." ICCV 2021
3. Shen, Wei, et al. "Object skeleton extraction in natural images by fusing scale-associated deep side outputs." CVPR 2016
4. Teo, Ching L., Cornelia Fermuller, and Yiannis Aloimonos. "Detection and segmentation of 2D curved reflection symmetric structures." ICCV 2015
5. Tsogkas, Stavros, and Iasonas Kokkinos. "Learning-based symmetry detection in natural images." ECCV 2012
6. Gareth Loy et.al. "Detecting symmetry and symmetric constellations of features." ECCV 2006.
7. Liu, Jingchen, et al. "Symmetry detection from realworld images competition 2013: Summary and results." Proceedings of the IEEE conference on computer vision and pattern recognition workshops. 2013.
8. Wertheimer, M. "Laws of organization in perceptual forms." In A source book of Gestalt psychology, pages 71-88. Routledge & Kegan Paul, London, 1938.
9. Tyler, C. W. "Empirical aspects of symmetry perception." Spatial Vision, 9(1):1-7, 1995.
10. Møller, A. P. and Thornhill, R. "Bilateral symmetry and sexual selection: a meta‐analysis." The American Naturalist, 151(2):174-192, 1998.
11. Giurfa, M., Eichmann, B., and Menzel, R. "Symmetry perception in an insect." Nature, 382(6590):458-461, 1996.
12. Wagemans, J. "Detection of visual symmetries." Spatial vision, 9(1):9-32, 1995.
13. Liu, Zhuang, et al. "A convnet for the 2020s." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022.
14. Huang, Kun, et al. "Learning to parse wireframes in images of man-made environments." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018.
15. Yu, Chen-Ping, et al. "3D blob-based brain tumor detection and segmentation in MR images." IEEE ISBI, 2014.
16. Li, Wai Ho, Alan M. Zhang, and Lindsay Kleeman. "Bilateral symmetry detection for real-time robotics applications." The International Journal of Robotics Research 27.7 (2008): 785-814.
17. Mo, Ningkai, et al. "ES6D: A computation-efficient and symmetry-aware 6D pose regression framework." CVPR, 2022.
18. Yang, Mingkun, et al. "Symmetry-constrained rectification network for scene text recognition." CVPR, 2019.
19. Mitra, Sinjini, and Yanxi Liu. "Local facial asymmetry for expression classification." CVPR, 2004.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This submission received two negative sores and a positive score after rebuttal. The remaining major concerns are about the weaker performance and unclear explanation of the proposed approach. After carefully reading the paper, the review comments, the AC can not recommend the acceptance of this submission, which is based the fact that the average score is under the threshold bar. The AC also recognizes the contributions confirmed by the reviewers, and encourages the authors to update the paper according to the discussion and submit it to the upcoming conference.