Paper: Warped Convolutional Neural Networks For Large Homography Transformation with $\mathfrak{sl}(3)$ Algebra (ICLR-2024)

Reviewer comment (Evaluation):
-- page 7: "improved performance from 0.69 to 14.72"

Author response:
**Q1:"page 7: "improved performance from 0.69 to 14.72."**

Thanks for your careful proofreading. It should be from 14.72 to 0.69. We will revise the manuscript accordingly.

Reviewer comment (Presentation):
-- The MACE of PFNet* in Table 2 is 1.20, while it is 1.21 in the main text of page 8.

Author response:
**Q2:"The MACE of PFNet\* in Table 2 is 1.20, while it is 1.21 in the main text of page 8."**

Thanks again and sorry for the typos. After proofreading and double checking, it should be 1.21. We will revise the manuscript accordingly.

Reviewer comment (Experiments):
Does the order of the decomposed subgroups affect the results?

Author response:
**Q3:Does the order of the decomposed subgroups affect the results?**

Yes, and this is an interesting question we haven't investigated thoroughly. 
Our initial finding is that translation must be resolved at first since other Lie parameters need to have a pivot before warping.
We use the current permutation because of the prior classical decomposition in [1]. 
The translation and rotation normally are larger than other parameters, so we put them in the first rank. 
We will investigate this in future work.

Reviewer comment (Reproducibility):
Can the estimators be learned in parallel instead of sequentially? The current implementation seems that later estimators need to be conditioned on earlier estimators.

Author response:
**Q4:Can the estimators be learned in parallel instead of sequentially? The current implementation seems that later estimators need to be conditioned on earlier estimators.**

We cannot learn them in parallel currently. Theoretically, the Lie algebra SL(3) is not commutative, which means their Lie elements are not purely independent. This is a very tough problem behind the phenomenon. We argue that this is one of the main reasons why it is hard to estimate the homography parameters.

However, it is a very meaningful suggestion and we have considered it before. This will certainly make the model efficient. One possible way now we think is to parallely estimate in a coarse-to-refine fashion.

Reviewer comment (Novelty):
In terms of novelty, this paper bears resemblance to the work of Zhan et al. (2022). Zhan et al. employed a similar approach, utilizing two groups for estimation, whereas this paper proposes the use of six groups for the same purpose. As a result, the contribution of this work can be seen as somewhat incremental. More clear discussion should be given.

Author response:
Thanks for your suggestion. The main differences between HDN [1] and our proposed method are summarized as follows:

- We aim at building a more general and unified network for the SL(3) group, purely on the algebra space. Meanwhile, HDN presents a method in decomposition to different groups and purely adopts the warp convolution proposed by Henriques et al. [2].

- Thereby, the prediction of our work is the unified variable for the whole warped image. We construct the homography from atomized elements on Lie algebra. However, HDN has to decompose it into two groups and predict the four corners offsets, which introduces the problems of this genre. 

- To the best of our knowledge, the additional warp function has not been proposed before. There are lots of excellent research works 
 [3,4,5] that focus on only one-parameter equivariant architectures, HDN instead does not create new warp functions.
We believe this work is helpful for the community in exploring the related theory on algebraic expression for geometric transformation. 

From the perspective of the application, we proposed a new homography estimation dataset S-COCO-Proj. 
None of the existing datasets emphasize the large transformation in homography estimation.
Most of the existing datasets only have local perspective distortion that is not consistent with the daily observations. 
We start with the Lie algebra, which is able to deal with large transformations in the real-world applications.

Reviewer comment (Writing):
It is desirable for this paper to provide additional elaboration on the mathematical aspects associated with the proposed method, with the aim of enhancing comprehension for individuals who are not familiar with this particular field. The inclusion of more accessible explanations and intuitive examples would be beneficial in ensuring that the content is more easily understood by a broader audience.

Author response:
We thank you for this helpful suggestion. To help the readers better understand our method, we will provide additional information.

- We will include more references to the relevant literature, i.e., group theory and Lie algebra [6,7], for those who want to learn more about the background and the motivation of our work.


- We will improve explanations and illustrative examples in the revised manuscript. We understand that the spatial projection and the correspondence are complex and non-linear, and they may not be easy to understand intuitively. 

Due to the space limitation, we could not include more details and examples in the main paper. 
We realize that our paper would benefit from more elaboration on the mathematical aspects of our method, as they are essential to understanding the underlying theory and the implementation of our approach.
We have provided some basic concepts and proofs in Appendix A and Appendix E, which can serve as supplementary materials for interested readers.