Paper: Sparse hyperbolic representation learning (ICLR-2024)

Reviewer comment (Theory):
The proposed method only works for Cartan-Hadamard manifolds.

Author response:
First, we emphasize that considering Cartan-Hadamard manifolds suffices for our paper's motivation: sparse learning in hyperbolic space. This is because hyperbolic space is a Cartan-Hadamard manifold. In fact, as we have mentioned in Section 8, we can extend our framework to other manifolds, such as spheres and tori. For example, suppose that we consider a two-dimensional sphere, like our planet, and we put the origin at the point with 0 longitude and 0 latitude without loss of generation. Then, sparse hyperplanes are 0-degree longitude lines and 0-degree latitude lines. The signed distance from those hyperplanes can easily be calculated on the sphere. This discussion applies to higher-dimensional spheres and tori. However, we omitted those cases since our motivation is not there but in hyperbolic space, and it involves unnecessary technical preliminaries if we include those types of space in the paper's discussion. 

Just in case, we remark that we did not limit the discussion to hyperbolic space but included a general Cartan-Hadamard manifold because it helps readers to compare the hyperbolic space case to the Euclidean space case. Here, this purpose does not require us to consider other types of manifolds, either.

Reviewer comment (Theory):
There is no convergence analysis for the proposed HISTA.

Author response:
This point is what we have mentioned in Section 8 as a limitation. As we have mentioned there, we cannot use the same technique as we use in Euclidean space. However, we claim that it should not be regarded as an incompleteness of our paper. One reason is that our paper has provided a framework of sparse learning in hyperbolic space, so we have no competitor to which we can compare the convergence rate of our method. Another reason is that the convergence analysis of optimizing an objective function on a general Riemannian manifold with a non-smooth regularization has not been pioneered at the moment. In fact, to the best of our knowledge, there has been no convergence analysis of closed-form algorithms with non-smooth regularizations in a general Riemannian manifold. Obviously, if we regard the regularization term as a part of the loss function, we can get a convergence rate of Riemannian stochastic gradient descent variants, but in this case the convergence rate is affected by the non-smoothness of the regularization term and bad. Also, it does not solve the oscillation issue. This is why the ISTA is proposed in Euclidean space and we are inspired by ISTA. The ISTA achieves a good convergence rate in Euclidean space by not regarding the regularization term as a part of the loss function. However, to the best of our knowledge, no research has succeeded in extending the convergence rate of ISTA to that of a closed-form algorithm in a general Riemannian manifold. The paper (Huang & Wei, 2022) is the only related one, but the algorithm is not given in a closed form in hyperbolic space, unfortunately. To wrap up, if we succeeded in providing a convergence analysis for our algorithm, it could be another independent breakthrough paper. Considering the above background, we believe that the lack of convergence analysis is not an obstacle for our paper to be published at the moment.

Reviewer comment (Experiments):
Currently the applications of the proposed sparse learning method are kind of limited.

Author response:
What we have proposed is a sparse learning framework in hyperbolic space based on regularization. Hence, it applies to any method with a loss function of parameters in hyperbolic space. In this sense, we claim that the application of our sparse learning method is not limited. The reason why we chose the graph embedding as an example application in the numerical experiments in the Appendix of our paper is that the simplicity of the task helps us to discuss the effect of our regularization. Indeed, we successfully discuss the relation between our method's performance and the graph structure there. This does not mean it cannot apply to other tasks. However, if we had applied it to more complicated tasks, such as regression using a multi-layer hyperbolic neural network, the evaluation of our sparse learning framework could not be as insightful as our experiments.

Reviewer comment (Presentation):
There lacks a graphical presentation of experimental results in the main body. Please optimize the article architecture for better readability.

Author response:
Thank you for your constructive suggestion. Our priority was to provide a framework for sparse learning in hyperbolic space, and we focused on its theoretical foundation in the main body. However, we agree that including the graphical presentation of experimental results would help readers' understanding if we had more space. Although the additional page is not granted in this rebuttal period, we will move the contents of Section F in the current version to the main body in the camera-ready version using the additional page.

Reviewer comment (Writing):
Symbols denoted in this paper are ambiguous sometimes. For example, the italic upper letter T represents the tangent space in the preliminary section but represents the final iteration number in Algorithm 1. Please check the uniqueness of notations.

Author response:
Thank you for pointing out the notation issue. We have updated the draft. In the revised version, $\mathscr{T}$, instead of $T$, is used to denote the tangent space and $T$ is now used only to represent the final iteration number.

Reviewer comment (Presentation):
Sections 2, 3, 7 and 8 are missed at the end of the introduction.

Author response:
Thank you for pointing out the format issues. We have updated the draft. In the revised version, 
- a) Sections 2, 3, 7, and 8 are now in the explanation of the remaining sections at the end of the Introduction section.
- b) Definition 2 and Example 2 do not start in new lines now.
- c) SHP mentioned in Definition 3 is now italic-bolded.
- d) The spelling of Poincaré at the end of Section 6 has now been corrected.

Reviewer comment (Presentation):
SHP mentioned in Definition 3 should be italic-bolded.

Author response:
Thank you for pointing out the format issues. We have updated the draft. In the revised version, 
- a) Sections 2, 3, 7, and 8 are now in the explanation of the remaining sections at the end of the Introduction section.
- b) Definition 2 and Example 2 do not start in new lines now.
- c) SHP mentioned in Definition 3 is now italic-bolded.
- d) The spelling of Poincaré at the end of Section 6 has now been corrected.

Reviewer comment (Writing):
There is an incorrect spelling of Poincar\'e at the end of section 6.

Author response:
Thank you for pointing out the format issues. We have updated the draft. In the revised version, 
- a) Sections 2, 3, 7, and 8 are now in the explanation of the remaining sections at the end of the Introduction section.
- b) Definition 2 and Example 2 do not start in new lines now.
- c) SHP mentioned in Definition 3 is now italic-bolded.
- d) The spelling of Poincaré at the end of Section 6 has now been corrected.