Hgformer: Hyperbolic Graph Transformer for Collaborative Filtering
Authors: Xin Yang
Xingrun Li
Heng Chang
Yang jinze
Xihong Yang
Shengyu Tao
Maiko Shigeno
Ningkang Chang
Junfeng Wang
Dawei Yin
Erxue Min
Venue: ICML 2025 poster (2025)
Keywords: Collaborative Filtering
Hyperbolic Representation Learning
Graph Transformer
Primary area: deep_learning->attention_mechanisms

Abstract: Recommender systems are increasingly spreading to different areas like e-commerce or video streaming to alleviate information overload.  One of the most fundamental methods for recommendation is Collaborative Filtering (CF), which leverages historical user-item interactions to infer user preferences. In recent years, Graph Neural Networks (GNNs) have been extensively studied to capture graph structures in CF tasks. Despite this remarkable progress, local structure modeling and embedding distortion still remain two notable limitations in the majority of GNN-based CF methods.  Therefore, in this paper, we propose a novel Hyperbolic Graph Transformer architecture, to tackle the long-tail problems in CF tasks. Specifically, the proposed framework is comprised of two essential modules: 1) Local Hyperbolic Graph Convolutional Network (LHGCN), which performs graph convolution entirely in the hyperbolic manifold and captures the local structure of each node; 2) Hyperbolic Transformer, which is comprised of hyperbolic cross-attention mechanisms to capture global information. Furthermore, to enable its feasibility on large-scale data, we introduce an unbiased approximation of the cross-attention for linear computational complexity, with a theoretical guarantee in approximation errors. Empirical experiments demonstrate that our proposed model outperforms the leading collaborative filtering methods and significantly mitigates the long-tail issue in CF tasks. Our implementations are available in \url{https://github.com/EnkiXin/Hgformer}.

Hgformer: Hyperbolic Graph Transformer for Collaborative Filtering
Xin Yang' Xingrum Li Heng Chang? Jinze Yang* Xihong Yang²
Shengyu Tao Erxue Min
Maiko Shigeno
Ningkang Chang?
Junfeng Wang
Dawei Yin
Abstract
Problem 1: Local structure modeling.
Recommender systems increasingly spread- ing to different areas like e-commerce or video streaming to alleviate information overload. One of the most fundamental methods for recommen- dation Collaborative Filtering (CF), which lever- ages historical user-item interactions infer user preferences. In recent years, Graph Neural Net- works (GNNs) have been extensively studied to capture graph structures in CF tasks. Despite this remarkable progress, local structure model- ing and embedding distortion still remain two notable limitations in the majority of GNN-based CF methods. Therefore, in this paper, we propose novel Hyperbolic Graph Transformer architec- ture, tackle the long-tail problems in CF tasks. Specifically, the proposed framework is com- prised of two essential modules: 1) Local Hyper- bolic Graph Convolutional Network (LHGCN), which performs graph convolution entirely in the hyperbolic manifold and captures the local struc- ture of each node; Hyperbolic Transformer. which comprised of hyperbolic cross-attention mechanisms to capture global information. Fur- thermore, to enable its feasibility on large-scale data, we introduce an unbiased approximation of the cross-attention for linear computational com- plexity, with theoretical guarantee approx- imation errors. Empirical experiments demon- strate that our proposed model outperforms the leading collaborative filtering methods and signif- icantly mitigates the long-tail issue in CF tasks. Our implementations are available in https: / //github. com/ Enw.iin/Hgforrmmeene
Euclidean distance
Distance the graph
Problem 2: High distortion!
www.............................
Items
Hierarchical structure
Users
Figure I. This figure highlights two key challenges that traditional GNNs often face when dealing with long tail items. The first one is the message- passing paradigm in local neighborhoods, which prevents the model items to most users, leading to poor recommendations for these items. The second one is that the embeddings in Euclidean space may fail to capture the long distance in graphs, and will lead to serious information distortion in graph structure.
transferring information from less popular
1. Introduction
Recommender systems have become an indispensable part of our daily life, serving as fundamental tools for person- alized information filtering and prioritization (Cheng et al., 2016; Davidson et al., 2010; Gong et al., 2020; Min et al., 2025; Xihong et al., 2025; Yang et al., 2025). The core of recommender system to predict whether user will en- gage with an item, such as by clicking, rating, purchasing it. In this context, Collaborative Filtering (CF) (Rendle et al., 2012; Xue et al., 2017; Zheng et al., 2018; He et al., 2020), which leverages past interactions between users and items to make these predictions, remains an essential component to deliver effective personalized recommendations. The inter- action patterns between and items in CF tasks naturally form graph structure, motivating researchers to investigate the use of Graph Neural Networks (GNNs) (He et al., 2020; Wang et al., 2019a; Sun et al., 2021), ), which has proven significant advantages in modeling graph structures(Kipf Welling, 2016; Hamilton et al., 2017).
Despite the significant attention and fruitful outcomes in this field, most existing GNNs normally assume that the degree of each node is balanced. However, data in the realm of recommender systems generally exhibit long-tail dis- tribution (Park & Tuzhilin, 2008; Park, 2012; Yin et al., 2012: Zhao et al., 2023): a small portion of items are highly popular with numerous users, whereas most other items
Universityy of Tsukuba Kyoto University "Tsinhhua University University Tokyo BBidu Inc.. Correspondence to: Erxue Min <erxue min gomi..com>..
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Hgformer: Hyperbolic Graph Transformer for Collaborative Filtering
attract relatively few users. Recent studies have also shown that GNNs-based methods perform well in recommending popular items (head items), but often struggle to perform as effectively with less popular items (tail items) (Sun et al., 2021; Yang et al., 2022a). This issue primarily arises from two factors, which are described in Fig. 1: i) Local struc- ture modeling: GNN-based models normally follow the neighborhood aggregation scheme and tend to be biased towards nodes with high degree (Liu et al., 2023). More pre- cisely, in the task of CF, head items that are interacted with by many users typically have higher-quality representations, while the vast majority of tail nodes with few interactions are likely to be underrepresented (Yun et al., 2022). Sev- eral GNN frameworks have been proposed to mitigate these degree biases by introducing designated architectures or training strategies specifically for low-degree nodes (Sun al., 2021; Yun et al., 2022; Zhao et al., 2023), but they still fail to capture the global information as transformers (Ying et al., 2021; Wu et al., 2022; 2023; 2024) and thus achieve sub-optimal results. ii) Embedding distortion: Most exist- ing methods encode items and users into Euclidean space (Wang al., 20;9a; He et al., 2020), which is a flat geometry with a polynomial expanding capacity. Data with the long- tail distribution can be traced back to hierarchical structures (Ravasz& & Barabási, 2003), whose number of neighbors increases exponentially. As result, encoding these data via Euclidean space naturally incurs information loss and could subsequently deteriorate the performance of downstream tasks. In contrast, hyperbolic manifold, a non- -uclldeid space characterized by constant negative curvature, allows the space expand exponentially with the radius, making it particularly well-suited for representing tree-like or hier- archical structures (Chami et al., 2019; Ganea et al., 2018) 8). For this reason, in recent years, significant advances have been made in hyperbolic neural networks to better handle the problem of long-tail distribution (Chami et al., 2019; Zhang& & Wu, 2023; Yue et al., 2023).
HICF (Sun et al., 2021; Yang et al., 2022b:a) require to first project embeddings in the hyperbolic manifold back to the tangent space for subsequent graph convolution, which causes information loss and limits their performance.
No well-defined hyperbolic self-attention mechanism for collaborative filtering. Although there are currently some definitions hyperbolic attention (Zhang et al., 202 Yang et al., 2024a), in the field of Collaborative Filtering, the user-item interaction structure is represented as bi- partite graph, existing methodologies are inadequate, and hyperbolic self-attention mechanism tailored for CF tasks has not yet been investigated.
Scalability issue of hyperbolic self-attention mecha- nism. In recommender systems, real-world graphs are often large-scale, which poses a significant challenge in efficiency when applying transformer architectures with quadratic time complexity. Although many studies have tackled the scal- ability problem of graph transformers in Euclidean space (Wu et al., 2022; 2023; 2024), the solution for linear com- putational complexity of hyperbolic self-attention is still under-explored.
To solve these challenges, we propose new Hyperbolic Graph Transformer framework called Hgformer. For the first challenge, we propose the Light Hyperbolic Graph Convolutional Network (LHGCN), which performs graph convolution entirely in the hyperbolic manifold. For the see- ond challenge, we propose a novel hyperbolic transformer architecture tailored for CF tasks, which consists of a cross- attention layer and hyperbolic normalization layer; For the last challenge, we propose unbiased approximation approach to reduce the computational complexity of hyper- bolic self-attention to the linear level. Numerical experi- ments show that our proposed model performs better than the leading CF models and remarkably mitigates the long- tail issue CF tasks. We summarize our contributions as follows:
So far, most research in recommender systems has focused on addressing either one of the two problems but has not managed to tackle both simultaneously. Inspired by the successful application of Graph Transformers in graph and node classification tasks (Ying et al., 2021; Wu et al., 2022: 2024; Yang et al., 2024a), this study proposes a novel Graph Transformer architecture in the hyperbolic manifold for Col- laborative Filtering, Although the idea of extending Graph Transformers to hyperbolic space for recommendations is intriguing, it poses several challenges that must be over- come:
We propose a novel graph convolution method, LHGCN, which removes the feature transformation and non-linear activation functions from HGCNs and ensures that the entire message passing can be per- formed fully in hyperbolic manifold without mapping back to Euclidean space.
We propose hyperbolic graph transformer framework for CF tasks, which can simultaneously address the challenges of local structure modeling and embedding distortion.
No well-defined parameter-free graph convolution in hyperbolic space. Parameter-frre message-passing paradigms such as LightGCN (He et al., 2020) have shown superior advantages in CF, however, most existing hyper- bolic variants of LightGCN, such as HGCF, HRCF, and
We propose kernel function approximation method in hyperbolic space, which reduces the computational complexity of hyperbolic self-attention to a línear scale and we theoretically prove that the approximation error remains within a controllable range.
2

---

Hgformer: Hyperbolic Graph Transformer for Collaborative Filtering
The numerical experiments show our proposed method is superior to leading CF models. Moreover, compared with traditional hyperbolic graph neural network meth- ods, our approach can further enhance the model's s performance on long-tail items.
efits and increases the difficulty of model training. There- fore, removing nonlinear feature transformation from GCNs is a well-accepted approach in CF. To perform message- passing in hyperbolic space, a straightforward solution is to first project embeddings in the hyperbolic manifold back to the tangent space at the north pole point, performing parameter-free graph convolution, and then map them back to the hyperbolic manifold (Sun et al., 2021; Yang et al., 2022b;a). However, since the tangent space at the north pole point is merely local approximation of the north pole point (Boumal, 2023) this can cause a certain degree of infor- mation loss. For this sake, we design a simple yet efficient graph convolution method tailed for CF called LHGCN. Similar to HGCF (Sun et al., 2021) and LightGCN (He et al., 2020). LHGCN does not have trainable parameters and all computations are performed entirely on the hyperbolic man- ifold, eliminating the need for transformations between the hyperbolic manifold and Fuclidean space. Specifically, we adopted hyperbolic centroid to aggregate the messages of neighbors:
2. Methods
In this section, we will elaborate our proposed method. All the notations of this paper are summerized in Ap- pendix 2. We formally define our tasks as follows: In- put: The interaction graph of users and items G = (Va. Vi, E S Va x V₁. Output: A learned function F= (u, i|g 9), where u E V E Vú and & denote the model parameters.
As is shown in Fig. 2(a), in our framework, we first map the users and items into embedding space according to their IDs and use an exponential map to project the embeddings into hyperbolic space. To capture the local structure of nodes in the user-item interaction graph, we design a Light Hy- perbolic Graph Convolutional Network (LHGCN) and to capture the global structure of the entire interaction graph, we propose a novel hyperbolic transformer, which is com- posed of hyperbolic cross-attentin mechanism with linear computation complexity and hyperbolic normalization layer( et al., 2023). In the final step, we aggregate the local structure information and the global information from both perspectives for prediction, optimized by a hyperbolic margin-rankinn loss.
u, (I+1) Centroid ( () kE N:))),
i, (1+1) Centroid (1, (ug
(0) k E N2H)),
where N₂ denotes the neighbors of node i. For detailed definition of the hyperbolic centroid, see Definition B.7 in Appendix B
Then the outputs of LHGCN are
u looat (f.) and glocat j())
2.1. Hyperbolic Embedding
where L denotes the number of layers of LHGCN.
We first encode each user and item into the embedding space, which is denoted as u = -fff:..... u and = :ifyl where the superscript means Euclidean space, N and M means the number of users and items. Then, we use an exponential map project the embeddings into the hyperbolic space:
2.3. Hyperbolic Transformer Model
In this section, to address LHGCN's limitations in capturing the global information of the interaction graph and taking into account the unique structure of the bipartite graph in CF, we design a novel hyperbolic cross-attention mechanism for modeling global user-item interactions. Furthermore, since this cross-attention requires operating on all user-item pairs, with computational complexity OMM-N), we propose an approximation approach to reduce the computational complexity to O(M + NN).
u Exp (10, n)), = Exp ((0,
For the detailed definition of the exponential map, see Defi- nition B.6 in Appendix B. Since in the subsequent sections, we will only use the embeddings within the hyperbolic man- ifold, for convenience and to avoid confusion, we ignore the superscript H and simply denote the embeddings of users and items in the hyperbolic manifold as u = |ui;...: uy] and = [i,:...;; :MM:
1. 1. HYPERBOLIC CROSS- ATTENIATTNNNOON
In CF tasks, there are only interactions between the user set and the item set, which form a bipartite graph. Intuitively, modeling the inner interactions among user groups or item groups would introduce noisy signals and thus deteriorate the performance (Min et al., 2022). For this reason, we introduce a cross-attention mechanism to model only all possible user-item interactions, as detailed in the structure presented in Fig. 2(c). Taking the i-th user vector as an example, firstly, the correlation between the i-th user vector
2.2. LHGCN: Light Hyperbolic Graph Convolutional Networks
In CF tasks, each node (user or item) is represented by unique ID, which lacks concrete semantics beyond being an identifier. In such scenarios, LightGCN (He et al., 2020) empirically demonstrated that performing multiple layers of nonlinear feature transformation does not provide ben-
3

---

Hgformer: Hyperbolic Graph Transformer for Collaborative Filtering
HGCF
LHGCN
Capture the global information
Hyperbolic Cross
Hyperbolic Norm
Attention
Predict by hyperbolic distance
geaun
Centroid
Add
Problem: T2 T only local approximation.
Aggregation
Prediction
Mean T
(b) Difference between HGCF and LHGCN
Target user
Take all items into consideration
Items
LHGCN
1L
Users
Aggregate hyperbolic mean
Interacted Items Non-interacted Items
(a) Overall architecture of Hgformer
(c) Hyperbolic Cross-Attention
Figure 2. Figure(a) shows the overall architecture of Hgformer. We first map the and item embeddings into hyperbolic manifold. Then, we use several layers of LHGCN to capture the local information of the interaction graph and use a hyperbolic transformer to capture the overall information of the interaction graph. Finally, we combine these information and make predictions on the hyperbolic manifold. Figure (b) shows the difference between HGCF and LHGCN in graph convolution. In HGCF, embeddings are mapped from the hyperbolic manifold to North Pole point's tangent space for aggregation, then back to hyperbolic manifold, causing information distortion due to the local approximation. LHGCN, on the other hand, aggregates information directly in the hyperbolic manifold. Figure (c) shows the process of hyperbolic cross-attention. Unlike GNN-based models that only consider items a user interacted with, Hyperbolic Cross-Attention takes both the interacted and non-interacted items into account.
(i E ; NN) and the j-th item vector (j E {I....,My) MY) under specific attention head is defined as:
where c ||l [| and is the curvature of the hyper- bolic manifold. After that, we aggregate the embeddings of different heads by the hyperbolic centroid defined in Definition. B.7 in Appendix B:
exp (q (h) .. )/T
(N) WE
(1)
E exp Sim (h) (1) 1/T
k/"
ug global Centroii ((1) :....""""
(())))..
(5)
where q. w (h) @K u, = @ i, and @ Q
We calculate all item embeddings ig in the same way.
golobal .j E {I....,Mf
denotes hyperbolic matrix multiplication, which defined in Definition B.8 in Appendix B. = is the temperature pa- rameter. Sim(:s) is function calculate the similarity of two vectors in the hyperbolic manifold, and it was generally defined as:
For all the embeddings = [ugiobal jalobal]] Finally, for numerical stability, adopted the definition of Hyperbolic Normalization from (Bdeir et al., 2023) and applied it to normalize the final embeddings x:
x)
+2))
PT 7.
4-00 [log (x))
HN(x) expx
pTk
where f(-) is a monotonically increasing function, such as exponential maps, linear functions, tanh, sigmoid, etc. and (--) is the distance function in hyperbolic manifold, which is detailed in Definition B.5 in Appendix B. Then, the representation of the i-th user is updated by aggregating all item embeddings with weights ai.j
voz +e
(6)
where μ = Centroid(xx), which Centroid of x in hy- perbolic manifold and 0² M ΣM+N (x;, H)) which is the variance in hyperbolic space, 3 and e are train- able vectors and PT - means the parallel transport from the point o to point f3, which is detailed in Definition.B B.9 in Appendix B.
M
(h)
(h)
(h)
(3)
2.3.2. TOWARDS LINEAR COMPLEXITY
where y w!! hh)
K 1,-
In this section, we introduce the hyperbolic self-attention with only linear computational complexity. Although the above Hyperbolic cross cross-attention mechanism can model the global interactions between all users and items, its quadratic time complexity prevents its application in real-world see-
To ensure that the embedding stays in the hyperbolic mani- fold, we need an extra coefficient to scale the embedding,
(h)
(N) cu;
(4)

---

Hgformer: Hyperbolic Graph Transformer for Collaborative Filtering
narios when there are numerous users or items. To able the application of Hyperbolic Transformers larger datasets, as Fig. 3 shown, the previous complexity of the similarity matrix of hyperbolic vectors is reduced to only O(MM + mm) by our mechanism, where the dimensions m and are much smaller than M and N.
Similarity Matrix
Space: DLMN Md)
M
Time: Of ((MNd)
X C
M
Since the most computationally intensive part of the model Eq. 3, in this section, our goal is reduce the compu- tational complexity of Eq. 3 to linear. Firstly, we rede- fined Eq. 2. Since K >> 0, arcosh(-) is monotonically increasing function, both (-dM( (:)-) and the Minkowski inner product (..)) could be used to compare the similar- ity between different vectors in hyperbolic space. Then for x,y Hd-1, we redefine the similarity function -im(--) as the Hyperbolic SoftMax similarity function, HSM(;,): He0+. Hd+1, + R:
Use hyperbolic positive random features + to approtimate
2
Same
Compute this part first
M
X C
Space: O(Nm + Md md)
HSM(X, y) e exp ((
Time: O N)md)
then Eq. 3 redefined as:
M
exp ((q (h) (h))
/T)
Figure 3. This figure shows how we reduce the complexity of hy- perbolic graph transformers to linear. We first use to transform the similarity matrix into the multiplication two smaller matrices and then, we change the order of computation order to reduce the computation complexity.
( Σ
M
(h)
(7)
1=1 exp ((a.") k (h)
M
/T)
n
n=1
where q; k(th) and wh follows the settings of Eq. (N)
Then, we use an estimation k(q (()) to approximate
exp
(h) q kg (e(")) (m) in Eq. 7 and we introduce Theorem
where wk N (Oa, Ia) is i..dd, m is a constant that could be chosen smaller than d. Then, we have:
4.1, which proves that the aforementioned estimation is unbiased estimation.
o(x) ( R xx
HSM(X. y).
(10)
For x,y E Hd+1KKK with x (zr; F1, ... rdd T. y (yo, y1.... yya) and x (x1, 22. x.) T ÿ (91, 12.
Theorem 3.1.
The proof of this Lemma is given in Appendix C.2. We adopted a positive random feature map x(-) Hd+1,K +
Maa) we have an esti-
mation function k(,,) Hd 1
1,K R:
T to approximate HSM function:
- (Jo + y0)2 22K
x(x,y) exp
2
HSM(x.y)/TEL
(1I)
1Ew. [exp(w] T (x))))]
(8)
where w ~ N (Oa, Ia) :.).))) is an unbiased estimation of the HSM fanction:
Eq. 11 is proved in Lemma 4.2 and for Hd+1k x (zo, ... xa) x = (I1, 22 xa)" the explicit form of positive random feature map with temperature parameter is defined as:
y( = HSM(x, y).
(9)
The proof of this thcorem is given in Appendix C.I.
Then, such an unbiased estimation function (Eq. 8) can be converted into a dot product of vector functions approxi- mately; the method of converting it is akin to kernel tricks shown as the following lemma:
X
exp
2r Vm
K K-22
W
X
W x
exp
exp
Then, can change the computation order and extract common factors by using Eq. 10 to convert the HSM func- tion into the dot product of two feature functions. Then the approximating aggregation function Eq. 12 only has linear complexity and the process is visualized in Fig. 3. Subsequently, the final form of the aggregation function
Lemma 3.1. Define the hyperbolic positive random candomfeatures <(.): : Hd+1,K + m
K-2 exp
o(x)=
(exp (w) x)
exp (w [x)
Vm
5

---

Hgformer: Hyperbolic Graph Transformer for Collaborative Filtering
proposed follows:
where À is non-negative hyperparameter. if the embeddings of the positive samples of this user and ifinal are the embeddings of negative samples of this user the same hyperbolic manifold. Positive samples refer to the items that the user has interacted with, while negative sam- ples refer to randomly sampled items that the user has not interacted with.
M
Φ (q./VT) δ (kg//VF)
1 (qi/v) a (kk/vi) Vy
(12)
(g. En M (ks/v7) (v5)
① (q:/v7 Ex= @ (kn/vll (T)
The error of approximation is bounded and have: Theorem 3.2: The error function of approximation
3. Experiments
In this section, we conduct extensive experiments on mul- tiple public datasets evaluate the proposed method and primarily address the following questions.
A = =|HSMM(X,y) - -((x) d(y)
is bounded by O (V
exp(((66-K) me
with the probability
RQI: How does Hgformer performs compared to baselines?
that:
RQ2: How does each module contributes to the perfor- mance?
exp cxp(3(6 -K) K))
1 1-EEP(A
(13)
Me
RQ3: Why does Hgformer perform better than other models?
assuming that ||x13 8 1 < for x, E Hd+1.K The proof is given in Appendix C.3.
To ensure accuracy and fairness in our experiments, all the models were tested on RecBole (Zhao et al., 2021), a widely accepted framework in the field of recommender systems. The dataset was split into training, validation, and test sets with an 8:1:1 ratio. During training, we adopted an early-stop mechanism for all models, stopping training if no improvement was observed on the validation set for 30 epochs and the best-performing parameters on the validation set were selected for final evaluation on the test set. Due to space limitations, the details of experimental settings are provided in Appendix D, and tail-item analysis and sensitivity analysis are detailed in Appendix E.
Since the upper bound of the error function depends only on the Euclidean norm 5, the curvature constant K, the number positive random features m, and the demanding error accuracy 6, we can reduce the error by normalizing the vectors and process them in a suitable hyperbolic space, or increase m.
2.4. Embedding Aggregation and Optimization
Embedding Aggregation. To aggregate both structural and global information, we map the embeddings back to Euclidean space using the Log function, and then perform weighted average:
3.1. Overall Performance Comparison (RQ1)
u final = Expo K (ar Log (notobal) + (1 - aa) Log (nlocal)
We compared Hgformer with traditional models, GNN- based models, hyperbolic GNN-based models, and graph transformer-based models across five datasets of different sizes in Table 1. We find the following observations:
ifinal Expo K (a Log (Hotobal) (1 aal-a) a) Log (flocat)) (14)
where a is a hyperparameter between and 1.
Prediction. Margin ranking loss has been extensively used in recommendation tasks (Sun al., 2021), which separates positive and negative pairs of user items by given margin. When the gap between negative and positive user-item pair exceeds this margin, neither pair contributes to the overall loss, enabling the optimization process to focus on the difficult pairs in the data set. In this work, we use the hyperbolic version of margin-ranking loss as prediction loss. The prediction loss is defined as:
1. Across all datasets, Hgformer demonstrates consistent performance improvements. The improvements are par- ticularly significant in the Amazon Book and Amazon CD datasets. On the Amazon Book dataset, compared to the second-best model, HICF, Hgformer achieves rela- tive improvements of 17.6%. 18.9%, 18.1% and 19.3% in Recall @ 10, Recall @ 20, NDCG@ 10, and NDCG@) 20, respectively. Similarly, on the Amazon CD dataset, the relative improvements are 18.3%, 15.7%, 15.9%, and 16.9% respectively.
2. It can be seen that all three hyperbolic GCN-based mod- els show noticeable improvements over traditional GNN models, demonstrating the advantages of hyperbolic GCN in handling recommendation tasks. This is at- tributed to the fact that the capacity of hyperbolic space
: Lu Sinal ffinel ffinal) max (dvelu/ina final iffinatz ippee
- dd dd ffnaaal f + 2.0).
6

---

Hgformer: Hyperbolic Graph Transformer for Collaborative Filtering
Table 1. Overview of performance. N@ 10 and @@ 10 are abbreviations for the metries NDCG@ 10 and Recall 10, respectively. *
represents the significance level p-value < 0.05. The highest scores for each dataset and metric are emphasized in bold, while the second best ones are underlined.
Dataset
Metrie
BPR
NGCF
LightGCN
HMLET
HGCF
HICF
HRCF
SGFormer
NodeFormer
Hypformer
Hgformer
Recall 10
0.0357
0.0427
0.0581
0.0347
0.0748
0.0766
0.0745
0.0545
0,0379
0.0376
0.0901
Recall
0.0538
0.0676
0.0910
0.0553
0.1089
0.1086
0,1080
0.0841
0.0602
0.0605
0.1291
Amazon Book
NDCG@ 10
0.0218
0,0250
0.0344
0.0201
0,0467
0.0485
0.0466
0.0332
0.0222
0.0215
0.0573
NDCG@20
0.0267
0.0317
0.0433
0.0256
0.0560
0.0571
0.0560
0,0412
0.0282
0.0277
0.0681
Recall 10
0.0473
0.0755
0.0770
0.0601
0,0797
0.0826
0.0824
0.0435
0,0454
0.0613
0,1977
Recall 20
0.0726
0.1151
0.1174
0,0874
081184
0.1211
0.1176
0.0695
0.0710
0.0936
0.1401
Amazon CD
NDCG@10 10
0.0268
0.0426
0.0437
0.0351
0.0451
0.0489
0.0477
0.0242
0.0253
0.0340
0.0567
NDCG@20
0.0334
0.0529
0.0542
0,0421
0.0552
0.0580
0.0577
0,0310
0.0319
0.0424
0.0678
Recall 10
0.0585
0.0580
0.0702
0.0491
0.0740
0.0740
0.0761
0.0560
0.0338
0.0294
0.0803*
Recall 20
0.0929
0.0913
0.1 106
0.0817
0.1110
0.1135
0.1157
0.0871
0,0575
0.0528
0.1203
Amazon Movie
NDCG@10
0.0362
0.0351
0.0440
0.0293
0.0464
0.0465
0.0481
0.0343
0.0200
0.0150
0.0503
NDCG@20 0.0455
0.0442
0.0549
0.0382
0.0566
0.0572
0.0589
0.0428
0.0265
0.0210
0.0612
Recall
0,1059
0.1280
0.1313
0.0926
0.1375
0,1388
0.1357
0.0886
0.0777
0,0446
0.1462*
2
0.1588
0,1832
0.1906
0.1440
0.1935
0.1938
0.1892
0.1349
0.1225
0.0772
0.2052
Douban Book
NDCG@10 10
0.0706
0,0864
0.0922
0,0630
0.0960
0.0968
0.0943
0.0610
0,0484
0.0289
0.1033
NDCG@20
0.0850
0,1013
0,1078
0.0768
0,1113
0.1117
0.1091
0.0734
0.0610
0.0380
0,1189*
Recall 10
0.0616
0,1373
0.1339
0,1178
0.1348
0.1370
0.1393
0.1259
0,1118
0.0678
0.1405
Recall
0.0970
0,2042
0.1989
0,1802
0.1992
0,2034
0,2036
0.1877
0,1722
0.1106
0.2068
Douban Movie
NDCG@10
0.0676
0.1256
0.1326
0.1227
0.1248
0.1297
0.1342
0.1263
0,1002
0,0780
0.1322
NDCG@20
0,0725
0.1389
0.1435
0.1325
0.1377
0.1424
0.1456
0.1364
0.1133
0,0845
0,1447
0.1084
0.1229
0.1258
0.0916
0,1218
0.1276
0.1271
0.1071
0,0791
0,0313
0,1386
Recall @: 20
0.1606
0,1803
0,1802
0,1418
0.1791
0.1843
0.1791
0.1599
0,1239
0.0531
0.1955
Douban Music
NDCG@10 10
0.0783
0.0899
0.0966
0.0727
0.0927
0,0942
0.0940
0.0842
0,0534
0.0253
0.1024
NDCG@20
0.0917
0.1045
0.1095
0,0840
0.1068
0.1085
0.1070
0.0966
0.0656
0.0306
0.1165
exponentially increases with radius, aligning well with the power-law distributed user-item interaction graph.
2. Removing the Hyperbolic Transformer also leads a re- markable decline in model performance. The Recall @ 10 metric decreased by 1 11.66 11.6%, 12.9%, and 10.8% on the Amazon Book, Amazon CD, and Douban Music datasets, respectively. Similarly, the NDCG@ 10 metric experi- enced reductions of 13.4%, 10.9%, and 7.9% on the Amazon Book, Amazon CD, and Douban Book datasets, respectively. Furthermore, we observed that incorporat- ing the Hyperbolic Transformer led to certain improve- ments in HGCF, which demonstrates its effectiveness.
3. We also compared our model with three graph trans- former models (SGFormer, NodeFormer, and Hyp- former). However, since these three models are designed for node classification tasks and compute attention for all node pairs, they tend to overemphasize the relationships between users and between items. This leads to poor performance in recommendation tasks.
3. Replacing LHGCN with HGCF also results in decline. Through the direct comparison between LHGCN and HGCF, we found that LHGCN outperforms HGCF on the majority of datasets (Amazon Book, Amazon CD, Ama- zon Movie, Douban Movie). The only exception is the Douban Book dataset, where LHGCN performs slightly worse than HGCF (Recall @ 10: 0.1368 vs. 0.1375; Re- call @ 20: 0.1916 vs. 0. NDCG@ 10: 0.0948 vs. 0.0960; NDCG @ 20: 0. 1096 vs. 0.1113), As mentioned in Section 2.2, HGCF requires mapping embeddings back to the tangent space at the North Pole point during message aggregation, which results in information distor- tion during this process. In contrast, LHGCN performs graph convolution entirely on the hyperbolic manifold.
3.2. Ablation Analysis (RQ2)
We conduct ablation studies on two main components of Hgformer. The results are shown in Fig. 4.We have the following observations:
1. The removal of LHGCN results in the most significant drop in performance. This indicates that LHGCN plays a dominant role in the CF task, and solely using the Hyperbolic Transformer to capture global information between users and items, while ignoring the inherent topological structure of the existing interaction graph, is not sufficient to effectively capture potential user-item relations. Therefore, in recommendation tasks or link prediction tasks, it is difficult to achieve good results by completely abandoning GNNs and relying solely on transformers for prediction. A better strategy for link pre- diction and recommendation tasks is to use transformers as a supplementary tool.
3.3. Case study (RQ3)
In this section, we present a case study to demonstrate the effectiveness of our model in addressing both the local struc-
7

---

Hgformer: Hyperbolic Graph Transformer for Collaborative Filtering
0.14
0.175
Gownt
0.125
1134
6-224
6.I OVTIN
10 0.06
8,300
20
0.075
0.850
0.02
0.025
0.00
Amazon Book
Amazon Movie
Douban Book
Douban Movie
Douhen Music
0.000
Aut Amason Book
Amason CD
Amazon Movie
Douban Book
Douban Movie
Douban Music
6360
0.34
1119
0.30
0.08
10000
0.06
Douss
8450
0:401
00900 pear
0.04
0.02
0.00
Amazon
Movie Douban
Douban Movie Douban Music
0.00
Amazon Book
Amazon CD
Amazon Movie Douban Book Douban Movie Douban Music
Hgformer
w/o Transformer
w/o LHGCN
HGCF-Transformer
HOCF
Figure 4. To validate the contribution of cach module in the model, we individually removed the LHGCN and the Hyperbolic Transformer for evaluation. In the figure, ww/o LHGCN' and "w/o Transformer' represent the effects after removing the LHGCN and the Hyperbolic Transformer, respectively. Furthermore, to validate the effectiveness of LHGCN, we also compared LHGCN with HGCF. Specifically, we replaced LHGCN in Hgformer with HGCF and tested performance and the figure, is noted as HGGC++Transoonin
ture modeling problem of GNN-based models and high distortion problem of models in Euclidean space. We se- lected a user who interacted with 28 items as the subject of the case study. By analyzing items recommended by Hgformer from both head and tail positions, aim to derstand why Hgformer recommended these items while other models did not. It can be observed in Fig 5 that both the Euclidean space models and hyperbolic space models ex- hibit similar performance for head items, with their rankings being relatively close. However, in the case of tail items, we notice that Euclidean space models tend rank these items lower, while hyperbolic space models can more effectively identify specific tail items that User_4 prefers. Nonetheless, HGCF fails to recommend items that are far from User 4 in the interaction graph (i.e., items with hops greater than 5). In contrast, Hgformer introduces the Hyperbolic self- attention mechanism, enabling the model to identify distant but relevant items effectively.
features Degree
Interacted items
item features Degrea Hops
Interacted items
Hops
133
2
37173
16
44671
Ranking:
2
Ranking: 156
119
25657
V
V
Ranking: 612
10950
28
12
Ranking:
8
Ranking: 657
Recommendded by LigheGNN
Recommended HCCF
16949
Recommended Haformer
Ranking: 1289
32
19
Degrea:
Figure 5. We selected number 4 from the Amazon CD dataset as the subject of analysis. We analyzed the items recommended by Hgformer in the @20 task.
Neural Network (FHNN) (Chen et al., 2021) resolves this by using Lorentzian linear transformations, while H2HGCN (Dai et al., 2021) and LGCN (Zhang et al., 2021b) employ Einstein midpoints and Lorentzian aggregation for fully hy- perbolic computations. Despite their efficacy, these methods introduce additional parameters, which are less suitable for collaborative filtering (CF). To address this, we propose LHGCN, parameter-free fully hyperbolic graph convolu- tion method tailored for CF.
4. Related Works
Hyperbolic Neural Networks. Hyperbolic manifolds, char- acterized by negative curvature, are effective for modeling hicrarchical structures and long-tail distributions (Ravasz & Barabási, 2003). Building on foundational works (Chami al., 2019; Ganea et al., 2018), various hyperbolic opera- tions linear layers, activation functions, and graph convo- lutions-have been introduced. However, many methods map embeddings back to Euclidean space for operations like self-attention and linear transformations. Fully Hyperbolic
Graph Neural Networks for CF. Graph Neural Networks (GNNs) effectively capture complex user-iten item relationships, making them widely adopted in CF (Wang et al., 2019a; He et al., 2020; Sun al., 2021; Chang al., 2025; Li et al.,
8

---

Hgformer: Hyperbolic Graph Transformer for Collaborative Filtering
2025; Wang al., 2025; Xie al., 2024; Chang et al., 2024; 2023). NGCF (Wang et al., 2019a) enhances expressiveness through multilayer graph convolutions, while LightGCN (He et al., 2020) simplifies GCN architectures by removing non-linear activations and self-loops, improving efficiency and performance. However, traditional GNNs struggle with hierarchical and scale-free graph structures. Hyperbolic GNNs address this limitation: HGCF (Sun et al., 2021) mitigates long-tail issues, HICF (Yang et al., 2022a) bal- ances head and tail recommendations via hyperbolic margin ranking, HRCF (Yang et al., 2022b) incorporates geometric regularization to prevent over-smo. and HCTS (Yang et al., 2024b) leverages hyperbolic contrastive learning for cross-domain CF. Despite these advances, existing hyper- bolic CF models primarily capture local structures, limiting their ability model the global user-item graph structure. This highlights the need for more expressive hyperbolic approaches in CF.
Acknowledgement
This work was supported by JST SPRING, Grant Number JPMISP2124.
Impact Statement
This research addresses the issue of insufficient attention to tail items in recommendation systems. This problem is related to fairness from the perspective of merchants. The model proposed in this study ensures that while maintaining recommendation accuracy for users, merchants' items also receive sufficient attention, thereby ensuring fairesss.
References
Bdeir, A., Schwethelm, K., and Landwehr, N. Fully hyper- bolic convolutional neural networks for computer vision. arXiv preprint arXiv:2303.159// 2023.
Graph Transformer. Graph transformers have demon- strated strong representation capabilities for capturing global graph structures and complex interactions (Yun al., 2019; Ying al., 2021; Wu al., 2022; 2023; 2024; Li al., 2023). However, their high computational complexity restricts their application to small graphs. Recent meth- ods have focused on reducing this complexity, enabling the use of graph transformers in large-scale graphs (Wu et al., 2022; 2024) and recommendation tasks (Li et al., 2023; Chen et al., 2024; Wei et al., 2023; Min et al., 2022). Cur- rently, Hypformer (Yang et al., 2024a) introduces a linear complexity hyperbolic transformer, designed for node clas- sification. The differences between their models with ours can be summerized as follows: We propose LHGCN for ef- fective local structure modeling in hyperbolic space, which not explored in their work. To achieve linear computa- tional complexity, Hypformer first swaps the multiplication order of space-like values in self-attention vectors and then recalibrates the time-like values. In contrast, we adopt the cross coossatttentonn for better performance in CF tasks and di- rectly extends the kernel function to hyperbolic space, which has a theoretical guarantee in approximation errors. Overall, we are the first to propose hyperbolic graph transformer in the context of collaborative filtering.
Boumal, N. An introduction to optimization on
smooth manifolds. Cambridge University Press, 2023. doi: 10. 00.1017/9781009166144. URL https:/ /tt:://wwww.... /ww.. n nicolase wccolaaoummaaao www./te/bbooooooo
Chami, I., Ying, Z., Ré, C., and Leskovec, J. Hyperbolic graph convolutional neural networks. Advances in neural information processing systems, 32, 2019.
Chang, H., Cai, J., and Li, J. Knowledge graph completion with counterfactual augmentation. In Proceedings of the ACM Web Conference 2023, pp. 2611 2611-2620, 2023.
Chang, H., Ye, J., Lopez-Avila, A., Du, J., and Li, J. Path- based explanation for knowledge graph completion. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 231-242, 2024.
Chang, H., Gu, L.. Hu, C., Zhang, Z., Zhu, H., Xu, Y., Fang, Y., and Chen, Z.. Separated contrastive learning for match- ing in cross-domain recommendation with curriculum scheduling. In Companion Proceedings of the ACM on Web Conference 2025, pp. 114-123, 2025.
Chen, S., Chen, J., Zhou, S., Wang, B., Han, S., Su, C., Yuan, Y., and Wang, C. Sigformer: Sign-aware graph transformer for recommendation. arXiv preprint arXiv:2404.11988 2024.
5. Conclusion
Considering the local structure modeling and embedding distortion issues in previous work of collaborative filtering, we proposed Hyperbolic Graph Transformer framework, which leverages LHGCN for graph structure modeling and hyperbolic cross-attention for global information modeling. Both theoretical analysis and empirical results demonstrate the superiority of our method, especially in mitigating the long-tail problems in collaborative filtering.
Chen, W., Han, X., Lin, Y., Zhao, H., Liu, Z., Li, P., Sun, M., and Zhou, J. Fully hyperbolic neural networks. arXiv preprint arXiv:2 14686.66....... 2021.
Cheng, H.-T., Koc, L., Harmsen, J., Shaked, T., Chandra, T., Aradhye, H., Anderson, G., Corrado, G., Chai, W., Ispir, M., et al. Wide deep learning for recommender
9

---

Hgformer: Hyperbolic Graph Transformer for Collaborative Filtering
systems. In Proceedings of the Ist workshop on deep learning for recommender systems, pp. 7-10, 2016.
Min, E.,, Rong. Y., Xu, T., Bian, Y., Luo, D., Lin, K., Huang. J., Ananiadou, S., and Zhao, P. Neighhour interaction based click-through rate prediction via graph-masked transformer. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 353-362, 2022.
Dai, J., Wu, Y., Gao, Z., and Jia, Y. A hyperbolic-to- hyperbolic graph convolutional network. In Proceedings of the IEEE/CVF conference computer vision and pat- tern recognition, pp.
2021.
Min, E., Huang, H.-Y., Yang. M., Yang, X., Jia, X., Wu, Y., Cai, H., Wang, S., and Yin, D. From prompting to alignment: A generative framework for query recommen- dation. arXiv preprint arXiv:2504. 10208, 2025.
Davidson, J., Liebald, B., Liu, J., Nandy, P., Van Vleet, T., Gargi, U., Gupta, S., He, Y., Lambert, M., Livingston, B., et al. The youtube video recommendation system. In Pro- ceedings of the fourth ACM conference on Recommender systems, pp. 293 993-2996779 2010.
Park. Y.J. The adaptive clustering method for the long tail problem of recommender systems. IEEE Transactions on Knowledge and Data Engineering, 2508): 2012.
Ganea, O., Bécigneul, G., and Hofmann, 'T Hyperbolic neu- ral networks. Advances neural information processing systems, 31, 2018.
Park, Y.J. and Tuzhilin, A. The long tail of recommender systems and how to leverage it. In Proceedings of the 2008 ACM conference on Recommender systems, pp- 18, 2008.
Gong, Y., Jiang, Z., Feng, Y., Hu, B., Zhao, K., Liu, Q., and recommender system on edge in mobile taobao. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pp. 2477-2484, 2020.
Ou, W. Edgerec:
Ravasz, E. and Barabási, A.-L. Hierarchical organization in complex networks. Physical review E, 67(2):026112 12, 2003.
Hamilton, W., Ying, Z., and Leskovec, J. Inductive repre- sentation learning on large graphs. Advances in neural information processing systems, 30, 2017.
Rendle, S., Freudenthaler, C., Gantner, Z., and Schmidt- Thieme, L. Bpr: Bayesian personalized ranking from implicit feedback. arXiv preprint arXiv: arXiv:/205.26/8. 2012.
He, X., Deng, K., Wang. X., Li, Y., Zhang, Y., and Wang, M. LightGCN: Simplifying and powering graph convolution network for recommendation. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, pp. 639-648, 2020.
Sun, J., Cheng, Z., Zuberi, S., Pérez, F., and Volkovs, M. Hgef: Hyperbolic graph convolution networks for collab- orative filtering. In Proceedings of the Web Conference 2021, pp. 593-601, 2021.
Kipf, T. N. and Welling, M. Semi-supervised classifica- tion with graph convolutional networks. arXiv preprint arXiv: /609/ 02907, 2016.
Wang, B., Li, J., Chang, H., Zhang, K., and Tsung, F. Het- erophilic graph neural networks optimization with causal message-passing In Proceedings of the Eighteenth ACM International Conference on Web Search and Data Min- ing. pp. 829-88288888 003-837, 2025.
Kong, T., Kim, T., Jeon, J., Choi, J., Lee, Y.-C., Park, N., and Kim, S.-W. Linear, or non-linear, that is the ques- tion! In Proceedings of the fifteenth ACM international conference on web search and data mining, pp. 517-525, 2022.
Wang, X., He, X., Wang, M., Feng, F., and Chua, T.-S. Neural graph collaborative filtering. In Proceedings of the 42nd international ACM SIGIR conference on Research and development in Information Retrieval, pp. 165-174, 2019a.
Li, C., Xia, L., Ren, X., Ye, Y., Xu, Y., and Huang, C. Graph transformer for recommendation. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1680- 1689, 2023.
Wang, X., He, X., Wang, M., Feng, F., and Chua, T.-S. Neural graph collaborative filtering, In Proceedings of the 42nd international ACM SIGIR conference on Research and development in Information Retrieval, pp. 165-174, 2019b.
Li, Y., Zhang, X., Luo, L., Chang, H., Ren, Y., King. I., and J. G-refer: Graph estrieval-augmente large language model for explainable recommendation. In Proceedings of the ACM on Web Conference 2025, pp. 240-251, 2025.
Wei, Y., Liu, W., Liu, F., Wang, X., Nie, L., and Chua, T.-S. Lightgt: A light graph transformer for multimedia rec- ommendation. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development Information Retrieval, pp. 1508-1517, 2023.
Liu, Z., Nguyen, T.-K., and Fang, Y. On generalized degree fairness in graph neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37,
pp. 4525 4533, 2023.
10

---

Hgformer: Hyperbolic Graph Transformer for Collaborative Filtering
Wu, Q. Zhao, W., Li, Z., Wipf, D. P., and Yan, J. Node- former: A scalable graph structure learning transformer for node classification. Advances in Neural Information Processing Systems, 15:27387-27400 2022.
on Knowledge and Data Engineering, 37(6)3332-33326 2025. doi: 10.1 109/TKKDE:2223.558886
Yin, H., Cui, B., Li, J., Yao, J., and Chen, C. Chal- lenging the long tail recommendation, arXiv preprint arXiv:/205.0 1205.6700 2012.
Wu, Q-. Yang, C., Zhao, W., He, Y., Wipf, D., and Yan, J. Difformer: Scalable (graph) transformers in- duced by energy constrained diffusion. arXiv preprint arXiv:2301.0944 2023.
Ying, C., Cai, T., Luo, S., Zheng, S., Ke, G., He, D., Shen, Y., and Liu, T.-Y. Do transformers really perform badly for graph representation? Advances neural information processing systems, 34:28877-28888, 2021.
Wu, Q. Zhao, W. Yang, C., Zhang, H., Nie, F., Jiang, H., Bian, Y., and Yan, J. Simplifying and empowering transformers for large-graph representations. Advances in Neural Information Processing Systems, 36, 2024.
Yue, Y., Lin, F,, Yamada, K. D., and Zhang, Z. Hyperbolic contrastive learning. arXiv preprint arXiv:2302.014420 2023.
Xie, B., Chang, H., Zhang, 7., Zhang, Z., Wu, S., Wang. X., Meng, Y., and Zhu, W. Towards lightweight graph neural network search with curriculum graph sparsification. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 3563-3573, 2024.
Yun, S., Jeong, M., Kim, R., Kang, J., and Kim, H. Graph transformer networks. Advances in neural information processing systems, 32, 2019.
Yun, S., Kim, K., Yoon, K., and Park, C. Lte4g: Long-tail experts for graph neural networks. In Proceedings of the st ACM International Conference on Information & Knowledge Management, pp. 2434-2443, 2022.
Xihong, Y., Heming, J., Zixing, Z., Jindong, W., Huakang, N., Shuaiqiang, W., Yu, L., Junfeng, W., Dawei, Y., Xin- wang, L., En, Z., Defu, L., and Erxue, M. Darec: A disentangled alignment framework for large language model and recommender system. In 2025 IEEE 41rd International Conference on Data Engineering (ICDE). IEEE, 2025.
Zhang, L. and Wu, N. Hgee: Enhancing hyperbolic graph convolution networks on heterogeneous collab- orative graph for recommendation. arXiv preprint arXiv:2304.0296/ 2023.
Zhang, Y., Wang, X., Shi, C., Jiang, X., and Ye, Y. Hyper- bolic graph attention network. IEEE Transactions Big Data, 8(6): 2021a.
Xue, H.-J., Dai, X., Zhang, J.. Huang. S., and Chen, J. Deep matrix factorization models for recommender systems. In IJCAI, volume 17, pp. 3203 3209.. Melbourne, Australia, 2017.
Zhang, Y., Wang, X., Shi, C., Liu, N., and Song, G. Lorentzian graph convolutional networks. In Proceedings of the web conference 2021, pp. 1249-1261, 2021b.
Yang, M., Z., Zhou, M., Liu, J., and King, Hief: Hyper- bolic informative collaborative filtering. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 2212-2211 22211, 2022a.
Zhao, W. X., Mu, S., Hou, Y., Lin, Z., Chen, Y., Pan, X., Li, K., Lu, Y., Wang. H., Tian, C., et al. Recbole: Towards a unified, comprehensive and efficient framework for rec- ommendation algorithms. In proceedings of the 30th acm international conference information & knowledge management, pp. 4653-4664, 2021.
Yang, M., Zhou, M., Liu, J., Lian, D., and King, I. Href: En hancing collaborative filtering via hyperbolic geometric regularization. In Proceedings of the ACM Web Confer- ence 2022, pp. 2462-2471, 2022b.
Zhao, 7., Zhou, K., Wang, X., Zhao, W. X., Pan, F., Cao, Z., and Wen, J.-R. Alleviating the long-tail problem in conversational recommender systems. In Proceedings of the 1 17th ACM Conference on Recommender Systems, pp. 375-385, 2023.
Yang. M., Verma, H., Zhang, D. C.. Liu, J.. King, I., and Ying, R. Hypformer: Exploring efficient hyperbolic transformer fully in hyperbolic space. arXiv preprint arXiv:2407.00 01 2900 2024a.
Yang, X., Chang, H., La, Z., Yang, J., Li, X., Lu, Y., Wang, S., Yin, D., and Min, F. Hyperbolic knowledge transfer in cross-domain recommendation system. arXiv preprint arXiv:2406.1728 2024b.
Zheng, L., Lu, C.-T. Jiang, F., Zhang, J., and Yu, P. S. Spectral collaborative filtering. In Proceedings of the 12th ACM conference on recommender systems, pp. 311- 319, 2018
Yang, X., Wang, Y., Chen, J., Fan, W., Zhao, X., Zhu, E., Liu, X., and Lian, D. Dual test-time training for out-of- distribution recommender system. IEEE Transactions

---

Hgformer: Hyperbolic Graph Transformer for Collaborative Filtering
Appendix
A. Notations
summerize all the notations of this paper as follows:
Table 2. Notations used in this paper
Notation
Description
VA
V;
The user and item set Number of users
N
M
Number of items
N₂
The neighbors of node
E
Interactions between users and items
θ
Model parameters
d
Dimension of latent embeddings.
uf & &
User&item embeddings in Euclidean space.
u &&
User&item embeddings in hyperbolic manifold.
Ug & i (1) 1()
k-th and I-th vector i
User&item embeddings after 1-th layer
LHGCN
4. (h)
& k (h) (h)
&
i-th query, j-th key and value of h-head
u local & itooal
User&item embeddings after LHGCN User&item embeddings of hyperbolic cross attention.
global & ialobat
wiy
Similarity score of i-th user and j-item of h-th head in the self-attention i-th user embedding of h-th head after Euclidean weighted sum.
ú (h)
u,
(h)
i-th user embedding of h-th head after hyperbolic weighted sum.
Number of random features.
B. Preliminaries of Hyperbolic Geometry
In this section, we summarize key concepts from hyperbolic geometry that form the foundation of our method. Definition B.1 (Minkowski Inner Product). Let
(:)) p=+1 x
gn+1 R
be the bilinear map defined by
(u, v)M == 0n +
JJJ,
where J = diag(-1, 1) ((=+))x(--), Although (. is not a true inner product (because J has a negative eigenvalue), it commonly referred the Minkowski (pseudo) inner product.
12

---

Hgformer: Hyperbolic Graph Transformer for Collaborative Filtering
For any K > the condition
(x,X)== = -K
i.c.,
40
3 K K,
this defines two connected components, determined by the sign of zp. Note that zo 2 K or 2q S K. The condition To > 0 selects one of them.
Definition B.2 (Hyperbolic Manifold), Consider the following subset of I=+
Hr.. {x R=+1 (x, X)M K and zo > 0}.
This is often called n-dimensional Hyperbolic Manifold with curvature - k
The defining function hxx) (x, x). K has differential Dh(x) 2(x, U\u (2Jx)"u.
Since J is invertible and J if and only if O, we have Dh(x) + for all x H Hence, Dh(x) is surjective, implying H is smooth embedded submanifold of R of dimension (see, e.g., (Boumal, 2023, Definition 3.10 & Theorem 3,15)).
Definition B.3 (Tangent Space). For any x E Hn.K, the tangent space is given by
T.H.K kerx) {u R=+1; (x, u)M 0} {u R1+1
Σ x4 r44u}.
Zo un
Restricting the Minkowski product (t,))M to each T& H...K provides a Riemannian metric on Hr..K Thus, H becomes a Riemannian manifold with constant negative sectional curvature equal to -
Definition B.4 (North Pole). Define the north pole Hn..K by
o = (VK, 0, .) E Hm..K
Its tangent space satisfies
To T.H (u E m+ (o, u)M =0, 0, uo = 0} {(0, ((), u' E R²) s R..
For fixed dimension n, different curvatures - - yield different manifolds Hm..K each with its own north pole but all sharing the same tangent space structure
Definition B.5 (Hyperbolic Distance). The distance function on H induced by the Minkowski product is
(x))) = VK arcosh (x
K
for all y E Hr.K
Well-definedness Hyperbolic Distance:
We define two vectors in R" x' (VK, I Z.,.,, y' (VK, yI...., B.).
Since x.y E Hm.K implies ||x||2 = =o > 0 and ||'' yu > 0, we have
a
M( Zo yo Σ
a. |x Σ Ft + K =x (x.y") + K,
where ( on the right is the usual inner product in R..
13

---

Hgformer: Hyperbolic Graph Transformer for Collaborative Filtering
By the Cauchy-Schwarz inequality,
|x| I|y'll - (x,y)
(x, 2 K
(x.y) MM K
2 1.
Hence, arcosh( (x
well-defined, making d'A (x, y) well-defined for all x, y E HriKK
Definition B.6 (Exponential and Logarithmic Maps). For x E Hink and v E TyHu. with v of O, the exponential map is
Expx K (v)
costi [ivl vk
x + VK sinh [i
VR ||v|l
and for y E H..K with y # x, the logarithmic map is
dk (x
Log 2k (y)
(Proix(y)ll) Projx(y).
Example: Mapping R Hr.KK Following (Chami et al., 2019), x E R a Euclidean vector and == (vK,0..... be the north pole H We interpret xf) as a vector T, H" Then, using the exponential map,
* expo ((0,x)) (VK cosh
x 12), VK sinh VK
|x VK
x
Definition B.7 (Hyperbolic Centroid). Wet Hd+1,K be finite set of points {x:} with corresponding weights {w:}. The weighted centroid с Hd+1.K is defined the minimizer of
Σ X.EN
(c
min CEHUILK
closed-form solution (cf. (Chami et al., 2019)) is
ExEN X |EXEEN Xi
Centroid(N, w) = VK
When all weights are equal, we have
Centroid(A)) = VK EXEE
Ex
Definition B.8 (Hyperbolic Matrix Multiplication). Let W be a matrix in Raxe and x E Hr.K Following (Chami et al., 2019), the hyperbolic matrix multiplication is
W Kxxx== = Expo K/ W Loge (x))
where o is the north pole.
Definition B.9 (Parallel Transport). If x, Hn.K are connected by unique geodesic, then the parallel transport of T Hm.K to Ty Hm, along this geodesic is
Px Px-y(V) = (logx(y), v)M (10gx(x) + 1ogy(x))
(x)))
These definitions equip us with the fundamental tools for hyperbolic geometry. They serve as the basis for our approach described in this paper.
14

---

Hgformer: Hyperbolic Graph Transformer for Collaborative Filtering
C. Proof of Theorems
C.1. Proof of Theorem 3.1
Proof.
We set w (Os, I2) Rd. For any x E Rd we have
(2x) [ exp (-1|w - x du =
(15)
and for any x E Hd+1 - ........0 -K
Then by definition of HSM function, we have:
HSM(x,y) = exp ((x,))
(16)
exp(-xo- yo) exp (-11x12/22 exp (11--11) v12l12) exp (-151/1))/
(17)
multiple 1 and utilize Eq. 15, we get:
d Σ
(2x) (ar)-4/- 4/2 exp (1 / exp (-||w (X + //)13/2) 2113//2dd dw
= eep
(2x0080 +
2
+ Σ ////2
(18)
1=1
expand inside:
1200000 + x K + yd
K)
(2T) d/2 / exp(-1|w1//2 + w T - || + 323/22 || + 15133/232
= exp
/2
2
(19)
simplify the expression:
= exp
((10 yo)² + 2K
(2x)
(2x) 4/2 / exp (-1wwl1//2 (xx+ du
(20)
2
show this integration a form similar to Eq. 15
((20 + yn) + 2k)
(2T)) d/2 / exp (-1wol1//2) exp (w) exp (w du
= exp
(2mr)-//2
(21)
2
So it is the expectation of [exp(wTT T (x + {)]:
(((xo + 50)2 + 2K
exp
Bu- [exp(w] (x+ +)]
(22)
2
which shows that an unbiased estimation of HSM function.
15

---

Hgformer: Hyperbolic Graph Transformer for Collaborative Filtering
C.2. Proof of Lemma 3.1
Proof.
By definition of function Φ(x):
2K 2
-Yo
1 m
(x)))(())
o(y) expl 2
Texp (w] (R (x+ , exp (w + 8))
(23)
since the error between exp(2K - 2 v0/2 and exp (= + y00² + 2K/2) are small, and the remaining part the sampling function of the expectation, then we can take the approximation:
(2o + yo)² 2K
exp
Bw--NiosTa) [exp(w>] T (x + y)]
(24)
which equals ixx y), and by Eq. 8, also equals HSM(x, y), then complete the proof.
C.3. Proof of Theorem 3.2
Proof.
First, by the definition of error function:
A(x, y) = lexp ((x,))m) o(x) [xy))
(25)
take an expansion:
A(x,)) y)
exp
2K
(No + y0)
exp (x 3) exp
2K
Yi
1
exp
w (X y))
(26)
2
2
we set
exp
(-2030)
(27)
2K
r
yo
X =xxp
explws T (x + f))
(28)
2
then we have:
2K 1 v/
B(X) exp
F Fow [exp (x ()))
(29)
By Markov's inequality:
exp(3(5 - K))
E ((X aB(X))?)] me
P
A
(30)
exp(3(6 K))
take an expansion:
16

---

Hgformer: Hyperbolic Graph Transformer for Collaborative Filtering
E (x²) + (u² 2a) B² me
1
(31)
exp(3(5 K))
plug in the expectation:
= exp(II + v113) exp(2x y)
(32)
(1 + (a² - 22)) exp(- - 1 + Tlli)) me/ (exp(316 K)))
Since exp (-z00)) & <e-K, K E (0, 2o). we get:
> 1 - eexp 1 + 2x 2x y - -(6
(33)
We assume that ||x12 $ 5.11y11 & 6, then
8-K K
8 K
| I
11 v
(34)
and also have:
8- K
+ + 312 & 2 K)
(35)
2
then the last inequality holds:
e exp (1|x + 3 +
2 - 3(6 K)) > 1
(36)
Finally, we prove the upper bound of the error with probability:
exp(3(d K))
P
21-6
(37)
me
which completes the proof.
D. Details of Experimental Setup
In this section, we will describe the detailed settings of our experiments.
D.1. Baselines
To demonstrate the effectiveness our proposed model, we compare our model with four categories of models: (A)
Traditional models BPR(Rendle al., 2012) and DMF(Xue al., 2017). (B) GNN-based models NGCF(Wang et al., 2019b).
LightGCN(He et al., 2020) and HMLET. (C) Hyperbolic GNN-based methods HGCF(Sun al., 2021), HRCF(Yang et al., 2022b) and HICF(Yang al., 2022a). (D) Graph Transformer-basee methods SGFormer(Wa al., 2024), NodeFormer(Wa et al., 2022) and Hypformer(Yang) et al., 2024b). Specifically, BPR, DMF, NGCF, LightGCN, and HMLFE utilize the RecBole implementations, while HGCF, HRCF, HICF, SGFormer, NodeFormer, and Hypformer are migrated from their original implementations into RecBole.
We introduce the baseline models detail as follows:
17

---

Hgformer: Hyperbolic Graph Transformer for Collaborative Filtering
BPR (Rendle et al., 2012) is foundational matrix factorization approach for recommendation. It optimizee pairwise ranking objective that directly models preferences by enforcing higher scores for the positively interacted items than for the negative ones.
NGCF (Wang et al., 2019b) is a GCN-based recommendation model that exploits high-order user-item interactions by iteratively propagating embeddings through the user-item graph, thus capturing collaborative signals beyond direct neighbors.
LightGCN (He et al., 2020) is simplified GCN model for recommendation that removes feature transformations and non-linear activation functions, focusing purely on neighborhood aggregation to reduce complexity while maintaining strong performance.
HMLET (Kong et al., 2022) combines both linear and non-linear propagation layers to capture various aspects of user-item relationships, yielding improved performance on general recommendation tasks.
HGCF (Sun al., 2021) extends graph-based recommendation into hyperbolic geometry, aiming better model hierarchical or like structures in the user-item graph.
HRCF (Yang et al., 2022b) also a hyperbolic graph neural network for recommendation, which incorporates geometric regularization further enhance representation learning in hyperbolic space.
HICF (Yang et al., 2022a) leverages margin ranking loss in hyperbolic space, explicitly exploiting geometric properties for more accurate recommendation performance.
SGFormer (Wu al., 2024) is a linear graph transformer model designed for large-scale graphs, utilizing kernelliike attention mechanism to reduce the quadratic complexity typically associated with transformers.
NodeFormer (Wu et al., 2022) is another kernel-based graph transformer model that achieves scalability through linearized attention computations, making it suitable for large and dense graphs.
Hypformer (Yang et al., 2024a) is a hyperbolic graph transformer model, bringing transformer-style attention into hyperbolic space to better capture hierarchical structures and geometric relationships.
D.2. Datasets
To validate the effectiveness of the model, we conducted experiments on six datasets with varying sizes and densities, including Amazon datasets' (Amazon Book, Amazon CD, and Amazon Movie), well Douban datasets? (Douban Book, Douban Movie, and Douban Music). Detailed statistics of the datasets are summarized in Table 3.
Table 3. Dataset Statistics
Metric
Amazon-Book
Amazon-CD
Amazon-Movis
Douban-Bac Book
Douban-Movie
Douban-Mun Musicc
#User 211,170
66,317
26,969
18,086
22,041
15,996
#ltem
163,789
58,869
18,564
33,068
25,802
39,749
#Interactions
5,069,747
952.547
762,957
809,248
2,553,305
1,116,984
Density 0.015%
0.024%
0.150%
0.140%
0.459%
0.176%
D.3. Definition of evaluation metries
We used two metries for the evaluation. The first metric is Recall@K, which measures the fraction of relevant items retrieved out of all relevant items. The second metric is NDCG@K, which is measure of ranking quality in which positions discounted logarithmically. It accounts for the position of the hits by assigning higher scores to hits at the top ranks. The formal definition is given follows: We employed two metrics for evaluation. i) Recall@ which measures the fraction of relevant items retrieved out of all relevant items, which is formally defined as:
'https:// /// jmcaulep ey. www.. edu /atta ammazon Pttpps: www. douban. com
18

---

Hgformer: Hyperbolic Graph Transformer for Collaborative Filtering
Amazon
Amazon Movie
Douban Movie
Douban Music
Aetoooooo Aall
97.5%
Altrost rncamnnntedd Iarta
15.7%
95.7%
53.5%
$4.0%
The Sema ricommended hyparboiic
relatbely
14.0%
4.5%
0.7%
57%
40%
LgghGNN
LHOON
Hyformer
LighCCN
HOGF
LMCCN
Hgformer
LIgHGNN
HOCF
LHGON
Hgformar
LightCN
LNCCN
Hgformer
Propertion head items
Proportion of Tail Items
Amazon CD
Amazon Movie
Douban Movie
Douban Music
Tail Recall@@
Imp:
0.0813
Imp-
0.1942
0.0692
0.1485
0.1496
Imp.
0.1281
0.9624
0.3011
10
0.0678
HOCF
LHGON
Hgformer
HCCF
LHOON
Ngformer
HGGF
LHGEN
Hyformer
HOOF
LHOON
Hiformer
Figure 6. We define the top 20% of items in terms of popularity as head items and the rest as tail items. We calculated the proportion of head items and tail items among all recommended items for each model. To further analyze the performance of Hgformer tail items, we evaluate the performance of three hyperbolic-hased models on tail items.
R(u) n R(u)
Recall @
Σ NEW
(38)
12/
[R(u)]
where 2/ is the set of all users, R(u) represent a ranked list of items that model produces, and R(u) represent a ground-truth of items that user has interacted with. NDCG@K K, which is measure of ranking quality where positions are discounted logarithmically. It accounts for the position of the hits by assigning higher scores hits top ranks and it is formally defined as:
K 8(i E R(u))
NDCG@K
1
Σ
(39)
W
(uin(R((s)..X)
loga
wEll
==1
(og.(++1)
where 5(-) is an indicator function. In this research, we set K as 10 and 20.
E. Extra experiments
We implement Sensitivity Analysis and Analysis Tail Items to address the following questions:
RQ4: How well does Hgformer perform on the head and tail items?
RQ5: Is Hgformer sensitive to different hyperparameters?
E.1. Tail-item ana)ysis(RQ4)
In this section, we analyze the results on tail items demonstrate Hgformer's capability to mitigate long-tail tail issues.
1. TAIL PERCENTAGE ANALYSIS
We calculate the proportion of tail items recommended by each model and tail items are defined as those whose popularity ranks in the last 80%. For this purpose, we designed a metric called tail percentage, which is formally defined as follows:
EiER. 5(i T))
TailPercentage@K
(40)
U
R.
NEU
19

---

Hgformer: Hyperbolic Graph Transformer for Collaborative Filtering
Recall@10
Recal1@20
NDCG@10
NDCG@20
0.14
0.20
0.12
0.14
0.15
0.18
NOOHT
0.10
0.14
0.08
0.06
0.87
0.04
0.04
0.15
0.20
0.14
0.13
0.18
0.10
0.11
0.16
0.10
0.10
0.08
0.08
0.07
0.05
0.30
0.15
0.20
0.25
0.30
0.65
0.10
0.15
0.20
0.25
0.50
0.05
0.10
0.15
0.20
0.25
0.30
0.06
Amazon CD
Amazon Movie
- Douban Book
+ Doubaa Movie
Douban Music
Figure 7. Sensitivity analysis number of LHGCN layers and aggregation weight
where R,, is the set of items recommended user u, T the set of tail items and an indicator function. This metric gives the proportion of head items and tail items among all the items recommended by the model. We conducted evaluations on the Amazon CD and Amazon Movie datasets. As shown in the upper part of Fig. 6, it can be observed that the Euclidean space-based model (LightGCN) recommends almost only head items to users in both datasets (97.8 99.8% in the Amazon CD) dataset & 95.1 % in the Amazon Movie dataset, 99.3% in Douban Movie dataset and 99.5% in Douban Music dataset), This indicates Euclidean space-based models tend to overlook tail items in CF tasks and aggravate the Matthew Effect. Conversely, this phenomenon significantly mitigated by the three hyperbolic space-based models, showing that hyperbolic space is more suitable for the long-tail setting CF. It can be observed that the model's emphasis on tail items differs between the Amazon and Douban datasets, which primarily due their significant differences in the density and data distribution.
E. 1.2. ANALYSIS OF MODEL'S PERFORMANCE ON TAIL ITEMS
To further investigate the performance of each hyperbolic-basee model on tail items, in the second experiment, we evaluate the models' performance solely on tail items by calculating the recall@ 10 metric. As shown in Fig. 6, Hgformer significantly outperforms the other two hyperbolic space-based models on tail items. This is because Hgformer not only leverages the hyperbolic manifold to capture the hierarchical structure of the data but also introduces hyperbolic cross-attention which beneficial for capturing global information. This allows the model gather more information for tail nodes during the message- message-passing process, thereby improving the accuracy of recommendation.
E.2. Sensitivity ana)ysis(RQ5)
To evaluate the stability of our model, we conducted sensitivity analysis on four hyperparameters of our model: number of LHGCN layers and aggregation weight The results shown in Fig. 7.
1. Our model remains relatively stable within a certain range for both aggregation weight and the number of LHGCN layers. In the sensitivity analysis of aggregation weights, our model demonstrated relative stability in the range of 0.2 0.3, achieving better performance within this interval.
3. For the number of LHGCN layers, we observe that the optimal performance range of the model differs between the Amazon dataset and the Douban dataset. For the Amazon dataset, the optimal number of LHGCN layers falls within the range of 7, whereas for the Douban dataset, the optimal performance achieved roughly between and layers.
HR@K
Σ 8(N(( n R(u) #
UU
WEU
20


Reviews:
[{"review_id": "XN6uBufglt", "reviewer": "Reviewer_UU62", "summary": "The paper proposes a Hyperbolic Graph Transformer architecture, to tackle the long-tail problems in CF tasks, which leverages LHGCN for graph structure modeling and hyperbolic cross-attention for global information modeling", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q: Inconsistency between two aggregation approaches**\n\n**A:** Sorry for the misunderstanding. These two types of aggregation serve different purposes in our model. The aggregation in Section 2.2 indicates gathering neighbor information for each node to capture multi-hop relationships, which is performed in multiple layers—thus, any distortion at this stage would accumulate significantly. In contrast, the aggregation of local and global information in  Section 2.4 is simply fusing information from the two representation space for each node, which is a much simpler operation and involves negligible information distortion. In fact, aggregation step in Section 2.4 can be easily replaced by the weighted centroid of two embeddings defined in Definition B4 in the Appendix, and we conducted additional comparative experiments:\n| **Dataset**       | **Metric**  | **Our method** | **Hyperbolic aggregation** |\n|-------------------|-------------|----------------|----------------------------|\n| **Amazon CD**     | Recall@10 | 0.0977         | 0.0978                     |\n|                   | Recall@20 | 0.1401         | 0.1398                     |\n|                   | NDCG@10   | 0.0567         | 0.0572                     |\n|                   | NDCG@20   | 0.0678         | 0.0680                     |\n| **Douban Book**   | Recall@10 | 0.1462         | 0.1459                     |\n|                   | Recall@20 | 0.2052         | 0.2045                     |\n|                   | NDCG@10   | 0.1030          | 0.0995                     |\n|                   | NDCG@20   | 0.1189         | 0.1186                     |\n\nWe observe that the differences between the two methods are marginal.\n\n**Q: Implementations of SGFormer, NodeFormer and Hypformer** \n\n**A:** We adopted the encoder parts of these models straightforwardly and replace their output layers with recommendation-specific modules (For NodeFormer, SGFormer we applied BPR loss, which is the same as LightGCN and NGCF and for Hypformer, we applied Hyperbolic Margin Ranking Loss, which is the same as our hyperbolic baselines). The detailed implementations are publicly available in the anonymized GitHub repository in the Abstract.\n\n**Q: Lack of mathematical justification for LHGCN** \n\n**A:** Thanks for pointing out this issue. In the following, we provide a theoretical analysis supporting our claims. \n\nFor a set of hyperbolic points $N \\subset H^{d+1, K}$, for each point $x_i \\in N$, the centroid is defined as the point $c \\in H^{d+1, K}$ that minimizes the sum of squared hyperbolic distances:\n\n$Centroid(N) = \\arg\\min_{c \\in H^{d+1, K}} \\sum_{x_i \\in N} (d^K(c,x_i))^2 = \\arg\\min_{c \\in H^{d+1, K}} f(c).$\n\nIts closed-form solution is given by\n\n$c^* = \\sqrt{K} \\frac{\\sum_{x_i \\in N} x_i}{| \\sum_{x_i \\in N} x_i |_M}.$\n\nAnother way is to aggregate at a north pole point $o$ \n\n$\\bar{c}' = \\frac{1}{|N|}\\sum_{x_i \\in N} log_o(x_i),$\n\nand then map them back:\n\n$\\bar{c}^* = exp_o(\\bar{c}') = cosh\\Bigl(\\frac{|\\bar{c}'|_M}{\\sqrt{K}}\\Bigr)o + \\sqrt{K},sinh\\Bigl(\\frac{|\\bar{c}'|_M}{\\sqrt{K}}\\Bigr)\\frac{\\bar{c}'}{|\\bar{c}'|_M}.$\n\nSince $f(c)$ is convex in hyperbolic space (see [1]), it follows that\n\n$f\\left(c^*\\right) \\le f\\left(\\bar{c}^*\\right)$\n\nThis means that aggregation in Euclidean space is only a suboptimal solution and will lead to distortion.\n\n**Q: Is there validation set? How do authors get the results?**\n\n**A:** Thanks for the questions. To ensure accuracy and fairness in our experiments, all the models were tested on RecBole, a widely accepted framework in the field of recommender systems. The dataset was split into training, validation, and test sets with an 8:1:1 ratio. During training, we adopted an early-stop mechanism for all models, stopping training if no improvement was observed on the validation set for 30 epochs and the best-performing parameters on the validation set were selected for final evaluation on the test set. \n\n**Q: No statistical significance tests (t-tests, confidence intervals)**\n\n**A:** We respectfully clarify that we calculate p-values for our experiments, which is depicted in the caption of table 1. Here, we provide the specific p-values and t-values:\n| Dataset       | p-value | t-value |\n|---------------|---------|---------|\n| Amazon Book   | 0.023   | 2.5     |\n| Amazon CD     | 0.019   | 2.6     |\n| Amazon Movie  | 0.021   | 2.7     |\n| Douban Book   | 0.018   | 2.9     |\n| Douban Movie  | 0.027   | 2.4     |\n| Douban Music  | 0.038   | 2.3     |\n\n **Q: Essential References Not Discussed**\n\n**A:** Thanks for pointing out the problem. For \"Hyperbolic Attention Networks\", \"Performers\" and \"Nyströmformer\", we would like to add them into Graph Transformer part in Related Works and for \"Causal Intervention for Leveraging Popularity Bias in Recommendation\", \"SASRec\" and \"BERT4Rec\", we would like to add an extra section in Appendix for discussion.\n\n **Reference:**\n\n[1]Bacák M. Computing medians and means in Hadamard spaces.", "claims_and_evidence": "Are the claims made in the submission supported by clear and convincing evidence?", "methods_and_evaluation_criteria": "The author mentions not using tangent space for aggregation, but in the \"Embedding Aggregation and Optimization\" section, they still use aggregation, which seems inconsistent and unreasonable.\n\nRegarding the baselines, SGFormer, NodeFormer, and Hypformer were not originally designed for recommendation tasks. It remains unclear how the authors adapted these models for recommendation tasks and comparison purposes.", "theoretical_claims": "While the paper claims LHGCN performs \"graph convolution entirely in hyperbolic manifold without mapping back to Euclidean space\" (p.2), the mathematical justification for why this preserves information better than alternative approaches is more empirically than theoretically supported.", "experimental_designs_or_analyses": "is there any validation set for these experiments? How does the author get these results?\n\nthere are no statistical significance tests (t-tests, confidence intervals) to verify if these differences are statistically meaningful or potentially due to random variation.\n\nmissing Euclidean space transformer-based recommenders like SASRec and BERT4Rec\n\nregarding the baselines, SGFormer, NodeFormer, and Hypformer were not originally designed for recommendation tasks. It remains unclear how the authors adapted these models for recommendation tasks and comparison purposes.", "relation_to_broader_scientific_literature": "The linear-complexity approximation builds upon the theoretical framework of Linformer but adapts it for hyperbolic manifolds—a mathematical contribution extending beyond recommendations.", "essential_references_not_discussed": "The paper proposes Hyperbolic Transformer but does not cite \"Hyperbolic Attention Networks\" (Gulcehre et al., ICLR 2019), which specifically established the theoretical foundations for attention mechanisms in hyperbolic space. \n\nthe paper overlooks \"Performers: Rethinking Attention with Performers\" which introduced a kernel-based approximation method achieving linear complexity in Euclidean space using random feature maps.\n\nSimilarly, \"Nyströmformer\" is not discussed despite offering another established approach to linear-complexity attention.\n\nFor their long-tail recommendation contribution, the paper fails to cite \"Causal Intervention for Leveraging Popularity Bias in Recommendation\", which specifically addressed popularity bias with a causal inference approach.", "other_strengths_and_weaknesses": "None", "other_comments_or_suggestions": "None", "questions_for_authors": "None", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "None"}}, {"review_id": "VuddXsSCDt", "reviewer": "Reviewer_n8SP", "summary": "The paper introduces Hgformer, a novel Hyperbolic Graph Transformer framework designed to address two critical limitations in GNN-based collaborative filtering (CF): local structure bias caused by neighborhood aggregation and embedding distortion in Euclidean space. The proposed method combines a parameter-free Local Hyperbolic Graph Convolutional Network (LHGCN), which performs graph convolution entirely within the hyperbolic manifold to avoid information loss from tangent space projections, with a Hyperbolic Cross-Attention mechanism that captures global user-item interactions in bipartite graphs. To ensure scalability, the authors propose a linear-complexity approximation for the cross-attention, supported by theoretical guarantees of unbiased estimation and controllable error bounds. The numerical experiments show that Hgformer is superior to leading CF models. Moreover, compared with traditional hyperbolic graph neural network methods, this approach can further enhance the model’s performance on long-tail items. Overall, the work presents a compelling integration of hyperbolic geometry and transformers, advancing CF research with both theoretical and practical contributions.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q: Empirical validation of training/inference time on large-scale data is missing.**\n\n**A:** Thank you for pointing this out. We have conducted empirical evaluations and reported the average computational time per epoch for several representative baselines and our proposed model on Amazon Book (our largest dataset) and Amazon Movie. In each cell, the first value represents the training time per epoch, while the second value represents the testing time.\n| Dataset       | BPR         | LightGCN    | HMLET       | HGCF        | Hgformer    |\n|---------------|-------------|-------------|-------------|-------------|-------------|\n| Amazon Book   | 0.46s/0.74s | 2.22s/1.33s | 12.63s/1.28s| 6.06s/1.84s | 14.93s/1.92s|\n| Amazon Movie  | 0.22s/0.56s | 1.41s/0.71s | 7.41s/0.5s  | 3.03s/0.78s | 9.8s/0.83s  |\n\n**Q: Limited comparison with 2023–2024 hyperbolic CF models. How does Hgformer compare to recent hyperbolic CF methods in terms of performance and scalability?**\n\n**A:** Thank you for pointing out the issue. We have looked into recent literature from 2023–2024 and plan to include HyperCL[1], which extend contrastive learning into hyperbolic space and applied it for collaborative filtering, as one of our baselines. The results are as follows:\n\n| Dataset         | Metric                   | Our Method | HyperCL  |\n|----------------|--------------------------|------------|----------|\n| **Amazon Book** | Recall@10                | 0.9010      | 0.0756   |\n|                | Recall@20                | 0.1291     | 0.1189   |\n|                | NDCG@10                  | 0.0573     | 0.0482   |\n|                | NDCG@20                  | 0.0681     | 0.0582   |\n| **Amazon CD**   | Recall@10                | 0.0977     | 0.0828   |\n|                | Recall@20                | 0.1401     | 0.1221   |\n|                | NDCG@10                  | 0.0567     | 0.0496   |\n|                | NDCG@20                  | 0.0678     | 0.0598   |\n| **Amazon Movie**| Recall@10                | 0.0803     | 0.0776   |\n|                | Recall@20                | 0.1203     | 0.1186   |\n|                | NDCG@10                  | 0.0503     | 0.0492   |\n|                | NDCG@20                  | 0.0612     | 0.0594   |\n| **Douban Book** | Recall@10                | 0.1462     | 0.1392   |\n|                | Recall@20                | 0.2052     | 0.1968   |\n|                | NDCG@10                  | 0.1030      | 0.0976   |\n|                | NDCG@20                  | 0.1189     | 0.1126   |\n| **Douban Movie**| Recall@10                | 0.1405     | 0.1386   |\n|                | Recall@20                | 0.2068     | 0.2046   |\n|                | NDCG@10                  | 0.1322     | 0.1316   |\n|                | NDCG@20                  | 0.1447     | 0.1435   |\n| **Douban Music**| Recall@10                | 0.1386     | 0.1268   |\n|                | Recall@20                | 0.1955     | 0.1826   |\n|                | NDCG@10                  | 0.1024     | 0.0937   |\n|                | NDCG@20                  | 0.1165     | 0.1076   |\n\n\n**Q: Has a hyperparameter sensitivity analysis been conducted (e.g., curvature K, m)?**\n\n**A:** Thanks for the question. We provided a sensitivity analysis in Appendix E.2. However, we did not conduct a sensitivity analysis for K and m. We have now supplemented our work with an analysis for these two hyperparameters, and the results can be found at the link below. https://anonymous.4open.science/r/Hgformer-2AE0/sensitivity_analysis_rebuttal.pdf\n\n**Q:Can you clarify the negative sampling strategies used for the margin-ranking loss?**\n\n**A:** We adopted the default approach provided by the RecBole framework. Specifically, negative samples were uniformly drawn from items with which the user had no previous interaction. To ensure a fair and consistent comparison across different methods, all baselines utilizing negative sampling followed this same strategy and pair each positive sample with exactly one negative sample.\n\n\n**Q:There is inconsistent formula formatting (e.g., Equation 12 and lines 304–307).**\n\n**A:** Thanks for pointing our this inconsistency! We acknowledge that they can indeed be easily misunderstood. In lines 304–307, the $u^{final}$ and $i^{final}$  are derived from Equation 14, where $u^{global}$ and $i^{global}$ are obtained from Equation 5. Meanwhile, Equation 12 is introduced to reduce the computational complexity of Equation 3 (which leads to Equation 5) to linear complexity. We will add further clarifications in the paper to improve its readability.\n\n**References:**\n\n[1] Qin Z, Cheng W, Ding W, et al. Hyperbolic Graph Contrastive Learning for Collaborative Filtering[J]. IEEE Transactions on Knowledge and Data Engineering, 2024.", "claims_and_evidence": "Claim 1: LHGCN outperforms tangent-space projection-based methods (e.g., HGCF).\nEvidence: Figure 2(b) illustrates LHGCN’s direct hyperbolic aggregation vs. HGCF’s tangent-space mapping. Table 1 and ablation studies (Figure 4) show LHGCN achieves higher Recall/NDCG on most datasets.\nClaim 2: Hyperbolic cross-attention effectively captures global user-item interactions. Evidence: Removing the transformer module causes significant performance drops (e.g., -11.6% Recall@10 on Amazon Book). Case studies (Figure 5) highlight its ability to recommend distant tail items. Additionally, tail-item analysis (Appendix E) shows that Hgformer improves the recommendation of long-tail items compared to models without hyperbolic cross-attention.\nClaim 3: The linear-complexity approximation maintains performance while reducing costs.\nEvidence: Theorems 3.1–3.2 provide theoretical guarantees. Figure 3 visualizes the approximation workflow. However, empirical validation of training/inference time on large-scale data is missing.", "methods_and_evaluation_criteria": "Method Soundness:\n1.LHGCN’s design aligns with LightGCN’s simplicity, leveraging hyperbolic centroids for neighbor aggregation.\n2.The unbiased estimation approach for cross-attention is theoretically grounded but may require tuning hyperparameters (e.g., random feature dimension m).\nEvaluation Adequacy:\n1.Metrics (Recall@K, NDCG@K) are standard for CF.\n2.Baselines include hyperbolic GNNs (HGCF, HICF) and graph transformers (SGFormer), but some (e.g., SGFormer) are not CF-specific, potentially affecting fairness.", "theoretical_claims": "Theorems 3.1 (unbiased estimation) and 3.2 (error bound) in Appendix C are logically sound.", "experimental_designs_or_analyses": "1.Comprehensive experiments on 5 datasets with ablation studies and case analyses.\n2.Code availability enhances reproducibility.", "relation_to_broader_scientific_literature": "The paper connects hyperbolic GNNs (e.g., HGCF, HICF) and graph transformers, advancing CF through hyperbolic-local-global fusion.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "Strengths:\nNovel integration of hyperbolic geometry and transformers for CF.\nStrong empirical results and theoretical contributions.\nWeaknesses:\nLimited comparison with 2023–2024 hyperbolic CF models.\nThere is inconsistent formula formatting (e.g., lines 303–307 and Equation 12).", "other_comments_or_suggestions": "No.", "questions_for_authors": "1.\tLimited comparison with 2023–2024 hyperbolic CF models. How does Hgformer compare to recent hyperbolic CF methods in terms of performance and scalability?\n2.\tHas a hyperparameter sensitivity analysis been conducted (e.g., curvature K, m)?\n3.\tCan you clarify the negative sampling strategies used for the margin-ranking loss?\n4.\tThere is inconsistent formula formatting (e.g., Equation 12 and lines 304–307).", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "q7ivkB5NZJ", "reviewer": "Reviewer_A1RV", "summary": "The authors propose Hgformer: Hyperbolic Graph Transformer for Collaborative Filtering\tand conduct extensive experiments to\tanalyze the proposed method.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q: The two key problems addressed in the paper have already been extensively studied in previous works.**\n\n**A:** Thank you for pointing out these issues. As you mentioned, these two problems have indeed been discussed in prior works, and we have acknowledged these discussions in our Introduction section. However, as we emphasize, these challenges remain unresolved in the recommendation domain. Specifically, as demonstrated in Section E.1 of the Appendix, our proposed method addresses long-tail problems better than previous approaches. Moreover, simultaneously tackling both issues is non-trivial, and as highlighted in the Challenges parts of our Introduction, there are still significant difficulties that need to be overcome. Additionally, extending kernel functions to hyperbolic space to enable the application of hyperbolic transformers in recommendation is a novel approach.\n\n**Q: The combination of hyperbolic space and transformers has also been explored.**\n\n**A:** Thank you for pointing out the issue. We have also noted the paper \"Hypformer: Exploring Efficient Transformer Fully in Hyperbolic Space\" and discussed the key difference in the Related Works section, which can be summarized as:\n1) They focus on node classification, whereas our Hgformer is specifically tailored for collaborative filtering.\n2) We propose LHGCN for effective local structure modeling in hyperbolic space, which is not explored in their work.\n3) To achieve linear computational complexity, Hypformer first swaps the multiplication order of space-like values in self-attention vectors and then recalibrates the time-like values. In contrast, we adopt the cross-attention for better performance in CF tasks and directly extends the kernel function to hyperbolic space, which has a theoretical guarantee in approximation errors.\n\nOverall, we are the first to propose a hyperbolic graph transformer in the context of collaborative filtering.\n\n**Q: Using hyperbolic space introduces additional computational overhead. Is this trade-off justified? How does the computational cost of the proposed method compare with existing approaches?**\n\n**A:** Thanks for your question. The operations in hyperbolic space do not differ significantly from those in Euclidean space in terms of computational complexity.  The main parts of our model include LHGCN and Hyperbolic Graph Transformer. For LHGCN parts, In Section 2.2, we define the convolution in hyperbolic space by\n\n$$\n\\mathbf{u}_i^{(l+1)} = \\mathbf{Centroid}\\bigl(\\{\\mathbf{u}_i^{(l)}, \\{\\mathbf{i}_k^{(l)} : k \\in N_i\\}\\}\\bigr)\n$$\n and\n $$ \\mathbf{i}_j^{(l+1)} = \\mathbf{Centroid}\\bigl(\\{\\mathbf{i}_j^{(l)}, \\{\\mathbf{u}_k^{(l)} : k \\in N_j\\}\\}\\bigr).$$\n\nAccording to Definition B.7 in Appendix B, the above Centroid formula still yields an overall complexity  $O(|E| \\times d)$ for an entire pass over the graph, where $|E|$ is the number of edges and $d$ is the embedding dimension and we find that the computational complexity is similar to that of the LightGCN convolution process. \n\nThe complexity of the Hyperbolic Graph Transformer is detailed in Figure 3 in our paper and we can find that it is consistent with that of the NodeFormer, which is a graph transformer model in Euclidean space and is capable of dealing large-scale graphs. Hence, the hyperbolic variant retains the same order of computational complexity as its Euclidean counterpart. \n\n**Q: The hyperparameter settings for the baseline models are not specified in the paper.**\n\n**A:** Thanks for pointing out the problem. Below we provide the hyperparameter tuning ranges for baselines.\n\n| **Model**    | **Hyperparameters (Ranges)**                                              |\n|--------------|---------------------------------------------------------------------------|\n| **BPR**      | learning_rate: (2e-3, 1e-3, 5e-4)                                         |\n| **NGCF**     | learning_rate: (2e-3, 1e-3, 5e-4); node_dropout: (0.0, 0.1); message_dropout: (0.0, 0.1) |\n| **LightGCN** | learning_rate: (2e-3, 1e-3, 5e-4); n_layers: (2, 3); reg_weight: (1e-4, 1e-5) |\n| **HMLET**    | learning_rate: (2e-3, 1e-3, 5e-4); n_layers: (3, 4); activation_function: (elu, leakyrelu) |\n| **HGCF**     | learning_rate: (2e-3, 1e-3, 5e-4); n_layers: (2–5); scale: 0.1; learner: (adam, rsgd); margin: (0.1, 0.15, 0.2) |\n| **HICF**     | learning_rate: (2e-3, 1e-3, 5e-4); n_layers: (2–5); scale: 0.1; learner: (adam, rsgd); margin: (0.1, 0.15, 0.2) |\n| **HRCF**     | learning_rate: (2e-3, 1e-3, 5e-4); n_layers: (3–7); scale: 0.1; learner: (adam, rsgd); margin: (0.1, 0.15, 0.2) |\n| **SGFormer** | learning_rate: (2e-3, 1e-3, 5e-4); n_layers: 1; weight_decay: (1e-5, 1e-4, 1e-3); dropout_ratio: (0.0, 0.2, 0.3); num_heads: (2, 3) |\n| **NodeFormer** | learning_rate: (2e-3, 1e-3, 5e-4); n_layers: (2, 3); rb_order: (1, 2); num_heads: (2, 3)                                     |\n| **Hypformer**  |  learning_rate: (2e-3, 1e-3, 5e-4); n_layers: 1; num_heads: (2, 3); margin: (0.1, 0.15, 0.2)", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "Yes.", "experimental_designs_or_analyses": "Yes.", "relation_to_broader_scientific_literature": "Good.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "Strengths\n\n1.\tThe\tpaper is well-structured.\n\n2.\tThe\texperiments are comprehensive.\n\nWeaknesses\n\n1. The two key problems addressed in the paper, local structure modeling and embedding\tdistortion, have already\tbeen extensively studied in previous works.\n\n2. The combination of hyperbolic space and transformers has also\tbeen explored\tbefore. For\tinstance,\tsee: Hypformer: Exploring efficient transformer fully in hyperbolic space, KDD 2024.\n\n3. Using hyperbolic space introduces\tadditional computational overhead. Is\tthis trade-off\tjustified?\t Moreover, how\tdoes\t the\tcomputational\tcost\tof the proposed\tmethod\tcompare\twith\texisting\tapproaches?\n\n4. The hyperparameter settings\t for\tthe baseline models are\tnot\tspecified\tin the paper.", "other_comments_or_suggestions": "See\t“Other\tStrengths\tAnd\tWeaknesses”", "questions_for_authors": "None", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: Post-rebuttal, reviewers have diverse evaluation scores (weak reject, weak accept, accept). Based on such scores, I recommend acceptance. Please ensure that the suggested revisions are incorporated into the final version, particularly raised by Reviewer UU62:
- Limited novelty: The use of the centroid as an average in Lorentz GCN appears to overlap with previous methods such as LGCN and related frameworks.
- Statement-implementation inconsistency: Although the paper discusses issues with the tangent space, the method still relies on tangent space operations when integrating global and local information.
- Unfair comparison: The implementation of baseline methods, as described by the authors, may not faithfully reflect their intended usage, potentially affecting the fairness of the comparisons.
Decision: Accept (poster)