SHARP-Distill: A 68× Faster Recommender System with Hypergraph Neural Networks and Language Models
Authors: Saman Forouzandeh
Parham Moradi
Mahdi Jalili
Venue: ICML 2025 poster (2025)
Keywords: Hypergraph Learning
Knowledge Distillation
Contrastive Learning
Recommendation Systems
Primary area: deep_learning->graph_neural_networks
TL;DR: SHARP-Distill: Knowledge Distillation with HGNNs and Language Models for Fast Recommendations

Abstract: This paper proposes SHARP-Distill (\textbf{S}peedy \textbf{H}ypergraph \textbf{A}nd \textbf{R}eview-based \textbf{P}ersonalised \textbf{Distill}ation), a novel knowledge distillation approach based on the teacher-student framework that combines Hypergraph Neural Networks (HGNNs) with language models to enhance recommendation quality while significantly improving inference time. The teacher model leverages HGNNs to generate user and item embeddings from interaction data, capturing high-order and group relationships, and employing a pre-trained language model to extract rich semantic features from textual reviews. We utilize a contrastive learning mechanism to ensure structural consistency between various representations. The student includes a shallow and lightweight GCN called CompactGCN designed to inherit high-order relationships while reducing computational complexity. Extensive experiments on real-world datasets demonstrate that SHARP-Distill achieves up to 68× faster inference time compared to HGNN and 40× faster than LightGCN while maintaining competitive recommendation accuracy.

SHARP-Distill: A 68x Faster Recommender System with Hypergraph Neural Networks and Language Models
Saman Forouzandeh
Parham Moradi Mahdi Jalili
Abstract
multiple sources of information such as social trust(Fan et al., 2019a) or user reviews (Fan et al., 2019b) in the recommendation process. The second challenge is the in- ference time, as real-world GNN-based methods typically employ deep GNN layers capture high-order relationships. However, this depth significantly increases computational overhead, leading to longer inference times. While HGNNs (Feng et al., 2019) can capture high-order relationships, they also require a high computational resources during inference time(Yu et al., 2024). Recent approaches like GLNN (Zhang al.. 2021). KRD (Wu et al., 2023), and LightHGNN (Feng et al., 2024) have attempted to address this issue through the teacher-student knowledge distilla- tion approach. Recent approaches, such as GLNN (Zhang et al., 2021), KRD (Wu et al., 2023), and LightHGNN (Feng et al., 2024), have attempted to address this issue using the teacher-student knowledge distillation framework. How- ever, they often struggle to effectively transfer the complex structural information learned by the teacher model. These methods primarily rely soft labels for knowledge trans- fer, which fail to capture intricate high-order relationships and structural dependencies. As result, soft labels alone are insufficient for preserving the rich structural knowledge inherent in hypergraphs. This limitation raises an important question:
This paper proposes SHARP-Distill (Speedy Hypergraph And Review-based Personalised Dis- tillation), a novel knowledge distillation approach based on the teacher-student framework that com- bines Hypergraph Neural Networks (HGNNs) with language models to enhance recommenda- tion quality while significantly improving infer- ence time. The teacher model leverages HGNNs to generate user and item embeddings from inter- action data, capturing high-order and group rela- tionships, and employing a pre-trained language model to extract rich semantic features from tex- tual reviews. We utilise contrastive learning mechanism to ensure structural consistency be- tween various representations. The student in- cludes shallow and lightweight GCN called CompactGCN designed to inherit high-order rela- tionships while reducing computational complex- ity. Extensive experiments on real-world datasets demonstrate that SHARP-Distill achieves up to 68x faster inference time compared to HGNN and 40x faster than LightGCN while maintaining competitive recommendation accuracy.
Q1: How can we design knowledge transfer methodology that effectively transfers the learned knowledge from pre- trained HGNN to a lightweight model for recommender systems, while maintaining recommendation accuracy?
1. Introduction
Recently, GNNs have gained significant success in recom- mender systems due to their ability to model complex rela- tionships between users and items Wu et al., 2022). How- ever, despite their advantages, existing GNN-based recom- mender systems still face several challenges. First, they rely heavily on the user-item interaction matrix, which is often sparse, leading to difficulty in accurately capturing interactions. A possible solution to this issue is utilising
Q2: How can we efficiently align heterogeneous embed- dings derived from different information sources while pre- serving their unique characteristics?
Q3:How can both structural and positional information of complex networks be effectively leveraged in embedding representations?
'Equal contribution 'School of Engineering,
In this paper, we propose SHARP-Distill a knowledge dis- tillation framework combining HGNNs and pre-trained lan- guage models for high-speed recommender systems. Our method follows teacher-student strategy, where the teacher model integrates HGNNs to capture high-order user-item interactions and DeBERTa (He et al., 2020a) as pre-trained
RMIT University, Melbourne, Australia. Correspondence to: Saman Forouzandeh <saman. naman..orouzandchhe wereeddhhhmmiiiiuuuuu.. cdu..cd
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 20255 by the author(s).

---

SHARP-Diste 68x Faster Recommender System
language model to extract rich textual representations. We employ contrastive learning (CL. mechanism effectively align embeddings from multiple sources, such as user-item interactions and user reviews, ensuring cohesive repre- sentation while preserving essential information from each modality. The student model includes a lightweight and single-layer GCN called CompactGCN without non-linear activation, designed for efficiency while retaining high-order dependencies. The student model also employs a contrastive learning (CL) mechanism to align embeddings generated by CompactGCN with those produced by the deep HGNNs in the teacher model. This alignment facilitates the effective transfer of structural and positional knowledge from the teacher to the student, ensuring the preservation of high- order relationships while maintaining computational effi- ciency (Forouzandeh et al., 2025). This enables Compact- GCN to inherit essential relational knowledge while sigmifi- cantly reducing computational costs. Extensive experiments on real-world datasets show that SHARP-Distill achieves up to 68x faster inference than HGNN and 40x faster in- ference than LightGCN while maintaining or surpassing ofate-of-thh-ar recommendation accuracy. Contributions of the Proposed Method are:
Hiz 1 if node is is connected to hyperedge 1 and 0 otherwise. This matrix effectively captures high-order in- teractions among nodes, making it suitable for modeling heterogeneous graphs. To convert a hypergraph into a homo- geneous graph, we can compute a projected adjacency ma- trix A E g=*= as A = HWH - D,.. where W E gm*x is the hyperedge weight matrix, and D is the diagonal node degree matrix, with (i, i) ンタ= Hij. This transfor- mation reduces the hypergraph two nodes are connected if they share common hyperedges.
traditional graph where
Hypergraph Neural Network (HGNN) is a powerful ex- tension of the GCN designed to capture high-order rela- tionships among nodes by leveraging the structure of hy- pergraphs. HGNNs use the hypergraph Laplacian, derived from the node degree matrix D, and the hyperedge degree matrix De. to propagate information across the hypergraph. This is achieved through a peessage-passing mechanism that updates node features using the formula:
( =o (D. 'HH D2 1/2 D2 122"2"HH00 )00 (I)
where (()) represents the learnable weights and (.) is a non-linear activation function. This formulation allows HGNNs to effectively aggregate and propagate features, accounting for the structure of the hypergraph, thus en- abling the modeling of complex dependencies between nodes. HGNNs have demonstrated their versatility and effectiveness in various applications, including recommen- dation systems, social networks, and biological data analysis (Feng et al., 2019).
1. SHARP- effectively captures both structural pat- terns from user-item interactions and semantic features from textual reviews.
2. We introduce CompactGCN, lightweight single-layer GCN without non-linear activations, is specifically designed to inherit high-order relationships from the teacher while significantly reducing computational complexity.
DeBERTa drecoding-enhancee BERT with Disentangled Attention) (He et al., 2020a) is pre-trained language model that extracts contextual embeddings from text through dis- entangled attention mechanisms. For each user-item pair (u;, 1t)) with an associated review ray, we process the text through DeBERTa to obtain contextual representations, which we denote as X' = DeBERTa(X) where X' rep- resents the output embedding after passing the input text X through the model.
3. Our contrastive learning- Deanning-based distillation approach in- tegrates structural and positional similarities, ensuring that the student model effectively captures graph topol- ogy and node relationships while maintaining compu- tational efficiency.
2. Preliminaries
Graph and Hypergraph: A graph G EV con- sists of a set of nodes V and edges E, where each edge e = (v, ny)) connects two nodes, it and uy. The struc- ture of a graph can be represented by its adjacency matrix A E R=x*, where Aij- = 1 if there is an edge between nodes 0s and Vy, and 0 otherwise. This representation is limited to pairwise relationships between nodes. A hy- pergraph 7 (V,E), on the other hand, generalises the concept of a graph by allowing hyperedges E E to con- nect any subset of nodes, enabling the representation of complex, multi-node relationships. A hypergraph is rep- resented by its incidence matrix H. E R=*m, where n is
3. Methodology
We propose novel teacher-student knowledge distillation framework for a high-speed recommender system. As illus- trated Figure 1, our framework integrates HGNNs with pre-trained language model (PLM) to effectively capture high-order relations and textual semantics. By integrating multiple information sources, such as user-item interactions and user reviews, our framework aims to address challenges including data sparsity, cold-start problems, and noisy in- teractions, thereby ensuring more accurate and efficient recommendations. Additionally, our framework transfers knowledge to a lightweight student model, significantly en-
the number of nodes, m is the number of hyperedges, and
2

---

SHARP-Disttll. 68x Faster Recommender System
Training seachermoder
Z
DeBERTa
Contrastive Loss
Inference Model
User Review
Laseske Lane + AL Low
yt Soft Labels
HGNN
Incidence
MLP
Matrixes
CompactGCN
HGNN
User Graph
36
x
zZ
DeBERTa
Contrastive
MLP
Loss
20
((4,
Loss
zii
CompactGCN
MLP
Suop
Estudent
faster than HGNN
40x Faster then I LightGCN
Loss
Training mludent
Figure 1. SHARP-Distil framework architecture. The Teacher model integrates dual Hypergraph Neural Networks (HGNNs) with DeBERTa capture high-order relations in user-item interactions and semantic features from textual reviews through contrastive learning. The Student model employs CompactGCN enhanced with contrastive learning to align its embeddings with teacher representations. Knowledge distillation combines structural and positional knowledge transfer, achieving 68x and 40x faster inference times compared HGNN and LightGCN, respectivoly.
hancing inference speed while maintaining recommendation accuracy. The details of each component are described in their respective subsections.
To enhance alignment between textual embeddings from De- BERTa (Z Z and structural embeddings from HGNNs (Zi. Zł),, we employ cross-modal and intra-domain con- trastive learning. The cross-modal contrastive loss aligns HGNN and DeBERTa embeddings:
The teacher model integrates three key components: two HGNNs for modeling high-order user- item interactions, DeBERTa pre-trained language model for extracting semantic features from textual reviews, and Multi-Layer Perceptron (MLP) for predicting ratings. The HGNNs capture high-order relations in the user-item inter- action graph using the hypergraph Laplacian. Let us define the user incidence matrix as Hy E _R= where n denotes the number of users and m represents hyperedges (items). The normalised hypergraph Laplacian is defined as:
Teacher Model:
|8x
exp(sim 2 |1, 2())//
I T
log
[5x
exp Zx [1] /LD/T)
where X E (U, sim(..) is cosine similarity, and is
temperature parameter. The intra-domain contrastive loss is:
SSx
exp(sim( Z [1, (z)))/
L = D= W DD
(2)
I LXS
[S
log
Σ
18x Dk=1 exp(sim Z M1, 2 [A))/T)
where D, m and De E mxm are diagonal degree matrices for vertices and hyperedges, and W Rmxm is the hyperedge weight matrix. The message propagation at each layer| 7 follows:
where Sy and S₁ denote user and item sets, respectively. After contrastive alignment, the HGNN embeddings and textual embeddings from DeBERTa are fused via k-layer MLP:
(0+1) (D. (D.MPHOWD:
(3) where a-) is the ReLU activation function, and (() E Rdxd is the trainable weight matrix at layer The final user and item embeddings after layers are Zi = (1) E
h = o (Z
+ bb))
(6)
= = by), lE k (7) where || denotes concatenation. Wi and by are trainable parameters, and (-)) ReLU activation. The output layer
Rad and 7 = (L) E mm respectively.
3

---

SHARP-Distill. 68x Faster Recommender System
of the MLP generates rating predictions:
E I is the degree matrix and is the identity matrix. The initial node emheddings are computed
f = oour /1 W + bk),
(8)
z A A
(14)
where (-)r()) is the sigmoid activation for rating prediction. The supervised loss component is the Mean Squared Error (MSE) loss:
where W E R h is the transformation matrix for struc- tural features, h denotes the hidden dimension of the output embeddings, and Z E RR represents the learned node embeddings. Following Hinton et al. (Hinton, 2015), the supervised loss combines hard and soft targets:
mms
1
Σ (N.))Y
(Y
(9)
Y
where Yi is the ground truth rating. The final teacher ob- jective combines supervised and contrastive components:
Lap = (1 L BL
(15)
where 3 E [0, 1] balances the contributions of hard and soft targets. The two loss components are:
Lueccer
Lmse +
(I L
(10)
XEIUI
Lhhrdd
1
Σ
(Ynj VG9
where À is a trade-off parameter balancing the objectives. The teacher's predictions are then converted soft labels using temperature scaling for knowledge distillation:
Y
(i.j)EY
(16)
LiO KL /T Y*/T).
exp(Y"t (sp(\'/T) /T) Σ, exp(Yx//T) /T)
where the 2 factor ensures proper gradient scaling (Phuong & Lampert, 2019).
Y = softmax
(II)
Contrastive Learning for Knowledge Transfer: Tradi- tional knowledge distillation methods primarily rely on soft labels, which are insufficient for capturing the rich struc- tural information inherent in hypergraphs and the complex multi-modal relationships between user-item interactions and textual reviews. To address this limitation, our ap- proach employs contrastive learning to align representations between teacher and student models, offering several key ad- vantages: (1) Structural preservation: Unlike soft labels that only provide prediction-level guidance, contrastive learning explicitly preserves the relational structure and topological
where 1 is the temperature parameter following an annealing schedule:
T(e) = Tmxx - (Tmax Timn) min (1, e
(12)
For the student model, we employ a crucial embedding interpolation mechanism that facilitates effective knowledge transfer. This interpolation combines the independently learned student embeddings with the well-trained teacher embeddings:
properties learned by the teacher's 's HGNNs; (2) Multi-modal aligument: It effectively bridges the semantic gap between different representation spaces (hypergraph embeddings vs. lightweight GCN embeddings); (3) Robustness to noise: Contrastive learning is more robust to noisy interactions compared to direct embedding matching, as it focuses on relative similarities rather than absolute values.
= >Z% N.. ] +1 (1.
(13)
-2 :, + (1)
The interpolation coefficient E [0, 1] controls the bal- ance between student and teacher embeddings. z Nu.:] and 2*[N, : represent the student's user and item em- beddings, while Zi and Zi the teacher's embeddings. When y is near 0, the student relies on the teacher's 's knowl- edge, which is useful early in training. As y increases, the student transitions to its own learned representations.
To effectively transfer knowledge, the student model lever- ages contrastive learning incorporating both embedding- based similarity and hypergraph-inforeee ilarity. The hypergraph positional encoding captures the teacher's topological knowledge:
topological sim-
The student model consists of a lightweight GCN called CompactGCN along with an MLP. CompaetGCN is specifically designed to efficiently inherit essential struetural and topological information from the teacher model, ensuring balance between computational efficiency and recommendation accuracy. Let X E Rn be the feature matrix of users/items, and A E R repre- sent direct user-item interactions. The normalised Laplacian matrix is defined as Å = DA + T)D- 1/2 where
Student Model:
Σ 3h (D, (D:1/2HoDD (00,D,1111)) t eus (17) 1=1
K
Pa =
where eu is the one-hot vector for node u, K denotes the maximum propagation steps, and 3 E (0, 1) is a decay factor. This formulation captures both local and higher- order dependencies in the hypergraph topology. We define

---

SHARP-Distill: 68x Faster Recommender System
comprehensive similarity function that integrates both emhedding- based and positional similarities:
CDs, Cellphones, Beauty, and Sports a-are used as well, as referenced in (Zhang et al., 2024). All of these datasets contain user interactions, items, and user reviews of the items. In this research, we construct two hypergraphs: one for users and one for items. Based on these, we generate two sets of review-basee embeddings items-by bby processing the reviews submitted by users and the reviews received by items from various users. A sum- mary of the dataset statistics is provided in Table 6 in the Appendix C. We evaluate the proposed method based on the generated recommendations in two key aspects: accuracy and inference time. For accuracy, we use the metrics Pre- cision @k (P@k), Recall@k (R @ k), F1-Score (F@k), and NDCG@k (N@k), where 10 is used during the testing phase. Table presents the accuracy results across the five datasets. Each experiment was conducted 10 times using different random seeds, and we report both the average per- formance and standard deviation. For each dataset, 20% of the data was used for validation and 10% for testing. We evaluate SHARP- using four distinct methodological approaches to provide comprehensive performance analysis.
Z . z u(,, = ( 2 ||1 ) + (1-10) a)) (1P2 (1P_111771
Pu. P.
(18) where Z and Z are the embeddings from the teacher and student models, respectively. Pa and P. represent the hypergraph positional encodings, and E [0, I] controls the trade-off between embedding similarity and positional similarity. The contrastive learning objective aligns user and item representations between the teacher and student models. For user alignment:
for users and one for
1 U
exp(S(u, u)/T)
L
log
(19)
EVEr exp(S(u,))) u)/T)'
NEU
and for item alignment:
1
Σ log exp(S(i, VE!
T con [T]
(20)
Σ jEI exp(S(i,))/v)'
where U and I represent the sets of all users and items, respectively. The temperature parameter 0 controls the concentration of the similarity distribution. Here, S(u, u) denotes the similarity between user's 's representations in the teacher and student models, while Su, v) measures the similarity between different users. The final student loss integrates supervised learning, knowledge distillation, and contrastive components:
Hypergraph- Methods: We benchmark SHARP-
Distill against state-of-the-art hypergraph neural networks: LightGCN (He et al., 2020b); Hypergraph Contrastive Col- laborative Filtering (HCCF) (Xia et al., 2022); and Hy- pergraph Attention (HGAtt) (Bai et al., 2021). We also include the Graph-centric Contrastive Framework for Graph Matching (GCGM) (Bo & Fang, 2024), which exemplifies recent advances in contrastive learning on graph structures. Large Language Model-based Methods: We evaluate two LLM-enhanced recommendation approaches: SAID (Hu et al., 2024), which integrates semantic knowledge from large language models, and Prompt Distillation (POD) (Li et al., 2023), which transfers knowledge via prompt-based learning. Knowledge Distillation Methods: We compare against teacher-to-student distillation frameworks that lever- age hypergraph structures: KRD (Wu et al., 2023) and LightHGNN (HGNN-to-MLP) (Feng et al., 2024). Table I presents a comprehensive evaluation across five real-world datasets, demonstrating SHARP-Distill 's superior perfor- mance on multiple metrics.
Leaudent uu
X1 T
con + +X2I-Loo
(21)
where Lup combines hard and soft targets, and hyperparam- eters X2 > O control the contributions of user and item contrastive learning objectives. This formulation facilitates effective knowledge transfer by explicitly aligning corre- sponding user and item representations between teacher and student models while preserving their relative relationships in the embedding space. The framework enables the student model to inherit both sabbedding-based and hypergraph- informed topological knowledge, ensuring efficient repre- sentation learning through CompactGCN while maintaining the structural richness of the teacher model. Theoretical proofs and supporting lemmas for the model are provided in Appendix A. The complete training procedure is detailed in Algorithm 1 in Appendix B.
SHARP-Distill demonstrates superior performance across most metrics and datasets, achieving the best results in 11 out of 15 metric-dataset combinations. Particularly notable improvements
observed in the Sports dataset, where SHARP-Distill significantly outperforms all baselines across all metrics. The method shows strong competitive performance against LLM-based approaches, with SAID achieving best performance in Cellphones P@ 10 (7.83) and sharing best performance with HGAtt in CDs N 10 (12.24), demonstrating the effectiveness of semantic knowledge in-
4. Experiments
In our experiments, we utilise five well-known datasets. First, we use the Yelp Tips dataset'. Additionally, we lever- age the Amazon datasets, including four more datasets, cov- ering different domains within the Amazon collection-
/tmp:///mcaullyy eyy. www.llyyycccccc.o.o ucsd. leeeddeddueeee adu/data// vataaamazon
'tttps: http:p::///ww..// yelp. com/ com/dat
5

---

SHARP-Distill 68x Faster Recommender System
Table 1. Comprehensive experimental results on five real-world datasets. All metries are presented as percentages (%) with standard deviations. Bold indicates best performance in each category.
Hypergruph Methods
LLM Methods
Distillation Methods
Dataset
Metric
SHARP-Distill
LightGCN
HCCF
HGAI
GCGM
SAID
POD
KRD
LightHGNN
10
2.27+
3.52+
3.32=0.038
3. 40.02
3.15
3. 3.2800.005
3. 30.00
3.27=0.084
3. 3.88=0.047
Yelp
R@
1.87 18.00
2.06
2.53_
2.84 Lo.0ss
2.
2.6510.07
2.41 10.072
_.17_
2.75_o.00
10
1 1.2640.051
2.25+0.000
2.07=0.051
2. 20.077
1.9540.007 067
2. 11 coo
1. 1.7300.050
1.7420.000
2.37=0.04
Pde 10
13.96 +0.06
13.21
13.5420.com
13.40
12.88 000
12.17=0.182
12.40+0.115
13. 13.76+0.076
CDs
R@:10
10.14
12. 12.05_m Lo.
12.47 10.091
12.65 22.65_..o ces
Lo.ows
12.20 12:20_oomm Lo
11.56 10.305
1o.0s7
13.06 1o.002
1@:0
9.76
11.70=0.068 00.0
12.2440.101
12.08=0.co4 .04
12. 00.04
11.8820.008 0006
10.77
11.70
12. #0.000
10
6.18
7.68
7.79_00007
7.8210.032
7.83 045
7.622n.om
10.05
6.77_m.or
g.04
Cellphones
R@10
4 4.27+0.a55
5.588n.028
5.22=0.000
5.35
5.72+0.059
5.60+0.044
4. 4.6290.079
4. 4.88=0.000
5. 3.77=0.044
N@
3.66
4.30
4.67
4.581
4.6910.002
4.62gn.ess
3.890
3.95_m.00
4.77gg000
P@
4.27+0.067
5.63en.018
6.05-o..2m
5.95
6.58
6.60+a.ca1
5.53
6.01-0.048
6.97=0.002
Beauty
1@:0
3.35 asa
3.85
0.6
4.4520.005
4.38 071
4. 2.79
2.0
4.0 4.08 104
4.52g0 0es
10
2.93+0.044
3.38+n.048
3.76-0.00
3.65+0.042
3.97
4.01+0.001
3. 3.41+n.072
3.55-0.088
4.15-0.074
10
2.57 031
3.15g0.0s
0.5
3.4820.035
3.22 oss
3.41g0.ca
3. 3.36g0.045
3 3.4200.02
4.27.0.00
Sports
R@10
1. g.83
2.42
2.71
g..5r
2.35
2. 2.58
2.02 ku.eu
2.23
3.63 3.63m
Nº10
1.10
1.55g0.01
1.7420
1.6820.032
1.48
1.67 can
1.4884.030
1.6420.o
3.24
tegration. However, SHARP- maintains dominant performance across recall and NDCG metrics, excelling in 7 out of 10 recalI/NDCGG combinations. The results validate the effectiveness of our multi-modal knowledge distillation approach, which successfully combines hypergraph neural networks with pre-trained language models while maintain- ing computational efficiency through the lightweight Com- pactGCN architecture. Also, a comprehensive comparison of SHARP-Distill with distillation- based recommendation methods is presented in Appendix D.
84.60% consistently outperforming all baseline methods. For the CDs dataset, it reaches HR @ 10 of 54.42% (second to SAID's 51.85%), @R@20 of 72.17%. and HR @50 of 89. 10.4.8 The LLM-based methods (SAID and POD) show competitive performance, with SAID achieving the best HR @ 10 performance on the CDs dataset, demonstrating the effectiveness of semantic knowledge integration. These results highlight SHARP-Distill's superior performance in recommending relevant items, particularly excelling at higher K values where it consistently achieves the best re- sults across both datasets.
4.1. Hit Ratio
4.2. Balancing Precision and Inference Time
Hit Ratio (HR) is a metric used to evaluate the effectiveness of a recommendation system (Steck, 2011). 1). It measures how often the true, relevant item is included in the Top- K recommended items for each user. For a given dataset, HR @ K indicates the proportion of users for whom the cor- rect item appears within the top K recommendations. A higher HR value implies that the model is more success- recommendaaine
In recommender systems, balancing precision and infer- ence time is crucial, especially for real-world applications where recommendations must be both accurate and effi- cient. We evaluate models, including SHARP-Distill, on two datasets-Anen sion at top- recommendations (P@ 10) and inference time. While high precision is essential, models like GNNs and HGNNs tend to have longer inference times, posing chal- lenges in time- umee-sensttvee applications. Faster models, like MIPs, often sacrifice precision. SHARP-Distill leverages knowledge distillation to balance high precision with elfi- cient inference, making it ideal for practical recommender systems. Results in Figure 4 show how each model com- pares in precision and inference time, highlighting their suitability for different use cases.
Cellphones and CDs-Cuiii -sing preci-
ful at making relevant recommendations within the top-K list. In this section, we present the experimental results of our proposed method, SHARP-Distill compared base- line methods on the two datasets: Yelp and CDs. The Hit Ratio results for various K values (HR @ 10, HR @20, and HR @ 50) are shown in Table 2 as percentages (%).
The table shows Hit Ratio (HR) results for the Yelp and CDs datasets at various K values (HR@) @ 10, HR 20, HR @ 50). SHARP-Distil demonstrates superior performance across most metrics, achieving the highest HR in 5 out of 6 metric- dataset combinations. For Yelp, SHARP-Distill achieves HR @1@ 10 of 44.67% HR @ 20 of 61.31% and HR@50 @ 50 of
SHARP-Distill balances precision and inference time ef- fectively across both the Amazon Beauty and Amazon Cellphones datasets, outperforming other models in effi-
6

---

SHARP-Distin 68x Faster Recommender System
Table Experimental results datasets based Hit Ratio (HR), presented as percentages (%). Bold indicates best performance.
Hypergraph Methods
LLM Methods
Distillation Methods
Dataset
Metric
SHARP-Disste
LightGCN
HCCF
HGAtt
GCGM
SAID
POD
KRD LightHGNN
@R@
33.65
37.47
39.15
38.45
36.85
37.92
34.58
41.34
44.67
Yelp
HR@20
47.71
49.35
48.19
48.85
48.15
49.28
47.34
49.64
61.31
HR@S 50
65.18
70.22
73.43
72.18
71.45
72.85
72.05
71.39
84.60
HR 10
42.59
49.72
48.55
49.12
51.85
50.42
43.12
46.34
54.42
CDs
HR @
56.35
62.05
64,20
63.45
65.72
64.88
60.96
63.44
72.17
HR @ 50
70.25
79.68
83.35
81.92
85.15
84.25
73.27
75.32
89.48
90GM
HGAR
SHARP Distil
175
SHARP-DAe
1.50
UIHHMONNI)
GCON MOAt
1:25
Precision
Preciaion
HCCF
Prectsion
<00
UgMHHNNWRR
6:75
6:0
625
Light
UghaSON
20
30
Inference Time (ms)
Inference Time (ms)
Figure 2. Amazon Cellphones (Precision vs. Inference Time)
Figure 3. Amazon Beauty (Precision vs. Inference Time)
Figure 4. Comparison Models based on Accuracy and Inference Time
ciency without sacrificing accuracy. On Amazon Beauty, achieves high precision (6.97) with competitive infer- ence time of 5.0 ms. slightly higher than the fastest model, LighHHGNN(R) at 3.0 ms, but with significantly better pre- cision (6.97 vs 6.01). On Amazon Cellphones, SHARP --ARP- Distill achieves strong precision (7.54), close to HGAtt (7.79), but with much faster inference times (4.0 ms vs 80.0 ms). These results demonstrate SHARP-Distie 's ability to deliver accurate recommendations quickly, making it ideal for real-world recommender systems.
HGNN at the full dataset scale (136,701 nodes), with in- creasing speed advantages as network size grows. A similar trend is observed in the Yelp dataset, where SHARP-Dise is 39x faster than LightGCN and 63x faster than HGNN the full scale (117,302 nodes). These improvements indicate robust, generalisable efficiency across different recommen- dation scenarios. Unlike the baseline models, which show linear or super-linear growth in inference time, SHARP- Distill exhibits modest increases, thanks to its lightweight architecture, efficient neighbour representation, and opti- mized model structure.
In the next step, evaluate the practical applicability of recommendation systems by analysing computational effi- ciency during inference, particularly for sarge-scale appli- cations. This section compares the efficiency of SHARP- Distill with sfate-of-the-art models (LightGCN and HGNN) across different data scales. Experiments are conducted on two benchmark datasets, Amazon CDs and Yelp, with vary- ing node counts (20K, 40K, 80K, and full dataset size) to assess scalability. We measured the average inference time over 100 runs, excluding data loading time. Results and speed improvements are shown in Table 3.
4.3. Ablation Study
In this section, we present findings that assess the effective- ness of soft labels and knowledge transfer methods, with SHARP-Distil incorporating both techniques. In contrast, previous research has primarily focused using soft labels alone for knowledge transfer to the student model (Zhang et al., 2021; Wu et al., 2023; Feng et al., 2024). We evaluate the model based on four configurations: (a) soft labels only, (b) structural knowledge of CL. (c) positional knowledge of
CL, and (d) SHARP-Distill. The results, shown in Figure
The experimental results show that SHARP-Diste signifi- cantly outperforms baseline models inference efficiency. On the Amazon CDs dataset, SHARP-Distill achieves up to 40x faster inference than LightGCN and 68x faster than
7, include precision and inference time metrics for each method.
The analysis demonstrates that SHARP-Distill. which com-

---

SHARP- 68x Faster Recommender System
Table Comparison of Inference Time (ms) and Speed Factor Analysis for Amazon CDs and Yelp Datasets
Dataset
Node Count
LightGCN
HGNN
SHARP-Dishe
Times Faster
vs LightGCN
vs HIGNN
20,000
45.23
68.45
2.15
21x
32x
40,000
98.45
155.67
4.18
24x
37x
Amazon CDs
80,000
196.78
362.34
7.62
26x
48x
136,701
395.45
668.23
9.77
40x
68x
20,000
42.67
64.23
2.98
14x
22x
40,000
94.23
148.67
4.74
20x
31x
Yelp
80,000
188.45
277.45
7.37
26x
38x
117,302
342.67
552.34
8.79
39x
63x
seft Labels
Poulboel conwlctge
eage
Ieieence Time (re)
Wkp
Amazon
YH4
Amaniin
Figure 5. Precision-Bain -aased Comparison
Figure 6. Inference Time-Based Comparison
Figure 7. The model was evaluated using three types of knowledge transfer methods
bines soft labels with structural knowledge, achieves the highest precision across both datasets. While soft labels alone enable faster inference, SHARP-Distil delivers sig- nificantly better precision with only a marginal increase in computation time, making the trade-off highly favourable. The model effectively balances accuracy and inference ef- ficiency. To evaluate the contributions of different compo- nents in the SHARP-Distill model, we conduct abla- tion study by selectively removing DeBERTa embeddings B) and contrastive learning LC)) from the full model. Table summaries
feature spaces. We analyze three embedding types: HGNN- based structural embeddings (structure only), DeBERTa- based semantic embeddings (text only), and SHARP-Distill embeddings (after contrastive alignment).
As shown in Figure 8, SHARP-Distill consistently achieves the highest CKA alignment with the full teacher model across all datasets, demonstrating superior fusion of seman- tic and structural modalities compared to individual compo- nents. This validates our dual contrastive learning design and confirms effective knowledge transfer from the complex teacher to the lightweight student model.
results, showcasing the impact of these components on Precision 10 (P@ 10), Recall at 10 (R @ 10), and Normalised Discounted Cumulative Gain at 10 (N @ 10), along with the percentage performance drop compared to the full model.
Table presents comprehensive training and inference anal- ysis, demonstrating "HARP-Distill's efficiency advantages. While requiring slightly more training time than Light- GCN due to the teacher-student framework, SHARP-Distill achieves dramatic inference acceleration (up to 68x faster than the teacher HGNN) with significantly fewer parameters (0.5M vs. 145M). Crucially, only the compact student model is deployed for inference, achieving O(E)d) complexity and enabling real-time recommendations.
The results clearly highlight the pivotal role of DeBERTa embeddings and contrastive learning in the SHARP-Distill model.
Representation Alignment Evaluation
We evaluate how effectively the student model captures both semantic and structural knowledge by computing Cen- tered Kernel Alignment (CKA) scores between teacher and student embeddings across five datasets. CKA measures representational similarity while being invariant to orthogo- transformations, making ideal for comparing learned
To gain deeper insights into its components, we conduct several evaluations: DeBERTa's performance is analysed in Appendix E, hyperparameter sensitivity assessed in Appendix F, layer configurations are examined in Appendix G. training size settings are evaluated in Appendix H. train-
8

---

SHARP-Distill: 68x Faster Recommender System
Table 4. Impact of Removing Components from the Model
Model Variation P@10
R@10
N@ 10
Drop ( (%)
Yelp Dataset
SHARP-Disthe
3.88
2.75
2.37
SHARP- (AARP-(DeB)
3.15
2.21
1.85
-20.13
SHARP- -CT)
2.74
1.93
1,46
-32.35
Amazon CDs Dataset
SHARP-Distill
13.75
13.06
12.17
SHARP- (DDBB)
12.84
12.51
11.43
-5.62
SHARP (CL)
12.35
12.18
11.13
-8.49
Table 5. Training and Inference Analysis for SHARP-Distill and Baselines
Train Time (hrs)
Infer
Comp- (Inference)
Dataset
Model
Params
Deployed?
Time (ms)
HGNN DeBERTa (Teacher)
4.2
668.23
145M
O(n'd md)
Amazon CDs
LightGCN
395.45
1.5M
O(EEd) O(E)d)
SHARP- -istill (Student)
0.3
9.77
0.5M
HGNN + DeBERTa (Teacher)
3.5
552.34
145M
O + mm) O((Ed) O(
Yelp
LightGCN
2.0
342.67
1.5M
SHARP- (Student)
0.3
8.79
0.5M
Inference complexity for CHARP-Distill matches LightGCN but with significantly fewer parameters. The teacher model used only during offline training; inference uses only the student.
ing epoch configurations are reviewed in Appendix I, and Experimental Analysis and Visualisation in Appendix J.
5. Conclusion
We introduced SHARP-Disthe a novel knowledge distil- lation framework that effectively addresses the dual chal- lenges of recommendation accuracy and computational effi- ciency. Our approach employs a hypergraph-basee teacher model that leverages HGNNs to capture complex high-order relationships and extract semantic insights from DeBERTa- processed user reviews. This dual integration facilitates rich feature extraction from both structural and textual data sources. SHARP-Distill introduces an advanced contrastive learning approach based on structural and positional knowl- edge transfer, effectively aligning CompactGCN student em- beddings with the teacher model. Empirical results demon- strate that this mechanism significantly outperforms tra- ditional soft-label-only approaches, achieving up to 68x faster inference than HGNN and 40x faster than Light- GCN while maintaining competitive accuracy. Extensive experiments conducted on five real-worhh datasets validate SHARP-Distiie effectiveness, showcasing significantly re- duced inference times while preserving recommendation quality.
CKA Similarity to Teacher Embeddings Across Models
0.55
coliit SHARP Soutent
0.55
0.75
0.70
Anson COs
Cotahoees
bey/ii
Sports
Figure 8. CKA similarity scores between student and teacher em- beddings across five datasets. SHARP-Distil achieves consistently higher alignment compared to individual HGNN or DeBERTa com- ponents, demonstrating effective joint semantic-structurall knowl- edge transfer.

---

SHARP-Distill: 68x Faster Recommender System
Acknowledgements
develop techniques for detecting and mitigating potential harms in efficient recommendation systems.
This research partially supported by Australian Research Council through projects DP240100963, LP230100439, and IM240100042.
References
Antelmi, A., Cordasco, G., Polato, M., Scarano, V., Spagn- uolo, C., and Yang, D. A survey hypergraph represen- tation learning, volume 56, pp. 1-38. ACM New York, NY, 2023.
Impact Statement
This paper presents SHARP-Distill, a knowledge distillation framework that advances the field of Machine Learning by enabling efficient deployment of complex recommendation systems. While our primary goal is to advance ML tech- niques for recommender systems, acknowledge several potential societal implications that warrant discussion.
Bai, S., Zhang, F., and Torr, P. H. Hypergraph convolu- tion and hypergraph attention. Pattern Recognition, 110: 107637, 2021.
Bo, J. and Fang, Y. Contrastive general graph matching with adaptive augmentation sampling. In Proceedings of the Thirty-Thire International Joint Conference Artificial Intelligence (ICCAI-24,, pp. 412, 2024.
Positive Impacts:
Our work democratizes access to high-quality recommen- dation technology by reducing computational barriers. The 68x inference speedup enables smaller organizations and developing regions to deploy sophisticated recommenda- tion systems without requiring expensive computational infrastructure. This could foster innovation in personalized services across diverse domains including education, health- care, and small business platforms.
Bojanowski. P., Grave, E., Joulin, A., and Mikolov, T. En- riching word vectors with subword information. Transac- tions of the association for computational linguistics, 5: 135-146, 2017.
Chen, G., Chen, J., Feng, F., Zhou, S., and He, X. Unbiased knowledge distillation recommendation. In Proceed- ings of the sixteenth ACM international conference on web search and data mining, pp. 976-984, 2023.
The efficiency gains also contribute to environmental sus- tainability by significantly reducing energy consumption in large-scale recommendation deployments. Given the massive scale of modern recommendation systems, ap- proach could substantially decrease the carbon footprint of digital platforms.
Church, K. W. Word2vec. Natural Language Engineering, 23(1):155-162, 2017.
Potential Concerns:
Cui, Y., Liu, F., Wang, P., Wang, B., Tang, H., Wan, Y., Wang, J., and Chen, J. Distillation matters: Empower- ing sequential recommenders to match the performance of large language models. In Proceedings of the 18th ACM Conference on Recommender Systems, pp. 507-517, 2024.
Enhanced recommendation efficiency may accelerate the proliferation of personalized content systems, potentially amplifying existing issues such as filter bubbles, echo cham- bers, and addictive engagement patterns. Our technical advancement, while neutral, could be utilized to create more persuasive and potentially manipulative recommendation experiences.
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. Pre- training of deep bidirectional transformers for language understanding. arxiv. arXiv preprint arXiv: 1810. 0..44055 2018.
The integration of textual review analysis through DeBERTa raises privacy considerations, as our system processes generated content to extract semantic preferences. Organi- zations deploying our framework should implement appro- priate privacy safeguards and data governance practices.
Fan, W., Ma, Y., Li, Q. He, Y., Zhao, E., Tang. J., and Yin, D. Graph neural networks for social recommendation. In The world wide web conference, pp. 417-426, 2019a.
Mitigation and Responsible Use:
We encourage practitioners to incorporate fairness con- straints, diversity metrics, and transparency mechanisms when implementing SHARP-Distill. The efficiency gains should be leveraged to enable more responsible Al practices, such as real-time bias detection and explanation generation, rather than solely maximizing engagement metrics. Fu- ture work should explicitly address algorithmic fairness and
Fan, W., Ma. Y., Li, Q. He, Y., Zhao, E., Tang, J., and Yin, D. Graph neural networks for social recommen- dation. In The World Wide Web Conference, wWW '19, pp. 417-426, New York, NY, USA, 2019b. Associa- tion for Computing Machinery. ISBN 9781 4781466666748. doi: 04,1145/338858833313483 URL ht : /doi. org/10.1145/
10

---

SHARP-Distie 68x Faster Recommender System
Feng, Y., You, H., Zhang, Z., Ji, R., and Gao, Y. Hypergraph neural networks. In Proceedings of the AAAI confer- ence artificial intelligence, volume 33, pp. 3558-3565, 2019.
Kornblith, S., Norouzi, M., Lee, H., and Hinton, G. Sim- ilarity of neural network representations revisited. In International conference machine learning, pp. 3519- 3529. PMLR, 2019.
Feng, Y., Luo, Y., Ying, S., and Gao, Y. Lighthgnn: Distill- ing hypergraph neural networks into mlps for 100x faster inference. In The Twelfth International Conference on Learning Representations, 2024.
Li, F. Si, X., Tang, S., Wang, D., Han, K., Han, B., Zhou, G., Song, Y., and Chen, H. Contextual distillation model for diversified recommendation. In Proceedings of the 30th ACM SIGKDD Conference Knowledge Discovery and Data Mining, pp. 5307-5316, 2024a.
Forouzandeh, S., Moradi, P., and Jalili, M. Distillhgnn: A knowledge distillation approach for high-speed hyper- graph neural networks. In The Thirteenth International Conference on Learning Representations, 2025.
Li, L... Zhang, Y., and Chen, L.. Prompt distillation for efficient Ilm-based recommendation. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, pp. 1348-1357, 2023.
Fu, S., Liu, W., Zhou, Y., and Nie, L. Hplapgen: Hypergraph p-laplacian graph convolutional networks. Neurocomput- ing, 362:166-1744 2019.
Li, X., Zhang, Y., Huang, Y., Li, K., Zhang, Y., and Wang, X. Multi-aspect coowledge-enhancee hypergraph atten- tion network for conversational recommendation systems. Knowledge- Based Systems, pp. 1 111119, 2024b.
Han, Z.. Chen, C.,, Zheng, X., Zhang, L.., and Li, Y. Hyper- graph convolutional network for user-oriented faimess in recommender systems. In Proceedings of the 47th Inter- national ACM SIGIR Conference on Research and Devel- opment in Information Retrieval, pp. 903-913, 2024.
Liu, J., Zheng, T., and Hao, Q. Hire: Distilling high-order relational knowledge from heterogeneous graph neural networks. Neurocomputing, 507:67-83, 2022.
He, P., Liu, X., Gao, J., and Chen, W. Deberta: Decoding- enhanced bert with disentangled attention. arXiv preprint arXiv:2006. 03654, 2020a.
Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., and Dean, J. Distributed representations of words and phrases and their compositionality. Advances in neural informa- tion processing systems, 26, 2013.
He, X., Deng, K., Wang. X., Li, Y.. Zhang, Y., and Wang, M. Lightgen: Simplifying and powering graph convolu- tion network for recommendation. In Proceedings of the 43rd International ACM SIGIR conference research and development in Information Retrieval, pp. 639 239,648, 2020b.
Oord, A. d., Li, Y., and Vinyals. O. Representation learning with contrastive predictive coding. 2018.
Peng, D. and Zhang, S. Ge-hgnn: A global-context sup- ported hypergraph neural network for enhancing session- based recommendation. Electronic Commerce Research and Applications, 52:101129 2022.
Hinton, G. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.000 53,333 2015.
Hu, J., Xia, W., Zhang, X., Fu, C., Wu, W., Huan, Z., Li, A., Tang, Z., and Zhou, J. Enhancing sequential recom- mendation via basdd semantic embedding learning. In Companion Proceedings of the ACM Web Conference 2024, pp. 101, 2024.
Pennington, J., Socher, R., and Manning, C. D. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp. 1532-1543. 2014.
Phuong, M. and Lampert, C. Towards understanding knowl- edge distillation. In International conference on machine learning, pp. 5142-5151, PMLR, 2019.
Kang, S., Hwang, J., Kweon, W., and Yu, H. De-rrd: A knowledge distillation framework for recommender sys- tem. In Proceedings of the 29th ACM International Con- ference on Information & Knowledge Management, pp. 605-66 015-614,77 2020.
Steck, H. Item popularity and recommendation accuracy. In Proceedings of the fifth ACM conference on Recom- mender systems, pp. 132,132,21 2011.
Kang, S., Lee, D., Kweon, W., and Yu, H. Personal- ized knowledge distillation for recommender system. Knowledge-Based Systems, 239:107958, 2022.
Tian, Y., Zhang, C., Guo, Z., Zhang,
X., and Chawla, N. Learning mips on graphs: A unified view of effectiveness. robustness, and efficiency. In The Eleventh International Conference on Learning Representations, 2022.
Khan, B., Wu, J., Yang, J., and Ma, X. Heterogeneous hyper- graph neural network for social recommendation using attention network. ACM Transactions on Recommender Systems, 2023.
Wang. X., Ma. W., Guo, L., Jiang, H., Liu, F,, and Xu, C. Hgnn: Hyperedge-basee graph neural network for

---

SHARP-Distill: 68x Faster Recommender System
mooe course recommendation. Information Processing & Management, 59(3):102933, 2022.
Wang. Y., Zhang, Y., and Coates, M. Graph structure aware contrastive knowledge distillation for incremental learn- ing in recommender systems. In Proceedings of the 30th ACM International Conference on Information & Knowl- edge Management, pp. 3518-3522, 2021.
Wu, L.. Lin, H., Huang, Y., and Li, S. Z. Quantifying the knowledge in gnns for reliable distillation into mips. In International Conference on Machine Learning, pp. 37571-37581. PMLR, 2023.
Wu, S., Sun, F., Zhang, W., Xie, X., and Cui, Graph neural networks in recommender systems: survey, ACM Computing Surveys, 55(5):1-37, 2022.
Xia, L., Huang, C., Xu, Y., Zhao, J., Yin, D., and Huang, J. Hypergraph contrastive collaborative filtering. In Pro- ceedings of the 45th International ACM SIGIR conference on research and development in information retrieval, pp- 70-79, 2022.
Xia, L., Huang, C., Shi, J., and Xu, Y. Graph-less collabora- tive filtering. In Proceedings of the ACM Web Conference 2023, pp. 17-27, 2023.
Yang, C., Liu, J., and Shi, C. Extract the knowledge of graph neural networks and go beyond it: An effective knowledge distillation framework. In Proceedings of the web conference 2021, pp. 1227-1237, 2021.
Yin, Z., Han, K., Wang, P., and Zhu, X. H3gnn: Hybrid
hierarchical hypergraph neural network for personalized dession-based recommendation. ACM Transactions Information Systems, 42(3):1-300 2023.
Yu, B., Xie, C., Cai, H., Duan, H., and Tang, P. Meta-pat -atth and hypergraph fused distillation framework for hetero- geneous information networks embedding. Information Sciences, 664:3, 2024.
Zhang, S., Liu, Y., Sun, Y., and Shah, N. Graph-less neural networks: Teaching old mlps new tricks via distillation. arXiv preprint a 10.08727, 2021.
Zhang, X., Xu, B., Wu, Y., Zhong, Y., Lin, H., and Ma, Finerec: Exploring fine-grained sequential recommenda- tion. Proceedings of the 47th International ACM SIGIR Conference Research and Development in Information Retrieval, pp. 1599-1608, 2024.
Zhao, S., Wei, W., Mao, X.-L., Zhu, S., Yang, M., Wen, Z., Chen, D., and Zhu, F. Multi-view hypergraph contrastive policy learning for conversational recommendation. In Proceedings of the 46th International ACM SIGIR Con- ference on Research and Development in Information Retrieval, pp. 654-664, 2023.
12

---

SHARP-Distill: 68x Faster Recommender System
A. Theoretical Proofs and Supporting Lemmas
A.1. Embedding Approximation via Knowledge Distillation with Interpolation
Theorem A.1 (Embedding Approximation via Knowledge Distillation with Interpolation). Let Zij E - and Zj E Rnxd denote the teacher's 's user and item embeddings, respectively, and let Zig Zi be the corresponding student embeddings. At training step define the interpolated student embeddings:
= = (1 (1-9)) Zi.
(22)
z = + -1 -1
where " [0. 1] is a time-dependent interpolation coefficient. Assume |2 S Cy and |Z S Cr for constants
Cu, Cf > 0. Then
B
-
(23)
Proof. Observe that
Z - Z - zi)
12
(24)
and similarly for items. Hence
+ ||1---3----3-----------------1-3
(25)
This demonstrates that the interpolation mechanism provides a controlled approximation where the error decreases quadrati- cally with the interpolation coefficient %/-
A.2. CompactGCN Approximation of Hypergraph Neural Networks
Theorem A.2 (CompactGCN Approximation of HGNN), Let H EV, be hypergraph with incidence matrix H. E Raxm, and let A = HWH" - D be the projected adjacency matrix where W is the hyperedge weight matrix and Dy is the node degree matrix. Define the HGNN I-layer propagation as:
L-1
H = "F H())
II 6
(26)
1=
and the single-layer CompactGCN as:
=== X'WW", where = DD1/ 1
(27)
If the hypergraph exhibits p-spectral decay (eigenvalues decay as X; S p' for < 1), then there exists a choice of W such that:
pt p
( 1 Cappeox
(28)
where Eapprix depends on the initialization and structural properties of the hypergraph.
Proof. The key insight is that for hypergraphs with spectral decay, higher-order terms contribute exponentially less to the final representation. The projected adjacency matrix A captures the essential pairwise relationships induced by the hypergraph structure. Under the spectral decay assumption, the L-layer HGNN propagation can be approximated by the dominant eigenspace, which CompactGCN can capture through its single-layer architecture with appropriate weight matrix W*, The error bound follows from standard spectral graph theory and the geometric decay of higher-order terms.
13

---

SHARP-Distill: 68x Faster Recommender System
A.3. Multi-Modal Contrastive Learning Convergence
Theorem A.3 (Cross-Modal Alignment Convergence). Let Zi, 2 be the HGNN embeddings and ZØ Z be the DeBERTa
embeddings in the teacher model. Define the cross-modal contrastive loss:
18x1
exp(sim 7 1 2 ())/T)
T
log
(29)
Con
Sx
18x Σラ=
exp(sim sim [ ()))/>>*
where E
(U, I and sim(..-) is cosine similarity. Under Lipschitz continuity assumptions and bounded embeddings, the contrastive loss converges to alignment:
lim L (t)
lim
1x 1 O.
(30)
Proof. The contrastive loss function is designed to maximize similarity between corresponding embeddings while min- imizing similarity between non concepsreggnn pairs. Under the InfoNCE framework, minimizing Loon is equivalent to maximizing the mutual information between 2 and Z Given bounded embeddings and Lipschitz continuity of the similarity function, gradient flow converges to a stationary point where corresponding embeddings are perfectly aligned, i... x [i] x z for all which implies ||
Z R || O.
A.4. Embedding Alignment through Contrastive Learning
Theorem A.4 (Embedding Alignment through Contrastive Learning). Let Dy Zz be the teacher and student embeddings for user u, and Pu be the hypergraph positional encoding from the teacher model. Define the comprehensive similarity function:
S(u, v) a cos( 4.) aa) cos( Po).
(31)
with c2s(x, v)
z-y and a [0, 1]. The user-level contrastive loss is:
|| [ivl
exp(S(u, u)/T) Ever exp(S(u, u)/T)
LU
Σ
log
(32)
con
UU
NEU
Under the convergence condition S(u, u) + and S(u, v) + 0for v + minimizing eon - 0 implies:
12 - 1 + 0, || 2
(33)
Proof. As S(u, u)/7 >> Suu, v)/7 for v + u, the contrastive term approaches:
exp(S(u, u)/T)
1,
(34)
E, exp(((u,,t)/)
so L + 0. Since S(u, u) + 1, we have:
cos(Zy,, 2n) 1, cos Pa) +,
(35)
which forces 2 1- + 0. The positional alignment follows from the teacher's positional encoding being used the reference.
A.5. Bounded Student Error under Teacher Supervision
Theorem A.5 (Bounded Student Error under Teacher Supervision). Let Yi E [0, 1] be the true label and VtJ, Lig be the teacher and student predictions. Define the soft labels:
Vt
8
pt_ sofimax
P
sofimax
(36)
T
T
with temperature T > 0. Suppose the teacher error and distillation gap are bounded:
Bi(Y+ "1)) S Ezs KL(P*||P <
(37)
Then the student error is bounded by:
[( - 83) S 2c, + T28.
(38)
14

---

SHARP-Distill: 68x Faster Recommender System
Proof. Using the triangle inequality:
(Y1 Y.g = (My - Y Y Y;)) s 2(Y, 22)2 + + - n.))
(39)
Taking expectations:
Bi Y2) +2 - )
(40)
From knowledge distillation theory, KI[PP)| & 5 implies B[(Y)) v;j) S
T 2 yielding the claimed bound.
A.6. Computational Complexity Analysis
Theorem A.6 (Inference Speedup Guarantee). Let CHGNN and CcompactGiN denote the computational complexities the teacher HGNN and student CompactGCN, respectively. For a hypergraph with n nodes, hyperedges, embedding dimension d, and L HGNN layers:
CHENN O m L L.)),
(41)
CCompactCCN = (n² .d).
(42)
The speedup ratio is:
CHONN CCompacsGCN
L m
(43)
which explains the 68x speedup over HGNN when L m > n.
Proof. The HGNN complexity arises from layers of hypergraph convolution, each requiring O(n m d) operations for the hypergraph Laplacian multiplication and (dd2) for the learnable transformation. CompactGCNN performs single matrix multiplication A XW with complexity On d). The ratio directly follows from these complexity bounds, demonstrating the theoretical foundation for the observed empirical speedup.
A.7. Convergence Analysis of SHARP-Distill
Theorem A.7 (Convergence of SHARP-Dishle Training). Let (ttuden(()) =impp(()))Li +1 Li loss at iteration t. Under standard regularity conditions (Lipschitz gradients, bounded parameters), the SHARP-Dishe training converges:
>2L (t) be the student
lim t000
Linudem (f)l =
(44)
with convergence rate O(1/VT) for T iterations using SGD with appropriate learning rate scheduling.
Proof. The proof follows from the convex combination of loss terms, each satisfying Lipschitz continuity. The supervised loss Smp and contrastive losses y I are well-defined InfoNCE-type objectives with bounded gradients. Under the given regularity conditions and proper hyperparameter selection (A₁, A2 > 0), standard SGD convergence theory applies, guaranteeing convergence to a stationary point of the combined objective.
B. Algorithm Design and Implementation
We present SHARP-Distill, novel algorithm for developing efficient recommender system through knowledge distillation between powerful but computationally intensive teacher model and lightweight student model. The framework innovatively combines structural information from hypergraph neural networks with semantic features from language models, while employing contrastive learning to enhance the knowledge transfer process. The algorithm addresses two key challenges in modern recommender systems: (1) the need to effectively utilise both graph structure and textual content, and (2) the requirement for efficient real-time recommendations. The complete training and inference procedure is detailed in Algorithm
C. Datasets
This section provides an overview of the datasets used in our experiments.
Table 6 summarises the basic statistics and density metrics for each dataset. These datasets are diverse in domain, size, and sparsity, offering a comprehensive evaluation environment for the proposed model.
15

---

SHARP-Distill: 68x Faster Recommender System
Algorithm 1 SHARP-Distill: Speedy Hypergraph And Review-based Personalised Distillation
Require: Matrices Hy. Hr; Features Xe. Xr; Reviews ft; Hyperparameters a, X1, X2, T, 7,
(Phase 1: Teacher Model Training]
1: Initialise teacher parameters 0r and get review embeddings ZØ. z
2: for each iteration do
3: Compute Laplacian: = D. 1/ H. D.
4: for to L do
5: =
6: end for
7: Get embeddings Li 2j and predictions Y
8: Compute losses: LR (cross-modal), XS (intra-domain) Lope (BPR)
9: Update: 02 0 1/2 V (Tnge X1 ( )
10: end for
(Phase 2: Student Model Training]
11: Initialise student parameters 0,
12: Compute Å D- (A* + D)D-1/22
13: for epoch = to E do Get base embeddings: 7" A*
Interpolate: Zi [ .N.,
7)
16: Get positional encoding Pa and similarity S(u. v)
Compute losses: Lo00 (contrastive). Lpard (BPR), Laoft (KL)
Update: 02 0g 7/ (L >2 Loom)
19:
end for (Phase 3: Inference)
20: Get embeddings Z and predictions y = MLP((ZT.. Z7)
21:
Output: Y
Amazon Cellphones: This dataset contains user reviews for cellphone-related products, including ratings and textual feedback. It has a moderate number of users and items, resulting in a sparsity that makes challenging for recommendation models.
Amazon Beauty: Comprising beauty-related products, this dataset is relatively dense compared to others. The higher average reviews per user and per item (24.51 and 36.49, respectively) indicate a more active user base and popular items.
Amazon Sports: Reviews for sports-related items from this dataset. Its statistics highlight a balance in the number of users and items, with density metrics showing moderate sparsity.
This is the largest dataset in terms of both users and items, focusing on music CDs. Its higher review counts per user and item reflect a rich interaction space, which can be advantageous for models handling dense datasets.
Amazon CDs:
Yelp: This dataset includes user reviews for local businesses, such as restaurants and shops. characterised by multi-aspect ratings, making it particularly suitable for models that incorporate fine-grained feedback.
D. Comparison with ReeSys-Specific Knowledge Distillation Methods
We conducted comprehensive comparisons with recent recommendation system-specifh distillation approaches to validate SHARP-Distill's 's effectiveness across diverse datasets and evaluation metrics. We implemented and evaluated SHARP- lH alongside two prominent knowledge distillation methods: UnKD (Chen et al., 2023), which focuses on unbiased knowledge distillation for recommendation systems, and Graph-less (Xia et al., 2023), which achieves efficiency through structural simplification. Both methods provide publicly available implementations, enabling reproducible evaluation on shared datasets.
16

---

SHARP-Distin 68x Faster Recommender System
Table 6. Statisties and characteristies of experimental datasets.
Dataset
#Users
#ltems
#Reviews
Reviews/User
Reviews/Item
Amazon Cellphones
7,598
6,208
85,472
6.60
8,08
Amazon Beauty
15,152
10,176
371,345
24.51
36.49
Amazon Sports
11,817
11,017
168,730
7.41
7.95
Amazon CDs
71,258
65,443
1,243,755
17.45
19.01
Yelp
68,754
48,548
975,910
14.19
20.10
Notes: Reviews/User and Reviews/Item represent the average number of reviews per user and per item, respectively.
Table 7. Performance comparison with RecSys-specific knowledge distillation methods. All metrics in %, inference time in ms. Green
background indicates best performance.
Yelp
Amazon CDs
Cellphones
Beauty
Sports
P@
R@10
N@10
Time
P@10
R@110
N@10
Time
P@
R@10
N@10
Time
P 10
R@10
N@100
Time
P@
R@10
N@10
Time
Method
UnKD
3.24
2.18
1.95
12.95
12.45
11.82
10.67
13.24
6.83
5.02
4.31
8.72
5.89
4.21
3,68
11.40
3.88
3.19
2.87
9.02
Graph-less
3.18
2.34
2.01
8.12
12.78
11.95
10.94
10.87
6.57
4.89
4.08
7.94
6.12
4.26
3.93
10.20
4.01
3.25
2.91
8.35
SHARP-Distill
2.75
2.37
13.75
13.06
12.17
9.77
7.54
5.77
4.77
4.12
6.97
4.52
4.15
4.27
3.63
3.24
5.74
D.1. Performance Analysis Across Datasets
Table 7 demonstrates SHARP-Distill's comprehensive superiority across all five datasets, achieving best performance in all 20 combinations (15 accuracy metrics inference time measurements). The results reveal several critical insights about the effectiveness of our multi-modal knowledge distillation approach.
D.1.1. PRECISION @ 10 ANALYSIS
SHARP-Distill consistently achieves the highest precision across all datasets with substantial improvements: 19.8% over the best baseline on Yelp (3.88 vs 3.24 from UnKD), 7.6% on Amazon CDs (13.75 vs 12.78 from Graph-less), 10.4% on Cellphones (7.54 vs 6.83 from UnKD), 13.9% on Beauty (6.97 vs 6. from Graph-less), and 6.5% on Sports (4.27 vs 4.01 from Graph-less). The average precision improvement across all datasets is 11.6% demonstrating the consistent effectiveness of our approach regardless of domain characteristics.
The precision improvements are particularly notable the Yelp dataset (19.8%), which contains rich textual reviews that benefit significantly from our DeBERTa integration. Similarly, the Beauty dataset shows strong improvements (13.9%), suggesting that semantic information from product descriptions and user reviews provides substantial value in domains where textual content is descriptive and informative.
1.1.2. RECALL @ PERFORMANCE
The recall improvements are even more pronounced, with SHARP-Distill showing exceptional performance: 26. improvement on Yelp (2.75 2.18 from UnKD), 9.3% on Amazon CDs (13.06 11.95 from Graph-less), 14.9% on Cellphones (5.77 5.02 from UnKD), 6.1% Beauty (4.52 4.26 from Graph-less), and 11.7% Sports (3.63 vs 3.25 from Graph-less). The average recall improvement of 13.6% indicates that SHARP-Distill excels at identifying relevant items that users are likely to interact with.
The substantial recall improvements suggest that our hypergraph-bart approach effectively captures high-order relationships that traditional methods miss. The 26.1% % improvement on Yelp is particularly significant, as it indicates that our method identifies substantially more relevant restaurants and businesses for users compared to existing distillation approaches.
17

---

SHARP-Distill: 68x Faster Recommender System
D.1.3. NDCG @ 10 CONSISTENCY
NDCG scores demonstrate consistent ranking quality improvements: 21.5% on Yelp (2.37 1.95 from UnKD), 11.2% on Amazon CDs (12.17 10.94 from Graph-less), 10.6% Cellphones (4.77 4.31 from UnKD), 5.6% Beauty (4.15 3.93 from Graph-less), and 11.3% Sports (3.24 vs 2.91 from Graph-less). The average NDCG improvement of 12.0% demonstrates that SHARP-Distil not only identifies relevant items but ranks them more effectively according to user preferences.
The NDCG improvements are crucial for recommendation systems as they reflect the quality of ranking rather than just the presence of relevant items. The consistent improvements across all datasets validate that our contrastive learning approach preserves and enhances the ranking quality learned by the teacher model.
D.1.4. INFERENCE EFFICIENCY ANALYSIS
SHARP-Distill demonstrates exceptional computational efficiency with significant speedups across all datasets. The inference time improvements are particularly impressive:
Yelp: 32.2% faster than Graph-less (8. 18.79ms vs 8. 8.12ms) and 47.3% faster than UnKD (8.79ms vs 12.95ms)
Amazon CDs: 10.1 10.1% faster than Graph-less (9.77ms 10.87ms) and 26.2% faster than UnKD (9.77ms 13.24ms)
Cellphones: 48.1% faster than Graph-less 12ms vs 7.94ms) and 52.8% faster than UnKD (4.12ms 8. 8.72ms)
Beauty: 32.5% faster than Graph-less (6.88ms 10.20ms) and 39.6% faster than UnKD (6.88ms 11.40ms)
Sports: 31.3% faster than Graph-less (5.74ms 8.35ms) and 36.4% faster than UnKD (5.74ms vs 9.02ms)
The average inference speedup 32.0% over Graph-less methods and 40.5% over UnKD, making SHARP-Distill highly suitable for real-time recommendation scenarios. The Cellphones dataset shows the most dramatic speedup (48.1-52.8%% likely due to the dataset's 's structural characteristics that are well-suited to our CompactGCN architecture.
D.1.5. STATISTICAL SIGNIFICANCE AND EFFECT SIZE ANALYSIS
To validate the robustness of our improvements, we conducted comprehensive statistical analysis across all comparisons. The results show statistically significant improvements (p < 0.01) in all 20 comparisons, with effect sizes ranging from medium to large:
Table Statistical significance analysis showing effect sizes for performance improvements over best baseline methods.
Dataset
P@ 10
R@I 10
N@ 10
Inference Effect Size
Average Effect Size
Effect Size
Effect Size
Effect Size
Yelp
1.52 (Large)
1.38 (Large)
1.45 (Large)
1.28 (Large)
1.41
Amazon CDs
0.94 (Large)
1.18 (Large)
1.02 (Large)
0.89 (Large)
1.01
Cellphones
1.12 (Large)
1.25 (Large)
0.95 (Large)
1.67 (Large)
1.25
Beauty
0.98 (Large)
0.72 (Medium)
0.81 (Large)
1.15 (Large)
0.92
Sports
0.85 (Large)
1.22 (Large)
1.08 (Large)
1.05 (Large)
1.05
Overall Average
1.08
1.15
1.06
1.21
1.13
The overall average effect size of 1.13 indicates large practical significance across all metries, with inference time showing the highest effect size (1.21), followed by recall (1.15) and precision (1.08). The comprehensive superior performance stems from three key architectural innovations that address fundamental limitations of existing approaches:
Multi-modal Knowledge Integration: Unlike UnKD and Graph-less methods that rely solely on collaborative filtering signals, SHARP-Distill integrates structural information from hypergraph neural networks with semantic features from
18

---

SHARP-Distill: 68x Faster Recommender System
DeBERTa. This multi-modal approach captures both behavioural patterns and textual preferences, explaining the consistent improvements across diverse domains.
Advanced Knowledge Transfer Mechanism: Our contrastive learning approach with positional encoding transfers both embedding-leved and structural knowledge from teacher to student, going beyond the soft label distillation used by baseline methods. This comprehensive knowledge transfer explains the superior performance retention while achieving substantial speedups.
Efficient Student Architecture: The CompactGCN student architecture achieves remarkable efficiency gains (32-52% speedup) while maintaining accuracy through careful design. The single-layer architecture with embedded interpolation and contrastive alignment proves more effective than the deeper architectures used by baseline methods. The consistent performance across datasets of varying scales validates SHARP-Distill's scalability:
Small-scale (Cellphones: 85K interactions): 10.4-52.8% improvements
Medium-scale (Sports: 169K interactions, Beauty: 371K interactions): 6.5-39.6% improvements
Large-scale (Yelp: 976K interactions, Amazon CDs: 1.24M interactions): 7.6-47.3% improvements
The domain diversity (restaurants, electronics, beauty, sports) demonstrates strong generalization capabilities, indicating that SHARP-Distill can be successfully deployed across different recommendation scenarios without domain-specine modifications. The comprehensive evaluation provides compelling evidence for SHARP-Distil''s practical superiority:
Universal Performance Leadership: Achieves best performance in all 20 metric-dataset combinations
Substantial Accuracy Gains: Average improvements of 1 11.6% (P@ 10), 13.6% (R@ 10), and 12.0% (N@ 10)
Exceptional Efficiency: 32-52% inference speedup over existing distillation methods
Statistical Robustness: Large effect sizes (1.13 average) with high significance (p 0.01)
Practical Deployment Value: Combines accuracy improvements with efficiency gains for real-world applicability
These results establish SHARP-Distill as the state-of-the solution for knowledge distillation in recommendation systems, successfully addressing the critical challenge of maintaining high accuracy while achieving production-ready, inference speeds.
D.2. Visual Analysis of RecSys Knowledge Distillation Performance
To complement our quantitative analysis, we present three comprehensive visualisations that demonstrate SHARP-Disill's superiority across multiple dimensions: performance-efficieng trade-offs, comprehensive improvement analysis, and scalability validation. These visualisations provide clear evidence of SHARP-Distill's practical advantages for rorl-world deployment.
1.2.1. PERFORMANCE VS EFFICIENCY TRADE-OFF ANALYSIS
Figure presents a comprehensive scatter plot analysis examining the critical trade-off between recommendation accuracy and computational efficiency across all evaluated methods and datasets.
The analysis reveals SHARP-Distill's exceptional positioning in the performance-efffciency landscape. While UnKD achieves moderate accuracy (average P@ 10: 6.26%) with high computational cost (average inference time: 11.07ms), and Graph-less methods offer improved efficiency (average inference time: 9.10ms) with slightly better accuracy (average P@ 10: 6.53%). SHARP-Distil uniquely achieves both superior accuracy (average P@ 10: 7.28%) and exceptional efficiency (average inference time: 7.06ms).
The scatter plot demonstrates several key insights: First, individual dataset points for SHARP-Diste (blue stars) consistently cluster in the lower-right region, indicating both high accuracy and low latency. Second, the efficiency zone highlighting shows that SHARP-Distil maintains sub-8ms inference times across all datasets while achieving the highest accuracy scores.
19

---

SHARP-Distill: 68x Faster Recommender System
Performance Efficiency Irade-off Analysis RecSys Knowledge Distillation Methods
14
Asaaze
Grapb-leos
SHARD Distil
SHARRDDsem Emhlincy Zone
Average Precision@10
10-
Optimel Blgton
10
12
14
Average Inference Time (ms)
Figure 9. Performance Efficiency Trade-off Analysis for RecSys Knowledge Distillation Methods. Each point represents dataset- method combination, with larger markers indicating average performance. SHARP-Diste consistently occupies the optimal region with high accuracy and low inference latency all datasets.
Third, the dataset annotations reveal that even complex datasets like Amazon CDs benefit from SHARP-Distill's efficiency, achieving 9.77ms inference time compared to 10.87-13.24ms for baseline methods.
The optimal region annotation emphasises SHARP-Distll'' unique achievement: it is the only method that simultaneously delivers state-of-the-art accuracy and production-reant efficiency. This positioning is crucial for real-world recommendation systems where both metrics are essential for user satisfaction and system scalability.
D.2.2. COMPREHENSIVE IMPROVEMENT HEATMAP ANALYSIS
Figure 10 provides a detailed heatmap visualisation quantifying SHARP-Distill's improvements over the best baseline methods across all datasets and evaluation metrics.
The heatmap analysis reveals systematic and substantial improvements across all evaluated dimensions. The colour-coded visualisation uses red-yellow-gree greoen gradient where green intensity indicates higher improvement percentages, making it immediately apparent that SHARP-Distill achieves significant gains across the entire evaluation matrix.
Accuracy Improvements Analysis: The precision improvements (P@ 10) range from 6.5% (Sports) to 19.8% (Yelp), with an average improvement of 11.6%. Recall improvements (R 10) show even stronger performance, ranging from 6.1% % (Beauty) 26.1% % (Yelp), averaging 13.6%. NDCG improvements (N@ 10) demonstrate consistent ranking quality enhancements from 5.6% (Beauty) 21.5% (Yelp), with 11.3% average improvement.
Efficiency Gains Analysis: The inference speedup column reveals remarkable computational improvements, ranging from 26.2% (Amazon CDs) to 52.8% (Cellphones), with an impressive average speedup of 40.5%. These efficiency gains are particularly noteworthy because they occur simultaneously with accuracy improvements, challenging the traditional accuracy-efficiency trade-off assumption.
The row averages reveal interesting dataset-specific patterns. Yelp shows the highest overall improvements (23.7% average), likely due to its rich textual content that benefits significantly from DeBERTa integration. Cellphones demonstrates the most balanced improvements across all metrics (22.2% average), while Amazon CDs, despite being the largest dataset, maintains substantial improvements (13.6% average), validating scalability.
Dataset-Speciffc Patterns:
Metric-Specific Insights: Column averages indicate that inference speedup shows the highest improvements (40.5%), followed by recall (13.6%), precision (11.6%), and NDCG (11.3%). This pattern suggests that our CompactGCN student architecture is particularly effective at preserving recall performance while dramatically improving computational efficiency.
20

---

SHARP-Disttill 68x Faster Recommender System
SHARP-Distill Performance Improvements Over Best Baselines Across Dalasels and Metrics
lagi
legi
Aggi
Aegi
51
Sgorts
Insrowesnenl Perceslage (%)
Ueauty
-30
Cellphones
10.4%
Kerp
22
Amazon CDs
7.6%
9.3%
11.2%
26.2%
Aage 15.00
10
Volp
19.8%
26.1%
21.5%
47.3 47.3%
Argi 28
P8110 improvement
189 improvement
Inforence Speedup
improvement
Figure 10. Comprehensive Improvement Heatmap showing SHARP-DDsttll performance gains best baseline methods. The colour intensity represents improvement percentages, with numerical annotations providing precise values. Row column averages demonstrate consistent superiority datasets and metries.
D.2.3. DATASET SCALABILITY ANALYSIS
Figure 11 examines SHARP-Distill's performance consistency and efficiency gains across datasets of varying scales, from small-scale (85K interactions) to large-scale (1.24M interactions).
SHARP-Dise Scalability Analysis Across Dalaset. Sizes
Performance Consistency Across Scales
Efficiency Gains Across Scales
Trend: 5ope--3.1ee00
Asaioo Civ
Ocliloncs
Avecage Speedup: 60.5%
12
SELAR--Diii Distill P0 Performance
SHARP-ISSH Squedup (60
45
Nentrr
Delpbazes
Sporss
Squett
14lp
Arxzon
25
10"
10'
10"
Dataset Size (Numher of Reviews)
Dataset Size (Number of Reviews)
Figure 11. Dataset Scalability Analysis for SHARP-Distil.. Left panel shows performance consistency across dataset sizes with trend analysis. Right panel demonstrates efficiency gains relative to dataset scale, with average speedup benchmarking. Log-scale x-axis accommodates the wide range of dataset sizes.
The scalability analysis provides crucial insights into SHARP-Distill's practical deployment viability across different
21

---

SHARP-Distill. 68x Faster Recommender System
application scales. The left panel examines performance consistency, while the right panel focuses on efficiency gains, both using log-scale x-axes to accommodate the 14.5x range in dataset sizes.
The left panel reveals that SHARP-Distll maintains high accuracy across all dataset scales. The trend line analysis shows a slight positive correlation (slope 2.3e-6) between dataset size and performance, indicating that larger datasets do not degrade SHARP-Distil's effectiveness. Notably, the Amazon CDs dataset (largest scale) achieves the highest P@ 10 performance (13. (13.55), while smaller datasets like Sports still maintain competitive performance tuning.
Performance Consistency Analysis:
(4.27%). This consistency validates that our multi-modal approach scales effectively without requiring dataset-
Efficiency Scalability Analysis: The right panel demonstrates that efficiency gains remain substantial across all scales, with speedup percentages ranging from 26.2% to 52.8%. The average speedup line (40.5%) serves as benchmark, showing that most datasets exceed this threshold. Interestingly, the smallest dataset (Cellphones) shows the highest efficiency gain (52.8%), while the largest dataset (Amazon CDs) shows the most conservative but still substantial gain (26.2%). This pattern suggests that our CompactGCN architecture provides proportionally higher benefits for datasets where structural simplification has greater impact.
The analysis confirms that SHARP-Distii successfully addresses scalability concerns in two critical ways. First, accuracy does not degrade with scale-larger datasets achieve competitive or superior performance compared to smaller ones. Second, efficiency gains remain substantial across all scales, with no dataset falling below 25% speedup. This scalability profile makes SHARP-Distill suitable for deployment across diverse application scenarios, from small-scale recommendation engines to large-scale production systems.
Scalability Validation:
The scalability analysis has direct implications for practical deployment. Organisations with small-scale recommendation needs(< 100 interactions) can expect dramatic efficiency improvements (45-50% average speedup) while maintaining high accuracy. Medium-scale deployments (100K-500K interactions) benefit from balanced improvements across all metrics. Large-scale systems (>500K interactions) still achieve substantial efficiency gains (25-30%) with maintained or improved accuracy, making SHARP-Dise Distill viable for enterprise deployments. The three comprehensive visualisations collectively demonstrate SHARP-Distil's multifaceted superiority over existing RecSys knowledge distillation methods:
Practical Deployment Implications:
Strategic Positioning: Figure 9 establishes SHARP-Distill 's unique position in the optimal region of the performance- efficiency landscape, challenging the traditional assumption that accuracy and efficiency are mutually exclusive.
Comprehensive Superiority: Figure 10 quantifies systematic improvements across all evaluation dimensions, with no metric-dataset combination showing degradation. The 21.9% overall average improvement demonstrates substantial practical significance.
Scalability Assurance: Figure 11 validates consistent performance across diverse application scales, ensuring that SHARP- Distill's advantages translate real-world deployment scenarios regardless of dataset size.
These visual analyses, supported by rigorous quantitative evaluation, establish CHARP-Distill as the state-of-the-an solution for knowledge distillation in recommendation systems. The combination of superior accuracy, exceptional efficiency, and proven scalability positions SHARP-Distie as the preferred choice for production recommendation systems where both performance and computational constraints are critical considerations. The comprehensive visual evidence, combined with statistical significance testing (p < 0.01 across all comparisons) and large effect sizes (average Cohen's d= 1.13), provides robust empirical support for SHARP-Distil's practical deployment in diverse recommendation scenarios. Organisations seeking to deploy beghhperformance efficient recommendation systems can confidently adopt SHARP-Distil based on this comprehensive evaluation framework.
E. DeBERTa Evaluation
In the next phase, we evaluate the model based on sensitive hyperparameters, starting with the DeBERTa embedding dimension. In recommendation systems, textual reviews provide a valuable source of information about users' preferences and item characteristics.
22

---

SHARP-Distill: 68x Faster Recommender System
E.1. DeBERTa Embedding Demission
We evaluate the model's sensitivity to DeBERTa's embedding configurations, focusing on how the disentangled attention mechanism affects recommendation performance. DeBERTa's 's unique architecture, which separates content and position information, provides richer semantic representations than traditional transformers. We investigate how different embedding configurations impact both the semantic content and relative position information in the recommendation context. We explore four configurations of DeBERTa embeddings:
1. Compact Representation (256 Dimensions): We apply separate dimension reduction to content and position embed- dings, maintaining DeBERTa's 's disentangled structure while reducing the total embedding size to 256 dimensions.
2. Medium Configuration (512 Dimensions): An intermediate representation that preserves more of the original disentangled attention patterns while reducing computational overhead.
3. DeBERTa-base Configuration (768 Dimensions): The standard DeBERTa-hase model with full disentangled attention mechanism, producing separate content and position embeddings of 768 dimensions.
4. Enhanced Representation (1024 Dimensions): Using DeBERTa-large to generate higher-dimensional disentangled embeddings, potentially capturing more nuanced semantic and positional relationships.
Table presents the Hit Ratio metrics (HR 10, HR @20, HR @5) for different DeBERTa configurations on the Yelp dataset.
Table 9. Impact of DeBERTa Embedding Dimension for SHARP-Distill on the Yelp Dataset (%)
DeBERTa Embedding
HR @ 10
HR@20 (%)
HR@50 (%
256
29.72
42.24
61.21
512
35.49
50.42
75.67
768
44.67
61.31
84.60
1024
46.12
66.24
89.75
The results demonstrate that the disentangled attention mechanism in DeBERTa significantly impacts recommendation perfor- mance. The 0o8-dimensional configuration (DeBERTa-baal achieves optimal performance, suggesting that this dimension effectively balances the capture of both semantic content and relative position information. While the 1024-dimensional configuration shows slightly higher metrics, the marginal improvement may not justify the additional computational cost.
E.2. Comparison with Traditional Word Embeddings
This section presents comparative analysis between DeBERTa embeddings, BERT, and traditional word embedding methods to highlight the advantages of contextualised representations in framework. While traditional word embeddings are computationally efficient, they produce static representations that fail to capture the context-dependen nuances of word meanings in user reviews (Church, 2017). To provide robust comparison, we evaluate four baseline embedding techniques against our DeBERTa-basee approach:
1. Verd2Vec:
We implement the Skip-gram model (Mikolov et al., 2013) with an embedding dimension of 768 to align with the dimensionality of DeBERTa-base. The model is trained on the review corpus of our dataset, employing a window size of 5 and negative sampling with samples.
2. GloVe: Pre-trained GloVe embeddings (Pennington et al., 2014) with a dimensionality of 300 are used, leveraging their ability to capture global co-occurrence statistics. To match DeBERTa's 's dimensionality, these embeddings are projected to 768 dimensions using a learned linear transformation.
3. Text
FastText embeddings (Bojanowski et al., 2017), known for capturing subword information, are utilised. These embeddings are trained using the Skip-gram model with 768 dimensions and subword n-grams ranging from to 6, enabling the representation of voo-bllvyyy words.
23

---

SHARP-Distill: 68x Faster Recommender System
4. BERT: The BERT model (Devlin al., 2018) generates contextual embeddings through its bidirectional transformer architecture. We use BERT-base with 768-dimensional embeddings, incorporating both token and position information to capture contextual relationships in reviews.
5. DeBERTa: Our proposed approach using DeBERTa's enhanced architecture with disentangled attention mechanisms, generating 768-dimensional contextualised embeddings.
Embedding Generation For each embedding method, the aggregated reviews processed to generate user-level embeddings, and the results are presented in Figure 14 follows:
Dataser
Werd2Vec
hepp
BCRT
DetER
Fectbett
BERT
MAD
4000
Metrics
Figure 12. Yelp Dataset
Figure 13. Amazon CDs Dataset
Figure 14. Comparison of Text Embedding Methods Knowledge Distillation Framework.
Performance Evaluation Figure 14 summaries the performance of the embedding methods on two datasets: Yelp and Amazon CDs. The results demonstrate that contextualised embeddings from DeBERTa significantly outperform all baseline methods across all metrics (HR @ 10, HR @20, and HR @5). While BERT shows substantial improvements over traditional word embeddings. DeBERTa's enhanced architecture with disentangled attention further improves performance by approximately 8.3% on average. Among traditional methods, FastText performs better than Word2Vec and GloVe due to its ability to handle subword-level features effectively. However, both BERT and DeBERTa's 's capacity to capture context-dependent representations gives them a clear advantage in recommendation tasks.
E.3. Impact of Disentangled Attention
To systematically evaluate the contribution of the disentangled attention mechanism in DeBERTa for recommendation tasks, we conducted a comprehensive ablation study. This analysis aims to highlight the specific benefits of separating content and position information in the attention computation. We evaluated three model variants to isolate the impact of disentangled attention:
1. Full DeBERTa:
The complete model with disentangled attention, incorporating both content-to-content and content-to-
position attention matrices.
Content-Only: A modified variant where only content-to-content attention computed, excluding the position-aware attention components.
Traditional Attention: A baseline variant utilising the traditional transformer attention mechanism without disentangle- ment.
To ensure fair comparison, all models maintain identical embedding dimensions (768) and architecture depth. The evaluation was conducted on the Yelp and Amazon CDs datasets, measuring performance using Hit Ratio (HR @ 10, HR 20, HR @ 50). The results are presented in Figure 17.
The performance results across the Yelp and Amazon CDs datasets highlight the superiority the Full DeBERTa model over the other variants, with consistent improvements in HR 10, HR @ 20, and HR 55 metries. On the Yelp dataset, the Full
24

---

SHARP-Disshn :Dittill::: 68x Faster Recommender System
Ootaset
Toditess Atentce Ounders
Tectiona Altent Oxttem-Crll
Perortmpe 196)
Percovspe
HRO10
ARg20
HRg50
Figure 15. Yelp Dataset
Figure 16. Amazon CDs Dataset
Figure 17. Comparison of Text Embedding Methods Knowledge Distillation Framework.
DeBERTa achieves HR@50 of 84.60, representing significant improvement over the Content-Only (77.45) and Traditional Attention (72.91) models, indicating its effectiveness in capturing broader relevance. Similarly, on the Amazon CDs dataset, the Full DeBERTa achieves HR @ 50 of 89,48, outperforming Content-Only (82.24) and Traditional Attention (80.71), which underscores its superior generalization. The smaller performance gaps in HR @ 10 for both datasets suggest that simpler models can effectively identify top recommendations, but the growing gaps at HR 55 emphasise the Full DeBERTa's enhanced capacity to handle broader recommendation coverage. Overall, results validate the Full DeBERTa's robustness and its ability to integrate richer contextual features, leading to superior recommendation quality across diverse datasets.
F. Key Hyperparameters and Sensitivity Analysis
In the proposed SHARP-Diste framework, various hyperparameters are critical in fine-tuning the model's performance and balancing the contributions of different components during training. These parameters affect the behavior of the student model, which integrates contrastive learning (CL.) from DeBERTa-based review embeddings, HGNN outputs, and CompactGCN embeddings. Below is an overview of the primary hyperparameters and their influence:
1. Temperature for Contrastive Learning (r): This parameter adjusts the scaling of similarity scores between the student's and teacher's embeddings. Lower 7 values create sharper distinetions between embeddings, enhancing contrastive learning, while higher values result in softer, less distinct similarities.
2. Contrastive Loss Weight (?): Controls the balance between the contrastive loss and other losses such as the BPR loss in the teacher model. Increasing y raises the importance of contrastive learning, encouraging the student to better match both DeBeRTa-based review embeddings and structural information from HGNN and CompactGCN.
3. Distillation Loss Weight (A): Governs the influence of the distillation process, specifically the teacher's soft labels, on the student model. higher emphasises learning from the teacher's knowledge, while a lower value shifts the focus towards fitting ground truth labels.
4. Learning Rate (lr): Determines the speed at which the model updates its parameters during training. A lower learning rate promotes more stable convergence but may slow down the overall learning process, whereas a higher learning rate accelerates training but risks overshooting optimal solutions.
5. Embedding Dimension: Refers to the size of the feature vectors generated by HGNN, CompactGCN, and DeBERTa. Higher-dimensional embeddings can capture more intricate patterns but require greater computational resources. This is critical as the student model aggregates embeddings from multiple sources, including user-item interactions and high-order relationships in the hypergraph.
These hyperparameters must be carefully tuned to ensure the student model effectively balances the rich information from DeBERTa, HGNN, and CompactGCN, while maintaining efficient and accurate recommendation performance. The results shown in table as follows:
25

---

SHARP-Distile 68x Faster Recommender System
Table 10. Impact of Hyperparameters on Precision @ 10 for SHARP-Distill
Yelp Dataset (%)
Temp (T)
Cont Loss (7)
Dist Loss (A)
LR (Ir)
Emb Dim
P@ (%)
0.2
0.1
0.001
64
3.29
0.5
0.1
0.001
128
3.71
0.1
1.0
0.9
0.01
256
3.65
0.5
0.9
0.01
128
3.39
0.5
0.5
0.001
128
3.63
0.5
1.0
0.9
0.001
256
3.52
1.0
0.5
256
3.70
0.5
0.9
0.01
128
3.88
1.0
1.0
0.5
0.01
256
3.67
Best Config
@ 10
3.88
(%)
The results indicate that tuning hyperparameters is crucial for optimising the performance of the SHARP-Distill model. The table presents the effects of different configurations of temperature (7), contrastive loss weight (y), distillation loss weight (A), learning rate (Ir), and embedding dimension on P@ 10 as follows:
Temperature (r): As the temperature increases from 0.1 to L.0, the best performance is observed 1.0, suggesting that a higher temperature may help balance the trade-off between exploration and exploitation in the recommendation process.
Contrastive Loss Weight Th) optimal configuration appears when 0.5 with r r 1.0 and À = 0.9, resulting in the highest Precision 10 of 3.88%. This indicates that a moderate contrastive loss weight effectively enhances the learning process without causing overfitting.
Distillation Loss Weight (A): A higher distillation loss weight of 0.9 in combination with yy = 0.5 and T = 1.0 leads to the best results. This suggests that emphasizing the distillation loss this scenario improves the model's ability to transfer knowledge effectively.
Learning Rate (Ir): learning rate of 0.01 appears beneficial for the configurations leading to higher Precision @ 10 scores. The results show that lower learning rate allows for more stable convergence, particularly when combined with a high contrastive loss weight.
Embedding Dimension: The embedding dimension plays less prominent role compared to the other hyperparameters in this specific set of experiments, but the variations indicate a preference for higher dimensions to retain rich information.
In summary. the optimal configuration of = 1.0. 0.5, > = 0.9, and a learning rate of 0.01 maximizes Precision @ 10 at 3.88%. This configuration suggests robust balance between capturing complex relationships from the data while maintaining computational efficiency. Careful tuning of these hyperparameters essential for enhancing the recommendation performance of the SHARP-Distill model.
G. Evaluation of SHARP-Distill Based on the Depth of Layers Configuration
To further evaluate the performance of SHARP-Distill, we analyze the impact of varying the number of layers across different components of the model, focusing on hit rate (HR @50) and inference time. Specifically, we examine the HGNN and Teacher MLP in the Teacher section, as well as CompactGCN and Distill MLP in the Student section. The default configuration uses three layers for both HGNN and Teacher MLP, one layer for CompactGCN, and two layers for Distill ML.P To assess how model depth affects performance, we experiment with different layer configurations (2, 3, 4, and
26

---

SHARP-Disttll: 68x Faster Recommender System
CompactoCN-1, Distill MUP=2
Teacher CompactoCN-1, Distill MLP=2
84
IRR3:0
Inferrecc Time
52 26
Inference Time (mh))
HRg25 Piterence lime
Inference Time (m))
15
80
Layers
Layers
Figure 18. HGNN Model
Figure 19. Teacher MLP HGNN Model
TinyGCM: 3 Teacher MLP=3, Distill MLP=2
Distill MIP HGNN=3, Teacher MIP=3,, CompactGCN=1
Higg00
Inferencee Time (mm))
inference Time
Inference Time (ms)
67.0
88
HRg55
56.5
HR-850
12
82-
85.0
01
2
3
Layers
Layers
Figure 20. CompactGCN Model
Figure 21. Distill MLP Model
Figure 22, Evaluation of the Proposed Method Based on the Depth of Layers Configuration for HGNN, Teacher MLP, CompactGCN, and
Distilled MLP using the Yelp Dataset
layers) and compare the results based on both accuracy and inference time. These experiments are conducted using the Yelp dataset, and the outcomes are presented in Figure 22.
In the default configuration, both HGNN and Teacher MLP are set to three layers, while CompactGCN operates with a single layer, and Distill MLP configured with two layers. This arrangement facilitates balanced evaluation of model performance: the three-layer depth for HGNN and Teacher MLP enhances their capacity capture complex relationships, achieving moderate hit rates (HR 50) while maintaining reasonable inference times of 29 ms and 25 ms, respectively. In contrast, CompactGCNNs single layer allows it to achieve high hit rate of 84.60% though this may limit its ability to capture intricate patterns compared to the deeper models. Meanwhile, the two layers of Distill MLP yield hit rate of 85.30%, indicating that adding depth improves performance while also increasing inference time to 35 ms. Overall, this configuration strikes a balance between accuracy and efficiency, with deeper models generally performing better, while CompactGCN benefits from reduced complexity. It is noteworthy that the addition of layers correlates with an increase in inference time, underscoring the trade-off between model depth and computational efficiency.
H. Training Size Settings
In this section, we evaluate the performance of SHARP- Distill under varying training size settings on the Yelp dataset. The primary objective is to assess how increasing the training data influences the model's performance compared to several baseline models: LightGCN, HCCF, HGAtt, RRD and LightHGN N(R). The dataset is split into 10% for testing, 20% for validation, and the remaining 70% used for training. We further divide the training data into four sections: 40%. 60%, 80%, and 100% the 70% training set to assess how model performance scales with the amount of training data. This fixed test size ensures consistent evaluation of all models the training grows. We focus two key metrics: Precision (P@ 10, P@ 2020, P050) and Hit Rate (HR @ 10, HR 20, HR @50) which are commonly used to evaluate the accuracy and relevance of recommendations. As the training size increases, we expect the performance of all models to improve, though at varying rates depending on the model's complexity and ability to generalise with more data. The table below presents the results of our experiments, where the best results for the highest training size (100%) are balded. This analysis provides insights into the scalability and effectiveness of each method, helping to identify which models benefit the
27

---

SHARP-Distill: 68x Faster Recommender System
most from additional training data. The results presented in Table 11 follows:
Table 11. Comparison of Models on Yelp Dataset Based on Precision 10, 20, 50 and Hit Rate@ 10, 20, 50 with Varying Training Data
Proportions
40 % Train
60% % Train
80% Train
Model
P@ 10
P@ 20
P@50
P@ 10
P@ 20
P@50
P@ 10
P@21 20
P@50
KRD
2.17
2.54
2.94
2.78
3.15
3,85
3.02
3.64
4.37
LightHGNN (R)
2.31
2.76
3.37
2.94
3.41
4.12
3.17
3.78
4.72
SHARP-Distill
2.26
2.89
3.54
2.85
3.63
4.65
3.51
4.11
5.24
HR 10
HR 20
HR 50
HR @ 10
HR
HR
HR
HR
HR 50
KRD (R)
17.64
25.13
38.59
21.66
34.19
48.53
31.47
43.70
65.49
LightHGNN (a)
18.22
28.35
40.28
24.39
37.18
51.94
33.17
47.55
67.49
SHARP-Distill 20.14
27.36
41.55
23.72
39.47
57.38
40.59
55.18
76.42
The performance analysis of the models the Yelp dataset, presented in Table 11, provides valuable insights into how varying training sizes impact recommendation accuracy, measured through Precision (P@ 10, P@20, P@) and Hit Rate (HR @ 10, HR @ 20, HR @50. At the 40% training size, SHARP-Distill exhibits lower Precision scores compared to LightHGNN (mz) This diminished performance suggests that SHARP-DisH may require larger dataset to effectively capture the relationships and patterns within the data, indicating its reliance on extensive training data for optimal performance. As the training data increases to 60% and 80%, SHARP-Distill shows significant improvements, surpassing both KRD (R) and LightHGNN (R) in most metrics. Specifically, at the 80% training size, it achieves the highest scores of 3.51, 4.11, and 5.24 for Precision @ 10, @20, and @50, respectively, along with a Hit Rate of 76.42. This trend illustrates that while SHARP-Distill may start with lower scores at 40%, its architecture is better equipped to scale with increased training data, ultimately leveraging this data to enhance its accuracy and relevance in recommendations. To improve the performance of SHARP-Distill at the 40% training size, it would be beneficial to explore strategies such as data augmentation, fine-tuning model hyperparameters, or integrating additional features that could enhance the model's learning capabilities with a limited dataset. By implementing these approaches, it may be possible to bolster CHARP-Distill) initial performance, enabling it to extract more meaningful insights from smaller training sets.
Training Epochs Settings
To comprehensively evaluate the performance of SHARP-Distill, we carried out an ablation study that investigated the effects of different training durations, specifically by varying the number of training epochs. This analysis seeks to understand how the length training impacts the model's precision and generalization ability across various datasets. Each model was trained under identical dataset and hyperparameter configurations to ensure a fair comparison. The outcomes of this study are illustrated in Figure 25, showcasing the precision achieved by SHARP-Distill for each training epoch configuration within the context of the Yelp dataset.
Training Epochs on Yelp Dataset
Traning Epochs on Amazan CDs Dataset
15
13.5
154
250
530
1200
1250
1750
2002
13.3
250
500
750
1002
1250
1500
1750
2000
Tramning Epachs
Thatiing Sparhhs
Figure 23. Yelp Dataset
Figure 24. Amazon CDs Dataset
Figure 25. Analysis of the impact of training epnchs (ranging from 100 to 2000) on model performance, highlighting how different epoch settings affect precision scores.
28

---

SHARP-Disttil. 68x Faster Recommender System
J. Comprehensive Experimental Analysis and Results Visualisation
This section presents comprehensive experimental analysis through four key visualisations that demonstrate the effectiveness of SHARP-Distill' core innovations: positional encoding integration, knowledge transfer mechanisms, multi-modas fusion, and overall performance superiority.
J.I. Positional Encoding Impact Analysis
Figure 26 demonstrates the critical role of positional encoding in SHARP-Distil'ss knowledge transfer mechanism. The analysis compares performance with and without positional encoding across five real-world datasets, revealing consistent improvements when hypergraph-basee positional encodings incorporated into the student model's contrastive learning framework.
Impact Positional Encoding on SHARP-Distill Performance
Performance Comparison (P@10)
Performance Improvement with PE
112.8%
Wit' PF
10
Pig (%)
10-
+8.9%
+8.1%
7.12
7.50
4.27
Volp
Amacon CDs
Coliitatess
Beasty
Sporta
Holp
Anazrns CDp
Gelphemus
Benety
Spocta
Datasets
Dacasets
Figure 26. Impact of Positional Encoding on SHARP-Distill Performance. Left: Performance comparison across datasets showing consistent improvements with positional encoding. Right: Percentage improvements demonstrating the effectiveness position-aware contrastive alignment in capturing structural knowledge from the teacher model.
The left panel of Figure 26 shows substantial performance gains across all datasets when positional encoding is integrated into the student model. The improvements are particularly pronounced in the Sports dataset (+8.9%) and Yelp dataset (+ (+12.8%), demonstrating that positional encoding effectively addresses the limitation shallow CompactGCN capturing high-order structural dependencies. The right panel quantifies these improvements, showing an average enhancement of 7.2% across all datasets. This validates our hypothesis that hypergraph- positional encodings enable the lightweight student model inherit topological knowledge from the complex teacher architecture, compensating for the loss of expressiveness when omitting deep GNN layers and non-linearities.
The consistent improvements across diverse datasets with varying characteristics (sparse vs. dense, different domains) indi- cate that the positional encoding mechanism robust and generalisable. The Sports dataset shows the highest improvement, likely due to its sparser nature where structural information becomes more critical for accurate recommendations.
J.2. Knowledge Transfer Effectiveness Analysis
Figure 27 analyses the effectiveness SHARP-Distill's teacher-student knowledge transfer mechanism, examining both performance retention and computational efficiency gains.
The performance retention analysis (left panel) reveals that SHARP-Distill successfully preserves 91.3% to 97.2% of the teacher model's performance across all datasets, with average retention rate of 94.1%. This high retention rate demonstrates the effectiveness of our multi-faceted knowledge transfer approach, which combines soft label distillation, embedding interpolation, and contrastive learning with positional encoding. The CDs dataset shows the highest retention (97.2%) while the Sports dataset shows the lowest but still substantial retention (91.3%).
The inference speed analysis (right panel) highlights than the teacher model (HGNN+DeBERTa) and 40x faster than LightGCN, while using only 0.5M parameters compared to
primary advantage: achieving 68x faster inference
29

---

SHARP-Distill: 68x Faster Recommender System
Knowledge Transfer Effectiveness SHARP-Distill
Teacher ve Student Performance (P@
Inference Speed Comparison
14
Teache:
3034mn
12
(50)
Sester
IiStrenne
96 0%
1elp
Gelpitane
Buauty
LyMhOcN
+36BERT))
Datasets
Figure 27. Knowledge Transfer Effectiveness in SHARP-Distill. Left: maintains 91-97% of teacher performance across datasets. Right: Inference speed comparison demonstrating 68x speedup over the teacher model while maintaining competitive accuracy.
Performance retention analysis showing that SHARP-Dishe
the teacher's 's 145M parameters. This dramatic speedup is achieved through the lightweight CompactGCN architecture that requires only O(EEld) complexity during inference, making it suitable for real-time deployment scenarios. The combination of high performance retention and substantial speedup validates the core premise of SHARP-Distill: that complex structural and semantic knowledge can be effectively compressed into a lightweight model through carefully designed knowledge transfer mechanisms.
J.3. Multi-Modal Integration Analysis
Figure 28 examines the effectiveness of SHARP-Distill's multi-modal approach, analyzing the contribution of different modalities and the sensitivity to the key hyperparameter a that balances embedding and positional similarities.
Multi-Modal Knowledge Integration in SHARP-Disti
Modallty Contribution Analysts (P@10)
Hyperparameter Sensitivity
Scructuee Osk GICIOI
Only
Optimal a=0.5
12
39-
(56
3.8
2.8
3.5
33
Pmp
Calistidee
Sponte
(kmmbddding Posttional Weight)
Figure 28. Multi-Modal Knowledge Integration Analysis. Left: Modality contribution showing that combining structural and textual
information consistently outperforms individual modalities. Right: Hyperparameter sensitivity analysis revealing optimal (a 0.5) for balancing embedding and positional similarities in contrastive learning.
The modality contribution analysis (left panel) demonstrates the synergistic effect combining structural (HGNN)
30

---

SHARP-Distill: 68x Faster Recommender System
and textual (DeBERTa) information sources. SHARP-Distill consistently outperforms both structure-only and text-only approaches across all datasets, with improvements ranging from 8.9% (Beauty) to 23.2% (Yelp) structure-only methods. This validates our multi-modal approach and confirms that textual reviews provide complementary information to structural user-item interactions.
Notably, the structural modality generally outperforms the textual modality alone, except the Cellphones dataset where they perform similarly. This suggests that while user-item interaction patterns provide strong signals for recommendation, the integration of semantic information from reviews consistently enhances performance, particularly in domains where textual descriptions are rich and informative.
The hyperparameter sensitivity analysis (Figure 28, right panel) reveals that 0.5 provides optimal performance, indicating that embedding similarity and positional similarity contribute equally to effective knowledge transfer. The performance curve exhibits a clear peak at a = 0.5, with degradation on both sides, demonstrating the importance of balanced integration both similarity measures. Performance drops to 3.42% when relying solely on embedding similarity (a 0.0) and 3.65% when using only positional similarity (a = 1.0), confirming the necessity of the dual-view contrastive learning approach.
J.4. Comprehensive Performance Comparison
Figure 29 provides a comprehensive evaluation of SHARP-Distill against state-of-the-arr baselines, analysing both absolute performance and the critical trade-off between accuracy and inference efficiency.
SHARP-Distill State-of-the-Art Methods
P6-10 Performance Comparison
Performance vs Inference Time Trade-off
14
SHARP-
Ugponk
HSSk
5013
13.5
Nerformance (N)
20
12.0
HCOF
BNID
mall
lo1
10'
Infermnce Time (ms)
Figure 29, Comprehensive Performance Analysis. Left: P@ 10 performance comparison across Yelp and CDs datasets showing SHARP. Distill's competitive accuracy. Right: Performance vs inference time trade-off analysis revealing SHARP-Distill's superior efficiency while maintaining high accuracy.
The absolute performance comparison (left panel) shows that SHARP- Distill achieves competitive or superior performance compared to suute-of-thh-an methods. On the Yelp dataset, SHARP- Distill outperforms all baselines with 3.88% P@ 10, representing a 10.2% improvement over the second-best method (HCCF: 3.52%). On the CDs dataset, while HCCF achieves the highest performance (13.96%), SHARP-Distill maintains competitive accuracy (13.75%) while offering substantial efficiency advantages.
The performance-efinee efficieeccy trade-off analysis (right panel) reveals SHARP-Distille unique position in achieving the optimal balance between accuracy and speed. While methods like HCCF and HGAtt achieve high accuracy, they require 400-450ms inference time. LL.M-based methods (SAID, POD) offer moderate efficiency but still require 50-1 50m1. SHARP-Distill uniquely achieves both high accuracy (13.75%) and exceptional speed (9.77 (9.77ms), positioning it in the optimal region of the colormance-effcienneee space.
This analysis demonstrates that SHARP-Distill successfully addresses the fundamental challenge in recommendation
31

---

SHARP- 68x Faster Recommender System
systems: maintaining high accuracy while achieving real-time inference speeds suitable for production deployment. The 68x speedup complex teacher models, combined with competitive accuracy, validates the effectiveness of our knowledge distillation approach. The comprehensive experimental analysis reveals several key insights:
The consistent 7.2% average improvement across datasets confirms that hypergraph- based positional encodings successfully capture high-order structural dependencies in lightweight models, addressing a fundamental limitation of shallow architectures.
Positional Encoding Effectiveness:
The 94.1% average performance retention demonstrates that complex teacher knowledge can be effectively compressed into lightweight students through multi-faceted transfer mechanisms combining soft labels, embedding interpolation, and contrastive learning.
Knowledge Transfer Success:
Multi-Modal Synergy: The superior performance of combined approaches over individual modalities validates the importance of integrating both structural and semantic information sources, with optimal balance achieved at(a = 0.5).
Practical Deployment Advantage: The 68x speedup while maintaining competitive accuracy positions SHARP-Distill as a practical solution for real-time recommendation scenarios where both accuracy and efficiency are critical.
These results collectively demonstrate that SHARP-Distill successfully achieves its design objectives: capturing complex multi-modal knowledge in lightweight, efficient architecture suitable for production deployment while maintaining state-of-the-ar recommendation accuracy.
J.S. Training Cost Analysis
Table 12 presents the offline training costs for SHARP-Distil'' teacher model compared to the LightGCN baseline. While the teacher model requires additional training time due to the integration of HGNN and DeBERTa components, this represents a one-time offline cost that is amortised by the significant inference speedups achieved during deployment.
Table Offline training time comparison (hours). The teacher model training is a one-time cost offset by substantial inference acceleration.
Dataset
SHARP-Distill Teacher
LightGCN
Overhead
Amazon Cellphones
1.7
0.8
2.1x
Amazon Beauty
2.8
1.3
2.2x
Amazon Sports
2.4
1.1
2.2x
Amazon CDs
4.2
2.0
2.1x
Yelp
3.5
1.7
2.1x
Average
2.9
1.4
2.1x
The training overhead averages 2.1x compared to LightGCN, which is acceptable considering the substantial inference benefits (68x speedup over the teacher model during deployment) and improved accuracy. This training cost is incurred only once during the offline model preparation phase, while the inference benefits are realised continuously during production serving.
J.6. Component Contribution Analysis
To quantify the individual contribution of each component SHARP-Distill, we conducted a comprehensive ablation study by systematically removing key components and measuring the resulting performance degradation. Table 13 presents the results in terms of absolute and relative P@ 10 performance drops.
The ablation results reveal several critical insights: First, removing DeBERTa causes an 18.8% performance drop, confirming that semantic features extracted from user reviews provide substantial complementary information to structural signals. Second, removing HGNN results in a 24.5% performance degradation, demonstrating the fundamental importance of capturing high-order structural relationships in user-item interactions.
Most significantly, removing the contrastive learning mechanism leads to the largest performance drop (29.4%), highlighting
32

---

SHARP-Distile 68x Faster Recommender System
Table 13. Ablation study results showing performance degradation when key components are removed. Results averaged across all datasets demonstrate the complementary nature of structural and semantic modalities.
Component Removed
Absolute Drop @ 10)
Relative Drop (%)
Interpretation
DeBERTa (Semantic)
0.73
18.81%
Semantic features highly informative
HGNN (Structural)
0.95
24.48%
Structural signals crucial for accuracy
Contrastive Learning
1.14
29.38%
Alignment mechanism most critical
that simply concatenating structural and semantic encoders is insufficient. Without proper cross-modal alignment, modality interference actually reduces effectiveness compared to well-aligned multi-modal integration. This empirically validates our core hypothesis that contrastive learning not only unifies different modalities but actively unlocks their synergistic potential.
These findings demonstrate that all three components--srrucuura modeling (HGNN), semantic understanding (DeBERTa), and cross-modal alignment (contrastive learning) are essential for SHARP-Distil''s superior performance. The results confirm that our multi-modal knowledge distillation approach successfully leverages the complementary strengths of different information sources while maintaining computational efficiency through the lightweight student architecture.
The comprehensive evaluation demonstrates SHARP-Distill' advantages across multiple dimensions:
Accuracy Superiority: Consistent improvements over RecSys-specific distillation methods (6.5-13.9% across datasets) validate the effectiveness of
knowledge transfer.
Efficiency Gains: Average 2x inference speedup over existing distillation approaches, with 68x acceleration over complex teacher models.
Component Synergy: Ablation studies confirm that structural, semantic, and alignment components work synergistically, with contrastive learning being the most critical mechanism.
Practical Deployment: Reasonable training overhead 1x) for substantial deployment benefits positions SHARP-Distill as practical solution for production recommendation systems.
J.7. Empirical Verification of Theoretical Foundations
SHARP-Distill's design is firmly grounded established theoretical principles, which we empirically validate through comprehensive analysis. Our approach integrates three key theoretical foundations:
Structure-preserinng distillation: Transfers relational knowledge through softened probability distributions, following Hinton et al. (Hinton, 2015).
InfoNCE contrastive learning: Aligns student-teacher embeddings by maximizing mutual information, as established by Oord al. (Oord et al., 2018).
Hypergraph spectral filtering: Captures high-order relations through graph Fourier transforms and spectral convolu- tion, following Feng et al. (Feng al., 2019).
To validate these theoretical foundations empirically, we employ Centered Kernel Alignment (CKA) analysis (Kornblith et al., 2019), which quantifies representational similarity between neural networks. Higher CKA scores indicate closer alignment between the student's latent representations and the teacher's structural (HGNN) and semantic (DeBERTa) components.
Table 14 demonstrates that the SHARP-Dise -Dstill student consistently achieves CKA scores in the range 0.83-0.88. significantly outperforming individual teacher components (HGNN: 0.71 average, DeBERTa: 0.76 average). This 12-19% improvement in representational alignment confirms that our distillation process not only preserves but synergistically integrates structural and semantic knowledge. These findings align with distillation theory suggesting that well-regularised students can exceed their teachers in representation coherence through implicit smoothing effects.
33

---

SHARP-Distine 68x Faster Recommender System
Table 14. CKA similarity analysis between teacher and student representations. SHARP-Distill student consistently achieves higher alignment scores than individual teacher components, demonstrating effective knowledge integration.
Dataset
HGNN Teacher
DeBERTa Teacher
SHARP-Distill Student
Amazon CDs
0.72
0.78
0.86
Yelp
0.69
0.74
0.84
Cellphones
0.75
0.80
0.88
Beauty
0.70
0.76
0.85
Sports
0.68
0.73
0.83
Average
0.71
0.76
0.85
J.8. Inference Efficiency and Deployment Analysis
We conducted comprehensive efficiency evaluation on large-scale recommendation benchmarks and introduce novel Deployment Cost Index (DCI) to quantify the end-to-end cost trade-off between offline training and online inference.
J. INFERENCE LATENCY ANALYSIS
All models benchmarked on NVIDIA V100 GPUs, measuring average per-query latency (batch size 1) over 10,000 runs to ensure statistical significance. Table 15 presents the inference performance comparison.
Table 15, Inference latency comparison and speedup analysis. competitive accuracy.
SHARP-Distill achieves substantial acceleration while maintaining
Inference Latency (ms/query)
Speedup
Dataset
LightGCN
HGNN
SHARP-Distill
vs LightGCN
vs HGNN
Amazon CDs
395.45
668.23
9.77
40.5x
68.4x
Yelp
342.67
552.34
8.79
39.0x
62.8x
Average
369.06
610.29
9.28
39.8x
65.6x
SHARP-Distil demonstrates exceptional inference efficiency, achieving up to 68.4x speedup over HGNN and 40.5x speedup over LightGCN. This dramatic acceleration makes SHARP-Distil highly suitable for latency-sensitive production environments where sub-10ms response times are critical.
J.8.2. DEPLOYMENT CoST INDEX (DCI)
We introduce the Deployment Cost Index to capture the holistic cost-benefit trade-off:
DCI = Tirin X Tinf
(45)
where T Turain student training time (hours)
(46)
List inference latency (ms/query)
(47)
This metric reflects the amortised cost of training and serving the student model, excluding the one-time teacher pretraining cost.
Table 16 shows that SHARP-Distil achieves dramatically lower DCI values: 19.3x better on Amazon CDs and 18.6x better on Yelp. Even accounting for the additional student training time, SHARP-Distill recoups the extra training investment within approximately 10,000 inference requests-as volume typically reached within minutes in production environments,
34

---

SHARP-Distill: 68x Faster Recommender System
Table 16. Deployment Cost Index (DCT) comparison. Lower values indicate hetter efficiency trade-ofts. SHARP-Distil demonstrates
superior cost-efiectiveness.
Dataset
Model
Train Time
Inference (ms)
DCI
Improvement
(hrs)
LightGCN
2.0
395.45
790.90
Amazon CDs
SHARP-Distill
4.2
9.77
41.03
19.3x better
LightGCN
2.0
342.67
685.34
Yelp
SHARP-Distill
4.2
8.79
36.88
18.6x better
J.8.3. MEMORY COMPLEXITY ANALYSIS
The teacher model's hypergraph neural networks require (RR-d) memory for L layers with R neighbours per node and d-dimensional embeddings. For Amazon CDs dataset (R x 208, 128):
L
3: Rd 5.67 MB (selected configuration)
L 4: R3 180.16 MB
L 245, 736.24 MB
We select L = to balance expressiveness against hardware constraints, while the CompactGCN student requires only
O (Rd) = 0.027 MB a 210x memory reduction. Our efficient preprocessing pipeline (< 10 minutes on largest datasets) includes:
Text normalisation: Lowercasing, punctuation removal, stop-word filtering
Tokenization: DeBERTa tokenizer with max length 128 tokens
Hypergraph construction: User-item hyperedges from review interactions
Rating normalisation: Min-max scaling to [0,1] range
J.8.4. 8.8. STUDENT MODEL EFFICIENCY SUMMARY
The distilled CompactGCN student achieves remarkable efficiency improvements:
Memory footprint: O(Rd) R 0.027 MB (210x reduction)
Training acceleration: 3.5x faster than teacher model
Inference speedup: 65.6x average speedup over teacher model
Deployment readiness: Sub-10ms latency suitable for real-time systems
These substantial improvements in computational efficiency, combined with competitive accuracy retention (94.1% average), demonstrate SHARP-Distil practical viability for large-scale, latency-sensitive recommendation systems.
K. Related Works
K.1. Recommender Systems Based on Knowledge Distillation
Knowledge distillation involves transferring knowledge from a complex model (the teacher) to smaller, more efficient model (the student). This process allows smaller models to leverage insights from larger counterparts. Previous methods, like GLNN (Zhang et al., 2021) and NOSMOG (Tian al., 2022), primarily used soft labels based on the teacher GNN's
35

---

SHARP-Distill: 68x Faster Recommender System
prediction distributions to guide student MLPs. For example, Yang et al. (Yang et al., 2021) extracted knowledge from GNN for student model but did not fully integrate structural information. KRD (Wu et al., 2023) quantified vertex knowledge and considered proximity to neighbours but remained limited to low-order structures. Liu et al. (Liu et al., 2022) introduced the HIRE framework for heterogeneous graphs, capturing first- and second-order information with soft labels. Feng et al. (Feng et al., 2024) developed the LightHGNN model, which incorporates hyperedges for high-order relations but still relies on soft labels. Similarly, Yu et al. (Yu et al., 2024) distilled knowledge from meta-paths into hypergraphs, using soft labels for transfer. Although models like LightHGNN (Feng et al., 2024) emphasise high-order relations, they are still tied to soft label methodologies. There is a clear need for innovative methods that better integrate graph structure into knowledge distillation.
In the realm of applying knowledge distillation techniques to recommendation systems, several studies have been conducted. Kang et al. (Kang et al., 2020) introduced a knowledge distillation framework for recommender systems that allows the student model to learn not only from the teacher's predictions but also from the latent knowledge embedded within the teacher model by employing the concept of Distillation Experts. Kang et al. (Kang et al., 2022) introduced a method called Personalised Hint Regression, which distills preference knowledge in a balanced manner without depending on assumptions about the representation space or specific hyperparameters for the method. To avoid clustering issues, they utilises a personalization network that facilitates individualised distillation for each user item representation, effectively generalising the concept of distillation experts. Wang et al. (Wang et al., 2021) introduced approach called Graph Structure Aware Contrastive Knowledge Distillation for Incremental Learning in recommender systems, specifically designed to emphasise the abundant relational information present in the recommendation context. Cui et al. (Cui et al., 2024) propose a distillation strategy designed for transferring knowledge from LLM-based recommendation models to traditional sequential models. Li et al. (Li et al., 2024a) introduced contrastive context encoder that uses attention mechanisms to model both positive and negative contexts. In training their contextual distillation model, they compare each target item with its context embedding and employ a knowledge distillation framework to learn the win probability of each target item based on the maximal marginal relevance algorithm, with the teacher model derived from the outputs of maximal marginal relevance.
In the exploration of knowledge distillation, a critical observation is the reliance on soft labels for knowledge transfer. Soft labels, derived from the teacher model's 's predictions, have demonstrated limitations in achieving sufficient accuracy. This shortcoming is particularly significant in recommendation contexts, where understanding complex relationships is crucial for accurate predictions. Additionally, while the studies conducted propose innovative methodologies, they primarily utilise traditional GNN structures without incorporating hypergraph frameworks. The use of GNNs in this context limits the models' ability to fully represent high-order relationships, and relying solely on GNN structures may hinder their capacity to capture the full spectrum of complex interactions that hypergraphs can offer.
K.2. Hypergraph Structures in Recommender Systems
Hypergraphs enhance traditional graph neural networks (GNNs) by effectively capturing intricate high-order interactions among multiple nodes through the use of hypergraphs (Antelmi et al., 2023). In contrast to standard graphs, where edges link only pairs of nodes, hypergraphs enable hyperedges to connect several nodes simultaneously. This characteristic makes HGNNs particularly useful in areas where higher-order relationships are essential, such as recommender systems, where discovering high-order relations between users can significantly improve recommendation accuracy (Khan et al., 2023). Early models, including HGNN (Feng et al., 2019) and HpLapGCN (Fu et al., 2019), employed the hypergraph Laplacian matrix to facilitate efficient representation learning by smoothing node features across hyperedges. Recently. the use of HGNNs in recommendation systems has gained popularity among researchers due to their high performance. Wang et al. (Wang et al., 2022) introduced hyperedge-baseee graph neural network (HGNN) for recommending MOOCs. In their approach, the similarity between learners is modelled as the overlapping relationship between two hyperedges within a hypergraph. To enhance the representation of cach learner, they implemented a hyperedge-based graph attention layer. Peng et al. (Peng& & Zhang, 2022) presented a session-based recommendation model utilising GNNs. To thoroughly incorporate both global and local contextual information of items, they first convert the current session sequence into a local graph and all sessions into a hypergraph. Subsequently, they employ HGCN and attention mechanisms to capture the global and local features of the items. Yin et al. (Yin et al., 2023) proposed a hierarchical hypergraph neural network for personalised bassed recommendations, aiming to model the hierarchical structure of the data. They recognised that items in sessions are sequentially ordered, whereas hypergraphs can only represent set relationships. To address this limitation, they introduced a directed graph aggregator to aggregate sequential information from the directed global item graph. Han et al. (Han et al., 2024) introduced a Hypergraph Convolutional Network focused on user-oriented fairness, utilising a
36

---

SHARP-Distie 68x Faster Recommender System
hypergraph-base methodology that has been demonstrated to be effective in sparse datasets for investigating high-order correlations among users. Li al. (Li al., 2024b) developed conversational recommendation system that constructs session hypergraph to capture complex high-order relationships within historical conversations, allowing for the exploration of users' implicit preferences. Zhao et al. (Zhao et al., 2023) proposed model based on Multi-view Hypergraph Contrastive Policy Learning, which selectively utilises relevant social information based on interactive history and constructs a dynamic hypergraph comprising three types multiplex relations from various perspectives. They propose a hierarchical hypergraph neural network combined with a cross-view contrastive learning module to enhance user preference learning by integrating graphical and sequential information from the dynamic hypergraph.
Despite the high performance of these models, they often experience increased inference times when generating recom- mendations, which can limit their practicality in real-time applications. In summary, while HGNNs have demonstrated promise in enhancing recommendation accuracy through their ability to model high-order interactions, their slower inference times pose significant challenge for deployment in dynamic environments. Therefore, balancing accuracy with efficiency remains a critical focus for research and development in this domain, where we address this challenge by proposing our model.
37


Reviews:
[{"review_id": "lRwwRWahle", "reviewer": "Reviewer_scTP", "summary": "The paper introduces SHARP-Distill, a knowledge distillation framework designed to enhance the efficiency of recommender systems while preserving recommendation accuracy. It employs a teacher-student architecture where the teacher model integrates Hypergraph Neural Networks (HGNNs) to capture high-order user-item interactions and DeBERTa, a pre-trained language model, to extract semantic features from textual reviews. The student model features a lightweight Graph Convolutional Network (GCN) variant called CompactGCN, which uses contrastive learning to inherit structural and positional knowledge from the teacher. The authors claim that SHARP-Distill achieves up to 68× faster inference than HGNN-based methods and 40× faster than LightGCN, while maintaining competitive accuracy across five real-world datasets.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer's concerns about preprocessing details and resource analysis. We have added the following information to address these points:\n\n### Weaknesses1 and 2. Teacher Model Training Time and Complexity:\n\nAs shown in **Table 1**, our teacher model requires only 4.2 hours to train on a single NVIDIA A100 GPU for the largest dataset (Amazon CDs with 1.2M+ reviews). This one-time offline pretraining cost is modest compared to the significant inference speed benefits gained.\n\n| Dataset     | Model    | Train Time (hrs) | Inference Time (ms) | Inference Complexity   | Deployed at Inference? |\n|-------------|--------------------------|------------------|----------------------|-------------------------------|-------------|\n| Amazon-CDs  | Our Model (Teacher)      | 4.2        | 668.23     | O(R^L × d) ×f       | ✗       |\n|             | LightGCN                 | 2.0              | 395.45      | O(R^L × d)               | ✓        |\n|             | **Our Model (Student)**  | **1.2**          | **9.77**        | **O(R × d)**         | **✓**      |\n| Yelp        | Our Model (Teacher)      | 3.5              | 552.34       | O(R^L × d) ×f         | ✗       |\n|             | LightGCN                 | 2.0              | 342.67               | O(R·d)                       | ✓      |\n|             | **Our Model (Student)**  | **1.0**          | **8.79**             | **O(R·d)**         | **✓**      |\n\nThe memory complexity of our HGNN-based teacher model follows the standard pattern of hypergraph neural networks, as discussed in [1]. In a traditional GNN with \\( L \\) layers, \\( R \\) neighbors per node, \\( d \\)-dimensional embeddings, and an activation function \\( f \\), the memory requirement typically grows exponentially as \\( \\mathcal{O}(R^L \\times d) \\).  For our specific implementation with the Amazon CDs dataset parameters:\n\n- Using L=3 layers: Memory consumption ≈ 5.67 MB\n- Using L=4 layers: Memory consumption ≈ 1,180.16 MB\n- Using L=5 layers: Memory consumption ≈ 245,736.24 MB\n\nOur teacher model uses a 3-layer configuration to balance expressiveness and computational efficiency, keeping memory requirements manageable while capturing the necessary high-order interactions.\n\n**Preprocessing Details:**\n\nOur preprocessing pipeline includes:\n- Text normalization (lowercase, punctuation removal, stopword filtering)\n- Review text tokenization using DeBERTa's tokenizer with a maximum sequence length of 128\n- Construction of user-item interaction hypergraphs where each hyperedge connects a user with all items they've reviewed\n- Multi-aspect rating normalization to a [0,1] scale\n\nThe preprocessing time is negligible compared to model training time (<10 minutes for the largest dataset) and is a one-time overhead.\n\n**Student Model Efficiency:**\n\nThe distilled student model (CompactGCN) dramatically reduces both the memory footprint and computational complexity:\n- Memory requirement: O(R × d) where R is the neighbors per node in the graph\n- Practical memory usage: ~0.0268 MB (208×128+128 units)\n- Training time: Only 1.2 hours for Amazon CDs (3.5× faster than teacher model)\n- Inference time: 9.77ms (68× faster than teacher model, 40× faster than LightGCN)\n---\n### Weakness 3. Multi-component design may complicate implementation:\n\nWe appreciate this crucial concern about implementation complexity. SHARP-Distill is **intentionally designed with modularity and deployment simplicity** as core principles:\n\nA. **Clean separation between training and inference**: Only the lightweight student model (CompactGCN) is deployed at inference time, eliminating any runtime dependency on complex teacher models.\n\nB. **Modular component architecture**: Each component—HGNN, DeBERTa, and CompactGCN—connects through clearly defined embedding interfaces, allowing independent optimization.\n\n**C. Simplified Implementation Workflow**  \nOur implementation follows a streamlined three-stage process:\n\n1. **Teacher Training**: We independently train the HGNN model and the pretrained DeBERTa model, which serve as the teacher components.  \n2. **Knowledge Embedding Generation**: We extract knowledge embeddings from both trained teacher models.  \n3. **Knowledge Distillation**: These embeddings are distilled into the CompactGCN student model using our unified loss function.\n\n[1] Zhang, S., Liu, Y., Sun, Y., & Shah, N. (2022). Graph-less neural networks: Teaching old MLPs new tricks via distillation. \\textit{ICLR 2022}.\n\n### Weakness 4. Lacks formal proofs or detailed justification:\n\nYou can find our full response to this comment at the following link:\n\n🔗 **[Full Response – Lacks formal proofs or detailed justification (Reviewer scTP)](https://github.com/1234554321-00/SHARP-Distill/blob/main/README.md)**\n\nTheoretical claims have been added to strengthen our framework. Please refer to the following link for details:\n\n🔗 **[Theoretical Extensions and Supporting Lemmas](https://github.com/1234554321-00/SHARP-Distill/blob/main/README.md)**", "claims_and_evidence": "The primary claim is that SHARP-Distill significantly improves inference speed without sacrificing recommendation quality. However, the evidence is empirical, lacking detailed computational resource specifics, which could affect practical speed claims.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "The paper lacks formal theorems, relying instead on established techniques.", "experimental_designs_or_analyses": "The experimental setup is reasonable.", "relation_to_broader_scientific_literature": "The proposed method is built on knowledge distillation and HGNNs. It uniquely combines these fields to tackle the speed-accuracy trade-off, a persistent challenge in the domain.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Strengths:\n1. Novel integration of HGNNs, DeBERTa, and contrastive learning.\n2. Dramatic inference speed improvements (up to 68×).\n3. Competitive or superior performance across datasets.\n4. CompactGCN offers a lightweight solution for real-time systems.\n\nWeaknesses:\n1. Missing details on preprocessing and resources analysis.\n2. No discussion of teacher model training time and complexity.\n3. Multi-component design may complicate implementation.\n4. Lacks formal proofs or detailed justification.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "See weaknesses.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6RNVmFnknY", "reviewer": "Reviewer_8pQM", "summary": "The paper proposes SHARP-Distill, a framework which uses DeBERTa language model as teacher model to distill HGNN-based recommenders to enhance recommendation performance and inference speed. A contrastive leaning mechanism is leveraged to efficiently inherit the structural and semantic knowledge. Experiments on multiple datasets demonstrate that SHARP-Distill achieves significant inference speed improvement compared to traditional methods like HGNNs and LightGCN, while competitive or even better performance.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### Claims And Evidence:\n\nWe appreciate the reviewer’s thoughtful critique regarding the fairness of our inference-time comparison and broader deployment considerations.\n\nFirst, we clarify that our comparison is primarily with other **distillation-based models** (e.g., KRD, LightHGNN), making the results in Table 1 a fair and meaningful evaluation. Regarding teacher model costs, we report the full training time (4.2 hours on a single A100 GPU) in Table 1 (as also noted in Reviewer scTP’s comments). This one-time overhead is amortized over deployment and does not impact end-user latency.\n\nOur inference speed claims (68× faster than HGNN, 40× faster than LightGCN) focus specifically on **deployment-time performance**, which is critical for real-time recommendation use cases. Moreover, our modular design supports efficient knowledge updates (e.g., retraining only HGNN or DeBERTa if needed), reducing long-term maintenance burdens.\n\nRegarding the concern that KD benefits may diminish over longer training, we conducted extended training experiments (3× standard epochs). We observed stable accuracy across all models, with our relative improvements holding within ±2% variance—indicating that the performance gains are not just due to warm-start effects but reflect genuine enhancements.\n\nWe have incorporated these extended training results and deployment discussions into the revised paper. We believe this addresses the reviewer’s concerns while reinforcing our core claim: **SHARP-Distill achieves substantial practical gains for real-world recommendation systems**.\n\n### Questions Q1) \n\nWe appreciate the reviewer's insightful question. To assess potential conflicts between semantic and structural features, we conducted an ablation study on the Yelp dataset, evaluating the impact of removing each core component from SHARP-Distill. Specifically, we examined four variants:\n\n- **SHARP-Distill:** Full model with HGNN-based structure, DeBERTa-based semantics, and contrastive learning  \n- **w/o DeBERTa:** Structure only (removes semantic encoder)  \n- **w/o HGNN:** Semantics only (removes structural encoder)  \n- **w/o Contrastive Loss:** Structure + semantics without alignment  \n\n**Table 3: Ablation Study on Yelp (P@10 / R@10 / N@10, %)**  \n| Variant              | P@10 | R@10 | N@10 | Inference (ms) |\n|----------------------|------|------|------|----------------|\n| SHARP-Distill        | **3.88** | **2.75** | **2.37** | 8.79           |\n| w/o DeBERTa          | 3.15 | 2.21 | 1.85 | 6.22           |\n| w/o HGNN             | 2.93 | 2.04 | 1.67 | 6.45           |\n| w/o Contrastive Loss | 2.74 | 1.93 | 1.46 | 6.01           |\n\nTo quantify the contribution of each component, we calculated the relative performance drop in P@10 with respect to the full SHARP-Distill model:\n\n**Δ Performance Compared to SHARP-Distill (P@10):**  \n| Component Removed       | Absolute Drop | % Drop   |\n|------------------------|----------------|----------|\n| DeBERTa (semantic)     | 0.73           | 18.81%   |\n| HGNN (structural)      | 0.95           | 24.48%   |\n| Contrastive Learning   | 1.14           | 29.38%   |\n\nThese results demonstrate the complementary nature of semantic and structural features, and the crucial role of contrastive learning in harmonizing them. Removing DeBERTa causes an 18.8% performance drop, indicating that semantic features are highly informative. Removing HGNN leads to a 24.5% drop, showing the importance of structural signals. Most notably, removing contrastive loss results in a 29.4% performance drop, the largest among all variants. This highlights that simply combining structural and semantic encoders is insufficient—without proper alignment, modality interference reduces effectiveness.\n\n**Our contrastive objective (Eqs. 6–7, 17–19)** explicitly resolves potential conflicts by aligning heterogeneous modalities into a shared latent space. The observed performance degradation without this alignment empirically validates our hypothesis: contrastive learning not only unifies the two modalities but also unlocks their synergy. These findings are now explicitly stated and discussed in our revised paper.\n\n### Questions Q2) \n\nYou can find our full response to this comment at the following link:\n\n🔗 **[Full Response – As discussed in Claims And Evidence section (Reviewer 8pQM)](https://github.com/1234554321-00/SHARP-Distill/blob/main/README.md)**\n\nTheoretical claims have been added to strengthen our framework. Please refer to the following link for details:\n\n🔗 **[Theoretical Extensions and Supporting Lemmas](https://github.com/1234554321-00/SHARP-Distill/blob/main/README.md)**", "claims_and_evidence": "The following claim could benefit from further clarification or additional evidence:\n\nClaim: 68× faster inference time compared to HGNN and 40× faster than LightGCN while maintaining competitive recommendation accuracy\n\nPotential issue: Directly compare the inference time of non-distilled models with distilled student models might not be fair. The training and inference cost of teacher models, maintaining cost of teacher models should also be considered, depending on the actual deployment infra system design. Besides, the recommendation accuracy improvement of KD often decrease after a longer training time in real-world cases as it provides warm-start advantages. Whether the degree of presented performance improvement can still hold requires more evidence.", "methods_and_evaluation_criteria": "-", "theoretical_claims": "-", "experimental_designs_or_analyses": "-", "relation_to_broader_scientific_literature": "-", "essential_references_not_discussed": "-", "other_strengths_and_weaknesses": "-", "other_comments_or_suggestions": "-", "questions_for_authors": "1. Are there any trade-offs or conflicts between the structural and textual features that need to be managed?\n2. As discussed in Claims And Evidence section, what's the training and inference cost of the teacher model? what's the ROI considering the cost of teacher and student as a whole compared with other models?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yR6DCdvu4u", "reviewer": "Reviewer_xTAj", "summary": "This paper introduces SHARP-Distill, a knowledge distillation framework combining Hypergraph Neural Networks (HGNNs) with language models to improve recommendation quality while reducing inference time. The teacher-student approach uses HGNNs for user-item embeddings and DeBERTa for extracting textual features, while the lightweight student model (CompactGCN) inherits structural knowledge through contrastive learning. Experiments show up to 68× faster inference than HGNN while maintaining competitive accuracy.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the thoughtful feedback and are grateful for the recognition of our work’s strengths, including inference speedup, empirical rigor, and the novel integration of hypergraphs with language models. \n\n## Weakness 1) Differentiating RecSys-specific distillation from general graph learning:\n\n**(Relation To Broader Scientific Literature)**\n\nWe appreciate this observation and have clarified the RecSys-specific contributions of our distillation strategy in Section~3.2 of the revised version. In particular:\n\n* **User-Item Bipartite Structure Exploitation:** Unlike general GNN distillation methods that operate on homogeneous graphs, SHARP-Distill explicitly models the bipartite structure of user-item interactions using dual incidence matrices $\\mathcal{H}_U$ and $\\mathcal{H}_I$ (Equations 2--3). This formulation captures asymmetric collaborative patterns unique to recommender systems.\n    \n* **Hypergraph Modeling of High-Order Interactions:** Our hypergraph-based formulation extends beyond pairwise user-item links to encode group-level behavioral patterns (e.g., co-engagement with semantically related item clusters), which are crucial for capturing latent preference signals in sparse recommendation data.\n    \n* **Preference-Aware Contrastive Transfer:** As detailed in Equations 17--18, our contrastive objective is designed to align user preference rankings rather than only topological proximity, which is essential for recommendation tasks.\n    \n* **Multi-Modal Distillation with Reviews:** SHARP-Distill uniquely integrates semantic information from textual reviews alongside interaction structures. This dual-source distillation enables robust generalization to cold-start scenarios—an issue more prominent in RecSys than in general graph tasks.\n\n## Weakness 2) Comparison with Prior Distillation Methods on Public Benchmarks:\n**(Essential References Not Discussed)**\n\nWe have implemented and evaluated SHARP-Distill alongside recent RecSys-specific distillation approaches: **UnKD [Chen et al., WSDM 2023]** and Graph-less Collaborative Filtering **[Zhang et al., NeurIPS 2023]**. Both authors released their code, enabling reproducibility on shared datasets.\n\nAs shown in Table 2, SHARP-Distill outperforms both in terms of accuracy (P@10 / R@10 / N@10) across Amazon Cellphones, Beauty, and Sports. Moreover, our approach significantly reduces inference time, with speedups of over 2× on average. Due to limited rebuttal-phase time, we could not include large-scale datasets (Amazon CDs, Yelp) for these two models. However, we plan to include these full comparisons in the final version.\n\n**Table7: Comparison on Amazon Cellphones, Beauty, and Sports. Metrics: P@10 / R@10 / N@10 (%). Inference time in milliseconds (ms).**\n\n| **Model**                 | **Cellphones**       | **Beauty**           | **Sports**           | **Inference Time (ms)** |\n|--------------------------|----------------------|----------------------|----------------------|--------------------------|\n| Chen et al. (UnKD)       | 6.83 / 5.02 / 4.31    | 5.89 / 4.21 / 3.68    | 3.88 / 3.19 / 2.87    | 9.72 / 11.4 / 9.02        |\n| Zhang et al. (Graph-less)| 6.57 / 4.89 / 4.08    | 6.12 / 4.26 / 3.93    | 4.01 / 3.25 / 2.91    | 8.94 / 10.2 / 8.35        |\n| **SHARP-Distill (Ours)** | **7.54 / 5.77 / 4.77**| **6.97 / 4.52 / 4.15**| **4.27 / 3.63 / 3.24**| **4.12 / 7.88 / 5.74**    |\n\nWhile UnKD effectively mitigates popularity bias, it relies on MF/LightGCN backbones and does not incorporate high-order hypergraph structures or textual modalities. SHARP-Distill transfers richer preference-aware signals using contrastive learning from both **HGNN structure** and **DeBERTa-based textual features**.\n\nGraph-less avoids structural modeling altogether, trading off explainability and some precision. In contrast, SHARP-Distill uses a **lightweight CompactGCN** student that distills both structural and semantic knowledge while remaining highly efficient at inference.\n\n## Weakness 3) Training efficiency discussion:\n\n**Table 6: Teacher Model Training Time (in hours)**\n\n| Dataset            | SHARP-Distill Teacher | LightGCN |\n|--------------------|-----------------------|----------|\n| Amazon Cellphones  | 1.7                  | 0.8     |\n| Amazon Beauty      | 2.8                  | 1.3    |\n| Amazon Sports      | 2.4                  | 1.1     |\n| Amazon CDs         | 4.2                   | 2.0     |\n| Yelp               | 3.5                   | 1.7      |\n\nThese measurements include both HGNN and DeBERTa training and reflect the cost of a one-time offline pretraining phase. For context, we've included baseline LightGCN training times, showing that while our teacher requires approximately 2x longer to train, this cost is amortized through significant inference speedups. Theoretical claims have been added to strengthen our framework. \n\n🔗 **[Theoretical Extensions and Supporting Lemmas](https://github.com/1234554321-00/SHARP-Distill/blob/main/README.md)**", "claims_and_evidence": "The paper addresses data sparsity and high computational costs of GNNs, demonstrating that traditional soft-label knowledge distillation is insufficient for preserving hypergraph structural knowledge. Figure 7 shows the superiority of the proposed approach combining soft labels with structural knowledge transfer. Performance improvements and speed claims are well-supported by comprehensive experiments.", "methods_and_evaluation_criteria": "SHARP-Distill combines HGNNs, language models, and contrastive learning in a knowledge distillation framework. The evaluation uses standard metrics (Precision, Recall, NDCG, Hit Ratio) across five datasets, with careful analysis of both recommendation quality and inference time.", "theoretical_claims": "There is no theoretical claim in the paper.", "experimental_designs_or_analyses": "The paper includes comprehensive experiments comparing against 11 baselines, with ablation studies, hyperparameter sensitivity analysis, and scalability assessments. Results consistently show SHARP-Distill effectively balances recommendation quality with computational efficiency.", "relation_to_broader_scientific_literature": "While the paper provides background on hypergraphs and knowledge distillation, it could better differentiate between recommendation-specific and general graph learning approaches. Additional references on distillation methods for graph-based recommendation would strengthen the literature review.", "essential_references_not_discussed": "The paper should further discusses more distillation methods for graph-based recommendation. Not only methods for general graph learning.\n[1] Unbiased Knowledge Distillation for Recommendation\n[2] Graph-Less Collaborative Filtering", "other_strengths_and_weaknesses": "Strengths;\n1. Significant inference speed improvements (68× faster than HGNN)\n2. Comprehensive experimental evaluation\n3. Novel integration of hypergraphs with language models\n4. Effective knowledge transfer through contrastive learning\n\nWeaknesses:\n1. Limited discussion of recommendation-specific knowledge distillation methods\n2. Could strengthen theoretical justification\n3. Less focus on training efficiency compared to inference efficiency", "other_comments_or_suggestions": "N/A", "questions_for_authors": "Please refer to my comments above", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "twgWskJN60", "reviewer": "Reviewer_zAKq", "summary": "This paper focuses on the teacher-student knowledge distillation. The teacher model use contrastive learning to combine HGNN and a pre-trained LLM, which generate collaborative and semantic features, respectively. The student model is a lightweight GCN. Both response-based and feature-based knowledge distillation loss are adopted to transfer structural and positional knowledge.", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Essential References Not Discussed**\n\nWe thank the reviewer for highlighting recent work on LLM-based recommendation and distillation, including DLLM2Rec [1], SAID [2], and POD [3]. These are valuable contributions to this fast-moving area, and we have now incorporated a discussion of them into our Related Work section.\n\nWhile SHARP-Distill employs a pretrained textual encoder, we clarify that our method is not an LLM-based recommender in the conventional sense. In contrast to DLLM2Rec, SAID, and POD—which rely on full-scale LLMs, prompt tuning, or autoregressive decoding—our framework uses a lightweight review encoder (DeBERTa) on short user-item reviews (~10–40 tokens), avoiding the latency and cost of LLM inference.\n\nTo ensure a fair comparison, we evaluated SHARP-Distill against **SAID** and **POD** using their official code and settings. DLLM2Rec was excluded only due to lack of code at submission time.\n\n**Table 8: Comparison with LLM-based Recommenders (Top-K Metrics and Inference Time)**\n\n| Dataset     | Metric | SAID (Distill) | POD (Prompt) | SHARP-Distill | SAID Time (ms) | POD Time (ms) | SHARP Time (ms) |\n|-------------|--------|----------------|---------------|----------------|----------------|----------------|------------------|\n| CDs         | P@10   | 13.40          | 12.88         | **13.75**      | 110.0          | 450.0          | **9.77**         |\n|             | R@10   | 12.65          | 12.20         | **13.06**      |                |                |                  |\n|             | N@10   | **12.24**      | 11.88         | 12.17          |                |                |                  |\n| Cellphones  | P@10   | **7.83**       | 7.62          | 7.54           | 51.0           | 210.0          | **4.12**         |\n|             | R@10   | 5.72           | 5.60          | **5.77**       |                |                |                  |\n|             | N@10   | 4.69           | 4.62          | **4.77**       |                |                |                  |\n| Beauty      | P@10   | 6.58           | 6.60          | **6.97**       | 60.0           | 240.0          | **7.88**         |\n|             | R@10   | 4.38           | 4.29          | **4.52**       |                |                |                  |\n|             | N@10   | 3.97           | 4.01          | **4.15**       |                |                |                  |\n\nSHARP-Distill achieves comparable or better accuracy than LLM-based baselines while offering **10–40× lower inference time**, thanks to its compact GCN-based student and contrastive fusion strategy. Unlike POD, which requires real-time LLM queries, or SAID, which retains large students, our model delivers **LLM-level quality with sub-10ms latency**, making it ideal for scalable deployment in time-sensitive environments. We appreciate the reviewer’s suggestion and believe this further clarifies SHARP-Distill’s positioning as an efficient, LLM-alternative framework for real-world recommendation.\n\n**Other Strengths and Weaknesses**\n\nThank you for the detailed critique. Due to space limits, we provide a full point-by-point response—including novelty clarifications, differences from [4–6], and positional alignment motivation—at the following link:\n\n🔗 **[Full Response – Other Strengths & Weaknesses](https://github.com/1234554321-00/SHARP-Distill/blob/main/README.md)**\n\nTheoretical claims have been added to strengthen our framework. Please refer to the following link for details:\n\n🔗 **[Theoretical Extensions and Supporting Lemmas](https://github.com/1234554321-00/SHARP-Distill/blob/main/README.md)**", "claims_and_evidence": "Yes, claims made in the submission are supported by clear and convincing evidence.", "methods_and_evaluation_criteria": "Yes, the proposed methods and evaluation criteria make sense.", "theoretical_claims": "There is neither theoretical claim nor proof in this paper.", "experimental_designs_or_analyses": "The teacher model requires the knowledge from a pre-trained LLM, which is transferred to the student through knowledge distillation. However, the baseline methods don't leverage the LLM. It seems to be unfair. By comparing results in Table 1 and Table 4, SHARP without LLM cannot outperforms most baseline methods.", "relation_to_broader_scientific_literature": "This paper confirms the feasibility of simultaneously distilling the knowledge in HGNN and LLM into a simple model.", "essential_references_not_discussed": "The teacher model in this paper combines a traditional recommendation model and a LLM. And the knowledge from LLM plays a important role. But it cited no related work. For example, [1, 2, 3]\n\n[1] \"Enhancing Sequential Recommendation via LLM-based Semantic Embedding Learning.\" WWW 2024\n\n[2] \"Distillation Matters: Empowering Sequential Recommenders to Match the Performance of Large Language Model.\" arXiv 2024\n\n[3] \"Prompt Distillation for Efficient LLM-based Recommendation.\" CIKM 2023I keep up with the literature in this area.", "other_strengths_and_weaknesses": "Weaknesses:\n\n1. The novelty is limited. The first contribution claimed by the authors is combining structural and semantic features. However, they simply concatenate them, followed by a MLP. The second contribution simply comes from their use of knowledge distillation. As for the third contribution, many previous works[4, 5, 6] have used contrastive learning for knowledge distillation. To sum up, the only novelty left is the use of positional similarity. However, there is little discussion in this paper on why it is useful.\n\n   [4] \"Contrastive Representation Distillation.\" ICLR 2020\n\n   [5] \"Contrastive Distillation on Intermediate Representations for Language Model Compression.\" EMNLP 2020\n\n   [6] \"Distilling Holistic Knowledge with Graph Neural Networks.\" ICCV 2021 \n\n2. The formal details about DeBERTa in Preliminaries are not used in the main text. It should be removed or moved to the appendix.\n\n3. There is no indication at all of the target task. Is it recommendation with implicit feedback or rating prediction? Eq.(8) claims that there is a ground truth \"rating\", but the evaluation metrics used in the experiments are typically for implicit feedback and do not include MSE.\n\n4. The author seems to have misunderstood some basics of recommender system. For example, Eq.(8) is MSE loss, not BPR loss.\n\n5. There are a lot of errors in the formulas. For example, Eq.(9) is inconsistent with the definition of $L_{teacher}$ in Figure 1. Moreover, 'j' in Eq.(7) is not defined. $N_u$ in Eq.(12) is not defined. And, $D$ used in the computation of $\\hat A^s$ should be specified to be the degree matrix of $A^s+I$, not $A^s$, to avoid confusion.", "other_comments_or_suggestions": "1. There should be more discussion on how Eq.(16) transfers positional knowledge to the student model.\n2. Figure 7 is a combination of Figure 5 and 6? It looks strange. Using subfigures is a better option.", "questions_for_authors": "I have no other questions.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposes SHARP-Distill, a knowledge distillation framework that combines a Hypergraph Neural Network (HGNN) and a language model (DeBERTa) as a teacher to generate high-order and semantic features, which are then distilled via contrastive learning into a lightweight student model, CompactGCN. The goal is to maintain competitive recommendation accuracy while achieving significant inference speedups—up to 68× over HGNNs and 40× over LightGCN.

The paper addresses a timely and practically important topic in efficient recommender systems. However, several critical concerns raised by the reviewers were not fully resolved, and the paper ultimately falls short of the bar for ICML. First, while the proposed framework is well-motivated and empirically validated, the core methodological contributions are limited. The combination of knowledge distillation, hypergraph modeling, and contrastive learning, though novel in this particular composition, largely repurposes established ideas without sufficiently deep theoretical or algorithmic innovation. This was particularly noted by Reviewer zAKq, who pointed out that components such as contrastive distillation and multimodal fusion are already well-explored in the literature.

Second, although the rebuttal addressed several technical inaccuracies and expanded on the theoretical aspects, these improvements came late in the process and were not acknowledged by multiple reviewers. Furthermore, the reliance on a strong teacher model that uses an LLM raises fairness issues in comparative evaluation, as several baseline models do not incorporate similar textual information. The authors’ clarification that DeBERTa is only used offline and on short reviews does mitigate this concern somewhat, but it remains unclear how generalizable the results are across deployment settings.

Third, the clarity and rigor of the paper could be further improved. Issues with notation, the mismatch between loss functions and evaluation metrics, and the inconsistent description of the target task (implicit vs. explicit feedback) suggest that parts of the submission were not polished to a high standard. While the authors made commendable efforts to clarify these during the rebuttal, the responses came too late to significantly alter the overall evaluation trajectory.
Decision: Accept (poster)