EduLLM: Leveraging Large Language Models and Framelet-Based Signed Hypergraph Neural Networks for Student Performance Prediction
Authors: Ming Li
Yukang Cheng
Lu Bai
Feilong Cao
Ke Lv
Jiye Liang
Pietro Lio
Venue: ICML 2025 poster (2025)
Keywords: Signed Hypergraph Neural Network
Hypergraph Learning
Graph Neural Networks
Large Language Models
Primary area: deep_learning->graph_neural_networks
TL;DR: An Exploration of LLMs and Signed Hypergraph Learning for Educational Data Analytics

Abstract: The growing demand for personalized learning underscores the importance of accurately predicting students' future performance to support tailored education and optimize instructional strategies. Traditional approaches predominantly focus on temporal modeling using historical response records and learning trajectories. While effective, these methods often fall short in capturing the intricate interactions between students and learning content, as well as the subtle semantics of these interactions. To address these gaps, we present EduLLM, the first framework to leverage large language models in combination with hypergraph learning for student performance prediction. The framework incorporates FraS-HNN ($\underline{\mbox{Fra}}$melet-based $\underline{\mbox{S}}$igned $\underline{\mbox{H}}$ypergraph $\underline{\mbox{N}}$eural $\underline{\mbox{N}}$etworks), a novel spectral-based model for signed hypergraph learning, designed to model interactions between students and multiple-choice questions. In this setup, students and questions are represented as nodes, while response records are encoded as positive and negative signed hyperedges, effectively capturing both structural and semantic intricacies of personalized learning behaviors. FraS-HNN employs framelet-based low-pass and high-pass filters to extract multi-frequency features. EduLLM integrates fine-grained semantic features derived from LLMs, synergizing with signed hypergraph representations to enhance prediction accuracy. Extensive experiments conducted on multiple educational datasets demonstrate that EduLLM significantly outperforms state-of-the-art baselines, validating the novel integration of LLMs with FraS-HNN for signed hypergraph learning.

EduLLM: Leveraging Large Language Models and Framelet-Based Signed Hypergraph Neural Networks for Student Performance Prediction
Ming Li
Yukang Cheng'
Lu Bai?
Feilong Cao
Ke Lu 45
Jiye Liang
Pietro Lio"
Abstract
features. EduLLM integrates fine-grained mantic features derived from LLMs, synergiz- ing with signed hypergraph representations to enhance prediction accuracy. Extensive experi- ments conducted on multiple educational datasets demonstrate that EduLLM significantly outper- forms state-of-the-art baselines, validating the novel integration of LLMs with FraS-HNN for signed hypergraph learning.
The growing demand for personalized learning underscores the importance of accurately predict- ing students' future performance to support tai- lored education and optimize instructional strate- gies. Traditional approaches predominantly fo- cus on temporal modeling using historical re- sponse records and learning trajectories. While effective, these methods often fall short in cap- turing the intricate interactions between students and learning content, as well as the subtle se- mantics of these interactions. To address these gaps, we present EduLLM, the first framework to leverage large language models in combina- tion with hypergraph learning for student perfor- mance prediction. The framework incorporates FraS-HNN (Framelet- based Signed Hypergraph Neural Networks), novel spectral-baas -baseddl-baedd model for signed hypergraph learning, designed to model interactions between students and multiple-choice questions. In this setup, students and questions are represented nodes, while response records encoded as positive and negative signed hyper- edges, effectively capturing both structural and se- mantic intricacies of personalized learning behav- iors. FraS-HNN employs framelet- based low-pass and high-pass filters to extract multi fulti-ffreuency
1. Introduction
Background. In recent years, the increasing emphasis on personalized learning has underscored the need for innova- tive strategies support individualized educational path- ways (Mayer, 2019). A critical aspect of this effort is the accurate prediction of student performance, which enables educators to tailor instructional strategies, optimize learning experiences, and improve educational outcomes (Dziuban et al., 2015). While adaptive learning technologies have demonstrated promise, they often struggle capture the intricate structural and semantic relationships inherent in the learning process, such interactions between students and educational content.
Existing approaches to student performance prediction predominantly rely on temporal modeling and graph neural networks (GNNs) to analyze historical answer- ing records, learning trajectories, and pairwise connections between learners and learning materials. Temporal mod- els focus sequential dependencies, while GNNs excel at representing "student-content" interactions (Zhou et al., 2020; Zhang & Chen, 2018). These methods have achieved considerable success by capturing local relationships within the learning ecosystem. However, their reliance on pairwise modeling limits their capacity to represent higher-order in- teractions involving multiple entities, such as a student's engagement with related questions or the interconnectedness of various knowledge points within course.
Motivation.
'Zhejiang Key Laboratory of Intelligent Education Technol- ogy and Application Zhejiang Normal University, Jinhua, China "School of Artifcial Intelligence, and Engineering Research Center Intelligent Technology and Educational Application, Ministry of Education, Beijing Normal University, Beijing, China "Scoool of Mathematical Sciences. Zhejiang Normal University. Jinhua, China "School of Engineering Science, University of Chinese Academy of Sciences, Beijing, China 'Peng Cheng Laboratory, Shenzhen, China KKy Laboratory Computational Intelligence and Chinese Information Processing of Ministry of Education, the School of Computer and Information Technology, Shanxi Uni- versity, Taiyuan, China 'Department of Computer Science and Technology, University of Cambridge, Cambridge, UK. Corre- spondence to: Lu Bai <bbiilu bnu.cdu.en>.
Hypergraph learning offers compelling alternative by ex- tending traditional graph structures to model higher-order relationships (Antelmi et al., 2023; Dai Gao, 2023; Kim et al., 2024). Unlike GNNs, hypergraphs can represent in-
Proceedings of the ndnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

EduLLM
teractions among multiple nodes simultaneously through hyperedges, providing richer representation of complex relationships (Li et al., 2024). For instance, student's responses to multiple-choice questions (MCQs) can be cap- tured as hyperedges, effectively modeling both structural and semantic associations. Despite this potential, the appli- methods in student performance
tions in existing approaches, this framework provides robust technological support for advancing personalized learning and facilitating precise educational interventions.
Contribution. The primary contributions of this work are summarized as follows:
cation of hypergraph- prediction remains limited. particularly in contexts that re- quire the integration of semantic insights and structural information.
Problem Formulation Perspective: We redefine the student performance prediction task as a hypergraph learning problem by introducing signed hypergraphs to represent interactions between students and questions, where response records are modeled as positive and negative signed hyperedges.
Mcanwhile, the advent of large language models (LLMs) has marked a transformative shift in natural language pro- cessing. LLMs excel at extracting -ne-grained semantic features from textual data, such as the content of MCQs, and offer a complementary approach to hypergraph learning by enhancing the semantic granularity of question representa- tions (Wang et al., 2024b). However, existing methods strug- gle to effectively integrate LLMs with hypergraph-band baased approaches, leading to feature embeddings that are noisy or fail to capture the full complexity of student-content interac- tions. Furthermore, many hypergraph learning frameworks lack mechanisms for multi-scale representation, which are critical for capturing both shared preferences among stu- dents and individualized distinctions in learning behaviors.
Model Development: We design a framelet-based signed hypergraph neural network (FraS-HNN) that leverages multiresolution signal processing to effec- tively learn hypergraph representations. This model employs low-pass filters to capture shared preferences among learners and high-pass filters to emphasize indi- vidual distinctions, facilitating a nuanced understand- ing of both structural and semantic relationships within the signed hypergraph.
We propose EduLLM, a uni- fied framework that synergizes hypergraph learning and LLM-based semantic analysis. By combining hypergraph- based structural modeling with the seman- tic granularity provided by LLMs, EduLLM achieves a comprehensive representation of student-conten in- teractions. This integration significantly improves the precision of student performance predictions, demon- strating the effectiveness of the framework across mul- tiple datasets.
Framework Design:
Our Methodology. To address these challenges, this paper introduces EduLLM, a unified framework that integrates hypergraph learning with LLMs to enhance the precision of student performance prediction. At its core, EduLLM formulates the student performance prediction task as a hypergraph learning problem, where interactions between students and MCQs are modeled using signed hypergraphs. In this formulation, students and questions are represented as nodes, and their response records are encoded as positive and negative signed hyperedges, capturing both structural and semantic intricacies of the learning process. A key innovation of EduLLM lies in its framelet-based signed hypergraph neural network (FraS-HNN), which leverages multiresolution signal processing to effectively learn hy- pergraph representations. This model employs low-pass filters to identify shared patterns in learning behaviors and high-pass filters to highlight individual differences, provid- ing a nuanced understanding of student-content interactions. Framelct-basee transformations further enable decomposi- tion and reconstruction in the frequency domain, enhancing the model's 'ss ability capture multi-scale features while re- ducing noise and improving data compression. Fine-grained semantic features extracted by LLMs are seamlessly inte- grated into this framework, enriching the representation of course content and improving prediction accuracy. Exten- sive experiments on multiple educational datasets demon- strate that EduLLM outperforms state-of-the-art baselines, validating the effectiveness of combining hypergraph leam- ing and LLM-based semantic analysis. By addressing limita-
2. Problem Formulation and Notation
In this section, we introduce how the interactions between students and multiple-choice questions (MCQs) can be rep- resented using a signed hypergraph. The interaction data includes not only students' responses MCQs but also detailed textual information associated with each question, such as the question stem, options, answers, and explana- tions. These interactions are naturally modeled as signed hypergraph, where nodes represent students and questions, and hyperedges capture the relationships between students and specific questions based on their responses.
Signed Hypergraph. The signed hypergraph is mathemati- cally defined as G (V, where V = U U Q represents the set of nodes, comprising the student set l and the ques- tion set Q. The hyperedge set E consists of two components: the positive hyperedge set E and the negative hyperedge set E- Specifically, a positive hyperedge et E E con- nects all students who answered question correctly, while
2

---

EduLLM
negative hyperedge e E E connects all students who answered the same question incorrectly.
and questions. While signed graphs are limited pair- wise interactions, signed hypergraphs enable hyperedges to connect multiple nodes simultaneously, capturing group interactions that are critical in educational settings. For example, a single hyperedge in a signed hypergraph can represent the collective response of multiple students to a specific question, providing a richer and more expressive representation of learning behaviors. This capability allows the signed hypergraph to encode non-local relationships, such as shared misconceptions among students or the inter- play of multiple knowledge points within a course, which signed graphs cannot adequately capture. These characteris- ties highlight the potential of signed hypergraphs for more comprehensive modeling of student-content interactions and the advancement of personalized learning systems.
To formally describe the structure of the signed hypergraph,
we use an incidence matrix H E {0, 1, - N X.M where N = [V] is the number of nodes and M = |E| is the number of hyperedges. The matrix element H (v, e) encodes the relationship between a node E V and hyperedge e E E: (i) H(U I. if v belongs to positive hyperedge et: (ii) (u,,c) = 11, if belongs to a negative hyperedge <:: (ii) ((),)) = 0 otherwise.
The degrees of nodes and hyperedges are represented by the diagonal matrices D. E RNN and De E IR MMM, respectively, as computed as follows:
u,(,, = > [((,,)), Do(e,e)=) - [H(u,))]- (1)
EEE
VEV
3. Method
Task Definition. Within this signed hypergraph framework, the student performance prediction task is formulated hyperedge sign prediction problem. Here, the objective is to predict the unknown signs of hyperedges, representing whether students correctly or incorrectly respond to spe- cific questions. Given a signed hypergraph G = (V, we aim to learn low-dimensional embeddings for students and questions, denoted as Zu, for student u, E U and wa, for question 9j. where Zu.; Wg, E Rd and d is the embed- ding dimension. The embeddings are used to predict the sign of an unknown hyperedge Ci via mapping function: [Zzm, ) + -1/+ 1.
3.1. Framework Overview
As shown in Figure 1, EduLLM consists of three key mod- ules: i) an LLM-based semantic extraction module, which extracts key components from MCQs (e.g., stems, options, answers, and explanations) and generates semantic embed- dings enriched with contextual knowledge; ii) a signed hy- pergraph construction module, which models interactions between students and MCQs by representing students and questions as nodes and encoding correct and incorrect re- sponses as positive and negative signed hyperedges; and iii) FraS-HNN a framelet-basee signed hypergraph neural net- work equipped with low-pass and high-pass filters for multi- frequency hypergraph learning. As the novel component of the framework, FraS-HNN is detailed in the following section, while further details about the first two components are provided in the Appendix D.
Remark 1. We note that our problem formulation, i.c., interactions through signed hy- pergraphs, differs fundamentally from conventional knowl- edge tracing (KT) (Abdelrahman et al., 2023) and cognitive diagnosis (CD) (Wang et al., 2024a) frameworks. KT and CD are typically centered on tracking students' mastery of underlying concepts over time and rely heavily on fine- grained mappings between questions and predefined con- cepts. In contrast, our formulation operates at the question level, leveraging signed hyperedges encode the correct- ness of student responses. This representation is inherently incompatible with the structure and assumptions of most KT/CD datasets, which are not designed to support signed or higher-order relational modeling at the question level. Technically, how to formulate signed hypergraphs within the KT/CD setting, particularly in the context of modeling concept-level mastery via Q-matrices, requires further in- vestigation. We leave this a potential direction for future work to connect signed hypergraph learning with traditional student modeling paradigms.
modeling
3.2. Construction of FraS-HNN
We propose FraS-HNN to analyze signed hypergraphs by decomposing their signals into multi-frequency components. This approach enables effective feature learning by leverag- ing framelet theory, capturing both shared (low-pass) and in- dividual (high-pass) features within the signed hypergraph.
Framelet Construction on Signed Hypergraphs. Let g = (V, E) denote signed hypergraph, where V repre- sents the set of N nodes, and È represents the signed hy- peredges (including both positive and negative ones). The signed hypergraph Laplacian L, encodes the structural re- lationships within G. Its eigendecomposstionn is expressed as: L, = MAM T, where M = [m,, m2,....... my]] is the matrix of eigenvectors, and A diag(8,, (8, 62,82,,.. (5....,5)) contains the eigenvalues.
The signed hypergraph framework offers a significant advantage over traditional signed graphs by its ability to model higher-order relationships among students
Remark 2.
Framelets are constructed using scaling and wavelet func- tions I = (()) ...., ((()), with associated filter banks
3

---

EduLLM
MCQs
Prompt
Respouse
planee
Task Objective
points from multipi-cchicee
Extract key knowdedge
conupeuentt
MCQ
"K
"Beyword "Rotation
Optina
Jugiter
Earth
C: Mart
Annesrr
Stem
which includer:
+ axpionatim
+Up to options
Keyworls recognition
Pariid",
purcsutas"":3
Rxplan
skortest
Requtremeents
"keyword" "Mapiter" proceetag"
alioa
appeessimataly
lours
Wonnar the sespouse in 1500
Total percentages squal
Calculate percentage
Assner
Hinchade the keywordd
Glave Word Embodding
MCQ:
Stu
Stukka Enrhedling
MCQ: Enrhedding
lowe
DCE
FraS-IINN
MLP
Lave
Framelet Based Signed Hypergraph Convolution
LowwPaso
flerel
High-Pass
devel 21
Fratere Matris
Figure 1. Schematic of proposed EduLLMM framework: The process begins with extracting semantic information from MCQs using an L.M,, where key components such as the question stem, options, answers, and explanations are identified and formatted into structured representation. The LLM generates semantic embeddings for the MCQs by recognizing keywords and calculating their relative contributions. These embeddings, combined with Glove word embeddings, are incorporated into signed hypergraph structure that models the interactions between students and MCQs. In this signed hypergraph, students and MCQs are represented as nodes, and their interactions are encoded as positive negative signed hyperedges, reflecting correct and incorrect responses, respectively. FraS-IINN, the proposed crumelet-based signed hypergraph neural network, processes this hypergraph using multi- multi-frrquency signal analysis. Specifically, the proposed spectral- -base signed hypergraph convolution module leverages framelet- bes-b low-pass and high-pass filters to capture shared patterns among nodes while highlighting individualized distinctions within the hypergraph structure. These processed features are then propagated through multiple layers of FraS-HNN to learn comprehensive embeddings for students and MCQs. The final prediction module employs a MLP with a binary cross coos-ennrropy loss function to predict student performance, completing the framework.
θ = ((g:h() (.). These filters satisfy:
construct the framelets. For system with two levels of decomposition (j = 1, 2) and one high-pass filter (r 1), the filters are defined as:
((2w) = ((w)@((w) (2w) = hr( (w)
where F(w) the Fourier transform of f. The scaling function & captures shared patterns (low-pass), while the wavelet functions vi(t) (r ,,....A)) highlight distinctions (high-pass). For node at scale j, the low-pass and high- pass framelets are defined as:
!
A
S
COs
COS
(5)
2
8
16
A
A
(b)
sin
(6)
2
16
A
(b1)
sin
8p
4
16
1 (u) =
mp mp(u)mp(u)
(3)
2
1=1
yielding one low-pass and two high-pass components. These filters enable efficient decomposition of signals into multi- frequency components.
0p 2
(r)
mn mp(u)mp(u).
(4)
p=1
Framelet-Based Signed Hypergraph Convolution. From the implementation perspective, eigendecomposi- tion of L. can be computationally expensive for large
Filter Design with Haar-Type Functions. For computa- tional efficiency, FraS-HNN employs Haar-type filters to

---

EduLLM
hypergraphs. FraS-HNN addresses this by approximating spectral filters using Chebyshev polynomials:
to predict the sign of the unobserved hyperedge ety. To accomplish this, concatenate the student embedding Zu; with with the question embedding wj. forming combined vector. This concatenated vector is then passed through a multi-layer perceptron (MI.) to predict the edge sign, as described by the following form:
((A) Σ Cn TA), (A)
Σ (A)) (8)
where (z) is the n-th Chebyshev polynomial. This ap- proximation reduces computational costs while retaining spectral properties.
Upred = MLP(zu, IWy).
(12)
Here, the predicted score Upred corresponds to the likelihood that the hyperedge sign is positive, with higher scores indi- cating stronger probability of positive sign, and lower scores suggesting a negative sign.
The framelet-basee signed hypergraph convolution operator is formally defined as:
F(X)
For the training objective, we adopt the binary cross-entropy loss function to optimize the hyperedge sign prediction task, that is:
(9)
diag diagg(o..))) To,
To, and Trrr are the low-pass and high-pass decom- position operators, respectively, f,r and o,,J are trainable spectral filters applied to the corresponding frequency com- ponents. The decomposition operators are defined as:
LCE = - logg (og((eeed)) + +11 y) log(1 - ypred), (13)
where y denotes the ground-truth label of the hyperedge sign, with mapping {-1,1} to {0, 13.
Additional theoretical properties of FraS-HNN, along with rigorous proofs, are provided in the Appendix B.
To., = @
M, Ty.- M (b((r)
M. (10)
2
21-1
These operators project the signal X onto the low low-pass and high-pass framelets, enabling multi-frequency analysis.
4. Experiments
In this section, we conduct series of experiments to evalu- ate the effectiveness of the proposed EduL_MM framework. The primary objectives are to address the following research questions:
Building FraS-HNN. FraS-HNN is constructed by stack- ing multiple layers of framelet-basse signed hypergraph convolution. To improve learning stability and mitigate the over-smoothing issue, FraS-HNN employs initial residual and identity mapping techniques introduced in (Chen al., 2020). Each layer is defined as:
Q1: How does EduLLM perform compared to state-of- the-art signed graph representation learning methods and classic graph neural network models?
x(++)== = o ((111 + axx
Q2: What are the contributions of high- sash-passssss infor- mation, low-pass information, and semantic feature embeddings to EduLLM's performance?
(11)
(1- (1- (,0()))))))) (1-
Q3: How do key hyperparameters impact the perfor- mance of EduLLM?
where (*) is the feature matrix at layer f, (0) is the initial feature matrix, F()) represents the framelet-based signed hypergraph convolution operator, (1) is a trainable diag- onal scaling matrix, ar E [0, 1] balances the contributions of the initial and current features, 3e e [o, 1] adjusts the im- pact of the identity mapping, (-)) is a non-linear activation function (e.g., ReLU).
Q4: How robust is the EduLLM model with respect to the L.M--inuued semantic representation embed- dings?
Q5: How does the proposed FraS-HNN module benefit the EduLLM framework from a hypergraph learning perspective?
3.3. Training Objective
4.1. Baselines and Evaluation Metrics
After obtaining the embeddings for each student i and ques- tion j from the signed hypergraph learning module equipped with the FraS-HNN model, we represent these embeddings as Zu: and W, E Rdd where d denotes the embedding di- mension. The function (Zu,, w,) -> {-1,+1] +13 is used
To address Q1, we evaluate EduLLM on five real-world datasets by comparing its performance against several base- lines, including Random Embedding, Graph Convolutional Network (GCN) (Kipf & Welling, 2017), Graph Attention
5

---

EduLLM
Networks (GAT) (Veličković et al., 2018), Signed Graph Convolutional Network (SGCN) (Dem al., 2018), Signed Bipartite Graph Neural Network (SBGNN) (Huang et al., 2021), Signed Bipartite Graph Contrastive Learning (SBCL) (Wang et al., 2024b), and its variant version LLMM-BCL. (Wang et al., 2024b), see more details in the Appendix E.I. For Q2 and Q3, we conduct ablation studies and sensitivity analyses examine the contributions of individual compo- nents and the influence of key hyperparameters. To answer Q4, we perform robustness tests to assess the reliability and applicability of the integrated large language models. For Q5, we replace the FraS-HNN module with other existing HNN models while keeping the remaining components of EduLLM unchanged. This allows for a convincing evalu- ation of the role that FraS-HNN plays in learning signed hypergraphs within the EduLLM framework.
Table I. Statistics of the five real-world datasets.
Biology
Law
Cardiff' Sydney19 Sydney23
U
761
528
383
382
198
V
380
5600
1171
457
748
E
760
11,200
2.342
914
1,496
F]
76,613
88,563
64,524
24,032
24,050
Pos Link
66.5%
93.1%
60.0%
53.1%
70.6%
Neg Link
33.5%
6.9%
40.0%
46.9%
29.4%
interact through hyperedges labeled to reflect the correct- ness of responses. The hypergraph captures the higher-order relationships between multiple students and questions, pro- viding a richer structural representation compared to the graph-based and signed graph-based based models.
Table provides a detailed comparison of the data charac- teristics for both signed hypergraphs and signed graphs con- structed from the five real-world datasets. In this table, the number of hyperedges within the signed hypergraph is de- noted by [g), which captures the higher-order relationships between multiple students and single question. In contrast, the number of edges within the signed graph is denoted by [b), representing the direct, individual interactions be- tween students and questions. Naturally, |E] is significantly smaller than |E}, reflecting the ability of hypergraphs to represent co-occurrence (higher-order) relationships among students and questions.
Given the imbalance between positive and negative links in the datasets, we employ two evaluation metrics: area under the curve (AUC) and binary-average Fl score (Binary-FI). Both metrics provide a balanced assessment of performance, with higher values indicating better results.
4.2. Datasetes
Multiple Choice Questions (MCQs) are a fundamental tool for assessing students' knowledge and learning progress, particularly on online education platforms where they play critical role in evaluating learners. In this study, we uti- lize real-world datasets containing interaction information between students and MCQs. Each interaction is recorded as either a correct response (marked as '+') or an incorrect response (marked as +"), enabling the construction of both signed graphs (to align with baseline models) and signed hypergraphs (to fit the problem formulation of EduLLM). The five datasets (Wang et al., 2024b) used in our exper- iments are derived from courses at three universities: the Biology and Law courses from the University of Auckland, the Cardiff20100 102 Medical School course from Cardiff Uni- versity, and the biochemistry courses Sydney and Syd- ney23146 from the University of Sydney.
Further details on the datasets and the process of formu- lating the original 'students-MCQs' data sources as signed hypergraphs are provided in Appendix C.
4.3. Experimental Setup
We note that, in our experiments, the semantic embeddings are pre-encoded and provided by (Wang et al., 2024b), rather than being directly derived from the original multiple-choice question (MCQ) data (e.g.. question stems, options, and ex- planations). As a result, is not feasible to directly compare the performance of EduLLM using different LLM mod- ules. To roughly investigate the impact of LLLM-induced embeddings, we conduct robustness study described in Section 4.7 (for Q4). Meanwhile, using the same pre- encoded semantic embeddings ensures a fair comparison when analyzing the specific contribution of the FraS-HNN module, as presented in Section 4.8 (for Q5).
Signed Hypergraph Construction. To construct the signed hypergraph for each dataset, we leverage the students' re- sponse records. Each student's response to a question is classified as either correct or incorrect. Since answering processes for questions are independent, the hypergraph structure naturally emerges from the data. Specifically: i) For each question node Oj E V, students who correctly answer the question form a positive hyperedge ('+1') con- necting the question node with the corresponding student nodes U, E V; ii) Similarly, students who answer the ques- tion incorrectly form negative hyperedge ('-1") with the same question node. Thus, each dataset is represented as a signed hypergraph, where student nodes and question nodes
The datasets are partitioned into three subsets to ensure a ro- bust evaluation: 85% for training capture underlying data patterns, 5% for validation to fine-tune hyperparameters, and 10% for testing to assess the model's generalization capability. Each model was trained for 300 epochs to en- sure convergence while reducing the risk of overfitting. To enhance the reliability of the results, all experiments were repeated ten times. The mean and standard deviation of the
6

---

EduLLM
Table 2. Performance comparison on five real-world educational datasets (average binary-F1 + standard deviation).
Biology
Law
Cardif120102
Sydney 1 19351
Sydney23146
Random
0.350 0.010
0.472 0.001
0.136 0.062
0.290 0.014
0.288 0.035
GCN
0.682 + 0.058
0.823 + 0.010
0.677 + 0.024
0.642 = 0.021
0.728 + 0.013
GAT
0.618 + 0.013
0.817 +: 0.050
0.371 0.013
0.564 0.022
0.608 + 0.020
SGCN
0.768 +: 0.040
0.840 + 0.013
0.607 0.033
0.635 0.044
0.726 0.040
SBGNN
0.753 3 0.014
0.861 + 0.034
0.712 0.016
0.673 0.016
0.712 = 0.021
SBCL
0.772 + 0.016
0.901 0.016
0.718 0.018
0.674 = 0.021
0.733 + 0.019
LLM-SBCL
0.787 + 0.014
0.908 + 0.018
0.734 + 0.023
0.694 0.021
0.760 + 0.022
EduLLM (Ours)
0.809 + 0.010 0.945 + 0.005
0.753 + 0.011
0.712 = 0.016
0.829 + 0.006
performance metrics were reported to account for variations due to data splits and random initialization, thereby ensuring robust and unbiased evaluation. Detailed reproducibility pa- rameters for the optimal results obtained in our experiments are provided in the Appendix F.2.
datasets, demonstrating its potential adaptability and appli- cability in varying educational contexts.
4.5. Ablation Study (Q2)
To address Q2, we conduct ablation study to evaluate the contributions of high-pas figh-pas information, low-pat informa- tion, and semantic feature embeddings to the performance of the EduLLM model. In particular, we compare the full model with three ablated variants: i) w/o High: This variant excludes the high-pass filter, retaining only the low-pass filter. It evaluates the role of high-frequency information in capturing fine-grained distinctions within the hypergraph; wi)w/o Low: This variant removes the low-pass filter, re- lying solely on the high-pass filter. It assesses the contri- bution of low-frequency information in modeling shared preferences and global patterns; iii) w/o LLM: This variant eliminates the semantic embeddings derived from the LLM and relies solely on structural embeddings. It investigates the significance of incorporating semantic information into the hypergraph representation.
4.4. Results and Discussion (Q1)
To address QI, we compare EduLLM with various baseline models on the five datasets. The comparison results, sum- marized in Table 2. lead the following key observations:
Superior Performance of EduLLM. Across all datasets, EduLLM consistently outperformed all baseline models, achieving the highest binary Fl scores. This result under- scores the advantages of hypergraph-based models in cap- turing multi-scale signed information. The use of framelet- based hypergraph convolution, which effectively integrates low-pass and high-pass information, was instrumental in achieving this superior performance.
I Limitations of Vanilla GNN Models. As evident from Ta- ble 2, traditional unsigned graph learning methods such as GCN and GAT demonstrate limited effectiveness in model- ing signed graphs or hypergraphs. These models are unable to fully capture the intricate signed interactions inherent in educational contexts. In contrast, EduLLM leverages its signed hypergraph structure to model these complex relationships, consistently delivering better predictive per- formance across all datasets.
Observation and Discussion. EduLLM's integration of both low-pass and high-pass features is theoretically moti- vated, as discussed in the Appendix B, where FraS-HNN is designed to comprehensively capture the diversity and complexity of signed hypergraph signals. This balanced integration of high- and cww-frequeccy information, com- bined with semantic embeddings, forms a robust founda- tion for modeling diverse patterns within the data. The results of the ablation study, presented in Table 3, demon- strate the substantial contributions of each component to the overall performance of EduLLM. Removing any of the key components leads to significant drop in binary-F1 scores across all datasets, underscoring their importance. Specifically, the high-pass component plays a vital role in capturing fine-grained distinctions, while the low-pass com- ponent effectively models global relationships. Together, these components of the framelet-based hypergraph convo- lution highlight the potential for designing more advanced hypergraph neural networks for signed hypergraph learn- ing. Furthermore, the semantic embeddings induced by the LLM significantly enhance the representation of knowledge
FraS-HNN vs. Signed GNNs Specifically, EduLLM also outperforms state-of-the-ar signed graph models, includ- ing SGCN, SBGNN, SBCL, and its variant LLM-SBCL, which integrates large language models. This significant im- provement highlights the strength of FraS- -Na's framelet- based hypergraph convolution, which effectively captures both local and global signed information. This enhanced higher-order feature expressiveness give Edul_MM a distinct advantage over these competing models.
Robustness and Stability. Relatively, EduLLM exhibits low standard deviations across all datasets, reflecting strong stability and generalization capability. Moreover, the model maintains solid performance despite the diversity of the

---

E
Table 3. Ablation study results (average binary-F1 standard deviation).
Full Model
w/o High
w/o Low
w/o LLM
Biology
0.809 0.010
0.801
0.005
0.799
0.005
0.795 0.017
Law
0.945 + 0.005
0.943 + 0.017
0.934 + 0.029
0.941 + 0.013
Cardiff'
0.753 + 0.011
0.731 + 0.019
0.741 + 0.017
0.740 + 0.001
Sydney19
0.712 0.016
0.705
0.014
0.693 0.006
0.703
+ 0.033
Sydney23
0.829
0.006
0.817
0.261
0.823 0.012
0.819 = 0.016
0.7
0.7
0.7
ATC
0.67
ADC
0.67
ATC
0.57
0.64
0.04
0.61
0.61
0.61
0.61
0.1
0.3
0.4
0.5
9.6
0.2
0.3
0.5
0.6
0.4
0.6
0.8
avaluuss
yvaluss
Avalues
Figure 2. Results for parameter sensitivity analysis on the Law dataset: a (LHS), 7 (middle), À (RHS), respectively.
points from the MCQs, demonstrating their ability to com- plement the structural embeddings. Overall, these findings highlight the individual and collective contributions of the high-/low-pans components, and the I.MM-indued semantic information, the overall merits of EduLLM.
4.7. Further Assessment for the Role of LLM-induced Semantic Representation (Q4)
To address Q4, we conduct additional studies to evaluate the robustness of EduLLM with respect to the semantic repre- sentations induced by the large language model. Due to the limitation that the semantic embeddings in our experiments are pre-encoded and provided by Wang et al. (Wang et al., 2024b), rather than derived directly from the original MCQ data (c.g.. question stems, options, and explanations), it is not feasible to directly compare EduLLM using different LLMs. As an alternative, we simulate variations seman- embeddings by introducing different levels of noise to the pre-encoded embeddings of the MCQs. This approach serves as a simple substitution for testing the robustness of the LLM backbone, the variance caused by the noise ap- proximates the differences in embeddings generated by dif- ferent LLMs. Specifically, we add additive white Gaussian noise M ~ (0,, o²) E Id to the semantic embedding ten- sor X E Rd The noise level o a = p(max((X)) min (X)).. where p E (0.01, 0.03, 0.05, 0.08, 0.13 indicates indicates the relative strength of the added noise.
4.6. Parameter Sensitivity Analysis (Q3)
To address Q3, we perform a detailed sensitivity analysis on three key hyperparameters: a, 7, and X. These hyper- parameters integral to balancing various components of the model and influence its performance: 1) regulates the balance between the current feature representation and the initial feature map; 2) controls the relative contribution of the two computational branches in the model: one based on random filtering and the other on adjacency matrix propaga- tion. It adjusts the interaction dynamies between these two pathways; 3) À modulates the weight of each layer's feature updates relative to the initial features through the scaling parameter 0. It affects the overall hierarchical learning of feature representations across layers.
To evaluate the impact of these byperaarameters, we conduct experiments by varying their values over predefined range. Due to page limitations, we present the sensitivity analysis results the Law dataset in Figure 2, while additional re- sults other datasets are included in the Appendix E.4. As shown clearly in Figure 2, EduLLM performs stably across a broad range of values for all three hyperparameters. While slight variations in performance are observed as parameter values change, the overall trends remain consistent, and no abrupt performance degradation occurs. This stability, to certain extent, highlights the robustness of EduLLM to parameter tuning, ensuring its adaptability and effectiveness across different configurations.
Following this design, comprehensively evaluate the pre- dictive performance of EduLLM under different noise levels. The results, presented in Fig. 3, show that as p increases, the model's performance remains relatively stable, with only minor declines observed at higher noise levels. This smooth degradation expected and indicates that EduLLM is robust to variations in the semantic embeddings, suggesting that re- placing the current LLM backbone with different one does not significantly impact the overall task performance. We should note that these experiments validate the robustness and reliability of EduLLM in handling noise interference, highlighting its ability to maintain consistent predictive ac-
8

---

EduLLM
Table 4. Performance comparison of Edul variants with different HNN modules.
Sydney 19351
Sydney23146
Biology
Cardif/20102
Law
EduLLM with HGNN
0.606 0.014
0.619
0.013
0.673
0.006
0.624 0.007
0.905 1 0.011
EduLLM with HyperGCN
0.620 + 0.012
0.650 + 0.038
0.631 + 0.016
0.625 + 0.023
0.901 + 0.032
EduLLM with AllDeepSets
0.626 + 0.017
0.660 + 0.012
0.697 + 0.008
0.637 + 0.023
0.898 + 0.009
EduLLM with AlISecTransformer
0.618 + 0.022
0.661 + 0.010
0.689 + 0.016
0.644 + 0.029
0.906 + 0.009
EduLLM with ED-HNN
0.662 0.023
0.708 + 0.032
0.715 0.039
0.673 0.026
0.910 I 0.011
EduLLM with SheafHyperGNN
0.684 + 0.023
0.711 0.030
0.732 - 0.022
0.687 = 0.025
0.914 = 0.013
EduLLM with
0.712 0.016
0.829 0.006
0.809 0.010
0.753 0.011
0.945 0.005
HNNs are well-suited for various real-world applications, including recommendation systems, social network anal- ysis, and bioinformatics. However, despite their promise, existing representative HNNS, such as HGNN (Feng et al., 2019). HyperGCN (Yadati et al., 2019), HNHN (Dong et al., 2020), HCHA (Bai et al., 2021), HCON (Wu al., 2023), UniGNN (Huang & Yang, 2021), AllDeepSets and AllSet- Transformer (Chien et al., 2022), ED-HNN (Wang et al., 2023), SheafHyperGCN and SheafHyperGNN (Duta et al., 2023), are primarily limited to undirected and unsigned hypergraph scenarios. Efforts to design HNNs for signed hypergraphs remain in the early stages. Meanwhile, the rapid development of LLMs has opened new opportunities to combine hypergraph learning with LLMs, particularly in applications where learning from raw data is challeng- ing due to the multimodal nature of the information and the complex, non-Euclidean relationships inherent in the data. LLMs are well-suited to handle the multimodal as- pect, while hypergraphs offer an effective framework for modeling non-Euclidean relationships. To the best of our knowledge, our work is the first to combine LLMs and signed hypergraph learning for student performance pre- diction in a real-world educational context. An extended version of the related works is provided in the Appendix A.
0.84
0.83
0.82
0.81
0.03
0.05
0.08
P
Figure 3. Performance demonstration for robustness analysis on the Sydney23 dataset.
curacy even when faced with perturbations in the semantic representations. This robustness underscores the potential of M to generalize across different LLM backbones and adapt to varying sources of semantic information. However, as a limitation, this empirical study may not fully reflect the role of LL.M-induced semantic representations in EduLLM.
4.8. Further Study on the Role of FraS-HNN
In order verify the advantages of FraS-HNN that ben- efit EduLLM framework, we have included comparisons with several HNN baselines, including HGNN (Feng et al., 2019), HyperGCN (Yadati et al., 2019), AllDeepSets (Chien et al., 2022), AlSSecTransformer (Chien et al., 2022), ED- HNN (Wang et al., 2023), and SheafHyperGNN (Duta et al., 2023), by replacing the FraS-HNN backbone of EduL.MM with each of these HNN modules. As shown in Table 4, EduLLM (with FraS-HNN consistently outperforms the variants equipped with each HNN module across all datasets, demonstrating the effectiveness of signed high-order interactions and its potential to benefit future research in (signed) hypergraph learning.
6. Conclusion
This paper proposes EduLLM, a framework that integrates LLMs with framelet-based signed hypergraph neural net- works (FraS-HNN) for student performance prediction. As novel spectral- based model for signed hypergraph learning, FraS-HNN leverages collaborative low-pass and high-pass filters to enable multi-frequency feature learning, with rig- orous theoretical guarantees for its multi-frequency analysis properties in both forward and inverse transforms. This design effectively captures both shared patterns among stu- dents and individual distinctions in their learning behaviors. By integrating LLM-induced semantic embeddings with this advanced hypergraph representation, EduLLM achieves superior performance across multiple datasets, addressing the structural and semantic complexities of personalized learning. For future work, we plan to develop more ad- vanced signed hypergraph neural networks and explore their applications in various domains.
in modeling
5. Related Works
Hypergraph neural networks and hypergraph representa- tion learning have emerged as powerful tools for capturing higher-order correlations in complex data, thereby extend- ing the capabilities of traditional graph neural networks (Antelmi et al., 2023; Kim et al., 2024). By modeling relationships among multiple entities through hyperedges,

---

EduLLM
Acknowledgments
Dai, Q and Gao, Y. Hypergraph Computation. Springer Nature, 2023.
This work was supported part by the "Pioneer" and "Lead- ing Goose" R&D Program of Zhejiang (No. 2024C03262), and by the National Natural Science Foundation of China (Nos. U21A20473, 62172370). L.. Bai acknowledges sup- port from the National Natural Science Foundation of China (No. T2122020). F. Cao acknowledges support from the Na- tional Natural Science Foundation of China (Nos. 62176244, 62032022). K. Lu acknowledges support from the National Natural Science Foundation of China (Nos. U23A20388, 62320106007).
Daud, A., Aljohani, N. R., Abbasi, R. A., Lytras, M. D., Abbas, F., and Alowibdi, J. S. Predicting student perfor- mance using advanced learning analytics. In Proceedings of the 26th International Conference on World Wide Web Companion, pp. 415-421, 2017.
Derr, T., Ma, Y., and Tang, J. Signed graph convolutional networks. In Proceedings of the 2018 IEEE International Conference on Data Mining, pp. 929 929-934, 2018.
Dong, Y., Sawin, W., and Bengio, Y. Hnhn: hypergraph networks with hyperedge neurons. In ICML Graph Rep- resentation Learning and Beyond Workshop, 2020.
Impact Statement
This paper aims to advance the field of Graph Machine Learning, focusing on key topics such as hypergraph//graph neural networks, hypergraph/graph representation learning, spectral- based hypergraph//grahh convolution, and their ad- vanced applications in intelligent education. There may be certain societal consequences of our work, none of which we feel must be specifically highlighted here.
Duta, I., Cassara, G., Silvestri, F., and Liò, P. Sheaf hyper- graph networks. In NeurIPS, pp. 12087-12099, 2023.
Dziuban, C., Moskal, P., Thompson, J., Kramer, L., De- Cantis, G., and Hermsdorfer, A. Student satisfaction with online learning: Is it psychological contract?. Online Learning,, 19(2):122-137) 137, 2015.
References
Feng, Y., You, H., Zhang, Z., Ji, R., and Gao, Y. Hypergraph neural networks. In AAAI, pp. 3558-35883303 1158-3565,222 2019.
AbdeIrahman, G., Wang, Q. and Nunes, B. Knowledge tracing: A survey, ACM Computing Surveys, 55(11):1-37, 2023.
Huang, J. and Yang, J. UniGNN: A unified framework for graph and hypergraph neural networks. IJCAI, pp. 2563-2569, 2021.
Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08874. 2023.
Huang, J., Shen, H., Cao, Q., Tao, S.,, and Cheng, X. Signed bipartite graph neural networks. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pp. 740-749, 2021.
Antelmi, A., Cordasco, G., Polato, M., Scarano, V. Spag- nuolo, C., and Yang, D. A survey on hypergraph repre- sentation learning. ACM Computing Surveys, 56(1) (1:1:1138,,, 2023.
Jung, J., Yoo, J., and Kang, U. Signed graph diffusion network, arXiv preprint UrXiv:20/2.141990 2020.
Bai, S., Zhang, F., and Torr, P. H. Hypergraph convolu- tion and hypergraph attention. Pattern Recognition, 110: 107637, 2021.
Kasneci, E., SeBler, K., Küchemann, S., Bannert, M.. Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. Chatgpt for good? opportunities and challenges of large language models for education. Learning and Individual Differences, 103: 102274, 2023.
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan. A., Shyam, P., Sastry, G., Askell, A., et al. Language models are few-shot learners. In Proceedings of the 34th International Conference on Neural Information Processing Systems, pp. 1877-1901, 2020.
Kim, J., Park, H., Lee, J.-E., and Kang, U. SIDE: Repre- sentation learning in signed directed networks. In Pro- ceedings of the 2018 World Wide Web Conference, pp. 509-518, 2018
Chen, M., Wei. Z., Huang, Z., Ding, B., and Li, Y. Simple and deep graph convolutional networks. In ICML, pp. 1725-1733 2020.
Kim, S., Lee, S. Y., Gao, Y., Antelmi, A., Polato, M., and Shin, K. A survey on hypergraph neural networks: An in-depth and step-by-step guide. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 6534 2024.
Chien, E., Pan, C., Peng, J., and Milenkovic, O. You are allset: multiset function framework for hypergraph neural networks. ICLR, 2022.
10

---

EduLLM
Kipf, T. N. and Welling, M. Semi-supervised classification with graph convolutional networks. In Proceedings of the International Conference on Learning Representations, 2017.
Shu, L.,, Du, E., Chang, Y., Chen, C., Zheng, Z., Xing, X., and Shen, S. SGCL: Contrastive representation learn ing for signed graphs. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pp. 1671-1680, 2021.
Li, H., Wei, H., Wang. Y., Song, Y., and Qu, H. Peer- inspired student performance prediction in interactive online question pools with graph neural network. In Proceedings of the 29th ACM International Conference on Information& & Knowledge Management, pp. 2589- 2596, 2020.
Susnjak, T. and McIntosh, T. R. Chatgpt: The end of online exam integrity? Education Sciences, 14(6):656, 2024.
Tan, K., Pang, T., Fan, C., and Yu, S. Towards apply- ing powerful large ai models in classroom teaching: Op- portunities, challenges and prospects. arXiv preprint Stti:2305503333. 2023.
Li, M., Li, Z., Huang, C., Jiang, Y., and Wu, X. Edu- graph: Learning path-based hypergraph neural networks for mooe course recommendation. IEEE Transactions Big Data, (06):706-719,2022
Thaker, K., Carvalho, P., and Koedinger, K. Comprehension Factor Analysis: Modeling student's reading behaviour: Accounting for reading practice in predicting students' learning in MOOCs. In Proceedings of the 9th Interna- tional Conference on Learning Analytics & Knowledge, pp. 111-115, 2019.
Li, M., Fang, Y., Wang, Y., Feng. H., Gu, Y., Bai, L., and Liò, P. Deep hypergraph neural networks with tight framelets. In AAAI, pp. 18385-18392, 2025a.
Veličković, P., Cucurull, G., Casanova, A., Romero, A.. Lio, P., and Bengio, Y. Graph attention networks. In Proceedings of the International Conference Learning Representations, 2018.
Li, M., Gu, Y., Wang, Y., Fang. Y.. Bai, L., Zhuang. X., and Lio, P. When hypergraph meets heterophily: New benchmark datasets and baseline. In AAAI, pp. 18377- 18384, 2025b.
Wang. F., Gao, W. Liu, Li, J., Zhao, G., Zhang, Z., Huang. Z., Zhu, M., Wang, S., Tong, W., et al. A survey of models for cognitive diagnosis: New developments and future directions. arXiv preprint 2407.05550 2024a.
Lin, Y. Chen, H., Xia, W., Lin, F., Wang, Z., and Liu, Y. A comprehensive survey on deep learning techniques ed- ucational data mining. arXiv preprint vrXiv:23090476.1 2023.
Malinka, K., Peresíni, M., Fire, A., Hujnák, O., and Janus, F. On the educational impact of chatgpt: Is artificial intelli- gence ready to obtain a university degree? In Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education, pp. 47-53, 2023.
Wang, P., Yang, S., Liu, Y., Wang, Z., and Li, P. Equivariant hypergraph diffusion neural operators. In ICLR, 2023.
Wang, S., Ni, L., Zhang, Z., Li, X., Zheng, X., and Liu, J. Multimodal prediction of student performance: A fu- sion of signed graph neural networks and large language models. Pattern Recognition Letters., 181:1 88, 2024b.
Mayer, R. E. Thirty years of research on online learning. Applied Cognitive Psychology. 33(2):152-155) 2019.
Wang, S., Xu, T., Li, H.,, Zhang, C., Liang, J., Tang. J., Yu, P. S., and Wen, Q. Large language models for education: A survey and outlook. arXiv preprint arXiv:2403.18105 2024c.
Nakagawa, H., Iwasawa, Y., and Matsuo, Y. Graph- based knowledge tracing: modeling student proficiency using graph neural network. In Proceedings of the IEEE/WIC/ACM International Conference on Web In- telligence, pp. 156-163, 2019.
Wei, H., Li, H., Xia, M., Wang, Y., and Qu, H. Predicting student performance interactive online question pools using mouse interaction features. In Proceedings of the 10th International Conference on Learning Analytics & Knowledge, pp. 645-654, 2020.
Pennington, J., Socher, R., and Manning, C. GloVe: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Lan- guage Processing, pp. 1532-1543, 2014.
Wu, H., Yan, Y., and Ng, M. K.PP. Hypergraph collabora- tive network on vertices and hyperedges. IEEE Transac- tions on Pattern Analysis and Machine Intelligence, 45 (3):3245-333 3258, 2023.
Piech, C., Bassen, J., Huang, J., Ganguli, S., Sahami, M., Guibas, L. J., and Sohk-Dicksteen, J. Deep knowledge tracing. In Proceedings of the 28th International Con- ference on Neural Information Processing Systems, pp- 505-513, 2015.
Yadati, N., Nimishakavi. M., Yadav, P., Nitin, V., Louis, A., and Talukdar, P. HyperGCN: new method for

---

EduLLM
training graph convolutional networks on hypergraphs. In NeurlPS, pp. 151 1511-1522, 2019.
Zhang, M. and Chen, Y. Link prediction based on graph neural networks, Proceedings of the 32nd International Conference on Neural Information Processing Systems, pp. 5171-5181, 2018.
Zhang, Z., Liu, J., Zheng, X., Wang, Y., Han, P., Wang, Y., Zha,, K., and Zhang, RSGNN: A model-agnostic approach for enhancing the robustness of signed graph neural networks. In Proceedings of the ACM Web Confer- ence 2023, pp. 60-70, 2023.
Zhou, J., Cui, G., Hu, S., Zhang, Z., Yang, C., Liu, Z., Wang, L., Li, C., and Sun, M. Graph neural networks: A review of methods and applications. Al Open., 1:57-81, 2020.
12

---

E
A. Extended Related Works
A.1. Student Performance Prediction
Student performance prediction involves leveraging historical data and advanced analytical methods to forecast a student's s future performance in academic tasks such as exams, assignments, and course completion. The primary objective is to predict students' performance based on their prior learning data, enabling educational institutions and instructors tailor support and guidance to individual learning needs (Lin et al., 2023).
Early models for student performance prediction relied heavily on traditional machine learning methods, such as logistic regression, decision trees, and support vector machines. However, these approaches faced significant limitations, particularly in their reliance on manually extracted features and their inability to capture the complexity of student behavior. With the rise of deep learning, classic models such as Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), Long Short-Term Memory (LSTM) networks, and Deep Neural Networks (DNN) have gained prominence due to their ability to extract more complex features (Lin al., 2023). Specifically, efforts to enhance personalized learning have resulted in two main categories of prediction models: static models and sequential models (Li et al., 2020; Thaker et al., 2019). Static models, such as those studied by Wei et al. (2020) and Daud et al. (2017), rely solely on historical data for predicting future performance. While effective, these models fail to account for temporal dependencies and sequential relationships within the learning process. On the other hand, sequential models like knowledge tracing and its variants (Piech et al., 2015; Nakagawa et al., 2019) model the evolving knowledge of students, allowing for dynamic updates to their knowledge base and providing more accurate predictions in certain settings.
However, despite these advances, many existing methods still overlook the complex, non-linear interactions among students and learning content, as well as the rich semantic information embedded in educational resources. For instance, many models focus on analyzing temporal dependencies or basic student-reooree interactions without considering how the deeper relationships between different learning materials influence student performance. In contrast, our proposed EduLLM framework not only leverages the content of the questions and learner- quenttro interaction data, but also integrates semantic embeddings from LLMs, capturing the rich semantic context of educational resources. By using both structural and semantic features, EduLLM overcomes the limitations of traditional methods, improving prediction accuracy and providing more nuanced insights into student learning behaviors.
A.2. Signed Graph Neural Networks
Student performance prediction can be framed as a link sign prediction task within a signed bipartite graph, where positive edges indicate correct answers and negative edges indicate incorrect ones (Derr et al., 2018; Zhang al., 2023). Early methods for signed graph embedding, such as SIDE (Kim et al., 2018) and SGDN (Jung et al., 2020), used random walks to generate node embeddings. While these methods successfully captured structural information, they struggled to model the nuanced relationships between the signs. Other techniques, such as signed Laplacian embedding (Derr et al., 2018) and matrix factorization (Zhang et al., 2023), utilized mathematical tools to extract structural information from signed graphs but were limited in handling the inherent balance and contrast of signed edges.
Recent advancements have incorporated neural network- betsork-based approaches to address these limitations. For instance, SGCN (Derr et al., 2018) extends GCNs (Kipf & Welling, 2017) by incorporating balance theory to optimize sign prediction. SiNE (Kim et al., 2018) combines triangular structures with balance theory to enhance signed graph representation learning. While these models made strides in capturing local structural information, they still fall short of exploiting the global features of signed graphs. More recent approaches, inspired by contrastive learning, such as Signed Graph Contrastive Learning (SGCL) (Shu et al., 2021), SBGNN (Huang et al., 2021), 1), and Signed Bipartite Graph Contrastive Learning (SBCI.) (Wang et al., 2024b), have focused on learning from signed graphs and signed bipartite graphs. However, these models primarily focus on extracting local structural information, addressing only pairwise relationships between nodes. They do not consider higher-order relationships, such co-occurrences or interactions among multiple nodes, which are critical in more complex scenarios. In contrast, our proposed EduLLM framework utilizes signed hypergraph models, which extend traditional graph structures by considering higher-order interactions through hyperedges. This allows EduLLM to capture richer, more complex relationships, addressing both local and global patterns within the data.
13

---

EduLLM
A.3. LLMs for Education
LLMs have revolutionized several areas of the education sector, particularly in personalized learning, automated assessment, and educational assistance (Kasneci et al., 2023; Wang al., 2024c). For example, Brown et al. (2020) demonstrate the potential of GPT-3 in few-shot learning, highlighting its ability to personalize learning by analyzing students' behaviors and language expressions. These models can generate individualized learning paths and resources, improving learning outcomes. However, practical applications face certain challenges, particularly the need for large amounts of labeled data for fine-tuning, which is often scarce and imbalanced in educational settings, limiting the models' generalization and deployment. The GPT-4 technical report (Achiam et al., 2023) demonstrates that LLMs can perform at a student level on standardized tests in various mathematics subjects, including physics and computer science, for both multiple-choie and open-ended questions. Furthermore, studies have shown that L.MSs can be effectively used as writing or reading assistants in educational contexts. For instance, recent study (Susnjak & McIntosh, 2024) finds that ChatGPT is capable of generating coherent and logically consistent responses across a range of disciplines, providing both depth and breadth in its answers. Another quantitative analysis (Malinka et al., 2023) indicate that students who use ChatGPT, refining or incorporating the model's outputs into their own answers, outperform their peers in certain computer security courses. Additionally, recent reviews (Tan et al., 2023; Wang et al., 2024c) have discussed potential applications LLMs educational settings, such as enhancing seacher-ttudent collaboration, enabling personalized learning, and automating assessment processes.
While there certain concerns associated with the use of LLMs in practical applications (Kasneci et al., 2023), such as issues related to plagiarism, potential biases in Al-generated content, overreliance on LLMs, and inequitable access for non-English speakers, LLMs have nonetheless been recogmized as powerful tool for extracting rich semantic information from multimodal educational resources. This implies that by leveraging the comprehensive semantic insights provided by LLMs, in conjunction with advanced learning models, researchers can develop more effective models capable of addressing complex tasks within specific domains, such as education. From this perspective, to the best of our knowledge, our proposed EduLLM framework is the first to combine LLMs and signed hypergraph learning for student performance prediction.
B. Theoretical Properties of FraS-HNN
Preliminaries and Key Notations. Consider signed hypergraph G. = (V, E) with N nodes and signed hypergraph Laplacian L. Let M = [mi, m2,... mv] denote the matrix of eigenvectors of L, and A = diag(0), , 6N) be the diagonal matrix of the corresponding eigenvalues. Framelets over the signed hypergraph are generated by a set of scaling functions S ( {60)) } C Li (R) associated with filter bank v) = (b10); ((2), b(k),, which satisfy the relations for any & E I:
((2 = ((0) (E))(E), (2E) = (((r) (E)
(B-1)
These functions are defined as follows (for clarity, Equations (3) and (4) from Section 3.2 of the main manuscript are restated here for reference):
0p
L j.r
(u)
my (u)
(B-2)
2
p=1
N
v
w =
bbtr
mp (v) (u).
(B-3)
20
where mv) represents the v-th component of the eigenvector mp-
For integers J, Jl such that J > J1, we define a tight framelet system on signed hypergraphs (denoted as TiFraS(S, starting from scale J1, as a non-homogeneous, stationary affine system:
Ti f | |OE u (1 /oEV,j= J, Jy...," 1
(B-4)
Theorem B1 (Properties of Tight Framelets on Hypergraphs). Let J2 1 be an integer, and consider the hypergraph framelet system TiFraSy, (S, my. 92) defined in (B-4), with hypergraph framelets Dy.v and y,,.. The following statements are equivalent:
14

---

EduLLM
Table B-I. Frequently used notations.
Notation
Description
G. (V,
Signed hypergraph with node set V and signed hyperedge set
M, A
Eigenvector matrix M and eigenvalue diagonal matrix A of the signed hypergraph Laplacian.
(()
Spectral low-pass filter (scaling function).
())
Spectral high-pass filter for the r-th wavelet function.
To
Framelet decomposition operator for low-pass components scale J.
T
Framelet decomposition operator for the r-th high-pass component at scale j.
F(F)
Framelet-based signed hypergraph convolution operator.
X
Signal or feature matrix on the signed hypergraph.
Feature matrix at layer and the initial feature matrix, respectively.
Trainable diagonal scaling matrix at layer L.
ag
Learnable parameter balancing contributions from initial features and the transformed features.
B
Learnable parameter balancing the contributions of identity mapping.
(()
Non-linear activation function (c.g.. RcLU).
Tn(x)
n-th Chebyshev polynomial used for spectral filter approximation.
Cn
Coefficients for approximating the low-pass filter (()) using Chebyshev polynomials.
((r)
Coefficients for approximating the high-pass filter (.())() using Chebyshev polynomials.
6p
Eigenvalue associated with the p-th node in the signed hypergraph Laplacian.
C
Signed hypergraph Laplacian operator.
Wrj. Wo.J
Framelet decomposition matrices for high-pass (r, and low-pass (0, J) components.
J
Number of decomposition scales (levels).
k
Number of high-pass filters.
(i) For each J1 l.....JJJ the framelet system on hypergraphs, TiFraSy, (S, Gz), a tight frame for (2.)... That is, vf l2(G)),
(5 f.
|1 = Σ (S, Jr
+ Σ
(B-5)
NEV
j=Ji re
iii For all f E (2.)) and for j -.....,J J - 1, the following identities hold:
k
f = Σ DJ,) L.
1,0) J,
(B-6)
NEV
r=1
Σ (f. 1 1141. = Σ
(f.
f, (1)) 3.0 w/)
(B-7)
NEV
PEV
(ii) For all f E (2.)) and for = -.....; J 1, the following identities hold:
1 = Σ
KR. BJ,) +
(I. 1..
(B-8)
NEV
r=1 VEV
Σ
Σ
(f. 1 f..))
(5, w/( p.
(B-9)
NEV
PEV
r=1 NEV
15

---

E
(iv) The scaling functions S = (p, {(b(r)
} y=
C L1 (R) satisfy
0q
2
6g
2
6
+
(b(r)
bq = -.......A
(B-10)
2
2
1=1
Ag
6g
0g
N, J-1 -
Φ
+
(t()
(B-11)
2
2
2
j =
The identities (B-10) hold and the filters in the filter bank v satisfy
5g
0g
2
b(0)
+
b(r)
1
Vq E 0 j=
== =......JJ
(B-12)
2
2
with
0 = q (l.....NN] : x//) + o).
Proofs. (ii). Let r(llow) = span (i) : v V} and r rehhigh.))
= span jo : v}. Define projections
pr(low)
pr (high.r) (with r r= 1,...,k) by
pr (low) (f) = Σ
(f. Dyo) 2.00
Pr (bight)) (f) = E
v(r) ji0t
fe (2G.).
(B-13)
wEV.
NEV
Since TiFraS (8, 17, Go) is tight frame signed hypergraphs for (2(),) (J1 I,...,)), obtain by polarization identity,
f = Ji ( Σ
pr
(high.r) ( = Pr(low) (f) + J1+1
Σ
Pl Pr (high.r) (f)
(B-14)
j=J1 r=1
for all f E (G.)) and for all J1 = 1, I,...,J.. J. Thus, for J1 = 1, 1....,J-J -
Pr = pr.(oo)) (f) + Σ
(hight)))) (f), J1
(B-15)
which is (B-7). Moreover, when J1 = J, (B-14) gives (B-6). Consequently, (i)= (ii). Conversely, recursively using (B-15) gives
k
pr(low) m+1 (f) = (l(l)o)) J.
(f) +
Σ Σ
Pr
(high.)) (f)
(B-16)
J
r=Jr==
for all Ji S m S Taking m = 1 together with (B-6), we deduce (B-14), which is equivalent to (B-5). Thus, (ii) (i).
(ii)< (iii). The equivalence between (ii) and (iii) simply follows from the polarization identity.
(ii) 4 (iv). By the orthonormality of mp.
8g Jg (z)),, (1, (.
0g 27
( (.0)
(r)
I mm(())
where f, = (f, m,)) is the Fourier coefficient of with respect to my. This together with (B-13), (B-2) and (B-3) gives, for j 2 1 and r = I,...,A. ks the Fourier coefficients for the projections Pr (low))
(f) and Pr (hight)))) (f):
8
2 fa.
prrshinh.o (P)shithh)))))
Pr
(r)
6p
2 Fg.
1g= N,
(B-17)
16

---

EduLLM
which implies that (B-6) and (B-7) are equivalent to (B-10) and (B-11) respectively. Thus, (ii)<(iiv)
(vv). Based on the relations (B-1) that (((E) = (E)) and eb(r) = B(r) (E)((() for any & E R, it can be
deduced that for q = 1,..., N and j 2 1,
P
(b(r)
5g
(1
0g
Σ
h
bg
)r){
0
6g
)r
2
2
2
2i+1
2i+1
==1
This shows that (B-11) is equivalent to (B-12). Therefore, (iv)
C. More Details on Datasets
In our experimental studies, utilize five real-world datasets derived from courses at three universities: the Biology and Law courses from the University of Auckland, the Cardif220102 Medical School course from Cardiff University, and the biochemistry courses Sydney/935/ and Sydney23146 from the University of Sydney. These datasets serve as the foundation for the student performance prediction task. To align with the problem formulation, we construct signed hypergraphs based on student-question interaction data. This appendix provides further details on the process of defining signed (bipartite) hypergraphs for each dataset, which serves as the input signal for the problem-solving framework (as illustrated Figure I). Additionally, to demonstrate the advantages of using signed (bipartite) hypergraphs over signed (bipartite) graphs, we also present the construction of signed graphs, which allows for comparative experimental studies and provides further insights into the merits of EduLLM (as detailed in the following Appendix E.3).
Formulating Original 'Students-MCQs' Sources as Signed Hypergraph. To capture the complex interactions between students and questions, we constructed signed hypergraphs using students' answer records. Signed hypergraphs are particularly effective in modeling higher-order relationships, as they allow us to categorize student responses into "correct" or "incorrect" answers, with these responses forming hyperedges in the data. Specifically:
For each question node fj. all students who correctly answered the question are connected to the question node via positive hyperedge, which is labeled as "+1".
In contrast, students who answered the question incorrectly are connected to the question node through a negative hyperedge, labeled as "-1".
This structure enables the representation of each dataset as signed hypergraph, providing a more expressive framework for modeling student-question interactions. By capturing higher-order relationships, signed hypergraphs offer a richer representation compared to traditional signed graphs, overcoming their limitations. For better clarity, toy example for signed hypergrpah construction is provided in the following Appendix D.
Formulating Original 'Students-MCQs' Sources as Signed Graph (as a Specific Comparison Case). In comparison, signed graphs represent a simpler structure by focusing on individual interactions between students and questions. For each dataset, we constructed a signed bipartite graph using the answer records:
If student u, answered question gj correctly, an edge with a positive sign ("+1") is established between the two nodes.
If the student's 's answer was incorrect, an edge with negative sign ("-1"") is created between the student node and the question node.
The bipartite structure of signed graphs more straightforward and is particularly suited for analyzing one-to-one relation- ships between students and questions. From our statistical analysis, we observe that the overall correctness rates for most courses ranged between 60% and 70%, except for the Law course, which exhibited impressive correctness rate of 93%.
D. A Additional Details on EduLLM's Key Components
As outlined in Section 3.1, prior to the implementation of -NNN the core module for signed hypergraph learning, it is crucial to first establish the modules for -baeed semantic extraction and signed hypergraph construction. This part provides further details on these two essential components.
17

---

EduLLM
MCQ
Key-Value Pairs
Question stem: Which planet has the shortest rotation perfod in Solar System?
Which planet shortest rotation period Solar System?
STEM
[A]: Earth
Jupiter [D]: Vens
Transform
Options
[C]: Mars
Jupiter
Vens
Question
Explan
Jupter has the shortest rotation in the Solar System,taking apporoximatoly 10 hours to complete rotation.
Explanation: Jupter shortest
-ation
rotation in the Solar System,taking apporoximately hours complete rotation.
Answer
Optin B
Flatten
MCQ "sentence"
STEM
Which..
Options
A: Earth
D: Vens
Explanation
Jupter has
Answer
Option B
Figure D-I. Flatten MCQ component key-value pairs into sequence form.
D.I. LLM-based Semantic Extraction
We recognize the significant potential of integrating semantic context into multiple-choice questions (MCQs), as it can reveal latent knowledge that is crucial for prediction tasks. To enhance the representation of MCQs, we employ LLMs to integrate semantic information, which enables the fusion of textual features with signed hypergraph data. We also note that the procedure for LLM- semantic extraction of questions is adapted from the related work by (Wang et al., 2024b). We revisit the general process below not only for clarity but also to help readers better understand how to apply different LLM tools or platforms similar contexts in their future work.
MCQs are cornerstone of educational assessments and widely used in both traditional education and crowdsourced learning platforms. However, conventional methods for representing MCQs often fail to leverage the full potential of their semantic content, thereby limiting their utility in predictive modeling tasks. By incorporating semantic context, the representation of MCQs can be significantly enriched, leading to improved performance in downstream prediction tasks. Despite the broad application of MCQs various educational contexts, the integration of textual and graph data to enhance MCQ representations has been underexplored the literature, underscoring the need for novel approaches to address this gap.
In our EduLLM framework, we use LLMs to enrich MCQ representations with semantic context. The process begins by transforming the various components of MCQ into flattened sentence format, shown in Figure D-1. Each question is reformulated into key-value pairs extracted from the question stem, answer options, and explanations. Using the LLM, we extract a set of knowledge point terms from each MCQ, denoted as k; and assign a corresponding weight to each term, denoted as h4. Next, we compute word vectors for each term in the knowledge point set using GloVe embeddings (Pennington et al., 2014). The word vectors are then averaged to obtain a representation for each knowledge point, denoted as The semantic representation of the entire MCQ is constructed by taking a weighted average of the knowledge points' vectors. The resulting semantic embedding for the question is denoted as Wj. which is computed as: kp; (E fi,))/m /m Wy EΣ= hxpp:)) (E", ha). Here, Si., denotes represents the embedding of the 1-th term in the i-th knowledge point. These semantic embeddings are then combined with structural information from the graph data, resulting in a richer representation for each node within the graph.
For the semantic extraction process, we utilize ChatGPT as the LLM, which allows us to extract n pairs of knowledge points (k; and h₂) single API request. Recall the schematic of EduLLM framwork shown in Figure 1, we design our prompts efficiently extract relevant keywords and their associated weights, with results returned in a structured JSON format. This setup ensures that
effectively handle large volumes of requests while maintaining the quality of the responses. The L.M outputs contain essential elements for understanding the MCQs, enabling it to identify specific knowledge areas related to the questions and generate meaningful semantic embeddings. A detailed description of the LLM API requests we have designed is provided in the following illustration.
18

---

EduLLM
Task Objective: Your task is to extract key knowledge points from an MCQ created by a first year student who studics Law in college. The question consists of stem, to five options, an answer, and an explanation provided by the student author.
Special Requirements: Please provide your response JSON format with the following keys and format:
"Keywords": ("keyword"; "keyword name", "percentage": OFFF
1. Each "keyword_name" should be word or short term with less than five words.
2. The percentages of all keywords should add up
3. Only include the top five keywords to avoid excessive keyword extraction.
In Context Learning: Here's an example response format:
["Keywords"; ["keyword"; "Legal person", "percentage"; 0.8),
["keyword"; "Intellectual property rights", "percentage": 0.2 }}
Context: Here's the MCQ:
Question stem: Which of the following situations is most likely to be considered a breach of contract?
[A]: A party realizes after signing the contract that they are financial difficulty fails to make timely payments.
[B]: The contract specifies the delivery of particular product, but the delivering party provides different substitute product.
[C]: party fails to fulfill their obligations on time due to force majeure (e.g.. a natural disaster).
[D]: Both parties mutually agree to terminate the contract during the performance phase.
Question answer:
Option B best fits the definition of breach of contract. A breach occurs when party fails to perform its obligations according the terms of the contract, performs manner inconsistent with the contract's requirements. In this case. providing substitute product instead of one specifically agreed upon constitutes violation Option A: While failing to make timely payment might lead to breach, "financial difficulty" may be subject to further analysis and is not automatically considered breach in every case. Option C: A failure to perform due to force majeure is generally regarded contract legitimate contractual decision and does not constitute breach.
Explanation:
contract terms.
excusable pon-performance, not breach of contract. Option D: Mutually agreed termination of the
LLM Response:
"Keywords";
("keyword"; "breach of contract", "percentage"; 0. 0.6),
("keyword": "substitute product", "percentage": 0.2),
["keyword": "contract terms", "percentage": 0.1),
("keyword": "ferce majeure", "percentage"; 0.05),
""kewword"; "mutually agreed termination", "percentage": 0.05)
D.2. Toy Example for Signed Hypergraph Construction
Before proceeding to the signed hypergraph learning module, FraS-HNN, we first need to explicitly construct the structure of the signed hypergraph, as outlined in Appendix C. This essential for capturing the higher-order relationships between students and MCQs. Specifically, construct a signed hypergraph hyperedge E, along with the incidence matrix H, which reflects the underlying structure the hypergraph:
(V,E), need define the vertex V and the
Vertex Set V: This set includes the set of students 24 and the set of questions Q. Specifically, / {Student 1, Student 2....)
represents the set of students, and Q = {Question 1, Question 2....) represents the set of questions. There is no overlap between students and questions, i.e., 24 I = 0.
Hyperedge Set E: The hyperedge set used to represent interactions between students and questions. Based on the answers provided by students, interactions are classified into positive and negative hyperedges. Specifically, positive hyperedges
19

---

EduLLM
connect all students who answered question correctly the corresponding question, while negative hyperedges connect all students who answered incorrectly the corresponding question. The positive hyperedge is denoted as E+, and the negative hyperedge set as = Each positive hyperedge et E E connects all students who answered correctly, and each negative hyperedge e E E connects all students who answered incorrectly.
Construction of the Incidence Matrix. To facilitate further processing and computation, we use an incidence matrix I {0, 1, -1-1ax to represent interactions between students and questions. Here, N is the number vertices (the sum of the number of students and the number of questions), and M the number of hyperedges (the total number of positive and negative hyperedges). The matrix construction as follows:
H(v,e*)= = 1, e belongs to a positive hyperedge et:
H( = -,, if e belongs to a negative hyperedge e
((u,e) = otherwise
For example, consider scenario with students and 2 questions. The interaction data between students and questions is presented in Table D-1. In this table, a checkmark (V) represents a correct answer, and a cross (x) represents an incorrect answer.
Table D-1. simple example for three students and two questions.
Student ID Question Question
student
student 2
student
Based on the above interaction data, the incidence matrix is constructed follows:
Students
MCQs
1
0
1 0 0 0
0
-1
1
H =
1
0
1
-1
0
0
0
-1
1
+
3
Figure D-2. Incidence matrix and illustration of the signed hypergraph.
Clearly, this signed hypergraph consists of five nodes and four hyperedges: ef (Student 1, Student 3, Question 1), e (Student 2, Student 3, Question 2), ej (Student 1, Question 2), and ej (Student 2, Question 1).
E. Additional Experimental Details and Further Results
E.1. Details for Baseline Models
In our experiments, we use several baseline models for performance comparison, including Random Embedding, Graph Convolutional Network (GCN) (Kipf & Welling, 2017), Graph Attention Network (GAT) (Veličković et al., 2018), Signed Graph Convolutional Network (SGCN) (Derr et al., 2018), Signed Bipartite Graph Neural Network (SBGNN) (Huang al., 2021), Signed Bipartite Graph Contrastive Learning (SBCL) (Wang et al., 2024b), and its variant version LLM-SBCL (Wang et al., 2024b). Below is brief description of each model:
Random Embedding: This method generates random d4-dimensional embeddings for both students (Zu, e R44 and questions (Eny E 1 These embeddings are concatenated, and a Logistic Regressor (LRR is trained on the
20

---

EduLLM
concatenated embeddings predict the sign in the test data. Since random embeddings do not incorporate any graph-related information, this model serves as a lower-bound benchmark (Huang al., 2021).
GCN (Kipf& & Welling, 2017): It utilizes pesssge-passiigg based graph convolution to aggregate features from nodes and their neighbors, capturing local graph structure information. Its hierarchical design, which allows stacking multiple convolutional layers, enables the extraction of deeper graph features, making GCN suitable for node classification and graph representation learning tasks.
GAT (Veličković et al., 2018): It introduces self-attention mechanism that assigns different weights to each node, enabling the model to adapt diverse graph structures for more flexible feature aggregation. This dynamic weighting mechanism allows GAT to explicitly assess and quantify the importance of neighboring nodes, resulting in more precise representation learning.
SGCN (Derr al., 2018): It leverages balance theory to incorporate negative links during aggregation. addresses the unique challenges posed by negative links, which carry distinet semantic meanings and form complex relationships with positive links, enabling more accurate modeling of signed graphs
SBGNN (Huang et al., 2021): It employs advanced techniques for modeling signed bipartite graphs. We use the publicly available code with default settings and 300 training epochs for our implementation.
SBCL (Wang et al., 2024b): It employs a single-headed GAT with two convolutional layers and 64-dimensional random embeddings. We use publicly available code 2 with default settings and 300 training epochs for this model.
LLM-SBCL (Wang et al., 2024b): It is variant of SBCL that integrates semantic embeddings for question components, which are generated by a large language model. This addition enhances the model's ability to better capture the semantic context of the questions.
E.2. Experimental Setup and Hyperparameter Settings
All experiments are conducted using the Py Torch framework and executed on single NVIDIA RTX A6000 GPU to ensure computational efficiency and consistency across all runs. To facilitate the reproducibility of results, we provide the corresponding hyperparameters that yield optimal performance in these experiments. These hyperparameter settings are detailed in Table E-1, which allows for the accurate reproduction of the experimental setup and comparison of performance results.
E.3. Further Study: EduLLM vs. EduLLM-SG (EduLLM with Signed GNNs)
The previous performance comparison between EduLLM and other baseline models has already demonstrated the superiority of EduLLM. This advantage stems from the new problem formulation using signed hypergraphs and the incorporation of key techniques, including both LLM--based semantic extraction and signed hypergraph learning. To further explore the benefits of signed hypergraphs over signed graphs in solving this problem, a natural question arises: "Assuming the influence of LLM excluded, how does EduLLM perform when using signed graph learning while still employing framelet-based convolutions?" To answer this, we revisit the problem formulation by replacing the signed hypergraph with signed graph (as described in Appendix C). In this case, EduLLM is refined by constructing framelet-basedd convolutions on the signed graph, resulting in variant model, which we term EduLLM-SG, illustrated in Figure E-1.
Specifically, the signed (bipartite) graph consists of two types of nodes: students and questions. Interactions between students and questions are represented by edges, where the sign of the edge indicates whether the student answered the question correctly (positive) or incorrectly (negative). This graph is denoted as 9 = (14, V, E), where ' = {ur, U2, Weyr } represents the set of students, V (u1, ......, , } represents the set of questions. and there is no overlap between the two sets (i.e., U V Ø). The edge set c represents the pair-wise relationships between students and questions, divided into positive edges (E+) and negative edges (E-), with E E U E and E n E 0
For fair comparison, we apply the same experimental setups and hyperparameter settings, as detailed in Table F-1, to the experimental study on EduLLM-SG. The results for both EduLLM-SG and EduLLM are presented in Table E-2. The
21

---

EduLLM
Table F-1. Hyperparameter settings for the five datasets used in the experiments.
Dataset
Hyperparameter Setting
Learning rate: 5e-3
Layers: 36
Weight decay: le-4
Alpha: 0.5
Biology
Hidden Size: 64
Gamma: 0.5
Dropout ratio: 0.2
Lambda: 0.8
Level:
Seed: 2000
Learning rale: 5e-3
Layers:
Weight decay: le-4
Alpha: 0.5
Law
Hidden Size: 64
Gamma: 0.5
Dropout ratio: 0.2
Lambda: 0.8
Level: 2
Seed: 2000
Learning rate: 5e-2
Layers: 24
Weight decay: le-2
Alpha:
Cardiff'
Hidden Size: 64
Gamma: 0.1
Dropout ratio: 0.2
Lambda: 0.5
Level:
Seed: 2000
Learning rate: le-2
Layers: 28
Weight decay: 1e-3
Alpha: 0.2
Sydney19
Hidden Size:
Gamma: 0.2
Dropout ratio: 0.2
Lambda: 0.6
Level:
Seed: 2000
Learning rate: 5e-2
Layers: 16
Weight decay: le-4
Alpha: 0.1
Sydney23
Hidden Size: 64
Gamma: 0.1
Dropout ratio: 0.2
Lambda: 0.5
Level: 2
Seed: 2000
Table E-2. Comparison of EduLLM and EduLLM-SC Performance
EduLLM-SG
EduLLM
Biology
0.807 + 0.007
0.809 + 0.010
Law
0.941 +: 0.030
0.945 1 0.005
Cardiff
0.748 0.003
0.753 0.011
Sydney19 19
0.715 0.012
0.712 1 0.016
Sydney23
0.823 + 0.014
0.829 = 0.006
experimental results clearly show that EduLLM consistently outperforms EduLLM-SG, validating the effectiveness of using signed hypergraphs in this specific educational application. By capturing higher-order interactions, the signed hypergraph framework enables the model to comprehensively represent the complex relationships between students and questions, offering richer and more expressive model than the signed graph approach. This advantage is especially evident in datasets such as Biology and Law. However, for the Sydney19 19 dataset, EduLLM-SG achieves slightly better performance than EduLLM. We hypothesize that this small difference may be due to the limited number of answer records and high data sparsity in the dataset. In such sparse scenarios, the higher-order relationships captured by hypergraphs may not provide enough discriminative power and may even introduce redundant information, leading to reduced performance. contrast, the simplicity of signed graphs can offer more effectiveness these conditions.
Interestingly, despite not outperforming EduLLM, EduLLM-SC still surpasses all other baseline models recorded in Table 2 in the main manuscript, especially those that also use signed graphs (such as SGCN (Derr et al., 201 8), SBGNN (Huang et al., 2021), SBCL (Wang et al., 2024b), and LLM-SBCL (Wang et al., 2024b)). This further highlights the benefits of the FraS-GNN approach (with the advantages of low-pass and high-pass filters inherited from FraS -NNN, as theoretically studied in Appendix B) in signed graph learning. Overall, the experimental results presented in Table E-2, along with the previous findings, convincingly demonstrate the effectiveness and advantages of Edul_LM this task.
22

---

E
MCQs
Students Embeddings
MCQ₂
LLM Response
Embeddings
Bipartito
Signed
3
6
Ze
ws
Graph
Embedding
MCQQ
layer
Prediction
MLP
FraS-GNN
Loss
5
Figure E-1. 1. The framework of EdULLM-SG
E.4. Additional Results Parameter Sensitivity Analysis for EduLLM and EdULLM-SG
In line with the results presented in Figure of the main manuscript, provide additional empirical results parameter sensitivity analysis for EduLLM in Figure E-2 and for EduLLM-SG in Figure E-3, Overall, both models demonstrate low sensitivity to variations in the parameters ?, and À across most cases.
F. Further Highlights
We emphasize that although EduLLM its current form is specifically designed for student performance prediction, an important problem in educational data analytics and mining, the underlying signed hypergraph neural network (FraS-HNN) has broader applicability. It holds strong potential for tackling analogous tasks in other domains and for advancing the development of hypergraph-baseet machine learning models and techniques (Li et al., 2025a;b). This is particularly true when signed hypergraph learning can offer effective solutions to the problem at hand. Signed hypergraphs are particularly effective for problems involving multi-party interactions, where both positive and negative relationships need to be modeled, making them suitable for various real-world applications. Thus, the potential for applying FraS-HNN or its future variants goes beyond educational contexts, offering promising avenues for research and application across various interdisciplinary fields.
For future work, foresee the application of signed hypergraph learning (SHL) to variety of specific tasks across different fields, provided that the data sources and problem formulations align well with the SHI. paradigm. Furthermore, to advance research in SHL, the development of benchmark datasets of signed hypergraphs across various domains is highly expected.
23

---

E
0.06
0.60
0.66
0.63
0.65
0.63
ADC
40C
9.5
0.5
6.1
vaaluess
Vvaltaes
Avalues
(a) Sydney23
0.64
0.64
0.64
AUC
0.62
0.62
0.62
0.6
0.58
0.08
0.98
0.1
0.2
0.3
0.4
0.5
0.6
0.1
0.2
0.3
0.4
0.5
0.6
0.2
6.4
0.6
0.8
vivalies
Vallars
A valuus
(b) Sydney19 19
0.63
0.63
0.68
ADC
ABC
0.6
0.6
0.6
0.57
0.57
0.57
0.1
0.2
033
0.
0.6
0.1
0.2
0.4
0.5
0.6
0.2
0.4
0.6 Avalues
0.8
avalues
vvaluues (c) Cardiff
ATC
ACCC
0.5
0.57
0.57
0.57
0.5
066
0.8
avalues
(d) Biology
Figure F-2, Parameter sensitivity analysis for Mdul.LM across the Sydney23, Sydney 19, Cardiff, and Biology datasets.
24

---

EduLLM
0.68
0.68
0.65
Alc
ATCC
ATC
0.62
0.39
0.99
0.59
0.6
0.8
vvalless
(a) Law
0.7
0.7
0.7
0.5G
AUC
0.6
0.53
0.5
0.5
0.1
0.2
033
0.4
0.3
0.6
0.1
0.2
0.3
0.4
0.5
0.6
0.2
0.1
0.5
0.8
avallcs
Vallles
Avalues
(b) Sydney23
0.7
0.7
0.68
0.68
0.68
0.64
0.64
0.64
0.62
0862
0.62
0.5
volouess
V vallues
À voLuee
(c) Sydney 19
0.7
0.7
0.7
0.68
0.08
0.69
ADC
ATC
0.66
0.66
0.65
0.61
0.62
0.62
0.62
0.2
6.5
0.6
0.1
0.4
0.2
0.5
0.8
vaalacs
Avallus
(d) Cardiff'
0.66
0.66
0.65
A3C
0.60
0.62
0.62
0.6
0.6
0.6
0.2
0.3
0.5
0.4
0.5
0.2
valluss
Y vallucs
1val Ics
(e) Biology
Figure E-3, Parameter sensitivity analysis for EduLLM-SG across Law, Sydney23, Sydney19, Cardiff, and Biology datasets.
25


Reviews:
[{"review_id": "CKnvXYsJH0", "reviewer": "Reviewer_stfm", "summary": "This paper introduces EduLLM, a framework that combines large language models (LLMs) with hypergraph learning to improve student performance prediction. Traditional methods mainly rely on historical response patterns but struggle to capture the complex interactions between students and learning content. To address this, EduLLM integrates FraS-HNN, a spectral-based model for signed hypergraph learning, where students and questions are represented as nodes, and response records are modeled as signed hyperedges to capture both structural and semantic relationships. FraS-HNN utilizes framelet-based filters to extract multi-frequency features, while EduLLM enhances predictions by incorporating fine-grained semantic features from LLMs. Experimental results on multiple datasets show that EduLLM outperforms existing approaches, demonstrating the effectiveness of combining LLMs with signed hypergraph learning.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for carefully reading our paper and providing detailed feedback. We respectfully offer the following clarifications:\n- **Further Clarification on Motivation:** While cognitive diagnosis models have explored learner-element relationships using graphs, our goal is to provide a new perspective by modeling **student-question interactions using signed hypergraphs**. Specifically, in the context of **student performance prediction**, where responses can be either correct or incorrect, we introduce **positive and negative hyperedges** to explicitly encode this polarity in group-level interactions. We believe this formulation brings a fresh and principled structural view to the problem, which forms the core of our motivation.\n- **Clarifying the Role of LLMs:**  We would like to note that, this submission is flagged under the **Application-Driven Machine Learning** type, where the focus is on solving a specific real-world task rather than developing LLMs itself. In our framework, LLMs are used as a semantic feature extraction tool to generate question embeddings, serving as initial node features in the signed hypergraph. One reason we did not explore different LLM selection or prompt strategies is that we aimed to ensure fairness in performance comparison with the baseline models. We kept the use of the same LLM module and processing pipeline for generating the preprocessed semantic embeddings. This manner ensures that any observed improvements are due to the merits of the proposed framework, particularly the signed hypergraph setting and advantages of FraS-HNN, rather than differences in the LLM processing itself. \n  \n  For your concern about **stability** and **proportional contribution** of LLM-induced embeddings, we actually conducted a robustness test in **Section 4.7**, where additive Gaussian noise was introduced to the semantic embeddings to simulate perturbations. The results (in Figure 3) indicate that performance degrades smoothly and moderately as the noise level increases, suggesting that EduLLM is generally **robust to fluctuations in LLM outputs**. To a certain extent, this empirical evidence supports the **stability** of the model with respect to semantic input variations. In addition, the **advantage of using LLM-induced semantic embeddings**, as opposed to initial embeddings, has already been validated in related works such as SBCL and LLM-SBCL.\n- **Comparisons and Baseline Selection:**  We would like to clarify that our problem formulation, i.e., modeling student-question interactions via **signed hypergraphs**, is fundamentally different from traditional **knowledge tracing** or **cognitive diagnosis** tasks. Specifically, cognitive diagnosis methods are designed around concept-level mastery modeling **over time** and typically require a fine-grained concept-question mapping. In contrast, our approach focuses on **question-level prediction** using **signed hyperedges** to represent correctness of responses, which is not directly supported by the data structures or assumptions of cognitive diagnosis datasets. Therefore, direct comparison would be misaligned and may lead to unfair conclusions. That said, we acknowledge the broader relevance of cognitive diagnosis research in educational modeling and will incorporate appropriate discussion in a future version. Additionally, based on your suggestion, we have included comparisons with hypergraph neural network (HNN) baselines, including **HGNN, HyperGCN, AllDeepSets, AllSetTransformer, ED-HNN, and SheafHyperGNN**, by replacing the FraS-HNN backbone of EduLLM with each of these HNN modules. As shown in the table below, EduLLM (with FraS-HNN) consistently outperforms the variants equipped with each HNN module across all datasets, demonstrating the effectiveness of FraS-HNN in modeling **signed high-order interactions** and its potential to benefit future research in **(signed) hypergraph learning**.\n\n|       Model\\Dataset       |   Sydney19351   |   Sydney23146   |     Biology     |  Cardiff20102   |       Law       |\n|--------------------|----------------|-----------------|-----------------|-----------------|-----------------|\n|        HGNN          |   0.606±0.014   |   0.619±0.013   |   0.673±0.006   |   0.624±0.007   |   0.905±0.011   |\n|      HyperGCN        |   0.620±0.012   |   0.650±0.038   |   0.651±0.016   |   0.625±0.023   |   0.901±0.032   |\n|     AllDeepSets      |   0.626±0.017   |   0.660±0.012   |   0.697±0.008   |   0.637±0.023   |   0.898±0.009   |\n|  AllSetTransformer   |   0.618±0.022   |   0.661±0.010   |   0.689±0.016   |   0.644±0.029   |   0.906±0.009   |\n|       ED-HNN         |   0.662±0.023   |   0.708±0.032   |   0.715±0.039   |   0.673±0.026   |   0.910±0.011   |\n|   SheafHyperGNN      |   0.684±0.023   |   0.711±0.030   |   0.732±0.022   |   0.687±0.025   |   0.914±0.013   |\n| **EduLLM (with FraS-HNN)**| **0.712±0.016** | **0.829±0.006** | **0.809±0.010** | **0.753±0.011** | **0.945±0.005** |", "claims_and_evidence": "Overall, the claims in this paper are fairly reasonable.", "methods_and_evaluation_criteria": "Overall, the proposed method appears relatively simple, which limits its level of innovation. Additionally, some design motivation lack sufficient explanation regarding their rationale and justification. For example:\n\n1.\tThe introduction of LLMs in the method is solely for preprocessing raw text data—extracting keywords from multiple-choice question descriptions to construct a dictionary and using GloVe to learn representations as model inputs. However, I am curious why LLMs are considered an integral part of the model when they are merely used as a tool. There is no representation learning, fine-tuning, or specifically designed prompt engineering involved, and the approach relies on the simplest GloVe embeddings. Moreover, how is the stability of LLM outputs ensured during data processing, and how is their contribution controlled proportionally?\n\n2.\tThe use of signed hypergraphs seems reasonable, but what is the specific novel design of this module in this paper? In particular, what is the motivation behind the framelet-based signed hypergraph convolution? How does this design specifically cater to the task of student performance prediction?", "theoretical_claims": "The theoretical section of this paper primarily discusses the properties of signed hypergraph neural networks, but it is unclear how these properties relate to the student performance prediction scenario being modeled.", "experimental_designs_or_analyses": "I believe the experiments are quite insufficient. First, there are too few comparison methods. Why are only graph-based methods compared, and why are some common approaches in student performance prediction, such as cognitive diagnosis, not included? Additionally, since you are comparing graph-based methods, why not include some state-of-the-art graph representation learning methods, including hypergraphs and graph transformers?", "relation_to_broader_scientific_literature": "To be honest, the motivation of this paper is unclear. The authors state, \"While effective, these methods often fall short in capturing the intricate interactions between students and learning content, as well as the subtle semantics of these interactions,\" which seems unreasonable. There has already been a substantial amount of research on cognitive diagnosis methods based on graph learning to explore the relationships between learners and learning elements. However, the authors do not mention or compare these existing approaches, which is quite surprising.", "essential_references_not_discussed": "[1] Gao W, Liu Q, Huang Z, et al. RCD: Relation map driven cognitive diagnosis for intelligent education systems[C]//Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval. 2021: 501-510.\n\n[2] Shao P, Yang Y, Gao C, et al. Exploring Heterogeneity and Uncertainty for Graph-based Cognitive Diagnosis Models in Intelligent Education[J]. arXiv preprint arXiv:2403.05559, 2024.", "other_strengths_and_weaknesses": "Overall, this paper needs improvement. At least in terms of its innovation and contribution to the specific field, it is limited, which is insufficient for ICML. Specifically, the paper has several notable weaknesses:\n\n1.\tThe writing lacks logical clarity, and the motivation is not adequately addressed. Many sections are overly redundant and verbose.\n\n2.\tThe innovation at the method level is limited, particularly in the special design and improvements related to student performance prediction that the authors emphasize. As I mentioned earlier.\n\n3.\tThe experiments are insufficient and lack necessary state-of-the-art methods in the field.\n\n4.\tThe descriptions and discussions of the experiments are limited, especially regarding the performance improvements of the model itself and the corresponding conclusions.", "other_comments_or_suggestions": "Refer to Strengths&Weaknesses.", "questions_for_authors": "Refer to Strengths&Weaknesses.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gH6C3Li24p", "reviewer": "Reviewer_Lfh6", "summary": "- This paper presents EduLLM, a novel method for predicting student performance by integrating LLM-based semantic understanding with structural modeling via a framelet-based signed hypergraph neural network (FraS-HNN). \n- Signed hypergraphs capture higher-order interactions and differentiate correct from incorrect student responses, while LLMs enhance the semantic representation of questions. \n- EduLLM leverages framelet transforms to extract both low- and high-frequency information from complex educational data.", "questions": "", "limitations": "", "rating": 5, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for the thoughtful evaluation. We are pleased that the novelty and effectiveness of EduLLM and FraS-HNN are recognized, along with the **theoretical soundness, strong empirical results, and broader contributions to hypergraph learning and educational data mining**. Below we provide point-by-point clarifications regarding the concerns raised:\n\n- **Generalization Beyond Educational Domains:**   We agree that it is important to consider the applicability of FraS-HNN beyond the student performance prediction task. Although the current work is tailored for modeling educational data, **the proposed signed hypergraph formulation and framelet-based representation learning are inherently general and applicable to domains involving higher-order and polarity-sensitive interactions**. For instance, tasks such as signed social network modeling, sentiment-based recommendation, or misinformation spread detection can similarly benefit from the ability to distinguish positive and negative multi-way relations. The design of FraS-HNN does not rely on domain-specific assumptions, which supports its potential transferability.\n\n- **Motivation for Haar-type Filter:** Mathematically, the choice of the Haar-type filter is based on its efficiency, orthogonality, and suitability for decomposing signals into coarse (low-frequency) and detail (high-frequency) components. In the context of signed hypergraphs, this allows the model to simultaneously capture common learning behaviors and student-specific deviations. While other filters (such as Daubechies or spline-based filters) may offer smoother basis functions or better frequency localization, the Haar-type was selected as a starting point due to its simplicity and proven utility in prior work on multiscale graph signal processing. Exploring alternative filters remains a promising direction for future extension.\n\n- **Visual Comparison with Signed Graphs:**  We appreciate the suggestion to better highlight the advantages of signed hypergraph modeling. To support this, we will provide illustrative examples showing how signed hypergraphs can represent multi-way interactions (e.g., groups of students responding to related questions) with polarity, in contrast to signed graphs that are limited to pairwise relations. \n\n- **Further Insights and Clarification for the Theoretical Properties:** The intention behind our theoretical analysis is to reveal how **the proposed FraS-HNN effectively captures both low-pass and high-pass components of node signals in signed hypergraphs**, enabling the model to differentiate shared and individualized patterns within complex educational interactions. To make these insights more accessible to the readers, we will summarize the key takeaways in the main text (in the updated version), such as: (1) how the framelet transform enables multiscale analysis on signed hypergraphs, (2) the specific role of positive and negative hyperedges in modulating spectral responses, and (3) how this facilitates richer representation learning compared to traditional graph-based or unsigned hypergraph approaches. \n\nAgain, we sincerely thank you for recognizing the merits of our proposed FraS-HNN module, which we believe contributes not only to the student performance prediction task **but also holds broader value for advancing (signed) hypergraph learning in general**. In response to **Reviewer stfm's** suggestions, we have conducted additional empirical studies that include comparisons with several representative hypergraph neural network (HNN) baselines, i.e., HGNN (AAAI, 2019), HyperGCN (NeurIPS, 2019), AllDeepSets (ICLR, 2022), AllSetTransformer  (ICLR, 2022), ED-HNN  (ICLR, 2023), and SheafHyperGNN (NeurIPS, 2023), by replacing the FraS-HNN backbone of EduLLM with each of these modules. Please refer to our response to Reviewer stfm for further details. These supplementary results further validate the effectiveness of the high-pass and low-pass filters in FraS-HNN, as positively noted in your comments. For future work, **we plan to construct more challenging signed hypergraph datasets beyond the educational domain (e.g., in social networks, biology, traffic systems)** to support the development of this emerging direction in terms of theoretical understanding, model design, and real-world applications. We believe our framework, along with the core FraS-HNN module, can benefit other complex tasks and applications where signed hypergraphs (or their variants) align well with the problem formulation.\n\nWe hope the above responses clarify the key concerns raised in your comments and questions. Please feel free to reach out with any further suggestions or points for discussion. We are happy to engage further to improve the clarity and impact of our work.", "claims_and_evidence": "- The claims in the submission are well-supported.", "methods_and_evaluation_criteria": "- EduLLM and evaluation criteria are well-suited for the student performance prediction,\n- and benchmark datasets effectively show its effectiveness.", "theoretical_claims": "- The theoretical proofs, formulations, and derivations are correct; however, the authors need to address some typos.", "experimental_designs_or_analyses": "- The use of diverse educational datasets, along with detailed comparisons and ablation analyses, provides solid evidence supporting the effectiveness.", "relation_to_broader_scientific_literature": "- The key contributions closely aligned with ongoing research in educational data mining, hypergraph learning, and the application of large language models. \n- FraS-HNN introduces a novel method for hypergraph learning, which can be considered a model-level contribution, not confined to a specific application. \n- FraS-HNN has the potential to inspire further advancements in hypergraph neural networks, hypergraph learning, and related applications.", "essential_references_not_discussed": "- This paper has appropriately cited relevant sources.", "other_strengths_and_weaknesses": "- Strengths \n   - The paper redefines student performance prediction as a hypergraph learning, which effectively model both correct and incorrect student responses through positive and negative edges. \n   - FraS-HNN applies multiscale signal processing with low-pass and high-pass filters to capture both shared patterns and individual differences within student interactions, enhancing the representation of complex educational relationships.\n   - EduLLM successfully combines hypergraph structural learning with semantic features from LLMs, leading to a more comprehensive understanding of student-content interactions and delivering improved prediction performance across multiple datasets.\n   - The theoretical analysis of FraS-HNN is sound.\n- Weaknesses:\n   - While the student performance prediction task is well-suited to the signed hypergraph learning formulation, a novel and insightful perspective proposed by the authors, EduLLM demonstrates strong performance on educational datasets. \n      - However, the paper offers limited discussion on its potential generalization to other domains beyond student performance prediction.\n   - The motivation for selecting the Haar-type filter in the framelet construction is not clearly explained. \n      - The authors should elaborate on why this filter was chosen and whether other filter types were considered. \n      - Additionally, could alternative filters offer advantages in capturing different patterns within the signed hypergraph?", "other_comments_or_suggestions": "- To better highlight the advantages of signed hypergraph modeling, it would be helpful to include visual examples that compare the structural differences between traditional signed graphs and signed hypergraphs using real student-question interaction data.", "questions_for_authors": "- The authors summarize the key insights of these properties in the main text.\n- I strongly suggest that the authors enhance the clarity of their Abstract by clearly stating their motivation and key insights.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6C0ffE1r5M", "reviewer": "Reviewer_xwCf", "summary": "This paper presents EduLLM, a novel framework that integrates Large Language Models (LLMs) with a Framelet-based Signed Hypergraph Neural Network (FraS-HNN) to address student performance prediction in personalized education systems. EduLLM models the complex structural and semantic relationships between students and multiple-choice questions (MCQs) by constructing signed hypergraphs, where positive and negative hyperedges capture correct and incorrect responses. LLMs provide fine-grained semantic embeddings for educational content, which are combined with the multi-frequency features extracted by FraS-HNN using low-pass and high-pass filters. Through comprehensive experiments on five real-world educational datasets, EduLLM demonstrates significant improvements over strong baselines, highlighting its effectiveness in both capturing high-order relationships and integrating semantic information for performance prediction tasks.", "questions": "", "limitations": "", "rating": 5, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for the encouraging feedback on our work. We are pleased that you recognized **the novelty and technical contributions of EduLLM, including its theoretical soundness, methodological innovation, and comprehensive experimental validation**. For your key concerns, please find our responses below:\n\n- **Generalization Beyond Educational Domains & Alternative Wavelet Designs:**  Thank you for noting these issues. For detailed clarifications regarding the potential generalization of FraS-HNN beyond educational applications and the discussion on exploring alternative wavelet designs, please kindly refer to our responses to **Reviewer Lfh6** (who is also curious about this).  In addition, we would like to emphasize that, as flagged in our submission type, this work is submitted under the **Application-Driven ML Submissions** category. Our particular focus is specifically on the student performance prediction task, a well-recognized and meaningful problem in the educational domain. The proposed EduLLM framework is newly developed for solving this specific problem. Therefore, we did not engage in studying broader generalization across domains, as that falls outside the intended scope of our application-driven submission. For more context, please refer to the **Supplementary Guidelines for Reviewing Application-Driven ML Submissions** (see https://icml.cc/Conferences/2025/ReviewerInstructions). \n\n  Hopefully, this clarifies the position of our work within the specific context of this year’s ICML submission categories..\n\n\n\n- **Framelets for Directed and Signed Hypergraphs:**  We appreciate the reviewer’s insightful question regarding the extension of framelet theory to directed and signed hypergraphs. Although there is very recent work defining the notion of **directed hypergraphs** (see: [https://openreview.net/forum?id=h48Ri6pmvi](https://openreview.net/forum?id=h48Ri6pmvi)), to the best of our knowledge, a formal and unified definition of **directed and signed hypergraphs** has not yet appeared in the literature. Developing framelet transforms for such structures would require extending spectral theory to handle both edge directionality and polarity, potentially involving non-symmetric Laplacians or incidence-based spectral operators. Key challenges include defining appropriate inner product spaces, preserving desirable mathematical properties (e.g., tightness and localization), and ensuring the interpretability of the resulting representations. While this extension is beyond the scope of the current work, we view it as an important direction for future investigation. Of course, we hope that our current work, particularly the key module FraS-HNN, **can motivate further follow-up studies on directed and signed hypergraphs, whether through model development or advanced applications**.\n\nAgain, we sincerely thank you for recognizing the merits of our proposed FraS-HNN module (also recognized by **Reviewer Lfh6**), which we believe contributes not only to the student performance prediction task **but also holds broader value for advancing (signed) hypergraph learning in general**. In response to **Reviewer stfm's** suggestions, we have conducted additional empirical studies that include comparisons with several representative hypergraph neural network (HNN) baselines, i.e. HGNN (AAAI, 2019), HyperGCN (NeurIPS, 2019), AllDeepSets (ICLR, 2022), AllSetTransformer  (ICLR, 2022), ED-HNN  (ICLR, 2023), and SheafHyperGNN (NeurIPS, 2023), by replacing the FraS-HNN backbone of EduLLM with each of these modules. Please refer to our response to Reviewer stfm for further details. These supplementary results further validate the effectiveness of the high-pass and low-pass filters in FraS-HNN, as positively noted in your comments. For future work, we plan to construct more challenging **signed hypergraph datasets beyond the educational domain (e.g., in social networks, biology, traffic systems)** to support the development of this emerging direction in terms of theoretical understanding, model design, and real-world applications. We believe our framework, along with the core FraS-HNN module, can benefit other complex tasks and applications where signed hypergraphs (or their variants) align well with the problem formulation.\n\n\nWe hope the above responses have clarified your questions and comments. We welcome any further discussion or suggestions.", "claims_and_evidence": "Yes, the claims made for EduLLM in this submission are well-supported by comprehensive evidence, including both rigorous theoretical analysis and extensive empirical results.", "methods_and_evaluation_criteria": "Yes, the proposed methods and evaluation criteria are well-aligned with the problem of student performance prediction.", "theoretical_claims": "Yes, I have reviewed the provided theoretical analysis and proofs, particularly those related to the framelet-based signed hypergraph neural network (FraS-HNN). The mathematical formulations, including the construction of framelets, spectral filtering, and the tight frame properties, are clearly presented and appear logically sound.", "experimental_designs_or_analyses": "Yes, I have reviewed the experimental design and analyses. The experiments are well-structured, using five real-world educational datasets that are appropriate for the student performance prediction task. The evaluation includes comparisons against strong baselines, thorough ablation studies to assess the contributions of key components, parameter sensitivity analyses, and robustness tests for LLM-induced semantic representations. \nOverall, the experimental designs and analyses are comprehensive.", "relation_to_broader_scientific_literature": "Technically, EduLLM advances the student performance prediction (which indeed is a classic educational data mining task) by incorporating LLM-generated embeddings to enrich the feature representations of questions, helping to model semantic nuances in student interactions. \nAlso, as for the problem-solving, prior studies have applied signed graphs to model positive and negative relationships (e.g., correct/incorrect answers) in educational scenarios. However, traditional signed graphs are limited to pairwise relations. EduLLM extends this by introducing signed hypergraphs, which can capture higher-order interactions (such as multiple students answering the same question) and better reflect the group dynamics present in educational settings.\nEduLLM builds on this by applying framelet theory to signed hypergraphs, enabling the simultaneous extraction of global (low-pass) and discriminative (high-pass) features from complex educational interactions. I think this may potentially motivate more follow-up works on signed hypergraphs.", "essential_references_not_discussed": "The paper cites a solid range of related works across knowledge tracing, signed graph learning, hypergraph neural networks, and educational data mining. The literature review is up to date.", "other_strengths_and_weaknesses": "S1: [New Problem Formulation] The paper introduces a well-defined and meaningful problem setting by modeling student performance prediction with signed hypergraphs to capture both correct and incorrect interactions in a high-order structure.\nS2: [Novel Framework] The proposed EduLLM framework uniquely combines LLM-based semantic embeddings with a framelet-based signed hypergraph neural network, offering an innovative solution to educational prediction tasks.\nS3: [Comprehensive Theoretical and Empirical Studies] The work provides both solid theoretical foundations and extensive empirical validation through detailed experiments, ablation studies and evaluation on multiple real-world datasets.\n\nW1: The application and evaluation are focused on educational data, with limited discussion on how the framework could generalize to other domains or tasks beyond student performance prediction. Any further insights?\nW2: Although I am familiar with framelets and wavelets, I am curious whether the current framework could be further enhanced by exploring alternative wavelet designs on hypergraphs. Specifically, would incorporating different types of wavelets lead to improved feature extraction or better adapt to various hypergraph structures?", "other_comments_or_suggestions": "See the above weaknesses", "questions_for_authors": "Q1: I am curious whether framelets can be designed specifically for directed and signed hypergraphs (I think this is a brand-new definition, to the best of my knowledge). Although this is beyond the scope of the current work, it would be interesting to understand the key challenges involved.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "c5j2jC80CG", "reviewer": "Reviewer_vEwM", "summary": "The paper introduces EduLLM, a new framework for student performance prediction that combines Large Language Models (LLMs) with a Framelet-based Signed Hypergraph Neural Network (FraS-HNN). FraS-HNN is a novel approach for signed hypergraph learning, utilizing high-pass and low-pass filters to extract multi-frequency interactions between students and questions. LLMs are used to enhance semantic representation, complementing hypergraph structural learning to improve predictive accuracy. The LLM-based semantic feature extraction is relatively standard, with limited discussion on LLM selection, fine-tuning, or task adaptation. It appears to mainly convert text into embeddings and integrate them with the hypergraph structure.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for the constructive feedback and for acknowledging the novelty and strengths of our proposed framework. Please find our detailed responses below:\n- **LLM Selection, Fine-tuning, and Task Adaptation:**  We appreciate the reviewer’s observation and agree that further analysis of LLM choices could be valuable in broader contexts. However, to clarify the scope of our work: LLMs in EduLLM are primarily used as a semantic information extraction tool to generate initial node features for questions in the student-question signed hypergraph, rather than being the core focus of technical innovation. **One reason** why we do not study in detail the effect of different LLM selections is that, as we would like to note, this submission is flagged under the **''Application-Driven Machine Learning''** category, where the focus is on solving a specific real-world task and not on advancing LLM development itself. **Another reason** is that we aimed to ensure fairness in the performance comparison with the baseline models, so we used the same LLM module and processing pipeline to generate the preprocessed semantic embeddings. This manner ensures that any observed improvements are due to the merits of the proposed framework, particularly the signed hypergraph setting and advantages of FraS-HNN, rather than differences in the LLM processing itself.\n\n  Alternatively, to approximately assess the impact of LLM-based embeddings on model performance, we conducted a robustness study presented in **Section 4.7**. The results, shown in **Figure 3**, demonstrate that the model’s performance degrades smoothly and only slightly at higher noise levels. This robustness suggests that EduLLM is relatively insensitive to variations in the LLM-induced embeddings, implying that replacing the current LLM backbone with a different one is unlikely to significantly affect task performance.\n- **Large-scale Dataset:**  We agree that evaluating on larger-scale datasets would strengthen the practical validation of EduLLM. Our current datasets are widely used and shared with baseline models, ensuring fairness and comparability. Meanwhile, our lab is constructing new large-scale datasets with MCQ texts from various subjects, which will allow us to further assess scalability and real-world applicability in future work.\n- **Theoretical Clarity and Intuition:**  While detailed proofs are provided in the appendix, we agree that offering more intuitive explanations would help clarify the motivation and benefits of our approach. In a future updated version, we will provide more intuitive explanations of FraS-HNN’s advantages over existing methods, which we believe will also benefit researchers working on **(signed) hypergraph learning**.\n- **Comparisons with Existing Hypergraph and KT Models:**  We would like to clarify that our problem formulation, predicting student performance at the question level using signed hypergraphs, is not entirely equivalent to the KT task, which (although relates to student performance prediction) typically focuses on modeling students' evolving mastery over concepts across time. We have added additional evaluations against representative hypergraph neural networks (HNNs), including **HGNN, HyperGCN, AllDeepSets, AllSetTransformer, ED-HNN, SheafHyperGNN**. Due to space constraints, the detailed results are provided in our response to **Reviewer stfm**, demonstrating clearly FraS-HNN's effectiveness in capturing **signed high-order interactions**.\n- **Computational Complexity and Scalability:**  We provide a comparative analysis of training complexity for several representative HNNs and our proposed FraS-HNN, summarized in the table below. In our formulation, $k$ (the number of high-pass filters), $J$ (the scale level in FraS-HNN), and $K$ (the largest number of non-zero values in the framelet transform matrices) are constants independent of specific hypergraph structures. In practice, $k$ and $J$ are small, and due to the sparsity of hypergraph framelets, $K$ is typically small and often comparable to $\\|H\\|_0$. As a result, FraS-HNN's overall complexity is comparable to models such as AllDeepSets and ED-HNN, without introducing significant overhead.\n| Model                | Computational Complexity |\n|---------------------|----------------------------------|\n| UniGCNII            | $\\mathcal{O}(TL(N+M+\\|H\\|_0)d + TLNd^2)$ |\n| Deep-HGCN           | $\\mathcal{O}(TLM'd + TLNd^2)$ |\n| AllDeepSets         | $\\mathcal{O}(TL\\|H\\|_0d + TL(N+M)d^2)$ |\n| ED-HNN              | $\\mathcal{O}(TL\\|H\\|_0d + TL(N+M)d^2)$ |\n| **FraS-HNN (ours)** | $\\mathcal{O}(TL(kJ+1)Kd + TL(N+M)d^2)$ |\n  \n  Here, $N$ and $M$ denote the number of nodes and hyperedges, respectively; $\\|H\\|_0$ is the number of non-zero entries in the incidence matrix; $T$, $L$, and $d$ denote the number of epochs, layers, and feature dimensions; $M^{'}$ is the number of edges in the clique expansion (when transforming the hypergraph into a graph).", "claims_and_evidence": "Claims\n1. EduLLM improves student performance prediction by integrating LLMs with hypergraph neural networks.\n2. FraS-HNN effectively models signed hypergraphs, capturing both structural and semantic information to enhance prediction performance.\n3. EduLLM outperforms existing state-of-the-art (SOTA) methods across multiple educational datasets.\nClaims 1 and 2 are supported by experiments but lack a detailed discussion on optimizing the LLM component. LLMs are primarily used for feature extraction without further fine-tuning or analysis.\nClaim 3 shows strong results on five datasets, but th comparison lacks larger-scale datasets and does not evaluate computational complexity.\nThe theoretical analysis of FraS-HNN is complex but lacks an intuitive explanation of its advantages over existing methods.", "methods_and_evaluation_criteria": "1. The paper models student-question interactions using signed hypergraphs and integrates LLM-enhanced semantic features, which provides a degree of novelty.\n2. FraS-HNN is mathematically analyzed, including its framelet-based hypergraph filtering approach.", "theoretical_claims": "Theoretical results demonstrate multi-frequency signal analysis and the design of high-pass/low-pass filters.However, comparative experiments do not show a clear advantage of FraS-HNN over traditional hypergraph GNNs (e.g., HyperGCN, HCHA).", "experimental_designs_or_analyses": "The model's performance is evaluated on five educational datasets, showing that EduLLM consistently achieves higher F1-score and AUC than SOTA methods.\nAblation studies analyze the contributions of high-pass filters, low-pass filters, and LLMs.\n\nLimitations:\n1. Lack of large-scale dataset evaluation—current datasets have limited diversity and scale.\n2. No computational complexity analysis—FraS-HNN's scalability remains unclear.", "relation_to_broader_scientific_literature": "1. EduLLM uses FraS-HNN for signed hypergraph learning, related to HyperGCN, HCHA, and UniGNN.\n2. The work can be seen as an extension of KT tasks, related to models such as DKT, DKVMN, AKT, and SAINT.\n3. The paper applies LLMs for semantic feature extraction, aligning with recent studies on ChatGPT in personalized learning.", "essential_references_not_discussed": "1. HyperGCN (Yadati et al., 2019): A strong baseline for hypergraph convolution, should be included in comparative experiments.\n2. AKT (Ghosh et al., 2020): A state-of-the-art knowledge tracing model, relevant for student performance prediction, should be compared against EduLLM.\n3. MOOC-related work (Piech et al., 2015; Nakagawa et al., 2019): Prior studies evaluating large-scale online learning datasets—EduLLM should be tested on similar datasets for practical validation.", "other_strengths_and_weaknesses": "Strengths:\n1. EduLLM achieves SOTA performance on the evaluated datasets.\n2. Integrating LLMs with hypergraph neural networks introduces a new perspective for student modeling.\n\nWeaknesses:\n1. The study only evaluates five small datasets, lacking tests on more challenging real-world datasets (e.g., MOOCs).\n2. The paper does not discuss the scalability of FraS-HNN on large-scale datasets.", "other_comments_or_suggestions": "Discuss LLM selection and optimization—a comparison between different LLMs (e.g., GPT-4, LLaMA) would strengthen the contribution.", "questions_for_authors": "How was the LLM chosen? Have different LLM architectures been considered?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper presents EduLLM, a framework that combines large language models (LLMs) with hypergraph learning to improve student performance prediction. Most reviewers found the paper is well organized and recognized the technical contributions of this work. Meanwhile, reviewers raised some concerns about motivation, technical details, baselines, paper writing, etc. Most of these concerns were successfully addressed by the author's rebuttal.
Decision: Accept (poster)