DisenGCD: A Meta Multigraph-assisted Disentangled Graph Learning Framework for  Cognitive Diagnosis
Authors: Shangshang Yang
Mingyang Chen
Ziwen Wang
Xiaoshan Yu
Panpan Zhang
Haiping Ma
Xingyi Zhang
Venue: NeurIPS 2024 poster (2024)
Keywords: Cognitive Diagnosis
Intelligent Education
Disentanglement Learning
Robustness
Graph Learning
Primary area: machine_learning_for_other_sciences_and_fields

Abstract: Existing graph learning-based cognitive diagnosis (CD) methods have made relatively good results, but their student, exercise, and concept representations are learned and exchanged in an implicit unified graph, which makes the interaction-agnostic exercise and concept representations be learned poorly, failing to provide high robustness against noise in students' interactions. Besides,  lower-order exercise latent representations obtained in shallow layers are not well explored when learning the student representation. 
To tackle the issues, this paper suggests a meta multigraph-assisted disentangled graph learning framework for CD (DisenGCD), which learns three types of representations on three disentangled graphs: student-exercise-concept interaction,  exercise-concept relation, and concept dependency graphs, respectively. 
Specifically,  the latter two graphs are first disentangled from the interaction graph. 
Then, the student representation is learned from the interaction graph by a devised meta multigraph learning module; multiple learnable propagation paths in this module  enable current student latent representation to access  lower-order exercise latent representations,
which can lead to  more effective nad robust student representations learned; 
the exercise and concept representations are learned on the relation and dependency graphs by graph attention modules. 
Finally, a novel diagnostic function is devised to handle three disentangled representations for prediction.  Experiments show better performance and robustness of DisenGCD than state-of-the-art CD methods and demonstrate the effectiveness of the disentangled learning framework and meta multigraph module.The source code is available at https://github.com/BIMK/Intelligent-Education/tree/main/DisenGCD.

DisenGCD: A Meta Multigraph-assisted Disentangled Graph Learning Framework for Cognitive Diagnosis
Shangshang Yang 1.2
Mingyang Chen 1,3
Ziwen Wang'
Xiaoshan Yu!
Panpan Zhang
Haiping 13
Xingyi Zhang 1.3
Key Laboratory of Intelligent Computing and Signal Processing of Ministry of Education, Anhui University. Hefei, Anhui 230601, P. R. China
2 Anhui Province Key Laboratory of Intelligent Computing and Applications, Department of Information Materials and Intelligent Sensing Laboratory of Anhui Province (yangshang0308, wzw12sir, yxsleo, zppan55, xyzhanghust @ggmai com q22201127@stu. ahu, edu. hpma@ahu edu.
Abstract
Existing graph learning-basedd cognitive diagnosis (CD) methods have made rel- atively good results, but their student, exercise, and concept representations are learned and exchanged in implicit unified graph, which makes the interaction- agnostic exercise and concept representations be Iearned poorly, failing to pro- vide high robustness against noise in students' interactions. Besides, lower-order exercise latent representations obtained shallow layers are not well explored when learning the student representation. To tackle the issues, this paper sug- gests a meta multigraph-assistee disentangled graph learning framework for CD (DisenGCD). which learns three types of representations graphs: student-exerrise--ommep interaction, exercise-concept relation, and con- cept dependency graphs, respectively. Specifically, the latter two graphs are first disentangled from the interaction graph. Then, the student representation is learned from the interaction graph by devised meta multigraph learning mod- ule; multiple learnable propagation paths in this module enable current student latent representation access lower-order exercise latent representations, which can lead to more effective nad robust student representations learned; the ex- ercise and concept representations are learned on the relation and dependency graphs by graph attention modules. Finally, a novel diagnostic function is de- vised to handle three disentangled representations for prediction. Experiments show better performance and robustness of DisenGCD than state-of-the-art CD methods and demonstrate the effectiveness of the disentangled learning frame- work and meta multigraph module. The source code is available at https; //github. com/ com/BIMK/ Intelligent Education http:/maanDDDieeee
three disentangled
Introduction
cognitive diagnosis (CD) plays crucial role in estimating students' mastery/proficiency each knowledge concept (1]. which mainly models the exercising process of students by predicting students' responses based on their response records/logs
In the realm of intelligent education 43 27
*Corresponding author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Cognitve
E
B
Disprosis
Hob
E
concepts
Figure I: Cognitive diagnosis based on student's response records and exercise-coneept relations.
and the exercise-coneept relations. As shown in Figure []] student Bob completed five exercises {ei, e2, e3, ed, es and got corresponding responses, and his diagnosis result can be obtained through CD based on his records and the plotted relations between exercises and concepts. With students' diag- nosis results, many intelligent education tasks can be benefited, such as exercise assembly 42), course
recommendation (34 45 student testing [8] and targeted training (2) and remedial instruction
In recent years, many cognitive diagnosis models (CDMs) have been suggested based on various techniques, generally classified into two rypes. The first type of CDMs is based on the theory of educational psychology, and the representatives include DINA 4.. (IRT) 13 and MIRT [20]. Considering that the simple diagnostic functions of this type of models fail to model complex student interactions, recently, artificial intelligence researchers have tried to employ neural networks (NNs) to enhance and invent novel CDMs. Therefore. the second type of CDMs can be further divided into two sub-types: focusing on inventing novel diagnostic functions and enhancing the input representations of CDMs, respectively. The representatives
the first sub-type of CDMs include the well-knoww 36, (38. For the second sub-type of CDMs, many NN long short-term memory networks for DIRT (3) graph neural networks (GNNs) for RCD 7 SCD 28 and GraphCDM (21).
NCD 255, KSCD [15), and so on [4 16 12
techniques are employed, including hierarchical attention NNs for ECD
48
Among existing CDMs, GNN-based ones exhibit significantly better performance than others, which is attributed to the rich information propagation and exchange brought by GNNs. These CDMs generally learn the student, exercise, and concept representations implicitly or explicitly on a unified graph 7 [28], which makes three types of representations be exchanged and aggregated fully even they are distant regarding relations, thus generating better representations and providing better performance. However, learning exercise and concept representations should be agnostic [3I] while the above CDMs' learning for exercise and concept representations is easily affected by students' interactions, and the representations will be learned poorly especially when there exists noise students' interactions. In short, existing GNN-based CDMs fail to provide high robustness against student interaction noise. In addition, these CDMs do not explore the use of lower-order exercise latent representations shallow layers for learning the student representation due to the intrinsic defect of their GNNs. The learned student representations could be more robust and not sensitive to student interaction data change.
Therefore, we propose meta caaligrrap--assister disentangled graph learning framework for CD (DisenGCD) to learn robust representations against interaction noise. The contributions include
(1) The disentangled graph learning framework DisenGCD learns three types of representations on three disentangled graphs. Specifically, the student representation is learned on the concept interaction graph; the exercise and concept representations are learned on two disentangle graphs: the exercise-concept relation graph and the concept dependency graph. By doing so, the latter two learned representations are interaction-agnoste and robust against student interaction noise.
(2) To make the best of lower-order exercise latent representations for learning the robust student representation, a modified meta multigraph module containing multiple learnable propagation paths is used, where propagation paths enable current student latent representation to access and use lower-order exercise latent representations. The exercise and concept representations are learned through common graph attention networks (GAT) relation and dependency graphs, respectively. Finally, a novel diagnostic function is devised to handle three learned representations for prediction.
(3) Extensive experiments show the superiority of the proposed DisenGCD to state-of-the-art (SOTA) models regarding performance and robustness, and the effectiveness of the disentangled graph learning framework and the devised meta multigraph module is validated.

---

2
Related Work
Related Work on Cognitive Diagnosis. The above has given the classification of CDMs, and we will briefly introduce typical CDMs, especially GNN-based ones. As representatives in educational psychology, IRT [33 (MIRT (20)) utilizes single (multiple) variable(s) to denote student's ability with logistic function for prediction. For NN-based representatives, NCD [25] and KSCD [IS] fed student, exercise, and concept vectors to an IRT-like NN as the diagnostic function for prediction;
NAS-GCD
44 is similar to NCD, but its diagnostic function is automatically obtained.
For GNN-based CDMs, their focus is on obtaining enhanced representations. For example, RCD learns the student representation along the relation of adjacent exercise nodes, learns the exercise representation along the relation of adjacent student and concept nodes, and learns the concept repre- sentation along the relation of adjacent exercise and concept nodes. Since each node's ' information can be propagated to any node, learning regarding three relations can be seen as learning on an implicit unified graph; Similarly, SCD (28) takes the same aggregation manner along the first two relations RCD for learning student and exercise representations for contrastive learning to mitigate long-tail problems. Despite the success of GNN-based CDMs, their learning manners of exercise and concept representations are not robust against student interaction noise, because the representations are learned together with student interactions in unified graph, where the noise will prevent the representations from being learned well.
Related Work on Meta Graph/Multirraph. As can be seen, existing GNN- CDMs intuitively adopt classical GNNs (GAT (24 or GCN(9) to learn three types of representations, These CDMs update the student representation by only using the exercise latent representation in the previous layer yet not exploring the use of lower-order exercise latent representations.
Compared to traditional GNNs, meta graph-based GNNs can make the target type of nodes access lower-order latent representations of their adjacent nodes. The meta graph is a directed acyelic graph built for a GNN, each node stores each layer's ' output (latent representations) of the GNN, and each edge between two nodes could be one of multiple types of propagation paths. The representatives include DifTMG (5] and GEMS S. The meta multigraph is the same as the meta graph, but its each edge could hold more than one type of propagation path, where the representative is PMMM [IO]. To make the best of low-order exercise latent representations for learning effective and robust student representations, this paper adopts the idea of meta multigraph and devises a modified meta multigraph learning module for the updating of the student interaction graph.
Related Work on Disentangling Graph Representation Learning. Learning potential representa- tions of disentangling in complex graphs to achieve model robustness and interpretability has been hot topic in recent years. Researchers have put forward many disentangiement approaches (c.g.. DisenGCN 11). DisenHAN (32), DGCF [29]. DCCF [35]. DeRee [33), etc.) to address this chal- lenge. For example, in DisenHAN 32), the authors utilized disentangled representation learning to account for the influence of each factor in an item. They achieved this by mapping the representation into different spatial dimensions and aggregating item information from various edge types within the graph neural network to extract features from different aspects; DeRee [33] disentangles the network into user-item domain and a user-user social domain. generating two views through data augmentation and ultimately obtaining a more robust representation via contrastive learning.
Despite many approaches suggested, they were primarily applied to bipartite graphs learn different representations from different perspectives, for learning more comprehensive representations. While this paper aims to leverage disentanglement learning to mitigate the influence of the interaction noise in the interaction graph, and thus we proposed a meta multigraph-asiittd disentangled graph cognitive diagnostic framework to learn three types of representations on three disentangled graphs. By doing so, the influence of the noise on exercise and concept learning can be well alleviated.
3 Problem Formulation
For easy understanding, two tables are created to describe all notations utilized in this paper including notations for disentangled graphs and notations for the meta multigraph, summarized in Table 4 and Table Due to the page limit, the two tables are included in Appendix A.
3

---

3.1 Disentangled Graph
This paper only employs the sundent-exercise-comee interaction graph Gr for students' representation learning, and disentangles two graphs from Gr. for the remaining two types of representation learning, which are the exercise-concept relation graph GR. and the concept dependency graph Gr.
Student- Exercixe--onceepp Interaction Graph. With students' response records R. exercise- concept relation matrix, and concept dependency matrix, the interaction graph Gz can be represented S U E J C the union of the student set S, exereise set
as ςI EV. The node E, and concept set C, while the edge set 3 Ree U Ree Uec contains three types of relations: rseej E Rye represents the student 8; E S answered exercise G E E, recjk E Rec denotes the exercise ej contains concept C E C, and rcckm Ree denotes concept C₁ relies on concept Cm.
Disentangled Relation Graph. To avoid the impact of students' interactions Rae on exercise representation learning, the exercise-concept relation graph GR is disentangled from Gz, denoted as GR = 9z/ (1, R { U C, Rec U Rec).
Disentangled Dependency Graph. Similarly, a concept dependency graph Gp is further disentangled from Gz to learn the concept representation without the influence of interactions Rec and exercises' relation Rec which is represented by GD = GR/ (E, Reo = {C, Ree).
3.2 Problem Statement
For the cognitive diagnosis task in an intelligent education online platform, there are usually three sets of items: set of N students S {s1. ..,...,......................... SN J. set of M exercises E {er, e2,,,,,..., h. and a set of K knowledge concepts (concept for short) C' = {ci, C2 .2,,.,, C. Besides, there commonly exists two matrices: exercise-concept relation matrix Q (Qjk {o. x (Q- matrix) and concept dependency matrix D (Qkm {0, 1))X*K to show the relationship of exercises to concepts and concepts to concepts, respectively. 2,k = denotes the concept e₁ is not included in the exercise Ej and Q5k 0 otherwise; similarly, Dam = 1 denotes concept ek relies on concept Cm and D km = 0 otherwise. All students' exercising reponse logs are denoted by Tay)]ss E S, E, r,j E {0, 1}}, where ray refers the response/answer of student R {(s, Cj Tay
8; on exercise ej. Fij = means the answer is correct and Fij = 0 otherwise.
The CD based on disentangled graphs is defined as follows: Given: students' response logs R and three disentangled graphs: interaction graph Gr, relation graph GR, and dependency graph Gp; Goal: revealing students' proficiency on concepts by predicting students' responses through NNs.
4 Method
For better understanding, Figure presents the overall architecture of the proposed DisenGCD, where three learning modules are used for learning three types of representations on three disentangled graphs and the diagnostic function makes the final prediction based on the learned representations. Specifically, these three learning modules are: (1) a meta multigraph-bar based student learning module, (2) GaT-based exercise learning module, and (3) GAT-based concept learning module. In each learning module, the corresponding graph's embedding is randomly first initialized. Then, the meta multigraph-based student learning module is employed to learn the student representation S; based on interaction graph Gr; the GAT-based exercise learning module is used to learn the exercise representation E, based on relation graph GR; while the concept representation C2 is learned by the GAT-based concept learning module on dependency graph GD or the naive embedding GD unavailable. Finally, devised novel diagnostic function receives three learned representations S,, E,, and C2 get the prediction rig of student s; on exercise ej-
Note that the first module employs modified meta multigraph aggregator to learn student representa- tions. Compared to traditional graph learning, the module contains multiple learnalle propagation paths, which enable the student latent representation be learned currently to access and use lower- order exercise latent representations, leading to more effective and robust student representation.

---

Embedding Layer
Disentangled
Graph Learning
Diagnostic Function
Framework
go
FD
API IP
Meta Multigraph-based
GAT-based Exercise Learning Module
GAT-based Concept Learning Module
Student Learning Module
AP
HIM
Interaction Graph
Relation graph
Dependency graph
AP Candidate propagation paths
C
AP,,
A ak Axe Age
hra
Disentangle
Disentangle
Meta-multigrahh Aggregator
GAT
GAT
H
Ags
Hi
Ay
S
ZCTO
Figure 2: Overview of the proposed DisenGCD: the disentangled graph learning framework is composed three learning modules: two GAT-based learning modules and a meta multigraph-baset learning module, where the latter's details are shown in the green part.
4.1 The Meta Multigraph-based Student Learning Module
Based on interaction graph Gr, this module is responsible for: learning the optimal meta multigraph structure in Gz, i.e., optimal multiple propagation paths, and updating student nodes' representations based on the learned meta multigraph structure to get the representation S; R xxd
To start with, all nodes (including students, exercises, and concepts) need be mapped into
d-dimensional hidden space. The student embedding 8; E IR. IRixd for student 8,, exercise embedding ef IR 1xd for exercise Ey. and concept embedding of 1gd for concept cæ can be obtained by
s, = x x W e/ xx x W cf = x x W W Rxd W E Mxd W E Rxdd (I)
x E (0. 1) |xN x E {0.1)IXM, and x E f0, 111xx are three one-hot vectors for student 8; exercise ej, concept Ck. respecitvely. W3. W and W are three learnable parameter matrices.
4.1.1 The Meta Multigraph Aggregator
To learn more useful representations from the above initial embedding, we construct meta multigraph MG AP) to specify set of propagation paths for all nodes' aggregation in graph learning.
As shown in Figure the meta multigraph is actually directed acyclic graph containing P hyper- nodes: H (H
(P)) ..... H()). (P)) {ss e Rxx e² E Mxd c² E k is set of latent representations stored in p-th hyper-node for all students, exercises, and concepts, whose row i-th row s R Ixd j-th row ef IR xd and k-th row c² R are the p-th latent representations of student s;, exercise Ej, and concept Ck2 respectively. Especially when p=l, S. ef,. and cf equal initial embedding s;, c/. and c/.
Each edge APu. E AP between two hyper-nodes (1)) and (v) contains multiple types (HR)) of propagation paths, which can be denoted as AP.. {(hr, HR, hr E [0, 1]]]1 a RR) Here hr, is a-th type of propagation path in candidate path set H R, and hr represents the weight of lrr, This paper adopts ft types of propagation paths as the candidate paths for RR: the students to exercises path A.c. 2) the exercises students path Aes. 3) the exercises to concepts path Aak, the concepts exercises path Age. 5) the concepts to concepts path Agk, 6) the identity path 1, 7) the zero path zero.
5

---

Here, path Ase means updating the exercise latent representation along the propagation path of student nodes exercise nodes, and other paths hold similar meanings. As a result, HR (Ase, Aes, Ack: Ake, Akk. 1, zero), and the edge set AP in Mg can be denoted by AP = (APael I < S P), containing [ R * * (P - 1)/2] propagation paths. With above propagation paths, (P) can be updated by
H EU(AP up H(m) 1 u u p),
where f()) is the aggregation function of GCN. f(AP up, H refers to that obtaining the latent representations (sf, eg. + in hyper-node H(P) based on all latent representations stored in previous hyper-nodes (i.e., H()) to H(P-1)), can be seen that the updating process of HPP) replies on latent
representations in multiple hyper-nodes and multiple propagation paths, which compose the modified meta multigraph aggregator together.
For better understanding, here is an example: when p=3, AP13 {zero} and AP23 {Aes 1}. the updating of three latent representations sy. ej.. and ci is denoted
AP1s
{s)(13)=0 * s!, (13)=0 {1))))))))))))))))000))))))))))) Mess(s o7))
e/ (13)== cz
s: (23) = Up(s?, Σ
AP23
JEN;
e2 = e 3 (23) = cx
s = s(3) + (23). of (13) + (23), cf = c + 223)
The first equation is to update three latent representations according to AP13, while the second is to get the updating based on AP23. The final latent representations are obtained by summing up both updated ones. Here Up(-) and Mess(-) are common update and sassage-passing functions.
4.1.2 Routing Strategy
To find suitable propagation paths, threshold ((u.)) is created for each pair of hyper-nodes (u, v):
7 (u A max(Softman(APPw)) + (1-b) A) (SoofSftmzzAAP....
is predefined. (4)
Softmar(AP) normalizes weights of each type of propagation path regarding their hei values. By doing so, the propagation paths of each pair of hyper-noder will remain if their hr values greater than the corresponding threshold. Thus the updating process of (()) can be rewritten as
(P)) = EE ((iPpppH(") H()) u < p}
(5)
AP = ((hra. ver ||br wei 2 (u.P), Vhra E APup)
Finally, the learned student representation sf in H are used for the diagnosis, i.e, sf is used as S.
4.2 The GAT-based Exercise Learning Module and GAT-based Concept Learning Module
GAT-based Exercise Learning Module. This module is responsible for learning the exercise representation Ej E I the relation graph GR via L-layer GAT network (24). Firstly, the embedding of exercises and concepts in GR. obtained in the manner same as Eq. through two learnalle matrices W E Mxd and W E R i.e of = =x W,
WR. cf x WR.
Afterward, the GAT neural network is applied aggregate neighbor information learn the exercise representation. The aggregation process of (-th layer (1 < L) can be represented as
R(I) ΣxEN,,
R(U) R(-1) 300)
+ - R(L-1)
Ck
(6)
n(O) ml-1) _( NgE ( e
B()) B((-1) 1) k(m)
R(I-1)"
+ EmeNse
Cm
+ c
The first equation is aggregate the information of exercise ej neighbors Ne, to get its 1-th layer's latent representation e,
R(()) Rixd, while the second is to update the I-th layer's concept latent
R() representation c₁ E 1g from its exercise neighbors Nee and concept neighbors Nec aj
R()

---

R()) and og R() oK
are the I-th layer's attention matrices for exercise ey concept neighbors, concept C s exercise neighbors, and c ss concept neighbors. They can be obtained in the same manner, and can be obtained by
the k-thh row of oz R(I)
R(!) Softmax(Fec R(1-1) (1o) Ck ()) Ak E Nej- [kk
[ is the concatenation and F-o(.) is a fully connected (FC) layer mapping * d vectors to scalars.
The latent representation ej
R(O) and Cg R(O) refer to of and of and the I--th layer output ej as Ej. By disentangling the interaction data, the learned exercise representation Ej will be more robust against interaction noise.
R(L) is used
GAT-based Concept Learning Module. Similarly, this module obtains the concept representation C& E Rixd by applying L-layer GAT network to the dependency graph Gp. After obtaining the initial embedding c X x W by a learnable matrix W R xxd this module updates the I-th
layer's latent representation C₁ by C₂
D(I) D(I)
EmeNse Nee a D(i D(I-1) D(1-1) D(I) denotes the
k(m)
Cm
+ Ck ag
attention matrix, which can be computed as same as 77.
D(0) is equal to cß. and the L-th layer output Cg (LL) is used as C₁. If graph Gp is unavailable, Here C this module will directly take the initial embedding cf as C₁. By further disentangling, the learned representation C₃ may be robust against noise in student interactions some extent.
4.3 The Diagnosis Module
To effectively handle the obtained three types of representations, novel diagnostic function is proposed to predict the response riş of student s; got on exercise ej as follows:
hiimi = o (h.i hej)) h.s = (S Ck), haj Fej C
(8)
rig = e Qk haims))
ΣQ= Qx
where F.i(.). F.)()), and (-ime()) are three FC layers mapping a a d-dimensional vector to another one, and Qu is a binary vector in the k-th row of Q-matrix.
Here has IR. 1xd can be seen the student's mastery of each knowledge concept; while the obtaining of hej E Ixd aims to get the exercise difficulty of each concept; hammi is to measure the similarity between h2: and haj via a dot-product followed by an FC layer and a Sigmoid function a(-) [1], where a higher similarity value in each bit represents higher mastery indicating a higher probability of answering the related exercises; the last equation follows the idea of NCD to compute the overall mastery averaged over all concepts contained in exercise ej.
each concept, further
We can see that the proposed diagnostic function has as high interpretability as NCD, IRT, and MIRT.
Model Optimization. With the above modules, the proposed DisenGCD are trained by solving the following bilevel optimization problem through Adam 339
miu Leat (Deat w (ax), o)), s.t. w (a) argmin, Ltrasn (D ww, o);
where (-aa(-) and Lirain denote the loss on validation dataset Deal and training dataset Derain- denotes all model parameters, and a denotes the weights of learnable propagation paths AP in meta multigraph MG. The cross-entropy loss [40] is used for Loot and Lirain.
Experiments
This section answers the following questions: RQ1: How about the performance of DisenGCD compared to SOTA CDMs? RQ2: How about the DisenGCD's robustness against noise and the disentangled learning framework's effectiveness DisenGCD? RQ3: How about the effectiveness of the devised meta multigraph learning module? RQ4: How does the learned meta multigraph on target datasets look like, and how about their generalization on other datasets? addition, more experiments to validate the proposed DisenGCD's effectiveness are in the Appendix.
7

---

Table 1: Statistics of three datasets: ASSISTments, Math, and SLP.
Datasets
Students
Exercises
Concepts 123
Logs
Avg logs per student 67 60 38
# ASSISTments
4.163
17,746
278,868 118,348/ 57,244
# Math / SLP
1,967 / 1,499 1,686 907
61 33
Table 2: Performance comparison between DisenGCD and five CDM in terms of AUC, ACC, and RMSE values, obtained on ASSISTment and Math. Four dataset-splitting ratios were adopted, and the best result of each column one dataset highlighted.
Datatset
Ratio
40%//0%%500%
50%
60%//0%%/30%
70%:/1%%/20%
Method
ACC1 RMSEJ AUCT ACC1 RMSEJ AUC1 ACC1 RMSEL AUC1 ACC1 RMSEJ AUC1
DINA
0.6388 0.4931 0.68740.6503 0.4862 0.4978 0.4820 0.7071 0.70710.6623 0.6623 0.4787 0.7126
MIRT
0.6954 0.4740 0.72540.7015 0.4689 0.73580.7090 0.4624 0.74690.71 10 0.4617 0.7514
ASSIST
NCD
0.7070
0.4443 0.7344 0.4370 0.7423 7.33 0.4365 0.75520.7285 0.4298 0.7603
ments
ECD
0.7154 0,4373 0.73620.7130 0.4373 0.74320.7274 0,4329 0.75430.7297 0.4296 0.7599
RCD
0.7232
0.4311 0.754600.253 0.4285 0.7605 0.4262 0.7663 0.4245 0.7687
Di.77GG
0.4255 0.7635 0.4238 0.76770.7335 0.4219 0.77230.7334 0.4209 0.7746
DINA
0.6691 0.4715 0.71170.6775
0.4674 0.7199/0.6813 0.4633 0. 0.72220.6812 0.4635 0.7231
MIRT
0.7229
0.4335 0.74270.7227
0.4299 0.74970.7279 0,4291 0.74790.7340 0.4256 0.7542
NCD
0.7394
0.4157 0.7624 0.4119 0.7660 0.4109 0.77060.7447 0.4084 0.7756
Math
ECD
0.7335
0.4154 0.76150.7424 0.413 0.7657 0.4114 0.76930.7484 0.4087 0.7761
RCD
0,7446
0.4100 0.77240.7489 0.4074 0.7551 0.4078 O. 0.78060.7534 0.4034 0.7866
Dis77G 0.4076 0.7802007513
0.4052
0.7832007527 0.4039 0.78670.7582 0.4004 0.7932
5.1 Experimental Settings
Datasets. To verify the DisenGCD's ' effectiveness, we conducted experiments on two public datasets ASSISTments [6] and SLP [14), and one private dataset Math, whose statistics are in Table [ Note that ASSISTments and Math (16] were used for most experiments to answer RQ1 RQ3, and SLP was used to answer RQ4. More details of these datasets are presented in Appendix A.2
Comparison CDMs and Metrics. To verify the effectiveness of DisenGCD, five SOTA CDMs were compared, including traditional CDMs DINA and MIRT, NN-based CDMs NCD and ECD, and GNN-based RCD. Here SCD not compared due to the failed run of its provided source code Three metrics were adopted to measure the performance of all CDMs (37, including area under the cure (AUC) 23 accuracy (ACC) 22 and rool mean square error (RMSE) [46]
Parameter Settings. For the DisenGCD model, its dimension was set to the number of concepts, P in the meta multigraph, the number of layers L in GAT, and À were set to 5, 2. and 0.8. For its training, the learning rate and batch size were lc-4 and 256. For comprehensive comparisons [7]. four splitting ratios 400%/10%/50% 50%/10%/40%, 60%//0%//30 adopted to get training, validation, and testing datasets. All compared CDMs followed the settings in their original papers, and all experiments were executed on NVIDIA RTX4090 GPU.
and 70%//0%%/2%4 were
5.2 Overall Performance Comparison (RQ1)
To address RQL, the DisenGCD was compared with DINA, MIRT, NCD, ECD, and RCD on ASSISTments and Math datasets. Table 2 summarizes their performance in terms of AUC, ACC, and RMSE obtained under four dataset-splittng settings, where the best result of each column on one dataset was highlighted in bold. Besides, Table in the Appendix compares DisenGCD with three SOTA CDMs, including SCD 28 KSCD [5]. and KaNCD 26).
We can observe from the results in both tables that, DisenGCD holds better performance than all compared CDMs. Besides, we can also obtain two observations from Table 2 Firstly, GNN-based CDMs (RCD and DisenGCD) hold significantly better than NN-based CDMs (NCD and ECD), indicating the importance of learning representations through graphs, and DisenGCD outperforming RCD validates the superiority of DisenGCD's graph learning manner. Secondly, as the ratio changes,
8

---

Table 3: Performance of DisenGCD, RCD, and its four variants on ASSISTments dataset.
DisenGCD (Is+Rec)
DisenGCD (Ise+Re)
DisenGCD (Isc+Re)
Metric
RCD
DisenGCD(I)
DisenGCD
ACC1
0.7291
0.7331
0,7321
0.7301
0.7333
0.7335
RMSEL
0.4262
0.4235
0.4259
0,4235
0.4231
0.4219
AUC1
0.7663
0.7678
0.7701
0.7678
0.7685
0.7723
DisanGCD(raivee
DisenGDD(mp)
DismGCD(mg)
DisenGCD
0.75
DisenGCO
DisenGCD
0.00
RCD
ACC
0.70
0.75
0.78
0.65
0.74
0.76
0.60
0.75
0.14
0.72
Internction Nosic Radio
Interaction Nodie Ratlo
30%
126
Math
(a)
(b)
Figure 3: (a): Performance RCD and DisenGCD under different noises. (b): Effectiveness the
meta multigraph learning module.
the performance change of DisenGCD and RCD is much smaller than NCD and ECD, which signifies DisenGCD and RCD are more robust to different-sparsity data. To validate its sparsity superiority, more experiments are summarized in Appendix B.1
5.3 Effectiveness of Disentangled Learning Framework of DisenGCD (RQ2)
To investigate DisenGCD's ' robustness against interaction noise, we conducted robust experiments on the ASSISTments dataset under the ratio of 60%//0%%/30% where certain amount of noise interactions were added to each student in the training and validation datasets. Figure 3a presents the ACC and AUC of DisenGCD and RCD under noise data of different percentages. As the noise data increases, the performance leading of DisenGCD over RCD becomes more significant, which reaches maximal when noise data of 50% was added. That demonstrates DisenGCD is more robust to student noise interactions than RCD, attributed to the disentangled learning framework of DisenGCD. Similar observations can be drawn from more experiments other two datasets in Appendix [B.2
To further analyze the framework effectiveness, four variants DisenGCD were created: Disen- GCD(I) refers to learning three representations only the interaction graph Gz; DisenGCD(ls+Red Rec) refers to learning student representation on Gz, but learning exercise and concept ones on the relation graph GR: DisenGCD(lse+RRo refers to learning student and exercise representations learning concept one on GR; while DisenGCD(Isc++R) learns student and concept representations on Gr but learns exercise variants on ASSISTments. As can be seen, the comparison between DisenGCD(I) and other variants indicates learning three representations in disentangled graphs is more effective than in one unified graph, especially with (extiDiseenCCD((s-Ree)) the comparison between DisenGCD(Is++ Rec) and DisenGCD indicates learning three representations in three disentangled graphs is more effective, further validating the above conclusion. Finally, we can conclude that the proposed disentangled learning framework effective in enhancing DisenGCD's performance and robustness.
GI but
GR. Table comparrs the performance of RCD, DisenGCD, and its four
5.4 Effectiveness of Meta Multigraph Learning Module in DisenGCD (RQ3)
In Table 3 both DisenGCD(I) and RCD learn three types of representations one unified graph, but DisenGCD(1) utilizes the meta multigraph aggregator. Therefore, the comparison between DisenGCD(I) and RCD proves the devised meta multigraph learning module is effective in improving the model performance some extent.
To further validate this, we created three variants DisenGCD: DisenGGCD(navve) refers to the meta multigraph module replaced by the naive embedding (i.c.. Sj equal to s; in Eq.(h); Dis-

---

0.46
85%
H
H
H3
H
H(S)
ASSISTments
0.45
AUC
RMSE
80%
0.44
AUC
RMSE
75%
0.43
H
H
H
H
H
Math
Ake
ke
05
0.42
70%
05
0.41
65%
ES
A1
A2
A3
M1
M2
M3
(b)
Figure 4: (a): Visualization of learned meta multigraph two datasets. (b): Generalization validation of six learned meta multigraphs.
enGCD(mp) refers to predefining the meta multigraph module's propagation paths according to DisenGCDDmgg refers to this module's meta multigraph replaced by meta graph. Fig- ure 3b presents the AUC and ACC values of DisenGDCN and its variants on two datasets under the ratio 60% 10%/30% As can be seen, the devised meta multigraph learning module enables DisenGDCN to hold significantly better performance than the naive embedding-base -baee variant. The comparison results between DisenGCD and DisenGCD(mg) as well as DisenGCD(mp) validates the effectiveness of using meta multigraph and the automatically learned propagation paths (i.e., learned propapppppaai
HAN 30
meta multigraph). Thus, the effectiveness of the devised meta multigraph module can be validated.
5.5
Visualization and Effectiveness Learned Meta Multigraph (RQ4)
The comparison between DisenGCD(mp) and DisenGCD has revealed the effectiveness of the learned meta multigraph on the target dataset. Therefore, an intuitive doubt naturally emerged: the learned meta multigraph still effective on other datasets? Before solving this, Figure 44 the structure visualization of two learned meta multigraphs. We can see two learned meta multigraphs hold two distinct structures. That may be because two datasets between exercises and concepts, where exercises in Math contain only one concept while exercises in ASSISTments may contain more than concept. Thus, another doubt naturally emerged: Is the learned meta multigraph on the target dataset ineffective on different type dataset?
bit different regarding the relations
To solve the doubts, we applied six learned meta multigraphs (A1, A2, A3, M1, M2, and M3) to the SLP. A1-A3 and M1-M3 were learned by DisenGCD three times on ASSISTments and Math. SLP dataset is similar to Math, whose exercises only contain concept. Figure 4b summarizes the results of RCD and six DisenGCD variants that utilize six given meta multigraphs. As can be seen, DisenGCD's performance under M1-M3 is promising and better than RCD, while DisenGCD under A1-A3 performed poorly. The observation can answer the above doubts to some extent: meta multigraphs learned by DisenGCD may be effective when the datasets be applied are similar to target datasets.
In addition to the above four experiments, more experiments were executed to validate the devised diagnostic functions, analyze the DisenGCD's parameter sensitivity, analyze its execution efficiency, and investigate the effectiveness of the employed GAT modules in Appendix B.3 Appendix B.4 Appendix B.5 and Appendix B.6
Conclusion
This paper proposed meta multigraph-assistee disentangled graph learning framework for CD, called DisenGCD. The proposed DisenGCD learned student, exercise, and concept representations on three disentangled graphs, respectively. It devised meta multigraph module to learn student representation and employed two common GAT modules to learn exercise and concept representations. Compared SOTA CDMs three datasets, the proposed DisenGCD exhibited highly better performance and showed high robustness against interaction noise.
10

---

Acknowledgements
This work was supported in part by the National Key R&D Program of China in part by the National Natural Science Foundation of China (No.62302010, No. No.62303013, No.62107001, 6o.62006053, No.61876162. No.621 36008, No. No.62276001, No.U21A2055 12), in part by China Postdoctoral Science Foundation (No.2023M740015), in part by the Postdoctoral Fellowship Program (Grade B) of China Postdoctoral Science Foundation 2002222202400,, and in part by the Anhui Province Key Laboratory of Intelligent Computing and Applications (No. AFZNIS2024KFO1)
References
[1] Ashton Anderson, Daniel Huttenlocher, Jon Kleinberg, and Jure Leskovec. Engaging with massive online courses. In Proceedings of the 23rd international conference on World wide web, pages 687-698, 2014.
[2] Joseph Beck. Difficulties in inferring student knowledge from observations (and why you should care). In Proceedings of the 13th International Conference of Artificial Intelligence in Education, pages 21 21-30, 2007.
[3] Song Cheng. Qi Liu, Enhong Chen, Zai Huang, Zhenya Huang, Yiying Chen, Haiping Ma. and Guoping Hu. Dirt: Deep learning enhanced item response theory for cognitive diagnosis. In Proceedings of the 28th ACM international conference on information and knowledge management, pages 2397-2400, 2019.
[4] Jimmy De La Torre. Dina model and parameter estimation: A didactic. Journal of educational and behavioral statistics, 34(1):1155 15--130 2009.
[5] Yuhui Ding, Quanming Yao, Huan Zhao, and Tong Zhang, Diffing: Differentiable meta graph search for heterogeneous graph neural networks. In Proceedings of the 27th ACM SIGKDD Conference Knowledge Discovery & Data Mining, pages 279-288, 2021.
[6] Mingyu Feng, Neil Heffernan, and Kenneth Koedinger. Addressing the assessment challenge with an online system that tutors as it assesses. User Modeling and User-adapted Interaction, (933):2432666 2009.
[7] Weibo Gao. Qi Liu, Zhenya Huang, Yu Yin, Haoyang Bi, Mu-Chun Wang, Jianhui Ma, Shijin Wang, and Yu Su. Red: Relation map driven cognitive diagnosis for intelligent education systems. In Proceedings of the 44th international ACM SIGIR conference on research and development information retrieval, pages 501-510, 2021.
[8] Zhenyu Han, Fengli Xu, Jinghan Shi, Yu Shang, Haorui Ma, Pan Hui, and Yong Li. Genetic meta- structure search for recommendation on heterogeneous information network. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pages 455 464, 2020.
[9] Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, YongDong Zhang, and Meng Wang. Lightgen: Simplifying and powering graph convolution network for recommendation. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '20, page 639-648, New York, NY, USA, 2020. Association for Computing Machinery.
[10] Chao Li, Hao Xu, and Kun He. Differentiable meta multigraph search with partial message propagation heterogeneous information networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 8518 8526, 2023.
[11] Qi Liu, Shiwei Tong, Chuanren Liu, Hongke Zhao Enhong Chen, Haiping Ma, and Shijin Wang. Exploiting cognitive structure for adaptive learning. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 627-635, 2019.
[12] Sannyuya Liu, Qing Li, Xiaoxuan Shen, Jianwen Sun, and Zongkai Yang. Automated dis- covery of symbolic laws governing skill acquisition from naturally occurring data. Nature Computational Science, pages 1-12, 2024.

---

[13] Frederique M Lord. Applications of Item Response Theory to Practical Testing Problems. LAWRENCE ERLBAUM ASSCCIAATES, 1980.
[14] Yu Lu, Yang Pian, Ziding Shen, Penghe Chen, and Xiaoqing Li. SLP: A multi- coaliiddmmesionall and consecutive dataset from K- K-12 education. In Proceedings of the 29th International Conference on Computers in Education Conference, volume 1, pages 261 261-266. Asia-Pacific Society for Computers Education, 2021.
[15] Haiping Ma, Manwei Li, Le Wu, Haifeng Zhang, Yunbo Cao, Xingyi Zhang, and Xuemin Zhao Knowledge-sensee cognitive diagnosis for intelligent education platforms. In Proceedings of the st ACM International Conference on Information & Knowledge Management, pages 1451-1460, 2022.
[16] Haiping Ma, Changgian Wang, Hengshu Zhu, Shangshang Yang, Xiaoming Zhang, and Xingyi Zhang. Enhaneing cognitive diagnosis using un-interacted exercises: A collaboration-awe awaree mixed sampling approach. In Proceedings of the AAAI Conference on Artificial Intelligence,
volume 38, pages 8877-8885, 2024.
[17] Haiping Ma, Yong Yang, Chuan Qin, Xiaoshan Yu, Shangshang Yang, Xingyi Zhang, and Hengshu Zhu, Hd-kt: Advancing robust knowledge tracing via anomalous learning interaction detection. In Proceedings of the ACM Web Conference 2024, pages 4479-4488, 2024.
[18] Haiping Ma, Yi Zeng, Shangshang Yang. Chuan Qin, Xingyi Zhang, and Limiao Zhang. novel computerized adaptive testing framework with decoupled learning selector. Complex & Intelligent Systems, pages 12 2023.
[19] Jianxin Ma, Peng Cui, Kun Kuang, Xin Wang, and Wenwu Zhu. Disentangled graph convolu- tional networks. International Conference on Machine Learning, 2019.
[20] Mark D. Reckase. Multidimensional Item Response Theory. Springer New York, 2009.
[21] Yu Su, Zeyu Cheng, Jinze Wu, Yanmin Dong, Zhenya Huang, Le Wu, Enhong Chen, Shijin Wang, and Fei Xie. Staahhbased cognitive diagnosis for intelligent tutoring systems. Knowledge- Based Systems, 253:109547, 2022.
[22] Jianwen Sun, Fenghua Yu, Sannyuya Liu, Yawei Luo, Ruxia Liang, and Xiaoxuan Shen. Adversarial bootstrapped question representation learning for knowledge tracing. In Proceedings of the 31st ACM International Conference on Multimedia, pages 8016-8022 2023.
[23] Jianwen Sun, Fenghua Yu, Qian Wan, Qing Li, Sannyuya Liu, and Xiaoxuan Shen. Interpretable knowledge tracing with multiscale state representation. In Proceedings of the ACM on Web Conference 2024, pages 3265-3276, 2024.
[24] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. arXiv preprint arXiv: 17.0. 10903, 2017.
[25] Fei Wang, Qi Liu, Enhong Chen, Zhenya Huang, Yuying Chen, Yu Yin, Zai Huang, and Shijin Wang. Neural cognitive diagnosis for intelligent education systems. In Proceedings of the AAAI conference artificial intelligence, volume 34, pages 6153-6161, 2020.
[26] Fei Wang, Qi Liu, Enhong Chen, Zhenya Huang. Yu Yin, Shijin Wang, and Yu Su. Neuraled: general framework for cognitive diagnosis. IEEE Transactions on Knowledge and Data Engineering, 35(8):8312-8327, 2022.
[27] Hao Wang, Tong Xu, Qi Liu, Defu Lian, Enhong Chen, Dongfang Du, Han Wu, and Wen Su. Mone: An end-to-end framework for learning multiple conditional network representations of social network. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery data mining, pages 1064-1072, 2019.
[28] Shanshan Wang, Zhen Zeng, Xun Yang, and Xingyi Zhang, Self-supervised graph learning for long-tailed cognitive diagnosis. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 110-118, 2023.
12

---

[29] Xiang Wang, Hongye Jin, An Zhang, Xiangnan He, Tong Xu, and Tat-Seng Chua. Disentangled graph collaborative filtering. Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, 2020.
[30] Xiao Wang, Houye Ji, Chuan Shi, Bai Wang, Yanfang Ye, Peng Cui, and Philip S Yu. Het- erogeneous graph attention network. In The world wide web conference, pages 2022-2032, 2019.
[31] Xinping Wang, Caidie Huang, Jinfang Cai, and Liangyu Chen. Using knowledge concept aggregation towards accurate cognitive diagnosis. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pages 2010-2019, 2021.
[32] Yifan Wang, Suyao Tang, Yuntong Lei, Weiping Song, Sheng Wang, and Ming Zhang. Disenhan: Disentangled heterogeneous graph attention network for recommendation. Proceedings of the 29th ACM International Conference on Information & Knowledge Management, 2020.
[33] Jiahao Wu, Wenqi Fan, Jingfan Chen, Shengcai Liu, Qing Li, and Ke Tang. Disentangled contrastive learning for social recommendation. New York, NY, USA, 2022, Association for Computing Machinery.
[34] Zhengyang Wu, Ming Li. Yong Tang, and Qingyu Liang. Exereise recommendation based on knowledge concept prediction. Knowledge-Basse Systems, 210: 11641,,48, 2020.
[35] Jiashu Zhao Dawei Yin Xubin Ren, Lianghao Xia and Chao Huang. Disentangled contrastive collaborative filtering. Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2023.
[36] Shangshang Yang, Haiping Ma, Ying Bi, Ye Tian, Limiao Zhang, Yaochu Jin, and Xingyi Zhang. An evolutionary multi-objective neural architecture search approach to advancing cognitive diagnosis intelligent education. IEEE Transactions on Evolutionary Computation, 2024.
[37] Shangshang Yang. Haiping Ma, Cheng Zhen, Ye Tian, Limiao Zhang, Yaochu Jin, and Xingyi Zhang. Designing novel cognitive diagnosis models via evolutionary multi-objective neural architecture search. arXiv preprint arXiv:2307.04423 2023.
[38] Shangshang Yang. Linrui Qin, and Xiaoshan Yu. Endowing interpretability for neural cognitive diagnosis by efficient kolmogorov- networks. arXiv preprint 1439992405.1339, 2024.
[39] Shangshang Yang, Xiangkun Sun, Ke Xu, Yuanchao Liu, Ye Tian, and Xingyi Zhang. Hy- brid architecture-biat basee evolutionary robust neural architecture search. IEEE Transactions Emerging Topics in Computational Intelligence, 2024.
[40] Shangshang Yang, Ye Tian, Cheng He. Xingyi Zhang, Kay Chen Tan, and Yaochu Jin. A gradient-guided evolutionary approach training deep neural networks. IEEE Transactions on Neural Networks and Learning Systems, (3(9):4861-4477 2021.
[41] Shangshang Yang, Ye Tian, Xiaoshu Xiang, Shichen Peng, and Xingyi Zhang. Accelerating ovolutionary neural architecture search via multifidelity evaluation. JEEE Transactions on Cognitive and Developmental Systems, 14(4): 2022.
[42] Shangshang Yang, Haoyu Wei, Haiping Ma. Ye Tian, Xingyi Zhang, Yunbo Cao, and Yaochu Jin. Cognitive diagnosis- personalized exercise group assembly via multi-objectivy evolutionary algorithm. IEEE Transactions on Emerging Topics in Computational Intelligence, 2023.
[43] Shangshang Yang, Xiaoshan Yu, Ye Tian, Xueming Yan, Haiping Ma, and Xingyi Zhang. Evolutionary neural architecture search for transformer in knowledge tracing. In Thirly-seventh Conference Neural Information Processing Systems, 2023.
[44] Shangshang Yang, Cheng Zhen, Ye Tian, Haiping Ma, Yuanchao Liu, Panpan Zhang, and Xingyi Zhang. Evolutionary multi-objective neural architecture search for generalized cognitive diagnosis models. In 2023 5th International Conference on Duta-driven Optimization of Complex Systems (DOCS), pages 1-10. IEEE, 2023.
13

---

[45] Mingjia Yin, Hao Wang, Wei Guo. Yong Liu, Suojuan Zhang, Sirui Zhao, Defu Lian, and Enhong Chen. Dataset regeneration for sequential recommendation. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 3954-3965, 2024.
[46] Mingjia Yin, Chuhan Wu, Yufei Wang, Hao Wang, Wei Guo, Yasheng Wang, Yong Liu, Ruiming Tang, Defu Lian, and Enhong Chen. Entropy law: The story behind data compression and Ilm performance. arXiv preprint (XX1::447.06644 2024.
[47] Xiaoshan Yu, Chuan Qin, Dazhong Shen, Shangshang Yang, Haiping Ma, Hengshu Zhu, and Xingyi Zhang. Rigl: A unified reciprocal approach for tracing the independent and group learning processes. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 4047-4058, 2024.
[48] Yuqiang Zhou, Qi Liu, Jinze Wu, Fei Wang. Zhenya Huang, Wei Tong, Hui Xiong. Enhong Chen, and Jianhui Ma. Modeling context-awar features for cognitive diagnosis student learning. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 2420-2428, 2021.
14

---

Appendix A More details about Proposed Method and Datasets
A.1
Notations Summary
For convenient reading and understanding, we have summarized all the notations discussed in the paper in the following two tables. Table 4 summarizes all the notations in the three disentangled graphs, while Table describes the notations of the meta multigraph.
Table 4: Notation of disentangled graphs
Notation
Description
Gz
the cuudent-exercise-ee concept interaction graph
8;
initial embedding of student in Gz
ef
initial embedding of exercise in Gz
c/
initial embedding of concept in Gz
S;
the final representation of the updated student node
OR.
the exercise-concept relation Graph
e)
initial embedding of exercise in GR
Cj
initial embedding of concept in GR
Ej
the final representation of the updated exercise node
GD
the concept dependency graph
cg
initial embedding of concept in GD the final representation of the updated concept node
C
Table 5: Notation of meta multigraph.
Notation
Description
Mg
meta multigraph
H())
the p-th hyper node meta multigraph
AP
edges between hyper podespprooagatioon paths)
APa
edges between (u)) and (v))
HR
the set of propagation path types the propagation path from student to exercise the propagation path from exercise to student the propagation path from concept exercise the propagation path from exercise to concept the propagation path from concept to concept
Ase
Aes
Ake
Aek
Agk
I
information is not updated to propagate to the next hyper-node
zero
no information propagation between two hyper nodes the weight of each type propagation path in AP... the threshold of candidate propagation paths in.1P Pae
wei
p(u,))
15

---

Table 6:
Performance comparison of recent CDMs (SCD, KaNCD, KSCD) and DisenGCD on the
Math dataset.
Method/Metric
SCD
KaNCD KSCD DisenGCD
ACC
0.7546
0.7519
0.7549
0.7582
RMSE
0.4025
0,4050
0.4040
0.4004
AUC
0.7882
0.7854
0.7890
0.7932
A.2 Statistics of Datasets
We evaluated our method three real-world datasets: ASSISTments, Math and SLP, which both provide student-exercise interaction records and the excreise-knowledge concept relational matrix.
ASSISTments a public dataset collected by the assistant online tutoring systems the 2009-2010 acadaemic year.
Math a private dataset collected by a well-known online learning platform that contains math practice records and test records for elementary and middle school students.
SLP is another publicdata set that collects data learners' performance in eight different subjects over three years of study, including maths, English, physics, chemistry, biology. history and geography.
Notably, in both the Math dataset and the SLP dataset, they provide relationships between concepts. For these three datasets, we filtered students with fewer than 15 answer records ensure there was enough data for the learning of the model. We compared our model with five previous diagnostic models, including two classic models based on educational psychology. DINA 4 and MIRT (2)) two neural network-basee models, NCD 25 and ECD
48 and a graph- diagnostic model,
RCD 77.
DINA 4 is classical CDM, which uses a binary variable to characterize whether and whether a student has mastered specific concept.
MIRT (00 is extension of the irt model, which uses multidimensional vectors characterize students' abilities and the difficulty of exercises, and uses linear functions to model the interactions.
NCD 25 to model higher-order student-exercise complex interactions.
of the recent CDMs based deep learning, which uses neural networks
ECD 48 incorporates the impact of the educational environment in students' historical answer records into the diagnostic model to achieve diagnostic enhancement. Here due to the lack of educational background, we use random initialization vectors to characterize cognitive states.
is one of the most advanced models, which introduces the relationship between concepts into cognitive diagnosis and models the relationship through graph structure.
RCD 7
Appendix B Additional Experiments
B.1
Experiments Dataset with Different Levels of Sparsity
As shown in Section 5.2 the proposed DisenGCD shows better performance than RCD even when the splitting ratios change. which indicates the better robustness of DisenGCD to different-sparsity data.
To further verify whether the proposed DisenGCD is robust against sparse data (i.e., missing data or sparse interaction patterns), we conducted corresponding experiments on ASSISTments and Math datasets under the splitting setting of 60%/ 10% / 30%. In the experiments, for each dataset, we randomly deleted 5%, 10%, and 20% of the students' answer records (i.e., the interaction data) in the training sets, respectively, thus providing three variant training sets; the proposed DisenGCD
16

---

Table 7: Performance comparison between DisenGCD, NCD, and RCD under sparse interaction data.
5% sparsity
10% sparsity
20% sparsity
Datasets
Models
ACC
RMSE
AUC
ACC
RMSE
AUC
ACC
RMSE AUC
NCD
0.7216 0.4337 0. 0.7487 0.7170 0.4365 0.7455 0.7146 0.4471 0.7398
ASSISTments
RCD
0.7264 0.4273 0.7636 0.7245 0.4295 0.7610
0.7241 0.4289 0.7569
DisenGCD
0.7282 0.4248 0.7676 0.7278 0.4254 0.7660
0.7273 0.4256 0.7631
NCD
0.7399
0.4122 0.7674 0.7383
0.4148 0.7651 0.7352 0.4162 0.7629
Math
RCD
0.7496 0.4071 0.7790 0.7481
0.4104 0.7781 0.7476
0.4075 0.7771
DisenGCD
0.7526 0.4030 0.7891 0.7503
0.4049 0.7832
0.7485
0.4058 0.7812
DisenGCD
RCD
0.60
0.7
ACC
RMSE
AUC
0.7
0.55
0.6
0.6
Math
0.5
0.50
0.5
0.4
0.45
0.4
0.3
0.40
10
30
50
30
50
1
10
30
50
NosieRatio Ratio
Nosie Rabio
Nosee Ratio
0.75
ACC
0.50
RMSE
0.80
AUC
0.70
0.48
0.75
0.46
0.70
SLP
0.65
0.44
0.65
0.60
0.42
0.60
0.55
0.55
10
10
30
10
30
50
Nose Raboo
Nosee Kabio
Nose Rabo
Figure Futher Robustness Validation of DisenGCD.
was compared with NCD and RCD, and their overall performance comparisons were summarized in Table B.I
As can be seen, under which deleting ratio, the proposed DisenGCD exhibits the best performance, and its performance does not drop very significantly when the deleting ratio increases. Even when deleting 20% of students' interaction data, DisenGCD still achieves AUC value of 0.7812, which is greatly higher than NCD and RCD. It demonstrates that the proposed DisenGCD is effective on sparse interaction data, which is mainly attributed to the devised melt-guut learning module. That module enables the DisenGCD to access and use lower-order exercise latent representations, thus providing more accurate and robust students' representations, especially when partial data is lacking.
In summary, the proposed DisenGCD is effective and more robust against sparse interaction data, which benefits from the devised meta- multiguutgg learning module, and thus the effectiveness of the meta-multigtaph learning module can be indirectly demonstrated.
B.2 More Experiments for Robustness Validation
we have validated the robustness of the proposed DisenGCD against interaction noise the ASSISTments dataset. To further show its robustness superiority, we executed the same experiments on other two datasets, i.e., Math and SL.P, under the same settings, where certain ratio of noise interactions were added to each student in the training and validation datasets. Figure 5
In Section 5.3
17

---

Table 8: Validation of the devised diagnostic function on Math dataset.
Splitting Ratios
70%/10%/20%
60%%/10%/30%
50% ACC
40%/10%/50%
Diagnostic Functions
ACC
RMSE AUC
ACC
RMSE AUC
RMSE AUC
ACC
RMSE AUC
MIRT
0.7340 0.4256 0.7542 0,4291 0.7479 0.4299 0.7497 0.7229 0,4335 0.7427
NCD
0.7447 0.4084 0.7756
0,4109 0.7706 0.7424 0.4119 O. 0.7660 0.7394 0,4157 0.7604
RCD
0.7534 0.4034 0.7866
0.4078 0.7806 0.7489 0.4074 0.7751 0.7446 0,4100 0.7724
DisenGCCD-MIRT
0.7396 0.4095 0.7338 0.4103 0.7705 0.7378
0.4118 0.7689 0.7367 0,4138 0,7656
DisenGCD-NCD
0.7501 0.4057 0.7796 0.7464
0.4076 0.7763
0.7430
0,4099 0.7714 0.7424 0,4127 0.7658
DisenGCD-RCD
0.7548 0.4045 0.7507 0,4066 0.7828 0.7482
0.4073 0. 0.7470 0,4098 0.7767
DisenGCD
0.7582 0.4004 0.7932 0.4039 0.7867
0.7513 0.4052
0.7832
0.7479
0.4076 0.7802
presents the results of DisenGCD and RCD obtained under different proportions of noise: the orange polyline denotes DsienGCD's results and the blue polyline denotes RCD's results.
As can be seen, the proposed DisenGCD always performs better than RCD in both datasets under different noise ratios. Besides, as the noise ratio increases, the performance leading of the proposed DisenGCD to RCD becomes more significant. That further demonstrates the proposed DisenGCD more robust against student interaction noise and thus validates the effectiveness of the proposed disentangled graph learning framework.
B.3 Validation of the Devised Diagnostic Function
In this paper, in addition the disentangled learning framework and the devised meta multigraph module, we also designed a novel diagnostic function to adopt them. To validate its effectiveness, we compared it with the diagnostic functions of MIRT, NCD, and RCD the Math dataset, where four splitting settings were considered. Table g summarizes their overall performance regarding AUC, ACC, and RMSE.
We can get the following two observations. Firstly, the comparisons of MIRT and DisenGCD- as well as RCD and DisenGCD- RCD show that the representations NCD and DisenGCD-NV learned by the proposed DisenGCD are effective. Secondly, the performance superiority of DisenGCD other variants indicates the devised diagnostic function is effective. To sum up, the effectiveness of the devised diagnostic function is validated.
B.4 Hyperarameter Sensitivity Analysis
For the proposed DisenGCD, there is an important hyperparameter, i.e., the number of hyper-nodes in the meta multigraph. To investigate its influence on theDisenGGDD, the hyper-node number was set to 4, 5, 6, and 7, respectively, and we executed the experiments on the ASSISTments and SLP datasets. Figures 6 and pplot the results regarding AUC and RMSE.
As can be seen in both two datasets, the DisenGCD can achieve optimal results in terms of RMSE and AUC, when the number of hyper-nodes set to 5. When the number of hyper-nodes less than 5, the meta multigraph contains too few propagation paths, adversely affecting the learning of student representations. This results in suboptimal performance compared to the scenario where the number of hyper-nodes is 5. Conversely, when the number of hyper-nodes is greater than 5, there is an abundance of propagation paths, making the aggregated information overly complex and hindering effective student representation learning, Besides, more hyper-nodes cause more computational complexity. Therefore, this paper set the number of hyper-nodes to 5 for the proposed disenGCD.
Furthermore, we aim to explore the influence of the number of hyper-nodes on the robustness of the proposed DisenGCD. Therefore, we also executed the experiments on Math datasets under different ratios of noise interaction( I (5.30% Figures 8 plot the results regarding AUC and ACC. As can be seen, when the number of hyper nodes is equal to 5, the robustness of DisenGCD the most promising, whose performance under different ratios of noises is balanced better than RCD,
18

---

num=4
num=5
num 6
num=7
AUC
RMSE
0.426
0.770
0.425
0.766
0.424
0.762
0.423
0.422
0.758
epoch
epoch
8
10
12
14
16
18
20
8
10
12
14
16
18
20
Figure 6: The impact of the number of hyper nodes in the meta-multigrahh on the ASSISTments dataset
num =4
num=5
num=6
num=7
AUC
RMSE
0.780
0.416
0.770
0.412
0.760
0.408
0.755
epoch
0.404
epoch
8
10
12
14
16
18
20
6
8
10
12
14
16
18
20
Figure 7: The impact of the number of hyper nodes in the meta-multigraph on the Math dataset
B.5 Efficiency Analysis DisenGCD
To investigate the computational efficiency of the proposed approach, we have compared it with RCD ASSISTments and Math datasets in terms of model inference time and training time. Tarsen]sresentts the overall comparison, where the time the proposed DisenGCD under different hyperparameters P is also reported.
As can seen, DisenGCD's inference time better than RCD's. This indicates that the proposed DisenGCD is more efficient than RCD, further signifying that it is promising to extend DisenGCD to
19

---

RCD
DCDe))CD(4)
Disen6Coj4
DisenGCD(S)
Disen6C3/31
DisenGCo1a
DiserGCD(6)
Disen6co111
DiscnGCD(7)
03
ADC
0.-
0.4
17
30%
50%
Figure 8: Robustness of DisenGCD under different P to different ratios of interaction noise (on the Math dataset).
Datasets
Models
Inference time (seconds)
RCD
0,07829
ASSISTments
DisenGCD
0.01661
RCD
0.10888
Math
DisenGCD
0.00791
ASSISTments
Math
Models
RMSE
AUC
Training time(s)
RMSE
AUC
Training time(s)
RCD
0.4245
0.7687
27161
0.4078
0.7806
22164
DisenGCD(4)
0.4224
0.7714
17880
0.404
0.7857
3045
DisenGCD(5)
0,4217
0.7722
18135
0.4039
0.786
3346
DisenGCD(6)
0.4223
0.7711
18924
0.4049
0.7836
3675
DisenGCD(6)
0.4223
0.7719
20004
0.404
0.7856
4681
Table 9: Upper: Inference time comparison between RCD and DisenGCD. Lower: Performance and computational efficiency (regarding training runtime seconds) of DisenGCD under different (equal to 4, 5, 6, and 7).
dynamic CD. It can be seen from Table Ill:Lower: although a larger P will make DisenGCD take more time to train the model, the proposed DisenGCD achieves the best performance on both two datasets when P=5 and its runtime does not increase too much, which much better than RCD.
Method/Metric
DisenGCD(GCN)
DisenGCDDGraphhagee)
DisenGCD(HAN)
DisenGCD
ACC
0.7428
0.7543
0.7435
0.7582
RMSE
0.4112
0.402
0.411
0.4004
AUC
0.7639
0.7881 I
0.7647
0.7932
Table 10:
Performance comparison of variants of DisenGCD using different GNNs, and DisenGCD
on the Math dataset
20

---

B.6 Ablation Experiments on Graph Representation Learning
To verify the superiority of the employed GAT on the proposed DisenGCD, three GNNs (GCN, GraphSage, and HAN) are used to replace the GAT in the proposed DisenGCD, which are termed DisenGCD(GCN), (isanGCD(GraphSugee) and DisenGCD(HAN), respectively. Then, we compared the three approaches with the proposed DisenGCD on the Math dataset. The comparison of them presented in Table T0 As shown in Table T0 the GCN-based DisenGCD achieves the worst performance, followed by DisenGCD(HAAN, The GraphSSgg-bbssd DisenGCD holds competitive performance to yet still worse than the proposed DisenGCD (based on GAT).
The above results show the GAT is a suitable and optimal choice among these four GNNs for the proposed DisenGCD, but we think the GAT may not be the best choice because there exist other types of GNNs that can be integrated with DisenGCD, showing better performance.
In summary, it is reasonable and effective for the proposed DisenGCD adopt the GAT to learn the representations.
Appendix C Limitation Discussion
There are still some limitations in the proposed DisenGCD.
High Computational Complexity and Poor Scalability. The complexity of DisenGCD consists of three parts: the aggregation of P-hypemode meta multigraph on N M + K- node interaction graph (average A₁ neighbors), L-layer GAT aggregation on the M + K- node relation graph (average Ag neighbors). and I.-layer GAT aggregation on the K-node dependency graph (average A3 neighbors). Suppose each node is d dimensional, L-layer GAT on K-node graph contains: computing nodes' attention from neighbors O(L x K x Aa d) and nodes' linear transformation ( x K x dd)), totally equalling O x K x d (A3+d)); meta multigraph on N + M + K -node graph contains: computing nodes' attention O(Px x (N + M + K) A1 d), computing path's attention O(Px (N + M + K) x d), and nodes' linear transformation O x XN (N+M++) x dP). As a result, DisenGCD's s complexity equals O M +
+d) (d)) +O K (d((++d))) As analyzed above, the time complexity of our model may be high and not applicable in some large data sets.
Task Transferability. The proposed method primarily targets cognitive diagnosis tasks and designed handle such graphs as the nudenttexerriss-cconee graph, which may not be readily applicable other tasks. It is specifically tailored for modeling tasks similar to the sudeet--xerrise-comee triad diagram.
Potentially Poor
21

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: [TODO]In the abstract and introduction section, we describe the motivation of our research and the problems to be optimized
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: [TTOD]]nn the appendix, we discuss some of the shortcomings of our model and the directions for further optimization.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sellsspeciiiationn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
22

---

Answer: [NA]
Justification: [TODO]not applicable
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: [TODO]In the experimental chapter and the appendix chapter, introduce the experimental environment and details in detail.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
23

---

reproducible, they Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: [TODO]We will include the required data set in the code that supplements the file
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/
Cooee//bdeeeSbmmmcia ssi onPolicy) for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: [' the experimental chapter and the appendix chapter, introduce the experimental environment and details in detail
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[No]
Justification: [TODO]We did not analyze the error in the experimental results
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative rates).
show in tables or
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: [TODO]We describe our experimental environment in detail in the experimen- tal chapter
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics http:////eerrpppccpppppbbcaaaaaa
Answer: [Yes]
Justification: [TODO]Our experiment conforms to the experimental requirements of NIPS
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: [TODO]Our experiments help the development of Al in the field of education
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
25

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
[TODO]not applicable
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: [TODO]The dataset is free from the interference of related issues introduce it in the experimental section
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
26

---

If assets are released, the license, copyright information, and terms of use in the thcode has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssit
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
[TODO]not applicable
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
[TODO]not applicable
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: [TODO]not applicable
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
27

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
28


Reviews:
[{"review_id": "gajUxVE1EL", "reviewer": "Reviewer_2m1M", "summary": "The paper presents DisenGCD, a novel cognitive diagnosis model designed to enhance the robustness and accuracy of student, exercise, and concept representations by leveraging a meta multigraph-assisted disentangled graph learning framework. DisenGCD constructs and disentangles three specific graphs for interactions, relations, and dependencies, thus mitigating the negative impact of noise in students’ interactions. The framework incorporates a meta multigraph learning module and GAT to improve the learning of latent representations, demonstrating superior performance and robustness compared to state-of-the-art CDMs on multiple datasets.", "questions": "1. Can the model be adapted to handle real-time data for dynamic cognitive diagnosis?\n2. How does the choice of hyperparameters affect the model’s performance and robustness?\n3. Can the disentangled graph learning framework be integrated with other neural network architectures for further performance enhancement?", "limitations": "NA", "rating": 5, "confidence": 1, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The use of a meta multigraph-assisted disentangled graph learning framework is a novel approach that significantly enhances the robustness of cognitive diagnosis models.\n2. DisenGCD shows better performance in terms of AUC, accuracy, and RMSE compared to other state-of-the-art CDMs.\n3. The model effectively handles noise in student interactions, ensuring that the learning process remains accurate and reliable.\n4. The experiments cover multiple datasets and different data splits, providing a thorough validation of the model’s effectiveness.\n5. The diagnostic function of DisenGCD maintains high interpretability, comparable to traditional models like NCD, IRT, and MIRT.", "weaknesses": "1. The implementation of the meta multigraph learning module and the disentangled graph learning framework could be complex and resource-intensive.\n2. The paper does not report error bars or other statistical significance measures, which are important for validating experimental results. \n3. The comparison does not include some recent models like SCD, which could provide a more comprehensive evaluation.\n4. While some limitations are discussed, a more detailed analysis of potential weaknesses and future improvements could enhance the paper’s transparency and reliability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Response to   Weak.1 & Q.1\n\nWe admit that the proposed DisenGCD seems complex and needs more resources for its implementation because more graphs need to be handled. In future work, we would like to design a more efficient paradigm to learn robust representation in a unified graph.\n\nAs for **Question 1**, to be honest, we have to admit that the proposed DisenGCD is not designed for real-time dynamic cognitive diagnosis. \nHowever, as shown in some research [R1,R2], the graph-based CD  approaches could be easily extended to knowledge tracing (KT, a task similar to dynamic cognitive diagnosis): for example, RKT [R1] is actually a variant of RCD in knowledge tracing, where the representations learned by graph NNs are attached as static exercise features used for sequence prediction.  \n\nDue to the time limit and our limited coding capabilities， we are sorry that we failed to extend our approach to dynamic cognitive diagnosis. \n\nHowever, to extend graph CD approaches to KT or dynamic CD, a crucial problem to be solved is the graph-based model's efficiency. Therefore,  to solve your concern indirectly, we make the statistics about the proposed DisenGCD's efficiency regarding its inference time.  To this end, the inference time (seconds) of  DisenGCD and RCD are compared in **Table III:Upper** in ***global.pdf***. \n\nAs can be seen, the inference time of DisenGCD is better than RCD. That indicates that the proposed DisenGCD is more efficient than RCD, further signifying that it is promising to extend DisenGCD to dynamic CD.\n\n```\n[R1] Pandey, S., & Srivastava, J. (2020, October). RKT: relation-aware self-attention for knowledge tracing. In Proceedings of the 29th ACM international conference on information & knowledge management (pp. 1205-1214).\n[R2] Yang, Y., Shen, J., Qu, Y., Liu, Y., Wang, K., Zhu, Y., ... & Yu, Y. (2021). GIKT: a graph-based interaction model for knowledge tracing. In Machine learning and knowledge discovery in databases.\n```\n\n\n## Response to Q.2\nThe most important hyperparameter is the number of hyper nodes (i.e., *P*) in the meta-multigraph. To solve your concern, we would like to explore its influence from **two aspects** (i.e., on performance and robustness): \n\n**On the one hand**, its influence on model performance has been discussed in ***Appendix B.4***.  As shown in Figures 6 and 7 in Appendix B.4, the proposed DisenGCD can achieve promising performance when the number of hyper nodes is 5, indicating setting *P* to 5 is reasonable.\n\n***On the other hand***, we also analyzed the influence of different *P* (set to 4, 5, 6, and 7 ) on the model robustness under the Math dataset (ASSISTments is not used due to the time limit). **Fig.1(a)&(b)** in ***global.pdf***  shows the results of RCD and DisenGCD with different *P* obtained under different ratios of noise interactions (1\\%, 30\\%, and 50\\%). As you can see, when the number of hyper nodes is equal to 5, the robustness of DisenGCD is the most promising, whose performance under different ratios of noises is balanced better than RCD.\n\nIn summary, it can be seen that the proposed DisenGCD can achieve good performance and promising robustness when setting the number of hyper nodes to 5.\n\n## Response to Q.3 \n\nOther NNs can be integrated with the proposed framework yet the performance of them are not sure.\n\nTo validate this, we used three NNs (GCN, GraphSage, and HAN) to replace the GAT in the proposed approach, respectively, which are denoted as DisenGCD(GCN), DisenGCD(GraphSage), and DisenGCD(HAN). Due to the time limit, we only compared them with the proposed approach on the Math dataset, and the comparison results are summarized in **Table I:Upper** in ***global.pdf***.  It can be seen that the GAT enables the proposed framework to hold the best performance than other three variants, where  DisenGCD(GraphSage) performs better than the other two.\n\nThe above comparison indicates that the proposed DisenGCD could be integrated with other neural network architectures and their performance may be improved when suitable NNs are adopted.\n\n## Response to Weak.3 \n\nAccording to your suggestions, we have added more recent CDMs, like SCD [R1], KaNCD [R2], and KSCD [R3] to the comparison. The comparison between them and DisenGCD is presented in  **Table I:Upper** in ***global.pdf***, where only the Math dataset is used due to the time limit.\n\nAs can be seen from  **Table I:Upper**, compared to these recent CDMs, the proposed DisenGCD still exhibits better performance. \n\nWe will add these CDMs to the comparison in the revised paper. \n\n```\n[R1] Wang S, Zeng Z, Yang X, et al. Self-supervised graph learning for long-tailed cognitive diagnosis[C]//Proceedings of the AAAI conference on artificial intelligence. 2023, 37(1): 110-118.\n[R2] F. Wang, Q. Liu, E. Chen, Z. Huang, Y. Yin, S. Wang, and Y. Su, “Neuralcd: a general framework for cognitive diagnosis,” IEEE Transactions on Knowledge and Data Engineering, 2022.\n[R3] H. Ma, M. Li, L. Wu, H. Zhang, Y. Cao, X. Zhang, and X. Zhao, “Knowledge-sensed cognitive diagnosis for intelligent education platforms,” in Proceedings of the 31st ACM International Conference on Information & Knowledge Management, 2022, pp. 1451–1460.\n```\n\n## We appreciate your valuable feedback and will revise the paper based on the above.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nlF0xdHQ9w", "reviewer": "Reviewer_9V3d", "summary": "This paper introduces DisenGCD, a new framework for cognitive diagnosis in educational contexts. The authors make several contributions:\n1) They propose a disentangled graph learning approach, separating the typically unified graph into three distinct graphs: student-exercise-concept interactions, exercise-concept relations, and concept dependencies.\n2) They develop a meta multigraph module for learning student representations, which innovatively allows access to lower-order exercise representations.\n3) They employ GAT-based modules to learn exercise and concept representations on the disentangled graphs.\n4) They design a new diagnostic function to effectively combine the learned representations from the different graphs.\nThe authors evaluate DisenGCD on multiple datasets, demonstrating performance improvements and enhanced robustness compared to state-of-the-art methods in cognitive diagnosis.", "questions": "1) How does DisenGCD adapt to dynamic changes in the knowledge structure, such as the introduction of new concepts or the discovery of new relationships between existing concepts? Does the disentangled structure present any challenges in updating the model under these scenarios? \n2) In Section 4.1, you describe the meta multigraph as containing P hyper-nodes, and in your experiments you set P to 5. How did you determine this value? Have you explored the impact of different values of P on the model's performance and computational efficiency?\n3) While your experiments demonstrate improved robustness to interaction noise, how does DisenGCD perform when noise is introduced in the concept dependency graph or exercise-concept relation graph? This could simulate errors in curriculum design or expert knowledge.\n4) Have you investigated whether the disentangled approach leads to any loss of performance in scenarios where complex, cross-entity relationships are crucial for accurate diagnosis? Are there cases where a unified graph approach might outperform DisenGCD?", "limitations": "N/A", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper presents several notable strengths:\n1) The proposed disentangled graph learning approach is a novel contribution to the field of cognitive diagnosis. By separating different types of information (student-exercise-concept interactions, exercise-concept relations, and concept dependencies) into distinct graphs, the authors have created a more robust model that appears less susceptible to noise in the data. This is a significant advancement over existing unified graph approaches.\n2) The meta multigraph module introduces an innovative mechanism for student representation learning. By allowing access to lower-order exercise representations, this module potentially captures more nuanced relationships in the data, which could be particularly valuable in educational contexts where student knowledge evolves over time.\n3) The empirical evaluation is comprehensive and well-executed. The authors conduct experiments on multiple datasets (ASSISTments, Math, and SLP), demonstrating consistent improvements in performance and robustness over state-of-the-art methods. This thorough evaluation strengthens the paper's claims and increases confidence in the generalizability of the proposed method.\n4) The paper includes detailed ablation studies and analyses that effectively validate the different components of DisenGCD. These studies provide valuable insights into the contribution of each component and help justify the design choices made by the authors.", "weaknesses": "1) While the disentangled graph approach is innovative, it may lose some valuable cross-entity interactions. The separation of student-exercise interactions from exercise-concept relations could potentially limit the model's ability to capture complex, multi-hop relationships that might exist in a unified graph structure.\n2) The meta multigraph module, although showing improvements, adds considerable complexity to the model. It's not clear if this added complexity is always justified, particularly for simpler datasets or scenarios with limited noise. A more thorough analysis of the trade-off between model complexity and performance gains would strengthen the paper.\n3) The paper lacks a comprehensive discussion on how DisenGCD handles cold-start problems, especially for new students or exercises that don't have established interactions in the graph structure. This is a common challenge in educational settings and needs to be discussed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Response to Q.1\nWe must admit that the design of the proposed DisenGCD did not consider the scenery of dynamic changes in knowledge structures. However, the proposed DisenGCD and RCD are trained in an inductive manner in graph learning. Therefore, it is feasible for DisenGCD and RCD to handle the scenery of introducing new concepts, but their performance is unknown.\n\nTo investigate this, we conducted the experiments on the Math dataset due to the time limit. Specifically, interactions and exercises related to 80% of knowledge concepts were not masked during the training, and we tested the model on the remaining interactions and exercises. The comparison results of the DisenGCD and RCD are presented in **Table I:Lower** in   ***global.pdf***.\n\nWe can see DisenGCD exhibits poor performance on such this scenery, which is poorer than RCD. That indicates that the proposed DisenGCD cannot adapt to dynamic changes in the knowledge structure.\n\nThe reason may be that the propagation paths learned in the meta-multigraph are not suitable for newly added knowledge concepts.  Since adding 20% of concepts changes the topology of the interaction graph, while different-topology graphs generally need to learn different propagation paths. This fact can be found in Figure 4(a) in the submitted paper, stated in [R1].\n\nIn short, the proposed DisenGCD may not adapt to the scenery of introducing new concepts well due to the utilized learnable meta multigraph. In future work, we would like to explore how to address this issue to handle dynamic changes in knowledge structures.\n\n```\n[R1] Chao Li, Hao Xu, and Kun He. Differentiable meta multigraph search with partial message propagation on heterogeneous information networks. \n```\n## Response to Q.2\n\nActually, setting *P* to 5 is by trial and error on the DisenGCD. In ***Appendix B.4***, we have discussed the influence of different *P* on the performance of DisenGCD. **Figures 6 and 7** plot the results of DisenGCD obtained under different *P* (4, 5, 6, and 7) on the two datasets. It can be found that  DisenGCD can achieve the optimal performance on both two datasets when setting *P* to 5.\n\nHowever, we did not explore its influence on the computational efficiency.  The most intuitive is that a larger *P* will cause worse computational efficiency. \n\nTo investigate this, **Table III:Lower** in ***global.pdf*** presents the performance and runtimes (for training) of RCD and DisenGCD under different *P*. As can be seen, it is consistent with the above assumption that a larger *P* will make DisenGCD take more time to train the model. The proposed DisenGCD achieves the best performance on both two datasets when *P*=5, and the runtime increase from  *P*=4 to *P*=5 is not too much and is acceptable.\nBesides, **Fig.1(a)&(b)** in ***global.pdf***  show that the proposed DisenGCD can obtain the most promising robustness when *P*=5.\n\nTherefore, in this paper, we set *P* to 5 for a comprehensive consideration of model performance, model robustness, and model efficiency.\n\n\n## Response to Q.3 \n\nTo figure out the robustness of the proposed DisenGCD to exercise-concept noise, we conducted experiments on the Math dataset, where 1\\%, 10\\%, 20\\%, and 30\\% of exercise-concept relations are randomly removed as or added as noise relations, respectively. (Note that larger dataset) **Fig. 1(c)&(d)** in ***global.pdf*** compares the performance of RCD and DisenGCD on Math under different ratios of exercise-concept noise.\n\nAs can be seen, it is unexpected that the proposed DisenGCD exhibits better robustness than RCD to the added exercise-concept noises. However, different from against interaction noise (as shown in Figure 3(a) in the submitted manuscript), the robustness superiority/leading of DisenGCD over RCD does not increase as the ratio of noise increases, where the leading performance becomes smaller when a larger portion of noises are added.\n\nIt is reasonable because the added exercise-concept noise affects the learning on two graphs (the interaction graph and the relation graph), which further affects the learning of student and exercise representations. Since the learning of the concept representation is not influenced, DisenGCD still outperforms RCD. However, when more noise is added, DisenGCD's robustness against exercise-concept noise will not be better than that of DisenGCD against interaction noise, where the learning of two graphs is affected.\n\n\n## Response to Q.4\n\nAs for the first sub-question, to be honest, we have not investigated the performance of the proposed approach in scenarios where complex, cross-entity relationships. To answer your concern, we have tried our best to check many educational datasets， but we indeed cannot find a suitable dataset that holds cross-entity relations. Therefore, we are sorry that we cannot answer your question through experiments.  However, as far as we are concerned, the DisenGCD can extend to handle cross-entity relations by disentangling more graphs. That may make promising performance, but its complexity and efficiency will be criticized. \nWe also would like to answer your question if you can recommend a specific dataset. \n\nAs for the second, we would like to state that the compared RCD approach is essentially a unified graph approach, where all representations are learned and exchanged in an implicit graph. As shown in the submitted manuscript, the proposed DisenGCD exhibits better performance than the RCD. In addition, we have also compared  DisenGCD with another unified graph-based approach, i.e., SCD [R1], on the Math dataset, where the results are presented in **Table I:Upper** in ***global.pdf***. We can find that the DisenGCD still outperforms SCD. In summary, there may exist other novel unified graph approaches outperforming DisenGCD, but existing unified graph-based CDMs (RCD and SCD) cannot.\n\n```\n[R1] Wang S, Zeng Z, Yang X, et al. Self-supervised graph learning for long-tailed cognitive diagnosis[C].\n```", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0Dir7qEFT8", "reviewer": "Reviewer_7tDb", "summary": "In this paper, the authors introduce a meta multigraph assisted disentangled graph cognitive diagnosis model. Its main contribution is to propose a disentangled graph framework, which disentangles the student-exercise-concept dependency graph into an exercise-concept interaction graph and a concept-concept relationship graph to improve the robustness of representation modeling. Additionally, the concept of meta multigraph is introduced in the student-exercise-concept dependency graph to assist in better modeling of student representation. DisenGCD has demonstrated good performance compared to previous models in multiple datasets, and has shown good robustness in the presence of noise interference or sparse datasets. Finally, the authors also discussed some limitations of DisenGCD in terms of computational complexity and task transferability.", "questions": "1. KaNCD seems to have been found to work equally well in some papers. Can you compare DisenGCD and KaNCD?\n2. The paper utilizes graph attention networks (GAT) to learn representations of exercises and concepts. However, it would be valuable to explore and compare alternative graph neural network (GNN) models, such as graph convolutional networks (GCN) or LightGCN, to assess their performance in the same context.\n3. How are the multiple learnable propagation paths in the meta-multigraph learning module designed and selected? Have you tried different numbers or types of propagation paths and how well have they worked?", "limitations": "\\", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.\tThe paper is well-organized and clearly written, making it accessible to readers. Besides, the authors also present mathematical formulations, detailed explanations of the proposed models.\n2.\tThe authors focus on improving the robustness of the CD model and provide a detailed analysis of some problems found in previous graph cognitive diagnosis works. \n3.\tThe experiments are very thorough. The authors demonstrate the robustness of the model through decoupling experiments, sparse dataset experiments, and noisy dataset experiments", "weaknesses": "1. Although DisenGCD has demonstrated excellent performance across multiple datasets, it still appears to have the drawback of high computational complexity, which may limit its application in practical scenarios.\n2. Although the model has shown improved performance, its interpretability may be lacking. In real-world applications within the education field, transparent and interpretable models are more likely to gain acceptance and trust. However, the paper does not address methods for enhancing the interpretability of the model.\n3. The paper lacks detailed discussion on the challenges and potential issues that may arise when applying the model in practical educational settings. For instance, it does not address the strategies for handling data privacy and security concerns, which are critical considerations in the implementation of such models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Response to Q.1\n\n\nAs you suggested, we added KaNCD [R1] to be compared with the proposed DisenGCD, and the comparison results are summarized in  **Table I:Upper** in ***global.pdf***, where only the Math dataset is used because there is no enough time to validate on other two larger datasets. \n\nAs you can see, in addition to KaNCD, we have also compared our approach with  SCD [R2] and KSCD [R3], which are other two recently published CDMs. As shown in  **Table I:Upper** in ***global.pdf***, the proposed DisenGCD still performs better than KaNCD and other two state-of-the-art CDMs, further validating the effectiveness of the proposed DisenGCD.\n\nIn the future revised paper, we will add these CDMs to the comparison on all three datasets. \n\n```\n[R1] F. Wang, Q. Liu, E. Chen, Z. Huang, Y. Yin, S. Wang, and Y. Su, “Neuralcd: a general framework for cognitive diagnosis,” IEEE Transactions on Knowledge and Data Engineering, 2022.\n[R2] Wang S, Zeng Z, Yang X, et al. Self-supervised graph learning for long-tailed cognitive diagnosis[C]//Proceedings of the AAAI conference on artificial intelligence. 2023, 37(1): 110-118.\n[R3] H. Ma, M. Li, L. Wu, H. Zhang, Y. Cao, X. Zhang, and X. Zhao, “Knowledge-sensed cognitive diagnosis for intelligent education platforms,” in Proceedings of the 31st ACM International Conference on Information & Knowledge Management, 2022, pp. 1451–1460.\n```\n\n\n## Response to Q.2\n\nTo solve your concern, three GNNs (GCN, GraphSage, and HAN) are used to replace the GAT in the proposed DisenGCD, which are termed DisenGCD(GCN), DisenGCD(GraphSage), and DisenGCD(HAN), respectively. Then, we compared the three approaches with the proposed DisenGCD on the Math dataset, where the other two datasets were not used due to the time limit. The comparison of them is presented in **Table I:Upper** in ***global.pdf***. \n\n\nAs shown in **Table I:Upper**, the GCN-based DisenGCD achieves the worst performance, followed by DisenGCD(HAN). The GraphSage-based DisenGCD holds a competitive performance to yet is still worse than the proposed DisenGCD (based on GAT). \n\nThe above results show the GAT is a suitable and optimal choice among these four GNNs for the proposed DisenGCD, but we think the GAT may be not the best choice because there exist other types of GNNs that can be integrated with DisenGCD, showing better performance.\n\nIn summary, it is reasonable and effective for the proposed DisenGCD to adopt the GAT to learn the representations.\n\n## Response to Q.3\n\nThe design of the learnable propagation paths  is inspired by the success of meta graph. And the propagation paths are selected by the adopted routing strategy, which learns a value for each candidate propagation path. Afterward, these propagation paths that satisfy the threshold will be automatically kept. Therefore, the selection of propagation paths is achieved by the model learning.\n\n\nAs for the influence of the propagation path number on the proposed approach, it can be found in line 179 that the path number is determined by the number of hyper nodes (i.e., *P*). Therefore, this influence can be investigated by analyzing the influence of the number of hyper nodes (i.e., *P*).\n\nIn fact, we have investigated the influence of  *P* on the proposed DisenGCD in ***Appendix B.4***. As shown in Figures 6 and 7 in Appendix B.4,  the proposed DisenGCD is not very sensitive to the number of hyper nodes, where DisenGCD obtains a promising performance when *P* is 5. That indirectly indicates that the proposed  DisenGCD is not sensitive to the number of candidate propagation paths.\n\nIn addition, for more convincing, the influence of different P on the robustness of the proposed DisenGCD is also analyzed as shown in **Fig.1(a)&(b)** in ***global.pdf***, where only the Math dataset is used due to the time limit. **Fig.1(a)&(b)** present the results of RCD and DisenGCD with different P obtained under different ratios of noise interactions (1%, 30%, and 50%). As can be seen, the robustness of DisenGCD is a bit sensitive to different *P*, and the DisenGCD achieves the most promising robustness when *P* is equal to 5. That indirectly indicates that the proposed  DisenGCD is sensitive to the number of candidate propagation paths.\n\nIn summary, the performance of the proposed DisenGCD is not sensitive to the propagation path number while its robustness is sensitive. Nevertheless, the proposed DisenGCD can obtain promising performance and robustness when *P* is set to 5.\n\n\n\n\n## We appreciate your valuable feedback and will revise the paper based on the above.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KSbZS1e0Gr", "reviewer": "Reviewer_53z4", "summary": "The paper studies cognitive diagnosis based on graph learning. The proposed method utilizes multiple graphs to assist the representation learning. Specifically, the authors further disentangle another two graphs from the most comprehensive student-exercise-concept interaction graph by removing certain node and edge types. Therefore, three kinds of disentangled representations are learned based on the three graphs. Lastly, a diagnostic function fuses the three representations for final predictions.", "questions": "Please refer to the weakness part.", "limitations": "N/A", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 1, "strengths": "1. The problem that the paper aims to solve is well formulated and it is of practical value. \n2. The authors give very detailed illustrations of the methodology part. \n3. Comprehensive experiments are conducted to demonstrate the effectiveness of the proposed method.", "weaknesses": "1. Since the proposed method learns the graph representations based on disentangled graphs, the authors are encouraged to add a section about disentangled graph representation learning in the related work part to summarize the recent works on this problem and illustrate the relation of this work with previous ones. \n2. A more detailed introduction to incorporated datasets should be given. Specifically, both accuracy and RMSE are employed as evaluation metrics, the authors should provide a clearer illustration of the target tasks of those datasets. \n3. The authors should include more mode variants to demonstrate the effectiveness of the added graphs (e.g., w/o G_R, w/o G_D, and w/o both). \n4. GAT is used in this work to learn exercise and concept representations. The authors are encouraged to introduce the reason for this choice or conduct experiments based on other graph model backbones. \n5. There are many typos in the paper, e.g., line 16 and line 254. The authors should check the grammar carefully.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Response to Q.1\n\nThe related work on disentangling graph representation learning is as follows, and we will add this to the revised paper:\n\n1. Learning potential representations of disentangling in complex graph networks to achieve model robustness and interpretability has been a hot topic in recent years. Researchers have put forward a lot of classic neural network disentanglement approaches (e.g.,  DisenGCN[R1], DisenHAN[R2], DGCF[R3], DCCF[R4], DcRec[R5], etc.) to address this challenge. Some representative approaches will be briefly introduced.\n\n2. For example, in DisenHAN[R1], the authors utilized disentangled representation learning to account for the influence of each factor in an item. They achieved this by mapping the representation into different spatial dimensions and aggregating item information from various edge types within the graph neural network to extract features from different aspects； In DCCF[R4], the authors introduced global intent disentanglement into graph contrastive learning, extracting more fine-grained latent factors from self-supervised signals to enhance model robustness； DcRec[R5] disentangles the network into a user-item domain and a user-user social domain, generating two views through data augmentation, and ultimately obtaining a more robust representation via contrastive learning.\n\n3. Despite many approaches suggested, they were primarily applied to bipartite graphs to learn different representations from different perspectives, for learning more comprehensive representations. While  this paper aims to leverage disentanglement learning to mitigate the influence of the interaction noise in the interaction graph, and thus we proposed a meta multigraph-assisted disentangled graph cognitive diagnostic framework to learn three types of representations on three disentangled graphs. By doing so, the influence of the noise on exercise and concept learning can be well alleviated. \n\n```\n[R1]Jianxin Ma, et al., Disentangled Graph Convolutional Networks. \n[R2]Yifan Wang, et al., DisenHAN: Disentangled Heterogeneous Graph Attention Network for Recommendation. \n[R3]Xiang Wang, et al., Disentangled Graph Collaborative Filtering. \n[R4]Xubin Ren, et al., Disentangled Contrastive Collaborative Filtering.\n[R5]Jiahao Wu, et al., Disentangled Contrastive Learning for Social Recommendation\n```\n\n## Response to Q.2\n\nWe would like to  answer your question from the following three aspects:\n\n- In the Appendix, we indeed provided a brief introduction to the three datasets (ASSISTments, Math, SLP). However, this introduction may not be detailed, thus we will give more information about the datasets in the revised paper, including their collectors, the contained subjects, and many other attributes.\n- The cognitive diagnosis task is to assess the true knowledge level of students. To achieve this, this task is generally solved by modeling the students' answer prediction task, which is a binary classification task. Therefore, we use the evaluation metrics of classification to measure the diagnosis accuracy.\n- The utilized metrics include **ACC**, **RMSE**, and **AUC**, which are widely used for the classification tasks. The three metrics are also widely adopted in many previous CD approaches.\n  \n## Response to Weak.3 \n\nAs you suggested, we added three variants of the DisenGCD to show the effectiveness of the added graphs, which are as\n- DisenGCD(w/o both) is same as DisenGCD but learns exercise and concept representations directly through **native embedding modules(NEMs)** .\n- DisenGCD(w/o $\\mathcal{G_R}$)  is same as DisenGCD but its exercise representation is learned through  a **NEM**.\n- DisenGCD(w/o $\\mathcal{G_D}$) is same as DisenGCD but its concept representation is learned through a **NEM**.\n\n**Table II** in ***global.pdf*** summarizes the results of all variants and the DisenGCD on the ASSISTments dataset. \n\nAs can be seen, the DisenGCD outperforms both DisenGCD(w/o $\\mathcal{G_D}$) and DisenGCD(w/o $\\mathcal{G_R}$), which indicates the effectiveness of learning exercise/concept representations by the  GAT on added graphs. Moreover, the performance leading of DisenGCD(w/o $\\mathcal{G_D}$) and DisenGCD(w/o $\\mathcal{G_R}$) over DisenGCD(w/o both) can further validate the effectiveness of the added graphs. In addition, the effectiveness of the DisenGCD can be indirectly validated through the comparisons of DisenGCD(w/o both) and DisenGCD(I), as well as DisenGCD(w/o $\\mathcal{G_D}$) and DisenGCD(Is+Rec).\n\nIn summary, the above comparison validates the effectiveness of the added graphs and the proposed disentangling learning framework.\n\n## Response to Q.4 \nAs you suggested, other three types ofGNNs (i.e., GCN, GraphSage, and HAN) are used as the graph model backbones to replace the GAT module in DisenGCD. The variants of DisenGCD, utilizing GCN, GraphSage, and HAN as the backbones, are denoted as DisenGCD(GCN), DisenGCD(GraphSage), and DisenGCD(HAN). \n\nTo observe their influence on the proposed approach, these three variants were executed on the Math dataset, whereas the other two datasets were not used due to the time limit. As a result,  **Table I:Upper** in ***global.pdf*** summarizes the results of these variants and the DisenGCD. We can find that the variant DisenGCD(GraphSage) can obtain competitive performance than state-of-the-art approaches, while DisenGCD(GCN) and DisenGCD(HAN) cannot. Despite its competitive performance, DisenGCD(GraphSage) is still worse than DisenGCD, which indicates that the GAT is most suitable for the proposed framework.\n\nIn summary, the results show the GAT is a suitable and optimal choice among these four GNNs for the proposed DisenGCD, and it is reasonable and effective for the proposed DisenGCD to adopt the GAT to learn the representations.\n\n## Response to Q.5 \nThanks for pointing out the topos.  We will correct these typos and double-check the paper.\n\n## We appreciate your valuable feedback and will revise the paper based on the above.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We sincerely appreciate the valuable feedback provided by all the reviewers. We have carefully addressed their questions and concerns in our response, aiming to provide satisfactory answers.

Here we uploaded a file named "global.pdf" to show some necessary results and comparisons, which contains three tables and one figure.

In all subsequent responses to reviewers, this file is termed ***global.pdf*** for easy presentation and convenient discussion.

Meta-review: The paper presents a meta multigraph-assisted disentangled graph learning framework for cognitive diagnosis (CD) in educational contexts, named DisenGCD. This new method constructs three distinct types of representations on separate disentangled graphs: student-exercise-concept interactions, exercise-concept relationships, and concept dependencies. Employing GAT-based modules, it effectively learns nuanced exercise and concept representations from these graphs. A specialized diagnostic function integrates these representations for predictive assessment. DisenGCD demonstrates superior performance over other state-of-the-art CD methods, excelling in AUC, accuracy, and RMSE metrics.

All four reviewers agree that the paper meets the acceptance criteria and have offered several constructive suggestions, including comparisons with more recent CD methods and conducting an ablation study to evaluate the impact of model parameters and the choice of graph neural network. The authors have addressed these points to the reviewers' satisfaction and are encouraged to include these additional results in the final version of the paper.
Decision: Accept (poster)