An Autoencoder-Like Nonnegative Matrix Co-Factorization for Improved Student Cognitive Modeling
Authors: Shenbao Yu
Yinghui Pan
Yifeng Zeng
Prashant Doshi
Guoquan Liu
Kim-Leng Poh
Mingwei Lin
Venue: NeurIPS 2024 poster (2024)
Keywords: Student Cognitive Modeling
Matrix Co-Factorization
Autoencoder
Primary area: machine_learning_for_other_sciences_and_fields
TL;DR: This paper presents an autoencoder-like nonnegative matrix co-factorization model of student cognition for better-predicting student exercise performance and estimating his or her knowledge proficiency in a subject.

Abstract: Student cognitive modeling (SCM) is a fundamental task in intelligent education, with applications ranging from personalized learning to educational resource allocation. By exploiting students' response logs, SCM aims to predict their exercise performance as well as estimate knowledge proficiency in a subject. Data mining approaches such as matrix factorization can obtain high accuracy in predicting student performance on exercises, but the knowledge proficiency is unknown or poorly estimated. The situation is further exacerbated if only sparse interactions exist between exercises and students (or knowledge concepts). To solve this dilemma, we root monotonicity (a fundamental psychometric theory on educational assessments) in a co-factorization framework and present an autoencoder-like nonnegative matrix co-factorization (AE-NMCF), which improves the accuracy of estimating the student's knowledge proficiency via an encoder-decoder learning pipeline. The resulting estimation problem is nonconvex with nonnegative constraints. We introduce a projected gradient method based on block coordinate descent with Lipschitz constants and guarantee the method's theoretical convergence. Experiments on several real-world data sets demonstrate the efficacy of our approach in terms of both performance prediction accuracy and knowledge estimation ability, when compared with existing student cognitive models.

An Autoencoder-Like Nonnegative Matrix Co-Factorization for Improved Student Cognitive Modeling
Shenbao Yu"
Yinghui Pan 2- Yifeng Zeng
Prashant Doshi 4
Guoquan Liu
Kim-Leng Poh Mingwei Lin
College Computer and Cyber Security, Fujian Normal University, China
2 National Engineering Laboratory for Big Data System Computing Technology, Shenzhen University, China
Department of Computer and Information Sciences, Northumbria University, UK Intelligent Thought and Action Lab, School of Computing, University Georgia, USA
Financial Technology Research Institute, Fudan University, China College of Design and Engineering, National University of Singapore, Singapore
(yushenbao, ,lmff jm edu. en, panyinghui@szu @szuu edu. cn yifeng. www.genotttuurriaacoomm uck pdoshi@uga edu liugq0fudan edu. en, pohkimlengonus. edu.
Abstract
Student cognitive modeling (SCM) is a fundamental task in intelligent education, with applications ranging from personalized learning to educational resource allo- cation. By exploiting students' response logs, SCM aims to predict their exercise performance well as estimate knowledge proficiency obtain high accuracy in predicting stu-
subject. Data mining
approaches such as matrix factorization dent performance on exercises, but the knowledge proficiency is unknown or poorly estimated. The situation is further exacerbated if only sparse interactions exist between exercises and students (or knowledge concepts). To solve this dilemma, we root monotonicity (a fundamental psychometric theory educational assessments) in a co-factorization framework and present an autoencoder- nonnegative ma- trix co-factorization (AE-NMCF), which improves the accuracy of estimating the student's knowledge proficiency via an encoder-decoder learning pipeline. The resulting estimation problem nonconvex with nonnegative constraints. We intro- duce projected gradient method based on block coordinate descent with Lipschitz constants and guarantee the method's theoretical convergence. Experiments on several real-world data sets demonstrate the efficacy of our approach in terms of both performance prediction accuracy and knowledge estimation ability, when compared with existing student cognitive models.
Introduction
With the explosion of open educational resources, student cognitive modeling is receiving growing attention. As illustrated in Figure 1, given set of exereises (could be recommended by a learning platform) with the finish the exercises and leaves the responses, Based on the response log, cognitive modeling aims to
knowledge concepts in a subject domain, a student is required to
Corresponding author
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

(a) estimate the student's cognitive levels on the knowledge concepts (i.e., cognitive diagnosis) and (6) predict some exercise performance. With comprehensive understanding of students, cognitive modeling is fruitful in applications such as computerized adaptive testing [1] and exercise recommen- dations [2]. To profile students' cognitive status, much progress has been made in the educational psychology area, where one popular avenue is to use cognitive diagnosis models (CDMs) [3]. While most CDMs provide detailed insights into students' cognitive states, the subjective handcraft features (e.g., the slip and guess of an exercise) may only partially capture the nuances of actual cognitive functioning, triggering cascading errors predicting student performance [4].
Exercise bank
Exercise
Knowledge
Student
Exi
Xc1
Kc5
Ex2
Kc2
Ex2
Kc3
TUTT
Rc3*
EX5
Ex3
Kc2
Ked
Test
choose exercises
Kc4
Ex6
Kc3
Kc5
Cognitive diagnosis
Performance prediction
Exercise content of Ex4 Consider function
Knowledge description
Responses
Bx1
xcl: T.kwwrr
What is the value of I(21?
Fanction
Bx
fix)=10, what 15 the value of
Student Cognitive Modeling
Exd
Figure 1: A schematic illustration of the student cognitive modeling problem. On the left is set of exercises with the expert-labeled knowledge concepts. The middle is a student's binary-value response log with missing values (c.g.. Exg missing) that is input to the modeling, and the top right illustrates the two cognitive tasks, which are the output of the modeling.
In fresh direction, several studies focus applying data mining techniques model students' learning status, of which the cornerstone is matrix factorization (MF) [5]. By transforming students' response logs into scoring matrix, MF-based models directly predict the missing response values via latent factors, thereby reducing cascading errors. In contrast to CDMs, MF--baed models enjoy high prediction accuracy and are inexpensive deploy [6]. On the other hand, the latent factors uncovered from the factorization techniques, which encode students' implicit learning ability, are unexplainable, i... the true knowledge components the students remain unknown the latent vectors. Recognizing this problem, follow-up scalable nonnegative matrix co- co- factorization (SNMCF) model [4] utilizes point coverage function to learn students' proficiency levels via pre-trained latent factors. However, SNMCF solves the two learning tasks separately, i.e., the generation of latent features is aimed at performance prediction without considering the target of improving cognitive diagnosis, thereby compromising the identification of student cognitive levels. As such, the fundamental issue of identifying students' knowledge proficiency remains an open problem.
In this paper, we envision reliable and interpretable data mining-based cognitive model with interlocking learning components. Learning latent factors that help pinpoint students' responses to exercises can guide the assessment of their knowledge proficiency, and the corresponding latent knowledge features, in turn, enable their success or failure on the exercises. To this end, several challenges exist: How can we specify and assess the students" knowledge proficiency since the ground truth the cognitive levels is unknown [1]? How can we frame the two learning tasks as the building blocks of an optimization framework while reducing cascading errors?
To mitigate these challenges, we leverage the known monotonicity [7] sidestep the issue of unknown knowledge proficiency. The monotonicity states that a student's 's knowledge proficiency has monotonic relationship with the probability of the right responses related exercises. Furthermore. by investigating the form of an autoencoder, our key observation reveals that its seff-reconstruction principle, which aims to reconstruct input data from the learned diwemmiiiionl representations, is amenable to the requirement of the monotonic constraint. Leveraging this observation, we root monotonicity in a co-factorization framework via the autoencoder mechanism. Consequently, an autoencoder-Hn like nonnegative matrix co- co-factorization (AF-NMCF) is presented, which enables an iterative link between students' knowledge proficiency and exercise performance, thereby enhancing prediction accuracy and diagnostic ability. As the resulting optimization problem is not convex and

---

has nonnegative constraints - which makes the complexity acute by an inverse link function (often called response functions in the case of general linear models [8]) we develop a projected gradient method based on block coordinate descent with Lipschitz constants and guarantee its theoretical convergence. The main contributions are:
We introduce a data mining-based model (AE-NMCF) for improved student cognitive modeling, which provides an end-to-end and data-driven way of specifying and assessing students' under- standing of a set of given knowledge concepts. This new model exploits the monotonicity in educational MF-based approaches for the first time.
To learn the model, we present a novel projected gradient method based on block coordinate descent with Lipschitz constants, for which theoretical convergence is guaranteed. This method accounts for the non-convexity of the optimization function with nonnegative constraints and the complexity of the inverse link function.
AE-NMCF provides a good fit to the students' knowledge proficiency while maintaining student performance prediction that is comparable other student cognitive models.
These contributions will potentially improve the automated comprehensive understanding of students' knowledge learning and benefit numerous intelligent educational tools.
2
Related Works
Student cognitive modeling has generally proceeded along two tracks: cognitive diagnostic models (CDMs) and data mining approaches. CDMs are of two types: continuous CDMs, an example of which is item response theory (IRT) [9]), and discrete CDMs such the deterministic inputs, noisy "and" gate (DINA) [10). IRT predicts a student's exercise performance based on single learning trait and exercise difficulty levels. In contrast, DINA probes student's binary cognitive status in different knowledge concepts, which assumes that a student can answer correctly if she has mastered all the required knowledge concepts. Traditional CDMs engender plethora of advanced models. For example, Cheng et al. [11] extend IRT using deep learning to enhance the diagnostic process. Noting the importance of the relation among knowledge concepts, Gao et al. [12] proposed a deep diagnosis framework that considers both the importance of and the interactions between knowledge concepts. Furthermore, Yang el al. [13] recently presented relationship-based CDM to explore implicit knowledge-exercise relations that educators ignore.
Along the data mining approach, MF has proven to be effective in understanding students' response processes [12], especially toward student performance prediction. In this study, classic models (c.g., nonnegative MF (NMF) [14]) and their variants such as the regularized NMF [15] were successfully applied. Because the latent trait of MF is not interpretable for knowledge estimation, Yu et al. [4] proposed SNMCF that utilizes coverage function to model students' knowledge states, thereby taking an important stride in data mining based student cognitive modeling. But, the coverage function often gives binary cognitive levels, failing discern the nuance between knowledge proficiencies.
Matrix co-factorization (MCF) [16] benefits from jointly exploiting multiple data sources. is well established for many applications such as convolutive source separation [17]. data spar- sity [18, 19]. and decision support systems [20]. Given a domain task, MCF improves performance by incorporating knowledge in additional matrices (e.g., trust relationship for social recommen- dation [21]), which share latent factors with the original one. This sharing mechanism facilitates entity-relation learning [22]. It motivates to develop in-depth understanding of students, exer- cises, and knowledge concepts, and facilitates an effective solution for the two learning tasks, which differs from existing MCF-based approaches that aim performance boost only.
Recently, the autoencoder architecture is being explored in dimensionality reduction [23], classification [24], and anomaly detection [25, 26]. For example, Wang et al. [23] proposed a deep version of autoencoder to explore manifold data structures. Gong el al. [25] augmented the autoencoder with memory module to mitigate anomaly reconstruction problems. For student cognitive modeling, given the reconstruction ability of autoencoder, our work is the first attempt to exploit this mechanism in MF-based approaches to estimate student knowledge proficiency.
3

---

3 AE-NMCF Model and Method
Given M students and N exercises, all students' responses to the exercises are recorded in a binary scoring matrix X [0]]]>*M, where Xam denotes student St,, 's answer on exercise Ex,- In addition, given K knowledge concepts, we have an expert-labeled Q-matrix Q E [0] 10]1NN where Qnk if Ex,, relates knowledge concept Kcş, otherwise Qnk With X and Q in hand, we aim to (a) learn students' proficiency in knowledge concepts from the responses, and (6) predict students' performance on exercises that they have never done.
3.1 Model Formulation
Figure 2 (from left to right) offers an overview of the approach, which includes an encoder and decoder. The encoder and decoder specify and diagnose students' cognitive levels, thereby enabling monotonicity. Specifically, the new framework receives the student scoring matrix (X) and the Q-matrix (Q). In the encoder process,
introduce the exercise- -nrecise-knowedgee association matrix (B) and then jointly decompose X and B to obtain three low-dimensional nonnegative matrices: the student proficiency matrix (U), the exercise characteristic matrix (E), and the knowledge requirement matrix (V). Note that the shared matrix E places U on the same scale as V, which shapes a pathway specify the students' knowledge proficiency (A). In the decoder process, introduce the exercise difficulty vector (M), which is combined with A and B to form cognitive factors. By re- re-fitting X. the decoder process ensures that students' knowledge proficiency monotonic with the probability of the correct exercise responses, which embodies our desire to maintain the monotonicity.
Inputs
Latent matrices
Cognitive factors
Output
29009
wyl--k0ll3
Viknowlalge requirenent matris
$09000
Xi the scoring
matrix
iopic-ecil
lcpi--ici2
lopic-akil
eecrnciee
E: charactersstc matrix
the exceciet- matris
A student- knowlolge proficiency mutris
difficalty vector
X: scocing
sppe-skl111
2ppt-akil2 2
Q the < -munii
3 3
Utstudent
proficiency manris
Encoder process
Decoder process
Nonnegative constraint
Sparse constraint
Decompose
Compose
The end-to-end pipeline of AE-NMCF. We start from the scoring matrix (X), which is also the ending module. The question marks (??") in X denote the absent responses that the students have never visited the exercises before. Here, we use the cell shadings to highlight the nonnegative constraints on the matrix blocks, wherein the dotted lines impose the sparse constraints. In addition, the solid and chain-dotted lines denote the decomposing and composing processes, respectively.
Figure 2:
The encoder process. Given X E [0] and Q [0] start the encoder process with optimization problem (I), where we have three low-dimensional nonnegative matrices: E E l8 NxT U e IR TxM and V E TxK each of which consists of T latent factors. The latent factors can be loosely viewed as a series of topic skills denoting high-level knowledge in a subject area, such as "spatial imagination" and "abstract summarization' in mathematics.
min I о (X EU) 2
+
QQ
о
(B B --VV
(1)
S.L.
B > 0,
U
0,
V

---

For the first term in problem (1), we use E and U to approximate X through the Frobenius norm and introduce weighted matrix W E [o]1]NxM to focus on the observed entries in X via the Hadama product (o). In the second term, considering that Q only stores the linkage between exercises and knowledge concepts with either true or false relations (failing to uncover their strength), we introduce the nonnegative matrix B E RNNK where B, is the degree to which exercise Ex, involves knowledge concept Kcg, with larger values denoting stronger involvement of the knowledge concept. Similarly, use E and V to approximate B, where the sparsity is imposed by Q.
In problem (1), X and B share the matrix E, which bridges the gap between students and knowledge concepts. In reality, given E, the approximation for X.m. is a linear accumulation of the columns of E, weighted by the components of U .m. and so does B we project the two nonnegative vectors U. and V., into the new basis E [4]. Since the latent factors are considered topic skills [27], we define Utm as the topic knowledge of student Stm on 1-th topic skill, as well as Vt as the topic requirement of Кc accordingly. Based on U.m and VIk, we specify students' knowledge proficiency via the matrix A V E IR. KxM, where Akm is the cognitive level of Stm on Kc,-
The decoder process. Recall that the matrix A specified by problem (1) does not give an off-the- shelf diagnostic solution due the ignorance of monotonic constraints. We remedy this void by reconstructing the scoring matrix X. Specifically, we first assume that exercise Ex, has an intrinsic difficulty level Hu R. which are stacked into a column vector M [21.M2:-' with A, B, and M, the probability that St, answers Ex, correctly is
.NN] Armed
Φ (Anm)
N(())
1 2
-2/22d1
where Anm = A.m + pmn indicates that St., 's response to Ex, is generated by a linear accumu- lation of required knowledge concepts. In addition, we use inverse link function Φ(x), which is often a response function in generalized linear models, to map A.m to the success probability of the binary response Xam- ((x) can be any monotonic differentiable function. Here, focus the commonly used probit link function with the probability density of the standard Gaussian distribution.
Given Eq. (2), we can maximize the likelihood of the observed data X Xam as
mm( 4(Amm) ) Xum
$ $ mm
| (1-X..)
and the likelihood finally yields the following optimization problem
N Σ
min
E+
2 2 n=1
B 1
(4)
B A Vnn.
where 8 E mn,m)en, log Pr(X,mm) is the log-likelihood term, and 22 {L, (L,,...,,,x(( M} contains indices of the observed responses in X. In addition, since one can arbitrarily increase the scale of the vector B.: while decreasing the scale of the vector A.m. (or V U.m) accordingly (and vice versa) without changing the likelihood, we gauge the vector B,: using the regularization term EN BI: with the regularization parameter > 0. To illustrate the encoder-decoder process further, provide example in Appendix A.
Objective function. By combining the encoder and decoder, the objective function (OAF) is
min B.U.E.V.
OAF - W
X
EU) 2 + ||Q (B (8-EV) +
7 2
Σ
B
1
(5)
s.t. B 0,
O,E 0, V 0.
is worth taking a few moments to study the form of problem (5) as enables the monotonicity from two viewpoints. First, the monotonicity is achieved by the monotonic formulation in Eq. (2); Second, the monotonicity is optimized by problem (4). They jointly guarantee that a large value of knowledge proficiency corresponds to a better chance of success on related exercises.
3.2 Model Solution
In problem (5), the first term -6 is convex for the probit link function [28]. The second and third terms convex in either B. U, E, or V only, but they are not convex in all the variables together.
5

---

Given the nonnegative constraints, we employ the projected gradient (PG) method [29]. Furthermore, concerning blocks of B and U U... we apply the PG via block coordinate descent (PG-BCD) approach. Hence, problem (5) can be expressed in block fashion as
min B.U.E.V.M
OAF -L+
W (X.m
EU m
2
QQ
Bn: E,, V)
s.t. B 0, O.U02.EE22 2.V20.2.
Accordingly, the subproblems of B U.m. E, V, and Mn constitute the iterations of PG-BCD for AE-NMCE. Next, we show the parameter solution for B. problem (6) below. For further details on the remaining parameters, refer to Appendix B.
min OA, Σ m
log Pr(Xmm)
2 B + QQ: @ (Bn E, V)I
(6)
To solve problem (6), we note that second-order methods do not scale well to high-dimensional problems due to the necessary computation of the Hessian, making explicit calculation difficult for the probit link function. Thus, we build our learning algorithm on first-order methods. To do so, we first derive the gradients of (A:(B as
VOAF(B,..) = = Xmmm ((Am)) U V 2 OB, Qn. V)] + +yB...
A/Amm) "J and we can employ the gradients above to search for the optimum point. In each iteration = 1,2, ... the gradient step is
where Emm 4(A.m)
B (+1) B(++)
B ( B....
VO AF B
where the half-wave rectifier [x]+ = max(k, x). k = 10-15 ensures the nonnegativity [30]. and (I)
(!)
suitable step size. For Eq. (7), a key issue is to choose the appropriate step size m..."" Vn.. a simple strategy is "Armijo rule along the projection arc" [31]. Although the convergence guaranteed, it time-consuming to search for feasible values. Motivated by Lan al. [6], determine the appropriate step sizes by Lipschitz constants [32]. A common approach that guarantees convergence of function f is to set (1()) = 1/L, where I. is the Lipschitz constant of V..
3.3 Algorithm and Theoretical Analysis
We start with Lemma 1 [6] to analyze the Lipschitz constant for problem (6). After that, present the parameter learning algorithm for problem (5) and conclude with its theoretical analysis.
Lemma 1. Let g(x) v(z) x E R. where (x)) is the probit link function. Then, for y, 2 E R. we have (g(y) - -((x)) < Lp z). Here, Lp is the scalar Lipschitz. constant of g(x).
('(x)
Since Eq. (3) can be rewritten as Pr(Xum)) = ((2X,m, am VT )n)) for $(.), we de- rive the following theorem which serves as bound the (vector) Lipschitz constant for problem (6), using the result in Lemma I.
V
Theorem For a given n, substituting Pr(Xam) in problem (6) with the right hand side expression above yields the following
(Bn:)
Σ
log (AAm)) + ? Bu: 12 ||Q @ (Bn: E,, V)
where Anm =22Xnm nm 1) V pn). For any vectors y, 2, have
$
K
V(Orll 3
L) +2
Σ
Q
+
Ily - 2||2.

---

To prove Theorem 1, we first derive the gradient of OAr(Br)) (B.)) based on the element-win operation of (V() and (-)). After that, we establish the upper bound of the l2 -n-norm of the gradient difference given two arbitrary points y and z. The derivation details refer to Appendix D. By comparing
with Theorem 1, we obtain the Lipschitz constant as ( + 2
km Q (>k
+7, where
(-) denotes the corresponding maximum singular value.
Armed with Eq. (7) with the step sizes determined by Theorem 1, Algorithm outlines the opti- mization process for AE-NMCF, named LG-BCD-Lipschizz.
Algorithm 1, we first initialize all parameters with random entries (line 1) and then optimize OAF in alternating fashion. Each outer iteration solves the inner subproblems (lines - 9). For each subproblem, we optimize the target parameter and hold others constant. For example, we hold U, E, and M constant and separately op- timize each block of variables B. The update order in the block case BJ: + Bg. -.... B The outer loop is terminated if the decrease in OA is smaller than a threshold e (lines 6 6- 8).
Algorithm 1 PG-BCDDLipschitz
Input: X, Q, and c.
Output: BR:. U U. E, V, and nn, IInnSN,Imm$1
1: Initialize / 0, U 0, E(0) 2 V 2 and (0) (ISHEN,1<MSM)
2: Calculate (0)
3: for /
0, 0, 1.. do
4: Update: B (ISn<N); U
(ISmSM); B(i+)); v(+!)); pn (ISnsN).
(+1)
(1+1)
5: Calculate AF
6: if (0!Ht1)
(0) S 6 then
AF
AF
7: Return (1+1) U ((+1)
E(+)), v(+!), and pnn
(1+1)
8: end if
9:
end for
We now establish the convergence guarantees of PG-BCD+Lipxchizz In fact, the development of rigorous statements for the convergence of B, U, E, V, and M to an optimum is not trivial, due to the block multi-convex nature. Nevertheless, we can establish the convergence of PG-BCD+Lipschit based on a prior analysis of BCD for multiconvex optimization [33]. To achieve this, for the sake of convenience, let θ (B, E, V, M), then we have the following theorem.
Theorem 2. Given any start point ((),, let { be the sequence of the factors from PG- BCD+Lipshhiz, where ( = 1,2,... are the outer iteration numbers, then the sequence {O(0) converges the finite the critical point of problem (5). In particular, if ( is close to the global point of problem (5), PG-BCD+ 4ipssiisc converges to the global optimum.
Since minimizing AE-NMCF follows multi-block coordinate descent solutions, which correspond BCDs with the update (1.3a) in [33], we can use the results laid by Xu and Yin [33, Lemma 2.6. Corollary 2.7, and Theorem 2.8] to prove Theorem 2, and the proof details refer to Appendix E. Note that we can not guarantee the global optimum convergence PGGssci+.ipschitzzz from an arbitrary point due to the multi-convex, but the use of multiple randomized initialization attempts can increase the change to reach the global optimal solution.
4 Experiments
Data set description. We use real-world students' response data with different sparsities and knowledge-exerriss relations, which are from diversified academic subjects, including (a) Math (FreSub, Junyi-s, and Quanlang-s), (b) Biology (SLP-Bio-s), (c) History (SLP-His-s). and (d) English (SLP-Eng). FreSub comprises of the fraction subtraction problem scores of 536 middle school students [10]. Junyi-s includes problem logs from an e-learning website based on the open-source code released by Khan Academy [34]. The private Quanlang-s s data set collected from mathematical exams given to junior schools supplied by QUANLANG education company. 2 Others include SLP-
"This data set was made available to us under an agreement with Quanlang education company (ttps://www.quaalanged..ccm whose terms included informed consent, privacy protection, and faimess.
7

---

Bio-s, -His-s, and -Eng, which provide unit test results of K-12 learners compiled by an online learning platform (smart learning partner. SLP) [35]. Statistics of the data set are summarized in Table III of Appendix F.
Baseline approaches. The baselines include data mining approaches and cognitive diagnosis models.
The former uses the well-known NMF [36], MCF [16], GNMF [37]. NMMF [38], and the advanced SNMCF [4]. For the latter, we consider the following competing models: (i) DINA [10], a classic
CDM that models students' knowledge levels by a binary attribute vector with the slip and guess factors of exercises; (ii) DIRT [11], an extended IRT model incorporating the deep learning technique
to enhance the diagnostic process: (iii) DeepCDF [12], a deep learning- -based CDM that considers the importance and relationships of knowledge concepts; and (ie) QRCDM [13]. which integrates
the implicit knowledge-exere exercise relations into CDMs. DIRT and DecpCDF are modified by excluding
text information. Our code is available at https: //gi thub. com /S////heeaabboo Yu/ Yu/Ao- AE-NMCF.
4.1 Results
In this section, we evaluate the effectiveness of AE-NMCF in the two learning tasks. Additional experiments, such as cognitive case studies, can be found in Appendix F.
We first compare student performance prediction. The evaluation metrics are the commonly used ACC and RMSE [4], which are calculated based on the ground truth of students' responses to exercises and corresponding predicted ones. Table I shows the prediction results with the best performances highlighted in boldface, the top 2 results are shaded, and use 't' to denote the standard deviations. The last column lists the average ranks of all models from the Friedman test (a rank-based method to validate the performance of multiple models multiple datasets) [39]. In Table 1, we observe that the data mining methods (especially SNMCF and AE-NMCF) perform well, and AE-NMCF lies in the 2 performers on all data sets except for Fre and SLP-EEg in terms of ACC and RMSE, respectively. Its rank of 1.33 on ACC and 1.67 on RMSE also confirm the competitiveness of AE-NMCF. The results indicate that our model can not only handle the students' response data that yields varied degrees of sparsity but also do so for diversified subject domains.
Table 1: Experimental results on student performance prediction
Metie
Model
Data set
Rank
FreSub
Junyi-s
Quanlang-s
SLP-Dio-s
SL
SLP-Ling
NMF
0.7564:20.0093
0.618620000233
0.6312:000075
0.6752=0.0105
0.716900.004
0.7222+0.011
5.83
MCF-GGa
0.5227+0.0126
0.5046+0.0219
0.5679+0.0229
0,5515+00119
0.8828-0.000
0.5640++00010
10.17
MCF-New
0.7066:10.0776
0.53271..0174
0.5743.10.009
0.5543:10.0096
0.5824-0.0043
0.5696.10.008
9.17
GNMF
0:7516+0.0112
0.6429+0.0305
0.5894+0.0109
0.6190-000143
06209-0.0056
0.6034+00 0.777
7.00
NMMF
0,77591000008
0.672910.0264
0.6477:000194
0.678010.0004
0.7002-0.0011
0.7007
ACC 1
SNMCF
0.8548:10.0043
0.68781000192
0.7417:00.001
0.735120.0108
0,8051
0.7456:100009
1.85
DINA
0.81562000033
0.5209=0.0078
0.6000=0.01443
1.968820.006
0.081400.0051
0.5950+0 0123
8,50
DIRT
0.615120.0076
0.574110.0208
0.612010.0100
0,522610.022
0.5952-0.0150
0.582310.0118
8.33
DeeCCDF
0.8115:0000081
0.47171000085
0.6956::0.0189 89
0.6763:0.0069
0.7850-0.0022
0.6676:20.0113
5.67
QRCDM
008308+000079
0.6406=000026
066111+0.0133
0.699++000091
08001-000085
0.7396+0
3.35
AE-NMCF
0.8267.1000088
0.706510.0225
0.0064
0.755310..001
0.0019
0.7632.10 0.0002
L.33
NMF
0.4102:000057
0.519211.0105
04812:10.0052
0.155810.0009
0.0068
0.4696.10
0.093
5.67
MCF-Gra
0.5677:20.0016
0.676220.0339
0.5393 00066
0,518720.0110
0.62210.0263
0.6675
10.50
MCF-New
0.4738.10.0128
0.590611.0442
0.5602.10.01 28
0.567410.0018
0.0897
0.6478 0.0097
9.83
GNMP
0.4153+0.0006
0.4980+0.0198
0.5012+-0.0056
0.5175+0.0004
0.5294-0.0011 0.0001
0.5468++0.0000 0.0060
7.83
NMMF
0.3986:2000032
0,470+20,0402
0.4749:20.007
0,456420
0.445500.002
0.4724 0226
5.35
RMSE
SNMCF
0.3349100.0029
0.4$3711.0019
0.426620.0076
0.423610.00022
0.3741=0.0069
0.58451000066
3.17
DINA
0.8027=000035
0.6179+0.0005
0.5756++0.0119 19
0.3332+0.0040
0.5281=0.0029
0.58766+0.0105
8.67
DIRT
0,811120.0014
0,191220.0075
04474420.0037
0,1983:000023
0.4872-0.0011
0.1797
6.85
DeepCDF
0.3522-10.0025
0.38741:.00336
0.4433100.004
0.3862-0.0012 0,457510.00440
3.00
QRCDM
0.3555+0.0036
0.00999
+.4607+0.0066
0.559++0.0000
0. 9..88-0000013
0.4213++0.01 96
3.50
AE-NMCF
0.3476110.0088
0.151410.0113
0,406710.0047
0.399620.0000
0.3665-00.024
0,126210.003
1.67
The MCF model with the gradient-hssed method.
The MCF model with the Newtoe- Raphson methed.
We proceed to discuss AE-NMCF's ability to estimate students' knowledge proficiency, which is our major concern. Since the ground truth of students' cognitive levels is unknown, we take cues from the area under curve and use ranking-based metric (knowledge-respoosee consistency coefficient, KRC) as an alternative way to evaluate the diagnostic results. Specifically, for knowledge concept Kcg, we first extract the pair set S- {Ex,,, Stm), n E [0, m E [0, M]} from the testing set D.
8

---

For each pair (Ex,: Stm), we record student Stm 's proficiency level of Kc and the true response score on exercise Ex,. Then, the KRC result on Kc is KRC(Kca) - (x- N (N++11
/(N+N-), N-)
where X Ex: R(n, m), and R(n, m) is the reordered position of the pair (Ex,, Stm)) based on the proficiency level. N+ (N-) denotes the number of records with correct (wrong) answers in S. Finally, we average the KRC values of all the knowledge concepts and denote the average as To- Higher values of rc indicate better performance.
1.2
AE-NMCF
DINA
DespCDD
QRCDM
0.6
0.4
PreSub
Jungles
Quanlang-s
SI.P-Bio-s
SI.P-Hiss
SL.-Eng
Data Set
Figure 3: Students' knowledge proficiency estimations.
Figure 3 illustrates the results of estimating students' knowledge proficiency. We exclude NME, MCF, GNMF, and NMMF due to their known poor capability of cognitive diagnosis. From the results, have: (a) the values in FreSub surpass the other data sets expected, which is mainly due to the strong and consistent connection between exercises and knowledge concepts. (b) Regardless of the relationship types, AE-NMCF delivers comparable slightly improved performance w.r.t. CDM-based approaches and rises well above SNMCF on all data sets. The resulting p-value given by Wilcoxon-ssined rank test between AE-NMCF and SNMCF is 0.031, which also confirms the improvement. (c) While QRCDM shows good diagnostic results, its predicting performance suffers for multiple data sets (see Table 1). This is mainly due the Enowledge-exereise relationship being one-to-one (or one- mne-to-many), which may impede the discovery of implicit correlations.
FreSab
AF-NMCF
0.75 Junyi-s
Quanlang-4
3.55
AE MMET
QÉCDM
3.65
QRCDM
QRCDÓ
DaspCti
0.3
SNMCF
3.55
DINA
DIRT
DespCIS
AMCY
0.29
DDeyyDDD
DIRT
0:55
DIRT
0.55
042
0.55
ACC
SIP- Hlo-s
AE-MMG
SI.-Hisss
AE NE-M
SLP-bog
0.00
ORCDM
3.5
QRCDM
8.1
ORCDM
DeepC
DIRT
$35
DINA
DesyCD
8.2
SNMCI
SMUU
83
DIKY
6.6
BNMCF
0.5
24
0,50
us
ACC
ACC
Figure 4: Model comparison in balancing the two learning tasks via bubble visualizations. The x(y)-axis denotes the prediction (estimation) performance in terms of ACC (re), and the bubble size measures the harmonic mean of ACC and The dash lines locate the models' average performance.
Furthermore, Figure 4 compares the model performance in balancing the two learning tasks, where we use a bubble's horizontal (vertical) position to note the ACC (r.) value for model. In spired by Fi score, we further visualize the bubble size based on the harmonic mean of ACC and r.. Hence, the closer to the upper right corner with larger bubble size, the better the balance achieved. As shown in Figure 4, SNMCF excels at student performance prediction but inadequate knowledge cognitive estimation. In addition, the comparatively low prediction performance of QRCDM compromises

---

its balance ability, especially on Junyi-s and Quanlang-s. AE-NMCF, in contrast, is well above the model average (indicated by dash lines) on all data sets, which achieves the best balance between prediction accuracy and diagnostic ability and works with multiple relation cases.
We close with the demonstration of the effectiveness of our encoder-decoder learning pipeline. As shown in Table 2, we conduct the ablation study by the use of two variants of AE-NMCF, i.e., AE-NMCF w/o Decoder (Encoder) that removes the decoder (encoder) module. The optimization approach is also PG-BCD with appropriate Lipschitz constants. According to Table 2, the ignorance of the encoder (or decoder) process leads to a degradation in predicting and estimating performances, and the performance losses of AE-NMCF w/o Encoder are lower than those of the variant that removes the decoder. The positive results not only suggest the performance boost of the decoder module but also prove the efficacy of the proposed encoder-decoder architecture, which aligns with our expectations to achieve the monotonicity.
Table 2: Ablation analysis of AE-NMCF in student cognitive modeling
Metric
Variant
Data set
FreSub
Junyiss
Quanlangss
SLP-Bic-s
SLP-His-s
SLP-Eng
AE-NMCF w/o Decoder
0.7523+D.0118
1.6261-0.0522
066177+0.0278
06682--0.009
0.7224-0.0062
0:5511=0.0184
ACC
AE-NMCF w/ Encoder
0.81562000000
0.6501:0.0141
0.7269:10.0066
0.7418:000040
0.7745-0.0011
0.7413=0.0126
AE-NMCF
0.8267:00.0048
0.7065-00.228
0.0034
0.7553-900101
0.8072-0.0011
0.7632::00002
AE-NMCF wob Decodar AE-NMCT wo Encoder
0,1197:0.0042
0.1953:2001-41
0.4782:40.081
0.485520.0041
0.4243=0.0025
0.560820.0113
RMSE
0.3668.10.0061
0.507621000338
0.4262:10.007
0.116010.0021
0.4193_000004
0.4494.00.0155
AE-NMCH
0.3476=+0.0088
0..514:00.0133
0.4067:1000477
0.3996=000066
0.3665
0.4262
NMCF wo Decoder AF-NMCF w/o Encoder
0.7202110.0089
0.636211000335
0.589410.0153
0.5730:10.0302
0.5130_00.0150
0.0279
081137+0.0140
0.2266-0.0628
0.5665--0.0963
0.5891
0.5653-0.000
0.4756-00.0225
AE-NMCF
0.8738:00.0177
0.7249:0003300
0.6456.10.0167
0.6875:0000199
0.6313_0.0116
0,7063=00.6578
4.2 Discussion on the Results
We summarize the key findings. First, AE-NMCF improves competing approaches two learning tasks across subject domains, data sparsities, and knowledge-exercinn relationships. Notably, the better estimation accuracy of knowledge proficiency benefits from the explicit encoding of the knowledge level for each student, which is then iteratively improved by the novel autoencoder machine that guarantees that knowledge proficiency can cumulatively cause success in exereises. Second, our purely data-driven model estimates interpretable factors to pinpoint a student's strengths and weaknesses, which is helpful for decision-making as we may tailor learning resources.
However, AE-NMCF's improved prediction and estimation accuracy over the baselines (SNMCF in particular) comes at a price of higher computational complexity (e.g., see Table VI in Appendix F). Nevertheless, AE-NMCF is well-suited to scale-based tests, which are common scenarios in the real world because students are often evaluated for a small set of knowledge concepts, and the need for confidence statistics is one of the critical factors. In addition, we observe that the diagnostic results for some knowledge concepts tend to be overoptimistic due to ignorance of the prerequisite structure (e.g., see Figure VI in Appendix F), and one part of ongoing work is exploiting the knowledge prerequisite structure for AE-NMCF to attenuate this problem.
5 Conclusion
This paper studies student cognitive modeling from a data mining perspective, in which students' knowledge proficiency estimation is our primary concern. To tackle this problem, we propose the AE-NMCF model. Specifically, we root monotonicity in a co-factorization via the carefully crafted encoder-decoder framework. It achieves the assessment of students' knowledge proficiency end-to- end. Considering the nonconvex nature of the objective function with nonnegative constraints, we develop a projected gradient method based on block coordinate descent with Lipsehitz to facilitate model learning, in which theoretical convergence is guaranteed. Experiments on real-world data sets show that AE-NMCF embraces the merit of satisfactory ability to measure students' knowledge proficiency while retaining good performance prediction accuracy. The future work is two-fold: (1) Considering the learning dependency of knowledge concepts; (2) Investigating other efficient parameter learning methods and exploring their scalability.
10

---

Acknowledgments
SY was supported by the Natural Science Foundation of Xiamen, China (Grant No. 3502220227180 YP was supported by the Natural Science Foundation of Guangdong Province, China (Grant No. the Social Science Planning Project of Jiangxi Province, China (Grant No. 19TQ06), and the Provincial Project on Teaching Reform in Higher Education Institutions in Jiangxi Province, China (Grant No. JXJG-18 (JG-118--3), PD was supported by grant from the National Science Foundation of USA (Grant No. IIS-271657). ML was supported by the Young Top Talent of Young Eagle Program of Fujian Province, China (Grant No. F211001 1202B01), This work was also supported by the National Natural Science Foundation of China (Grant No. 62276168, 62472363, 62176225).
2023A1515010880
References
[1] Yan Zhuang, Qi Liu, GaanHao Zhao, Zhenya Huang, Weizhe Huang, Zachary Pardos, Enhong Chen, Jinze Wu, and Xin Li. A bounded ability estimation for computerized adaptive testing. In the 37th Annual Conference on Neural Information Processing Systems, volume 36, 2023.
[2] Shi Dong, Xueyun Tao, Rui Zhong, Zhifeng Wang, Mingzhang Zuo, and Jianwen Sun. Advanced mathematics exercise recommendation based on automatic knowledge extraction and multi-layer knowledge graph. IEEE Transactions on Learning Technologies, 17:776-793, 2023.
[3] Louis V. Dibello, L. Roussos, and W. Stout. 31a review of cognitively diagnostic assessment and a summary of psychometric models. Handbook of Statistics, 26:979-1030, 2006.
[4] Shenbao Yu, Yifeng Zeng, Yinghui Pan, and Fan Yang. Snmef: a scalable non-negative matrix co-factorization for student cognitive modeling. IEEE Transactions Knowledge and Data Engineering, pages 1-14, 2023.
[5] Nguyen Thai-Nghe, Lucas Drumond, Tomás Horváth, Artus Krohn- Grimbbrr,,,,,,he, Alexandros Nanopoulos, and Lars Schmidt- Thieme. Factorization techniques for predicting student perfor- mance. In Educational Recommender Systems and Technologies: Practices and Challenges, pages 153. IGI Global, 2012.
[6] Andrew S Lan, Andrew E Waters, Christoph Studer, and Richard G Baraniuk. Sparse factor analysis for learning and content analytics. Journal of Machine Learning Research, 15(57):1959- 2008, 2014.
[7] Paul R Rosenbaum. Testing the conditional independence and monotonicity assumptions of item response theory. Psychometrika, 49:425-433 435,, 1984.
[8] Antoine Guisan, Thomas C Edwards Jr, and Trevor Hastic. Generalized linear and generalized additive models in studies of species distributions: setting the scene. Ecological Modelling, (772233):99-100 2002.
[9] Ronald K Hambleton and Linda Cook. Latent trait models and their use in the analysis of educational test data. Journal of Educational Measurement, 14(2):75-96 1977.
[10] Jimmy De La Torre. Dina model and parameter estimation: didactic. Journal of Educational and Behavioral Statistics, 1451):115-130, 2009.
[11] Song Cheng, Qi Liu, Enhong Chen, Zai Huang. Zhenya Huang, Yiying Chen, Haiping Ma, and Guoping Hu. Dirt: deep learning enhanced item response theory for cognitive diagnosis. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pages 2397-2400, 2019.
[12] Lina Gao, Zhongying Zhao, Chao Li, Jianli Zhao, and Qingtian Zeng. Deep cognitive diagnosis model for predicting students' performance. Future Generation Computer Systems, 126:252-
262, 2022.
[13] Haowen Yang, Tianlong Qi, Jin Li, Longjiang Guo, Meirui Ren, Lichen Zhang, and Xiaoming Wang. novel quantitative relationship neural network for explainable cognitive diagnosis model. Knowledge-Baat Based Systems, 250:109 109156, 2022.

---

[14] Michel C Desmarais and Rhouma Naceur. A matrix factorization method for mapping items skills and for enhancing expert-based q g-matrices. In International Conference on Artificial Intelligence in Education, pages 141-450, 2013.
[15] Ke Xu, Rujun Liu, Yuan Sun, Keju Zou, Yan Huang, and Xinfang Zhang. Improve the prediction of student performance with hint's s assistance based on an efficient non-negative factorization. IEICE Transactions Information and Systems, 100(4):768-77) 0008-777.200 2017.
[16] Ajit P Singh and Geoffrey J Gordon. Relational learning via collective matrix factorization. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 650-658, 2008
[17] Farnaz Sedighin, Massoud Babaie-Zadeh, Bertrand Rivet, and Christian Jutten, Multimodal soft nonnegative matrix co-factorization for convolutive source separation. IEEE Transactions on Signal Processing, 65(12):3179 -1909, 2017.
[18] Adrian Flanagan, Were Oyomno, Alexander Grigorievskiy, Kuan Tan, Suleiman A Khan, and Muhammad Ammad-Ud-Din. Federated multi-view matrix factorization for personalized recommendations. In Machine Learning and Knowledge Discovery in Databases; European Conference, pages 324 224-347,1 2021.
[19] Wen Wen, Wencui Wang, Zhifeng Hao, and Ruichu Cai. Factorizing site-eeterogeneous markov transition for temporal recommendation. Neural Networks, 159:84-96, 2023.
[20] Panagiotis Symeonidis, Luca Bellinazzi, Chemseddine Berbague, and Markus Zanker. Safe and effective recommendation of drug combinations based on matrix co-factorization. In 2023 IEEE 36th International Symposium on Computer-based Medical Systems, pages 634-639, 2023.
[21] Bo Yang. Yu Lei, Jiming Liu, and Wenjie Li. Social collaborative filtering by trust. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1633-1663144, 2016.
[22] Jiho Yoo and Seungjin Choi. Bayesian matrix co-factorizzaton: variational algorithm and cramér- bound. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 537-552, 2011.
[23] Wei Wang, Yan Huang, Yizhou Wang, and Liang Wang. Generalized autoencoder: a neural network framework for dimensionality reduction. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 490-497, 2014.
[24] Pengfei Liu, Xiaoming Sun, Yang Han, Zhishuai He, Weifeng Zhang, and Chenxu Wu. Ar- rhythmia classification of Istm autoencoder based on time series anomaly detection. Biomedical Signal Processing and Control, 71:103228, 2022.
[25] Dong Gong, Lingqiao Liu, Vuong Le, Budhaditya Saha, Moussa Reda Mansour, Svetha Venkatesh, and Anton van den Hengel. Memorizing normality to detect anomaly: memory- augmented deep autoencoder for unsupervised anomaly detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1705-1714, 2019.
[26] Viet-Tuan Le and Yung Kim. Attention-based residual autoencoder for video anomaly detection, Applied Intelligence, 53(3):340-3325, 2023.
[27] Michel Desmarais. Mapping question items to skills with non-negative matrix factorization. ACM SIGKDD Explorations Newsletter, 13(2):30-36, 2012
[28] Eric R Ziegel. The elements of statistical learning. Technometrics, 45(3):267--2) 2003.
[29] Prateek Jain and Purushottam Kar. Non-convex optimization for machine learning. Foundations and Trends in Machine Learning, 10(3-4):122333,, 2017.
[30] Chih-Jen Lin. Projected gradient methods for nonnegative matrix factorization. Neural Compu- tation, 19(10))775-2777 27799,6-777.22 2007.
[31] Dimitri P Bertsekas. Nonlinear programming. Journal of the Operational Research Society, 48(3):334-334, 1997.
12

---

[32] Juha Heinonen. Lectures on Lipschitz Analysis. University of Jyväskylä, 2005.
[33] Yangyang Xu and Wotao Yin. A block coordinate descent method for regularized multiconvex optimization with applications to nonnegative tensor factorization and completion. SIAM Journal Imaging Sciences, 6(3):1758-1789) 2013.
[34] Haw-Shiuan Chang, Hwai-Jung Hsu, and Kuan-Ta Chen. Modeling exercise relationships in e-learning: a unified approach. In International Conference on Educational Data Mining, pages 532-535, 2015.
[35] Lu, Yang Pian, Ziding Shen, et al. Slp: multi--iimensioall and consecutive dataset from k-12 education. In Proceedings of the 29th International Conference on Computers in Education Conference, pages 261-266, 2021.
[36] Daniel Lee and H. Sehastian Seung. Algorithms for non-negative matrix factorization. In International Conference on Neural Information Processing Systems, pages 556-562, 2000.
[37] Hyekyoung Lee and Seungjin Choi. Group nonnegative matrix factorization for eeg classifica- tion. In Artificial Intelligence and Statistics, pages 320-327. PMLR, 2009.
[38] Koh Takeuchi, Katsuhiko Ishiguro. Akisato Kimura, and Hiroshi Sawada. Non-negative multiple matrix factorization. In Proceedings of the 23rd International Joint Conference on Artificial Intelligence, pages 1713-1720, 2013.
[39] Janez Demšar. Statistical comparisons of classifiers over multiple data sets, Journal of Machine Learning Research, 7:1-30, 2006.
[40] Edwin F Beckenbach and Richard Bellman. Inequalities, volume 30. Springer Science Business Media, 2012.
[41] Roger A Horn, Roger A Horn, and Charles R Johnson. Topics in matrix analysis. Cambridge University Press, 1994.
[42] Krzysztof Kurdyka. On gradients of functions definable in o-minimal structures. In Annales de I'institul Fourier, volume 48, pages 769-783, 1998.
[43] Steven G Krantz and Harold R Parks, primer of real analytic finctions.. Springer Science Business Media, 2002.
13

---

Appendix
A
An Example of the Encoder-Decoder Process
Figure provides an example the encoder (Left) and decoder (Right) processes AE-NMCR. First, the encoder targets the specification of student Stz 's knowledge proficiency vector (A.3). Here, we omit matrix B for conciseness. Second, the decoder reconstructs St3's response on exercise Ex3 (i.e., X33) via the specified knowledge proficiency. Thus, by explicitly encoding the knowledge level for each student the encoder, which then iteratively improved by the decoder that guarantees that knowledge proficiency can cumulatively cause success in related exercises, monotonicity can be achieved.
Encoder process
Decoder process
BIT
A-3
08.
0. DIN D.
KGT Ko2 KoB
X
3-t Bd Ed Est Eu6
0.303 0.000110
A.
0.5
M
Det Fx? DVD EAS
DUU SH
Ket KrP 00
09
Mor Kocko
KG1 KcB 160 0 OLd D.
Low-dimensional matrices
Cognitive factors (knowledge proficiency)
Low-dimensional matrices
Input data
Input data
Cognitive factors
(the scoring matrix)
Figure An illustration of the encoder and decoder processes, where we highlight the target entries in the color cells.
B Parameter Learning for AE-NMCF
The parameters AE-NMCF include B, U, E, V, and M, where B B BT,,"... ....... ,BJ.J,
U U,, U.2..... ... U. and M [un, M2, AN]] T Because of the nonnegative constraints on U. E, and V, we employ the projected gradient via block coordinate descent (PG-BCD) for the parameters solution, and the subproblems of B.:., U. U.m E, and pn are
min OAF(BR:) Σ Bm:20
log Pr(X )
2 B 112 + Q O (B.: E., V)Il2
(a)
2
min 20 20
Σ
log Pr(X nm + W O (X.m EU.
(b)
min E20
OE) = W о X
EUJ
Q c (B EV)
(c)
O V) Σ - log Pr (x(mm) + ||Q ② (B EV
min V20
(d)
R,
min ONF(HN) = Σ
log Pr( Pr(XX.m)
(e)
m
where Pr(X,,)) = ( mm
(1-Xmm) X ( (1-XX..)) and A,m. = B.. + pn- Considering
the computation burden of the Hessian and the calculation difficulty for the probit link function when employing second-order approaches, we build our parameter learning algorithm on first-order methods. Hence, the gradients for problems (a) (e) are
VO VOAK (Bn:) = -E Eum
Enm Xnm Φ ((Anm)) JUT V + 2 O Br,
V)] + +
m
14

---

VOAF(Umm - 3m Xmm p(4mm) VB m + +2E [ Wim O EU m)) W x
72
VOAF(E)= = 2[ @ (EU) W X + 2/Q e (EV) QQB
VOAF(V) - Σ
nm X (( ( U, 2E 2 [QV(EV) Q
(n,m)
VOAF(Rn) E [Xmm ((Amm))
where Enm 7(anm am Based on the gradients above, searching for the optimum point
MiA...
easy. Taking B,: as an example, in each iteration
- 1,2, ..., the gradient step is
B (1+1)
( n JBB,.. AF (Bn.)
(1) VO
n:
where we use half-wave rectifier [x]+ max 10-15 to ensure the nonnegativity, and (I)
is a suitable step size for B.., which is determined by the Lipschitz constant in this paper. BB..
C Complexity Analysis
This section discusses the time complexity of PG-BCIDLLipschit.. The analysis is based on the update rules of B.:- Umm E, V, and N/: (ISnEN, 1 M). For simplicity of exposition, the case where the number of students is larger than that of knowledge concepts, which commonly occurs. For each block B, the Lipschitz constant takes (MK?2 operations, and the operations of the gradient are bounded by ((КTТ) for each iteration." 3 Consequently, the cost of the variable B, which contains N blocks, is bounded by O(MN32). Other parameters can analyzed similarly, summarized in Table I. Hence, the overall cost is the number of iterations needed for convergence times OMMK3)), where the latter term is the complexity of each iteration.
consider
Table Computational operations (U, E, V, M) for each iteration
U
E
V
M
O(NNKT)
O(min(M, NJ-MN)
(mma((KT, min(M, NJ)-MN)
O(MNKT)
D Proof of Theorem 1
To prove Theorem 1, we first introduce scalar Lipschitz constant in Lemma [6].
(x)) Lemma Let g(z) (())+ x IR, where ((x) the inverse function. Then, for y, 2 E IR, we have
- Loly- z/,
where Lp = 1 is the scalar Lipschitz constant for g(x).
Next, we prove Theorem 1. For the sake of brevity, we assume that all entries in the student scoring matrix X are observed, i.c., Qg = {I..... .......,,.x... .N)x(1.... (1..... .M:: the extension to the case with missing entries in X is straightforward. In what follows, (N.)) and ((-) are assumed to operate element- wise on the vector or matrix. We start with the gradient of OFF (B.:)..) in Theorem 1, as shown below
M
([(Am)
VOAF( = Σ
(2X 1)UT m +2(Q.. - Qn:O V) + ByBas ((A,m)
B...)
"Since we focus on the computational time in terms of the size of the input data, the complexity of the probit link function is omitted.
15

---

where Am =2X nm 1) V mm + M2)). The first term O (B,:) can be rearranged as
(A
(2X,11 - -V +... (( (2XmM- I)UM ((AnM)
VOl (Bn:)
@(Am1)
(2Xm1 - 1)
[A(1))
((An1)
((AnM)
(2X I)UM V
N(An))
Cns
D(An:)
where An: [Aml: ... AnM), and we have
An: = (2X,11 (Bx V U.
V. + pn), (2X 1), V V U. un)l
BB, (2X 1)V U. (2XnM 1)v UM] + [(2Xn1
1)pm. ,2X 1)mm)]
=B,, + X pm
Therefore, the gradient AO Bn:) can be rewritten
(Bn: ((B,,. n: CT X Cn + 2.Q...B.. QE, V)) + +yB.:
(B. CT X
We can now establish an upper bound of the 12-morm of the difference between the gradients at two arbitrary points y and z of VO follows
[[VOaf(y) AZ
N(yč + Pn Cn + 2 Oy - Q V)
p(yC + +Xm
NacC X
Cn 2[Qn: Cz Qn: Em: V)] + vy -0z
D(2C + x
2
< 1|-1| N(y + Xun)
N(CCT + Xua)
C, ((zC,T + X
C
( + kn
+ +| - Qn: V)] O0... - Qn: En: V)
7/ly - 2/12
(f)
N(yC + X
N(xC, + +X
(C.)
4 X (2C,, + +
+2 +2| - QmlOz/2 + -
-
(g)
(on (yč, + (())) (aC,) + X n:
+ Qnellllly z/12 + +llyy z112
(h)
< 1 (Cn)ly 2112 + 2/ 2n:Qullelly - 2||2 + 7lyyy - 2||2
ii)
K
L +2
Q Σ
+ Ily 2/12-
Here, (f) uses the triangle inequality of a norm. (g) follows the Hölder inequality [40], and an (-) denotes the corresponding maximum singular value. The bounds (h) follow from Lemma i and the inequality of Hadamard products (c.g., see [41, Section 1.). The final inequality (i) follows from the fact that flipping the signs of the rows (or columns) a matrix does not affect its singular values.
Note that this proof assumes that the scoring matrix X is fully populated. We can easily adapt to the case of missing entries in X, by replacing the matrix C, to OZ which is the matrix containing the
16

---

rows of Cn corresponding to the observed entries indexed by the set I = {m (n,m) E Ωo}. We omit the details for the sake of brevity.
The proofs for the remaining subproblems for Umm E, V, and Pn (I I< follow analogously, and Table summarizes the Lipschitz constants of all parameters, where 1 MN denotes the M X N all-ones matrix.
Table II: The Lipschitz constants of parameters
Parameter
The Lipschitz constant
1 Oni
B
Lpo V) +
Σk
+y
Umm
[VB
(VB + (B)
N
W
E
(B)
1M.N) + (E)
Lp Em== (2Xmm 1)2
Pn
E Proof of Theorem 2
Since minimizing AE-NMCF follows the multi-block coordinate descent solution, and the subprob- lems can correspond to BCDs with update (1.3a) [33], we use the results laid by Xu and Yin [33, Lemma 2.6, Corollary 2.7, and Theorem 2.8] to prove the convergence of PG-BCD BCD++ipschhh,,z To this end, we show that the objective function of AE-NMCF, i.c., problem (j) meets all assumptions needed for the convergence results in [33].
N Σ
min OAF ++ B.U.E.V.
W ④ X - - + |QQ (B EVI
B
B
1
2
n=1
s.t. B 0,00 0, 2 0.
We start by discussing Assumptions 1 and in [33]. For Assumption 1, since all the terms in problem (j) nonnegative, we have OAK -C0, which has a lower bound of O. For Assumption by inspecting the form of the individual subproblems, we see that they are strongly convex. Therefore, Assumptions and in [33] met.
We then provide that problem (i) also meets the additional assumptions in [33, Lemma 2.6), which requires (a) the Lipschitz continuous of the gradient of the block multi-convex function ff on any bound set and (b) the Kurdyka--ojasiewiicz (KL) inequality [42]. To do so, for (a), let
(B, U, E, V, M), and
rewrite the objective function of problem (i) as follows
OxF(O)= = - W 3 X - EU + |QQ (B
---
7 +
Bu:
1
2
1=1
+ Σ
5(0 <0)
Ee
(ir(O)
Σ ((O))) <0),
where Ozy denotes the i-th element of θ 5(2) is an indicator function, and we have 8(2) = 00 < 0 and O otherwise. We now show that the gradients of the smooth part of Oxr(O), i.e. (O). Lipschitz continuous in dom O
17

---

Let @² @ E dom OAP), we have
|VO V
VOL+(O
Σ (VO):(%)
VO (O))))
Σ
|0
(L/)² | O
where = = m J. ( [B, V, M). Recall that the bounds on the Lipschitz constant
corresponding to E and V are shown Table II. For B, let B = [B B ". BN.]] we have
IF (B%) VO AF (B2)
OF (B)) VO (B
K
L (VV +)+2 Σ
Q
+7
[B%
B
s (L,)) VI + 2//Qn: + +7) | B=112
where the last line states that the maximum singular value of a matrix is no greater than its Frobenius norm. Similarly, the Lipschitz constants for U and M are L,P VB I E W Wmlllz and
Lpl 12X..
1, respectively. Therefore. (O) is Lipschitz continuous in dom (FF).
For (b), using [6. Lemma 7], the first term (i.e., the negative log-probit likelihood function - of OAF in problem (j) is real analytic, which is based on the fact that compositions of real analytic functions are real analytic [43]. In addition, the second and third terms with Frobenius norms OAF. plus the regularizer, are all polynomial functions, therefore also real analytic. Hence, the objective function OA is real analytic and satisfies the KI. inequality, consequence of [33, Section 2.2]. By setting the extrapolation weight of = 0 in [33], we can conclude that the PG-seDtLipsehhtz algorithm converges a local minimum. Furthermore, PG-BCD+Lipschitz converges globally if the initial point is close to the global minimum [33].
F Extended Details of Experiments
In this section, conduct follow-up experiments to enhance the effectiveness of AE-NMCF. The statistics of the data sets are summarized in Table III, and the implementation details are described briefly below:
We deploy the competing models using the best publicly available implementation with Python 3.8 on an Ubuntu server with Core 19-10908 3.7 GHz and 128 GB memory.
For AE-NMCF, set the number of iterations and the stopping threshold 500 and 5 to guarantee convergence. The hyperparameters and y are set in Section F.6.
For each dataset, reshape the response logs to the scoring matrix and utilize 80% train/test split. All models' performances are averaged over 5 repeated trials to ensure faimness.
F1 Statistical Hypothesis Test
We first conduct the hypothesis test for the student performance prediction and the knowledge proficiency estimation. Table IV shows the details of the paired t-test results, where each entry denotes the p-value of the AE-NMCF with the baseline in terms of a given metric. According to Table IV, AE-NMCF shows significant difference at the 5% level with the baselines in most cases. We can conclude that the prediction (estimation) performance of AE-NMCF significantly different from that of the competitive models.
18

---

Table III: The statistics of data sets
Statistics
Data Set
FieSub
Junyi-s
Quanlang-s
SLP-Bio-s
SLP-His-s
SLP-Eng
# Student
536
1,091
50
100
1057
360
# Exercise
20
9
107
129
326
362
# Knowledge concept
9
14
7
14
19
Subject
Math
Math
Math one-to-many 68.67%
Biology
History one-to-many 84.28%
English
Relations'
many-to-many
one-to-one 75.03%
one-to-many 54.92%
one-to-many
Sparsity
0%
96.92%
The relationships between knowledge concepts exercises.
The sparsity of student scoring matrix.
Table IV: Paired t-test for the prediction (estimation) results of AE-NMCF with other methods
Data set
Metric
Comparison
FreSub
Junyi-s
Quanlang-s
SLP-Bio-s
SLP-Hisss
SLP-Eng
NMF
0.000
0.006'
0.000'
0.000'
0,000'
0.001
MCF-Gra
0.000
0.000'
0.000'
0.000'
0,000'
0.000'
MCF-New
0.000
0.000
0.000'
0.000
0.000
0.000
GNMF
0.000
0.008
0.000'
0.000"
0.000
0.000'
ACC
AE-NMCF
NMMF
0.000
0.040
0.000'
0.000
0.000'
SNMCF
0.002
0.274
0.009'
0.001
0.238
0.041
DINA
0.023
0.000
0.000'
0.000
0.000
0.000"
DIRT
0.000
0.000
0.000
0.000
0.000
0.000
DeepCDF
0,001
0.000
0.002
0.002
0.000
0.000
QRCDM
0.140
0.009
0.000
0.000
0.049
0.047
NMF
0.000
0.003
0.000
0.000
0.000
0.000
MCF-Gra
0.000
0.000
0.000
0.000
0.000
0.000
MCF-New
0.000
0.001
0.000
0.000
0.000
GNMF
0.000
0.003
0.000
0.000
0.000
0.000
RMSE
AE-NMCF
NMMF
0.000
0.334
0.000
0.000
0.000
0.012
SNMCF
0.072
0.866
0.022
0.001
0.060
0.014
DINA
0.000
0.000
0.000
0.000
0.000
0.000
DIRT
0,000
0.019*
0.000'
0.000
0.000
0.000"
DeepCDF
0.248
0.002
0.000'
0.000'
0.000'
0.000
QRCDM
0.060
0.020
0.000'
0.000
0.193
0.435
SNMCF
0.000
0.000
0.005'
0.000"
0.000
DINA
0.000
0.000
0.000
0.000"
0.000
0.441
KRC (r.)
AE-NMCF
VS.
DIRT
DeepCDFF
0.000
0.000
0.001
0.008
0.010'
0.001'
0.000'
0.001
0.000
0.000
0.006'
0.001
QRCDM
0.009
0.012
0.226
0.023
0.009
0.141
Significunt difference the 5% level.
F.2 Nemenyi Test
We conduct the Nemenyi test [39] to present the comparison of the proposed AE-NMCF model with the baseline approaches. The Nemenyi test shows the differences between the average ranks among all the compared methods, and any two of which are significantly different their average ranks differ by at least one crucial difference (5% in this paper). As illustrated in Figure II, it is obvious that the AE-NMCF model performs the best in terms of ACC and RMSE, which demonstrates its effectiveness in student performance prediction.
F.3 A Case Study for Diagnostic Comparison
To get a sense of diagnostic improvement of AE-NMCF for data mining techniques (compared with SNMCF), we present the diagnostic results for case students on FreSub, Quanlang-s, and Junyi-s, which covers all typical crowledge-exerciss relationships.
The mastery degree of each knowledge concept is normalized to range of [0, 1).
19

---

Crtical Distance-6.180 1639
Criccal Disanceed
MCF-Now
DeepCDF
(a) ACC
(b) RMSE
Figure II: The CD diagrams of all the methods in terms of ACC and RMSE.
KG
0.0149
0.2882
0.0300
KG
1,000
1:400
LIXX
Kc
38
Ke
K2
03455
0.32M4
0,9799
0.0000
Kg
0.0078
1,0000
2000
0.7125
KC
0J448
0.3286
0,0650
0.0000
Ko
0.0143
1,000
1000
110000
K0g
K0
004477
0.3274
0.0220
0.0000
03
1,000
1,0000
1.0000
1,0000
92
Kn-
0
0.2674
0.9421
0.0000
Kg
0,0000
0,0000
1:000
0
KG
KG
0.1
51ys
Si
813
127
100
(a) Diagnostic results
(b) Diagnostic results
(AE-NMCF)
(SNMCF)
Figure III: Case students cognitive diagnostic results (AE-NMCF vs. SNMCF) FreSub.
FreSub. Figure III compares four case students' diagnostic results from AE-NMCF and SNMCE, respectively, where each numerical value student's knowledge proficiency specific knowledge concept. In addition, we show the corresponding scoring matrix Table V. For student St we can observe that both AE-NMCF and SNMCF give suitable diagnostic results since she answers all the exercises correctly. However, for students Stys and S1342, Table V shows that they only give the right answer Exg and fail in the remaining exercises, which means that Stşs and St342 not grasp all knowledge concepts. The diagnostic result given by the AE-NMCF model confirms this fact, while SNMCF gives confusing result. In addition, the response log of St107 indicates that she needs to continuously make progress on most of the knowledge concepts to improve proficiency levels, but SNMCF argues that St1o7 has mastered most of the knowledge concepts, which does not square with the facts. In summary, the diagnostic outputs provided by AE-NMCF align with our expectations.
Table V: The corresponding scoring matrix on FreSub
Student
Student
Exercise
Exercise
Stys
St107
Styos
St342
Stys
St107
Sti
St342
Ex
O
Ex1
0
0
O
Ex2
0
O
Ex12
0
0
(
Ex3
O
Ex13
0
0
0
Ex4
0
1
1
0
Ex14
0
0
0
Ex5
0
O
Ex15
0
0
0
EX6
0
1
O
Ex16
0
0
0
Ex7
O
Ex17
0
0
0
Exs
0
1
O
Ex1s
0
0
0
Ex9
0
Ex19
0
0
Ex10
0
0
0
Ex20
0
0
0
Quanlang-s. Figure IV shows three case students" knowledge proficiency based on radar charts. To facilitate comparison, also label the student's answer accuracy rates (the ratio of correctly
20

---

AB NMCF
SNMCF
AECF
SNMCF
AEC
SNMCF
KcS
Ko4
KcS
Ked
Kc5
Kcl
Kc6
Kc3
Kc6
Kc3
Kc6
Kc3
Kc7
Kc2
Kc7
Ke2
Kc7
Kc2
Kc8
0.2 0.6
1.0
Kel
Ke8
0.0
0.6
Kel
Kc8
0.0 0.6
0,9
Kel
Kc9
Kc14
Kc9
Kc14
Kc9
Kc14
Ke10
Kc13
Kc10
Kc13
Kc10
Kc13
Kc11 Kc12
Kcll Kc12
Kcll Kc12
(a) St50 (46.67%)
St33
(66.36%)
(c) St42 (84.62%)
Figure IV: Diagnosis results of three case students between AE-NMCF and SNMCF Quanlang-s.
answering all exercises), e.g., 46.67% for student Stso- Intuitively, the proficiency levels of St42 should be the highest because of the top accuracy rate, and Sts0 is at the lowest level accordingly. However, SNMCF gives extreme estimation, which overestimates the ability of Stso (or Sta3); and consequently, the cognitive diagnostic ability is limited. Instead, the proposed model gives reasonable results. We conclude that AE-NMCF provides richer information the diagnosis than SNMCE.
Junyi-s. Different from FreSub, there is substantial missing data in the scoring matrix for Junyi-s, with only 24.97% of its entries observed. Given that the relationship between the knowledge concepts and exercises is one-to-one, we show each knowledge proficiency (provided by AE-NMCF and SNMCF, respectively) with its corresponding answer record of three case students in Figure V, where each subgraph consists of two parts the response (left) and the knowledge proficiency (right). From Figure V, we have the following observations:
63716
kah(220)
Koe(be))
0345
(ed)))))
DAIDS
Kol (Txf)
Red (Tc))
60030
kel (555)
6344
(222
(21)
(21
Kcl (ct(2s))
Response
proficinccy
BReepussc
Knowllagee proficiancy
BRepusse
Knowlolge profiiiayy
(a) Styis (AF-NMCF)
(b) Sto1 (AF-NMCF)
(c) Sto11 (AE-NMCF)
0002
Ket (Dud)
1ea0s
Kes (ba)
(2:0)
kc7 (b2))
127
3000
Ko7
Team
Kes (b4))
1,00
Schibol
Kan
Kc5 (b6)
1600
Xct (Bc5
4000
Kc5
32k
(o4()
0083
Kc
Kct (P45)
(235)
Sid (2x2)
Kel
(c1(
Kil (21()
Responec
Knowlolge proficincy
BReepunsc
Knowlalgge proficiancy
Reepoose
KKnowladge prodiieeecy
(d) Str1s (SNMCF)
(e) Sta1 (SNMCF)
Stsıı (SNMCF)
Figure V: Case students' cognitive diagnostic results and the corresponding answer record on Junyi-s.
First, Figure Va and Figure Vd illustrate that both AE-NMCF and SNMCF provide reasonable diagnostic results since student Sty18 has only wrong answer record, which lacks far too much information available for diagnosis.
Second, it can be seen from Figure Vb and Figure Ve that although student Stgı responds correctly to the given exercises (i.e., Exg, Ex,, and Ex,), there are still some exercises that Stg1 has never answered before (c.g., Ex,). However, the SNMCF model asserts that the student has completely mastered all knowledge concepts. In contrast, AE-NMCF makes more sense than SNMCF because the new model considers the uncertainty of the missing values of the unanswered exercises.
Finally, for student Stş11 (see Figure Ve and Figure Vf), we observe that the SNMCF model still gives more illogical diagnostic results than AE-NMCF because the response log shows that Stzn makes mistakes in some exercises (e.g., Ex4), which indicates that she needs to timely learn the corresponding knowledge concepts (c.g., Kc₃).
21

---

F.4 Cognitive Diagnosis Visualization
We proceed to visualize and investigate the diagnostic results of a student as case study, which provides useful insight into the estimation outcomes of the proposed model. Figure VI displays the student's knowledge proficiency with the corresponding answers on Quanlang-s. As observed, AE-NMCF gives interpretative and meaningful diagnostic results, based on which the student can determine her strengths and shortcomings. For example, the student has good grasp of all knowledge concepts except for Ke12 (exponentiation of rational numbers). Observing her responses related to Ke12, we notice that the student only tries very few relevant exercises. It suggests a timely study of KC for the student. Based this visualization, Al-based tutoring systems could provide her with personalized remedy plans for improvement.
However, we see that the diagnostic result of Keg is overoptimistic, not only because she made many mistakes in the related exercises but also due her low proficiency in the prerequisite knowledge concepts (e.g., Kc7). Recognizing this limitation, an intuitive work-around is to exploit the knowledge prerequisite structure for AE-NMCF attenuate this problem.
65
Knowledge Proficiency
Knowledge Name
0340
0,817
0.798
0,778
01,709
(0.778
0.79)
Kc2
Kel
rasional numbers
positive negative numbers
0.5-
Kc3
addition of rational numbers
Kcl
opposite numbers
Kcd
number line
0.632
Kc6
comparing rational numbers
numbers
Kel
Kc3
Kc5
Kc7
Ke9
Keto Kell
Ke12 Kel3 Kel4
Kc8
Resparsess
Ke9
operations addition
Kc10
multiplication of national numbers
Kell
recipmeca
Kc12
exponentiation of rational numbers
Kc13
division of rational numbers
Kc14
mixed operations with rational numbers
Figure VI: Diagnosis visualization of a case student Quanlang-s via AE-NMCF. The bottom left shows her responses to related exercises. The circles with green (red) colors represent right (wrong) responses, and the hollow circles denote the absent responses.
F.5 Comparison of the Step-Size Search Methods
As noted earlier, the "Armijo rule along the projection arc" (Armijo rule) is another step-size solution. In this section, we show the compared performance between the Lipschitz search and the Armijo search on FreSub, Junyi-s, and Quanlang-s, which covers all types of knowledge-exercise relationships. We first check the convergence in Figure VII, which sees that both the search solutions converge to stationary point; however, the Armijo search at first quickly decreases the objective function value but slows down in sequence, which takes more time to converge.
Niterations
#sterations
#iterations
Armijo search
Armioo search
Armio search
Objeative Value
Lipschitz search
Doppective Value
Lipschitz scarch
Objectivee Value
Value
Lipchitz search
Objective Value
Objectio
#iterations
#
Fitetatione
(a) FreSub
(b) Junyi-s
(c) Quanlang-s
Figure VII: The number of iterations objective values for the Lipschitz search and Armijo search on FreSub, Junyi-s, and Quanlang-s.
Furthermore, we the number of iterations (the smallest one the two strategies) and present the compared performance in Table VI. We observe that all the methods exhibit similar performance for
22

---

Table VI: The comparative results between the Lipschitz search and Armijo search
Dataset
Strategy
#iterations
Time (minutes)
Objective value
ACC
Armijo search
525,41
4496.3772
0.8307
0.8889
FreSub
Lipschitz search
3.83
4100.9518
0.8344
0.8951
Armijo search
425.02
2134.1798
0.7477
0.6205
Junyi-s
25
Lipschitz search
0,46
1215.7141
0.7383
0.7243
Armijo search
172.70
3238.7009
0.7127
0.5942
Quanlang-s
45
Lipschitz search
0.73
2050.8577
0.7336
0.6438
student cognitive modeling, while given the same number of iterations, the Lipschitz search achieves the fastest convergence while maintaining relatively small objective function value.
A00
-CC
600
ACC &
ACC
oud
0.5
(a) FreSub
(b) Junyi-s
(c) Quanlang-5
M-0CC
-000
ACC
445
(d) SLP--io-S
(e) SLP-His-s
SLP-Eng
Figure VIII: Sensitivity analysis of parameter T on the data sets.
ACC
6.44
AGC
ACC
121
671
(a) FreSub
(b) Junyi-s
(c) Quanlang-s S
AC0
077
0.5
ACC
ACC
677
011
(d) SLP-Bio-s
(e) SLP-His-s
(I) SLP-Eng
Figure IX: Sensitivity analysis of parameter ? on the data sets.
23

---

F.6 Parameter Sensitivity Analysis
Finally, there are two parameters in the AE-NMCF model: (1) the number of latent factors T (i.e.. the rank of the nonnegative matrix co- co-factorization) and (2) the regularization parameter 7. Since T leads role in achieving the approximation effect, begin by discussing T, followed by y.
Effect of parameter T. We use the grid search rule to tune the value of parameter
min(N, M, K), and consider the effect in terms of ACC and Tz- The results summarized in Figure VIII. It can be seen from the figure that as the value of T increases, ACC and Te share a similar decreasing tendency. Therefore, choose the value that balances the two types of tasks, ie., we set T = 3, 3, 2, 1, for FreSub, Junyi-s, Quanlang-s, SLP-Bio-S, SLP-His-s, and SLP-Eng respectively as the tuning results.
Effect of parameter y,. Based on the best value, we proceed to find the best value for 7, which controls the degree of avoiding the ill-posed problem for B. For all the data sets, we perform the grid 2, 3, 4, 5, 10, 15, 20, 25, 50}. By observing the results in Figure IXa, we that the ACC (rc) leads drop the beginning, followed by sharp rise after y (7 4), and then slightly fluctuates in the sequence. Therefore, we set y = 10 for FreSub. For Junyi-s and Quanlang-s, considering the fluctuation for the value, choose Y = 2, 4 respectively avoid too much regularization. Similarly, we use 7 2, 1, 3 as the tuning result for SLP-Bio-s, SLP-His-S, and SLP-Eng, respectively.
search with the range of 21,2,
24

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The research scope is clearly defined, and we list the main contributions the end of the introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We discuss the limitations (see Section 4.2). In addition, since student cognitive models have made diversified assumptions about how knowledge concepts contribute to generating a student's response an exercise, including disjunctive, conjunctive, and additive hypotheses, the choice of assumptions depends on the context and measurement scenario, which still open question. This paper adopts the common additive hypothesis (see the decoder process in Section 3.1).
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model vell-specificattin, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
25

---

3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: We provide proof details of Theorem 1 and Theorem 2 in Appendix D and Appendix E, respectively.
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provide the data set and baseline descriptions in Section 4, well the implementation details and the parameter sensitivity analysis Appendix F.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
also be provided via detailed
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may tricky in some cases, in which case authors welcome describe the particular way they provide for reproducibility.
26

---

reproducible, they In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification:
We attach the code and data sets to the supplemental material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.c cc/ Stalcccguidee//odeeeeeee
cemiionoollyy for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: / /nips. htc/publicgguuuesss...eeepse (rodeeubmissiooPooo
for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/eetails
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Please see the implementation details and parameter sensitivity analysis in Appendix F.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification:
We report the error bars (standard deviations) of the results in predicting student performance (Table 1) and knowledge proficiency estimations (Figure 3) in the main manuscript. In addition, we provide the results of statistical hypothesis test in Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
27

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Please see the implementation details in Appendix F.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics http:: /ttpppip////uiis...... httpr//p.....ppoeeectccccccceeee
Answer: [Yes]
Justification: Except for the public data sets used in the paper, we also use private data set (i.e., Quanlang-s), which was made available to under agreement with Quanlang education company privacy protection, and fairness.
whose terms included informed consent,
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
28

---

The new cognitive model will potentially improve the automated compre- hensive understanding of students' knowledge learning and benefit numerous intelligent educational tools.
Justification:
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g.. pretrained language models, image generators, scraped datasets)?
Answer: [NA].
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best. faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All the data sets and baselines used in the paper are cited by listing the references and websites.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
29

---

The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswi wwwodee com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's ' creators.
13. New Assets
Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Question:
Answer: [NA].
Justification:
The paper describes the proposed new cogmitive model in detail (see Section 3).
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the datasct/cod//model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA].
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA].
Justification: The paper does not involve crowdsourcing research with human subjects.
Guidelines:
30

---

The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
31


Reviews:
[{"review_id": "zSMMR59u9S", "reviewer": "Reviewer_Qt1t", "summary": "The paper studies the problem of predicting student grades based on the past student response to the question. The proposed method falls into the research of matrix completion. The novelty lies in (1) a new matrix co-factorization, and (2) the proof of the convergence of the proposed gradient descent method.", "questions": "1. Could authors compare the proposed methods to other matrix completion methods, which were originally developed to solve Netflix challenges?", "limitations": "See the weakness section.", "rating": 7, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "1. The paper is easy to understand, and the proof seems to be correct.\n2. The proposed method is tested in the real-world dataset, and it shows promising results", "weaknesses": "1. Not sure if the proposed approach can be applied to other datasets.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank for your positive evaluation. For your concerns regarding the performance comparison, we compare our model (AE-NMCF) with several variants of collective matrix factorization (MF) methods suggested by Reviewer Px4s, including CMF [Singh2008], GNMF [Lee2009], and NMMF [Takeuchi2013]. In particular, CMF and NMMF are designed for rating predictions on e.g., Netflix Prize data. We report the compared results in Table 2 in the global PDF file. The experimental results show that AE-NMCF achieves the optimal performance across all data sets, and the performance is noteworthy in the sparse cases, especially in SLP-His-s and SLP-Eng. \n\nHence we have compared our new method with the latest ones and those used for solving Netflix challenges.\nIn addition, the data sets are representative of many relevant applications.\n\n[Singh2008] Singh, A. P., \\& Gordon, G. J. (2008, August). Relational learning via collective matrix factorization. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 650-658).\n\n[Lee2009] Lee, H., \\& Choi, S. (2009, April). Group nonnegative matrix factorization for EEG classification. In Artificial Intelligence and Statistics (pp. 320-327). PMLR.\n\n[Takeuchi2013] Takeuchi, K., Ishiguro, K., Kimura, A., \\& Sawada, H. (2013, June). Non-negative multiple matrix factorization. In Twenty-third International Joint Conference on Artificial Intelligence.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iOgaIvw4wK", "reviewer": "Reviewer_LdEU", "summary": "The authors propose a novel approach to student cognitive modeling based on nonnegative matrix factorization via the use of sparse autoencoders. The authors propose an algorithm to solve the estimation problem in their framework, and provide a formal analysis including a convergence proof of the propose approach. They conduct empirical experiments to validate the proposed method, on five real-world student response datasets covering four academic subjects (math, biology, history, english).", "questions": "See above.", "limitations": "Yes.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "Posting my full review here.\n\n# Overall\n\nOverall, this paper appears to be an improvement on a domain that is not often discussed in NeurIPS despite its relevance to the NeurIPS community, and strong potential for real-world impact. The paper includes relevant theoretical results (which I did not fully check, but appear correct) and, perhaps even more important, solid empirical results on real-world student datasets. Applied results as consistent as these are not common in the computational educational literature, and it is nice to see an improvement. However, I feel that the paper's potential for impact is currently limited by issues with the clarity of the writing and presentation. I am open to raising my score if the authors can demonstrate a clear, comprehensive, and achievable plan for improving the paper in their response.\n\n# Major comments\n\n* *More introduction*: While I feel the topic of this paper is appropriate for NeurIPS and it is good to see this issue being brought to the NeurIPS audience, this conference has less of a history of publishing work related to student modeling. As a result, it is likely less familiar to readers, and most NeurIPS readers would benefit from more background and motivation than the paper currently provides (Figure 1 is quite useful, for example, but I don't feel it is given adequate discussion in the current text). It would be useful to give details which may seem obvious to the authors, such as: what data is cognitive modeling applied to? Where does that data derive from? In what applications are cognitive models currently used? What is the potential impact of improved cognitive modeling? The answers to these questions may seem obvious to the authors, and I understand that space is limited, but they are likely *not* obvious to most readers and would provide considerable motivation for the general direction and the specific methods and results of the paper.\n\n* *Figures:* Related to the above note, there are several figures which make some aspects of the proposed methods clear (Figures 1, 2, I) but which do not clarify the mechanics of the model itself: how do the various matrices interact, what constraints are applied to which matrices, and how does one go from \"inputs\" to \"outputs\"? This can be understood from diving into the theoretical details in the paper, but given that the figures seem intended to clarify exactly this (but currently fail to do so, for me at least), I think that improvements to the figures to make the modeling process clear would substantially improve it.\n\n* *Clarity of writing and presentation:* I found the writing to be difficult to follow in many places. The sentence structures were unnecessarily complex (and often include the passive voice), the authors make claims which are not verifiable (\"The new model (AE-NMCF) achieves sophisticated knowledge prificiency explanation\"), and the text is quite vague in certain places (\"the coverage function only scratches the surface of understanding students' cognitive levels\"). Individually, these are small issues, but collectively, they make the paper much harder to follow than it needs to be. The entire paper could use a thorough revision for clarity and succinctness. Some, but not all, of the writing issues I noted are highlighted in the minor comments and typos sections below.\n\n* *Causal claims about performance gains:* In 4.3, the authors make causal claims about specific components of model which don't appear to be supported by the experiments without ablation studies. Specifically, they say \"[t]he boost in performance prediction is due to the co-factorization that facilitates implicit feature learning collaboratively\". The experiments do not support this claim: there are other methods that use matrix factorization (e.g. SNMCF) in their experiments but which do not achieve similar performance gains, and more importantly, there are multiple aspects of the new method that could be the source of the observed improved performance (nonnegativity, monotonicity, training hyperparameters and other details). Only an ablation study which removed co-factorization (but kept the other components) would provide direct evidence for this claim. Such ablation studies would be nice, but in my opinion are not required (I would suggest to just remove those claims about which specific aspects of the model lead to the performance gains). \n\n* I did not see a mention of whether the authors plan to publicly release their code. Please clarify in the response whether you plan to do so -- this would greatly support reproducibility of the proposed method, at least on the publicly available data sources.\n\n# Minor comments\n\n* The abstract could do a better problem of introducing readers to the problem being solved: what data is used? What are the inputs and outputs? In what contexts/applications is this useful? What matrices are being factorized? The beginning of the intro clarifies some -- but not all -- of these questions, but many readers will rely solely on the abstract to assess the paper, and currently it is not effective at outlining the high-level problem the paper addresses.\n\n* \"Cascading errors\" are mentioned twice in Section 1 without ever being defined. Please clarify.\n\n* L35-37: in one sentence, the authors state that SNMCF \"learn[s] students' proficiency levels\", and in the following sentence says it does not (\"without considering the target of improving cognitive diagnosis\") - please clarify.\n\n* It would be useful to explain why previous approaches do not leverage monotonicity, as it seems like an obvious constraint but is also framed as a novel contribution of the current method.\n\n* L50-52 are supposed to highlight the authors' \"key observation\", but the sentence is convoluted and difficult to understand; it obscures what this observation is. Please clarify.\n\n* Figure 2 is nice (the row and column signifiers are quite helpful), but what does the direction of the arrows signify in Figure 2? Also, could this figure indicate which matrices are constrained (i.e. nonnegative)?\n\n* Equation (1) is inserted into the text without any introduction or framing beforehand. Please clarify. Perhaps it needs a new bolded paragraph.\n\n* It would be helpful to explain the *prediction task* being performed, prior to the discussion of the metrics in 4.1. Specifically: accuracy and RMSE are measured with respect to what targets? This is discussed a bit later but needs to appear earlier, as it is essential to understanding the experimental setup and metrics.\n\n* Please clarify what the cell shading and bolded text in Table 1 signify. Also, a CD diagram comparing the average ranks would be quite useful.\n\n# Typos etc.\n\n* L25: \"the subjective handcraft features\": what does this refer to? Please clarify.\n\n* The paper includes some unnecessary italics, such as \"Lipschitz\" and \"Armijo\" -- while non-English text is typically italicized, this does not apply to names.\n\n* L138: what does it mean that X and B \"share the matrix E\"?\n\n* Figure 4 caption: \"dash lines\" -> dashed lines", "weaknesses": "See above.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your positive feedback. We are grateful for the suggestions for improving the quality of our manuscript in terms of writing and presentation. We have considered all of your concerns, including but not limited to enriching the background, updating the figures, simplifying the expression, and clarifying the ambiguous information. Regarding the questions you raised, our point-by-point responses are:\n\n>**Q1.** More introduction of background, motivation, and data descriptions.\n\n**A1.** Following your advice, we will enrich the Abstract and Introduction:\n\nIn Abstract, we will include the descriptions: (**Background**) Student cognitive modeling (SCM) is a fundamental task in intelligent education, with applications ranging from personalized learning to educational resource allocation. (**In-outputs**) By exploiting students' response logs, SCM aims to predict their exercise performance as well as to estimate knowledge proficiency in a subject. (**Technical details**)  To solve this dilemma ..., which jointly factorizes student-exercise interactions and exercise-knowledge linkages to ...\n\nIn Introduction, we will include the descriptions: (**Data**) We use two types of data: (a) The expert-labeled Q-matrix. (b) The students' scoring matrix, which could be accessed either online or offline assessments. For example, in online scenarios, students are required to finish tests (unit tests or term tests). A test normally contains several questions in the form of e.g., multiple-choice or fill-in-the-blank. The students' responses to these questions are automatically recorded and stored by the online learning platforms. (**Applications**) With a comprehensive understanding of students, cognitive modeling could be applied to numerous educational applications, including personalized learning and resource allocation.\n\n> **Q2.** Figures and Tables.\n\n**A2.** For Figure 2, the green (blue) arrows denote the decomposition (composition) process. As you suggested, in the global file, we have modified Figure 2 by adding different types of arrows to highlight the matrix interactions, and adding nonnegative constraints with cell shading. For Figure 1, we have reorganized the input/output flow. Besides, we have added the CD diagrams in Figure 3. For Table 1 in the manuscript, we use bolded text and cell shading to denote the best and top-2 performance, respectively.\n\n>**Q3.** Clarity of writing and presentation.\n\n**A3.** We will concise the expressions and clarify the ambiguous information throughout the manuscript. For *The new model (AE-NMCF) ...*, we rephrase it as *The new model (AE-NMCF) provides a good fit to the students' knowledge proficiency*.  For *the coverage function ...*, we rephrase it as *the coverage function often gives binary cognitive levels, failing to discern the nuance between knowledge proficiencies.*\n\n>**Q4.** Causal claims about performance gains.\n\n**A4.** Thank you for your advice, we will remove the ambiguous claim. Also as suggested by Reviewer RqUb, we have conducted an ablation study to see the impact of co-factorization by removing the encoder (see AE-NMCF w/o Encoder in Table 1 in the global pdf file), and the results confirm your suggestions. Hence, this inappropriate message will be modified.\n\n>**Q5.** Open source code.\n\n**A5.** The source code will be publicly available and we welcome all discussions.\n\n>**Q6.** Cascading errors.\n\n**A6.** Most cognitive diagnosis models learn student learning behavior with expert-designed handcraft features, which are often coupled between the two cognitive tasks. However, due to the limited scope and potential bias, the failure in one task may negate the performance in another task, namely causing cascading errors.\n\n>**Q7.** L35-37: the authors state that SNMCF learn[s] ...\n\n**A7.** Although SNMCF can measure students' proficiency by a coverage function, the latent features used are pre-trained, and the training objective function is designed for student performance prediction but not aimed at cognitive diagnosis.\n\n>**Q8.** The use of monotonicity.\n\n**A8.** Previous cognitive approaches have mainly differed along two dimensions: cognitive diagnosis models (CDMs) and matrix factorization (MF)-based models. Most CDMs can follow the monotonicity principle, which however is unable in MF. The reason is that the latent features decomposed from MF are unable to be explained, i.e., the specific knowledge concepts cannot be clearly described via the features, let alone using monotonicity. To get around this problem, we propose AE-NMCF.\n\n>**Q9.** The key observation.\n\n**A9.** Our key observation is that the self-construction principle of the autoencoder, which reconstructs the students’ response data from the learned low-dimensional representations of students and knowledge concepts, is amenable to the requirement of the monotonic constraint. Therefore, the autoencoder mechanism provides a first-cut approach to achieve the monotonicity for student cognitive modeling.\n\n> **Q10.** Eq.(1) is inserted .. without any introduction ...\n\n**A10.** We start with the sentence *Given X ... with optimization problem (1) ...* to introduce Eq.(1) (see lines 126--130). Here, the problem (1) corresponds to Eq.(1).\n\n>**Q11.** ... explain the prediction task being performed ...\n\n**A11.** As you suggested, we will detail the prediction tasks before describing the experimental results. We use ACC and RMSE from regression and classification aspects, respectively, which are commonly used in predicting student performance. ACC/RMSE are calculated based on the ground truth of students' responses and corresponding predicted ones.\n\n>**Q12.** Typos etc.\n\n**A12.** The  *handcraft features* refer to those that are manually designed by domain experts (e.g., the slip and guess of an exercise). For *share*, we use E and U to approximate X, which means E is one of the latent features of X. Similarly, E is also one of the latent features of B. Hence we call it *share*.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CXPMv5rZEq", "reviewer": "Reviewer_RqUb", "summary": "The authors present a novel model of student cognition to improve the prediction of student exercise performance and the estimation of their knowledge proficiency in a subject. Current approaches such as matrix factorization perform well in predicting student performance on exercises, but the knowledge proficiency is often unknown or poorly estimated. These problems are amplified if only sparse interactions between exercises and students are available. To address this, the authors develop an autoencoder-like nonnegative matrix co-factorization (AE-NMCF) framework based on monotonicity. AE-NMCF improves the accuracy of predicting the student's knowledge proficiency through an end-to-end learning pipeline that makes the estimation problem nonconvex with nonnegative constraints. AE-NMCF uses a projected gradient method based on block coordinate descent with Lipschitz constants, ensuring the method's theoretical convergence. Experiment results show the superiority of AE-NMCF in predicting student exercise performance and estimating student knowledge proficiency when compared to state-of-the-art student cognitive models.", "questions": "N/A", "limitations": "The authors adequately addressed the limitations of their work.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "-The authors used an adequate number of datasets and baselines for their experiments.", "weaknesses": "-The authors don't perform any ablation studies for their work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable feedback to enhance the quality of our paper. As you suggested, we have conducted ablation experiments to verify the encoder-decoder architecture in AE-NMCF. In the ablation study, we use two variants of AE-NMCF, including (*a*) AE-NMCF w/o Decoder that removes the decoder, and (*b*) AE-NMCF w/o Encoder that ignores the encoder. The optimization approach is also PG-BCD with appropriate Lipschitz constants.\n\nThe experimental results of the proposed AE-NMCF and its variants are reported in Table 1 and Figure 4 in the global pdf file. According to Table 1, the ignorance of the encoder (or decoder) process leads to a degradation in predicting performance, the observation of Figure 4 comes to a similar result for student knowledge estimation.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nrJEK9kPjO", "reviewer": "Reviewer_Px4s", "summary": "The paper introduces an autoencoder-like nonnegative matrix co-factorization framework (AE-NMCF) to enhance predictions of student exercise performance and assessments of knowledge proficiency even when data is sparse. The authors offer a projected gradient method employing block coordinate descent and Lipschitz constants, ensuring theoretical convergence for parameter estimation. The effectiveness of this approach is validated through experiments on several real-world datasets.", "questions": "I think that almost similar modeling can be done by considering the factorization of matrix Q without introducing matrix B. For example, I'm guessing that co-factorization as $X \\approx EU$, $Q \\approx EV$ following [Singh2008][Takeuchi2013] shows good performance. So my two questions are as follows. \n\n(Q-1) What is the advantage of the proposed method over variants of existing collective matrix factorization methods such as [Singh2008][Lee2009][Takeuchi2013]?\n\n(Q-2) Does the estimation performance (ACC/RMSE) of the proposed method outperform the above existing collective factorization methods (with appropriate loss function and constraints)?\n\n[Singh2008] Singh, A. P., & Gordon, G. J. (2008, August). Relational learning via collective matrix factorization. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining (pp. 650-658).\n\n*EDIT: My concerns have been addressed in the author-discussion phase and so I updated my score.", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "(Strength-1) The paper is well written, except for the lack of an explanation justifying the proposed factorization form (See Weaknesses)\n\n(S-2) The authors provide a theoretical analysis of the proposed projected-gradient algorithm.", "weaknesses": "I have three concerns about matrix B, the objective function, and related works:\n\n(W-1) The matrix parameter B seems to make parameter estimation difficult. Unlike other matrices such as E and V, the size of matrix B is N x K, where N is the number of exercises and K is the number of knowledge. I think introducing such a large parameter is undesirable for the model.\n\n(W-2) From the view of probabilistic modeling, the objective function Eq. (5) seems inappropriate; it includes two terms of log-likelihood for the same X, one related to the log-likelihood of X using the inverse link function in the decoder process and another related to the squared error loss, which can be regarded as the negative log-likelihood when the elements of X follow Gaussian, in the encoder process. That is equivalent to thinking that two Xs are generated, which is not an appropriate modeling. \n\n(W-3)  Although the authors appear to state that co-factorization is used only for performance improvement, there are studies that extract intuitive and interpretable patterns by introducing nonnegative constraints, e.g.,[Lee2009][Takeuchi2013]. The superiority of this method over these methods needs to be demonstrated.\n\n[Lee2009] Lee, H., & Choi, S. (2009, April). Group nonnegative matrix factorization for EEG classification. In Artificial Intelligence and Statistics (pp. 320-327). PMLR.\n\n[Takeuchi2013] Takeuchi, K., Ishiguro, K., Kimura, A., & Sawada, H. (2013, June). Non-negative multiple matrix factorization. In Twenty-third international joint conference on artificial intelligence.\n\nMinor Comments\n\n-I feel a little strange that the proposed method is named autoencoder, even though it does not use neural networks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your attention and comments on our paper. Your valuable feedback means a lot to us. Regarding the questions you raised, we have carefully considered each point and have made the following responses:\n\n>**Q1.** The matrix B seems to make parameter estimation difficult.\n\n**A1.** Despite the computational burden of optimizing B, as a data-driven method, we consider introducing B because:\n\n1. The subjective tendency of the Q-matrix (Q). In fact, building Q from a domain is a non-trivial task [Desmarais2013], and it is widely recognized that expert-designed Q may e.g. consist of some misspecifications [Chiu2013][Yang2022] due to the blind spot of domain experts.\n\n2. The binary Q only indicates whether an exercise requires a knowledge concept, failing to uncover the strength (lines 133-134). However, the strength can benefit a lot of downstream educational tasks such as learning resource recommendations.\n\nHence, we use Q as a sparse prior and introduce B to specify the exercise-knowledge strength. B can be optimized based on the encoder-decoder framework in a data-driven manner (using the scoring matrix X). This is also the reason that we only reconstruct X in the decoder but do not consider Q.\n\nBesides, although the size of B is N by K, it is still reasonable and can be applied to many educational applications because scale-based tests, where students are evaluated for a small set of knowledge concepts, are common scenarios, especially in unit tests and home quizzes.\n\n>**Q2.** From the view of probabilistic modeling, the objective function Eq.(5) seems inappropriate ...\n\n**A2.** From the generative model perspective, although the two terms of likelihood for the scoring matrix X seem different ostensibly, their interpretations of the approximation for X are the same in cognitive-psychometric modeling. Specifically:\n\n1. For the encoder process, we cannot read off the students' knowledge proficiencies via X, which comprises the information only about students and exercises. Therefore, we use the interaction between the E and U with the Frobenius term to approximate the value of X. The use of this approximation is postulated on the belief that the low-dimensional latent space of E (or U) can correspond to the space of high-level knowledge components (i.e., the top skills we refer to in the manuscript, see lines 128-130) that can explain variability in student performance.\n\n2. For the decoder process, we can now assess the students' fine-grained knowledge proficiency through the matrix A, and we have specified the degree to which an exercise involves a knowledge concept in the matrix B. Armed with A and B, we can approximate the probabilities that the students answer the exercises correctly by the inverse link function. The use of this approximation follows the additive assumption that is widely used in the educational psychology area.\n\nHence, the implication that the feature matrices E and U in the encoder for student performance predictions are different from A and B in the decoder, the two types of generation processes differ.\n\n>**Q3.** Demonstrating the superiority of the method.\n\n**A3.** First, regarding the performance gains of co-factorization, as suggested by Reviewer LdEU and RqUb, we conduct the ablation study to investigate the prediction performance of co-factorization by removing the decoder of the proposed AE-NMCF model, i.e., AE-NMCF w/o Decoder. The experimental results are shown in Table 1 in the global pdf file. Second, we show the prediction performance of CMF [Singh2008], GNMF [Lee2009], and NMMF [Takeuchi2013] as you suggested in Table 2 in the global pdf file. From Table 1 and Table 2, we have the following conclusion:\n\n1. In Table 1, the prediction performance of AE-NMCF w/o Decoder is behind AE-NMCF w/o Encoder, suggesting that the boost may not come from the co-factorization. Hence, as also suggested by Reviewer LdEU, we will modify this ambiguous claim.\n\n2. By comparing the prediciton results of AE-NMCF w/o Decoder in Table 1 with those of CMF, GNMF, and NMMF in Table 2, we observe that except for NMMF, the performance of co-factorization is still noteworthy, at least, over the performance of CMF, and on par with GNMF.\n\n3. In Table 2, the ACC/RMSE of AE-NMCF rises well above that of all compared models on all data sets, especially in sparse scenarios such as SLP-Bio-s and SLP-Eng.\n\n>**Q4.** Why the method is named autoencoder.\n\n**A4.** We call the proposed approach an *Autoencoder-like* model because our design draws inspiration from the architecture of the autoencoder, which consists of two components: an encoder that encodes the low-dimensional representation of input data (E, U, and V in our model), and a decoder that reconstructs the input data (the scoring matrix X) from the encoded representations.\n\n>**Q5.** The advantage of the proposed method over ...\n\n**A5.** The advantages are:\n\n1. (**Matrix B**) The consideration is the subjective tendency of the expert-labeled Q-matrix, which may be inappropriate using the factorization Q $\\approx$ EV directly. Hence, we use Q as a sparse prior and introduce B that can be optimized based on the proposed encoder-decoder framework based on students' response logs (see the gradient of B in the manuscript, which not only considers Q but also takes into account of X).\n\n2. (**Motivation**) Different from [Singh2008] [Lee2009] [Takeuchi2013] that focus on a single task such as link prediction, our factorization framework enables two educational tasks simultaneously, i.e., not only the student performance prediction but also knowledge concept estimation which is our major concern. The estimation results can be helpful for the interpretation of the prediction results.\n\n3. (**Model implication**) Based on our carefully designed encoder-decoder mechanism, we utilize the relationships among the low-dimensional matrices to better understand students' latent knowledge proficiency, which is unable in most MF-based methods.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We deeply appreciate all the reviewer's evaluations and comments on our paper. Their thoughtful feedback has provided valuable insights that have significantly contributed to improving the quality of the manuscript. In the global response, **we have included an attached global file**, where some figures and tables will be referenced when replying to each reviewer. Thanks again to the reviewers for taking the time and effort.

The references occurred in the rebuttal are listed as follows:

- **[Desmarais2013]** Michel C Desmarais and Rhouma Naceur. A matrix factorization method for mapping items to skills and for enhancing expert-based q-matrices. In *Artificial Intelligence in Education: 16th International Conference*, pages 441–450. Springer, 2013.

- **[Chiu2013]** Chia-Yi Chiu. Statistical refinement of the q-matrix in cognitive diagnosis. *Applied Psychological Measurement*, 37(8):598–618, 2013.

-  **[Yang2022]** Haowen Yang, Tianlong Qi, Jin Li, Longjiang Guo, Meirui Ren, Lichen Zhang, and Xiaoming Wang. A novel quantitative relationship neural network for explainable cognitive diagnosis model. *Knowledge-Based Systems*, 250:109156, 2022.

Meta-review: This paper proposes a novel model for improving the prediction of student performance and the estimation of proficiency. 
The incorporation of monotonicity in the modeling process, along with significant improvements in performance prediction accuracy and knowledge estimation ability, particularly in sparse data scenarios, has a substantial impact on student cognitive modeling in this area.
Through discussions with reviewers, additional experiments were conducted, and clarifications were made, which further enhanced the quality of the paper.
Decision: Accept (poster)