# A Unified Interpretable Intelligent Learning Diagnosis Framework for Learning Performance Prediction in Intelligent Tutoring Systems

Authors: Zhifeng Wang; Wenxing Yan; Chunyan Zeng; Yuan Tian; Shi Dong
Journal: International Journal of Intelligent Systems
 (2023)

---

## Abstract

Intelligent learning diagnosis is a critical engine of intelligent tutoring systems, which aims to estimate learners’ current knowledge mastery status and predict their future learning performance. The significant challenge with traditional learning diagnosis methods is the inability to balance diagnostic accuracy and interpretability. Although the existing psychometric-based learning diagnosis methods provide some domain interpretation through cognitive parameters, they have insufficient modeling capability with a shallow structure for large-scale learning data. While the deep learning-based learning diagnosis methods have improved the accuracy of learning performance prediction, their inherent black-box properties lead to a lack of interpretability, making their results untrustworthy for educational applications. To settle the abovementioned problem, the proposed unified interpretable intelligent learning diagnosis framework, which benefits from the powerful representation learning ability of deep learning and the interpretability of psychometrics, achieves a better performance of learning prediction and provides interpretability from three aspects: cognitive parameters, learner-resource response network, and weights of self-attention mechanism. Within the proposed framework, this paper presents a two-channel learning diagnosis mechanism LDM-ID as well as a three-channel learning diagnosis mechanism LDM-HMI. Experiments on two real-world datasets and a simulation dataset show that our method has higher accuracy in predicting learners’ performances compared with the state-of-the-art models and can provide valuable educational interpretability for applications such as precise learning resource recommendation and personalized learning tutoring in intelligent tutoring systems.


---

## Body

## 1. Introduction

The integration of modern information technology with traditional teaching and education methods has promoted the emergence of smart education, which refers to a new form of education that relies on modern information technology in the education field [1]. It applies new technologies such as artificial intelligence, virtual reality, cloud computing, and big data to improve the quality and effectiveness of education [2]. As a vital part of smart education, the intelligent tutoring system can support monitoring and measuring the learning process and digging out the learning patterns of learners [3], as well as effectively presenting the results to teachers and learners [4]. Correspondingly, intelligent learning diagnosis [5] is one of the key components of the intelligent tutoring system [6], and it supports the intelligent tutoring system through data mining and educational theories technically. To provide learners with personalized learning tutoring services, the intelligent learning diagnosis employs educational psychology and data mining technologies to diagnose the learner’s learning state and predict possible future learning performances by collecting and analyzing the exercise records and behavioral data, which are generated by the learner during the learning process. In both online and offline education scenarios, the number of teachers is far lower than the number of learners. There is an urgent need for intelligent means to assist teachers in improving efficiency and provide targeted guidance for learners to bridge this resource mismatch. As a result, intelligent learning diagnosis has been applied in many e-learning systems such as Coursera, EDX, Khan Academy, and offline classes, and it has brought great progress to the education industry [7]. While facilitating the renewal of teachers’ teaching philosophy and driving smart learning systems to provide large-scale personalized learning services, the intelligent learning diagnosis can help teachers provide appropriate guidance based on learners’ characteristics, as well as help learners gain a more objective and clear understanding of their learning progress and status, enabling learners to regulate and standardize their learning independently.The key to intelligent learning diagnosis is to mine the learner’s potential learning state, including their mastery state of knowledge points and learning ability, then predict the learner’s performance on particular learning tasks. For this pivotal problem, two types of solutions are widely developed at present: psychometrics-based methods [8–15] and deep learning-based methods [16–23]. First, psychometrics-based methods appeared earlier and have been developed for more than 40 years. They usually describe the features of the learner and the exercises based on a specific cognitive perspective and employ an empirical response function to express the relationship between learners’ performance on exercises and their knowledge status. Psychometrics-based methods are easy to understand, and the predefined parameters and the connections between them have clear actual significance and interpretability. Second, deep learning diagnosis methods appeared with the development of deep learning technology, which collects the learning resource-related information and learners’ learning data, then models learners’ learning process and their knowledge status through deep neural networks such as recurrent neural networks [16], convolutional neural networks [18], memory neural networks [19], and graph neural networks [21]. With support from deep neural networks’ powerful representation learning capabilities, deep learning-based diagnosis methods achieve higher accuracy in learning performance prediction.The abovementioned methods are well developed for intelligent learning diagnosis and have been applied in different scenarios of intelligent tutoring systems. However, there are still some significant challenges. First, traditional psychometric learning diagnosis models introduce a few specific cognitive parameters to model learners from a single perspective, making them difficult to fit the actual learning situation perfectly. For example, the Deterministic Input, Noisy “And” gate (DINA) model can only diagnose the learner’s mastery of knowledge in a relatively rough way [12]. When different learners have not mastered a certain knowledge point, it cannot reflect more detailed differences. Second, due to the black-box characteristics of deep learning methods, the interpretability of deep learning diagnosis is relatively low. Given an example, the deep knowledge tracing (DKT) model summarizes a learner’s state of all knowledge concepts in one hidden state, so that it is difficult to know exactly how much a student has mastered a certain knowledge concept [16].In this paper, we proposed a novel unified interpretable intelligent learning diagnosis framework that can effectively solve the above problems. First, multichannel psychometric learning diagnosis models are introduced to initialize the learning diagnosis of learners from a multicognitive perspective. Second, the learner representation network and learning resource representation network are constructed separately using the stacked autoencoder (SAE) network. Then, the learner-learning resource response network is designed to extract deep learning features. Subsequently, deep learning features are fused with shallow learning features, and a self-attention mechanism is introduced to weigh the fused features. Finally, a learning performance prediction network is designed through convolutional neural networks. While obtaining better learning performance predictions, the proposed framework provides interpretability in three aspects: cognitive parameters, learner-resource response network, and weights of self-attention mechanism. Our main contributions can be summarized as follows:(i)
Framework: The proposed unified interpretable intelligent learning diagnosis framework, which benefits from the powerful representation learning ability of deep learning and the interpretability of psychometrics, achieves a better performance of learning prediction and provides interpretability from three aspects: cognitive parameters, learner-resource response network, and weights of self-attention mechanism. Furthermore, the proposed framework is a generic learning diagnosis pattern that can incorporate a variety of different psychometric models and deep learning diagnosis methods into this framework.(ii)
Mechanisms: Based on the proposed framework, a 2-channel learning diagnosis mechanism LDM-ID is implemented based on the fusion of IRT and DINA on the one hand, and a 3-channel learning diagnosis mechanism LDM-HMI is implemented based on the fusion of Ho-DINA, MIRT, and IRT on the other hand. The proposed two learning diagnosis mechanisms provide reference examples for developing new learning diagnosis models in different educational scenarios. The codes of these two mechanisms are released athttps://github.com/CCNUZFW/LDM-ID-HMI.(iii)
Application: The proposed two mechanisms are compared with 11 state-of-the-art methods in two publicly available online class databases and a self-built offline class database, and the proposed method in this paper obtains better learning prediction accuracy (AUC absolute improves by about 5%), while the stability of learning prediction is also better (RMSE relatively decreases by about 7%), and is comparable to current deep learning diagnosis methods in terms of prediction time cost. In addition, this paper constructs and open-sources a database of participants in offline courses named CL21 to verify the effectiveness of this method in both online and offline classes. CL21 is released athttps://github.com/CCNUZFW/CL21.The rest of the paper is organized as follows: Section2 briefly reviews some related work on intelligent learning diagnosis. Section 3 presents the intelligent learning diagnosis model, including the basic assumptions, definitions, and problem formula. Section 4 introduces our proposed unified interpretable intelligent learning diagnosis framework in detail, and Section 5 describes two specific implementation methods under the proposed framework. Then, experiments are conducted on two real-world datasets and one virtual dataset, and the results of the proposed method are compared with several baseline methods in Section 6. Finally, we conclude the paper and identify future research that could be carried out based on our new findings in Section 7.


## 2. Related Work

This section briefly reviews the main research work related to intelligent learning diagnosis, which can be divided into two categories: (1) learning diagnosis based on psychometrics and (2) learning diagnosis based on deep learning. While the psychometrics-based learning diagnosis methods aim at mining learners’ potential cognitive situation, the deep learning-based learning diagnosis methods focus on the deep representation learning of learning data and the accurate prediction of learning performance.


### 2.1. Learning Diagnosis Based on Psychometrics

This category of learning diagnosis methods is derived from psychometrics [24]. These methods are based on cognitive psychological theories, using modern statistical methods to construct learning diagnosis models and combining estimation and inference to diagnose the cognitive structure and state of learners [25]. Psychometrics-based learning diagnosis methods can be divided into two subcategories: (1) continuous learning diagnosis models and (2) discrete learning diagnosis models.


#### 2.1.1. Continuous Learning Diagnosis Models

Continuous learning diagnosis models which are based on item response theory (IRT) [26] assume learners have a “latent trait” and learners’ responses to exercises are determined by the exercise’s factors and the learner’s potential features jointly. The learner’s knowledge state diagnosis result is demonstrated by modeling the relationship among the learner’s potential features, the exercise’s factor, and the learner’s answering response [8]. Typical methods, such as IRT [26] use logical functions to describe the relationship between learner parameters and responses, but the learner characteristic parameters in this model are an expression of the learner’s overall mastery level and cannot be refined to every knowledge point. As a result, a multi-dimensional IRT (MIRT) model [10] was developed, which estimates the ability of learners in multiple dimensions at the same time, and considers the relationship between each ability dimension. Furthermore, since MIRT does not consider the hierarchical relationship between multidimensional ability traits, De la Torre et al. proposed a high-order item response model (Ho-IRT) [11], which can directly deal with the potential traits of learners with a hierarchical structure.


#### 2.1.2. Discrete Learning Diagnosis Models

Discrete learning diagnosis models assume that the ability value is discontinuous and the latent knowledge space is composed ofK dichotomous variables. Therefore, there are 2K knowledge mastery states. The learners are divided into these mastery states according to their exercise responses and distinguish their cognitive status of knowledge accordingly. For example, the DINA [12] model uses guess and slip parameters to define the attributes of the exercise, and it employs a knowledge mastery mode vector that takes the value of 0 or 1 to represent the learner’s mastery of each knowledge point. It is assumed that the learner must master all the knowledge points corresponding to the exercise to answer correctly based on the noncompensation assumption. Correspondingly, based on compensation assumption, the Deterministic Input, Noisy “Or” gate (DINO) [13] model assumes that the learner can make the correct answer after mastering at least one of the knowledge points. Furthermore, for more fine-grained modeling, the Noisy Input, Deterministic “And” gate (NIDA) [14] model defines the parameters at the knowledge points level of the exercises, and refines the scope of the parameters compared with DINA and DINO models. To be able to report both the macroscopic general ability and the microscopic cognitive state at the same time, the High-order DINA (Ho-DINA) model [15] is proposed, which assumes that the knowledge points are independent of each other and subordinate to higher-order general capabilities. Assuming that the learner’s response matrix is determined by the inner product of the learner’s knowledge proficiency matrix and the knowledge point vector contained in the exercise, the probabilistic matrix factorization-based learning diagnosis model (PMF-LDM) is established through matrix decomposition [27].Although psychometric-based learning diagnostic methods can provide explanations of learning diagnosis in terms of cognitive parameters, current methods define targeted cognitive parameters for a specific learning scenario. These learning diagnostic models based on specific cognitive parameters can only interpret learning diagnostic results from a single perspective, so it becomes important to construct a unified learning diagnosis framework that includes multiple cognitive parameters from integrated perspectives and can be applied to multiple learning scenarios.


### 2.2. Learning Diagnosis Based on Deep Learning

Deep learning methods have powerful representation capabilities and have been widely used in computer vision [28, 29], speech processing [30, 31], natural language understanding [32], and other fields [33].Currently, advanced deep learning methods have been widely studied in diagnosis and prediction tasks. For intelligent fault diagnosis, the advent of deep learning has constructed an end-to-end diagnosis procedure [34]. Furthermore, out-of-distribution detection-assisted trustworthy machinery fault diagnosis approach is developed to enhance the reliability and safety of intelligent diagnosis models [35]. For the wheel wear status diagnosis of high-speed trains, a multiplex local temporal fusion architecture is combined with the transformer architecture [36].Deep learning is also extensively used in the learning diagnosis task of learners in the field of intelligent tutoring systems [37]. Deep learning-based learning diagnosis usually models the learner’s learning process based on their answering records sequence through an end-to-end trainable neural network. Classified by the network structures, the learning diagnosis methods based on deep learning could be divided into five categories: (1) the recurrent neural network-based learning diagnosis, (2) the convolutional neural network-based learning diagnosis, (3) the memory network-based learning diagnosis, (4) the graph neural network-based learning diagnosis, and (5) the deep neural network-based learning diagnosis.


#### 2.2.1. The Recurrent Neural Network-Based Learning Diagnosis

The DKT proposed by Piech et al. [16] first applied the recurrent neural network to learning diagnosis, which could break the restriction of empirical response function by simulating the interaction between the learner and the exercise with recurrent neural networks. However, DKT can only simply predict whether a learner has mastered or not mastered a specific knowledge point, and can do nothing about the intermediate state of learner knowledge mastery. Furthermore, Su et al. [17] proposed the exercise-enhanced recurrent neural network (EERNN) that predicts the learner’s performance through the learner’s answer record and the text information of each exercise. For multifeatures, Xiong et al. [38] proposed an extended DKT model, adding more answering features such as answering time, question difficulty, and learners’ previous knowledge to assist in predicting learning performance. In terms of the improvement of the model structure, Pandey and Karypis [39] proposed the Self-Attentive Knowledge Tracing (SAKT) model, which is the first to apply the attention mechanism to the learning diagnosis task. Besides, Ghosh et al. [40] used a monotonic attention mechanism to limit the weight of questions that are far away from the learner’s current answer. Based on the attention mechanism, Choi et al. [41] further applied the encoder-decoder model to the learning diagnosis task.


#### 2.2.2. The Convolutional Neural Network-Based Learning Diagnosis

Different from models based on recurrent neural networks, to achieve personalized modeling, Shen et al. [18] proposed the convolutional knowledge tracing (CKT) method, which utilizes the sensitivity of convolutional neural networks for the temporal and spatial information to consider the prior knowledge and learning rate for each learner. Furthermore, Wang et al. [42] combined a convolutional neural network and a recurrent neural network into an integrated model to build convolutional recurrent knowledge tracing (CRKT).


#### 2.2.3. The Memory Network-Based Learning Diagnosis

Since DKT only uses a latent state to represent the learner’s knowledge mastery state and cannot simulate the dynamic change process of learners’ mastery of different knowledge points, Zhang et al. [19] proposed Dynamic Key-Value Memory Networks (DKVMN), which draws on the idea of memory networks, uses a key matrix for representing the knowledge and a value matrix for indicating the learner’s mastery of each knowledge point and predicts learner’s performance according to the key matrix and value matrix. Furthermore, Tsutsumi et al. [20] add exercises’ difficulty and learners’ ability to DKVMN to improve the accuracy and interpretability.


#### 2.2.4. The Graph Neural Network-Based Learning Diagnosis

Inspired by cognitive theory, the structural information between knowledge points is also helpful for learning diagnosis. Therefore, Nakagawa et al. [21] explored the knowledge graph structure and applied it to learning diagnosis with graph neural networks. Besides, Yong et al. [22] considered the exercise-knowledge correlations for learning diagnosis through graph convolutional networks. At the same time, Tong et al. [43] are more focused on using graph neural networks to capture the relationship between different knowledge points, such as similarity or prerequisite relationship. Furthermore, Abdelrahman and Wang [44] used the latent knowledge graph to improve the reading and writing process of the memory matrix of the DKVMN model.


#### 2.2.5. The Deep Neural Network-Based Learning Diagnosis

Neural Cognitive Diagnosis Model (NeuralCDM) [45] is proposed to project students and exercises to factor vectors and incorporates neural networks to obtain complex learning interactions. Besides, deep matrix factorization-based learning diagnosis (DeepMFLD) [46] acquires the learners’ embedding and the exercises’ embedding separately by deep matrix decomposition, then fuses them with a deep neural network, and finally predicts the learner’s performance by a fully connected layer. Furthermore, item response ranking (IRR) framework [47] introduces pairwise learning into learning diagnosis to sufficiently model the monotonicity between learning interactions.Although the accuracy of learning diagnosis can be improved by introducing deep learning technology, its black-box and end-to-end properties in learning modeling [23] lead to its inability to provide supporting explanatory information and model intermediate feedback for instructional and learning decisions.


### 2.3. Research Questions and Motivation

Based on the research of the related work, we summarized the research questions and motivation of this paper as follows:(i)
RQ1: How to build a unified learning diagnosis framework that incorporates multidimensional cognitive parameters and can be applied to a broader range of learning scenarios?Inspired by the multimodal approach, we fused multiple psychometrics-based learning diagnosis models containing different cognitive parameters into a unified multimodal learning diagnosis model. On the one hand, this allows for multidimensional interpretable cognitive parameters. On the other hand, it also enables the proposed unified learning diagnosis framework to be extended to a more practical and broader range of educational application scenarios.(ii)
RQ2: How to effectively characterize learners and learning resources and appropriately establish effective interactive responses between learner and learning resource features?Inspired by the representational power of deep learning methods, on the one hand, we constructed a learner network and a learning resource network based on shallow learning features to extract better representational deep learner features and deep learning resource features; on the other hand, we characterized the interactive responses between them by constructing a learner-learning resource response network. Furthermore, the prediction network is constructed by combining shallow and deep learning features to obtain better representativeness as well as to provide interpretable cognitive parameters.(iii)
RQ3: How to effectively improve learning performance predictions while also providing appropriate explanatory information that can be used to improve the acceptability and usability of learning diagnosis?Inspired by the interpretable AI approach, first, we provided self-contained explanatory information describing learners’ cognitive states from multidimensional cognitive parameters; second, we provided explanatory information on the interaction responses between learners and learning resources by constructing learner-learning resource response networks for discovering which learning interaction channels are more effective; and finally, we obtained global explanatory information on the importance of different learning features in terms of decision making due to the attention mechanism for identifying essential learning cues.


## 2.1. Learning Diagnosis Based on Psychometrics

This category of learning diagnosis methods is derived from psychometrics [24]. These methods are based on cognitive psychological theories, using modern statistical methods to construct learning diagnosis models and combining estimation and inference to diagnose the cognitive structure and state of learners [25]. Psychometrics-based learning diagnosis methods can be divided into two subcategories: (1) continuous learning diagnosis models and (2) discrete learning diagnosis models.


### 2.1.1. Continuous Learning Diagnosis Models

Continuous learning diagnosis models which are based on item response theory (IRT) [26] assume learners have a “latent trait” and learners’ responses to exercises are determined by the exercise’s factors and the learner’s potential features jointly. The learner’s knowledge state diagnosis result is demonstrated by modeling the relationship among the learner’s potential features, the exercise’s factor, and the learner’s answering response [8]. Typical methods, such as IRT [26] use logical functions to describe the relationship between learner parameters and responses, but the learner characteristic parameters in this model are an expression of the learner’s overall mastery level and cannot be refined to every knowledge point. As a result, a multi-dimensional IRT (MIRT) model [10] was developed, which estimates the ability of learners in multiple dimensions at the same time, and considers the relationship between each ability dimension. Furthermore, since MIRT does not consider the hierarchical relationship between multidimensional ability traits, De la Torre et al. proposed a high-order item response model (Ho-IRT) [11], which can directly deal with the potential traits of learners with a hierarchical structure.


### 2.1.2. Discrete Learning Diagnosis Models

Discrete learning diagnosis models assume that the ability value is discontinuous and the latent knowledge space is composed ofK dichotomous variables. Therefore, there are 2K knowledge mastery states. The learners are divided into these mastery states according to their exercise responses and distinguish their cognitive status of knowledge accordingly. For example, the DINA [12] model uses guess and slip parameters to define the attributes of the exercise, and it employs a knowledge mastery mode vector that takes the value of 0 or 1 to represent the learner’s mastery of each knowledge point. It is assumed that the learner must master all the knowledge points corresponding to the exercise to answer correctly based on the noncompensation assumption. Correspondingly, based on compensation assumption, the Deterministic Input, Noisy “Or” gate (DINO) [13] model assumes that the learner can make the correct answer after mastering at least one of the knowledge points. Furthermore, for more fine-grained modeling, the Noisy Input, Deterministic “And” gate (NIDA) [14] model defines the parameters at the knowledge points level of the exercises, and refines the scope of the parameters compared with DINA and DINO models. To be able to report both the macroscopic general ability and the microscopic cognitive state at the same time, the High-order DINA (Ho-DINA) model [15] is proposed, which assumes that the knowledge points are independent of each other and subordinate to higher-order general capabilities. Assuming that the learner’s response matrix is determined by the inner product of the learner’s knowledge proficiency matrix and the knowledge point vector contained in the exercise, the probabilistic matrix factorization-based learning diagnosis model (PMF-LDM) is established through matrix decomposition [27].Although psychometric-based learning diagnostic methods can provide explanations of learning diagnosis in terms of cognitive parameters, current methods define targeted cognitive parameters for a specific learning scenario. These learning diagnostic models based on specific cognitive parameters can only interpret learning diagnostic results from a single perspective, so it becomes important to construct a unified learning diagnosis framework that includes multiple cognitive parameters from integrated perspectives and can be applied to multiple learning scenarios.


## 2.1.1. Continuous Learning Diagnosis Models

Continuous learning diagnosis models which are based on item response theory (IRT) [26] assume learners have a “latent trait” and learners’ responses to exercises are determined by the exercise’s factors and the learner’s potential features jointly. The learner’s knowledge state diagnosis result is demonstrated by modeling the relationship among the learner’s potential features, the exercise’s factor, and the learner’s answering response [8]. Typical methods, such as IRT [26] use logical functions to describe the relationship between learner parameters and responses, but the learner characteristic parameters in this model are an expression of the learner’s overall mastery level and cannot be refined to every knowledge point. As a result, a multi-dimensional IRT (MIRT) model [10] was developed, which estimates the ability of learners in multiple dimensions at the same time, and considers the relationship between each ability dimension. Furthermore, since MIRT does not consider the hierarchical relationship between multidimensional ability traits, De la Torre et al. proposed a high-order item response model (Ho-IRT) [11], which can directly deal with the potential traits of learners with a hierarchical structure.


## 2.1.2. Discrete Learning Diagnosis Models

Discrete learning diagnosis models assume that the ability value is discontinuous and the latent knowledge space is composed ofK dichotomous variables. Therefore, there are 2K knowledge mastery states. The learners are divided into these mastery states according to their exercise responses and distinguish their cognitive status of knowledge accordingly. For example, the DINA [12] model uses guess and slip parameters to define the attributes of the exercise, and it employs a knowledge mastery mode vector that takes the value of 0 or 1 to represent the learner’s mastery of each knowledge point. It is assumed that the learner must master all the knowledge points corresponding to the exercise to answer correctly based on the noncompensation assumption. Correspondingly, based on compensation assumption, the Deterministic Input, Noisy “Or” gate (DINO) [13] model assumes that the learner can make the correct answer after mastering at least one of the knowledge points. Furthermore, for more fine-grained modeling, the Noisy Input, Deterministic “And” gate (NIDA) [14] model defines the parameters at the knowledge points level of the exercises, and refines the scope of the parameters compared with DINA and DINO models. To be able to report both the macroscopic general ability and the microscopic cognitive state at the same time, the High-order DINA (Ho-DINA) model [15] is proposed, which assumes that the knowledge points are independent of each other and subordinate to higher-order general capabilities. Assuming that the learner’s response matrix is determined by the inner product of the learner’s knowledge proficiency matrix and the knowledge point vector contained in the exercise, the probabilistic matrix factorization-based learning diagnosis model (PMF-LDM) is established through matrix decomposition [27].Although psychometric-based learning diagnostic methods can provide explanations of learning diagnosis in terms of cognitive parameters, current methods define targeted cognitive parameters for a specific learning scenario. These learning diagnostic models based on specific cognitive parameters can only interpret learning diagnostic results from a single perspective, so it becomes important to construct a unified learning diagnosis framework that includes multiple cognitive parameters from integrated perspectives and can be applied to multiple learning scenarios.


## 2.2. Learning Diagnosis Based on Deep Learning

Deep learning methods have powerful representation capabilities and have been widely used in computer vision [28, 29], speech processing [30, 31], natural language understanding [32], and other fields [33].Currently, advanced deep learning methods have been widely studied in diagnosis and prediction tasks. For intelligent fault diagnosis, the advent of deep learning has constructed an end-to-end diagnosis procedure [34]. Furthermore, out-of-distribution detection-assisted trustworthy machinery fault diagnosis approach is developed to enhance the reliability and safety of intelligent diagnosis models [35]. For the wheel wear status diagnosis of high-speed trains, a multiplex local temporal fusion architecture is combined with the transformer architecture [36].Deep learning is also extensively used in the learning diagnosis task of learners in the field of intelligent tutoring systems [37]. Deep learning-based learning diagnosis usually models the learner’s learning process based on their answering records sequence through an end-to-end trainable neural network. Classified by the network structures, the learning diagnosis methods based on deep learning could be divided into five categories: (1) the recurrent neural network-based learning diagnosis, (2) the convolutional neural network-based learning diagnosis, (3) the memory network-based learning diagnosis, (4) the graph neural network-based learning diagnosis, and (5) the deep neural network-based learning diagnosis.


### 2.2.1. The Recurrent Neural Network-Based Learning Diagnosis

The DKT proposed by Piech et al. [16] first applied the recurrent neural network to learning diagnosis, which could break the restriction of empirical response function by simulating the interaction between the learner and the exercise with recurrent neural networks. However, DKT can only simply predict whether a learner has mastered or not mastered a specific knowledge point, and can do nothing about the intermediate state of learner knowledge mastery. Furthermore, Su et al. [17] proposed the exercise-enhanced recurrent neural network (EERNN) that predicts the learner’s performance through the learner’s answer record and the text information of each exercise. For multifeatures, Xiong et al. [38] proposed an extended DKT model, adding more answering features such as answering time, question difficulty, and learners’ previous knowledge to assist in predicting learning performance. In terms of the improvement of the model structure, Pandey and Karypis [39] proposed the Self-Attentive Knowledge Tracing (SAKT) model, which is the first to apply the attention mechanism to the learning diagnosis task. Besides, Ghosh et al. [40] used a monotonic attention mechanism to limit the weight of questions that are far away from the learner’s current answer. Based on the attention mechanism, Choi et al. [41] further applied the encoder-decoder model to the learning diagnosis task.


### 2.2.2. The Convolutional Neural Network-Based Learning Diagnosis

Different from models based on recurrent neural networks, to achieve personalized modeling, Shen et al. [18] proposed the convolutional knowledge tracing (CKT) method, which utilizes the sensitivity of convolutional neural networks for the temporal and spatial information to consider the prior knowledge and learning rate for each learner. Furthermore, Wang et al. [42] combined a convolutional neural network and a recurrent neural network into an integrated model to build convolutional recurrent knowledge tracing (CRKT).


### 2.2.3. The Memory Network-Based Learning Diagnosis

Since DKT only uses a latent state to represent the learner’s knowledge mastery state and cannot simulate the dynamic change process of learners’ mastery of different knowledge points, Zhang et al. [19] proposed Dynamic Key-Value Memory Networks (DKVMN), which draws on the idea of memory networks, uses a key matrix for representing the knowledge and a value matrix for indicating the learner’s mastery of each knowledge point and predicts learner’s performance according to the key matrix and value matrix. Furthermore, Tsutsumi et al. [20] add exercises’ difficulty and learners’ ability to DKVMN to improve the accuracy and interpretability.


### 2.2.4. The Graph Neural Network-Based Learning Diagnosis

Inspired by cognitive theory, the structural information between knowledge points is also helpful for learning diagnosis. Therefore, Nakagawa et al. [21] explored the knowledge graph structure and applied it to learning diagnosis with graph neural networks. Besides, Yong et al. [22] considered the exercise-knowledge correlations for learning diagnosis through graph convolutional networks. At the same time, Tong et al. [43] are more focused on using graph neural networks to capture the relationship between different knowledge points, such as similarity or prerequisite relationship. Furthermore, Abdelrahman and Wang [44] used the latent knowledge graph to improve the reading and writing process of the memory matrix of the DKVMN model.


### 2.2.5. The Deep Neural Network-Based Learning Diagnosis

Neural Cognitive Diagnosis Model (NeuralCDM) [45] is proposed to project students and exercises to factor vectors and incorporates neural networks to obtain complex learning interactions. Besides, deep matrix factorization-based learning diagnosis (DeepMFLD) [46] acquires the learners’ embedding and the exercises’ embedding separately by deep matrix decomposition, then fuses them with a deep neural network, and finally predicts the learner’s performance by a fully connected layer. Furthermore, item response ranking (IRR) framework [47] introduces pairwise learning into learning diagnosis to sufficiently model the monotonicity between learning interactions.Although the accuracy of learning diagnosis can be improved by introducing deep learning technology, its black-box and end-to-end properties in learning modeling [23] lead to its inability to provide supporting explanatory information and model intermediate feedback for instructional and learning decisions.


## 2.2.1. The Recurrent Neural Network-Based Learning Diagnosis

The DKT proposed by Piech et al. [16] first applied the recurrent neural network to learning diagnosis, which could break the restriction of empirical response function by simulating the interaction between the learner and the exercise with recurrent neural networks. However, DKT can only simply predict whether a learner has mastered or not mastered a specific knowledge point, and can do nothing about the intermediate state of learner knowledge mastery. Furthermore, Su et al. [17] proposed the exercise-enhanced recurrent neural network (EERNN) that predicts the learner’s performance through the learner’s answer record and the text information of each exercise. For multifeatures, Xiong et al. [38] proposed an extended DKT model, adding more answering features such as answering time, question difficulty, and learners’ previous knowledge to assist in predicting learning performance. In terms of the improvement of the model structure, Pandey and Karypis [39] proposed the Self-Attentive Knowledge Tracing (SAKT) model, which is the first to apply the attention mechanism to the learning diagnosis task. Besides, Ghosh et al. [40] used a monotonic attention mechanism to limit the weight of questions that are far away from the learner’s current answer. Based on the attention mechanism, Choi et al. [41] further applied the encoder-decoder model to the learning diagnosis task.


## 2.2.2. The Convolutional Neural Network-Based Learning Diagnosis

Different from models based on recurrent neural networks, to achieve personalized modeling, Shen et al. [18] proposed the convolutional knowledge tracing (CKT) method, which utilizes the sensitivity of convolutional neural networks for the temporal and spatial information to consider the prior knowledge and learning rate for each learner. Furthermore, Wang et al. [42] combined a convolutional neural network and a recurrent neural network into an integrated model to build convolutional recurrent knowledge tracing (CRKT).


## 2.2.3. The Memory Network-Based Learning Diagnosis

Since DKT only uses a latent state to represent the learner’s knowledge mastery state and cannot simulate the dynamic change process of learners’ mastery of different knowledge points, Zhang et al. [19] proposed Dynamic Key-Value Memory Networks (DKVMN), which draws on the idea of memory networks, uses a key matrix for representing the knowledge and a value matrix for indicating the learner’s mastery of each knowledge point and predicts learner’s performance according to the key matrix and value matrix. Furthermore, Tsutsumi et al. [20] add exercises’ difficulty and learners’ ability to DKVMN to improve the accuracy and interpretability.


## 2.2.4. The Graph Neural Network-Based Learning Diagnosis

Inspired by cognitive theory, the structural information between knowledge points is also helpful for learning diagnosis. Therefore, Nakagawa et al. [21] explored the knowledge graph structure and applied it to learning diagnosis with graph neural networks. Besides, Yong et al. [22] considered the exercise-knowledge correlations for learning diagnosis through graph convolutional networks. At the same time, Tong et al. [43] are more focused on using graph neural networks to capture the relationship between different knowledge points, such as similarity or prerequisite relationship. Furthermore, Abdelrahman and Wang [44] used the latent knowledge graph to improve the reading and writing process of the memory matrix of the DKVMN model.


## 2.2.5. The Deep Neural Network-Based Learning Diagnosis

Neural Cognitive Diagnosis Model (NeuralCDM) [45] is proposed to project students and exercises to factor vectors and incorporates neural networks to obtain complex learning interactions. Besides, deep matrix factorization-based learning diagnosis (DeepMFLD) [46] acquires the learners’ embedding and the exercises’ embedding separately by deep matrix decomposition, then fuses them with a deep neural network, and finally predicts the learner’s performance by a fully connected layer. Furthermore, item response ranking (IRR) framework [47] introduces pairwise learning into learning diagnosis to sufficiently model the monotonicity between learning interactions.Although the accuracy of learning diagnosis can be improved by introducing deep learning technology, its black-box and end-to-end properties in learning modeling [23] lead to its inability to provide supporting explanatory information and model intermediate feedback for instructional and learning decisions.


## 2.3. Research Questions and Motivation

Based on the research of the related work, we summarized the research questions and motivation of this paper as follows:(i)
RQ1: How to build a unified learning diagnosis framework that incorporates multidimensional cognitive parameters and can be applied to a broader range of learning scenarios?Inspired by the multimodal approach, we fused multiple psychometrics-based learning diagnosis models containing different cognitive parameters into a unified multimodal learning diagnosis model. On the one hand, this allows for multidimensional interpretable cognitive parameters. On the other hand, it also enables the proposed unified learning diagnosis framework to be extended to a more practical and broader range of educational application scenarios.(ii)
RQ2: How to effectively characterize learners and learning resources and appropriately establish effective interactive responses between learner and learning resource features?Inspired by the representational power of deep learning methods, on the one hand, we constructed a learner network and a learning resource network based on shallow learning features to extract better representational deep learner features and deep learning resource features; on the other hand, we characterized the interactive responses between them by constructing a learner-learning resource response network. Furthermore, the prediction network is constructed by combining shallow and deep learning features to obtain better representativeness as well as to provide interpretable cognitive parameters.(iii)
RQ3: How to effectively improve learning performance predictions while also providing appropriate explanatory information that can be used to improve the acceptability and usability of learning diagnosis?Inspired by the interpretable AI approach, first, we provided self-contained explanatory information describing learners’ cognitive states from multidimensional cognitive parameters; second, we provided explanatory information on the interaction responses between learners and learning resources by constructing learner-learning resource response networks for discovering which learning interaction channels are more effective; and finally, we obtained global explanatory information on the importance of different learning features in terms of decision making due to the attention mechanism for identifying essential learning cues.


## 3. Intelligent Learning Diagnosis Model

This section introduces some assumptions and definitions, and then details the problem definition.


### 3.1. Assumptions and Definitions of Intelligent Learning Diagnosis

First, this paper is based on the following assumptions and definitions:Assumption 1.
The obtained online and offline learning data is used with permission.Assumption 2.
The data is authentic and complete. The learners participating in the data collection are required to take the course seriously and use their initiative to complete each exercise, to ensure that there is no halfway exit leading to incomplete data or inconsistency in answering results. And the data can reflect the actual level of learners.Assumption 3.
The learner’s answer results are partially independent. On the one hand, it means that the learner’s answering results of course exercises will not be affected by other people. At the same time, it will not affect others. On the other hand, it means that the answering results of learners in different exercises will not be affected by each other.Definition 1.
Shallow feature of learning resource.
For the learning resources involved in the course (mainly refer to the exercises performed by learners), it obtains descriptive parameters for them based on different educational interpretation meanings to form a parameter matrixEC∈Re×de, where e and de represent the number of exercises and the dimension of shallow learning resource feature, respectively.Definition 2.
Shallow feature of learners.
Combining multiple focal points of different cognitive diagnosis models, it describes learners from different angles and forms a parameter matrixSC∈Rs×ds, where s and ds, respectively, represent the number of learners and the dimension of shallow learner feature.Definition 3.
Deep feature of learning resources and learners.
Based on the two descriptive parameters, it introduces deep learning technology to mine higher-order bottleneck features, which models and characterizes the response between learning resources and learners more effectively.


### 3.2. Problem Description

According to the above definitions and assumptions, the unified interpretable intelligent learning diagnosis task is described as follows. Given thatS learners study the same course and complete the same exercise E corresponding to course C, the multiple psychometric models are integrated to obtain the learner and learning resource parameter matrix, and the learner’s answer performance and cognitive state are predicted based on the parameter matrix. Among them, S is a set of learners, E is a set of exercises, K is a set of knowledge points, and the answer record of learner s is Rs=e1,r1,e2,r2,…,et,rt. Among them, et is the exercises performed by the learner s at time t, rt is the corresponding answer result, rt∈0,1, rt=1 means the answer is correct, and rt=0 means the answer is wrong. qi is the set of knowledge points involved in exercise i and qij=1 means that there is a correlation between exercise i and knowledge point j, otherwise, it is irrelevant. The set of answer records of all learners R=R1,R2,…,Rs consists the answer matrix R, and Q=q1,q2,…,qi consist of the exercise-knowledge point association matrix Q. Then, the problem can be defined as follows:Definition 4.
Intelligent learning diagnosis problem.
Given the exercising logs of each learner and the knowledge points involved in each exercise, our goal is threefold: (1) extract interpretable parameters of learners and learning resources; (2) construct a learner-resource response network and deep-shallow feature fusion mechanism to obtain the explanatory information on the importance of learning features; and (3) predict the learning performancert+1^ of the specific learner on time t+1 and offer interpretable information for decision-making.


## 3.1. Assumptions and Definitions of Intelligent Learning Diagnosis

First, this paper is based on the following assumptions and definitions:Assumption 1.
The obtained online and offline learning data is used with permission.Assumption 2.
The data is authentic and complete. The learners participating in the data collection are required to take the course seriously and use their initiative to complete each exercise, to ensure that there is no halfway exit leading to incomplete data or inconsistency in answering results. And the data can reflect the actual level of learners.Assumption 3.
The learner’s answer results are partially independent. On the one hand, it means that the learner’s answering results of course exercises will not be affected by other people. At the same time, it will not affect others. On the other hand, it means that the answering results of learners in different exercises will not be affected by each other.Definition 1.
Shallow feature of learning resource.
For the learning resources involved in the course (mainly refer to the exercises performed by learners), it obtains descriptive parameters for them based on different educational interpretation meanings to form a parameter matrixEC∈Re×de, where e and de represent the number of exercises and the dimension of shallow learning resource feature, respectively.Definition 2.
Shallow feature of learners.
Combining multiple focal points of different cognitive diagnosis models, it describes learners from different angles and forms a parameter matrixSC∈Rs×ds, where s and ds, respectively, represent the number of learners and the dimension of shallow learner feature.Definition 3.
Deep feature of learning resources and learners.
Based on the two descriptive parameters, it introduces deep learning technology to mine higher-order bottleneck features, which models and characterizes the response between learning resources and learners more effectively.


## 3.2. Problem Description

According to the above definitions and assumptions, the unified interpretable intelligent learning diagnosis task is described as follows. Given thatS learners study the same course and complete the same exercise E corresponding to course C, the multiple psychometric models are integrated to obtain the learner and learning resource parameter matrix, and the learner’s answer performance and cognitive state are predicted based on the parameter matrix. Among them, S is a set of learners, E is a set of exercises, K is a set of knowledge points, and the answer record of learner s is Rs=e1,r1,e2,r2,…,et,rt. Among them, et is the exercises performed by the learner s at time t, rt is the corresponding answer result, rt∈0,1, rt=1 means the answer is correct, and rt=0 means the answer is wrong. qi is the set of knowledge points involved in exercise i and qij=1 means that there is a correlation between exercise i and knowledge point j, otherwise, it is irrelevant. The set of answer records of all learners R=R1,R2,…,Rs consists the answer matrix R, and Q=q1,q2,…,qi consist of the exercise-knowledge point association matrix Q. Then, the problem can be defined as follows:Definition 4.
Intelligent learning diagnosis problem.
Given the exercising logs of each learner and the knowledge points involved in each exercise, our goal is threefold: (1) extract interpretable parameters of learners and learning resources; (2) construct a learner-resource response network and deep-shallow feature fusion mechanism to obtain the explanatory information on the importance of learning features; and (3) predict the learning performancert+1^ of the specific learner on time t+1 and offer interpretable information for decision-making.


## 4. Proposed Unified Interpretable Intelligent Learning Diagnosis Framework

Based on the assumptions and definitions of the above learning diagnosis model, this paper proposes a unified interpretable learning diagnosis framework, which is versatile and can flexibly formulate specific strategies according to different needs to realize the interpretable learning diagnosis and performance prediction of learners. As shown in Figure1, the proposed framework mainly consists of five components: initial learning diagnosis, learner and learning resources representation network, learner-resource response network, deep-shallow learning feature fusion, and learning performance prediction network. The framework that takes advantage of the powerful representation learning ability of deep learning and the interpretability of psychometrics, achieves a good performance of learning prediction and provides interpretability from three aspects: cognitive parameters, learner-resource response network, and weights of self-attention mechanism.Figure 1
The inputs of the proposed unified interpretable intelligent learning diagnosis framework are the learners’ response matrixR and the exercise-knowledge point matrix Q. The proposed framework contains five main components: (1) initial learning diagnosis, (2) learner and learning resource representation network, (3) learner-resource response network, (4) deep-shallow learning feature fusion, and (5) learning performance prediction network. The interpretability can be obtained from three aspects: (1) the interpretability from cognitive parameters, (2) the interpretability from LRRNet, and (3) the interpretability from attention weights.


### 4.1. Initial Learning Diagnosis

The main purpose of the initial learning diagnosis module is to construct the multichannel cognitive parameter set.The cognitive parameter set construction process is shown in Figure1. Based on the learner’s historical learning records and learning resource information, the multichannel psychometric model is used to make a preliminary diagnosis of the learner, and to estimate the parameters of the learning resource at the same time, thereby constructing the learning resource parameter set EC and learner parameter set SC.Taking into account the interpretability of the learner’s cognitive state, it is worth noting that the parameters are highly interpretable with practical significance in cognitive theory and psychometrics, such as the difficulty and discrimination of the exercise, the ability of the learner, which differs from learning parameters through network training in the deep learning-based learning diagnosis methods.(1)EC=PM1R,Q,PM2R,Q,…,PMnR,Q,SC=PM1R,Q,PM2R,Q,…,PMnR,Q.PMn is the nth psychometric model. Then, based on the answer record Rs=e1,r1,e2,r2,…,et,rt of learner s, we obtained the learner parameter vector SCs and the learning resource parameter vector EC1 to ECt related to learner s.


### 4.2. Learner and Learning Resources Representation Network

This module consists of two parts: learners representation network and learning resources representation network.Specifically, given the learner parameter vectorSCs of learner s and the learning resource parameter vector ECe of exercise e, firstly, in order to eliminate the difference between the ranges of various parameters, the parameter values are discretized and one-hot encoded, and then SCs is coded as a d0 dimensional 0-1 vector xs and ECe is coded as a d1 dimensional 0-1 vector xe.To better represent learners and learning resources based on cognitive parameters, the learner representation network and learning resource representation network are designed, respectively, which take advantage of auto-encoder to excavate the deep learning features of learners and learning resources. The SAE can copy the input to the output through a hidden layer. It consists of two parts: an encoder that can be represented by the functionh=f⋅ and a decoder that generates reconstruction information y=g⋅, by training the auto-encoder to reproduce the input, h can obtain some useful features, to achieve the purpose of feature learning for the learner’s parameter vector. The learners and learning resource auto-encoder designed in this paper can be expressed as follows:(2)hs=fxs=σW1s×xs+b1s,ys=ghs=σW2s×hs+b2s,he=fxe=σW1e×xe+b1e,ye=ghe=σW2e×he+b2e,W,b, and σ⋅ are weights, bias, and activation function, respectively. We take the intermediate results hs and he of the coding layer as the feature learning results, which can represent a more valuable part of the information in the original parameter vector.


### 4.3. Learner-Resource Response Network

Then, the learner-resource response network is used to simulate the answer process in the actual situation. It takes the deep characteristics of learners and learning resources as the input and outputs the deep characteristics after the interaction between learners and learning resources.(3)f=concaths,he,fd=σW3×f+b3,where σ is the tanh activation function and h and y are the encoder and the decoder, respectively. In addition, the learner parameter encoder’s weight matrix W1s∈Rd2×d0, the bias vector b1s∈Rd2, the decoder’s weight matrix W2s∈Rd0×d2, bias vector b2s∈Rd0, the learning resource parameter encoder’s weight matrix W1e∈Rd3×d1, bias vector b1e∈Rd3, decoder’s weight matrix W2e∈Rd1×d3, and the bias vector b2e∈Rd1 are the parameters of the learner representation network and learning resource representation network. The weight matrix W3∈Rd2+d3×d4 and the bias vector b3∈Rd2+d3 are the parameters of the learner-resource response network. fd is the deep feature of learners and learning resources.


### 4.4. Deep-Shallow Learning Feature Fusion

To improve the performance and interpretability of the model, we regard the original cognitive parameter vector as shallow features and the deep characterization features as deep features, and then we combine the two features as the overall features for diagnosis and prediction.(4)f=concatenatefd,SCs,ECe,where the dimensions of f are d5, in which d5=d4+de+ds.In terms of interpretability for the prediction process, we focus on the reasons why the model makes such predictions. Therefore the prediction network in this paper employs the self-attention mechanism to process the fusion features of the learner and the learning resource, in order to fully mine the correlation and importance information between the features of each dimension, so as to provide interpretability from the feature level for the final diagnosis and prediction results, in other words, we can know exactly which features have a more significant impact on the final prediction result.A convolutional layer is used with a convolution kernel size of 1 to obtain the query vector matrix Query, the key vector matrix Key, and the value vector matrix value throughf as follows:(5)Query=Convf,Key=Convf,Value=Convf.The weight of each dimension of data in the feature is calculated by calculating the dot product between the query vector matrix and the key vector matrix, and the weight indicates the degree of relevance between the task to be queried and each input data.(6)similarityQuery,Keyi=Query•Keyi.On the one hand, numerical conversion of the degree of relevance can be carried out to normalize and organize the original calculated scores into a probability distribution with the sum of weights being 1. On the other hand, it can also highlight the weights of essential elements.(7)ai=SoftMaxsimilarityi=esimilarityi∑j=1dsesimilarityj.The input data is weighted and summed to obtain the fused feature data.(8)fa=∑i=1d5ai⋅Valuei.


### 4.5. Learning Performance Prediction Network

With the features acquired from the feature extraction process, we constructed a learner performance prediction network based on a convolutional neural network, used learner feature vectors and learning resource feature vectors to predict the probability of learners answering a specific exercise correctly, and diagnosed the knowledge mastery of learners and analyze the learning resources through the feature information of learners and learning resources.The convolutional layer is used to extract the spatial information of the deep-shallow fusion feature, the convolution kernel size is set to 3, and the step size is set to 1. Then, the ReLU activation function is added to reduce the dependence between parameters, overfitting is avoided, and the maximum pooling layer is used to reduce the feature dimension, then the predicted valuep of the learner’s answer is obtained under a specific learning task accordingly. It can be expressed as follows:(9)fc=Convfa,fre=Relufc,fp=MaxPoolfre,p=σW4×fp+b4,where fc, fre, and fp are the intermediate variables of the prediction network.To learn the various parameters of the model, the cross-entropy loss function is used between the student’s real answer valuert and the predicted answer value pt as the objective function L of the model as follows:(10)L=−∑i=1Nrtlogpt+1−rtlog1−pt.


### 4.6. Interpretability of the Proposed Framework

The interpretability of our proposed framework includes three aspects, which correspond to the three modules of the framework:


#### 4.6.1. Interpretability from Cognitive Parameters

Based on the learner’s historical learning records and learning resource information, the multichannel psychometric model is used to make a preliminary diagnosis of the learner, and to estimate the parameters of the learning resource at the same time, thereby constructing the learning resource parameter setEC and learner parameter set SC. Taking into account the interpretability of the learner’s cognitive state, it is worth noting that the parameters are highly interpretable with practical significance in cognitive theory and psychometrics, such as the difficulty and discrimination of the exercise and the ability of the learner, which differs from learning parameters through network training in the deep learning-based learning diagnosis methods.


#### 4.6.2. Interpretability from Learner-Resource Response Networks

In this module, we first characterize the learners and learning resources, and then construct the learner-resource response network. The learner-resource response network is used to simulate the complex learning activities in the process of learners’ answers, that is, the interactive response between learners and learning resources, which is closer to the actual situation and has a better interpretability of the learning process.


#### 4.6.3. Interpretability from Weights of Self-Attention Mechanism

In terms of interpretability for the prediction process, we focused on the reasons why the model makes such predictions, therefore the prediction network in this paper employs the self-attention mechanism to process the fusion features of the learner and the learning resource, in order to fully mine the correlation and importance information between the features of each dimension, so as to provide interpretability from the feature level for the final diagnosis and prediction results, in other words, we can know exactly which features have a greater impact on the final prediction result.


## 4.1. Initial Learning Diagnosis

The main purpose of the initial learning diagnosis module is to construct the multichannel cognitive parameter set.The cognitive parameter set construction process is shown in Figure1. Based on the learner’s historical learning records and learning resource information, the multichannel psychometric model is used to make a preliminary diagnosis of the learner, and to estimate the parameters of the learning resource at the same time, thereby constructing the learning resource parameter set EC and learner parameter set SC.Taking into account the interpretability of the learner’s cognitive state, it is worth noting that the parameters are highly interpretable with practical significance in cognitive theory and psychometrics, such as the difficulty and discrimination of the exercise, the ability of the learner, which differs from learning parameters through network training in the deep learning-based learning diagnosis methods.(1)EC=PM1R,Q,PM2R,Q,…,PMnR,Q,SC=PM1R,Q,PM2R,Q,…,PMnR,Q.PMn is the nth psychometric model. Then, based on the answer record Rs=e1,r1,e2,r2,…,et,rt of learner s, we obtained the learner parameter vector SCs and the learning resource parameter vector EC1 to ECt related to learner s.


## 4.2. Learner and Learning Resources Representation Network

This module consists of two parts: learners representation network and learning resources representation network.Specifically, given the learner parameter vectorSCs of learner s and the learning resource parameter vector ECe of exercise e, firstly, in order to eliminate the difference between the ranges of various parameters, the parameter values are discretized and one-hot encoded, and then SCs is coded as a d0 dimensional 0-1 vector xs and ECe is coded as a d1 dimensional 0-1 vector xe.To better represent learners and learning resources based on cognitive parameters, the learner representation network and learning resource representation network are designed, respectively, which take advantage of auto-encoder to excavate the deep learning features of learners and learning resources. The SAE can copy the input to the output through a hidden layer. It consists of two parts: an encoder that can be represented by the functionh=f⋅ and a decoder that generates reconstruction information y=g⋅, by training the auto-encoder to reproduce the input, h can obtain some useful features, to achieve the purpose of feature learning for the learner’s parameter vector. The learners and learning resource auto-encoder designed in this paper can be expressed as follows:(2)hs=fxs=σW1s×xs+b1s,ys=ghs=σW2s×hs+b2s,he=fxe=σW1e×xe+b1e,ye=ghe=σW2e×he+b2e,W,b, and σ⋅ are weights, bias, and activation function, respectively. We take the intermediate results hs and he of the coding layer as the feature learning results, which can represent a more valuable part of the information in the original parameter vector.


## 4.3. Learner-Resource Response Network

Then, the learner-resource response network is used to simulate the answer process in the actual situation. It takes the deep characteristics of learners and learning resources as the input and outputs the deep characteristics after the interaction between learners and learning resources.(3)f=concaths,he,fd=σW3×f+b3,where σ is the tanh activation function and h and y are the encoder and the decoder, respectively. In addition, the learner parameter encoder’s weight matrix W1s∈Rd2×d0, the bias vector b1s∈Rd2, the decoder’s weight matrix W2s∈Rd0×d2, bias vector b2s∈Rd0, the learning resource parameter encoder’s weight matrix W1e∈Rd3×d1, bias vector b1e∈Rd3, decoder’s weight matrix W2e∈Rd1×d3, and the bias vector b2e∈Rd1 are the parameters of the learner representation network and learning resource representation network. The weight matrix W3∈Rd2+d3×d4 and the bias vector b3∈Rd2+d3 are the parameters of the learner-resource response network. fd is the deep feature of learners and learning resources.


## 4.4. Deep-Shallow Learning Feature Fusion

To improve the performance and interpretability of the model, we regard the original cognitive parameter vector as shallow features and the deep characterization features as deep features, and then we combine the two features as the overall features for diagnosis and prediction.(4)f=concatenatefd,SCs,ECe,where the dimensions of f are d5, in which d5=d4+de+ds.In terms of interpretability for the prediction process, we focus on the reasons why the model makes such predictions. Therefore the prediction network in this paper employs the self-attention mechanism to process the fusion features of the learner and the learning resource, in order to fully mine the correlation and importance information between the features of each dimension, so as to provide interpretability from the feature level for the final diagnosis and prediction results, in other words, we can know exactly which features have a more significant impact on the final prediction result.A convolutional layer is used with a convolution kernel size of 1 to obtain the query vector matrix Query, the key vector matrix Key, and the value vector matrix value throughf as follows:(5)Query=Convf,Key=Convf,Value=Convf.The weight of each dimension of data in the feature is calculated by calculating the dot product between the query vector matrix and the key vector matrix, and the weight indicates the degree of relevance between the task to be queried and each input data.(6)similarityQuery,Keyi=Query•Keyi.On the one hand, numerical conversion of the degree of relevance can be carried out to normalize and organize the original calculated scores into a probability distribution with the sum of weights being 1. On the other hand, it can also highlight the weights of essential elements.(7)ai=SoftMaxsimilarityi=esimilarityi∑j=1dsesimilarityj.The input data is weighted and summed to obtain the fused feature data.(8)fa=∑i=1d5ai⋅Valuei.


## 4.5. Learning Performance Prediction Network

With the features acquired from the feature extraction process, we constructed a learner performance prediction network based on a convolutional neural network, used learner feature vectors and learning resource feature vectors to predict the probability of learners answering a specific exercise correctly, and diagnosed the knowledge mastery of learners and analyze the learning resources through the feature information of learners and learning resources.The convolutional layer is used to extract the spatial information of the deep-shallow fusion feature, the convolution kernel size is set to 3, and the step size is set to 1. Then, the ReLU activation function is added to reduce the dependence between parameters, overfitting is avoided, and the maximum pooling layer is used to reduce the feature dimension, then the predicted valuep of the learner’s answer is obtained under a specific learning task accordingly. It can be expressed as follows:(9)fc=Convfa,fre=Relufc,fp=MaxPoolfre,p=σW4×fp+b4,where fc, fre, and fp are the intermediate variables of the prediction network.To learn the various parameters of the model, the cross-entropy loss function is used between the student’s real answer valuert and the predicted answer value pt as the objective function L of the model as follows:(10)L=−∑i=1Nrtlogpt+1−rtlog1−pt.


## 4.6. Interpretability of the Proposed Framework

The interpretability of our proposed framework includes three aspects, which correspond to the three modules of the framework:


### 4.6.1. Interpretability from Cognitive Parameters

Based on the learner’s historical learning records and learning resource information, the multichannel psychometric model is used to make a preliminary diagnosis of the learner, and to estimate the parameters of the learning resource at the same time, thereby constructing the learning resource parameter setEC and learner parameter set SC. Taking into account the interpretability of the learner’s cognitive state, it is worth noting that the parameters are highly interpretable with practical significance in cognitive theory and psychometrics, such as the difficulty and discrimination of the exercise and the ability of the learner, which differs from learning parameters through network training in the deep learning-based learning diagnosis methods.


### 4.6.2. Interpretability from Learner-Resource Response Networks

In this module, we first characterize the learners and learning resources, and then construct the learner-resource response network. The learner-resource response network is used to simulate the complex learning activities in the process of learners’ answers, that is, the interactive response between learners and learning resources, which is closer to the actual situation and has a better interpretability of the learning process.


### 4.6.3. Interpretability from Weights of Self-Attention Mechanism

In terms of interpretability for the prediction process, we focused on the reasons why the model makes such predictions, therefore the prediction network in this paper employs the self-attention mechanism to process the fusion features of the learner and the learning resource, in order to fully mine the correlation and importance information between the features of each dimension, so as to provide interpretability from the feature level for the final diagnosis and prediction results, in other words, we can know exactly which features have a greater impact on the final prediction result.


## 4.6.1. Interpretability from Cognitive Parameters

Based on the learner’s historical learning records and learning resource information, the multichannel psychometric model is used to make a preliminary diagnosis of the learner, and to estimate the parameters of the learning resource at the same time, thereby constructing the learning resource parameter setEC and learner parameter set SC. Taking into account the interpretability of the learner’s cognitive state, it is worth noting that the parameters are highly interpretable with practical significance in cognitive theory and psychometrics, such as the difficulty and discrimination of the exercise and the ability of the learner, which differs from learning parameters through network training in the deep learning-based learning diagnosis methods.


## 4.6.2. Interpretability from Learner-Resource Response Networks

In this module, we first characterize the learners and learning resources, and then construct the learner-resource response network. The learner-resource response network is used to simulate the complex learning activities in the process of learners’ answers, that is, the interactive response between learners and learning resources, which is closer to the actual situation and has a better interpretability of the learning process.


## 4.6.3. Interpretability from Weights of Self-Attention Mechanism

In terms of interpretability for the prediction process, we focused on the reasons why the model makes such predictions, therefore the prediction network in this paper employs the self-attention mechanism to process the fusion features of the learner and the learning resource, in order to fully mine the correlation and importance information between the features of each dimension, so as to provide interpretability from the feature level for the final diagnosis and prediction results, in other words, we can know exactly which features have a greater impact on the final prediction result.


## 5. Multichannel Intelligent Learning Diagnosis Mechanisms

Within the proposed unified interpretable intelligent learning diagnosis framework in Section4, two specific multichannel intelligent learning diagnosis mechanisms are implemented in this section. As shown in Figure 2, the first implementation is a two-channel learning diagnosis mechanism based on the fusion of IRT and DINA (LDM-ID), and the other is a three-channel learning diagnosis mechanism based on the fusion of Ho-DINA, MIRT, and IRT (LDM-HMI).Figure 2
Based on the proposed unified interpretable learning diagnosis framework, two learning diagnosis mechanisms were developed. LDM-ID was based on IRT and DINA, which extracted two categories of learner cognitive parameters and four categories of exercise cognitive parameters. LDM-HMI was based on Ho-DINA, MIRT, and IRT, which extracted four categories of learner cognitive parameters and seven categories of exercise cognitive parameters, respectively.


### 5.1. Two-Channel Learning Diagnosis Mechanism Based on the Fusion of IRT and DINA (LDM-ID)

Generally, LDM-ID applies the IRT and DINA models for initial learning diagnosis, in which the DINA model uses a mastery vector to represent the master of the learner on each knowledge component. The dimension of the mastery vector is equal to the number of knowledge, and the slip rate and guess rate are employed to represent the exercise parameters. The IRT model uses only one value to represent the overall ability of the learner and describes the exercises with difficulty and discrimination. The information on the parameters used to describe the learners and learning resources in these two models is integrated to obtain the original input parameter setsEC and SC, and then it performs subsequent learning diagnostics.


#### 5.1.1. Initial Diagnosis Models of LDM-ID

The process of learning diagnosis from the initial parameter set has been described in detail in Section4.The response function of the IRT model is as follows:(11)PsI_θ=eI_gues+1−eI_gues1+e−DeI_discsI_θ−eI_diff,where sI_θ is the learner’s learning ability parameter. The parameter eI_guess is generally called the “guessing parameter”, that is, when the learner’s ability value is very low (for example, close to negative infinity), it is still possible to give the right answer to this problem. eI_diff is the problem difficulty parameter and eI_disc is the problem discrimination parameter.The response function of the DINA model is as follows:(12)PsD_α=eD_gues1−η⋅1−eDslipη,where sD_α is the learner’s mastery vector for each piece of knowledge, η is the learner’s ideal answer situation under the corresponding mastery state, eD_slip and eD_gues are the guess and slip parameters of the exercises, respectively, and the slip parameter indicates that the learner has mastered all the knowledge required by the exercises he answered, but answered incorrectly. The guessing parameter is the probability that the learner has not mastered all the knowledge needed to answer the exercises or even not mastered one knowledge, but answered correctly. The specific parameter estimation process is shown in steps 1–7 of algorithm 1.Algorithm 1: Two-channel learning diagnosis mechanism based on the fusion of IRT and DINA (LDM-ID).
(i)
Input: Student exercise response matrix R and question-knowledge correlation matrix Q(ii)
Output: Student exercise score prediction and interpretable parameters(1)
Initialize parameters matrixEC and SC:(2)
EC0=eI_diff0,eI_disc0,eD_gues0,eD_slip0, SC0=sI_θ0,sD_α0(3)
while not converge or not reach iterations do(4)
LR=∏u=1ULRu=∏u=1U∑l=1LPRuαlPαl(5)
LRδ,π=∏k=1N∏j=1Mpπkδjrjk1−pπkδj1−rjk(6)
//update parameters in DINA(7)
eD_slip=e^D_slip,eD_gues=e^D_gues←∂logLR/∂eD_slip=0,∂logLR/∂eD_gues=0(8)
//update parameters in IRT(9)
eI_disc=e^I_disc,eI_diff=e^I_diff,sI_θ=s^I_θ←∂logLR/∂eI_disc=0,∂logLR/∂eI_diff=0,∂logLR/∂sI_θ=0(10)
end while(11)
for student s and exercise edo(12)
//extract student features and exercise features via auto-encoder(13)
hs=fSCs=σW1s×SCs+b1s(14)
ys=ghs=σW2s×hs+b2s(15)
he=fECe=σW1e×ECe+b1e(16)
ye=ghe=σW2e×he+b2e(17)
hs⋅,ys⋅=argminhs⋅,ys⋅hs−SCs‖2,he⋅,ye⋅=argminhe⋅,ye⋅he−ECe‖2(18)
SCs,hs,ECe,he⟶exercise,student,f=concatexercise,student(19)
//calculate keys queries and values in attention function(20)
forl in lenfdo(21)
Query=ConvfKey=ConvfValue=Convf(22)
fa=∑j=1dsesimilarityi/∑j=1dsesimilarityj⋅Valuei(23)
similarityQuery,Keyi=Query•Keyi(24)
end for(25)
end for(26)
fp=MaxPoolReluConvfa(27)
p=σW3×fp+b3(28)
returnp, EC, SC, attention weightsAccordingly, a parameter set of learners and a parameter set of learning resources are constructed based on the learner’s learning record data and learning resource data as follows:(13)EC=eI_diff,eI_disc,eD_gues,eD_slip,SC=sI_θ,sD_α.


#### 5.1.2. Representation and Prediction Network of LDM-ID

We modeled learners and learning resources separately and constructed a performance prediction network to complete the learning diagnosis task as described in Section4 of the paper. The process is described in steps 12–28 of Algorithm 1.Algorithm 2: Three-channel learning diagnosis mechanism based on the fusion of Ho-DINA, MIRT, and IRT.
(i)
Input: Student exercise response matrix R and question-knowledge correlation matrix Q(ii)
Output: Student exercise score prediction and interpretable parameters(1)
Initialize parameters matrix EC and SC:(2)
EC0=eI_diff0,eI_disc0,eH_slip0,eH_gues0,eM_disc0,eM_gues0,eM_diff0, SC0=sI_θ0,sH_θ0,sM_α0,sH_α0(3)
while not converge or not reach iterations do(4)
pθ^←θ=minpαθ,λ^pθ^/pαθ,λ^pθ,1, pα^←α=minLs,g;α^pα^θθ,λ^/Ls,g;αpαθ^,λ^,1, ps^,g^←s,g=minLs^,g^;α^ps^pg^/Ls,g;α^pspg,1(5)
L1R=∏k=1N∏j=1Mpπkπpπkδjrkj1−pπkδj1−rkj, L2R=∏i=1I∏j=1Jerijajθi+dj/1+erijajθi+dj(6)
//update parameters in Ho-DINA(7)
eH_slip=s^,eH_gues=g^,sH_θ=θ^,sH_α=H_α^(8)
//update parameters in IRT(9)
eI_disc=e^I_disc,eI_diff=e^I_diff,sI_θ=s^I_θ←∂logL1R/∂eI_disc=0,∂logL1R/∂eI_diff=0,∂logL1R/∂sI_θ=0(10)
//update parameters in MIRT(11)
eM_gues=e^M_gues,eM_diff=e^M_diff,eM_disc=e^M_disc,sM_α=s^M_α←∂logL2R/∂eM_gues=0,∂logL2R/∂eM_diff=0,∂logL2R/∂eM_disc=0,∂logL2R/∂sM_α=0(12)
end while(13)
ECfinal=eI_difffinal,eI_discfinal,eH_slipfinal,eH_guesfinal,eM_discfinal,eM_guesfinal,eM_difffinal(14)
SCfinal=sI_θfinal,sH_θfinal,sM_αfinal,sH_αfinal(15)
for student s and exercise edo(16)
hs=fSCs=σW1s×SCs+b1s(17)
ys=ghs=σW2s×hs+b2s(18)
he=fECe=σW1e×ECe+b1e(19)
ye=ghe=σW2e×he+b2e(20)
hs⋅,ys⋅=argminhs⋅,ys⋅hs−SCs‖2,he⋅,ye⋅=argminhe⋅,ye⋅he−ECe‖2(21)
SCs,hs,ECe,he⟶exercise,student,f=concatexercise,student(22)
forl in lenfdo(23)
Query=ConvfKey=ConvfValue=Convf(24)
fa=∑i=1dsesimilarityi/∑j=1d5∑j=1d5esimilarityj⋅Valuei(25)
similarityQuery,Keyi=Query•Keyi(26)
end for(27)
end for(28)
fp=MaxPoolReluConvfa(29)
p=σW3×fp+b3(30)
returnp, EC, SC, attention weights


### 5.2. Three-Channel Learning Diagnosis Mechanism Based on the Fusion of Ho-DINA, MIRT, and IRT (LDM-HMI)

LDM-HMI is an implementation of a three-channel learning diagnosis mechanism under our proposed intelligent learning diagnostic framework, which employs the Ho-DINA model, MIRT model, and IRT model for initial learning diagnosis. Based on the knowledge state vector of learners in the DINA model, the Ho-DINA model constructs high-order potential abilities for learners to describe learners further. Compared with the one-dimensional ability value of the IRT model, the MIRT model uses a multidimensional ability vector to represent learners’ knowledge states. Similarly, the parameter information used to describe learners and learning resources in the three models is integrated to obtain the original input parameter set, which can be used for subsequent learning diagnosis.


#### 5.2.1. Initial Diagnosis Models of LDM-HMI

The Ho-DINA model assumes that the cognitive knowledgeαk is independent (partially independent) under a given ability sH_θ. Cognitive knowledge has the following relationship as follows:(14)Pα∣sH_θ=∏k=1KPαk∣sH_θ,Pαk∣sH_θ=eλ0k+λ1ksH_θ1+eλ0k+λ1ksH_θ,where λk represents the loading of knowledge k on the ability sH_θ, that is, under the DINA model, it is assumed that knowledge is locally independent and subordinate to a higher-order ability.The response function of the MIRT model is as follows:(15)PsM_α=eM_gues+1−eM_gues1+e−D⋅eM_discsM_α+eM_diff,where D is a constant; sM_α is the learner’s multidimensional ability vector, and each dimension is defined on a specific knowledge point; eM_disc is the multidimensional discrimination vector of exercise, which is also defined in the dimension of the knowledge point; eM_gues and eM_diff are, respectively, guessing degree and difficulty parameters in the dimension of the question. The parameter estimation process is described in steps 1 to 18 in algorithm 2. The IRT model is the same as in Section 5.1. Based on this, we constructed a parameter set of learners and learning resources as follows:(16)EC=eI_diff,eI_disc,eH_slip,eH_gues,eM_disc,eM_gues,eM_diff,SC=sI_θ,sH_θ,sM_α,sH_α.


#### 5.2.2. Representation and Prediction Network of LDM-HMI

The learners and learning resources are modeled separately, and the performance prediction network is constructed to complete the learning diagnosis task in the same way as described in Section4. The detailed steps are shown in steps 19–33 in algorithm 2


## 5.1. Two-Channel Learning Diagnosis Mechanism Based on the Fusion of IRT and DINA (LDM-ID)

Generally, LDM-ID applies the IRT and DINA models for initial learning diagnosis, in which the DINA model uses a mastery vector to represent the master of the learner on each knowledge component. The dimension of the mastery vector is equal to the number of knowledge, and the slip rate and guess rate are employed to represent the exercise parameters. The IRT model uses only one value to represent the overall ability of the learner and describes the exercises with difficulty and discrimination. The information on the parameters used to describe the learners and learning resources in these two models is integrated to obtain the original input parameter setsEC and SC, and then it performs subsequent learning diagnostics.


### 5.1.1. Initial Diagnosis Models of LDM-ID

The process of learning diagnosis from the initial parameter set has been described in detail in Section4.The response function of the IRT model is as follows:(11)PsI_θ=eI_gues+1−eI_gues1+e−DeI_discsI_θ−eI_diff,where sI_θ is the learner’s learning ability parameter. The parameter eI_guess is generally called the “guessing parameter”, that is, when the learner’s ability value is very low (for example, close to negative infinity), it is still possible to give the right answer to this problem. eI_diff is the problem difficulty parameter and eI_disc is the problem discrimination parameter.The response function of the DINA model is as follows:(12)PsD_α=eD_gues1−η⋅1−eDslipη,where sD_α is the learner’s mastery vector for each piece of knowledge, η is the learner’s ideal answer situation under the corresponding mastery state, eD_slip and eD_gues are the guess and slip parameters of the exercises, respectively, and the slip parameter indicates that the learner has mastered all the knowledge required by the exercises he answered, but answered incorrectly. The guessing parameter is the probability that the learner has not mastered all the knowledge needed to answer the exercises or even not mastered one knowledge, but answered correctly. The specific parameter estimation process is shown in steps 1–7 of algorithm 1.Algorithm 1: Two-channel learning diagnosis mechanism based on the fusion of IRT and DINA (LDM-ID).
(i)
Input: Student exercise response matrix R and question-knowledge correlation matrix Q(ii)
Output: Student exercise score prediction and interpretable parameters(1)
Initialize parameters matrixEC and SC:(2)
EC0=eI_diff0,eI_disc0,eD_gues0,eD_slip0, SC0=sI_θ0,sD_α0(3)
while not converge or not reach iterations do(4)
LR=∏u=1ULRu=∏u=1U∑l=1LPRuαlPαl(5)
LRδ,π=∏k=1N∏j=1Mpπkδjrjk1−pπkδj1−rjk(6)
//update parameters in DINA(7)
eD_slip=e^D_slip,eD_gues=e^D_gues←∂logLR/∂eD_slip=0,∂logLR/∂eD_gues=0(8)
//update parameters in IRT(9)
eI_disc=e^I_disc,eI_diff=e^I_diff,sI_θ=s^I_θ←∂logLR/∂eI_disc=0,∂logLR/∂eI_diff=0,∂logLR/∂sI_θ=0(10)
end while(11)
for student s and exercise edo(12)
//extract student features and exercise features via auto-encoder(13)
hs=fSCs=σW1s×SCs+b1s(14)
ys=ghs=σW2s×hs+b2s(15)
he=fECe=σW1e×ECe+b1e(16)
ye=ghe=σW2e×he+b2e(17)
hs⋅,ys⋅=argminhs⋅,ys⋅hs−SCs‖2,he⋅,ye⋅=argminhe⋅,ye⋅he−ECe‖2(18)
SCs,hs,ECe,he⟶exercise,student,f=concatexercise,student(19)
//calculate keys queries and values in attention function(20)
forl in lenfdo(21)
Query=ConvfKey=ConvfValue=Convf(22)
fa=∑j=1dsesimilarityi/∑j=1dsesimilarityj⋅Valuei(23)
similarityQuery,Keyi=Query•Keyi(24)
end for(25)
end for(26)
fp=MaxPoolReluConvfa(27)
p=σW3×fp+b3(28)
returnp, EC, SC, attention weightsAccordingly, a parameter set of learners and a parameter set of learning resources are constructed based on the learner’s learning record data and learning resource data as follows:(13)EC=eI_diff,eI_disc,eD_gues,eD_slip,SC=sI_θ,sD_α.


### 5.1.2. Representation and Prediction Network of LDM-ID

We modeled learners and learning resources separately and constructed a performance prediction network to complete the learning diagnosis task as described in Section4 of the paper. The process is described in steps 12–28 of Algorithm 1.Algorithm 2: Three-channel learning diagnosis mechanism based on the fusion of Ho-DINA, MIRT, and IRT.
(i)
Input: Student exercise response matrix R and question-knowledge correlation matrix Q(ii)
Output: Student exercise score prediction and interpretable parameters(1)
Initialize parameters matrix EC and SC:(2)
EC0=eI_diff0,eI_disc0,eH_slip0,eH_gues0,eM_disc0,eM_gues0,eM_diff0, SC0=sI_θ0,sH_θ0,sM_α0,sH_α0(3)
while not converge or not reach iterations do(4)
pθ^←θ=minpαθ,λ^pθ^/pαθ,λ^pθ,1, pα^←α=minLs,g;α^pα^θθ,λ^/Ls,g;αpαθ^,λ^,1, ps^,g^←s,g=minLs^,g^;α^ps^pg^/Ls,g;α^pspg,1(5)
L1R=∏k=1N∏j=1Mpπkπpπkδjrkj1−pπkδj1−rkj, L2R=∏i=1I∏j=1Jerijajθi+dj/1+erijajθi+dj(6)
//update parameters in Ho-DINA(7)
eH_slip=s^,eH_gues=g^,sH_θ=θ^,sH_α=H_α^(8)
//update parameters in IRT(9)
eI_disc=e^I_disc,eI_diff=e^I_diff,sI_θ=s^I_θ←∂logL1R/∂eI_disc=0,∂logL1R/∂eI_diff=0,∂logL1R/∂sI_θ=0(10)
//update parameters in MIRT(11)
eM_gues=e^M_gues,eM_diff=e^M_diff,eM_disc=e^M_disc,sM_α=s^M_α←∂logL2R/∂eM_gues=0,∂logL2R/∂eM_diff=0,∂logL2R/∂eM_disc=0,∂logL2R/∂sM_α=0(12)
end while(13)
ECfinal=eI_difffinal,eI_discfinal,eH_slipfinal,eH_guesfinal,eM_discfinal,eM_guesfinal,eM_difffinal(14)
SCfinal=sI_θfinal,sH_θfinal,sM_αfinal,sH_αfinal(15)
for student s and exercise edo(16)
hs=fSCs=σW1s×SCs+b1s(17)
ys=ghs=σW2s×hs+b2s(18)
he=fECe=σW1e×ECe+b1e(19)
ye=ghe=σW2e×he+b2e(20)
hs⋅,ys⋅=argminhs⋅,ys⋅hs−SCs‖2,he⋅,ye⋅=argminhe⋅,ye⋅he−ECe‖2(21)
SCs,hs,ECe,he⟶exercise,student,f=concatexercise,student(22)
forl in lenfdo(23)
Query=ConvfKey=ConvfValue=Convf(24)
fa=∑i=1dsesimilarityi/∑j=1d5∑j=1d5esimilarityj⋅Valuei(25)
similarityQuery,Keyi=Query•Keyi(26)
end for(27)
end for(28)
fp=MaxPoolReluConvfa(29)
p=σW3×fp+b3(30)
returnp, EC, SC, attention weights


## 5.1.1. Initial Diagnosis Models of LDM-ID

The process of learning diagnosis from the initial parameter set has been described in detail in Section4.The response function of the IRT model is as follows:(11)PsI_θ=eI_gues+1−eI_gues1+e−DeI_discsI_θ−eI_diff,where sI_θ is the learner’s learning ability parameter. The parameter eI_guess is generally called the “guessing parameter”, that is, when the learner’s ability value is very low (for example, close to negative infinity), it is still possible to give the right answer to this problem. eI_diff is the problem difficulty parameter and eI_disc is the problem discrimination parameter.The response function of the DINA model is as follows:(12)PsD_α=eD_gues1−η⋅1−eDslipη,where sD_α is the learner’s mastery vector for each piece of knowledge, η is the learner’s ideal answer situation under the corresponding mastery state, eD_slip and eD_gues are the guess and slip parameters of the exercises, respectively, and the slip parameter indicates that the learner has mastered all the knowledge required by the exercises he answered, but answered incorrectly. The guessing parameter is the probability that the learner has not mastered all the knowledge needed to answer the exercises or even not mastered one knowledge, but answered correctly. The specific parameter estimation process is shown in steps 1–7 of algorithm 1.Algorithm 1: Two-channel learning diagnosis mechanism based on the fusion of IRT and DINA (LDM-ID).
(i)
Input: Student exercise response matrix R and question-knowledge correlation matrix Q(ii)
Output: Student exercise score prediction and interpretable parameters(1)
Initialize parameters matrixEC and SC:(2)
EC0=eI_diff0,eI_disc0,eD_gues0,eD_slip0, SC0=sI_θ0,sD_α0(3)
while not converge or not reach iterations do(4)
LR=∏u=1ULRu=∏u=1U∑l=1LPRuαlPαl(5)
LRδ,π=∏k=1N∏j=1Mpπkδjrjk1−pπkδj1−rjk(6)
//update parameters in DINA(7)
eD_slip=e^D_slip,eD_gues=e^D_gues←∂logLR/∂eD_slip=0,∂logLR/∂eD_gues=0(8)
//update parameters in IRT(9)
eI_disc=e^I_disc,eI_diff=e^I_diff,sI_θ=s^I_θ←∂logLR/∂eI_disc=0,∂logLR/∂eI_diff=0,∂logLR/∂sI_θ=0(10)
end while(11)
for student s and exercise edo(12)
//extract student features and exercise features via auto-encoder(13)
hs=fSCs=σW1s×SCs+b1s(14)
ys=ghs=σW2s×hs+b2s(15)
he=fECe=σW1e×ECe+b1e(16)
ye=ghe=σW2e×he+b2e(17)
hs⋅,ys⋅=argminhs⋅,ys⋅hs−SCs‖2,he⋅,ye⋅=argminhe⋅,ye⋅he−ECe‖2(18)
SCs,hs,ECe,he⟶exercise,student,f=concatexercise,student(19)
//calculate keys queries and values in attention function(20)
forl in lenfdo(21)
Query=ConvfKey=ConvfValue=Convf(22)
fa=∑j=1dsesimilarityi/∑j=1dsesimilarityj⋅Valuei(23)
similarityQuery,Keyi=Query•Keyi(24)
end for(25)
end for(26)
fp=MaxPoolReluConvfa(27)
p=σW3×fp+b3(28)
returnp, EC, SC, attention weightsAccordingly, a parameter set of learners and a parameter set of learning resources are constructed based on the learner’s learning record data and learning resource data as follows:(13)EC=eI_diff,eI_disc,eD_gues,eD_slip,SC=sI_θ,sD_α.


## 5.1.2. Representation and Prediction Network of LDM-ID

We modeled learners and learning resources separately and constructed a performance prediction network to complete the learning diagnosis task as described in Section4 of the paper. The process is described in steps 12–28 of Algorithm 1.Algorithm 2: Three-channel learning diagnosis mechanism based on the fusion of Ho-DINA, MIRT, and IRT.
(i)
Input: Student exercise response matrix R and question-knowledge correlation matrix Q(ii)
Output: Student exercise score prediction and interpretable parameters(1)
Initialize parameters matrix EC and SC:(2)
EC0=eI_diff0,eI_disc0,eH_slip0,eH_gues0,eM_disc0,eM_gues0,eM_diff0, SC0=sI_θ0,sH_θ0,sM_α0,sH_α0(3)
while not converge or not reach iterations do(4)
pθ^←θ=minpαθ,λ^pθ^/pαθ,λ^pθ,1, pα^←α=minLs,g;α^pα^θθ,λ^/Ls,g;αpαθ^,λ^,1, ps^,g^←s,g=minLs^,g^;α^ps^pg^/Ls,g;α^pspg,1(5)
L1R=∏k=1N∏j=1Mpπkπpπkδjrkj1−pπkδj1−rkj, L2R=∏i=1I∏j=1Jerijajθi+dj/1+erijajθi+dj(6)
//update parameters in Ho-DINA(7)
eH_slip=s^,eH_gues=g^,sH_θ=θ^,sH_α=H_α^(8)
//update parameters in IRT(9)
eI_disc=e^I_disc,eI_diff=e^I_diff,sI_θ=s^I_θ←∂logL1R/∂eI_disc=0,∂logL1R/∂eI_diff=0,∂logL1R/∂sI_θ=0(10)
//update parameters in MIRT(11)
eM_gues=e^M_gues,eM_diff=e^M_diff,eM_disc=e^M_disc,sM_α=s^M_α←∂logL2R/∂eM_gues=0,∂logL2R/∂eM_diff=0,∂logL2R/∂eM_disc=0,∂logL2R/∂sM_α=0(12)
end while(13)
ECfinal=eI_difffinal,eI_discfinal,eH_slipfinal,eH_guesfinal,eM_discfinal,eM_guesfinal,eM_difffinal(14)
SCfinal=sI_θfinal,sH_θfinal,sM_αfinal,sH_αfinal(15)
for student s and exercise edo(16)
hs=fSCs=σW1s×SCs+b1s(17)
ys=ghs=σW2s×hs+b2s(18)
he=fECe=σW1e×ECe+b1e(19)
ye=ghe=σW2e×he+b2e(20)
hs⋅,ys⋅=argminhs⋅,ys⋅hs−SCs‖2,he⋅,ye⋅=argminhe⋅,ye⋅he−ECe‖2(21)
SCs,hs,ECe,he⟶exercise,student,f=concatexercise,student(22)
forl in lenfdo(23)
Query=ConvfKey=ConvfValue=Convf(24)
fa=∑i=1dsesimilarityi/∑j=1d5∑j=1d5esimilarityj⋅Valuei(25)
similarityQuery,Keyi=Query•Keyi(26)
end for(27)
end for(28)
fp=MaxPoolReluConvfa(29)
p=σW3×fp+b3(30)
returnp, EC, SC, attention weights


## 5.2. Three-Channel Learning Diagnosis Mechanism Based on the Fusion of Ho-DINA, MIRT, and IRT (LDM-HMI)

LDM-HMI is an implementation of a three-channel learning diagnosis mechanism under our proposed intelligent learning diagnostic framework, which employs the Ho-DINA model, MIRT model, and IRT model for initial learning diagnosis. Based on the knowledge state vector of learners in the DINA model, the Ho-DINA model constructs high-order potential abilities for learners to describe learners further. Compared with the one-dimensional ability value of the IRT model, the MIRT model uses a multidimensional ability vector to represent learners’ knowledge states. Similarly, the parameter information used to describe learners and learning resources in the three models is integrated to obtain the original input parameter set, which can be used for subsequent learning diagnosis.


### 5.2.1. Initial Diagnosis Models of LDM-HMI

The Ho-DINA model assumes that the cognitive knowledgeαk is independent (partially independent) under a given ability sH_θ. Cognitive knowledge has the following relationship as follows:(14)Pα∣sH_θ=∏k=1KPαk∣sH_θ,Pαk∣sH_θ=eλ0k+λ1ksH_θ1+eλ0k+λ1ksH_θ,where λk represents the loading of knowledge k on the ability sH_θ, that is, under the DINA model, it is assumed that knowledge is locally independent and subordinate to a higher-order ability.The response function of the MIRT model is as follows:(15)PsM_α=eM_gues+1−eM_gues1+e−D⋅eM_discsM_α+eM_diff,where D is a constant; sM_α is the learner’s multidimensional ability vector, and each dimension is defined on a specific knowledge point; eM_disc is the multidimensional discrimination vector of exercise, which is also defined in the dimension of the knowledge point; eM_gues and eM_diff are, respectively, guessing degree and difficulty parameters in the dimension of the question. The parameter estimation process is described in steps 1 to 18 in algorithm 2. The IRT model is the same as in Section 5.1. Based on this, we constructed a parameter set of learners and learning resources as follows:(16)EC=eI_diff,eI_disc,eH_slip,eH_gues,eM_disc,eM_gues,eM_diff,SC=sI_θ,sH_θ,sM_α,sH_α.


### 5.2.2. Representation and Prediction Network of LDM-HMI

The learners and learning resources are modeled separately, and the performance prediction network is constructed to complete the learning diagnosis task in the same way as described in Section4. The detailed steps are shown in steps 19–33 in algorithm 2


## 5.2.1. Initial Diagnosis Models of LDM-HMI

The Ho-DINA model assumes that the cognitive knowledgeαk is independent (partially independent) under a given ability sH_θ. Cognitive knowledge has the following relationship as follows:(14)Pα∣sH_θ=∏k=1KPαk∣sH_θ,Pαk∣sH_θ=eλ0k+λ1ksH_θ1+eλ0k+λ1ksH_θ,where λk represents the loading of knowledge k on the ability sH_θ, that is, under the DINA model, it is assumed that knowledge is locally independent and subordinate to a higher-order ability.The response function of the MIRT model is as follows:(15)PsM_α=eM_gues+1−eM_gues1+e−D⋅eM_discsM_α+eM_diff,where D is a constant; sM_α is the learner’s multidimensional ability vector, and each dimension is defined on a specific knowledge point; eM_disc is the multidimensional discrimination vector of exercise, which is also defined in the dimension of the knowledge point; eM_gues and eM_diff are, respectively, guessing degree and difficulty parameters in the dimension of the question. The parameter estimation process is described in steps 1 to 18 in algorithm 2. The IRT model is the same as in Section 5.1. Based on this, we constructed a parameter set of learners and learning resources as follows:(16)EC=eI_diff,eI_disc,eH_slip,eH_gues,eM_disc,eM_gues,eM_diff,SC=sI_θ,sH_θ,sM_α,sH_α.


## 5.2.2. Representation and Prediction Network of LDM-HMI

The learners and learning resources are modeled separately, and the performance prediction network is constructed to complete the learning diagnosis task in the same way as described in Section4. The detailed steps are shown in steps 19–33 in algorithm 2


## 6. Experiment Evaluation

In this section, we evaluated the learning performance prediction and interpretability attained by the proposed LDM-ID and LDM-HMI. We also provided some potential applications of the proposed method in the field of intelligent tutoring systems. The details of the experiments will be discussed below, while the codes for the experiments in this paper are released athttps://github.com/CCNUZFW/LDM-ID-HMI.


### 6.1. Experimental Datasets

To verify the effectiveness of the proposed intelligent learning diagnosis model, three datasets were employed to carry out the experiments. These datasets include two real-world datasets Math11, CL212, and a virtual dataset Synthetic-53, of which Math1 and Synthetic-5 are both open datasets that are widely used in intelligent learning diagnosis tasks and are of moderate size, containing information such as learner-response matrices and question-knowledge point matrices that are commonly used in online intelligent learning diagnosis. CL21 is a smaller dataset of real data collected in four actual university offline classrooms and can be used to validate the applicability of the LDM framework in small offline classrooms. The details of experimental datasets are as follows:(i)
Math1: This dataset comes from several objective and subjective test questions, and answer records are collected by Wu et al. [48] from a college mathematics final test. For the consistency between datasets, this experiment selects the records of the learners’ answers to the objective questions, which contains 4209 learners’ answers on 15 exercises with 11 knowledge points.(ii)
CL21: This dataset is collected by one of our authors, Dr. Chunyan Zeng, who is serving as the instructor of a C programming language class at the Hubei University of Technology in 2021. The dataset is collected through an intelligent tutoring system, named Chaoxing. This dataset includes the answer records of 93 learners on 36 exercises with 12 knowledge points.(iii)
Synthetic-5: This is a virtual dataset that contains the answer records of 2,000 learners on 50 questions with 5 knowledge points.Table1 shows a summary of these three datasets. The Q matrices were labeled by domain experts, and the historical response matrices R were collected by the intelligent tutoring systems. With the availability of the above data, the experiments were carried out with an NVIDIA GeForce GTX 2080Ti GPU in the environment of Python 3.6 and Pytorch 1.9.0.Table 1
Summary of experimental datasets.
DatasetAnswering recordsNumber of learnersKnowledge pointsNumber of exercisesMath16313542091115Synthetic-51000002000550CL213348931236


### 6.2. The Methods to Be Compared

Three proposals need to be evaluated: the effectiveness of learner and learning resource representation networks, the effectiveness of deep and shallow learning feature fusion, and the effectiveness of attention mechanisms. In addition, the proposed models are compared with 11 state-of-the-art methods such as item response theory (IRT) [26], multidimensional item response theory (MIRT) [10], deterministic input noise and door model (DINA) [12], Ho-DINA [15], PMF-LDM [27], neural cognitive diagnosis model (NeuralCDM) [45], DeepMFLD [46], and IRR [47], which are listed as follows:(i)
IRT [26]: The IRT model is a psychometrics learning diagnosis model that uses logic functions to model the relationship between the learner’s learning state, the distinction, and difficulty of test questions, and the learner’s response.(ii)
MIRT [10]: The MIRT model can estimate learners’ abilities in multiple dimensions at the same time, and consider the relationship between different dimensions of ability. Therefore, MIRT can estimate multidimensional abilities more effectively.(iii)
DINA [12]: The DINA model is also a psychometrics learning diagnosis model. DINA describes the learner as a multidimensional knowledge point mastering vector and judges the learner’s answering results according to the question-knowledge correlation.(iv)
Ho-DINA [15]: Based on the DINA model, the Ho-DINA model assumes that the mastery state of knowledge points is also related to the learner’s general ability, which is the learner’s high-level ability.(v)
PMF-LDM [27]: PMF-LDM is an intelligent learning diagnosis method based on probability matrix decomposition, which assumes that the learner’s response matrix is determined by the inner product of the learner’s knowledge proficiency matrix and the knowledge point vector contained in the exercise, and uses the random gradient descent method to constantly diagnose the learning performance.(vi)
NeuralCDM [45]: NeuralCDM is a deep learning-based learning diagnosis method that combines neural networks to learn the complex answering process, projecting learners and exercises as latent space vectors, and using multiple neural layers to model its interaction process to predict learners answering results.(vii)
DeepMFLD [46]: DeepMFLD is a matrix decomposition learning diagnosis method based on deep learning. It obtains the learners’ embedding and the exercises’ embedding separately by deep matrix decomposition, then fuses them with a deep neural network, and finally predicts the learner’s performance by a fully connected layer.(viii)
IRR-IRT, IRR-MIRT, IRR-DINA, and IRR-NeuralCDM [47]: The IRR is a general framework, which introduces pairwise learning into learning diagnosis. Four specific methods based on the IRR framework are used as the baseline: IRR-IRT, IRR-MIRT, IRR-DINA, and IRR-NCDM. The first three methods enhance the traditional psychometric diagnostic models IRT, MIRT, and DINA based on the IRR framework, while the NeuralCDM is also used to integrate with the IRR framework to form IRR-NeuralCDM.


### 6.3. Evaluation Metrics

Since the diagnosis results of the learners’ knowledge mastery state cannot be judged objectively and accurately, the diagnosis results are usually used to predict the probability that the learner will answer correctly and compare it with the true value to evaluate the performance of the model. This paper uses AUC (area under curve) and RMSE (root mean square error) as the evaluation indicators of the experimental results. AUC is the area under the indicator curve. The AUC value of 0.5 represents the randomly obtainable score. The higher the AUC score, the more accurate the prediction result. RMSE is used to measure the deviation between the estimated value and the true value. The smaller the value, the closer the mining result is to the true value. The calculation formula is as follows:(17)RMSEyi,y^i=1m∑i=1myi−y^i2,where m represents the size of the test data set, yi is the true value of the student’s answer, and y^i is the predicted value of the student’s answer.


### 6.4. Experimental Results and Analysis

In our experiments, a 5-fold cross-validation is performed, and the average AUC and RMSE of five times experiments are used as the evaluation metrics. Furthermore, 80% samples of each dataset are employed as the training set, and 20% of the data are left for the test to check whether the proposed method is generalized well.


#### 6.4.1. Comparison with the State-of-the-Art Methods

For the LDM-ID and LDM-HMI methods, detailed settings are as follows: Sigmoid is used as the activation function of the fully connected layer, and the CNN in the prediction network consists of a convolutional layer, a ReLU activation layer, and a maximum pooling layer, where the convolutional kernel size is 3. The model is parameterized by the Adam optimizer, the learning rate is set to 0.001, and the dropout ratio is set to 0.2. In the learner representation network and learning resource representation network, the learner potential vector and learning resource potential vector dimensions are set to 128 and 64, respectively. In addition, in the baseline models, 3PL-IRT is used for comparison, and the capability latitude value is set to 3 in the MIRT model.As shown in Table2, in terms of the accuracy of learning performance prediction, among the traditional psychometric-based learning diagnostic models, MIRT outperformed IRT, DINA, Ho-DINA, and PMF-LDM overall on the three datasets. The performance of NeuralCDM, DeepMFLD, IRR, and the proposed methods LDM-ID and LDM-HMI generally outperformed the psychometric-based learning diagnostic models, which indicates that the deep network-based learning diagnostic methods outperform the traditional psychometric-based learning diagnostic methods. Usually, deep networks have more complex structures and more parameters, and the information they can characterize will be richer, while psychometric-based methods usually have simpler structures, fewer parameters, and limited characterization capabilities.Table 2
Comparison with the state-of-the-art methods in terms of learning performance prediction.
ModelCL21Math1Synthetic-5AUCRMSEAUCRMSEAUCRMSEIRT [26]73.12%0.436568.31%0.516478.52%0.4824MIRT [10]73.24%0.435269.83%0.494679.80%0.4733DINA [12]69.92%0.485964.46%0.529773.80%0.4968Ho-DINA [15]70.37%0.472366.58%0.521572.45%0.4988PMF-LDM [27]63.74%0.651568.03%0.488462.40%0.6038NeuralCDM [45]76.88%0.427472.90%0.453379.77%0.4756DeepMFLD [46]80.40%0.422677.74%0.435676.68%0.4897IRR-IRT [47]71.23%0.439574.10%0.450572.65%0.4996IRR-MIRT [47]81.29%0.421380.72%0.423181.35%0.4073IRR-DINA [47]75.51%0.427181.40%0.423481.03%0.4087IRR-NeuralCDM [47]75.02%0.427978.56%0.427875.85%0.4917Proposed LDM-ID86.44%0.410581.35%0.417188.02%0.3667Proposed LDM-HMI85.91%0.412281.89%0.416390.25%0.3528The significance of bold values means the best learning performance prediction on each dataset.As shown in Figure3, the LDM-ID and LDM-HMI methods proposed in this paper outperformed all baseline models in terms of the accuracy of learning performance prediction. On the three datasets, their AUC improved by about 11.45% and RMSE decreased by about 0.07 compared to MIRT on average, their AUC improved by about 8.57% and RMSE decreased by about 0.06 compared to NeuralCDM on average. On the one hand, the cognitive parameters in this paper provide more effective input information for the diagnosis and prediction tasks from the source. On the other hand, this paper constructs a more capable network of learners and learning resources and explores the potential characteristics of learners and learning resources more effectively.Figure 3
The proposed methods were compared with 11 state-of-the-art methods. In this paper, 11 state-of-the-art learning diagnosis methods were replicated and experiments were conducted on two publicly available datasets and one self-built dataset, and evaluated using two metrics, AUC and RMSE. A larger AUC indicates higher prediction accuracy, and a smaller RMSE indicates better stability of the methods.
(a)(b)(c)(d)(e)(f)Furthermore, among the two implementations of the LDM framework proposed in this paper, LDM-HMI performs better on the larger-scale datasets Math1 and Synthetic-5, with an AUC about 1.39% higher and an RMSE about 0.007 lower than that of LDM-ID over the two datasets, while on the smaller-scale dataset CL21, LDM-ID performs better, with an AUC about 0.53% higher and an RMSE about 0.002 lower than that of LDM-HMI. This indicates that the LDM-ID method may be more suitable for small data volume scenarios. In other words, under the LDM framework of this paper, the specific implementation can be flexibly adjusted according to the requirements to achieve better adaptation to the actual scenarios.As shown in Table3, the LDM-ID and LDM-HMI proposed in this paper are comparable to the deep learning diagnostic method NeuralCDM, DeepMFLD, and IRR in terms of time loss. The prediction time loss of the proposed method on the medium-sized dataset Math1 is comparable to that of the traditional psychometric methods. On the small dataset CL21 and large dataset Synthetic-5, the time loss of the proposed method is one order of magnitude higher compared to the traditional psychometric methods. Overall, the prediction time of the proposed method is about 0.2 ms on a single sample, which is sufficient for practical educational application scenarios.Table 3
Comparison with traditional methods in terms of prediction time cost.
ModelTime cost for predictionCL21Math1Synthetic-5IRT [26]16 ms203 ms141 msMIRT [10]90 ms385 ms597 msDINA [12]16 ms31 ms47 msHo-DINA [15]57 ms198 ms124 msPMF-LDM [27]15 ms45 ms61 msNeuralCDM [45]156 ms216 ms3359 msDeepMFLD [46]270 ms352 ms3822 msIRR-IRT [47]192 ms401 ms3591 msIRR-MIRT [47]229 ms435 ms3603 msIRR-DINA [47]223 ms462 ms3609 msIRR-NeuralCDM [47]218 ms316 ms3583 msProposed LDM-ID187 ms277 ms3397 msProposed LDM-HMI203 ms303 ms3374 ms


#### 6.4.2. Effectiveness of Learner and Learning Resource Representation Networks

To verify the effectiveness of the learner and learning resource characterization network in the interpretable intelligent learning diagnosis framework proposed in this paper, experiments are conducted to compare the auto-encoder-based characterization network in this paper with the fully connected deep neural network in the traditional approach.As shown in Table4, in the LDM-ID and LDM-HMI methods, compared with the fully connected deep neural network, the auto-encoder-based representational network improves the AUC of the model by about 0.85% and 0.9% on average over the three data sets, respectively, and reduces the RMSE by about 0.002 and 0.003, respectively. It proves that the auto-encoder-based learner and learning resource representation network is more effective. Among them, the encoder learns to retain as much relevant information in the potential space and discard irrelevant parts, and the decoder learns the potential space information to reconstruct the input, which achieves an effective denoising effect and contributes to a more effective potential feature representation of learners and learning resources.Table 4
Learning performance prediction experiments for learner and learning resource representation networks.
ModelNetwork structureCL21Math 1Synthetic-5AUCRMSEAUCRMSEAUCRMSELDM-IDDNN84.78%0.417979.26%0.422386.15%0.3691SAE85.32%0.415780.47%0.418486.96%0.3682LDM-HMIDNN81.56%0.421979.74%0.421588.60%0.3579SAE82.65%0.420181.02%0.416888.93%0.3562The significance of bold values means the best learning performance prediction on each dataset.


#### 6.4.3. Effectiveness of Deep and Shallow Learning Feature Fusion

To verify the effectiveness of the fusion of deep and shallow features in the interpretable intelligent learning diagnosis framework proposed in this paper, experiments are conducted in three forms: using only shallow features, using only deep features, and fusing deep and shallow features.As shown in Table5, the fusion of deep and shallow features has the most significant effect on the model effectiveness on the average over the three datasets, with the AUC of the fusion of deep and shallow features improving by about 1.46% and 1.73% and the RMSE decreasing by about 0.011 and 0.016 in the LDM-ID and LDM-HMI approaches relative to the use of only shallow features, relative to the use of only deep features, the AUC of deep and shallow feature fusion is increased by about 0.42% and 0.72%, and the RMSE is reduced by about 0.003 and 0.011. Therefore, the fusion of deep and shallow features helps to improve the performance of the model, because the deep and shallow features provide different valid information from the high-dimensional space and low-dimensional space, respectively, to achieve better prediction results. Also, the fusion of deep and shallow features has better interpretation compared to using only deep features.Table 5
Learning performance prediction experiments with a fusion of deep and shallow features.
ModelRepresentationCL21Math 1Synthetic-5AUCRMSEAUCRMSEAUCRMSELDM-IDShallow feature83.16%0.429177.41%0.438885.23%0.3749Deep feature84.31%0.420678.69%0.426585.91%0.3718Shallow + deep feature84.78%0.417979.26%0.422386.15%0.3691LDM-HMIShallow feature79.66%0.443578.33%0.439786.73%0.3664Deep feature81.04%0.435979.21%0.434287.48%0.3638Shallow + deep feature81.56%0.421979.74%0.421588.60%0.3579The significance of bold values means the best learning performance prediction on each dataset.


#### 6.4.4. Effectiveness of Attention Mechanism

Based on the deep and shallow feature fusion, considering that each dimension of deep features and shallow features may have different effects on the performance of model prediction, giving different degrees of attention to different dimensional features may produce better results. Therefore, this paper introduces an attention mechanism in the prediction network to further process the deep and shallow fused features, and conducts experimental validation.As shown in Table6, after introducing the attention mechanism, the AUC is improved by about 0.92% and 1.42% and the RMSE is reduced by about 0.004 and 0.005 on average in the three datasets in the LDM-ID and LDM-HMI methods, which fully illustrates that the introduction of the attention mechanism can improve the performance of the model. The attention mechanism assigns different weights to the features of different dimensions, which not only helps to improve the accuracy of the prediction results but also can provide explanatory information for the prediction results. It can help us to know the potential connection between the features and the output.Table 6
Experiments on the prediction of learning performance of attention mechanism.
ModelAttention settingCL21Math 1Synthetic-5AUCRMSEAUCRMSEAUCRMSELDM-IDWithout attention84.78%0.417979.26%0.422386.15%0.3691With attention85.39%0.412379.84%0.419287.71%0.3669LDM-HMIWithout attention81.56%0.421979.74%0.421588.60%0.3579With attention84.70%0.413480.33%0.418589.14%0.3537The significance of bold values means the best learning performance prediction on each dataset.In addition, this paper also verifies the effect of the joint action of the auto-encoder-based learner and learning resource representation network and attention mechanism based on deep and shallow feature fusion, which is conducted in three cases, namely, introducing the auto-encoder-based learner and learning resource representation network alone, introducing the attention mechanism alone, and introducing the auto-encoder-based learner and learning resource representation network and attention mechanism simultaneously. The experiments were conducted.As shown in Table7, the model has the best performance when the auto-encoder-based learner and learning resource representation network and the attention mechanism are introduced simultaneously. The AUC improves by 1.02% and 1.82%, and the RMSE decreases by 0.003 and 0.004 in the LDM-ID and LDM-HMI methods on average of the three datasets compared to the auto-encoder-based learner and learning resource representation network alone. Compared with the attention mechanism alone, the AUC of the model increased by 0.96% and 1.29%, and the RMSE decreased by 0.001 and 0.001. This indicates that the model performs best when the auto-encoder-based learner and learning resource representation network and the attention mechanism are combined, which can effectively represent the learners and learning resources while assigning different weights to the features of different dimensions.Table 7
Learning performance prediction experiments for the representation network and attention mechanism.
ModelAttention settingCL21Math 1Synthetic-5AUCRMSEAUCRMSEAUCRMSELDM-IDSAE85.32%0.415780.47%0.418486.96%0.3682Attention85.39%0.412379.84%0.419287.71%0.3669SAE with attention86.44%0.410581.35%0.417188.02%0.3667LDM-HMISAE82.65%0.420181.02%0.416888.93%0.3562Attention84.70%0.413480.33%0.418589.14%0.3537SAE with attention85.91%0.412281.89%0.416390.25%0.3528The significance of bold values means the best learning performance prediction on each dataset.


#### 6.4.5. Interpretable Analysis and Educational Applications

The interpretable intelligent learning diagnostic framework proposed in this paper has good interpretability, which comes from three main aspects: cognitive parameters, learner-resource response network, and weighting information of attention mechanisms.1) Interpretability of cognitive parameters: In the initial part of the model, this paper conducts a preliminary diagnostic evaluation of learners based on different cognitive diagnostic model, and obtains some cognitive parameters that have practical significance in the educational field, such as the multidimensional ability values of learners in the MIRT model, the binary vector of learners’ knowledge mastery in the DINA model, the difficulty, differentiation, the skipping rate and guessing rate of learning resources. These parameters can complement each other and form a comprehensive and multidimensional description of the learners and learning resources, which can be used as input to the subsequent prediction network to provide a better explanation.For example, for learners, the proposed interpretable intelligent learning diagnostic framework can be combined with multiperspective cognitive diagnostic theories to evaluate learners’ learning states, which can not only understand the differences in learners’ multidimensional and fine-grained potential cognition status but also obtain the overall ability level of learners. In addition, it can provide refined tutoring information for subsequent teaching practice. Figure4(a) shows the knowledge mastery of four randomly selected learners in the CL21 dataset and their overall proficiency level. It shows that learner S1 has a low mastery of knowledge points K8 and K10. Figure 4(b) shows the overall proficiency level of the four learners, and we can see that the overall proficiency level of learner S4 is low, indicating that his foundation may be weak and needs to consolidate his basic knowledge. Figure 5(a) visualizes six cognitive parameters of 20 exercises on the CL21 dataset. This information can be effectively applied in various aspects, such as learner profiling and learner proficiency analysis.Figure 4
There are two sets of interpretable cognitive parameters for learners, the knowledge mastery cognitive parameters and the learning ability parameters. For the knowledge mastery cognitive parameter, 0 represents no mastery of the knowledge, and 1 represents full mastery of the knowledge. For the cognitive parameter of learning ability, the range of values is from −1.5 to 1.5, representing low to high learning ability. The higher the learning ability parameters, the higher the probability of answering the exercise correctly. (a) Knowledge Mastery status parameters of learners 1∼4 on knowledge points K1∼K12. (b) Learning ability parameters of learners 1∼4.
(a)(b)Figure 5
As in Figure (a), 6 groups of the relevant cognitive parameters of 20 exercises are intercepted from CL21. The darker the color is, the greater the value is. For Figure (b), the correlation coefficients between learner embedding and learning resource embedding are extracted in the learner-resource response network. For Figure (c), the weight factors of the self-attention mechanism on the learning feature vector are used to speculate the answering results.
(a)(b)(c)Second, for learning resources, this method can provide comprehensive parameter characterization information of learning resources. Based on these parameters, a more accurate learning resource recommendation service can be provided for learners. For example, when learners have a higher mastery of knowledge pointa, we can recommend more difficult learning resources to help learners master knowledge point a more firmly, or when there is a small difference in the diagnostic results of different learners’ mastery of knowledge point b, we can select more differentiated learning resources about b to provide detailed learning diagnosis for these learners. This rich information about learning resources can assist in practical applications such as educational resource modeling and test question quality analysis.2) Interpretability of learner-resource response network: After deep characterization of learners and learning resources, this paper constructs a learner-learning resource response network to model the learner-learning resource response process in the actual scenario, simulating the interrelationship between the potential learning state of learners and the response results in the response process, which is more suitable for the actual situation and provides explanations in the modeling method. The heat map of the Pearson coefficient correlation between the learner characteristics and the learning resource characteristics is constructed, and Figure5(b) shows that there is a more significant connection between the two. The learner-learning resource response network can be applied to learning resource recommendation, learning interaction analysis, and adaptive learning systems.3) Interpretability of attention mechanisms: Based on the model’s deep and shallow integration features of learners and learning resources, an attention mechanism is introduced to obtain the difference in the degree of influence of features of different dimensions on the actual response results of learners, so that the importance level of features provides retrospective explanatory information for predicting results and provides support for educational cue analysis, educational visualization research, etc.Second, from the perspective of learning resources, this method can provide comprehensive parameter characterization information for learning resources. Figure5(c) shows the exercise parameter information of the CL21 data set. Based on these exercise parameters, it is possible to provide learners with more accurate learning resource recommendation services. For example, when the learner has a high degree of mastery of knowledge point a, we can recommend difficult topic resources about knowledge point a to help learners grasp this knowledge point more firmly, or when different learners have a small difference in the diagnosis result of the mastery of knowledge point b, we can make a more detailed learning diagnosis for these learners by selecting exercise resources with a greater degree of discrimination.


## 6.1. Experimental Datasets

To verify the effectiveness of the proposed intelligent learning diagnosis model, three datasets were employed to carry out the experiments. These datasets include two real-world datasets Math11, CL212, and a virtual dataset Synthetic-53, of which Math1 and Synthetic-5 are both open datasets that are widely used in intelligent learning diagnosis tasks and are of moderate size, containing information such as learner-response matrices and question-knowledge point matrices that are commonly used in online intelligent learning diagnosis. CL21 is a smaller dataset of real data collected in four actual university offline classrooms and can be used to validate the applicability of the LDM framework in small offline classrooms. The details of experimental datasets are as follows:(i)
Math1: This dataset comes from several objective and subjective test questions, and answer records are collected by Wu et al. [48] from a college mathematics final test. For the consistency between datasets, this experiment selects the records of the learners’ answers to the objective questions, which contains 4209 learners’ answers on 15 exercises with 11 knowledge points.(ii)
CL21: This dataset is collected by one of our authors, Dr. Chunyan Zeng, who is serving as the instructor of a C programming language class at the Hubei University of Technology in 2021. The dataset is collected through an intelligent tutoring system, named Chaoxing. This dataset includes the answer records of 93 learners on 36 exercises with 12 knowledge points.(iii)
Synthetic-5: This is a virtual dataset that contains the answer records of 2,000 learners on 50 questions with 5 knowledge points.Table1 shows a summary of these three datasets. The Q matrices were labeled by domain experts, and the historical response matrices R were collected by the intelligent tutoring systems. With the availability of the above data, the experiments were carried out with an NVIDIA GeForce GTX 2080Ti GPU in the environment of Python 3.6 and Pytorch 1.9.0.Table 1
Summary of experimental datasets.
DatasetAnswering recordsNumber of learnersKnowledge pointsNumber of exercisesMath16313542091115Synthetic-51000002000550CL213348931236


## 6.2. The Methods to Be Compared

Three proposals need to be evaluated: the effectiveness of learner and learning resource representation networks, the effectiveness of deep and shallow learning feature fusion, and the effectiveness of attention mechanisms. In addition, the proposed models are compared with 11 state-of-the-art methods such as item response theory (IRT) [26], multidimensional item response theory (MIRT) [10], deterministic input noise and door model (DINA) [12], Ho-DINA [15], PMF-LDM [27], neural cognitive diagnosis model (NeuralCDM) [45], DeepMFLD [46], and IRR [47], which are listed as follows:(i)
IRT [26]: The IRT model is a psychometrics learning diagnosis model that uses logic functions to model the relationship between the learner’s learning state, the distinction, and difficulty of test questions, and the learner’s response.(ii)
MIRT [10]: The MIRT model can estimate learners’ abilities in multiple dimensions at the same time, and consider the relationship between different dimensions of ability. Therefore, MIRT can estimate multidimensional abilities more effectively.(iii)
DINA [12]: The DINA model is also a psychometrics learning diagnosis model. DINA describes the learner as a multidimensional knowledge point mastering vector and judges the learner’s answering results according to the question-knowledge correlation.(iv)
Ho-DINA [15]: Based on the DINA model, the Ho-DINA model assumes that the mastery state of knowledge points is also related to the learner’s general ability, which is the learner’s high-level ability.(v)
PMF-LDM [27]: PMF-LDM is an intelligent learning diagnosis method based on probability matrix decomposition, which assumes that the learner’s response matrix is determined by the inner product of the learner’s knowledge proficiency matrix and the knowledge point vector contained in the exercise, and uses the random gradient descent method to constantly diagnose the learning performance.(vi)
NeuralCDM [45]: NeuralCDM is a deep learning-based learning diagnosis method that combines neural networks to learn the complex answering process, projecting learners and exercises as latent space vectors, and using multiple neural layers to model its interaction process to predict learners answering results.(vii)
DeepMFLD [46]: DeepMFLD is a matrix decomposition learning diagnosis method based on deep learning. It obtains the learners’ embedding and the exercises’ embedding separately by deep matrix decomposition, then fuses them with a deep neural network, and finally predicts the learner’s performance by a fully connected layer.(viii)
IRR-IRT, IRR-MIRT, IRR-DINA, and IRR-NeuralCDM [47]: The IRR is a general framework, which introduces pairwise learning into learning diagnosis. Four specific methods based on the IRR framework are used as the baseline: IRR-IRT, IRR-MIRT, IRR-DINA, and IRR-NCDM. The first three methods enhance the traditional psychometric diagnostic models IRT, MIRT, and DINA based on the IRR framework, while the NeuralCDM is also used to integrate with the IRR framework to form IRR-NeuralCDM.


## 6.3. Evaluation Metrics

Since the diagnosis results of the learners’ knowledge mastery state cannot be judged objectively and accurately, the diagnosis results are usually used to predict the probability that the learner will answer correctly and compare it with the true value to evaluate the performance of the model. This paper uses AUC (area under curve) and RMSE (root mean square error) as the evaluation indicators of the experimental results. AUC is the area under the indicator curve. The AUC value of 0.5 represents the randomly obtainable score. The higher the AUC score, the more accurate the prediction result. RMSE is used to measure the deviation between the estimated value and the true value. The smaller the value, the closer the mining result is to the true value. The calculation formula is as follows:(17)RMSEyi,y^i=1m∑i=1myi−y^i2,where m represents the size of the test data set, yi is the true value of the student’s answer, and y^i is the predicted value of the student’s answer.


## 6.4. Experimental Results and Analysis

In our experiments, a 5-fold cross-validation is performed, and the average AUC and RMSE of five times experiments are used as the evaluation metrics. Furthermore, 80% samples of each dataset are employed as the training set, and 20% of the data are left for the test to check whether the proposed method is generalized well.


### 6.4.1. Comparison with the State-of-the-Art Methods

For the LDM-ID and LDM-HMI methods, detailed settings are as follows: Sigmoid is used as the activation function of the fully connected layer, and the CNN in the prediction network consists of a convolutional layer, a ReLU activation layer, and a maximum pooling layer, where the convolutional kernel size is 3. The model is parameterized by the Adam optimizer, the learning rate is set to 0.001, and the dropout ratio is set to 0.2. In the learner representation network and learning resource representation network, the learner potential vector and learning resource potential vector dimensions are set to 128 and 64, respectively. In addition, in the baseline models, 3PL-IRT is used for comparison, and the capability latitude value is set to 3 in the MIRT model.As shown in Table2, in terms of the accuracy of learning performance prediction, among the traditional psychometric-based learning diagnostic models, MIRT outperformed IRT, DINA, Ho-DINA, and PMF-LDM overall on the three datasets. The performance of NeuralCDM, DeepMFLD, IRR, and the proposed methods LDM-ID and LDM-HMI generally outperformed the psychometric-based learning diagnostic models, which indicates that the deep network-based learning diagnostic methods outperform the traditional psychometric-based learning diagnostic methods. Usually, deep networks have more complex structures and more parameters, and the information they can characterize will be richer, while psychometric-based methods usually have simpler structures, fewer parameters, and limited characterization capabilities.Table 2
Comparison with the state-of-the-art methods in terms of learning performance prediction.
ModelCL21Math1Synthetic-5AUCRMSEAUCRMSEAUCRMSEIRT [26]73.12%0.436568.31%0.516478.52%0.4824MIRT [10]73.24%0.435269.83%0.494679.80%0.4733DINA [12]69.92%0.485964.46%0.529773.80%0.4968Ho-DINA [15]70.37%0.472366.58%0.521572.45%0.4988PMF-LDM [27]63.74%0.651568.03%0.488462.40%0.6038NeuralCDM [45]76.88%0.427472.90%0.453379.77%0.4756DeepMFLD [46]80.40%0.422677.74%0.435676.68%0.4897IRR-IRT [47]71.23%0.439574.10%0.450572.65%0.4996IRR-MIRT [47]81.29%0.421380.72%0.423181.35%0.4073IRR-DINA [47]75.51%0.427181.40%0.423481.03%0.4087IRR-NeuralCDM [47]75.02%0.427978.56%0.427875.85%0.4917Proposed LDM-ID86.44%0.410581.35%0.417188.02%0.3667Proposed LDM-HMI85.91%0.412281.89%0.416390.25%0.3528The significance of bold values means the best learning performance prediction on each dataset.As shown in Figure3, the LDM-ID and LDM-HMI methods proposed in this paper outperformed all baseline models in terms of the accuracy of learning performance prediction. On the three datasets, their AUC improved by about 11.45% and RMSE decreased by about 0.07 compared to MIRT on average, their AUC improved by about 8.57% and RMSE decreased by about 0.06 compared to NeuralCDM on average. On the one hand, the cognitive parameters in this paper provide more effective input information for the diagnosis and prediction tasks from the source. On the other hand, this paper constructs a more capable network of learners and learning resources and explores the potential characteristics of learners and learning resources more effectively.Figure 3
The proposed methods were compared with 11 state-of-the-art methods. In this paper, 11 state-of-the-art learning diagnosis methods were replicated and experiments were conducted on two publicly available datasets and one self-built dataset, and evaluated using two metrics, AUC and RMSE. A larger AUC indicates higher prediction accuracy, and a smaller RMSE indicates better stability of the methods.
(a)(b)(c)(d)(e)(f)Furthermore, among the two implementations of the LDM framework proposed in this paper, LDM-HMI performs better on the larger-scale datasets Math1 and Synthetic-5, with an AUC about 1.39% higher and an RMSE about 0.007 lower than that of LDM-ID over the two datasets, while on the smaller-scale dataset CL21, LDM-ID performs better, with an AUC about 0.53% higher and an RMSE about 0.002 lower than that of LDM-HMI. This indicates that the LDM-ID method may be more suitable for small data volume scenarios. In other words, under the LDM framework of this paper, the specific implementation can be flexibly adjusted according to the requirements to achieve better adaptation to the actual scenarios.As shown in Table3, the LDM-ID and LDM-HMI proposed in this paper are comparable to the deep learning diagnostic method NeuralCDM, DeepMFLD, and IRR in terms of time loss. The prediction time loss of the proposed method on the medium-sized dataset Math1 is comparable to that of the traditional psychometric methods. On the small dataset CL21 and large dataset Synthetic-5, the time loss of the proposed method is one order of magnitude higher compared to the traditional psychometric methods. Overall, the prediction time of the proposed method is about 0.2 ms on a single sample, which is sufficient for practical educational application scenarios.Table 3
Comparison with traditional methods in terms of prediction time cost.
ModelTime cost for predictionCL21Math1Synthetic-5IRT [26]16 ms203 ms141 msMIRT [10]90 ms385 ms597 msDINA [12]16 ms31 ms47 msHo-DINA [15]57 ms198 ms124 msPMF-LDM [27]15 ms45 ms61 msNeuralCDM [45]156 ms216 ms3359 msDeepMFLD [46]270 ms352 ms3822 msIRR-IRT [47]192 ms401 ms3591 msIRR-MIRT [47]229 ms435 ms3603 msIRR-DINA [47]223 ms462 ms3609 msIRR-NeuralCDM [47]218 ms316 ms3583 msProposed LDM-ID187 ms277 ms3397 msProposed LDM-HMI203 ms303 ms3374 ms


### 6.4.2. Effectiveness of Learner and Learning Resource Representation Networks

To verify the effectiveness of the learner and learning resource characterization network in the interpretable intelligent learning diagnosis framework proposed in this paper, experiments are conducted to compare the auto-encoder-based characterization network in this paper with the fully connected deep neural network in the traditional approach.As shown in Table4, in the LDM-ID and LDM-HMI methods, compared with the fully connected deep neural network, the auto-encoder-based representational network improves the AUC of the model by about 0.85% and 0.9% on average over the three data sets, respectively, and reduces the RMSE by about 0.002 and 0.003, respectively. It proves that the auto-encoder-based learner and learning resource representation network is more effective. Among them, the encoder learns to retain as much relevant information in the potential space and discard irrelevant parts, and the decoder learns the potential space information to reconstruct the input, which achieves an effective denoising effect and contributes to a more effective potential feature representation of learners and learning resources.Table 4
Learning performance prediction experiments for learner and learning resource representation networks.
ModelNetwork structureCL21Math 1Synthetic-5AUCRMSEAUCRMSEAUCRMSELDM-IDDNN84.78%0.417979.26%0.422386.15%0.3691SAE85.32%0.415780.47%0.418486.96%0.3682LDM-HMIDNN81.56%0.421979.74%0.421588.60%0.3579SAE82.65%0.420181.02%0.416888.93%0.3562The significance of bold values means the best learning performance prediction on each dataset.


### 6.4.3. Effectiveness of Deep and Shallow Learning Feature Fusion

To verify the effectiveness of the fusion of deep and shallow features in the interpretable intelligent learning diagnosis framework proposed in this paper, experiments are conducted in three forms: using only shallow features, using only deep features, and fusing deep and shallow features.As shown in Table5, the fusion of deep and shallow features has the most significant effect on the model effectiveness on the average over the three datasets, with the AUC of the fusion of deep and shallow features improving by about 1.46% and 1.73% and the RMSE decreasing by about 0.011 and 0.016 in the LDM-ID and LDM-HMI approaches relative to the use of only shallow features, relative to the use of only deep features, the AUC of deep and shallow feature fusion is increased by about 0.42% and 0.72%, and the RMSE is reduced by about 0.003 and 0.011. Therefore, the fusion of deep and shallow features helps to improve the performance of the model, because the deep and shallow features provide different valid information from the high-dimensional space and low-dimensional space, respectively, to achieve better prediction results. Also, the fusion of deep and shallow features has better interpretation compared to using only deep features.Table 5
Learning performance prediction experiments with a fusion of deep and shallow features.
ModelRepresentationCL21Math 1Synthetic-5AUCRMSEAUCRMSEAUCRMSELDM-IDShallow feature83.16%0.429177.41%0.438885.23%0.3749Deep feature84.31%0.420678.69%0.426585.91%0.3718Shallow + deep feature84.78%0.417979.26%0.422386.15%0.3691LDM-HMIShallow feature79.66%0.443578.33%0.439786.73%0.3664Deep feature81.04%0.435979.21%0.434287.48%0.3638Shallow + deep feature81.56%0.421979.74%0.421588.60%0.3579The significance of bold values means the best learning performance prediction on each dataset.


### 6.4.4. Effectiveness of Attention Mechanism

Based on the deep and shallow feature fusion, considering that each dimension of deep features and shallow features may have different effects on the performance of model prediction, giving different degrees of attention to different dimensional features may produce better results. Therefore, this paper introduces an attention mechanism in the prediction network to further process the deep and shallow fused features, and conducts experimental validation.As shown in Table6, after introducing the attention mechanism, the AUC is improved by about 0.92% and 1.42% and the RMSE is reduced by about 0.004 and 0.005 on average in the three datasets in the LDM-ID and LDM-HMI methods, which fully illustrates that the introduction of the attention mechanism can improve the performance of the model. The attention mechanism assigns different weights to the features of different dimensions, which not only helps to improve the accuracy of the prediction results but also can provide explanatory information for the prediction results. It can help us to know the potential connection between the features and the output.Table 6
Experiments on the prediction of learning performance of attention mechanism.
ModelAttention settingCL21Math 1Synthetic-5AUCRMSEAUCRMSEAUCRMSELDM-IDWithout attention84.78%0.417979.26%0.422386.15%0.3691With attention85.39%0.412379.84%0.419287.71%0.3669LDM-HMIWithout attention81.56%0.421979.74%0.421588.60%0.3579With attention84.70%0.413480.33%0.418589.14%0.3537The significance of bold values means the best learning performance prediction on each dataset.In addition, this paper also verifies the effect of the joint action of the auto-encoder-based learner and learning resource representation network and attention mechanism based on deep and shallow feature fusion, which is conducted in three cases, namely, introducing the auto-encoder-based learner and learning resource representation network alone, introducing the attention mechanism alone, and introducing the auto-encoder-based learner and learning resource representation network and attention mechanism simultaneously. The experiments were conducted.As shown in Table7, the model has the best performance when the auto-encoder-based learner and learning resource representation network and the attention mechanism are introduced simultaneously. The AUC improves by 1.02% and 1.82%, and the RMSE decreases by 0.003 and 0.004 in the LDM-ID and LDM-HMI methods on average of the three datasets compared to the auto-encoder-based learner and learning resource representation network alone. Compared with the attention mechanism alone, the AUC of the model increased by 0.96% and 1.29%, and the RMSE decreased by 0.001 and 0.001. This indicates that the model performs best when the auto-encoder-based learner and learning resource representation network and the attention mechanism are combined, which can effectively represent the learners and learning resources while assigning different weights to the features of different dimensions.Table 7
Learning performance prediction experiments for the representation network and attention mechanism.
ModelAttention settingCL21Math 1Synthetic-5AUCRMSEAUCRMSEAUCRMSELDM-IDSAE85.32%0.415780.47%0.418486.96%0.3682Attention85.39%0.412379.84%0.419287.71%0.3669SAE with attention86.44%0.410581.35%0.417188.02%0.3667LDM-HMISAE82.65%0.420181.02%0.416888.93%0.3562Attention84.70%0.413480.33%0.418589.14%0.3537SAE with attention85.91%0.412281.89%0.416390.25%0.3528The significance of bold values means the best learning performance prediction on each dataset.


### 6.4.5. Interpretable Analysis and Educational Applications

The interpretable intelligent learning diagnostic framework proposed in this paper has good interpretability, which comes from three main aspects: cognitive parameters, learner-resource response network, and weighting information of attention mechanisms.1) Interpretability of cognitive parameters: In the initial part of the model, this paper conducts a preliminary diagnostic evaluation of learners based on different cognitive diagnostic model, and obtains some cognitive parameters that have practical significance in the educational field, such as the multidimensional ability values of learners in the MIRT model, the binary vector of learners’ knowledge mastery in the DINA model, the difficulty, differentiation, the skipping rate and guessing rate of learning resources. These parameters can complement each other and form a comprehensive and multidimensional description of the learners and learning resources, which can be used as input to the subsequent prediction network to provide a better explanation.For example, for learners, the proposed interpretable intelligent learning diagnostic framework can be combined with multiperspective cognitive diagnostic theories to evaluate learners’ learning states, which can not only understand the differences in learners’ multidimensional and fine-grained potential cognition status but also obtain the overall ability level of learners. In addition, it can provide refined tutoring information for subsequent teaching practice. Figure4(a) shows the knowledge mastery of four randomly selected learners in the CL21 dataset and their overall proficiency level. It shows that learner S1 has a low mastery of knowledge points K8 and K10. Figure 4(b) shows the overall proficiency level of the four learners, and we can see that the overall proficiency level of learner S4 is low, indicating that his foundation may be weak and needs to consolidate his basic knowledge. Figure 5(a) visualizes six cognitive parameters of 20 exercises on the CL21 dataset. This information can be effectively applied in various aspects, such as learner profiling and learner proficiency analysis.Figure 4
There are two sets of interpretable cognitive parameters for learners, the knowledge mastery cognitive parameters and the learning ability parameters. For the knowledge mastery cognitive parameter, 0 represents no mastery of the knowledge, and 1 represents full mastery of the knowledge. For the cognitive parameter of learning ability, the range of values is from −1.5 to 1.5, representing low to high learning ability. The higher the learning ability parameters, the higher the probability of answering the exercise correctly. (a) Knowledge Mastery status parameters of learners 1∼4 on knowledge points K1∼K12. (b) Learning ability parameters of learners 1∼4.
(a)(b)Figure 5
As in Figure (a), 6 groups of the relevant cognitive parameters of 20 exercises are intercepted from CL21. The darker the color is, the greater the value is. For Figure (b), the correlation coefficients between learner embedding and learning resource embedding are extracted in the learner-resource response network. For Figure (c), the weight factors of the self-attention mechanism on the learning feature vector are used to speculate the answering results.
(a)(b)(c)Second, for learning resources, this method can provide comprehensive parameter characterization information of learning resources. Based on these parameters, a more accurate learning resource recommendation service can be provided for learners. For example, when learners have a higher mastery of knowledge pointa, we can recommend more difficult learning resources to help learners master knowledge point a more firmly, or when there is a small difference in the diagnostic results of different learners’ mastery of knowledge point b, we can select more differentiated learning resources about b to provide detailed learning diagnosis for these learners. This rich information about learning resources can assist in practical applications such as educational resource modeling and test question quality analysis.2) Interpretability of learner-resource response network: After deep characterization of learners and learning resources, this paper constructs a learner-learning resource response network to model the learner-learning resource response process in the actual scenario, simulating the interrelationship between the potential learning state of learners and the response results in the response process, which is more suitable for the actual situation and provides explanations in the modeling method. The heat map of the Pearson coefficient correlation between the learner characteristics and the learning resource characteristics is constructed, and Figure5(b) shows that there is a more significant connection between the two. The learner-learning resource response network can be applied to learning resource recommendation, learning interaction analysis, and adaptive learning systems.3) Interpretability of attention mechanisms: Based on the model’s deep and shallow integration features of learners and learning resources, an attention mechanism is introduced to obtain the difference in the degree of influence of features of different dimensions on the actual response results of learners, so that the importance level of features provides retrospective explanatory information for predicting results and provides support for educational cue analysis, educational visualization research, etc.Second, from the perspective of learning resources, this method can provide comprehensive parameter characterization information for learning resources. Figure5(c) shows the exercise parameter information of the CL21 data set. Based on these exercise parameters, it is possible to provide learners with more accurate learning resource recommendation services. For example, when the learner has a high degree of mastery of knowledge point a, we can recommend difficult topic resources about knowledge point a to help learners grasp this knowledge point more firmly, or when different learners have a small difference in the diagnosis result of the mastery of knowledge point b, we can make a more detailed learning diagnosis for these learners by selecting exercise resources with a greater degree of discrimination.


## 6.4.1. Comparison with the State-of-the-Art Methods

For the LDM-ID and LDM-HMI methods, detailed settings are as follows: Sigmoid is used as the activation function of the fully connected layer, and the CNN in the prediction network consists of a convolutional layer, a ReLU activation layer, and a maximum pooling layer, where the convolutional kernel size is 3. The model is parameterized by the Adam optimizer, the learning rate is set to 0.001, and the dropout ratio is set to 0.2. In the learner representation network and learning resource representation network, the learner potential vector and learning resource potential vector dimensions are set to 128 and 64, respectively. In addition, in the baseline models, 3PL-IRT is used for comparison, and the capability latitude value is set to 3 in the MIRT model.As shown in Table2, in terms of the accuracy of learning performance prediction, among the traditional psychometric-based learning diagnostic models, MIRT outperformed IRT, DINA, Ho-DINA, and PMF-LDM overall on the three datasets. The performance of NeuralCDM, DeepMFLD, IRR, and the proposed methods LDM-ID and LDM-HMI generally outperformed the psychometric-based learning diagnostic models, which indicates that the deep network-based learning diagnostic methods outperform the traditional psychometric-based learning diagnostic methods. Usually, deep networks have more complex structures and more parameters, and the information they can characterize will be richer, while psychometric-based methods usually have simpler structures, fewer parameters, and limited characterization capabilities.Table 2
Comparison with the state-of-the-art methods in terms of learning performance prediction.
ModelCL21Math1Synthetic-5AUCRMSEAUCRMSEAUCRMSEIRT [26]73.12%0.436568.31%0.516478.52%0.4824MIRT [10]73.24%0.435269.83%0.494679.80%0.4733DINA [12]69.92%0.485964.46%0.529773.80%0.4968Ho-DINA [15]70.37%0.472366.58%0.521572.45%0.4988PMF-LDM [27]63.74%0.651568.03%0.488462.40%0.6038NeuralCDM [45]76.88%0.427472.90%0.453379.77%0.4756DeepMFLD [46]80.40%0.422677.74%0.435676.68%0.4897IRR-IRT [47]71.23%0.439574.10%0.450572.65%0.4996IRR-MIRT [47]81.29%0.421380.72%0.423181.35%0.4073IRR-DINA [47]75.51%0.427181.40%0.423481.03%0.4087IRR-NeuralCDM [47]75.02%0.427978.56%0.427875.85%0.4917Proposed LDM-ID86.44%0.410581.35%0.417188.02%0.3667Proposed LDM-HMI85.91%0.412281.89%0.416390.25%0.3528The significance of bold values means the best learning performance prediction on each dataset.As shown in Figure3, the LDM-ID and LDM-HMI methods proposed in this paper outperformed all baseline models in terms of the accuracy of learning performance prediction. On the three datasets, their AUC improved by about 11.45% and RMSE decreased by about 0.07 compared to MIRT on average, their AUC improved by about 8.57% and RMSE decreased by about 0.06 compared to NeuralCDM on average. On the one hand, the cognitive parameters in this paper provide more effective input information for the diagnosis and prediction tasks from the source. On the other hand, this paper constructs a more capable network of learners and learning resources and explores the potential characteristics of learners and learning resources more effectively.Figure 3
The proposed methods were compared with 11 state-of-the-art methods. In this paper, 11 state-of-the-art learning diagnosis methods were replicated and experiments were conducted on two publicly available datasets and one self-built dataset, and evaluated using two metrics, AUC and RMSE. A larger AUC indicates higher prediction accuracy, and a smaller RMSE indicates better stability of the methods.
(a)(b)(c)(d)(e)(f)Furthermore, among the two implementations of the LDM framework proposed in this paper, LDM-HMI performs better on the larger-scale datasets Math1 and Synthetic-5, with an AUC about 1.39% higher and an RMSE about 0.007 lower than that of LDM-ID over the two datasets, while on the smaller-scale dataset CL21, LDM-ID performs better, with an AUC about 0.53% higher and an RMSE about 0.002 lower than that of LDM-HMI. This indicates that the LDM-ID method may be more suitable for small data volume scenarios. In other words, under the LDM framework of this paper, the specific implementation can be flexibly adjusted according to the requirements to achieve better adaptation to the actual scenarios.As shown in Table3, the LDM-ID and LDM-HMI proposed in this paper are comparable to the deep learning diagnostic method NeuralCDM, DeepMFLD, and IRR in terms of time loss. The prediction time loss of the proposed method on the medium-sized dataset Math1 is comparable to that of the traditional psychometric methods. On the small dataset CL21 and large dataset Synthetic-5, the time loss of the proposed method is one order of magnitude higher compared to the traditional psychometric methods. Overall, the prediction time of the proposed method is about 0.2 ms on a single sample, which is sufficient for practical educational application scenarios.Table 3
Comparison with traditional methods in terms of prediction time cost.
ModelTime cost for predictionCL21Math1Synthetic-5IRT [26]16 ms203 ms141 msMIRT [10]90 ms385 ms597 msDINA [12]16 ms31 ms47 msHo-DINA [15]57 ms198 ms124 msPMF-LDM [27]15 ms45 ms61 msNeuralCDM [45]156 ms216 ms3359 msDeepMFLD [46]270 ms352 ms3822 msIRR-IRT [47]192 ms401 ms3591 msIRR-MIRT [47]229 ms435 ms3603 msIRR-DINA [47]223 ms462 ms3609 msIRR-NeuralCDM [47]218 ms316 ms3583 msProposed LDM-ID187 ms277 ms3397 msProposed LDM-HMI203 ms303 ms3374 ms


## 6.4.2. Effectiveness of Learner and Learning Resource Representation Networks

To verify the effectiveness of the learner and learning resource characterization network in the interpretable intelligent learning diagnosis framework proposed in this paper, experiments are conducted to compare the auto-encoder-based characterization network in this paper with the fully connected deep neural network in the traditional approach.As shown in Table4, in the LDM-ID and LDM-HMI methods, compared with the fully connected deep neural network, the auto-encoder-based representational network improves the AUC of the model by about 0.85% and 0.9% on average over the three data sets, respectively, and reduces the RMSE by about 0.002 and 0.003, respectively. It proves that the auto-encoder-based learner and learning resource representation network is more effective. Among them, the encoder learns to retain as much relevant information in the potential space and discard irrelevant parts, and the decoder learns the potential space information to reconstruct the input, which achieves an effective denoising effect and contributes to a more effective potential feature representation of learners and learning resources.Table 4
Learning performance prediction experiments for learner and learning resource representation networks.
ModelNetwork structureCL21Math 1Synthetic-5AUCRMSEAUCRMSEAUCRMSELDM-IDDNN84.78%0.417979.26%0.422386.15%0.3691SAE85.32%0.415780.47%0.418486.96%0.3682LDM-HMIDNN81.56%0.421979.74%0.421588.60%0.3579SAE82.65%0.420181.02%0.416888.93%0.3562The significance of bold values means the best learning performance prediction on each dataset.


## 6.4.3. Effectiveness of Deep and Shallow Learning Feature Fusion

To verify the effectiveness of the fusion of deep and shallow features in the interpretable intelligent learning diagnosis framework proposed in this paper, experiments are conducted in three forms: using only shallow features, using only deep features, and fusing deep and shallow features.As shown in Table5, the fusion of deep and shallow features has the most significant effect on the model effectiveness on the average over the three datasets, with the AUC of the fusion of deep and shallow features improving by about 1.46% and 1.73% and the RMSE decreasing by about 0.011 and 0.016 in the LDM-ID and LDM-HMI approaches relative to the use of only shallow features, relative to the use of only deep features, the AUC of deep and shallow feature fusion is increased by about 0.42% and 0.72%, and the RMSE is reduced by about 0.003 and 0.011. Therefore, the fusion of deep and shallow features helps to improve the performance of the model, because the deep and shallow features provide different valid information from the high-dimensional space and low-dimensional space, respectively, to achieve better prediction results. Also, the fusion of deep and shallow features has better interpretation compared to using only deep features.Table 5
Learning performance prediction experiments with a fusion of deep and shallow features.
ModelRepresentationCL21Math 1Synthetic-5AUCRMSEAUCRMSEAUCRMSELDM-IDShallow feature83.16%0.429177.41%0.438885.23%0.3749Deep feature84.31%0.420678.69%0.426585.91%0.3718Shallow + deep feature84.78%0.417979.26%0.422386.15%0.3691LDM-HMIShallow feature79.66%0.443578.33%0.439786.73%0.3664Deep feature81.04%0.435979.21%0.434287.48%0.3638Shallow + deep feature81.56%0.421979.74%0.421588.60%0.3579The significance of bold values means the best learning performance prediction on each dataset.


## 6.4.4. Effectiveness of Attention Mechanism

Based on the deep and shallow feature fusion, considering that each dimension of deep features and shallow features may have different effects on the performance of model prediction, giving different degrees of attention to different dimensional features may produce better results. Therefore, this paper introduces an attention mechanism in the prediction network to further process the deep and shallow fused features, and conducts experimental validation.As shown in Table6, after introducing the attention mechanism, the AUC is improved by about 0.92% and 1.42% and the RMSE is reduced by about 0.004 and 0.005 on average in the three datasets in the LDM-ID and LDM-HMI methods, which fully illustrates that the introduction of the attention mechanism can improve the performance of the model. The attention mechanism assigns different weights to the features of different dimensions, which not only helps to improve the accuracy of the prediction results but also can provide explanatory information for the prediction results. It can help us to know the potential connection between the features and the output.Table 6
Experiments on the prediction of learning performance of attention mechanism.
ModelAttention settingCL21Math 1Synthetic-5AUCRMSEAUCRMSEAUCRMSELDM-IDWithout attention84.78%0.417979.26%0.422386.15%0.3691With attention85.39%0.412379.84%0.419287.71%0.3669LDM-HMIWithout attention81.56%0.421979.74%0.421588.60%0.3579With attention84.70%0.413480.33%0.418589.14%0.3537The significance of bold values means the best learning performance prediction on each dataset.In addition, this paper also verifies the effect of the joint action of the auto-encoder-based learner and learning resource representation network and attention mechanism based on deep and shallow feature fusion, which is conducted in three cases, namely, introducing the auto-encoder-based learner and learning resource representation network alone, introducing the attention mechanism alone, and introducing the auto-encoder-based learner and learning resource representation network and attention mechanism simultaneously. The experiments were conducted.As shown in Table7, the model has the best performance when the auto-encoder-based learner and learning resource representation network and the attention mechanism are introduced simultaneously. The AUC improves by 1.02% and 1.82%, and the RMSE decreases by 0.003 and 0.004 in the LDM-ID and LDM-HMI methods on average of the three datasets compared to the auto-encoder-based learner and learning resource representation network alone. Compared with the attention mechanism alone, the AUC of the model increased by 0.96% and 1.29%, and the RMSE decreased by 0.001 and 0.001. This indicates that the model performs best when the auto-encoder-based learner and learning resource representation network and the attention mechanism are combined, which can effectively represent the learners and learning resources while assigning different weights to the features of different dimensions.Table 7
Learning performance prediction experiments for the representation network and attention mechanism.
ModelAttention settingCL21Math 1Synthetic-5AUCRMSEAUCRMSEAUCRMSELDM-IDSAE85.32%0.415780.47%0.418486.96%0.3682Attention85.39%0.412379.84%0.419287.71%0.3669SAE with attention86.44%0.410581.35%0.417188.02%0.3667LDM-HMISAE82.65%0.420181.02%0.416888.93%0.3562Attention84.70%0.413480.33%0.418589.14%0.3537SAE with attention85.91%0.412281.89%0.416390.25%0.3528The significance of bold values means the best learning performance prediction on each dataset.


## 6.4.5. Interpretable Analysis and Educational Applications

The interpretable intelligent learning diagnostic framework proposed in this paper has good interpretability, which comes from three main aspects: cognitive parameters, learner-resource response network, and weighting information of attention mechanisms.1) Interpretability of cognitive parameters: In the initial part of the model, this paper conducts a preliminary diagnostic evaluation of learners based on different cognitive diagnostic model, and obtains some cognitive parameters that have practical significance in the educational field, such as the multidimensional ability values of learners in the MIRT model, the binary vector of learners’ knowledge mastery in the DINA model, the difficulty, differentiation, the skipping rate and guessing rate of learning resources. These parameters can complement each other and form a comprehensive and multidimensional description of the learners and learning resources, which can be used as input to the subsequent prediction network to provide a better explanation.For example, for learners, the proposed interpretable intelligent learning diagnostic framework can be combined with multiperspective cognitive diagnostic theories to evaluate learners’ learning states, which can not only understand the differences in learners’ multidimensional and fine-grained potential cognition status but also obtain the overall ability level of learners. In addition, it can provide refined tutoring information for subsequent teaching practice. Figure4(a) shows the knowledge mastery of four randomly selected learners in the CL21 dataset and their overall proficiency level. It shows that learner S1 has a low mastery of knowledge points K8 and K10. Figure 4(b) shows the overall proficiency level of the four learners, and we can see that the overall proficiency level of learner S4 is low, indicating that his foundation may be weak and needs to consolidate his basic knowledge. Figure 5(a) visualizes six cognitive parameters of 20 exercises on the CL21 dataset. This information can be effectively applied in various aspects, such as learner profiling and learner proficiency analysis.Figure 4
There are two sets of interpretable cognitive parameters for learners, the knowledge mastery cognitive parameters and the learning ability parameters. For the knowledge mastery cognitive parameter, 0 represents no mastery of the knowledge, and 1 represents full mastery of the knowledge. For the cognitive parameter of learning ability, the range of values is from −1.5 to 1.5, representing low to high learning ability. The higher the learning ability parameters, the higher the probability of answering the exercise correctly. (a) Knowledge Mastery status parameters of learners 1∼4 on knowledge points K1∼K12. (b) Learning ability parameters of learners 1∼4.
(a)(b)Figure 5
As in Figure (a), 6 groups of the relevant cognitive parameters of 20 exercises are intercepted from CL21. The darker the color is, the greater the value is. For Figure (b), the correlation coefficients between learner embedding and learning resource embedding are extracted in the learner-resource response network. For Figure (c), the weight factors of the self-attention mechanism on the learning feature vector are used to speculate the answering results.
(a)(b)(c)Second, for learning resources, this method can provide comprehensive parameter characterization information of learning resources. Based on these parameters, a more accurate learning resource recommendation service can be provided for learners. For example, when learners have a higher mastery of knowledge pointa, we can recommend more difficult learning resources to help learners master knowledge point a more firmly, or when there is a small difference in the diagnostic results of different learners’ mastery of knowledge point b, we can select more differentiated learning resources about b to provide detailed learning diagnosis for these learners. This rich information about learning resources can assist in practical applications such as educational resource modeling and test question quality analysis.2) Interpretability of learner-resource response network: After deep characterization of learners and learning resources, this paper constructs a learner-learning resource response network to model the learner-learning resource response process in the actual scenario, simulating the interrelationship between the potential learning state of learners and the response results in the response process, which is more suitable for the actual situation and provides explanations in the modeling method. The heat map of the Pearson coefficient correlation between the learner characteristics and the learning resource characteristics is constructed, and Figure5(b) shows that there is a more significant connection between the two. The learner-learning resource response network can be applied to learning resource recommendation, learning interaction analysis, and adaptive learning systems.3) Interpretability of attention mechanisms: Based on the model’s deep and shallow integration features of learners and learning resources, an attention mechanism is introduced to obtain the difference in the degree of influence of features of different dimensions on the actual response results of learners, so that the importance level of features provides retrospective explanatory information for predicting results and provides support for educational cue analysis, educational visualization research, etc.Second, from the perspective of learning resources, this method can provide comprehensive parameter characterization information for learning resources. Figure5(c) shows the exercise parameter information of the CL21 data set. Based on these exercise parameters, it is possible to provide learners with more accurate learning resource recommendation services. For example, when the learner has a high degree of mastery of knowledge point a, we can recommend difficult topic resources about knowledge point a to help learners grasp this knowledge point more firmly, or when different learners have a small difference in the diagnosis result of the mastery of knowledge point b, we can make a more detailed learning diagnosis for these learners by selecting exercise resources with a greater degree of discrimination.


## 7. Conclusion

To address the challenge of balancing diagnostic accuracy and interpretability in traditional learning diagnostic methods, this paper proposed a unified interpretable intelligent learning diagnosis framework that integrates the interpretability of cognitive diagnosis methods and the powerful representation learning ability of deep learning methods, which uses learners’ learning records and learning resources related information to perform learning diagnosis and learning performance prediction for learners. Using the diagnostic results of multichannel prior cognitive diagnosis as the basis for embedding representation, a learner-learning resource response network is constructed to simulate the process of learners’ answering behaviors. More important influencing factors for prediction results are mined through the self-attention mechanism, making the model more interpretable.Specifically, in the modeling process, we first constructed cognitive parameter sets to form shallow features based on educational theories from multiple cognitive diagnosis models. Then, we designed a learner representation network and learning resource representation network to mine the deep feature of learners and learning resources, to facilitate the use of shallow features in combination with deep features. In the diagnosis process, a fusion self-attention mechanism based on convolutional neural network architecture is used for learners’ performance prediction, which is helpful in finding the features that have a greater impact on the learner’s performance. Experiments on real-world data sets verify the effectiveness of the proposed method, and the analysis of the experimental results can prove to a certain extent the possibility of the method proposed in this paper in real education scenarios.Our method can obtain the predicted value of the correct probability of the learner’s answer, diagnose the learner’s overall knowledge point mastery through the characteristic information of the learner and the learning resource, conduct the parameter representation of the learning resource, and provide reference information for the individualized learning of the learner and the intelligent recommendation of the learning resource of the learning platform. Future research can be carried out from the following aspects: (1) in the intelligent learning diagnosis framework of this paper, more learning diagnosis models could be introduced, and different methods and perspectives could be used to model learners and learning resources; (2) it could monitor the learner’s learning process more macroscopically and evaluate the learner’s learning in periods, and dynamically observe the change process of the learner’s knowledge mastery status and learning ability during learning; and (3) it could introduce multimodal data, such as the text, video, and audio information in the learning resources. The learner’s behavior information could be used to enrich the input data of the model to obtain more accurate learning evaluation results.



---

*Source: 4468025-2023-02-20.xml*