SlowGuess/NeurIPS_2023_OCR huggingface dataset:
where  $\mathcal{V}_1$  means labeled exercise set. For semi-supervised DRL in the vision field, it's usually enough with a single alignment module for a few labeled samples. However, for cognitive diagnosis, the annotation vector (i.e., the column of Q-matrix) of exercise is very sparse with only one or two knowledge concepts per exercise. The challenge is that utilizing a few labeled exercises to do alignment is too hard to keep such sparsity in relevance representation for unlabeled exercises. To this end, the alignment of unlabeled exercises is necessary.
Numerous Unlabeled Exercises. For an exercise  $v_{j} \in \mathcal{V}_{2}$ , we have no idea any knowledge concept of it. But we know there are only a few knowledge concepts of each exercise. The feature of sparsity
in  $\pmb{\mu}_{v_j}^r$  would help infer missing knowledge concepts. We just need to guarantee that there are only a few elements to be one and other elements to be zero in  $\pmb{\mu}_{v_j}^r$ . In this work, we employ a margin loss and an L2 loss to achieve this target, which can be expressed as follows:
$$
\mathcal {L} _ {u l} = \sum_ {v _ {j} \in \mathcal {V} _ {2}} \left(\lambda_ {1} \max  (0, m - \left(\boldsymbol {\mu} _ {v _ {j}} ^ {r} [ \text {m a x} \# d _ {1} ] - \boldsymbol {\mu} _ {v _ {j}} ^ {r} [ \text {m a x} @ \left(d _ {1} + d _ {2} + 1\right) ]\right)\right) + \lambda_ {2} \| \boldsymbol {\mu} _ {v _ {j}} ^ {r} \| _ {2} ^ {2}\left. \right), \tag {7}
$$
where  $m$  is the margin,  $\lambda_{1}$  is the hyperparameter to control margin loss, and  $\lambda_{2}$  is the hyperparameter to control L2 loss. We use  $\pmb{\mu}_{v}^{r}[max@k]$  and  $\pmb{\mu}_{v}^{r}[max\# k]$  to denote the  $k$ -th largest element of  $\pmb{\mu}_{v}^{r}$  and the top  $k$  largest elements of  $\pmb{\mu}_{v}^{r}$ , respectively. In Eq. (7), we first sort  $\pmb{\mu}_{v_j}^{r}$  by descending order and then split the sorted one into three parts of length  $d_{1}, d_{2}, K - d_{1} - d_{2}$ . We argue that knowledge concepts in the last part are the least likely to be the ground-truth knowledge concepts of exercise  $v_{j}$  and knowledge concepts in the first part are the most likely to be the ground-truth knowledge concepts of exercise  $v_{j}$ . Therefore, we adopt a margin loss between knowledge concepts in the first part and knowledge concepts in the last part, which requires that all values in the first part (i.e.,  $\pmb{\mu}_{v_j}^{r}[max\# d_1]$ ) is at least  $m$  greater than the largest value in the last one (i.e.,  $\pmb{\mu}_{v_j}^{r}[max@(d_1 + d_2 + 1)]$ ). For the middle part, we do not constrain anything on it. Intuitively, it's reasonable that adopt an L1 loss to constraint the value in the last part (i.e.,  $\pmb{\mu}_{v_j}^{r}[min\# (K - d_1 - d_2)]$ ) to be 0. In our initial attempt, we first directly try to use L1 loss for numerous unlabeled exercises, and we find it does not show competing performance (detailed in Appendix A.3). We speculate a possible reason is as follows: with L1 loss, after model initialization, it is very hard to revise these incorrectly labeled exercises. In contrast, L2 loss are sensitive to outliers, and would have a larger loss when the corresponding knowledge is incorrectly predicted.
4.4 Decoder Module
After aligning the exercise relevance representation with the real knowledge concepts, the decoder module predicts scores of students doing exercises, which could be described as a triplet set  $T = \{(u_i, v_j, X_{ij}) | X_{ij} \neq -1\}$ . The object function can be described as follows:
$$
\begin{array}{l} \mathcal {L} _ {m} = - \mathbb {E} _ {q _ {\phi_ {u}} (\mathbf {z} _ {u} | \mathbf {x} _ {u}), q _ {\phi_ {v}} ^ {d} (\mathbf {z} _ {v} ^ {d} | \mathbf {x} _ {v}), q _ {\phi_ {v}} ^ {r} (\mathbf {z} _ {v} ^ {r} | \mathbf {x} _ {v})} [ \log p (\mathbf {X} | \mathbf {z} _ {u}, \mathbf {z} _ {v} ^ {d}, \mathbf {z} _ {v} ^ {r}) ] \\ = \sum_ {\left(u _ {i}, v _ {j}, X _ {i j}\right) \in T} B C E \left(\sigma \left(\left(\mathbf {z} _ {u _ {i}} - \mathbf {z} _ {v _ {j}} ^ {d}\right) \otimes \mathbf {z} _ {v _ {j}} ^ {r}\right), X _ {i j}\right), \tag {8} \\ \end{array}
$$
where  $BCE(\cdot, \cdot)$  is the binary cross entropy loss function,  $\otimes$  and  $\sigma(\cdot)$  denote inner product operation and sigmoid function, respectively.
Similar to existing CD works [38, 39, 21], the interaction function (i.e., the decoder) keeps the monotonicity assumption [39], which guarantees the interpretability of student proficiency representation (i.e., the higher the value of the latent factor means the better proficiency on the corresponding knowledge concept, detailed in Appendix A.4). In addition, the element-wise operation in the decoder module propagates the alignment (i.e., each latent factor corresponds to each real knowledge concept) on exercise relevance representation to both student proficiency representation and exercise difficulty representation, which is also the same as existing works.
**Optimization.** We summarized all object functions in all modules and combine them for final optimization. We set different hyperparameters to balance each loss function. The final object function can be summarized as follow:
$$
\underset {\Theta = \left[ \phi_ {u}, \phi_ {v} ^ {d}, \phi_ {v} ^ {r} \right]} {\arg \min } \mathcal {L} = \mathcal {L} _ {m} + \alpha \mathcal {L} _ {l} + \mathcal {L} _ {u l} + \sum_ {i = 1} ^ {L} \left(\beta^ {i} \sum_ {\mathbf {z} \in \left\{\mathbf {z} _ {u}, \mathbf {z} _ {v} ^ {d}, \mathbf {z} _ {v} ^ {r} \right\}} \mathcal {L} _ {d} ^ {i} (\mathbf {z})\right) + \sum_ {\mathbf {z} \in \left\{\mathbf {z} _ {u}, \mathbf {z} _ {v} ^ {d}, \mathbf {z} _ {v} ^ {r} \right\}} \mathcal {L} _ {p} (\mathbf {z}), \tag {9}
$$
where  $\Theta = [\phi_u,\phi_v^d,\phi_v^r ]$  is the parameter set in the whole model,  $\alpha$  is the hyperparameter for alignment of labeled exercises, and  $\beta^i$  denotes the weight for disentanglement term corresponding to the  $i$ -th level in the knowledge concept tree.
Datasets. Our experiments are conducted on three real-world datasets, i.e., Matmat $^2$ , Junyi [4] and NIPS2020EC [42], all of which contain knowledge concepts of the tree structure. The statistics of these datasets are summarized in Table 1. We adopt five-fold cross-validation to avoid randomness. The details about datasets and implementation $^3$  are depicted in the Appendix B.
Table 1: The statistics of three datasets.
dataset	#students	#exercises	#concepts at different level	#leaf concepts per exercise	#right records : #error records	sparsity
Matmat	7,067	1,847	[1,5,11]	1.00	5.50	97.7%
Junyi	8,852	720	[1,8,39]	1.00	2.16	87.4%
NIPS2020EC	11,857	6,509	[1,4,29,64]	1.05	2.00	99.1%
Metrics. To evaluate the effectiveness of DCD in terms of score prediction and interpretability, we adopt three widely used prediction metrics: AUC [3], ACC, RMSE, and one interpretability metric: Degree of Agreement (DOA) [38]. DOA is the most commonly used interpretability metric in the CD, which measures the degree of agreement between cognitive results and response records.
Baselines. The baselines including the classical data mining model Probabilistic Matrix Factorization (PMF) [33], the latent factor models IRT [10], MIRT [35], and the representative CD models DINA [8], NCDM [38], KaNCD [39] and KSCD [30]. In addition, we also explore a simplified version of our method. Specifically, we remove the exercise difficulty module and margin loss, and focus solely on the fine-grained disentanglement module. This simplified version serves as a baseline, which we refer to as  $\beta$ -TCVAE in the following analysis.
Existing non-interpretable models such as PMF, IRT, and MIRT cannot obtain a student's proficiency for each knowledge concept [38]. Consequently, DOA cannot be calculated for these models. Interpretable models such as DINA, NCDM, KaNCD, and KSCD cannot directly apply to few-labeled scenarios. To ensure fairness in comparison, we propose a pre-filling algorithm (detailed in Appendix B.2) for the missing Q-matrix before applying these models.
5.2 Performance Comparison
To evaluate the effectiveness of our proposed DCD model in few-labeled scenarios, we have conducted experiments on datasets with  $10\%$  Q-matrix (few-labeled exercises),  $20\%$  Q-matrix (few-labeled exercises) and  $100\%$  Q-matrix (fully-labeled exercises). The detailed performance comparison on three datasets is displayed in Table 2.
Prediction Comparison. 1) Our proposed model consistently outperforms all interpretable baseline models across the different scenarios, demonstrating its effectiveness. DINA, which only considers binary space, performs the worst. Both KSCD and KaNCD implicitly model relation among knowledge concepts and has a better performance than NCDM. 2) However, all non-interpretable models achieve the best prediction metrics over interpretable models on Junyi and NIPS2020EC datasets except Matmat dataset. This may be because there are far fewer knowledge concepts in the Matmat dataset than in the other two datasets. To meet the interpretability of larger knowledge concept space, interpretable models lose partial prediction accuracy. This explanation is supported by the fact that the prediction results of interpretable models show a noticeable improvement in the few-labeled scenario compared to the fully labeled scenario, indicating that the trade-off between interpretability and accuracy is more significant in a larger knowledge concept space.
Interpretability Comparison. 1) Table 2 demonstrates that our method achieves the best interpretability in all scenarios. Especially, for the  $10\%$  and  $20\%$  Q-matrix (few-labeled scenarios), our model significantly outperforms interpretable baselines in terms of DOA, demonstrating its superior interpretability in this scenario. 2) In few-labeled scenarios, KaNCD and KSCD show a remarkable improvement in DOA compared to DINA and NCDM. This is because DINA and NCDM can only
$^{2}$ https://github.com/adaptive-learning/matmat-web
$^{3}$ Our code is available at https://github.com/kervias/DCD
Table 2: Comparison in both fully-labeled and few-labeled scenarios. We divide models into non-interpretable and interpretable models. The best scores are in bold for two kinds of models. Noted that non-interpretable models keep same results in any scenario as they do not utilize Q-matrix.
Model	100% Q-matrix (fully-labeled)	20% Q-matrix (few-labeled)	10% Q-matrix (few-labeled)
AUC↑	ACC↑	RMSE↓	DOA↑	AUC↑	ACC↑	RMSE↓	DOA↑	AUC↑	ACC↑	RMSE↓	DOA↑
Matmat
PMF	0.759	0.857	0.334	-	0.759	0.857	0.334	-	0.759	0.857	0.334	-
IRT	0.747	0.851	0.339	-	0.747	0.851	0.339	-	0.747	0.851	0.339	-
MIRT	0.750	0.853	0.337	-	0.750	0.853	0.337	-	0.750	0.853	0.337	-
DINA	0.696	0.816	0.368	0.828	0.698	0.774	0.380	0.766	0.700	0.770	0.372	0.735
NCDM	0.742	0.839	0.348	0.850	0.738	0.837	0.350	0.781	0.740	0.846	0.346	0.738
β-TCVAE	0.753	0.847	0.337	0.847	0.733	0.843	0.342	0.743	0.730	0.842	0.343	0.733
KSCD	0.753	0.824	0.348	0.833	0.745	0.828	0.348	0.789	0.744	0.838	0.345	0.769
KaNCD	0.760	0.857	0.335	0.860	0.752	0.852	0.338	0.800	0.751	0.852	0.338	0.783
DCD	0.763	0.857	0.334	0.861	0.767	0.857	0.333	0.819	0.764	0.855	0.334	0.796
Junyi
PMF	0.817	0.776	0.394	-	0.817	0.776	0.394	-	0.817	0.776	0.394	-
IRT	0.822	0.779	0.391	-	0.822	0.779	0.391	-	0.822	0.779	0.391	-
MIRT	0.820	0.777	0.392	-	0.820	0.777	0.392	-	0.820	0.777	0.392	-
DINA	0.737	0.684	0.462	0.862	0.748	0.658	0.459	0.638	0.754	0.663	0.456	0.599
NCDM	0.760	0.715	0.445	0.851	0.791	0.743	0.418	0.632	0.804	0.753	0.409	0.594
β-TCVAE	0.770	0.732	0.423	0.827	0.803	0.764	0.406	0.699	0.805	0.763	0.407	0.673
KSCD	0.767	0.725	0.426	0.809	0.804	0.754	0.409	0.670	0.809	0.760	0.405	0.650
KaNCD	0.775	0.754	0.422	0.835	0.807	0.767	0.400	0.719	0.815	0.773	0.395	0.691
DCD	0.787	0.754	0.414	0.873	0.811	0.768	0.400	0.733	0.814	0.771	0.397	0.717
NIPS2020EC
PMF	0.814	0.760	0.407	-	0.814	0.760	0.407	-	0.814	0.760	0.407	-
IRT	0.819	0.762	0.402	-	0.819	0.762	0.402	-	0.819	0.762	0.402	-
MIRT	0.822	0.765	0.400	-	0.822	0.765	0.400	-	0.822	0.765	0.400	-
DINA	0.764	0.705	0.451	0.856	0.767	0.687	0.453	0.740	0.766	0.684	0.453	0.693
NCDM	0.795	0.730	0.423	0.853	0.803	0.751	0.411	0.736	0.805	0.750	0.411	0.687
β-TCVAE	0.792	0.744	0.417	0.854	0.797	0.740	0.416	0.781	0.801	0.745	0.413	0.774
KSCD	0.798	0.716	0.421	0.830	0.809	0.754	0.410	0.787	0.809	0.755	0.410	0.777
KaNCD	0.797	0.750	0.423	0.856	0.811	0.759	0.403	0.783	0.812	0.760	0.404	0.761
DCD	0.801	0.752	0.415	0.861	0.812	0.761	0.405	0.793	0.813	0.762	0.404	0.786
diagnose knowledge concepts labeled in the training set. When incorrect knowledge concepts are filled in, KaNCD and KSCD are less affected, while DINA and NCDM cannot accurately diagnose the true knowledge concepts. In contrast, our model avoids this issue by modeling the distribution of students' proficiency overall knowledge concepts.
5.3 Disentanglement Analysis
To investigate the effectiveness of disentanglement, we conducted an experiment where we compared our method at different disentanglement weights  $\beta$  in the few-labeled scenario. Fig. 3 (a-d) illustrates the results, the x-axis shows the value of  $\beta$  from 0 to 2, which corresponds to the strength of disentanglement. The AUC is displayed on the left y-axis and the DOA is displayed on the right y-axis. The AUC(i) and DOA(i) denote the results by disentanglement only according to i-th level of knowledge concept tree. There are some observations from Fig. 3: 1). The disentanglement according to the last level of tree (i.e., the most fine-grained disentanglement) can not have an improvement in terms of DOA as the rise of  $\beta$ . The phenomenon is consistent with the assumption that knowledge concepts are correlated. Despite there being an improvement in AUC, we assume it's the result of interpretability losses, as mentioned in the prediction comparison in section 5.2. 2) The coarse-grained disentanglement could simultaneously improve the AUC and DOA compared to no disentanglement. We observe that the performance increased initially with the implementation of disentanglement and then decreased as the strength of disentanglement became excessive, leading to counterproductivity.
5.4 Alignment Analysis
To evaluate the effectiveness of the limited-labeled alignment module, we conducted an experiment with different hyperparameters on Junyi dataset with  $10\%$  Q-matrix, namely  $\alpha$ ,  $\lambda_1$ ,  $\lambda_2$  and  $d_2$ , as shown in Fig. 3 (e-h). Due to the number of leaf concepts per exercise being close to 1 for all datasets,
Figure 3: AUC and DOA results of DCD with hyper-parameters in the group-based disentanglement (a-d) and limited-labeled alignment (e-h) modules.
we set the  $d_{1} = 1$  as the default value. The default margin is 0.5. There are some observations: 1)  $\alpha$  is the weight of the alignment loss corresponding to labeled exercises. Fig. 3 (e) shows that the performance of our model initially improves and eventually stabilizes as  $\alpha$  increases. 2)  $\lambda_{1}$  is the weight of margin loss for unlabeled exercises. As illustrated in Fig. 3 (f), the employment of margin loss is effective according to results from zero to non-zero of  $\lambda_{1}$ . The excessive  $\lambda_{1}$  would decrease the performance. 3)  $\lambda_{2}$  is the weight of L2 loss of unlabeled exercises exercise relevance representation. We can observe that the employment of the L2 loss plays an essential role from Fig. 3 (g). 4) We can observe that the employment of  $d_{2}$  is effective from Fig. 3 (h). As  $d_{2}$  increases from 0 to 8, the performance increases first and then drops. Overall, the limited-labeled alignment module has a positive effect on performance, and the hyperparameters of the alignment module need to be certainly adjusted.
As shown in Table 3, to prove the effectiveness of our proposed disentanglement and alignment modules, we conduct the ablation study on Junyi dataset under the  $10\%$  Q-matrix scenario. The margin Loss in alignment module ensures few elements to be one in exercise relevance repre
sensation, which achieves an improvement of  $0.53\%$  in terms of DOA. The L2 loss in alignment module ensures the sparsity of exercise relevance representation, which achieves an improvement of  $13.48\%$  in terms of DOA. It proves that utilizing a few labeled exercises to do alignment is too hard to keep such sparsity in relevance representation for unlabeled exercises. The disentanglement module alone achieves  $0.56\%$  in terms of DOA. Overall, both the proposed disentanglement and alignment modules achieve an improvement in terms of prediction and interpretability metrics.
Table 3: Ablation study of DCD on Junyi
Model	10% Q-matrix(few-labeled)
AUC↑	ACC↑	RMSE↓	DOA↑
DCD w/o Alignment-Margin	0.8140	0.7692	0.3976	0.7136
DCD w/o Alignment-L2 loss	0.8101	0.7681	0.3977	0.6322
DCD w/o Disentanglement	0.8130	0.7682	0.3983	0.7134
DCD	0.8143	0.7713	0.3965	0.7174
In this paper, we present a novel approach, called DCD, to tackle the interpretability problem of cognitive diagnosis with limited exercise labels. Inspired by semi-supervised DRL, we learn disentangled representations and align them with real limited labels by group-based disentanglement and limited-labeled alignment. Extensive experiments on widely used benchmarks demonstrate the superiority of our proposed method in few-labeled scenarios.
This work was supported in part by grants from the National Key Research and Development Program of China (Grant No. 2021ZD0111802), the National Natural Science Foundation of China (Grant No. 61972125, 72188101, 61932009, 62376086, and U22A2094).
[1] Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. TPAMI, 35(8):1798-1828, 2013.
[2] Haoyang Bi, Haiping Ma, Zhenya Huang, Yu Yin, Qi Liu, Enhong Chen, Yu Su, and Shijin Wang. Quality meets diversity: A model-agnostic framework for computerized adaptive testing. In ICDM, pages 42–51. IEEE, 2020.
[3] Andrew P Bradley. The use of the area under the roc curve in the evaluation of machine learning algorithms. Pattern recognition, 30(7):1145-1159, 1997.
[4] Haw-Shiuan Chang, Hwai-Jung Hsu, and Kuan-Ta Chen. Modeling exercise relationships in e-learning: A unified approach. In *EDM*, pages 532-535, 2015.
[5] Ricky TQ Chen, Xuechen Li, Roger Grosse, and David Duvenaud. Isolating sources of disentanglement in vaes. In NeurIPS, pages 2615-2625, 2018.
[6] Song Cheng, Qi Liu, Enhong Chen, Zai Huang, Zhenya Huang, Yiying Chen, Haiping Ma, and Guoping Hu. Dirt: Deep learning enhanced item response theory for cognitive diagnosis. In CIKM, pages 2397-2400, 2019.
[7] Elliot Creager, David Madras, Jorn-Henrik Jacobsen, Marissa Weis, Kevin Swersky, Toniann Pitassi, and Richard Zemel. Flexibly fair representation learning by disentanglement. In ICML, pages 1436-1445, 2019.
[8] Jimmy De La Torre. Dina model and parameter estimation: A didactic. Journal of educational and behavioral statistics, 34(1):115-130, 2009.
[9] Janez Demšar. Statistical comparisons of classifiers over multiple data sets. JMLR, 7:1-30, 2006.
[10] Susan E Embretson and Steven P Reise. Item response theory. Psychology Press, 2013.
[11] Weibo Gao, Qi Liu, Zhenya Huang, Yu Yin, Haoyang Bi, Mu-Chun Wang, Jianhui Ma, Shijin Wang, and Yu Su. Rcd: Relation map driven cognitive diagnosis for intelligent education systems. In SIGIR, pages 501-510, 2021.
[12] Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C. Courville, and Yoshua Bengio. Generative adversarial nets. In NeurIPS, pages 2672–2680, 2014.
[13] Irina Higgins, Loic Matthew, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual concepts with a constrained variational framework. In ICLR, 2017.
[14] Zhenya Huang, Qi Liu, Chengxiang Zhai, Yu Yin, Enhong Chen, Weibo Gao, and Guoping Hu. Exploring multi-objective exercise recommendations in online education systems. In CIKM, pages 1261-1270, 2019.
[15] Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv:1611.01144, 2016.
[16] Ilyes Khemakhem, Diederik P. Kingma, and Aapo Hyvarinen. Variational autoencoders and nonlinear ica: A unifying framework. In ICAIS, 2019.
[17] Hyunjik Kim and Andriy Mnih. Disentangling by factorising. In ICML, pages 2649-2658, 2018.
[38] Fei Wang, Qi Liu, Enhong Chen, Zhenya Huang, Yuying Chen, Yu Yin, Zai Huang, and Shijin Wang. Neural cognitive diagnosis for intelligent education systems. In AAAI, pages 6153-6161, 2020.
[39] Fei Wang, Qi Liu, Enhong Chen, Zhenya Huang, Yu Yin, Shijin Wang, and Yu Su. Neuralcd: A general framework for cognitive diagnosis. TKDE, 2022.
[40] Xin Wang, Hong Chen, and Wenwu Zhu. Multimodal disentangled representation for recommendation. In ICME, pages 1-6, 2021.
[41] Xinping Wang, Caidie Huang, Jinfang Cai, and Liangyu Chen. Using knowledge concept aggregation towards accurate cognitive diagnosis. In CIKM, pages 2010-2019, 2021.
[42] Zichao Wang, Angus Lamb, Evgeny Saveliev, Pashmina Cameron, Yordan Zaykov, José Miguel Hernández-Lobato, Richard E. Turner, Richard G. Baraniuk, Craig Barton, Simon Peyton Jones, Simon Woodhead, and Cheng Zhang. Instructions and guide for diagnostic questions: The neurips 2020 education challenge, 2021.
[43] Le Wu, Xiangnan He, Xiang Wang, Kun Zhang, and Meng Wang. A survey on accuracy-oriented neural recommendation: From collaborative filtering to information-rich recommendation. TKDE, 35(5):4425-4445, 2023.
[44] Le Wu, Peijie Sun, Yanjie Fu, Richang Hong, Xiting Wang, and Meng Wang. A neural influence diffusion model for social recommendation. In SIGIR, pages 235-244, 2019.
[45] Hengshi Yu and Joshua D Welch. Michigan: sampling from disentangled representations of single-cell data using generative adversarial networks. Genome biology, 22(1):1-26, 2021.
[46] Yin Zhang, Ziwei Zhu, Yun He, and James Caverlee. Content-collaborative disentanglement representation learning for enhanced recommendation. In RecSys, pages 43-52, 2020.
[47] Yuqiang Zhou, Qi Liu, Jinze Wu, Fei Wang, Zhenya Huang, Wei Tong, Hui Xiong, Enhong Chen, and Jianhui Ma. Modeling context-aware features for cognitive diagnosis in student learning. In SIGKDD, pages 2420-2428, 2021.
A.1 Comparison on Inferring Missing Knowledge Concepts
To evaluate the degree of alignment from exercises' relevance, we adopt Hit Ratio (HR) [44] to measure whether the top-k ranked elements in exercise relevance representation hit a ground-truth knowledge concept. The algorithm about filling missing Q-matrix for baselines (detailed in Appendix B.2) is compared with our exercise relevance encoder. The filling results of HR@2 is illustrated in Table 4. We can observe that: 1) It's rational that inferring missing knowledge concepts according to response records of exercises. The filling algorithm and our method show a obvious improvement than randomly assigning knowledge concepts for unlabeled exercises. 2) Our method demonstrates the best HR@2 on Matmat dataset, but shows a worse performance on other datasets. This is because the number of knowledge concepts in Matmat has a smaller scale than other datasets and our model has a better inferring performance in small scale knowledge concepts. Despite the worse filling results than filling algorithm for baselines, our method still shows the best performance in prediction metrics and interpretable metrics, which demonstrate the robustness of the proposed DCD approach. This may be attributed to the inference of student proficiency of our student encoder module.
Table 4: HR@2 comparison on inferring missing knowledge concepts
Matmat	Junyi	NIPS2020EC
20% Q-matrix	10% Q-matrix	20% Q-matrix	10% Q-matrix	20% Q-matrix	10% Q-matrix
Random	0.1818	0.1818	0.0513	0.0513	0.0313	0.0313
Filling Algorithm	0.6876	0.6006	0.2483	0.2382	0.5864	0.4548
DCD	0.9292	0.8693	0.2892	0.1898	0.4943	0.3849
A.2 Experiments under other few-labeled scenarios