# Cognitive Diagnosis Method via Q-Matrix-Embedded Neural Networks
## 3 Experiments and Results
### Simulation Studies
#### 3.2.2 Results
\begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Data Type** & **Data Name** & \(g\)_s_ **levels** & **Q-Matrices** & **Sim Models** \\ \hline \multirow{3}{*}{sim data} & HSD1 & low & \multirow{3}{*}{\(\mathbf{Q}_{1}\in\mathbb{R}^{10\times 3}\)} & DINA \\  & LSD1 & high & & \\ \cline{1-1} \cline{2-5}  & HSD2 & low & & \\ \cline{1-1} \cline{2-5}  & LSD2 & high & & \\ \hline \hline \end{tabular} The value of the guessing and slipping parameters \(g,s\in\mathrm{U}(0,0.15)\) is called “low”, indicating that the simulated data were defined as high-quality data. \(g,s\in\mathrm{U}(0.15,0.30)\) is the opposite. The Q-matrices \(\mathrm{Q}_{1}\) and \(\mathrm{Q}_{2}\) were derived from Equations (17) and (18).  
\end{table} 
Table 1: Details of the simulated dataset. Figure 4: Training Dynamics: Loss, AAR, and PAR values of neural networks on simulated data generated by the GDINA model with Q-matrix \(\mathbf{Q}_{2}\). Subplots (**a–c**) show the results of the model training process on the high-quality dataset (i.e., \(P(0),1-P(1)\in\mathrm{U}(0,0.15)\)), and subfigures (**d–f**) show the results of the model training process on the low-quality dataset (i.e., \(P(0),1-P(1)\in\mathrm{U}(0.15,0.30)\)). tic models such as DINA and GDINA, and nonparametric models like NPC and GNPC. The scores of each method for AAR and PAR are presented in Tables 2 and 3. In the case of the three neural-network-based methods, MLP, ANN, and Dual Q-Net, the hyperparameters were consistent with those outlined in the previous section. The model parameters for all three were initialized by pre-training with a mixture of the four simulated datasets, followed by fine-tuning tailored to datasets of different sample sizes. The performance of these fine-tuned and traditional models was tested on the test dataset, with the average results from dozens of repetitions detailed in Tables 2 and 3. In the following, we detail the performance of each model using the AAR metric across the various simulated datasets. On the whole, firstly, our method, Dual Q-Net, consistently outperformed all other models under the various conditions. Secondly, across all datasets, the neural-network-based methods achieved higher scores for AAR compared with the classical parametric (e.g., DINA and GDINA) and non-parametric (e.g., NPC and GNPC) methods. It is noteworthy that this superiority was particularly pronounced for datasets with lower quality and more attributes. For instance, on all sub-datasets of LSD2, the scores of the classical methods for AAR were consistently below 0.9, with some even falling below 0.8, whereas the neural-network-based methods consistently surpassed the 0.9 threshold. In terms of the impact of the number of attributes on AAR performance, when the number of attributes was small (e.g., the HSD1 and LSD1 datasets), the differences between the classical models were minimal, with the DINA model having a slight edge over the other three classical models. Among the nonparametric methods, NPC slightly outperformed GNPC. In the realm of the neural-network-based methods, Dual Q-Net and the ANN were similar and ahead of the MLP. As the number of attributes increased (e.g., the HSD2 and LSD2 datasets), it became evident that GDINA and GNPC performed relatively well among the classical methods. In contrast, DINA and NPC did not perform as strongly, both scoring below 0.8 for AAR. Among the neural-network-based methods, our method, Dual Q-Net, further extended its lead over ANN and MLP. To investigate the impact of the quality (i.e., the guess and slip rates) of the datasets on the performance of the models for AAR, we compared the AAR scores of the models on high-quality datasets (e.g., HSD1 and HSD2) and low-quality datasets (e.g., LSD1 and LSD2). All methods were affected by the guessing and slip parameters, but the neural-network-based methods were more robust than the classical methods.