# Cognitive Diagnosis Method via Q-Matrix-Embedded Neural Networks
## 3 Experiments and Results
### Simulation Studies
#### 3.2.2 Results  
This section addresses two crucial pivotal aspects of validating model performance. Firstly, we compare the performance of the Dual Q-Net proposed by us with the cognitive diagnostic methods based on MLP by [11] and ANN by [24] during training on simulated data. The experimental results indicated that Dual Q-Net showed relatively superior performance across the various scenarios. Subsequently, we conducted a comprehensive evaluation of our proposed method by comparing the performance of neural-network-based cognitive diagnostic models with classical cognitive diagnostic methods on simulated datasets of different scales. Result I: training process results of different neural network models  
Before starting the analysis of the experimental results, there is some common information that needs to be illustrated. MLP contains one input layer with dimensions \(N\times J\), and one output layer with the dimensions \(J\times K\), indicating that the total number of parameters in the MLP network is \(K(J+1)\). Among the models, ANN consists of an input layer with dimensions \(N\times J\), a hidden layer with dimensions \(K\times K\), and an output layer with dimensions \(N\times K\), resulting in a total parameter count of \(K(J+K+2)\) for the network. The meanings denoted by the symbols \(N\), \(J\), and \(K\) here are the same as those denoted in the foregoing. The structure of Dual Q-Net is illustrated in Figure 2.  
In Figures 3 and 4, the training processes of the three methods, MLP, ANN, and Dual Q-Net, are depicted, where the green curve represents Dual Q-Net, the yellow line represents ANN, and the blue curve represents MLP. Each figure consists of six subplots arranged in two rows and three columns. Among them, the first, second, and third columns represent the convergence processes of the loss, AAR, and PAR during model training, respectively. The first row and second row depict the convergence process of model training on simulated datasets generated by the same Q-matrix and parametric cognitive diagnostic model with low (i.e., \(P(0),1-P(1)\in\mathrm{U}(0,0.15)\)) and high (i.e., \(P(0),1-P(1)\in\mathrm{U}(0.15,0.30)\)) levels of guessing and slip. Figure 3 shows the trend in loss, AAR, and PAR values with the number of training iterations for the three models, MLP, ANN, and Dual Q-Net, on the simulated data, which had a sample size of 500 and were generated by the DINA model with the Q-matrix \(\mathbf{Q}_{1}\). Notably, in the Dual Q-Net network, the skill interaction matrix \(\mathbf{Q}^{\prime}\) was designed as a \(4\times 3\) randomized binary matrix with non-repeating rows. This design signified the presence of four mutual influence interactions among the three skills examined in the Q-matrix \(\mathbf{Q}_{1}\). In particular, the sample size of the test set was 20% of the training set. The hyperparameter  
learning rates, batch size, and epochs for the trained models were 0.01, 64, and 100 for the high-quality dataset. However, the hyperparameters were adjusted to 0.005, 128, and 200 for the low-quality dataset. In Figure 3, we analyze how well the models performed overall across the test datasets. The overall performance of Dual Q-Net surpassed that of MLP and ANN in all cases. Firstly, Dual Q-Net achieved a lower loss than MLP and ANN on both high- and low-quality datasets, as shown in the sub-plot in Figure 3. Moreover, Dual Q-Net's loss on the high-quality datasets converged faster and had smaller values than that of MLP and ANN. Furthermore, the same trend was observed with the low-quality datasets, where the loss performance of Dual Q-Net was more prominent than on the high-quality datasets. Secondly, in terms of model accuracy, Dual Q-Net was ahead of MLP and ANN for both AAR and PAR, as shown in sub-plots b, e, c, and f in Figure 3. In addition, it can also be observed that this advantage was particularly prominent on the low-quality datasets.