# Computer Adaptive Testing Using Upper-Confidence Bound Algorithm for Formative Assessment
## 3 Results
### Assessment of Real Students  
In Figure 3 are presented the results from the experiment with real students (\(n=104\)). A formative assessment quiz included items from 15 topics, with 4 question items in each topic, for 60 total questions. Results show that in this scenario using traditional testing methods quiz length could be shortened from 60 questions to 55 questions if the goal was 95% accuracy in weakest topic identification. With use of UCB adaptive assessment, however, quiz length could be almost halved to 32 questions. Empirical results (orange and blue plots) are in line with projections drawn from simulations (grey plot). For this particular quiz and group of students UCB would offer a reduction in quiz length by 23 questions if we aim for same (>95%) accuracy. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline \(\mathbf{Q}\) & \(\mathbf{T}\) & \(\mathbf{4}\) & \(\mathbf{8}\) & \(\mathbf{16}\) & \(\mathbf{32}\) & \(\mathbf{64}\) & \(\mathbf{128}\) & \(\mathbf{256}\) & \(\mathbf{512}\) \\ \hline
2 & 33\% & 0\% & 8\% & 0\% & 24\% & \(-2\%\) & 22\% & 24\% \\
4 & 0\% & 29\% & 31\% & 35\% & 36\% & 44\% & 43\% & 52\% \\
8 & & 0\% & 40\% & 45\% & 50\% & 54\% & 58\% & 60\% \\
16 & & & 0\% & 57\% & 50\% & 59\% & 67\% & 71\% \\
32 & & & & 0\% & 65\% & 52\% & 64\% & 73\% \\
64 & & & & & 0\% & 62\% & 56\% & 66\% \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Reduction in quiz length necessary to achieve 95% accuracy conferred by using UCB algorithm with optimal exploration constant over traditional random question approach. All numbers are given for the optimal C value, where T—number of topics; Q—total number of questions. Green color indicates large reduction in quiz length, and red indicates lesser benefit. Figure 3: Average accuracy of formative assessment for 104 students. Simulated Upper-Confidence Bound (UCB) line is for 104 synthetic students to show that synthetic experiments are congruent with reality. The error bars represent SEM. The ease at which the weakest topic of a student can be identified is dependent on how strongly the answers on the same topic are correlated. At answer variance equal to zero, it is sufficient to ask only one question to know the student's knowledge of the rest of the items within the topic. However, when answers are uncorrelated, estimation must be harder, and might defeat entire premise of the proposed UCB testing model. Thus an experiment to measure the effect of knowledge correlation on items within one topic on method effectiveness was carried out. The impact of answer variance on testing accuracy can be seen in Figure 4.  
As anticipated, identifying weak topics is trivial for unrealistically strongly correlated answers. However, even for uncorrelated answers UCB performs twice as good as random questioning. Also plotted in the Figure 4 are variances calculated from answers of real students, \(N=104\) (15 topic 60 question quiz). Correlation matrix for answers between different questions has been constructed and is shown in Figure 5. It indicates relatively low answer correlation even for inter topic questions. Answers on each topic where grouped together for this illustration (i.e., answers 1, 2, 3, 4 all belong to same topic). The matrix in accordance with Figure 4 shows low general correlation of answers within one topic especially for items 20-24 (questions about the physical layer of OSI model). Figure 4: The impact of answer variance within topics on testing accuracy (number of questions needed to achieve 95% accuracy in a 60 question 15 topic quiz). The green markers indicate variance calculated using data from real tests, square and circle position along x axis represents average variance. Figure 5: Correlation matrix (Pearson coefficient) for quiz answers. To assess shorter quiz lengths and algorithm behavior at 95% accuracy value a more detailed look at accuracy distribution is provided in Figure 6. As seen from the figure for 60 question, 15 topic quiz the algorithm rarely displays accuracies between 50% and 99% for individual students.