# The Accurate Measurement of Students' Learning in E-Learning Environments
## 6 Results
### Percentage Scores vs. IRT Scores  
Figure 5 presents the comparison between the percentage scores and IRT scores. The IRT scores for both the pre- and post-tests are located on a common metric because those scores come from FPC. Figure 5a presents the scatter plot between percentage scores (\(y\)-axis) and IRT scores (\(x\)-axis) for both the pre- (colored blue) and post-tests (colored orange). This plot clearly shows that the same percentage score can be further differentiated using the IRT scores. For example, the percentage scores of around 60 corresponded to the IRT scores of around \(-1\) in the post-test (colored orange), whereas the same percentage scores of around 60 corresponded to the IRT scores of around 0 in the pre-test (colored blue). Because the IRT scores from FPC were located on a common metric, the differences in the IRT scores between the pre- and post-tests can be attributed to the students' actual learning without worrying about the confounding between the students' abilities and the items' difficulties.  
Figure 4: Percentages of correct responses to items and persons in the pre- and post-tests.  
Figure 5(b) presents the scatter plot between the two gain scores based on the percentage scores (\(y\)-axis) and IRT scores (\(x\)-axis). There was a strong correlation between the two gain scores (\(r=0.96\)), indicating that the order structure of the gain scores was generally preserved. However, the two gain scores produced quite different classification results in terms of students' learning. In Figure 5(b), the green dots represent a students who had positive learning gains based on the percentage scores, but had negative learning gains based on the IRT scores. Table 1 presents the cross-tabulation between the two gain scores. There were 349 students who had positive learning gains based on the percentage gain scores, whereas 206 students had positive learning gains based on the IRT gain scores. There were 140 students who had positive learning gains based on the percentage scores, but had negative learning gains based on the IRT scores. The classification based on the IRT scores should be more reliable because the IRT scores for the pre- and post-tests were placed on a common metric and, therefore, were free from the confounding between the students' abilities and the items' difficulties.  
Figure 6 illustrates the percentage and IRT scores between the pre- and post-tests using box plots. By visual inspection, the percentage scores seemed to show significant learning gains, whereas the IRT scores seemed not to show significant learning gains. This observation was confirmed by the repeated-measures \(t\)-tests presented in Table 2, which suggested that the learning gains based on the percentage scores might be spurious.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline  & & \multicolumn{3}{c}{**Final-Pre**} \\ \cline{3-5}  & & **Negative** & **Positive** & **Total** \\ \hline \multirow{2}{*}{Final-Pre} & Positive & 140 & 209 & 349 \\ \cline{2-5}  & Negative & 66 & 0 & 66 \\ \hline \multirow{2}{*}{Final-Pre} & Total & 206 & 209 & 415 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Gain scores based on the percentage and IRT scores.  
Figure 5: Comparison between the percentage and IRT scores (**a,b**).