# The Accurate Measurement of Students' Learning in E-Learning Environments
## 7 Discussion  
The measurement of learning is an essential task in E-learning. For the valid measurement of learning in E-learning, scores must be placed on a common metric or equated. Despite the crucial role of a common metric, however, less attention has been paid to establishing a common metric in the measurement of learning in E-learning. This study aimed to fill this gap in the E-learning literature. Among various equating methods that have been used in the field of educational measurement, we proposed to use FPC, which establishes a common metric in a way that fits well in E-learning settings. A common metric was expected to allow us (a) to define students' learning properly, (b) to evaluate students fairly, (c) to reuse E-learning items efficiently, and (d) to prevent cheating in E-learning. We used the data from the MOOC \"Introduction to Psychology as a Science\" offered through Coursera collaboratively by Georgia Institute of Technology (GIT) and Carnegie Mellon University (CMU) in 2013 to answer our specific research questions: Does a common metric make meaningful changes in students' abilities and gain scores? Are the IRT scores more robust to missing data then percentage scores? The results from our analysis showed that a common metric made substantial changes in the students' abilities and gain scores. These results suggested that IRT scores are preferable in E-learning. As presented in Figure 4(a), the same percentage score in the pre- and post-tests was mapped into different IRT scores in the pre- and post-tests, which implies that the same percentage score (say 60% of correct responses) in the pre- and post-tests may represent different levels of students' abilities (RQ1). This makes sense because the pre- and post-tests included different sets of items, and therefore, the same percentage score in the pre- and post-tests may represent different levels of the students' abilities. The different IRT scores corresponding to the same percentage scores were the result of removing confounding between the students' abilities and the items' difficulties in the IRT models. In our analysis, the percentage scores tended to overestimate the students' learning gains \begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & **M(SD) for Pre** & **M(SD) for Post** & **Mean Difference** & **t** & _p-Value_ \\ \hline Percentage scores & 55.204 (17.819) & 77.562 (16.620) & 22.358 & 20.325 & \textless{}0.001 \\ IRT scores & 0.079 (0.803) & 0.093 (0.942) & 0.013 & 0.246 & 0.805 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Repeated-measures _t_-tests. Figure 6: Learning gains between the pre- and post-tests. (RQ2). According to Table 1, 140 students (about 40%) who were classified as having positive learning gains based on percentage scores were reclassified as students having negative learning gains based on the IRT scores. The repeated-measures _t_-tests in Table 2 also demonstrated that the overall learning gains shown in the percentage scores (mean difference = 22.358, _p_-value = 0.000) disappeared when using the IRT scores (mean difference = 0.013, _p_-value = 0.805). One of the limitations of our study was the splitting of our MOOC data to mimic the multiple administrations of a MOOC. FPC requires previously calibrated item parameters to fix some of the item parameters during the calibration. In real settings, those previously calibrated item parameters can be obtained from the previous administrations of a MOOC. However, in our study, our access to the data was restricted to the 2013 MOOC data. Therefore, we randomly split our data into two subsets representing data from the past and current year. A study using real longitudinal data would be an interesting future work. Assessment is the key to identify factors that can promote students' learning in E-learning. For valid assessment, scores must be placed on a common metric or equated. This study contributes to the E-learning literature by highlighting the importance of a common metric and also by introducing FPC as a simple equating procedure.