# Gamifying Massive Online Courses: Effects on the Social Networks and Course Completion Rates
## 4 Results
### Predictive Models  
Because for both groups, the first component (F1) includes most variables, we analyzed whether F1 (factor scores) can be used as a predictor of the probability of learning success. Remember that F1 is a weighted sum of variables. As the variable \"Certificate\" can have only two values (\"\(1\)\" for Certificate and \"\(0\)\" for course fail), testing the relationship is difficult, as shown in Figure 6A that plots Certificate vs. F1. Even though the relationship exists with R\({}^{2}\) = 0.33, the binary nature of the Certificate makes the appreciation of the relationships hard to see. For a better characterization of the relationships, we introduced a new measure called the _success probability_, or in this case, the probability of getting the certificate. To estimate this probability, we ordered the dataset by F1 and sampled the certificate values above and below a given value of the variable F1, taking \(n\) samples that created a sliding window of samples. In essence, we considered each result (certificate or not) in the sample as a Bernoulli experiment and used the sample mean of the Certificate variable as an estimator for the probability, which is a well-known-maximum-likelihood estimator [44]. This estimator is similar to the Krichevsky-Trofimov estimator, which is used as a conditional estimator of the outcome of the next Bernoulli experiment [45]. However, since our dataset is not a time series and our objective is to create a global model for the success probability as a function of the variable F1, we used the sample mean of the current window, as shown in [46; 47]. In general, \(n\) data points from the plots are lost (\(n\)/2 points at the beginning of the plot and \(n\)/2 at the end). Initially, we used \(n\) = 11, which is 11 samples from the ordered list, as the window size. The samples are taken, and then the number of successes (or certificates) is counted. Assume \(A(i)\) is a measure which indicates that horizontal point \(x_{i}\) is a success (\(A(i)~{}=~{}1\)) or not (\(A(i)~{}=~{}0\)). When the horizontal coordinate \(x_{i}\) is ordered, the probability of success \(P_{s}(i)\) at the horizontal point \(i\) is estimated using Equation (1). \[P_{s}(i)~{}=~{}\frac{1}{n}\sum_{j~{}=~{}i-\frac{(n-1)}{2}}^{j~{}=~{}i-\frac{( n-1)}{2}}A(j) \tag{1}\]  
Equation (1): Probability of success (getting the certificate) at point \(i\). Figure 6B plots the probability of Certificate vs. F1 with \(n\) = 11. Notice how the R\({}^{2}\) statistic improved significantly (R\({}^{2}\) = 0.77). The plot and the fit provide mathematical models to describe the student's performance. Figure 6B shows that the probability of getting a certificate improves significantly when participation in the course increases, as measured by the variables that are part of F1. It is noteworthy that the plot of this figure has R\({}^{2}\) of only 0.334 (Figure 6A) and that by using the certificate-probability estimate, the R\({}^{2}\) increases to 0.774 with \(n\) = 11 (Figure 6B), which is a significant increase in model prediction capability. The sample size used to estimate the success probability was small, so tests were made with different values of \(n\) (Figure 6B-D). It was found that the behavior of the probability estimate was about the same, showing stability in the estimate. Naturally, the bigger the sample, the more precise the estimate will be. In Figure 7, we show the success probability estimator against F1 for the control group. We can appreciate that F1 can be used as a predictor of learning success for both groups. We also see that the steepness of the slope is higher in the experimental group (0.05 for experimental and 0.03 for control). This means that the probability of getting the certificate increases more rapidly in the experimental groups for students who engage in getting higher scores in the variables that are part of F1. Figure 6: The plot of Certificate probability vs. F1 with the linear fit for the experimental group with different sample sizes (n) of the probability estimate.