# A Predictive Analytics Infrastructure to Support a Trustworthy Early Warning System
## 4 The Predictive Analytics Infrastructure and the Early Warning System
### The Early Warning System  
#### 4.2.1. Profiled Gradual At-Risk Model  
This section presents the predictive model developed for the EWS to detect at-risk learners on individual courses. The model, denoted as the Profiled Gradual At-Risk model (PGAR), aims to detect the likelihood to fail a course on the basis of the grades of the continuous assessment activities and some profiling information of the learners. This model is an evolution of the Gradual At-Risk model described in [77] by adding the  
Figure 5: Example of creation of a dataset from a query where (**a**) shows the data stored in the MongoDB, (**b**) defines the query to create the dataset for training a predictive model, and (**c**) shows the obtained dataset from data in MongoDB. learner's profile. These last features improve the identification of at-risk situations by knowing the different kinds of enrolled learners. The profiling information in the UOC data mart is currently quite limited, and only academic data can be used. For a learner, the model uses the number of enrolled courses in the current semester to estimate the learner' workload, the number of times they have repeated the course where the prediction is performed to know their aptitude within the course, whether they are new in the university to evaluate self-regulation capacity, and the GPA (global point average) that provides an estimation about their average performance during previous semesters at the institution. A PGAR model is built for each course, and it is composed of a set of predictive models defined as submodels. A course has a submodel for each assessment activity that features the profiling information previously defined and the grades of the activities until the current one. The model's outcome is to fail the course, and it is a binary variable (i.e., fail or pass). **Example 1**. : _Let us describe the PGAR model for a course with four assessment activities (AA). In such a case, the PGAR model contains four submodels:_  
\(Pr_{AA1}\)_(Fail?) = (Profile, Grade\({}_{AA1}\))_  
\(Pr_{AA2}\)_(Fail?) = (Profile, Grade\({}_{AA1}\), Grade\({}_{AA2}\))_  
\(Pr_{AA3}\)_(Fail?) = (Profile, Grade\({}_{AA1}\), Grade\({}_{AA2}\), Grade\({}_{AA3}\))_  
\(Pr_{AA4}\)_(Fail?) = (Profile, Grade\({}_{AA1}\), Grade\({}_{AA2}\), Grade\({}_{AA3}\), Grade\({}_{AA4}\))_  
_where \(Pr_{AAn}\)(Fail?) denotes the name of the submodel to predict whether the learner will fail the course after the activity AAn. Each submodel \(Pr_{AAn}\)(Fail?) uses the learner's profile (Profile) and the grades (\(Grade_{AA1}\), \(Grade_{AA2}\),..., \(Grade_{AAn}\)), that is, the grades from the first activity until the activity AAn._  
Each submodel is evaluated on the basis of different accuracy metrics. We use four metrics [8]:  
\[TNR=\frac{TN}{TN+FP}\ \ \ ACC=\frac{TP+TN}{TP+FP+TN+FN} \tag{1}\]  
\[TPR=\frac{TP}{TP+FN}\ \ \ F_{1.5}=\frac{\left(1+1.5^{2}\right)TP}{(1+1.5^{2})TP +1.5^{2}FN+FP}\]  
where \(TP\) denotes the number of at-risk learners correctly identified, \(TN\) the number of non-at-risk learners correctly identified, \(FP\) the number of at-risk learners not correctly identified, and \(FN\) the number of non-at-risk learners not correctly identified. These four metrics are used for evaluating the global accuracy of the model (\(ACC\)), the accuracy when detecting at-risk learners (true positive rate--\(TPR\)), the accuracy when distinguishing non-at-risk learners (true negative rate--\(TNR\)), and a harmonic mean of the true positive value (precision) and the \(TPR\) (recall) that weights correct at-risk identification (\(F\) score--\(F1.5\)). A non-balanced \(F\) score with a value higher than one gives more emphasis on correctly identify at-risk learners. In our case, this metric is more significant than \(ACC\) due to the class imbalance and the EWS's aim, which focuses on detecting those at-risk learners.