# Systematic Literature Review of Predictive Analysis Tools in Higher Education +
## 3 Results
### Early Warning Systems
The system performed analyses of student activity month by month, identifying students with unexpectedly low activity traces. These students could be flagged for being completely inactive during a specific month, or for low relative activity compared to the classroom average. The EWS was tested in three different undergraduate mathematics courses, with a total sample size of 362 students, achieving an average recall of 66% when identifying dropout students. A Mann-Whitney U test confirmed that studentswho failed a course received more inactivity alerts than those who passed. This was also true for students who dropped out of their degree studies the following year compared to those who did not. [PERSON] et al. built an EWS intended to work together with BookRoll, an e-book management system used in several Asian universities that provides access to course materials [36]. This tool collected student interactions with BookRoll as input, in the form of Experience API (xAPI) statements. Tracked information included logs regarding e-book navigation, page highlighting, and note taking. Using these data, the EWS applied a predictive model in order to identify students at risk of failing the course. This study was clearly geared towards experimentation, as 13 different prediction algorithms were tested. Additionally, three ways of processing the input data were tried: raw, where numeric input data were used as is; transformed, where percentile rank transformation was used to convert raw data to values between 0 and 1; and categorical, where transformed data were discretized into \"Low\", \"Medium\" and \"High\" categories. The system was tested with a cohort of 90 students in a 16-week elementary informatics course. It was observed that the best performing algorithms were Random Forest for raw data, C4.5 decision trees for transformed data, and Naive Bayes for categorical data. Additionally, the accuracy obtained with transformed data was lower than with raw or categorical data. From the third week of the course onward, both RF with raw data and NB with categorical data were able to correctly predict over 80% of at-risk students. Finally, [PERSON] et al. implemented an EWS at Vrije Universiteit Amsterdam in order to support student counselors [37]. This study involved 758 first-year students from 12 different study programs, as well as 34 counselors. The tool estimated a dropout probability for each student based on progress indicators, as reflected by elements such as grades or number of credits obtained. The calculated dropout probability, along with extra information regarding student motivation and performance, was presented to the counselor via an analytics monitor. The outcome of this experiment showed that, while the early identification of at-risk students was useful for counselors, the EWS-assisted counseling sessions did not make an impact on student dropout in any noticeable way. This hints at the existence of an underlying problem that causes underperformance, which cannot be solved only with the identification of at-risk students.