# Systematic Literature Review of Predictive Analysis Tools in Higher Education +
## 3 Results
### Early Warning Systems  
Among the reviewed instances of EWS, there is one in particular that stands out above the rest: Course Signals, documented by [PERSON] and [PERSON] [23]. This is the earliest application that was considered in this study, being documented in a paper in 2012, but the EWS itself has been used in courses at Purdue University since the late 2000s. Course Signals has been highly influential to many other EWS developed after it, becoming one of the most referenced systems by researchers in the community. Course Signals works in conjunction with the LMS used at Purdue University, Blackboard Vista. The EWS works using student-related data regarding demographics, performance in tasks and exams, effort indicators (measured by the interaction with online course materials), and prior academic history. With these data, the system is able to estimate each individual student's risk of failing a course. Risk is represented in a three-level scale, color coded similar to a traffic light, with green meaning low or no risk of failing, yellow representing a mild risk, and red being high likelihood of failing the course. This kind of multi-level representation is easy for instructors and students to understand, and ended up becoming a staple of other EWS to come. Once the level of risk has been assessed, instructors can implement an intervention plan of their choice, including actions such as sending e-mails to the student or scheduling a face-to-face meeting. According to the authors, there was an improvement of around 15% in student retention after Course Signals was introduced at Purdue University, and the tool garnered an overall positive reception by students and instructors. Internally, Course Signals uses what the authors call a \"Student Success Algorithm\" (SSA) in order to process the input data. This algorithm assigns specific weights to each of the input categories and produces a single score, representative of the perceived level of risk [24]. A screenshot of the Course Signals user interface [25] is shown in Figure 2.  
As aforementioned, many EWS developed after Course Signals took heavy inspiration from it. An important example is Student Explorer, presented by [PERSON] et al. [26], which implements a similar three-level scale to assess students' risk of failing a course. Student Explorer mines LMS data regarding performance, in terms of points earned in tasks and exams, and engagement, as in the number of accesses to the course site. These data are weighted to produce three categories to classify students: \"encourage\", \"explore\", and \"engage\", in increasing order of risk. The way Student Explorer classifies students regarding their performance and engagement indicators is represented in the original paper as a table, reproduced in Figure 3.  
Student Explorer is presented as a supporting tool for student advisors, making the task of identifying struggling students much easier. The application was first implemented in STEM courses  
Figure 3: Classification criteria in Student Explorer [26]. Figure 2: Student overview screen in Course Signals [25]. at universities in Maryland and California, contributing to an overall improvement in student performance, as reflected by the general increase of GPA scores. More interestingly, Student Explorer has seen improvements and further development in the years after it was first introduced. The following are add-ons and studies centered around this EWS:  
* [PERSON] and [PERSON] added LMS resource use to the existing input data in Student Explorer [27]. This includes information on access to lecture notes and completion of assignments. This system was tested across 10 consecutive semesters, involving a total of 8762 students in an introductory chemistry course. The authors observed the existence of a significant correlation between resource use and the final grade obtained in the course, using logistic regression as analysis method. The activities that were most influential in the final grade were those related to exam preparation. * [PERSON] et al.