# Systematic Literature Review of Predictive Analysis Tools in Higher Education +
## 4 Discussion
Engagement and effort are most commonly measured by tracking student activity in educational online platforms, such as the institutional LMS. However, there are also examples of engagement being assessed via direct surveys to students, such as the study by [PERSON] et al. (2012). Engagement indicators have the advantage of being easy to access and collect in most cases, and they provide high volumes of data as well: for a given course, tens of thousands of activity records can be collected from a LMS. Other kinds of input data are less common, but not necessarily less significant. Some studies incorporate demographic information about the students, such as gender, age, or ethnicity. On the other hand, studies that target several different kinds of courses often include input data regarding course characteristics, such as the type of contents and their value in credits. Some of the studies target very specific contexts, and thus are able to work with unique kinds of input data that are unavailable otherwise. For example, [PERSON] analyzed notes taken by students during lectures ([PERSON] et al., 2016), [PERSON] et al. collected information regarding student positioning in the classroom ([PERSON] et al., 2016), and [PERSON] focused on crime and violence indicators ([PERSON] et al., 2016). Lastly, it is important to know that testing these predictors and EWS in real higher education scenarios is essential in order to assess their utility. It is difficult to compare these studies in terms of how well predictors and EWS were tested, since optimal test scenarios vary depending on the context for which the tool was designed. However, one aspect that could help understand whether results from tests are reliable or not is the number of students who participated in the study. A higher number of students implies more volume--and usually, more variety--of input data, which can increase the credibility of measured statistics such as the accuracy of a predictive model. Figure 6 showcases the number of students who participated in testing procedures for each predictor and EWS. This excludes papers that do not provide specific number of students: the documents presenting Course Signals ([PERSON] et al., 2016) and Student Explorer ([PERSON] et al., 2016) focus on describing the EWS itself rather than specific cases of application, while the study by [PERSON] ([PERSON] et al., 2016) used data at an institutional level, rather than related to specific students. In the high end of the scale, there are tools tested with several thousand students. Two studies stand out above the rest in terms of population size: [PERSON] and [PERSON] had data of 8658 students belonging to 13 different courses ([PERSON] and [PERSON], 2010), while [PERSON] and [PERSON] collected data regarding 8762 students across 10 semesters ([PERSON] et al., 2016). On the other hand, the system presented by [PERSON] was tested on a class of only 38 students ([PERSON] et al., 2016), while [PERSON] et al. included only 90 students in their study ([PERSON] et al., 2016), which can be regarded as insufficient in order to make a solid evaluation of a predictor's performance. \begin{table}
\begin{tabular}{c c} \hline \hline
**Input Data Type** & **Appearances in Studies** \\ \hline Student demographics and background & 6 \\ Student engagement and effort & 15 \\ Student performance and academic history & 19 \\ Course, degree or classroom characteristics & 4 \\ Others & 6 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Number of observed use instances of input data types.