# A Scoring Algorithm for the Early Prediction of Academic Risk in STEM Courses
## 3 Materials and Methods  
This study is based on data collected from the LMS platform Merlin, an implementation of the open-source system Moodle (version 4.2). The dataset comprises user activity logs, quiz scores, overall course performance metrics, and responses to two questionnaires conducted in the first week of the course. The first questionnaire evaluated students' prior mathematical knowledge based on three key indicators: high school mathematics grades and state matriculation examination results. The second questionnaire used the adapted academic motivation scale, translated and adapted into Croatian by [PERSON] [31]. This 24-item instrument measured six motivation factors, which encompassed three types of extrinsic motivation (external regulation, introjected motivation, and identified motivation) and three forms of intrinsic motivation (motivation for knowledge, achievement, and stimulation). The responses collected provided insight into students' academic backgrounds and motivational profiles, which were later analyzed in relation to their course performance. Data collection was carried out in two phases. The first phase was a retrospective analysis based on data from the 2023/2024 academic year, which included 76 students enrolled in the compulsory course Mathematics 1 at the Faculty of Engineering, Universityof Rijeka. Among them, 26 students did not complete the course successfully. The second phase was a prospective analysis conducted on 70 students enrolled in the same course during the academic year 2024/2025. All records were anonymized prior to thhe analysis to ensure compliance with ethical standards and data protection regulations, thereby safeguarding student privacy. The preprocessing phase included handling missing values and transforming independent variables for statistical analysis. In the retrospective dataset, students with missing data for any of the required variables were excluded from the analysis, resulting in a final dataset of 43 students for model evaluation. Independent variables (as described in Section 4) were originally numeric and converted to categorical dichotomous variables. The optimal cut-off point for dichotomization was determined using the Youden index, ensuring a statistically robust separation between categories. The Youden index [32] is a measure of effectiveness that combines sensitivity and specificity and it is calculated as follows:  
\[\text{Youden index}=\text{sensitivity}+\text{specificity}-1\.\]  
Specificity is a measure of the correct identification of negative cases, while sensitivity is a measure of the correct identification of positive cases. The Youden index ranges from 0 to 1. A value close to 1 indicates high model efficiency in distinguishing students who will pass the course from those who will not. The variable _Success_ in the retrospective analysis was defined as a dichotomous variable, where _Success_ = 1 indicated that the student passed the course, and _Success_ = 0 indicated failure. In the prospective analysis, _Success_ was based on passing the first mid-term exam. The prospective dataset from 2024/2025 did not require handling missing values, as all required data were available to all students. The preprocessing was performed using R version 4.3.3. [PERSON]'s chi-square test was used to identify the most relevant dichotomous predictors of _Success_. The [PERSON] chi-square test [33] is a statistical test used to determine whether there is a significant association between two categorical variables. It compares the observed frequencies (\(O\)) with the expected frequencies (\(E\)) to check if differences are due to chance. The value of the Pearson chi-square statistic (\(\chi^{2}\)) is defined as follows:  
\[\chi^{2}=\sum\frac{(E_{ij}-O_{ij})^{2}}{E_{ij}}. \tag{1}\]  
The LMS variables considered relevant on a weekly basis were those for which the corresponding \(p\)-value of Pearson's chi-square test was below the significance level of 0.05.