# Predicting Student Performance with Virtual Resources Interaction Data

Alex Martínez-Martínez, Raul Montoliu, Jesús Aguiló Salinas, Inmaculada Remolar

► To cite this version:

Alex Martínez-Martínez, Raul Montoliu, Jesús Aguiló Salinas, Inmaculada Remolar. Predicting Student Performance with Virtual Resources Interaction Data. 19th IFIP International Conference on Artificial Intelligence Applications and Innovations (AIAI), Jun 2023, León, Spain. pp.463-474, 10.1007/978-3-031-34111-3_39. hal-05449780

HAL Id: hal-05449780

https://inria.hal.science/hal-05449780v1

Submitted on 9 Jan 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# Predicting Student Performance with Virtual Resources Interaction Data

Alex Martínez-Martínez✉, Raul Montoliu✉, Jesús Aguiló Salinas, and Inmaculada Remolar✉

Institute of New Imaging Technologies (INIT), Universitat Jaume I, Avda. de Vicent
Sos Baynat, s/n, 12071 Castelló de la Plana, Spain
{alemarti,montoliu,al395312,remolar}@uji.es

**Abstract.** E-learning can be able to act where traditional education cannot, thanks to its ease of interaction with virtual resources. In this work, the possibility of predicting the final outcome of students based solely on their interaction with virtual resources will be tested. The study aims to evaluate the effectiveness of various machine learning and deep learning models in predicting the performance of students based on their interactions with these virtual resources. The OULA dataset will be used to evaluate the proposed models to predict not only whether the student will pass or fail, but also whether the student will receive a distinction or will drop out of the course prematurely. Some of the models trained in this paper, such as Random Forest, have achieved high accuracy levels, up to 96% for binary classification and up to 80% for multiclass classification. These results indicate that it is possible to predict the performance of students based exclusively on their interactions during the duration of the course and to make predictions for each course individually. They also demonstrate the effectiveness of the proposed models and the potential of virtual resources in predicting the performance of students.

**Keywords:** Performance Prediction · E-learning · Machine Learning

## 1 Introduction

Recent years have shown the importance of distance learning (e-learning) as it can be used in a variety of contexts, such as higher education and corporate training [20]. Its use has spread throughout the world thanks to its time and geographical flexibility. It allows both, teachers and students, to be connected whenever they need it and allows them to use more educational resources than in traditional learning. E-learning also provides the opportunity to improve students' engagement and learning experiences by using different digital tools [19].

Its increasing popularity has motivated the research of different learning analytics, whose objectives encompass other goals, such as behavioral or performance predictions [22]. This has given rise to educational data mining, a new field of data analysis [4], supported by the use of various techniques for its study, including deep learning and machine learning [1,15].
Although distance education is starting to become a feasible option for learn-
ing, it still has many challenges to overcome [16]. Currently, there are sufficient
virtual resources that allow the student to be able to complete the teaching
and learning process. However, unlike traditional education, distance education
allows a larger number of students to be brought together, which complicates
the interaction between students and teachers. Despite the advantages that vir-
tual education can offer, the dropout rate is usually higher than face-to-face
education [13].

Having tools to help students in their learning process is important. Each student may have specific needs, so it can be difficult for a teacher to be able to adapt the teaching to each student during classes. This problem is the focus of most research carried out, which tries to solve it by recognizing the characteristics of each student to complement the teacher's work [2].

This work proposes to predict the final result of a student using only the
interactions carried out during the course with the available virtual resources. It
is likely that a greater number of interactions or visiting certain key resources is
important in deciding the student's final outcome. In order to achieve this, the
Open university learning analytics dataset (OULAD) will be used [12], which
contains information on 22 different courses and the interaction of more than
30,000 students. This work focuses on predicting all four possible outcomes of
the student's final results, which are, distinction, pass, fail, or withdraw.

This research aims to measure the effectiveness of using different prediction
models to predict the outcome of students using the interaction data of virtual
resources. It also aims to find models that work well regardless of the course they
evaluate. Each course may have a different set of resources and present different
structures, so it may be important to design a model that is able to work in any
situation. Then, the objectives to be achieved in this study are as follows:

- To determine the effectiveness of machine and deep learning models in predicting students' performance.

- To analyze the generated interaction data of the students with the virtual resources to evaluate their performance.

- To determine the effectiveness of learning models for early prediction of stu-
dent performance.

The proposed contribution of this work is to achieve the listed objectives
but to study and make predictions for each course separately. As far as we
know, this is the first work evaluating all four possible outcomes in the same
learning process for each course individually. Most of the work using the OULAD
for predictions is focused on other problems, such as binary classification or
withdrawn prediction [8].

The rest of the paper has been organized as follows. Section 2 describes exist-
ing performance prediction literature. Section 3 describes the dataset and dis-
cusses the machine and deep learning models used for the prediction. Sections 4
and 5 present the experimental results and discussion. Finally, Sect. 6 provides
the concluding remarks and proposes possible future directions.
## 2 Related Work

Student performance is a broad area of research as it covers various criteria, such as predicting dropouts or withdrawals, identifying at-risk students, and evaluating final course outcomes. The first step in creating prediction models is to conduct a data analysis process. Some works, such as Waheed et al. [23] have explored different analytics applicable to e-learning.

Early predictions are crucial in this domain due to the lower retention rate of learners in virtual environments compared to traditional ones. Xing et al. [24] identified the factors contributing to the existing dropout rates. Hone and El Said [9] studied how course content, teacher-student interaction, and the platform used to deliver the course impact student performance.

Marbouti et al. [17] used several models to predict at-risk students as early as possible during a course, including Logistic Regression, Support Vector Machines, Multi-Layer Perceptron, and others. Hassan et al. [8] used a Long Short Term Memory model, a version of Recurrent Neural Networks, to predict student withdrawals using the clickstream data to make the predictions. Katarya [11] used a derivative of an autoencoder, combined with techniques like feature selection, to predict the final result of a student. Li et al. [14] used a Graph Neural Network to predict the final result of the student by combining interaction data and student attributes.

The research on the OULA dataset is similar to the previously mentioned works, mostly focusing on performance prediction. Hussain et al. [10] used the dataset to predict student engagement. Some works, like Chen et al. [2] used the dataset to create a recommender system. Waheed et al. [21] used a Generative Adversarial Network to simulate student behavior and generate synthetic data.

Most papers tend to group interactions and make predictions for all courses together, which may pose a problem since courses may differ from one another. This paper will make predictions for each course separately and focus on the classification of the four possible outcomes. Additionally, this paper will only use student interactions and not include demographic data.

## 3 Material and Methods

This section will describe the data, its pre-processing, and the different models used for the predictions.

### 3.1 Dataset

The OULA dataset gathers information about the demographic, clickstream behavior, and assessment performance of 32,593 students [12]. There are 22 different courses, from 2013 to 2014, where each course consists of a series of assessments and virtual resources. Each student has a bunch of graded evaluations, from which the final grade for the course can be obtained. The interactions they have had with each virtual resource are also available.
This work aims to predict the final result of each student using only the interactions with the virtual resources. The required information can be found in the *student info* and *student vle* tables. No other student data or type of virtual resource accessed is required for this work. The final result of the students is classified into four labels; distinction (3,024), pass (12,361), fail (7,052), and withdrawn (10,156), the problem to be solved is, therefore, a multiclass classification.

## 3.2 Problem Definition

For this problem, a set of courses, C, is considered, each with a set of enrolled students, $S^i$. Each course is defined as:

$$c_i \in C \quad i \in [1, \dots, n_c] \qquad (1)$$

and each student is defined as:

$$s_j^i \in S^i \quad j^i \in [1, \dots, n_s^i] \qquad (2)$$

Here, $n_c$ is the number of courses and $n_s^i$ is the number of students enrolled in the i-th course. Each course contains a set of activities, $A^i$, with each activity defined as:

$$a_k^i \in A^i \quad k^i \in [1, \dots, n_a^i] \qquad (3)$$

where $n_a^i$ is the number of activities for the i-th course.

Each student has a set of interactions, $x_j^i$, with the activities of each given course. These interactions, also called feature or input vectors, can be defined as:

$$x_j^i = [a_{1(j)}^i, \dots, a_{k^i(j)}^i] \qquad (4)$$

For each feature vector, there is an associated label $y_j^i$, called output, which represents the final result of the student. The task is to find a model f that maps the input vector $x_j^i$ to a predicted output $\hat{y}_j^i$, i.e.,

$$f \rightarrow \hat{y}_j^i = f(x_j^i, \Omega) \qquad (5)$$

where $\Omega$ represents the model parameters chosen by hyperparameter tuning.

The predicted label $\hat{y}_j^i$ must be equal to the original label, $y_j^i$, corresponding to the input vector $x_j^i$. The effectiveness of each model is measured through accuracy, defined as the ratio of the number of correct predictions to the total number of predictions:

$$acc^i = \frac{\sum_{j=1}^{n_s^i} (y_j^i = \hat{y}_j^i)}{n_s^i} \qquad (6)$$

In addition, the area under the ROC curve (AUC) is also considered. It is defined as the ratio between the true positive rate (well-predicted positives) and the false positive rate (wrongly predicted positives) against original positives and negatives, respectively. The objective of the model f is to maximize the accuracy across all courses:

$$\max \left( \frac{\sum_i acc^i}{n_c} \right) \qquad (7)$$
## 3.3 Data Preparation

The data on student interaction is collected by tracking the number of clicks made by each student for each resource. However, since a student can access a resource multiple times, it is important to determine how to collect the information for each resource. Two options are available. The first one is adding up the number of clicks on the same resource, which can represent the time spent on the resource. The second one is counting the number of times a student returns to the same resource after visiting a new one, which may represent the student's behavior. For this study, the second option has been chosen.

<figure><img src="image_3.png" /><figcaption>Fig. 1. Relation for course interaction grouped by the final result of each student.</figcaption></figure>

Figure 1 shows the relationship between the course-interaction pair and the final result obtained by each student. The figure indicates that the number of interactions is dependent on both the course and the final result. Therefore, it is justifiable to use a machine learning model for predictions and to separate the learning processes for each course. Since the data for each course may be different, the results obtained may not always be satisfactory.

To verify this, an analysis was conducted to evaluate the relationship between interactions and the final result obtained as well as the course. The interaction data were grouped by course and the final result, and the normality and variance were checked. The analysis indicated that the data was not normal and had unequal variance. Therefore, the Kruskal-Wallis method was used to determine if there were significant differences between the populations. The analysis confirmed that there were significant differences between the interaction populations with more than 95% confidence, with a p-value lower than 0.05 and very close to 0, justifying the separation of the courses on an individual basis.
## 3.4 Classifiers

This section will describe all the different machine learning models that have been used to make predictions.

**KNN.** This method is a non-parametric, instance-based supervised learning algorithm [7]. It classifies an instance based on the majority class of its K-nearest data points in the training set using a defined distance metric (e.g. Euclidean). It is popular due to its simplicity and effectiveness on large datasets but has a high computational cost and can be sensitive to the choice of K and distance metrics.

**Random Forest.** This method is a popular machine learning algorithm that belongs to the ensemble learning family [3]. It works by training decision trees on bootstrapped samples of the input data and random subsets of features and outputs the mode of the classes from individual trees. This combination is used to reduce overfitting and it makes this model a powerful tool for predictive modeling.

**XGBoost.** This algorithm is an optimized implementation of the gradient boosting algorithm [5]. It builds decision trees to minimize a loss function (e.g. cross-entropy) by adding trees based on the negative gradient of the loss. It includes regularization options, making it computationally efficient and effective for handling large-scale data.

**SVM.** This method finds a hyperplane that separates linearly separable data into classes by maximizing the margin, and distance between hyperplane and support vectors, in classification problems [18]. To solve non-linearly separable problems it uses the kernel trick by projecting data into a higher dimensional space where a linear separation is possible. The algorithm can handle high-dimensional data and is robust to outliers.

**CNN.** This type of neural network is a deep learning algorithm [6]. It uses learnable filters to extract local features from some input data through a process called convolution. The data is then reduced in spatial dimension through pooling in order to build a hierarchy of features. Then normalized through batch normalization for faster and more stable training. Finally, it ends with a fully connected layer that outputs the final prediction.

## 4 Experiments and Results

Two large test batteries were conducted in the study. The first consisted of binary classification of the results, grouping pass and distinction on the one hand, and
fail and withdrawn on the other. The second consisted of performing a multiclass classification for each of the results. For each test battery all the models listed in Sect. 3.4 have been carried out.

## 4.1 Experimental Setup

Once the data and models to be used were established, the next step was to define the structure of the CNN model and the hyperparameters to be obtained for each model. The CNN architecture, shown in Fig. 2, consists of two convolutional layers followed by normalization layers, with a max pooling layer added afterward. The output of these layers is then passed through a series of fully connected layers to produce predictions for both two-class and four-class classification problems. Each model was fitted with its respective tuning process, carried out in combination with a 10-fold cross-validation process. The parameters to be tuned for each model can be observed in Table 1 and the values tested were the most common and reasonable values that are usually used.

<figure><img src="image_4.png" /><figcaption>Fig. 2. CNN built for predictions.</figcaption></figure>

## 4.2 Binary Classification

The binary classification aims to predict if the student will pass the course or not. The final results were converted to ‘Pass’, composed of ‘Distinction’ and ‘Pass’ and ‘Fail’, composed of ‘Withdrawn’ and ‘Fail’. The results obtained with each model can be observed in Fig. 3.

The results indicate that all five models perform similarly, with KNN being the worst performer in most cases and the Random Forest having the highest mean accuracy of 92%. Figure 4 displays the results of the accuracy and
<table><caption>**Table 1.** Parameters tuned for each machine learning model.</caption><thead><tr><td>Model</td><td>Parameters</td></tr></thead><tbody><tr><td>KNN</td><td>Number of neighbors, Distance metric</td></tr><tr><td>Random Forest</td><td>Number of estimators, Max depth, Criterion</td></tr><tr><td>SVM</td><td>Regularization parameter (C), Kernel, Degree of poly kernel</td></tr><tr><td>XGBoost</td><td>Number of estimators, Max depth, Learning rate</td></tr><tr><td>CNN</td><td>Layer Neurons, Normalization, Layer activation</td></tr></tbody></table>

<figure><img src="image_5.png" /><figcaption>**Fig. 3.** Accuracies obtained by all binary models for each course.</figcaption></figure>

AUC for each binary model. The SVM classifier obtains the highest accuracy of 96.19%, proving that the Random Forest is not always the best model for all courses. The Kruskal-Wallis test was performed to confirm the independence of the results obtained by the models. The test results reveal that the models are not independent, with a confidence of 95% and a p-value close to 0, indicating that each model can yield completely different results. Therefore, it is important to evaluate each model's performance for each course independently.

## 4.3 Multiclass Classification

The multiclass classification aims to predict all four possible final results of the students. The classifiers were used to predict 'Distinction', 'Pass', 'Withdrawn', and 'Fail'. The results obtained can be observed in Fig. 5.
<figure><img src="image_6.png" /><figcaption>Fig. 4. Results of accuracy and AUC for each binary model.</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Fig. 5. Accuracies obtained by all multiclass models for each course.</figcaption></figure>

The results indicate that all five models perform similarly, with KNN and CNN being the worst performers in most cases and the Random Forest having the highest mean accuracy of 70.18%. Although the highest accuracy is obtained by the Random Forest, for some of the courses this is not the case and other models achieve better results. Figure 6 displays the results of the accuracy and AUC for each binary model. These results suggest that there is no significant difference between the results for each model, so an ANOVA test was performed to verify this. The result of this test confirms that the results obtained by the models are not independent, with a confidence of 95% and a p-value close to 0, so that each model can assume completely different results.
<figure><img src="image_8.png" /><figcaption>Fig. 6. Results of accuracy and AUC for each multiclass model.</figcaption></figure>

# 5 Discussion

The comparison between multiclass and binary classification reveals some differences in the results. In multiclass prediction, classifiers deal with a smaller amount of data for each possible result, leading to lower performance compared to binary classification. Despite the uneven results, some models show consistency across tasks. For instance, KNN tends to be the worst model in most cases, while Random Forest achieves the best performance. Additionally, SVM and XGboost yield similar results in both cases. One major distinction between binary and multiclass classification is that CNN does not perform as well in multiclass prediction.

Figures 3 and 5 emphasize the importance of evaluating each course independently. It is challenging to find a machine learning model that can generalize well across different courses. None of the models used in either type of classification consistently outperformed the others. The performance of the models in binary and multiclass classification was relatively similar, with no clear winner. However, Random Forest seems to be the most promising model, as it achieved the best results in most cases.

# 6 Conclusions

Predicting a student's performance is essential for both students and teachers. Identifying students who are struggling can be extremely beneficial. This study proposes predicting a student's final result using only their interaction data with virtual resources, using machine learning and deep learning models.

The results of this study reveal that it is possible to predict a student's performance by analyzing their interaction data with virtual resources. It can help identify whether the student will pass or fail, or if they will drop out of the course. Furthermore, this study highlights that each course should be evaluated independently, as different courses may have unique structures and information. These predictions can be achieved using both machine learning and deep learning models, with up to 80% accuracy in some multiclass classification and up
to 96% in binary classification. These results demonstrate the effectiveness of predicting student performance; however, there are still limitations that need to be addressed, such as the size and quality of data, which need to be studied in greater depth to enhance the accuracy of predictions.

In the future, a wider range of models should be evaluated to determine if there is an algorithm that can improve the predictions further for each course. It would also be interesting to examine different databases that include student interactions with virtual resources, to see if other types of interactions or structures can help determine a student's potential final performance. It would also be valuable to make predictions for different stages of the course, limiting the number of interactions to those carried out up to those points, instead of utilizing all the data to make predictions.

**Acknowledgement.** Research supported by the e-DIPLOMA, project number 101061424, funded by the European Union. Views and opinions expressed are, however, those of the authors only and do not necessarily reflect those of the European Union or the European Research Executive Agency (REA). Neither the European Union nor the granting authority can be held responsible for them.

<figure><img src="image_9.png" /></figure>

Funded by
the European Union

Research is also supported by the Valencian Graduate School and Research Network of Artificial Intelligence (valgrAI).

## References

1. Bhardwaj, P., Gupta, P., Panwar, H., Siddiqui, M.K., Morales-Menendez, R., Bhaik, A.: Application of deep learning on student engagement in e-learning environments. Comput. Electr. Eng. **93**, 107277 (2021)

2. Chen, H., Yin, C., Li, R., Rong, W., Xiong, Z., David, B.: Enhanced learning resource recommendation based on online learning style model. Tsinghua Sci. Technol. **25**(3), 348–356 (2019)

3. Dietterich, T.G., et al.: Ensemble learning. Handb. Brain Theory Neural Netw. **2**(1), 110–125 (2002)

4. Dutt, A., Ismail, M.A., Herawan, T.: A systematic review on educational data mining. IEEE Access **5**, 15991–16005 (2017)

5. Friedman, J.H.: Stochastic gradient boosting. Comput. Stat. Data Anal. **38**(4), 367–378 (2002)

6. Gu, J., et al.: Recent advances in convolutional neural networks. Pattern Recogn. **77**, 354–377 (2018)

7. Guo, G., Wang, H., Bell, D., Bi, Y., Greer, K.: KNN model-based approach in classification. In: Meersman, R., Tari, Z., Schmidt, D.C. (eds.) OTM 2003. LNCS, vol. 2888, pp. 986–996. Springer, Heidelberg (2003). https://doi.org/10.1007/978-3-540-39964-3_62

8. Hassan, S.U., Waheed, H., Aljohani, N.R., Ali, M., Ventura, S., Herrera, F.: Virtual learning environment to predict withdrawal by leveraging deep learning. Int. J. Intell. Syst. **34**(8), 1935–1952 (2019)

9. Hone, K.S., El Said, G.R.: Exploring the factors affecting mooc retention: a survey study. Comput. Educ. **98**, 157–168 (2016)
10. Hussain, M., Zhu, W., Zhang, W., Abidi, S.M.R.: Student engagement predictions in an e-learning system and their impact on student course assessment scores. Comput. Intell. Neurosci. **2018** (2018)

11. Katarya, R.: Deep auto encoder based on a transient search capsule network for student performance prediction. Multimedia Tools Appl., 1-25 (2022)

12. Kuzilek, J., Hlosta, M., Zdrahal, Z.: Open university learning analytics dataset. Sci. Data **4**(1), 1–8 (2017)

13. Lee, Y., Choi, J.: A review of online course dropout research: implications for practice and future research. Educ. Technol. Res. Dev. **59**, 593–618 (2011)

14. Li, M., Wang, X., Wang, Y., Chen, Y., Chen, Y.: Study-GNN: a novel pipeline for student performance prediction based on multi-topology graph neural networks. Sustainability **14**(13), 7965 (2022)

15. Lykourentzou, I., Giannoukos, I., Nikolopoulos, V., Mpardis, G., Loumos, V.: Dropout prediction in e-learning courses through the combination of machine learning techniques. Comput. Educ. **53**(3), 950–965 (2009)

16. Maatuk, A.M., Elberkawi, E.K., Aljawarneh, S., Rashaideh, H., Alharbi, H.: The covid-19 pandemic and e-learning: challenges and opportunities from the perspective of students and instructors. J. Comput. High. Educ. **34**(1), 21–38 (2022)

17. Marbouti, F., Diefes-Dux, H.A., Madhavan, K.: Models for early prediction of at-risk students in a course using standards-based grading. Comput. Educ. **103**, 1–15 (2016)

18. Noble, W.S.: What is a support vector machine? Nat. Biotechnol. **24**(12), 1565–1567 (2006)

19. Pan, Z., Cheok, A.D., Yang, H., Zhu, J., Shi, J.: Virtual reality and mixed reality for virtual learning environments. Comput. Graph. **30**(1), 20–28 (2006)

20. Van Raaij, E.M., Schepers, J.J.: The acceptance and use of a virtual learning environment in China. Comput. Educ. **50**(3), 838–852 (2008)

21. Waheed, H., et al.: Balancing sequential data to predict students at-risk using adversarial networks. Comput. Electr. Eng. **93**, 107274 (2021)

22. Waheed, H., Hassan, S.U., Aljohani, N.R., Hardman, J., Alelyani, S., Nawaz, R.: Predicting academic performance of students from vle big data using deep learning models. Comput. Human Behav. **104**, 106189 (2020)

23. Waheed, H., Hassan, S.U., Aljohani, N.R., Wasif, M.: A bibliometric perspective of learning analytics research landscape. Behav. Inf. Technol. **37**(10–11), 941–957 (2018)

24. Xing, W., Chen, X., Stein, J., Marcinkowski, M.: Temporal prediction of dropouts in moocs: reaching the low hanging fruit through stacking generalization. Comput. Human Behav. **58**, 119–129 (2016)