# An Automated Tool to Support an Intelligence Learner Management System Using Learning Analytics and Machine Learning

Shareeful Islam, Haralambos Mouratidis, Hasan Mahmud

► To cite this version:

Shareeful Islam, Haralambos Mouratidis, Hasan Mahmud. An Automated Tool to Support an Intelligence Learner Management System Using Learning Analytics and Machine Learning. 17th IFIP International Conference on Artificial Intelligence Applications and Innovations (AIAI), Jun 2021, Hersonissos, Crete, Greece. pp.494-504, 10.1007/978-3-030-79150-6_39. hal-03287708

HAL Id: hal-03287708

https://inria.hal.science/hal-03287708

Submitted on 15 Jul 2021

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# An automated tool to support an intelligence learner management system using Learning Analytics and Machine Learning

Shareeful Islam¹, Haralambos Mouratidis², Hasan Mahmud ¹,³

¹School of ACE, University of East London, London, E16 2RD, UK

²Centre for Secure, Intelligent and Usable Systems, University of Brighton, UK

³Department of Computer and System Science, University of Stockholm, Sweden

shareeful@uel.ac.uk, hasan.tvu@gmail.com, h.mouratidis@brighton.ac.uk

**Abstract.** Learner Management Systems (LMSs) are widely deployed across the industry as they provide a cost-saving approach that can support flexible learning opportunities. Despite their benefits, LMSs fail to cater for individual learning behavior and needs and support individualised prediction and progression. Learning Analytics (LAs) support these gaps by correlating existing learner data to provide meaningful predictive and prescriptive analysis. The industry and research community have already recognised the necessity of LAs to support modern learning needs. But a little effort has been directed towards the integration of LA into LMSs. This paper presents a novel automated Intelligence Learner Management System (iLMS) that integrates learner management and learning analytics into a single platform. The presented iLMS considers Machine Learning techniques to support learning analytics including descriptive, predictive and perspective analytics.

**Keywords:** Machine Learning, Learning Analytics, Predictive Analytics, Perspective Analytics.

## 1 Introduction

The demand for using technology to effectively support teaching and learning is constantly increasing. Technology enhanced Learning emerges quickly and many institutions rapidly adopted the eLearning where web-based Learner Management Systems (LMSs) have been embedded [1]. LMSs are rapidly deployed across the industry with any provider size and the market is expected to reach a value of $25.7 billion by 2025[2]. Despite of the wider adoption, the traditional LMS cannot support the modern learning trends in terms of understanding individual learning behavior and needs, engagement, and prediction of assessment outcomes. Learning Analytics (LAs) can effectively support for a better learning experience by analyzing and correlating learner data to predicate future needs [11,12]. LA is an emerging field of research based on
ideas from other domains such as technology-enhanced learning, data visualisation and integration [3]. There are works that consider various techniques to support LAs, mainly descriptive and limited predicated analytics [3, 4, 6.7, 9,11]. However, there is a lack of focus on how LA can be integrated into the LMS based on the existing learner data generated from the LMS.

This paper presents a novel automated Intelligence Learner Management System(iLMS) that integrates learner enrolment, learner content management and learning analytics into a single platform. Initial work for ILMS started through a Knowledge Transfer Partnership (KTP) project between the University of East London and Medi-prospects[11]. This paper extends our previous work by providing an automation of iLMS and its unique features[11,12]. iLMS uses various indicators such as learner highest education, assessment data and engagement logs and considers Machine Learning(ML) algorithms, i.e., Logistic Regression(LR), KNN, and decision tree for the Learning Analytics(LA). The LA part of iLMS initially focuses on the descriptive and predictive analysis based on the learner data generated by LMS and finally provides various actions decisions as a part of perspective analytics to support overall teaching and learning support.

## 2 Related Work

Technology-enhanced learning plays an important role for quality teaching and learning. The LMS and eLearning platform market is among the busiest in the technology industry nowadays [1,2]. Several contributions focus on developing various techniques for learning analytics and underlying issues. A literature review performed regarding relevant data sources for LA by [6] emphasizes that the choice of data sources depends on the purpose of the learning analytics and data integration is one of the main challenges for the LA. Students' data are stored in different platforms in different formats as they leave tracks while using different systems [7]. Educational Data Mining (EDM) is considered to discover knowledge from data originating from educational environments [5,6]. Educational Data Mining uses many techniques such as Decision Trees, Neural Networks, Naïve Bayes, K-Nearest neighbor, and many others. EDM is used to analyse data produced during the learning process to predict students' behavior to take an actionable decision. For improving the students' learning progress using huge education data, modern intelligent learning analytics depends on appropriate machine learning models [8]. However, the integration of a single ML model to different sources is challenging and depends on excessive computing power. ML models are strongly data-driven and require systems that have the capacity of collecting data and proposing actionable decision for the stakeholders [8]. A study by Wong reviews how LA has been used by 43 higher education institutions and results show the benefits of using LA in terms of Improving student retention, supporting informed decision making, cost effectiveness and learning behavior [13]. The same study summarizes various predicate models relating to students' academic performance, engagement and early alert. Avella et al review the literature for methods, benefits, and challenges of LA in Higher Education sector [14]. The LA process focuses on the tracking the analytical information from the students to the stakeholders, specifically data related to learners'
interactions with course content, other students, and instructors. Despite of benefits,
several challenges relating to tracking, collection, evaluation and analysis of data and
learning environment optimization, and issues concerning ethics and privacy are men-
tioned. Recently a study by Chen et al assesses the impact of AI on the administration,
instruction, and learning areas of the education sector [15]. The study highlights that
AI learning system playing an important role for improving independent learning ca-
pabilities based on intelligence education technologies using machine learning, learn-
ing analytics and data mining. Busch et al develop a predictive model to identify stu-
dents at risk using apache web server log data of student engagement from a large co-
hort [16]. The work identifies influential predicate attribute such as grade prediction
based on various machine learning algorithms such as decision tree, random forest and
support vector machines. The result shows that support vector machines provides the
highest accuracy among the three chosen algorithm. Akçapınar et al also use learning
analytics to predicate student at risk using ML algorithm using 76 student records [17].
The result shows that the KNN algorithm provides highest accuracy for measuring end-
of-term academic performance. The work investigates whether end-of-term perfor-
mance of students could be predicted in earlier weeks using the selected algorithm,
features, and data transformation techniques.

To summarise, the current literature propose a number of techniques to support Learn-
ing Analytics and reviews the suitability of AI based approached for technology en-
hanced learning. However, little effort has taken place related to how LA can be inte-
grated into LMS. Additionally, there is lack of guidelines and approaches for consider-
ing descriptive, predicative and prescriptive analytics for an effective LA. Our work
contributes to address these limitations and presents an automated intelligence Learner
Management System.

3 Machine Learning for Learning Analytics

Learning Analytics(LA) is defined as the measurement, collection, analysis, and report-
ing of data about learners for purposes of understanding and optimizing learning and
the environments in which it occurs [9]. The purpose of LA is to collect data from
various sources and analyse it to support different institutional needs related to poten-
tial learner engagement and progression and other issues. Machine Learning (ML) can
effective support to analyse the learners data in order to discover pattern and correlating
among the data for the prediction and future actions. LA requires extracting useful
information from the large sets of educational data sets and correlate the data for various
purposes. A common characteristic of all ML algorithms is that they run on and learn
from data, to find rules, hidden patterns and to predict future behaviors [10]. ML mod-
els are trained with learners’ data for predictive analysis which institution can use for
making actionable decisions.

LA requires analysing existing learners to understand patterns and predication for
the overall teaching and learning support. Traditional LMS is not capable of answering
what will happen from the existing learner data despite managing a vast amount of
learner data relating to learner enrolment, progression, engagement and other relevant
areas. Additionally, it is difficult to provide a personal learning experience through
traditional LMS. The goal of individualizing learning to each student's needs is not wholly new [11], yet education is still a long way from achieving this goal. ML models are trained with learners' data for predictive analysis which institution can use for making actionable decisions. But with the help of ML, LMS will be able to drip feed content to the students according to their understanding and performance. The iLMS uses three ML algorithms for the LA purpose. An overview of the algorithms is given below:

* LR: Logistic Regression is a statistical analysis technique which is used for predictive analysis. It uses binary classification to reach specific outcomes and models the probabilities of default classes. We have used LR for predicting student's course outcome either as pass or fail and given us important insight into learner's success rate

* KNN: KNN algorithm is used in industrial applications in tasks such as when a user wants to look for similar items in comparison to others. It is a supervised machine learning algorithm can be used for both classification and regression. We have compared our results obtained from the logistic regression for performance tuning.

* Decision tree: The decision tree algorithm creates a tree-like structure for the data feed and process outcome at the leaf. For our case, we used the categorical variable decision tree to predict students result either as pass or fail.

# 4 An overview of Intelligence Learner management System

This section provides an overview of iLMS and its key features.

## 4.1 Conceptual View of iLMS

Figure 1 shows the conceptual view of the iLMS which consists of two main components, i.e., learner data management and analytics. The data management part mainly feeds the data from both internal and external sources. The internal sources include enrolment system, learner management including assessment and engagement and external includes mainly data from awarding body and funding agency. Data can be in various formats; therefore, data integration plays an important role in the learning analytics. The LA consider three types of analytics in our case, i.e., descriptive, predictive, and perspective analytics using multiple set of indicators such as engagement, age, gender, and location. The upper part of Figure 1 shows the traditional LMS system, which is also used by the project industry partner, i.e., Mediprospects. It includes learner enrolment, management, and reporting. The data integration from various sources certainly is a challenging task for the existing system. The bottom part of the Figure 1 shows the proposed iLMS. It integrates the data from various sources including the learner enrolment data and external sources into learner record system and performs Exploratory Data Analysis (EDA) to clean the data for the analytics. Finally, the Machine Learning algorithms are used to analyse and correlated the data to discover various patterns depending on the stakeholder needs. The outcome from the predictive analytics supports undertaking various actionable decision as a part of perspective
analytics to improve overall teaching and learning support. Therefore, iLMS uses various indicators explain in the later sub sections for both predicative and perspective analytics.

<figure><img src="image_2.png" /><figcaption>Fig 1: Conceptual View of iLMS</figcaption></figure>

## 4.2 Integration of Learning Analytics into iLMS

Learning Analytics (LA) provide measurement, analysis and reporting of the gathered data and their context to understand the patterns and suitable measurement. LA is one of the key features of iLMS that supports descriptive, predictive and perspective analytics.

* Descriptive analytics Descriptive analytics provides an overall view of existing learner status. Past data is analysed to provide insight into what has happened. Existing works that consider LA mainly focuses on descriptive analytics[1]. Tratitional business intelligence system uses previous data to visualize them for better understanding of previous results. This stage is also the preparation stage for the predictive analysis. Existing data is explored using Exploratory Data Analysis (EDA). EDA aims to understand the data sets by summarizing their main characteristics often plotting them visually. Through the EDA process, we can understand the data and its domain or definition on our very important data set. During EDA we try to discover a pattern and try to detect anomalies in the data set. EDA will help to choose our feature(s) by knowing the data set, this will improve the accuracy of the predictive models.

* Predictive analytics: Predictive analysis is mainly based on the descriptive analysis report. It forecasts various institutional needs by following three ML algorithms i.e., LR, KNN and Decision Tree. The ML algorithms are applied at this stage to predicate any outcome relating to teaching and learning outcomes. ML algorithms are mainly used for the predication. For instance, the models can predict learner
performance for a specific course and identified learners at risks. We have consid-
ered the outcome and the accuracies of the different algorithms. iLMS considers
various indicators for the predictive analytics such as :

o Engagement log: Students' engagement with system leave behind huge activity logs. This log is helpful to create a relationship between engagement and outcome. As stated previously we have considered various general indicators for the prediction of engagement. The log data containing student's login information, online duration, resource access logs, assignment upload logs, quiz and other online assessment tools using logs. These logs correlate with student's performance data.

o Assessment data: Students' on course assessment is very important indi-
cators of their outcome. Based on their results throughout the academic,
trainers will have an idea on their risk of failing or passing. Apart from
these results from the previous academic year is also an important factor,
at the same time results of different groups can predict a trend based on
learner's demographic and historical data.

o Drop out: Students' drop out is another important indicator to take an ac-
tionable decision. Drop out data can be tagged with students on course
behavior to identify students at risk. On the other hand, predicting dropout
rate for a future cohort based on the current trend can also be achieved.

o Career progression: Predictive analysis can also help to gauge students'
career progression. Based on the historical record predictive model can
suggest possible future progression.

o Demographic Distribution: The system will predict student success rate based on the demographic background of the existing learner.

o Highest Education: We have considered student's previous academic achievement, during the enrolment students provide this information.

*   **Prescriptive analytics:** Prescriptive analytics suggests a possible measure to take to intervene in the outcome. iLMS considers several actionable decisions for prescriptive analytics such as awareness, personalized support, new course offering, target marketing, performance evaluation. Awareness helps both the learners and instructors to identify the current status of the learners and identifies any unnoticed issue. Such awareness for learners supports to reflect changing their learning behavior. The instructors can also aware of the learners' interest, activity, and additional learner needs. Awareness can also support the personalized learning facility for the learner. It is one of the key requirements from learning analytics by providing support achieving personal learning goals, additional flexible online learning materials, virtual one to one learning session, summative feedback and encouragement. Instructors may also require additional support specifically when a large number of the learner is enrolled for a course or adjustment of learning and assessment content. Prescriptive analytics also support taking business decision in terms of future course offering and target marketing for the potential learners.
## 4.3 System Architecture

The iLMS is fully deployed and managed through the Platform as a Service (PaaS) cloud-based infrastructure and accessible through https://ilms.mediprospects.ac.uk. Note that only authorised users are allowed to access the platform. It combines learner enrolment, management and analytics through a single platform. The overall iLMS architecture is presented in Figure 2. It consists of five different levels of abstraction, i.e., user interface, security, application, learning record store, and analytics engine. The first layer of abstraction is the user interface to allow legitimate users to access the system using pre-created authentication details. There are different categories of users such as learner, admin, instructor and management with different access rights. The second layer authenticates the users based on their roles and directs them to specific dashboards. iLMS uses security features for identity and access management and data encryption in transit and rest. The enrolment layer is responsible for managing learner enrolment and holds detailed information related to learners. Learners are assigned to different groups based on their enrolment data and cohort and they are managed through this group structure. In the learning management system, each group has a group leader as an instructor responsible for delivering the course. The LMS has different types of functionality like content management, communication, course delivery, notification, assignment, feedback, forum and resource sharing. Learner activity logs are recorded in the Learning Record Store (LRS). The Learning Analytic Engine (LAE) processes all the logs and performs data analytics using the chosen ML algorithms. The outcomes of the analytics are a number of detailed reports and data visualization of the various analytics report. LAE includes three main components, i.e., Exploratory Data Analytics (EDA), Machine Learning Classifiers and Data Visualization. EDA cleans the data and extracts the features and feds into the ML algorithms for descriptive, perspective and predictive analytics. To improve the accuracy feature engineering is done through EDA (refer to figure 1). Data Visualization modules produce visualization of the past and future possible outcome.
<figure><img src="image_3.png" /><figcaption>Fig2: iLMS System Architecture</figcaption></figure>

4.4 Key Features

There are several features of the iLMS which are managed through users dashboard.
Each functionality contains important components of learner management and learning
analytics. This section presents the key iLMS features.

**User Interface and Security:** iLMS allows users to interact with the system based on the roles, i.e., management, instructors, admins and learners. The user interfaces have some standard functionalities which are irrespective of user role i.e., user profile, messaging system, course navigation, and shared course contents. Additional functionalities are added based on the specific role, for instance, management interface includes audit logs, instructors' performance against courses, retention rate, withdrawal rate, new enrolment rate, and so on. iLMS supports system access to the external stakeholder such as funding agency and awarding body, with limited capabilities, they will able to view learners progress, performance and course work. The application is hosted on a SSL enabled cloud platform. User will be given access details after their enrolment. The system authenticates users based on multiple factors. The system will support single sign-on and two factors authentication to enhance security measures. The system provides real-time traffic monitoring which offers regular backup and audit trails. Figure 3 shows the snapshot of the e-enrolment dashboard. The left-hand side of the interface is the navigation for the system and the content is in the middle of the web page. The enrolment dashboard provides a quick snapshot of the enrolment activities. By clicking on the icon, the top right corner, the user can access their profile and message related the users.
<figure><img src="image_4.png" /><figcaption>Fig 3: E-Enrolment Dashboard</figcaption></figure>

**Learning support session:** The system supports one to one and group session as required by the quality framework. The instructor can assign the task to individual learners where the learner will be able to upload/submit his/her work. This will support the company to adhere to quality assurance guidelines. An instructor will be able to produce a review report, observation report of individual learners. In traditional LMS courses are offered in group settings, our system also supports that. We have added TODO, FILES and COMMENTS. Comments support one to one or specific group communication. A trainer will be able to share additional materials with the learner according to his/her needs and trainer will be able to assign additional tasks to the learner. The learner will be able to upload his/her work. This feature will improve engagement and decrease the risk of dropping out.

**Report dashboard:** iLMS provides customised report generation facility depending on the various stakeholders' needs. In particular, reports that are necessary for the teaching and learning team are different from the report that needs for the management and external stakeholder. iLMS generates several reports such as the course progress, individual learner progression, learner's activity, learner success rate, learner's engagement, and learner's assessment. Figure 4 shows a snapshot of course reports and we have masked for two-column which contains identifiable information about learners. From this report, an instructor can view course activities. He/she can send individual email right from the report page by clicking the E-mail button on the right column. The button on the bottom left corner enables easy to export data in CSV format.
<figure><img src="image_5.png" /><figcaption>Fig 4: Course Report</figcaption></figure>

**Learning Analytics:** Learning analytics, important features of the iLMS, allows the user to get important information regarding the overall system and the user's activities. As the system support released access control, the analytics are shown based on user role. An instructor will be able to see the group-wise progress report, individual learners performance and corresponding activity logs. An instructor will be able to flag the students early based on their performance and who is on the risk of dropping out. Based on this analytics instructor will be to intervene in the learner early with additional and personalized learning support. The management will be able to redesign their course offerings and course contents, they will also able to offer student with an additional course based on their performance in the current course. The analytics features will also be available for the learners as they will be able to see their performance data act based on those.

Figure 5 shows the prediction engine in the test environment as a part of predictive analytics. The prediction engine is developed based on the data cleaned by EDA and the ML algorithms. As mentioned earlier, we have used LR, KNN and Decision Tree algorithm. The decision tree provides the highest accuracy among the three classifiers. It takes three inputs as based on the three selected features i.e. assessment score, highest qualification and engagement score. With the information provided the system predict the outcome. In the screenshot, we have entered student qualification of A-Level, average score of 82 (the average score is calculated based on the marks obtained in different assessments ) and engagement score18615 ( this score is near our threshold value of 2000 ). The system accurately predicts the learner will pass which establishes our finding in the descriptive analysis.
<figure><img src="image_6.png" /><figcaption>Fig4: Prediction engine interface</figcaption></figure>

**5 Conclusion**

Technology-enhanced learning has already become one of the key elements for teaching and learning support. LMS is now widely adopted across the industry to support the teaching and learning needs. The steep market growth and the post-COVID-19 will change the LMS landscape. This paper presents an automated intelligence learner management system (iLMS) and its key features. iLMS integrates learner enrolment, management and learning analytics through a single platform. The learning analytics correlates the learners data generated from the LMS and provides descriptive, predictive and perspective analytics. We believe iLMS can effectively support the institutions to understand the learners' needs and undertake the necessary actions to improve the overall learning support. The iLMS development and testing phase is recently completed. We are planning to deploy the iLMS into the project context based on the chosen Cohort. Upon completion of the deployment and users feedback, iLMS will be fully operational into the project industrial partner context. Therefore, an empirical investigation of the iLMS and its finding to demonstrate its applicability are next step of this work.

**References**

1. Bezovski, Z. and Poorani, S., 2016, March. The evolution of e-learning and new trends. In Information and Knowledge Management (Vol. 6, No. 3, pp. 50-57). IISTE.

2. LMS Market by Component (Solution and Services), Delivery Mode (Distance Learning, Instructor-Led Training and Blended Learning), Deployment Type, User Type (Academic and Corporate), and Region - Global Forecast to 2025, https://www.marketsandmarkets.com/PressReleases/learning-management-systems.asp

3. Kumar, K. and Vivekanandan, V., 2018. Advancing learning through smart learning analytics: a review of case studies. Asian Association of Open Universities Journal.
4. Herodotou, C., Rienties, B., Boroowa, A., Zdrahal, Z. and Hlosta, M., 2019. Large-scale implementation of predictive learning analytics in higher education: the teachers' role and perspective. Educational Technology Research and Development, 67(5), pp.1273-1306.

5. Viberg, O., Hatakka, M., Bälter, O. and Mavroudi, A., 2018. The current landscape of learning analytics in higher education. Computers in Human Behavior, 89, pp.98-110.

6. Papamitsiou, Z., & Economides, A. (2014). Learning Analytics and Educational Data Mining in Practice: A Systematic Literature Review of Empirical Evidence. Educational Technology & Society, 17 (4), 49–64.

7. Rosé, C.P., McLaughlin, E.A., Liu, R. and Koedinger, K.R., 2019. Explanatory learner models: Why machine learning (alone) is not the answer. British Journal of Educational Technology, 50(6), pp.2943-2958.

8. C. Carmen, B. Davis, AND E.D Wagner, “The Evolution of LMS: From Management to Learning”. The eLearning Guild, Santa Rosa, CA, 2009

9. Siemens, G. and Long, P., 2011. Penetrating the fog: Analytics in learning and education. EDUCAUSE Review, 46(5), p.30.

10. Andriotis, N., 2017. Will Artificial Intelligence Bring Real Smarts To Elearning?. [online] eFront Blog. Available at: <https://www.efrontlearning.com/blog/2017/06/artificial-intelligence-elearning.html> [Accessed 17 May 2020].

11. Islam, S. and Mahmud, H., 2020. Integration of Learning Analytics into Learner Management System using Machine Learning. In: 2020 The 2nd International Conference on Modern Educational Technology ICMET, ACM

12. Islam, S. and Mahmud, H., An Intelligence Learner Management System using Learning Analytics and Machine learning, 12th International Conference on Education Technology and Computers ICETC 2020, ACM.

13. Wong, B.T.M., 2017. Learning analytics in higher education: an analysis of case studies. Asian Association of Open Universities Journal.

14. Avella, J. T., Kebritchi, M., G. Nunn S., Kanai, T (2016), “Learning analytics methods, benefits, and challenges in higher education: a systematic literature review”, Online Learning, Vol. 20

15. L.; Chen, P., Lin, Z., (2020) Artificial Intelligence in Education: A Review, IEEE Access, Vol(8), 2020

16. Busch, J., Hanna, P., O'Neill, I., McGowan, A., Collins M., (2017) Can machine learning on learner analytics produce a predictive model on student performance?, Innovative and Creative Education and Technology International Conference (ICETIC).

17. Akçapınar G., Altun A., Aşkar P. (2019), Using learning analytics to develop early warning system for at-risk students, International Journal of Educational Technology in Higher Education, Springer.