# Learning Environments in the 21st Century: A Mapping of the Literature

Ilias Pappas, Dag Håkon Oslen, Tumaini Kabudi

► To cite this version:

Ilias Pappas, Dag Håkon Oslen, Tumaini Kabudi. Learning Environments in the 21st Century: A Mapping of the Literature. International Working Conference on Transfer and Diffusion of IT (TDIT), Dec 2020, Tiruchirappalli, India. pp.67-76, 10.1007/978-3-030-64849-7_7. hal-03701808

HAL Id: hal-03701808

https://inria.hal.science/hal-03701808

Submitted on 22 Jun 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Learning Environments in the 21st century: A Mapping of
the Literature

Tumaini Kabudi¹, Ilias Pappas¹ and Dag Håkon Oslen¹

1 University of Agder, Kristiansand, Norway

**Abstract.** Education has been transformed by significant breakthroughs in AI, mobile internet, cloud computing and Big Data technologies. More personalized educational settings are developed by increasingly integrating contemporary learning environments with new technologies. However, few examples of executed AI enabled learning interventions have been identified. Therefore, a mapping of literature on AI enabled learning systems was done. 121 studies published in the last five years were analyzed. This paper presents a discussion regarding on what mainly AI enabled contemporary learning environments are designed to achieve. The major contribution of the study is bringing awareness to researchers and system developers on the purposes of AI enabled contemporary learning environments. This review will act as a guide for future studies on how to better design AI enabled learning environments.

**Keywords:** Contemporary learning environments, AI, Systematic Literature Mapping, Adaptive Learning Systems.

## 1 Introduction

AI enabled learning environments are gaining traction as they can deliver learning content and adapt to individual student needs. [1]. These contemporary learning systems are characterized by students thriving in a digital environment, where current technologies shape expectations of students and “*their abilities to access, acquire, manipulate, construct, create and communicate information*” [2]. The resources in AI enabled learning environments (both physical and virtual) are developed to deliver effective learning, by helping students construct their knowledge. Adaptive learning systems, Intelligent tutoring Systems and Recommender systems are good examples of recent AI enabled learning environments. Research on intelligent tutors, Big Data analytics, learning analytics and educational data mining techniques developed these AI enabled learning systems. Recommender systems are “*software tools based on machine learning and information retrieval techniques that provide suggestions for potential useful items to someone's interest*”[3]. On the other hand, Adaptive learning systems are platforms for personalized learning that adapt to students’ learning strategies, the sequence and difficulty of the task, learning styles, abilities, time of feedback and preferences [4]. The advantages of using adaptive learning systems include efficiency, interaction, support and effectiveness [5]. These systems encourage students to own their learning
journey via automated feedback cycles that allow the students to progress inde-
pendently from the course instructor.

Advancement in new data analytical techniques has brought more successful learn-
ing systems in the education sector. Furthermore, several studies have discussed on how
AI techniques can be applied to learning systems. Nevertheless, few examples of exe-
cuted AI enabled learning interventions have been identified [6]. Consequently, imple-
mentation of these systems in education settings seems to be at its early stage. It is
stated, "*Many of these learning systems as well as Intelligent Tutoring Systems (ITS)
are described in the literature, and their effectiveness has been proved. However, these
systems are rarely used in real educational settings practices in ordinary courses.*" [7].
ITS are basically designed to enable tutoring and personalized support. In Addition,
another significant challenge is bridging the gap between pedagogy and emerging AI
techniques.

The main aim of this study was to map latest literature and present the summarized findings related to contemporary learning systems. The main research question of this study is: “What are the AI enabled learning environments designed to do?” The topic discussed in this paper is AI-enabled learning systems, which deliver solutions for overcoming challenges students face. These challenges include inefficiently capturing student proficiency, lacking metacognition to self-assess students’ knowledge decay, lack of accommodation of practical skills in the systems, facing student disengagement and poor student motivation and utilizing outdated adaptation technologies [8], [9]. The next section introduces the systematic literature mapping (SLM) process, search protocol and results from the execution of the SLM. Results and Analysis are presented in Section 3. Section 4 concludes the paper with summarized discussions of findings from retrieved literature and recommendations.

## 2 Research Methodology

This study was conducted using the systematic literature review methodology
guidelines proposed by [10]. The researchers utilized EndNote X9, NVivo 11 and Excel
spreadsheets to extract publication outlets, find duplicates and organize the information.
A review protocol was used to guide the whole research method. This study selected
previous works published in the past five years to avoid outdated research.

The first step in conducting the systematic review is to formulate the search strat-
egy. This strategy is formulated based on review protocol to reduce research bias. The
search strategy was formulated by following and expanding the research question. Then
search keywords were identified and search strings generated to minimize the number
of articles. Synonyms and substitute spellings were identified too. Focus was placed
on two main terms of interest to perform database searches. These two main terms of
interest included “adaptive learning system” and “artificial intelligence”. Two parallel
searches were conducted as the two main terms of interest are sometimes used inter-
changeably. “Adaptive learning ecosystem,” “adaptive learning environment,” “adap-
tive learning platform,” “adaptive learning setting” and “adaptive learning technology”
were used as synonyms for adaptive learning system. The Boolean operators *OR* and
*AND* were used.
The search was conducted between November and December 2019. The search was done on eight databases i.e. ACM, Web of Science, EBSCO Host, Wiley, SAGE Journals, IEEE Xplore, Scopus, and Taylor and Francis. These eight databases were chosen as they have wide selection of relevant and recent articles in this SLM/SLR. The databases include numerous AI-related academic journals such as Journal of Artificial Intelligence and Soft Computing Research, IEEE Transactions on Pattern Analysis and Machine Intelligence, British Journal of Educational Technology, and International Journal of Intelligent Systems. 1699 articles were retrieved using the above-mentioned search strategies. These number of articles were retrieved also by choosing the publication years between 2015 and 2019. To reduce the number of retrieved articles, the study went under further refinement. Several articles were selected by considering certain criteria. This was done to ensure that the selected articles are relevant and answer the RQs. Consequent to the search, all documents retrieved went through duplicate removal using EndNote software. Figure 1 shows steps of the study selection criteria, which comprises inclusion and exclusion criteria.

## 3 Results and Analysis

The 121 selected papers were published between 2014 and 2019. In terms of publication channels, 63 of the papers were academic journals, 55 were conference papers, and 3 were published in conference proceedings. The selected papers were categorized based on the type of research approach used. This categorization is based on an existing research approach classification by Wieringa et al. (2006). Evaluation research was the most utilized research approach (33 articles), followed by validation research (30 articles). The third and fourth used research approach were philosophical approach and literature review approach, with 24 and 22 papers, respectively. The distribution of documents per year is shown in Table 1. A note to the reader: The complete database that include all the 121 references in RIS format can be downloaded

<table><caption>**Table 1.** The Distribution of papers per year</caption><thead><tr><th></th><th>2014</th><th>2015</th><th>2016</th><th>2017</th><th>2018</th><th>2019</th><th>2020</th></tr></thead><tbody><tr><td>Evaluation Research</td><td>4</td><td>4</td><td>7</td><td>7</td><td>4</td><td>9</td><td>35</td></tr><tr><td>Literature Review</td><td>0</td><td>3</td><td>4</td><td>9</td><td>2</td><td>4</td><td>22</td></tr><tr><td>Validation Research</td><td>2</td><td>0</td><td>7</td><td>6</td><td>8</td><td>7</td><td>30</td></tr><tr><td>Philosophical Paper</td><td>2</td><td>3</td><td>3</td><td>6</td><td>5</td><td>5</td><td>24</td></tr><tr><td>Solution Proposal</td><td>0</td><td>2</td><td>0</td><td>2</td><td>2</td><td>3</td><td>9</td></tr><tr><td>Opinion Paper</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>1</td><td>1</td></tr></tbody></table>
<figure><img src="image_2.png" /><figcaption>Fig. 1 The Systematic Literature Mapping Process</figcaption></figure>

## 3.1 RQ: What AI enabled learning environments designed to do

The study's RQ aimed to identify what are AI enabled learning environments designed to do. As shown in Table 1, there are different types of AI enabled learning environments. Majority of these learning environments in the selected papers were intelligent tutoring systems and adaptive learning systems.

<table><caption>Table 2. Kinds of AI enabled systems and their examples</caption><thead><tr><td>Kind of System</td><td>Examples</td><td>References</td></tr></thead><tbody><tr><td>Intelligent tutoring system</td><td>English Tutor, AutoTutor, Passive Voice Tutor, INTUITEL, the DARPA Tutor, Ms. Lindquist, Crystal Island, Wayang Outpost, ANDES, Guru, ACTIVEMATH,</td><td>[7], [11]–[17]</td></tr></tbody></table>
<table><tr><td>Adaptive Learning System</td><td>Fishtree, System designed by Tseng, S., Su J., Hwang, G.J. Tsai C. J, Quiz-Bot, Adaptive Mobile Learning System (AMLS), Personalized Adaptive Learning Dashboard (PALD), "MostSaRT" system, Smart Sparrow, System developed by Realizeit, OPERA, LearnSmart, Personal Assistant for Life-Long Learning (PAL3), INSPIREus, ProSys INSPIRE</td><td>[13], [18]–[27]</td></tr><tr><td>Adaptive system</td><td>Student Diagnosis, Assistance, Evaluation System based on Artificial Intelligence (StuDiAsE), The LeaPTM system, ADAPTIVE INSTRUCTOR OPERATING SYSTEMS</td><td>[22], [28], [29]</td></tr><tr><td>AI enabled system (machine learning)</td><td>The Early Recognition System, MeuTutor</td><td>[30]</td></tr><tr><td>A Knowledge grid based intelligent system personalized E-Learning system</td><td>KGTutor</td><td>[31]</td></tr><tr><td>Adaptive digital teaching and learning environment</td><td>Connect™</td><td>[32]</td></tr><tr><td>Intelligent and adaptive learning environment</td><td>FIT Java Tutor</td><td>[15]</td></tr><tr><td>Web-based intelligent computer-assisted language learning (iCALL) system</td><td>E-Tutor</td><td>[15]</td></tr><tr><td>Adaptive and intelligent web based educational systems (AIWBES)</td><td>SQL-Tutor, ALEA, Quiz Guide and Flip</td><td>[33]</td></tr><tr><td>Adaptive learning environment</td><td>ACTIVEMATH</td><td>[15]</td></tr><tr><td>Context-aware adaptive mobile learning system</td><td>Units of Learning mobile (UoLmP)</td><td>[21]</td></tr><tr><td>Intelligent learning system</td><td>Yixue</td><td>[34]</td></tr><tr><td>Intelligent system</td><td>BOXFiSH</td><td>[35]</td></tr><tr><td>Recommendation system</td><td>NetCoach</td><td>[22]</td></tr></table>

Table 2 highlights the various categories and themes of the designed aims of these AI enabled learning environments. Many of the published papers identified that the AI enabled learning environments were designed to assist with teaching several courses. These courses included mathematics, physics, psychology, nursing, computer literacy and biology. It was also identified that these systems were designed as platforms to teach and learn languages. The identified languages that were taught in these systems include English, German, and Greek. Another category of what AI enabled learning
environments were designed to do is improve students' performance through Personalization of Learning. These systems were designed to act as platforms to provide personalized content based on their level. Also, the AI enabled learning environments are designed to teach and learn programming languages such as SQL and Java. The rest remaining identified themes are shown in the table below.

<table><caption>**Table 3.** Categories of what AI enabled learning environments designed to do</caption><thead><tr><td>Category</td><td>Examples of the Mentioned Systems</td><td>References</td></tr></thead><tbody><tr><td>Teach Courses (17)</td><td>System developed by Realzeit, OPERA, ACTIVEMATH, AutoTutor, Ms. Lindquist, UZWEBMAT, AutoTutor, Crystal Island, Oscar<br/>Wayang Outpost, ANDES, Guru, ACTIVEMATH, English Tutor, Student Diagnosis, Assistance, Evaluation System based on Artificial Intelligence (StuDiAsE), Yixue, Lumilo</td><td>[13], [17], [23], [24], [28], [34], [36]–[39]</td></tr><tr><td>Platforms for Teaching and Learning Languages (6)</td><td>QuizBot, AutoTutor, Passive Voice Tutor, BOXFiSH, E-Tutor</td><td>[11], [13], [15], [35], [40]</td></tr><tr><td>Improve Students' Performance through Personalization of learning (8)</td><td>Adaptive Mobile Learning System (AMLS), INSPIREus MeuTutor Knewton, INSPIRE, Units of Learning mobile (UoLmP), An Online Web-based Adaptive Tutoring System, Connect™</td><td>[18], [21], [27], [41]–[43]</td></tr><tr><td>Platform for Quizz, Exercises, Training (6)</td><td>Smart Sparrow, Tamaxtil, affective tutoring system (ATS), QuestionIT</td><td>[16], [18], [19], [44]–[46]</td></tr><tr><td>Teach and Help with Programming Language (5)</td><td>SQL-Tutor, The intelligent Teaching Assistant for programming (ITAP), ALEA, QuizGuide and Flip, FIT Java Tutor, Gerdes' tutor</td><td>[15], [33]</td></tr><tr><td>Evaluate and Improve Students' Knowledge (4)</td><td>LearnSmart, Personal Assistant for Life-Long Learning (PAL3), DeepTutor, Protus</td><td>[15], [25], [26], [37]</td></tr><tr><td>Consider and Examine Learners Requirements (4)</td><td>Personalized Adaptive Learning Dashboard (PALD) "MostSaRT" system, INTUITEL, KGTutor</td><td>[7], [21], [31], [47]</td></tr><tr><td>Identify and Inform Students (2)</td><td>The LeaPTM system, The Early Recognition System</td><td>[30], [47]</td></tr></tbody></table>
# 4 Discussion and Conclusion

There were some interesting insights revealed in the conducted SLM. Regarding types of AI enabled learning environments, intelligent tutoring systems and adaptive learning systems were the most utilized. Our mapping of the literature revealed that the use of both these types of systems play an important role to improve performance of students through personalization of their learning, and mitigating poor levels of motivation [15], [48]–[50]. A possible reason of adaptive learning systems and intelligent tutoring systems to be mostly used is the recent acknowledgement of AI enabled learning environments in assisting teaching languages, several courses and programming languages. The increasing usage of these systems is in agreement with [8] and [9] on using AI enabled learning environments to improve students’ performance, improve students’ knowledge, and examine learners requirements such as previous knowledge and the learners’ targets. Our review indicated that however there is low number of implemented interventions designed to address major concerns and problems identified in the papers. Difficulty in attaining learners’ skills, background and profile of student issues are some of the overlooked concerns. This observation is in agreement with [51] who identified that designers of adaptive learning systems give little attention to courses that have practical skills as a prerequisite. This research gap should be addressed. Designing and assessing adaptive courses issues, in our review, are problems that are not highly considered. The low usage of adaptive learning systems concern is noted by [7]. The authors stated, “*Many of these learning systems as well as Intelligent Tutoring Systems (ITS) are described in the literature, and their effectiveness has been proved. However, these systems are rarely used in real educational settings practices in ordinary courses.*” This concern needs to be addressed in the research of AI enabled learning environments in the future. The issue of better designing adaptive learning systems and adaptive courses should be given attention to increase the usage of these systems in real educational settings. Only three papers out of 121 papers identified this concern to be addressed [52]–[54].

The contribution of this study is the identification of AI enabled learning interventions utilized. It is to be noted that the AI enabled learning environments are still under-researched. The other contribution of this study is to bringing awareness to researchers and system developers on contemporary learning systems. Hopefully, the study will provide important insights for practitioners in education settings who are interested in adaptive learning systems. The authors recommend for future research, more adaptive learning systems, frameworks and models should be developed. More systems should be developed in order to provide solutions for overcoming the overlooked problems that students face, especially attaining skills [55]. The existing systems lack novel AI techniques or tools for students to practice and master their learning skills. This lack is because of complex measurement models for skills and limited learning support. Nevertheless, this challenge can be addressed by boosting the learning content in the adaptive learning system using AI techniques [56]. A system that accommodates the required skills and requirements for students is also needed. Similarly, to address the issue of complexity, current adaptive learning models should be enhanced by AI techniques building on learning analytics [55].
Due to the nature of research, this study was related with several limitations. The search words, strings and databases may have limited the review, despite the recommendations by [10] were followed. The key strings were limited to adaptive learning systems only. The key strings may be extended to other types of contemporary learning environments, such as recommender systems. Also, the selected databases, inclusion criteria and exclusion criteria may have excluded some studies. In addition, research questions like what problems and concerns did the AI enabled contemporary learning environments address and types of interventions that utilized AI in addressing the problems were not reviewed. These research questions can be reviewed in future research.

## References

[1] E. Mousavinasab, N. Zarifsanaiey, S. R. Niakan Kalhori, M. Rakhshan, L. Keikha, and M. Ghazi Saeedi, "Intelligent tutoring systems: a systematic review of characteristics, applications, and evaluation methods," *Interact. Learn. Environ.*, pp. 1–22, 2018.

[2] T. D. Green and L. C. Donovan, "Learning Anytime, Anywhere through Technology," in *The Wiley Handbook of Teaching and Learning*, 2018, pp. 225–256.

[3] T. A. Syed, V. Palade, R. Iqbal, and S. S. K. Nair, "A personalized learning recommendation system architecture for learning management system," *IC3K 2017 - Proc. 9th Int. Jt. Conf. Knowl. Discov. Knowl. Eng. Knowl. Manag.*, vol. 1, pp. 275–282, 2017.

[4] D. Johnson and D. Samora, "The Potential Transformation Of Higher Education Through Computer-Based Adaptive Learning Systems," *Glob. Educ. J.*, vol. 2016, no. 1, pp. 1–17, 2016.

[5] A. Hasanov, T. H. Laine, and T. S. Chung, "A survey of adaptive context-aware learning environments," *Journal of Ambient Intelligence and Smart Environments*, vol. 11, no. 5. pp. 403–428, 2019.

[6] E. Wakelam, A. Jefferies, N. Davey, and Y. Sun, "The potential for using artificial intelligence techniques to improve e-learning systems," 2015, pp. 762–770.

[7] E. Verdú et al., "Intelligent tutoring interface for technology enhanced learning in a course of computer network design," in *Proceedings - Frontiers in Education Conference, FIE*, 2015, vol. 2015-Febru, no. February.

[8] A. J. Hampton, B. D. Nye, P. I. Pavlik, W. R. Swartout, A. C. Graesser, and J. Gunderson, "Mitigating knowledge decay from instruction with voluntary use of an adaptive learning system," vol. 10948 LNAI. Springer Verlag, pp. 119–133, 2018.

[9] R. Tommy, N. Amala, S. Ram, B. Kumar, and H. Jose, "Intelligent and adaptive test system," in *Lecture Notes in Electrical Engineering*, 2016, vol. 376, pp. 895–902.

[10] B. Kitchenham and S. Charters, "Guidelines for performing systematic literature reviews in software engineering," in *Technical report, Ver. 2.3 EBSE*
*Technical Report. EBSE*, 2007.

[11] V. Bradáč and K. Kostolányová, "Intelligent tutoring systems," *J. Intell. Syst.*, vol. 26, no. 4, pp. 717-727, 2017.

[12] A. Lippert, J. Gatewood, Z. Cai, and A. C. Graesser, "Using an Adaptive Intelligent Tutoring System to Promote Learning Affordances for Adults with Low Literacy Skills," in *Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)*, 2019, vol. 11597 LNCS, pp. 327-339.

[13] S. Ruan et al., "QuizBot: A Dialogue-based Adaptive Learning System for Factual Knowledge," in *Conference on Human Factors in Computing Systems - Proceedings*, 2019.

[14] K. W. Brawner and A. J. Gonzalez, "Modelling a learner's affective state in real time to improve intelligent tutoring effectiveness," *Theor. Issues Ergon. Sci.*, vol. 17, no. 2, pp. 183-210, Mar. 2016.

[15] A. T. Bimba, N. Idris, A. Al-Hunaiyyan, R. B. Mahmud, and N. L. B. M. Shuib, "Adaptive feedback in computer-based learning environments: a review," *Adapt. Behav.*, vol. 25, no. 5, pp. 217-234, 2017.

[16] J. D. Fletcher, "Adaptive Instructional Systems and Digital Tutoring," in *Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)*, 2019, vol. 11597 LNCS, pp. 615-633.

[17] K. Almohammadi, H. Hagras, D. Alghazzawi, and G. Aldabbagh, "A survey of artificial intelligence techniques employed for adaptive educational systems within e-learning platforms," *J. Artif. Intell. Soft Comput. Res.*, vol. 7, no. 1, pp. 47-64, 2017.

[18] V. Kasinathan, A. Mustapha, and I. Medi, "Adaptive learning system for higher learning," in *ICIT 2017 - 8th International Conference on Information Technology, Proceedings*, 2017, pp. 960-970.

[19] G. Natriello, "The Adaptive Learning Landscape," *Teach. Coll. Rec.*, vol. 119, no. 3, 2017.

[20] B. Pandey, R. B. Mishra, and A. Khamparia, "CBR based approach for adaptive learning in E-learning system," 2014.

[21] F. E. Louhab, A. Bahnasse, F. Bensalah, A. Khiat, Y. Khiat, and M. Talea, "Novel approach for adaptive flipped classroom based on learning management system," *Educ. Inf. Technol.*, 2019.

[22] M. Liu, E. McKelroy, S. B. Corliss, and J. Carrigan, "Investigating the effect of an adaptive learning intervention on students' learning," *Etr&D-Educational Technol. Res. Dev.*, vol. 65, no. 6, pp. 1605-1625, 2017.

[23] P. Johannes and L. Lagerstrom, "Adaptive learning: The premise, promise, and pitfalls," 2017, vol. 2017-June.

[24] M. Oliveira, A. Barreiras, G. Marcos, H. Ferreira, A. Azevedo, and C. V. De Carvalho, "Collecting and analysing learners data to support the adaptive engine of OPERA, a learning system for mathematics," in *CSEDU 2017 - Proceedings of the 9th International Conference on Computer Supported Education*, 2017, vol. 1, pp. 631-638.
[25] Q. Sun, Y. Abdourazakou, and T. J. Norman, "LearnSmart, adaptive teaching, and student learning effectiveness: An empirical investigation," *J. Educ. Bus.*, vol. 92, no. 1, pp. 36-43, 2017.

[26] A. J. Hampton, B. D. Nye, P. Pavlik, and W. Swartout, "Mitigating Knowledge Decay from Instruction with Voluntary Use of an Adaptive Learning System GIFT Multi-Agent Architecture View project Operation ARA View project SEE PROFILE," *Springer*, vol. 10948 LNAI, pp. 119-133, 2018.

[27] L. Dounas, C. Salinesi, and O. El Beqqali, "Requirements monitoring and diagnosis for improving adaptive e-learning systems design," 2019.

[28] M. Samarakou et al., "Evaluation of an intelligent open learning system for engineering education," *Knowl. Manag. E-Learning*, vol. 8, no. 3, pp. 496-513, 2016.

[29] L. M. Milham, J. A. Pharmer, and A. W. Fok, "Adaptive instructor operating systems: Design to support instructor assessment of team performance," in *Proceedings of the Human Factors and Ergonomics Society*, 2015, vol. 2015-Janua, pp. 1801-1805.

[30] M. Ciolacu, A. F. Tehrani, L. Binder, and P. M. Svasta, "Education 4.0 - Artificial Intelligence Assisted Higher Education: Early recognition System with Machine Learning to support Students' Success," *2018 IEEE 24th Int. Symp. Des. Technol. Electron. Packag. SIITME 2018 - Proc.*, pp. 23-30, 2019.

[31] M. Mudrak, "Analysis and implementation of adaptive course in Moodle," *ICETA 2017 - 15th IEEE Int. Conf. Emerg. eLearning Technol. Appl. Proc.*, 2017.

[32] K. Kakish and L. Pollacia, "Adaptive learning to improve student success and instructor efficiency in introductory computing course," 2018, pp. 72-78.

[33] A. H. Jonsdottir, A. Jakobsdottir, and G. Stefansson, "Development and use of an adaptive learning environment to research online study behaviour," 2015.

[34] S. Wang, M. Feng, M. Bienkowski, C. Christensen, and W. Cui, "Learning from an adaptive learning system: Student profiling among middle school students," in *CSEDU 2019 - Proceedings of the 11th International Conference on Computer Supported Education*, 2019, vol. 1, pp. 78-84.

[35] W. Cui, Z. Xue, and K. P. Thai, "Performance Comparison of an AI-Based Adaptive Learning System in China," in *Proceedings 2018 Chinese Automation Congress, CAC 2018*, 2019, pp. 3170-3175.

[36] S. Sosnovsky and I. A. Chacon, "Semantic gap detection in metadata of adaptive learning environments," in *Proceedings - IEEE 14th International Conference on Advanced Learning Technologies, ICALT 2014*, 2014, pp. 548-552.

[37] F. J. Gallego-Durán, R. Molina-Carmona, and F. Llorens-Largo, "Measuring the difficulty of activities for adaptive learning," *Univers. Access Inf. Soc.*, vol. 17, no. 2, pp. 335-348, 2017.

[38] K. W. Brawner and A. J. Gonzalez, "Modelling a learner's affective state in real time to improve intelligent tutoring effectiveness," *Theor. Issues Ergon. Sci.*, vol. 17, no. 2, pp. 183-210, 2016.

[39] B. du Boulay, "Escape from the Skinner Box: The case for contemporary
intelligent learning environments,” *Br. J. Educ. Technol.*, vol. 50, no. 6, pp. 2902–2919, Nov. 2019.

[40] A. Lippert, J. Gatewood, Z. Cai, and A. C. Graesser, “Using an Adaptive Intelligent Tutoring System to Promote Learning Affordances for Adults with Low Literacy Skills,” vol. 11597 LNCS. Springer Verlag, pp. 327–339, 2019.

[41] S. J. De Santana, R. Paiva, I. I. Bittencourt, P. E. Ospina, R. De Amorim Silva, and S. Isotani, “Evaluating the impact of mars and venus effect on the use of an adaptive learning technology for Portuguese and mathematics,” *Proc. - IEEE 16th Int. Conf. Adv. Learn. Technol. ICALT 2016*, pp. 31–35, 2016.

[42] C. Wongwatkit, “An Online Web-based Adaptive Tutoring System for University Exit Exam on IT Literacy,” in *International Conference on Advanced Communication Technology, ICACT*, 2019, vol. 2019-Febru, pp. 563–568.

[43] K. Kakish and L. Pollacia, “Adaptive learning to improve student success and instructor efficiency in introductory computing course,” in *Proceedings of the 34th Information Systems Education Conference, ISECON 2018*, 2018, pp. 72–78.

[44] G. Padron-Rivera, C. Joaquin-Salas, J. L. Patoni-Nieves, and J. C. Bravo-Perez, “Patterns in Poor Learning Engagement in Students While They Are Solving Mathematics Exercises in an Affective Tutoring System Related to Frustration,” vol. 10880 LNCS. Springer Verlag, pp. 169–177, 2018.

[45] M. Houa and C. Fidopiastis, “A generic framework of intelligent adaptive learning systems: from learning effectiveness to training transfer,” *Theor. Issues Ergon. Sci.*, vol. 18, no. 2, pp. 167–183, 2017.

[46] L. M. Milham, J. A. Pharmer, and A. W. Fok, “Adaptive instructor operating systems: Design to support instructor assessment of team performance,” *Proc. Hum. Factors Ergon. Soc.*, vol. 2015-Janua, no. 1, pp. 1801–1805, 2015.

[47] M. Liu, E. McKelroy, S. B. Corliss, and J. Carrigan, “Investigating the effect of an adaptive learning intervention on students’ learning,” *Educ. Technol. Res. Dev.*, vol. 65, no. 6, pp. 1605–1625, Dec. 2017.

[48] S. El Janati and A. Maach, “Towards a new adaptive E-learning framework for adapting content to presentation,” *2017 Intell. Syst. Comput. Vision, ISCV 2017*, 2017.

[49] G. Padron-Rivera, C. Joaquin-Salas, J. L. Patoni-Nieves, and J. C. Bravo-Perez, “Patterns in Poor Learning Engagement in Students While They Are Solving Mathematics Exercises in an Affective Tutoring System Related to Frustration,” in *Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)*, 2018, vol. 10880 LNCS, pp. 169–177.

[50] M. Maravanyika, N. Dlodlo, and N. Jere, “An adaptive recommender-system based framework for personalised teaching and learning on e-learning platforms,” *2017 IST-Africa Week Conf. IST-Africa 2017*, 2017.

[51] H. Xie, H. C. Chu, G. J. Hwang, and C. C. Wang, “Trends and development in technology-enhanced adaptive/personalized learning: A systematic review of journal publications from 2007 to 2017,” *Comput. Educ.*, vol. 140, Oct. 2019.
[52] A. T. Bimba, N. Idris, A. Al-Hunaiyyan, R. B. Mahmud, and N. L. B. M. Shuib, "Adaptive feedback in computer-based learning environments: a review," *Adapt. Behav.*, vol. 25, no. 5, pp. 217-234, 2017.

[53] W. Chen, C. Joe-Wong, C. G. Brinton, L. Zheng, and D. Cao, "Principles for assessing adaptive online courses," 2018.

[54] R. Y. Tsarev et al., "An Approach to Developing Adaptive Electronic Educational Course," in *Advances in Intelligent Systems and Computing*, 2019, vol. 986, pp. 332-341.

[55] Z. Papamitsiou, A. A. Economides, I. O. Pappas, and M. N. Giannakos, "Explaining learning performance using response-Time, self-Regulation and satisfaction from content: An fsQCA approach," in *ACM International Conference Proceeding Series*, 2018, pp. 181-190.

[56] M. N. Giannakos, K. Sharma, I. O. Pappas, V. Kostakos, and E. Velloso, "Multimodal data as a means to understand the learning experience," *Int. J. Inf. Manage.*, vol. 48, pp. 108-119, 2019.