# Analysis of Creative Engagement in AI Tools in Education Based on the #PPai6 Framework

Dea Puspita Septiani, Panos Kostakos, Margarida Romero

► To cite this version:

Dea Puspita Septiani, Panos Kostakos, Margarida Romero. Analysis of Creative Engagement in AI Tools in Education Based on the #PPai6 Framework. MIS4TEL 2023 - International Conference in Methodologies and intelligent Systems for Techhnology Enhanced Learning, Jul 2023, Guimaraes, Portugal. pp.48 - 58, 10.1007/978-3-031-42134-1_5. hal-04593531

HAL Id: hal-04593531

https://hal.science/hal-04593531

Submitted on 29 May 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
5c21-AI, a five-level framework of creative engagement in AI
tools in education

Dea Puspita Septiani¹ [0009-0001-9780-5782], Panos Kostakos
[0000-0002-8545-599X]², Margarida Romero¹ [0000-0003-3356-8121]

¹ Université Côte d'Azur, LINE, Nice, France

² University of Oulu, Finland, and Kaunas University of Technology, Lithuania
dea-puspita.septiani@etu.univ-cotedazur.fr

Panos.Kostakos@oulu.fi
margarida.romero@univ-cotedazur.fr

**Abstract.** Human creativity is a complex process that can be evaluated in a wide range of domains and tasks. The domain and task-specificity of human creativity challenge the process of designing AI-based tools to support teachers' and learners' creative engagement. In this study, we introduce the 5c21-AI framework to look at the 21st-century skills that modern education needs to teach, with a focus on creativity and critical thinking. After introducing the 5c21-AI framework, we analyze 41 studies using AI in education in order to identify the level of creative engagement they are able to support from primary education to Higher Education. The results show the most usual way of supporting learners' creative engagement is through intelligent tutoring systems (ITS), which rely on the second level of creative engagement of the 5c21-AI framework. In this second level, the AI tool shows adaptive behavior based on the learners' interactions but does not engage the learners in creating new ideas or solutions. We analyze 13 cases where learners and teachers get help with their own creative processes, but only two cases of collective creativity support at the individual level. None of the AI tools in education supports collective creativity among teachers.

**Keywords:** Artificial intelligence, creativity, creative engagement,
education, 21st-century competencies

# 1. Introduction

In the last five decades, AI tools have been developed for a variety of educational purposes. However, these tools were mostly used in limited situations, often for research purposes. The review by Feng and Law [1] on the studies developed during 2010–2019 has permitted two main educational technologies being supported by AI for educational purposes: Intelligent Tutoring Systems (ITS) and massive open online courses (MOOCs). The popularization of the use of AI tools for education has benefited over the past year from the public availability of the *ChatGPT*. In 2022, 242 papers were published on *ChatGPT*, and 1470 in the first three months of 2023. The
5c21-AI, a five-level framework of creative engagement in AI
tools in education

Dea Puspita Septiani¹ [0009-0001-9780-5782], Panos Kostakos
[0000-0002-8545-599X]², Margarida Romero¹ [0000-0003-3356-8121]

¹ Université Côte d'Azur, LINE, Nice, France

² University of Oulu, Finland, and Kaunas University of Technology, Lithuania
dea-puspita.septiani@etu.univ-cotedazur.fr

Panos.Kostakos@oulu.fi
margarida.romero@univ-cotedazur.fr

**Abstract.** Human creativity is a complex process that can be evaluated in a wide range of domains and tasks. The domain and task-specificity of human creativity challenge the process of designing AI-based tools to support teachers' and learners' creative engagement. In this study, we introduce the 5c21-AI framework to look at the 21st-century skills that modern education needs to teach, with a focus on creativity and critical thinking. After introducing the 5c21-AI framework, we analyze 41 studies using AI in education in order to identify the level of creative engagement they are able to support from primary education to Higher Education. The results show the most usual way of supporting learners' creative engagement is through intelligent tutoring systems (ITS), which rely on the second level of creative engagement of the 5c21-AI framework. In this second level, the AI tool shows adaptive behavior based on the learners' interactions but does not engage the learners in creating new ideas or solutions. We analyze 13 cases where learners and teachers get help with their own creative processes, but only two cases of collective creativity support at the individual level. None of the AI tools in education supports collective creativity among teachers.

**Keywords:** Artificial intelligence, creativity, creative engagement, education, 21st-century competencies

# 1. Introduction

In the last five decades, AI tools have been developed for a variety of educational purposes. However, these tools were mostly used in limited situations, often for research purposes. The review by Feng and Law [1] on the studies developed during 2010–2019 has permitted two main educational technologies being supported by AI for educational purposes: Intelligent Tutoring Systems (ITS) and massive open online courses (MOOCs). The popularization of the use of AI tools for education has benefited over the past year from the public availability of the *ChatGPT*. In 2022, 242 papers were published on *ChatGPT*, and 1470 in the first three months of 2023. The
media's coverage of ChatGPT's effects has made more people aware of how this tool
could be used in education. However, it has also raised some concerns about the
effects of this type of tool on academic integrity and on teachers' and students' ability
to be creative in their educational activities. In this paper, we consider an
emergent-based approach to technology-enhanced learning (TEL), in which AI tools
are part of activity systems that are not only defined by the technology tools but also
by how they mediate the human activity systems towards a certain goal [2], [3]. While
some educators are concerned about how AI tools can hinder learners' creative
processes by delegating part of the process to AI tools, we should also consider how
these tools can support AI-human collaboration for individual, group-based, and
large-group collaboration.

1.1. Creativity as one of the 21st-century skills to be supported in AI-human
learning activities

Creativity is now widely acknowledged as a crucial skill that distinguishes human labor from that performed by robots in an environment where automatization and artificial intelligence are having an increasing effect [4]–[6]. According to Florida [4], creativity is a factor in the social division of modern societies into "creative classes," who establish professions where creativity is a decisive element in their sophisticated problem-solving activities, and "creative workers," who do not. In postindustrial knowledge societies, different types of jobs increasingly depend on the “creative class” [4], and those who work in it "engage in complex problem-solving that involves a great deal of independent judgment and requires high levels of education or human capital" (p. 8). For this reason, supporting creative processes is essential in 21st-century education. AI tools should be part of the efforts to support human creativity development at the school, but also in creative professions. Teachers are among the professionals who face challenges related to the need to support learners' 21st-century competencies as well as a better understanding of how AI technologies operate and can support the teaching and learning process.

Creativity is one of the five key transversal competencies for the 21st century education #5c21 model [7], in which critical thinking is the ability to develop independent, critical thought in order to analyze ideas, knowledge, and processes based on a human-based value system and judgment. While creativity can be considered either human or "artificial creativity" when an AI system develops useful, novel, and original ideas or artifacts [8]; critical thinking can only be performed by humans because this competency relies on human criteria and takes into account factors such as the cultural context and interpersonal relationships in a certain socio-cultural context [9]. In the Horizon AugMentor project [10], we aim to support learners' creativity by developing a pedagogical framework permitting the use of LMS AI tools to support learners and teachers in their creative processes.
media's coverage of ChatGPT's effects has made more people aware of how this tool
could be used in education. However, it has also raised some concerns about the
effects of this type of tool on academic integrity and on teachers' and students' ability
to be creative in their educational activities. In this paper, we consider an
emergent-based approach to technology-enhanced learning (TEL), in which AI tools
are part of activity systems that are not only defined by the technology tools but also
by how they mediate the human activity systems towards a certain goal [2], [3]. While
some educators are concerned about how AI tools can hinder learners' creative
processes by delegating part of the process to AI tools, we should also consider how
these tools can support AI-human collaboration for individual, group-based, and
large-group collaboration.

1.1. Creativity as one of the 21st-century skills to be supported in AI-human learning activities

Creativity is now widely acknowledged as a crucial skill that distinguishes human labor from that performed by robots in an environment where automatization and artificial intelligence are having an increasing effect [4]–[6]. According to Florida [4], creativity is a factor in the social division of modern societies into "creative classes," who establish professions where creativity is a decisive element in their sophisticated problem-solving activities, and "creative workers," who do not. In postindustrial knowledge societies, different types of jobs increasingly depend on the “creative class” [4], and those who work in it "engage in complex problem-solving that involves a great deal of independent judgment and requires high levels of education or human capital" (p. 8). For this reason, supporting creative processes is essential in 21st-century education. AI tools should be part of the efforts to support human creativity development at the school, but also in creative professions. Teachers are among the professionals who face challenges related to the need to support learners' 21st-century competencies as well as a better understanding of how AI technologies operate and can support the teaching and learning process.

Creativity is one of the five key transversal competencies for the 21st century education #5c21 model [7], in which critical thinking is the ability to develop independent, critical thought in order to analyze ideas, knowledge, and processes based on a human-based value system and judgment. While creativity can be considered either human or "artificial creativity" when an AI system develops useful, novel, and original ideas or artifacts [8]; critical thinking can only be performed by humans because this competency relies on human criteria and takes into account factors such as the cultural context and interpersonal relationships in a certain socio-cultural context [9]. In the Horizon AugMentor project [10], we aim to support learners' creativity by developing a pedagogical framework permitting the use of LMS AI tools to support learners and teachers in their creative processes.
<figure><img src="image_1.png" /><figcaption>Fig. 1. Five key competencies for the 21st-century education (#5c21)</figcaption></figure>

## 1.2. The limits of AI tools for supporting cognitive architecture to support creativity

While creativity can manifest as either human or "artificial creativity" when an AI system generates useful, novel, and original ideas or artifacts, critical thinking is inherently human. It depends on human criteria and factors such as cultural context and interpersonal relationships within a specific socio-cultural environment.

Human cognitive architecture allows for the incorporation of cultural context and interpersonal relationships within a specific socio-cultural environment. Presently, large Language Models (LLMs) like OpenAI's ChatGPT are incapable of engaging in critical thinking and human-like creativity because they do not possess a "cognitive architecture capable of self-reflection and metacognitive judgments. This absence of cognitive architecture led some GPT3 architects to develop a new model, Anthropic, which aims to develop a cognitive architecture capable of critical thinking. GPT4 develops a higher short memory, which better supports the integration of socio-cultural prompts and creates results that are perceived by the end-user as more creative than prior releases.

## 1.3. Creative engagement: learners' and teachers' perspective

Learners' engagement is a requirement for correctly developing the learning activities designed by the teachers. Engagement is defined as the learners' "involvement in learning activities in terms of attention, participation, effort, intensity, or persistence" [11, p. S16].

For example, the *English-ABLE* system [12] has the main pedagogical objective of supporting the learners' engagement with three artificial agents based on the Open Students Model (OSM) which are designed to engage the learners' in a teaching process of the OSM. Through the learners' engagement with the OSM agents, they are expected to develop their English as a Second Language (ESL) competencies. The agents have been designed in a way to engage in different objectives, such as
<figure><img src="image_2.png" /><figcaption>Fig. 1. Five key competencies for the 21st-century education (#5c21)</figcaption></figure>

**1.2. The limits of AI tools for supporting cognitive architecture to support creativity**

While creativity can manifest as either human or "artificial creativity" when an AI system generates useful, novel, and original ideas or artifacts, critical thinking is inherently human. It depends on human criteria and factors such as cultural context and interpersonal relationships within a specific socio-cultural environment.

Human cognitive architecture allows for the incorporation of cultural context and interpersonal relationships within a specific socio-cultural environment. Presently, large Language Models (LLMs) like OpenAI's ChatGPT are incapable of engaging in critical thinking and human-like creativity because they do not possess a "cognitive architecture capable of self-reflection and metacognitive judgments. This absence of cognitive architecture led some GPT3 architects to develop a new model, Anthropic, which aims to develop a cognitive architecture capable of critical thinking. GPT4 develops a higher short memory, which better supports the integration of socio-cultural prompts and creates results that are perceived by the end-user as more creative than prior releases.

**1.3. Creative engagement: learners' and teachers' perspective**

Learners' engagement is a requirement for correctly developing the learning activities designed by the teachers. Engagement is defined as the learners' "involvement in learning activities in terms of attention, participation, effort, intensity, or persistence" [11, p. S16].

For example, the *English-ABLE* system [12] has the main pedagogical objective of supporting the learners' engagement with three artificial agents based on the Open Students Model (OSM) which are designed to engage the learners' in a teaching process of the OSM. Through the learners' engagement with the OSM agents, they are expected to develop their English as a Second Language (ESL) competencies. The agents have been designed in a way to engage in different objectives, such as
grammar feedback. We can consider the learners’ engagement in the *English-ABLE* system as creative because they are producing novel, useful, and original interactions with artificial agents. They are not only selecting pre-established answers (non-creative engagement), but they are also creatively engaged in the interaction with these three artificial agents. For supporting these creative engagements, the system is based on an "adaptive sequencing of activities and adaptive feedback mechanisms" [12, p. 382]. So, it's important to model the learner, the activities, and the feedback system while keeping in mind the learner's creative process for this particular task. The external regulation of the teachers can increase learners' creative engagement. In the context of the *English-ABLE* solution, dynamic recommendations that enable teachers to make decisions about how to support the learners' process support the teachers' creative engagement in the supervision process.

## 2. 5c21-AI levels of creative engagement in AI for education

Technology alone is not sufficient for driving innovation or improvement in education [13], [14]. The learning activities are complex systems in which technological and cultural artifacts mediate individual and collective activity oriented towards a certain goal and are constrained by the socio-cultural and situational elements within which the learning activity is developed [15]. An important consequence of considering learning activities as an intertwined system in which there is an emergent use of technologies based on the learners' motivations, the task constraints, and the situational aspects of the activity is that the same technology could be used in a very diverse way. For example, the *Teachable Machine* tool for creating machine learning models can be used in a lecture in which the teacher shows an example of its use to a group of learners who are not engaged in the activity. In this context, the learners are "passive consumers" of the lecture on *Teachable Machine*. But the teacher can also engage the learners in a group-based activity in which the children will create a model for improving the recycling system of their neighborhood in collaboration with the shop owners around the school. In this participatory solution co-creation, *Teachable Machine* could be used as a tool to integrate when tinkering with a solution that is possible to develop thanks to the participatory creative process engaged by the community participating in this objective. *Teachable Machine* is only a tool with the potential to support the creative engagement process, depending on the pedagogical scenario of the teaching activity proposed by the teacher.

### 2.1. From passive consumption to participatory content co-creation

In Fig. 2, five levels of creative engagement with AI tools in education are identified, which are on a continuum from simple to complex, and reflect the degree of creative engagement a learner can experience as a socio-cognitive process:

* *Level 1: Passive consumer.* The learner just consults AI-generated content with no creative engagement on their side.

* *Level 2: Interactive consumer.* The learner interacts with an AI system that adapts the feedback and activity progression based on the learners’ and task
grammar feedback. We can consider the learners’ engagement in the *English-ABLE* system as creative because they are producing novel, useful, and original interactions with artificial agents. They are not only selecting pre-established answers (non-creative engagement), but they are also creatively engaged in the interaction with these three artificial agents. For supporting these creative engagements, the system is based on an "adaptive sequencing of activities and adaptive feedback mechanisms" [12, p. 382]. So, it's important to model the learner, the activities, and the feedback system while keeping in mind the learner's creative process for this particular task. The external regulation of the teachers can increase learners' creative engagement. In the context of the *English-ABLE* solution, dynamic recommendations that enable teachers to make decisions about how to support the learners' process support the teachers' creative engagement in the supervision process.

## 2. 5c21-AI levels of creative engagement in AI for education

Technology alone is not sufficient for driving innovation or improvement in education [13], [14]. The learning activities are complex systems in which technological and cultural artifacts mediate individual and collective activity oriented towards a certain goal and are constrained by the socio-cultural and situational elements within which the learning activity is developed [15]. An important consequence of considering learning activities as an intertwined system in which there is an emergent use of technologies based on the learners' motivations, the task constraints, and the situational aspects of the activity is that the same technology could be used in a very diverse way. For example, the *Teachable Machine* tool for creating machine learning models can be used in a lecture in which the teacher shows an example of its use to a group of learners who are not engaged in the activity. In this context, the learners are "passive consumers" of the lecture on *Teachable Machine*. But the teacher can also engage the learners in a group-based activity in which the children will create a model for improving the recycling system of their neighborhood in collaboration with the shop owners around the school. In this participatory solution co-creation, *Teachable Machine* could be used as a tool to integrate when tinkering with a solution that is possible to develop thanks to the participatory creative process engaged by the community participating in this objective. *Teachable Machine* is only a tool with the potential to support the creative engagement process, depending on the pedagogical scenario of the teaching activity proposed by the teacher.

### 2.1. From passive consumption to participatory content co-creation

In Fig. 2, five levels of creative engagement with AI tools in education are identified, which are on a continuum from simple to complex, and reflect the degree of creative engagement a learner can experience as a socio-cognitive process:

* *Level 1: Passive consumer.* The learner just consults AI-generated content with no creative engagement on their side.

* *Level 2: Interactive consumer.* The learner interacts with an AI system that adapts the feedback and activity progression based on the learners’ and task
models. In this second level, the AI system adapts to the learner, but the learner isn't doing anything creative; they're just moving forward based on how the system is set up.

* Level 3: Individual content creation. The learner can engage creatively in proposing different ideas or solutions that are not predetermined by the AI system.

* Level 4: Collaborative content creation. A dyad or small group of learners are engaged in a joint creative activity to propose different ideas or solutions which are not pre-determined by the AI system.

* Level 5: Participatory knowledge co-creation. A group of learners, in collaboration with other participants outside their learning group, engage in a creative participatory activity engaged in a complex problem-solving situation.

<figure><img src="image_2.png" /><figcaption>**Fig. 2.** Five levels of creative engagement in AI in education (#5c21-ai)</figcaption></figure>

The first level of the #5c21-AI is similar to the "passive" level of Chi and Wylie's ICAP framework [16]. The biggest difference between our model and the ICAP is the consideration of interactivity. While "interactive mode of engagement" is the highest level of cognitive engagement within the ICAP framework, our model considers "participatory knowledge co-creation". Participatory knowledge co-creation engages the participants not only in an interactive and socio-constructive situation, but also engages learners in the identification, understanding, and problem-solving processes of a problematic situation within their learning or neighborhood community, linking the team-based co-creation process with a participatory process wherein a team of learners engages in their learning community in order to improve a real-world problem or value community initiatives [2], [17].

The first two levels are not engaging the learners' in a creative activity. In the first level, the learners use what is made available to them without any interaction. In the second level, learners interact with an AI system that responds to their actions based on a model of the learning task and a model of the learner that is built into the AI system. The AI tool has a predefined set of options that lead to interactions governed by a "programmed instruction" approach, harking back to Pressey's teaching machine. ITS are the most common AI tools in this second level of creative engagement.
models. In this second level, the AI system adapts to the learner, but the learner isn't doing anything creative; they're just moving forward based on how the system is set up.

* Level 3: Individual content creation. The learner can engage creatively in proposing different ideas or solutions that are not predetermined by the AI system.

* Level 4: Collaborative content creation. A dyad or small group of learners are engaged in a joint creative activity to propose different ideas or solutions which are not pre-determined by the AI system.

* Level 5: Participatory knowledge co-creation. A group of learners, in collaboration with other participants outside their learning group, engage in a creative participatory activity engaged in a complex problem-solving situation.

<figure><img src="image_3.png" /><figcaption>**Fig. 2.** Five levels of creative engagement in AI in education (#5c21-ai)</figcaption></figure>

The first level of the #5c21-AI is similar to the "passive" level of Chi and Wylie's ICAP framework [16]. The biggest difference between our model and the ICAP is the consideration of interactivity. While "interactive mode of engagement" is the highest level of cognitive engagement within the ICAP framework, our model considers "participatory knowledge co-creation". Participatory knowledge co-creation engages the participants not only in an interactive and socio-constructive situation, but also engages learners in the identification, understanding, and problem-solving processes of a problematic situation within their learning or neighborhood community, linking the team-based co-creation process with a participatory process wherein a team of learners engages in their learning community in order to improve a real-world problem or value community initiatives [2], [17].

The first two levels are not engaging the learners' in a creative activity. In the first level, the learners use what is made available to them without any interaction. In the second level, learners interact with an AI system that responds to their actions based on a model of the learning task and a model of the learner that is built into the AI system. The AI tool has a predefined set of options that lead to interactions governed by a "programmed instruction" approach, harking back to Pressey's teaching machine. ITS are the most common AI tools in this second level of creative engagement.
The third level of creative engagement with AI tools in education is to get the learner to make texts, photos, or videos that are related to a certain learning moment or situation. Whereas the fourth and fifth levels of creative engagement with AI tools engage learners in a co-creation process that supports the knowledge construction process [18]. The fifth level gets students involved in finding a problem in their learning or neighborhood community, understanding it, and coming up with a solution. In this fifth level, the co-creation participatory process is oriented toward the community as well as real-life problem-solving [2]. The participatory and community-oriented (or based) knowledge co-creation values local community initiatives, promotes diversity, and regenerates intergenerational and intercultural links that are often missed in our current societies [19].

## 3. Methodology

For the analysis of the five levels of creative engagement in AI in education, we revised the *International Journal of Artificial Intelligence in Education* (AIED) within the last 3 years. We selected 41 papers in order to analyze the different levels of creative engagement, considering not only the learners' and teachers' perspectives on creative engagement but also the domain of application and the educational level of the study.

For papers integrating more than one AI solution, we evaluate them separately. For example, *HOWARD* and *BioWorld* tools in the paper of Lajoie [20], we integrated two records in the analysis of creative engagement levels in order to report the specificities of these different AI solutions that were studied.

Studies in which both learner and teacher activity is supported are also integrated as two separate entries in order to consider the level of creative engagement for each of these perspectives.

## 4. Results

### 5.1. Levels of creative engagement

The assessment of the five levels of creative engagement across the 41 selected studies has revealed that the majority of creative engagement focuses on the learner perspective (n=32), with only five instances of AI tools supporting teachers' creative engagement (Fig. 3). In three studies (*BioWorld & HOWARD Platform* [20], *MiWRITE* [21], *English-ABLE & CBAL* [12]), the tool supported both the teacher and the learner's creative engagement process.
The third level of creative engagement with AI tools in education is to get the learner to make texts, photos, or videos that are related to a certain learning moment or situation. Whereas the fourth and fifth levels of creative engagement with AI tools engage learners in a co-creation process that supports the knowledge construction process [18]. The fifth level gets students involved in finding a problem in their learning or neighborhood community, understanding it, and coming up with a solution. In this fifth level, the co-creation participatory process is oriented toward the community as well as real-life problem-solving [2]. The participatory and community-oriented (or based) knowledge co-creation values local community initiatives, promotes diversity, and regenerates intergenerational and intercultural links that are often missed in our current societies [19].

## 3. Methodology

For the analysis of the five levels of creative engagement in AI in education, we revised the *International Journal of Artificial Intelligence in Education* (AIED) within the last 3 years. We selected 41 papers in order to analyze the different levels of creative engagement, considering not only the learners' and teachers' perspectives on creative engagement but also the domain of application and the educational level of the study.

For papers integrating more than one AI solution, we evaluate them separately. For example, *HOWARD* and *BioWorld* tools in the paper of Lajoie [20], we integrated two records in the analysis of creative engagement levels in order to report the specificities of these different AI solutions that were studied.

Studies in which both learner and teacher activity is supported are also integrated as two separate entries in order to consider the level of creative engagement for each of these perspectives.

## 4. Results

### 5.1. Levels of creative engagement

The assessment of the five levels of creative engagement across the 41 selected studies has revealed that the majority of creative engagement focuses on the learner perspective (n=32), with only five instances of AI tools supporting teachers' creative engagement (Fig. 3). In three studies (*BioWorld & HOWARD Platform* [20], *MiWRITE* [21], *English-ABLE & CBAL* [12]), the tool supported both the teacher and the learner's creative engagement process.
<figure><img src="image_3.png" /><figcaption>Fig. 3. Learners’ and teachers’ levels of creative engagement</figcaption></figure>

Only one study has developed a solution in which there is no creative engagement from the learners. The study of Lawson, et al. [22] only looked at learners’ emotional expressions to animated instructions during a presentation.

We can observe that the majority of studies (n=21) support the second level of creative engagement, “interactive consuming”. In this level, the learner uses an AI tool to help with a well-defined learning activity, and the system adapts to the learner's needs. Most of the systems in this context are self-identified as Intelligent Tutoring Systems (ITS).

Individual creative engagement, the third level of the 5c21-AI model, is the only level in which we can observe both the learner’s (n=8) and teacher’s (n=6) perspectives, which are in some cases supported independently, as in the case of *Tuglet* [23], *Physics Playground* [24], *C2STEM* [25], *TopoMath* [26], etc. More interestingly, the *BioWorld & HOWARD Platform* [20], *MiWRITE* [21], *English-ABLE* and *CBAL* [12] tools support both the teachers and the learners in their creative processes.

The fourth level, collective creative engagement, can be observed in two studies. The first one is the study of *NoRILLA* [27], a specialized Augmented Reality (AR) for STEM education where learners are encouraged to do an exploratory construction together. The learners’ using the AI tool *NoRILLA* are engaged in building blocks that should not fall during a strong motion. First, the learners advance individually, and once they have achieved a certain progression, they are able to engage in teams for an exploratory construction collaboratively with the tool. The second study is using the *HOWARD Platform* [20] where learners and instructors can simultaneously engage in monitoring and responding in a problem-based learning (PBL) environment in which discussion is encouraged.

Participatory content co-creation, the fifth level of the 5c21-AI framework, is not observed in any of the 41 studies revised.
<figure><img src="image_4.png" /><figcaption>Fig. 3. Learners’ and teachers’ levels of creative engagement</figcaption></figure>

Only one study has developed a solution in which there is no creative engagement from the learners. The study of Lawson, et al. [22] only looked at learners’ emotional expressions to animated instructions during a presentation.

We can observe that the majority of studies (n=21) support the second level of creative engagement, “interactive consuming”. In this level, the learner uses an AI tool to help with a well-defined learning activity, and the system adapts to the learner's needs. Most of the systems in this context are self-identified as Intelligent Tutoring Systems (ITS).

Individual creative engagement, the third level of the 5c21-AI model, is the only level in which we can observe both the learner’s (n=8) and teacher’s (n=6) perspectives, which are in some cases supported independently, as in the case of *Tuglet* [23], *Physics Playground* [24], *C2STEM* [25], *TopoMath* [26], etc. More interestingly, the *BioWorld & HOWARD Platform* [20], *MiWRITE* [21], *English-ABLE* and *CBAL* [12] tools support both the teachers and the learners in their creative processes.

The fourth level, collective creative engagement, can be observed in two studies. The first one is the study of *NoRILLA* [27], a specialized Augmented Reality (AR) for STEM education where learners are encouraged to do an exploratory construction together. The learners’ using the AI tool *NoRILLA* are engaged in building blocks that should not fall during a strong motion. First, the learners advance individually, and once they have achieved a certain progression, they are able to engage in teams for an exploratory construction collaboratively with the tool. The second study is using the *HOWARD Platform* [20] where learners and instructors can simultaneously engage in monitoring and responding in a problem-based learning (PBL) environment in which discussion is encouraged.

Participatory content co-creation, the fifth level of the 5c21-AI framework, is not observed in any of the 41 studies revised.
**5. Discussion**

AI tools can be used in a diversity of pedagogical scenarios and situations with different degrees of creative engagement for the learner or the teacher. In this study, we have analyzed 41 studies in order to identify creative engagement with AI tools. The results show the majority of educational uses of AI are individual (n=33 for the learners n=6 for the teachers), with only two uses to support the learners' co-creativity. We can observe that most of the uses of AI tools do not engage the learners creatively. Most of the studies engage learners in the second level of the 5c21-AI framework, "interactive consuming", through intelligent tutoring systems (ITS) which permit the system to adapt to the learners' inputs based on the learner model and the learning task model integrated into the AI-tool. Developing a learner and task model is already a complex process that requires domain-specific expertise and computer modeling efforts. Computer-supported collaborative learning (CSCL) activities are more complex in relation to the group dynamics that are emerging not only at the individual level but at the collective level. The lack of standardization on these group dynamics ontologies and modes is one of the potential reasons why we observe a limited number of collaborative uses of AI tools in education.

We can observe that some of the AI tools are Digital Game-Based Learning (DGBL) solutions. The characteristics of DGBL as systems engaging the learners' by providing feedback and supporting the actions towards a learning objective are aligned with the characteristics of AI tools in education, which share these two characteristics. Most DGBL solutions support the second level of creative engagement. Among the DGBL AI tools, two address STEM education, such as C2STEM [25] and NoRILLA [27].

None of the studies supports participatory content co-creation, the fifth level of the 5c21-AI framework. We can consider not only the higher degree of complexity of AI systems but also their ability to support users whose behavior does not correspond to the teacher-and-learner model. In the context of Maison de l'intelligence artificielle (MIA) in Sophia Antipolis, the types of activities in which the learners are engaged correspond to the fifth level. The learners are engaged to co-create solutions addressing different Sustainable Development Goals (SDGs) in collaboration with the AI experts in the MIA, their teachers, and other learners.

The analysis of the levels of creative engagement will permit the researchers and computer engineers in the Augmentor Project to consider the types of AI solutions that can better support 21st-century competencies and permit the learners and teachers to deploy their human creativity with the support of AI.

Beyond the creative integration of AI tools in education, AI tools cannot be considered as creative as humans at the current stage. Consider a more robust cognitive architecture that will allow AI models to better simulate human-like cognition, allowing them to process cultural contexts, interpersonal relationships, and long-term memory more effectively. This would be particularly valuable for collaborative open-ended learning activities and CSCL pedagogical scenarios on the Level 5 participatory content co-creation, where context and history play an essential role in decision-making and problem-solving processes. Incorporating a cognitive architecture in AI models would also help address challenges related to biases, ethics, and transparency, as it would allow AI systems to make decisions and judgments
# 5. Discussion

AI tools can be used in a diversity of pedagogical scenarios and situations with different degrees of creative engagement for the learner or the teacher. In this study, we have analyzed 41 studies in order to identify creative engagement with AI tools. The results show the majority of educational uses of AI are individual (n=33 for the learners n=6 for the teachers), with only two uses to support the learners' co-creativity. We can observe that most of the uses of AI tools do not engage the learners creatively. Most of the studies engage learners in the second level of the 5c21-AI framework, "interactive consuming", through intelligent tutoring systems (ITS) which permit the system to adapt to the learners' inputs based on the learner model and the learning task model integrated into the AI-tool. Developing a learner and task model is already a complex process that requires domain-specific expertise and computer modeling efforts. Computer-supported collaborative learning (CSCL) activities are more complex in relation to the group dynamics that are emerging not only at the individual level but at the collective level. The lack of standardization on these group dynamics ontologies and modes is one of the potential reasons why we observe a limited number of collaborative uses of AI tools in education.

We can observe that some of the AI tools are Digital Game-Based Learning (DGBL) solutions. The characteristics of DGBL as systems engaging the learners' by providing feedback and supporting the actions towards a learning objective are aligned with the characteristics of AI tools in education, which share these two characteristics. Most DGBL solutions support the second level of creative engagement. Among the DGBL AI tools, two address STEM education, such as C2STEM [25] and NoRILLA [27].

None of the studies supports participatory content co-creation, the fifth level of the 5c21-AI framework. We can consider not only the higher degree of complexity of AI systems but also their ability to support users whose behavior does not correspond to the teacher-and-learner model. In the context of Maison de l'intelligence artificielle (MIA) in Sophia Antipolis, the types of activities in which the learners are engaged correspond to the fifth level. The learners are engaged to co-create solutions addressing different Sustainable Development Goals (SDGs) in collaboration with the AI experts in the MIA, their teachers, and other learners.

The analysis of the levels of creative engagement will permit the researchers and computer engineers in the Augmentor Project to consider the types of AI solutions that can better support 21st-century competencies and permit the learners and teachers to deploy their human creativity with the support of AI.

Beyond the creative integration of AI tools in education, AI tools cannot be considered as creative as humans at the current stage. Consider a more robust cognitive architecture that will allow AI models to better simulate human-like cognition, allowing them to process cultural contexts, interpersonal relationships, and long-term memory more effectively. This would be particularly valuable for collaborative open-ended learning activities and CSCL pedagogical scenarios on the Level 5 participatory content co-creation, where context and history play an essential role in decision-making and problem-solving processes. Incorporating a cognitive architecture in AI models would also help address challenges related to biases, ethics, and transparency, as it would allow AI systems to make decisions and judgments
based on a more comprehensive understanding of the inter-group context and human
values.

**Acknowledgments.** This research is developed within the AugMentor project, funded by the HORIZON-CL2-2021 TRANSFORMATIONS-01-05 program.

References

[1] S. Feng and N. Law, "Mapping Artificial Intelligence in Education Research: a Network-based Keyword Analysis," *Int. J. Artif. Intell. Educ.*, vol. 31, no. 2, pp. 277–303, Jun. 2021, doi: 10.1007/s40593-021-00244-4.

[2] G. Isaac, M. Romero, and S. Barma, "Understanding co-creativity in real-world problem solving in project-based learning in higher education," *Rev. Int. CRIRES Innover Dans Tradit. Vygotsky CRISAS Int. J. Vygotskys Herit. Innov. Educ.*, vol. 6, no. 3, pp. 86–99, 2022, doi: 10.51657/ric.v6i2.51585.

[3] Y. Engstrom, “Activity theory as a framework for analyzing and redesigning work,” *Ergonomics*, vol. 43, no. 7, pp. 960–974, Jul. 2000, doi: 10.1080/001401300409143.

[4] R. L. Florida, *The rise of the creative class: and how it's transforming work, leisure, community and everyday life*, Nachdr. New York, NY: Basic Books, 2006.

[5] E. De Bono, *Serious creativity: using the power of lateral thinking to create new ideas*. New York, NY, USA: Penguin Random House, 2015.

[6] R. J. Sternberg and P. A. Frensch, *Complex problem solving: Principles and mechanisms*. Psychology Press, 2014.

[7] M. Romero, B. Lille, and A. Patino, Eds., *Usages créatifs du numérique pour l'apprentissage au XXIe siècle*, vol. 1. Québec: PUQ, 2017.

[8] C. Moruzzi, “Measuring creativity: an account of natural and artificial creativity,” *Eur. J. Philos. Sci.*, vol. 11, no. 1, p. 1, Mar. 2021, doi: 10.1007/s13194-020-00313-w.

[9] M. Romero, “Développer la pensée informatique pour démystier l’intelligence artificielle,” *1024 – Bull. Société Inform. Fr.*, no. 12, pp. 67–75, 2018.

[10] augMENTOR, “Augmented Intelligence for Pedagogically Sustained Training and Education (augMENTOR),” Jan. 01, 2023. https://cordis.europa.eu/project/id/101061509 (accessed Mar. 25, 2023).

[11] T. K. F. Chiu, “Applying the self-determination theory (SDT) to explain student engagement in online learning during the COVID-19 pandemic,” *J. Res. Technol. Educ.*, vol. 54, no. sup1, pp. S14–S30, Jan. 2022, doi: 10.1080/15391523.2021.1891998.

[12] D. Zapata-Rivera, “Open Student Modeling Research and its Connections to Educational Assessment,” *Int. J. Artif. Intell. Educ.*, vol. 31, no. 3, pp. 380–396, Sep. 2021, doi: 10.1007/s40593-020-00206-2.

[13] M. Romero, T. Laferriere, and T. M. Power, “The Move is On! From the Passive Multimedia Learner to the Engaged Co-creator,” *eLearn*, vol. 2016, no. 3, p. 1, Mar. 2016, doi: 10.1145/2904374.2893358.
based on a more comprehensive understanding of the inter-group context and human
values.

**Acknowledgments.** This research is developed within the AugMentor project, funded by the HORIZON-CL2-2021 TRANSFORMATIONS-01-05 program.

References

[1] S. Feng and N. Law, “Mapping Artificial Intelligence in Education Research: a Network-based Keyword Analysis,” *Int. J. Artif. Intell. Educ.*, vol. 31, no. 2, pp. 277–303, Jun. 2021, doi: 10.1007/s40593-021-00244-4.

[2] G. Isaac, M. Romero, and S. Barma, “Understanding co-creativity in real-world problem solving in project-based learning in higher education,” *Rev. Int. CRIRES Innover Dans Tradit. Vygotsky CRISAS Int. J. Vygotskys Herit. Innov. Educ.*, vol. 6, no. 3, pp. 86–99, 2022, doi: 10.51657/ric.v6i2.51585.

[3] Y. Engstrom, “Activity theory as a framework for analyzing and redesigning work,” *Ergonomics*, vol. 43, no. 7, pp. 960–974, Jul. 2000, doi: 10.1080/001401300409143.

[4] R. L. Florida, *The rise of the creative class: and how it's transforming work, leisure, community and everyday life*, Nachdr. New York, NY: Basic Books, 2006.

[5] E. De Bono, *Serious creativity: using the power of lateral thinking to create new ideas*. New York, NY, USA: Penguin Random House, 2015.

[6] R. J. Sternberg and P. A. Frensch, *Complex problem solving: Principles and mechanisms*. Psychology Press, 2014.

[7] M. Romero, B. Lille, and A. Patino, Eds., *Usages créatifs du numérique pour l'apprentissage au XXIe siècle*, vol. 1. Québec: PUQ, 2017.

[8] C. Moruzzi, “Measuring creativity: an account of natural and artificial creativity,” *Eur. J. Philos. Sci.*, vol. 11, no. 1, p. 1, Mar. 2021, doi: 10.1007/s13194-020-00313-w.

[9] M. Romero, “Développer la pensée informatique pour démystier l’intelligence artificielle,” *1024 – Bull. Société Inform. Fr.*, no. 12, pp. 67–75, 2018.

[10] augMENTOR, “Augmented Intelligence for Pedagogically Sustained Training and Education (augMENTOR),” Jan. 01, 2023. https://cordis.europa.eu/project/id/101061509 (accessed Mar. 25, 2023).

[11] T. K. F. Chiu, “Applying the self-determination theory (SDT) to explain student engagement in online learning during the COVID-19 pandemic,” *J. Res. Technol. Educ.*, vol. 54, no. sup1, pp. S14–S30, Jan. 2022, doi: 10.1080/15391523.2021.1891998.

[12] D. Zapata-Rivera, “Open Student Modeling Research and its Connections to Educational Assessment,” *Int. J. Artif. Intell. Educ.*, vol. 31, no. 3, pp. 380–396, Sep. 2021, doi: 10.1007/s40593-020-00206-2.

[13] M. Romero, T. Laferriere, and T. M. Power, “The Move is On! From the Passive Multimedia Learner to the Engaged Co-creator,” *eLearn*, vol. 2016, no. 3, p. 1, Mar. 2016, doi: 10.1145/2904374.2893358.
[14] A. Tricot, "L'innovation pédagogique : Mythes et réalités," 2017.

[15] M. Romero and S. Barma, "Analysing an Interactive Problem-Solving Task Through the Lens of Double Stimulation," *Can. J. Learn. Technol.*, vol. 48, no. 1, Art. no. 1, Aug. 2022, doi: 10.21432/cjlt28170.

[16] M. T. H. Chi and R. Wylie, "The ICAP Framework: Linking Cognitive Engagement to Active Learning Outcomes," *Educ. Psychol.*, vol. 49, no. 4, pp. 219-243, Oct. 2014, doi: 10.1080/00461520.2014.965823.

[17] R. Lukyanenko and J. Parsons, "Extending Participatory Design Principles to Structured User-generated Content," presented at the Scandinavian Conference on Information Systems, Oulu, Finland, 2015.

[18] J. Zhang, M. Scardamalia, M. Lamon, R. Messina, and R. Reeve, "Socio-cognitive dynamics of knowledge building in the work of 9-and 10-year-olds," *Educ. Technol. Res. Dev.*, vol. 55, no. 2, pp. 117-145, 2007.

[19] G. Cucinelli, A.-L. Davidson, M. Romero, and T. Matheson, "Intergenerational Learning Through a Participatory Video Game Design Workshop," *J. Intergenerational Relatsh.*, vol. 16, no. 1-2, pp. 146-165, 2018.

[20] S. P. Lajoie, "Student Modeling for Individuals and Groups: the BioWorld and HOWARD Platforms," *Int. J. Artif. Intell. Educ.*, vol. 31, no. 3, pp. 460-475, Sep. 2021, doi: 10.1007/s40593-020-00219-x.

[21] J. Wilson, Y. Huang, C. Palermo, G. Beard, and C. A. MacArthur, "Automated Feedback and Automated Scoring in the Elementary Grades: Usage, Attitudes, and Associations with Writing Outcomes in a Districtwide Implementation of MI Write," *Int. J. Artif. Intell. Educ.*, vol. 31, no. 2, pp. 234-276, Jun. 2021, doi: 10.1007/s40593-020-00236-w.

[22] A. P. Lawson, R. E. Mayer, N. Adamo-Villani, B. Benes, X. Lei, and J. Cheng, "Do Learners Recognize and Relate to the Emotions Displayed By Virtual Instructors?," *Int. J. Artif. Intell. Educ.*, vol. 31, no. 1, pp. 134-153, Mar. 2021, doi: 10.1007/s40593-021-00238-2.

[23] T. Käser and D. L. Schwartz, "Modeling and Analyzing Inquiry Strategies in Open-Ended Learning Environments," *Int. J. Artif. Intell. Educ.*, vol. 30, no. 3, pp. 504-535, Oct. 2020, doi: 10.1007/s40593-020-00199-y.

[24] V. J. Shute et al., "The Design, Development, and Testing of Learning Supports for the Physics Playground Game," *Int. J. Artif. Intell. Educ.*, vol. 31, no. 3, pp. 357-379, Sep. 2021, doi: 10.1007/s40593-020-00196-1.

[25] N. M. Hutchins, G. Biswas, N. Zhang, C. Snyder, Á. Lédeczi, and M. Maróti, "Domain-Specific Modeling Languages in Computer-Based Learning Environments: a Systematic Approach to Support Science Learning through Computational Modeling," *Int. J. Artif. Intell. Educ.*, vol. 30, no. 4, pp. 537-580, Nov. 2020, doi: 10.1007/s40593-020-00209-z.

[26] K. VanLehn, C. Banerjee, F. Milner, and J. Wetzel, "Teaching Algebraic Model Construction: A Tutoring System, Lessons Learned and an Evaluation," *Int. J. Artif. Intell. Educ.*, vol. 30, no. 3, pp. 459-480, Oct. 2020, doi: 10.1007/s40593-020-00205-3.

[27] N. Yannier, S. E. Hudson, and K. R. Koedinger, "Active Learning is About More Than Hands-On: A Mixed-Reality AI System to Support STEM Education," *Int. J. Artif. Intell. Educ.*, vol. 30, no. 1, pp. 74-96, Mar. 2020, doi: 10.1007/s40593-020-00194-3.
[14] A. Tricot, "L'innovation pédagogique : Mythes et réalités," 2017.

[15] M. Romero and S. Barma, "Analysing an Interactive Problem-Solving Task Through the Lens of Double Stimulation," *Can. J. Learn. Technol.*, vol. 48, no. 1, Art. no. 1, Aug. 2022, doi: 10.21432/cjlt28170.

[16] M. T. H. Chi and R. Wylie, "The ICAP Framework: Linking Cognitive Engagement to Active Learning Outcomes," *Educ. Psychol.*, vol. 49, no. 4, pp. 219-243, Oct. 2014, doi: 10.1080/00461520.2014.965823.

[17] R. Lukyanenko and J. Parsons, "Extending Participatory Design Principles to Structured User-generated Content," presented at the Scandinavian Conference on Information Systems, Oulu, Finland, 2015.

[18] J. Zhang, M. Scardamalia, M. Lamon, R. Messina, and R. Reeve, "Socio-cognitive dynamics of knowledge building in the work of 9-and 10-year-olds," *Educ. Technol. Res. Dev.*, vol. 55, no. 2, pp. 117-145, 2007.

[19] G. Cucinelli, A.-L. Davidson, M. Romero, and T. Matheson, "Intergenerational Learning Through a Participatory Video Game Design Workshop," *J. Intergenerational Relatsh.*, vol. 16, no. 1-2, pp. 146-165, 2018.

[20] S. P. Lajoie, "Student Modeling for Individuals and Groups: the BioWorld and HOWARD Platforms," *Int. J. Artif. Intell. Educ.*, vol. 31, no. 3, pp. 460-475, Sep. 2021, doi: 10.1007/s40593-020-00219-x.

[21] J. Wilson, Y. Huang, C. Palermo, G. Beard, and C. A. MacArthur, "Automated Feedback and Automated Scoring in the Elementary Grades: Usage, Attitudes, and Associations with Writing Outcomes in a Districtwide Implementation of MI Write," *Int. J. Artif. Intell. Educ.*, vol. 31, no. 2, pp. 234-276, Jun. 2021, doi: 10.1007/s40593-020-00236-w.

[22] A. P. Lawson, R. E. Mayer, N. Adamo-Villani, B. Benes, X. Lei, and J. Cheng, "Do Learners Recognize and Relate to the Emotions Displayed By Virtual Instructors?," *Int. J. Artif. Intell. Educ.*, vol. 31, no. 1, pp. 134-153, Mar. 2021, doi: 10.1007/s40593-021-00238-2.

[23] T. Käser and D. L. Schwartz, "Modeling and Analyzing Inquiry Strategies in Open-Ended Learning Environments," *Int. J. Artif. Intell. Educ.*, vol. 30, no. 3, pp. 504-535, Oct. 2020, doi: 10.1007/s40593-020-00199-y.

[24] V. J. Shute et al., "The Design, Development, and Testing of Learning Supports for the Physics Playground Game," *Int. J. Artif. Intell. Educ.*, vol. 31, no. 3, pp. 357-379, Sep. 2021, doi: 10.1007/s40593-020-00196-1.

[25] N. M. Hutchins, G. Biswas, N. Zhang, C. Snyder, Á. Lédeczi, and M. Maróti, "Domain-Specific Modeling Languages in Computer-Based Learning Environments: a Systematic Approach to Support Science Learning through Computational Modeling," *Int. J. Artif. Intell. Educ.*, vol. 30, no. 4, pp. 537-580, Nov. 2020, doi: 10.1007/s40593-020-00209-z.

[26] K. VanLehn, C. Banerjee, F. Milner, and J. Wetzel, "Teaching Algebraic Model Construction: A Tutoring System, Lessons Learned and an Evaluation," *Int. J. Artif. Intell. Educ.*, vol. 30, no. 3, pp. 459-480, Oct. 2020, doi: 10.1007/s40593-020-00205-3.

[27] N. Yannier, S. E. Hudson, and K. R. Koedinger, "Active Learning is About More Than Hands-On: A Mixed-Reality AI System to Support STEM Education," *Int. J. Artif. Intell. Educ.*, vol. 30, no. 1, pp. 74-96, Mar. 2020, doi: 10.1007/s40593-020-00194-3.