<figure><img src="image_1.png" /></figure>

HAL
open science

# Synthesis on AI in Education: How can AI support teachers at their job?

Jiajun Pan, Azim Roussanaly, Anne Boyer

<figure><img src="image_2.png" /></figure>

## To cite this version:

Jiajun Pan, Azim Roussanaly, Anne Boyer. Synthesis on AI in Education: How can AI support teachers at their job?. Loria; Université de Lorraine (France). 2022. hal-04425100

HAL Id: hal-04425100

https://hal.science/hal-04425100v1

Submitted on 29 Jan 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_3.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_4.png" /></figure>

AI4T
AI FOR TEACHERS

**Work Package 2:**

**Training architecture and resources**

**Deliverable 2.2**

**Synthesis on AI in Education**

**How can AI support teachers at their job?**

<figure><img src="image_5.png" /></figure>

Co-funded by the
Erasmus+ Programme
of the European Union

<figure><img src="image_6.png" /></figure>
<figure><img src="image_7.png" /></figure>

AI4T
AI FOR TEACHERS

<table><tr><td>Project Title:</td><td>Artificial Intelligence for and by Teachers</td></tr><tr><td>Project Acronym:</td><td>AI4T</td></tr><tr><td>Project Number</td><td>626154</td></tr><tr><td>Grant Agreement Number</td><td>626154-EPP-1-2020-2-FR-EPPKA3-PI-POLICY</td></tr><tr><td>Instrument:</td><td>Erasmus Plus, Key Action 3</td></tr><tr><td>Deliverable number</td><td>D2.2</td></tr><tr><td>Work Package</td><td>2 – Training architecture and resources</td></tr><tr><td>Work Package Leader</td><td>University of Nantes</td></tr><tr><td>Work Package Partners</td><td>INRIA, Université de Lorraine, MENJS, MIZS, University of Maribor, H2, DWEC, SCRIPT, ITD-CNR, MI, FEI</td></tr><tr><td>Dissemination level</td><td>Public</td></tr><tr><td>Delivery date</td><td>2022-11</td></tr><tr><td>Status</td><td>Final</td></tr><tr><td>Version</td><td>3.0</td></tr><tr><td>Author(s)</td><td>Jiajun PAN, Azim ROUSSANALY, Anne BOYER</td></tr><tr><td>List of contributors</td><td>All partners</td></tr><tr><td>Deliverable Manager / Reviewers</td><td>France Education International</td></tr></table>

Disclaimer

This project has been funded with support from the European Commission. This publication reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein.

<figure><img src="image_8.png" /></figure>

Co-funded by the
Erasmus+ Programme
of the European Union

<figure><img src="image_9.png" /></figure>
<figure><img src="image_10.png" /></figure>

<table><tr><td>ABSTRACT</td><td>The application of Artificial Intelligence (AI) in education is a rapidly developing field that is revolutionizing the way we learn and teach. This report aims to provide a comprehensive overview of the current state of AI applications in education, including its definition and role, application scenarios and common technologies, strengths and weaknesses, challenges faced, and future outlook. Through an organized structure, this report delves into various aspects of the application of AI in education, on the one hand to answer the questions that teachers would like to know about and to use it in the training material in the AI4T project, especially in parts of the MOOC, and on the other hand to help other members of the AI4T project to have a better understanding of the current state of the art of the use of AI in education.<br/><br/>This report defines the concept of AIER (Artificial Intelligence Educational Resources). AIERs are AI tools or resources used to support educational tasks. Central to this definition is the clarity of the scope of the work of teachers supported by AIERs. Based on this definition, we propose a template for characterizing the work of AI-supported teachers. In the report, we categorize the current state of the art, research trends and possible developments in AIER, based on the template, for the current and near future. Application scenarios, advantages and disadvantages of AIER are presented. Teachers' and other educators' concerns about AI are included.<br/><br/>Based on our survey of teachers' concerns, we discuss how teachers should use AIER, and how to measure the performance and credibility of AIER. We conclude with a summary of questions that teachers would like to know about using AIER in their work and answer them using the content of the previous chapters.</td></tr><tr><td>KEYWORDS</td><td>Artificial Intelligence; AI in education; Educational tasks.</td></tr></table>

<table><thead><tr><th colspan="3">Dissemination level</th></tr></thead><tbody><tr><td>PU</td><td>Public</td><td>X</td></tr><tr><td>PP</td><td>Restricted to project partner (including the Commission)</td><td></td></tr><tr><td>RE</td><td>Restricted to a group defined by the consortium (including the Commission)</td><td></td></tr><tr><td>CO</td><td>Confidential, only for members of the consortium (including the Commission)</td><td></td></tr></tbody></table>

<figure><img src="image_11.png" /></figure>

<figure><img src="image_12.png" /></figure>
<figure><img src="image_13.png" /></figure>

Introduction<fnref n="1" />

Artificial Intelligence in Education (AIEd) is one of the emerging areas in education today. However, it is still unclear for most teachers and educators how it can be fully leveraged and what its benefits are. In the ERASMUS+ AI4T project, from the reports [4][5][26], it can be noticed that compared to the increase of AIER (Artificial Intelligence Educational Resources), there is a lack of understanding and trust in AIER among our primary service group, teachers. This lack also exists in related groups, students and educational institutions. To address this issue and explore the current status and future development of AIEd, this paper focuses on how AI (Artificial Intelligence) can support the work of teachers.

This document is organized as follows:

In the Section Background and Definition, we focus on finding a general consensus of what is the teachers' job and investigated the need for teachers to be supported by AI. We summarized this definition of teachers' jobs through descriptions of teachers' work on major jobs search websites and investigated various views on the need for teachers to have AI support for their work. By summarizing these, we propose a template for the characteristics of AI support for teachers' jobs. In this document, we present AIERs categorized by this template, which makes it easy for teachers to understand the support that AI can provide, and for AIER designers to define teachers' needs for AI. This template is a supplement to the template provided in document "Template for AI characterization" to describe tools.

In the Section Now and Future, we present the state of the art of AIER today and in the near future (about 5 years from now). AIER has a broad meaning and can be an educational platform or an online application. In this section, we classify AIERs according to the template proposed in the previous section. By summarizing the current state of the art of research in the AIEd field, we predict the trends of AIERs in the next 5 years and how they might support teachers' jobs.

In the Section Advantages and Concerns, following the conclusion of the previous section, we summarized the advantages offered by the use of AI in education today. On the other hand, we also summarized the concerns and worries of teachers about the use of AI in education. For teachers and educational institutions, it is worth noting what AIER can do and how it can support their work. For researchers in the AIEd field, it is worth noting what advantages and disadvantages are associated with using AIER.

In the Section Usage and Metric, we focus on how teachers should use and select AI in education. In this section, we describe scenarios in which teachers use AIER in their educational work. We also recommend several metrics for teachers to choose AIER in their work, including metrics to measure AI performance, and metrics to measure AI trustworthiness.

<figure><img src="image_14.png" /></figure>

<footnote n="1">¹ This document is made for policy makers in the AI4T project.</footnote>
<figure><img src="image_15.png" /></figure>

AI4T
AI FOR TEACHERS

Finally in the Section Questions and Template, we summarize the problems that teachers want to know when using AIER in their work. We will use the content from previous sections to answer the common questions about the use of AI in education and use the template from another AI4T project's 'Template for AI characterization' report to answer the questions about the use of specific AIER in teachers' job.

<figure><img src="image_16.png" /></figure>
<figure><img src="image_17.png" /></figure>

# 1. Background and Definition.

## What are teachers' jobs and what kind of Artificial intelligence (AI) support teachers need.

Fast evolution of digital technology in the information age has made today's educational work increasingly related to Artificial intelligence (AI) technology and should become an important part of today's teachers' work. However, in a survey of 140 Estonian K-12 teachers about their understanding of AI [26], the results showed that the majority of the participating teachers indicated that they had limited (47%) or average (35%) knowledge about AI. About 4% of teachers responded that they had never heard of AI before, and 8% said they were not sure what AI was. In another survey of 37 K-12 teachers who are already interesting in AI [27], most of them (80.5%) still do not have experiences with teaching AI yet. In contrast to the trend of AI becoming increasingly important in education, teachers lack understanding and trust in AI technology. This issue leads to a growing gap between the development of teachers' work and the development of modern AI technologies. Therefore, this document will focus on how AI can support teachers' jobs to help them understand and trust the use of AI in education.

Before investigating what kind of support AI can provide for teachers, it should be clear what AI is and what teachers' jobs are. In this section, we will introduce the background of AI and Artificial Intelligence Educational Resources (AIER) in the following steps:

* The definition and scope of AI in this document when we discuss AIED.

* The definition of teachers' jobs in this document when we discuss teachers' job.

* From the definitions of AI and teachers' jobs that we have chosen, the tasks in teachers' jobs that AIER needs to support.

After these steps, we will propose a template for classifying and describing AIERs based on these definitions.

### 1.1 The definition of AI in education

From a 2020 review for "AI in education" [12], we can see the general definition of AI is the ability of digital computers or computer-controlled robots to perform tasks normally associated with intelligent people, while the author defines the AI as the ability of machines to adapt to new situations, deal with emerging situations, solve problems, answer questions, device plans. This definition and the later elaboration in this review reflect the fact that the definition of AI is highly dependent on the tasks that AI needs to accomplish. This is the reason why in this chapter we limit the discussion of AI to accomplishing educational tasks.

<figure><img src="image_18.png" /></figure>
<figure><img src="image_19.png" /></figure>

From another review on the impact of AI in education [10], in addition to the above definition, the authors describe AI as the ability of computer to recognize patterns, make choices adapt to change and learn from experience. This definition is related to the different technologies and algorithms of AI. Machine learning and data mining are closely related to AI, while learning analytics is an important direction for AI technologies in education [10]. Another point worth noting is that this definition refers to the ability to learn from data, which is related to the collection and management of data by AIER.

As seen above, in many different articles discussing the impact of AI on education [4][5][12][13], the authors explore different perspectives, so much so that they define AI differently and have different scopes for AI. Therefore, in this paper we will limit the discussion of AI in AIED to AIERs that support educational tasks.

When we discuss AI to support education, we do not include those AI that can be used for general tasks in educational and non-educational scenarios, for example, AI translators that can be used to translate mathematics textbooks and translate non-educational fiction. In this document, we focus on AI that is only used for educational tasks, which we refer to as AIER.

Please note that the AI techniques used in AIER can be not only AI methods specifically designed for educational tasks, but also general AI methods, and we will not limit the definition of AIER by technology.

## 1.2 Teachers' jobs

In our opinion, for discussing the support of AIER for teachers' jobs, the definition should be highly related to the educational tasks in teachers' jobs. To complete the definition of AIER, we should point out what are the educational tasks in the work of teachers.

In order to find a general consensus about teacher jobs and to ensure that they are international and widely acceptable, we investigated the descriptions of teacher jobs from the most popular jobs search sites, such as Betterteam, Indeed, etc. [1] [2]. After filtering out the skills required for a teacher's job and focusing on the educational tasks teachers are expected to do, we summarized four main categories of activities in teacher's jobs:

Developing and issuing educational content:

* Designing lectures, notes, tests, and assignments.

* Setting homework, assignments, and tests.

Delivering personalized instruction to each student:

* Supervising students.

* Answering student questions.

<figure><img src="image_20.png" /></figure>
<figure><img src="image_21.png" /></figure>

Assessing and recording the learning process of students:

* Developing and marking examinations to measure students' progress.

* Assessing and reporting on students' academic achievements and behavior.

Supervising students' learning environment:

* Encouraging student participation and performance.

* Attending parent-teacher meetings.

Besides those, there are more detailed aspects of teachers' jobs. But for the purposes of this document, this is the definition of teachers' jobs we want to keep. Therefore, the four parts of this definition should be the main domains of which AI needs to support teachers' jobs.

## 1.3 The definition of educational task

After clarifying the definitions of AI and teachers' work, it is time now to discuss what AI can offer to support the teachers' jobs.

Firstly, AIER is the core of AIED, which can be a wide range of resources and tools. AIERs include educational applications, educational content design software, online course platforms, and so on. Nowadays, there are many different kinds of AIERs for different educational tasks. Both surveys from eLearning Industry in 2019[4] and from Datamation in 2021[5] point out four main roles of AI in education: Personalized learning programs; chat-bot as teaching assistants or software-driven tutors; 24/7 access and instant feedback for students; Learning analytic tools for teachers or educationalists. In a 2014 TeachThought survey [3], 10 roles of AI in education were proposed, and we summarized them with the previously mentioned roles and listed the main types of AIERs offered to teachers.

* Automatic generation of educational content.

* Helping to improve educational content.

* Providing personalized instruction for each student.

* Fast feedback to students.

* Assistance in monitoring students.

* Assesses students' learning behavior and learning routes.

* Records the student's learning process.

* Helping learning analysis and providing improvement.

* Facilitate communication between teachers, students, parents and relevant groups.

<figure><img src="image_22.png" /></figure>
<figure><img src="image_23.png" /></figure>

Note that when these surveys describe the role of AIER, sometimes they are talking about the technology used by AI and sometimes they are talking about the educational tasks supported by AI. This has caused some confusion, and in order to more clearly and effectively describe the support that AIER provides for teachers' job, a distinction is made in the different layers of the templates later in this document.

Second, educational tasks are at the core of teachers' work. As we mentioned before, various educational tasks belong to the four domains of teachers' work, and each of them requires the support of AIER. In a survey of 140 Estonian K-12 teachers about the challenges they face [26], the authors identified the educational tasks that teachers need the most support for by investigating the superpowers they would like to have. In another survey of 34 Indian teachers about the use of programs in education [28], the authors identified educational tasks that needed support by surveying teachers about what kind of help they expected the programs to offer. The results can be summarized as follows:

* Planning educational content;

* Preparation of materials;

* Finding misconceptions;

* Personalized instruction to each student;

* Monitoring students;

* Fast feedback to students;

* 24/7 feedback to students;

* Learning analytical skills.

It is worth noting that there is much overlap with the educational tasks we have already proposed and those supported by AIER as mentioned earlier. This suggests that the specific educational task the AIER supports is a noteworthy characteristic when describing AIER.

## 1.4 The Template of AIER

To summarize the above investigation, we propose a 5-layer template to describe the support that teachers need from AI. This template will also be used to describe the current AIER and future AI support for teachers' work. The meaning of each layer of this template is given below.

* Description: The first layer contains basic information about AIER, including the name of AIER and its company or publisher. If there is a website for the product, it will be attached to this layer.

* User: This layer declares the final user, which could be teacher, student, educator, stakeholder, and so on. As additional information, this layer will also declare who pays for the AIER or if the AIER is free for the final user.

<figure><img src="image_24.png" /></figure>
<figure><img src="image_25.png" /></figure>

AI4T
AI FOR TEACHERS

* Usage: This layer declares the main domains of teachers' jobs that this AIER supports. This layer is one or more options selected from 4 main domains: Developing and issuing educational content; Delivering personalized instruction to each student; Assessing and recording the learning process of students; Supervising students' learning environment.

* Task: This layer declares the educational task supported by this AIER.

* Technology: This layer declares the AI technology used in this AIER, which explains how this AIER support the educational task. The detail of this layer contains the AI method or algorithms used in this AIER, the related public and the personal data used.

<figure><img src="image_26.png" /></figure>

<figure><img src="image_27.png" /></figure>
<figure><img src="image_28.png" /></figure>

## 2. Now and Future

# The current situation of AIERs and the future development of AIEd.

In this section we present the main AI technologies used in AIER nowadays, describe some real-world AIER cases we have investigated with the template proposed in previous section, summarize the current top research trends in AIER, and attempt to make predictions about future trends in AIER. The details are divided into three subsections as follows:

1. Firstly, we introduce most commonly used AI techniques in AIER and the main research in AIEd. This subsection provides a brief overview of the current status of AIER.

2. Secondly, we classify and describe popular AI applications or products that are mainly used in education in K12 countries, based on the template presented in the previous section. This subsection focuses on what the AIERs available now can do to support teachers' work.

3. Finally, we summarise and analyse the current state of research within the development areas related to AIER. And based on the current trends in the popularity of AIER and AIEd, the relationship between AI technologies and educational tasks, we analyse the future trends of AIER and try to make predictions. This subsection focuses on what support AIER will provide for teachers' work in the future.

### 2.1 The main research in AIEd

From the 2020 review on AIEd [12], the authors identified three main categories of technologies most relevant to AI in education: machine learning, learning analytics, and data mining.

* Machine learning is the parsing process of knowledge discovery, model discovery, and performance improvement of task sets based on sampled datasets. Such techniques use students' past learning records and learning behaviors to recommend course choices for students, to help instructors adjust their teaching methods, for example.

* Learning analytics is the measurement, collection, analysis, and reporting of data about learners and their contexts with the goal of understanding and optimizing learning and education [17]. In AIEd, learning analytics is combined with AI technology and applied to the non-technical world of education. Its primary use is to provide early intervention or feedback to students who are at risk for learning. Artificial intelligence techniques help learning analytics gain the possibility to consider more parameters and generate faster early warning systems.

<figure><img src="image_29.png" /></figure>
<figure><img src="image_30.png" /></figure>

* Data mining is the process of extracting and discovering patterns in large data sets [18]. Data mining in AIED is the mining of educational data to identify the association rules inherent in it to provide knowledge to students or teachers. In today's AIED research, data mining is often used to discover patterns in student learning behavior, predict future student performance to help teachers make adjustments, and help teachers improve curriculum development and the design of educational systems.

* From this review, we can see that the main development trend of AIER will be around these three main directions, with different specific directions based on different specific AI technologies and algorithms.

## 2.2 The state of the art of AIER

An increasing number of AIERs are now being published around the three main directions mentioned in the last subsection. We mainly surveyed the status of AI tools and applications used in K12 classes. Based on the 4 main domains of teacher's work in our template, we grouped them into the following 4 categories:

<table><thead><tr><td>Usage</td><td>Task</td><td>AIER</td></tr></thead><tbody><tr><td rowspan="3">Developing and issuing educational content</td><td>Automatic generation for customized education content</td><td>Content Technologies, In c. (CTI) [19]</td></tr><tr><td>Recognize the speech and generate the text</td><td>Nuance [20]</td></tr><tr><td>Help with the collection of materials for designing educational content</td><td>Not really one for now</td></tr><tr><td rowspan="3">Assess and record the learning process of students</td><td>Recording students' learning traces</td><td>Thinkster Math [21], Cognii[22]</td></tr><tr><td>Evaluating students' learning behavior</td><td>Cognii</td></tr><tr><td>Generating the reports based on learning analysis</td><td>Cognii</td></tr><tr><td rowspan="2">Delivering personalized instruction to each student</td><td>AI assistance for answering students' questions</td><td>Cognii, Brainly[24], Georgia Tech(Jill Social Agent) [23]</td></tr><tr><td>AI assistance for teachers to provide personalized instruction</td><td>Thinkster Math, Cognii</td></tr><tr><td rowspan="3">Supervising students' learning environment</td><td>Virtual Reality for engaging students</td><td>ClassVR[25]</td></tr><tr><td>Assistance in monitoring classroom.</td><td>ClassVR</td></tr><tr><td>Chatbot between teachers, students, parents or relevant groups.</td><td>Not really one for now</td></tr></tbody></table>

<figure><img src="image_31.png" /></figure>
<figure><img src="image_32.png" /><figcaption>**Table 1 : the map of the AIER support for 4 main domains of teachers' job**</figcaption></figure>

In Table 1, we list the various types of educational tasks supported by the investigated AIERs.

The first column shows the usage, i.e., which major area of the teacher's work this educational task belongs to. (we identify it with different colors); the second column is the task in our proposed template, i.e., the specific content of the educational task; the third column is example of AIERs that support this educational task.

We describe these AIERs according to the template proposed in the previous section, as follows:

<table><thead><tr><th colspan="3">Content Technologies, Inc. (CTI)[19]</th></tr></thead><tbody><tr><th scope="row">Description</th><td colspan="2">Content Technologies, Inc(CTI) provide software which can analyze course materials, textbooks, syllabus, and other resources to create textbooks, study guides, and multiple-choice tests.(https://contenttechnologiesinc.com/)</td></tr><tr><th scope="row">User</th><td>Used by teacher</td><td>Payed by educational institution</td></tr><tr><th scope="row">Usage</th><td colspan="2">Developing and issuing educational content.</td></tr><tr><th scope="row">Task</th><td>Automatic generation for customized education content</td><td>Cram101:overview study guides<br>JustTheFacts101:highlight important information<br>NursingEd101:find the critical information</td></tr><tr><th scope="row">Technology</th><td colspan="2">Deep learning NLP model for text generation. This model is called Jasper.AI based on GPT-3(ussler, Frederik (July 21, 2020). "Will GPT-3 Kill Coding?". Towards Data Science. Retrieved August 1, 2020.)<br>The education content generation without users' personal information but based on the users' selected textbook and CTI's library.</td></tr></tbody></table>

<table><thead><tr><th colspan="2">Nuance[20]</th></tr></thead><tbody><tr><th scope="row">Description</th><td>Nuance is speech recognition software used by students and faculty. It help for students who struggle with writing or have accessibility needs<br>(https://www.nuance.com/dragon/industry/education-solutions.html.)</td></tr></tbody></table>

<figure><img src="image_33.png" /></figure>
<figure><img src="image_34.png" /></figure>

<table><tr><td>User</td><td>Used by student</td><td>Payed by educational institution</td></tr><tr><td>Usage</td><td colspan="2">Developing and issuing educational content.</td></tr><tr><td>Task</td><td colspan="2">Recognize the speech and generate the text and correctly read them back.</td></tr><tr><td>Technology</td><td colspan="2">Speaker-dependent speech recognition called Dragon NaturallySpeaking. (https://www.nuance.com/asset/en_us/collateral/dragon/whitepaper/wp-helping-students-reach-their-full-potential-en-us.pdf)<br/>The users' voice is collected and used for voice profile.</td></tr></table>

<table><tr><td colspan="3">Cognii [21]</td></tr><tr><td>Description</td><td colspan="2">Cognii is a provider of AI based educational technologies. Their products include a virtual learning assistant, for K-12 and higher education institutions, as well as corporate training organizations. (https://www.cognii.com/)</td></tr><tr><td>User</td><td>Used by student, teacher.</td><td>Payed by educational institution</td></tr><tr><td>Usage</td><td colspan="2">Delivering personalized instruction to each student.<br/>Assessing and recording the learning process of students.</td></tr><tr><td>Task</td><td colspan="2">Cognii Virtual Learning Assistant: answering student questions.<br/>Cognii Virtual Teaching Assistant: evaluating students' responses to questions to assess deep understanding and critical thinking.<br/>Cognii EdTech Analysis: generating report of knowledge gaps and mastery overview at the class, student, problem or specific concept level.</td></tr><tr><td>Technology</td><td colspan="2">For Cognii Virtual Assistant, Cognii use Conversational EdTech based on Personalized Adaptive Learning which include NLP and Cognitive Computing. There are some feature about this NLP algorithm but no linked public.<br/>For assess the user and learning analysis, Cognii use data mining and machine learning on the users' behaviors, but does not state which algorithms they use.</td></tr></table>

<figure><img src="image_35.png" /></figure>
<figure><img src="image_36.png" /></figure>

<table><thead><tr><th colspan="3">Thinkster Math [22]</th></tr></thead><tbody><tr><th scope="row">Description</th><td colspan="2">Thinkster Math is an online application for students and their parents. The application trace the steps of students learning on math problems and match the teacher to students based on their learning path.<br>(https://hellothinkster.com/online-math-tutor.html.)</td></tr><tr><th scope="row">User</th><td>Used by student.</td><td>Payed by students' parents</td></tr><tr><th scope="row">Usage</th><td colspan="2">Delivering personalized instruction to each student.<br>Assessing and recording the learning process of students.</td></tr><tr><th scope="row">Task</th><td colspan="2">Learning analysis for the learning path based on the trace of users' behaviors;<br>Delivering personalized courses and match the teacher based on the learning path.</td></tr><tr><th scope="row">Technology</th><td colspan="2">Automatic Information Processing Model for Learning analysis. No more detail;<br>Courses and teachers are recommended for students, the parents select the teacher.<br>No more detail and no sure the ranking is based on result of learning analysis.<br>The users' personal data and trace of behaviors are collected and used for learning analysis.</td></tr></tbody></table>

<table><thead><tr><th colspan="3">Georgia Tech(Jill Social Agent)[23]</th></tr></thead><tbody><tr><th scope="row">Description</th><td colspan="2">Georgia Tech proposed a Jill Watson AI framework. The first produce is Jill Social Agent, which is a teaching assistant using their own custom "knowledge base" to answer basic student questions 24/7.(https://emprize.gatech.edu/.)</td></tr><tr><th scope="row">User</th><td>Used by student</td><td>Payed by educational institution</td></tr><tr><th scope="row">Usage</th><td colspan="2">Delivering personalized instruction to each student.</td></tr><tr><th scope="row">Task</th><td colspan="2">Answering student questions 24/7.</td></tr></tbody></table>

<figure><img src="image_37.png" /></figure>
<figure><img src="image_38.png" /></figure>

<table><tr><td>Technology</td><td>Jill Watson AI framework, similarity matching and knowledge-based adaptation(Wang Q, Jing S, Camacho I, et al. Jill Watson SA: Design and evaluation of a virtual agent to build communities among online learners[C]//Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems. 2020: 1-8.)<br/>The users' personal data (location, timezone, hobby, class schedule, and so on.) are collected and used for matching.</td></tr></table>

## Brainly for Schools and Teachers[24]

<table><tr><td>Description</td><td colspan="2">Brainly is an online space that offers a supportive message board setting for peer-to-peer learning and homework help. The part of Brainly for Schools and Teachers expand classroom into online earning environment.<br/>(https://brainly.com/schools-and-teachers?source topbar.)</td></tr><tr><td>User</td><td>Used by student, teacher</td><td>Payed by student</td></tr><tr><td>Usage</td><td colspan="2">Delivering personalized instruction to each student.</td></tr><tr><td>Task</td><td colspan="2">Recommend the answer for student questions.</td></tr><tr><td>Technology</td><td colspan="2">Content based and collaborative filtering methods. No more details on their website.<br/>The users' input content and behavior trace will be used for the recommendation. Beside that, the users' personal data (email address, age, and phone number) are collected and used for this online platform.</td></tr></table>

## ClassVR[25]

<table><tr><td>Description</td><td>ClassVR is a fully integrated, classroom-ready device for provide the virtual reality courses to students by teacher.</td></tr></table>

<figure><img src="image_39.png" /></figure>
<figure><img src="image_40.png" /></figure>

<table><tr><td colspan="3">(https://classvr.com)</td></tr><tr><td>User</td><td>Used by student, teacher</td><td>Payed by educational institution</td></tr><tr><td>Usage</td><td colspan="2">Supervising students' learning environment</td></tr><tr><td>Task</td><td colspan="2">Virtual reality for engaging student participation and performance.<br/>Assistance in monitoring classroom.</td></tr><tr><td rowspan="3">Technology</td><td colspan="2">ClassVR provide ClassVR Headsets as a VR hardware with VR and AR technology.</td></tr><tr><td colspan="2">ClassVR provide simple-to-use teacher controls software to streamline learning with VR & AR. The software helps to manage and monitor lessons.</td></tr><tr><td colspan="2">There are no public papers related to the technology on their website. The personal data will not be used in ClassVR.</td></tr></table>

As can be seen from the various AIERs mentioned above, AIERs currently cover almost all four main parts of teachers' work, yet there are still some gaps between them. It is worth noting that in addition to AIERs specifically designed to fulfil educational tasks, generic AI technologies also bring some support to teachers' work, such as in Table 1 for 'Helping to collect materials for designing educational content', generic AI search technologies can support this task, as some AI generic tools can also provide support to teachers in collecting materials when writing educational content.

## 2.3 The research in AIEd and the future of AIER

In the previous section, we surveyed and summarized the AIER applications or tools that are now available on the market and proposed what should be the future development of AIERs by analyzing the relationship between existing AIERs and the educational tasks proposed in our survey. In order to further analyze the trends in the future development for AIER, it is important not only to investigate the resources that are being developed now, but also to determine what kind of resources could be available in the future.

From the 2019 report on "A Year of Artificial Intelligence in K-12 Education"[8], the 2020 review on AIEd [12] and 2021 review on AIEd [13], we can see that AI algorithms and systems in education are gaining interest year by year. In the 2020 review [12], it is mentioned that the number of papers published from

<figure><img src="image_41.png" /></figure>
<figure><img src="image_42.png" /></figure>

Web of Science and Google scholar under the topics "Artificial Intelligence" and "Education" have increased from 60 papers in 2010 to 200 papers in 2019 [12]. Besides the three main research directions, which are Machine learning, Learning analytics, and Data mining, there are several new research trends in the AIED.

From the review on AIED in 2021 [13], the authors additionally point out several popular research directions in the field of AIED:

* Integration with the development of emerging computer technologies: increase the integration of AI with the development of emerging computer technologies, such as robot control, sensing devices, quantum computing, wearable devices, 5G wireless communication technologies and mobile communications [14]. These technologies provide new opportunities and chances for the application of artificial intelligence in learning and instructional design; facilitating student learning and creating an environment where teachers can teach more effectively, such as E-learning [15].

* Increasing the use of non-formal education: The development of AI technologies has facilitated the development of non-formal education. Therefore, AI technology is also a noteworthy development in non-formal education. For example, the management of students' courses and assignments outside the school, distance learning as mentioned before, etc.

* AI research related to specific disciplines: AI technologies have a natural relevance to education in some specific disciplines because of their natural association with technologies related to data processing. Examples include research on the intersection between AI, mathematics education (ME), and computational thinking (CT); the processing of medical graphics and other related developments in medical education curricula; and so on.

From the above report and review, we can see that the current research direction in AIED complements and develops the AIER that is already available today. Following the four main domains of teachers' work proposed in this paper, we can fill in table 2 with the state of art of AIED research directions according to the relevant application scenarios and related AI technologies:

<figure><img src="image_43.png" /></figure>
<table><caption>Table 2: the map of the AIER research technologies support for 4 main domains of teachers' job</caption><thead><tr><th>Usage</th><th>Task</th><th>Related Research technology</th></tr></thead><tbody><tr><td rowspan="4">Developing and issuing educational content</td><td>Automatic generation for customized education content</td><td>Academic data mining and pattern mining</td></tr><tr><td>Recognize the speech and generate the text</td><td>Speech recognition</td></tr><tr><td>Help with the collection of materials for designing educational content</td><td>Machine learning. Recommendation System</td></tr><tr><td>Smart lesson system</td><td>Machine learning and data mining</td></tr><tr><td rowspan="4">Assess and record the learning process of students</td><td>Evaluating papers and exams</td><td>Image recognition, computer vision, prediction system</td></tr><tr><td>Recording students' learning trace</td><td>Learning analysis, Data mining</td></tr><tr><td>Evaluating students' learning behaviour</td><td>Machine learning, data mining</td></tr><tr><td>Generating the reports based on learning analysis</td><td>Machine learning, learning analysis with AI</td></tr><tr><td rowspan="2">Delivering personalized instruction to each student</td><td>AI assistance for answering students' questions</td><td>Adaptive learning</td></tr><tr><td>AI assistance for teachers to provide personalized instruction</td><td>Adaptive learning and personalized learning</td></tr><tr><td rowspan="4">Supervising students' learning environment</td><td>Virtual Reality for engaging students</td><td>Virtual Reality, Augmented Reality</td></tr><tr><td>Assistance in monitoring classroom.</td><td>Face recognition, speech recognition</td></tr><tr><td>Chatbot between teachers, students, parents or relevant groups.</td><td>Natural language processing</td></tr><tr><td>Online and mobile education</td><td>Edge computing, virtual personalized assistants, real-time analysis</td></tr></tbody></table>

In Table 2, we list the various types of educational tasks supported by the technologies associated with the current top trends in AIEd research. The first column is the use in our proposed template, i.e., which main domains of teacher work this educational task belongs to. (We identify them with different colors); the second column is the task in our proposed template, i.e., the specific content of the educational task; and

<figure><img src="image_44.png" /></figure>
<figure><img src="image_45.png" /></figure>

the third column is the technology in our proposed template, i.e., the relevant technology of AIEd research that supports this educational task.

It is worth noting that Table 2 is almost identical to Table 1, which confirms what we know about AIER. And by comparing Tables 1 and 2, and taking into account the educational tasks we presented in the previous section, we can see that AIER will move in three main directions in the foreseeable future:

1. **Filling the gaps.** As we can see in Table 1, the existing AIER is still not enough, there are still gaps in some educational tasks. And in Table 2, there is already some research looking at solutions to fill these gaps.

2. **Better performance.** For the same educational task, there are different AIERs in Table 1 and different research trends in Table 2. From this fact we can see that they will compete with each other and become more effective as technology develops. For example, online answers to students' questions will be faster and more accurate in the future.

3. **More extensive.** In Table 1, there are different research trends related to the same AIER. This may mean that in the future, a single AIER will have a wider range of functions and effects. For example, in Table 1 Thinkster Math and Cogii provide personalized instruction for each student by assessing and recording the results of learning analytics on the student learning process. Their related research has involved learning analytics on student behaviour, improving educational content based on student feedback and so on. The multiple research directions will allow the future of AIER to be broader and not limited to one major part of a teacher's work. The various parts of a teacher's work are coordinated by AI, and in the future, the support AIER provides to teachers should be more extensive and accessible.

<figure><img src="image_46.png" /></figure>
<figure><img src="image_47.png" /></figure>

AI4T
AI FOR TEACHERS

## 3. Advantages and Concerns

### Why should teachers use AIER in their work and why they did not use.

Artificial Intelligence in Education (AIEd) is one of the emerging areas in education today. Although it has been around for about 30 years, it is still unclear to teacher how it can be fully leveraged to influence teaching and learning and how it can be used to its pedagogical advantage on a larger scale. In this section, we will introduce the impact of using AIER in the work of teachers in two sides:

* In the first subsection, we will follow the conclusions of the previous section on what AIER can do now and in the future, summarizing the advantages that AIER offers teachers in helping to accomplish educational tasks.

* In the second subsection, on the opposite side, we will summarize teachers' concerns and worries about using AI in their work.

#### 3.1 Advantage of using AIER in the work of teachers

From the 2016 report on the "Centennial Study of Artificial Intelligence"[7] and the 2019 report on "A Year of Artificial Intelligence in K-12 Education"[8], we can see an increasing use of AI in education in general. A survey of K12 education in Europe in 2021 reported [8] that 72% of countries are interested in AIEd. Over 47% of learning management tools will be AI-enhanced within the next three years, according to the survey by eLearning Industry [4]. From these reports, we can see a gradual increase in the use of AI in education and a deepening of its importance.

From the 2016 report [7] and another 2020 report [10] on "Advantages and Disadvantages of Artificial Intelligence", it appears that AI improves our lives through the following advantages:

* AI can automatically and efficiently complete predefined tasks to save time and other cost.

* AI can be programmed, self-written, self-modified.

* AI can be deployed easily.

* AI can support the work and give feedback fast and 24/7.

* AI can support the online work and digital documents.

In addition to these advantages, the 2019 report [8] lists powerful and immersive applications of AI in education in the near future. The authors cite the role of data mining, learning analytics, and machine learning in education and the mutually reinforcing developments when combined with each other. And the development of these technologies will help enable innovation in education beyond its current state. AI can
<figure><img src="image_48.png" /></figure>

help with the lifelong length learning recording outcomes, ongoing assessment and learning companions.
In another 2021 report on the current state of AI in education in China [11], it is stated that AI offers three
irreplaceable and revolutionary advantages: the first is the replacement of repetitive educational labor with
AI, consisting mainly of the recording and assessment of student learning behavior; the second is the help
in personalizing education, with developing and issuing educational content across the whole learning path
and deliver personalized instruction to students; the third is to break through the boundaries of schools,
classes and subjects and to rebuild the learning environment through online education and other AI
technology, helping teachers to better supervising students' learning environment. In the 2021 review on
AIED[13], the advantage of AI in the education sector includes: easy and fast access to information; distance
education and learning; delivering personalization of education; globalization of knowledge provided by AI
translators; assistant monitoring of students' learning behavior; environmental protection due to digital
education; AI making information more accessible to special (disabled) students than it would be otherwise;
reduction of errors due to repetitive tasks.

From the above report, we can see that the advantages of using AIER to support teachers in their work, classified according to the four main parts of teachers' job, are as follows.

## Developing and issuing educational content:

* As we mentioned in the support from Content Technologies, Inc. (CTI) and other similar AIER, the AI helps to automatically generate the course materials and generate the quiz or test based on the courses that the teacher prepared.

* AI helps teachers access information easily and quickly.

* AI helps the teacher to design the educational content based on the whole learning path of student.

* AI helps the teacher to improve the educational content based on the result of learning analysis or pattern mining from other educational resources.

## Assessing and recording the learning process of students:

* AI helps to trace the students' learning behaviors when students use the applications or learn online.

* AI-driven lifelong learning companions and AI-enabled record of lifelong learning achievements.

* AI helps to automatically correct and measure the test result of students based on the teachers' setting, which will save the time for teachers' evaluating and reduce the error.

* AI-enabled continuous assessment.

* Learning analysis will be easier with AI technologies, such as machine learning or data mining.

## Delivering personalized instruction to each student:

<figure><img src="image_49.png" /></figure>
<figure><img src="image_50.png" /></figure>

* AI assistance for answering students' questions.

* AI assistance for teachers to provide personalized instruction.

## Supervising students' learning environment

* Assistance in monitoring classroom.

* Online learning environment.

* Chatbot between teachers, students, parents or other relevant groups.

As can be seen from the above, the use of AIER for a variety of educational tasks in all major aspects of teachers' work can help teachers.

## 3.2 Teachers' concerns about using AIER in their work.

There are two sides to everything, and the use of AI in education certainly brings some drawbacks, not just advantages and benefits. Note that, here we do not discuss the disadvantages of AI due to technology and common applications, such as the increase in education costs due to expensive devices, technology addiction, negative health effects due to incorrect use, negative impact on social life due to students' dependence on the Internet, etc. [10][13]. We focus on the concerns and worries that come with teachers' perceptions and understanding of AI, which is a major issue that the AI4T project needs to address.

In a survey of 140 Estonian K-12 teachers, their concerns about the use of AI in education were summarized and analyzed. Participants' perceptions of using AI in education were negative in three ways: more than half of the participants (55%) thought the biggest problem with using AI was the extra effort required to learn how to use AIER; 34% did not believe AI could perform tasks without errors; and 11% thought AI could replace teachers' jobs and lead to job loss. It is worth noting that even on the positive side, 41% of the participants were of the opinion that their jobs require human involvement, and because of their distrust of AI, they do not think their jobs will be replaced.

In a 2022 study on concerns of teachers of using AI in education [29], the author conceptualized teachers' concerns about a new pedagogical innovation into 5 categories:

* Consequence (C): The concerns about the impacts on student learning and teacher professional development.

* Evaluation (E): The concerns about the worthiness and possibility of introducing the AI.

* Information (I): The concerns about the involved tasks and demands related to using the AI in education.

* Management (M): The concerns about the practical issues related to the actual implement of using AI in education.

<figure><img src="image_51.png" /></figure>
<figure><img src="image_52.png" /></figure>

* Refocusing (R): The concerns about the further improvement and enhancement.

The participants of this study were 38 Hong Kong high-school teachers who attended a 2-hour face-to-face AIEd induction seminar voluntarily. The result shows that the teachers have encouraging concerns about C and R but very strong discouraging concerns about E, I and M.

These two studies showed that teachers had three main concerns about using AIER in education: first, they were concerned about the difficulty of learning to use AIER and the unrewarded effort and time spent, second, they questioned the effectiveness and necessity of using AIER, and third, they suspected that their role as teachers would be replaced or changed by using AIER.

In addition to these concerns, from the review [11][12], the authors also point out that some of the problems with the use of AI in education from the viewpoint of educational institutions include: confusion in student management that may result from distance education; inequality in non-formal education that may result from the conditions of different devices; data security issues for students and teachers; consistency and laziness of teachers in creating educational literature due to the assistance of AI tools, and loss of creativity.

It can be seen that the sources of these above concerns and worries about the use of AIER in education can be summarized as follows.

* The cost of learning using AIER

* Lack of knowledge about AI

* Concerns about the transparency of AIER

* Concerns about the security of AIER data

* Concerns about the decision level teachers have when using AIER

It is worth noting that the first two points are the main issues that the AI4T project is trying to address, while the last three points are described and summarized in the AI4T project's report on "Template for AI characterization" specifically for these three features of AIER for teachers and educational practitioners.

<figure><img src="image_53.png" /></figure>
<figure><img src="image_54.png" /></figure>

## 4. Usage and Metric

### How to use and select the AIER for teacher in education.

In the previous section we could see that there are both advantages and disadvantages to using AIER in education and many teachers have concerns as a result. These concerns may be reduced after receiving training on knowledge of AI in education. In addition to an introduction to what AI is, educating teachers on how to use and select AI in education is an important part of this training. In this section, we will introduce this essential part of training for teachers in two subsections:

* In the first subsection, we will follow up on what AIER can do, describing scenarios in which teachers use AIER in their educational work.

* In the second subsection, we will introduce the general metrics for measuring the AI and recommend teachers several metrics for selecting the AIER in their work. While metrics that measure AI performance are important, there are a number of metrics that measure AI trustworthiness that are worth considering.

#### 4.1 Scenarios of using AIER

In this subsection, we present different scenarios in which teachers are using AI in their educational work. As we mentioned before the use of AIER is growing in classrooms across Europe and AIER is being used in different ways to support educational tasks. From the Usage layer in our proposed templates, we can see that a common distinction to AIER is made between 'student-oriented', 'teacher-oriented' and 'system-oriented'. For these different AIERs, teachers can use them in different scenarios and get support for their own work.

The scenarios in which AI tools are used in education can be divided into the following four groups:

**Teaching Support:** This type of scenario is the use of AI applications to teach students in education. AIERs in this scenario are designed for students and also generally have extended features specifically designed for teachers. First, using such AIERs can help teachers share the burden of teaching a particular course. Second, when using such AIERs, teachers can gain better control over students' learning in the course through features such as class management and monitoring of students' learning progress. AIERs in such scenarios support the teachers' jobs including delivering personalized instruction to each student and assess and record the learning process of students. The common examples of Teaching Support include:

* Intelligent tutoring system, such as Thinkster Math[22];

<figure><img src="image_55.png" /></figure>
<figure><img src="image_56.png" /></figure>

* Dialogue-based tutoring systems, such as Georgia Tech[23], Brainly[24];

* Language learning applications, such as Doulingo[33].

**Learning Support** - This type of scenario uses AIERs to support student learning. Unlike the previous type of scenario, AIERs in this type of scenario are designed not only for students, but also for teachers or to support learning environments. Some of the AIERs used in these scenarios help students and teachers communicate in different situations, such as distance learning or learning in a VR environment. Other AIERs help address difficulties students encounter in communicating with teachers or in learning environments. AIERs in such scenarios support teachers' jobs, including tasks such as supervising students' learning environment. The common examples of Learning Support include:

* Exploratory learning environments, such as ClassVR[25];

* AI-supported collaborative learning, such as 360Learning[34];

* Learning analytics to aid learning, such as learning path analysis and assistive tools[36]

**Teacher Support** - This type of scenario uses AIER to directly support teachers. In this type of scenario, AIERs are designed for teachers to support their work. This type of AIER is typically designed to support the teacher's direct work, for example helping the teacher with repetitive tasks such as lesson plan preparation and grading assignments. AIERs in this type of scenario support teachers' work with tasks such as assessing students' learning process, developing and issuing educational content. The common examples of Teacher Support include:

* AI teaching assistants, such as Cognii[21];

* Pedagogical resource recommendation, such as Content Technologies, Inc. (CTI)[19].

* Learning analytics to aid teacher, such as descriptive cards of student characters [37].

* Diagnosing learning difficulties for warning teachers, such as ERA (Early Risk Alert In A Distance Learning)[38];

**System Support** - This type of scenario uses AI to support diagnostic or system-wide planning. In this type of scenario, AIERs are designed for schools or government educational institutions to support teachers from a global planning perspective. This type of AIER is usually designed for educational support of whole information systems, such as educational data mining systems, and diagnostic learning difficulty systems. Although these AIERs are mainly used by educational institutions such as schools, their output is used to support the work of teachers. AIERs in this type of scenario support the work of teachers, such as Assess

<figure><img src="image_57.png" /></figure>
<figure><img src="image_58.png" /></figure>

and record the learning process of students and supervising students' learning environment. The common examples of System Support include:

* Educational data mining for resource allocation, such as Wisconsin Information System for Education (WISE) [35];

* Diagnosing learning difficulties, such as DEWS (Dropout Early Warning System); Student Mapping Tool (SMT) [35].

* Organization for the educational activities, such as the BacAnalytics.

From the above scenarios, it is clear that teachers can use AIER to support their work, regardless of the type of user it is designed for. What teachers need to know is how to approach and use AIERs in these scenarios. In the next subsection we will describe how teachers should choose to use AIERs when faced with AIERs that perform the same function in the same scenario.

## 4.2 Measurements of AIER

In this subsection, we will present some measures of AIER performance and trustworthiness. Note that, in this document, we will not compare or rank the AIER nowadays, but to help the teachers to know about what measurements they should consider when select the AIER in the educational work.

From [30], the authors state that, in general, there are three groups of metrics that measure the performance of AI in education. Metrics that measure the generic performance of the algorithm (GAM), performance metrics for completing the intended task (CTM), and metrics for generic cost of using AI (GCM). The first two of these three groups of indicators, GAM and CTM, are functional which means they are related to the specific function of the AIER, while the last group, GCM, is non-functional and not related to the function of the AIER. The following is a detailed explanation of these three groups of metrics:

* Metrics that measure the generic performance of the algorithm (GAM): The GAM is a group of functional metrics that are commonly used in AI algorithms to measure the performance of the algorithm. For example, accuracy is the most common and important GAM for classification machine learning algorithms, in addition to Confusion Matrix, precision, recall, F-1 score AU-ROC are all common GAMs for classification algorithms [32]. After specifying the algorithm used in AIER, it is possible to find the GAM for this algorithm and use the common metric in GAM to measure the performance of this AIER.

* Metrics for completing the intended task (CTM): The CTM is a group of functional metrics that, unlike the GAM, which has a clear definition and a uniform standard, is defined according to the tasks of the AIER. For example, for an early warning system to detect learner problems at risk of failure, accuracy as a GAM is important, however, how early alerts are generated is also an

<figure><img src="image_59.png" /></figure>
<figure><img src="image_60.png" /></figure>

important indicator. An early warning system that is less accurate but produces alerts earlier may be better. In this case, a CTM metric such as harmonic mean (HM) [30] can be found which expresses the relationship between accuracy and time metrics. The CTM is closely related to the task, and once a technical or educational task has been assigned to an AIER, it is possible for the technical staff involved to help the user find a suitable CTM to measure the performance of that AIER.

* Metrics for generic costs for using AI (GCM): The GCM is a group of non-functional metrics which is not related to the function or technology of AIER but the cost for using this AI tools in education. The GCM includes the time cost of running the AIER, the memory space, and even the price of purchasing the AIER, etc. The selection and the important level of GCM are different from user to user and from use case to use case. After specifying the user and the education environment, user could select the GCM by the requirement from themselves.

As can be seen from the above explanation of the three groups of metrics that measure the performance of an AIER, functional metrics require the help of a technician to find them, and non-functional metrics can be chosen by the user. All these metrics help the user to select an AIER according to its performance.

Besides the performance of the AIER, the trustworthiness of the AIER is an important criterion for the selection of an AIER [34]. Firstly, it is worth pointing out that AIER must be legal and ethical when used in the field of education. Subject to human agency and oversight, subject to diversity, non-discrimination and fairness, and subject to humanity are prerequisites for using AIER in education. Non-compliant AIERs are out of the scope of our discussion. Secondly, subject to the above conditions, there are several measures of the trustworthiness of the AIER based on the guidelines on the use of AI for Educators [31]:

* Technical robustness and security: This metric measures the robustness and security of the AI technologies and algorithms used in AIER, including resilience to attacks, security and general safety, accuracy, reliability and repeatability. Specific values of this metric can be presented to teachers by technicians who find algorithm-related metrics based on the algorithms used in the AIER.

* Privacy and data management. This metric measures AIER's use and management of private data, including respect for privacy, what data is used, the quality and integrity of the data, and the encrypted management of and access to the data. In Europe, the use of private data is subject to the provisions of the General Data Protection Regulation (GDPR). In addition to this regulation, for AIER private data can be processed in different ways, such as anonymization, encryption and denial of access. The different ways in which data is handled provide different levels of data security. This metric should be published by the supplier of AIER to teachers.

* Transparency: This metric measures the level of understandability of how the AIER works, including traceability, interpretability, and communication. This indicator is more related to the educational task that the AIER is intended to fulfil. For example, if the educational task is assessed

<figure><img src="image_61.png" /></figure>
<figure><img src="image_62.png" /></figure>

by the student's learning behaviors, the internal mechanism of this AIER is about which learning behaviors are selected and which parameters are used to generate the final report. If the final assessment report given to the teacher shows these parameters, or if the introductory website of the AIER explains the internal mechanisms of this AIER, this will help the teacher to understand the possibilities, limitations and risks of this AIER in education. This metric depends on what information the supplier of the AIER has published to the teacher.

As can be seen, these metrics are either interpreted by technicians or depend on public announcements by the AIER supplier. It is difficult for teachers to measure the trustworthiness of an AIER and choose one independently without other help. Therefore, this kind of help should be provided to them. This is one of the issues that we are addressing in the AI4T project. Please note that these three metrics are also described in dedicated layers in the template from AI4T project's 'Template for AI characterization' report to help teachers understand AIERs and make their choices.

<figure><img src="image_63.png" /></figure>
<figure><img src="image_64.png" /></figure>

# 5. Questions and Template

## What teachers want to know when they use AI tools.

In this section, we will summarize the problems that teachers want to know when using AIER in their work. We will answer the common confusion about the use of AI in education through the content of this article. At the same time, we have another AI4T project's 'Template for AI characterization' report to answer the confusion of using specific AIER.

### 5.1 The common questions about AI in education

After the survey and analysis of AIEd, we summarize and answer the following questions which are what teachers want to know and worry about AI in education for common AIER:

#### What is AI in education?

In the Section Background and Definition, we denote that AI in education is the ability of a digital computer or computer-controlled robot to perform educational tasks normally associated with teachers or other related educational group through digital information technology. The core is the educational task it supports and the digital information technology it use. In this document, we limit the definition into AIER (Artificial Intelligence Educational Resources).

#### How AI help my work?

The teachers' work includes four domains, which are developing and issuing educational content; delivering personalized instruction to each student; assessing and recording the learning process of students; supervising students' learning environment. For each domain, there are several educational tasks. AI will support teacher with different educational tasks and help teacher in four group of scenarios: Teaching Support, Learning Support, Teacher Support, System Support. There are different scenarios in which AI tools help teachers with different aspects of their work and there are different ways in which teachers can use AI tools or decide on the output of AI tools.

What kinds of AIERs are teachers able to use now and in the future to get support? And how do they work?

<figure><img src="image_65.png" /></figure>
<figure><img src="image_66.png" /></figure>

In the Section Now and Future, we have classified, described and analyzed the current common AIER according to the proposed template. Based on the comprehensive analysis of current AIER and AIEd research, we try to predict the development direction of AIER in the next five years.

## How should I choose and trust AI tools?

In the Section Usage and Metric, we introduce the scenarios for teacher to use AIER in their work and we recommend several measurements in terms of performance and trustworthiness of AIER. For performance, we present metrics to measure the generic performance of the algorithm (GAM), performance metrics to complete the intended task (CTM), and generic cost metrics to use the AI (GCM). And we point out which parts of these metrics require technical staff support to present to faculty. For trustworthiness, we discuss the robustness and security of the technology, privacy and data management, and transparency. And we point out the support that these metrics need from AIER providers and technical staff to present to teachers.

## 5.2 Template for AI characterization

<figure><img src="image_67.png" /><figcaption>Figure 1: The relation between this paper and 'Template for AI characterization' report</figcaption></figure>

For better describe and classifying the AIER, in the AI4T project, we design a 3-class 7-layer template for AI characterization.

* First class answers how and why use this AIER: Usage layer to describe how the AI tool can help the teacher, and Decision Level layer to describe how the teacher interacts with the AI tool, and the proportion of the final outcome that is attributed to the teacher.

<figure><img src="image_68.png" /></figure>
<figure><img src="image_69.png" /></figure>

* Second class answers the teacher's concerns about credibility: Personal Data, Data security and Transparency layer to describe the input data used and the secure handling of that data, to make it clear that AI tool s demonstrate the transparency and to help teachers trust in the security of AI tools.

* Third class answers how the AIER work: Technology and Algorithm layer to describe the approaches the AI tools used to reach its purpose and to help teachers understand how it works.

After the survey and analysis, we summarize the questions below are what teachers want to know about a particular AI tool. In parentheses after each question is the corresponding layer of the template proposed "Template for AI characterization" report, which can be used as a reference to answer this question.

* What is this AI tool? (Basic Information)

* Who is this AI tool for? (Usage)

* What aspect of my work can this AI tool help? (Usage)

* In what scenarios can I use this AI tool? (Usage, Technology)

* How much decision power do I have in using this AI tool? (Decision Level)

* What kind of personal data will this AI tool use? (Personal Data)

* Is this AI tool safe to use? Is it reliable? (Data security, Transparency)

* Is this AI tool trustworthy? Where can I get the relevant supporting evidence? (Transparency)

* What are the advantages of this AI tool when I use it in my work? (Technology, Algorithm)

* What AI technology does this AI tool work through? (Technology, Algorithm)

The template could be fulfill based on the information from the AIER public, then easily help teacher to get the answer to the questions above. More detail of this template could be found in the 'Template for AI characterization' report.

<figure><img src="image_70.png" /></figure>
<figure><img src="image_71.png" /></figure>

## 6. References

[1] Betterteam "Teacher jobs Description", September 2022, Betterteam, https://www.betterteam.com

[2] Indeed "Teacher jobs Description: Top Duties and Qualifications", September 2022, Indeed, https://www.indeed.com

[3] TeachThought "10 Roles For Artificial Intelligence In Education", July, 2014, TeachThought, https://www.teachthought.com

[4] Amit Gautam "Role Of Artificial Intelligence In Shaping The Future Workforce", July 7, 2019, eLearning Industry, https://elearningindustry.com

[5] Sarah Hunt "How Al is Being Used in Education", October 10, 2021, www.datamation.com

[6] Copeland, B.J.. "Artificial intelligence". Encyclopedia Britannica, 24 Aug. 2022, https://www.britannica.com/technology/artificial-intelligence. Accessed 20 September 2022.

[7] Stone P, Brooks R, Brynjolfsson E, et al. Artificial intelligence and life in 2030: the one hundred year study on artificial intelligence[J]. 2016.

[8] Touretzky D, Gardner-McCune C, Breazeal C, et al. A year in K-12 AI education[J]. AI Magazine, 2019, 40(4): 88-90.

[9] European Schoolnet . "Artificial Intelligence Role in K12 Education: Agile Collection of Information", 2021, Brussels, Belgium.

[10] Bhbosale S, Pujari V, Multani Z. Advantages And Disadvantages Of Artificial Intellegence[J]. Aayushi International Interdisciplinary Research Journal, 2020, 77: 227-230.

[11] Ouyang F, Jiao P. Artificial intelligence in education: The three paradigms[J]. Computers and Education: Artificial Intelligence, 2021, 2: 100020.

[12] Chen L, Chen P, Lin Z. Artificial intelligence in education: A review[J]. Ieee Access, 2020, 8: 75264-75278.

[13] Nalbant K G. The importance of artificial intelligence in education: a short review[J]. Journal of Review in science and engineering, 2021, 2021: 1-15.

[14] Hwang G J, Tu Y F. Roles and research trends of artificial intelligence in mathematics education: A bibliometric mapping analysis and systematic review[J]. Mathematics, 2021, 9(6): 584.

[15] Nagao K. Artificial intelligence in education[M]//Artificial intelligence accelerates human learning. Springer, Singapore, 2019: 1-17.

<figure><img src="image_72.png" /></figure>
<figure><img src="image_73.png" /></figure>

[16] Mitchell, Tom (1997). Machine Learning. New York: McGraw Hill. ISBN 0-07-042807-7. OCLC 36417892.

[17] Siemens, George. "What Are Learning Analytics?" Elearnspace, August 25, 2010. [1] Archived 2018-06-28 at the Wayback Machine

[18] Han, Jaiwei; Kamber, Micheline; Pei, Jian (2011). Data Mining: Concepts and Techniques (3rd ed.). Morgan Kaufmann. ISBN 978-0-12-381479-1.

[19] Content Technologies, Inc. (CTI) https://contenttechnologiesinc.com/

[20] Nuance https://www.nuance.com/dragon/industry/education-solutions.html

[21] Cognii https://www.cognii.com/

[22] Thinkster Math https://hellothinkster.com/online-math-tutor.html.

[23] Georgia Tech https://emprize.gatech.edu/

[24] Brainly https://brainly.com/schools-and-teachers?source(topbar)

[25] ClassVR https://classvr.com

[26] Chounta I A, Bardone E, Raudsep A, et al. Exploring teachers' perceptions of Artificial Intelligence as a tool to support their practice in Estonian K-12 education[J]. International Journal of Artificial Intelligence in Education, 2022, 32(3): 725-755.

[27] Lindner A, Romeike R. Teachers' perspectives on artificial intelligence[C]//12th International conference on informatics in schools, "Situation, evaluation and perspectives", ISSEP. 2019.

[28] Vazhayil A, Shetty R, Bhavani R R, et al. Focusing on teacher education to introduce AI in schools: Perspectives and illustrative findings[C]//2019 IEEE tenth international conference on Technology for Education (T4E). IEEE, 2019: 71-77.

[29] Jong M S Y. Pilot Study on Concerns of Teachers of Using Artificial Intelligence in Learning and Teaching[C]//2022 IEEE 5th Eurasian Conference on Educational Innovation (ECEI). IEEE, 2022: 209-211.

[30] LOLA: La notion de scénario, Chahrazed Labba, Azim Roussanaly, Anne Boyer

[31] European Education Area, "Ethical guidelines on the use of artificial intelligence (AI) and data in teaching and learning for Educators", 25 Oct 2022, European Education Area, https://education.ec.europa.eu/

[32] Scikit-learn: Machine Learning in Python, Pedregosa et al., JMLR 12, pp. 2825-2830, 2011.

[33] Doulingo https://www.doulingo.com

<figure><img src="image_74.png" /></figure>
<figure><img src="image_75.png" /></figure>

[34] 360learning https://360learning.com

[35] Vélez White C M. Inteligencia artificial en el sector de la educación. Informe 5[J]. 2022.

[36] Zhang Z, Brun A, Boyer A. New measures for offline evaluation of learning path recommenders[C]//European Conference on Technology Enhanced Learning. Springer, Cham, 2020: 259-273.

[37] Treuillier C, Boyer A. A New Way to Characterize Learning Datasets[C]//CSEDU (2). 2022: 35-44.

[38] Soussia A B, Roussanaly A, Boyer A. Toward An Early Risk Alert In A Distance Learning Context[C]//2022 International Conference on Advanced Learning Technologies (ICALT). IEEE, 2022: 206-208.

<figure><img src="image_76.png" /></figure>