A Systematic Literature Review of Research on
AI Literacy from 2020 to 2024
First Author1,2*, Second Author2,3† and Third Author1,2†
1*Department, Organization, Street, City, 100190, State, Country.
2Department, Organization, Street, City, 10587, State, Country.
3Department, Organization, Street, City, 610101, State, Country.
*Corresponding author(s). E-mail(s): <PRESIDIO_ANONYMIZED_EMAIL_ADDRESS>;
Contributing authors: <PRESIDIO_ANONYMIZED_EMAIL_ADDRESS>; <PRESIDIO_ANONYMIZED_EMAIL_ADDRESS>;
†These authors contributed equally to this work.
Abstract
To be added
Keywords: to be added
1 Introduction
2 Related Work
2.1 AI Literacy
- AI Literacy Definitions & Importance
- What is AI Literacy? Competencies and Design Considerations (Long & Magerko)
- other AI Literacy Frameworks (AI Comp- https://nexteducation.org/downloads/2023-12-14-AIComp-FutureSkills-Modelleng.pdf)(Digital
Promise - https://digitalpromise.dspacedirect.org/server/api/core/bitstreams/c09bc1b5-
a869-4fc3-b47b-c3bfd6575ca0/content)
1
2.2 Systematic Literature Reviews on AI Literacy
- https://link.springer.com/article/10.1186/s40594-023-00418-7
- https://www.sciencedirect.com/science/article/pii/S2666557324000107
- https://www.sciencedirect.com/science/article/pii/S2666920X2200056X
- https://www.sciencedirect.com/science/article/pii/S2666557324000144
- https://www.sciencedirect.com/science/article/pii/S2666920X21000357
3 Methodology
3.1 Method
We collected 50 AI-related papers from the Internet (https://dl.acm.org), which were
divided into 10 groups of 5 papers each, with no repetition of content in each paper.
Based on these papers, we created an Excel, which containing the paper IDs (from
number 1 to 50, with each number representing a paper), the paper links (clicking on
the links will redirect you to the corresponding paper pages for reading), 17 columns
of AI Literacy descriptions (used for the evaluators to pick out the corresponding
AI Literacy information after reading the paper. These 17 items are: Competency 1
(Recognizing AI), Competency 2 (Understanding Intelligence), Competency 3 (Interdisciplinarity), Competency 4 ( General vs. Narrow), Competency 5 (AI’s Strengths &
Weaknesses), Competency 6 (Imagine Future AI), Competency 7 (Representations).
Competency 8 (Decision-Making), Competency 9 (ML Steps), Competency 10 (Human
Role in AI), Competency 11 (Data Literacy), Competency 12 (Learning from Data),
Competency 13 (Critically Interpreting Data), Competency 14 (Action & Reaction),
Competency 15 (Sensors), Competency 16 (Ethics), Competency 17 (Programmability)), the number of authors, the nationality of the authors, the author’s field of study,
and the main research methods used in the paper.
A total of 12 people were involved into the task of reading papers and coding
them. These 12 people were all from Duisburg Essen University and were students of
the ”AI Literacy Seminar”. The 12 people were divided into several groups of two.
Each group was assigned to a different paper group, and two people in the same
group read five papers from the same paper group. Based on the content of the paper,
the two people selected the AI Literacy in Excel that matched the paper and labeled
it with a ”1” under the competency number. If the competency does not match the
content of the paper, then mark ”0”. This is the process of coding. After they read
each paper, they also need to fill in the areas like how many authors were in this
paper, all these areas listed after column Competency 17 in Excel.
3.2 Dataset
- describe the data (how we collected data, how many papers we had initially, how
many papers did we end up with)
2
3.3 Analysis
- describe the analytical process (what statistical methods we used,how did we calculate agreement, how were the final codes specified)?
4 Results
We used the AI Literacy Taxonomy proposed by [1].
4.1 Demographics
4.1.1 How many authors per paper?
The statistical analysis shows the following results:
• Average number of authors per paper: 3.73
• Standard deviation of the number of authors per paper: 2.05
Fig. 1 Distribution of the Number of Authors per Paper
Paper IDs per Category of Number of Authors
The provided figure and table offer an insightful overview of the distribution of the
number of authors per paper. Figure 1 displays a bar chart showing how papers are
distributed according to the number of authors involved. The chart indicates that the
3
Number of Authors Paper IDs
1 13, 32, 38, 39, 41, 46
2 16, 24, 27, 28, 31, 33, 42, 43
3 1, 9, 12, 14, 17, 19, 22, 23, 26, 29, 40, 44, 47, 48, 49
4 3, 5, 20, 36
5 7, 10, 15, 21, 25, 35
6 18, 30, 45
7 6, 8, 11, 37, 50
8 4
9 34
Table 1 Paper IDs per Category of Number of Authors
most common number of authors per paper is three, followed by two and four authors,
while papers with one, five, six, and seven authors are less common. Papers with eight
and nine authors are relatively rare.
Table 4.1.1 complements this visual data by listing specific paper IDs categorized
by the number of authors. This detailed breakdown shows the exact distribution of
papers for each category, making it easier to identify and reference specific papers
within each author group. For instance, papers with three authors include IDs such
as 1, 9, 12, 14, etc., while papers with a single author are identified by IDs like 13, 32,
38, etc. This combination of graphical and tabular data helps to understand the collaborative patterns in the research, highlighting a tendency towards moderate-sized
author groups rather than single authors or very large teams.
4.1.2 How many papers per country
We looked at the affiliation country of the first author of each collected paper and
listed them in Table 2. We also listed the amount of papers released by each country
in Fig. 2. Most of the papers we took into account for in our literacy review come from
the USA, with a total of 28 papers published. Out of the 48 papers published, 7 papers
were published in Germany. The rest of the countries published one or two papers.
Country Paper IDs
Australia 32
Canada 37
Chile 31
China 30, 38
Cyprus 12
Denmark 10, 18
Germany 16, 17, 24, 40, 43, 47, 50
Italy 14, 19
Malta 7
Spain 29
UK 34
USA 1, 3, 4, 5, 6, 8, 9, 11, 13, 15, 20,
21, 22, 23, 25, 26, 27, 28, 33, 35
36, 39, 41, 42, 44, 45, 48, 49
Table 2 Paper IDs per Country
4
Fig. 2 Distribution of the Number of Papers per Country
4.1.3 How many papers per year?
Publication Year Paper IDs
2020 22, 27, 30, 34, 44, 48
2021 4, 8, 10, 17, 19, 24, 26, 28, 33, 40, 42, 49
2022 1, 5, 11, 20, 21, 23, 29, 36, 41, 50
2023 3, 6, 7, 9, 12, 13, 15, 18, 31, 35, 38, 39, 43, 46
2024 14, 16, 25, 32, 37, 45, 47
Paper IDs
per Category of years
5
Fig. 3 Distribution of the Number of Papers per Country
4.2 What is AI?
Competency 1. [Recognizing AI...]
In the context of AI Literacy, the ability to distinguish between technological artifacts
that utilize Artificial Intelligence (AI) and those that do not is crucial. Out of the 48
assessed papers, 6 (12,5%) addressed this competency (inter-rater agreement: Cohen’s
kappa 0.78 - indicating substantial agreement) (Paper ID 3 [2], 4 [3], 12 [4], 24 [5],
41 [6], 45 [7]). For instance, in the context of fostering public AI literacy, paper 3 [8]
explores a hands-on exhibit called ”Knowledge Net” where people build network tiles
- a type of AI knowledge that represents relationships between objects. Learners were
able to consider differences between human intelligence and the network Paper 4 [3]
introduces an innovative AI Book Club model for professional development aimed at
building a foundational knowledge of AI for teachers, all while fostering a supportive
community. One key weekly meeting topic directly addressed ”What is AI - Examples
of AI”, aiming to equip educators with core AI literacy skills. Similarly, paper 12 [4]
highlights the importance of recognizing AI in everyday life. It describes an eight-week
course, ”AI in everyday life”, aimed at educating the public about AI technologies
and their implications. Paper 24 [5] focuses on developing a framework of core AI
competencies for non-computer science students. These competencies cover a range of
perspectives, from basic AI understanding to more complex knowledge.
Competency 2. [Understanding Intelligence]
This skill involves analyzing and discussing the features that make an entity intelligent, distinguishing between human, animal, and machine intelligence. Definitions
vary, with some emphasizing foresight and adaptability, or highlighting traits like
6
communication and learning. It also includes understanding empirical versus rational
approaches to intelligence. In total, 10 out of 48 papers (20%) addressed this competency (kappa: 0.35 / Fair Agreement) (Paper IDs: 7[9], 12[4], 15[10], 29[11], 32[12],
33[13], 36[14], 38[15], 45[7], 48[16], ). For example paper 48[16] seeks to identify gaps
in the current explainable AI (XAI) algorithmic work and the questions users might
have to these intelligent systems.
Competency 3. [Interdisciplinarity] Understand that AI is used in many different
areas and is built with various technologies. Recognize different technologies used to
develop AI. This competency was studied in 22 out of 50 papers (PaperIDs 1[17], 4[3],
7[9], 8[18], 9[19], 10[20], 15[10], 16[21],19[22], 21[23], 22[24], 24[5], 29[11], 31[25], 33[13],
34[26], 36[14], 37[27], 39[28], 44[29],45[7], 47[30]) This corresponds to 44%. An interrater agreement of 3.9 was calculated for this statistic, which is a fair agreement. This
means the evaluators generally had consistent opinions when determining whether a
paper studied this competency. In simpler terms, this fair agreement suggests that
different evaluators mostly agreed on which papers covered the topic, indicating that
their assessments were reasonably reliable but not perfect. This helps ensure that the
reported statistic—that 44% of the papers discussed AI development technologies—is
trustworthy. For example, paper 8 mentions AI being used in smartphones, social
media, and school surveillance[18]. This corresponds to the Competency ”interdisciplinarity” because it mentions different areas in which AI is being used. Similarly,
paper 36[14] specifically describes where Generative AI is being used.
Competency 4. [General vs. Narrow AI] aims to address the distinction between
general AI and narrow AI. General AI refers to systems with the ability to perform
any intellectual task that a human can, while narrow AI is designed to handle specific
tasks and lacks the generalization capabilities of a human mind. However distinguishing between general and narrow AI can help learners grasp the current capabilities
of AI and the challenges that lie ahead in achieving more advanced forms of artificial intelligence,in total, 0 out of 48 papers addressed this competency (kappa: 0).
This absence of coverage suggests that distinguishing between general and narrow AI
might not be a primary focus in the current discourse on AI literacy, which could be
due to factors like: the practical focus of AI literacy on current narrow AI applications, the speculative and complex nature of general AI theories, and the possibility
that this differentiation is an emerging topic not yet fully integrated into educational
frameworks.
4.3 What AI can do?
Competency 5. [...]
Competency 6. [Imagine future AI] The competency ”Imagine Future AI” focuses
on the ability to envision and anticipate the advancements and applications of artificial intelligence in the coming years. It involves foreseeing novel AI technologies,
analyzing emerging trends, considering ethical and societal implications, and developing strategic plans to leverage AI for competitive advantage. This competency also
encompasses anticipating future challenges and devising innovative solutions, as well
as understanding the integration of AI with other technologies and disciplines to
7
drive progress and innovation. 12 out of 48 articles (24%) were identified as addressing Competency 6, especially those with Article-IDs: 6 [31], 12 [4], 14 [32], 15 [10],
17 [33], 19 [22], 25 [34], 33 [13], 37 [27], 38 [15], 39 [28], and 45 [7].. The Cohen’s
Kappa coefficient was calculated to be k= 0.62, indicating a substantial agreement
between Rater1 and Rater2. The focus here is on AI knowledge and its effects on
human-AI collaboration. The article explores how AI knowledge can enhance task
performance but also reduce the intention to continue using AI, highlighting the need
for context-specific AI education. It suggests that better understanding the nuances
of AI knowledge can lead to more effective human-AI collaboration and proposes
future research directions to optimize AI education and upskilling programs. The
article 25 outlines a curriculum designed to engage students with AI concepts through
practical and ethical explorations. It emphasizes the importance of exposing students
to AI-enhanced careers and real-world applications of AI. The curriculum includes
interactive simulations and projects that help students understand AI mechanisms,
recognize their strengths, and envision future career paths in AI-related fields.
4.4 How does AI work?
Competency 7. [(Knowledge) Representations] Knowledge needs to be modeled so
that machines are capable of understanding it. This is done using Knowledge Representations, which is an integral part of every AI system. Understanding the concepts
of Knowledge Representation is essential to grasp how computers possess knowledge
and to understand how Knowledge gets lost in representations. This competency has
been addressed by 10 out of 48 papers (around 20%) with an inter-rater agreement of
0.51 kappa-value, which indicates moderate agreement. [17] [2] [18] [19] [21] [33] [22]
[25] [30] [16]
Papers [17], [2], [18] and [19] focus on how to teach this competency through the use
of a prototype activity called ’knowledge net’. This activity also focuses on teaching
users that real world knowledge will get lost in knowledge representations.
On the other hand, Papers [21] [22] [25] [30], focus on how to enable teachers to better
teach AI literacy competencies like Knowledge Representations through examining
existing frameworks for teaching AI competencies or suggesting new ones. Another
Paper [33], also suggests that teaching Knowledge Representations to users will make
users delegate tasks to AI more efficiently, which we think is an important skill to
have in a world were people collaborate more and more with AI systems.
Competency 8. [Decision-Making] Competency ”Decision Making” is aims to to
recognize and describe examples of how computers reason and make decisions. Of the
48 papers reviewed, 9 papers specifically focused on this competency, that takes 19% of
all the papers, showing an inter-rater agreement with a kappa-value k=0,4 (The nine
papers are 3 [2], 9 [19], 28 [35], 32 [12], 33 [13], 34 [26], 40 [36], 48 [16], 50 [37]). Paper
3[2], for example, the point ”Creature Features,” discussed in Section 3.2 of the article,
directly explores how agents represent knowledge and make decisions, which are central aspects of this competency. In ”Creature Features” it mentions “learners can use
a card deck and “weight tokens” to provide training data to a feature-based machine
8
learning algorithm that classifies animals as birds.” This point is specifically intended
to communicate competencies related to decision-making in AI, by letting participants
adjust the training data and observe changes in how the algorithm classifies information based on those adjustments. The article discusses how this setup allows learners
to understand the impacts of their decisions on the algorithm’s performance, aligning
directly with Competency 8’s focus on understanding AI decision-making processes.
In a similar side, paper 48 [16]touches on AI Literacy’s Competency 8, which is centered on how AI systems make decisions, particularly through the lens of explainable
AI (XAI). From the paragraphs “BACKGROUND Explainable artificial intelligence
(XAI)” it shows the following statements: “In much of the ML literature, XAI aims
to make the reasons behind a ML model’s decisions comprehensible to humans.” This
direct statement highlights how AI systems reason and make decisions, fitting squarely
with Competency 8’s requirements. From introduction part, it mentions, “To close the
gap between XAI algorithms and user needs for effective transparency, the HCI community has called for interdisciplinary collaboration and user-centered approaches to
explainability.” This suggests a focus on how AI’s decision-making processes can be
made understandable and meaningful to users, aligning with the need for recognizing
and describing examples of AI decision-making.
We believe it is crucial for Competency 8 [Decision-Making].
Competency 9. [Machine Learning steps] ML is not only the AI sub-field but
also overlaps each other. AI is the intersection of cognitive systems, robotics, and
machine learning [9]. Some recent research focuses on how to teach ML to individuals
without a CS background and to non-experts to perform ML algorithms so that they
won’t be surprised that ML needs human decision-making. Having students physically display algorithms to avoid misunderstanding learners should know computers
learn from their data.[21]
Undergraduate and non-computer science students need to learn some specific ML
competencies and how ML techniques and their main algorithms work.
Advanced ML like neural networks and deep learning is beneficial for students. It can
be adopted in the political science domain, and medical students should understand
how AI systems can be introduced in their work and distinguish their responsibilities
from the machine’s. Journalist students need to understand AI ethical concerns.
Data literacy is really important when it comes to understanding machine learning.
You can’t really understand AI, especially from a machine learning perspective,
without being data literate. In the evaluation stage, it’s important for students to be
able to train a model and get how biases in the training data can affect the accuracy
and strength of a machine learning model.[5]
Competency 10. [Human Role in AI] is indispensable, as they are fundamental
to its operation. Humans creates datasets, has an important role on every activity,
set ethical guidelines and provide the creativity and critical thinking necessary to
innovate and address complex problems that AI aims to solve.
Furthermore, humans are essential in training AI models, such as choosing and working with available data [10] and, as they evolve, human oversight remains crucial to
9
monitor, interpret, and guide these technologies, ensuring they serve society’s best
interests and reflect human values. As for the future generations, since AI is increasingly present nowadays and will be used even more in the future, there is an increasing
need to prepare them for the future workforce and everyday life, for example, with
Machine Learning (ML), that can revolutionize practices of knowledge discovery [23].
Although it seems to be completely positive, when you think about how AI works,
you can see that the role of humans can actually be negative. As discussed in [17],
people can teach AI false information and, in the study in this article, the participating groups used a bad dataset to understand the consequences of this.
Since it is a competency strongly related to AI, after a 4-step verification, it could be
concluded that 17 of the 48 studied articles addressed this competency, 35% of them
all. The inter-rater agreement (kappa-value) on the third step was 0.26 and then the
last verification was made to get to the final results, since kappa-value was lower then
0.4.
Therefore, considering the positive and negative influence of humans in AI, the
transmission of humanistic ideas remains an important function of education. The
development of AI can not ignore the essence of education [35]
Competency 11. [Data literacy] The competency ”Data Literacy” focuses on the
understanding, evaluation, and effective use of data. This competency encompasses
skills in data acquisition, processing, analysis, and interpretation, alongside ethical
data management and visualization. Of the 48 papers reviewed, 4 papers were identified as addressing Competency 11, accounting for 8.3% of all papers, with an inter-rater
agreement kappa value of 0.13, indicating a low level of agreement between Rater 1
and Rater 2. (The four papers are 7[9], 16[21], 27[38], and 47[30]).
For instance, paper 7[9]demonstrates how to effectively teach data literacy in primary and secondary education through the design and implementation of the ”ArtBot”
game. Specifically, through hands-on practice and reflection, students gain a deep
understanding of and ability to apply data analysis skills. Furthermore, by helping
students recognize how human values and biases are reflected in AI applications, the
game fosters ethical awareness and a sense of responsibility in data handling.
Similarly, the review results of paper 16[21] show that existing AI literacy frameworks either barely cover or do not cover competencies related to data literacy at all.
However, data literacy is crucial for understanding AI, especially machine learning.
To address this gap, the article suggests that the frameworks be expanded to more
comprehensively promote both AI and data literacy.
Competency 12. [Learning from Data] It’s crucial for learners to understand that
AI gains knowledge from the data it receives.Understanding how computers learn
from their data is crucial for critically reviewing and interpreting machine learning
models, discovering and mitigating bias, supporting ethical AI practices, and involving learners in useful and applicable ways. This fundamental knowledge ensures that
learners are more than passive AI users; they are informed and critical thinkers able
to responsibly navigate and shape the AI-driven environment. Being able to identify
when their data is utilized to teach machine learning is an important aspect of data
10
literacy in the context of AI[1]. In total, 14 out of 48 papers (29%) addressed this
competency. The inter-rater agreement is 0.37, which indicates fair agreement.
For example, Paper ”Learn to Machine Learn: Designing a Game Based Approach
for Teaching Machine Learning to Primary and Secondary Education Students”[9],
through the implementation of the game ArtBot in the LearnML project, helps users
understand that AI gains knowledge from the data. ArtBot allows children to engage
with AI programming interactively. By manipulating data inputs and observing the
outcomes, learners can see firsthand how AI systems use data to learn and make
decisions.
Similarly, the paper ”Artificial Intelligence in Compulsory K-12 Computer Science Classrooms: A Scalable Professional Development Offer for Computer Science
Teachers”[16] focuses on developing and implementing a professional development
(PD) program to prepare computer science (CS) teachers to teach artificial intelligence (AI) in K-12 education. The professional development (PD) programme for
K-12 computer science teachers contains learning objectives in machine learning and
data handling. The program guarantees that teachers comprehend how AI systems
use data to learn and judge by teaching these subjects. The progress test developed for
evaluating teachers includes questions regarding various machine-learning approaches
and algorithmic bias. Addressing algorithmic bias in the PD program emphasises the
relevance of data quality and features while training AI systems. Teachers learn that
biases in data can result in biased AI outputs, emphasising the importance of data in
AI learning processes. This emphasis helps teachers understand that data is essential
for training AI models and determines their behaviour and performance.
Competency 13. [...]
Competency 14. [...]
Competency 15. [Sensors] focuses on the role of sensors in AI systems. Learners
should understand the definition and functions of sensors in AI, identify various
types of sensors, comprehend the data representations and reasoning processes they
support, and recognize and explain sensors in everyday devices. Additionally, learners
should grasp how sensors integrate with AI algorithms to interpret data, make decisions, or perform actions. This competency is essential for a thorough understanding
and application of AI technology.
Out of 48 reviewed papers, four of them specifically addressed this competency,
the paper with competency Sensors“ takes part of 8.3% of all 48 papers. With an
inter-rater agreement kappa value of 0.38 (the four papers ID are 9[19], 13[8], 19[22],
50[37]).
For instance, paper 9 [19]discusses the importance of understanding sensors, their
integration with AI, and their roles in decision-making and action, which aligns closely
with the described competency. From paragraph 8.2.7 Integrating technology of paper
9 we can find following statements:
”Sensors and actuators in particular seemed to elicit a lot of participant interest/dialogue throughout the workshop. The success of the paper-based prototypes at
facilitating learning dialogue is promising for the development of ’unplugged’ AI
learning experiences for low-cost learning interventions” .
11
These words addresses the role of sensors in AI systems, highlighting their importance in educational contexts and their ability to engage learners. This aligns with the
competency’s focus on understanding the functions of sensors and their types, further
more it can also show how they integrate with AI to support data interpretation and
decision-making processes.
Paper 50 [37] explores user involvement in the learning of intelligent agents, describing how smart home systems can associate sensor information with user-provided
activity labels to recognize user activities, thereby enhancing users’ understanding and
control over Al systems.
”Participants were briefly explained that the smart home would connect the information on which activity is being executed with sensory data of motion sensors and
smart devices in the home. This way, it would later be able to tell from similar sensory information which activity is executed by the resident. In fact, our prototype did
not yet actually collect any sensory information. Instead, we had fake motion sensors
installed in the apartment to make the prototypical smart home look fully realistic.”
As the words comes in 4.1 Method, it describes how the smart home system integrates sensory data to recognize and react to user activities. It addresses the definition
and function of sensors in AI, their role in data collection and interpretation, and
their application in everyday devices, which are key components of Competency 15.
4.5 How should AI be used?
Competency 16. [Ethics] The rapid advancement of AI has raised critical ethical
questions. From potential biases in decision-making algorithms to concerns about data
privacy, ensuring responsible AI development is essential. In total, 17 papers out of
48 (35,4%) addressed ethical inquiries. For a more comprehensive overview, a table is
provided to summarize the frequency of individual concepts
Concept Frequency Percentage
Bias 12 25%
Ethical decision-making 5 10%
Fairness 5 10%
Data privacy 5 10%
Misinformation 4 8%
Diversity 2 4%
Discrimination 2 4%
Transparency 2 4%
Accountability 1 2%
Table 3 Breakdown of concepts and their frequency
12
Fig. 4 AI ethics word cloud
Bias: Papers IDs 1 [17], 3 [2], 4 [3], 7 [9], 9 [19], 10 [20], 11 [39], 26 [40], 28 [35],
29 [11], 33 [13], 35 [41]. Most papers highlighted the existence of algorithmic biases.
These biases are frequently associated with the inherent biases present in the training dataset. Paper 10 [20] pointed out that there is increasing concern about how the
opacity and complexity of ML systems can result in systemic racial and gender biases.
Additionally, paper 35 [41] showed that it is necessary to understand accessibility in
relation to AI, as most AI courses often neglect disability.
Ethical decision making: Papers IDs 7 [9], 9 [19], 10 [20], 28 [35], 29 [11]. In addition to
the advantages and benefits of AI, there are concerns about the potential drawbacks
and challenges related to the decisions these systems might make, the autonomy of
AI agents, and the values and biases built into their design [9]. Paper 28 [35]pointed
out that the current ML technologies have their limitations, such as their tendency
to reflect and reinforce existing societal biases and discrimination, their potential to
pick up spurious correlations rather than causal relationships, and their performance
in generalizing to external data. The inclusion of diverse human perspectives in the
training of AI models used for critical decision-making, especially those that disproportionately impact marginalized groups is needed[19].
Fairness: Papers IDs 5 [42], 26 [40], 33 [13], 34 [26], 35 [41]. In technical AI/ML courses,
fairness is a frequently discussed topic, often paired with bias. The primary focus of
these courses is on promoting fairness in AI systems [40]. Paper 33 [13] introduces
Let’s Chance, a novel computational microworld extending the Scratch Programming
Language to help children explore the ethical issue of AI, including fairness. Paper 34
[26] shows how art can offer alternative approaches to developing public AI literacy,
focusing on fairness by providing creative and tangible interactions to better understand AI processes beyond technical methods.
Data privacy: Papers IDs 3 [2], 6 [31], 9 [19], 10 [20], 26 [40]. Increasing worries regarding AI’s contribution to misinformation, and data privacy violations underscore the
necessity for technology users to acquire new skills to engage with AI more critically
and thoughtfully. Paper 6 [31] introduces the AI Educator Make-a-Thon as a model
that enhances educators’ engagement and investment in AI literacy. To engage teachers with new perspectives on AI technologies, expert panels regarding data privacy and
13
other issues were organized. Paper 26 [40] examines the integration of ethics into AI
education through standalone courses and embedding ethics with technical AI courses
at US universities. Privacy was specifically addressed in 32% of the courses, often in
combination with other topics such as bias, regulation, and automation. Additionally,
two courses included at least one lecture on differential privacy.
Misinformation: Papers IDs 1 [17], 3 [2], 4 [3], 9 [19]. Recent studies highlight
widespread public concerns regarding AI in everyday applications, such as the
propagation of viral misinformation [19]. Misinformation is the consequence of misunderstanding or inadequately regulating AI’s decisions [17].
Diversity: Papers IDs 9 [19], 14 [32]. The new advances in AI technology can help
people from disadvantaged groups to better integrate and communicate with others.
For example paper 14 [32] looks into teachers’ perspective on the implementation of
an AI education curriculum and how it can stimulate reflections around inclusion and
diversity.
Discrimination: Papers IDs 3 [2], 28 [35]. Similarly as in Misinformation explained
AI can be used with malicious intent to discriminate others. Moreover the AI-models
themself can be discriminatory, if not properly supervised. To get a deeper insight
into the reliance of laypeople on ML models, paper 28 [35] studies the effects of user
tutorials on this subject.
Transparency: Papers IDs 10 [20], 27 [38]. To better understand how to use a particular AI it is important to clearly and transparently communicate to the users the
capabilities and limitations of such systems. To study this aspect, paper 27 [38] looks
at how children come to see machine intelligence after training custom machine learning models and creating smart programs that use them.
Accountability: Papers ID 10 [20]. Research driven by AI ethical concerns focuses
on increasing accountability in ML models by improving transparency, fairness, and
robustness, and ensuring that ML system designers communicate and justify their
decisions to make them accountable for their impact on broader social systems.
4.6 How do people perceive AI?
Competency 17. [Programmability]
Understand that agents are programmable is foundational for understanding how
AI works [1], since it is built and controlled by human-written code and logic. However,
it is important for designers to keep in mind that prerequisite coding skills can be a
barrier to entry into the AI world. [1]. In total, 12 out of 48 papers (25%) addressed
this competency. The inter-rater agreement is 0.38, which indicates fair agreement
(PaperIDs 1[17], 7[9], 11[39], 12[4], 14[32], 27[38], 31[25],33[13], 39[28], 45[7], 47[30],
50[37]).
Paper IDs 12[4] discusses that there is a clear need for developing and implementing
separate courses that are specifically designed for various segments of the public. In
this way, all citizens will learn about AI and the impact intelligent algorithms have
on their lives, at a level that they can actually understand and relate to without
necessarily having a technical background. However, understanding what exactly is
14
needed when designing AI education for the broad public is an important part of the
process, and to our best knowledge, there is still work to be done in the .
Most of the papers studied mentioned children’s perceptions of AI. Since they
are future users of it, it is crucial to consider their opinions so its use can be more
accessible and effective in the future. There are multiple factors that influence children’s views of AI: age, gender, cultural background (since it can lead to lack of prior
experience) and more. The form of AI might make a difference in perception as well.
Some works can contribute to understanding how different types of AI technologies
can influence youth users’ understanding and judgments about fairness. [39]
Paper ”What is AI Literacy? Competencies and Design Considerations” [1] mentions that children have different ways of perceiving things compared to adults, such as
their tendency to focus on observable characteristics (e.g., success) rather than unobservable ones (e.g., strategy) when assessing AI. This makes it challenging to help users
not only understand but also know how to apply AI. Paper ”Learn to Machine Learn:
Designing a Game Based Approach for Teaching Machine Learning to Primary and
Secondary Education Students”[9] implemented a game called ArtBot, which allows
children to play while practicing AI programming in a guided manner, enhancing their
attention. Their goal in the LearnML project and through the game ”ArtBot” is to
develop educational AI and ML support materials while stimulating students’ critical
thinking about the aspects, factors, and biases that can shape AI agents and systems.
5 Discussion
6 Conclusion
Appendix A Section title of first appendix
References
[1] Long, D., Magerko, B.: What is ai literacy? competencies and design considerations. In: Proceedings of the 2020 CHI Conference on Human Factors in
Computing Systems, pp. 1–16 (2020)
[2] Long, D., Padiyath, A., Teachey, A., Magerko, B.: The role of collaboration,
creativity, and embodiment in ai learning experiences. In: Proceedings of the 13th
Conference on Creativity and Cognition (2021)
[3] Lee, I., Zhang, H., Moore, K., Zhou, X., Perret, B., Cheng, Y., Zheng, R., Pu, G.:
AI Book Club: An Innovative Professional Development Model for AI Education,
pp. 202–208 (2022)
[4] Kasinidou, M., Kleanthous, S., Otterbacher, J.: Artificial intelligence in everyday
life: Educating the public through an open, distance-learning course. In: Proceedings of the 2023 Conference on Innovation and Technology in Computer Science
Education V. 1, pp. 306–312 (2023)
15
[5] Ten´orio, K., Romeike, R.: Ai competencies for non-computer science students in
undergraduate education: Towards a competency framework. In: Proceedings of
the 23rd Koli Calling International Conference on Computing Education Research
(2024)
[6] Khadivi, P.: Ai and society: Teaching ai to non-stem students, pp. 18–27 (2023)
[7] Song, Y., Tian, X., Regatti, N., Katuka, G.A., Boyer, K.E., Israel, M.: Artificial intelligence unplugged: Designing unplugged activities for a conversational
ai summer camp. In: Proceedings of the 55th ACM Technical Symposium on
Computer Science Education V. 1, pp. 1272–1278 (2024)
[8] Long, D., Blunt, T., Magerko, B.: Co-designing ai literacy exhibits for informal
learning spaces. In: Proceedings of the ACM on Human-Computer Interaction,
pp. 1–35 (2021)
[9] Voulgari, I., Zammit, M., Stouraitis, E., Liapis, A., Yannakakis, G.N.: Learn to
machine learn: Designing a game based approach for teaching machine learning
to primary and secondary education students. In: IDC ’21: Proceedings of the
20th Annual ACM Interaction Design and Children Conference (2021)
[10] Tseng, Y., Xiao, R., Bogart, C., Savelka, J., Sakr, M.: Assessing the efficacy of ˇ
goal-based scenarios in scaling ai literacy for non-technical learners. In: SIGCSE
2024: Proceedings of the 55th ACM Technical Symposium on Computer Science
Education V. 2 (2024)
[11] Schaper, M., Malinverni, L., Valero, C.: Robot presidents: Who should rule
the world?: Teaching critical thinking in ai through reflections upon food
traditions. In: NordiCHI ’20: Proceedings of the 11th Nordic Conference on
Human-Computer Interaction: Shaping Experiences, Shaping Society (2020)
[12] Wan, X.: A study on the current development of artificial intelligence in education industry in china. In: ICETT ’21: Proceedings of the 2021 7th International
Conference on Education and Training Technologies (2021)
[13] Dhariwal, M., Dhariwal, S.: Let’s chance: Playful probabilistic programming for
children. In: CHI EA ’20: Extended Abstracts of the 2020 CHI Conference on
Human Factors in Computing Systems (2020)
[14] Nithithanatchinnapat, D.J. Maurer: Future business workforce: Crafting a generative ai-centric curriculum today for tomorrow’s business education. In: ACM
SIGMIS Database: the DATABASE for Advances in Information Systems,
Volume 55, Issue 1 (2024)
[15] Wang, H.: Integration path of artificial intelligence technology and education
and risk avoidance. In: ICETM ’22: Proceedings of the 2022 5th International
Conference on Education Technology Management (2023)
16
[16] Liao, Q.V., Gruen, D., Miller, S.: Questioning the ai: Informing design practices
for explainable ai user experiences. In: Proceedings of the 2020 CHI Conference
on Human Factors in Computing Systems (2020)
[17] Long, D., Teachey, A., Magerko, B.: Family learning talk in ai literacy learning
activities. In: Proceedings of the 2022 CHI Conference on Human Factors in
Computing Systems (2022)
[18] Duri, A.-D.H.N.R.M. Rollins: Fostering ai literacy with embodiment & creativity:
From activity boxes to museum exhibits. In: IDC ’23: Proceedings of the 22nd
Annual ACM Interaction Design and Children Conference (2023)
[19] Long, D., Blunt, T., Magerko, B.: Co-designing ai literacy exhibits for informal
learning spaces. Proc. ACM Hum.-Comput. Interact. (2021)
[20] Bilstrup, K.E.K., Kaspersen, M.H., Assent, I., Enni, S., Petersen, M.G.: From
demo to design in teaching machine learning. In: FAccT ’22: Proceedings of the
2022 ACM Conference on Fairness, Accountability, and Transparency (2022)
[21] Olari, V., Romeike, R.: Addressing ai and data literacy in teacher education: A
review of existing educational frameworks. In: Proceedings of the 16th Workshop
in Primary and Secondary Computing Education (2021)
[22] Polak, S., Schiavo, G., Zancanaro, M.: Teachers’ perspective on artificial intelligence education: an initial investigation. In: IUI ’22: Proceedings of the 27th
International Conference on Intelligent User Interfaces (2022)
[23] Xiaoyu Wan, Z.Y.C.K.M.Z.B. Xiaofei Zhou: Smileycluster: supporting accessible
machine learning in k-12 scientific discovery. In: IDC ’20: Proceedings of the
Interaction Design and Children Conference (2020)
[24] Stephanie Tena-Meza, A.A. Miroslav Suzara: Coding with purpose: Learning ai in
rural california. In: ACM Transactions on Computing EducationVolume 22Issue
3 (2022)
[25] Quiroz, P., Gutierrez, F.J.: Scratch-nb: A scratch extension for introducing k12 learners to supervised machine learning. In: Proceedings of the 55th ACM
Technical Symposium on Computer Science Education V. 1 (2024)
[26] Drew Hemment, S.B.J.E.A.R.C.S.M.V.R.H.H.W. Morgan Currie: Ai in the public
eye: Investigating public ai literacy through ai. In: FAccT ’23: Proceedings of the
2023 ACM Conference on Fairness, Accountability, and Transparency (2023)
[27] Guerzhoy, M., Neumann, M., Virtue, P., Anderson, C.J., Singla, Y.K., Orchard,
A., Shah, R.R.: Eaai-23 blue sky ideas in artificial intelligence education from
the aaai/acm sigai new and future ai educator program. AI Matters 9(2), 24–29
(2023)
17
[28] Thomas, G.: Redesign of an elective introductory artificial intelligence course in
a credit-limited computer science curriculum. Journal of Computing Sciences in
Colleges 37(2), 98–104 (2021)
[29] Jianlei Yang, W.Z. Xiaopeng Gao: Towards systems education for artificial intelligence: A course practice in intelligent computing architectures. In: GLSVLSI
’20: Proceedings of the 2020 on Great Lakes Symposium on VLSI (2020)
[30] Jetzinger, F., Baumer, S., Michaeli, T.: Artificial intelligence in compulsory k-12
computer science classrooms: A scalable professional development offer for computer science teachers. In: Proceedings of the 55th ACM Technical Symposium
on Computer Science Education V. 1 (2024)
[31] Sperling, K., Stenberg, C.-J., McGrath, C., ˚Akerfeldt, A., Heintz, F., Stenliden,
L.: In search of artificial intelligence (ai) literacy in teacher education: A scoping
review. Computers and Education Open, 100169 (2024)
[32] Gibellini, G., Fabretti, V., Schiavo, G.: Ai education from the educator’s perspective: Best practices for an inclusive ai curriculum for middle school. In: Extended
Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems,
pp. 1–6 (2023)
[33] Pinski, M., Adam, M., Benlian, A.: Ai knowledge: Improving ai delegation
through human enablement. In: Proceedings of the 2023 CHI Conference on
Human Factors in Computing Systems (2023)
[34] Lee, I., Ali, S., Zhang, H., DiPaola, D., Breazeal, C.: Developing middle school
students’ ai literacy. In: Proceedings of the 52nd ACM Technical Symposium on
Computer Science Education, pp. 191–197 (2021)
[35] Chun-Wei Chiang, M.Y.: Exploring the effects of machine learning literacy
interventions on laypeople’s reliance on machine learning models. In: Extended
Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems
(2022)
[36] Jakob Schoeffer, Y.M. Niklas Kuehl: “there is not enough information”: On the
effects of explanations on perceptions of informational fairness and trustworthiness in automated decision-making. In: FAccT ’22: Proceedings of the 2022 ACM
Conference on Fairness, Accountability, and Transparency (2022)
[37] Leonie Nora Sieger, A.S.S.H.C.M.C.-C.H.A.D. Julia Hermann: User involvement
in training smart home agents: Increasing perceived control and understanding.
In: HAI ’22: Proceedings of the 10th International Conference on Human-Agent
Interaction (2022)
[38] Druga, S., Ko, A.J.: How do children’s perceptions of machine intelligence change
when training and coding smart programs? In: IDC ’21: Proceedings of the 20th
18
Annual ACM Interaction Design and Children Conference (2021)
[39] Solyst, J., Xie, S., Yang, E., Stewart, A.E.B., Eslami, M., Hammer, J., Ogan, A.E.:
“i would like to design”: Black girls analyzing and ideating fair and accountable
ai. In: CHI ’23: Proceedings of the 2023 CHI Conference on Human Factors in
Computing Systems (2023)
[40] Garrett, N., Nathan, B., Fiesler, C.: More than ”if time allows”: The role of ethics
in ai education. In: AIES ’20: Proceedings of the AAAI/ACM Conference on AI,
Ethics, and Society (2020)
[41] Tseng, C.E., Jung, S.H., Elglaly, Y.N., Liu, Y., Ludi, S.A.: Exploration on integrating accessibility into an ai course. In: SIGCSE 2022: Proceedings of the 53rd
ACM Technical Symposium on Computer Science Education - Volume 1 (2022)
[42] Xie, S., Solyst, J., Ogan, A.E., Hammer, J.: Booklet-based design fiction to
support ai literacy. In: SIGCSE 2023: Proceedings of the 54th ACM Technical
Symposium on Computer Science Education V. 2 (2023)
19



This is my report (work in progress). My next task is: 

The related work chapter should provide information to the reader about the following:

a summary of other systematic literature reviews on the topic of AI Literacy.


I will show you the literature reeview that you should summarize.