# Impact of game-based learning on STEM learning and motivation: Two case studies in Europe

Nour El Mawas, Peter Trúchly, Pavol Podhradský, Martin Medvecký, Cristina Hava Muntean

<figure><img src="image_1.png" /></figure>

## To cite this version:

Nour El Mawas, Peter Trúchly, Pavol Podhradský, Martin Medvecký, Cristina Hava Muntean. Impact of game-based learning on STEM learning and motivation: Two case studies in Europe. Knowledge Management & E-Learning: an international journal, 2022, 14 (4), pp.360-394. 10.34105/j.kmel.2022.14.020. hal-04041216

HAL Id: hal-04041216

https://hal.science/hal-04041216v1

Submitted on 14 May 2025

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Impact of game-based learning on STEM learning and motivation: Two case studies in Europe

Nour El Mawas

University of Lille, Lille, France

Peter Trúchly

Pavol Podhradský

Martin Medvecký

Slovak University of Technology in Bratislava, Slovakia

Cristina Hava Muntean

National College of Ireland, Dublin, Ireland

<figure><img src="image_3.png" /></figure>

Knowledge Management & E-Learning: An International Journal (KM&EL)

ISSN 2073-7904

**Recommended citation:**

El Mawas, N., Trúchly, P., Podhradský, P., Medvecký, M., & Muntean, C. H. (2022). Impact of game-based learning on STEM learning and motivation: Two case studies in Europe. *Knowledge Management & E-Learning, 14*(4), 360–394. https://doi.org/10.34105/j.kmel.2022.14.020
# Impact of game-based learning on STEM learning and motivation: Two case studies in Europe

## Nour El Mawas*

CIREL (EA 4354)
University of Lille, Lille, France
E-mail: nour.el-mawas@univ-lille.fr

## Peter Trúchly

Faculty of Informatics and Information Technologies
Slovak University of Technology in Bratislava, Slovakia
E-mail: peter.truchly@stuba.sk

## Pavol Podhradský

Faculty of Electrical Engineering and Information Technology
Slovak University of Technology in Bratislava, Slovakia
E-mail: pavol.podhradsky@stuba.sk

## Martin Medvecký

Faculty of Electrical Engineering and Information Technology
Slovak University of Technology in Bratislava, Slovakia
E-mail: martin.medvecky@stuba.sk

## Cristina Hava Muntean

School of Computing
National College of Ireland, Dublin, Ireland
E-mail: cristina.muntean@ncirl.ie

**Abstract:** The number of science, technology, engineering, and mathematics (STEM)-related jobs is increasing all around the world and especially in Europe. However, teachers face many difficulties in making STEM related classes more attractive and motivating the students to learn. This paper presents two case studies involving 116 students from two European countries (Slovakia and Ireland). The studies investigated the impact of a new educational computer game called Final Frontier on learning process and students' motivation. We found that there are no significant differences between the two countries in terms of students' interest in STEM, although the students in Slovakia achieved slightly better grades than those in Ireland. We also found that in both countries, the students using the Final Frontier game outperformed those not using the game in improving their subject knowledge after the study. Furthermore, the impact of game-based learning on students' motivation for

<footnote>*Corresponding author</footnote>
STEM differed in the two countries.

**Keywords:** Technology-enhanced learning; Educational game; STEM; Motivation; Knowledge gain

**Biographical notes:** Dr. Nour El Mawas received a PhD degree in Computer Science from the Université de Technologie de Troyes at Troyes in 2013. Her work was about the design of serious games for expertise training in complex situations like Crisis and Sustainable development. She did postdoctoral research at the Université du Maine about the design of instructional scenarios on Learning Management Systems (LMS). She was also a researcher in the Institut Mines-Télécom, working on MOOCs personalization in a lifelong learning perspective. She was also a researcher in the National College of Ireland, and she was working on innovative pedagogical methods and educational games used to teach STEM subjects. Currently, Nour El Mawas is an Associate Professor at Université de Lille. Her research interests focus on the Technology Enhanced Learning Field.

Dr. Peter Trúchly is an associate professor at the Slovak University of Technology, Faculty of Informatics and Information Technologies in Bratislava. He achieved his Master's degree in informatics in 1995 and his PhD in applied informatics in 2004 at this university. He is interested in communication technologies, mobile applications, the development of educational materials and e-learning platforms, and their pilot testing. His research also covers the performance analysis of communication protocols in wireless and satellite systems. Currently, he is a member of a research group focused on vehicular ad hoc networks.

Dr. Pavol Podhradský is a full professor in Telecommunications at the Slovak University of Technology in Bratislava and serves as a Deputy Director of the Multimedia Information and Communication Technologies Institute, Faculty of Electrical Engineering and Information Technology. He was born in Banská Štiavnica, Slovakia, in 1943. He achieved his Master's degree in the field of telecommunications in 1965 and his PhD in telecommunications in 1980 at this university. He is interested in communication technologies, ICT network architectures and services, and communication protocols as well as in the development of educational materials, also as e-learning.

Dr. Martin Medvecký is an Associate Professor in the Institute of Multimedia Information and Communication Technologies, Faculty of Electrical Engineering and Information Technology at the Slovak University of Technology in Bratislava, Slovakia. His research interests are focused primarily on advanced traffic routing techniques in modern networks, software-defined networking, and network function virtualization. He participated in several projects related to the use of ICT in education and the use of new progressive methods in education.

Dr. Cristina Hava Muntean is an Associate Professor with School of Computing, National College of Ireland and leader of the Usabiliy Lab. She holds a B.Eng. degree in Computer Science and a Ph.D. degree awarded by Dublin City University, Ireland in 2005. Dr. Muntean has been constantly involved in various research related activities over the past 20 years fostering and promoting research in Ireland and abroad, building academic collaborations with European and Wordwide Universities, leading research projects, supervising PhD and MSc students and publishing in excess of 130 publications in international peer-reviewed books, journals and conferences. She has broad and extensive research expertise in the ICT domain in general
and in particular in technology enhanced learning, adaptive and personalised
multimedia delivery, data analytics, end-user Quality of Experience and data
communication and networking. Dr. Cristina Muntean chaired or served as
technical program committee member for top international conferences and
acted as reviewer for several journals. She is an Associate Editor of the IEEE
Transactions on Broadcasting journal.

# 1. Introduction

Science, Technology, Engineering, and Mathematics (STEM) is universally recognized as the application of knowledge in STEM areas to understand how the world works and solve problems (Vilorio, 2014). It is estimated that STEM-related jobs will reach about 9 million by 2022, representing a growth of about a million jobs in a 10-year period from 2012. STEM-related jobs are growing faster than all other jobs and generally pay higher than the median for all other positions (Vilorio, 2014). Therefore, when choosing their majors, children should be introduced to STEM subjects in primary and secondary schools and not only in high schools and universities.

The lack of education and careers' popularity in the STEM field is due to numerous reasons. The first reason is related to the fact that STEM perception is constantly criticized; we need to consider STEM as a part of human history and culture and a key element to access a variety of careers (Brzozowy et al., 2017). The second reason is due to the poor STEM and technology-oriented teaching methodology in primary and secondary schools despite various useful and interesting research findings in the STEM education field (Osborne & Dillon, 2008). The third reason is the student's perception that STEM is a difficult topic. The fourth reason is the lack of information about the STEM competencies required by employers for the new positions. The fifth reason is the low participation of stakeholders in political decision-making and educational system development in order to address STEM issues (Brzozowy et al., 2017).

Students must not learn STEM subjects only from textbooks and through teacher-
driven lectures (Herder & Rau, 2022). Active learning increases student performance
across the STEM disciplines (Freeman et al., 2014): they need to perform practical
activities that provide hands-on experiences related to real-world situations (Gura, 2012).
According to Bhoje (2015), motivation is an important factor that universities and
teachers need to focus on in order to improve students' learning. It is also a necessary
element of education quality (Palmer, 2007). Weiner (1992) defines motivation as the
study of the determinants of thought and action - it addresses why the behaviour is
initiated, persists, and stops, as well as what choices are made. In general, most teachers
are aware of the importance of keeping students motivated.

The educational computer game is one of the technology-enhanced learning
environments that support concrete examples of interactive and constructivist learning (El
Mawas, 2014; Papadakis, 2018) in formal and informal education. Educational games
enable students to participate actively and invest in course materials differently from the
classical learning session (Hussain et al., 2016; Papadakis & Kalogiannakis, 2018).
Educational games are often used in classrooms due to their ability to increase student
motivation, foster STEM learning (Clark et al., 2016; Morris et al., 2013; Trúchly et al.,
2019) and allow improvements in the educational curriculum (Smith & Munro, 2009; Xie
et al., 2021). Bruner confirms that 21st-century learners' skills are increased when taught
through learning and game-based activities (Bruner, 2009). This has a direct effect on the
sense of responsibility and independence of learners. However, the design and use of educational games in classrooms to assess the learning and the motivation of students have not been extensively studied (Hoffman & Nadelson, 2010).

The research work presented in this paper was carried out in the context of a European project named NEWTON (NEWTON, 2016) that developed the NEWTELP educational platform. The NEWTON project integrates diverse novel technologies in education such as Augmented Reality and Virtual Reality (AR/VR), adaptive and personalised multimedia and multiple sensorial media, personalisation and gamification and interactive educational computer-based games (Bogusevschi, Bratu, et al., 2018; Bogusevschi, Muntean, et al., 2018; Muntean et al., 2019). These technologies were combined with self-directed teaching pedagogies such as problem-based learning, flipped-classroom and game-based learning in order to provide support for students (Muntean et al., 2018).

In this context, the aim of the paper is to address the learning and motivation impact of a new educational computer game designed and developed as part of the NEWTON project, in the STEM field. The research paper presents two case studies that involved the use of the Final Frontier educational game to teach concepts about planets in primary schools in Slovakia and Ireland. These countries have been selected on the basis of the origin of the coordinator and project partner's location. The research findings show that students from the two countries have different perceptions regarding their feelings about STEM, knowledge gained through the Final Frontier game, and the game's motivation impact. This research work is dedicated to Technology-Enhanced Learning (TEL) community and more specifically to pedagogical engineers, researchers, and teachers in Irish / Slovak primary schools who encounter difficulties in engaging students in STEM subjects.

The paper is organized as follows. Section 2 presents research work done in the STEM field, focusing on existing educational games that aim to teach STEM subjects in Slovakia and Ireland. Section 3 introduces the Final Frontier game with its learning objectives. Section 4 presents an analysis of the outcomes of the case study in the two European countries. Section 5 compares and discusses the results. Section 6 concludes the paper and presents further research perspectives.

## 2. Related work

### 2.1. STEM education in Slovakia

Based on analysis published in (Šveda et al., 2018) the current state of education in primary and secondary Slovak schools is not satisfactory because it does not reflect actual trends and it is not flexible to rapidly changing requirements of a modern 21st century company. Education is highly accessible in Slovakia; more than 90% of the population finishes at least secondary school level. However, employers and companies emphasize the lack of qualified people in the labour market and call for a change in the educational system that cannot produce graduates with the required skills and competencies.

The research done by Trexima (Trexima, 2017) also acknowledges this discrepancy between the knowledge of Slovak graduates and the needs of the labour market. 57% of tertiary school graduates and even 63% of secondary school graduates are not employed in a branch they graduated from. When we concentrate on STEM study
specializations, i.e., subjects with high labour market demand, 25.8% of students in Slovakia only studied STEM subjects in 2016, which is below the EU average (30.52%), and this ratio has stagnated since 2013 (MŠVVaŠ, 2018). Based on Piatkowski's study (Piątkowski, 2020) only 15 STEM HEI (higher education institution) graduates per 1000 individuals were recorded in Slovakia (1.5% vs 1.8% in the EU). However, it is expected that the demand for STEM-oriented job positions in the EU will grow until 2025 (Caprile et al., 2015). The educational system in Slovakia experienced curriculum reform and transformation (from 2008), although no evident focus on STEM education was included (Kearney, 2016). Although Slovakia is an industrial country, the tertiary school system produces a large number of graduates in social sciences and humanities every year.

Nowadays young people want to study and work in sectors that match their values and interests, but STEM areas are not of interest to them. Employers and tertiary schools have been observing a descending level of mathematical skills in secondary school graduates for more than ten years (Šveda et al., 2018). The critical situation is especially in STEM-related specializations that are based on mathematical literacy. This was negatively influenced when the math subject was stopped being mandatory in a secondary school leaving examination.

The results of national tests conducted by the Ministry of Education in 2018 among primary school students also show large differences in educational outcomes between different regions and districts of Slovakia. The unsatisfied situation is also noticed among Slovak girls' interest in STEM subjects. European studies show a deteriorating interest in studying math, physics, chemistry, natural science, and informatics from the age of 12 years and rapidly starts to fall at the age of 15 years (Trotman, 2017). The interest of Slovak girls in STEM is even lower than the EU average. Researchers indicated as main reasons lack of women role models in science and practical experience with science, lack of teachers who can motivate students via attractive educational activities, the insufficient link between acquired knowledge and real-life scenarios, as well as insufficient personalization of education.

### 2.1.1. Review of educational games used to teach STEM in Slovakia

Currently, thousands of primary and secondary school teachers in Slovakia make use of learning objects provided by a virtual library available on the Zborovna.sk portal. This portal shares a large collection of educational materials, PowerPoint presentations, video clips, exercises, worksheets, and tests for many subjects. However, if teachers want to enrich their teaching with computer-based applications (such as games and virtual laboratories), they have to visit other portals.

Several educational websites, such as Matika.in (Probst, 2018) or Vieme matiku (Vieme to, 2018) exist for teaching mathematics concepts and resources are also available in the Slovak language. Resources can be used by both teachers and students from primary and secondary schools, as well as parents. Students are provided with several math problems, exercises, and 2D games for each school grade or topic. Teachers can work with a large number of problems, manage classes, track students' progress, and print problems. The resources available on the Matika.in website are based on the Hejny method of education (Hejný, 2012) and they integrate gamification elements such as badges and feedback to increase student motivation. However, clear studies on student motivation and knowledge acquisition when using those resources are missing. The authors of the Vieme matiku website run some studies but focused mainly on questions related to difficulty aspects of their adaptive educational system.
*GalaxyCodr* is an educational game that interactively teaches children algorithmic thinking and the basics of programming (GalaxyCodr, 2018). Children from the age of 8 years can learn the principles of information technologies on ten planets. The game is available in Slovak, English, German, and Czech languages. *GalaxyCodr* is also the name of a portal that allows teachers to manage classes, track students' progress, and work with didactical guides for different levels. Other portals teaching programming and providing similar functions are Code.org (Code.org, 2018) and Vieme programovat (Vieme to, 2018). Code.org offers students and teachers from K-12 the opportunity to create their own games. Unfortunately, there are no evaluation studies performed for games provided on these portals and for the *GalaxyCodr* game, which analyse concrete benefits in the case of Slovak students.

The *MapPie* application allows students (of all levels) to learn geography (Csicsolová et al., 2018). It is a free Android-based educational application with game elements and contains more than 90 maps, quizzes, memory games, encyclopaedias, and challenges. It supports the English and Slovak language and is suitable for students from primary schools, secondary schools, and even universities. The authors tested the application with a group of 7–12-year-old students and received positive feedback. However, the authors did not perform statistical analysis on their results.

### 2.1.2. Summary of STEM educational games in Slovakia

The reduced interest of the Slovak Ministry on STEM education is reflected in a lack of STEM-qualified people on the labour market. Trends show a decreasing level of mathematical skills among secondary school graduates and decreased motivation (for both boys and girls) to study technically oriented subjects. A limited number of national and international initiatives took place that aimed to train STEM teachers on how they can effectively extend their teaching in order to increase student motivation and knowledge acquisition.

<caption>**Table 1**</caption>

Students' knowledge gain and motivation in learning when using games and portals in Slovakian schools

<table><thead><tr><td>Game/portals</td><td>Knowledge gained through game-based learning</td><td>Motivation for learning when using an educational game</td></tr></thead><tbody><tr><td>Matika.in</td><td>No evaluation*</td><td>No evaluation*</td></tr><tr><td>Vieme matiku</td><td>Only partial evaluation</td><td>Only partial evaluation</td></tr><tr><td>GalaxyCodr</td><td>No evaluation</td><td>No evaluation</td></tr><tr><td>Code.org</td><td>No evaluation for Slovak students</td><td>No evaluation for Slovak students</td></tr><tr><td>Vieme programovat</td><td>Only partial evaluation</td><td>Only partial evaluation</td></tr><tr><td>MapPie</td><td>No evaluation</td><td>No statistical evaluation</td></tr></tbody></table>

*Note.* *Authors rely on the recognized Hejný method, but extensive studies are missing*

Slovak primary and secondary school teachers can improve their teaching using computer-based educational games, but these games are simple, two-dimensional, with limited freedom, and no or limited gamification elements. High-quality 3D computer games that focus on teaching STEM subjects and available in the Slovak language are still missing. Many teachers integrate some of the aforementioned games into their teaching, but quantitative research studies investigating the impact of game-based learning on knowledge acquisition in Slovak schools are missing, and teachers often rely
on their experience or generally spread information (e.g., games should motivate students to learn). Table 1 summarizes the data that support these conclusions.

## 2.2. STEM education in Ireland

There are an inadequate number of graduates to meet the workforce needs in STEM-related fields, which is why concerns are growing in Ireland. The reasons behind the insufficiency of graduates are the result of the low adoption of STEM topics and courses by students in secondary schools and third-level education. The lack of adoption of STEM topics is directly related to students' attitude towards STEM. Irish students demonstrate a considerable lack of interest in STEM at a young age because they do not see any relevance in their day-to-day living (Prendergast & Roche, 2018).

The problem of a low enrolment rate of students in the STEM area and the related shortfall of STEM graduates are addressed in existing research work. Factors that contribute to this challenge include structural issues in the Irish educational system, such as some topics that are not covered by all schools (Smyth & Hannan, 2006) and inadequate time allocated to STEM classes (Prendergast & O'Meara, 2017) as well as lack of access to technology (e.g., computers) in schools. Previous research work shows that the education system exacerbates the challenge of low enrolment rates in STEM courses. Nevertheless, (Osborne et al., 2003) and (Papanastasiou, 2000) affirm that students' approach toward STEM discipline with prejudices does not facilitate the situation. In addition, there is a difficulty in Ireland related to the responsibility of the post-primary education system that plans ahead young citizens to succeed in a "knowledge-based economy" (Quinn, 2012). The Irish government has recognized that STEM education is essential to the progress of the economic system, as well as a way to develop informed citizens in a technological society (OConnell, 2013).

Compared to the European Union with an average number of STEM graduates of 1.8%, 3.3% of the graduated Irish population got a degree in the STEM field (Piątkowski, 2020). Until 2008, Irish higher education institutions received increased funding to evolve knowledge in STEM and stimulate innovation (Hazelkorn & Moynihan, 2010). Since the global financial meltdown, Ireland adopted a strategy to increase the number of graduated people in STEM and facilitate access to careers' research in STEM (Roche et al., 2016).

Note that Irish STEM education is directly connected to old didactic teaching styles that focus on knowledge transmission with a passive role of students in the learning process and without any link with pragmatic examples from our daily life problems (Girvan et al., 2016).

Irish national plan for the period 2021 – 2026 mentions STEM Policy of Ireland that aims to foster the development of a more engaged and digitally-savvy society and a highly-skilled workforce. It also seeks to facilitate entry into STEM related career fields for all and to promote the development of computational thinking and digital skills (Jākobsone, 2021).

### 2.2.1. Review of educational games used to teach STEM in Ireland

The Lego Mindstorms series of kits (https://www.lego.com/en-us/mindstorms) provide effective resources for developing educational games. There are one of the educational games used in Ireland to teach STEM. They are made up of programs and equipment to build robots based on learning needs. Among the equipment, programmable bricks are
used to manage the robots, while sensors, electrical motors, and Lego bricks are used to
build them. Note that Lego Mindstorms kits are the fruit of cooperation between Lego
and the MIT Media Laboratory. In the context of the Empowering Minds project funded
by the Higher Education Authority of Ireland, (Martin et al., 2000) prove that the LEGO
Mindstorms kits have an important role for kids in terms of getting further insights and
for teachers helping them to be more creative than in a classic STEM session where
books are used.

The Department of Computer Science at Maynooth University created the Programming and Algorithms together teach Computational Thinking (PACT) group in collaboration with Irish post-primary schools (Mooney et al., 2014). The aim of the PACT group is to review the state of the Irish Computational Thinking education and develop resources and content for schoolteachers. This has been done through two workshops. The research study has shown that teachers and students appreciated the practical aspect of the materials that help learners better understand important concepts about computational thinking.

*Noughts and crosses* are an applet-based application used to teach Java programming language in Irish pre-secondary schools (Gibson, 2003). The application allows children to discover programming through rules, classes, and objects. It combines educational and funny aspects with a problem-based learning style to teach the Java language. To the best of our knowledge, there are no research papers on the evaluation of this applet application and its uses from a learning perspective.

*Math Elements* is a game developed by (Kiili et al., 2014) to introduce the mathematics curriculum for pre-secondary students. The learner has control over the pace of his mathematics learning by learning math concepts through an avatar. The game is used during normal class as a learning activity. The researchers evaluated the user experience of the game and found that the Math Elements game increased the motivation of children to learn maths. However, no research studies are conducted on its learning outcome.

2.2.2. Summary of STEM educational games in Ireland

The current state shows that there are some existing research works in the STEM field focusing on the design, development, and use of educational technologies and games in Ireland to teach STEM subjects. However, unfortunately, there is very limited work on the evaluation of these educational games from the motivation of the learner and the perspectives of STEM knowledge gained (see Table 2) when used in Irish primary schools.

In summary, and with the recognition of the local educational policy, the integration of technology-enhanced learning tools into STEM teaching in Irish primary schools is mandatory. This can be achieved more specifically through the design, development, implementation, and assessment of TEL tools. That is why this paper introduces an educational game developed for primary school students, and the results are presented and discussed in terms of the evaluation of the motivation of the learner and the learning outcomes of the game.

Based on this state-of-the-art, we can notice that there is a real need to support
STEM education in Slovakia and Ireland at primary school level not only by developing
educational games but also by assessing and adapting the developed games to ensure
improvement of children's STEM learning outcome and the enhancement of their motivation to learn such topics.

<table><caption>**Table 2**
Students' knowledge gain and motivation in learning via technologies and games in Irish schools</caption><thead><tr><td></td><td>Knowledge gained through the game</td><td>Motivation for learning with the game</td></tr></thead><tbody><tr><td>The Lego Mindstorms PACT initiative</td><td>No evaluation<br/>No evaluation, the researchers note that they use a declarative questionnaire for students, but there are no published results and analysis.</td><td>No evaluation<br/>No evaluation</td></tr><tr><td>The Noughts and Crosses applet app</td><td>No evaluation</td><td>No evaluation</td></tr><tr><td>Math Elements game</td><td>No evaluation</td><td>Researchers evaluate flow and playability in the game through a questionnaire. The results show that the players were very engaged with the game at the beginning, but the level of flow experience of the students decreased over time.</td></tr></tbody></table>

The next section presents the proposed new educational game called Final Frontier that supports learning concepts about planets from the Solar system based on the geography curriculum available in Slovakia and Ireland.

### 3. Final Frontier game

NEWTON is a large-scale EU Horizon 2020 project that aims to develop, integrate, and disseminate innovative technology-enhanced learning (TEL) methods and tools, and create new or interconnect existing state-of-the-art teaching labs (NEWTON, 2016). The NEWTON platform integrates and deploys many new and emerging mechanisms and TEL methodologies. Game-based learning and, more specifically, the Final Frontier interactive educational computer game is one of the TEL solutions designed, developed, and deployed in the context of the NEWTON project (El Mawas, Bratu, et al., 2019).

<figure><img src="image_4.png" /><figcaption>**Fig. 1.** Screenshots from the game: (a) the player on Mercury; (b) Information room for the Venus planet</figcaption></figure>
Final Frontier (see Fig. 1) is an interactive adventure-based educational computer game for children aged 10-12 years. The game was developed by the NEWTON Project consortium partner, National College of Ireland (NCI), and teaches concepts about space and the Solar system. The concepts and learning outcomes covered by the game are presented in Table 3. The game has a linear story and aims to immerse and motivate learners through direct experience, challenges, and fun. Four small planets, four gas giant planets, the Moon satellite, and the organization of the planets in the solar system are covered in the two parts of the game. The game supports two learning modes: through game levels, tasks/problems to be solved, and quizzes per level to be completed; or through a virtual interactive library, which exists on the spaceship, describing the concepts to be learned about the planets and the solar system. The levels are designed to give an accurate view of the planets in a cartoon-style form. On the screen, there is a game heads-up display (HUD) which shows information to the player, such as the star counter, meteorite counter, coolant time, and the game objective. It also displays the text box of what the avatar is saying and a teacher is present in the game telling the player what to do.

<caption>**Table 3**</caption>

Topics and expected learning outcomes covered in the Final Frontier game

<table><thead><tr><th>Planet</th><th>Learning outcomes</th></tr></thead><tbody><tr><td colspan="2"><strong>Final Frontier 1</strong></td></tr><tr><td>Mercury</td><td><ul><li>Closest planet to the Sun</li><li>Most cratered planet</li><li>Smallest planet</li></ul></td></tr><tr><td>Venus</td><td><ul><li>Hottest planet due to Greenhouse effect</li><li>Spins opposite direction to Earth</li><li>High Gravity cannot jump high</li></ul></td></tr><tr><td>Moon</td><td><ul><li>Solar Eclipse</li><li>Gravity differences</li><li>First person on the Moon</li></ul></td></tr><tr><td>Mars</td><td><ul><li>Liquid Water</li><li>Reddish in Colour Called the Red Planet</li><li>Smaller than Earth but Further Away from the Sun</li></ul></td></tr><tr><td colspan="2"><strong>Final Frontier 2</strong></td></tr><tr><td>Jupiter</td><td><ul><li>Largest Planet in the Solar System you can fit 1321 Earth's inside of Jupiter</li><li>5 times as far from the Sun as Earth and takes 12 years to go around the Sun.</li><li>The first of the Gas Giant planets.</li></ul></td></tr><tr><td>Saturn</td><td><ul><li>Famous for its rings, the rings are made up of small lumps of ice and dust</li><li>Second largest planet in the Solar system</li><li>Has 18 Moons</li></ul></td></tr><tr><td>Uranus</td><td><ul><li>It is the coldest planet</li><li>It is tipped on its side and spins sideways compared to Earth (like a barrel)</li><li>First planet discovered by a telescope</li></ul></td></tr><tr><td>Neptune</td><td><ul><li>Furthest planet from the Sun</li><li>18 hours to make one rotation</li><li>Very active climate – lots of storms</li></ul></td></tr></tbody></table>

The game begins with the player entering a spaceship and being told to go to the first planet, which is Mercury. On Mercury, the player must collect five meteors using the jetpack to get in and out of the craters; once they collect five, they are brought back to the
Spaceship. The player now must complete a puzzle about Mercury, and if they get it right, they gain access to the next planet Venus. On Venus, the player must cross the environment within the coolant time. Once the player gets to the last building, they are teleported back to the Spaceship, and they must complete a puzzle about Venus to unlock the next level, which is the Moon. On the Moon, the player must collect ten stars by jumping on platforms. Once they collected ten stars, they are teleported back to the Spaceship, where they must complete a puzzle about the Moon. The player then gains access to the next and final level Mars. On Mars, they must go through the Maze and find the liquid water. The player is teleported back to the Spaceship where they must complete a quiz. Similar stages are available for other planets as well. In this game, the students learn facts about each individual astronomical object presented in the game. More details on how educational principles have been incorporated into the functional design and architecture of this game can be found in (El Mawas et al., 2020).

The game was deployed through the NEWTON platform and tested in different European primary and secondary schools with more than 300 students. Various case studies that involved pilots run over 8-10 week duration have investigated the benefits of NEWTON technology-based education, including the use of educational games in different European schools (El Mawas, Truchly, et al., 2019). The aim of the research reported in this publication is to assess and analyse the student motivation and learning outcomes of the Final Frontier game across two European countries: Slovakia and Ireland.

# 4. Case studies: Research method and results analysis

This section presents the research and evaluation method applied, the case studies set-up and the analysis of the results for the collected data.

## 4.1. Case study 1: Slovakia

### 4.1.1. Research method

Lamač primary school from Bratislava was chosen to carry out the Slovak research study investigating the impact of game-based learning on the acquisition of students' knowledge, as well as their motivation and satisfaction with the learning process. This school is open to new learning technologies and teaches grades from the 1st class to the 9th class. The analysis of student motivation to learn, satisfaction with the learning process, and engagement in learning was performed using quiz questionnaires, observations, and interviews. Both standard teaching and educational game-based teaching approaches were used in the case study on two different groups of students.

For analysis and comparison of knowledge acquisition, for the standard and game-based learning approach, two classes (the same grade, students 12-13 years old) were selected. One randomly selected class of students was taught in a standard way, i.e., by the teacher explaining the Solar system topic using a PowerPoint presentation. This group is called a control group. The other class of students, called the experimental group, experienced a new learning approach through the Final Frontier game. The control group contained 25 children (16 boys and 9 girls) and the experimental group contained 24 children (12 boys and 12 girls). The case study consisted of two lessons; the first one dealt with rocky planets, while the second one focused on giant gas planets. The Final Frontier game is also divided into two corresponding parts, thus supporting the structure of the two lessons. The knowledge obtained by the students was measured using
knowledge tests. Students from the experimental and the control groups took the same pre-test at the beginning of each lesson and same post-test at the end of each lesson (see Fig. 2). These tests consisted of four single choice or simple answer questions with the aim of being finished in a few minutes. Table 5 illustrates the set of questions used in this study.

<figure><img src="image_5.png" /><figcaption>**Fig. 2.** Learning processes and class activities for experimental and control groups</figcaption></figure>

Fig. 2 illustrates the entire process implemented within our research study. It also depicts the differences in the learning steps of the experimental group and the control group. The assessment methodology was defined by the NEWTON Pedagogical Assessment Committee (Montandon et al., 2018). The questionnaires allow us to collect demographic information and analyse changes in students’ opinions, feelings, and attitudes toward various aspects after the learning process (lesson). They are based on the commonly used Likert scale and principles analysed in (Lovelace & Brickman, 2013). Pre- and post-tests consist of several multiple choices and short answer questions used to assess student knowledge. In the beginning, all students in the experimental group were informed about this study and data management. To meet ethical requirements, necessary approval was obtained from the relevant Ethics committee. Consent forms were delivered to all parents and corresponding Assent forms were given to students, which also took pre-study questionnaires (demographic and motivation). Then all students took a pre-test of Lesson 1. Students of the experimental group started to play (individually) the Final Frontier game (part 1) in self-directed mode (the teacher was involved only in case of technical problems). Students of the control group learned in a standard way (with the teacher teaching using presentation). At the end of lesson 1, all students took a post-test 1
and, moreover, experimental group students also took some post-study questionnaires (motivation and usability). The same steps for particular groups were performed within the second-class session. Selected students from the experimental group were interviewed at the end of the study. Students of the control group did not take any questionnaires because we concentrated on the impact of a new educational approach (game-based learning) on students learning and motivation and only the experimental group experienced it.

The entire learning process was supported via the NEWTELP learning management platform developed as part of the NEWTON project.

### 4.1.2. Demographic and pre-study motivation analysis

As mentioned and illustrated in Fig. 2, students of the experimental group took two types of the questionnaire before the lesson started. The results are summarized in Table 4 and Fig. 3. These results represent the attitude of the students towards school and the motivation to learn in the school and about the STEM fields. Only one student said that he likes school, but one-third of them do not. However, more than 84% always (or sometimes) get good marks (grades). More than half of the students are very or extremely interested in science classes. And the same number of students (44%) is somewhat confident or very/extremely confident in solving any/all of the problems and challenges they experience in their science classes. Regarding technology usage, almost all students use smartphones on a daily basis. 88% of students have access to a computer at home. Smartphones are used mainly for communication. About 70% search the Internet using smartphones and computers. And 48% of students use both technologies to play games and smartphones for help with homework.

<table><caption>**Table 4**
Demographic information of the experimental group in the Lamač school from Slovakia</caption><thead><tr><th>Question</th><th colspan="4">Choices/Answers</th></tr></thead><tbody><tr><td>How do you feel about school?</td><td>I don't like it (at all).</td><td>It's OK.</td><td>I like/love it.</td><td></td></tr><tr><td>How do you feel about learning STEM?</td><td>32%</td><td>64%</td><td>4%</td><td></td></tr><tr><td>Do you get good marks in STEM?</td><td>16%</td><td>52%</td><td>32%</td><td></td></tr><tr><td>How often do you use a smartphone?</td><td>Terrible/low marks<br>4%</td><td>Average marks<br>12%</td><td>Yes, sometimes/always<br>84%</td><td></td></tr><tr><td>You use a smartphone to:</td><td>Never<br>0%</td><td>Very rarely<br>0%</td><td>Once a week<br>0%</td><td>A few times a week<br>4%</td><td>Every day<br>96%</td></tr><tr><td>You use PC to:</td><td>Search Internet</td><td>Play games</td><td>Communicate with friends/family</td><td>Do schoolwork</td><td>Help you with homework</td></tr><tr><td>Do you play games on a gaming console?</td><td>68%<br>72%</td><td>48%<br>48%</td><td>84%<br>24%</td><td>20%<br>32%</td><td>48%<br>36%</td></tr><tr><td>Please rate how interested you are in your science classes?</td><td>Never<br>24%</td><td>Very rarely<br>32%</td><td>Occasionally<br>12%</td><td>Yes, sometimes<br>28%</td><td>Yes, as often as possible<br>4%</td></tr><tr><td>How confident are you that you can solve any/all of the problems and challenges you experience in your science classes?</td><td>Not at all<br>0%</td><td>Slightly interested<br>24%</td><td>Somewhat interested<br>20%</td><td>Very interested<br>44%</td><td>Extremely interested<br>12%</td></tr><tr><td></td><td>Not at all confident<br>0%</td><td>Slightly confident<br>12%</td><td>Somewhat confident<br>44%</td><td>Very confident<br>40%</td><td>Extremely confident<br>4%</td></tr></tbody></table>
<figure><img src="image_6.png" /><figcaption>**Fig. 3.** Pre-study motivation for learning STEM (experimental group)</figcaption></figure>

Fig. 3 shows the distribution of student responses to questions related to STEM fields and classes. More than half of the students would like to learn without textbooks. And the same percentage of students is not very enthusiastic about using textbooks. About half of the students are really interested in science classes, but only 44% enjoy learning in them. However, about 60% of students would like to use more technology in classrooms and want STEM lessons to be more active. Only 12% would change nothing about their STEM lessons. 40% of all students find learning STEM difficult and do not agree that it is boring. In general, an important number of students (e.g., 30 - 40%) had a neutral stance in answers.

### 4.1.3. Knowledge gain analysis

The applied methodology based on pre- and post- test knowledge tests (Table 5) allows us to evaluate the acquisition of knowledge for particular groups. Pre- and post-tests assess similar concepts (Table 6) based on the learning outcomes of the game presented in Table 3. For example, Q1 in the pre-test Final Frontier 1 and Q4 in the post-test Final Frontier 1 assess knowledge related to Venus, while Q1 in the pre-test Final Frontier 2 and Q2 in the post-test Final Frontier 2 assess knowledge related to Neptune.

Fig. 4 depicts the average scores of the pre- and post-tests for the control and experimental groups of students for both Final Frontier lessons. In general, it can be seen that both groups achieved higher average post-test scores in comparison with pre-test scores for both lessons. In other words, positive knowledge gain was achieved in all cases
and these improvements are also statistically significant (for paired t-tests for means with $\alpha = 0.05$), as can be seen in Table 7. When comparing the results between the two groups of students, the experimental group achieved a greater knowledge gain in both lessons.

<table><caption>**Table 5**
Pre- and post-test questions deployed in Final Frontier 1 and in Final Frontier 2 lessons</caption><thead><tr><th>Pre-test lesson 1 questions</th><th>Post-test lesson 1 questions</th><th>Pre-test lesson 2 questions</th><th>Post-test lesson 2 questions</th></tr></thead><tbody><tr><td><p><strong>Q1) Is Venus similar in size to the Earth?</strong></p><ul><li>a) Yes</li><li>b) No</li><li>c) I don't know</li></ul><p><strong>Q2) Which Planet has liquid water on it?</strong></p><ul><li>a) Mercury</li><li>b) Venus</li><li>c) Mars</li><li>d) I don't know</li></ul><p><strong>Q3) What does Mercury have a lot of?</strong></p><ul><li>a) Craters</li><li>b) Mountains</li><li>c) Water</li><li>d) I don't know</li></ul><p><strong>Q4) Neil Armstrong is the first person on:</strong></p><ul><li>a) Type answer:</li><li>b) I don't know</li></ul></td><td><p><strong>Q1) Which planet is called the Red Planet?</strong></p><ul><li>a) Mercury</li><li>b) Mars</li><li>c) Venus</li></ul><p><strong>Q2) Which Planet is closest to the Sun?</strong></p><p>Type answer:</p><ul><li>Q3) Can you jump much higher on Moon than on the Earth?</li></ul><ul><li>a) Yes</li><li>b) No</li></ul><p><strong>Q4) What is the temperature on Venus?</strong></p><ul><li>a) Very hot</li><li>b) Very cold</li><li>c) Like on Earth</li></ul></td><td><p><strong>Q1) Is Neptune the furthest planet from the Sun?</strong></p><ul><li>a) Yes</li><li>b) No</li><li>c) I don't know</li></ul><p><strong>Q2) Which Planet is the largest planet in the Solar System?</strong></p><ul><li>a) Neptune</li><li>b) Jupiter</li><li>c) Uranus</li><li>d) I don't know</li></ul><p><strong>Q3) How many moons Saturn has?</strong></p><ul><li>a) 15</li><li>b) 18</li><li>c) 20</li><li>d) I don't know</li></ul><p><strong>Q4) Which planet is the first planet discovered by a telescope?</strong></p><ul><li>a) Type answer:</li><li>b) I don't know</li></ul></td><td><p><strong>Q1) Which planet is famous for its rings made up of small lumps of ice and dust?</strong></p><p>Type answer:</p><ul><li>Q2) What planet is the furthest planet from the Sun?</li><li>a) Mercury</li><li>b) Neptune</li><li>c) Uranus</li></ul><p><strong>Q3) Is Jupiter the largest planet in the Solar System?</strong></p><ul><li>a) Yes</li><li>b) No</li></ul><p><strong>Q4) What planet is tipped on its side, like a barrel?</strong></p><ul><li>a) Uranus</li><li>b) Saturn</li><li>c) Earth</li></ul></td></tr></tbody></table>

<figure><img src="image_7.png" /><figcaption>**Fig. 4.** Average knowledge test scores of both groups in both Final Frontier lessons</figcaption></figure>
<table><caption>**Table 6**
Different knowledge concepts addressed in pre-test and post-test questions</caption><thead><tr><th rowspan="2">Concept</th><th colspan="2">Final Frontier 1</th><th rowspan="2"></th><th colspan="2">Final Frontier 2</th></tr><tr><th>Pre-test question</th><th>Post-test question</th><th>Pre-test question</th><th>Post-test question</th></tr></thead><tbody><tr><td>Mercury</td><td>Q3</td><td>Q2</td><td>Jupiter</td><td>Q2</td><td>Q3</td></tr><tr><td>Venus</td><td>Q1</td><td>Q4</td><td>Saturn</td><td>Q3</td><td>Q1</td></tr><tr><td>Moon</td><td>Q4</td><td>Q3</td><td>Uranus</td><td>Q4</td><td>Q4</td></tr><tr><td>Mars</td><td>Q2</td><td>Q1</td><td>Neptune</td><td>Q1</td><td>Q2</td></tr></tbody></table>

<table><caption>**Table 7**
Average knowledge test scores and t-test results for both groups in both Final Frontier lessons</caption><thead><tr><th rowspan="2"></th><th colspan="4">Experimental group</th><th colspan="4">Control group</th></tr><tr><th colspan="2">Final Frontier 1</th><th colspan="2">Final Frontier 2</th><th colspan="2">Final Frontier 1</th><th colspan="2">Final Frontier 2</th></tr><tr><th></th><th>pre-test</th><th>post-test</th><th>pre-test</th><th>post-test</th><th>pre-test</th><th>post-test</th><th>pre-test</th><th>post-test</th></tr></thead><tbody><tr><td>Mean</td><td>76.32%</td><td>89.47%</td><td>69.44%</td><td>90.28%</td><td>85.42%</td><td>93.75%</td><td>70.19%</td><td>87.50%</td></tr><tr><td>SD</td><td>26.97%</td><td>15.17%</td><td>26.50%</td><td>15.19%</td><td>22.01%</td><td>13.29%</td><td>21.24%</td><td>22.64%</td></tr><tr><td>Knowledge gain</td><td>13.15%</td><td></td><td>20.84%</td><td></td><td>8.33%</td><td></td><td>17.31%</td><td></td></tr><tr><td>t / p (t-test results)</td><td>-2.127 / 0.046</td><td></td><td>-3.589 / 0.002</td><td></td><td>-2.326 / 0.0291</td><td></td><td>-5.196 / 0.000</td><td></td></tr></tbody></table>

However, it is necessary to note that the students in the control group started with a higher level of knowledge about rocky planets and the Moon at the beginning of the Final Frontier 1 lesson. Students in this group had already been informed of some of this information on other events before the study was conducted. In the case of the second lesson, both groups started with the same level of initial knowledge about gas giant planets. Using both pedagogical approaches (based on games and standard), a high level of knowledge was achieved after both lessons (higher than 88%). Based on the results of a t-test for independent groups calculated on post-test scores for Final Frontier 1 (t(36)=2.028, p=0.339, Cohen's d=-0.302) and Final Frontier 2 (t(42)=2.018, p=0.628, d=0.139) at a confidence level of 0.05, we can say that both groups achieved a statistically comparable level of knowledge in both lessons.

Based on the results of the ANCOVA analysis (F(1,40)=0.235, p=0.631, effect size = 0.006 for the Final Frontier 1 lesson) and (F(1,41)=0.38, p=0.541, effect size = 0.009 for the Final Frontier 2 lesson), we can also conclude that both groups do not differ significantly on post-test scores in both lessons after adjusting for pre-test scores.

Fig. 5 shows average scores for pre- and post-test questions for both Final Frontier lessons and particular groups of students. These results allow us to analyse the difficulty of individual questions, and some of them can represent feedback for future improvement of knowledge tests (or learning objects). Low scores for post-test questions indicate that the questions are too difficult for students and it is necessary to focus more on them during learning. In our case, the majority of post-test questions achieved scores higher than 80%. On the other hand, high scores on pre-test questions can indicate high initial knowledge of students or too easy (well-known) knowledge in the field. We can see that only Question 4 in the Final Frontier 2 pre-test achieved a low score of about 32% (experimental group)/27% (control group). All other pre-test questions reported quite high scores, i.e., students, in general, had a certain level of knowledge in this field.
<figure><img src="image_8.png" /><figcaption>Fig. 5. Average scores for pre- and post-test questions for each class session</figcaption></figure>

4.1.4. *Student post-study motivation analysis*

The motivation and satisfaction of the students with the game-based learning approach
were investigated via pre- and post-study questionnaires that also contained mutually
related paired questions. In this way, we could compare the attitudes of the students
before and after the lessons. The concept of questionnaire questions was based on a five-
level Likert-type scale with items: Strongly disagree, Disagree, Neutral, Agree, and
Strongly agree. The analysis substituted these words for values from 1 to 5 (1
corresponds to Strongly disagree and 5 to Strongly agree).

<figure><img src="image_9.png" /><figcaption>**Fig. 6.** Average score for questionnaire questions (experimental group)</figcaption></figure>

Fig. 6 shows the results for four questions (statements) that the students in the experimental group answered. The most important change in students’ feelings was expected after the Final Frontier 1 (i.e., first) lesson. The analysis of results indicates a statistically significant positive change in the feelings of the students about the NEWTON lessons, a really interesting result compared to the standard science classes experienced by the students at the school. We also observed a statistically positive change in the motivation of students to learn with NEWTON. It is confirmed by a negative
decrease for the statement: I would prefer to learn without NEWTON. More students agreed that the NEWTON classes have made them more interested in STEM. And there is also a statistically positive change in students' opinion that learning STEM is not boring. For three questions, we can see a slight decrease in average scores after the Final Frontier 2 lesson (compared to average scores after the Final Frontier 1 lesson). It probably relates to the fact that Final Frontier 2 can be more characterized as a virtual library. In this library, students find various textual information about gas planets in contrast to Final Frontier 1 where they can freely explore and complete interactive tasks on the rocky planets and achieve awards (e.g., collect stars).

### 4.1.5. Discussion

The question *What is knowledge acquisition by students?* is one of the important questions that need to be answered when a new pedagogical approach is tested. The pedagogical approach used in this case study involved self-directed game-based learning and was tested in a Slovakian primary school with children aged 12-13 years of age. The results of the knowledge tests applied within this research study showed that this pedagogical approach can achieve comparable efficiency in comparison with a standard pedagogical approach and if the game used in the learning process provides a balance between the amount of textual and non-textual information, provides freedom of movement and gamification elements, then it can report even better assessment results.

Another critical aspect of learning is related to the problem of how to motivate students to learn and make them happy with the learning experience. In this research study, the Final Frontier game teaching the Solar system subject was used. Based on the student's responses to the questionnaire, we can see that this approach really attracted most of the students. They expressed that they would not like to learn only using textbooks but also using the Final Frontier game. They saw the learning experience as fun and entertaining and, as a consequence, it increased their interest in STEM fields. However, the core of success lies in the quality of a game used in the learning process, and a very important focus must be put on its development.

## 4.2. Case study 2: Ireland

### 4.2.1. Research method

A large number of students from two primary schools, randomly divided into two groups, participated in a case study conducted in Dublin, Ireland. One group of children was taught using the Final Frontier game in the class (experimental group). Another group of children was taught the same concepts by the teacher through PowerPoint slides (control group). The learning activities took place in the class, during the regular hours of study in two different sessions. A total of 67 children of 9-10 years of age from Saint Patrick Boys National School and Corpus Christi Girls' National School located in Dublin, Ireland, participated in the case study. Children from the two schools were randomly divided into two groups: 29 children were in the control group (14 girls and 15 boys), and 38 children (17 girls and 21 boys) were in the experimental group. Researchers from the NEWTON project at National College of Ireland and Dublin City University have prepared and helped to perform the tests.

The same research methodology deployed in the Slovakian school and described above was deployed in the case study run in Ireland. However, the directors of the two
schools requested the game also be provided to the control group for fairness reasons after running the case study. That is why all children (even in the control group) answered the pre-study questionnaires. After answering the post-test Part 2 questions, the control group played the game before answering the post-study questionnaires.

### 4.2.2. Demographic and pre-study motivation analysis

The distribution of the students' answers (both control and experimental groups) for some parts of the pre-study questionnaires (dedicated to demographic data, the attitude of students towards a school and motivation to learn STEM fields) is detailed in Table 8 and Fig. 7. Only 6% of the students said they do not like school, while more than 79% always (or sometimes) get good marks (grades). More than 73% of students are very or extremely interested in science classes. However, fewer students (28%) are somewhat confident or very/extremely confident in solving any/all of the problems and challenges they experience in their science classes. Regarding technology usage, 53% of students are almost daily users of smartphones. 79.1% of students have access to a computer at home. Smartphones are used primarily to play games. Many students use smartphones (52%) and computers (54%) to search on the Internet. Also, students use PCs more than smartphones to play games, and on average 27% of students use both smartphones and PCs to help with homework.

<figure><img src="image_10.png" /><figcaption>**Fig. 7.** Distribution of students' answers for the STEM-related questions in the pre-study questionnaire (all students)</figcaption></figure>

The distribution of students' answers to questions related to STEM fields and STEM classes is shown in Fig. 7. It shows that 47% of students would prefer to learn without textbooks. 31% of students disagree that using textbooks makes them more enthusiastic. 81% of students are really interested in science classes and feel really positive about learning in these classes. Also, 76% of students would like to use more technology in classrooms and 73% of students would like STEM lessons to be more active. 46% of the students would change nothing about their STEM lessons. 9% of all students find learning STEM difficult and 83% of students do not agree that it is boring.
### 4.2.3. Knowledge gain analysis

An analysis of the students' responses in the pre- and post-tests allows us to evaluate knowledge acquisition and compare results between the control and experimental groups. The average scores of pre- and post-tests for the two groups of students for Final Frontier 1 and 2 lessons are presented in Fig. 8. In general, except for the pre-test in the Final Frontier 1 lesson for the control group, all groups in both lessons achieved a higher average of correct answers in post-test than in the pre-test. This means that the use of the Final Frontier game is the reason for statistically significant improvements in knowledge gain (paired t-tests for means with $\alpha = 0.05$). After the use of the Final Frontier 1 lesson, the experimental group achieved a statistically significant higher knowledge gain (32%) compared to the control group (-6%). This is also confirmed by a t-test for independent groups calculated over the knowledge gain with the results $t(66)=1.997, p=2.85E-07$, and Cohen's $d=1.311$.

<table><caption>**Table 8**
Demographic information of all students</caption><thead><tr><th rowspan="2">Question</th><th colspan="4">Choices/Answers</th></tr><tr><th colspan="4">K<br>(K1, K2)</th></tr></thead><tbody><tr><td>How do you feel about school?</td><td>I don't like it (at all).<br>6%<br>(3%, 8%)</td><td>It's OK.<br>31%<br>(28%, 34%)</td><td>I like/love it.<br>63%<br>(69%, 58%)</td></tr><tr><td>How do you feel about learning STEM?</td><td>1%<br>(0%, 3%)</td><td>15%<br>(21%, 11%)</td><td>84%<br>(79%, 87%)</td></tr><tr><td>Do you get good marks in STEM?</td><td>Terrible/low marks<br>2%<br>(3%, 0%)</td><td>Average marks<br>19%<br>(28%, 13%)</td><td>Yes, sometimes/always<br>79%<br>(69%, 79%)</td></tr><tr><td rowspan="3">How often do you use a smartphone?</td><td>Never<br>16%<br>(14%, 18%)</td><td>Very rarely<br>20%<br>(14%, 24%)</td><td>Once a week<br>11%<br>(14%, 8%)</td><td>A few times a week<br>37%<br>(41%, 34%)</td><td>Every day<br>16%<br>(17%, 16%)</td></tr><tr><td>Search<br>Internet</td><td>Play games</td><td>Communicate<br>with<br>friends/family</td><td>Do schoolwork</td><td>Help you with<br>homework</td></tr><tr><td>You use a smartphone to:</td><td>52%<br>(62%, 45%)</td><td>55%<br>(41%, 66%)</td><td>42%<br>(48%, 37%)</td><td>10%<br>(10%, 11%)</td><td>30%<br>(24%, 34%)</td></tr><tr><td>You use PC to:</td><td>54%<br>(59%, 50%)</td><td>55%<br>(55%, 55%)</td><td>12%<br>(17%, 8%)</td><td>13%<br>(10%, 16%)</td><td>27%<br>(24%, 29%)</td></tr><tr><td>Do you play games on a gaming console?</td><td>Never</td><td>Very rarely</td><td>Occasionally</td><td>Yes, sometimes</td><td>Yes, as often as<br>possible</td></tr><tr><td rowspan="2">Please rate how interested you are in your<br>Science classes?</td><td>12%<br>(7%, 16%)</td><td>20%<br>(17%, 21%)</td><td>21%<br>(17%, 24%)</td><td>31%<br>(52%, 16%)</td><td>16%<br>(7%, 24%)</td></tr><tr><td>Not at all<br>interested<br>2%<br>(3%, 0%)</td><td>Slightly<br>interested<br>9%<br>(14%, 5%)</td><td>Somewhat<br>interested<br>16%<br>(14%, 18%)</td><td>Very interested</td><td>Extremely interested</td></tr><tr><td>How confident are you that you can solve<br>any/all of the problems and challenges you<br>experience in your science classes?</td><td>Not at all<br>confident<br>5%<br>(10%, 0%)</td><td>Slightly<br>confident<br>15.5%<br>(7%, 24%)</td><td>Somewhat<br>confident<br>52.5%<br>(66%, 39%)</td><td>Very confident</td><td>Extremely confident</td></tr></tbody></table>

Note. K = the percentage of all students selecting the choice. K1 = the percentage of the students in the control group selecting the choice. K2 = the percentage of the students in the experimental group selecting the choice.
<figure><img src="image_11.png" /><figcaption>**Fig. 8.** Average knowledge test scores of both groups in both Final Frontier lessons</figcaption></figure>

The ANCOVA analysis of post-test data for the Final Frontier 1 lesson (F(1,72)=0.235, *p*=8.8E-05, effect size = 0.194) also showed that the experimental and control groups differ significantly in post-test scores when pre-tests are treated as a covariate variable. At the end of the Final Frontier 2 lesson, both groups achieved positive knowledge gain. However, the control group achieved a gain more than two times higher (50%) than the experimental group (23.3%) (the results for the independent sample t-test are *t*(62)=1.999, *p*=0.0012, *d*=-0.805). Based on the results of the ANCOVA analysis (F(1,70)=0.789, *p*=0.377, effect size = 0.011) performed for the Final frontier 2 lesson we can also see that both groups (experimental and control groups) do not differ significantly in post-test scores (when pre-test scores represent a covariate).

These results can be explained as follows: (1) the learning outcomes for children that have been exposed to Final Frontier 1 are higher than the learning outcomes of children that have been taught via the classic teaching approach, and (2) the learning outcomes for children that have been exposed to Final Frontier 2 (explore the virtual library and read and learn about gas planets) are lower than the learning outcomes of children that have been taught by the teacher via the classic approach. The calculated values of the means, standard deviations, knowledge gains, and the t and p values of pair t-tests can be found in Table 9.

<caption>**Table 9**</caption>

Average knowledge test scores and t-test results for both groups in both Final Frontier lessons

<table><thead><tr><th rowspan="3"></th><th colspan="4">Experimental group</th><th colspan="4">Control group</th></tr><tr><th colspan="2">Final Frontier 1</th><th colspan="2">Final Frontier 2</th><th colspan="2">Final Frontier 1</th><th colspan="2">Final Frontier 2</th></tr><tr><th>pre-test</th><th>post-test</th><th>pre-test</th><th>post-test</th><th>pre-test</th><th>post-test</th><th>pre-test</th><th>post-test</th></tr></thead><tbody><tr><th>Mean</th><td>52.72%</td><td>84.78%</td><td>55.11%</td><td>78.41%</td><td>71.55%</td><td>65.52%</td><td>34.48%</td><td>84.48%</td></tr><tr><th>STD</th><td>24.28%</td><td>20.05%</td><td>20.59%</td><td>26.19%</td><td>21.88%</td><td>18.19%</td><td>25.37%</td><td>19.38%</td></tr><tr><th>Knowledge gain</th><td>32.06%</td><td></td><td>23.30%</td><td></td><td>-6.03%</td><td></td><td>50.00%</td><td></td></tr><tr><th>t / p (t-test results)</th><td>-7.112 / 6.95-E09</td><td></td><td>-4.566 / 4.13-E05</td><td></td><td>1.229 / 0.229</td><td></td><td>-0.0078 / 3.86E-09</td><td></td></tr></tbody></table>

The average scores for pre- and post-test questions for Final Frontier 1 and 2 for the control and experimental groups are presented in Fig. 9. These results allow us to analyse each question in the pre- and post-tests. Low scores for post-test questions may indicate that the concepts checked by the question were not clearly understood by the
students in the game/classic teaching approach. Note that students can also face
difficulties related to understanding the questions.

<figure><img src="image_12.png" /><figcaption>Fig. 9. Average scores for pre- and post-test questions</figcaption></figure>

Regarding Final Frontier 1, the knowledge gain for the experimental group is 79.83% for the Mars planet, 32.48% for the Venus planet, 19.5% for the Mercury planet, and 1.38% for the Moon. The knowledge gain for the control group is 31.05% for the Mercury planet, 4.05% for the Venus planet, -5.17% for the Mars planet, and -54.97% for the Moon. On the basis of these results, we can conclude that the overall knowledge gain is higher in the experimental group than in the control group. The learning outcome of the game-based approach is higher than that of the classic teaching approach. We believe that the high knowledge gain of Mercury in the control group is related to the fact that Mercury was the first concept explained by the teacher that is why students pay attention and have better learning outcomes than other concepts in the classic learning approach.

Regarding Final Frontier 2, the knowledge gain for the experimental group is 52.6% for the Uranus planet, -1.72% for the Neptune planet, -8.39% for the Saturn planet, and -34.97% for the Jupiter planet. The knowledge gain for the control group is 75% for the Saturn planet, 69.29% for the Uranus planet, 38.81% for the Neptune planet, and 17.62% for the Jupiter planet. On the basis of these results, we can conclude that the overall knowledge gain is higher in the control group than in the experimental group: the learning outcome of the classic teaching approach is higher than in the interactive library where students were required to read information about planets.

4.2.4. *Student post-study motivation analysis*

In order to measure the motivation of Irish students, the same post-study questionnaire
provided in the Slovakia case study was used.
Regarding the control group, students' responses to the post-study questionnaire are presented in Fig. 10. After students' exposure to Final Frontier 1 at the end of the case study, the results do not report a statistically significant change in students' feelings about learning STEM/Science compared to a classic teaching approach. After Final Frontier 2, students found the NEWTON lessons less interesting than the classic teacher-based lessons and Final Frontier 1 lesson. A statistically positive change in motivation to learn with NEWTON is observed after Final Frontier 1 with a slight decrease in student motivation after Final Frontier 2. Students consider that NEWTON classes have made them more interested in STEM after Final Frontier 1 than after Final Frontier 2. Fig. 10 shows that students find the classic teaching approach more boring than the NEWTON classes after Final Frontier 1 and even less boring than the NEWTON classes after Final Frontier 2.

<figure><img src="image_13.png" /><figcaption>Fig. 10. Control group's perceptions about STEM learning after exposure to the games</figcaption></figure>

<figure><img src="image_14.png" /><figcaption>Fig. 11. Experimental group's perceptions about STEM learning after exposure to the games</figcaption></figure>
Fig. 11 depicts the responses of the students in the experimental group to the post-study questionnaire. Results do not report statistically significant changes in students' feelings that NEWTON lessons are really interesting compared to classic teacher-based classes. We also observed a statistically positive change in the motivation to learn with NEWTON. This is confirmed by a negative decrease for the statement: I would prefer to learn without NEWTON. The results did not report a statistically significant change in the feeling that the NEWTON classes have made them more interested in STEM. Additionally, the results do not report a statistically significant change in the feeling that the NEWTON classes have made them more interested in STEM. Students find Final Frontier 2 more boring after Final Frontier 1 than the classic teaching approach and even more boring than NEWTON classes after Final Frontier 1.

Based on the post-study questionnaire results, we can conclude that students prefer Final Frontier 1 because they can play and interact freely with different objects (stars, meteors, jetpack, etc.) and with the non-player character in a gamified learning environment. They appreciate less the Final Frontier 2 with reading activities in the digital library. Furthermore, based on Fig. 10 and Fig. 11, students from the control group found learning STEM through NEWTON lessons less boring than students from the experimental group. This may be due to the fact that the students in the experimental group participated in all the evaluation steps of the game, including the knowledge tests, while the students in the control group played the games only for fairness reasons and had already studied the concepts taught by the teacher in the class.

### 4.2.5. Discussion

*What is the added value for students in terms of knowledge acquisition?* is one of the important questions to be addressed in the evaluation of a new pedagogical approach. The pedagogical approach used in this case study involved self-directed game-based learning and was tested in an Irish primary school on children aged 9-10 years. The results of the knowledge tests applied within this research study showed that the game-based approach can achieve a higher learning outcome compared to a standard pedagogical approach, while the standard pedagogical approach can achieve a higher learning outcome compared to an interactive library. This stresses the positive impact of movement freedom and gamification on learning (Kirginas & Gouscos, 2017).

*What is the impact on motivation?* is another important question to be addressed in the evaluation of a new pedagogical approach. In this research study, the Final Frontier game teaching the Solar system subject was used. Based on the student's responses to the questionnaire, we can see that this approach did not specifically attract them to STEM lessons compared to classic teacher-based classes. Their preference to learn without textbooks and by using the Final Frontier game increased after being exposed to Final Frontier 1 and Final Frontier 2. The game did not particularly change their interest in STEM fields. However, they found STEM lessons less boring after being exposed to Final Frontier 1.

## 5. Discussion and comparison of the two case studies

This section discusses and compares the two case studies that were conducted in three primary schools: one mixed school located in Bratislava, Slovakia, and two boys' and girls' schools only located in Dublin, Ireland.
The first research question we tried to answer when analysing the two case studies is: *Do students from the two countries have the same feeling about learning STEM?*

Regarding the students involved in the research study, Table 10 shows that the students in Ireland like/love school and STEM a lot more than the students in Slovakia. The percentages are calculated by subtracting the results obtained in Ireland from the results in Slovakia (Slovakia-Ireland). However, students from Slovakia achieved slightly better grades than students from Ireland and they use smartphones every day, which explains their frequent use to search the Internet, communicate with friends/family, do schoolwork, and help them with homework. In this sense, a study by the Slovak Savings Bank shows that Slovak students have started using cell phones since the age of six. This result may also be related to the lack of access to technology in Irish schools (Prendergast & O'Meara, 2017) and the low use of smartphones by Irish students (see Table 8).

<caption>**Table 10**</caption>

Comparison of the students' responses to the pre-study questionnaires (Slovakia – Ireland)

<table><thead><tr><th rowspan="2">Question</th><th colspan="4">Choices/Answers K</th></tr><tr><th>I don't like it (at all).</th><th>It's OK.</th><th>I like/love it.</th><th></th></tr></thead><tbody><tr><td>How do you feel about school?</td><td>26%</td><td>33%</td><td>-59%</td><td></td></tr><tr><td>How do you feel about learning STEM?</td><td>15%</td><td>37%</td><td>-52%</td><td></td></tr><tr><td>Do you get good marks in STEM?</td><td>Terrible/low marks<br>2%</td><td>Average marks<br>-7%</td><td>Yes, sometimes/always<br>5%</td><td></td></tr><tr><td>How often do you use a smartphone?</td><td>Never<br>-16%</td><td>Very rarely<br>-20%</td><td>Once a week<br>-11%<br>Communicate<br>with<br>friends/family</td><td>A few times a week<br>-33%</td><td>Every day<br>80%</td></tr><tr><td></td><td>Search<br>Internet</td><td>Play games</td><td>Do schoolwork</td><td>Help you with<br>homework</td><td></td></tr><tr><td>You use a smartphone to:</td><td>16%</td><td>-7%</td><td>42%</td><td>10%</td><td>18%</td></tr><tr><td>You use PC to:</td><td>18%</td><td>-7%</td><td>12%</td><td>19%</td><td>9%</td></tr><tr><td>Do you play games on a gaming console?</td><td>Never<br>12%</td><td>Very rarely<br>12%</td><td>Occasionally<br>-9%</td><td>Yes, sometimes<br>-3%</td><td>Yes, as often as<br>possible<br>-12%</td></tr><tr><td>Please rate how interested you are in your science classes?</td><td>Not at all<br>-2%</td><td>Slightly<br>interested<br>15%</td><td>Somewhat<br>interested<br>4%</td><td>Very interested<br>-10%</td><td>Extremely<br>interested<br>-7%</td></tr><tr><td>How confident are you that you can solve any/all of the problems and challenges you experience in your science classes?</td><td>Not at all<br>-5%</td><td>Slightly<br>confident<br>-4%</td><td>Somewhat<br>confident<br>-7%</td><td>Very confident<br>16%</td><td>Extremely<br>confident<br>0%</td></tr></tbody></table>

*Note.* K = the percentage of the students from Slovakia selecting the choice minus the percentage of the students from Ireland selecting the choice

Students from Ireland play games more often on PC and smartphones than students from Slovakia. This result is not surprising as the Assistant general secretary of the Association of Secondary Teachers Ireland (ASTI), Moira Leydon, said that teachers had been “expressing concern about the overuse and amount of time children are spending gaming”. In general, there is no significant difference between the two countries in terms of interest in Science classes. In addition, students from Slovakia feel more confident about solving any/all of the problems and challenges that they experience in their science classes. The positive impact of problem-solving learning on the confidence of Slovakian students has already been proven in previous studies (Tick, 2007). Note that
(Ríordáin & Hannigan, 2011) explains that Irish teachers do not consider themselves sufficiently qualified to teach STEM; this can have an impact on students' confidence.

Students from Slovakia agree more that learning STEM is boring/difficult, that they would prefer to learn without textbooks, and that the Science classes have made them more interested in STEM (Fig. 12). However, students from Ireland would like STEM lessons to be more active and Science classes to include more technology use due to old didactic teaching styles of Irish STEM education that focus on knowledge transmission with a passive role of the students in the learning process (Girvan et al., 2016). Students from Slovakia disagree more about the facts that: they feel positive about learning in science classes, they find them really interesting, and they enjoy learning during these classes. This is consistent with European studies on the decrease in interest of Slovak students in studying STEM (Trotman, 2017).

<figure><img src="image_15.png" /><figcaption>**Fig. 12.** Distribution of students' answers for other questions in the pre-study questionnaires</figcaption></figure>

Therefore, the answer to the question (*Do students from the two countries have the same feeling about learning STEM?*) is No.

Students in the two countries do not have the same feelings. Some differences result from many aspects (learning concepts used in science classes, teacher approach, IT skills, social situation, student interests, age, etc.). Students in Ireland are more interested in STEM; most of them enjoy science classes, see them as interesting, and are really positive about learning them. They like school. However, there is a smaller number of
students from Slovakia (compared to Ireland) interested in STEM. Only half of all students are interested in science classes, and even fewer enjoy learning science or feel really positive about it. They are less motivated to like school and learning STEM. On average, one-third of students often have a neutral attitude to questions asked.

This result can be explained by the fact that the Slovak Ministry has a weak focus on STEM education, while Irish educational policy acknowledges the importance of integrating technology meaningfully into STEM teaching and learning (see Section ‘Related work’).

However, more than half of students from both countries would like to have more technology and more interactive activities as part of STEM lessons, and half of the students (on average) would prefer to learn without textbooks. The same findings can be found in other studies such as (Ezeoguine & Augustine, 2021; Yien et al., 2011; Rosas et al., 2003).

The second research question that we have tried to address as part of this research work is the following: *Do students from the two countries achieve the same knowledge gain after using the Final Frontier game?*

<figure><img src="image_16.png" /><figcaption>**Fig. 13.** Comparison of knowledge gains between the two countries</figcaption></figure>

Fig. 13 shows the improvement of the average knowledge between post- and pre-tests in the two countries. Regarding lesson 1, the experimental group in Ireland had a higher knowledge gain than the experimental group in Slovakia, while in the case of control groups, the control group in Slovakia achieved a bigger improvement. Regarding lesson 2, the experimental groups in both countries achieved a comparable positive improvement in knowledge gain, while for the control groups, the control group in Ireland showed a bigger improvement.

Table 11 shows data of the t-test for independent groups calculated over the post-tests for the same type of groups (experimental and control groups) of students from both countries and for both lessons. The experimental groups of both countries during the Final Frontier 1 lesson achieved a statistically comparable level of knowledge (even though the level of knowledge gained was different). During the Final Frontier 2 lesson, the experimental groups acquired a high but statistically different level of knowledge. They achieved the same knowledge gain, but a different character of the Final Frontier 2
game probably did not provide high-level motivation (engagement) to learn. The control groups taught by the classical approach showed significantly different levels of knowledge (related to average post-test scores, see Table 11) after the Final Frontier 1 lesson, and the Irish control group achieved even negative knowledge gain. The reason behind the negative knowledge gain may be related to the old didactic teaching styles of Irish STEM education that focus on knowledge transmission with a passive role of the students in the learning process (Girvan et al., 2016). After the Final Frontier 2 lesson, the control groups achieved a statistically comparable level of knowledge (average post-test scores) supported by positive knowledge gains. The knowledge gain for the Irish control group was markedly higher because they started with a very low level of knowledge (the average pre-test score was 34.48%).

<table><caption>**Table 11**
Independent sample t-test results on post-tests</caption><thead><tr><th></th><th>Means (SK/IE)</th><th>Variances (SK/IE)</th><th>t value</th><th>p value</th><th>Cohen's d</th></tr></thead><tbody><tr><td>EG FF1</td><td>89.5% / 84.8%</td><td>2.3% / 4.02%</td><td>t(44)=2.015</td><td>0.31</td><td>0.25</td></tr><tr><td>CG FF1</td><td>93.8% / 65.5%</td><td>1.76% / 3.31%</td><td>t(50)=2.009</td><td>3.43E-08</td><td>1.75</td></tr><tr><td>EG FF2</td><td>90.3% / 78.4%</td><td>2.3% / 6.85%</td><td>t(53)=2.005</td><td>0.03</td><td>0.503</td></tr><tr><td>CG FF2</td><td>87.5% / 84.5%</td><td>5.13% / 3.76%</td><td>t(50)=2.009</td><td>0.6</td><td>0.144</td></tr></tbody></table>

*Note.* EG: experimental group; CG: control group of both countries; SK: Slovakia; IE: Ireland; FF1: Final Frontier 1; FF2: Final Frontier 2

Research analysis shows that student's part of the experimental groups (i.e., students who played Final Frontier 1 and 2 games) from both countries achieved positive and statistically significant knowledge improvement. These findings correspond with results acquired in (Liu & Chen, 2013; Ezeoguine & Augustine, 2021; Yien et al., 2011). However, (Lin & Liu, 2009) emphasised that the design of the educational game must efficiently combine entertainment and education aspects. The results also confirmed that game-based learning can provide comparable knowledge acquisition, such as the classic teacher-based approach in both countries. The knowledge gain achieved in both countries slightly differs because students started with a different level of initial knowledge, a different attitude to school, STEM learning, age, etc.

The third question addressed in this research is *Does the Final Frontier game have the same motivation impact on the students from the two countries?* As can be seen in Fig. 14, the Final Frontier game does not have the same motivational impact on the students from the two countries. Students from Slovakia reported a (statistically significant) positive change in three motivation-related questions after playing the Final Frontier 1 game. This finding correlates with an increase in motivation reported in studies from other countries (Tüzün et al., 2009; Lim, 2008; Ezeoguine & Augustine, 2021; Sedighian & Sedighian, 1996). They said that playing the game was more interesting than classic classes, they did not want to learn without that game, and that the game-based class was not boring. It reflects that only less than half of the students said they enjoyed learning science classes and more than half of the students wanted more technology and activity (and fun as well) inside STEM lessons. However, Slovakian students did not report a significant change in increased interest in STEM. Students would like to combine game-based learning with teacher-based teaching (as some of them said in interviews). The same outcome was concluded in (Lin & Liu, 2009).

The majority of students from Ireland expressed in pre-study questionnaires that they enjoy learning in their science classes, and that is probably a reason why they consider NEWTON classes (i.e., Final Frontier game-based learning) equally interesting
and they did not report any change in their increased interest in STEM. The same results, that is, no changes in the attitudes of students toward a course with educational computer games achieved authors in (Çankaya & Karamete, 2009). Before the course, the students also had a positive attitude towards that course. However, they would not like to learn without the Final Frontier game because they still require more technology in classes.

<figure><img src="image_17.png" /><figcaption>**Fig. 14.** Difference between average scores for the (pre-/post-study and post-/post-study) questionnaire questions (experimental group) in the two countries</figcaption></figure>

**6. Conclusion**

This research study investigates the motivation and learning outcomes of an interactive computer-based educational game in two European countries: Ireland and Slovakia. The interactive computer game called Final Frontier was designed, implemented and tested with children (boys and girls) from two Dublin-based schools and one Bratislava-based school. The game covers concepts related to the rocky planets, the giant gas planets, the Moon satellite and the organization of the planets in the Solar system. The game supports two learning modes. In the first mode (Final Frontier 1), the student learns through game levels, while in the second mode (Final Frontier 2) through a virtual, interactive library that requires reading and looking at images. An analysis of the results collected in a case study conducted on a group of 49 students in Bratislava and 67 students in Dublin was also presented in the paper. Knowledge acquisition and learning motivation through the Final Frontier game were investigated through a survey. The research outcomes show that students' interest in science classes is similar in Slovakia and Ireland. However, the case study reveals that the knowledge gain is different during and after exposure to the game in the two countries. On the other hand, students of both countries taught by our
game (Final Frontier 1) achieved a high and comparable level of acquired knowledge (calculated as post-test scores). Therefore, in both countries, game-based learning can represent a very competitive learning approach to standard learning with the teacher. The impact of the game on students’ motivation is not the same in the two countries because it was probably influenced by the attitude of the students before the courses.

The Final Frontier game was already developed and available in six different languages (English, Slovak, Czech, Spanish, Italian, and Romanian). Future research work will aim to investigate the benefits of the educational game in other European schools. In addition, the game is also suitable for learners with special needs (El Mawas, Bratu, et al., 2019). Our future objective is also to compare the learning impact of the game on students with special needs in different countries. Note that our game is also suitable for children in this critical situation and lockdown due to COVID-19. Children can play at home for entertainment and learn some concepts implemented in the game and based on the school curriculum.

## Author Statement

The authors declare that there is no conflict of interest.

## Acknowledgements

The research published in the paper was financially supported by the H2020 project NEWTON, Grant Agreement no. 688503. Special thanks go to all students and teachers who participated in these studies in Slovakia and Ireland.

## ORCID

Nour El Mawas https://orcid.org/0000-0002-0214-9840

Peter Trúchly https://orcid.org/0000-0002-4775-5158

Pavol Podhradský https://orcid.org/0000-0001-7389-6504

Martin Medvecký https://orcid.org/0000-0002-5243-6573

Cristina Hava Muntean https://orcid.org/0000-0001-5082-9253

## References

Bogusevschi, D., Bratu, M., Ghergulescu, I., Muntean, C. H., & Muntean, G.-M. (2018). Primary school STEM education: Using 3D computer-based virtual reality and experimental laboratory simulation in a physics case study. In *Proceedings of the Ireland International Conference on Education, Innovative Pedagogies for Effective Technology-Enhanced Learning (IPETeL) Workshop*. Ireland International Conference on Education, Innovative Pedagogies for Effective Technology-Enhanced Learning (IPETeL) Workshop.

Bogusevschi, D., Muntean, C. H., Gorji, N. E., & Muntean, G.-M. (2018). Earth course: A primary school large-scale pilot on STEM education. In *Proceedings of the International Conference on Education and New Learning Technologies,*
EDULEARN18. https://doi.org/10.21125/edulearn.2018.0958

Bruner, J. S. (2009). *The process of education* (Revised). Harvard University Press.

Brzozowy, M., Hołownicka, K., Bzdak, J., Tornese, P., Lupiañez-Villanueva, F., Vovk, N., Torre, J. J. S. de la, Perelló, J., Bonhoure, I., Panou, E., Bampasidis, G., Verdis, A., Papaspirou, P., Kasoutas, M., Vlachos, I., Kokkotas, S., & Moussas, X. (2017). Making STEM education attractive for young people by presenting key scientific challenges and their impact on our life and career perspectives. In *Proceedings of the International Technology, Education and Development Conference (INTED)*. https://doi.org/10.21125/inted.2017.2374

Çankaya, S., & Karamete, A. (2009). The effects of educational computer games on students attitudes towards mathematics course and educational computer games. *Procedia - Social and Behavioral Sciences, 1*(1), 145-149. https://doi.org/10.1016/j.sbspro.2009.01.027

Caprile, M., Palmén, R., Sanz, P., & Dente, G. (2015). *Encouraging STEM studies for the labour market* (p. 44). *Policy Commons*. Retrieved from https://policycommons.net/artifacts/1336503/encouraging-stem-studies-for-the-labour-market/1943728/

Clark, D. B., Tanner-Smith, E. E., & Killingsworth, S. S. (2016). Digital games, design, and learning: A systematic review and meta-analysis. *Review of Educational Research, 86*(1), 79-122. https://doi.org/10.3102/0034654315582065

Code.org. (2018). *Code.org-Learn Computer Science*. Retrieved from https://studio.code.org/courses

Csicsolová, I., Karolčík, Š., Takács, M., & Demko, M. (2018). Mobilná aplikácia MapPie a možnosti jej využitia vo výučbe geografie. *Geografia, 26*(1), 10-12.

El Mawas, N. (2014). Designing learning scenarios for serious games with ARGILE. *Knowledge Management & E-Learning, 6*(3), 227-249. https://doi.org/10.34105/j.kmel.2014.06.016

El Mawas, N., Bratu, M., Caraman, D., & Muntean, C. H. (2019). Investigating the learning impact of game-based learning when teaching science to children with special learning needs. In *Proceedings of the Society for Information Technology & Teacher Education International Conference*.

El Mawas, N., Tal, I., Moldovan, A.-N., Bogusevschi, D., Andrews, J., Muntean, G.-M., & Muntean, C. H. (2020). Investigating the impact of an adventure-based 3D solar system game on primary school learning process. *Knowledge Management & E-Learning, 12*(2), 165-190. https://doi.org/10.34105/j.kmel.2020.12.009

El Mawas, N., Truchly, P., Podhradský, P., & Muntean, C. H. (2019). The effect of educational game on children learning experience in a Slovakian school. In *Proceedings of the 11th International Conference on Computer Supported Education*. https://doi.org/10.5220/0007764104650472

Ezeoguine, E. P., & Augustine, S. E. (2021). Game-based learning influence on primary students' learning outcomes in basic science. *European Journal of Research and Reflection in Educational Sciences, 9*(5), 1-8.

Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014). Active learning increases student performance in science, engineering, and mathematics. *Proceedings of the National Academy of Sciences, 111*(23), 8410-8415. https://doi.org/10.1073/pnas.1319030111

GalaxyCodr. (2018). *Programovanie hravou formou*. Retrieved from https://galaxycodr.com/sk

Gibson, J. P. (2003). A noughts and crosses Java applet to teach programming to primary school children. In *Proceedings of the 2nd International Conference on Principles and Practice of Programming in Java*. https://doi.org/10.1145/957289.957315

Girvan, C., Wickham, C., & Tangney, B. (2016). Constructionism and microworlds as
part of a 21st century learning activity to impact student engagement and confidence.
*In Proceedings of the 4th International Constructionism Conference.*

Gura, M. (2012). Lego robotics: STEM sport of the mind. *Learning & Leading with Technology, 40*(1), 12–16.

Hazelkorn, E., & Moynihan, A. (2010). Ireland: The challenges of building research in a binary higher education culture. In S. Kyvik & B. Lepori (Eds.), *The Research Mission of Higher Education Institutions Outside the University Sector: Striving for Differentiation* (pp. 175–197). Springer Netherlands. https://doi.org/10.1007/978-1-4020-9244-2_10

Hejný, M. (2012). Exploring the cognitive dimension of teaching mathematics through scheme-oriented approach to education. *Orbis Scholae, 6*(2), 41–55.
https://doi.org/10.14712/23363177.2015.39

Herder, T., & Rau, M. (2022). Representational-competency supports in an educational video game for undergraduate astronomy. *Computers & Education, 190*: 104602.
https://doi.org/10.1016/j.compedu.2022.104602

Hoffman, B., & Nadelson, L. (2010). Motivational engagement and video gaming: A mixed methods study. *Educational Technology Research and Development, 58*(3), 245–270.
https://doi.org/10.1007/s11423-009-9134-9

Hussain, A., Lunn, J., Khalaf, M., Al-Jumeily, D., Pich, A., & McCarthy, S. (2016). The use of serious gaming for Open Learning environments. *Knowledge Management & E-Learning, 8*(1), 39–54. https://doi.org/10.34105/j.kmel.2016.08.004

Jākobsone, M. (2021). *Ireland—STEM education policy*. Digital skills and jobs platform. Retrieved from https://digital-skills-jobs.europa.eu/en/actions/national-initiatives/national-strategies/ireland-stem-education-policy

Kearney, C. (2016). *Efforts to increase students' interest in pursuing mathematics, science and technology studies and careers*. National measures taken by 30 countries – 2015 (p. 96). European Schoolnet.

Kiili, K., Ketamo, H., Koivisto, A., & Finn, E. (2014). Studying the user experience of a tablet based math game. *International Journal of Game-Based Learning, 4*(1), 60–77.
https://doi.org/10.4018/IJGBL.2014010104

Kirginas, S., & Gouscos, D. (2017). Exploring the impact of freeform gameplay on players’ experience: An experiment with maze games at varying levels of freedom of movement. *International Journal of Serious Games, 4*(4), 53–69.
https://doi.org/10.17083/ijsg.v4i4.175

Lim, C. P. (2008). Global citizenship education, school curriculum and games: Learning Mathematics, English and Science as a global citizen. *Computers & Education, 51*(3), 1073–1093.
https://doi.org/10.1016/j.compedu.2007.10.005

Lin, C.-H., & Liu, E. Z.-F. (2009). A Comparison between drill-based and game-based typing software. In Z. Pan, A. D. Cheok, W. Müller, & M. Chang (Eds.), *Transactions on Edutainment III* (pp. 48–58). Springer. https://doi.org/10.1007/978-3-642-11245-4_5

Liu, E. Z. F., & Chen, P.-K. (2013). The effect of game-based learning on students’ learning performance in science learning – A case of “Conveyance Go.” *Procedia - Social and Behavioral Sciences, 103*, 1044–1051.
https://doi.org/10.1016/j.sbspro.2013.10.430

Lovelace, M., & Brickman, P. (2013). Best practices for measuring students' attitudes toward learning science. *CBE Life Sciences Education, 12*(4), 606–617.
https://doi.org/10.1187/cbe.12-11-0197

Martin, F. G., Butler, D., & Gleason, W. M. (2000). Design, story-telling, and robots in Irish primary education. In *Proceedings of the IEEE International Conference on Systems, Man and Cybernetics*. https://doi.org/10.1109/ICSMC.2000.885082
Montandon, L., Playfoot, J., Ghergulescu, I., Bratu, M., Bogusevschi, D., Rybarova, R., & El Mawas, N. (2018). Multi-dimensional approach for the pedagogical assessment in STEM technology enhanced learning. In *Proceedings of the EdMedia + Innovate Learning* (pp. 378–383). Retrieved from https://www.adaptemy.com/multi-dimensional-approach-for-the-pedagogical-assessment-in-stem-technology-enhanced-learning/

Mooney, A., Duffin, J., Naughton, T. J., Monahan, R., Power, J. F., & Maguire, P. (2014). PACT: An initiative to introduce computational thinking to second-level education in Ireland. In *Proceedings of the International Conference on Engaging Pedagogy*.

Morris, B. J., Croker, S., Zimmerman, C., Gill, D., & Romig, C. (2013). Gaming science: The «Gamification» of scientific thinking. *Frontiers in Psychology, 4*: 607. https://doi.org/10.3389/fpsyg.2013.00607

MŠVVaŠ. (2018). *Annual report about tertiary education status for year 2017* (p. 46). The Ministry of Education, Science, Research and Sport of the Slovak Republic.

Muntean, C. H., Bogusevschi, D., & Muntean, G.-M. (2019). *Innovative technology-based solutions for primary, secondary and tertiary STEM education*. Retrieved from https://norma.ncirl.ie/4100/

Muntean, C. H., El Mawas, N., Bradford, M., & Pathak, P. (2018). Investigating the impact of an immersive computer-based Math game on the learning process of undergraduate students. In *Proceedings of the IEEE Frontiers in Education Conference (FIE)*. https://doi.org/10.1109/FIE.2018.8659005

NEWTON. (2016). *H2020: Networked labs for training in sciences and technologies for in-formation and communication*. Retrieved from http://www.newtonproject.eu/

OConnell, C. (2013). *Major review of STEM education in Ireland to take place*. Retrieved from https://www.siliconrepublic.com/discovery/major-review-of-stem-education-in-ireland-to-take-place

Osborne, J., & Dillon, J. (2008). *Science education in Europe: Critical reflections* (p. 32). Nuffield Foundation. Retrieved from https://www.nuffieldfoundation.org/about/publications/science-education-in-europe-critical-reflections/

Osborne, J., Simon, S., & Collins, S. (2003). Attitudes towards science: A review of the literature and its implications. *International Journal of Science Education, 25*(9), 1049–1079. https://doi.org/10.1080/0950069032000032199

Palmer, D. (2007). What is the best way to motivate students in science? *The Journal of the Australian Science Teachers Association*, 53(1), 38–42. https://doi.org/10.3316/aeipt.161785

Papadakis, S. (2018). The use of computer games in classroom environment. *International Journal of Teaching and Case Studies, 9*(1), 1–25. https://doi.org/10.1504/IJTCS.2018.090191

Papadakis, S., & Kalogiannakis, M. (2018). Using gamification for supporting an introductory programming course. The case of ClassCraft in a secondary education classroom. In A. L. Brooks, E. Brooks, & N. Vidakis (Eds.), *Interactivity, Game Creation, Design, Learning, and Innovation* (pp. 366–375). Springer International Publishing. https://doi.org/10.1007/978-3-319-76908-0_35

Papanastasiou, C. (2000). Effects of attitudes and beliefs on mathematics achievement. *Studies in Educational Evaluation, 26*(1), 27–42. https://doi.org/10.1016/S0191-491X(00)00004-3

Piątkowski, M. J. (2020). Expectations and challenges in the labour market in the context of Industrial Revolution 4.0. The agglomeration method-based analysis for Poland and other EU member states. *Sustainability, 12*(13): 5437. https://doi.org/10.3390/su12135437

Prendergast, M., & O'Meara, N. (2017). A profile of mathematics instruction time in
Irish second level schools. *Irish Educational Studies*, 36(2), 133–150.
https://doi.org/10.1080/03323315.2016.1229209

Prendergast, M., & Roche, J. (2018). Trinity Walton Club: What is its potential for promoting interest in STEM. *European Journal of STEM Education*, 3(1): 01. https://doi.org/10.20897/ejsteme/83659

Probst, A. (2018). *Matika.in. Have fun with mathematics—Training and practicing math exercises for children in primary schools*. Retrieved from https://www.matika.in/en/

Quinn, R. (2012). The future development of education in Ireland. *Studies: An Irish Quarterly Review*, 101(402), 123–138.

Ríordáin, M. N., & Hannigan, A. (2011). Who teaches mathematics at second level in Ireland? *Irish Educational Studies*, 30(3), 289–304.
https://doi.org/10.1080/03323315.2011.569117

Roche, J., O'Neill, A., & Prendergast, M. (2016). An inquiry-based learning intervention to support post-primary engagement with science, technology, engineering and mathematics. *European Journal of Science and Mathematics Education, 4*(4), 431–439.
https://doi.org/10.30935/scimath/9482

Rosas, R., Nussbaum, M., Cumsille, P., Marianov, V., Correa, M., Flores, P., Grau, V., Lagos, F., López, X., López, V., Rodriguez, P., & Salinas, M. (2003). Beyond Nintendo: Design and assessment of educational video games for first and second grade students. *Computers & Education, 40*(1), 71–94. https://doi.org/10.1016/S0360-1315(02)00099-4

Sedighian, K., & Sedighian, A. S. (1996). Can educational computer games help educators learn about the psychology of learning mathematics in children? *BibSonomy*. Retrieved from
https://www.bibsonomy.org/bibtex/f8a28726788e9ba402f3b243e7beced9

Smith, D. R., & Munro, E. (2009). Educational card games. *Physics Education, 44*(5), 479–483. https://doi.org/10.1088/0031-9120/44/5/004

Smyth, E., & Hannan, C. (2006). School effects and subject choice: The uptake of scientific subjects in Ireland. *School Effectiveness and School Improvement, 17*(3), 303–327. https://doi.org/10.1080/09243450600616168

Šveda, D., Repovský, M., & Ftáčnik, M. (2018). *Analýza požiadaviek vysokých škôl a trhu práce na absolventov stredných škôl z pohľadu matematickej gramotnosti a návrh opatrení na skvalitnenie matematického vzdelávania* (p. 85). RÚZ.

Tick, A. (2007). Application of problem-based learning in classroom activities and multimedia. In *Proceedings of the 5th Slovakian Hungarian Joint Symposium on Applied Machine Intelligence and Informatics*. Retrieved from
https://www.semanticscholar.org/paper/Application-of-Problem-Based-Learning-in-Classroom-Tick/fe8ba7e73bd1461f2783f19b39eeeddd7b634eb9

Trexima. (2017). *Forecasts of labor market development in SR*. Trexima.

Trotman, A. (2017). *Why don't European girls like science or technology? Microsoft News Centre Europe*. Retrieved from
https://news.microsoft.com/europe/features/dont-european-girls-like-science-technology/

Trúchly, P., Medvecký, M., Podhradský, P., & El Mawas, N. (2019). STEM education supported by virtual laboratory incorporated in self-directed learning process. *Journal of Electrical Engineering, 70*(4), 332–344. https://doi.org/10.2478/jee-2019-0065

Tüzün, H., Yılmaz-Soylu, M., Karakuş, T., İnal, Y., & Kızılkaya, G. (2009). The effects of computer games on primary school students' achievement and motivation in geography learning. *Computers & Education, 52*(1), 68–77. https://doi.org/10.1016/j.compedu.2008.06.008

Vieme to. (2018). Vieme to – Online precvičovanie školského učiva. Retrieved from
https://www.viemeto.org/

Vilorio, D. (2014). STEM 101: Intro to tomorrow's jobs. *Occupational Outlook Quarterly*, 58(1), 2-12.

Xie, J., Wang, M., & Hooshyar, D. (2021). Student, parent, and teacher perceptions towards digital educational games: How they differ and influence each other. *Knowledge Management & E-Learning*, 13(2), 142-160.
https://doi.org/10.34105/j.kmel.2021.13.008

Yien, J.-M., Hung, C.-M., Hwang, G.-J., & Lin, Y.-C. (2011). A game-based learning approach to improving students’ learning achievements in a Nutrition course. *Turkish Online Journal of Educational Technology - TOJET*, 10(2), 1-10.