# Children Emotions Detection and Learning using a Fairy Tale Serious Game

Chadi Fouad Riman, Eric Monacelli

► To cite this version:

Chadi Fouad Riman, Eric Monacelli. Children Emotions Detection and Learning using a Fairy Tale Serious Game. American Journal of Engineering and Applied Sciences, 2022, 15 (1), pp.126-131. 10.3844/ajeassp.2022.126.131. hal-04772382

HAL Id: hal-04772382

https://hal.science/hal-04772382v1

Submitted on 22 May 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Original Research Paper

# Children's Emotions Detection and Learning using a Fairy Tale Serious Game

¹Chadi Fouad Riman and ²Eric Monacelli

¹Department of Computer Engineering, American University of the Middle East, Egaila, Kuwait

²Department of Robotics, University of Paris-Saclay, France

## Article history

Received: 27-01-2022

Revised: 14-03-2022

Accepted: 20-03-2022

## Corresponding Author:

Chadi Fouad Riman
Department of Computer
Engineering, American
University of the Middle East,
Egaila, Kuwait
Email: chadi.riman@aum.edu.kw

**Abstract:** Children learn emotional ways of expressing their feelings in several ways. They learn at home as well as in their interactions with their friends at school. The problem is that sometimes they have a difficulty in learning emotions. This study aims to help children learn about emotions and help other children learn about emotions as well. The scope is generalized to all children but will be initially applied to a specific age. The used approach is by using "Serious Games". A Serious Game was written using C# language, to be used by two groups of children: "Teachers" group and "students' group. This serious game describes a fairy tale with pictures, text, facts, and questions. It is customizable for any fairy tale that you may want. The initial work includes the "The Ugly Duckling" tale. In the future, the written game will be tested and the results will be analyzed.

**Keywords:** Serious Game, Children Education, Emotions Learning.

## Introduction

A serious game or applied game is a game designed for a primary purpose other than pure entertainment. The "serious" adjective is generally prepended to refer to video games used by industries like defense, education, scientific exploration, health care, emergency management, city planning, engineering, and politics.

Feelings and Emotions are tricky things for young children and toddlers. They are overwhelming and hard to understand. A child can be helped to learn about feelings and emotions by playing fun and engaging children's games. Also, a great way to help kids learn about feelings is to discuss how characters in books or TV shows may feel. Pause to ask, "How do you think he feels right now?" Then, discuss the various feelings the character may be experiencing and the reasons why.

Centers for childhood mental health try to help build a strong mental health foundation for children and families. Such a center (CECMCH, 2022) adapted Creating Teaching Tools to provide teachers with practical strategies that are known to be successful in helping young children with problem behavior.

In this study, a serious game that will help teachers and therapists build children's emotional skills is introduced.

The rest of this study is organized as follows. In the

next section, a brief survey of the relevant literature review is shown. The following section presents the proposed serious game method. It also contains the technical details related to our game platform. Then another section describes the results and discussion that will be done in future work. Finally, the paper is concluded in the last section with possible future work.

## Literature Review

Many serious games were developed to help human beings in different areas, such as rehabilitation, and cognitive skills, among others. A serious game was developed by Muñoz et al. (2022) to help people living with dementia promote their physical activities. Another work (Yang et al., 2021) was done to help maintain cognitive functions for elderly people using video games. A work by Keith et al. (2021) used video games to build teamwork. Team work's productivity increased after using these games. The multiple works done by (Dufort et al., 2015; Dufort, 2016) implemented a cultural pervasive game to teach historical cultures of a society. Another author (Lavigne, 2016) showed the educational relevance of serious games, after testing 30 of them. It was stated that the educational benefits come at a big-time cost which schools are not willing to waste. The paper (Djaouti, 2014) used two metrics to assess
two serious games. The two metrics are Google Analytics and Playtomic. The work done by Marfisi-Schottman (2013) suggested an easy platform to help teachers create their educational serious games without having to ask programmers to do it. These free educational games' automatic creators will accelerate the use of serious games for education. The work done by (De Gloria et al., 2014) introduced the pedagogical theories and models relevant to Serious Games and their implications, with a schema for proper integration of games in education. The work also analyzed a set of well-established Serious Games and formats. The work done by (Darwesh, 2016) suggested a model of a serious game which included principles of an educational model, then exposed how a game can be used in education.

## Proposed Serious Game Methods

The first version of this Serious Game is completed. It is hundred percent configurable with the first tale “The Ugly Duckling”.

### Concept

We aim to teach emotions to children and help them teach each other. To do so, the development of a serious game that involves storytelling based on a fairy tale is done. The concept is similar to a role-playing game. The first child will move through images one by one and read the text attached to each image. Based on the story and individual images, he will react and choose related facts and questions to each image. Then another child takes his turn. The other child checks the chosen facts and questions by the first child while watching each image and he is requested to answer the questions. A therapist will be available to register the children's interactions with the tale. Figure 1 shows the fairy tale serious game main screen.

### Technical Information

The files include the main game and a data folder. The data folder has the images folder and the files: Parameter, Facts, Questions, Text and SAVED. A detailed explanation of each part is as follows:

1. Images folder: All JPG types images numbered image1 until the end (image 20 in our example) as in next Fig. 2

2. Parameter: General configuration file that includes several images and delay between each image. For example

- Images = 20 (total images in the story are 20)

- Delay = 500 (this is the delay between two consecutive images measured in milliseconds)

Figure 3 below shows the sample “Param file”:

3. Facts: List of facts, one fact per line. These will be chosen by the user according to the current image

4. Questions: List of questions, one question per line. These will also be chosen by the user according to the current image

5. Text: List of texts accompanying the images. One text per line related to one image in order

6. SAVED: If this file exists, it includes the results saved in the previous session and will be saved at the end of the current session. The results are open at the beginning of a session and saved at the end of that session. Figure 4 below shows a list of the default files available in the system with the “Images” subfolder

The system is fully configurable. You can put a story with any set of images and the related image text. Then you can add the facts and questions for the user to choose from. The general parameter file "Param" is used to specify the number of images and the delay time between them when it is run as a slide show. The trial is saved in the “SAVED” file.

### How does it Work?

In our serious game, each picture includes a large list of questions and facts shown with checkboxes. The test subject chooses which ones suit him in the displayed image. If he is satisfied, he clicks the left mouse button and moves to the next picture. If he wants to cancel his changes, he makes a right mouse click that takes him to the previous picture. When he finishes all the pictures, he can test his work automatically with the START button. This will make a slide show of the tail with the text. Each picture will show only the chosen questions and facts. These chosen values will be communicated to a therapist for analysis and conclusion about the child's learning progress. Figure 5 below shows the selection phase for questions and facts for one image. After the start button is pressed, the images will flow automatically and only the chosen questions and facts for each image will be shown, as in Fig. 6.

<figure><img src="image_2.png" /><figcaption>Fig. 1: Serious game: Fairy tale</figcaption></figure>
<figure><img src="image_3.png" /><figcaption>Fig. 2: List of Image Files</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>Fig. 3: Game parameters</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>Fig. 4: List of game files</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>Fig. 5: Game page during the selection phase</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Fig. 6: Game page in action during a running phase. Only previously selected questions/facts are shown</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>Fig. 7: Partial list of game figures</figcaption></figure>
<table><caption>**Table 1: Tale image descriptions**</caption><thead><tr><th>The original Text (French)</th><th>The translated text</th></tr></thead><tbody><tr><td>1LE VILAIN PETIT CANARD</td><td>The ugly duckling</td></tr><tr><td>2.</td><td></td></tr><tr><td>3. Il était une fois...</td><td>Once upon a time</td></tr><tr><td>4. dans une belle et généreuse nature, dans sa portée, une canne à un œuf différent...</td><td>In a beautiful nature, duck had a different egg</td></tr><tr><td>5. qui s'avère être un petit trop différent dès le départ des autres.</td><td>Which gave a duckling different from all the others</td></tr><tr><td>6. La Cane fait des efforts pour l'aimer quand même</td><td>The moth duck tried to love that duckling</td></tr><tr><td>7. Rejeté de tous, et même attaqué,</td><td>The duckling was rejected from all and even attacked</td></tr><tr><td>8.</td><td></td></tr><tr><td>9. le vilain petit Canard échappe au pire en allant seul et loin</td><td>The ugly duckling escapes and goes alone for a long distance</td></tr><tr><td>10. Il se trouve confronté à des péripéties sans protection</td><td>Facing dangers without protection</td></tr><tr><td>11. La mort frappe Mais il en ressort indemne et surpris,</td><td>Almost dead, but he survives</td></tr><tr><td>12.</td><td></td></tr><tr><td>13. tout n'est pas contre lui</td><td>It seems not everything was against him</td></tr><tr><td>14. Il rencontre des Oies un peu semblables, non hostiles</td><td>He meets friendly geese</td></tr><tr><td>15. Continuant sa route, il rencontre enfin des Cygnes, qui le reconnaissent et qui l'accueillent comme un des leurs</td><td>Going on his way, he finally meets swans who recognize him as one of theirs</td></tr><tr><td>16. Tellement beaux, il est prêt à mourir pour rejoindre cette beauté</td><td>They were so beautiful that he wanted to be like them</td></tr><tr><td>17.</td><td></td></tr><tr><td>18. Il prend conscience enfin de sa vraie valeur positive.</td><td>Finally, he discovers what he really was</td></tr><tr><td>19.</td><td></td></tr><tr><td>20. La vie valait donc bien la peine d'être vécu face à la joie permanente de la découverte de sa véritable nature fondamentale dans sa différence</td><td>Life was worth living after discovering his true Fundamental nature in its difference</td></tr></tbody></table>

<table><caption>**Table 2: Tale facts**</caption><thead><tr><th>Fact (French)</th><th>The translated Fact</th></tr></thead><tbody><tr><td>Vilain</td><td>Ugly</td></tr><tr><td>Petit</td><td>Small</td></tr><tr><td>Canard</td><td>Duck</td></tr><tr><td>Trop différent</td><td>Very Different</td></tr><tr><td>Rejeté de tous</td><td>Rejected from all</td></tr><tr><td>Ne t'approche pas de nous</td><td>Don't come near</td></tr><tr><td>Attaqué</td><td>Attacked</td></tr><tr><td>Sa beauté</td><td>His beauty</td></tr><tr><td>Magnifique</td><td>Magnificent</td></tr><tr><td>Elégant</td><td>Elegant</td></tr></tbody></table>

<table><caption>**Table 3: Tale questions**</caption><thead><tr><th>Question (French)</th><th>The translated Question</th></tr></thead><tbody><tr><td>Comment est qualifié le personnage?</td><td>How is the character qualified?</td></tr><tr><td>Quelle est l'émotion que ressent le personnage?</td><td>What emotion does the character feel?</td></tr><tr><td>Que ressentent les autres personnages?</td><td>How do the other characters feel?</td></tr><tr><td>Quelle est l'émotion ressentie?</td><td>What is the felt emotion?</td></tr><tr><td>Ca te fait quoi de lire cette histoire ?</td><td>How do you feel reading this story?</td></tr><tr><td>Qu'est ce qui t'intéresse toi-même dans cette histoire?</td><td>What does interest you in this story?</td></tr><tr><td>De quoi as-tu envie de parler de cette histoire?</td><td>What do you want to talk about this story?</td></tr><tr><td>Et plus particulièrement qu'est ce qui t'intéresse personnellement dans cette histoire?</td><td>And more specifically what interests you personally about this story?</td></tr><tr><td>Ça te fait penser à quoi?</td><td>What does that make you think of?</td></tr><tr><td>Est-ce que ça te plait ou pas?</td><td>Do you like it or not?</td></tr><tr><td>Est-ce que ça te rappelle quelque chose dans le passé ?</td><td>Does this remind you of anything from the past?</td></tr><tr><td>Est-ce que tu y vois des similitudes dans le présent?</td><td>Do you see any similarities in the present?</td></tr></tbody></table>

## Default Story

The initial story is "Ugly Duckling". This story includes twenty images with different text, facts, and questions. It is a classic nineteenth-century fairy tale that tells the

story of a duckling who, when hatched along with his brothers and sisters, is bullied because they perceive him as ugly. He wanders alone through the fall and winter and suffers from fear, loneliness, and sadness. In the spring he meets up with a group of swans and
realizes that he too has become a beautiful swan. Following in Fig. 7 is a partial list of images used for this tale. There are twenty images in total.

The descriptive text for each image is listed next both in French and English. The original game was created in the French language because it will be first applied in France. Table 1 shows the tale image descriptions.

Given the images in series with their descriptions, the child has to choose facts and questions related to each image. The facts related to the ugly duckling story are as follows in Table 2.

The questions related to the Ugly Duckling story are as follows in Table 3.

## Results and Discussion

The system is not tested yet. It is planned to be tested by a child psychologist in a hospital or therapy center for children. The results will be analyzed by the therapist and checked for consistency. As a first trial, there will be two groups of children working on the system. The older group is the ones specifying the questions and facts. The younger group are the ones running the game and reacting to each picture with the choice of meaningful questions and facts that relate to what they feel.

This game will be tested in a coming work soon. The work should be tested on a large number of subjects, otherwise, it will not give adequate results. It is hoped that this study will overcome the deficiencies of previous work, especially concerning children's emotional education.

## Conclusion

In this study, a serious game based on fairy tale storytelling is proposed. This game is used to assess and teach emotions to children. The platform was built with the initial "The ugly duckling" story. This study will be tested in future work. The testing will check if the work is valid and helps children learn emotions.

## Acknowledgment

The authors would like to thank the American University of the Middle East for providing its computers at the service for the software's development.

## Author's Contributions

Both authors contributed to the writing of the manuscript. The first author gave more attention to the software's technical description.

## Ethics

This article is original. The author declares that there are no ethical issues that may arise after the publication of this manuscript.

## References

CECMCH. (2022). CECMCH, Georgetown university center for child and human development, center for early childhood mental health consultation", https://www.ecmhc.org/about.html.

Darwesh, A. (2016). Concepts of Serious Game in Education. International Journal of Engineering and Computer Science, 4(12), 15229-15232. doi.org/10.18535/Ijecs/v4i12.25

De Gloria, A., Bellotti, F., & Berta, R. (2014). Serious Games for education and training. International Journal of Serious Games, 1(1). doi.org/10.17083/ijsg. v1i1.11

Djaouti, D. (2014). Utilisation de plateformes génériques de mesure analytique pour l'évaluation de Serious Games: Une expérimentation. Sciences et Technologies de l'Information et de la Communication pour l'Éducation et la Formation, 21(1), 483-517. https://www.persee.fr/doc/stice_1764-7223_2014_num_21_1_1109

Dufort, D. (2016). Outils sémantiques d'aide à la conception de jeux pervasifs pour la médiation culturelle (Doctoral dissertation, Université de Franche-Comté). https://tel.archives-ouvertes.fr/tel-01538605/

Dufort, D., Tajariol, F., & Roxin, I. (2015). Jeux pervasifs culturels: conception d'un outil descriptif et taxonomique. Questions de communication, (2), 19-41. https://www.cairn.info/revue-questions-de-communication-2015-2-page-19.htm

Marfisi-Schottman, I., Labat, J-M., Carron, T. (2013). Approche basée sur la méthode pédagogique des cas pour créer des Learning Games pertinents dans de nombreux domaines d'enseignement. Environnements Informatiques pour l'Apprentissage Humain, EIAH'2013, Toulouse, France. pp.67-78. hal-00845537.

Keith, M. J., Dean, D. L., Gaskin, J., & Anderson, G. (2021). Team Building Through Team Video Games: Randomized Controlled Trial. JMIR serious games, 9(4), e28896. doi.org/10.2196/28896

Lavigne, M. (2016). Les faiblesses ludiques et pédagogiques des serious games. https://halshs.archives-ouvertes.fr/halshs-02078300
Muñoz, J., Mehrabi, S., Li, Y., Basharat, A., Middleton, L. E., Cao, S., & Boger, J. (2022). Immersive virtual reality exergames for persons living with dementia: User-centered design study as a multistakeholder team during the COVID-19 Pandemic. JMIR Serious Games, 10(1), e29987. doi.org/10.2196/29987. PMID: 35044320

Yang, C., Han, X., Jin, M., Xu, J., Wang, Y., Zhang, Y., & Piao, C. (2021). The Effect of Video Game-Based Interventions on Performance and Cognitive Function in Older Adults: Bayesian Network Meta-analysis. JMIR Serious Games, 9(4), e27058. doi.org/10.2196/27058