# Assessing Social Engagement in a Digital Role-Playing Game

Guillaume Bonvin, Eric Sanchez, Pierre-Antoine Champin, Rémi Casado, Nathalie Guin, Marie Lefevre

► To cite this version:

Guillaume Bonvin, Eric Sanchez, Pierre-Antoine Champin, Rémi Casado, Nathalie Guin, et al. "Assessing Social Engagement in a Digital Role-Playing Game. Open Conference on Computers in Education (OCCE), Jun 2018, Linz, Austria. pp.161-166, 10.1007/978-3-030-23513-0_16." [inriaHAL](https://inria.hal.science/hal-02370923). [hal-02370923](https://hal.inria.fr/hal-02370923).

HAL Id: hal-02370923

https://inria.hal.science/hal-02370923

Submitted on 19 Nov 2019

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Assessing Social Engagement in a Digital Role-Playing Game
## Changes over Time and Gender Differences

Guillaume Bonvin, Eric Sanchez

University of Fribourg, CERF, Fribourg, Switzerland, {guillaume.bonvin, eric.sanchez}@unifr.ch

Pierre-Antoine Champin, Rémi Casado, Nathalie Guin, Marie Lefevre

University of Lyon, CNRS, Lyon, France, {pierre-antoine.champin, remi.casado, nathalie.guin, marie.lefevre}@univ-lyon1.fr

**Abstract.** Classcraft is a digital role-playing game dedicated to classroom management. Teachers can create teams and assign an avatar to students, as well as points and ‘powers’ as rewards for desired behaviour. We conducted a study in 4 classrooms from Switzerland. The study aimed at assessing to what extent the game fosters the social component of students’ engagement. The detection of socially-engaged behaviours is based on the monitoring of players’ behaviours. We collected and analysed players’ digital traces with kTBS4LA, a platform dedicated to playing analytics. The data collected shows that social engagement varies across time or gender. This variation seems to be linked to specific features of the game and also depends on how the game is played.

**Keywords.** Playing analytics, gamification, classroom management, social engagement, Classcraft

# 1 Introduction

## 1.1 Managing the classroom with a role-playing game

Classcraft is a digital role-playing game dedicated to classroom management [1]. The objective of Classcraft is to transform the classroom into a role-playing game for the duration of the school year. Teachers can create teams and assign avatars to students. They also assign points and ‘powers’ as rewards depending on students’ behaviour, defined by specific game rules. In order to acquire powers, a player must demonstrate behaviour expected by the school, such as doing homework or being on time for class. The students play as warriors, mages or healers and, depending on their avatar, they can acquire and use specific powers that have impact on real life. For example, a student being late to class may be saved from punishment if one of his teammates uses the power called “Protect_1’. The teacher is the game-master. He deducts points or rewards depending if students’ behaviours fit (or not) the classroom rules. Collaborative behaviours are expected from students, and players own individual or collaborative powers.
## 1.2 Classcraft as ludicisation of classroom management

Classcraft aims to convert an ordinary class into a playful situation. Each player, depending on his avatar, plays a specific role and owns specific powers. Players are rewarded or punished by the game master. Each week, a random event provokes positive or negative consequences, such as acquiring or losing powers. This use of game features, in order to convert non-game context into a game, is called gamification [2]. However, Classcraft is not limited to the use of game features in a mechanical way. Classcraft is based on a metaphor that changes the meaning of the action performed by the students. The actions themselves are not altered. The students attend class on time, do their homework or collaborate for learning activities. However, the meanings and the motives of these actions are changed when Classcraft is played. The classroom becomes a battle, where players have to overcome difficulties and to develop specific skills. Sanchez et al. called ludicisation [1] the conversion of a non-game context into a playful situation aiming at fostering gameful experiences and students' engagement.

# 2 Objectives of the study: assessing students' social engagement

There is an ambiguity about the concept of players’ engagement. Different definitions have been provided [3] and, due to the ambiguity of the related concepts and their context-dependent definitions, assessing players’ engagement might be difficult. Our approach is based on Self-Determination Theory (SDT) [4], which states that motivation results from innate psychological needs. Based on SDT, Bouvier et al. [3] consider that players’ engagement encompasses four components: the environmental component in relation to autonomy needs; the self-component that relates to autonomy needs; the action component linked with competence and with autonomy needs. The social component is in relation to relatedness.

By playing Classcraft, students are expected to help other students and to collaborate with teammates. Thus, Classcraft aims to foster players’ social engagement. As a result, the objective of our study aims at assessing the social component of the players’ engagement. For this study, we examine the evolution of the student’s social engagement during a school year. We also examine gender differences.

# 3 Methodology of the study

## 3.1 Monitoring the player: playing analytics

In order to assess a player's social engagement, we developed a specific methodology based on playing analytics [5]. We detect engaged-behaviours by monitoring players. Players' digital traces are players' actions performed during a digitally-mediated activity. Obsel (observed elements) are automatically collected. Obsels [6] are elementary players' actions (like buying or using powers). Each obsel is characterised by a type of event, a timestamp (beginning and end of the event) and information that is useful to derive meaning (attributes and relations with other obsels). Some obsels, like using
collaborative versus individual powers, are inherent to players' social engagement. Indeed, we consider that the use of collaborative powers (powers that have a positive impact on a teammate) brings information on the participation of the player to a collaborative play, where the outcomes of the game depend on the capacity of players to take their teammates into consideration. Thus, we consider that the use of collaborative powers demonstrates that a player is socially engaged with his/her teammates.

## 3.2 Data collection and data processing

The data collected come from 11 classes in Switzerland. For each class, the data consist of 8 JavaScript Object Notation (JSON) files. The main file (logbook.JSON) is a list of *obsels* collected during a session dedicated to play. The logbook.JSON file is completed with information coming from the other files: gender of the players; different teams involved; and game-levels reached by a given player. These operations are carried out with the data collected from the different classes. However, four sets of data have been selected for this exploratory study. The data analysis is performed with a specific digital platform called kernel Trace Based System for Learning Analytics (kTBS4LA) [7]. The data collected are uploaded, and different functionalities dedicated to data processing are available. Some of them were developed for the needs of this study. First, a kTBS4LA export model is created. This model gives a precise description of the digital traces uploaded on the platform. The platform allows for selecting specific *obsels* from the data collected. The *obsels* that are considered to be relevant for the study are extracted from the whole dataset. kTBS4LA enables the visualisation of these *obsels* along a timeline and different colours and shapes can be used for making apparent specific features. In order to obtain consistent results, the same protocol is applied to each dataset. We designed and recorded several scenarios for the visualisation of *obsels* such as the use of collaborative powers or individual powers according to gender and teams.

## 4 Results and discussion

The 4 classes selected for this preliminary study have an average of 16 students, 7 girls and 9 boys, and 3 teams per class. We collected data for 4 to 7 months, depending on the different classes. Table 1 shows that 'the activity' of the different classes (measured by the number of *obsels* collected and powers used by students) varies a lot. Class 4 is a specific case. The data collected show that the teacher gave all the powers to everyone, right from the beginning of the game.

<table><caption>**Table 1.** Information about the different classes of the study</caption><thead><tr><th>Traces</th><th>Class 1</th><th>Class 2</th><th>Class 3</th><th>Class 4</th></tr></thead><tbody><tr><th scope="row">Obsels</th><td>640 obsels</td><td>568 obsels</td><td>2352 obsels</td><td>7923 obsels</td></tr><tr><th scope="row">Use of powers</th><td>44 powers</td><td>52 powers</td><td>184 powers</td><td>467 powers</td></tr></tbody></table>
## 4.1 Variation of players' social engagement during the school year

Figure 1 represents collaborative (green) versus individual (red) powers used by the students during the school year (time line). For classes 1, 2 and 4, collaborative powers are increasingly used during the school year. However, it is not the case for class 3. For this class, there is a balance between individual and collaborative powers used by the students. As a result, for 3 classes, students' social engagement increases during the school year. The students are increasingly involved in collaborative play. The game seems to have the expected influence on students: fostering collaborative behaviours. However, the differences observed with class 3 show that the influence of the game is complex, and might depend on different factors. Students' social engagement also probably depends on how the game is played by the students and orchestrated by the teacher.

<figure><img src="image_2.png" /><figcaption>**Figure 1.** Powers used by the students from the different classes during the school year</figcaption></figure>

Playing analytics do not provide information regarding this issue, so further classroom observations or/and teachers' and students' interviews are needed.

## 4.2 Variation of social engagement depending on students' gender

The data collected from the 4 classes selected for the preliminary study also show differences between how girls and boys play. While boys use more collaborative than individual powers, they still use a lot of individual powers in comparison to girls. Girls are less active but more efficient, and use few individual powers (see Table 2). Social engagement varies according to the students' gender. Girls are more involved in collaboration than boys.
<table><caption>**Table 2.** Uses by gender: collaborative powers versus individual powers</caption><thead><tr><th></th><th>Male</th><th>Female</th></tr></thead><tbody><tr><td>Collaborative</td><td>(304/493) - 61.7%</td><td>(228/254) - 89.8%</td></tr><tr><td>Individual</td><td>(189/493) - 38.3%</td><td>(26/254) - 10.2%</td></tr></tbody></table>

## 5 Conclusion

Based on the data collected from this preliminary study, there are arguments to state that the ludicisation of classroom management with Classcraft helps students to develop positive classroom behaviour. The game influences how students collaborate with their teammates and, therefore, fosters students' social engagement. Changing the meaning of the situation experienced by the students helps them to take into account their teammates. Students' social engagement varies along the school year and according to gender. However, the differences observed for different classes also show that the way the game is orchestrated by the teacher influences how the game is played by the students. Playing analytics do not offer any information regarding this issue and further investigations are needed. Thus, we plan to continue to address this issue with a mixed methodology. The project will consist of collaborative work with voluntary teachers for the implementation of the game in different classes. The combination of learning analytics and classroom observations should enable the characterisation of students' social engagement and the identification of the game elements that foster students' collaboration. Therefore, we expect to gain a better understanding of how ludicisation might be applied for classroom management.

## References

1. Sanchez, E., Young, S., JounEAu-Sion, C.: Classcraft: from gamification to ludicization of classroom management. Education and Information Technologies, 20(5), 497-513 (2016)

2. Kapp, K.: The gamification of learning and instruction. San Francisco, CA: Pfeiffer (2012)

3. Bouvier, P., Lavoué, E., Sehaba, K., George, S.: Identifying Learner's Engagement in Learning Games - A Qualitative Approach based on Learner's Traces of Interaction. In O. Foley, M.T. Restivo, J.O. Uhomoibhi, M. Helfert (Eds.), Proceedings of the 5th International Conference on Computer Supported Education, Aachen, Germany, 339-350 (2013)

4. Ryan, R.M., Deci, E.L.: Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55, 68-78 (2000)

5. Sanchez, E., Mandran, N.: Exploring Competition and Collaboration Behaviors in Game-Based Learning with Playing Analytics. In É. Lavoué, H. Drachsler, K. Verbert, J. Broisin, M. Pérez-Sanagustín (Eds.), Data Driven Approaches in Digital Education, 12th European Conference on Technology Enhanced Learning, Tallinn, Estonia, 467-472 (2017)

6. Cordier, A., Lefevre, M., Champin, P., Mille, A.: Connaissances et raisonnement sur les traces d'interaction. Revue d'Intelligence Artificielle, 28(2-3), 375-396 (2014)

7. Casado, R., Guin, N., Champin, P., Lefevre, M.: kTBS4LA: une plateforme d'analyse de traces fondée sur une modélisation sémantique des traces. Papier présenté lors de l'Atelier ORPHEE Rendez-Vous, Font-Romeu, France (2017)