# SIMVA: Serious Games Learning Analytics Based on Standards and Open Code

Santilario-Berthilier Julio, Calvo-Morata Antonio, Martinez-Ortiz Ivan, Marc Spaniol, Fernandez-Manjon Baltasar

► To cite this version:

Santilario-Berthilier Julio, Calvo-Morata Antonio, Martinez-Ortiz Ivan, Marc Spaniol, Fernandez-Manjon Baltasar. SIMVA: Serious Games Learning Analytics Based on Standards and Open Code. Joint Proceedings of LAK 2025 Workshops co-located with 15th International Conference on Learning Analytics and Knowledge (LAK 2025), Dublin, Ireland, March 3-4, 2025, Mar 2025, Dublin, Ireland. hal-05266583

HAL Id: hal-05266583

https://hal.science/hal-05266583v1

Submitted on 18 Sep 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# SIMVA: Serious Games Learning Analytics Based on Standards and Open Code

Julio Santilario-Berthilier<fnref n="1" />,*,†, Antonio Calvo-Morata<fnref n="1" />,*,†, Ivan Martinez-Ortiz<fnref n="1" />,*,†, Marc Spaniol<fnref n="2" />,† and Baltasar Fernandez-Manjon<fnref n="1" />,†

## Abstract

Game Learning Analytics (GLA) collects game player interaction data that can be analyzed to offer insights into player behavior and support stakeholders such as developers and educators in understanding the learning process. However, current GLA methods are usually complex and costly to implement, with proprietary data formats and software, and too technology-dependent, limiting their applicability in real scenarios. To address these issues, we created SIMVA (SIMple VAlidator), an open-source, standards-based platform designed to simplify GLA applied to serious games. SIMVA uses the IEEE xAPI standard to collect and store data, facilitating player tracking and basic analysis/visualization. Our approach also allows for easy integration of various data storage systems (i.e. Learning Record Store) and different trackers, offering full data ownership to stakeholders. SIMVA also enables and simplifies game validation processes, supporting different experimental designs (e.g., control groups, A/B testing) and integrates pre-post questionnaires linked to player analytics data via anonymized codes. This integration supports the analysis of both quantitative and qualitative data, enhancing the assessment of the educational impact of games. Additionally, SIMVA is designed to simplify compliance with EU GDPR data privacy requirements. Overall, SIMVA offers a comprehensive open-code platform for both GLA and serious game validation. SIMVA will contribute to improving game effectiveness and to provide credible evidence of serious games educational value.

## Keywords

learning analytics, serious games

## 1. Introduction

Games are complex software systems designed to entertain, typically by integrating elements such as rules, goals, reward mechanisms, and challenges within a responsive and interactive framework (usually supported by a narrative). When these components are effectively combined through thoughtful design, they create engaging experiences for players[1]. Many of these elements have also proven effective in educational contexts, giving rise to the concept of serious games[2].

In education, those serious games have shown significant potential as tools that motivate learners [3], provide experiential learning with immediate feedback[4], and encourage active engagement in the learning process[5]. However, despite their proven benefits, the use of games in the classroom remains limited. This is largely due to several challenges, including difficulties in assessing student performance within games, technical issues related to deployment and classroom hardware, limited digital skills among teachers, and a lack of conclusive evidence on the effectiveness of educational games [6], [7]. Many of these barriers can be addressed with appropriate tools that better support teachers, enable real-time classroom monitoring and provide reliable assessment methods [8].

<figure><img src="image_2.png" /></figure>

<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /></figure>

<footnote n="1">¹ Universidad Complutense de Madrid Av. Complutense, s/n, Moncloa - Aravaca, 28040 Madrid, España</footnote>

<footnote n="2">² Université de Caen Normandie, Esp. de la Paix, 14000 Caen, France</footnote>

<footnote n="1">¹Joint Proceedings of LAK 2025 Workshops, co-located with 15th International Conference on Learning Analytics and Knowledge (LAK 2025), Dublin, Ireland, March 3-7, 2025.</footnote>

<footnote>*Corresponding author.</footnote>

<footnote>†These authors contributed equally.</footnote>

<footnote>jsantila@ucm.es (J. Santilario Berthilier); acmorata@ucm.es (A. Calvo Morata); imartinez@fdi.ucm.es (I. Martinez</footnote>

<footnote>Ortiz); marc.spaniol@unicaen.fr (M. Spaniol); balta@fdi.ucm.es (B. Fernandez Manjon)</footnote>

<footnote>00009-0003-2520-507X (J. Santilario Berthilier); 0000-0001-8701-7582 (A. Calvo Morata); 0000-0001-6595-5690 (I.</footnote>

<footnote>Martinez Ortiz); 0000-0002-5094-4523 (M. Spaniol); 0000-0002-8200-6216 (B. Fernandez Manjon)</footnote>
One promising approach to supporting the use of serious games in the classroom is the application of Game Learning Analytics (GLA) [9]. GLA involves collecting and analyzing data on players' interactions within a game to gain insights into both learner behavior and the actual learning process. This information can be used by different stakeholders to improve the learning experience, optimize educational content, evaluate game effectiveness, and assess overall learning outcomes. By using interaction data, GLA enables more informed instructional decisions, and supports teachers by providing actionable data to assess student progress and address learning challenges in real time. Additionally, it contributes to the continuous improvement of both teaching strategies and game design, it allows developers and researchers to validate educational game design and effectiveness.

Despite all the proven advantages of applying analytics, Game Learning Analytics (GLA) remains a complex, costly, and often fragile process that poses significant implementation challenges for many game developers. Consequently, most educational games are still evaluated using traditional methods such as interviews and self-reported questionnaires. Furthermore, existing GLA solutions are typically ad hoc and proprietary, limiting their adaptability and reusability across different games and educational contexts. To address these challenges, we propose SIMVA, a standards-based, open-source platform that provides a suite of tools to simplify the application of GLA, as well as the design, deployment, and validation of serious games through experimental methodologies.

## 2. SIMVA

Our open-source platform, SIMVA (SIMple VAlidator), is designed for third-party deployment on local servers to collect and analyze player interaction data. It is a complex software infrastructure, but the idea is that it can be easily installed from our GitHub repository<fnref n="2" /> using a Docker-based infrastructure and with a turn-key approach (see Figure 1).

SIMVA integrates a set of software components to streamline the application of Game Learning Analytics (GLA), including: a reverse proxy (Traefik), a Single Sign-On service (Keycloak), a survey manager (LimeSurvey), a data storage system for player interactions (MinIO), a trace processing engine (Kafka), a URL shortener (Shlink), our main web application (Simva), a trace reallocator (Simva Trace Allocator) and a basic data analysis and visualization tool (T-Mon). We also provide trackers (in Unity and JavaScript) to simplify the data collection by communicating in-game interaction data and where data move from or to SIMVA. By capturing detailed player interactions and combining them with qualitative data from surveys, SIMVA enables a comprehensive assessment of learning outcomes and player experience[10].

Figure 1 shows a diagram of the infrastructure and summarizes how each of the clients that can be accessed by users works.

* Dashboards (green color): SIMVA provides students, teachers, researchers and developers with a visualization system (TMON) to explore and analyze the data generated by the players. Through this framework (1), it is possible to visualize the list of available activities (2) and obtain information and visualizations that allow stakeholders to understand the data collected in each learning activity (3), but only if the stakeholder is owner of this activity and study data (researcher, teacher or developer) or is the user who generated the data (student).

<footnote n="2">² https://github.com/e-ucm/simva-infra/</footnote>
<figure><img src="image_5.png" /><figcaption>**Figure 1:** Simva Infrastructure and flow of data.</figcaption></figure>

* Game clients (blue color): SIMVA provides telemetry libraries (trackers) that games can integrate to record player actions. This makes it possible to establish a direct integration between the game and the platform (1), to trigger surveys before and after the gameplay
session (2,4) and to send interaction traces while students play (3). This data generated by the
surveys and games will be displayed on the dashboards. Only students with studies
permissions via Keycloak can send data from the game to SIMVA. In this way, valuable data
is collected to assess student performance and to evaluate the effectiveness of serious games.
While SIMVA specializes in integration with serious games, it can also be integrated with
other types of activities, whether digital or not.

* SIMVA client (purple color): Designed for teachers and researchers, this client facilitates the creation and management of game sessions. It allows organizing class groups and generate tokens by Keycloak (1) allowing anonymization of user information at source. Users can obtain the token lists for participants (2), access to the results of the activities carried out (4) and generate short links that simplify the distribution and access to the games in the classroom (5). In addition, it offers functionality to configure pre- and post-game surveys (3). Pre- and post-questionnaires evaluations together with the interaction data strengthen the validation of educational games.

The following sections describe in more detail what each of these clients allows.

**2.1. Activities and users' management**

SIMVA simplifies the validation of serious games [11] because it supports various experimental designs, including control groups and A/B testing. It also enables integrated data collection from multiple sources, such as gameplay, pre- and post-questionnaires or other types of educational resources such as articles, online presentations or videos with lessons. All data is linked to a player through a unique (anonymized) learner token, ensuring consistent and integrated tracking across activities.

**2.1.1. GamePlay Activity**

Gameplay activity, which is the core of the experiment, where students interact with the serious
game. While they play, SIMVA collects data on their in-game actions. Teachers and researchers are
able to monitor the learning process using an analytic panel and store the data securely for later
analysis - either in MinIO, as users backup or both, depending on the configuration shown in Figure
2. By setting the Game Uri parameter, the system knows where to redirect students and use this url to
track and associate player data correctly. The two main types of activities, but not the only ones, are
those related to surveys and games.

**2.1.2. Survey Activity**

The **Survey Activity** component encapsulates LimeSurvey<fnref n="3" />, a free and open-source software tool for creating and running online surveys. LimeSurvey strimelines tasks such as questionnaire deployment, participant management, status monitoring, and automated player response collection. To ensure flexibility and efficiency in survey management, the system provides multiple methods for survey preparation:

* Sometimes teachers or researchers want to **reuse a survey that they have already designed and tested**. This approach ensures consistency across experiments and allows comparison between studies. SIMVA supports this by allowing direct selection of existing surveys, either by knowing the unique identifier (ID) of the survey, by choosing from a list or by duplicating the existing test for another class, copying also the survey as shown in Figure 3.

<footnote n="3">³ https://www.limesurvey.org/</footnote>
* In some cases, stakeholders need to **create new surveys specifically designed** for the current study or context. This approach enables the inclusion of customized questions that align with the learning goal of the experiments.

* When surveys have been developed outside of the system, using a standardized template or prior survey, they can be imported directly. Those **pre-prepared survey designs** imply less work for the stakeholders, save time and ensure the carefully validated instrument can be reused across different experimental settings.

<figure><img src="image_6.png" /><figcaption>Figure 2 : Add GamePlay Activity</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Figure 3 : Add Survey Activity</figcaption></figure>

### 2.1.3. Users' management

Furthermore, all data collection is conducted anonymously to ensure compliance with European data privacy regulations, such as the EU GDPR [12], [13]. This is achieved through the use of anonymized tokens generated by the SIMVA platform (as shown in Figure 4), which are distributed to students via printed token reports.

<figure><img src="image_8.png" /><figcaption>Figure 4 : Token Generation (A), Group View (B) and Print (C)</figcaption></figure>
## 2.2. Trackers

A **tracker** is the software component that deals with the communication of the player interactions (traces) to a permanent external storage to enable analysis of behavior and learning outcomes. Two trackers are available: **JSTacker**, implemented for JavaScript, and **Xasu**, designed for use with Unity.

Both trackers support and implement xAPI standard so they can be used by a game programmer for sending out traces without requiring knowing the xAPI standard. xAPI is an international IEEE standard that provides a JSON format to track user actions. More specifically, the trackers support and implement an application profile of xAPI specially designed for representing relevant events in serious games [14].

Adopting the xAPI standard for representing interactions, along with a flexible platform for building custom components, greatly enhances the reusability of Game Learning Analytics (GLA) data [15]. It allows stakeholders to create, reuse, or modify system components while retaining full ownership of their data. For example, various trackers can be reused across projects, and new trackers can be developed to support different game platforms.

### 2.2.1. Xasu Unity Tracker

Xasu<fnref n="4" /> is a Unity plugin that includes a tracker capable of storing player interaction data either locally or on a server. It offers straightforward tracking functionality and flexible tools for configuring the collection of xAPI traces in a robust manner. Xasu can also send backups of interaction data to the server. To connect Xasu to a specific research study, users can download the **Xasu Config** from the SIMVA platform, represented as a blue dot cube in the gameplay activity (Figure 5). This configuration file provides the necessary settings to Xasu and automatically launches the game with the correct setup.

<figure><img src="image_9.png" /><figcaption>**Figure 5** : SIMVA console view with a study example. This study is composed of 3 activities to be carried out in the following order: pre-game questionnaire, game session and post-game questionnaire.</figcaption></figure>

### 2.2.2. JSTacker Javascript

JsTracker<fnref n="5" /> is a tracker designed for JavaScript-based web games. When integrated into a game, it sends player interaction data to the server and can also create a backup of this tracking data. To

<footnote n="4">⁴ https://github.com/e-ucm/xasu/</footnote>

<footnote n="5">⁵ https://github.com/e-ucm/js-tracker/</footnote>
streamline the experimental research process, the game’s URL can be added to SIMVA, allowing the scheduler to launch the game at the appropriate stage of the experiment (e.g., once the user has completed the pregame survey).

## 2.3. Data Storage and Access

If the trace Storage option has been activated in GamePlay activity shown in Figure 6, the data is stored in our database Minio as a json file - one json trace per line- accessible and downloadable by the stakeholders in the SIMVA web study page. If the backup was activated, there is an extra copy of that information to make the process more resilient to possible connectivity problems (this is quite frequent in actual school environments where we test our games).

To improve interoperability, SIMVA allows for using a standard xAPI data store that is usually known as a Learning Record Store (LRS) in xAPI vocabulary. From the game the data can also be sent to an existing LRS like Veracity<fnref n="6" /> or ADL LRS<fnref n="7" />.

<figure><img src="image_10.png" /><figcaption>**Figure 6 :** GamePlay Activity</figcaption></figure>

## 2.4. T-Mon Data Analysis and Visualization

Storing in-game user traces using the xAPI application profile for serious games (xAPI-SG) streamlines data analysis [16]. T-Mon<fnref n="8" /> is a lightweight data analysis and visualization tool specifically designed to process data in the xAPI-SG format [17]. It can load xAPI-SG statements from a JSON file (either a single file or one JSON trace per line) or directly from SIMVA storage via a connected user account. T-Mon then analyzes the data and generates a default set of visualizations, offering a quick overview and facilitating the creation of charts to represent game-based learning data. Originally developed as a Jupyter Notebook<fnref n="9" />, T-Mon is currently being migrated to a web-based

<footnote n="6">⁶ https://lrs.io/home/</footnote>

<footnote n="7">⁷ https://lrs.adlnet.gov/</footnote>

<footnote n="8">⁸ https://github.com/e-ucm/t-mon/</footnote>

<footnote n="9">⁹ https://jupyter.org/</footnote>
Python application using Dash<fnref n="10" />, to improve performance and streamline report generation, particularly for large datasets.

Additionally, users have the option to download the data and perform custom analyses or create their visualizations outside of SIMVA using their preferred tools.

## 3. Conclusions

SIMVA significantly streamlines the process of validating serious games, offering an integrated, standards-based platform that reduces the technical and methodological barriers often faced by researchers and developers. Traditionally, validating educational games requires complex coordination across data collection, participant management, assessment, and analysis— usually relying on fragmented or proprietary tools. SIMVA addresses this by providing a unified, open-source infrastructure that supports the entire research workflow: from deploying controlled experiments and capturing in-game behavior, to integrating survey data and generating actionable insights through built-in analytics tools like T-Mon.

By simplifying these processes, SIMVA helps educators, researchers, and developers to conduct serious games with rigorous, reproducible studies with minimal setup, while maintaining full ownership and control of their data. It allows for the creation of well-structured experimental designs, automated complex data collection, and standardized trace formats (xAPI-SG) making comparing results across studies and contexts easier.

Therefore, SIMVA lowers the barrier to entry for serious game validation, accelerates the production of reliable evidence, and, in the long run, can help to promote the wider adoption of serious games as credible, data-driven tools for learning, training, and skills development.

## Acknowledgements

This work has been partially funded by the Spanish Ministry of Education (PID2020-119620RB-I00; PID2023-149341OB-I00) and by the Telefónica-Complutense Chair of Digital Education and Serious Games.

## Declaration on Generative AI

During the preparation of this work, the author(s) used OpenAI ChatGPT-4 and DeepL in order to: Grammar and spelling check. After using these tool(s)/service(s), the author(s) reviewed and edited the content as needed and take(s) full responsibility for the publication's content.

## References

[1] H. Schoenau-Fog, "The Player Engagement Process – An Exploration of Continuation Desire in Digital Games," *DiGRA Digital Library*, vol. 2011, no. 1, Jan. 2011, doi: 10.26503/dl.v2011i1.540.

[2] Y. Zhonggen, "A Meta-Analysis of Use of Serious Games in Education over a Decade," *International Journal of Computer Games Technology*, vol. 2019, no. 1, p. 4797032, Jan. 2019, doi: 10.1155/2019/4797032.

[3] P. Wouters and H. van Oostendorp, "Overview of Instructional Techniques to Facilitate Learning and Motivation of Serious Games," *Instructional Techniques to Facilitate Learning and Motivation of Serious Games*, pp. 1–16, 2017, doi: 10.1007/978-3-319-39298-1_1.

[4] C. Burgers, A. Eden, M. D. Van Engelenburg, and S. Buningh, "How feedback boosts motivation and play in a brain-training game," *Comput Human Behav*, vol. 48, pp. 94–103, Jul. 2015, doi: 10.1016/J.CHB.2015.01.038.

<footnote n="10">¹⁰ https://dash.plotly.com/</footnote>
[5] J. Rugelj, "Serious Computer Games Design for Active Learning in Teacher Education," *Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST*, vol. 161, pp. 94–102, 2016, doi: 10.1007/978-3-319-29060-7_15.

[6] F. Bellotti, R. Berta, and A. De Gloria, "Designing Effective Serious Games: Opportunities and Challenges for Research," *International Journal of Emerging Technologies in Learning (IJET)*, vol. 5, pp. 22–35, Nov. 2010, doi: 10.3991/IJET.V5ISI3.1500.

[7] A. Dimitriadou, N. Djafarova, O. Turetken, M. Verkuyl, and A. Ferworn, "Challenges in Serious Game Design and Development: Educators' Experiences," *Simul Gaming*, vol. 52, no. 2, pp. 132–152, Apr. 2021, doi: 10.1177/1046878120944197.

[8] V. Shute and M. Ventura, "Stealth Assessment: Measuring and Supporting Learning in Video Games Valerie," *Computer Games, Simulations & Education*, p. 91, 2013.

[9] M. Freire, Á. Serrano-Laguna, B. Manero Iglesias, I. Martínez-Ortiz, P. Moreno-Ger, and B. Fernández-Manjón, "Game Learning Analytics: Learning Analytics for Serious Games," *Learning, Design, and Technology*, pp. 3475–3502, 2023, doi: 10.1007/978-3-319-17461-7_21.

[10] C. Alonso-Fernández, M. Freire, I. Martínez-Ortiz, and B. Fernández-Manjón, "Improving evidence-based assessment of players using serious games," *Telematics and Informatics*, vol. 60, p. 101583, Jul. 2021, doi: 10.1016/J.TELE.2021.101583.

[11] P.-C. I. Jose, P.-C. V. Manuel, M.-O. Ivan, F.-M. Manuel, and F.-M. Baltasar, "Simplifying Serious Games Authoring and Validation with Adventure and SIMVA," Institute of Electrical and Electronics Engineers (IEEE), Aug. 2020, pp. 106–108. doi: 10.1109/icalt49669.2020.00039.

[12] A. Alhazmi and N. A G Arachchilage, "A Serious Game Design Framework for Software Developers to Put GDPR into Practice," *ACM International Conference Proceeding Series*, Aug. 2021, doi: 10.1145/3465481.3470031; JOURNAL: JOURNAL:ACMOTHERCONFERENCES; CTYPE:STRING:BOOK

[13] M. Freire, A. Calvo-Morata, I. Martinez-Ortiz, and B. Fernandez-Manjon, "Bootstrapping serious games to assess learning through analytics," *IEEE Global Engineering Education Conference, EDUCON*, vol. 2023-May, 2023, doi: 10.1109/EDUCON54358.2023.10125177.

[14] Á. Serrano-Laguna, I. Martínez-Ortiz, J. Haag, D. Regan, A. Johnson, and B. Fernández-Manjón, "Applying standards to systematize learning analytics in serious games," *Comput Stand Interfaces*, vol. 50, pp. 116–123, Feb. 2017, doi: 10.1016/J.CSI.2016.09.014.

[15] C. Alonso-Fernandez, A. Calvo, M. Freire, I. Martinez-Ortiz, and B. Fernandez-Manjon, "Systematizing game learning analytics for serious games," *IEEE Global Engineering Education Conference, EDUCON*, pp. 1111–1118, Jun. 2017, doi: 10.1109/EDUCON.2017.7942988.

[16] C. Alonso-Fernández, A. Calvo-Morata, M. Freire, I. Martínez-Ortiz, and B. Fernández-Manjón, "Game Learning Analytics:: Blending Visual and Data Mining Techniques to Improve Serious Games and to Better Understand Player Learning," *Journal of Learning Analytics*, vol. 9, no. 3, pp. 32–49, Dec. 2022, doi: 10.18608/jla.2022.7633.

[17] C. Alonso-Fernandez, A. Calvo-Morata, M. Freire, I. Martinez-Ortiz, and B. F. Manjon, "Data science meets standardized game learning analytics," *IEEE Global Engineering Education Conference, EDUCON*, vol. 2021-April, pp. 1546–1552, Apr. 2021, doi: 10.1109/EDUCON46332.2021.9454134.