# A Learning Analytics Approach in Web-Based Multi-user Learning Games

Matthias Ehlenz, Thiemo Leonhardt, Ulrik Schroeder

► To cite this version:

Matthias Ehlenz, Thiemo Leonhardt, Ulrik Schroeder. A Learning Analytics Approach in Web-Based Multi-user Learning Games. Open Conference on Computers in Education (OCCE), Jun 2018, Linz, Austria. pp.167-171, 10.1007/978-3-030-23513-0_17. hal-02370920

HAL Id: hal-02370920

https://inria.hal.science/hal-02370920

Submitted on 19 Nov 2019

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# A Learning Analytics Approach in Web-based Multi-user Learning Games

Matthias Ehlenz, Thiemo Leonhardt, Ulrik Schroeder

RWTH Aachen University, Learning Technology Research Group, Germany, {Ehlenz, leonhardt, schroeder}@cs.rwth-aachen.de

**Abstract.** As technology changes, learning games are adapted to target audience and available devices. Analytics methods must keep up with keeping the learner in focus. This work presents the Multi-Touch Learning Game (MTLG) framework, designed to implement cross platform educational games with support for cooperative, collaborative and competitive settings. It shows adaption of a user-centred learning analytics data model, the learning data context model, to fit circumstantial requirements of multi-user settings on a shared device in games implemented using the MTLG framework. A first field study has been conducted, and the results, challenges and lessons learned are discussed.

**Keywords.** Game-based learning, learning analytics, multi-touch, educational games, learning context

## 1 A new style of learning

Learning nowadays requires educators around the globe to understand current developments of everyday habits of their students to provide methods of knowledge transfer via alternative channels, for example, game-based learning. To avoid unnecessary overheads in development processes, the method of choice for producers is often web-based. Hypertext markup language (HTML) and JavaScript based content can be viewed on a range of devices and can even be compiled without much effort in native applications (apps) for nearly all the common target platforms. Though this is carried out on an everyday basis, most learning analytics standards do not reflect those different settings. This paper aims to fill the gap between context awareness in learning analytics and their application in game-based learning. This challenge is approached by extending the Learning Context Data Model to fit modern collaborative contexts of "playing together" and specifics regarding educational games, to offer a broad range of analytic possibilities in game-based learning.

The paper has three parts. First, we explain the Multi-Touch Learning Framework to integrate learning analytics in web-based educational games and applications. Second, we introduce the Learning Context Data Model as an alternative to the Experience Application Program Interface (API), as it focuses on the learner and not only on the learning event. Third, we present our first field study as a proof of concept.
## 2 Multi-touch learning games

Educational games play an important role in modern education. It is widely accepted that engaging games enhance learning in a wide variety of ways [1]. As progress leads to a wide spread in platforms, developers are often either forced to stick with their area of expertise, thus limiting the reachable audience, gather an expensive team, or risk the time-consuming effort of getting accustomed to new areas. An alternative is to follow the more modern path of a cross-platform approach.

Most of those used on an everyday basis by the target audience of educational games, mainly children and teenagers, are not desktops or notebooks but tablets and smartphones are dominating this area [2]. Touch interaction has outrun keyboard and mouse and a big share of newly-introduced notebooks feature touch functionality. Thus, educational games make use of the possibilities of gestures and ten finger input that students have grown accustomed to. Furthermore, technology has advanced and produced touchscreens in growing sizes and capabilities. This opens up opportunities far beyond single learners behind their own devices and enables content creators to provide true collaborative learning environments where all users have per definition equal rights and chances of participation.

The Multi-Touch Learning Game framework (MTLG) developed by the Learning Technologies Research Group at RWTH Aachen University aims to assist university students, teachers and other developers to create suitable user experience on devices and for their demanding clientele, while maintaining core features of educational games [3]. In this paper, we focus on discussing the analytics component, as it represents the foundation of understanding behavioural patterns in multiuser multi-touch learning environments, and describe the challenges those collaborative settings bring.

## 3 Intention of the learning context data model

Learning analytics is a current trend, but is not a new and revolutionary topic. Since Learning Management Systems (LMSs) have been around, educators have striven to understand their learners and improve their systems to optimise user experience and, in the long run, maximise learning efficiency.

After early approaches to standardise a data format for interchange and storage of learning records like Tin Can API, the successor, Experience API () has become a de-facto standard for learning record stores and is implemented in all major LMSs.

xAPI represented a huge improvement compared to Tin Can regarding support of game-based learning, platform transitions and learning in teams, but still is mostly fitting for traditional e-learning scenarios. xAPI is a specification with the set main goal of interoperability in mind and providing a widely generic “one size fits all” solution which proves applicable in most settings.

Using xAPI to store the whole context of a user's learning environment stretches the bounds of this ambiguity beyond the point of good conscience, especially when interoperability is not the priority in the intended project. Thus, during the Prime Project, Thüs et al.[4] developed the Learning Context Data Model (LCDM), which is a learning record data definition aiming to provide a structural way to collect
information not only on the learning process itself but also on the context of the learner [5, 6]. Furthermore, this model is not only platform agnostic, it provides a structured pattern for storing the platform, source, session, action and general learning setting (work, academic, private); it introduced a hierarchical category system, with entities to model user interests [7] and generic entities.

The LCDM was implemented for usage in professional settings, but with the intention to allow self-reflection on daily tasks and learning activities [8]. Thus, it is not an event-centric or application-centric data model, but focuses on the learner. This user-centric idea emphasises applicability of this data model in multi-user settings as context might play an important role in analysis of gathered data. For use in MTLG games, the LCDM has been ported by a Representational State Transfer (REST)-Interface using a simple token-based authentication system. During implementation, an adaptive REST endpoint was created to facilitate exports of all events in xAPI format to keep future opportunities for using analytics toolkits, relying on the said standard.

## 4 First field studies and challenges

As proof of concept and a first research prototype, a four-user learning game for the introduction of regular expressions (a central topic in theoretical computer science and foundation for formal languages), has been implemented as the first game using the data collector and the LCDM back end.

<figure><img src="image_2.png" /><figcaption>**Figure 1.** PoC Prototype (Source: Own work - http://tabula-content.informatik.rwth-aachen.de/games/dem1-RegEx/)</figcaption></figure>

The basic idea shown in Figure 1 is that each player has her/his own drop area, associated with a regular expression. In the common area of the screen, word cards appear, either matching (one of) those regular expressions or not. Players have to touch matching cards and drop them in their drop area. Correctly assigned cards are awarded two points, while mismatched cards reduce the score by one. Depending on the setting,
there are either individual scores or a team score, individual expressions or the same for all players, and the play modes consist of two versus two and cooperative playing.

One goal of the study was to explore the practicability of the data model in real usage scenarios and investigate reliability of the produced data set. Information gathered for each touch interaction consisted of the result (success or not), drop area, regular expression on the drop area, timestamps and duration as well as session identifier (id).

Random sample checks comparing event data with the video recording of those sessions suggests that the data set is complete, and no interaction events have been lost. Nevertheless, there is a good amount of unassigned word card interactions since the first approach was guessing player id from the drop area, and cards moved in the common area could not be resolved that way, thus being assigned to the unknown player "John Doe" for later assignment in post-processing.

Another problem surfaced during observation, and shown by analysis of the visual recordings. Both in cooperative and competitive settings (less in the former, often in the latter), cards have been dropped in areas of different players, in the first case usually during the process of mutual explanation, and in the second, in taking advantage of the score malus for wrong cards for the opposite team for dropping those cards across the table. While this could be seen from a mostly negative perspective, some researchers [9] indicate that such a change in behavioural pattern can be interpreted as emotional involvement and thereby immersion in the playing experience. Still, this renders the collected data effectively useless in an unprocessed state in the attempt of user-centric analysis and demands of approaches either in post-processing or future data collection.

While this first field study merely aimed, from the technological perspective, at showing the capabilities of the data collector and back end, it left us with valuable insights and optimism regarding the planned approaches on the expected challenges.

Regarding further field studies in laboratory conditions, post-processing in regard to user assignment will be rendered obsolete. The TABULA Project enrichens learning experiences with so-called tangibles [10], objects that can be detected by any capacitive touchscreen device and adds a haptic aspect to the process.

While the users in our first case study have already been using the first version of passive tangibles for comparable user experiences, those tangibles are not distinguishable from each other. In the next, those tangibles will become active. By adding a microcontroller board with Bluetooth capabilities, the tool in the hands of each student will provide options like a direct and discreet feedback channel to each user, uniquely identifiable and linked (in the context of this prototype) to a specific user for the whole game session, which will empower us to assign "cross reaching" events as well as "unassigned" events to a specific user.

# 5 Conclusion and outlook

The first field study shows that the LCDM fulfils the needs of multi-user multi-touch learning games. There are adjustments to make to fit specific use cases, but mostly at a semantic level, like extending the range of values for a platform (by a MTTABLE value), or minor categories (by touch interaction events). Introducing a fourth category type GAME is in discussion. Software tools are currently being implemented enabling
analysts to synchronise a recorded video with an LCDM event stream for easy, time-
efficient post-processing for general study use without availability of tangibles.

The next iterations will collect an advanced set of interaction data including the start and end coordinates of each touch interaction, opening up the possibility to answer straightforward questions like “Does this user collect from the whole common area or just from within arm’s reach” to sophisticated ones like “Does a user become more hesitant over [time/failed attempts/difficulty]?” by looking at pixel distance to duration ratios. Further research will be done, aiming to implement automated, personalised feedback, continuing and transferring the groundwork laid in a previous study [11].

**Acknowledgments.** The authors acknowledge the financial support by the Federal Ministry of Education and Research of Germany in the call ELE (Project TABULA).

References

1. Hamari, J., Shernoff, D.J., Rowe, E., Coller, B., Asbell-Clarke, J., Edwards, T.: Challenging games help students learn: An empirical study on engagement, flow and immersion in game-based learning, Computers in Human Behavior, 54, 170-179 (2016)

2. Medienpädagogischer Forschungsverband Südwest mpfs: JIM-Studie 2017. Jugend, Information, (Multi-) Media. Basisuntersuchung zum Medienumgang 12- bis 19-Jähriger. Medienpädagogischer Forschungsverband Südwest mpfs, Stuttgart, Germany (2017)

3. Ehlenz, M., Leonhardt, T, Schroeder, U.: Spielend leicht Lernspiele entwickeln - Ein Framework für Multitouch-Lernspiele. In: Igel, C., Ullrich, C., et al. (Hrsg.) Bildungsräume 2017. Gesellschaft für Informatik, Bonn, Germany, 297-302 (2017)

4. Thüs, H., Chatti, M.A., Yalcin, E., Pallasch, Ch., Kyryliuk, B., Mageramov, T., Schroeder, U.: Mobile learning in context. International Journal of Technology Enhanced Learning, 4(5/6), 332–344 (2012)

5. Thüs, H., Chatti, M.A., Greven, C., Schroeder, U.: Kontexterfassung, modellierung und auswertung in Lernumgebungen. In Trahasch, S., Plötzner, R., et al. (Eds.) Proceedings der 12. e-Learning Fachtagung Informatik, DeLFI, Gesellschaft für Informatik, Freiburg, Germany, 157-162 (2014)

6. Thüs, H., Chatti, M.A., Schroeder, U., Kammer, T.: Kontexterfassung zur Anreicherung mobiler Anwendungen. Presented at the Mobile Learning Workshop, DeLFI, Bremen, Germany (2013)

7. Thüs, H., Chatti, M.A., Brandt, R., Schroeder, U.: Evolution of Interests in the Learning Context Data Model. In Conole, G., Klobucar, T., et al. (Eds.) Proceedings of the 10th European Conference on Technology Enhanced Learning, Toledo, Spain. Springer-Verlag, Heidelberg, Germany, 479-484 (2015)

8. Thüs, H., Soworka, M., Brauner, P., Schroeder, U.: Smart Experience Sampling in Android. In S. Rathmayer, H. Pongratz (Eds.) Proceedings of the Mobile Learning Workshop, DeLFI, München, Germany, 213-221 (2015)

9. Jabbar, A.I.A., Felicia, P.: Gameplay Engagement and Learning in Game-Based Learning: A Systematic Review. Review of Educational Research, 85(4), 740-779 (2015)

10. Voelker, S., Cherek, C., Thar, J., Karrer, T., Thoresen, C., Øvergård, K.I., Borchers, J.: PERCs: persistently trackable tangibles on capacitive multi-touch displays. Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology, Charlotte, NC. ACM, New York, NY, 351-356 (2015)

11. Herding, D.: The Tutor-in-the-Loop Model for Formative Assessment. Dissertation, RWTH Aachen University, Aachen, Germany (2013)