DOI: 10.5281/zenodo.2564815
title: RAGE_Pilot2_SPL-ITAlert

description: <strong>General description: </strong>

The data are gathered during an evaluation of the first prototypical version of the game IT Alert within the RAGE project.

This evaluation investigated the quality of the ecologically valid educational setting with the game (i.e. game & debriefing) for acquiring team skills (communication and collaboration). The quality of the game was examined by analysing participants’ opinions on usability, studyability, and perceived usefulness of the game for learning. In addition, more objective measurement of participants’ learning progress was gathered via in-game traces (i.e., in-game scores on two variables regarded important in IT team functioning while maintaining the integrity of a networked system). The main research question targeted in this evaluation was: Does the educational setting with the game (i.e., game & debriefing) lead to learning progress, and by which effort and satisfaction of the learner (task load, motivation)?

In total, 47 students installed and started IT Alert, but only 24 students truly participated in the most relevant part of the evaluation, completing two game sessions and the evaluation questionnaires.

According to the results, performance gains occurred from the educational setting with the game.

<strong>Topic</strong><br>
ACM CSS 2012: Applied computing~Interactive learning environments   • <em>Human-centered computing~Usability testing</em> <br>
PsycINFO Classification: 3500 Educational Psychology - 3530 Curriculum & Programs & Teaching Methods; 3550 Academic Learning & Achievement

<strong>Name entitites</strong><br>
Organizational information: Open University of the Netherlands<br>
Geographical information: The Netherlands<br>
Time information: February 2018 – November 2018

<strong>Types</strong>: SPSS-file (version 24)

<strong>RAGCS target group:</strong> demand side – end-users - adults- vocational school

<strong>Evaluation dimensions</strong><br>
Evaluation object: IT Alert game<br>
Methodology/design: The research focuses on the combined role of game and debriefing [three variants] (using game-traces from session 1 [as pre-test] and session 2 [as post-test]) on the development of the individual in team functioning (looking at individual performance & team performance and considering collaboration and communication as team skills).<br>
Evaluation variables: Communication skills, Self-perceived communication knowledge, Communicatin attitude, Usability, Studyability, Task load, Motivation

<strong>Instruments:</strong> Communication skills (in-game-traces); Self-perceived communication knowledge (ICKA, Wilkins et al., 2015); Communication attitude (CSAS, Laurence et al., 2012); Usability (SUS, Brooke, 1996); Studyability (Nadolski & Hummel, 2017); Task load (NASA-TLX, Hart & Staveland, 1988); Motivation (IMI, Ryan & Deci, 2000)

<strong>Knowledge/skill elements</strong><br>
RAGCS skills: skills - cognitive skills – applying; skills - affective skills - interpersonal

ESCO skills: coordinate communication within a team (http://data.europa.eu/esco/skill/cd85312f-0e0b-460b-addd-57d6df800595)

<strong>Relationships</strong>: D8.4 Second RAGE Evaluation Report