<figure><img src="image_1.png" /></figure>

HAL
open science

# Digital Gamification to Enhance Vaccine Knowledge and Uptake: Scoping Review

Ilaria Montagni, I. Mabchour, Christophe Tzourio

► To cite this version:

Ilaria Montagni, I. Mabchour, Christophe Tzourio. Digital Gamification to Enhance Vaccine Knowledge and Uptake: Scoping Review. JMIR Serious Games, 2020, 8 (2), pp.e16983. 10.2196/16983. hal-03132570

HAL Id: hal-03132570

https://hal.science/hal-03132570

Submitted on 5 Feb 2021

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Review

# Digital Gamification to Enhance Vaccine Knowledge and Uptake: Scoping Review

Ilaria Montagni¹,², PhD; Inass Mabchour³, MBBS; Christophe Tzourio¹, MD, PhD

¹Bordeaux Population Health U1219, Inserm-University of Bordeaux, Bordeaux, France

²Institute of Public Health, Epidemiology and Development (ISPED), University of Bordeaux, Bordeaux, France

³Faculty of Medicine Hyacinthe Bastaraud, University of Antilles-Guyane, Pointe-à-Pitre, Guadeloupe

**Corresponding Author:**

Ilaria Montagni, PhD
Bordeaux Population Health U1219
Inserm-University of Bordeaux
146 rue Léo Saignat
Bordeaux, 33000
France
Phone: 33 0547304281
Email: ilaria.montagni@u-bordeaux.fr

## Abstract

**Background:** Vaccine hesitancy is a growing threat to population health, and effective interventions are needed to reduce its frequency. Digital gamification is a promising new approach to tackle this public health issue.

**Objective:** The purpose of this scoping review was to assess the amount and quality of outcomes in studies evaluating gamified digital tools created to increase vaccine knowledge and uptake.

**Methods:** We searched for peer-reviewed articles published between July 2009 and August 2019 in PubMed, Google Scholar, Journal of Medical Internet Research, PsycINFO, PsycARTICLES, Psychology and Behavioral Sciences Collection, and SocINDEX. Studies were coded by author, year of publication, country, journal, research design, sample size and characteristics, type of vaccine, theory used, game content, game modality, gamification element(s), data analysis, type of outcomes, and mean quality score. Outcomes were synthesized through the textual narrative synthesis method.

**Results:** A total of 7 articles met the inclusion criteria and were critically reviewed. Game modalities and gamification elements were diverse, but role play and a reward system were present in all studies. These articles included a mixture of randomized controlled trials, quasi-experimental studies, and studies comprising quantitative and qualitative measures. The majority of the studies were theory-driven. All the identified gamified digital tools were highly appreciated for their usability and were effective in increasing awareness of vaccine benefits and motivation for vaccine uptake.

**Conclusions:** Despite the relative paucity of studies on this topic, this scoping review suggests that digital gamification has strong potential for increasing vaccination knowledge and, eventually, vaccination coverage.

(JMIR Serious Games 2020;8(2):e16983) doi: 10.2196/16983

## KEYWORDS

gamification; vaccination; vaccine hesitancy; digital tools; scoping review

## Introduction

Vaccination is one of the most cost-effective methods of preventing the spread of infectious diseases. The rates of people receiving vaccinations have recently declined in developed countries. If vaccination coverage falls below the thresholds that are safe for the prevention of epidemic transmission, the incidence of vaccine-preventable diseases increases [1]. One of the most illustrative examples of this phenomenon is the measles

outbreak that returned over the past 2 decades. In the first 6 months of 2019, reported measles cases were the highest they had been in any year since 2006, indicating a concerning and continuing upsurge in the overall measles burden worldwide [2].

Reasons why some people do not get vaccinated are as varied as they are complex and include a sense of complacency, difficulty in accessing vaccines, mistrust of health or medical authorities, spread of misinformation, underestimation of risks,
and limited knowledge of the benefits of vaccination and how it works [3,4]. According to a 2019 World Health Organization report, "vaccine hesitancy" (ie, the reluctance or refusal to be vaccinated) is one of the top 10 threats to global health [5]. Interventions to address vaccine hesitancy are urgently needed in order to promote vaccine acceptance and uptake in developed countries.

Many researchers have explored different ways to deliver fair information on vaccine risks and benefits through ad hoc interventions addressing different targets, ranging from parents [6] to adolescents [7], and concerning different types of vaccines, ranging from human papilloma virus (HPV) vaccination [8] to measles, mumps, and rubella vaccination [9]. In order to better design and implement such interventions, Willis et al [10] proposed a classification of 7 important items to be used in immunization communication methods: inform or educate, remind or recall, teach skills, provide support, facilitate decision making, enable communication, and enhance community ownership.

Gamification is defined as the use of game design elements in non-game contexts [11]. It encompasses several features and dimensions like fun interfaces, immediate success or continuous progress feedback, reward systems (point scores, badges, levels), challenges and competitions, team playing, avatars, and quizzes. These features echo the 7 items described by Willis et al [10] for efficacious immunization communication. Previous studies have been conducted worldwide on the use of gamification as a means to increase the initiation and retention of desired health behaviors [12-14]. By using game-based mechanisms, gamification stimulates participants' involvement and facilitates their learning about health [15]. Serious games and mobile or tablet applications with game-based features are increasingly used to not only train health professionals but also deliver prevention and health promotion messages to the general population [16,17]. In detail, serious games are defined as full-blown digital games applied to train and educate players and are not predominantly or exclusively intended for entertainment purposes [18,19]. On the other hand, gamified digital tools (eg, apps) are not a full game experience but just contain gaming elements such as scoring of points, in-game rewards, or engaging in quests [20]. Thus, gamification is a broader concept including but not limited to serious games. A literature review of empirical studies on gamification [21] has provided evidence on the effectiveness of the game-based approach on the user, particularly on motivation and engagement.

Given its increasing use in the public health domain, gamification might be a useful approach for interventions aimed to sensitize populations to the relevance of vaccination acceptance. Including game-based features might improve vaccination campaign strategies by educating individuals, explaining the risks they face if they are not vaccinated and encouraging them to keep their vaccination records up to date. Interventions using fun and interactive approaches and leveraging digital technologies to deliver positive views on vaccination are increasingly requested [22]. A previous systematic review identified 16 serious games related to vaccination developed from 2003 to 2015 [23]. However, data

on the effectiveness of these tools were not fully provided nor compared. Among the 16 serious games, only 2 games were formally evaluated [23,24]. Furthermore, other vaccination-related gamified digital tools, like mobile apps or quizzes, were not taken into account. A study on the evaluation of existing gamified digital tools for vaccination not limited to serious games would bridge this research gap.

The aim of this scoping review was to identify gamified digital tools that have been implemented and evaluated across diverse populations and types of vaccines in an effort to tackle issues of vaccine hesitancy. The effectiveness of identified tools in terms of impact on users' knowledge and behavior towards vaccination as well as their usability/acceptability were also synthetized. Thus, the overarching goal of this study was to respond to the need for information on evidence-based interventions that could help design and implement future gamified digital tools to address vaccination hesitancy.

# Methods

## Search Strategy

We conducted a scoping review using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement as a more robust methodological approach [25]. The search was performed between July 2019 and August 2019.

Before starting the review, we manually checked for relevant articles in the authors' languages (ie, French, Italian, and Spanish) on the first 20 pages of Google Scholar. Since we did not find any article on "gamification" AND "vaccine" in corresponding languages, we confirmed our initial choice to restrict our search to English-language papers. The following search terms and related variations were used: all ("vaccine*" OR "vaccination" OR "immunization*") AND all ("serious game*" OR "video game*" OR "therapeutic game*" OR "online game*" OR "game*" OR "game app*" OR "mobile game*" OR "digital game*" OR "gamif*").

Studies were selected from a search of the following major electronic databases: PubMed, Google Scholar, Journal of Medical Internet Research, PsycINFO, PsycARTICLES, Psychology and Behavioral Sciences Collection, and SocINDEX via EBSCOhost. A supplementary manual search was performed to identify additional relevant publications by reviewing the reference lists of the included articles and using ResearchGate.

## Selection Criteria

Only peer-reviewed studies written in English in the decade 2009-2019 were included, regardless of the location of the study, type of vaccination under study, and study population. All types of study designs were included (eg, quantitative, qualitative, and mixed-methods studies; systematic reviews; meta-analyses; randomized controlled trials; and other experimental studies like pretests and posttests). Posters, preprints, and conference proceedings were excluded. Studies were included only if they used a gamification technique or tool to deliver informative or educative messages on vaccination. All types of digital games or gamified elements were included, from serious games (content gamification) to gamified Web-based quizzes (structural gamification) [26]. Articles not presenting the description and
evaluation of a concrete gamified digital tool were excluded (eg, articles reporting the results of a survey on users' needs and perspectives on games for vaccination).

## Data Extraction and Quality Assessment

Records identified in the literature search were evaluated in a 3-step approach. First, all identified titles and abstracts were screened for eligibility and coded by one researcher according to the selection criteria. Second, relevant articles were retrieved, and full-text articles were read independently by the researcher in charge of coding and extracting all data and by a second researcher. Third, a final list of publications for full-text review was established and validated by the 2 researchers. Any discrepancies were reviewed by a third researcher and finally resolved through consensus.

The 2 researchers conducted a quality assessment using the quality assessment method presented by Connolly et al [27]. This method assesses the overall weight of empirical evidence for the positive impact and outcomes of games. Each final paper included in the review was given a score of 1 (low), 2 (medium), or 3 (high) across the following 5 criteria: (1) appropriateness of the research design, with a score of 3 for randomized controlled trials, 2 for quasi-experimental controlled studies, and 1 for case studies, single subject-experimental, pretest/posttest, and other types of quantitative and qualitative studies; (2) appropriateness of methods and analysis; (3) representativeness and generalizability of the findings; (4) relevance of the focus of the study; and (5) relevance of the findings and their discussion. The total score for each paper was calculated by adding the scores of all 5 dimensions, resulting in a range from 5 to 15 points. Following the studies by Connolly et al [27] and Johnson et al [13] using the same quality assessment method, we categorized articles with a score of ≤8

points as weaker evidence, articles with a rating >8 to 12 points as moderate evidence, and articles with a rating >12 points as stronger evidence. We calculated the average score for each study and measured the weighted Cohen's kappa coefficient to test interrater reliability.

Data were sorted in categories, including author, year of publication, country, journal, research design, sample size and characteristics, type of vaccine, theory used, game content, game modality, gamification element(s), data analysis, type of outcomes, and mean quality score. For game modality and gamification elements, we based our coding on the work by Hamari et al [21].

## Outcome Measures

To assess the effectiveness of gamified digital tools for vaccination, we took into account 3 types of outcomes: behavior (eg, real actions like receiving a vaccination or intent to get vaccinated), cognition (eg, increased knowledge of the topic or vaccine literacy [28]), and usability/acceptability (eg, appreciation of the intervention). Furthermore, detailed outcomes of the evaluation of each game were individually reported.

## Results

A total of 2432 records were identified through database searches. After duplicates were removed, remaining papers were assessed using the described selection criteria. The same tool was presented by 2 different papers [29,30], with similar results. We decided to select only the most recent study [30] since it provided more in-depth information, including qualitative data. As a result, 7 articles were finally included. Figure 1 shows the flowchart of the study selection process.

<figure><img src="image_3.png" /><figcaption>**Figure 1.** Flow diagram of the literature search according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement.</figcaption></figure>
**General Description of the Studies**

Each study presented 1 gamified digital tool, with the exception of 1 study that described and evaluated 2 tools: one on vaccination and one on antibiotics [31]. Only data concerning the vaccination game were taken into account for this scoping review.

The final 7 articles eligible for review were rated for the quality of the evidence: 2 articles were categorized as providing weaker evidence [24,32], 2 articles were categorized as providing moderate evidence [33,34], and 3 articles were categorized as providing stronger evidence [29,31,35]. The weighted Cohen's kappa coefficient was 0.71, suggesting good agreement between raters [36]. Concerning methodologies, 2 studies used a randomized controlled trial design [29,34], 1 study used a controlled experiment with posttest questionnaires [32], 1 study used only focus groups [33], and 1 study used only pretest and posttest questionnaires [24]. A mixed methods approach was used by 3 studies [30,31,35], balancing quantitative data from questionnaires with qualitative data from interviews or focus groups.

Game modalities and gamification elements were diverse (eg, avatars, challenges, informative feedback, points, levels, leaderboards, storytelling), but role play and reward systems were present in all studies. Of the 7 games, 4 were Web-based [24,31,32,34], whereas the other 3 were mobile apps [30,33,35]. One study [35] also used a social networking site. Of the 7 studies, 3 described tools that were cocreated by several stakeholders including health professionals, developers, and end users [24,33,35]. All gamified digital tools were promoted and funded through health authorities for public health purposes and universities for educational purposes. Of the 7 gamified

digital tools, 5 were serious games [24,31,33-35], and the other 2 were implemented as a quiz [30] and website [32].

Infectious diseases were the specific focus of 6 studies: 3 studies addressed HPV [33-35], 2 studies addressed influenza [24,32], and 1 study addressed the combination of measles, mumps, and rubella [30]. The remaining study addressed vaccination in general [31].

Tools were mostly aimed at young players: 3 tools were developed for school-aged children [24,31,33], and 2 tools were developed for university students [32,34]. For 1 of the tools developed for children [33], evaluation data were collected also from parents. The remaining 2 tools were developed for the general population [30,35].

All studies but one [27] explicitly incorporated one or more behavioral theories: self-determination theory [33], health belief model [33,34], self-concept theory [34], theory of reasoned action and planned behavior [34], game theory [24,32], social value orientation [32], nudge theory [35], and empowerment model [30].

Positive effects of gamified interventions were reported in all 7 studies across at least one of the 3 outcomes we considered (ie, behavior, cognition, and usability/acceptability). For behavior, data were available for 5 studies [24,30-32,34]; for cognition, data were available for 3 studies [30,31,35]; and for usability/acceptability, data were available for 4 studies [30,31,33,35].

Table 1 outlines the characteristics of the 7 articles included in the review presented in chronological order. See Multimedia Appendix 1 for a description of the games and outcomes.
<table><caption>**Table 1.** Summary of the characteristics of the 7 studies.</caption><thead><tr><th>Author(s) (year of publication)</th><th>Country</th><th>Journal</th><th>Research design</th><th>Sample size</th><th>Sample characteristics</th><th>Type of vaccine</th></tr></thead><tbody><tr><td>Bertozzi et al (2013) [24]</td><td>United States</td><td>International Journal of Gaming and Computer-Mediated Simulations</td><td>Preplay and postplay assessments through face-to-face questionnaires</td><td>12</td><td>Lower-income, school-aged children</td><td>Influenza</td></tr><tr><td>Böhm et al (2014) [32]</td><td>Germany</td><td>Journal of Economic Behavior & Organization</td><td>Controlled laboratory experiment (8 sessions) with postexperiment quantitative data collected through questionnaires</td><td>180 (124 women)</td><td>College-aged social science students, mean age 23.37 years (SD 4.09 years)</td><td>Influenza</td></tr><tr><td>Ruiz-Lopez et al (2016) [35]</td><td>Norway</td><td>Journal of Medical Internet Research Serious Games</td><td>Mixed-methods with beta testing, focus groups (2 sessions), and self-administered questionnaires</td><td>40 for the beta testing, 6 women for the first focus group, 23 (10 girls) for the second focus group</td><td>First focus group age range 40-60 years, second focus group (high school students) age range 16-18 years</td><td>HPV<sup>a</sup></td></tr><tr><td>Cates et al (2018) [33]</td><td>United States</td><td>Games for Health Journal</td><td>Focus group (5 sessions)</td><td>16 preteens (5 girls) and 9 parents (7 women)</td><td>11-12 years old (preteens)</td><td>HPV</td></tr><tr><td>Darville et al (2018) [34]</td><td>United States</td><td>Simulation & Gaming</td><td>Randomized controlled trial (2x2 fully crossed between subjects) with post-experimental quantitative data collected through questionnaires (eg, Likert scales)</td><td>108</td><td>College-aged male students (18-26 years old)</td><td>HPV</td></tr><tr><td>Fadda et al (2017) [30]</td><td>Italy</td><td>Journal of Medical Internet Research mHealth and uHealth</td><td>Mixed methods, including a randomized controlled trial, Web-based survey (including the Mobile App Rating Scale [37]), and qualitative telephonic interviews</td><td>140 (138 women) for the survey, 60 for the telephone interviews</td><td>Parents</td><td>MMR<sup>b</sup></td></tr><tr><td>Eley et al (2019) [31]</td><td>United Kingdom</td><td>Journal of Medical Internet Research Serious Games</td><td>Mixed methods including pre- and post-experiment questionnaires, focus groups (26 sessions), and open-ended questions in the post-experiment questionnaire</td><td>473 (123 juniors and 350 seniors) for the questionnaire, 126 for the 26 focus groups</td><td>Students 7-16 years old</td><td>Infectious diseases</td></tr></tbody></table>

# Description of the Evaluation Outcomes of Individual Games

## Flu Busters!

The outcomes of this game [24] concerned behavior in terms of real action (ie, receiving vaccination). The game was deployed with 12 school-aged children in clinic waiting rooms before their appointments. A group of physician and midwives asked them to play the game and collected their feedback. Preliminary testing demonstrated that the game was reasonably successful in achieving its stated goal (ie, 10 of the 12 children were vaccinated for the flu after playing the game). Therefore,

92% of those who played the game agreed to be vaccinated compared to 42% (5/12) of a group of 12 clinic patients at the same premises who did not play the game. This demonstrates the real-world effectiveness of the game, despite the small sample size. Of the children, 27% (3/12) was able to answer how the vaccine worked after the game, while all of the children understood that germs invading the body and altering its cells were bad and that the vaccine could help. Children especially appreciated the animations of the protagonist *Vaccine Man* wrestling with the flu virus. Through feedback, the researchers also realized that the game was effective in communicating how prevalent the flu virus is in heavily populated environments.

<footnote><sup>a</sup>HPV: human papillomavirus</footnote>

<footnote><sup>b</sup>MMR: measles, mumps, and rubella</footnote>
## I-Vax Game

Key findings of this game [32] concerned behavior. In fact, this was a game model to simulate vaccination behavior based on real-world vaccination decisions. I-Vax underwent a controlled laboratory experiment involving 4 sessions with 2 treatment groups each (n=96 total), 3 different sessions with 2 treatment groups each (n=84 total), and 1 session with 1 control group. Results showed that the reaction to the game varied by personality, but that the game could contribute to a better understanding of vaccination behavior and vaccine hesitancy. Those with a positive attitude were vaccinated more often and did not alter their vaccination behaviors, while those with a neutral attitude had higher switching rates (P<.001). Pro-socials were more likely to get vaccinated than pro-selfs (P<.001). Participants experiencing adverse effects after vaccination within the game were more likely to change their attitude towards vaccination (P<.001). Overall, participants reported an intent to change health behaviors after playing.

## FightHPV

Outcomes of this game-based learning tool [35] reflected both cognition and usability/acceptability. Feedback was collected from 40 participants (employees of the Cancer Registry of Norway), a focus group of 6 participants aged 40-50 years (members of the Norwegian Women's Public Health Association), and a focus group of 23 high school students aged 16-18 years. HPV knowledge and cognition before and after playing the game was evaluated in the 22 participants from the second focus group who returned a questionnaire. Gameplay data from the beta testing study were collected using Google Analytics. Concerning cognition, after playing the game, concepts about HPV vaccination were better understood, and an increase in HPV knowledge was observed (P=.001). As for usability/acceptability, all those who returned the questionnaire stated that FightHPV was an appealing educational tool, 69% (18/26) reported that they liked the game, and 81% (21/26) stated that the game was challenging. Google Analytics showed that the game was easy to access and use but that players stopped the game when it became too hard.

## Land of Secret Gardens

This interactive videogame [33] presented outcomes only on usability/acceptability. Data were collected qualitatively through 3 focus groups conducted with a total sample of 16 boys and girls aged 11-12 years distributed among the focus groups as well as 2 parallel focus groups with a total sample of 9 parents distributed among the focus groups. Input on game design and perspectives on the game concept were investigated. Land of Secret Gardens was considered acceptable by both the preteens and parents. Preteens especially liked the mixture of entertaining and instructional elements and the opportunity to earn tokens and advance levels. Parents also favored game levels that were contingent on correct answers to HPV knowledge questions. However, some parents expressed hesitancy around games as motivational tools. In general, the game was appreciated as an opportunity to enhance communication about HPV between preteens and parents.

## VAX!

This study [34] studied outcomes on behavior only. This randomized control trial involved 24-29 students per condition group. The effects of avatar characters (assigned or customized) and perception of self (ideal or actual) on HPV risk perception, HPV vaccine self-efficacy, and behavioral intent to receive the HPV vaccine were tested. Outcomes of the evaluation were perception of self, self-efficacy, and HPV vaccine intention. Although results were not statistically significant (P values ranging from .581 to .001), data analysis indicated an increase in gain scores for risk perception, self-efficacy, and behavioral intention when participants were able to customize their avatar to look like their ideal or actual self. After playing the game, participants declared their intent to get vaccinated.

## Morbiquiz

The outcomes of Morbiquiz [30] concerned behavior, cognition, and usability/acceptability. The game was evaluated through a randomized controlled trial and mixed-methods with a Web-based survey (n=140) and qualitative telephonic interviews (n=60) to explore participants' experiences with the app. Objective outcomes were measured using an adapted version of the Mobile App Rating Scale [37] corresponding to 4 objective qualities (engagement, functionality, aesthetics, and information quality) and 2 subjective qualities (star-rated question and possible recommendation of the app). In addition, 3 items were included to assess participants' perceived impact of the app on their knowledge, on their help seeking, and the perceived likelihood of an actual change in the target health behavior. Concerning behavior, players reported significantly higher intention to vaccinate (P=.03) and more confidence in the decision (P=.006). When compared with the control group (empowerment and knowledge intervention), those receiving the app intervention were more likely to actually change their behavior and look for health information to opt for vaccination. As for cognition, all experimental groups reported a significant increase in their vaccination knowledge compared with the control group (P<.001). Concerning usability/acceptability, functionality and aesthetics scored high. The results of the focus group were a general appreciation of the design and content of the app. Participants defined the app as useful, trustworthy, innovative, and engaging and described their experience as fun and pleasant. Most participants reported that MorbiQuiz was highly convenient, meaning that it is handy, quick, noninvasive, easily accessible, and functional, and stressed that the game invited users to seek information actively thanks to its gamified approach.

## Stop the Spread

The outcomes of Stop the Spread [30] concerned behavior, cognition, and usability/acceptability. A total of 123 junior-aged students and 350 senior-aged students (age range, 7-16 years) from 5 UK educational provisions completed knowledge and evaluation questionnaires before and after using the game. Focus groups with 126 students were also conducted. As for behavior, in some focus groups, students reported an intent to change their health behavior. Concerning cognition, after playing, participants reported that their knowledge about sneezing behaviors and vaccinations increased significantly (P<.05) for both age groups.
Concerning usability/acceptability, the mean enjoyment score for Stop the Spread was 6.2/10 for juniors and 5.1/10 for seniors; participants found that Stop the Spread was fast-paced and challenging. Overall, many students reported positive perceptions of their user experience, with a few suggestions for improvement.

# Discussion

## Principal Findings

We identified 7 studies reporting data on the evaluation of the effectiveness of digital tools using gamification on the topic of vaccination. All 7 studies presented positive results in terms of pre-established outcomes (ie, behavior, cognition, and usability/acceptability) confirming that gamified digital tools can facilitate communication of vaccination-related messages and contribute to increased vaccination uptake. These results agree with those of previous studies demonstrating that gamification can contribute to changed behaviors and increased knowledge as well as be appreciated by users with regard to health-related topics [38]. When combining gamification and emerging technologies, results might be even more promising [39]. Digital tools have the advantage of being ubiquitous without time and space constraints [40], and the inclusion of gamified features might increase their appeal and acceptability, as reported in our study. Based on these assumptions, the design of future interventions should consider the use of both new technologies and gamification.

Previous research described existing serious games used for vaccination [23]. However, selected tools dated back to the year 2015 and were just listed and fully described without any appraisal of their effectiveness. Furthermore, only serious games were analyzed in this previous study [23], without considering other digital tools that were not full-blown games but used gamified features [20]. Through our review, we provide an update, through the year 2019, of existing digital games for vaccination and an evaluation of these games. Given the speed at which technology changes and improves, monitoring new digital tools is essential. Most importantly, to our knowledge, our study is the first scoping review synthesizing data on the evaluation of the effectiveness of different types of digital tools using gamification, not exclusively serious games.

We classified games per type of outcome. The 4 studies [24,30,32,34] reporting data on behavior showed a remarkable increase in the intent to get vaccinated and a positive attitude towards vaccination. Similar results on behavior change have been found in serious games for oral hygiene [41], asthma [42], and fruit and vegetable consumption [43]. Effectiveness in terms of behavior change might be explained by the fact that gamification tends to improve the involvement and motivation of users who feel more convinced of their decision after playing [44,45]. Games are thought to provide a good medium for increasing self-efficacy and changing behavior as they offer the opportunity for a new experience in a safe environment, without real-life consequences to making wrong decisions [46].

Positive findings on cognition were reported by 3 studies [30,31,35], corresponding to increased knowledge and literacy

about vaccination. This agrees with a meta-analysis of serious digital games for healthy lifestyle promotion [46]. According to this previous scientific work, health knowledge is easier to influence than other outcomes, but the impact of a change in knowledge is not as strong as influencing a person's intent to change behavior. These positive results might be explained by the learning-by-doing approach used in gamified digital tools where players learn through exploration and experimentation.

In terms of usability/acceptability, results on this outcome were reported by 3 studies [30,31,35]. Effectiveness in terms of usability is potentially justified by the co-construction process. Including different stakeholders like health professionals and game developers is time-consuming but is useful for developing games that will appeal to the target audience [47]. One game promoting healthy eating in children [48] and one game to help young people quit smoking [49] underlined the advantages of using a co-construction approach to increase commitment to and acceptability of the games.

While all 7 gamified digital tools were effective across the 3 types of outcomes, they presented different characteristics (ie, game features, targeted audience, and targeted disease for vaccination). Among successful game features, role play or characterization, earning and losing tokens, and advancing levels were the 3 modalities that were the most used and accepted by users across all studies. As for role play or characterization, FightHPV [35] and Stop the Spread [31] were appreciated by users because they included a set of characters that were considered really amusing. In particular, the avatars in Vax! [34] allowed users to better appraise risk perception for infectious diseases as well as to increase one's intent to receive vaccination. Presenting a story with a character with which users could identify might be preferred because the character refers to one's doubts and knowledge and helps virtually measure the impact of one's decision. Character identification can improve risk perception and encourage vaccination uptake [50], since, within the game, the harms of nonvaccination can be virtually self-experimented. Furthermore, following a herd immunity approach [51], some of the games under study made users interact and confront with other imaginary characters to explain the collective dimension and community-level impact of vaccination. Immersive story telling could also have enhanced engagement and subsequent retention of key messages. As for earning and losing tokens, like in I-Vax [32], Land of Secret Gardens [33], and Morbiquiz [30], the presence of tokens in a game might encourage the players and maintain their motivation. The explanation can be found in behavioral theories: reinforcement and punishment contingencies are equally effective as long as they challenge the user [52]. Participants feel their role in the game is active, which makes them more engaged with the game. As for advancing levels, we know from the Stop the Spread game [31] that players enjoy the steady increase in difficulty as the game progresses. However, if games are too difficult, like the first version of FightHPV [35], users stop playing and cannot learn if they do not advance. Thus, it is important to design the game so that players feel challenged but not frustrated. Like earning and losing tokens, advancing levels maintains the interest and motivation of the player. Finally, results showed that using a serious game format, like
in most of the studies [24,31,33-35], was not more effective
than the other two formats (ie, quiz [30] and website [32]), thus
confirming that using gamifying features might be as effective
as a full-blown game. Future research including more studies
is needed to validate this hypothesis.

As for the targeted audience, apart from 1 study involving parents who were a little skeptical about the trustworthiness of a gamified approach [33], all other studies targeted young people (children, students, young parents) who were attracted by digital gamified tools and represented the most captive audience. About 90% of teenagers play video games [53], and millennials are simultaneously technologically adept with and shaped by technology [54]. As future parents and adults making decisions about their health and the health of their relatives, young people represent a core target group for interventions addressing vaccination coverage.

As for the type of targeted vaccine, all games except Stop the
Spread [31] targeted a specific vaccine. This might have been
a good strategy to clearly explain one vaccine at a time without
confusing the player. Targeting other vaccines like hepatitis,
for instance, with new games might be interesting to have a
complete spectrum of vaccines explained through gamified
digital tools. On the other hand, the design of games seems to
be independent from the type of vaccine. This means that the
same game might be transferable to all types of vaccine. This
could be the case of games like Morbiquiz [30], VAX! [34],
and FightHPV [35], whose designs can be easily adapted to
other vaccines or vaccine-related diseases.

Finally, the main findings of this review include the relevance
of incorporating behavioral theories within the game conception.
Almost all gamified digital tools under study were based on
solid theories and proved to be effective in facilitating
understanding and appraisal of information about vaccines and
behavior change endpoints. As suggested by previous literature,
theory-driven interventions are more efficacious than those that
are not [55], which might further explain the positive outcomes
of synthesized tools supported by the gamification approach.

Limitations of this review were the comparative paucity of
included studies and the marked heterogeneity of their study
design and contents. Publication bias should also be considered,
as small studies with negative findings would likely not be
published. Furthermore, the scores of some articles were low
[33,34] for various reasons including a limited study focus, a
research design that was not completely appropriate,
nongeneralizable findings, or small sample sizes. Only 3 articles
[30,31,34] presented a solid evaluation methodology relying
on quantitative and qualitative pretest and posttest data. We
also relied on the work of a single coder, which might have
introduced systematic bias. However, by omitting a second
coder, we wished to ensure consistency in the study selection.
Finally, all 7 studies presented positive short-term outcomes,
while no long-term impact assessment of the games was
conducted.

Recommendations for the Design of Future
Interventions

Based on our study, we can suggest how to improve the design
of future interventions. Especially during this period of
disinformation about vaccination circulating on the internet,
gamified digital tools can help provide more accurate
information, while being fun and engaging. These
recommendations might ensure that produced games are
attractive, validated, and effective, especially if they are
produced with researchers and professionals in the vaccine
domain.

First, the educational perspective is fundamental for all games,
but it must be implicit. Interventions should be focused on
increasing knowledge and influencing behavior about
vaccination without clearly presenting their final aim. Users
might unconsciously learn by playing without having the feeling
that they are following a training class or didactic presentation.
It is important to use a narrative approach with an appropriate
story line to engage, motivate, and empower users throughout
the learning process. However, the information provided must
be factual and trustworthy.

Second, emotional engagement between the player and the
environment should be created. Fun contributes to such
engagement. Using characters and avatars is a good strategy to
capture and maintain the attention of the players. Avatars can
self-represent the player in a simulation or role-playing game
so that players are immersed in the game; if users are active,
their chances to learn increase. Emotional engagement is also
reinforced with levels and tokens. Their use is highly
recommended for future gamified digital tools. Provide a
leaderboard for competition not only with other players but also
with oneself as a personal challenge. However, game functions
like recovery aids and the pace of the games should be adapted
to the public, to avoid frustration with levels that are too
difficult.

Third, a balance should be found between simplicity and attractiveness. Future games should pay attention to the design and aesthetics of the game. Using concise text might help with retention of information and knowledge about vaccination, while not stopping the flow of the game. Amusing characters and animations are necessary to make the game appealing.

Fourth, games should take into consideration their target audience. The best solution to achieve this is to co-construct the games with all concerned stakeholders, including end users, through a design-thinking approach [56]. In general, games addressing children and young adults (especially boys) have the advantage of targeting a population who is familiar with video games and computers in general. This might not be the case for older adults.

Finally, before designing a gamified digital intervention, it is
important to consider its costs, which are usually very high (at
least US $10,000).

Implications for Future Research

Our review suggests that there is a need to continue developing
gamified digital tools but also to evaluate the impact of existing
and future tools. Great enthusiasm for these games has led to
many being produced with insufficient validation of their
effectiveness. To better evaluate gamified digital tools in the
short term, medium term, and long term, preferred study designs
are randomized controlled trials or any other experimental
design with a control group, combined with longitudinal data
collection. Using a mixed methods design would also be
beneficial to comprehensively capture users' opinions and
satisfaction with the game. Like in our scoping review, the
outcomes to measure might be behavioral indicators like
individual vaccination records or intent to get vaccinated,
cognitive indicators like an increase in knowledge or vaccine
literacy, and indicators of usability/acceptability of the
intervention. If possible, all 3 types of outcomes should be
measured to better assess the overall qualities of the games.

Instruments like the Mobile App Rating Scale [37] could be
used for this purpose. These evaluation studies will eventually
help us understand how gamified digital tools can change
vaccination uptake and coverage.

Conclusions

Gamification is an innovative and promising option to consider when designing vaccination-related interventions addressed to the general public and young people in particular, especially for those who are hesitant about vaccination. Based on the findings of this review, health professionals, health promotion and prevention specialists, and developers are encouraged to use game-based features in interventions aimed to endorse vaccination uptake in order to increase their acceptability and consequent effectiveness. Theory-driven gamified digital tools are preferred.

Acknowledgments

The authors are indebted to the i-Share project team for their support throughout the study.

**Conflicts of Interest**

None declared.

Multimedia Appendix 1

Summary of the findings on gamified digital tools in the 7 studies.

[DOCX File, 16 KB-Multimedia Appendix 1]

References

1. WHO UNICEF. Global and regional immunization profile. 2019 Dec 10. URL: https://www.who.int/immunization/monitoring_surveillance/data/gs_gloprofile.pdf?ua=1 [accessed 2020-04-08]

2. World Health Organization. New Measles Surveillance Data from WHO. Geneva; 2019 Aug. URL: https://www.who.int/immunization/newsroom/new-measles-data-august-2019/en/ [accessed 2020-04-08]

3. Kester LM, Zimet GD, Fortenberry JD, Kahn JA, Shew ML. A National Study of HPV Vaccination of Adolescent Girls: Rates, Predictors, and Reasons for Non-Vaccination. Matern Child Health J 2012 Jun 23;17(5):879-885. [doi: 10.1007/s10995-012-1066-z]

4. Paterson P, Chantler T, Larson HJ. Reasons for non-vaccination: Parental vaccine hesitancy and the childhood influenza vaccination school pilot programme in England. Vaccine 2018 Aug;36(36):5397-5401. [doi: 10.1016/j.vaccine.2017.08.016]

5. World Health Organization. WHO bulletin. Ten threats to global health in 2019 URL: https://www.who.int/emergencies/ten-threats-to-global-health-in-2019 [accessed 2020-03-06]

6. Trivedi D. Cochrane review summary: Face-to-face interventions for informing or educating parents about early childhood vaccination. Prim Health Care Res Dev 2014 Oct;15(4):339-341. [doi: 10.1017/S1463423614000322] [Medline: 25146256]

7. Golden SD, Moracco KE, Feld AL, Turner KL, DeFrank JT, Brewer NT. Process Evaluation of an Intervention to Increase Provision of Adolescent Vaccines at School Health Centers. Health Educ Behav 2014 Apr 30;41(6):625-632. [doi: 10.1177/1090198114531773]

8. Kharbanda EO, Stockwell MS, Fox HW, Andres R, Lara M, Rickert VI. Text message reminders to promote human papillomavirus vaccination. Vaccine 2011 Mar;29(14):2537-2541. [doi: 10.1016/j.vaccine.2011.01.065]

9. Hawe P, McKenzie N, Scurry R. Randomised controlled trial of the use of a modified postal reminder card on the uptake of measles vaccination. Arch Dis Child 1998 Aug 01;79(2):136-140 [FREE Full text] [doi: 10.1136/adc.79.2.136] [Medline: 9797594]

10. Willis N, Hill S, Kaufman J, Lewin S, Kis-Rigo J, De Castro Freire SB, et al. Erratum to: "Communicate to vaccinate": the development of a taxonomy of communication interventions to improve routine childhood vaccination. BMC Int Health Hum Rights 2013 Sep 18;13(1). [doi: 10.1186/1472-698x-13-37]

11. Groh F. Gamification: State of the Art Definition and Utilization. 2012 Presented at: 4th Seminar on Research Trends in Media Informatics; 02/14/2012; Ulm.

12. Sardi L, Idri A, Fernández-Alemán JL. A systematic review of gamification in e-Health. J Biomed Inform 2017 Dec;71:31-48 [FREE Full text] [doi: 10.1016/j.jbi.2017.05.011] [Medline: 28536062]
13. Johnson D, Deterding S, Kuhn K, Staneva A, Stoyanov S. Hides L. Gamification for health and wellbeing: A systematic review of the literature. Internet Interv 2016 Nov;6:89-106 [FREE Full text] [doi: 10.1016/j.invent.2016.10.002] [Medline: 30135818]

14. Lister C, West JH, Cannon B, Sax T, Brodegard D. Just a fad? Gamification in health and fitness apps. JMIR Serious Games 2014 Aug;2(2):e9 [FREE Full text] [doi: 10.2196/games.3413] [Medline: 25654660]

15. Cugelman B. Gamification: what it is and why it matters to digital health behavior change developers. JMIR Serious Games 2013 Dec;1(1):e3 [FREE Full text] [doi: 10.2196/games.3139] [Medline: 25658754]

16. Mandryk RL, Birk MV. The Potential of Game-Based Digital Biomarkers for Modeling Mental Health. JMIR Ment Health 2019 Apr 23;6(4):e13485. [doi: 10.2196/13485]

17. Raiff BR, Fortugno N, Scherlis DR, Rapoza D. A Mobile Game to Support Smoking Cessation: Prototype Assessment. JMIR Serious Games 2018 Jun 07;6(2):e11. [doi: 10.2196/games.9599]

18. Sawyer B, Rejeski D. Serious Games: Improving Public Policy through Game-Based Learning and Simulation. In: Woodrow Wilson International Center for Scholars. Washington: Woodrow Wilson International Center for Scholars; 2002.

19. Alvarez J, Damien D. An introduction to serious games: Definitions and concepts. In: Serious Games & Simulation for Risk Management. Montagret: LARSEN Science; 2011.

20. Fleming TM, Bavin L, Stasiak K, Hermansson-Webb E, Merry SN, Cheek C, et al. Serious Games and Gamification for Mental Health: Current Status and Promising Directions. Front Psychiatry 2016 Jan;7:215 [FREE Full text] [doi: 10.3389/fpsyt.2016.00215] [Medline: 28119636]

21. Hamari J, Koivisto J, Sarsa H. Does gamification work? - A literature review of empirical studies on gamification. 2014 Presented at: Annual Hawaii International Conference on System Sciences; 6-9 January 2014; Manoa. [doi: 10.1109/hicss.2014.377]

22. Wilson K, Atkinson K, Crowcroft N. Teaching children about immunization in a digital age. Hum Vaccin Immunother 2017 May 04;13(5):1155-1157 [FREE Full text] [doi: 10.1080/21645515.2016.1271519] [Medline: 28165917]

23. Ohannessian R, Yaghobian S, Verger P, Vanhems P. A systematic review of serious video games used for vaccination. Vaccine 2016 Aug;34(38):4478-4483. [doi: 10.1016/j.vaccine.2016.07.048]

24. Bertozzi E, Krilov L, Walker D. Successful game development partnerships between academics and physicians: Two case studies. Int J Gaming Comput Simulations 2013. [doi: 10.4018/jgcms.2013070107]

25. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. J Clin Epidemiol 2009 Oct;62(10):1006-1012. [doi: 10.1016/j.jclinepi.2009.06.005] [Medline: 19631508]

26. Kapp KM. What Is Gamification? In: The Gamification of Learning and Instruction: Game-Based Methods and Strategies for Training and Education. Hoboken: John Wiley & Sons; 2012.

27. Connolly TM, Boyle EA, MacArthur E, Hainey T, Boyle JM. A systematic literature review of empirical evidence on computer games and serious games. Computers & Education 2012 Sep;59(2):661-686. [doi: 10.1016/j.compedu.2012.03.004]

28. Lorini C, Santomauro F, Donzellini M, Capecchi L, Bechini A, Boccalini S, et al. Health literacy and vaccination: A systematic review. Hum Vaccin Immunother 2018 Feb 01;14(2):478-488 [FREE Full text] [doi: 10.1080/21645515.2017.1392423] [Medline: 29048987]

29. Fadda M, Galimberti E, Fiordelli M, Romanò L, Zanetti A, Schulz PJ. Effectiveness of a smartphone app to increase parents' knowledge and empowerment in the MMR vaccination decision: A randomized controlled trial. Hum Vaccin Immunother 2017 Nov 02;13(11):2512-2521 [FREE Full text] [doi: 10.1080/21645515.2017.1360456] [Medline: 29125783]

30. Fadda M, Galimberti E, Fiordelli M, Schulz PJ. Evaluation of a Mobile Phone-Based Intervention to Increase Parents' Knowledge About the Measles-Mumps-Rubella Vaccination and Their Psychological Empowerment: Mixed-Method Approach. JMIR Mhealth Uhealth 2018 Mar 07;6(3):e59. [doi: 10.2196/mhealth.8263]

31. Eley CV, Young VL, Hayes CV, Verlander NQ, McNulty CAM. Young People's Knowledge of Antibiotics and Vaccinations and Increasing This Knowledge Through Gaming: Mixed-Methods Study Using e-Bug. JMIR Serious Games 2019 Feb 01;7(1):e10915. [doi: 10.2196/10915]

32. Böhm R, Betsch C, Korn L. Selfish-rational non-vaccination: Experimental evidence from an interactive vaccination game. Journal of Economic Behavior & Organization 2016 Nov;131:183-195. [doi: 10.1016/j.jebo.2015.11.008]

33. Cates JR, Fuemmeler BF, Diehl SJ, Stockton LL, Porter J, Ihekweazu C, et al. Developing a Serious Videogame for Preteens to Motivate HPV Vaccination Decision Making: Land of Secret Gardens. Games Health J 2018 Feb;7(1):51-66 [FREE Full text] [doi: 10.1089/g4h.2017.0002] [Medline: 29161529]

34. Darville G, Anderson - Lewis C, Stellefson M, Lee Y, MacInnes J, Pigg RM, et al. Customization of Avatars in a HPV Digital Gaming Intervention for College-Age Males: An Experimental Study. Simulation & Gaming 2018 Oct 17;49(5):515-537. [doi: 10.1177/1046878118799472]

35. Ruiz-López T, Sen S, Jakobsen E, Tropé A, Castle PE, Hansen BT, et al. FightHPV: Design and Evaluation of a Mobile Game to Raise Awareness About Human Papillomavirus and Nudge People to Take Action Against Cervical Cancer. JMIR Serious Games 2019 Apr 08;7(2):e8540. [doi: 10.2196/games.8540]

36. Landis JR, Koch GG. The Measurement of Observer Agreement for Categorical Data. Biometrics 1977 Mar;33(1):159. [doi: 10.2307/2529310]
37. Stoyanov SR, Hides L, Kavanagh DJ, Zelenko O, Tjondronegoro D, Mani M. Mobile app rating scale: a new tool for assessing the quality of health mobile apps. JMIR Mhealth Uhealth 2015 Mar;3(1):e27 [FREE Full text] [doi: 10.2196/mhealth.3422] [Medline: 25760773]

38. Thompson D. Designing Serious Video Games for Health Behavior Change: Current Status and Future Directions. J Diabetes Sci Technol 2012 Jul;6(4):807-811. [doi: 10.1177/193229681200600411] [Medline: 22920806]

39. Tozzi AE, Gesualdo F, D'Ambrosio A, Pandolfi E, Agricola E, Lopalco P. Can Digital Tools Be Used for Improving Immunization Programs? Front Public Health 2016 Mar 08;4:36 [FREE Full text] [doi: 10.3389/fpubh.2016.00036] [Medline: 27014673]

40. Atkinson KM, Wilson K, Murphy MS, El-Halabi S, Kahale LA, Laflamme LL, et al. Effectiveness of digital technologies at improving vaccine uptake and series completion – A systematic review and meta-analysis of randomized controlled trials. Vaccine 2019 May;37(23):3050-3060. [doi: 10.1016/j.vaccine.2019.03.063]

41. Aljafari A, Rice C, Gallagher JE, Hosey MT. An oral health education video game for high caries risk children: study protocol for a randomized controlled trial. Trials 2015 May 28;16(1). [doi: 10.1186/s13063-015-0754-6]

42. Greer M, Lin L, Atkinson RK. Using a computer game to teach school-aged children about asthma. Interactive Learning Environments 2016 Jan 28;25(4):431-438. [doi: 10.1080/10494820.2015.1135469]

43. Jones BA, Madden GJ, Wengreen HJ. The FIT Game: preliminary evaluation of a gamification approach to increasing fruit and vegetable consumption in school. Prev Med 2014 Nov;68:76-79. [doi: 10.1016/j.ypmed.2014.04.015] [Medline: 24768916]

44. Theng Y, Lee JWY, Patinadan PV, Foo SSB. The Use of Videogames, Gamification, and Virtual Environments in the Self-Management of Diabetes: A Systematic Review of Evidence. Games Health J 2015 Oct;4(5):352-361. [doi: 10.1089/g4h.2014.0114] [Medline: 26287926]

45. Deterding S. Gamification. interactions 2012 Jul 01;19(4):14. [doi: 10.1145/2212877.2212883]

46. DeSmet A, Van RD, Compernolle S, Baranowski T, Thompson D, Crombez G, et al. A meta-analysis of serious digital games for healthy lifestyle promotion. Prev Med 2014 Dec;69:95-107. [doi: 10.1016/j.ypmed.2014.08.026] [Medline: 25172024]

47. Muniesa F. How Users Matter: The Co-construction of Users and Technologies. Br J Sociology 2004 Dec;55(4):598-598. [doi: 10.1111/j.1468-4446.2004.00040_10.x]

48. Ptakauskaite N, Chueng-Nainby P, Pain H. Supporting social innovation in children: Developing a game to promote health eating. 2016 Presented at: The 15th International Conference on Interaction Design and Children; June 2015; Manchester. [doi: 10.1145/2930674.2935980]

49. Scholten H, Granic I. Use of the Principles of Design Thinking to Address Limitations of Digital Mental Health Interventions for Youth: Viewpoint. J Med Internet Res 2019 Jan 14;21(1):e11528 [FREE Full text] [doi: 10.2196/11528] [Medline: 31344671]

50. Baranowski MT, Lu PAS, Buday R, Lyons EJ, Schell J, Russoniello C. Stories in Games for Health: More Pros or Cons? Games Health J 2013 Oct;2(5):256-263 [FREE Full text] [doi: 10.1089/g4h.2013.0052] [Medline: 26196925]

51. Metcalf C, Ferrari M, Graham A, Grenfell B. Understanding Herd Immunity. Trends Immunol 2015 Dec;36(12):753-755. [doi: 10.1016/j.it.2015.10.004] [Medline: 26683689]

52. Donaldson JM, DeLeon IG, Fisher AB, Kahng S. Effects of and preference for conditions of token earn versus token loss. J Appl Behav Anal 2014 May 21;47(3):537-548 [FREE Full text] [doi: 10.1002/jaba.135] [Medline: 24844817]

53. Garell C. Health games - healthy in what way? A systematic review of the current state of research in health games for children and adolescents. In: Thesis. Gothenburg: University of Gothenburg; 2015.

54. Lynch-Sauer J, Vandenbosch TM, Kron F, Gjerde CL, Arato N, Sen A, et al. Nursing students' attitudes toward video games and related new media technologies. J Nurs Educ 2011 Sep;50(9):513-523. [doi: 10.3928/01484834-20110531-04] [Medline: 21627050]

55. Moore G, Cambon L, Michie S, Arwidson P, Ninot G, Ferron C, et al. Population health intervention research: the place of theories. Trials 2019 Jun 11;20(1). [doi: 10.1186/s13063-019-3383-7]

56. Montagni I. Does design thinking really work? Views on co-creation practices in research. 2018 Presented at: XXI Congrès de La SFSIC; 13-15 June 2018; Paris.

Abbreviations

HPV: human papillomavirus.

MMR: measles, mumps, and rubella.
Edited by G Eysenbach; submitted 11.11.19; peer-reviewed by K Wilson, T Skousen; comments to author 07.03.20; revised version received 13.03.20; accepted 20.03.20; published 18.05.20

Please cite as:

Montagni I, Mabchour I, Tzourio C

Digital Gamification to Enhance Vaccine Knowledge and Uptake: Scoping Review

JMIR Serious Games 2020;8(2):e16983

URL: http://games.jmir.org/2020/2/e16983/

doi: 10.2196/16983

PMID:

©Ilaria Montagni, Inass Mabchour, Christophe Tzourio. Originally published in JMIR Serious Games (http://games.jmir.org), 18.05.2020. This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Serious Games, is properly cited. The complete bibliographic information, a link to the original publication on http://games.jmir.org, as well as this copyright and license information must be included.