# Designing Gamified Interventions for Autism Spectrum Disorder: A Systematic Review

Murilo C. Camargo, Rodolfo M. Barros, Jacques D. Brancher, Vanessa Barros, Matheus Santana

► To cite this version:

Murilo C. Camargo, Rodolfo M. Barros, Jacques D. Brancher, Vanessa Barros, Matheus Santana. Designing Gamified Interventions for Autism Spectrum Disorder: A Systematic Review. 1st Joint International Conference on Entertainment Computing and Serious Games (ICEC-JCSG), Nov 2019, Arequipa, Peru. pp.341-352, 10.1007/978-3-030-34644-7_28. hal-03652013

HAL Id: hal-03652013

https://inria.hal.science/hal-03652013

Submitted on 26 Apr 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Designing Gamified Interventions for Autism Spectrum Disorder: A Systematic Review

Murilo C. Camargo¹, Rodolfo M. Barros¹, Jacques D. Brancher¹, Vanessa T. O. Barros¹, and Matheus Santana¹

¹ State University of Londrina, Londrina, PR 86057 970, Brazil
{murilocrivellaric, rodolfomdebarros, ss.matheus.94}@gmail.com,
{jacques, vanessa}@uel.br

**Abstract.** Serious games and gamified interventions have become increasingly popular among researchers and therapists dealing with the autistic audience. The number of studies on technology for autism has multiplied, with the aim to foster independence and improve learning outcomes. Nevertheless, designing interventions for Autism Spectrum Disorder is challenging, due to the complex clinical conditions and the broad range of symptoms covered by the disturbance. Thus, this systematic review investigates the current status of gamification resources for autism, with a special interest in the gamification elements and the User Interface design. We describe the planning and the searching procedures and present the data extracted from 30 primary sources. The studies analyzed show a multitude of gamification elements and a plethora of methods and strategies to support decision-making and improve accessibility in the development of autism-specific software. It is concluded that the existence of methodological gaps related to the definition of the target audience and the conduction of testing may impose additional challenges to the development process, whilst the combination of gamification elements is generally positive.

**Keywords:** Gamification, Autism Spectrum Disorder, Systematic Review.

## 1 Introduction

According to the Diagnostic and Statistical Manual of Mental Disorders – DSM-5 [1], autism is characterized by persistent limitation in social communication/interaction, restrictive and repetitive behavioral patterns, and (verbal and non-verbal) communication deficits. As the autism diagnosis evolved to cover a broader spectrum of symptoms, there is emerging evidence of increasing epidemic rates. Numbers may vary depending on the metrics used for evaluation, but there are studies reporting on rates as high as 1 in 150 children being diagnosed with Autism Spectrum Disorder (ASD) [2].

Accessibility has played an important role in providing autistic individuals with opportunities to overcome their limitations by fostering independence and assisting them in social relationships [3]. Despite the fact that much information on autism etiology remains to be uncovered, accessibility advocates (researchers, psychologists,
therapists, teachers, among others) have been thriving to propose feasible and interest-
ing intervention solutions, which can be confirmed by the growing number of com-
puter-based resources published and available to the community.

Serious games and gamification strategies have been largely used in the treatment of psychiatric conditions, such as depression, eating disorders, substance use, demen- tia, and, also, Autism Spectrum Disorder [4]. Gamification is a powerful resource to increase motivation and engage participants, thus favoring the teaching-learning pro- cess or training for specific situations. Design elements of serious games and/or gami- fied interfaces often include storylines, mid-term and long-term goals, increasing level of difficulty, feedback and/or rewards, and provision of choice [5].

Recent studies suggest improvements in handling disorder-related symptoms through the use of serious games or gamified interventions [6]. The same applies to autism: gamification may be a potentially effective approach due to the high visual processing skills the ASD individuals possess. On the other hand, developers may face challenges. According to [5], little evidence exists that ASD individuals subject- ed to computer-based interventions are able to apply such learning to real life. This can be partly explained by the lack of fundamental gamification elements during the design process.

In light of this situation, there is an undeniable need to understand the current sta-
tus of serious gaming targeted to Autism Spectrum Disorder. Through a systematic
review, this paper collects data on serious games and/or gamified interventions for
autism, with a focus on the gamification elements used and their visual representation
in terms of the User Interface design. We believe that this study may help uncover
best practices used and shed light on the graphic design approach to deliver them at
the same time.

This paper is organized as follows: in section 2, the review protocol is presented, including research questions, inclusion criteria, search strings, and databases; in section 3, results from the review are described and discussed; finally, section 4 brings conclusions and future perspectives regarding the design of gamified interventions for the autistic audience.

## 2 Review Protocol

This systematic review applies a three-phase process, including the design of the re-
view protocol, the conducting of the search, and the data extraction [7]. We search,
identify, analyze, and summarize evidence on computer-based gamification resources
targeted to the ASD audience. In order to prevent biased or mixed results, the scope is
limited to resources in which the User Interface is the primary output channel. Robot-
ics, hardware-only ubicomp solutions, or any interface-less devices are not included
in this study, because their examination would require a completely alternative ap-
proach.

Preliminary studies and consultations with field experts helped pilot the review
protocol. The protocol was designed to provide comprehensive guidance through the
review process, with respect to the inclusion criteria and the nature of the results. It
intends to promote the identification of all relevant primary sources and assist the data
extraction process at the same time. The research questions are: what is/are the learn-
ing objectives? What is the development process/methodology/approach employed?
Who is the target audience? What gamification elements are used to improve learning
skills? What representation methods are applied in the User Interface design?

Then, a list of keywords was defined to support the searching process. Keywords
should cover three aspects of major importance, namely: (1) the target audience, that
is, autistic individuals; (2) the object of study – software development; and (3) the
intervention approach – computer-based serious games or gamified interventions.
Similar keywords were grouped to form a search string, using Boolean "ANDs" and
"ORs": ("Autism spectrum disorder" OR "autistic individuals" OR "autism" OR
"high-functioning autism") AND ("computer-assisted learning" OR "computer-
assisted intervention" OR "computer-enhanced learning" OR "therapy software")
AND ("serious games" OR "gamification" OR "gamified resources").

In order to be included, papers had to adhere to the sensitivity inclusion criteria: describe a novel contribution to computer-based serious games for ASD; have the software development as the primary object of study; provide relevant information on the gamification elements used during development; and rely on the User Interface as the main output channel. Secondary studies and hardware-only interventions were not considered. Also, publications older than 10 years were discarded, thus prioritizing the recent literature.

Six sources were selected in this study: (1) IEEE Xplore Digital Library, (2) ACM Digital Library, (3) SpringerLink, (4) SCOPUS, (5) ScienceDirect, and (6) Google Scholar. Together, they grant access to a large collection of relevant resources for Software Engineering. Besides, important journals on autism research can be accessed through these sources, such as the Journal of Autism and Developmental Disorders (Springer) and the Autism (SCOPUS).

Searching was performed using a combination of the search string described previ-
ously. Multiple trials of the keywords were performed in order to reach as many pri-
mary studies as possible. The search resulted in a combined total of 2,995 papers,
which were downloaded. After the exclusion of duplicated papers, the total narrowed
down to 2,642 studies. Then, two assessors examined titles and abstracts to determine
the relevance of the subject matter. Papers that failed to meet the inclusion criteria
were eventually withdrawn from this review, resulting in 30 eligible studies.

3 Results and Discussion

All 30 selected papers were carefully examined according to the review protocol.
Data extracted was compiled in quality reports and double-checked whenever asses-
sors had conflicting recommendations. Quality reports included data on the learning
objectives, the team organization, the software development process applied, the tar-
get audience, gamification elements, and the User Interface design.

The selected papers present a variety of learning objectives. Many aim to improve
communication skills (46.6%), mostly by teaching new vocabulary and phrases while
improving semantic meaningfulness. Other learning objectives are social interaction (13.3%), basic education (10%), and daily activities (10%). Two studies focused on facial recognition [14, 18] and only one aimed at refining motor skills [10]. Six of them (20%) presented multiple learning purposes [9, 13, 17, 19, 25, 32]. All of them involved the major impairments faced by ASD individuals: difficulty of socialization, imagination impairment, communication deficit, and behavioral issues [12].

<figure><img src="image_2.png" /><figcaption>Fig. 1. Learning objectives extracted from primary studies.</figcaption></figure>

Popular development approaches are the Participatory Design [15, 17, 30, 32] and the User-Centered Design [22, 23, 24, 31]. Both are techniques that aim to involve external help (experts, parents, teachers, and/or end-users) to assist the development process. Other approaches mentioned are the Applied Behavior Analysis [12], the ADDIE approach (Analysis, Design, Development, Implementation, Evaluation) [33], and the Extreme Programming [20]. Some studies reported on the use of an "iterative" [13, 14, 16, 21, 27, 29, 35] and/or "agile" approach [21, 27, 36], although they did not provide detailed information on it.

The team organization is either cross-functional or participatory. *Cross-functional teams* are composed by the development staff and stakeholders, who are deeply involved in the development lifecycle. On the other hand, *participatory teams* also engage stakeholders, but they are involved only in specific moments during the process (requirements specification or testing, for example). In any case, this reveals an effort to understand the needs and preferences of the end-user. Two studies did not provide any information on the methodological approach or the team organization [10, 34].

The vast majority of studies analyzed (76.6%) refer to their target audience simply as "autistic children" or some variation of the term. Only four studies report on the development of software for severity-specific audiences – that is, high-functioning and/or low-functioning autism [24, 28, 32, 36]. Two studies (6.6%) referred to the audience as "autistic individuals", claiming that the proposed intervention would serve for ASD adolescents or adults as well [16, 34]. There is also one study [20] that designed an intervention for children with "specific disorders", such as ASD, Down syndrome, mental retardation, etc. According to the author, this is possible due to the
use of the PECS (Picture Exchange Communication System), which is an educational resource largely used in interventions for several disturbances, including autism.

<table><caption>**Table 1.** Gamification elements and User Interface design.</caption><thead><tr><th rowspan="2">Reference</th><th colspan="10">Gamification elements</th><th colspan="8">User Interface design</th><th rowspan="2">Objective</th></tr><tr><th>Feedback</th><th>Rewards</th><th>Virtual reality</th><th>Level of difficulty</th><th>Personalization</th><th>Multimedia</th><th>Storytelling</th><th>Monitoring</th><th>Custom learning</th><th>Othezr</th><th>Illustration</th><th>Photographs</th><th>Words / Text</th><th>Use of colors</th><th>Sounds</th><th>Voice</th><th>Video / VR</th></tr></thead><tbody><tr><td>[11]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>[12]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>[15]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>[16]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>[20]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>[21]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>[22]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>[23]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>[24]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>[26]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>[29]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>[30]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>[31]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>[35]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>[9]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>[13]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>[17]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>[19]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>[25]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>[32]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>[16]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>[26]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>[27]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>[34]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>[8]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>[28]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>[37]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>[30]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>[33]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>[36]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>[14]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>[18]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>[10]</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr></tbody></table>
Table 1 summarizes data on the gamification elements applied in the software and the User Interface design, that is, what visual representation methods were used to deliver the interface. Also, it clusters the primary studies according to the learning objective(s) they aim to achieve. “C” stands for Communication; “M” is Multiple, meaning that the study intended to cover more than one learning objective at the same time; “SI” means Social Interaction; “BE” means Basic Education; “DA” stands for Daily Activities; “FR” means Facial Recognition; and “MS” is an abbreviation for Motor Skills.

## 3.1 Gamification Elements

According to Table 1, many of the studies use feedback (50%) and/or rewarding systems (33%) to appeal to the audience. Those can be either virtual output stimuli, such as visual cues or auditory feedback, or real-life tokens (a fruit, a game, or anything that the user would associate with having positively accomplished a task). Rewards have been largely used in ASD-specific interventions, often applied immediately after the task completion, in order to reinforce certain behavioral patterns [36].

Four studies applied Virtual Reality environments within the software [9, 10, 11, 12]. There has been increasing interest in using VR for autistic intervention, since this technology provides an immersion environment that improves interventionist control and promotes engagement [10]. In addition to that, many studies (60%) increased the level of difficulty as the user progresses. This feature includes either mid-term or long-term goals, and is used to foster motivation and interest in using the software.

Most of the studies also include some degree of personalization to the software they propose. This allows the child (or teachers, or parents) to customize the graphics and/or the content according to their particular needs and preferences. On the top level, it may allow for changes in the color scheme, graphics size, and layout arrangement. Besides that, it may include changes in content presentation: photographs instead of cartoon-like illustrations, text messages or picture-only content, background music on and off, etc. A few studies combine personalization with a multimedia approach, including text, audio and video options [1, 3, 11, 13, 14, 37]. These features are particularly interesting if we take into account that each ASD individual has unique interests and preferences, and that the autism covers a broad range of symptoms [8].

In terms of monitoring, many studies apply strategies to customize learning outcomes and provide parental control (56.6% combined). This is often reached by synchronizing child-parent / child-teacher devices or having multiple interface profiles. These studies report on the use of an integrated system that allows teachers, therapists or parents to monitor their children progress, and select custom learning objectives depending on individual cognitive responses. Again, ASD individuals can benefit from having tasks tailored specifically for each of them and, at the same time, teachers and parents are entitled with more control over the learning outcomes.

A few studies also reported on the use of storytelling [15, 17, 26, 29, 33]. This method uses multiple media formats (text, images, sounds, animation, and video) in order to encourage students to create their own narratives. In ASD children, this may
help improve their communication skills and the symbolic function, thus promoting
an autonomous attitude [27]. Other features found on the studies include familiariza-
tion (pictures of the surroundings and/or voice recording) [18, 34], information shar-
ing [16], use of virtual avatars [13], active experimentation [27], and remote control
[29].

By clustering the information in relation to the game objective, a few more insights emerge. When dealing with communication, there is a preference to work with personalization, monitoring, custom learning, and increasing level of difficulty. This may be explained by the communication process itself, which requires a certain vocabulary level in order to make sentences, and so on. The same applies to Social Interaction, since it involves cognitive skills rather than physical abilities. In fact, objectives that cope with physical development, such as Daily Activities and Motor Skills, tend to apply feedback, rewards, and also personalization strategies, which provides a more immediate response to the user. In turn, games that included multiple objectives often used a wide variety of strategies (although rewards and level of difficulty are more frequent), perhaps because a single approach would not be sufficient to comply with all the objectives at once.

## 3.2 User Interface Design

The User Interface design methods reported by the studies include visual presentation (colors, illustration, photographs, video), textual elements (words or phrases) and even sounds, voice/narration, and virtual 3D environments. The most popular approaches, however, are the use of illustrations (76.6%), words/text (80%), and colors (73.3%) to express feelings and guide the user through the interface. The items check-marked (refer to Table 1) represent the topics described in more detail in the papers, which does not mean unmarked items were not covered by the study.

In relation to illustrations, there are multiple approaches available. Reports range from the use of simple, sketchy, basic shapes (squares, triangles, and circles) [9, 16, 17, 22, 37] to more sophisticated illustrations [25, 26, 31]. None of them report on the use of realistic drawings. Instead, whenever the authors wanted a more realistic style, they would rely on real-life pictures or photographs. The main explanation provided in those cases is that real-life pictures would be easier to recognize, especially if the parents/teachers use the built-in camera to upload their own images. It would also look more familiar to the user, and may, in some cases, be more effective [38].

The use of colors appeared as one of the major concerns in the development pro-
cess. Again, there are various approaches. Some of the studies inform on the use of
more vibrant colors [16, 22, 25, 28], while others suggest a calm and half-toned color
scheme [23, 36]. There is no apparent revaluation of colors based on users' feedback.
A similar situation arises from the use of words/text: there is not much information on
why authors chose one strategy over another. The differences in communication defi-
cits experienced by ASD individuals impose a challenge because some of them are
able to read while others are not [1]. Thus, using an amount of written information
that fits the communication skills of the target audience is of major importance to
achieve success.
The use of sounds is also reported in many of the studies (56.6%), though voice recording features or narration are much scarcer (10%). Sounds, in this case, include background music and/or other sound effects (clicking a button, for instance). Sounds can be interesting decorative strategies to increase motivation and reinforce learning activities [39]. A few studies report on the use of voice recording or narration in order to add customization attributes or improve familiarization [14, 24, 30]. Around 30% of the studies also include video (self-recorded or not) and virtual environments as means to promote immersion experiences.

Once again, clustering the information according to the objectives leads to other insights. Games with communication objectives largely apply illustration, photographs, and words/text. Surprisingly, however, not many of the studies use sounds, voice recording or video modeling, which could potentially help users become familiar with the sounds of the words and practice pronunciation. The same applies to Social Interaction and Basic Education – they could greatly benefit from having sound reinforcing positive behavior. In turn, video modeling and Virtual Reality appears in all of the studies handling Facial Recognition and Motor Skills, probably because those strategies offer opportunities to simulate real situations – which could also be applied to Daily Activities.

## 4 Conclusions and Future Perspectives

The objective of this review was to investigate the current status of research on ASD-specific gamified interventions, emphasizing gamification elements and design representation methods applied in the software development process. Data were also collected on the target audience, the learning objectives, and the software development process. As results show, there are a good number of studies addressing the topic, which suggests more awareness of the disturbance and a willingness to move toward accessibility.

Clinically, the causes of autism are unclear, and as medicine advances more complexity is added to its diagnosis, treatment, and prognostic. On the one hand, parents find it difficult to handle autism inside their homes, because many times they have little or no assistance on how to behave with their autistic children. On the other, accessibility is often seen as unnecessary and expensive in the corporate environment. That is precisely why it became so important: development teams endure the task to develop effective solutions for autism and, at the same time, prove they are feasible from a productive, financial, and corporate perspective.

In this context, this review shows a wide collection of gamification strategies to cope with the development of autism-specific software. Despite that, many challenges emerge from dealing with the ASD audience. Studies included in this review share some common characteristics. Overall, they provide information on the development process, although the procedures are often not described in full detail (with only a few exceptions). We believe this does not mean the development process was neglected, but rather that the authors decided to highlight some of the aspects they assumed to be the most relevant. However, the lack of information is disappointing, since a more
detailed description would add consistency to the studies, and also help readers under-
stand and replicate the methods and strategies used.

The target audience is, most of the cases, referred to simply as "autistic children"
or "autistic individuals". As mentioned before, autism covers a broad spectrum, that
is, different symptoms appear in different severity levels in ASD individuals [1]. For
this reason, it is hard to assume a single software product would be equally effective
to the entire spectrum. Having a specific target audience (including the comprehen-
sive description of symptoms) would potentially help optimize the development pro-
cess and also serve as guidance for therapists in managing the software. Furthermore,
this has a negative impact on testing, since results may reflect mixed ASD profiles.

In relation to the development process / design approach, there is a clear focus on applying methodologies that prioritize the end-user. On the one hand, cross-functional teams often present a better performance, since their particular expertise helps improving the software product in terms of usability. On the other, involving external agents (teachers, parents, and the ASD users) is fundamental to collect feedback and confirm its efficiency on a User Experience level. Even though most studies did not present the design process comprehensively, they surely reveal an effort to meet the end-user's needs and preferences.

Despite the issues with the process detailing, all of the studies show a good use of multiple techniques (gamification elements and UI design strategies). The combination of such methods for the development of serious gaming applications often results in better usability and acceptance levels [30]. We should remember, once more, that ASD individuals have complex clinical conditions. Thus, combining multiple strategies and forms of presentation may have a huge impact on how they use and benefit from the software.

Unfortunately, most of the studies analyzed did not provide consistent testing pro-
cedures or clear results achieved with the games. Many studies used indirect forms of
assessment, which is valuable, but hardly represents the end-user realistically. In cas-
es when authors did test the software with autistic children [7, 10, 11, 22, 23, 25, 26, 27, 28], samples were often small, revealing fragility in the results achieved. In addi-
tion to that, none of the studies reported having used control groups within the testing
procedures. This leaves us with no reference for comparison on how the audience
would have behaved with and without the serious game. Of course, the testing efforts
are important but, most of the cases, it is hard to assume they are unquestionable evi-
dence of success.

Results, in turn, are generally presented in a superficial manner. Mostly, they are
described as “good/not good” or “positive/negative”, without providing further info-
mation on the analysis criteria, reference metrics, or a comprehensive review process.
Only a small number of studies inform on the specific positive/negative feedback
received, as well as how they would apply the suggestions on later improvements [9,
13, 16, 17, 26, 32]. Exclusively negative or unsuccessful outcomes have not been
found.

The path towards accessibility for autism is obviously not straightforward. Since
the clinical literature is still limited, practitioners are also learning from empirical
evidence. In this sense, one of the main challenges in developing serious games for
autism is to reach out the end-user, in order to understand the complexity of condi-
tions they have. In addition to that, the development process must be scrutinized. As
showed in this systematic review, there are issues related to the selection of the target
audience (which may lead to a poor definition of learning objectives), and the conduc-
tion of testing procedures (which may lead to biased or mixed results). Finally, devel-
opers must be aware of the user-centered approaches – involving autism advocates
along the process may be a good strategy to prevent usability disasters.

The studies included in this review are undeniably valuable, and we believe the au-
thors did their best with the limitations they might have faced and with the resources
they had available at the time. Results achieved and presented in the studies reinforce
that. Notwithstanding, scientific and methodological procedures must be taken seri-
ously given the complexity of the subject matter. This review also intends to open
future discussions on scientific rigor and integrity involving the development of ASD-
specific software. Efforts must continue toward accessibility, effective learning, and
social inclusion.

References

1. American Psychiatric Association (2013). *Diagnostic and Statistical Manual of Mental Disorders, DSM-5*. Artmed, 5th edition.

2. Venkatesh, S., Greenhill, S., Phung, D., and Adams, B. (2011). Cognitive intervention in autism using multimedia stimulus. *ACM Multimedia 2011*.

3. United Nations (2013). *Accessibility and development: mainstreaming disability in the post-2015 development agenda*. Department of Economics and Social Affairs Division for Inclusive Social Development.

4. Chandran, S., Prakrithi, S. N., and Kishor, M. (2018). Gamifying education and mental health. *Archives of Medicine and Health Sciences*, 6(2): 284-289.

5. Whyte, E. M., Smyth, J. M., and Scherf, K. S. (2015). Designing serious game interventions for individuals with autism. *Journal of Autism and Developmental Disorders*. 45(1): 3820-3831. DOI: 10.1007/s10803-014-2333-1

6. Lau, H. M., Smit, J. H., Fleming, T. M., and Riper, H. (2017). Serious games for mental health: are they accessible, feasible, and effective? A systematic review and meta-analysis. *Frontiers in Psychiatry*, 7(1): 1-13. DOI: 10.3389/fpsyt.2016.00209

7. Kitchenham, B. (2004). Procedures for performing systematic reviews. *Kelee, UK, Kelee University* 33(2004):1-26.

8. Kamaruzaman, M. F., Ranic, N. M., Norb, H. M., and Azaharia, M. H. H. (2016). Devel-
oping user interface design application for children with autism. *Social and Behavioral
Sciences*, 217(1):887-894. DOI: 10.1016/j.sbspro.2016.02.022

9. Cai, Y., Chia, N. K. H., Thalmann, D., Kee, N. K. N., Zheng, J., and Thalmann, N. M. (2013). Design and development of a virtual dolphinarium for children with autism. *Transactions on neural systems and rehabilitation engineering*, 21(2):208-217. DOI: 10.1109/TNSRE.2013.2240700

10. Finkelstein, S., Nickel, A., Barnes, T., and Suma, E. (2010). Astrojumper: Motivating children with autism to exercise using a VR game. *28th ACM Conference on Human Factors in Computing Systems*, pages 4189–4194.

11. Winoto, P., Xu, C. N., and Zhu, A. A. (2016). Look to remove: A virtual reality applica-
tion on word learning for Chinese children with autism. *10th International Conference on*
*Universal Access in Human-Computer Interaction*, pages 257–264. DOI: 10.1007/978-3-319-40238-3_25

12. Silva, C. A., Fernandes, A. R., and Grohmann, A. P. (2014). Star: Speech therapy with augmented reality for children with autism spectrum disorders. *16th International Conference on Enterprise Information Systems*, pages 379–396. DOI: 10.1007/978-3-319-22348-3_21

13. Konstantinidis, E. I., Luneski, A., Nikolaidou, M. M., Hitoglou-Antoniadou, M., and Bamidis, P. D. (2009). Using affective avatars and rich multimedia content for education of children with autism. *Pervasive Technologies Related to Assistive Environments (PETRA) 2009*.

14. Tan, C. T., Harrold, N., and Rosser, D. (2013). Can you CopyMe?: an expression mimicking serious game. *40th International Conference on Computer Graphics and Interactive Techniques SIGGRAPH 2013*.

15. Wadhwa, B. and Jianxiong, C. (2013). Collaborative tablet applications to enhance language skills of children with autism spectrum disorder. *11th Asia Pacific Conference on Computer Human-Interaction - APCHI 2013*, pages 39–44. DOI: http://dx.doi.org/10.1145/2525194.2525297

16. Zhang, L., Fu, Q., Swanson, A., Weitlauf, A., Warren, Z., and Sarkar, N. (2018). Design and evaluation of a collaborative virtual environment (CoMove) for autism spectrum disorder intervention. *Transaction on Accessible Computing*, 11(2). DOI: https://doi.org/10.1145/3209687

17. Hourcade, J. P., Bullock-Rest, N. E., and Hansen, T. E. (2012). Multitouch tablet applications and activities to enhance the social skills of children with autism spectrum disorders. *Personal and Ubiquitous Computing*, 16(1):157–168. DOI: 10.1007/s00779-011-0383-3

18. Washington, P., Voss, C., Kline, A., Haber, N., Daniels, J., Fazel, A., De, T., Feinstein, C., Winograd, T., and Wall, D. (2017). SuperpowerGlass: A wearable aid for the at-home therapy of children with autism. *ACM on Interactive, Mobile, Wearable and Ubiquitous Technology*, 1(3). DOI: 10.1145/3130977

19. Venkatesh, S., Phung, D., Duong, T., Greenhill, S., and Adams, B. (2013). TOBY: Early intervention in autism through technology. *Conference on Human Factors and Computing Systems - CHI 2013*, pages 3187–3196.

20. Fotjik, R. (2010). Agile methodology and development of software for users with specific disorders. *International Multiconference on Computer Science and Information Technology*, pages 687–691.

21. Shminan, A. S., Adzani, R. A., Sharif, S., and Lee, N. K. (2017). AutiPECS: Mobile based learning of picture exchange communication intervention for caregivers of autistic children. *International Conference on Computer and Drone Applications*, pages 49–54.

22. Alvarado, C., Munoz, R., Villarroel, R., Acuña, O., Barcelos, T. S., and Becerra, C. (2017). Valpodijo: Developing a software that supports the teaching of Chilean idioms to children with autism spectrum disorders. *12th Latin American Conference on Learning Technologies (LACLO) 2017*.

23. De Leo, G., Gonzales, C. H., Battagiri, P., and Leroy, G. (2011). A smart-phone application and a companion website for the improvement of the communication skills of children with autism: Clinical rationale, technical development and preliminary results. *Journal of Medical Systems*, 35(1):703–711. DOI: 10.1007/s10916-009-9407-1

24. Winoto, P., Cao, V. L., and Tang, E. M. (2017). A highly customizable parent-child word-learning mobile game for Chinese children with autism. *11th International Conference on Universal Access in Human-Computer Interaction*, pages 545–554. DOI: 10.1007/978-3-319-58706-6_44
25. Yan, F. (2011). A sunny day: Ann and Ron's world an iPad application for children with autism. *International Conference on Serious Games Development and Applications*, pages 129–138.

26. Wojciechowski, A. and Al-Musawi, R. (2017). Assistive technology application for enhancing social and language skills of young children with autism. *Multimedia Tools and Applications*, 76(1):5419–5439. DOI: 10.1007/s11042-016-3995-9

27. Chatzara, K., Karagiannidis, C., Mavropoulou, S., and Stamatis, D. (2014). Digital storytelling for children with autism: Software development and pilot application. *Research on e-Learning and ICT in Education: Technological, Pedagogical and Instructional Perspectives*, pages 287–300. DOI: 10.1007/978-1-4614-6501-0_19

28. Iradah, I. S. and Rabiah, A. K. (2011). EduTism: An assistive educational system for the treatment of autism children with intelligent approach. *International Visual Informatics Conference*, pages 193–204.

29. Zaffke, A., Jain, N., Johnson, N., Alam, M. A. U., Magiera, M., and Ahamed, S. I. (2014). iCanLearn: A mobile application for creating flashcards and social stories for children with autism. *International Conference on Smart Homes and Health Telematics*, pages 225–230. DOI: 10.1007/978-3-319-14424-5_25

30. Hayes, G. R., Hirano, S., Marcu, G., Monibi, M., Nguyen, D. H., and Yeganyan, M. (2010). Interactive visual supports for children with autism. *Personal and Ubiquitous Computing*, 14(1):663–680. DOI: 10.1007/s00779-010-0294-8

31. Signore, A., Balasi, P., and Yuan, T. (2014). You talk! – you vs. autism. *16th International Conference on Computers Helping People with Special Needs ICCHP 2014*, pages 506–512.

32. Malinverni, L., Mora-Guiard, J., Padillo, V., Valero, L., Hervás, A., and Pares, N. (2017). An inclusive design approach for developing video games for children with autism spectrum disorder. *Computers in Human Behavior*, 71(1):535–549. DOI: http://dx.doi.org/10.1016/j.chb.2016.01.018

33. Kurniawan, R., Purnamasari, W. M., Rakhmawati, R., and Jalaputra, D. P. E. (2018). Development of game for self-help toilet learning for children with autism. *CommIT (Communication and Information Technology) Journal*, 12(1):1–12.

34. Tsiopela, D. and Jimoyiannis, A. (2016). Pre-vocational skills laboratory: designing interventions to improve employment skills for students with autism spectrum disorders. *Universal Accessibility in the Information Society*, 16(1):609–627. DOI: 10.1007/s10209-016-0488-6

35. Cabelles-Hernández, D., Pérez-Pérez, J.-R., Paule-Ruiz, M. P., and Fernández-Fernández, S. (2016). Specialized intervention using tablet devices for communication deficits in children with autism spectrum disorders. *Transactions on Learning Technologies*, 10(2):182–193. DOI: 10.1109/TLT.2016.2559482

36. Barros, V. T. O., de Almeida Zerbetto, C. A., Meserlian, K. T., Barros, R., Camargo, M. C., and de Carvalho, T. C. P. (2014). Interactive and customizable use of mobile technology in the cognitive development process of children with autistic spectrum disorder. *8th International Conference on Universal Access in Human-Computer Interaction*, 1(1):443–453. DOI: https://doi.org/10.1007/978-3-319-07440-5_41

37. Hulusic, V. and Pistoljevic, N. (2012). LEFCA: Learning framework for children with autism. *Virtual Worlds for Serious Applications (VS-GAMES '12)*, 15(1):4–16. DOI: 10.1016/j.procs.2012.10.052

38. Harrold, N., Tan, C. T., and Rosser, D. (2012). Towards and expression recognition game to assist the emotional development of children with autism spectrum disorders. *7th Inter-*
national Conference on Wireless Algorithms, Systems and Applications, WASA 2012, pages 33–37.

39. Banire, B., Jomhari, N., and Ahmad, R. (2015). Visual hybrid development learning system (VHDLS) framework for children with autism. *Journal of Autism and Developmental Disorders, 45*(1):3069–3084. DOI: 10.1007/s10803-015-2469-7