# Code Your Own Game: The Case of Children with Hearing Impairments

Michail N. Giannakos, Letizia Jaccheri

► To cite this version:

Michail N. Giannakos, Letizia Jaccheri. Code Your Own Game: The Case of Children with Hearing Impairments. 13th International Conference Entertainment Computing (ICEC), Oct 2014, Sydney, Australia. pp.108-116, 10.1007/978-3-662-45212-7_14 . hal-01408511

HAL Id: hal-01408511

https://inria.hal.science/hal-01408511

Submitted on 5 Dec 2016

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# Code your own Game: the Case of Children with Hearing Impairments

Michail N. Giannakos and Letizia Jaccheri

Department of Computer and Information Science, Norwegian University of Science and Technology (NTNU), Trondheim, Norway
mgiannakos@acm.org; letizia@idi.ntnu.no

**Abstract.** It is well known in the computer science community that is important to encourage children to acquire coding skills and become creators of their own experiences and not only mere game consumers. Different children have different needs when approaching coding and making activities. Specifically, Deaf and Hard of Hearing (DHH) children, even when provided with accessible visual translations through sign language interpreters or real-time captions, need customized support. In our approach we have designed, implemented, and evaluated a workshop program of 12 children total, with the final goal of exploring and improving the design of appropriate workshops using the current learning environments. This paper presents an initial exploratory evaluation of a coding experience for children with hearing impairments and the development of a set of guidelines for improving the teaching of coding to children with DHH difficulties. An initial set of best practices was first developed through a focus group with experts; and afterwards, by employing content analysis, a revised set of guidelines was obtained. The results should be useful for special education teachers, curriculum designers and developers for K-12 education environments for DHH.

**Keywords:** Accessibility, Coding, Design Guidelines, Deaf, Hearing Impairments, Empirical Evaluation, Focus Groups, Games, Programming, Workshops.

## 1 Introduction

Currently, several efforts to broaden participation in programming and introduce computational literacy to young students [2], [9] are in progress. Children interact with visual programming tools like Scratch [14] to learn how to code by creating interactive stories, games, animations, and simulations. Sesame workshop [15] has given new insights into how programming for children needs to be approached; in order to be both educational and entertaining. The process for achieving this mix relies on a development model that integrates expertise in media production, educational content (or curriculum), and research with children. Sesame Workshop philosophy [15] identify some of the challenges and solutions in designing interactive educational activities that can be used by children. Buechley et al. [2] argue that there is a need to make children programming a far more informal, approachable, and natural activity.
Although, programming activities for children have drawn great interest in the last years, little information is available on how to introduce computational literacy to young students taking into account children with special needs and impairments. Children with disabilities face certain difficulties with the current approaches and methods to learn programming [3], [4], [10]. In particular, children with Deaf and Hard of Hearing (DHH) encounter challenges when learning how to code. Teachers and curriculum designers need to be aware and pay particular attention in these challenges.

In this paper, we present our experience from a game coding workshop focusing on children with DHH. With the knowledge extracted from this experience we aim to explore how design and technology can contribute to improve current learning practices for the benefit of children with DHH. This paper focuses on our efforts to develop a coding workshop that will allow DHH children to overcome their difficulties and explore their potential interest in game development and coding. Hence, we provide some first insights on: *How to design environments for facilitating coding for children with hearing impairments?*

In our efforts to investigate how game coding workshops could be designed to allow DHH children to overcome their difficulties, we designed, implemented, and evaluated a workshop program of 12 children with DHH. After the workshop, we organized a focus group with experts in DHH in order to capture their ideas and experiences with regard to the game coding workshop. Next, we employed a content analysis technique [11] in order to organize the data. As the final step of the process we used the structured data and derived guidelines for improving the design of the game coding workshop that address DHH difficulties.

## 2 Background

DHH is an impairment that can result from many reasons at different ages. DHH Children are a challenging target group for designers [8] [10]. Not only because it is harder to design environments for children rather than adults [9], but the fact that these children have DHH creates even more designing particularities. Most children today have hearing aid or a cochlear implant, but they do have special communication needs. In fact they can communicate orally, but only to a certain extent [12]. Often they miss a fluent mother tongue, which results in a lack of written and spoken language skills [12]. The primary form of communication within the deaf children is the sign language [1]. Sign language is not a visual form of the respective language (in our case Norwegian) but it is a different language with its own unique grammatical and syntactical structure.

Therefore, the lack of written and spoken language skills, which is common in children with DHH, has an impact on how they can be involved in different learning contexts [10] and therefore how these contexts can be designed.

This design of learning (in our case programming) environments to support children with special needs (in our case DHH) comes with additional challenges. For example, there are diverse stakeholders (e.g., special educators, designers, DHH ex-
perts) that need to be involved in the design of these environments and technologies, and from the children themselves and their families. Likewise, educators, researchers and practitioners with a variety of expertise need to work together to develop practical solutions with a chance of succeeding in realistic contexts.

## 3 Game Coding Experience

Twelve 12-year old children with DHH from the Deaf school of Trondheim in Norway participated in the game coding workshop. The workshop took part in the Norwegian Deaf Museum (see figure 1, for the context of the workshop). The schedule, the infrastructures and the main goal of the workshop were based from knowledge we obtained from prior similar experience [5] [6].

<figure><img src="image_2.png" /><figcaption>Fig. 1. The context of the game coding workshop</figcaption></figure>

In particular, the children attended the workshops were instructed and assisted by a programming artist with an interpreter and during the workshop children worked with the Scratch programming environment. In order to better organize the workshop we closely collaborate with six experts (see table 1), which, after observing the workshop, responded to a survey and formed up a focus group. In selecting the experts, we focused on people's professional expertise in the domains of DHH, children and education.

<table><caption>Table 1. Participants and their expertise</caption><thead><tr><th colspan="2">Participants and their expertise</th></tr></thead><tbody><tr><td>Norwegian Deaf Museum Curator</td><td>Norwegian Deaf Museum Director of Education</td></tr><tr><td>Trondheim Deaf School teacher</td><td>Trondheim Deaf School teacher</td></tr><tr><td>Artist-Instructor Programmer</td><td>HCI researcher</td></tr></tbody></table>
Throughout the workshop, children worked in dyads and developed in total six in-
teractive projects. Children worked collaboratively with the assistance of the visual
programming language (see figure 2); record of children’s activities was kept through
photographs, videos, observation-reports and surveys from the experts; this infor-
mation was used to evaluate the workshop and as an input experience for the next
phase of the study.

<figure><img src="image_3.png" /><figcaption>**Fig. 2.** Children worked collaboratively and communicate with the assistance of the visual programming language</figcaption></figure>

One of the goals of our study is to perform an exploratory evaluation of the game
coding workshop and justify the special attention needed for children with DHH diffi-
culties. As such, we used a quantitative survey-based approach. The survey was hand-
ed out to the six experts after the workshop and included the measures (factors) of
children's: a) Enjoyment, b) Control and c) Easiness with respect to the programming
workshop. In particular, the six experts were asked to rate, on a 7-point scale survey:
a) how children seemed to enjoy the workshop, b) how much control of the workshop
children had, and c) how easy was the workshop for children. Each one of the three
factors was measured based on the literature with 2 or 3 questions (see Figure 3).

- The child seemed to enjoy programming during the workshop (Enj1)

- The child seemed interested in actively exploring programming in the workshop (Enj2)

- The child seemed entertained by the workshop in general (Enj3)

- The child was able to follow the tasks of the workshop (Cont1)

- The child has the skills and the ability to follow the tasks of the workshop (Cont2)

- The workshop was easy for the child (Easy1)

- The workshop was flexible for the child (Easy2)

- The concept of the workshop was clear and understandable from the child (Easy3)
<figure><img src="image_4.png" /><figcaption>Fig. 3. Experts’ responses to 7-point Likert-scale</figcaption></figure>

With the quantitative approach we attempt to investigate DHH children difficulties
during a game coding workshop. Based on experts’ observations and survey respons-
es (figure 3) we can agree that children enjoyed the workshop, however, their control
towards the workshop was low. In addition, experts indicated that children found
many concepts of the workshop hard to follow and unclear. Although this game cod-
ing workshop has been validated and improved through many user studies (e.g. [5]
[6]), there is a lot of work need to be done in order to address DHH needs and justify
them through design patterns into the current programming practices and environ-
ments.

The findings from the exploratory evaluation clearly demonstrate the need for im-
proving DHH children programming experiences. To do so we need to design more
accessible and closer to their needs environments by addressing a variety of visual
child-programming environment interactions (e.g. Figure 4). Taking this into account,
in the next section we analyze qualitative data in order to give some first insights.

<figure><img src="image_5.png" /><figcaption>Fig. 4. Child-Programming Environment Interaction with the assistance of the camera</figcaption></figure>
## 4 Extracted Guidelines via Focus Group and Content Analysis Processes

Building on the experience from the game coding workshop, a focus group consisted of the six experts (see table 1 above) brainstormed and organized ideas and best practices for improving programming activities design. Using a focus group enables a wide variety of collective views and often leads to results based on a consensus among participants [11].

Content analysis is a technique used to categorize data (e.g., interviews, ideas) through a protocol. Content analysis enables the researchers to sift through large volumes of data and systematically identify properties, attributes and patterns embedded. The technique is considered useful for identifying and analyzing issues in gathered data [11]. In order to investigate how the experts' advices and best practices could be specifically relevant for children, the content analysis connected those, with three central "design" components for children [7]: *approach*, *settings* and *means*.

In particular, based on [7] 1) approach category was defined as all the attitudes and acts that professionals should aim to perform in order to contribute to a successful workshop; 2) settings category includes all the preconditions that help children enroll with the workshop more willingly and 3) means category includes the best practices referring to the concrete needs methods and materials.

The best practices and ideas of the experts could be relevant for several purposes, as such it was considered useful to sort them in a more generic way under the aforementioned three categories (see table 2).

<table><caption>**Table 2.** The extracted guidelines under the three main categories and the justification-example</caption><thead><tr><th>Categories</th><th>Guidelines</th><th>Justification-Example</th></tr></thead><tbody><tr><td rowspan="3">Approach</td><td>Follow children learning</td><td>Instructor should wait the interpreter to finish, give enough time to children to read information and repeat when children do not follow the instructions (as many times as needed).</td></tr><tr><td>Use personal approach</td><td>The instructor should treat every child uniquely, and consider his/her individual difficulties. The instructor should keep eye contact with all the children to ensure that they are following him.</td></tr><tr><td>Provide practical information</td><td>Reduce the amount of the provided information, by focusing to the practical information. Support the recall of communication patterns rather than building new ones and provide few but distinct choices to the child.</td></tr><tr><td rowspan="2">Settings</td><td>Different sessions with clear goals</td><td>The workshop should be well-structured with different and clear (IT-programming) competencies on each session and many breaks between the sessions.</td></tr><tr><td>Very well prepared</td><td>Interpreters and instructors need to work together in advance, in order to reduce potential difficulties in</td></tr></tbody></table>
<table><tr><td></td><td>interpreters</td><td>communication and the vocabulary (sign language vocabulary is limited). Interpreters need to have some knowledge in the field (e.g., programming).</td></tr><tr><td rowspan="2">Means</td><td>Many, clear and big visual aids</td><td>Big screens, projectors and other visual aids are essential on assisting children communication (e.g., on figure 1 the big screen behind the instructor). Text should have clear large headings, and different notions should be distinguished with different colors and shapes.</td></tr><tr><td>Support children-computer interaction via visual tools</td><td>Children-computer (program) interaction should be supported with various visual means. For instance children-program interaction through the web-camera (figure 4) motivates them to optimize their code.</td></tr></table>

## 5 Conclusions And The Way Ahead

In this paper we presented the results from the design, deployment and evaluation of a game coding workshop for children with DHH. Our results provide an initial attempt to exploit knowledge from experts in DHH and model this knowledge into useful guidelines for designers and developers who aim to address children with DHH as participants of programming learning workshop. Our research is characterized by a close collaboration between special educators, HCI researchers, and DHH experts. The study described in this paper has led to a set of guidelines for designing programming learning activities for children with DHH. The guidelines were backed by addressed experts' ideas and best practices and has been exposed to several stages of validation and organization (focus group, content analysis), which should provide some assurance of their validity. Based on this, seven design guidelines under three main categories have been proposed.

We want to emphasize that our findings are clearly preliminary with inevitably limitations. One important limitation is the absence of children's voices in this work. However, capturing, crossing and analyzing the experiences of the six experts allow us to portray design issues derived from hundreds of workshop sessions and teaching hours. Our future research will concentrate on further refinement of the proposed guidelines by applying and evaluating them on real conditions. Furthermore, educators, practitioners and researchers in the areas of 1) technology-enhanced STEM learning and 2) children with DHH should evaluate the proposed guidelines in order to ensure their understanding and seek suggestions and extensions. In the next step of this ongoing project we will continue our research with evaluating these guidelines with a mixed methods approach, and aim to improve and optimize them.
**Acknowledgements.** The authors would like to express their gratitude to all of the students for volunteering their time. Our very special thanks go to A. Eriksen, H. Mellemsether and L. Nordsveen.

## References

1. Borgna, G., et al.: Enhancing deaf students' learning from sign language and text: Metacognition, modality, and the effectiveness of content scaffolding. Journal of deaf studies and deaf education, 16(1), 79-100 (2011)

2. Buechley, L., Eisenberg, M., Catchen, J., and Crockett. A.: The LilyPad Arduino: using computational textiles to investigate engagement, aesthetics, and diversity in computer science education. In: CHI 2008, ACM, 423-432 (2008)

3. Deibel, K.: Studying our inclusive practices: course experiences of students with disabilities. In: ITiCSE 2007, ACM Press, 266-270 (2007)

4. Gellenbeck, E.: Integrating accessibility into the computer science curriculum. Journal of Computing Sciences in Colleges, 21(1), 267-273 (2005)

5. Giannakos, M. N., Jaccheri, L.: An Enriched Artifacts Activity for Supporting Creative Learning: Perspectives for Children with Impairments. In: Entertainment Computing-ICEC 2013, Springer LNCS 8215, 160-163 (2013)

6. Giannakos, M.N., and Jaccheri, L.: Designing creative activities for children: the importance of collaboration and the threat of losing control. In: IDC 2013, ACM, 336-339 (2013)

7. Høiseth, M., Giannakos, M.N., and Jaccheri, L.: Research-derived guidelines for designing toddlers' healthcare games. In: EA CHI 2013, ACM, 451-456 (2008)

8. Brashear, H., Henderson, V., Park, K., Hamilton, H., Lee, S., Starner, T.: American sign language recognition in game development for deaf children. In: ACM SIGACSEE 2006. ACM Press, 79-86 (2006).

9. Jacobs, J., and Buechley, L.: Codeable objects: computational design and digital fabrication for novice programmers. In: CHI 2013, ACM Press, 1589-1598 (2013)

10. Liffick, B.W.: An adaptive technologies course in a CS curriculum. In: ACM SIGACCESS 2005, ACM Press, 192-193 (2005)

11. Maguire, M., Bevan, N.: User requirements analysis: A review of supporting methods. In: IFIP WCC 2002, 133-148 (2002)

12. Moeller, M.P.: Early intervention and language development in children who are deaf and hard of hearing. Pediatrics 106 (3) (2000)

13. Nakatsu, R., Rauterberg, M., & Vorderer, P.: A new framework for entertainment computing: From passive to active experience. In Entertainment computing—ICEC 2005, Springer LNCS 3711, 1–12 (2005)

14. Resnick, M., Maloney, J., et al.: Scratch: programming for all. Communications of the ACM, 52 (11), 60-67 (2009)

15. Revelle. G. L.: Educating via entertainment media: the Sesame Workshop approach. Comput. Entertain. ACM Press, 1, 1, Art. 7. (2003)