# A Tale of Two Puppets, Two Avatars and Two Countries

Yoram I. Chisik, Monchu Chen, M. Corriea Martins

► To cite this version:

Yoram I. Chisik, Monchu Chen, M. Corriea Martins. A Tale of Two Puppets, Two Avatars and Two Countries. 14th International Conference on Human-Computer Interaction (INTERACT), Sep 2013, Cape Town, South Africa. pp.658-665, 10.1007/978-3-642-40477-1_44. http-01504919

HAL Id: hal-01504919

https://inria.hal.science/hal-01504919

Submitted on 10 Apr 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# A Tale of Two Puppets, Two Avatars and Two Countries

Yoram I. Chisik, Monchu Chen and M. Clara Corriea Martins

University of Madeira, Caminho da Penteada, 9020-105 Funchal, Portugal

ychisik@uma.pt, monchu@uma.pt, mccmzz@gmail.com

**Abstract.** We describe observations from studies conducted with school children in Nepal and Portugal with the aim of exploring two questions: 1. Can we develop an interactive game that will continue to engage the child in gameplay when the electricity supply fails? 2. What are the discernable differences in the ways children in Nepal and Portugal interact with the game given the huge socio-economic and cultural differences between them? The studies highlight the potential of the design in fostering continued engagement in the game when the electricity supply fails. They also show surprising little difference between the children in Nepal and Portugal. The observations suggest game mechanism and basic interactions can be regarded as universal across cultures and levels of income and can thus be designed and tested in any location and be expected to work and be understood by children from any culture and background.

**Keywords:** Games, Play, Children, Electricity, Design, Interaction, HCI4D

## 1. Introduction

Designing for children is notoriously hard as adults are indelibly separated from children by a generation gap and thus cannot possibly imagine the world from their perspective [6,7]. Designing for children in developing countries is doubly hard, as designers have to bridge a set of environmental, cultural and socio-economic perceptual gaps in addition to the aforementioned generation gap [8,4]. To confound matters further the physical distance and other logistical difficulties make it hard to employ the participatory and emphatic design practices that enable designers to bridge generational and environmental gaps in their perception [5,9].

The work presented in this paper was born out of an invitation to organize a children entertainment workshop in Nepal, a country with severe infrastructure limitations where even in the capital city electricity is only available for a few hours a day.

Our goal in this study was two fold. First we wanted to see if we could transcend Nepal resource limitation by developing a game that would be equally engaging with or without electrical power and will thus always have the power to engage and entertain its players. Second, we wanted to see whether the socioeconomic and technology use differences between Nepal and Portugal (Our home country) has any impact on
the ways in which children in these countries engaged with the computer and played
the game as suggested by recent studies [14,13].

## 2. The Game

For the purpose of the study we developed a simple game consisting of two screen
emojis, a singer and a conductor (Fig. 1) that are controlled by two sock puppets op-
erated by two players (Fig. 2), and governed by two self-discoverable rules.

<figure><img src="image_2.png" /><figcaption>Fig. 1. The avatars.</figcaption></figure>

<figure><img src="image_3.png" /><figcaption>Fig. 2. The sock puppets.</figcaption></figure>

We based our design on the natural curiosity and inventiveness of children [12,3] and on the malleability of flexible tangible objects such as sock puppets [10,2]. We strove to make the visual design of the avatars and the activities they perform as universal as possible so any child will be able to relate to them.

The game has a simple set of mechanics. When the singer sock puppet opens his mouth the singer avatar starts singing, and when the conductor puppet opens his mouth the conductor avatar waves his baton. In addition if the conductor avatar waves his baton at the same time the singer avatar is singing the singer will change his tune.

The game is governed by a set of two rules:

The first rule is that the puppet (operated by a player) controls the avatar appearing on the screen. To enact this rule the player has to interact with the computer.

The second rule is that the player/puppet/ avatar control each other. To enact this rule the players have to interact with each other.

We hypothesized that if the children can discover/learn these rules in the digital
game they would be more likely to enact them when the electricity supply fails and
the computer ceases to function and that the sock puppets coupled with the learned
rules and a sense of play would provide an engaging (and hopefully an equally engag-
ing) experience.

The game was developed using Quartz Composer [1] and reacTIVision [11]. The motion of the sock puppets and the evaluation of the open/close mouth events were detected using a webcam connected to a computer and fiducial markers affixed to the sock puppets (Fig. 2).
## 3. Putting the Game to the Test

To put the game to the test we conducted three field studies in schools. Our aim in these studies was not to conduct a formal study of the game but to gauge the overall concept. We primarily wanted to see whether the children found the overall experience fun and engaging, how they related to the avatars, whether they would be able to discover the rules of the game and what they would do when the computer ceased to function due to a simulated power failure.

Initially we had planned to conduct two field studies, one in Nepal and one in Portugal so that we can not only investigate the ways in which the game is being played but also see if there are any differences in the ways Nepalese and Portuguese children engaged with the game given the huge disparity in terms of culture, geography and economic wellbeing between them. However in light of the great socio-economic disparity between different schools in Nepal we decided to conduct two field studies in Nepal, one in a private school catering to the children of the well to do and the other in a public school catering to village children so that we can gain a more representative image of the child population of Nepal.

### 3.1. The Schools

The first trial was conducted with 14 students (10 Females and 4 Males) between the ages of 13-14 at the Ullens School in Kathmandu, Nepal (Fig. 3, left), a private school offering well-equipped classrooms to students coming from well endowed families capable of paying the high fees charged by the school.

The second trial was conducted with 12 students (8 Females and 4 Males) between the ages of 10-13 at the Shree Rudrayanee School (Fig. 3, middle) located on the outskirts of Kathmandu, a government run school offering free public education in bare bones facilities to the general population.

The third trial was conducted with 24 children (12 boys and 12 girls) between the ages of 10 to 11 in the Escola Basica dos 2º e 3º Ciclos dos Louros (Fig. 3, right) located in Funchal, Madeira, Portugal, a government run school offering free public education in well equipped classrooms to the general population.

In all schools the school selected the participating students and no remuneration was provided to either the school or the students.

<figure><img src="image_4.png" /><figcaption>Fig. 3. The three schools in which the studies were conducted.</figcaption></figure>
## 3.2. Setup and Method

We aimed to keep the setup of the environment and the procedure followed by the facilitators the same in each of the schools as much as possible and for the most part we were successful.

In each of the schools the field study was conducted using a laptop computer set up in one of the classrooms. In Kathmandu the study was conducted in English by two facilitators one interacting with the children and the other serving as an observer, note taker and cameraman. The Funchal study was conducted in Portuguese by three facilitators two interacting with the children and the third serving as an observer, note taker and cameraman.

Gameplay was conducted with a pair of children at a time. The pairs were always composed of two girls or two boys (in Kathmandu this was a result of self-organizing by the children themselves, in Funchal the pairs were matched by one of the facilitators). Each pair spent about 10-15 minutes with the game.

The children were not provided with any information about the game or the rule set. They were only instructed to choose one of the two puppets and to point their puppet towards the camera embedded at the top of the laptop screen.

Each field test session consisted of two conditions, a power condition in which the computer was functioning and the participants could interact with it and a no power condition in which the computer was not functioning and the players had to solely rely on the sock puppets and their own intuition in order to play the game. The power condition always preceded the no power condition so that the children would have a chance to play the game and discover the rules. The children were not told about the no power condition, the screen simply went black.

The facilitators not only observed the reactions of the children to the game they also engaged in a conversation with the children to see whether they recognized the avatars and made any correlations between their actions and the actions of the avatars. All the gameplay sessions were video recorded so that they could be further analyzed at a later date. In the Funchal study the children were debriefed after the gameplay session by one of the facilitators to gauge their reactions to the game and their understanding of the elements and rules of the game.

## 4. Observations

Perhaps one of the most interesting observations of all is the striking lack of difference in the reactions of the children to the game and its constituent elements despite the immense differences in terms of culture, language and environment between the Kathmandu and Funchal and the wide socio-economic disparity between the two schools in Kathmandu. Without exception all of the children found the game engaging and fun, they immediately took to moving the characters around the screen, recognizing who controls which avatar and making the singer sing. We therefore report our observations of the children reactions to the game in a unified form marking the differences between the two cities and three schools where necessary.
**4.1. Discovering the Rules of the Game**

**Discovering the First Rule.** All the children were able to map their sock puppet to the corresponding avatar (white sock puppet to the singer, black sock puppet to the conductor) and recognized the relationship between the actions they performed with the sock puppet and the activity of the avatar on the screen. Thus all of the children easily discovered the first rule (the puppet controls the avatar) although there were some minor differences in the speed in which the children came to these realizations all did so within the first two minutes of playing the game.

**Discovering the Second Rule.** In both schools in Kathmandu only a small number of the children discovered the second rule (the player/puppetGlass/Glass/Glass control each other). In the game the singer avatar sang nonsense words. When the singer sang without the direction of the conductor he repeated the word “oolalah”, when the conductor started twirling his baton he changed the word to “oogagah”. We assume the difference in tone between the two words was not very distinct so some of the children did not notice the difference in singing and thus only saw the singer and the conductor as independent actors sharing the same stage. For the test in Funchal we changed the sounds so when the singer sang without the direction of the conductor he warmed his voice by singing la...la...la...la...and when the conductor started twirling his baton the singer started humming the tune of Beethoven’s *Ode to Joy* in order to make the tonal shift between the two scenarios very distinct. This change had the desired effect with all of the children in Funchal discovering the second rule.

**4.2. Interacting with the Technology**

The reacTIVision Fiducial Markers used to control the avatars were both a prominent and highly visible feature of the sock puppets and a source technical difficulties as the reacTIVision software has difficulties with fast moving objects. Although the children were not given any information about the game mechanics or the technology behind the game none of them had any difficulties in figuring out the mechanics and none were fazed by technical glitches instead they incorporated a form of experimentation into their gameplay for example:

* Exploring the line of sight of the camera and seeing at what point the avatar stopped responding to their hand (puppet) movement.

* Playing with the open and close mouth actions. Some of the children constantly held the mouth open so that the open mouth fiducial marker was visible as this achieved the desired result. Others experimented with hiding the fiducial marker with their other hand to test whether the visibility of the marker was the controlling element

* When the system did not respond as expected such as when the movement of a fast moving hand was not effectively reflected in the movement of the avatar the children most frequently responded by repeating the same action over and over to validate whether the avatar would respond as expected. In face of a non-compliant avatar response, the children would attempt at either moving the hand faster or/and moving the
puppet closer to the camera – at times within a few millimetres of it – to try to get the desired response

### 4.3. Interacting with the Avatars

In some cases the relationship between the child and the avatar was more personal in both Kathmandu and Funchal we observed children talking to the avatars by saying “hello” to the conductor at the beginning of the game or telling the singer to “shut up” when he was singing his warm up song. Some of the children also tried to guide the avatars by placing their sock puppet next to the avatar on the screen and trying to nudge him in the direction they wanted him to go.

As was hoped the children also brought their own meanings and interpretation into the game. In both the Funchal and Kathmandu studies some of the children referred to the conductor as the as the magician and to the singer as the ghost. One pair in Funchal thought the objective of the game was for the characters to run after each other another pair suggested that the conductor has a baton so he could hit the ghost with it.

### 4.4. The No Power Condition

The initial reaction of most children to the no power scenario was to try and get help pointing out that the computer is no longer working, when no help was forthcoming they adopted a wide range of approaches:

* Some of the children adapted the same approach they used in case of a slow response, i.e. continue to play and position the sock puppets in different ways to try and get a response from the system despite the fact that in the no power condition the screen was entirely black.

* A few tried to fix the system by pressing some of the keys or looking around the computer for the reset button. In these cases it was usually one of the children who was the adventurous one trying to fix the system while the other child was the cautious one, either telling his partner not to touch the system or physically preventing him from touching anything.

* The inability to get the system back to work did not, in most cases, deter the children from continuing to play-act with the socks for a while. In some cases the children followed the narrative of the game and even sang the singing avatar's tune, in others they followed their own narratives some of which were created from elements of the game. For example, In both Funchal and Kathmandu a pair of boys actually play acted a scuffle with the sock puppets explaining that that is how they see the game as they assumed the objective of the game is for the conductor to catch the singing ghost and that the conductor baton was a weapon to be used on the ghost.

Most of the children play acted for a short period of time as they were self conscious someone was watching them and some simply waited patiently for the facilitators to help them when all other attempts at either getting attention from the system or the facilitators failed.
**5. Concluding Remarks**

In this study we set out to examine two questions: 1. Can we create an interactive game that would prove to be engaging with and without electrical power? 2. Can we discern any differences in the ways children in Nepal and Portugal interacted with the game given the huge socio-economic and cultural differences between them?

The first question remains to be fully answered as the no-power condition proved to be too contrived to provide a definitive answer to the question. However there were clear signs of engagement highlighting the potential of the approach. We intend to incorporate the lessons learned in this study into the next version of the game.

The second question yielded a fascinating set of observations. The first and perhaps the most significant one is the striking lack of difference in the reactions of the children to the game despite the immense differences in terms of culture, language, socio-economics, and exposure to technology between Nepal and Portugal and the socio-economic and technology exposure disparities between the schools in Nepal.

This finding is of primary importance as it implies that the design of mechanism and low-level basic interactions of technologies can be regarded as universal across cultures and levels of income and can thus be designed and tested in any location and be expected to work and be understood by kids of any culture and background. Furthermore, having done and tested their initial designs at home field study teams can move beyond basic interactions and mechanics and focus on aspects of environment, culture and language where participatory design truly comes into its own thereby maximizing the benefits of the field study and minimizing the overall cost of development.

The second significant observation is that the children often did not regard the failure of the technology as a failure but as an opportunity. Although the fast hand movements of the children led at times to a slow response by the game as the software struggled with the fast motion, the children were not frustrated by these failures but actively sought out ways to get around these difficulties by experimenting with different hand motions, speeds, positions and collaborations to get the system to perform as they wished. Thus the children saw what we perceived as technical failures as a sort of challenge to be solved. This perception can be effectively integrated into the game design process as a means of dealing with the foreseen and unforeseen challenges of developing technologies for developing and resource challenged countries.

Our future work will follow up on the insights gleaned in this study in a number of tracks: 1. We intend to continue with the development of the game towards the ultimate goal of creating an equally engaging game. 2. We intend to explore how children perceive and engage with the hidden “magic” inside the machine and what are the different methodologies they adopt in their approaches to problem solving. 3. We will continue to replicate the follow up studies in a number of countries so as to further explore whether ideas and observations gleaned using participatory and emphatic design methodologies can be applied across the board and if not where do the boundaries lie.
## Acknowledgements

We would like to express our outmost thanks and extreme gratitude to Ishwor Hamal, Veronika Halupka, Monique Park, Joana Gomes, Sophia Gomes and Rui Branco for the help and assistance they provided us during the field studies. We thank Diana Xu and Janet Read for comments they made on an early draft of this paper. Last but by no means least we would like to thank all the children and teachers who participated in the study for playing our game and sharing their insights with us.

## References

1. Apple Quartz Composer https://developer.apple.com/technologies/mac/graphics-and-animation.html. (2013).

2. Caillois, R. *Man, Play and Games*. Champaign, Illinois: University of Illinois Press. (1961).

3. Caillois, R. *Man, Play and Games*. Champaign, Illinois: University of Illinois Press. (1961).

4. Chetty, M., and Grinter, R.E. HCI4D: HCI challenges in the global south. In *CHI '07 Extended Abstracts on Human Factors in Computing Systems (CHI EA '07)*. ACM, New York, NY, USA, 2327-2332. (2007).

5. Dell, N., Vaidyanathan, V., Medhi, I., Cutrell, E. and Thies, W. "Yours is better!": participant response bias in HCI. In *Proceedings of the 2012 ACM annual conference on Human Factors in Computing Systems (CHI '12)*. ACM, New York, NY, USA, 1321-1330. (2012).

6. Gilutz, S., & Black, J. B. Child and Design Factors interacting in Children's HCI Helping children focus on the content, not the interface. In: *Designing for Children* February 2-6, 2010, Mumbai, India (2010).

7. Hourcade, J. P. Interaction Design and Children. In *Foundations and Trends in Human-Computer Interaction*, 1(4), pp. 277-392. (2008).

8. Hussain, S., Sanders, E., and Steinert, M. Participatory Design with Marginalized People in Developing Countries: Challenges and Opportunities Experienced in a Field Study in Cambodia. *International Journal of Design* 6:2. (2012).

9. Kolko, B., Putnam, C., Rose, E., and Johnson, E. Reflection on research methodologies for ubicomp in developing contexts. In *Personal Ubiquitous Computing* 15, 6, 575-583. (2011).

10. Marco, J., Cerezo, E., Baldassarri, S., Mazzone, E. and Read, J. C. Bringing tabletop technologies to kindergarten children. In *Proceedings of the 23rd British HCI Group Annual Conference on People and Computers: Celebrating People and Technology (BCS-HCI '09)*. British Computer Society, Swinton, UK, 103-111. (2009).

11. Reactivision. http://reactivision.sourceforge.net/.

12. Sutton-Smith, B. *The Ambiguity of Play*. Cambridge, MA: Harvard University Press. (2001).

13. Toyama, K. Technology as amplifier in international development. In *Proceedings of the 2011 iConference (iConference '11)*. ACM, New York, NY, USA, 75-82. (2011)

14. Yardi, S. and Bruckman, A. Income, Race, and Class: Exploring Socioeconomic Differences in Family Technology Use. In *Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12)*. ACM, New York, NY, USA, 3041-3050. (2012).