# Towards the Applicability of NAO Robot for Children with Autism in Pakistan

Muneeb Imtiaz Ahmad, Suleman Shahid, Anam Tahir

► To cite this version:

Muneeb Imtiaz Ahmad, Suleman Shahid, Anam Tahir. Towards the Applicability of NAO Robot for Children with Autism in Pakistan. 16th IFIP Conference on Human-Computer Interaction (INTER-ACT), Sep 2017, Bombay, India. pp.463-472, 10.1007/978-3-319-67687-6_32. hal-01717220

HAL Id: hal-01717220

https://inria.hal.science/hal-01717220

Submitted on 26 Feb 2018

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Towards the Applicability of NAO Robot for Children with Autism in Pakistan

Muneeb Imtiaz Ahmad<fnref n="1" />, Suleman Shahid², and Anam Tahir²

**Abstract.** In this paper, we present a HRI study that reports on the potential of NAO as a socially assistive robot in Pakistan. Our findings generated through interviewing 2 parents and 5 teachers on the plausibility of using NAO robot as an interaction partner show that both groups welcomed the use of NAO at schools. They, however, were sceptical due to missing NAO's facial expressions and certain body parts such as nose and lips. They also emphasised the importance of creating natural text to speech interface for the Urdu Language. Our findings taken from 7 autistic children to measure their level of social interaction during HRI revealed that children positively engaged with the NAO robot and showed a significant number of both verbal and non-verbal behaviours.

**Keywords:** Autism Spectrum Disorder; Children-Robot Interaction; Socially Assistive Robots.

## 1 Introduction

Autism Spectrum Disorder (ASD) is a pervasive, behavioural development disorder that typically manifests in the first three years of life [7]. Children with Autism show signs of social, emotion, communication impairment. Based on the level of their impairment, children are regarded as suffering from Low- or High-Functioning ASD. The prevalence of ASD is growing all over the world. However, most of this research on ASD, both at understanding the causes of Autism and developing digital solutions to support children with Autism, is witnessed in the western world [1] and a lack of research has been found in countries from the eastern world such as Pakistan. It is estimated that 350,000 children in Pakistan suffer from autism and this number is increasing day by day [12].

The integration of technology-based interventions including use of mobile tablets [21], robots [18], computer-based interventions [10] to help children with ASD can be witnessed widely in the western world. Socially Assistive Robots (SARs), as introduced by [18], is a commonly used term that encompasses all types of robots that can be used to assist people with special needs. These SARs are being used to support individuals suffering from autism, elderly with Alzheimer's and dementia and several other impairments [13].

<footnote n="1">¹ MARCS Institute, Human Machine Interaction Group, Western Sydney University; Australia, ² Department Of Comp. Science, Lahore University of Management Sciences; Pakistan</footnote>
SARs have been successfully utilised in helping children with autism de-
velop communication and social skills [16]. However, the potential impact of
such robots has been under-studied in the countries that are part of the eastern
world such as Pakistan. To the best of our knowledge, SARs have never been
used in the Pakistani context to support Autistic children or other children with
special needs but we do find a handful of studies conducted at schools in Pak-
istan where technology-based interventions were fruitful in improving the social
and emotional skills of children with Autism [6] [5]. We understand that due to
socio-cultural differences, findings from studies where SARs were used with these
children in the western world cannot be generalised to the Pakistani context [1].
Therefore, before designing and conducting interaction studies with SARs in
Pakistan, different factors are needed to be taken into consideration. These fac-
tors include technology acceptance, understanding parents and teachers views
on the use of SARs in schools or at homes.

We find numerous studies conducted with parents and teachers on their views on the acceptance of SARs in the western world [14] [16]. However, one of the issues with these studies is that the effect of parents’ or teachers’ hypothetical knowledge about robots has an effect on their opinions about future robotic design. Different studies have addressed this aspect and showed its impact on the quality of results [3]. Therefore, our research focuses on collecting observations from teachers and parents who first interacted with the NAO robot before providing feedback on their applicability and informing on their future designs and capabilities.

Another important aspect is to study the initial response of autistic children
to a social robot, based on the level of their autism. Studies have been con-
ducted where social engagement of children with autism has been measured in
an interaction with the NAO robot [19]. However, we don't find any study that
focuses on child's level of autism while investigating their interaction with social
robots. We, therefore, conducted a HRI study with autistic children to measure
the difference between their levels of social interaction and engagement during
their interaction with the NAO robot depending on the level of autism.

Keeping these aforementioned aspects, we, in this paper, report an initial
HRI study conducted with parents, teachers to inform their views on acceptance
towards using a NAO robot and its future design interaction with their children.
In addition, we also report a study conducted with autistic children to measure
the effect of NAO robot's interaction on children's social interactions and overall
engagement. To the best of our knowledge, this is the first study with a SAR in
this context that has been conducted in Pakistan.

## 2 Method

In our present study, we have followed the procedure as described by [17] in
which children were exposed to a NAO robot to evaluate their initial responses.
However, we conducted our study with parents, teachers to get their views on
the potential of NAO robot as a helping assistant for children with autism. In
addition, we also measured social interaction of autistic children during their
interaction with the NAO robot.

**2.1 Interaction Scenario**

We programmed NAO robot to autonomously display behaviours during the interaction, however, speech recognition was controlled via Wizard of Oz (WoZ). NAO was capable of autonomously showing three emotions (Happiness, boredom and sadness) through displaying gesture as reported by [9].

NAO robot began the one-to-one interaction with the participant through
asking introductory questions: (Hello, I am NAO, What is your name, “How are
you today”, “Have you interacted with a robot before”, “Do you like me” and
“Let me show you my emotions”. After the dialogue, NAO displayed sad emotion
through playing a sound with a gesture and uttering “I am feeling sad, Can you
smile for me”. Another gesture displaying boredom was played with saying “I am
feeling bored, can you smile a bit more”. Lastly, a gesture displaying happiness
was played while saying “I am happy to see you smiling”. At the end, NAO said
“Good Bye! I had fun interacting with you.” and waved goodbye gesture.

**2.2 Participants**

We conducted our study with two parents (1-male, 1-female), five teachers (2-males and 3-females) and seven children with autism spectrum disorder (6-males, 1-female) at a school for special children in Lahore, Pakistan. The ages of the parents, teachers were between 25-35 respectively whereas the children aged between 10-12 years. None of the participants had ever interacted with a robot.

Although we were informed that all participating children were diagnosed
with high functioning ASD, we, however, conducted a Gilliam Autism Rating
Scale-Second Edition (GARS-2) test to identify the level of autism [8]. The
test informs about the participant's Autism Index (AI). In case, if AI is 85 or
above, the person is very likely to be an autistic, and if it is 70-84, the person
is possibly autistic. 3 out of 7 participants were found to be likely autistic while
others belonged to the possibly autistic group.

**2.3 Procedure**

Our study was conducted in two phases. In the first phase, NAO interacted with parents that lasted for 3 minutes followed by a 5-minute interview. NAO began with an introductory dialogue followed by an interaction showing three different emotions (happiness, sadness and boredom) through displaying gestures simultaneously with LED lights. Lastly, the robot thanked the parents for participation and displayed the bye gesture along with a dialogue. An interview followed the interaction where we inquired about the potential of NAO robot as a helping agent for autistic children.

In the second phase, NAO interacted with autistic children in the presence
of teachers. The teachers monitored the sessions and played a calming role for
the autistic children. The interaction also lasted for 3 minutes following the same procedure as for the parents. After the interaction, one of the researchers interviewed the teacher about the potential of NAO as an assistive robot.

## 2.4 Setup and Materials

The set up of the study required using two portions. The first portion was for children and parents to engage with the NAO. We conducted our study in a quiet room, and as shown on the left side of Figure 1 below. The room was divided into two portions. On one side, the child and the parent independently interacted with NAO that had been placed on a table in front of the participant. On the other side, one of the researchers was controlling the speech recognition capabilities of NAO through a Wizard of Oz (WoZ) setup.

<figure><img src="image_2.png" /><figcaption>**Fig. 1.** Setup: A parent (left) and a child (right) interacting with NAO robot.</figcaption></figure>

We used NAO robot designed and developed by Aldebaran robotics. It is a humanoid robot measuring 58 cm in height. It provides researchers with a platform to design various applications driven by their creativity and requirements.

The interview questions were designed to discuss the potential of NAO as a tool to provide edutainment. In addition, we inquired about the activities where NAO can provide assistance. Moreover, what in their opinion are the missing capabilities of NAO and in what ways, these gaps can be covered. Lastly, we also asked them about the applicability of the NAO robot at homes and overall acceptance of such a robot in general.

## 3 Results

In this section, we present both quantitative and qualitative results. Our quantitative findings are based on measuring social interaction and the initial reaction of children with different level of autism during the HRI. Our qualitative findings are a reflection of both teachers and parents views on the potential of NAO as an assistive agent for their children.
**3.1 Quantitative Results**

Our goal of the study was to measure the amount of Social Interaction (SI) and Engagement (E) during our interaction sessions of both most likely and possibly autistic children. We predict that there will be a significant effect of autism level of the child on the SI or E. In order to measure SI or E, we analysed videos for all the sessions.

**Video Analysis** In order to measure the amount of SI or E, we coded both verbal and non-verbal behaviours during the sessions. We followed the coding scheme as reported by [4] and [11] respectively. We measured the frequencies and durations of eye gazes, verbal responses, and facial expressions of all the participants during the sessions. Eye-gaze referred to an event when the child gaze was facing towards the robot. Verbal response referred to the number of times the child spoke with the robot or answered robot query. Facial expression referred to the number of time a child smiled back at the robot.

We conducted an independent-samples t-test to compare the frequencies and durations of eye gazes, verbal responses and facial expressions in most likely and possibly autistic children determined through conducting the GARS-2 test [8]. We found a significant difference for the number eye gazes of most likely (M= 13, SD=118) and possibly (M= 26.67, SD=24.67) autistic children t(-3.34) = 0.01, *p* < 0.02. We also found a significant difference for the frequency of facial expressions of most likely (M= 1.75, SD=12.75) and possibly (M= 8.33, SD=34.67) autistic children t(-2.79) =0.01, *p* < 0.02. Our results witnessed a higher number of eye gaze and facial expressions for the possibly autistic children. In case of durations, we found a significant difference for eye gazes of most likely (M= 7.18, SD=4.13) and possibly (M= 4.13, SD=5.11) autistic children t(2.46) = 0.03, *p* < 0.04. We also witnessed significant difference for the durations of facial expressions of most likely (M= 0.9, SD=3.36) and possibly (M= 2.75, SD=0.38) autistic children t(-2.79388) =0.01, *p* < 0.02. The durations of both eye gaze and facial expressions were also found to be higher for possibly autism children as compared to more likely autistic children.

<figure><img src="image_3.png" /><figcaption>**Fig. 2.** Mean values for the frequencies (left) and durations (right) of eye-gaze, facial expressions and verbal response.</figcaption></figure>
On the other hand, we didn't find significant difference for the number of verbal responses of most likely (M= 9.5, SD=59) and possibly (M= 10.67, SD=162.67) autistic children $t(-0.22) = 0.41, p = 0.41$. We also didn't find significant difference for the durations of verbal responses of most likely (M= 0.81, SD=0.92) and possibly (M= 1.39, SD=0.47) autistic children $t(-1.43) = 0.10, p = 0.10$. We only witnessed a minor difference for the frequencies and durations of verbal response among two groups as shown in the mean value plot of eye-gaze, verbal response and facial expressions in figure 2.

**Discussion:** Our results confirmed our prediction and showed that children with the different level of autism showed different level of social interaction and engagement during HRI. However, we only found the number of non-verbal interactions to be significant among two groups of autistic individuals with varying autism based on the GARS-2 test results. We didn't find differences in the verbal interaction. We conjecture that the reason for this difference may be due to the social skills of each individual. As literature shows that high-functioning autistic children may also have varying social skills among each other [15]. We, however, also believe that more research studies with higher number of participants are required to consolidate these findings. In general, results showed that all children showed a certain level of interaction and engagement with the robot. This confirms the potential of NAO as a SAR in Pakistan and also directs for more research in the future.

## 3.2 Qualitative Results

In order to analyse data taken from the interviews, we performed content analysis on the transcriptions generated from audio recordings. We present the main themes resulting from the qualitative analysis performed on the interview data with parents and teachers. To keep the identities anonymous, the parents are labelled as P1 and P2, while teachers are listed as T1, T2, T3, T4, and T5 respectively.

**Anthropomorphism** Our findings show that both parents and teachers were comfortable with the use of NAO to help their children and students. However, both teachers and parents emphasised on the need of facial expressions for NAO. One of the parents mentioned: "Facial expressions are missing. For example, if one teacher is wearing a veil and the other one has her face uncovered children will connect better with the second one (P1)". Teachers mentioned as "the body parts and facial expressions are missing. If NAO uses voice or gestures to explain emotions, it'll be necessary that a teacher is present to translate those emotions to the child (T3)."

On the positive side, both teachers and parents welcomed the inclusion of LED lights to display different emotions as an alternative solution. However, they considered it a short-term solution. They reported: "Yes, it will but will be short term. (P2)".

**NAO for Edutainment:** In general, the response of parents and children towards using NAO for edutainment was very positive. One father mentioned that his child will be attracted towards NAO, however, it depends on the type
of autism of the child. One parent mentioned: "It varies on the autism level. My elder daughter is high functioning she will be attracted because of the fact that it talks. Younger one is destructive she won't accept it (P1)." Teachers, on the other hand, focused on the importance of the level of exposure a child may have with the NAO robot. They believed that in the case of more exposure, children will get comfortable and in a long-term interaction, this will result in a positive way. One teacher mentioned: "It can but for that, we need to give children exposure. If they interact with it twice or thrice a week they will get comfortable with it (T1)."

Teachers were of the view that it may take 6 to 7 interactions on average to get familiar with the NAO robot. They commented: "It may take up to 6-7 days on a regular basis. If they interact with it daily for a week, he will start responding to its commands next week. (T1)" Another mentioned that it may take them less time as compared to the humans to familiarise. They commented: "The child will get comfortable with the robots faster as compared to humans as interaction with toys is usually better (T2)." Others also mentioned that the familiarisation time may depend on the type of autism. They commented: "It also depend on the autism level. children with mild to moderate autism level might get comfortable with it within a month others might take longer (T5)."

We witnessed that both parents and teachers speculated that NAO can help children develop speech, vocabulary and basic cognition. One parent said: "NAO can assist my child to develop Language, vocabulary, and speech (P2)." Teachers reported as "It can help develop basic cognition, object recognition such as fruit names (T1)." One of the teachers also pointed that NAO can help autistic children with fears about human body and expressions learn about non-verbal communications, however, they were sceptical due to missing facial expressions of NAO. One teacher reported: "Some children have fear of human body and expressions so this might work with these children. But, the absence of facial expression is an opposing element (T3)."

**NAO for Collaborative Learning:** We found that both parents and teachers welcomed the application of NAO in a collaborative task. They were presented with a situation and were also asked about their opinions. The situation involved three children working in a group and NAO asks the first one to touch its head and in response to that it stands up. Later, NAO asks the second one to touch its feet and it sits down and Lastly NAO asks the third one to touch its hand and it waves.

All of them were positive towards the utilisation of NAO in this manner however, they focused on the need of the presence of a teacher to monitor the activity. They reported: "It will be good because when it responds to their action children will enjoy it and it will fascinate them. (T4)" and "It will work but will require a teacher to monitor the scenario (T5)."

**Applicability of NAO at Homes:** In general, both parents and teachers wanted to utilise NAO at schools before it may be used at homes. One parent said: "Starting from school first (P1)."
**Future Possibilities with NAO:** Teachers emphasised on the need of adopting the voice of NAO though changing its tone in combination with facial expressions. One teacher mentioned: "Facial expressions and voice tone variation are important. Children not making an eye contact with the robot because they don't find anything attractive. If I ask him to smile and with that, I am smiling and my tone is varying the child will respond accordingly (T2)."

Teachers mentioned that the robotic voice of NAO may not be suitable for autistic children. They emphasised on the need of a regional voice for NAO that should be in the Urdu language. In addition, the text to speech conversion needs to be slow as children may not understand the speech.

**Discussion:** The key lessons we learned about conducting research in future with NAO robot in Pakistan with children suffering from autism are as follows:

**The need for the natural text-to-speech feature for social robots** In literature, the challenge of an unnatural and too paced voice of a robot has been reported [2]. Our results also indicated that teachers wanted a robot to speak in the Urdu language in a naturalistic way. It, therefore, directs researchers towards creating text-to-speech interfaces in the Urdu language that are currently non-existent.

**Design and Evaluate the User's Perception of existing facial expressions for social robots in Pakistan** Most recently, an exploratory study was performed with SAR in Pakistan [20] where autistic children participated in a collaborated play. It resulted in positive findings towards improving social interaction. Otherwise, we don't find interaction studies where robots have been used to help autistic children understand a certain emotion through a gesture. Therefore, we believe research is needed to evaluate the perception of children with autism in Pakistan on existing gestures used to display emotions. In addition, based on these result, future interaction on helping autistic children understand emotions can be designed.

**Long-term interaction with a Social robot** Our results showed that children with autism may require time to get familiar with the robot. However, the familiarisation time may vary depending on the different factors such as type and level of autism. Therefore, we need to find empirical evidence on the time it may require by a child with varying degree of autism to get familiar with a robot.

**Acceptance of Socially Assistive Robots** We need to conduct more studies in Pakistan to assess the acceptance of robot in Pakistan. Our results, however, showed that robots were accepted but we need to interview more parents and teachers to consolidate our findings.

## 4 Conclusion, Limitations and Future Work

In this paper, we presented our results on parent's and teacher's opinion on the potential of NAO as a SARs in Pakistan. Teachers and parents reacted positively towards using robots as a tool for edutainment, helping them understand basic human emotions, and develop speech and basic cognition. In addition, teachers
pointed towards the missing facial expressions in NAO and focused on design-
ing robots with expressions in the future. Our results of our interaction study
with autistic individuals also showed that robot technology has a positive effect
towards developing social skills in both children suffering from different levels of
autism. Children were able to display both verbal and non-verbal skills during
HRI. In general, results generated from parents, teachers and children showed
the potential of NAO robot as a therapist agent in Pakistan.

One of the argued limitations can be a relatively smaller group of participants
but, our study was conducted in a school and it is a common group size for studies
conducted with autistic children at schools [19] [1]. We tried to control the effect
of hypothetical knowledge of our participants through an interaction with NAO,
however, some results can still be based on their previous knowledge of robots.

References

1. Ahmad, M.: Design and Evaluation of a Mobile Learning application for Autistic Children in a Developing Country. Ph.D. thesis, University Paderborn (2013)

2. Ahmad, M.I., Mubin, O., Orlando, J.: Children views’ on social robot’s adaptations in education. In: Proceedings of the 28th Australian Conference on Computer-Human Interaction. pp. 145–149. ACM (2016)

3. Ahmad, M.I., Mubin, O., Orlando, J.: Understanding behaviours and roles for social and adaptive robots in education: teacher’s perspective. In: Proceedings of the Fourth International Conference on Human Agent Interaction. pp. 297–304. ACM (2016)

4. Ahmad, M.I., Mubin, O., Orlando, J.: Adaptive social robot for sustaining social engagement during long-term children-robot interaction. International Journal of Human-Computer Interaction (just-accepted) (2017)

5. Ahmad, M.I., Shahid, S.: Design and evaluation of mobile learning applications for autistic children in Pakistan. In: Human-Computer Interaction. pp. 436–444. Springer (2015)

6. Ahmad, M.I., Shahid, S., Maganheim, J.S.: A game-based intervention for improving the communication skills of autistic children in Pakistan. In: International Conference on Computers for Handicapped Persons. pp. 513–516. Springer (2014)

7. Frith, U., Happé, F.: Autism spectrum disorder. Current biology 15(19), R786–R790 (2005)

8. Gilliam, J.E.: GARS-2: Gilliam autism rating scale. Pro-ed (2006)

9. Häring, M., Bee, N., André, E.: Creation and evaluation of emotion expression with body movement, sound and eye color for humanoid robots. In: 2011 RO-MAN. pp. 204–209. IEEE (2011)

10. Hetzroni, O.E., Tannous, J.: Effects of a computer-based intervention program on the communicative functions of children with autism. Journal of autism and developmental disorders 34(2), 95–113 (2004)

11. Hourcade, J.P., Williams, S.R., Miller, E.A., Huebner, K.E., Liang, L.J.: Evaluation of tablet apps to encourage social interaction in children with autism spectrum disorders. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp. 3197–3206. ACM (2013)

12. Irfan Haider, M Maroof Qureshi, Q.F.A.: The autism puzzle. https://www.dawn.com/news/1173610 (2015), [Online; accessed 12-April-2017]
13. Leite, I., Martinho, C., Paiva, A.: Social robots for long-term interaction: a survey. International Journal of Social Robotics 5(2), 291–308 (2013)

14. Mundy, P., Institute10, M.: Development of socially assistive robots for children with autism spectrum disorders (2009)

15. Rao, P.A., Beidel, D.C., Murray, M.J.: Social skills interventions for children with asperger's syndrome or high-functioning autism: A review and recommendations. Journal of autism and developmental disorders 38(2), 353–361 (2008)

16. Scassellati, B., Admoni, H., Matarić, M.: Robots for use in autism research. Annual review of biomedical engineering 14, 275–294 (2012)

17. Shamsuddin, S., Yussof, H., Ismail, L.I., Mohamed, S., Hanapiah, F.A., Zahari, N.I.: Initial response in hri-a case study on evaluation of child with autism spectrum disorders interacting with a humanoid robot nao. Procedia Engineering 41, 1448–1455 (2012)

18. Tapus, A., Mataric, M.J., Scassellati, B.: Socially assistive robotics. IEEE Robotics and Automation Magazine 14(1), 35 (2007)

19. Tapus, A., Peca, A., Aly, A., Pop, C., Jisa, L., Pintea, S., Rusu, A.S., David, D.O.: Children with autism social engagement in interaction with nao, an imitative robot: A series of single case experiments. Interaction studies 13(3), 315–347 (2012)

20. Tariq, S., Baber, S., Ashfaq, A., Ayaz, Y., Naveed, M., Mohsin, S.: Interactive therapy approach through collaborative physical play between a socially assistive humanoid robot and children with autism spectrum disorder. In: International Conference on Social Robotics. pp. 561–570. Springer (2016)

21. Venkatesh, S., Phung, D., Duong, T., Greenhill, S., Adams, B.: Toby: early intervention in autism through technology. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp. 3187–3196. ACM (2013)