# Social Robots in Learning Experiences of Adults with Intellectual Disability: An Exploratory Study

Alicia Mitchell, Laurianne Sitbon, Saminda Sundeepa Balasuriya, Stewart Koplick, Chris Beaumont

Koplick, Chris Beaumont

► To cite this version:

Alicia Mitchell, Laurianne Sitbon, Saminda Sundeepa Balasuriya, Stewart Koplick, Chris Beaumont. Social Robots in Learning Experiences of Adults with Intellectual Disability: An Exploratory Study. 18th IFIP Conference on Human-Computer Interaction (INTERACT), Aug 2021, Bari, Italy. pp.266-285, 10.1007/978-3-030-85623-6_17. hal-04330964

HAL Id: hal-04330964

https://inria.hal.science/hal-04330964v1

Submitted on 8 Dec 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# Social robots in learning experiences of adults with intellectual disability: an exploratory study

Alicia Mitchell<fnref n="1" />, Laurianne Sitbon<fnref n="1" />, Saminda Sundeepa Balasuriya<fnref n="1" />, Stewart Koplick<fnref n="2" /> and Chris Beaumont<fnref n="2" />

**Abstract.** The use of social robots has the potential to improve learning experiences in life skills for adults with intellectual disabilities (ID). Current research in the context of social robots in education has largely focused on how children with Autism Spectrum Disorder (ASD) interact with social robots primarily without a tablet, with almost no research investigating how beneficial social robots can be in supporting learning for adults with IDs. This research explores how interactions with a social humanoid robot can contribute to learning for communities of adults with ID, and how adults with ID want to engage with these robots. This exploratory study involved observation and semi-structured interviews of eleven participants with ID (in three groups, supported by their support workers) receiving information from a semi-humanoid social robot and interacting with the robot via its tablet. Two robot applications were developed to deliver content based on the participating disability support organization's life skills curriculum for healthy lifestyle choices and exercise, considering a variety of modalities (visual, embodied, audio). The study identified four ways in which participants interact, and our findings suggest that both the physical presence of the robot and the support of the tablet play a key role in engaging adults with ID. Observation of participant interactions both with the robot and with each other shows that part of the robot's value in learning was as a facilitator of communication.

**Keywords:** Intellectual Disability; Young Adults; Communication; Learning; Social Robot; Human Robot Interaction, Tablet Applications; Support Workers; Pepper Humanoid Robot.

## 1 Introduction

Around 70% of the Australian National Disability Insurance Scheme (NDIS) users have an Intellectual Disability (ID) or learning disability, and one in eight will access community services for learning and life skills [1]. People with intellectual disability are keen learners, who learn best when presented with embodied and visual information.

<footnote n="1">¹ Faculty of Science, Queensland University of Technology, Brisbane, Australia
alicia.mitchell@hdr.qut.edu.au, l.sitbon@qut.edu.au,
samindasundeepa.balasuriya@hdr.qut.edu.au</footnote>

<footnote n="2">² Endeavour Foundation, Brisbane, Australia
{s.koplick, c.beaumont}@endeavour.com.au</footnote>
These people are attracted to technology, and prior research suggested that social robots could spark interest and collaboration [2]. Humanoid social robots, such as Pepper (see Fig. 1), combine embodied communication with movement and visual information via a tablet. They have been designed and used as assistants in nursing [3] and hospitality [4], and utilized in clinical contexts for diagnosis and treatment of Autism Spectrum Disorder (ASD) [5]. The purpose of this research is to evaluate how a social humanoid robot, such as Pepper, can contribute to the learning experiences of communities of adults with ID.

People with ID often seek support in community services, where they can learn and practice life skills while socializing. Community services are typically attended by people with diverse abilities, interests and communication preferences. They offer group learning experiences that respect people's individuality. The disability support organization involved in this study only supports people who identify as people who require support and development of life skills. This approach respects the fact that many people with ID do not present with a specific diagnosis. Some people have co-morbidities, such as physical disability or ASD.

The use of social robots, have the potential to contribute to the learning experiences for adults with varying ID. However, the needs and perceptions of support workers must also be considered for that potential to be realized, as they will play a key role in facilitating access to and use of technologies by people they support.

<figure><img src="image_3.png" /><figcaption>Fig. 1: Pepper robot.</figcaption></figure>

This project explores how adults with intellectual disabilities (IDs) engage with Pep-
per, a social humanoid robot which can communicate with users through movement,
speech, voice and visuals on a tablet (Fig. 1). The project is being undertaken in con-
sultation with an independent, for-purpose organization that provides support to people
with intellectual disabilities. This study begins to investigate how and why social robots
can improve informational experiences and support communities of people with ID,
with the primary purpose to identify how a robot can add value and to raise questions
for future research. More specifically, we explore how social robots can use a variety
of modalities (visual, embodied, audio) to support communities of people with ID (also
including support workers) and improve learning experiences. The research seeks to
explore the following questions:
* How do adults with ID want to engage with a social robot?

* How can a social robot contribute to the support network of adults with ID in a learning context?

We approached these questions by porting existing curriculum (both session scripts and visuals), that was about to be introduced in the community centers, in a format that would both be appropriate for and make use of Pepper's abilities. This curriculum forms part of life skills education and focuses on outcomes relating to healthy lifestyle choices (nutrition and exercise). The project included both adult participants with varying IDs, and their support workers. We observed and interviewed them as they took part in, or facilitated, group learning sessions with Pepper.

We first introduce related work which has informed our questions and methodology. We then detail our study methodology and the applications that we created to support our investigation. We present our findings, and finally discuss the implications of this research.

# 2 Related Work

A literature review was undertaken to investigate existing research involving social robots, education, training and people with intellectual disabilities. Information gathered from this review was used to inform and support the detailed design of this study and the research methodology including identification of the variables, objective and subjective measures.

## 2.1 Human Robot Interaction

"Social robots are technologies designed to engage with humans on an emotional level through play, sometimes therapeutic play, and perhaps even companionship" [6]. Current publications about social robots in the context of adults with ID and support workers may include studies involving only pictures and concepts of robots, not actual interaction with robots [7, 8]. There has been very little published research involving the Pepper robot, however, an analysis of over approximately 300 studies involving social robots in education, found that around 50% of research involved SoftBank Robotics' small humanoid robot, NAO, which does not have a tablet [9], supporting the perceived research value of humanoid robots. Some papers express concern about artificial interactions and attachment to robots, stating that robots should be designed to assist and support activities, and care should be taken not to try to replace human interaction. Further research investigating actual versus safe imagination is required in relation to how the robot is framed when introduced to users [10]. Currently in many disability support service centers, support workers provide support to groups of 3 to 4 people with ID at a time. Robots to support learning may allow more time for support workers to interact with individuals and could actually increase care and social interactions. As such, it is important to understand the role of support workers and caretakers, and their relationship with the robot.
## 2.2 Social Robots and Education

Current research in the context of social robots in education has largely focused on social robots primarily without a tablet as platforms for teaching languages [11] and children with ASD [12, 13]. Where a tablet has been connected to a robot, it is often as an additional tool for shared communication.

Social robots have been able to teach children with autism and people with ID to recognize and produce gestures [14]. The children could then generalize what they learned to different scenarios but were unable to reproduce the gestures appropriately when interacting with a human model.

Previous research has observed that people with intellectual disability engage for long periods of learning time when robots were involved in the learning experience [15], however the extent to which delivering information via the robot can improve learning outcomes and knowledge is unknown.

In terms of social skills training, previous studies have shown that robots can be used to improve participation and verbal communication among students with autism or ID [16]. The participants in this study were eager to work with NAO and intrinsic interest in the robot helped them bring about improvements in their communication skills.

A long term study that took place in a public school in South Australia with students with autism or ID showed that the student's eagerness to interact with NAO did not diminish even after 24 months [17]. Skills that they learned from their time with the robot, like turn taking, self-expression, confidence and patience were transferred to general activities.

Parents and teachers of children with ID and autism [17] noticed certain benefits of the children's interactions with NAO and Paro (a furry seal style of robot), stating that the appearance and non-judgmental behavior of the robots were important aspects of their success. The predictability of the robot's behavior and their ability to repeat functions were important to reduce the students' stress and anxiety. A robot can be perceived as an intelligent agent, so it can be used to simulate human behavior [12].

## 2.3 Social Robots and Intellectual Disability

In a study involving the NAO robot across four activities including guessing and modelling emotions, eliciting behavior such as touch the robot's head, dance choreography and responses to robot characteristics, the researchers observed that participants exhibited a reduced level of disability behavior in comparison to normal situation behaviors, even when the robot was not interacting with them [5]. The findings have supported that of other researchers that attention is comparatively higher during sessions that involve a robot [5, 18]. The six participants aged 33 – 67, represented conditions ranging from moderate to severe intellectual disability with two also having ASD, two with affective disorder and one with Down Syndrome. The comparative disability behaviors were measured using a questionnaire based on the Gilliam Autism Rating Scale-Second Edition (GARS-2) supplementary screening tool for ASD; the World Health Organization Disability Assessment Schedule 2 (WHODAS 2.0) a tool for assessing functioning
and impairment; and the AAMR Adaptive Behaviour Scale (ABS-RC: 2) for assessing
adaptive behaviour in mentally handicapped adults.

IDs can cause reduced understanding, challenges in communication, memory recall, focus and attention. Different teaching strategies are required for students with IDs [5]. Disability research often focuses on medical diagnosis narrative however these intellectual disabilities typically occur on a spectrum [5]. Where a robot may be situated in a class with multiple students with varying needs, it is perhaps more appropriate to take a non-clinical approach to describing participants and shift to describing the social narrative in relation to daily living and participation in society [7, 19].

There is limited research for teachers to refer to about using robots to support learn-
ing in children with intellectual disabilities, and a preliminary review of existing liter-
ature indicates that there has been almost no research investigating how beneficial so-
cial robots can be in supporting education for adults with IDs [15]. Despite not being
specifically focused on supporting people with IDs, studies of children with ASD also
highlighted that the robots offered a number of qualities that also supported suitability
for use with children with other IDs, including the predictability, engagement and safety
[15]. One user-centered design activity based on the Pepper robot, has reported that
people with IDs working in a manufacturing facility felt that robots like Pepper would
be useful in demonstrating tasks and explaining activities as the workers rotate through
workstation activities [4].

## 2.4 Social Robots and Support Workers or Caregivers

The social network of people with intellectual disabilities mostly consists of family
members and support workers [20]. Support workers that work in disability community
activity centers help people with intellectual disabilities exercise control over their daily
activities [21]. They are involved in designing and implementing activities that improve
the quality of life of people with ID and have many responsibilities including develop-
ing skills, conducting leisure activities, daily personal care and monitoring health [22].

A literature review exploring risk and the role of support workers in facilitating tech-
nology use by adolescents and adults identified that perceived risk and safety concerns
are emerging factors that influence access to technologies [23]. The perceptions of ease
of use and safety is linked to uptake of technology [24], and as such, if a support worker
is not confident about supervising their clients when using a technology such as Pepper,
trial and adoption may be inhibited.

A month-long in-home study involving a social robot with educational content, was situated in a home with young children with ASD revealed that they wanted the robot to stay with them for an extended period of time [25]. Parents of the children stated that they were comfortable leaving their children to interact with the robot independently.

Another study via an online survey, found that whilst support workers saw the po-
tential for robots to perform repetitive tasks, they were concerned about the level of
supervision required, and were worried about the robot breaking [8]. They highlighted
that robots should not and could not replace human interactions. It is important to note
that this survey utilized hypothetical questions and did not show a working model, so
the perceptions of participants may change if given an opportunity to experience a live
interaction with a physical robot [26].

Support workers watched videos of people with intellectual disability interacting with a small robot named Cozmo and stated that the participants were much more interested in interreacting with Cozmo than they were during the normal workshops [2]. The support workers also commented on behaviors of the participants and mentioned whether they were typical of that person, or a new behavior brought about by Cozmo. For some participants, Cozmo was able to enhance aspects of their personality like competitiveness according to the support workers. But they also noticed moments that they did not expect like some participants who are usually very quiet that engaged in helping their partner out.

## 2.5 Social Robots – Embodiment and Anthropomorphism

People respond better to robots than computer tablets delivering healthcare instructions, and healthcare apps have been shown to improve patient's adherence to treatment and provide health education information [27].

Comparative studies have found that physically embodied robots show more promise than virtually embodied robots and avatars [28-30], for example one study involving a robot in the form of an exercise coach for the elderly, where the robot acts as a traditional exercise instructor, demonstrates and asks users to imitate, and also involves user activity recognition for the robot to imitate the user [28].

The physical embodiment of the robot makes it more interesting for people to interact with as it allows for tactile exploration and physical movements. The ability to touch is not available in virtual avatars and previous research has shown that children prefer a physical robot over an avatar on a device [31].

Past research with people with ID interacting with Cozmo has shown how they anthropomorphize the robot [2]. They attributed feelings, emotions and a sense of autonomy to Cozmo.

In research incorporating robots with a separate tablet, the tablet appears to be more of a shared communication device, comparable to writing on a board to be observed by the learner, rather than an extension of the robot itself [32, 33]. Pepper's tablet is bedded on the chest, forming a part of the robot rather than a remote device. Users may feel more connected and engaged with the robot when they can touch it [34, 35]. Certainly, the findings by de Wit et al. 2018 [36] and Vogt et al. 2019 [37] support the value of a tablet for user input and display. The use of the tablet not only provides an interactive mechanism for user input, but enhances the multimodal communication capabilities of the robot.
# 3 Methodology

## 3.1 Overview

The user study involved running a learning session where participants' interaction with the robot was observed. This exploratory study forms part of a research project which has received ethical clearance by the Queensland University of Technology (1400000673). With consent from all participants and support workers, the sessions were recorded on video to allow further review and analysis by the researchers after the session had been completed. The session was designed in a flexible manner to allow support worker's some autonomy and choice in how to run the session and facilitate use of the robot. Following an introduction about the session approach and objectives, all participants received an interactive induction about how to use Pepper. The user study was then divided into two key activities:

- Learning About Healthy Choices. Participants with ID receive the healthy eating life skills education curriculum module via Pepper. This content was developed independently by the disability service organization and adapted for Pepper by the research team. In this session, Pepper delivers some of the content verbally, and some visually (using images on the tablet). Participants can interact with Pepper by touching the screen.

- Practicing Healthy Exercise. Participants with ID participate in an exercise class directed by Pepper, mainly based on stretching movements. In this session, Pepper guides participants using speech and displays a video of the full movement on the tablet, and, in one activity, accompanies the video with synchronized arm movements. There is no interaction with Pepper during this session, however participants were encouraged to mimic the movements.

## 3.2 Apparatus

Robot. The robot used for this study was SoftBank Robotics' Pepper (Model v1.8a). Pepper is a semi-humanoid robot. At 120cm tall, Pepper is much taller than other robots commonly used in research, but not so tall as to be perceived as threatening. Pepper is sometimes perceived to be equivalent to a 12-year-old child.

Pepper can detect faces and follows people's faces with head movement. The eyes are animated with color change only. A speaker is located with the head, and in this research was programmed with the default female-like voice. There is a microphone to enable speech or sound recognition, however it was not activated during our research. The arms are animated, either with random communication-like movement during speech, or with movement programmed by an application. There are no animated legs, but travel is possible using wheels situated at its base. A 10.1" tactile screen (tablet) is situated on the robot's chest.

Recording Equipment. All sessions were video recorded using three cameras placed in various parts of the room: a Ricoh Theta S 180-degree camera, a Kodak Pixpro SP360 360o camera, and an action camera. An iPhone 8 was used as a backup for recording audio. Having several cameras also ensures that there is always one actively
recording. The benefit of having wide angle or 360 cameras for this research is to fully capture all participants reactions and align them with the robot's action naturally. Additionally, these types of cameras are small and unobtrusive, and their quality is sufficient for participant observations.

Furniture. Chairs were available in the room for participants to use if they chose to during the robot interactions, in between interactions and during group discussions.

## 3.3 Participants

We recruited participants from three of the disability support organization's center locations. They travelled to a University Campus where the sessions were held in order to take part in the research. A total of three trial sessions were run. Each trial had 3-4 participants with IDs and 1-2 support workers. Across the trial there were a total of 11 participants with IDs and 4 support worker participants. Table 1 details the participants and the session which they were involved in.

<table><caption>**Table 1. Participants.**</caption><thead><tr><th>Group</th><th>Participant Ref</th><th>Gender</th><th>Session</th></tr></thead><tbody><tr><td rowspan="10">Participants with ID</td><td>PA1</td><td>F</td><td>A</td></tr><tr><td>PA2</td><td>M</td><td>A</td></tr><tr><td>PA3</td><td>M</td><td>A</td></tr><tr><td>PA4</td><td>M</td><td>A</td></tr><tr><td>PB1</td><td>M</td><td>B</td></tr><tr><td>PB2</td><td>F</td><td>B</td></tr><tr><td>PB3</td><td>M</td><td>B</td></tr><tr><td>PC1</td><td>M</td><td>C</td></tr><tr><td>PC2</td><td>F</td><td>C</td></tr><tr><td>PC3</td><td>M</td><td>C</td></tr><tr><td>PC4</td><td>F</td><td>C</td></tr><tr><td rowspan="4">Support worker participants</td><td>SWA1</td><td>F</td><td>C</td></tr><tr><td>SWA2</td><td>M</td><td>A</td></tr><tr><td>SWB1</td><td>F</td><td>A</td></tr><tr><td>SWC1</td><td>M</td><td>B</td></tr></tbody></table>

**Participants with Intellectual Disability.** Participants were young adults (20 – 30) with varying abilities and levels of intellectual functioning who attend the support organization's centers/hubs. All participants were physically able to interact with the robot and use the applications via the robot's tablet, despite some displaying limited verbal and reading capabilities. The cohort selected by the support organization were all familiar with and had access to technology such as tablets and computers.
The participants represented conditions ranging from moderate to severe intellectual
disability with two also with ASD, and one with Down Syndrome. Whilst it may seem
important to provide details about the medical diagnosis and nature of each participant's
condition, we argue that this provides a narrow view of the person. The diversity of the
participants reflects what is typically observed in disability support centers. As such,
we prefer to focus on the evaluation of the robot applications, engagement with the
robot and on enhancing informational experiences across a spectrum of individual
needs, in the context of group learning.

**Support Workers of Adults with Intellectual Disability.** An important aspect of this study was to involve the support workers in all aspects of the session. Participants were staff of the support organization, familiar with the people who attended the trials with them. Support workers provide support and supervision through a range of activities and excursions aimed at helping people develop independent life skills, amongst other social and learning skills. It is important to note that support workers typically do not have educational or teaching qualifications or teaching-based backgrounds.

**3.4 Applications**

The two curriculum applications were developed with varying characteristics and en-
gagement methods to provide insight into the comparative benefits of engagement style.
The applications were designed without time limits to complete interactive activities,
instead allowing the participant in conjunction with the direction of their support
worker to set the pace [38].

Whilst the robot can be programmed to incorporate speech recognition inputs, there are a number of factors that can impact the success of speech interactions, such as background noise, user position in relation to the robot's microphone, timing of speech, choice of words and pronunciation. In this experiment, all programs were run locally on the robot, with no link to external cloud services which may deliver more advanced speech functionality. Without more advanced speech recognition systems and comprehensive user and environmental testing, such technical constraints could result in a high rate of unsuccessful speech interactions. This could, in turn, be discouraging for users and impact their confidence for continuing trying to interact. Thus, a decision was made not to use speech input for this experiment. Instead, we were able to observe where participants maybe were inclined to speak to Pepper, and the language they used, without it impacting the overall success of their interaction and while respecting their varying verbal preferences and abilities. Throughout the applications, an animated speech function was active, where the robot aligns contextual movement and gestures of its body with the content of the speech delivered. When the robot was not speaking, the movement would cease.

**Healthy Eating Application.** The healthy eating application involved an interactive
series of questions which could be answered by an individual user, followed by ques-
tions for group discussion with the support worker.
The three interactive questions each gave the user a choice of two pictures for the participants to select the healthy option answer by tapping the screen. For a correct answer, Pepper displayed a green tick on the screen and waved its arms whilst playing a sound of a crowd cheering (see Fig. 2). If the incorrect answer was selected, Pepper displayed a red cross and asked the user to try again. At the end of the three questions, Pepper asked if the user wanted to do the quiz again or move onto the next activity which was the group discussion. eLearning guidelines [39] suggest that frequent learner interaction is necessary to sustain attention. Repeating concepts through discussion can help to facilitate the learning process [38] so the discussion activity was designed with the intent that the support worker would facilitate discussion with the group or with individuals, asking questions about the healthy food pyramid, with the picture displayed on Pepper's tablet (see Fig. 2), and asking participants to identify how many serves of foods were recommended for different genders and ages of people from a table displayed on the tablet. To help reinforce learning through check-point questions [40], the final activity was a facilitator led discussion about what people would like to include in their own food pyramids.

<figure><img src="image_4.png" /><figcaption>Fig. 2. Pepper robot healthy eating application.</figcaption></figure>

**Healthy Exercise Application.** This application involved five exercise activities. Each exercise activity was structured in an “I do, We do, You do” format – essentially a basic education strategy for delivery of content. First Pepper showed a video of a person to the participants to demonstrate the exercise, then Pepper asked the participants to try the exercise along with the video, and finally, Pepper asked the participants to try the exercise on their own, whilst displaying a still image of the exercise. The fifth exercise was a repeat of the first exercise, with the addition of a custom animation where Pepper demonstrated the swinging arms movement in time with the video (as shown in Fig. 3).
<figure><img src="image_5.png" /><figcaption>Fig.3. Pepper robot exercise application trial.</figcaption></figure>

**3.5 Qualitative data**

The sessions involved observation of participant interactions and engagements with the robot, and with other participants in the group. Semi-structured interviews were carried out during the session and at the end of the session to gather information about the participants’ previous experience with robots, their thoughts about the robot, if they would like to interact with the robot again, what parts of the robot applications they liked most, and how the robot may be most useful to them. Observations were noted through the session and a thematic analysis [41] was undertaken to look for variances in engagement styles and synthesis across the three study groups. The video and audio recordings were leveraged to look back and verify the observations and gain further insights.

# 4 Findings

**4.1 Overview of Findings**

Observing the sessions, we reviewed how participants were engaging with Pepper to look for common themes across the groups. We considered “engagement” to encompass interacting with the robot, displaying interest in the robot, and initiating joint attention such as talking about the robot with others in the group. Engagement was observed both during a participant’s turn to interact with Pepper, when other participants were taking their turn, and during group discussion when the application activities had been completed. We identified four commonalities in engagement style:

1. High intensity engagement and emotional attachment

2. Intermittent focus

3. Quiet and obedient

4. Wanting more/distant
Because Pepper is a well-known social robot often appearing in news media, partic-
ipants may have prior attitudes towards Pepper. None of the participants had interacted
with Pepper before.

4.2 Researcher Observations of Participants with Intellectual Disability

**High Intensity Engagement and Emotional Attachment.** Some participants at-
tempted to engage Pepper in a social interaction when they first saw it. These partici-
pants displayed the highest intensity of engagement

PA1 was clearly drawn to Pepper throughout the session. She went straight up to Pepper at the beginning of the session and stayed close by up until everyone sat down to talk about the healthy eating pyramid. PA1 asked Pepper to shake hands, when no response was received, she threw her hands up a little, then turned away. She then turned to look at Pepper repetitively, often nodding encouragingly, and sometimes speaking, “yes, yes” or “it’s ok Pepper” or “that’s a good boy, Pepper”. During the initial setup, when PA2 had started an application, Pepper gave the instruction to tap its foot. PA1 touched the top of Pepper’s bumper, but when nothing happened, she said, “OK, Pepper, please”, when nothing happened again, she said, “What are you doing to me Pepper”. PA1 seemed fascinated by all parts of Pepper. PA1 answered many of the questions about the healthy eating application correctly, meaning that she paid attention to Pepper’s presentation, even when she was not directly involved in the interaction. Even though PA1 had noticed that the robot applications were programmed from the researcher’s laptop, she spoke of the robot as an autonomous being, and spoke of “teaching” the robot, rather than programming it, and starting off with simple instruc- tions so that “the robot could get its head around it” to start with. When we asked PA1 what she would like a robot to do for her, she stated that she wanted a robot that could help her around the house, reach high things and push her wheelchair. PA1 also stated that she would trust Pepper more than a doctor to give her medical advice particularly because of her non-threatening face. She also said that she would prefer to talk and give instructions to Pepper verbally than using the tablet.

PC1 was the first in his group to interact with Pepper. He called out "Hello Pepper". In the Healthy Eating quiz, PC1 selected his answers without waiting for the full in- struction from Pepper and gave himself a clap. When the robot cheered, he went to touch Pepper's hand. By the third question, where the robot asked "How do you feel? PC1 responded "I feel good" and selected the corresponding picture. At the food pyra- mid, he verbally answered the questions, then tapped to the next question. At the last question, he knelt down to look closely at the tablet, seemingly not wanting to look away from the robot. Twice during the session, he referred to Pepper as his "best friend" and he thought that robot seemed nice.

PB1 seemed very excited to be meeting a robot for the first time. In the healthy eating application, he seemed to be very concerned with getting the answer correct and seemed surprised when he got the right answer. He loved getting the right answer and seeing Pepper cheer. Unlike PC1 and PA1 who spoke to Pepper more like a friend, PB1 gener- ally referred to Pepper as the “robot” but was amazed when Pepper was “brought back to life”.
Intermittent Focus. Some participants switched from highly engaged, and focused on Pepper, to distracted and looking around the room. This may be attributed to a number of factors that have very little to do with the robot, such as the unfamiliar environment, the level of difficulty of the content, and the individual's usual ability to stay on task.

PA4 arrived late at the session and missed out on the earlier play with Pepper. After introducing himself to everyone in the room, he went to look at Pepper and asked, “May I touch it?”. PA1 introduced Pepper to him. PA4 was very interested in having a turn to change the application with the support worker. When changing the app, Pepper went into sleep mode. He followed PA1’s lead, saying “wakey wakey”. When Pepper was awake, he said “Hi Pepper”. He found tapping Pepper’s screen to be easy. PA4 often sought guidance from SWA2 during his interactions. At the end of each question, PA4 tended to lose focus on the robot. He was less engaged during the exercise class and stated that the exercises were too easy. This could be because of his experience playing basketball.

PB2 celebrated when she got the right answers in the healthy eating application and repeated the activity twice. During the healthy food pyramid section of the application, she walked away from the robot a few times because she thought it was the end of her turn, but it was not.

Quiet and Obedient. Initially PA2 stood back and watched Pepper and PA1, whilst also looking around the room. PA2 pressed icons on Pepper’s tablet without being prompted to do so to operate the applications. He seemed very familiar with using tablet functions. He followed along with Pepper’s instructions very well, and whilst he seemed quite fascinated by the robot’s hands, the majority of his engagement was focused on the tablet.

PB3 did not seem particular interested in Pepper, however SWB1 said that he spoke more to Pepper than in his usual interactions with people.

Wanting More/Distant. Some of the participants did not really engage with Pepper unless asked to. PC2 looked bored and did not seem to be very interested when interacting with Pepper. During the healthy eating pyramid questions, she skipped through to the next question without saying a response. When PC3 got the wrong answers in the healthy eating application, he tried to put Pepper to sleep.

PA3 was the least interested in Pepper. He only engaged in an activity and interacted with Pepper when encouraged by SWB1. He wasn’t called over to engage with Pepper until the Healthy Eating pyramid discussion questions. He exhibited a high ability in reading but may have a lower ability to comprehend and contextualize as he said the fruits that he saw, but when asked what the red tick meant, he seemed to have stopped paying attention and for the next two questions he answered incorrectly “fruit” before tapping the next button. He then moved away for PA4 to have their turn and returned to the back of the room. During the Healthy Eating discussion, he answered questions again relating to fruit. When it was time for the exercise class, Pepper asked “Are you ready? And PA3 said yes. We may observe that this participant responded to more active participatory questions rather than quiz questions. He participated in the exercises and was more engaged throughout the exercise application than the healthy eating application. PA3 was quite restless during the session and spent a lot of time walking
around the back of the room, which may be due to environmental factors and certain
aspects of this participant's disability.

## 4.3 Support Worker Perspectives

The support workers in each session engaged quite differently. In the first group, SWA2 opted to run the healthy eating application activities as a group, with PA2, PA1 and PA4 each taking a turn to answer a question. He then invited PA3 and PA4 to answer some of the healthy eating pyramid discussion. PA4 asked if he could change the application, and SWA2 then asked him to do it with him.

In the second group, SWB1 was very interactive, discussing the healthy eating topics and encouraging each participant to take their turn and sit down if they liked. SWB1 noted that she was very drawn to Pepper's non-verbal cues, particularly the robot's eyes. SWB1 was active in the exercise application and felt it best for the participants with IDs to be in control of the robot and moving. In the third group, SWC1 allowed the participants to each take a full turn of the whole healthy eating application. SWC1 and SWB1 actively invited the participants to learn to change the application.

The first session involved multiple support workers. SWA1 watched the application, but did not interact with Pepper, encouraging SWA2 to run the session. They agreed that Pepper has a lot of potential to run classes at the support organization's centers while they facilitate. They stated that the robot has the potential to hold the participants attention much better than a human teacher. But they also stated that the lessons will have to be changed up because the novelty factor might wear off after some time. An overall theme was that Pepper would be of benefit to support the consistent delivery of curriculum information. SWA2 stated that a robot with a female voice is better as it is more calming than a male voice.

## 4.4 Application structure and design

**Healthy Eating Application.** Initially it was thought that each participant would have a turn completing all three questions, however, the first group completed this activity as a group, taking a turn with 1 question each. Some sessions were run with the participants each taking their turn to do all 3 questions of the healthy eating quiz, then they took turns to talk about the healthy eating pyramid. This meant that they had to do the quiz again to get to the discussion questions. In the first session, only three participants had a chance to answer an interactive question.

The image used for the healthy eating pyramid [42] was quite detailed for a small screen. Some participants had difficulty reading the content which may have been in part due to the screen size, but also because the image text and background colors were not web accessibility compliant. Participants were generally able to identify foods, e.g. bananas, but some of the visuals were not as easily identifiable from the images used. When asked to identify a healthy food, PA2 didn't pick an item from the screen, after he hesitated, PA1 asked what he had for lunch, to which he responded "sandwiches". The question, what does the red cross mean, was also challenging for some participants,
and SWB1 suggested that a saltshaker may have been more easily identifiable than the imagery used.

This application involved close interaction from the participants to Pepper. Participants in the first group all stood up during this application, but participants in the other group sat in a chair to interact with Pepper. During this time, most participants were looking at the screen, except where Pepper cheered.

**Healthy Exercise Application.** In all sessions, the participants joined in while the robot was showing the initial demonstration. At the third stage, where participants were supposed to do the exercise by themselves, there was some confusion and participants copied the still image displayed on the tablet. By the fourth exercise, engagement of all participants was starting to wane, however at the fifth exercise, where the robot physically did the exercise as well, all participants reengaged, and except for PC4 and PB1, all participants performed the exercise with more gusto. Most didn't appear to be consciously aware that the robot was doing the activity too. PC4 was so amazed that she didn't do the exercise. PB1 seemed distracted by the fact that the videos showed the researcher doing the exercise.

# 5 Discussion

We discuss some of the factors that played a role in how participants engaged with Pepper, namely its physical presence and its social features. We also consider the implications of this work in practice, and the limitations of this exploratory study to be considered in future work.

## 5.1 Social nature of the robot and physical presence

Participants who wanted to engage with Pepper when they first saw it, may have done so because of its anthropomorphic shape.

Participants were more engaged during the interactive parts of the applications, as interactive activity attracts higher levels of attention [43]. Participants seemed to become less engaged in the exercise application and this may be in part because participants were not required to touch Pepper. This may have a negative effect on participants feelings of social presence and connection to Pepper [34, 44]. As noted by SWB1, the interactive screen is more engaging than playing a video, supporting the findings of other research [5]. In many ways, the tablet is central to this study, and its information delivery and interactions are supplemented by a social robot. Information needs to be visual to appeal to and be easily understood by people with ID. A social robot, offering a personality dimension, can add to user's sense of efficacy and trust in the presented information [45]. While we did not directly measure this in the study, the styles of engagement offer a starting point for future studies examining this aspect more specifically for users responding in a particular way.

The physical presence of the robot performing the arm movements for the final exercise correlated to an improvement in performance and engagement of the participants,
even if they did not appear to be consciously following the physical movement of the robot. This supports the finding of other their research which examined the benefits of interactions with physically present robots over video-displayed agents [29].

The most highly engaged participants were also the ones who seemed to be the most emotionally attached, talking both to, and about the robot like a friend. Whilst PA1 and PA2 communicated that they knew that the robot was a machine that could be programmed or rebooted, PA3 did not make such statements. This highlights potential areas of concern about artificial interactions and attachment to the robot [8]. Further research investigating actual versus safe imagination and attachment is required in relation to how the robot is framed when introduced to users.

## 5.2 Applicability to support centers context

This project started with an opportunity for support workers to be highly engaged and interactive alongside the robot, however, the support workers opted to be less engaged in running the session than was anticipated. The way in which support workers would use the robot, or spend any time and capacity freed up by the robot warrants further exploration, with consideration of factors of perception and acceptance of the robot as a co-facilitator.

The learning material delivered in this study was structured in line with support organization's curriculum requirements, and focus on highly visual and interactive modes of delivery. Pepper's tablet can be used to display video, and this enhances the potential for the robot to be a one stop shop for consistent and repeatable curriculum delivery. Something to consider for future activities is the design and complexity of content displayed on the tablet and the position of users interacting with the robot. Because the tablet is relatively small, it is suggested that content be simple, easy to read, and highly interactive. The participants in the first and second sessions thought that Pepper had potential for supporting sign language education and translation. SWB1 suggested that Pepper may be helpful in guiding meditations and personal hygiene activities. There is an opportunity to explore how robots with tablets may be used to provide information in an unstructured way, including flexible modalities for people to engage with Pepper in the way that best suits them using the tablet, speech or other touch sensor inputs. Beyond traditional curriculum delivery about independent life skills, literacy and numeracy, a social robot like Pepper can foster social relationships and play a role in modeling roles and social stories to support community and social engagement skills development.

As discussed in Lytridis et al.'s 2019 [46] review on measuring engagement, social interactions can be multi-modal and complex. Not all behavioral cues can be detected by equipment, some require trained specialists to detect and assess engagement indicators such as expression, inspection, approach and communications towards the robot. The engagement styles we have identified offer a non-clinical structure for evaluation of robot interactions that follows a social narrative. Observation and thematic analysis to determine styles of engagement enables the nuances of an individual's behavioral cues to be assessed, whilst providing common terms and a way to group participants in a useful way for community-based group learning. This may be useful to determine the
kinds of group configurations, types of interaction and curriculum contexts where robot
assisted community-based learning may deliver the most benefits, while acknowledg-
ing that the robot may not be of interest to everyone.

5.3 Avenues for Future Research

This study involved three separate sessions, with three different groups of participants. Each session was held on the university campus in the morning to align with the participants' morning tea and lunch schedule. Whilst a single interaction session lasting only 5-10 minutes for each user per activity is of research value [5, 26], as only one session was run with each group, there are limitations on the conclusions that can be drawn. Although the application content was developed from the support organization's standard curriculum, the content may have been of an incompatible complexity for some participants, as such a differentiated model of content warrants further exploration.

The results observed in the experiment may in part be due to the novelty effect of
the robot, rather than the interactions with it. In future research, it would be interesting
to investigate the long-term effects of the interaction with Pepper. For instance, how
well do the participants with ID remember the presented information about the food
pyramid? How would their perception of their relationship with the robot change,
would they still consider Pepper a friend (PC1) after multiple interactions?

It is also important to note also that the experiments were held at a university, not
the participant's regular center environment. As a result of this, further research is
needed to determine if there is a difference in interaction and engagement when partic-
ipants can engage with the robot in familiar surroundings, not just with familiar groups.
In the case of participants who appeared to be wanting more/distant, or quiet and obe-
dient, passiveness within the first session may not equate to disinterest over time where
participants are given an opportunity to become more familiar and confident in inter-
acting with the robot.

Disability support service organizations are typically for-purpose entities with lim-
ited funding. A social robot such as Pepper is currently a significant investment, beyond
the initial price of the hardware. At this time there are a very limited range of off the
shelf applications available. Longer term studies to observe changes in engagement and
use of the robot within a center environment are also required in order to evaluate the
novelty effect and intervals at which content should be iterated or changed to maintain
results. Once a broader base of participants has had a chance to become familiar with
the robot, co-design of applications could be explored.

This research has focused on younger adults and the results may not be generalized
across a span of all ages [28]. There is a gap in research about social robots to support
older adults (e.g. 45+) with intellectual disability and further research is warranted to
examine how social robots can support learning experiences of older people with IDs.
The participants invited by the support organization may be more tech savvy than the
general population of people and support workers attending the centers, and the ap-
plicability of the results to a wider audience should be interpreted with this in mind.
# 6 Conclusion

This exploratory study provides a new lens on the diversified use of social robots in life skills learning contexts for adults with intellectual disability. We have observed a variety of engagement styles for participants, but also a variety of uses by the learning facilitators. Our study uniquely placed the robot as a co-contributor to the learning experience, rather than as an exclusive learning partner, thus supplementing the social context rather than replacing it. Our findings suggest a way forward for social robots in disability services through engagement, and perhaps co-design with each network of learners and learning facilitators (in this case, support workers), as each group came up with a unique configuration and role for the robot. Many of our participants are eager to further explore the possibilities in the longer term, which we will investigate through a variety of experiences over long periods of time in the future.

**Acknowledgements.** This research is supported by the Australian Research Council under grant LP160100800, jointly with Endeavour Foundation. We acknowledge the time and dedication of our participants. Gavin Suddrey, Nicole Robinson and Belinda Ward of the ARC Centre of Excellence for Robotic Vision contributed to the development of the robot applications.

## References

1. Australian Institute of Health and Welfare: Disability support services: services provided under the National Disability Agreement 2017–18. AIHW, Canberra, Australia (2019).

2. Balasuriya, S.S., Sitbon, L., Brereton, M., Koplick, S.: How can social robots spark collaboration and engagement among people with intellectual disability? In: Proceedings of the 31st Australian Conference on Human-Computer-Interaction, pp. 209–220. ACM, New York (2019).

3. Ahn, H.S., Lee, M.H., MacDonald, B.A.: Healthcare robot systems for a hospital environment: CareBot and ReceptionBot. In: Proceedings of the 24th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), pp. 571–576. IEEE, Kobe (2015).

4. Collins, G.R.: Improving human-robot interactions in hospitality settings. International Hospitality Review 34(1), 61-79 (2020).

5. Shukla, J., Cristiano, J., Amela, D., Anguera, L., Vergés-Llahí, J., Puig, D.: A case study of robot interaction among individuals with profound and multiple learning disabilities. In: Tapus, A., André, E., Martin, J.-C., Ferland, F., Ammi, M. (eds.) Social Robotics. ICSR 2015. Lecture Notes in Computer Science, vol. 9388. pp. 613-622. Springer, Cham (2015).

6. Shaw-Garlock, G.: Loving machines: Theorizing human and sociable-technology interaction. In: Lamers, M.H., Verbeek, F.J. (eds.) Human-Robot Personal Relationships. HRPR 2010. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol. 59. pp. 1-10. Springer, Berlin, Heidelberg (2011).

7. Williams, A.B., Williams, R.M., Ronald E. Moore, McFarlane, M.: AIDA: a social co-robot to uplift workers with intellectual and developmental disabilities. In: 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 584–585. IEEE, Daegu (2019).

8. Wolbring, G., Yumakulov, S.: Social robots: views of staff of a disability service organization. International Journal of Social Robotics 6(3), 457-468 (2014).
9. Belpaeme, T., Kennedy, J., Ramachandran, A., Scassellati, B., Tanaka, F.: Meta analysis of robots for learning literature. Science Robotics, vol. 3. Science Robotics (2018).

10. Tanaka, F., Kimura, T.: The use of robots in early education: A scenario based on ethical consideration. In: The 18th IEEE International Symposium on Robot and Human Interactive Communication, pp. 558-560. IEEE, Toyama (2009).

11. Belpaeme, T., Vogt, P., van den Berghe, R., Bergmann, K., Göksun, T., de Haas, M., Kanero, J., Kennedy, J., Küntay, A.C., Oudgenoeg-Paz, O., Papadopoulos, F., Schodde, T., Verhagen, J., Wallbridge, C.D., Willemse, B., de Wit, J., Geçkin, V., Hoffmann, L., Kopp, S., Krahmer, E., Mamus, E., Montanier, J.-M., Oranç, C., Pandey, A.K.: Guidelines for designing social robots as second language tutors. International Journal of Social Robotics 10(3), 325-341 (2018).

12. Pennazio, V.: Social robotics to help children with autism in their interactions through imitation. Research on Education and Media 9(1), 10-16 (2017).

13. Pennisi, P., Tonacci, A., Tartarisco, G., Billeci, L., Ruta, L., Gangemi, S., Pioggia, G.: Autism and social robotics: a systematic review. Autism Research 9(2), 165-183 (2016).

14. So, W.-C., Wong, M.K.-Y., Lam, C.K.-Y., Lam, W.-Y., Chui, A.T.-F., Lee, T.-L., Ng, H.-M., Chan, C.-H., Fok, D.C.-W.: Using a social robot to teach gestural recognition and production in children with autism spectrum disorders. Disability and Rehabilitation: Assistive Technology 13(6), 527-539 (2018).

15. Hedgecock, J., Standen, P.J., Beer, C., Brown, D., Stewart, D.S.: Evaluating the role of a humanoid robot to support learning in children with profound and multiple disabilities. Journal of Assistive Technologies 8(3), 111-123 (2014).

16. Silvera-Tawil, D., Roberts-Yates, C., Bradford, D.: Talk to me: the role of human-robot interaction in improving verbal communication skills in students with autism or intellectual disability. In: Proceedings of the 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), pp. 1-6. IEEE, Nanjing (2018).

17. Silvera-Tawil, D., Roberts-Yates, C.: Socially-assistive robots to enhance learning for secondary students with intellectual disabilities and autism. In: Proceedings of the 27th IEEE International Symposium on Robot and Human Interactive Communication, pp. 838-843. IEEE, Nanjing (2018).

18. Standen, P., Brown, D., Roscoe, J., Hedgecock, J., Stewart, D., Galvez Trigo, M.J., Elgajiji, E.: Engaging Students with Profound and Multiple Disabilities Using Humanoid Robots. vol. 8514, pp. 419-430. Springer Verlag, Cham (2014).

19. Sitbon, L., Hoogstrate, M., Yule, J., Koplick, S., Bircanin, F., Brereton, M.: A non-clinical approach to describing participants with intellectual disability. In: Proceedings of the 30th Australian Conference on Human Computer Interaction, pp. 128-132. ACM, New York (2018).

20. McVilly, K.R., Stancliffe, R.J., Parmenter, T.R., Burton-Smith, R.M.: "I get by with a little help from my friends": adults with intellectual disability discuss loneliness. Journal of Applied Research in Intellectual Disabilities 19(2), 191-203 (2006).

21. Davy, L.: Philosophical inclusive design: intellectual disability and the limits of individual autonomy in moral and political theory. Hypatia 30(1), 132-148 (2015).

22. Iacono, T.: Addressing increasing demands on Australian disability support workers. Journal of Intellectual & Developmental Disability 35(4), 290-295 (2010).

23. Seale, J.: The role of supporters in facilitating the use of technologies by adolescents and adults with learning disabilities: a place for positive risk-taking? European Journal of Special Needs Education 29(2), 220-236 (2014).

24. Bröhl, C., Nelles, J., Brandl, C., Mertens, A., Schlick, C.: TAM reloaded: a technology acceptance model for human-robot cooperation in production systems. In: Stephanidis, C. (ed.)
HCI International 2016 – Posters' Extended Abstracts. HCI 2016. Communications in Computer and Information Science, vol. 617. Springer, Cham (2016).

25. Clabaugh, C., Becerra, D., Deng, E., Ragusa, G., Matarić, M.: Month-long, in-home case study of a socially assistive robot for children with autism spectrum disorder. In: Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction (HRI '18), pp. 87–88. ACM, (2018).

26. Shukla, J., Cristiano, J., Oliver, J., Puig, D.: Robot assisted interventions for individuals with intellectual disabilities: impact on users and caregivers. International Journal of Social Robotics 11, (631-249 (2019).

27. Mann, J.A., MacDonald, B.A., Kuo, I.H., Li, X., Broadbent, E.: People respond better to robots than computer tablets delivering healthcare instructions. Computers in Human Behavior 43, (112-117 (2015).

28. Fasola, J., Matari, M.J.: A socially assistive robot exercise coach for the elderly. Journal of Human-Robot Interaction 2(2), 3-32 (2013).

29. Bainbridge, W., Hart, J., Kim, E., Scassellati, B.: The benefits of interactions with physically present robots over video-displayed agents. International Journal of Social Robotics 3(1), 41-52 (2011).

30. Belpaeme, T., Kennedy, J., Ramachandran, A., Scassellati, B., Tanaka, F.: Social robots for education: a review. Science Robotics 3(21), (2018).

31. Sinoo, C., Pal, S.v.d., Henkemans, O.A.B., Keizer, A., Bierman, B.P.B., Looije, R., Neerincx, M.A.: Friendship with a robot: children's perception of similarity between a robot's physical and virtual embodiment that supports diabetes self-management. Patient Education and Counselling 101(7), 1248-1255 (2018).

32. Lemaignan, S., Jacq, A., Hood, D., Garcia, F., Paiva, A., Dillenbourg, P.: Learning by Teaching a Robot: The Case of Handwriting. IEEE Robotics & Automation Magazine 23(2), 56-66 (2016).

33. Hood, D., Lemaignan, S., Dillenbourg, P.: When Children Teach a Robot to Write: An Autonomous Teachable Humanoid Which Uses Simulated Handwriting. Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction, pp. 83–90. Association for Computing Machinery, Portland, Oregon, USA (2015).

34. Lee, K.M., Jung, Y., Kim, J., Kim, S.R.: Are physically embodied social agents better than disembodied social agents?: The effects of physical embodiment, tactile interaction, and people's loneliness in human-robot interaction. International Journal of Human-Computer Studies 64(10), 962-973 (2006).

35. Andreasson, R., Alenljung, B., Billing, E., Lowe, R.: Affective Touch in Human-Robot Interaction: Conveying Emotion to the Nao Robot. International Journal of Social Robotics 10(4), 473-491 (2018).

36. Wit, J.d., Schodde, T., Willemsen, B., Bergmann, K., Haas, M.d., Kopp, S., Krahmer, E., Vogt, P.: The Effect of a Robot's Gestures and Adaptive Tutoring on Children's Acquisition of Second Language Vocabularies. Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction, pp. 50–58. Association for Computing Machinery, Chicago, IL, USA (2018).

37. Vogt, P., Berghe, R.v.d., Haas, M.d., Hoffman, L., Kanero, J., Mamus, E., Montanier, J.M., Oranç, C., Oudgenoeg-Paz, O., García, D.H., Papadopoulos, F., Schodde, T., Verhagen, J., Wallbridgell, C.D., Willemsen, B., Wit, J.d., Belpaeme, T., Göksun, T., Kopp, S., Krahmer, E., Küntay, A.C., Leseman, P., Pandey, A.K.: Second Language Tutoring Using Social Robots: A Large-Scale Study. In: 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 497-505. IEEE, (2019).
38. Lopez-Basterretxea, A., Mendez-Zorrilla, A., Garcia-Zapirain, B., Madariaga-Ortuzar, A., Lazcano-Quintana, I.: Serious games to promote independent living for intellectually disabled people: Starting with shopping. In: 2014 Computer Games: AI, Animation, Mobile, Multimedia, Educational and Serious Games (CGAMES), pp. 1-4. IEEE, Louisville (2014).

39. Food and Agriculture Organization of the United Nations: E-learning methodologies: a guide for designing and developing e-learning courses (2011). http://www.fao.org/3/i2516e/i2516e.pdf.

40. Australian e-Learning Association: Australian e-learning best practice guidelines: elearning performance criteria – learning methodology and framework (2016). https://www.australianelearningassociation.com.au/wp-content/uploads/2016/06/eLearning-Performance-Criteria-Learning-Methodology-and-Framework.pdf.

41. Braun, V., Clarke, V.: Using thematic analysis in psychology. Qualitative Research in Psychology 3(2), 77-101 (2006).

42. Australian Nutrition Foundation Inc.: Healthy eating pyramid. (2015).

43. Shukla, J., Cristiano, J., Anguera, L., Vergés-Llahi, J., Puig, D.: A comparison of robot interaction with tactile gaming console stimulation in clinical applications. Proceedings of Robot 2015: Second Iberian Robotics Conference. Advances in Intelligent Systems and Computing, vol. 418. pp. 435-445. Springer, Cham (2015).

44. Lee, K.M.: Presence, explicated. Communication Theory 14(1), 27-50 (2004).

45. Lopatovska, I., Korshakova, E., Brown, D., Li, Y., Min, J., Pasiak, A., Zheng, K.: User Perceptions of an Intelligent Personal Assistant's Personality: The Role of Interaction Context. Proceedings of the 2021 Conference on Human Information Interaction and Retrieval, pp. 15-25. Association for Computing Machinery, Canberra ACT, Australia (2021).

46. Lytridis, C., Bazinas, C., Papakostas, G.A., Kaburlasos, V.: On Measuring Engagement Level During Child-Robot Interaction in Education. In: Merdan, M., Lepuschitz, W., Koppensteiner, G., Balogh, R., Obdržálek, D. (eds.) Robotics in Education. RiE 2019. Advances in Intelligent Systems and Computing, vol. 1023. pp. 3-13. Springer, Cham (2020).