R obot-assisted therapy (RAT) offers potential advantages for improving the social skills of children with autism spectrum disorders (ASDs). This article provides an overview of the developed technology and clinical results of the EC-FP7-funded Development of Robot-Enhanced therapy for children with AutisM spectrum disorders (DREAM) project, which aims to develop the next level of RAT in both clinical and technological perspectives, commonly referred to as robot-enhanced therapy (RET). Within this project, a supervised autonomous robotic system is collaboratively developed by an interdisciplinary consortium including psychotherapists, cognitive scientists, roboticists, computer scientists, and ethicists, which allows robot control to exceed classical remote control methods, e.g., Wizard of Oz (WoZ), while ensuring safe and ethical robot behavior. Rigorous clinical studies are conducted to validate the efficacy of RET. Current results indicate that RET can obtain an equivalent performance compared to that of human standard therapy for children with ASDs. We also discuss the next steps of developing RET robotic systems. ASDs are identified by widespread abnormalities in social interactions and communication, together with restricted interests and repetitive behavior [1] . For children with ASDs, these symptoms can be efficiently reduced through early (cognitive) behavioral intervention programs, ideally starting at the preschool age [2] . This type of intervention is taught on a one-to-one basis in school and/or at home by caregivers (therapists, teachers, and parents) and must be intensive and extensive [2] , [3] . This process requires a significant amount of human workload to carry out therapeutic sessions as well as to manage a child's performance data. Using robots for autism therapy has received considerable attention over the past two decades [4] . Similar to animals and computers, robots can provide simple and predictable interactions, during which people with ASDs generally feel comfortable; however, they have several advantages over classical therapies [5] , such as the repeatability of the medium' s behavior, embodiment of the medium, and hygienic safety. RAT enables embodied interactions, such as increasing engagement and attention and decreasing social anxiety [6] , which are appealing to many children with ASDs. During a childrobot interaction, RAT robots can simultaneously provide social cues while maintaining simplicity and predictability [7] . These robots are diverse in their appearances, ranging from mobile platforms to humanoid robots. Although RAT robots have shown advantages, most of the studies are exploratory and have methodological limitations [4] , e.g., a low number of participants or numerous protocol breaches. Regarding this developing technology, most RAT studies are limited to the WoZ technique in which robots are remotely controlled-unbeknownst to the child-by a human operator [ Figure 1 (a)] [2], [7] . The WoZ technique allows human therapists to achieve a high level of social interaction without having to use a complex robotic system. However, it requires a significant amount of human workload and not suitable in the long term [8] . It is necessary to increase the level of robot autonomy in RAT research to lessen the human workload and deliver consistent therapeutic experiences [2] , [3] . Full autonomy [Figure 1(b)] indicates that the robot makes decisions and adapts its actions to any situation by itself. This is not feasible at this point because the robot's actions must be compliant with the therapeutic goals, interaction context, and state of the child, while its action policies cannot be perfect. Furthermore, fully autonomous robotic systems can raise some critical ethical concerns and are not socially accepted by the general public in the context of interacting with children [9] , [10] . However, a "supervised autonomy" in which the robot works independently toward achieving given therapeutic goals under a supervisor's guidance, is achievable, as shown in Figure 1 (c). When necessary, the supervisor can override the robot's actions before execution to ensure that only therapeutically valid actions are executed. The DREAM project aims at implementing RET, the next generation of RAT, for children with ASDs. This approach calls upon a robot's ability to assess a child's behavior by inferring the child's psychological disposition and mapping the behavior to appropriate actions within specified constraints under the supervision of a therapist (i.e., supervised autonomy). Thus, the therapist is not replaced, but rather takes full control of the therapeutic environment with an effective tool, acting as a mediator [3] . This article examines this developing technology and validates a supervised autonomous robotic system for ASD therapy. The project's consortium includes cognitive scientists, roboticists, computer scientists, psychotherapists, and ethicists who are collaboratively involved in the development of the system and follow requirements from different perspectives. The system is validated in a clinical study that assesses the effectiveness of socially assistive robots in enhancing social skills, e.g., imitation, turn taking, and joint attention. Most importantly, no full-scale randomized clinical trials have been carried out in previous research, which has been one of the main goals of the DREAM project. In this article, we also investigate therapists' attitudes toward the DREAM system as well as ethical issues related to using (supervised) autonomous robots in ASD therapy. Using robots in social therapies requires a highly interdisciplinary collaboration. In the DREAM project, all parties (i.e., psychotherapists, engineers, and ethicists) have been involved throughout the system development process in a concurrent manner. A robotic system used in RET should meet the requirements from both therapeutic and robotic perspectives. Key elements of the requirements are illustrated in Figure 2 and summarized in this section as follows [11] . First, the system should enable the robot to generate taskbased social behaviors to achieve therapeutic goals, which is the ultimate goal of using robots in therapeutic contexts. Second, the robot control should be shared with human therapists to ensure safe and ethical behaviors. Third, the system should be applicable to various therapeutic scenarios and robot platforms that reduce engineering workload, e.g., reprogramming a robot' s actions. Lastly, the system should analyze data (e.g., the user' s performance history and robot operation) recorded in structured forms and provide it to different parties. These requirements serve as guidelines and evaluation criteria for RET systems. Some of the system design principles used to obtain these requirements are 1) multilayered behavioral organization (for generating task-based and social behaviors), 2) personalization (for providing personalized interaction), and modularity (to help with applying the system to different scenarios and robot platforms [12] ). During development of the DREAM project, we adopted some of these design principles to establish a supervised autonomous system for different tasks in autism therapy (see the "Supervised Autonomous System" section). To assess socially assistive robots' effectiveness in enhancing social skills in children with ASDs, certain behaviors have been frequently targeted by therapeutic interventions. Among them (and for the specific goal of the DREAM project), we have targeted the following behaviors: imitation, turn taking, and joint attention. These behaviors, including communication and social interaction deficits, could be considered as possible mechanisms that underlie the general clinical picture and will be taught by a social robot during repeated therapy sessions of interactive games. Maneuvering a robot to deliver a therapy is a complex task, and in the case of supervised autonomous RET, it requires procedures that 1) sense the state and performance of the child and 2) select and execute an action for the robot (according to a therapeutic plan) while providing oversight of the robot's behavior to the therapist. This process is engineered by an interconnected network of components, as shown in Figure 3 . These components are responsible for sensing and interpreting the surrounding environment, classifying a child's behavior, and controlling robot behavior. The system also provides an intuitive graphical user interface (GUI), which allows the therapist to supervise the system operation and ensures efficient robot behavior. All system of the components were released [28] under the GNU General Public License v3 and documented, which allows researchers to replicate, modify, or expand the DREAM system for different target applications. An advanced sensory system translates multisensory data into meaningful information about the child-robot interaction, e.g., a child' s movement, gaze, vocal prosody, emotion expression, and typical ASD behaviors. Different techniques have been applied to raw images captured by red, green, blue (RGB) cameras and Microsoft' s Kinect sensors for gaze estimation, skeleton joint-based action recognition, face and facial expression recognition, object tracking, and audio data processing. Gaze estimation is vitally important for identifying shared attention in child-robot interactions during joint-attention tasks. The challenges to gaze estimation that emerge during therapeutic sessions are related to head movement, illumination variation, and eyelid occlusion. Feature points on the face are located by a supervised descent method, which is based on the best view of the child' s face. The head pose is calculated by an object pose-estimation method. Iris centers are localized by a hierarchical adaptive-convolution method [see the red dots in Figure 4 Human action recognition, i.e., a child's actions during interaction, plays a key role in evaluating imitation tasks performed by the child. A novel skeleton joint descriptor that uses a 3D moving trend and geometry property is applied on skeleton data extracted from Kinect's depth sensors [Figure 4(b)] [14] . The descriptor is then used to recognize actions (e.g., waving, touching the head with two hands, moving the arms to imitate an airplane, or covering the eyes) by a linear support vector machine (SVM) classification algorithm. Facial expression recognition provides an understanding of the child's emotions, e.g., anger, disgust, fear, happiness, sadness, and surprise. This is achieved by using a frontalization method to recover frontal facial appearances from unconstrained nonfrontal facial images, followed by a local binary patterns feature-extraction method applied to three orthogonal planes to represent facial appearance cues. Finally, we applied an SVM to identify and classify those facial expressions [15] , achieving a recognition rate of 63.71% under real-life conditions. We found it very difficult to achieve a clear partition of emotions because children tend to exhibit a combination of emotions. However, we obtained better results than other state-of-the-art algorithms. Object tracking helps to observe the child's behavior regarding the toys (e.g., a plane, flower, and cup) on the intervention table. A blob-based Otsu object-detection method is first employed to detect the objects. Then, a Gaussian mixture probability hypothesis density tracker is used to effectively detect and track objects in real time, even when being occluded by hands [ Figure 4 (c)] [16] . Audio processing provides information for the robot to perform social attention tasks and evaluate the child's verbal response. Speech recognition and sound direction are based on Kinect's Software Development Kit. Voices from the therapist and the child are labeled by classifiers such as the Gaussian mixture model, and vector quantification in combination with mel-frequency cepstrum coefficients, and linear predictive coding features [17] .  With the goal of achieving a supervised autonomous system, the current behavior of the child must be appraised, which occurs in two phases ( Figure 5 ). During the first phase, data is collected from the sensory system and mapped onto the child's identified behaviors. This mapping is based on training and validation sets of child-robot interactions that were previously annotated by knowledgeable therapists. From this process, the classifiers provide the probability that each behavior, among all of them, is currently observed. These probabilities are used during the second phase, where, based on the interaction history, the system attempts to infer the child's level of engagement, motivation, and performance of the task at hand. This second phase relies heavily on the semantic interaction knowledge of the therapists to provide insights into expected patterns. Classifiers within this system aim at generating real-time annotations of a therapy session that therapists would normally create. Therefore, these autogenerated annotation files may be submitted to therapists for verification and are compared to existing annotations from therapists using standard interrater agreement measures. The outcomes of these classifiers are fed into the robot behavior controller, which enables supervised autonomous operation of the robot. Additionally, these classifiers may offer other benefits, such as being used as a diagnostic tool or relieving therapists of some of their burden. Similar developments have been published, however, as a binary classification (e.g., non-ASD versus ASD) [18] . Intermediate degrees of severity of ASD, ranging from "typical of the general population" to "severely atypical, " should be accurately identified. Within the DREAM project, we have begun the development of a diagnostic tool based on these classifiers using neurocomputational mechanisms, which can be used for learning a large number of dynamical patterns, known as conceptors [19] . The robot behavior controller enables the robot to generate task-based and social behaviors and share control with the human therapist in a supervised autonomous manner. The behavior generation is organized into three layers, i.e., attentionReaction, deliberative, and self-monitoring, as shown in Figure 3 . Behaviors and therapeutic scripts are abstract and nonrobot specific, and later translated into robotspecific motor commands. This allows the system to be platform and scenario independent. The entire system operation is supervised by a human therapist via a GUI (Figure 6 ). The attention-reaction system provides the robot with lifelike behaviors, e.g., eye blinking, micromotions, and gaze [20] , all of which are essential in social robots. In this system, status information coming from the sensing system is immediately acted upon with appropriate motor outputs. The system also enables the robot to react to the relevant stimulus in the surrounding environment by directing its gaze toward their source. This is achieved by a combination of perceptual-and task-related attention as well as a target-selection algorithm. The deliberative system is responsible for producing taskbased behaviors that follow therapeutic scripts defined by therapists. These scripts detail step by step the high-level desired behaviors of the robot. There are, however, occasions when that the interaction does not go as planned, and the proposed script-based action is not the most appropriate one to perform. For instance, if the child has a low level of engagement with the task, the script-following process is paused. The robot then seeks appropriate actions for reengagement and returns to the script-following process. In case the action autonomously decided by the robot is not proper, the therapist can deny the suggested action and manually select a more appropriate one. We have proposed a learning-from-demonstration method called supervised progressively autonomous robot competencies (SPARC), so that the robot can learn from the manual actions of the therapist and improve its suggested actions for the next interaction [21] . As shown in Figure 7 , SPARC aims at maintaining high level of performance throughout the interaction (e.g., in WoZ) while ensuring a light workload for the therapist (i.e., autonomous learning). The self-monitoring system attempts to overcome possible technical and ethical limitations. This system currently acts as a logging mechanism and is connected using the therapist's supervisory interface. The therapist can overrule the robot's proposed actions via the GUI. In future applications and based on a set of rules, it would act as an alarm system that is triggered when the robot detects technical limitations and ethical issues. This system also provides recorded data (e.g., a child's performance and the robot's operation) for therapists and engineers to evaluate the efficacy of a RET system. From a clinical perspective, this project seeks to determine how much RET can improve joint attention, imitation, and turn-taking skills in ASD children as well as how the gains obtained within these interactions compare to standard interventions. Therefore, the clinical experiments were divided into two phases: one using RAT robots under a WoZ system, and another using RET within a supervised autonomous system. Both phases have been compared to standard human treatment (SHT) conditions. The experiments were conducted using a classical singlecase alternative treatment design. Children participated in six to eight baseline sessions, followed by eight SHT sessions and eight WoZ or RET sessions. Within the baseline sessions, the child interacts with a human partner who does not offer any feedback regarding the child's performance. The purpose of these sessions is to identify the initial level of skills and their variability before the child receives any of the two interventions (i.e., SHT or RET, where either the human or robotic partners give feedback that is based on the child' s performance). The conditions were randomized to mitigate the ordering effect. After the baseline sessions, the order for each intervention session (either SHT or RAT/RET) was established based on a random schedule that contained a random sequence indicating which session should be performed next. The schedule was different for each child. Before the baseline session, we used the Autism Diagnostic Observation Schedule (ADOS) instrument [22] to confirm children's diagnosis of autism and assess which were their social and communication abilities. We also employed ADOS as a measurement tool to quantify-before and after interventions-the differences in the scores. After the initial ADOS measurements were taken and the baseline session completed, children interacted with either a robot or a human, with an additional person always acting as a mediator between the child and interaction partner. The tasks to be tested were implemented following a discrete trial format, i.e., within a highly structured environment, the behaviors broken into discrete subskills, and a child taught to respond to explicit prompting (e.g., "Do you like me?"). We employed the humanoid robot NAO [29] to assess our hypothesis. For certain tasks, we used the electronic Sand Tray therapy kit [23], a 26-in capacitive touchscreen and associated control server where images can be manipulated by dragging (on the side of the human partner) or simulated dragging (on the side of the robot partner). Moreover, an intervention table was designed to capture sensory information (shown in Figure 8 ) by employing three RGB cameras and two Kinect sensors. Children's performance of the task was assessed by measuring their performance based on task-solving accuracy (e.g., accuracy during the imitation task, correct gazing during the joint-attention task, and appropriate pauses during the turn-taking task). As stated previously, during the first phase of the experiments we used RAT robots remotely controlled under a WoZ setup. Results from these experiments were used as a basis for the developing the supervised autonomous system. During this phase, 11 participants with ASDs between the ages of three and five years were recruited from the Autism Transylvanian Association. After completing this first phase, we obtained mixed results. These results were different depending on the task at hand. During the turn-taking task, the WoZ setup seemed to achieve gains as good as or even better than under SHT conditions, especially for children with low skill levels. Regarding joint attention, RAT and SHT yielded similar outcomes for the majority of participants. Specific to the RAT intervention, the results also suggest that the level of prompting offered by the robot mediator has a direct impact on the performance of ASD children, with more prompts resulting in improved performance [24] . For the imitation task, most of the children previously demonstrated good performances in baseline sessions, and the RAT condition did not enhance these skills. In the second phase of the experimental investigations, we compare the efficacy of RET and SHT using a randomized clinical trial design [30] . For this purpose, 27 children were recruited to participate from different organizations and institutions (most of which are located in Cluj-Napoca, Romania) that provide educational and/or psychotherapeutic services to children with autism. To date, 21 of the participants have completed the full protocol. The last-observation-carried-forward scores indicated that both groups showed signs of improvement within a significant time effect, i.e., Wilks' lambda = 0.62, F(8,16) = 3.19, p = 0.023, and . , 62 h = However, posttest differences between groups were not significant for joint attention, even when controlling for baseline scores. In future studies, this outcome will be carefully monitored and more children and sessions may change it in the expected direction. Both interventions also had a positive impact on the clinical ASD symptoms, with children in both groups who had completed the final assessment reporting lower ADOS severity scores at the end of the treatment, i.e., t(5) = 3.50, p = 0.017 in SHT, and t(4) = 3.25, p = 0.031 in RET. We conducted an interview to determine therapists' attitudes toward the DREAM system. Four therapists that have been working with the system for an average of six months were interviewed via email. During the interview, we used open-ended questions and a short usability survey. A screenshot of the GUI was used to elicit memories about their experiences with the system. The DREAM system was generally appreciated by the therapists (n = 4). According to the questionnaire results (a fivepoint Likert scale), the therapists showed positive attitudes toward the system, i.e., useful (M = 4.1, Min = 4.0, and Max = 4.1), satisfying (M = 3.6, Min = 2.4, and Max = 4.0), easy to use (M = 4.2, Min = 4.2, and Max = 4.4), and easy to learn (M = 3.8, Min = 2.0, and Max = 5.0). They expressed that the interface of the GUI is easy to use and helps them deliver an intervention that is both attractive and effective. According to the interview results, the automatic detection of behaviors was useful in treating children with ASDs because it reduced the therapists' potential burden of intervention. They also found the system to be safe and acceptable. Yet, some improvements are needed; e.g., increasing the accuracy of recognition, reducing the technical complexity of the system, and simplifying the GUI. Regarding the possibility of using the system for other types of therapies, they suggested having customized GUIs for different applications. Lately, research in the ethics of social robotics has increased significantly as it pertains to health care and children [25] . What are the specific problems raised by autonomous interaction with mentally disabled children? How can we protect children from exploitation? What if the robot gets the behavior assessment of the child wrong? How and when does the therapist need to overrule the behavior of the robot when needed? These questions all raise important ethical concerns. Within the DREAM project, we have conducted several studies to explore these and other ethical issues. In one of the studies, Coeckelbergh et al. [9] attempted to understand the opinions of parents and therapists about the appropriateness and benefits of social robots being used in therapy for children with ASDs. An important finding was the high acceptability of these robots for helping children with autism (85%). During the study, among the 416 subjects, 22% were parents of children with ASDs and 16% were therapists or teachers of children with ASDs. They were surveyed with questions such as "Is it ethically acceptable that social robots are used in therapy for children with autism?" or "Is it ethically acceptable to use social robots that replace therapists for teaching skills to children with autism?" This survey indicated the importance of stakeholder involvement in the process with a focus on specific health-care issues. In another study developed within DREAM, Peca [10] explored whether age, gender, education, previous experience with robots, or involvement with persons with ASDs influences people's attitudes about the use of robots in RET. Results show that these socialdemographic factors have a relevant impact on how social robots are perceived, e.g., men seem to have a higher level of ethical acceptability compare to women, younger participants seem to be more open to accepting the use of social robots in RET for ASDs compared to older participants. In terms of the involvement of the participant with children with ASDs and the use of social robots in RET, the study suggests that parents who are not involved directly with ASD children have a higher ethical acceptability level than those who are directly involved. Finally, Richardson et al. [26] have addressed a debate that discusses the risks and challenges of developing research by a multidisciplinary research team with a vulnerable population, such as children with autism. Given the different backgrounds, research goals, assumptions, and practices, each multidisciplinary research team would approach the research topic from different perspectives, i.e., experimental, clinical, engineering, philosophical, and anthropological. Each discipline has its own history, terminology, methods, and preferences, therefore, synthesizing these approaches can be challenging. With the DREAM project, we attempted to implement RET in children with autism interventions. In this article, we have highlighted the technical development and clinical validation of this approach. Given the sensitive environment where RET is utilized, the DREAM system was developed by taking into account the requirements from both therapeutic and robotic perspectives (see the "Requirements for RET Systems" section). The supervised autonomous system follows a multilayered behavioral organization for generating task-based and social behaviors. It was engineered by following a modular approach so that, along with being an open source software, the system may be easily used, adapted, and/or extended by other research teams for use with different therapeutic scenarios and robotic platforms. Our system reaches a performance on par with human therapies commonly used today in clinical studies. Despite the mixed results obtained during the single-case experiments, these studies offered valuable insights into the variability of the response of ASD children to RET and pointed to some important issues that should be accounted for when developing such interventions (e.g., the need for personalized interventions that match each child's skill level). Further exploration must be done using variables involved in outcomes, such as social engagement, positive and negative emotions, adaptive and maladaptive behaviors, and rational and irrational beliefs [27] . These variables have previously been studied for the first phase of the clinical trial. In terms of social engagement, the children showed more interest in the robot partner for the duration of the intervention. Positive emotions appeared more often while interacting with the robot during the imitation and joint-attention tasks. The presence of the robot usually acts as a behavioral activator, so that both adaptive and maladaptive behaviors seem to appear more often in the WoZ condition compared to the SHT condition. The same study is currently being done for the second phase, and it will offer better answers regarding the relative efficacy of RET and ASD interventions. Future investigations should aim for new research questions other than determining whether RET is more or less effective than standard treatments, e.g., is RET faster than therapist-mediated interventions and which children could benefit the most from RET interventions and under which conditions. As a final clinical conclusion and supported by data obtained during the experiments, we can say that RET is a promising approach that could be as efficient as (or even more efficient than) classical interventions for a large variety of outcomes for children with ASDs. Given its technical requirements, unfortunately, very few end users may benefit from the system developed in this project. For that reason, a simplified version of the DREAM system has been implemented as one of the Ask NAO [31] Tablet applications (Figure 9 ). Its functionality is as follows. The caregiver uses the application as an administrator (which allows him/her to monitor NAO's activities and access the control panel), and the child uses another tablet that can interact only with the information that NAO sends. This way, the caregiver never has to physically move away from the child and the robot to set up activities. This solution facilitates the ability of the caregiver (as an observer) to retrieve answers and send encouraging messages, while the robot is interacting with the child. Currently, 13 Ask NAO Tablet applications have been developed in the DREAM project. These applications will be tested on children with ASDs following the testing protocol created by our therapists. The results and feedback from therapists after this testing will be used to update the Ask NAO Tablet applications.
As stated by the authors, the technical requirements were too complex to be beneficial for most end users REF .
Robot-Enhanced Therapy: Development and Validation of Supervised Autonomous Robotic System for Autism Spectrum Disorders Therapy
IEEE Robotics & Automation Magazine
IEEE Robotics & Automation Magazine
Engineering Computer Science