Learning Word Groundings from Humans Facilitated by Robot Emotional Displays (2020)

Abstract: In working towards accomplishing a human-level acquisition and understanding of language, a robot must meet two requirements: the ability to learn words from interactions with its physical environment, and the ability to learn language from people in settings for language use, such as spoken dialogue. In a live interactive study, we test the hypothesis that emotional displays are a viable solution to the cold-start problem of how to communicate without relying on language the robot does not–indeed, cannot–yet know. We explain our modular system that can autonomously learn word groundings through interaction and show through a user study with 21 participants that emotional displays improve the quantity and quality of the inputs provided to the robot.

## introduction in any first language acquisition task, three questions must be resolved: to answer the first question, we note that colocated spoken dialogue interaction is the fundamental setting of first language acquisition for humans @xcite and that children generally tend to focus on physical objects first, as evidenced by age-of-acquisition datasets. for this reason, concrete words that denote physical objects are learned earlier than abstract words @xcite . this informs the answer to the second question: the model of semantics should be able to connect language with the physical world, which is part of the goal of grounded semantics (e.g., grounding a color word like green with visual information). this still leaves the third question: how can a system learn word groundings in a physical, colocated setting without using words it has yet to learn? in answering this, there is evidence that having a physical body is a requirement for bootstrapping semantic learning of concrete word denotations @xcite . therefore, a system that can use extra-linguistic cues through physical signals can potentially overcome the cold-start problem and learn words without uttering words it has never heard. in this paper, we test the hypothesis that emotional displays, specifically confusion and understanding displays performed by an embodied robot, are a viable solution to the cold-start problem. our reasons are two-fold: emotional displays can relate the robot's state to its human teacher, and emotional displays are developmentally appropriate for the most common language acquisition setting (i.e., an adult teaching a child) @xcite , and would therefore not lead a human user to make incorrect assumptions regarding the robot's level of comprehension. in an interactive study with 21 participants, our robot independently and autonomously explored a physical setting and elicited relevant word references and feedback from the participants, who were tested both with a robot that displayed emotions and a robot that did not. for grounded semantics, we opted for a model that is incremental (i.e., operates at the word level), that can map individual words to physical features, and that can learn a mapping between a word and physical features using only a few examples-the words-as-classifiers model (wac) @xcite . in the wac model, each word is represented by its own classifier trained on "not / is" examples of real-world referents. the wac model has been used in interactive dialogue scenarios with robots before @xcite . importantly, our system not only learned word groundings as it interacted with participants, it also incorporated a reinforcement learning model to learn from positive or negative participant feedback which emotional valence (either understanding or confusion) to display. analyzing the results from the surveys and the learned wac classifiers, we discovered that the use of emotional displays improved the quantity and quality of the inputs provided to the robot, with the effect modulated by the valence and frequency of the emotional displays. ## background & related work it has been shown that people assign anthropomorphic characteristics, social roles and models when interacting with robots @xcite , which has implications for the kinds of settings and tasks that robots can carry out with human collaborators. one dimension that people anthropomorphically assign to robots is emotion. we cannot prevent users from making emotional judgements of a robot's behavior @xcite . instead, if a robot's behavior were designed to take these emotional judgements into account, the robot could be made more predictable and more interpretable by humans in a complex environment @xcite . indeed, emotional features can make a robot appear more lifelike and believable to humans, thereby making humans more prone to accept and engage with them @xcite . of course, the choice of emotions must be taken with care; @xcite showed that happiness and sadness emotional displays during primary tasks (e.g., such as transporting an object) could confuse human interlocutors as robot actions (e.g., jerkiness, activity, gaze), and robot movement are also emotionally interpreted. similar to conversational grounding, jung (2017) explained how affective grounding-the coordination on content and process of affect-occurs between robots and human users. we handle this particular phenomenon by only considering a positive and negative valence of a single affective type (i.e., confusion vs. understanding), and by establishing through an evaluation that they are indeed interpreted the way we expect before we use them in a language learning task. robots have been used in many language grounding tasks; @xcite gives an overview of the recent literature. in some cases the cold-start problem is handled by wizard-of-oz paradigm studies where a robot that knows no word denotations interacts with human participants, but the robot is in fact being controlled by a confederate. in this paper, our robot is fully autonomous and has no pre-programmed language production capabilities; that is, the robot will never utter words it hasn't encountered within an interaction. beyond word learning, our approach attempts to ground language and learn which emotions to display. this work builds on @xcite which outlined the approach we take for a reinforcement-learning based on "polarized user appraisals gathered throughout the course of a vocal interaction between a machine and a human". their work outlined the design of a hypothetical experiment; we have taken this a step further by actually implementing this design in a live interactive study. we take user feedback to be the explicit reward signal (those user inputs that match the explicit positive or negative feedback). like their work, our approach does require a lengthy explore phase at the outset. ## system in this section we explain our choice of robot, and how we modeled the dialogue for language learning with integrated robot modules. choice of robot: anki cozmo cozmo is small, has track wheels for locomotion, a lift, and a head with and oled display which displays its eyes. the head has a small camera and a speaker with a built-in speech synthesizer (with a "young"sounding voice). with a python sdk, we can easily access cozmo's sensors and control it. importantly for our study, we will make use of cozmo's camera for object detection, human face recognition, and locomotion functionality for navigation between objects. cozmo does not have an internal microphone-we make use of an external one. the choice of robot affects how humans will treat it, and it is important for our study that users perceive the robot as a young language learning child. we opted for the anki cozmo robot because @xcite showed that participants in their study perceived cozmo as young, but with potential to learn. cozmo's affordances are likewise consistent with this perceived age and knowledge-level. cozmo is also a good option for this work because it has been recently demonstrated that humans perceive the same emotions and positive or negative valences from cozmo's over 940 pre-scripted be-haviors @xcite . taken together, these studies show that (1) we can safely assume that human participants will treat cozmo at an appropriate age level, and (2) we can assume that human participants will properly interpret cozmo's behaviors as displays of emotion. indicating objects if cozmo is to learn denotations for physical objects, then objects must be present in the environment that cozmo and a person share. also, the person needs to be able to identify the object that cozmo is attending to. once these requirements are met, then cozmo can learn the correct denotations for objects. noting that @xcite has been able to successfully use deictic gestures to isolate objects, we assume participants will denote objects that the robots are already attending to, which is what adults do for children learning their first language @xcite (that is, the perspective cozmo takes is egocentric). more practically, cozmo is small, which places its camera very low to the surface of the shared environment. therefore, cozmo must be very close to objects to "see" them through its camera, which effectively isolates objects without the need for deictic gestures from the robot. when cozmo does need to indicate an object, cozmo moves its lift up and down while directly in front of the object of intended reference. @xcite , cozmo needs to exhibit minimal "socially adept" behaviors if language learning is going to take place. we identify two behaviors that we incorporate into cozmo: (1) eye contact; that is, in certain states (e.g., cozmo is looking for feedback from the user) cozmo looks up and turns in place until it finds a face, and (2) motion; that is, cozmo must nearly always be moving-for several reasons, first to signal to an interlocutor that cozmo is still functional and second, children who are learning language rarely sit still. these random motions occur outside of the task actions (explained below) and give priority to those task actions when they occur. ## evaluation in this section, we explain how we evaluated our model with real human participants to determine if emotional displays increase engagement for language learning. we used two versions of our system: one which only performed the language learning task, and one which additionally included displays of emotion-the choice of which emotion was decided by a rl model. our evaluation included objective measures logged by the system, as well as subjective measures collected using participant questionnaires. ## conclusion we conducted an experiment with twenty-one participants who had to rely on the robot's displays of confusion and understanding and their own performance in a language acquisition task as context. we analyzed our results by comparing the participants' survey responses and the robots' grounded semantics classifiers between the experimental and control trials. we found that a robot that displayed a combination of confused and understanding emotional displays -positive-and negatively-valenced emotion -gathered more inputs, and more useful inputs (positive feedback), than a robot that only engaged in task-specific actions (i.e., orienting to objects; seeking out the user's face). this in turn led to the robot making more word proposals, which did not lead to greater engagement. user estimations of the robot were generally more positive estimations, supporting our choice of the anki cozmo robot for this task. emotional displays did not incline participants to over-estimate the robot's language understanding. we can conclude that emotion is an important aspect in handling the cold-start problem where a system can only use words it has heard. in future work, we will test different policies for the reinforcement learning regime including measures for novelty rewards (i.e., hearing new words) as well as repeated words. another aspect that demands further investigation would be the timing of emotional displays in the language learning interaction. importantly, we will go beyond the two basic emotions explored here and incorporate additional emotions (e.g., the 8 valence pairs used in mcneill and @xcite ) as the basis for additional engagement and perhaps use emotional states as features for the grounded classifiers.