Unsupervised Online Grounding of Natural Language during Human-Robot Interactions (2020)

Abstract: Allowing humans to communicate through natural language with robots requires connections between words and percepts. The process of creating these connections is called symbol grounding and has been studied for nearly three decades. Although many studies have been conducted, not many considered grounding of synonyms and the employed algorithms either work only offline or in a supervised manner. In this paper, a cross-situational learning based grounding framework is proposed that allows grounding of words and phrases through corresponding percepts without human supervision and online, i.e. it does not require any explicit training phase, but instead updates the obtained mappings for every new encountered situation. The proposed framework is evaluated through an interaction experiment between a human tutor and a robot, and compared to an existing unsupervised grounding framework. The results show that the proposed framework is able to ground words through their corresponding percepts online and in an unsupervised manner, while outperforming the baseline framework.

## introduction an increasing number of service robots is employed in human-centered complex environments and interacts with humans on a regular basis. this creates a need for robots that are able to understand instructions provided in natural language, such as bring a glas of water or pick up a box, to execute them appropriately and thereby enable efficient collaboration with humans. to this end, connections between words, i.e. abstract symbols, and their corresponding percepts, i.e. meanings, need to be created because according to the "symbol grounding problem", which was proposed in 1990 by @xcite , abstract knowledge and language only has meaning, if it is linked to the physical world through mappings from words to corresponding percepts. grounding approaches can in general be separated into supervised and unsupervised approaches. the former utilize the guidance of a human tutor, while the latter do not require any supervision and try to use co-occurrence information to identify through which percepts a word is grounded. previous studies, such as @xcite @xcite , that investigated unsupervised grounding employed algorithms that only work offline, i.e. these algorithms need to be trained before deployment with in advance collected perceptual data and words, which prevents these algorithms from being used in real-time human-robot interactions. additionally, most previous studies did not consider ambiguous words, although the sentences humans produce are often ambiguous due to homonymy, i.e. one word refers to several percepts, and synonymy, i.e. one percept can be referred to by several different words. the latter do not need to be true synonyms, i.e. words that refer to the exact same meaning, instead, words only need to be synonyms as references to a percept in a particular set of situations, e.g. coca cola or lemonade instead of bottle. in this paper, a recently proposed unsupervised online grounding framework @xcite is extended to handle real percepts obtained during human-robot interactions. more specifically, the learning framework is extended to first convert obtained percepts through clustering to an abstract representation, which is then used to ground all non-auxiliary words 1 of the encountered natural language instructions through cross-situational learning. each shape, color, and action is referred to by at least two synonymous words, which need to be mapped to their corresponding geometric characteristics, color histograms, or kinematic features of the robot joints during action execution, to investigate the ability of the used frameworks to handle synonymous words. the grounding performance of the proposed framework is evaluated by comparing it to the grounding performance of a bayesian grounding framework that has been used in several previous studies, e.g. @xcite @xcite the rest of this paper is structured as follows: sections (2 and 3) provide a brief overview of crosssituational learning and related work. afterwards, an overview of the proposed unsupervised online grounding framework as well as the unsupervised bayesian baseline framework is given in sections (4 and 5). the experimental design and obtained results are described in sections (6 and 7). finally, section (8) concludes the paper. ## background cross-situational learning (csl) refers to the process of learning the meaning of words across multiple exposures to handle referential uncertainty. the basic idea is that a set of candidate meanings, i.e. mappings from words to percepts, can be created for every situation or context a word is used in and that the correct meaning can be obtained by determining the intersection of the sets of candidate meanings @xcite . thus, the correct mapping between a word and its corresponding percepts, i.e. its meaning, will reliably reoccur across situations @xcite . a number of experimental studies have confirmed that humans use csl for word learning, when no prior knowledge of language is available @xcite @xcite . since csl requires more than one exposure to learn a word, it belongs to the group of slow-mapping mechanisms through which most words are acquired @xcite . in contrast, fast-mapping allows words to be acquired through a single exposure, but it is only used for a limited number of words and can neither be explained nor achieved through csl @xcite . many different algorithms have been proposed to simulate csl in humans and enable artificial agents, such as robots, to learn the meaning of words by grounding them through percepts (section 3). ## related work grounding is used to obtain the meaning of an abstract symbol, e.g. a word, by linking it to perceptual information, i.e. the "real" world @xcite . there exist many different approaches for grounding. @xcite grounded higher level symbols through already grounded lower level symbols with the help of a dialog system. since the system requires a sufficiently large set of grounded lower level symbols as well as a professional tutor to answer its questions, its usefulness is limited. the need for a human tutor that knows the correct mappings also limits the applicability of the naming game, which allows an agent to quickly learn word-percept mappings @xcite . in contrast to the previous approaches, cross-situational learning (section 2), which assumes that one word appears several times together with the same perceptual feature vector so that a corresponding mapping can be created, does not require a human tutor for grounding @xcite . previous studies investigated the use of cross-situational learning for grounding of objects, actions, and spatial concepts @xcite . in most studies, grounding was conducted offline, i.e. perceptual data and words were collected in advance, which prevents these approaches from being used in realtime human-robot interactions. in contrast to these approaches, the framework used in this study learns the correct mappings from words to percepts online while interacting with humans and does not require separate training and test phases. additionally, the majority of employed models were not able to handle ambiguous words, although, the sentences humans produce are often ambiguous due to homonymy and synonymy. one recent study showed that grounding of known synonyms does not require semantic or syntactic information and that such information can even have a negative effect, depending on the characteristics of the used information and how it is applied @xcite . therefore, the online grounding mechanism employed in this study uses no additional semantic or syntactic information to ground synonyms. ## grounding framework the employed grounding framework consists of four parts: (1) 3d object segmentation component, which segments objects into point clouds to determine their geometric characteristics and colors, (2) action recording component, which creates action feature vectors by recording the states of several joints while the robot is executing actions, (3) percept clustering component, which obtains an abstract representation of percepts through clustering, and (4) cross-situational learning based grounding component, which identifies auxiliary words and maps percepts to non-auxiliary words and phrases. the inputs and outputs of the individual parts are highlighted below, and described in detail in the following subsections. ## baseline framework the baseline framework consists of three parts: (1) 3d object segmentation component as described in section (4.1), (2) action recording component as described in section (4.2), and (3) bayesian learning model, which identifies auxiliary words and grounds non-auxiliary words and phrases through corresponding percepts. since the perceptual data extraction components are the same for both frameworks, any difference in grounding performance can only be due to the different grounding algorithms, i.e. component three and four of the proposed framework (sections 4.3 and 4.4) and component three of the baseline framework, which is described in the remainder of this section. the probabilistic learning model, described in this section, is based on the model used in @xcite , since the experimental setup employed in this study (section 6) is also based on the scenario used in @xcite . in general, the model has been chosen as a baseline because similar models have previously been em-figure 2 : graphical representation of the probabilistic model. indices i, s, c, and a denote the order of words, object shapes, object colors, and actions, respectively. ## results and discussion the proposed cross-situational learning framework (section 4) is evaluated through a human-robot interaction scenario (section 6) and the obtained grounding results are compared to the groundings achieved by an unsupervised bayesian grounding framework (section 5). figure (4) shows how the mean number of correct and false mappings changes, when the proposed grounding framework encounters the employed situations one after the other. it also shows that all 45 correct mappings are obtained, when all 125 situations are used for training, while on average only 43 correct mappings are obtained, when only 60% of the situations are used for training. the figure also illustrates the online grounding capability of the model, i.e. that it updates its mappings with every new encountered situation, as well as its transparency because it allows to check at any time through which percept a word is grounded at that moment. based on the collected co-occurrence information it would also be possible to calculate a confidence score for every mapping to understand how likely it is that a false mapping disappears or a correct mapping persists. the described transparency of the proposed framework can be helpful to understand and debug responses to instructions provided by a human, when the framework is used to control an artifical agent interacting with a human, especially when the responses are incorrect or inappropriate. in contrast, the baseline model requires an explicit training phase so that no corresponding figure, illustrating the number of correct and false mappings, can be created. thus, to allow a comparison between the two models, the mappings of the proposed model are extracted after 125 and 75 situations, depending on the used train/test split. two different train/test splits are analyzed in this study. for the first split, all situations are used for training and testing to see how well the frameworks perform when all test situations have been encountered before. for the second split, only 60% of the used situations are provided for training, while the remaining situations are used for testing. in this case, it is possible that some words never occur during training or only a limited number of times, e.g. once or twice. if a word does not occur during training, the proposed model is not able to obtain a corresponding mapping which leads to an accuracy of 0% as shown in figure (6c ) for the words coffee and sweets, which both only exist once in the dataset and are thus only present during train-ing or testing, but not both. the word accuracies shown in figure (6 ) were calculated by dividing the number of times a word was correctly grounded through the number of times the word was encountered during testing. similar to the proposed model, the baseline model was also not able to ground the words coffee and sweets correctly, when only 60% of the situations were used for training. however, the baseline model also seems to require in general a higher minimum number of occurrences to successfully ground words, since there are many words that achieved a mean accuracy of 0%, when only 60% of the situations were used for training (figure 6d ). @xcite shows that the proposed model achieves perfect grounding, when the same situations are provided for training and testing, which confirms that it is able to obtain all correct mappings as shown in figure (4 ). however, if only 60% of the situations are used for training and the remaining 40% unknown situations for testing the grounding accuracy drops for both models. for the proposed model the largest accuracy decrease is seen for auxiliary words, while still more than 95% of the obtained shape, color and action groundings are correct. for the baseline framework the largest drop in accuracy is seen for shapes, from more than 95% to less than 2%. the reason might be that every shape word has 5 synonyms, thus, if words would be equally distributed among all situations and specifically among the training and test sets, the decrease might not be as sharp. however, figure 3 shows that the number of occurrences is not necessarily the reason for the drop because the words bmw and narnia occured on average 7 and 2.5 times during training, respectively, and narnia achieved an accuracy of about 5%, while the accuracy of bmw was 0% (figure 6d ). in contrast, the proposed model shows a more stable performance, since it was able to ground all non-auxiliary words that occured at least one time during training with a mean accuracy of more than 70%, while only the auxiliary word please achieved a lower mean accuracy of 30%. overall the evaluation shows that the proposed model outperforms the baseline model based on its auxiliary word detection and grounding accuracy. interestingly, the performance difference is larger, when only 60% of the situations are used for training, although this scenario is artificially harming the proposed model by preventing it to learn during testing, since it does not require explicit training. in addition to the better grounding performance, the proposed model is also more transparent, which becomes important when robots are interacting with humans in complex and unrestricted environments, especially if some actions of the robots can cause harm to humans. ## conclusions and future work this paper investigated a multimodal framework for grounding synonymous shape, color and action words through the visual perception and proprioception of a robot during its interaction with a human tutor. the cross-situational learning model was set up to learn the meaning of shape and color words of objects as well as action words using geometric characteristics and color information of objects obtained from point cloud information as well as kinematic features of the robot joints recorded during action execution. the proposed model allowed auxiliary word detection and online grounding of synonyms through real percepts in an unsupervised manner and without the use of any syntactic or semantic information. additionally, it outperformed the baseline model based on the accuracy of the obtained groundings, its capability to process new situations online and its transparency. in future work, different mechanisms will be investigated to improve the sample efficiency of the algorithm, which will become relevant, if a larger number of words is used or words occur less often. additionally, it will be verified whether the framework can handle homonyms. finally, supervised grounding methods will be integrated so that the robot is able to use human feedback, but does not require it.