Turn-taking is essential to the structure of human teamwork. Humans are typically aware of team members' intention to keep or relinquish their turn before a turn switch, where the responsibility of working on a shared task is shifted. Future co-robots are also expected to provide such competence. To that end, this paper proposes the Cognitive Turn-taking Model (CTTM), which leverages cognitive models (i.e., Spiking Neural Network) to achieve early turn-taking prediction. The CTTM framework can process multimodal human communication cues (both implicit and explicit) and predict human turn-taking intentions in an early stage. The proposed framework is tested on a simulated surgical procedure, where a robotic scrub nurse predicts the surgeon's turn-taking intention. It was found that the proposed CTTM framework outperforms the state-of-the-art turn-taking prediction algorithms by a large margin. It also outperforms humans when presented with partial observations of communication cues (i.e., less than 40 of full actions). This early prediction capability enables robots to initiate turn-taking actions at an early stage, which facilitates collaboration and increases overall efficiency.
Physical turn-taking refers to the process where a hybrid human-robot team take turns on a physical task. Turn-taking has been studied in robotics and HRI to regulate shared resources among team members. Those resources include time (i.e., only one person can work on the shared task at a time) and space (i.e., only one person can access the working space at a time). In the context of human robot interaction, the types and usage frequencies of various implicit communication cues have been studied in a robot-assisted assembly task @cite_13 . The timing in multimodal turn-taking (i.e., speech, gaze, and gesture) was investigated through a collaborative Towers of Hanoi challenge accomplished by a hybrid human robot team @cite_1 . Nonverbal cues for timing coordination in physical turn-taking were studied in manufacturing @cite_9 . Such research focuses on turn-taking process modelling and the robot control, while neglecting predictions of the turn-taking timing and intention.
@cite_1 @cite_9 @cite_13
2553961644 2011685492
Nonverbal communication cues play an important role in human-human interaction and are expected to take a similar role in human-robot collaboration. In current industrial practice, human-robot turn-taking is explicitly human controlled, via a command channel such as switch or button. However, such a master-slave approach does not permit collaborative interaction, and requires the human to focus on both controlling the robot's behavior and on the task, thereby affecting overall performance. In this paper, implicit, nonverbal communication cues are examined as a non-explicit communication channel during a turn-taking task context. The aim of this study is to characterize the types and frequencies of nonverbal cues important to regulating turn taking during an assembly-task-type collaboration. This analysis will guide the selection of cues that can be expressed by the robot as implicit user inputs while human and robot complete a shared task. The goal of this work is to develop computational models of social intelligence that enable robots to work side by side with humans, solving problems and achieving task goals through dialogue and collaborative manipulation. A defining problem of collaborative behavior in an embodied setting is the manner in which multiple agents make use of shared resources. In a situated dialogue, these resources include physical bottlenecks such as objects or spatial regions, and cognitive bottlenecks such as the speaking floor. For a robot to function as an effective collaborative partner with a human, it must be able to seize and yield such resources appropriately according to social expectations. We describe a general framework that uses timed Petri nets for the modeling and execution of robot speech, gaze, gesture, and manipulation for collaboration. The system dynamically monitors resource requirements and availability to control real-time turn-taking decisions over resources that are shared with humans, reasoning about different resource types independently. We evaluate our approach with an experiment in which our robot Simon performs a collaborative assembly task with 26 different human partners, showing that the multimodal reciprocal approach results in superior task performance, fluency, and balance of control. Humans naturally use the multiple modalities of speech, gesture, and gaze when they communicate. They also engage in turn-taking to manage the seizing and yielding of the speaking oor, a process that controls the execution of turns comprising discrete speech, gesture, and gaze events. We are interested in modeling such interaction processes for a social robot to use that can be transferred between dierent domains. Timed Petri nets (TPNs) are currently uncommon in human-robot interaction (HRI) but oer an attractive representation for modeling concurrency and synchronization when controlling behavior for multiple modalities. Their representation also permits the intuitive and modular combination of rule-based behavior expression with statistical timing models. We describe their utility in application to multimodal interaction and compare them to nite state In the real world, noisy sensors and limited communication make it difficult for robot teams to coordinate in tightly coupled tasks. Team members cannot simply apply single-robot solution techniques for partially observable problems in parallel because they do not take into account the recursive effect that reasoning about the beliefs of others has on policy generation. Instead, we must turn to a game theoretic approach to model the problem correctly. Partially observable stochastic games (POSGs) provide a solution model for decentralized robot teams, however, this model quickly becomes intractable. In previous work we presented an algorithm for lookahead search in POSGs. Here we present an extension which reduces computation during lookahead by clustering similar observation histories together. We show that by clustering histories which have similar profiles of predicted reward, we can greatly reduce the computation time required to solve a POSG while maintaining a good approximation to the optimal policy. We demonstrate the power of the clustering algorithm in a real-time robot controller as well as for a simple benchmark problem.