A Multimodal HumanRobot Interaction Manager for Assistive Robots With rapid advances in social robotics, humanoids and autonomy, robot assistants appear to be within reach. However, robots are still unable to effectively interact with humans in activities of daily living. One of the challenges is in the frequent use of multiple communication modalities when humans engage in collaborative activities. In this paper, we propose a Multimodal Interaction Manager, a framework for an assistive robot that maintains an active multimodal interaction with a human partner while performing physical collaborative tasks. The heart of our framework is a Hierarchical Bipartite ActionTransition Network (HBATN), which allows the robot to infer the state of the task and the dialogue given spoken utterances and observed pointing gestures from a human partner, and to plan its next actions. Finally, we implemented this framework on a robot to provide preliminary evidence that the robot can successfully participate in a taskoriented multimodal interaction.

Reactive Interaction Through Body Motion and the PhaseStateMachine Between humans, body motion is an intuitive and ubiquitous means to coordinate interactions, but so far collaborative robots have not embraced this mode of communication. One reason for this is that conventional behavior generation systems use finite state machines which make it exceptionally difficult to ingest and produce the inherently continuous and concurrent information flow that body motion provides. We propose a new, reactive motion generator based on a dynamical system instead. It mimics a conventional state machine except that transitions are not instantaneous but timeextended, reversible, and have phases. Moreover, consecutive states and transitions may overlap during execution. Most notably, more than one transition can be active at the same time if they share predecessor state. Together, these unique features enable instantaneous and gradual responses to novel information and continuous decision processes without changing the state graph itself. We demonstrate the systemxe2x80x99s capabilities in an object handover task coordinated by body motion.