Intuitive human-robot cooperation

This thesis results from research on human-robot cooperation within the context of the collaborative research center “Humanoid robots—learning and cooperating multimodal robots”. It presents a haptic interface called “tactile language” that was designed to provide an additional non-verbal interaction modality. Furthermore, a method for proactive planning and execution of robot tasks based on the estimated human intention was devised. In order to integrate both methods into the existing simple robot control, an adequate robot architecture was conceived and implemented that comprises the existing robot control augmented by the additionally required architectural components. The main structure of the robot architecture consists of three layers, where the upper two layers consisting of task planner and execution supervisor represent the main new contributions. The job of the task planner is to select the task that should be executed next. It was designed such that the robot can receive commands from all man-machine interfaces on the one hand, and from the proactive execution module on the other hand. To that end, an algorithm was defined to select the appropriate command in any situation. After the selection of a command, the execution supervisor ensures the correct execution of the task. The tactile language developed in the course of this research allows a human to control the robot via its artificial skin or a touchpad. The processing system of the tactile language can be divided in three sections: First, it comprises the sensor interfaces and the tactile image processing; second, the recognition of the input symbols and characters of the language; and third, the interpretation of the input and the connection to the robot control. The tactile language offers the user four different input modes to choose from: The first two are a direct and an indirect control mode to move and rotate the robot’s TCP and head. The third is an abstract mode where symbolic, complex robot commands can be issued by entering appropriate multifinger symbols. Furthermore, by integration of a freely available character recognition software, the user can enter handwritten alphabet characters. Lastly, there is a button mode that works like a touchscreen with active regions on the tactile surface that directly issue robot commands. The tactile language is very expressive since multiple parameters like direction, distance, and speed can be decoded from a single human finger stroke which can be used as attributes to the input symbol. Moreover, it is approximately real-time capable, thus making it possible to control the robot directly and recognize symbols instantly as well. The interpretation is done by a deterministic finite automaton that defines the tactile language and checks whether or not a tactile input makes sense. In case of a valid input the data is transmitted to the robot control. A traditional service robot is controlled exclusively by explicit commands from the human. The paradigm of “proactive execution” that is newly established in this thesis contrasts this with an implicit command mode that aims at facilitating a more intuitive human-robot cooperation. Based on the estimated intention of the human cooperation partner the robot selects an appropriate action and starts to execute it unless the human has given an explicit command. In doing so, the robot risks selecting and executing the wrong action. Should this become evident, the robot thus aborts the currently proactively executed action and chooses another task based on the changed situation. The input of the module, the probability density distribution over the human intentions, is first filtered to smooth it. Then the set Γ of possible intentions of all N intentions is determined by sorting them according to their probabilities and identifying the largest difference between them. The appropriate actions for the individual intentions are predefined by the system designer. They are assigned action tendencies depending on the cardinality of Γ. In the unambiguous case (|Γ| = 1) or the undecidable case (|Γ| = 0 or |Γ| > 3) the procedure is clear. The interesting case of ambiguity occurs for 1 < |Γ| ≤ 3. Instead of idling, an action is executed proactively. The goal is to minimize the uncertainty of the intention estimation and spare the user an explicit verbal intervention. In the proactive case, four criteria influence the action tendencies: The conditional entropy H(I|A) as a measure of the uncertainty of the system, the expected utility E×V of an action, the necessary safety requirements, and the most frequent action sequences so far. The action tendencies are accumulated each cycle in an action selection filter. Finally, the action is selected that exhibits the maximum accumulated action tendency.