multimodal interface within a simulator of service robotic applications
The use of a multimodal interfaces for human-robot communication is an open issue, with a true market in service robotic applications. This paper describes a service robotic simulator which uses a multimodal interface (audio-video, speech-prosodic communication based) that demonstrates the usefulness of integrating many "modes" of interaction. A typical service robotic architecture is multi-layered: cognitive, deliberative, reactive layers are the most referred terms in literature. The design of each level requires an accurate testing methodology; the system development is thus accelerated and simplified using a simulator. The paradigm we take into account consists of robots and human operators continuously linked to a supervising centre. The operator's natural mode of communication is dialogue-based: so far the system we propose uses a prosody-based visuo-dialogical interface with the robot by means of keywords and prosody extracted from spoken commands. The simulator is based on this paradigm and allows to interact by means of conversational and visual interfaces. The former is composed by: speech recognition, prosody detection, keyword extraction, text to speech conversion; the latter by a visual interface and VRML 3D visualization.
multimodal interface service robotics robotic simulation prosody vocal dialogue