Abstract-Multimodal attention is a key requirement for humanoid robots in order to navigate in complex environments and act as social, cognitive human partners. To this end, robots have to incorporate attention mechanisms that focus the processing on the potentially most relevant stimuli while controlling the sensor orientation to improve the perception of these stimuli. In this paper, we present our implementation of audio-visual saliency-based attention that we integrated in a system for knowledge-driven audio-visual scene analysis and object-based world modeling. For this purpose, we introduce a novel isophote-based method for proto-object segmentation of saliency maps, a surprise-based auditory saliency definition, and a parametric 3-D model for multimodal saliency fusion. The applicability of the proposed system is demonstrated in a series of experiments. Index Terms-audio-visual saliency, auditory surprise, isophote-based visual proto-objects, parametric 3-D saliency model, object-based inhibition of return, multimodal attention, scene exploration, hierarchical object analysis, overt attention, active perception
To this end, we introduced auditory surprise in REF , which is based on the accumulated Bayesian surprise [19] of all frequencies.
Multimodal saliency-based attention for object-based scene analysis
2011 IEEE/RSJ International Conference on Intelligent Robots and Systems
2011 IEEE/RSJ International Conference on Intelligent Robots and Systems
Computer Science