  The ability to develop a high-level understanding of a scene, such as
perceiving danger levels, can prove valuable in planning multi-robot search and
rescue (SaR) missions. In this work, we propose to uniquely leverage natural
language descriptions from the mission commander in chief and image data
captured by robots to estimate scene danger. Given a description and an image,
a state-of-the-art deep neural network is used to assess a corresponding
similarity score, which is then converted into a probabilistic distribution of
danger levels. Because commonly used visio-linguistic datasets do not represent
SaR missions well, we collect a large-scale image-description dataset from
synthetic images taken from realistic disaster scenes and use it to train our
machine learning model. A risk-aware variant of the Multi-robot Efficient
Search Path Planning (MESPP) problem is then formulated to use the danger
estimates in order to account for high-risk locations in the environment when
planning the searchers' paths. The problem is solved via a distributed approach
based on Mixed-Integer Linear Programming. Our experiments demonstrate that our
framework allows to plan safer yet highly successful search missions, abiding
to the two most important aspects of SaR missions: to ensure both searchers'
and victim safety.
