# Thermal, Multispectral, and RGB Vision Systems Analysis for Victim Detection in SAR Robotics
## Appendix 1```
Source: Authors. \begin{table}
\begin{tabular}{l l} \hline \hline
**Proposed Individual Evaluation Metrics** \\ \hline \(\alpha\) & Outdoors \\ \(\beta\) & Poor Light Conditions \\ \(\gamma\) & Indoors \\ \(\delta\) & Processing Time \\ \(\epsilon\) & Heat Sources Presence \\ \(\theta\) & Totally covered victim \\ \(\phi\) & Partially covered victim \\ \(\omega\) & Clothes colour \\ \(\lambda\) & Summer/Fire conditions \\ \(\tau\) & Changing Light Conditions \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Quantitative metrics proposed for the evaluation of vision systems. \[ \begin{split} Score_{SAR}(Thermal-RGB-Mult)&=\alpha+1.75* \beta+\gamma+(100-\delta)+\\ 0.8*\epsilon+2*\theta+\phi+\omega+\lambda+1.5*\tau\end{split}  \tag{4}\]

## 4 Results and Discussion

### Mission Execution in Indoor-Outdoor Environments

To evaluate this study, the proposed metrics (\(\alpha,...,\tau\)), and the overall system presented, a series of missions were conducted indoors and outdoors, as synthesized in Figure 6. This figure illustrates various instances of traversal in different environments over distinct time intervals and perspectives on the progression of data collection phases. Figure 6: Indoors–outdoors exploration process at the University of Malaga and Center for Automation and Robotics facilities. Source: authors. Figure 6b-e depict the traversal sequence for the indoor scenario, spanning from time 0 to a median time of 200 s. A blue circle denotes the robot's position at each instant. In this scenario configuration, 20 tests were conducted, with five victims distributed throughout, including covered and partially covered cases. On the other hand, Figure 6e-h showcase the tunnel exploration carried out during the \"XVI Jornadas Internacionales de la Universidad de Malaga sobre Seguridad, Emergencias y Catastrofes\". This exercise was executed once in a single pass due to the logistical complexity involved. The figures highlight the individual explorations conducted by ARTU-R and Spot robots. Figure 6i,k provide the perspective captured by a UAV during the mission's development. In contrast, Figure 6j,l depict the corresponding thermal footprints captured by the aerial vehicle. Finally, Figure 6m,p illustrate the robot capturing images of a person acting as a victim for processing. These images were acquired from various perspectives assigned by the operator. ### System Performance Evaluation

#### 4.2.1 Evaluation of Victim Identification in SAR Missions Performed

Figure 7 illustrates the detections conducted in the missions of the scenarios presented in Figure 6. The figure highlights a notably high precision rate, exceeding an average of 87%, for identifying both victims and first responders. Figure 7a,d showcase the detection results with a high average precision rate exceeding 91% for indoor scenarios, employing thermal and multispectral images. Notably, the system demonstrates the effective detection of torsos and legs. On the other hand, Figure 7b,c,e pertain to outdoor scenarios, emphasizing bounding boxes that highlight various body parts of identified victims, achieving a precision rate exceeding fifty percent. #### 4.2.2 Individual Evaluation of Systems Using the Proposed Metrics

The metrics outlined in Table 3 have been evaluated based on victim detection quality data in the conducted missions. The evaluation considers the mean precision values (mAP)

Figure 7: Evaluation of three vision methods for victim detection in different scenarios. Source: authors. for inferences under the ten specified conditions across repetitions of missions indoors and subsequent evaluations outdoors. Figure 8 compiles the normalized percentage values from 0 to 100 obtained for each of the three types of images, according to the ten indices established in this study. This figure provides an intuitive and straightforward overview of the strengths of each image type relative to its counterparts.