# Research and Conceptual Design of Sensor Fusion for Object Detection in Dense Smoke Environments
## 1 Introduction  
According to several reports [1; 2; 3; 4; 5], about 60% of fire accidents occur in indoor environments, and more than 70% of firefighter deaths and injuries are caused by smoke inhalation, burns, and being trapped in indoor fires. In particular, extremely poor visibility from dense smoke triggers mental panic in firefighters and limits even their normal behavior [6; 7; 8; 9]. Typically, the visibility in dense smoke is less than 1 m. Therefore, these facts motivated research on visual aid systems based on a sensor fusion that can provide useful visualization information to firefighters, as shown in Figure 1.  
Various robotic and visual assistance platforms have been developed and studied to enable firefighters to successfully perform their missions in these situations [10; 11; 12; 13; 14; 15; 16; 17; 18; 19; 20; 21; 22; 23; 24; 25; 26; 27; 28]. Figure 1: Motivation: Provide visual information to firefighters: (**a**) normal environment (everything can be seen), (**b**) object detection through sensor fusion to visualize objects in a dense smoke environment (nothing can be seen). Most of these studies on firefighting robots and firefighter visual aids have utilized visual and IR night vision cameras for perception sensors. Due to their short wavelength, these sensors are not suitable for use in dense smoke environments. In addition, the camera is not suitable for the high temperatures of dense smoke. Therefore, in those studies, a temperature sensor was used to compensate for these shortcomings. Dense smoke contains various combustion gases such as CO\({}_{2}\), CO, H\({}_{2}\)S, SO\({}_{2}\), NH\({}_{3}\), HCN, COCl\({}_{2}\), and HCl. In addition, because these gases combine with water vapor, the absorption, reflection, and scattering properties of radiant energy in dense smoke differ from normal environments, and they are also very different from the characteristics of white smoke. The dense smoke also creates a high temperature gradient in an indoor air space. In these conditions, a sensor with a longer wavelength is advantageous for environmental perception. Therefore, thermal IR cameras, FMCW radars, and ultrasonic sensors are more suitable for use in dense smoke environments than visual and/or IR night vision cameras because of their longer wavelengths, as shown in Figure 2[29]. For these reasons, several studies have attempted to detect objects in dense smoke environments by combining thermal IR cameras, FMCW radars, or ultrasonic sensors, individually or in combination with other general environmental sensors, as shown in Table 1. However, these studies are insufficient for firefighting support due to the limitations and characteristics given in Table 1 and Figure 3[30]. Figure 3: Characteristics of suitable sensors for use in dense smoke: Thermal IR cameras, Ultrasonic sensors, and Radars. Figure 2: The relationship between visibility and wavelength of different perceptual sensors. Additionally, although there are several studies on sensor fusion for firefighting robots [31; 32; 33; 34; 35], these studies are related not to object detection in dense smoke environments but to flame detection or navigation in a general fire environment. In other words, studies on sensor fusion for object detection in dense smoke environments are very rare. As an alternative to existing studies, we propose the novel concept of a sensor fusion system that can provide 3D position information on objects with a visibility of less than 1 m. This system uses a single thermal IR camera, a single FMCW radar, and multiple ultrasonic sensors simultaneously to maximize the strengths of each sensor, as shown in Figure 2, as well as compensate for its weaknesses.