# Research and Conceptual Design of Sensor Fusion for Object Detection in Dense Smoke Environments
## 1 Introduction
Based on this sensor fusion, this paper contributes to enabling new perspectives and methodologies that can detect and visualize indistinguishable objects in dense smoke environments as follows: (1) 3D global position estimation for multiple objects through two types of 2D information and one type of 3D information, (2) application of image transformation theory to general data information transformation, (3) 3D position estimation of an object using 1D ultrasonic sensor information, (4) extension of the directional angle and detection area of the ultrasonic sensor system, (5) rejection \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Studies** & **Used Sensors** & & **Limitations** & **Advantages** \\ \hline \multirow{4}{*}{[11]} & \multirow{4}{*}{Ultrasonic sensor} & - & Impossible to measure 3D & \multirow{4}{*}{-} & \multirow{4}{*}{-} \\  & & position & & \\  & & Increase of measurement error & & & \\  & & with respect to temperature & & & \\  & & variation/gradient & & & \\ \hline \multirow{4}{*}{[12; 13]} & \multirow{4}{*}{Single thermal IR camera + Lidar + + UV sensor + normal camera} & - & Impossible to measure distance & \multirow{4}{*}{-} & \multirow{4}{*}{-} \\  & & & due to using only single camera & & \\ \cline{1-1}  & & & (Compensated by other sensors) & & \\ \cline{1-1}  & & & Other sensors: inappropriate to & & \\ \cline{1-1}  & & use in dense smoke & & & \\ \hline \multirow{4}{*}{[14; 15; 21]} & \multirow{4}{*}{Stereo thermal IR camera + Radar} & - & Ghost effect due to stereo & \multirow{4}{*}{-} & \multirow{4}{*}{-} \\  & & & misalignments & & \\ \cline{1-1}  & & & Full-resolution image processing & & \\ \cline{1-1}  & & time taken is long & & & \\ \cline{1-1}  & & & Object detection failure due to & & \\ \cline{1-1}  & & & high-temperature thermal & & \\ \cline{1-1}  & & & saturation & & \\ \hline \multirow{4}{*}{[27]} & \multirow{4}{*}{Stereo thermal IR camera + Radar} & - & Same to above & \multirow{4}{*}{-} & \multirow{4}{*}{-} \\  & & & - The longer distance, the bigger & \multirow{4}{*}{-} \\ \cline{1-1}  & & & the measurement error & & \\ \cline{1-1}  & & Long-distance accuracy increased & & & \\ \cline{1-1}  & & by radar & & & \\ \cline{1-1}  & & & Continued short-range object & & \\ \cline{1-1}  & & detection failure & & \\ \hline \multirow{4}{*}{[28]} & \multirow{4}{*}{RGB camera + Radar + Stereo thermal IR camera} & - & The same problems to & \multirow{4}{*}{-} & \multirow{4}{*}{-} \\ \cline{1-1}  & & & [14; 15; 21; 27] & & \\ \cline{1-1}  & & & [14; 15; 21; 27] & & \\ \cline{1-1}  & & & & in white smoke and & \\ \cline{1-1}  & & & low-temperature smoke & & normal environment \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Research on detecting objects in dense smoke and its limitations. of measurement error from ultrasonic sensors caused by temperature change in dense smoke, (6) object detection and classification through image processing from a thermal IR camera. As a result, the system presented in this paper can provide visualized information about objects in a dense smoke environment to firefighters or firefighting robots, as shown in Figure 1.  
The remainder of this paper is organized as follows. In Section 2, the theoretical methodologies of proposed sensor fusion are presented for object detection using ultrasonic sensors, FMCW radar, and thermal IR cameras. In Section 3, we describe the conceptual H/W and S/W design to implement the proposed methodologies. Finally, in Section 4, we draw conclusions and give an outlook to future works. Theoretical Methodology for Object Detection Using Ultrasonic Sensors, FMCW Radar, and Thermal IR Camera