# Development of IoT-Based Real-Time Fire Detection System Using Raspberry Pi and Fisheye Camera
## 3 Methods
### Deep-Learning-Based Fire Image Detection and Location Tracking  
The main purpose of this study is to estimate the location of the detected fire on a map, but, for this purpose, the technique of properly detecting the fire is also important. The desktop computer will access the images in the Firebase database sent from the Raspberry Pi to perform real-time forest fire detection. In order to accurately recognize fire in the image, You Only Look Once version 4 (YOLO v4), a deep-learning-based object detection  
Figure 3: (**a**) Polar image taken using a fisheye camera for a (**b**) transparent hemisphere with latitude indicated. Figure 2: (**a**) Concept of real-time forest fire detection system combining Raspberry Pi and fisheye camera, and (**b**) photo of the system mounted on a tripod. technology, was applied. YOLO v4 consists of three parts: backbone, neck, and head [31]. The backbone is a pretrained convolutional neural network (CNN) from input images. The backbone acts as a feature extraction network that computes a feature map. The neck consists of a spatial pyramid pooling (SPP) module and a path aggregation network (PAN) and connects the backbone and head. The neck concatenates the feature maps from different layers of the backbone network and sends them as inputs to the head, which then processes the aggregated features and classifies the objects. In this study, a pretrained CNN was used as the base network. In addition, transfer learning was performed to configure the YOLO v4 deep learning network for a new fire data set. For a data set consisting of about 2500 fire images, we labeled rectangular regions of interest (ROIs) for object detection and created a custom YOLO v4 detection network. Figure 4a shows a picture taken of real fire and red-colored objects using a fisheye lens, and it was confirmed that fire can be detected by distinguishing it from red-colored objects. Figure 4b shows a picture taken in a relatively dark environment, and it was confirmed that fire can be detected by distinguishing it from bright light. If the size of an object in an image is too small, detection may be difficult. Because objects are compressed in polar images, fire detection was performed by converting polar images into panoramic images for effective detection and location estimation. In this study, the center coordinates of the anchor box were extracted and used to calculate the geographic location of the detected fire. The purpose of this study is to specify the location of the fire point on the map. First, it is necessary to know the location of the developed system. For this, the location of the installed system was measured using GPS. Python code was written to calculate the azimuth angles from the center of the polar image to the center of the detected fire. In addition, the elevation angle of the fire can be calculated. If you know the azimuth angles, you can know the direction of the fire. However, if you do not know the elevation angles or topography of the target area, you cannot know the distance from the system to the  
Figure 4: (**a**) Fire detection in a picture containing fire and red objects (day), and (**b**) fire detection in a picture containing fire and bright lights (night). fire. In this study, DSM, which represents the height of the surrounding terrain, was used to calculate the distance to the fire point. Figure 5a shows an example of a contour map extracted from the DSM, and it is assumed that forest fires broke out at two points in this terrain. If the azimuth calculated from the polar image (Figure 5b) and the location of the system are known, the fire is located somewhere above the yellow line shown in Figure 5a. Figure 5c shows cross-sections from the observation point where the camera is installed to forest fires A and B. If you know the height of each point and the elevation angle obtained from the polar image, you can calculate the distance to the forest fires.