# A Bayesian Network-Based Information Fusion Combined with DNNs for Robust Video Fire Detection
## 3 Proposed Method  
### Network Architecture and Timing Diagram  
We proposed a deep learning-based fire detection method, which imitates the human fire detection process, called DTA [3]. Using this method, we could discriminate fire or smoke from an evening glow, clouds, and chimney smoke, which are difficult to differentiate from still images. In this paper, the new extended architecture (DTAIF) can be divided into five components as shown in Figure 1.  
The first three sections are identical to DTA, i.e., Faster R-CNN for fire object detection in the video frames in the first section (Refer Section 3.2), the learned spatial features accumulation by LSTM and short-term fire decision in the second section, and the majority voting stage for long-term fire decision in the third section (Refer Section 3.4). In the extended approach of DTAIF, we added a location classifier (Refer Section 3.3) and smoke detector as the fourth component. The location classifier was based on the extracted features from the backbone ResNet152 structure of Faster R-CNN, which also provides smoke detection results. The categories of the locations examined are \"building,\" \"traffic (car, train, ship),\" \"street,\" \"forest or mountain,\" \"other places (bonfire, garbage, and so on),\" and \"irrelevant place for fire (sky, sea, and so on).\" Once it was established that the location was one that has a probability of fire (i.e., all locations except \"irrelevant place for fire\"), the input was treated as a dangerous place in the corresponding node of the Bayesian net. Location identified in every frame but the averaged probability of each location was taken throughout the video. The smoke detection state was \"_yes_\" if more than half the frames include smoke objects during the period. In the fifth section of Bayesian net (Refer Section 3.5), all available information was combined to make the final decision regarding the presence of fire. There were two groups of information sources in this section of the Bayesian net: visual analysis of a video clip and other sensors. The results of the majority voting for a long-term decision, the location classifier, and the result of the smoke detector from Faster R-CNN were based on the information obtained from video analysis. Many different sensors may have aided in fire detection, measuring temperature, wind speed, and humidity. However, the temperature is weakly related with the fire occurrence, and the wind speed affects the spreading of a fire after breaking. Therefore, we only considered humidity in our simple Bayesian network. Note that the humidity is not only a critical factor of a fire, but also gives an important clue to prevent false fire-decision from cloud, haze, mist, or fog. Because we did not gather enough concurrent data to learn the Bayesian net, the structure and its associated conditional probability tables were reasonably presumed. Additionally, the specific domain knowledge where the fire detection system operates could be successfully considered in the Bayesian net. The timing of the proposed method was important to install in a real system and could be adjusted depending on the situation where it operates. Figure 2 presents a timing diagram that shows the decision period for each block. All timing periods were the same as in the previous paper [3], except the timing for Bayesian inference. The fire objects with smoke were detected for each frame of video, and the CNN spatial features of Faster R-CNN were temporally accumulated for a period to make a short-term decision. (Refer  
Figure 1: The proposed network architecture [26]. Section 3.2) The short-term fire decision for every \(T_{LSTM}\) was involved in the majority voting process for every time-period \(T_{vot}\)., which implies that the fire decision based only on visual analysis of a video clip was repeated for every \(T_{vot}\).