# A Bayesian Network-Based Information Fusion Combined with DNNs for Robust Video Fire Detection
## 3 Proposed Method
### Bayesian Network for Incorporation of all Information into Final Decision  
In general, people determine the existence of a fire not only based on its spatial and temporal behavior, but also about the place where it happens and environmental factors such as humidity and other weather conditions. For robust fire decisions, various types of data measured from diverse sensors can be combined with video data. We propose to use a Bayesian network to combine all this information to improve fire decisions. According to the types of sensors and the situations where they operate, however, there could be many domain-specific Bayesian networks. Therefore, we confined to the outdoor fires where only crude humidity information was assumed to be known. Due to the lack of domain-specific data, we tried to extract all visual information from video frames as much as possible, including the long-term fire decision, the place category, and the existence of smoke. If there was a sufficient number of concurrent data, the Bayesian network could also be trained by the machine learning algorithm. Unfortunately, we did not have such data, so a simple Bayesian net was constructed as an example to validate our suggestion. In general, the Bayesian network can also reflect the domain-specific knowledge that is essential to an accurate determination of fire. In addition, location can be finely identified at the point of interest (POI) level within the specific scene where a fire-like event always occurs. For example, a factory chimney that produces smoke can be visually identified and properly reflected in the construction of a domain-specific Bayesian network. In this paper, however, we should have exploited generic domain knowledge to design a simple Bayesian net with four nodes, as an example shown in Figure 7. The nodes were \"Fire (\(F\)),\" \"Cloud/Smoke (\(S\)),\" \"Dangerous Place (\(D\)),\" and \"Humidity (\(H\)).\" Here, to simplify the conditional probability table, we assumed that nodes \(F\), \(S\), and \(D\), has binary states, such that \"\(F=yes\) or no,\" \"\(S=yes\) or no,\" and \"\(D=yes\) or no\". The exception is \(H\), which has ternary states, including \"_Wet_, _Normal_, or _Dry_\". A priori probability of fire, P(\(F\)) is decided from the ratio of majority voting. Note that the ratio of the summed probability of SoftMax at the end of the LSTM stage could be alternatively taken for P(\(F\)). If the maximum averaged probability of location is for \"building,\" \"traffic (car, train, ship),\" \"street,\" \"forest or mountain,\" \"etc. (bonfire, garbage, and so on),\" then \(D=yes\). In general, almost all location categories are prone to fire except \"irrelevant place for fire (sky, sea, and so on).\" If the Faster R-CNN detects smoke in more than half the frames during \(t_{\mathit{vot}}\), then \(S=yes\). All these state values are obtained from video analysis. One may obtain the state value from a humidity sensor, but there was no concurrent sensor data to the  
Figure 6: The LSTM network for fire decision [26]. video in our experiment. So, the state of \(H\) was set to \"Normal\" whenever no additional description is given in the experiment. Examples of conditional probability tables are assumed as in Tables 1-3. We attempted to set reasonable conditional probabilities according to the general situations where it happens. Tables 1 and 2 show the probabilities of humidity and location conditioned on the fire decision based on the analysis of visual input from a video clip. Table 3 shows the conditional probability of smoke. In the table, the fog on the sea or lake is imagined, and the fire in wet conditions well produces smoke in rows (1) and (2), respectively, as examples. If the situation is ambiguous, then the equal probabilities are assumed in the table. Note that if the concurrent data were sufficiently collected, the structure and corresponding conditional probability table could be determined by training.