## 2 Materials and Methods
### AI Inference and Post-Processing  
The field-optimal model that learns virtual data is inferred using an AI device at the edge end placed at each site. Devices that can be linked to the NVIDIA TAO include the NVIDIA Jetson's Xavier series and Orin series. In this study, the Jetson Xavier NX was used.  
DeepStream is a GStreamer plug-in and library that can accelerate the inference process using deep learning. The models learned through the Transfer Learning Toolkit (TLT) can be distributed in Deepstream. DeepStream internally uses the TensorRT inference engine. Therefore, the encoded model file distributed by TAO is converted into an engine file for TensorRT using a Tao converter. By forming a GStreamer pipeline in a deepstream application for inference, the result of the final fire detection can be derived from input video decoding, such as CCTV, and post-processing to solve potential problems in the experimental field to be described later, and then sent to the outside by file or RTSP through an on-screen display (OSD). The inference result on the video, sink, or inference result can be sent to an external data server using a message broker. In this study, an external Kafka server was built and fire detection information was transmitted to the server as a Kafka message. Figure 13 shows the process from the model learned in TAO to the inference and result data of the Kafka message transmission.  
Specifically, the message containing the fire detection information is composed as shown in Figure 14. For the detected object (fire), classID of fire or smoke, detection rectangle, and how confident the model was about this detection result were included, and the ID value of the edge device was attached as metadata.  
Figure 12: Inference from testing data.  
Figure 13: The process of utilizing models deployed in TAO in deepstream for inference and post-processing.  
For both future research and the purpose of counting the number of fires, fire tracking is crucial. Consequently, post-processing was developed to accomplish this. We stored the difference between the diagonal length of the first bounding box of the tracked fire and the diagonal length of the subsequent bounding boxes and computed a \(d_{cumulative}\) from the 15 most recent differences to verify the fire movement against a specific threshold.  
\[d_{t}=|p_{0_{RB}}-p_{0_{LT}}|-|p_{t_{RB}}-p_{t_{LT}}|;d_{cumulative}=\sqrt{ \sum_{t=1}^{n}(d_{0}-d_{t})^{2}} \tag{1}\]  
As shown in Figure 15, from the center of the first bounding box of the tracked fire, we find the cosine of the angle between the vector \(v_{0_{LT}}\) to the top left of the first bounding box and the vector \(v_{t_{LT}}\) to the top left of the \(t_{th}\) bounding box \(\theta_{t_{LT}}\) and check whether it falls within a certain threshold to verify whether it is a fire. Based on our experiment, we found that 0.9-0.98 was the best threshold to verify whether it was a fire.  
\[cos\theta_{t_{LT}}=\frac{v_{0_{LT}}^{\rightarrow}.v_{t_{LT}}^{\rightarrow}} {|v_{0_{LT}}^{\rightarrow}||v_{t_{LT}}^{\rightarrow}|};cos\theta_{t_{RB}}= \frac{v_{0_{RB}}^{\rightarrow}.v_{t_{RB}}^{\rightarrow}}{|v_{0_{RB}}^{ \rightarrow}||v_{t_{RB}}^{\rightarrow}|} \tag{2}\]  
Figure 14: A message about the detection results sent by deepstream.  
Figure 15: Analysis between two bounding boxes for tracking fire.