# Thermal Threat Monitoring Using Thermal Image Analysis and Convolutional Neural Networks
## 3 Materials and Methods
### Proposed Methodology  
The main objective of the proposed solution was to use a CNN model that allows quick detection of hands and hot objects at thermovision images at close range. As a result, the system can provide a warning of burns. An illustration showing the device mounting is presented in Figure 3. From the point of view of the capabilities of currently available devices [8], it is possible to attach the devices to the user's chest as in [19; 20]. On the other hand, taking into account the convenience and comfort of use, a better solution is to use integrated devices [9], assuming that in the future they will provide appropriate hardware resources for effective image analysis. A smartphone and camera installed at chest level was used in the present study. Figure 4 below presents the block diagram of the proposed algorithm and discusses the individual stages. At the beginning, the thermal imaging camera acquires raw temperature matrices at a frequency of 8 Hz with resolution 320 \(\times\) 240. In the next step, they are converted to a temperature matrix in degrees Celsius and scaled to resolution 160 \(\times\) 120. The algorithm checks whether there are objects with increased temperatures in the image and detects hands. There are two possible scenarios that pose a potential threat. The first one is when only a hot object appears in the image and the other one is when the hot object and the user's hands appear in the image. Figure 4, on the left, shows a situation when the algorithm detected a hot object but there is no risk of burns (Scenario 1) and a high-temperature warning should be generated. On the right side, there is a situation when the left hand is near a hot object (Scenario 2). In this case, a risk of burn alarm should be generated. In the final stage, the threat classification block decides which warning to generate. The main element of the developed method is an image analysis block using convolutional networks--CNN. Methods [26; 28] also use CNN models, but they are characterized by an extensive structure resulting from the greater complexity of the tasks performed (e.g., multiple scale detection or multiclass detection and classification). The model from Method [26] contains 5 convolutional layers. Behind the 1 st and 2 nd convolutional layers, there are ReLU, INR and Max-pooling layers. Behind the conv3, conv4, conv5 layers, there are ReLU layers. Additionally, the outputs of the conv3, conv4 and conv5 layers are connected to three layers, the so-called ROI pooling from which data is transferred to two fully connected layers. At the end, there is a SoftMax layer that scales the model results into probabilities. The structure of the model used in Method Method [28] is much more complex because it can detect objects of different classes and sizes. The YOLOv7 model used is divided into 3 main blocks: Backbone, Neck and Head. The Backbone block contains 4 convolutional layers and 3 Max-pooling layers and is used to generate image features. The Neck block is used to combine features and contains convolutional, pooling and upsampling layers. The last block performs prediction by classifying image areas as appropriate classes. After getting acquainted with the selected network structures [29; 30] and bearing in mind the fact that the hand detection process would not require complexity similar to multi-class classification, a network structure consisting of 15 layers and similar to [31] was proposed. Initial tests were carried out with this model after its adaptation to  
Figure 3: Mounting of a smartphone or goggles with an IR camera. IR images. Due to the high-detection efficiency, an attempt was made to optimize it while maintaining high efficiency--Figure 5. The network receives as input an image matrix containing temperature data in the 160 \(\times\) 120 \(\times\) 1 format (optical resolution, number of channels). The images are additionally normalized using zero-center normalization. The input layer was the first to be modified to ensure compatibility with the thermal images.