# Thermal Threat Monitoring Using Thermal Image Analysis and Convolutional Neural Networks
## 2 Related Works
First, a Gaussian model representing the background was prepared, thanks to which the approximate area of the hand was determined. In the next step, five areas were generated inside the hand area and the temperature distribution models were created. After analyzing the image by the five prepared models, the resulting masks were combined into one resulting mask. The effectiveness of the operation was determined as the ratio of the bounding box marked by the expert to the bounding box determined by the algorithm and amounted to 86%. Another application of hand segmentation in thermal imaging is biometrics [18]. The goal in this publication was to segment the hand and determine the vein system. The authors proposed several approaches based on thermovision and a combination of visible light and thermovision. In the case of thermal images, the active shape model method was used. In cases where segmentation for visible light images was combined with thermal images, masks established in visible light were used in thermal imaging to preselect the hand region. In this scenario, it was important to match the mask obtained from the visible image to the shape of the hand in the thermal image. In previous studies, the authors proposed a hand detection method with an SVM classifier using the proposed geometric features and texture parameters in the hand area [19]. A set of over 5100 images was prepared, containing the user's hands and arms in various situations and with objects with increased temperatures. The hand detection efficiency reached the level of Acc = 0.89 and high-temperature detection was correct for all images in the test set. In the hand detection process, thermal images are also used in combination with visible light and depth image information [20]. In the case of 3D methods, the information about the colour and depth (RGB-D) are used. The use of a thermal camera, in this case, increased efficiency by reducing the impact of variable external lighting on segmentation results. The authors annotated ground truth bounding boxes for RGB, depth and thermal images and the Fast R-CNN object detector was used to analyze the images. The algorithm was trained on a set of several thousand images (2000 RGB images, 1000 thermal images, and 1000 depth images) recorded from two cameras and a depth sensor. The authors noted that RGB images, then depth images and finally thermal images had the greatest impact on the detection efficiency, but they did not indicate the efficiency values obtained by the proposed method. Another example is the publication [21] where deep neural networks were proposed for the detection and classification of gestures on sequences of combined colour images, depth images and stereo-IR images. The proposed 3D recurrent network was trained on the Sport1M set containing video sequences with 487 types of sports activities. The algorithm did not directly detect hands and the gesture classification efficiency reached 98.2%. In publication [22], the authors again applied a deep convolutional network to recognize hand gestures in static images. It was noted that hand detection in the case of a complex background is difficult and classic methods are not always effective; therefore, the proposed model simultaneously functioned as a detector and classifier. Relatively few images were used: 1600 training and 400 test images. Gesture classification efficiency reached approximately 94.7%. Another method related to hand analysis was proposed in paper [23] where the goal was to locate the hand and detect skin in visible light images. The performance of RCNN [24] and Fast-RCNN [25] combined with skin segmentation was compared on several image sets (over 13,200 images containing the hand). Hand detection efficiency on various image sets reached maximum values of 96-97%. As a further improvement, the authors indicated increased resistance to poor lighting, shadows or image blur, because in the case of other image sets, the effectiveness was very low, even around 30-40%.