# Artificial Intelligence Assistive Fire Detection and Seeing the Invisible Through Smoke using Hyperspectral and Multi-spectral Images

Ahed Alboody, Saeed Mian Qaisar, Gilles Roussel

► To cite this version:

Ahed Alboody, Saeed Mian Qaisar, Gilles Roussel. Artificial Intelligence Assistive Fire Detection and Seeing the Invisible Through Smoke using Hyperspectral and Multi-spectral Images. 13th International Conference on Electronics and Information Technologies (ELIT), IEEE, Sep 2023, Lviv, Ukraine. hal-04220125

HAL Id: hal-04220125

https://ulco.hal.science/hal-04220125

Submitted on 27 Sep 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Artificial Intelligence Assistive Fire Detection and Seeing the Invisible Through Smoke using Hyperspectral and Multi-spectral Images

Ahed ALBOODY

CESI-LINEACT

Nice, 06200, France

aalboody@cesi.fr

Saeed MIAN QAISAR¹,²

¹CESI-LINEACT, Lyon, 69100, France

²Electrical and Computer Engineering
Department, Effat University, 22332,
Jeddah, KSA

smianqaisar@cesi.fr

Gilles ROUSSEL

Laboratoire LISIC, Université du

Littoral Côte d'Opale, Calais, 62228,
France

gilles roussel@univ-littoral.fr

**Abstract**— The global warming has serious impact on our climate. Due to this, the frequency and the intensity of forest fires is increasing. It has shown serious challenges such as the protection of resources, human and wild life, health, and property. This study focuses on developing an artificial intelligence assistive innovative solution for active fire detection in the context of smart cities and vicinities. This paper addresses spectral analysis, detection and classification of active fires and seeing the invisible through smoke and thin clouds. The appealing applications are in urban surveillance, smart cities, future industries, forests and earth observation. The idea is realizable by using an intelligent hybridization of machine/deep learning models and using multi-sensor images (aerial, satellite). For this purpose, we use hyperspectral images (Visible, Near Infra-red (NIR) and Short-Wave Infrared (SWIR)) from AVIRIS aerial and Multi-Spectral Sentinel-2 satellite images. AVIRIS images are 224 spectral bands of wavelengths with a spatial resolution of 15 meters, which varies from 366nm (nanometers) up to 2500nm. However, AVIRIS image studied for their spectral richness of wavelengths not yet completely exploited by machine and deep learning and in SWIR to detect active fires. While, Sentinel-2 image has 13 spectral bands (Visible, NIR and SWIR) with three spatial resolutions (10, 20 and 60 meters). First, we explain and describe the preparation phase of hyperspectral and multispectral image databases of forest fires. These databases contain hyperspectral and multispectral endmembers data of different sites for forest fires. Then, we conduct a spectral analysis from these endmembers to characterize the hyperspectral/multispectral reflectance of active fires to identify the distinct wavelengths for fire detection. We identify the wavelengths that can be used for an effective identification of fire and to see through fires smoke and thin clouds. Onward, the selected feature set is processed by robust machine/deep learning algorithms and their performance is compared for automated identification of fire and invisible vision amelioration. The proposed machine/deep learning method secured an overall test accuracy of 99.1%.

**Keywords**— Deep learning; Machine learning; Classification; Semantic segmentation; Hyperspectral and Multi-spectral image; Active fire detection; Spectral analysis; Earth observation; Smart urban surveillance

## I. INTRODUCTION

In our days, forest fires have been increasing dramatically in fire intensity and frequency in many countries in Europe and Canada. Hyperspectral remote sensing systems were used to detect, identify and characterize forest fires [1, 2]. Hyperspectral systems collect spectral information of wavelengths where the wavelengths vary in spectral range from Visible (400-750 nm) to Near Infra-Red (NIR,

750-1100 nm) and Short-Wave Infra-Red (SWIR, 1100-2500 nm) [2, 3, 4, 5, 6, 7]. Despite being an expensive and complex system, hyperspectral system is robust to detect and identify fires [8, 9]. Indeed, it has been shown that NIR/SWIR hyperspectral systems between 1400nm and 2500nm are promising to identify forest fires based on fire index [8] because the spectral reflectance of active fires have distinct features in this range [6, 7, 8, 9].

Recently, machine learning methods, such as Support Vector Machine (SVM) [7], Artificial Neural Network (ANN) [10], and Random Forests [11, 12] were used to classify images (hyper-spectral [9], multispectral [10, 13, 14] and RGB) in order to detect and classify fires. In [7], authors used hyperspectral images from Hyperspectral PRISMA Italian satellite for fire identification in Australia forests. They explored classification technique based on SVM combined with visual interpretation of PRISMA image for validation as ground truth.

Compared to machine learning models, deep learning models based upon convolutional neural networks (CNN) are proposed to classify fires in [9, 11, 15, 16]. For example, 1-Dimensional Convolution Neural Network (1D-CNN) architectures were developed, trained on hyperspectral PRISMA images to classify wildfires [8, 9]. In [13], authors proposed a Fire-Net deep learning framework. Fire-Net is trained on Landsat-8 multispectral satellite images for the classification of active fires and burned areas. Specifically, three optical spectral bands (Red, Green and Blue) are fused with thermal spectral bands of Landsat-8 images for a more effective detection of active fires.

The main contributions of this paper are the following points: (1) Discussing the advantages of hyperspectral and multispectral images over spectral analysis of Visible/NIR/SWIR spectral bands for hyperspectral active fire detection; (2) Presenting the potential of machine learning models (KNN, SVM, ANN) and deep learning models based on 1-Dimensional Convolution Neural Network to detect and classify active fires. Then the results will be compared and discussed. (3) Discussing the possibility and the benefit to integrate the hyperspectral imaging embedded systems (as similar as to AVIRIS and PRISMA) coupled with machine and deep learning models can open new research opportunities for fire detection in application security of urban surveillance, smart cities, and industrial plants.

The rest of the paper is structured as follows. Section II addresses the description of the study areas, the benchmark
datasets of AVIRIS hyperspectral and Sentinel-2 multispectral image, the spectral analysis for active fire detection. In Section III, we will apply a supervised machine and 1D-CNN deep learning models for active fire classification. While in Section IV, the results of the proposed models are presented with a critical discussion. Conclusions are given in Section V.

## II. STUDY AREAS, BENCHMARK DATASETS AND SPECTRAL ANALYSIS FOR ACTIVE FIRE DETECTION

### A. Areas of Interest and Datasets for Active Fire Detection

The first datasets utilized in this paper are Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) hyperspectral images. AVIRIS is an instrument in the real-time of Earth Observation and Remote Sensing. AVIRIS sensor delivers calibrated images of spectral radiance in 224 spectral bands with wavelengths from 366 to 2500 nanometers and a spatial resolution of 15 meters. A hyperspectral image can be represented by a data-cube of two spatial dimensions (rows and colons) and third dimension as a spectral dimension for the number of spectral bands. The Shortwave Infra-red (SWIR) range covering wavelengths from 1400 to 2500 nm, can include significant emitted radiance from fire. The utility of hyperspectral remote sensing images are evaluated for active fire detection [8, 9], and in particular, NIR/SWIR remote sensing images. We used AVIRIS data for the study area of the 2019 California wildfire season that burned across the state of California in US (Fig. 1).

Fig. 1. a) RGB True color composite image from AVIRIS over parts of Sheridan fire in the Prescott National Forest in Arizona, USA on August 21, 2019. This composite used the spectral bands: 647 nm (Red), 550 nm (Green) and 472 nm (Blue); b) color-infrared (CIR) image by dividing the spectral range into three bands: 859 nm near infrared (NIR), 647 nm Red, and 550 nm Green bands; c) and d) False color composite inputting active fires monitoring in NIR and SWIR at: 2176 nm (red), 1561 nm (green) and 956 nm (blue)

All AVIRIS images are available free on AVIRIS Data Portal<fnref n="1" />. For AVIRIS hyperspectral dataset preparation and the purpose of the accurate evaluation of the detection performance of metrics, we discriminate between active fire pixels and non-fire pixels (smoke, burned area, vegetation, bare soil, and water). To do this discrimination, we select manually pixels with endmembers where each endmember is given by the mean value of spectral reflectance associated with each image patch of size (3×3×224) pixels. The mean value is calculated over all its 9 (3×3) pixels and converted into a vector data of size 1x224 (number of AVIRIS bands). These endmembers are the ground truth that used to train/test the supervised machine and deep learning models for the classification task in Section III. This step of manual selection and classification is needed to determine the ground truth pixels for the implementation of automatic classification based on machine and deep learning. We select these image patches of 9 pixels by exploring the false color composite (Fig. 1c and 1d) and looking at the AVIRIS hyperspectral reflectance, which was comparable with the corresponding classes in [6, 7, 8, 9]. Specifically, Non-Fires class contains five subclasses: smoke, burned areas, vegetation, bare soil and water. To prepare the ground truth of training datasets, we selected 528 endmembers, which

represent pixels of two classes: one for active fires/Fires class and the other for Non-Fires class. These 528 endmembers are divided into the following number of labeled endmembers: 270 endmembers for active fires, 114 endmembers representing smoke, 18 endmembers for burned areas, 27 endmembers for the bare soil class, 63 endmembers for vegetation, and 36 endmembers for water. We grouped the endmembers of smoke, bare soil, vegetation, burned areas and water into Non-Fires class. Finally, for learning phase, training/validation datasets have 270 endmembers of active fires for Fires class and 258 endmembers for Non-Fires class of training data (See Table I). Of the ground truth, we considered a five-fold cross-validation for the training/validation datasets. Then, we selected 106 endmembers for test datasets which are completely different of training/validation datasets. Test datasets are divided into the following number of labeled endmembers: 65 endmembers for active fires, 8 endmembers representing smoke, 13 endmembers for burned areas, 3 endmembers for the bare soil, 6 endmembers for vegetation, and 11 endmembers for water. As similar to training data, test datasets have 65 endmembers of active fires for Fires class and 41 endmembers for Non-Fires class for prediction phase.

The second datasets utilized in this paper are Sentinel-2 multispectral satellite image for Canada and Greece's multiple wildfire in July 2023. Multispectral Imager (MSI) of Sentinel-2 satellite delivers 13 spectral bands with three spatial resolution (10, 20 and 60 meters) [14, 15, 17]. These 13 spectral bands range from the Visible (VNIR) and Near Infra-Red (NIR) to the Short Wave Infra-Red (SWIR). Four spectral bands (Blue (B2), Green (B3), Red (B4), and Near-Infrared (B8)) have a 10-meter spatial resolution and these bands are centered at the following central wavelengths (in nanometers) respectively: 490 nm, 560 nm, 665 nm, 842 nm. Next, six spectral bands in VNIR and SWIR spectral range are given as follows: red edge (B5, 705nm), near-infrared NIR (B6, 740 nm; B7, 783 nm; and B8A, 865 nm), and short-wave infrared SWIR (B11, 1610 nm; and B12, 2190 nm) which have a 20-meter spatial resolution. Finally, the coastal aerosol (B1, 443 nm), water vapour band (B9, 940 nm), and cirrus (B10, 1375 nm) spectral bands have a 60-meter spatial resolution. Where for the correction of atmospheric effects (e.g., aerosols, cirrus or water vapor), three bands B01, B09 and B10 are used. The remaining ten spectral bands are primarily intended to land use and land cover applications. All Sentinel-2 images are available free on Copernicus Open Access Hub². For the purpose of active fire detection, we use Sentinel-2 level-2A images that are atmospherically corrected surface reflectance in cartographic geometry. At this level, these ten spectral bands, which explored in this study, are B2, B3, B4, B5, B6, B7, B8, B8A, B11 and B12 with spatial resolution of 20-meter where four bands B2, B3, B4 and B8A are resampled to 20-meter spatial resolution. For the selected area of interest, we obtained the corresponding Sentinel-2 level2A images at the start day or/and 5 days after of the set fire providing reference data for active fires class, several days before providing ground truth for vegetation (of selected area of interest) as non-fires class, and after the set fire start date for burned areas as non-fires class. As similar to AVIRIS visual interpretation in Fig. 1, false-color images of the SWIR (bands B11 and B12) and NIR (band B8A) are so useful for visual fire detection

<footnote n="1">1 JPL | nasa.gov: https://aviris.jpl.nasa.gov/dataportal/</footnote>
(Fig. 2). Similar to AVIRIS datasets, for Sentinel-2 multispectral datasets preparation, Sentinel-2 images are resampled and converted into image patches. Each image is divided into a grid of image patches of size 3×3 pixels. Since, ten VNIR/SWIR spectral bands are considered and the total number of image patches of size (3×3×10) pixels are 1652 manually selected. Then, we select *endmembers* divided into 372 *Fires and 384 Non-Fires* classes from these image patches where each endmember of 9 pixels is calculated by the mean multispectral reflectance associated with each image patch of size (3×3×10) pixels. The mean value is calculated over all its 9 (3×3) pixels leading to a vector data of size 1x10 (number of VNIR/SWIR bands of Sentinel-2 images). With supervised machine and deep learning models, we divide the datasets into two parts of *endmembers*: (1) 288 *Fires and 284 Non-Fires* classes for training/validation datasets, and (2) the rest for test datasets. The Sentinel-2 multispectral reflectance corresponding to six classes: *active fire, smoke, burned areas, vegetation, bare soil and water* are presented in Fig. 2c.

<figure><img src="image_2.png" /></figure>

<figure><img src="image_3.png" /><figcaption>MultiSpectral Reflectance of Sentinel-2 MSI Level-2A Images for "Active Fires" and "Non-Fire class"</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>Fig. 2. a) True color composite image from the Sentinel-2 over wildfires burning on the Greek island of Rhodes on July 23, 2023. The RGB composite used the bands centered at 665 nm (B4, Red), 560 nm (B3, Green) and 490 nm (B2, Blue); b) False color composite inputting active fires monitoring in NIR and SWIR at wavelengths 2190 nm (band B12) (red), 1610 nm (band B11) (green) and 865 nm (band B8A). c) Sentinel-2 Multi-Spectral Reflectance corresponding to six classes: active fire, bare soil, burned areas, smoke, vegetation, and water.</figcaption></figure>

## B. Spectral Analysis of Hyperspectral and Multispectral Datacubes for Active Fire Detection

In this subsection, we perform a spectral analysis of endmembers to characterize the hyperspectral/multispectral reflectance of active fire. Based on the training and test datasets, endmembers were selected by examining the false color composite (Figs. 1c and 1d) and considering the AVIRIS hyperspectral reflectance (see Fig. 3 and Fig. 4). We found spectral discrimination features with specific hyperspectral/multispectral bands in these NIR/SWIR wavelengths: from 1950nm to 2450nm; from 1511nm to 1800nm; from 1166nm to 1332nm; and from 966nm to 1100nm; with very good discrimination features to see through smoke and detect active fires in real-time applications. This spectral analysis shows that the spectral reflectance of fires obtained is similar to that in [7, 8, 9, 10] using PRISMA hyperspectral images. For the detection of active fire using hyperspectral images in industrial environments, we need a NIR/SWIR hyperspectral camera with spectral range from 1100 nm to 2500 nm. For smoke detection in the VNIR range, smoke can be quite easily detected by considering the Visible-NIR bands/wavelengths reported in Fig. 2c and Fig. 3.

Fig. 3. AVIRIS hyperspectral reflectance corresponding to six classes: active fire, bare soil, burned areas, smoke, vegetation, and water. Red bracket number 0 and yellow bracket number 1 indicate the overlapping bands of VNIR-SWIR; Black brackets number 2 and 3 indicate atmospheric attenuation by water vapor, which can occur at 1400 nm and 1900 nm respectively; bracket number 4 indicates the CO2 absorption bands around 2000 nm.

Fig. 4. AVIRIS reflectance for *Fire class* endmember against the background (*Non-Fire classes*) showing CO2 absorption bands around 2000 nm and 2010 nm ($\lambda_n$), and around 2050 nm and 2060 nm ($\lambda_n$). The locations of the wavelengths corresponding to the “peaks features” of active fires are indicated as $\lambda_1$ (absorption features to the left of $\lambda_n$) and $\lambda_2$ (absorption features to the right of $\lambda_n$), respectively.

## III. MACHINE LEARNING (ML) AND CNN DEEP LEARNING (DL) MODELS FOR ACTIVE FIRE DETECTION

In this section, we will apply a supervised machine and deep learning models to classify active fires using hyperspectral and multispectral images. For this end, we propose to apply three well-known supervised machine learning models (K-Nearest Neighbor classification (KNN), Support Vector Machines (SVM) and Artificial Neural Networks (ANN)) and convolutional deep learning models to classify the endmembers of the training datasets for the learning phase, and the test datasets for the prediction phase, which are described in Section II.A. We describe the proposed supervised machine and deep learning models for active fires detection. For the learning phase in this work, the stopping condition as 30-epochs is defined for all models. The test datasets were used to evaluate the trained models where metrics were calculated in the prediction phase. Finally, we test four trained models with real hyperspectral images to detect active fires.

<footnote n="2">2 Copernicus Open Access Hub: https://scihub.copernicus.eu/dhus/#/home</footnote>
### A. Machine Learning Models (ML)

Based on the training and test datasets, imbalanced classification and weakly supervised learning are challenges for predictive classification because most supervised machine learning models used for classification task were designed around the assumption of an equal number of samples for each class. To take into account these limitations, we have selected three supervised machine-learning models: KNN, SVM and ANN. For each model, there are hyper-parameters to be optimized to determine the best fine-tuning of the classification model by using the training endmembers in the learning phase and then to evaluate its performance with the test endmembers in the prediction phase. To fine-tune the hyper-parameters of the classification model with challenges of imbalance class and weakly supervised learning, Hyper-Parameter (HP) optimization methods, Bayesian and Random research, offer possibilities to automatically select a classification model with optimized fine-tuning hyper-parameters.

For the issue of fire classification, we therefore propose to apply the main three following phases: (1) in the first phase, PCA dimensionality reduction by feature extraction is applied on the training datasets; (2) in the learning phase, serval models of supervised machine learning as KNN, SVM and ANN classification models are applied with HP optimization methods of fine-tuning hyper-parameters to determine the best validation accuracy as in [3]. Five-fold cross-validation is considered to protect the trained models against overfitting. This scheme partitions the training datasets into five disjoints fold. Each fold is used once as a validation-fold and the others form a set of training-folds. That allows calculating the size of validation by the 20 percent of the training datasets (Table I).

<table><caption>TABLE I. AVIRIS DATASETS FOR ACTIVE FIRE DETECTION</caption><thead><tr><th rowspan="2">Endmembers extracted from AVIRIS images</th><th colspan="2">Number of Endmembers for Two classes</th><th rowspan="2">Total number of endmembers for</th></tr><tr><th>Fires</th><th>Non-Fires</th></tr></thead><tbody><tr><td>Training endmembers</td><td>270</td><td>258</td><td>528 (83.3 % of Total endmembers)</td></tr><tr><td>Validation endmembers (five-fold cross-validation)</td><td>54 (20 % of training)</td><td>52 (20 % of training)</td><td>106 (16.7 % of Total endmembers)</td></tr><tr><td>Test endmembers</td><td>65</td><td>41</td><td>106 (16.7 % of Total endmembers)</td></tr><tr><td>Total endmembers</td><td>335</td><td>299</td><td>634 (100%)</td></tr></tbody></table>

For each validation-fold, the classification model is trained using the training-folds and the validation classification accuracy is assessed using the validation-fold. The average validation accuracy is then calculated over all the folds and is used to optimize the fine-tuning parameters of the classification model. These hyper-parameters are determined by an automatic hyper-parameter optimization using two optimization methods: Bayesian and random research. The final validation accuracy have a high estimation of the predictive accuracy of the classification model. (3) In the prediction phase, we test the trained models obtained during the learning phase to the test endmembers to determine the overall test accuracy of the trained classification model. Here, the test datasets are completely different of the training datasets. Based on these three phases, Table II presents the top ten-classification models that we tested with different dimensions of the feature subspace obtained by PCA (128, 96, and 64) and with two HP optimization methods. The first column of this Table II gives

the name of the tested classification model, the second one indicates the dimension of the features subspace and the third column gives the name of the used HP optimization method. The goal of optimization method is to find a combination of HP values that minimizes an objective function, here the classification error rate. To find this combination, the iteration number of the used method is fixed to 30. The fourth column of Table II describes the determined optimized hyper-parameters. This column is divided into several cells whose number depends on the classification model. For each tested classification model, the validation accuracy computed with the training datasets and the test accuracy computed with the test datasets appear in the fifth and sixth columns, respectively. Accuracy is given as the percentage of endmembers (training or test) that are correctly classified.

### B. CNN Deep Learning Model

In this subsection, we propose a lightweight Convolutional Neural Network (CNN) for classification of active fire in hyper/multispectral images that improves the performance of fire detection. The proposed deep learning model based on a 1-Dimensional Convolution Neural Network (called 1D-CNN) presented in Fig. 6 and trained on AVIRIS hyperspectral images to classify active fires. The 1D-CNN model includes three convolutional layers, 2 Fully Connected (FC) layers, and one max-pooling (Max-Pooling) layer. At the end of the 1D-CNN model, a softmax activation function is applied. This classification model is inspired by the one described in [8, 9]. The input of 1D-CNN model is the endmember (obtained in Section II.A) comprising the Visible/NIR/SWIR spectral bands of AVIRIS. The input layer of the 1D-CNN model is the 1x224 input features during the training/learning phase. Thus, it is a vector data of size 1xC where ($C_1=224$ is the number of spectral bands for AVIRIS, and $C_2=10$ for Sentinel-2). The first layer is one Dimensional (1D) convolutional layer (Conv1) with kernel size equal to 1, number of filters equal to $n_1=224$, same padding with "PaddingValue" equal to "replicate", leakyReluLayer(value=0.1) activation function. The second layer is 1D convolutional layer (Conv2) with kernel size equal to 3, number of filters equal to $n_2=128$, same padding with "PaddingValue" equal to "replicate", leakyReluLayer(value=0.1) activation function. After these two 1D convolutional layers, a Max-pooling layer is connected to the Conv2 output with these parameters: pooling size of 2 and stride of 2 (with respect Fig. 6, note that $n_3=n_2=128$). The output of Max-pooling layer is passed to the third layer of 1D convolutional layer (Conv3) with kernel size equal to 3, number of filters equal to $n_2=64$, same padding with "PaddingValue" equal to "replicate", ReluLayer activation function. Then, the result of third convolutional layer (Conv3) is passed and connected to the first fully connected layer (FC1) of 32 unites with leakyReluLayer(value=0.1) activation function. The last layer is fully connected layer (FC2) of 2 unites and softmax activation function to classify the output into classes. The 1D-CNN model is trained on single GPU using Adam optimizer. The objective function of the 1D-CNN model is the categorical cross-entropy loss function of the classification output layer. All hyper-parameters are given in Table II. Models (KNN, SVM, ANN, 1D-CNN) have been implemented using MATLAB R2021a Machine Learning and Deep Learning Toolbox.
IV. RESULTS AND DISCUSSION

In this section, we present the performance evaluation of active fire detection using the top ten machine and deep learning models. Metrics as confusion matrix for test datasets (Fig. 8) and the validation/test accuracy assessment are given in Table II. The whole results for the proposed models are summarized in Table II with the comparison of the different results. Experimental results, given in Fig. 8, show the test confusion matrix and overall test accuracy for fire classification with four models: (1) Model1-KNN, (2) Model 5-SVM, (3) Model 8-ANN Bi-layered Neural Network, (4) Model 1D-CNN. an overall test accuracy on the test datasets is 99.1% for three different models (Model1-KNN, Model8-ANN, Model 1D-CNN) and 100% for the Model5-SVM, while using the proposed model 1D-CNN achieves high-accuracy to detect active fires on real AVIRIS images as shown as in Fig. 9. Generally, these results are higher than

97.83% where 1D-CNN used in [8, 9] for validation datasets from PRISMA hyperspectral images.

<figure><img src="image_5.png" /><figcaption>Fig. 6. Architecture of 1D-CNN model for active fires classification of AVIRIS images</figcaption></figure>

<table><caption>TABLE II. PERFORMANCE EVALUATION OF ACTIVE FIRE DETECTION USING THE TOP TEN MACHINE AND DEEP LEARNING MODELS</caption><thead><tr><th colspan="3">AVIRIS Machine / Deep Learning Models for Active Fire Classification</th><th>Value of PCA</th><th>Optimizer Method</th><th colspan="4">Fine-tuning / Optimized Hyper-Parameters (HP)</th><th>Validation accuracy</th><th>Test accuracy</th></tr><tr><th></th><th></th><th></th><th></th><th></th><th>Number of neighbors</th><th>Distance metric</th><th>Distance weight</th><th>Standardize data</th><th></th><th></th></tr></thead><tbody><tr><td rowspan="4">KNN</td><td rowspan="4"></td><td>Model 1-KNN</td><td>PCA disabled</td><td>Bayesian optimization</td><td>258</td><td>Cosine</td><td>Squared inverse</td><td>false</td><td>99.4%</td><td>99.1%</td></tr><tr><td>Model 2-PCA128-KNN</td><td>128</td><td>Random search</td><td>7</td><td>Minkowski (cubic)</td><td>inverse</td><td>false</td><td>98.5%</td><td>99.1%</td></tr><tr><td>Model 3-PCA96-KNN</td><td>96</td><td>Random search</td><td>1</td><td>Euclidean</td><td>Inverse</td><td>false</td><td>98.5%</td><td>99.1%</td></tr><tr><td>Model 4-PCA64-KNN</td><td>64</td><td>Random search</td><td>2</td><td>Correlation</td><td>Equal</td><td>true</td><td>98.3%</td><td>82.1%</td></tr><tr><td rowspan="3">SVM (Kernel scale: 1)</td><td rowspan="3"></td><td>Model 5-SVM</td><td>PCA disabled</td><td>Bayesian optimization</td><td>Multi-class method</td><td>Box constraint level</td><td>Kernel function</td><td>Standardize data</td><td></td><td></td></tr><tr><td>Model 6-PCA128-SVM</td><td>128</td><td>Bayesian optimization</td><td>One-vs-All</td><td>26.867</td><td>Linear</td><td>true</td><td>99.8%</td><td>100.0%</td></tr><tr><td>Model 7-PCA96-SVM</td><td>96</td><td>Bayesian optimization</td><td>One-vs-All</td><td>980.8977</td><td>Linear</td><td>false</td><td>95.8%</td><td>93.3%</td></tr><tr><td rowspan="4">Artificial Neural Network ANN</td><td rowspan="4"></td><td>Model 8-ANN Bi-layered Neural Network</td><td>PCA disabled</td><td>Bayesian optimization</td><td>Number of FC layers: 2</td><td>Activation: Relu</td><td>Regularization strength (Lambda): 5.0497e-08</td><td>Standardize data: yes</td><td>98.7%</td><td>99.1%</td></tr><tr><td rowspan="2">Model 9-PCA128-ANN Tri-layered Neural Network</td><td rowspan="2">128</td><td rowspan="2">Random search</td><td>First layer size: 10</td><td>Second layer size: 10</td><td>Third layer size: 0</td><td>Iteration limit: 1000</td><td>90.9%</td><td>83.6%</td></tr><tr><td>Number of FC layers: 3</td><td>Activation: ReLU</td><td>Regularization strength (Lambda): 0</td><td>Standardize data: yes</td><td>90.9%</td><td>83.6%</td></tr><tr><td rowspan="2">1D-CNN</td><td rowspan="2"></td><td rowspan="2">Model 10-1D-CNN</td><td rowspan="2">PCA disabled</td><td rowspan="2">Adam Optimizer</td><td>First layer size: 10</td><td>Second layer size: 10</td><td>Third layer size: 10</td><td>Iteration limit: 1000</td><td>99.4%</td><td>99.1%</td></tr><tr><td>Number of FC layers: 2; Convolution layers: three</td><td>Number of Max-Pooling layer: one</td><td>Activation: LeakyRelu/ ReLU; Softmax for output layer</td><td>Regularization strength (l2norm): 1.0e-4</td><td>Mini Batch Size: 12</td><td>Learning Rate: 3e-4</td><td>99.4%</td><td>99.1%</td></tr></tbody></table>

The segmentation map obtained by the prediction phase of the four proposed model are reported in Fig. 9 for the area of interest over parts of Sheridan fire in the Prescott National Forest in Arizona, USA on August 21, 2019. We found that Model 5-SVM and Model 8-ANN model returned false alarms of active fires detection when we tested on real AVIRIS images, while Model 1D-CNN and Model1-KNN give best performance for test and real AVIRIS images.

The results of this paper demonstrates the potentialities of hyperspectral data for active fire detection. The availability of hyperspectral reflectance allows analyzing the hyperspectral information in order to detect and identify fires in smart cities and urban environment. The possibility to use

a VNIR/SWIR hyperspectral camera embedded on drone [18, 19] or robot for smart surveillance of fires in urban and industrial environments is one of the bigger advantages of hyperspectral images when we talking about active fire detection in early-warning, real-time smart surveillance and to be considered for future mission dedicated for climate changes and environmental analysis of smart cities.
<figure><img src="image_6.png" /></figure>

<figure><img src="image_7.png" /></figure>

<figure><img src="image_8.png" /></figure>

<figure><img src="image_9.png" /><figcaption>Fig. 8. Test confusion matrix and overall test accuracy for fire classification with: (1) Model1-KNN, (2) Model 5-SVM, (3) Model 8-ANN Bi-layered Neural Network, (4) Model 1D-CNN</figcaption></figure>

<figure><img src="image_10.png" /></figure>

<figure><img src="image_11.png" /><figcaption>Fig. 9. Segmentation map results (fires in yellow color): (a) RGB AVIRIS aerial, and (b) False-colored images, and the results of the classification prediction from four proposed models: (c) Model1-KNN, (d) Model 5-SVM, (e) Model 8-ANN, and (f) Model 1D-CNN</figcaption></figure>

## V. CONCLUSION

For active fire detection, this paper has shown the interest of spectral analysis and machine/deep learning coupled with VNIR/SWIR hyperspectral and multispectral images. The results of the proposed models demonstrate that VNIR/SWIR hyperspectral/multispectral images allows to perform many different analysis in order to classify active fires and see through smoke by looking at different spectral bands in NIR/SWIR spectral range. Then, an automatic classification of active fire using supervised machine and deep learning models based on a one-dimensional convolutional layers has been performed. This paper showed also that machine and deep learning models coupled with VNIR/SWIR hyperspectral/multispectral images embedded on an industrial robot or Unmanned Drone is a promising solution to identify and detect active fires in application security of urban surveillance, smart cities, and industrial environments. In future applications, robot and drone's VNIR/SWIR camera embedded may be enable us to easily identify hotspots of active fires.

## REFERENCES

[1] M. K. Griffin, S. M. Hsu, H. H. K. Burke and J. W. Snow, "Characterization and delineation of plumes, clouds and fires in hyperspectral images," In Proc. IEEE International Geoscience and Remote Sensing Symposium '02, vol.2, 2000, pp. 809-812.

[2] P. E. Dennison and D. A. Roberts, "Daytime fire detection using airborne hyperspectral data," *Remote Sensing of Environment*, vol. 113, pp. 1646-1657, 2009.

[3] A. Alboody, N. Vandenbroucke, A. Porebski, R., F. Viudes, P. Doyen, and R. Amara, "A New Remote Hyperspectral Imaging System Embedded on an Unmanned Aquatic Drone for the Detection and Identification of Floating Plastic Litter Using Machine Learning," *Remote Sensing*, vol.15, no. 14: 3455, 2023.

[4] S. Veraverbeke, P. Dennison, I. Gitas, G. Hulley, O. Kalashnikova, T. Katagis, L. Kuai, R. Meng, D. Roberts, and N. Stavros, "Hyperspectral remote sensing of fire: State-of-the-art and future perspectives," *Remote Sensing of Environment*, vol. 216, pp. 105-121, 2018.
[5] R. S. Allison, J. M. Johnston, G. Craig, and S. Jennings, "Airborne Optical and Thermal Remote Sensing for Wildfire Detection and Monitoring," *Sensors*, vol. 16, no. 8:1310, 2016.

[6] E. Vangi, G. D'Amico, S. Francini, F. Giannetti, B. Lasserre, M. Marchetti, and G. Chirici, "The New Hyperspectral Satellite PRISMA: Imagery for Forest Types Discrimination," *Journal of Sensors*, vol. 21, no. 4:1182, 2021.

[7] S. Amici, and A. Piscini, "Exploring PRISMA Scene for Fire Detection: Case Study of 2019 Bushfires in Ben Halls Gap National Park, NSW, Australia," *Remote Sensing*, vol. 13, no 8: 1410, 2021.

[8] D. Spiller, L. Ansalone, S. Amici, A. Piscini, and P. P. Mathieu, "Analaysis and detection of wildfires by using PRISMA hyperspectral imagery," *In International Archives of the Photogrammetry Remote Sensing and Spatial Information Sciences*, XLIII-B3, 215-222, 2021.

[9] K. Thangavel, D. Spiller, R. Sabatini, S. Amici, S. T. Sasidharan, H. Fayek, and P. Marzocca, "Autonomous Satellite Wildfire Detection Using Hyperspectral Imagery and Neural Networks: A Case Study on Australian Wildfire," *Remote Sensing*, vol. 15, no. 3: 720, 2023.

[10] J. Florath, and S. Keller, "Supervised Machine Learning Approaches on Multispectral Remote Sensing Data for a Combined Detection of Fire and Burned Area," *Remote Sensing*, vol. 14, no. 3: 657, 2022.

[11] N. Maeda, and H. Tonooka, "Early Stage Forest Fire Detection from Himawari-8 AHI Images Using a Modified MOD14 Algorithm Combined with Machine Learning," *Sensors*, vol. 23, no. 210, 2023.

[12] K. G. Madhwaraj, V. Asha, A. Vignesh and S. Akshay Shinde, "Forest Fire Detection using Machine Learning," IEEE 12th International Conference on Communication Systems and Network Technologies'23, 2023, pp. 191-196.

[13] S. T. Seydi, V. Saeidi, B. Kalantar, N. Ueda, and A. Abdul Halin, "Fire-Net: A Deep Learning Framework for Active Forest Fire Detection," *Journal of Sensors*, vol. 2022, no.14, ID 8044390, 2022.

[14] D. A. G. Dell'Aglio, M. Gargiulo, A. Iodice, D. Riccio and G. Ruello, "Active Fire Detection in Multispectral Super-Resolved Sentinel-2 Images by Means of Sam-Based Approach," IEEE 5th International forum on Research and Technology for Society and Industry '19, 2019, pp. 124-127.

[15] M. Gargiulo, D. A. G. Dell' Aglio, A. Iodice, D. Riccio and G. Ruello, "A CNN-based Super-resolution Technique for Active Fire Detection on Sentinel-2 Data," IEEE Photonics & Electromagnetics Research Symposium - Spring (PIERS-Spring), 2019, pp. 418-426.

[16] M. A Akhloufi, R. B. Tokime, and H. Elassady, "Wildland fires detection and segmentation using deep learning. In Pattern recognition and tracking," In Proc. SPIE, vol. 10649, pp.1-12, 2018.

[17] A. Alboody, M. Puigt, G. Roussel, V. Vantrepotte, C. Jamet and T. - K. Tran, "Deepsen3: Deep Multi-Scale Learning Model For Spatial-Spectral Fusion Of Sentinel-2 And Sentinel-3 Remote Sensing Images," In Proc. IEEE Workshop on Hyperspectral Imaging and Signal Processing: Evolution in Remote Sensing, Italy, 2022, pp. 1-5.

[18] M. A. Akhloufi, A. Couturier, and N. A. Castro, "Unmanned Aerial Vehicles for Wildland Fires: Sensing, Perception, Cooperation and Assistance," *Drones*, vol. 5, no. 1:15, 2021.

[19] Ghali, R.; Akhloufi, M.A.; Mseddi, W.S. "Deep Learning and Transformers Approaches for UAV Based Wildfire Detection and Segmentation," *Sensors*, vol. 22, no. 5:1977, 2022.