# Fusion of the Multisource Datasets for Flood Extent Mapping Based on Ensemble Convolutional Neural Network (CNN) Model

Authors: Seyd Teymoor Seydi; Vahideh Saeidi; Bahareh Kalantar; Naonori Ueda; J. L. van Genderen; Fattah Hatami Maskouni; Farzad Amini Aria
Journal: Journal of Sensors
 (2022)

---

## Abstract

Floods, as one of the natural hazards, can affect the environment, damage the infrastructures, and threaten human lives. Due to climate change and anthropogenic activities, floods occur in high frequency all over the world. Therefore, mapping of the flood areas is of prime importance in disaster management. This research presents a novel framework for flood area mapping based on heterogeneous remote sensing (RS) datasets. The proposed framework fuses the synthetic aperture radar (SAR), optical, and altimetry datasets for mapping flood areas, and it is applied in three main steps: (1) preprocessing, (2) deep feature extraction based on multiscale residual kernel convolution and convolution neural network’s (CNN) parameter optimization by fusing the datasets, and (3) flood detection based on the trained model. This research exploits two large-scale area datasets for mapping the flooded areas in Golestan and Khuzestan provinces, Iran. The results show that the proposed methodology has a high performance in flood area detection. The visual and numerical analyses verify the effectiveness and ability of the proposed method to detect the flood areas with an overall accuracy (OA) higher than 98% in both study areas. Finally, the efficiency of the designed architecture was verified by hybrid-CNN and 3D-CNN methods.


---

## Body

## 1. Introduction

Recently, climate and human impact on the ecosystem trigger natural hazards such as drought, floods, and wildfire with higher frequency [1–3]. Among these types of disasters, floods are considered as the most devastating and costly disaster worldwide [4, 5]. Mainly, flood is triggered by downpour, while other factors such as topography and altitude, slope, distance from river, and sediment transport index (STI) can affect the occurrences of the flood [6–8]. For the past years, many lives and agriculture crops were affected by floods in the whole world [9, 10]. Therefore, accurate and timely flood mapping is vital for emergency response planning during and after a flood [11–14]. Moreover, the flood mapping can be used as a base map in the future events for rescue, compensation insurance, and prediction of flood extension.Advances in remote sensing (RS) technologies and evolution in smart sensors and charge-coupled device cameras providing higher resolution, wider coverage, lower cost, continuous information, and timely revisiting mean that earth observation and its regular monitoring become viable [15–18]. RS is deployed in many applications such as disaster mapping [19–22], environment monitoring [23, 24], land use/cover mapping [25–30], and forest mapping [31, 32]. Due to improvement of spatial and temporal resolution of satellite imagery and availability of synthetic aperture radar (SAR) dataset, disaster mapping based on RS data has been converted into a hot topic [33, 34].Meanwhile, algorithm development and computer sciences revolutionized the accuracy and time of different sensors’ data processing in various fields such as RS, prediction of natural disaster, feature detection, and biomedical [35, 36]. The machine learning (ML) algorithms are among the most popular methods in the image processing and computer vision. The ML learns from the feature patterns within the input dataset (data samples as training dataset). In other words, the ML is a mathematical expression that represents data in the context of a problem. The ML methods are applied in two main categories: (1) supervised method by predicting some output variable associated with each input sample and (2) unsupervised method that does not need any sample data and provides a prediction by considering input feature dataset. The ML methods are widely deployed in many applications based on different sensors and datasets such as quasidistributed smart textile [37], simultaneous assessment of magnetic field intensity [38], paddy rice seed classification [39, 40], anime film visualization [41], eggplant seed classification [42], regional digital construction [43], flood mapping [44], and flood prevention [45]. Although the ML methods have provided promising results in many abovementioned applications, they suffer from lower coverage and generalization [1]. Accurate performance of ML highly depends on the input feature map which is a time-consuming process to extract suitable and informative features, especially in hazardous situations. In essence, flood hazard occurs in vast areas, and availability of various sensors from satellite platform can be an optimal solution for flood management. In this scenario, development of deep learning- (DL-) based methods led to overcome most of ML methods’ limitations by using multilayer architectures [8]. DL methods are capable of extracting deep feature from the big input data automatically and are suitable for multisensor dataset classification in a vast area such as flooded zones.Flood area mapping is one of the most important applications of the satellite imagery in natural hazard monitoring [46–49]. Recently, much research has been done for flood mapping based on multispectral optical dataset and SAR satellite imagery. For example, Cian et al. [50] proposed rapid flood monitoring algorithm based on the normalized difference flood index (NDFI) and normalized difference flood in vegetated areas index (NDFVI). The indices were applied on multitemporal SAR imagery for predicting flood areas. Then, the flood map was generated by thresholding of predicted areas and postprocessing manner. D’Addabbo et al. [51] designed a flood mapping toolbox (DAFNE) based on Bayesian networks. The DAFNE approach fused the multisource RS dataset for flood mapping and equipped by several modules for probability flood mapping: (1) image segmentation by K-means algorithm, (2) electromagnetic modeling module to compute the probability of flood occurrence based on input data, (3) image modeling module for estimating the probability of a pixel of image with respect to flood conditions, (4) ancillary data (geomorphic flooding index and distance from the river) module for modeling the ancillary data contribution with respect to the flood condition, and (5) probabilistic flood map computation module (the final steps of the DAFNE) to estimate the flood probability. Sharma et al. [52] described the use of computational modules to design event-driven flood management. Uddin et al. [53] proposed an operational methodology for flood mapping based on Sentinel-1 imagery. For this end, the RGB (red-green-blue) clustering technique was used for segmentation of the dataset, and then, a rule-based classifier was deployed for making decision on the clusters to classify flood or nonflood polygons. Annis and Nardi [54] showed how by integrating VGI and 2D hydraulic models into a data assimilation framework helped to improve real-time flood forecasting and mapping. Sarker et al. [55] presented a flood extent mapping method based on the convolution neural network (CNN) using Landsat-5 imagery. The proposed CNN included three convolution layers for deep feature extraction with a filter size of the first and second layer as 3×3 and for the last layer as 1×1. Feng et al. [56] showed a flood mapping framework by employing the optical dataset boarded on an unmanned aerial vehicle (UAV). The proposed framework was applied in three steps: (1) preprocessing (registration, mosaicking, and orthorectification), (2) texture feature extraction, and (3) image classification based on random forest (RF) classifier. Li et al. [57] presented a real-time flood mapping by the Suomi national polar-orbiting partnership- (Suomi-NPP-) visible infrared imaging radiometer suite (VIIRS) dataset. The algorithm tried to remove nonflood objects (shadow, cloud, ice, etc.) and map the flood areas based on change detection and decision tree (DT) classification. In a recent paper by Prins and Niekerk [58], they used Sentinel-2 imagery, aerial imagery, and LiDAR (light detection and ranging) for crop-type mapping. By fusing these datasets, together with ML algorithms, they could improve the overall accuracy (OA) [58]. The ML-based algorithms are capable of extracting complex features [59]. Gebrehiwot et al. [60] employed CNN for flood extent area mapping by using a UAV dataset. The used architecture was based on visual geometry group (VGG) fully convolutional network (FCN-16s). Denbina et al. [61] compared the performance U-Net and SegNet architectures for flood extent mapping with the UAV-SAR dataset. Their result demonstrated the higher accuracy of the U-Net model rather than SegNet. Mateo-Garcia et al. [7] evaluated the performance of simple CNN and U-Net architecture for flood extent mapping using Sentienl-2 satellite imagery. The performance of the DL-based method was compared with flood mapping by thresholding the normalized difference water index (NDWI). The results of flood mapping showed that the U-Net algorithm had higher efficiency compared to thresholding of the NDWI.Similarly, many other researchers widely used RS satellite datasets for flood mapping [62–64]. The results of flood mapping demonstrated the high potential of the multispectral RS dataset for flood mapping. However, there are some challenges in flood mapping using optical data that can affect the accuracy of mapping. Mainly, the challenges of flood mapping include (1) discrimination between the permanent water bodies and streams from flood; (2) timely and accurate detection of flooded areas as the flooded zones are usually covered by cloud; (3) constant thresholding for flood area detection in most of the algorithms (while the diversity of objects in a heterogeneous region does not support the constant thresholding for the whole region); (4) for change detection-based methods using RS imagery, access to optimal revisiting time dataset especially after flood is hard, leading to a delay on rapid flood mapping; and (5) the handcrafting feature (textural feature) extraction is a time-consuming process. Therefore, fusion of multisensor datasets and exploiting a robust model are primary solutions [65–67].To minimize the abovementioned challenges, the present DL framework is proposed. This research demonstrates a novel framework for flood mapping based on fusion of optical satellite imagery, SAR satellite dataset, and altimetry data. The Sentinel-1 SAR satellite imagery can penetrate clouds and has high potential for detection of water bodies. The Sentinel-2 satellite imagery is considered as an optical multispectral dataset with good temporal resolution. Therefore, the fusion of Sentinel-1 and Sentinel-2 can be complementary for flood mapping. Generally, the fusion of multisource datasets can be applied at three levels: (1) pixel level, (2) decision level, and (3) feature level [67–70]. The pixel-level fusion usually is used for enhancing of the spatial resolution of a low-resolution dataset by the pan dataset, such as pan-sharpening techniques [71]. The decision-level fusion is applied for integrating the result of many decisions [72, 73]. One of the limitations of decision-level fusion is its complexity to select a suitable method for fusion and set their parameters. The feature-level fusion is one the best techniques for fusion of the multisource datasets for specific application such as classification [29, 70, 74], flood mapping [75, 76], and damage mapping [77–80]. Due to the big structure of datasets in the present study, the best solution for multisource dataset integration is feature-level fusion. DL-based methods provide promising results as a robust tool in RS and image processing communities [24, 81]. The DL algorithm can automatically extract the deep features from the input data for flood mapping. In this study, the flood mapping is applied in three main steps: (1) preprocessing, (2) deep feature extraction based on multiscale kernel convolution and CNN’s parameter optimization by fusing the datasets, (3) and flood mapping based on optimized model. Mainly, the flood mapping is applied based on a single postflood dataset, while it is not sufficient to extract permanent water bodies during the flood events. For this end, presenting novel framework for flood mapping based on fusion (deep feature extraction from multisource datasets) of freely available datasets is more effective. Moreover, finding the postevent dataset without cloud coverage is a big challenge due to a specific atmospheric condition in such hazardous areas. The proposed framework uses the SAR dataset with capability to penetrate clouds and rains. Thus, the proposed method has high potential for flood mapping. Based on many studies and observations, the flood occurs in low-height-level areas (i.e., area adjacent to rivers). Therefore, the altimetry dataset is the key source information in accurately detecting such areas. It helps to improve the result of flood mapping that was underestimated by most studies. Generally, flood mapping by conventional methods works in two parts: feature extraction, and classification by ML methods. These methods require more time for processing, while the proposed method is applied in an end-to-end framework without any additional processing in a timely manner. Furthermore, the most methods fuse the multisource dataset in lower level; however, it is proved that the fusion of deep feature in a deeper level can improve the result of mapping [82]. Thus, the proposed method is able to extract deep features by parallel computing of CNN networks and fusing them in another network with a higher level of deep features. The key contributions of this research are as follows:
(I)
Presenting novel framework for flood mapping based on fusion (deep feature extraction from multisource datasets) of freely available datasets(II)
Deploying multidimensional kernel convolution for deep feature extraction(III)
Improving the results by fusing altimetry data in heterogeneous change detection for flood mapping(IV)
Deep feature extraction by combining spatial and spectral features instead of single spectral features(V)
Proposing an end-to-end framework without any additional processing


## 2. Study Area and Datasets

### 2.1. Study Area

In this study, two study areas were selected (Figure1(a)). The first one is located in the Aq Qala province in the north of Iran. The Gorganrood River is one of the most important rivers crossing Aq Qala city [83]. This province has a temperate climate with occasional heavy rainfall with a maximum height of 2100 m above mean sea level. On the 19th March 2019, the area was flooded after a heavy rainfall. The flood led to the loss of more than 20 people’s lives, and high damage to buildings, infrastructures, and wildlife was reported. Due to the special topography (lower slope toward the sea side) of Aq Qala province, the flood lasted for 20 days in the region worsening the damages. Figure 1(b) shows the location of the first study area. The second area is located in Khuzestan province, south of Iran. Karun River is one of the largest rivers in Khuzestan province [84, 85], and the maximum elevation of the region is 250 m above mean sea level. Similarly, after a heavy rainfall on 20th March 2019, flooding occurred in this province, as well. Mainly, the flood was as a result of human activities and construction in Karun River basin. The second study area is presented in Figure 1(c).Figure 1
Map of study areas: (a) map of Iran; (b) first study area and location of flood areas in the Aq Qala county; (c) second study area and location of flood areas in the Khuzestan county.
(a)(b)(c)


### 2.2. Datasets

This research used altimetry data, Sentinel-1, and Sentinel-2 datasets for flood mapping. Figure2 illustrates the datasets for mapping the flood areas in the Aq Qala province. The second study area is located in the Khuzestan province covering a large area (Figure 3).Figure 2
The used dataset in flood mapping for Aq Qala province: (a) Sentinel-1; (b) Sentinel-2; (c) DEM; (d) distribution of sample data.
(a)(b)(c)(d)Figure 3
The used dataset in flood mapping for Khuzestan province: (a) Sentinel-1 for postflood; (b) Sentinel-2 for preflood; (c) DEM; (d) distribution of sample data.
(a)(b)(c)(d)


#### 2.2.1. Sentinel-1 Data

Sentinel-1 is a constellation of two satellites A and 1B lunched on 3th April 2014 and 16th April 2016, respectively [86, 87]. The sensor is considered as an active sensor capturing data in a C-band, and it is equipped by two channels including two polarizations (VV and VH). As an active sensor, it is capable of day and night data collection in all weather conditions. For this research, the Level-1 ground range detected (GRD) product was used for flood mapping.


#### 2.2.2. Sentinel-2 Data

The Sentinel-2 mission comprises a constellation of two polar-orbiting satellites with an optical multispectral instrument [88]. Sentinel-2 collects data in 13 spectral bands in visible, near infrared (NIR), and shortwave infrared (SWIR) domains at different spatial resolutions from 10 m to 60 m. This satellite has the revisiting time of 5 days and is freely available at Sentinel Scientific Data Hub (https://scihub.copernicus.eu). Description of the datasets can be seen in Table 1.Table 1
The descriptions of datasets for flood mapping in study areas.
DataAq QalaKhuzestanPixel size (m)Sentinel-1 (postflood)2019-03-292019-04-0810Sentinel-2 ( preflood)2019-03-112019-03-1710ALOS altimetry data version 3.2Released on January 202130Data size5226×819213416×10580—


#### 2.2.3. Altimetry Data

The ALOS (advanced land observing satellite) World-3D-30m (AW3D30) provides a global digital surface model (DSM) dataset [89]. This dataset is obtained by Panchromatic Remote-sensing Instrument for Stereo Mapping [90]. The spatial resolution of this product is approximately 30 m. This product is freely available as well (https://www.eorc.jaxa.jp/ALOS/en/aw3d30/data/index.htm).


## 2.1. Study Area

In this study, two study areas were selected (Figure1(a)). The first one is located in the Aq Qala province in the north of Iran. The Gorganrood River is one of the most important rivers crossing Aq Qala city [83]. This province has a temperate climate with occasional heavy rainfall with a maximum height of 2100 m above mean sea level. On the 19th March 2019, the area was flooded after a heavy rainfall. The flood led to the loss of more than 20 people’s lives, and high damage to buildings, infrastructures, and wildlife was reported. Due to the special topography (lower slope toward the sea side) of Aq Qala province, the flood lasted for 20 days in the region worsening the damages. Figure 1(b) shows the location of the first study area. The second area is located in Khuzestan province, south of Iran. Karun River is one of the largest rivers in Khuzestan province [84, 85], and the maximum elevation of the region is 250 m above mean sea level. Similarly, after a heavy rainfall on 20th March 2019, flooding occurred in this province, as well. Mainly, the flood was as a result of human activities and construction in Karun River basin. The second study area is presented in Figure 1(c).Figure 1
Map of study areas: (a) map of Iran; (b) first study area and location of flood areas in the Aq Qala county; (c) second study area and location of flood areas in the Khuzestan county.
(a)(b)(c)


## 2.2. Datasets

This research used altimetry data, Sentinel-1, and Sentinel-2 datasets for flood mapping. Figure2 illustrates the datasets for mapping the flood areas in the Aq Qala province. The second study area is located in the Khuzestan province covering a large area (Figure 3).Figure 2
The used dataset in flood mapping for Aq Qala province: (a) Sentinel-1; (b) Sentinel-2; (c) DEM; (d) distribution of sample data.
(a)(b)(c)(d)Figure 3
The used dataset in flood mapping for Khuzestan province: (a) Sentinel-1 for postflood; (b) Sentinel-2 for preflood; (c) DEM; (d) distribution of sample data.
(a)(b)(c)(d)


### 2.2.1. Sentinel-1 Data

Sentinel-1 is a constellation of two satellites A and 1B lunched on 3th April 2014 and 16th April 2016, respectively [86, 87]. The sensor is considered as an active sensor capturing data in a C-band, and it is equipped by two channels including two polarizations (VV and VH). As an active sensor, it is capable of day and night data collection in all weather conditions. For this research, the Level-1 ground range detected (GRD) product was used for flood mapping.


### 2.2.2. Sentinel-2 Data

The Sentinel-2 mission comprises a constellation of two polar-orbiting satellites with an optical multispectral instrument [88]. Sentinel-2 collects data in 13 spectral bands in visible, near infrared (NIR), and shortwave infrared (SWIR) domains at different spatial resolutions from 10 m to 60 m. This satellite has the revisiting time of 5 days and is freely available at Sentinel Scientific Data Hub (https://scihub.copernicus.eu). Description of the datasets can be seen in Table 1.Table 1
The descriptions of datasets for flood mapping in study areas.
DataAq QalaKhuzestanPixel size (m)Sentinel-1 (postflood)2019-03-292019-04-0810Sentinel-2 ( preflood)2019-03-112019-03-1710ALOS altimetry data version 3.2Released on January 202130Data size5226×819213416×10580—


### 2.2.3. Altimetry Data

The ALOS (advanced land observing satellite) World-3D-30m (AW3D30) provides a global digital surface model (DSM) dataset [89]. This dataset is obtained by Panchromatic Remote-sensing Instrument for Stereo Mapping [90]. The spatial resolution of this product is approximately 30 m. This product is freely available as well (https://www.eorc.jaxa.jp/ALOS/en/aw3d30/data/index.htm).


## 2.2.1. Sentinel-1 Data

Sentinel-1 is a constellation of two satellites A and 1B lunched on 3th April 2014 and 16th April 2016, respectively [86, 87]. The sensor is considered as an active sensor capturing data in a C-band, and it is equipped by two channels including two polarizations (VV and VH). As an active sensor, it is capable of day and night data collection in all weather conditions. For this research, the Level-1 ground range detected (GRD) product was used for flood mapping.


## 2.2.2. Sentinel-2 Data

The Sentinel-2 mission comprises a constellation of two polar-orbiting satellites with an optical multispectral instrument [88]. Sentinel-2 collects data in 13 spectral bands in visible, near infrared (NIR), and shortwave infrared (SWIR) domains at different spatial resolutions from 10 m to 60 m. This satellite has the revisiting time of 5 days and is freely available at Sentinel Scientific Data Hub (https://scihub.copernicus.eu). Description of the datasets can be seen in Table 1.Table 1
The descriptions of datasets for flood mapping in study areas.
DataAq QalaKhuzestanPixel size (m)Sentinel-1 (postflood)2019-03-292019-04-0810Sentinel-2 ( preflood)2019-03-112019-03-1710ALOS altimetry data version 3.2Released on January 202130Data size5226×819213416×10580—


## 2.2.3. Altimetry Data

The ALOS (advanced land observing satellite) World-3D-30m (AW3D30) provides a global digital surface model (DSM) dataset [89]. This dataset is obtained by Panchromatic Remote-sensing Instrument for Stereo Mapping [90]. The spatial resolution of this product is approximately 30 m. This product is freely available as well (https://www.eorc.jaxa.jp/ALOS/en/aw3d30/data/index.htm).


## 3. Methodology

The proposed framework is applied in three steps according to the flowchart in Figure4. First, image preprocessing is applied in two main steps: (1) spectral correction such as calibration and atmospheric correction and (2) spatial correction that refers to geometry of pixels (e.g., registration and topographic correction). Secondly, to build optimum model by model parameter tuning based on a sample dataset, the proposed network is trained by reference sample data. The sample data is usually divided into three parts: (1) training data, (2) validation data, and (3) testing data. The training and validation datasets are exploited in the training process. Besides, the testing dataset evaluates the performance of the network. After training of the proposed network, the model is used for mapping the flooded areas. Finally, the whole flooded zones are mapped based on tuning and training process.Figure 4
Overview of the general framework for flood mapping.


### 3.1. Image Preprocessing

The image preprocessing is one of the most important steps in flood mapping. This process is applied to the Sentinel-1 and Sentinel-2 datasets, separately. In Snap software, the Sentinel-1 dataset preprocessing phases include applying orbit file, thermal noise removal, border noise removal, calibration, despeckle, and terrain correction [91]. The sentinel-2 Level-1-C product was preprocessed (i.e., geometric correction and radiometric correction) by the provider, and for this study, atmospheric correction was performed [92, 93] by the Sen2corr module in the Snap software.


### 3.2. Proposed Deep Learning Architecture

The proposed DL-based architecture for flood mapping is illustrated in Figure5. Accordingly, the framework has two parts: (1) ensemble deep feature extraction models and (2) classification. Mainly, the flood mapping is applied based on a single postflood dataset, while this dataset is not sufficient to extract permanent water bodies during the flood events. The main differences between the proposed architecture and other CNN frameworks are (1) utilizing multistream deep feature extraction instead of low-level feature concatenating that improves the performance of CNN in the extraction of deep features, (2) fusion of high-level deep features by a separate CNN for more deep feature exploration, (3) taking advantage of 3D kernel convolution to exploit the spectral information content of the Sentinel-2 imagery and then 2D kernel convolution for extracting high-level deep features, (4) employing multiscale kernel convolution layers which have a better performance against the scale variations and increasing the robustness of network against the change of object sizes.Figure 5
The structure of the proposed deep learning based framework for flood mapping.


#### 3.2.1. Deep Feature Extraction

The main core of the CNN method is convolution layers to extract the deep features, automatically [94, 95]. Convolution layers investigate the spatial and spectral features by embedding filters [94]. The extracted deep features from first layers are fed into the next layers in a hierarchical manner. Therefore, the qualities of deep extracted features depend on the design of the architecture of the network. In this study, Figures 6(a)–6(d) show the four CNN models for deep feature extraction.Figure 6
The details of the proposed CNN framework for flood area mapping.
(a)
First CNN model for Sentinel-1(b)
Second CNN model for Sentinel-2(c)
Third CNN model for altimetry data(d)
Fourth CNN fusion deep features and classificationMainly, feature extraction from multisource datasets by the traditional method is based on staking datasets and then extracting the features [65, 96, 97]. The stacking layer generates high-dimensional data leading to dimensionality and overfitting problems, and it might affect the quality of the extracted features and, as a consequence, fail to detect some details [98]. Therefore, this research presents a novel architecture based on ensemble deep feature extraction models. The proposed architecture includes three CNN models (i.e., CNN-1, CNN-2, and CNN-3) for deep feature extraction (separately for each dataset) and one CNN model (i.e., CNN-4) for classification. Then, the concatenate function stacks the extracted deep features from three previous models and feeds it to the next layers for extracting high-level deep features. The CNN-1 and CNN-3 models investigate deep features based on 2D kernel convolutions from SAR and altimetry datasets, respectively. The CNN-2 model extracts the deep features based on hybrid 2D and 3D kernel convolutions from the optical dataset. The main differences between CNN-3 and CNN-2 and CNN-1 are in the number of convolution layers and the type of kernel convolution. The difference of 2D and 3D kernel convolution is presented in Figure 7. The 3D kernel convolution allows to fully deploy the content of spectral information [99]. The final concatenate function explores the deep features based on 2D CNN layers by taking the outputs of three CNN models and stacking the deep features.Figure 7
The main difference of 2D and 3D kernel convolution: (a) 2D kernel convolution; (b) 3D kernel convolution.
(a)(b)


#### 3.2.2. Classification

After feature extraction, the deep features are converted into a one-dimensional vector by a flatten layer. The feature vectors are fed, first to fully connected layers, and then are transformed to fully connect layers. The latest layer is “soft-max” that assigns probabilities to each class for the input pixel. Figure6(d) presents the classification procedure for this framework.


#### 3.2.3. Evaluation Indices and Accuracy Assessment

Accuracy assessment is necessary in any RS dataset analysis and modeling [15]. Hence, this research evaluates the result of flood mapping based on numerical and visual analysis. The numerical analysis has been conducted by measurement indices. The most common quantitative assessment metrics, namely, OA, kappa coefficient (KC), user accuracy (UA), and producer accuracy (PA), are selected for evaluating the results of flood mapping. The accuracy assessment is applied by evaluating the sample data (Figures 2(d) and 3(d)) derived from the reference map. Furthermore, the sample data is divided into three parts, namely, the training data (65% of the samples), validation (15% of the samples), and testing (20% of the samples) (Table 2) [2]. In addition, the performance of the proposed method is compared with two state-of-the-art deep learning-based methods, the hybrid-CNN- [90, 100] based method and 3D-CNN [101, 102].Table 2
The number of training data for two classes.
Study areaName classWhole sampleTraining (65%)Validation (15%)Testing (20%)Aq QalaFlood45672969685913Nonflood10459679815692092KhuzestanFlood7340477111011468Nonflood12962842519442593


## 3.1. Image Preprocessing

The image preprocessing is one of the most important steps in flood mapping. This process is applied to the Sentinel-1 and Sentinel-2 datasets, separately. In Snap software, the Sentinel-1 dataset preprocessing phases include applying orbit file, thermal noise removal, border noise removal, calibration, despeckle, and terrain correction [91]. The sentinel-2 Level-1-C product was preprocessed (i.e., geometric correction and radiometric correction) by the provider, and for this study, atmospheric correction was performed [92, 93] by the Sen2corr module in the Snap software.


## 3.2. Proposed Deep Learning Architecture

The proposed DL-based architecture for flood mapping is illustrated in Figure5. Accordingly, the framework has two parts: (1) ensemble deep feature extraction models and (2) classification. Mainly, the flood mapping is applied based on a single postflood dataset, while this dataset is not sufficient to extract permanent water bodies during the flood events. The main differences between the proposed architecture and other CNN frameworks are (1) utilizing multistream deep feature extraction instead of low-level feature concatenating that improves the performance of CNN in the extraction of deep features, (2) fusion of high-level deep features by a separate CNN for more deep feature exploration, (3) taking advantage of 3D kernel convolution to exploit the spectral information content of the Sentinel-2 imagery and then 2D kernel convolution for extracting high-level deep features, (4) employing multiscale kernel convolution layers which have a better performance against the scale variations and increasing the robustness of network against the change of object sizes.Figure 5
The structure of the proposed deep learning based framework for flood mapping.


### 3.2.1. Deep Feature Extraction

The main core of the CNN method is convolution layers to extract the deep features, automatically [94, 95]. Convolution layers investigate the spatial and spectral features by embedding filters [94]. The extracted deep features from first layers are fed into the next layers in a hierarchical manner. Therefore, the qualities of deep extracted features depend on the design of the architecture of the network. In this study, Figures 6(a)–6(d) show the four CNN models for deep feature extraction.Figure 6
The details of the proposed CNN framework for flood area mapping.
(a)
First CNN model for Sentinel-1(b)
Second CNN model for Sentinel-2(c)
Third CNN model for altimetry data(d)
Fourth CNN fusion deep features and classificationMainly, feature extraction from multisource datasets by the traditional method is based on staking datasets and then extracting the features [65, 96, 97]. The stacking layer generates high-dimensional data leading to dimensionality and overfitting problems, and it might affect the quality of the extracted features and, as a consequence, fail to detect some details [98]. Therefore, this research presents a novel architecture based on ensemble deep feature extraction models. The proposed architecture includes three CNN models (i.e., CNN-1, CNN-2, and CNN-3) for deep feature extraction (separately for each dataset) and one CNN model (i.e., CNN-4) for classification. Then, the concatenate function stacks the extracted deep features from three previous models and feeds it to the next layers for extracting high-level deep features. The CNN-1 and CNN-3 models investigate deep features based on 2D kernel convolutions from SAR and altimetry datasets, respectively. The CNN-2 model extracts the deep features based on hybrid 2D and 3D kernel convolutions from the optical dataset. The main differences between CNN-3 and CNN-2 and CNN-1 are in the number of convolution layers and the type of kernel convolution. The difference of 2D and 3D kernel convolution is presented in Figure 7. The 3D kernel convolution allows to fully deploy the content of spectral information [99]. The final concatenate function explores the deep features based on 2D CNN layers by taking the outputs of three CNN models and stacking the deep features.Figure 7
The main difference of 2D and 3D kernel convolution: (a) 2D kernel convolution; (b) 3D kernel convolution.
(a)(b)


### 3.2.2. Classification

After feature extraction, the deep features are converted into a one-dimensional vector by a flatten layer. The feature vectors are fed, first to fully connected layers, and then are transformed to fully connect layers. The latest layer is “soft-max” that assigns probabilities to each class for the input pixel. Figure6(d) presents the classification procedure for this framework.


### 3.2.3. Evaluation Indices and Accuracy Assessment

Accuracy assessment is necessary in any RS dataset analysis and modeling [15]. Hence, this research evaluates the result of flood mapping based on numerical and visual analysis. The numerical analysis has been conducted by measurement indices. The most common quantitative assessment metrics, namely, OA, kappa coefficient (KC), user accuracy (UA), and producer accuracy (PA), are selected for evaluating the results of flood mapping. The accuracy assessment is applied by evaluating the sample data (Figures 2(d) and 3(d)) derived from the reference map. Furthermore, the sample data is divided into three parts, namely, the training data (65% of the samples), validation (15% of the samples), and testing (20% of the samples) (Table 2) [2]. In addition, the performance of the proposed method is compared with two state-of-the-art deep learning-based methods, the hybrid-CNN- [90, 100] based method and 3D-CNN [101, 102].Table 2
The number of training data for two classes.
Study areaName classWhole sampleTraining (65%)Validation (15%)Testing (20%)Aq QalaFlood45672969685913Nonflood10459679815692092KhuzestanFlood7340477111011468Nonflood12962842519442593


## 3.2.1. Deep Feature Extraction

The main core of the CNN method is convolution layers to extract the deep features, automatically [94, 95]. Convolution layers investigate the spatial and spectral features by embedding filters [94]. The extracted deep features from first layers are fed into the next layers in a hierarchical manner. Therefore, the qualities of deep extracted features depend on the design of the architecture of the network. In this study, Figures 6(a)–6(d) show the four CNN models for deep feature extraction.Figure 6
The details of the proposed CNN framework for flood area mapping.
(a)
First CNN model for Sentinel-1(b)
Second CNN model for Sentinel-2(c)
Third CNN model for altimetry data(d)
Fourth CNN fusion deep features and classificationMainly, feature extraction from multisource datasets by the traditional method is based on staking datasets and then extracting the features [65, 96, 97]. The stacking layer generates high-dimensional data leading to dimensionality and overfitting problems, and it might affect the quality of the extracted features and, as a consequence, fail to detect some details [98]. Therefore, this research presents a novel architecture based on ensemble deep feature extraction models. The proposed architecture includes three CNN models (i.e., CNN-1, CNN-2, and CNN-3) for deep feature extraction (separately for each dataset) and one CNN model (i.e., CNN-4) for classification. Then, the concatenate function stacks the extracted deep features from three previous models and feeds it to the next layers for extracting high-level deep features. The CNN-1 and CNN-3 models investigate deep features based on 2D kernel convolutions from SAR and altimetry datasets, respectively. The CNN-2 model extracts the deep features based on hybrid 2D and 3D kernel convolutions from the optical dataset. The main differences between CNN-3 and CNN-2 and CNN-1 are in the number of convolution layers and the type of kernel convolution. The difference of 2D and 3D kernel convolution is presented in Figure 7. The 3D kernel convolution allows to fully deploy the content of spectral information [99]. The final concatenate function explores the deep features based on 2D CNN layers by taking the outputs of three CNN models and stacking the deep features.Figure 7
The main difference of 2D and 3D kernel convolution: (a) 2D kernel convolution; (b) 3D kernel convolution.
(a)(b)


## 3.2.2. Classification

After feature extraction, the deep features are converted into a one-dimensional vector by a flatten layer. The feature vectors are fed, first to fully connected layers, and then are transformed to fully connect layers. The latest layer is “soft-max” that assigns probabilities to each class for the input pixel. Figure6(d) presents the classification procedure for this framework.


## 3.2.3. Evaluation Indices and Accuracy Assessment

Accuracy assessment is necessary in any RS dataset analysis and modeling [15]. Hence, this research evaluates the result of flood mapping based on numerical and visual analysis. The numerical analysis has been conducted by measurement indices. The most common quantitative assessment metrics, namely, OA, kappa coefficient (KC), user accuracy (UA), and producer accuracy (PA), are selected for evaluating the results of flood mapping. The accuracy assessment is applied by evaluating the sample data (Figures 2(d) and 3(d)) derived from the reference map. Furthermore, the sample data is divided into three parts, namely, the training data (65% of the samples), validation (15% of the samples), and testing (20% of the samples) (Table 2) [2]. In addition, the performance of the proposed method is compared with two state-of-the-art deep learning-based methods, the hybrid-CNN- [90, 100] based method and 3D-CNN [101, 102].Table 2
The number of training data for two classes.
Study areaName classWhole sampleTraining (65%)Validation (15%)Testing (20%)Aq QalaFlood45672969685913Nonflood10459679815692092KhuzestanFlood7340477111011468Nonflood12962842519442593


## 4. Results

The results of flood mapping and the performances of our proposed method for the two study areas are discussed in this section.


### 4.1. Parameter Setting

The DL-based methods have some parameters to be set. These parameters are set manually based on trial and error. Here, the optimum values of the proposed method’s parameters are as follows: input patch size11×11; batch size 1000; number of neurons in the first and second layer 1500 and 500, respectively; and number of epochs 300 and learning rate 10-3. The proposed method is trained by an Adam (Adaptive Moment Estimation) optimizer [103] in a backpropagation manner, and all weights are initialized by Glorot initialization [104]. Furthermore, the shuffle technique is deployed for increasing the performance of the network during the training process. Moreover, these set values are applied for the hybrid-CNN and 3D-CNN methods.


### 4.2. Aq Qala Flood

The results of flood mapping for the first study area are presented in Figure8. The reference map is presented in Figure 8(d). The visual inspection of the proposed method (Figure 8(c)) shows that it provides a better performance compared to the hybrid-CNN and 3D-CNN methods (Figures 8(a) and 8(b)) in the Aq Qala region. There are some false pixels in the flood map resulting from the hybrid-CNN method, especially in the north-west and south-east of the study area. Besides, the effect of flooded river (north-east) can be seen in the result of the proposed method, while the hybrid-CNN method fails to point out those flooded pixels. Generally, the proposed method exhibits a less “pepper and salt” look.Figure 8
The result of flood mapping for Aq Qala county: (a) 3D-CNN; (b) hybrid-CNN; (c) proposed Flood-Net; (d) ground truth (reference map).
(a)(b)(c)(d)The numerical result of flood mapping for the Aq Qala province is provided in Table3. The best OA is obtained by the proposed method (98.07%) followed by hybrid-CNN (97.27%) and 3D-CNN (95.59%). Predominantly, all methods provide an accuracy better than 76% against OA, F1-score, precision, and recall. Vividly, the performances of the proposed method are higher and more accurate (up to 9.57%) than the hybrid-CNN and 3D-CNN against all indices. On the other hand, 3D-CNN exhibits the less certain classification of the flooded area by a kappa of 0.705.Table 3
Accuracy assessment of flood mapping for Aq Qala province.
MethodOA (%)Precision (%)Recall (%)F1-score (%)KC3D-CNN95.5965.8593.7877.370.705Hybrid-CNN97.2776.8094.6684.800.833Proposed method98.0777.8598.4586.940.859


### 4.3. Khuzestan Flood

The results of flood mapping for Khuzestan province are shown in Figure9. Similarly, the proposed method provides higher performance for mapping the flooded areas compared with the reference map (Figure 9(d)). In general, the 3D-CNN (Figure 9(a)) and hybrid-CNN (Figure 9(b)) have similar performance in the detection of flood areas, misclassifying many of nonflood pixels as flood pixels. This pattern is obvious in vast areas in the town (south, south-east, and south-west) in Khuzestan region showing many false pixels. Consequently, the false positive in the flood map might distract the analysis and decision making during the hazard. In contrast, the proposed method effectively detects the stream, river, and water bodies (nonflood) pixels. Specifically, south-west and southern parts of Khuzestan belong to the Shadegan wetland which hybrid-CNN and 3D-CNN methods mistakenly classified as flooded areas. Furthermore, there are many noisy pixels and “salt and pepper” appearances in the result of hybrid-CNN and 3D-CNN methods, whereas the map produced by our proposed method has a smooth and clear appearance and the effects of noisy pixels are insignificant.Figure 9
The result of flood mapping for Khuzestan county: (a) 3D-CNN, (b) hybrid-CNN, (c) proposed Flood-Net, and (d) reference map.
(a)(b)(c)(d)The accuracy assessments of flood mapping against the evaluation indices are shown in Table4. Unlike the previous study area, the hybrid-CNN and 3D-CNN methods show disappointing results, while the proposed method proves to be highly accurate and certain. The accuracy of the proposed method (OA=99.82%, precision=97.94) is considerably higher than that of hybrid-CNN (OA=80.66% and precision=29.1%) and 3D-CNN (OA=79.77% and precision=27.06%) referring to higher level of reliability and robustness in the result.Table 4
Accuracy assessment of flood mapping for Khuzestan province.
MethodOA (%)Precision (%)Recall (%)F1-score (%)KC3D-CNN79.7727.0681.8240.670.0302Hybrid-CNN80.6629.189.2043.880.357Proposed method99.8297.9499.7098.960.984


## 4.1. Parameter Setting

The DL-based methods have some parameters to be set. These parameters are set manually based on trial and error. Here, the optimum values of the proposed method’s parameters are as follows: input patch size11×11; batch size 1000; number of neurons in the first and second layer 1500 and 500, respectively; and number of epochs 300 and learning rate 10-3. The proposed method is trained by an Adam (Adaptive Moment Estimation) optimizer [103] in a backpropagation manner, and all weights are initialized by Glorot initialization [104]. Furthermore, the shuffle technique is deployed for increasing the performance of the network during the training process. Moreover, these set values are applied for the hybrid-CNN and 3D-CNN methods.


## 4.2. Aq Qala Flood

The results of flood mapping for the first study area are presented in Figure8. The reference map is presented in Figure 8(d). The visual inspection of the proposed method (Figure 8(c)) shows that it provides a better performance compared to the hybrid-CNN and 3D-CNN methods (Figures 8(a) and 8(b)) in the Aq Qala region. There are some false pixels in the flood map resulting from the hybrid-CNN method, especially in the north-west and south-east of the study area. Besides, the effect of flooded river (north-east) can be seen in the result of the proposed method, while the hybrid-CNN method fails to point out those flooded pixels. Generally, the proposed method exhibits a less “pepper and salt” look.Figure 8
The result of flood mapping for Aq Qala county: (a) 3D-CNN; (b) hybrid-CNN; (c) proposed Flood-Net; (d) ground truth (reference map).
(a)(b)(c)(d)The numerical result of flood mapping for the Aq Qala province is provided in Table3. The best OA is obtained by the proposed method (98.07%) followed by hybrid-CNN (97.27%) and 3D-CNN (95.59%). Predominantly, all methods provide an accuracy better than 76% against OA, F1-score, precision, and recall. Vividly, the performances of the proposed method are higher and more accurate (up to 9.57%) than the hybrid-CNN and 3D-CNN against all indices. On the other hand, 3D-CNN exhibits the less certain classification of the flooded area by a kappa of 0.705.Table 3
Accuracy assessment of flood mapping for Aq Qala province.
MethodOA (%)Precision (%)Recall (%)F1-score (%)KC3D-CNN95.5965.8593.7877.370.705Hybrid-CNN97.2776.8094.6684.800.833Proposed method98.0777.8598.4586.940.859


## 4.3. Khuzestan Flood

The results of flood mapping for Khuzestan province are shown in Figure9. Similarly, the proposed method provides higher performance for mapping the flooded areas compared with the reference map (Figure 9(d)). In general, the 3D-CNN (Figure 9(a)) and hybrid-CNN (Figure 9(b)) have similar performance in the detection of flood areas, misclassifying many of nonflood pixels as flood pixels. This pattern is obvious in vast areas in the town (south, south-east, and south-west) in Khuzestan region showing many false pixels. Consequently, the false positive in the flood map might distract the analysis and decision making during the hazard. In contrast, the proposed method effectively detects the stream, river, and water bodies (nonflood) pixels. Specifically, south-west and southern parts of Khuzestan belong to the Shadegan wetland which hybrid-CNN and 3D-CNN methods mistakenly classified as flooded areas. Furthermore, there are many noisy pixels and “salt and pepper” appearances in the result of hybrid-CNN and 3D-CNN methods, whereas the map produced by our proposed method has a smooth and clear appearance and the effects of noisy pixels are insignificant.Figure 9
The result of flood mapping for Khuzestan county: (a) 3D-CNN, (b) hybrid-CNN, (c) proposed Flood-Net, and (d) reference map.
(a)(b)(c)(d)The accuracy assessments of flood mapping against the evaluation indices are shown in Table4. Unlike the previous study area, the hybrid-CNN and 3D-CNN methods show disappointing results, while the proposed method proves to be highly accurate and certain. The accuracy of the proposed method (OA=99.82%, precision=97.94) is considerably higher than that of hybrid-CNN (OA=80.66% and precision=29.1%) and 3D-CNN (OA=79.77% and precision=27.06%) referring to higher level of reliability and robustness in the result.Table 4
Accuracy assessment of flood mapping for Khuzestan province.
MethodOA (%)Precision (%)Recall (%)F1-score (%)KC3D-CNN79.7727.0681.8240.670.0302Hybrid-CNN80.6629.189.2043.880.357Proposed method99.8297.9499.7098.960.984


## 5. Discussion

Accurate and timely flood mapping plays a significant role in disaster management, and RS satellite imagery provides valuable information wherever direct data collection is not viable. This research conducted a novel framework for flood mapping based on DL and multisource data fusion. According to the qualitative and quantitative assessment of the algorithms, all methods classified the first study area (Aq Qala, Golestan) with high accuracy and certainty on the sample data. However, the two CNN models failed to detect the flooded areas in the second zone (Khuzestan). Our finding was in agreement with [55] using support vector machine (SVM) to detect flooded areas. The reason for the misclassified areas in Khuzestan by the two aforementioned models is the presence of wetlands and ponds in the region which created a complex scene to be identified by other algorithms. The higher performances of the proposed method verified the robustness of this ensemble model using 10 m resolution satellite datasets in flood mapping in the two study areas in the north and south of Iran.One of the most important issues in rapid flood mapping based on satellite imagery is the availability of RS datasets. Mainly, the flood-prone areas are often cloudy, and the optical imagery cannot provide a good coverage, reversing the speed and accuracy of flood mapping. The fusion of multisource datasets can be the best solution to meet this challenge. This research used the feature-level fusion for mapping the flooded areas. Three CNN-based architectures were designed. The deep features were extracted by the three models, and then, the stacked results were fed to the latest CNN model and classification layer to reduce the dimensionality of the big data. The hybrid-CNN- and 3D-CNN-based methods stack the multisource datasets directly and then extract the deep features and classify them. This kind of fusion might fail to detect the details leading to unreliable result. The proposed method fused the high-level deep features leading to reliable flood map results. Moreover, the ensemble 3D and 2D structure and design of the proposed model rather than only 3D kernel in 3D-CNN could significantly improve the accuracy of the flood detection.Mainly, the flood mapping is conducted based on moderate spatial and high temporal resolution dataset such as MODIS and VIRIS. These datasets have sufficient revisit time for earth observation and hazard monitoring, but they suffer from low spatial resolution (300 m) and some small areas (rivers prone to flood with width under 300 m) are not detected in the aforementioned RS datasets due to low resolution and missing pixels. The Sentinel series of satellite imagery with high spatial resolution (10 m) proves to be effective in the mapping of flood areas with more details. The altimetry dataset also proves promising results in many RS applications. This research takes advantages of altimetry data and fusion in flood mapping by providing promising results. Most of the algorithms consider the water bodies as flood in the flood mapping, which is very subjective, while the proposed methodology overcomes this challenge by fusion of multisource data in the complex and heterogeneous scene and wetlands.The CNN-based methods are applied by the different architectures for flood mapping. The suitable design architecture plays the key role in obtaining accurate results in flood mapping. Based on presented results, hybrid-CNN, 3D-CNN, and the proposed method use the deep features, but they provide different results that emphasize the benefits of hybrid 2D and 3D convolution, instead of only 3D convolution in flood mapping. Editing some false-negative or false-positive pixels based on postprocessing filter such as majority voting and refining based on professional experts (relies on expert knowledge from the region) would be heavily time-consuming and costly [105]. The presences of false positives and misclassifications in both hybrid-CNN and 3D-CNN methods increase the postclassification processing and decrease the reliability of decision making at the time of the hazard. It refers to the importance of deep feature extraction manner in DL-based methods. This research successfully designed the novel architecture based on ensemble CNN models intended to extract high-level features to obtain high accuracy and reliability for flood mapping and support decision making. Therefore, the use of the robust ensemble algorithm (instead of hybrid-CNN and 3D-CNN) for such dynamic phenomena is recommended.


## 6. Conclusion

Accurate, reliable, and timely detection of flooded areas is vital in disaster management. This study presents a novel flood mapping framework based on fusion of multisource datasets at the feature level. The preflood Sentinel-2 dataset, postflood Sentinel-1, and the altimetry dataset were used for flood area mapping, and an ensemble of 2D and 3D CNN models were deployed for deep feature extraction. The performance of the proposed method was compared with other state-of-the-art methods (e.g., hybrid-CNN) in two flooded regions. The accuracy assessment of the results showed that the multisource RS datasets have a high potential in flood area detection. Therefore, the fusion of active and passive RS datasets is one of the best solutions in areas with high cloud coverage and rainfall. The result of this study proved that the proposed method has high efficiency in mapping of flooded areas and more certainty in heterogeneous zones. Generally, the proposed method has many advantages in flood mapping compared to other methods including (1) high accuracy in mapping flood areas and a better fit with multisource datasets and big data; (2) applying in an end-to-end framework without any additional processing such as manual feature extraction; (3) providing lower false or miss detection pixels in mapping flood areas, in addition to high sensitivity to permanent water areas (e.g., lake and river); and (4) the need to use a lower number of sample data compared to other architecture such as semantic segmentation based on DL methods.



---

*Source: 2887502-2022-03-19.xml*