Deforestation estimation and fire detection in the Amazon forest poses a significant challenge due to the vast size of the area and the limited accessibility. However, these are crucial problems that lead to severe environmental consequences, including climate change, global warming, and biodiversity loss. To effectively address this problem, multimodal satellite imagery and remote sensing offer a promising solution for estimating deforestation and detecting wildfire in the Amazonia region. This research paper introduces a new curated dataset and a deep learningbased approach to solve these problems using convolutional neural networks (CNNs) and comprehensive data processing techniques. Our dataset includes curated images and diverse channel bands from Sentinel, Landsat, VI-IRS, and MODIS satellites. We design the dataset considering different spatial and temporal resolution requirements. Our method successfully achieves high-precision deforestation estimation and burned area detection on unseen images from the region. Our code, models and dataset are open source: The Amazon forest, also known as Amazonia, encompasses approximately 40% of the remaining rainforests in the world. Unfortunately, deforestation in the Amazon is reaching dangerous levels. In 2012, Brazil achieved an unprecedented feat among tropical countries by reducing deforestation rates in Amazonia by 84% (4,571 km2) compared to the historical peak of 2004, when 27,772 km 2 of forests were clear-cut [27]. In August 2022, deforestation in the Amazon was 1,661 km2, an amount 81% higher than that recorded in August last year [31]. In this context, multimodal satellite imagery offer a powerful alternative for estimating deforestation and detecting wildfire in such regions. Some of the most important research centers are the (INPE), by Satellite (PRODES) and Real-time Deforestation Detection System (DETER) [12,15]. The recent advances in machine learning (ML) allow us to improve traditional image processing using deep convolutional neural networks (CNNs). These techniques show already promising results for this task [11,14,19,21,22,32]. As part of this effort, we can find the MultiEarth 2023 [10] challenges that aim to monitor the Amazon rainforest in all weather and lighting conditions using a multimodal remote sensing dataset, which includes a time series of multispectral and synthetic aperture radar (SAR) images. In this work we discuss a deep learning framework to support the interpretation and analysis of the Amazon rainforest (and other areas in South America) at different time and weather conditions. We tackle the two core tasks: 1. Deforestation segmentation: Using images from the satellites provided by the organizers, we aim to accurately segment the deforested areas. 2. Fire segmentation: We suggest to use additional well maintained satellite data products to segment burned areas. In the deep learning (DL) era, we can use deep convolutional neural networks (CNNs) to automatically detect and recognize patterns in images. In this work we focus on image segmentation, the problem of identifying with a pixellevel accuracy the areas where an object (e.g. person, car, building, tree, specific area) is present in an image. Multiple works [9,24] on remote sensing proposed to leverage the growing availability of open Earth Observation (EO) data and the advances in deep learning research to tackle Earth system problems. Ban et al. introduced a method for near real-time wildfire monitoring using deep learning and images from Sentinel-1 satellite i.e. Synthetic Aperture Radar (SAR) [6]. Prapas et al. also study wildfire tracking with deep learning as a segmentation problem over time [21,22]. Using such remote sensing techniques, we can collect multiple observations per day at different weather conditions. Multiple works show the potential of using Sentinel-1 to detect wildfires and capture their temporal progression, also related to deforestation tracking [6,19,23,25]. Similarly, Lee et al. used diverse channels from three satellites, Sentinel-1, Sentinel-2 and Landsat-8 to train deep neural networks for estimating deforestation in the Amazon Forest via image segmentation [19]. Datasets Despite the importance of this problem, there is a limited amount of open-access datasets, especially for wildfire where we need variables related to the fire drivers and burned areas [5,10,22]. Kondylatos [18] et al. introduced a dataset of wildfire forecasting in the Eastern Mediterranean. We also find datasets that cover US areas [13,28]. We use as baseline the MultiEarth 1 dataset [10,11]. This is a multimodal remote sensing dataset with manually annotated binary masks for deforestation, in a small region in the Amazon rainforest. Our early experiments show that using the provided Sentinel-1, Sentinel-2, and Landsat-8 (as indicated in [10]) we can successfully train neural networks to detect deforestation obtaining accurate results. However, for the particular problem of wildfire detection, we found fundamental (as we show in Section 4) to use additional satellite data. We summarize the proposed dataset in We include images captured from the different satellites in Figures 3, 4 and 5. We refer the reader to [10] for more information about the data from these satellites [1,29] 1. Satellite Specifications. We provide the time periods available for each sensor, and the selected band intervals. See also [11]. 2019-06-06 2019-06-22 2019-07-08 2019-07-24 VNP13A1 [4] provides vegetation indices (EVI, EVI2, NDVI) at 500 meter resolution. This was designed after the MODIS vegetation indices. Along with the three vegetation layers, also includes layers for NIR reflectance -three shortwave infrared (SWIR)-; red, blue, and green reflectance. MCD15A2H [2] 8-day composite dataset with 500 meter pixel resolution. We can obtain the Leaf Area Index (LAI) and the Fraction of Photosynthetically Active Radiation (FPAR) metrics. Their algorithm chooses the best pixel available from all the acquisitions of both MODIS sensors in the 8-day period. We use a common deep learning pipeline to solve deforestation and burned area segmentation, this is illustrated in We use a state-of-the-art image segmentation architecture based on U-Net [26] with EfficientNet-B3 [30] as encoder. The model is illustrated in We use this model to solve both tasks, with necessary changes in the data loaders. The training procedure is very similar, using the corresponding ground truth binary masks for each task. To solve this task, we use only the provided datasets from Sentinel 1-2 and Landsat 8 (as in [10,19]).
The Landsat 8 images were upsampled to match the 10-meter pixel resolution of the masks. We believe the additional vegetation indices (Enhanced Vegetation Index, Fraction of Photosynthetically Active Radiation, etc.) and related metrics would be useful, however the resolution difference is too large. Given ideal weather conditions and continuous observations the deforestation segmentation is trivial, missing satellite data and frequent cloud occlusion makes it a challenging task. Instead of training separate models for each input satellites and apply post-processing later as [19], we propose training a single deep neural network on multi-channel images combining all available input dates and bands. Since we did not have exact timestamps for the segmentation label (binary masks), we extracted all relevant bands considering a +/-2 months window. Validation As the annotated binary masks were created for a small region across several years, the samples show high spatial and temporal correlation. To simulate the train/test separation we used spatial grid for cross validation . In the training region there is still a large forested area where deforestation has not happened yet; the models could learn this quickly, therefore, we downsampled this region to avoid "easy regions" that could boost the validation score while tuning network parameters. Training The model (see Section 4.1) was trained by minimizing the (BCE) with batch size 32, using AdamW optimizer with cosine scheduler, during 10 epochs. We used simple non-destructive augmentations from the dehidral group e.g. horizontal and vertical flips, transpose and 90  rotation. Further rotation or scaling did not help the generalization to the unseen test region. As the test queries had even more missing data we implemented satellite input dropout, up to half of the available imagery dates were forgotten randomly during training. The complete implementation details can be consulted in our github project page. Results Each experiment takes 2-3 hours using 3 NVIDIA Quadro RTX 6000 cards. Our best single model [26] segmentation model with EfficientNet [30] encoder. The output binary mask (1-channel image) is generated with a simple convolutional block (segmentation head). We show the number of channels for each decoder convolutional block. The number of input channels N varies between experiments, from 88 to 240, depending on the number of bands and recent images available. Team achieves 90.4 Pixel Accuracy, 0.871 F1 and 0.792 IoU on the unseen test area . Training several similar models with small configuration changes and blending raw predictions before binarization gives small performance boost up to 90.8 Pixel Accuracy, 0.876 F1 and 0.799 IoU. Half of the test queries were collected before 2013, we could only use Landsat 5 images from the provided datasets. The infrequent images and cloud occlusions made much more challenging to detect monthly burned areas accurately. We had a few similar binary segmentation experiments with moderate results (AUC 0.75). The ground truth FIRE CCI dataset has much lower resolution than the provided satellite imagery; it allows us to use external datasets with higher temporal resolution. All additional datasets (w.r.t Multiearth [10]) are still maintained allowing us to train global fire detection systems. For a larger experiment we downloaded 8 years satellite imagery covering all South America. Thanks to the lower resolution, the total dataset takes less than 100GB, allowing us to train segmentation models for full South America. Every dataset was upsampled to match the Fire CCI [5] resolution and the whole region were split into 256x256 tiles keeping only the relevant regions with at least 1/5 land surface. As we do not need manual annotations we suggest temporal validation split and use 2020 as validation set. Using all suggested the proposed dataset 3, from the target month and the previous three months, we are able to detect burned areas with 0.95 AUC for South America. We present a deep learning model for wildfire and deforestation detection. Our deforestation model achieves great performance combining all provided data sources. The proposed pipeline is easy to generalize and adapt for other satellite imagery tasks. Additionally, we propose new datasets for rapid fire detection at global scale. Early fire segmentation experiments show promising results by reaching 0.95 pixel-wise AUC for the South America region. Using the studied datasets, it is feasible to build global fire hazard prediction models.