SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Table 1: Results of the fire danger forecasting track
Model	Precision	Recall	F1	AUPRC
LSTM [61]	0,763	0,812	0,786	0,853
Transformer[62]	0,802	0,759	0,780	0,856
GTN [63]	0,781	0,790	0,786	0,858
Figure 2: A wildfire danger map of the Mediterranean, produced by the Transformer model. The fire danger is indicated by the softmax probabilities of the trained model.
Results. The results of the models are shown in Table 1. The promising results from all the models show that they can distinguish between high and low fire danger instances. It should be noted that the optimal performance model varies for each metric, thus making it difficult to define the overall best-performing model. In addition to the quantitative metrics, we also provide a visualization map generated by the Transformer model, presented in Figure 2. The map displays the model's softmax probabilities, indicating fire danger levels. This provides an example of how a daily fire danger map can be generated for operational scenarios in the Mediterranean region. The high spatial resolution of the dataset employed in this study could enable improved fire management strategies and demonstrate potential operational benefits. In this example, concerning fire danger for a day in the summer of 2022 notable variations in fire danger are observed across different areas of the Mediterranean, as well as within individual regions of each country.
4.2 Track B: Final Burned Area Prediction
Task formulation. This track focuses on predicting the likely extent of a wildfire's final burned area, given the ignition point and a set of variables available at the time of ignition, representing the fire drivers inside a neighborhood around the ignition point. These fire drivers encompass factors that influence fire behavior and spread. Thus, the objective of the ML task is to estimate the likelihood of the neighboring pixels surrounding the ignition point, to be eventually contained within the final burned area of the wildfire. The final burned area prediction is treated as a segmentation task, with two classes, indicating whether a pixel will experience burning or remain unaffected by the ignited fire. The resulting softmax probabilities of the trained model serve as indicators of the likelihood of a pixel being burnt.
Dataset extraction. For every fire event, we extract  $64km \times 64km$  patches, that are centered around the fire's ignition point, usually containing the whole burned area of a given fire event. The extracted samples include all the values of the variables of the datacube for the date of the fire's occurrence.
Experimental Setup. The  $64 \times 64$  patches are randomly cropped to  $32 \times 32$  during the training process. This approach ensures that the ignition point remains within the patch while preventing the model from generating a bias towards fire expansion solely from the central cell. As this is a segmentation task, we use the U-Net architecture [64] with an EfficientNet-B1 [65] encoder. Different input variables are stacked as separate channels. The cross-entropy loss is used to train the models' parameters. To define a baseline for the task, we train an additional model that uses as input only the ignition points. We do a temporal split to avoid leaking data from fire events happening close in time, using 2006 - 2019 for training (12550 samples), 2020 for validation (1781 samples), and 2021 - 2022 for testing (3527 samples). The loss in the validation is used for early stopping. Input variables are scaled with the minimum, and maximum values in the range  $[0,1]$  before being served as inputs to the model. As evaluation metrics, we report the CE loss and the AUPRC. The exact architecture and the values of the hyperparameters are provided in the Supplementary Material.
Table 2: Results of the final burned area prediction track
Model	CE Loss	AUPRC
U-Net (only ignitions)	0.0177	0.394
U-Net (all variables)	0.0166	0.418
Figure 3: Two examples of predictions (softmax values for the positive class) from the U-Net using all variables, and the U-Net using only ignition points. The predictions are presented together with binary maps representing the initial ignition points and the ground truth burned areas.
Results. The experimental results are presented in Table 2, which shows that the model incorporating all variables slightly outperforms the baseline model relying only on the ignition as input. In Figure 3 we see two examples from the models' predictions in the test set. It is notable that models predict higher spread danger around the ignition point, with the model that uses input from all variables showing some enhanced skill compared to the no-skill baseline. To improve on the task, it might be important to include variables before and after the fire starts and not only from the date of ignition. Notably, Coffield et al. [66] find that the performance can be improved using weather data forecasts for  $1 - 5$  days after the start of a fire. The dataset extraction script that accompanies the track includes the capabilities to extract datasets that include time series of arbitrary length before and after the start of each fire.
5 Potential other tracks to explore
Mesogeos incorporates burned areas and their sizes, as well as ignition points as predictands, and thus can be used for various ML applications. Thus, users can address additional tracks beyond those presented in the current study. The following ideas serve as suggestions for potential paths of research.
Fire size prediction. This application involves predicting the final size of fires rather than the presence of burned areas, which was addressed in this work. This can be framed as either a regression or a multiclass classification task. As a regression task, the target variable would be the burned area in hectares and therefore predictive models could be developed to estimate the final size of fires. As a classification task, a model could be employed to categorize fires into small, medium, or large. Notably, the dataset initially created for the wildfire danger forecasting track can be used as is for these two tasks.
Extreme events forecasting. Forecasting extreme events holds significant importance for effective fire management strategies. Concerning the Mediterranean, most of the damage caused by wildfires is the result of only a few large fires [67]. Therefore, anticipating these wildfires is crucial for ecosystem preservation and optimization of fire management. Mesogeos includes extreme events, such as the 5 massive fires in Greece in the summer of 2021, that burned nearly 94,000 hectares [68], the extreme events in Portugal in October 2017, with 5 events burning more than 18,000 hectares each [69] and the fire in Valencia, Spain in 2012 that burned over 50,000 hectares [70]. Extracting a dedicated dataset for this track offers an opportunity to develop models explicitly predicting these extreme events. As these extreme events are rare, one potential approach to tackle this task would be treating it as an outlier detection problem.
Wildfire susceptibility mapping. Wildfire susceptibility is defined as the static probability of wildfires in a certain area, depending on the characteristics of the terrain and prevailing meteorological conditions [71]. This task can be framed as a binary classification task. In this context, the dataset extraction process would involve identifying positive samples as the number of all pixels affected by fire over multiple years. Negative samples would be obtained from pixels that have never experienced burning, i.e. not belonging to any burned area of the dataset. An ML model could then be employed to distinguish between the fire-susceptible and non-susceptible samples.
Self-supervised learning. The vast amount of data in the datacube remain untapped when extracting task-specific datasets. Self-supervised learning (SSL) [72] offers a promising approach to take advantage of the full capabilities of these data. SSL allows for acquiring a representation that can be utilized across various downstream tasks, including those mentioned earlier. Concerning the SSL track, extracting specific datasets is unnecessary, as the training samples can be directly extracted from the datacube during the data loading process. Careful engineering is essential when selecting and extracting samples to minimize the time required for the model to be trained.
Modeling at different spatio-temporal scales. Mesogeos has a resolution of  $1km \times 1km \times$  daily, enabling the examination of problems at that specific temporal and spatial scale. However, the flexibility of the datacube format allows for resampling in various temporal and spatial dimensions through appropriate aggregations. This would enable the treatment of other tasks in coarser temporal or spatial scales, like seasonal or sub-seasonal fire modeling.
Beyond traditional ML. When doing ML-based wildfire modeling for decision support it is many times important to dive deeper into understanding the underlying processes that drive the models' predictions. In that respect, Mesogeos can foster the development of explainable AI techniques [73] toward a better understanding of models and subsequently the interactions of the fire drivers that result in wildfires. Additionally, causal inference methods [74] could be used to assess the effects of human controls, such as agricultural practices or land use on wildfire regimes. Moreover, considering the stochastic nature of fire processes, noise commonly appears on the labels. Methods that take into account the noisy labels [75] and especially those estimating the inherent aleatoric uncertainty [76], could enhance the reliability of the models and support the decision-making. When existing layers in the datacube are not enough, the datacube can be easily enhanced with extra information such as socio-economic factors, settlement, and infrastructure.
Despite the advantages of Mesogeos, we would like to acknowledge certain limitations. Firstly, the datasets inherit inaccuracies of the original data sources. Factors such as the satellites' spatial resolution and missing data resulting from cloud cover can influence the precise determination of the fires' location and size. Additionally, the acquisition of the fire ignition date is challenging and prone to deviations, as discussed in Section 3. Another limitation arises from the types of fires included in the burned areas' products. It is possible that there are fires resulting from prescribed burning or agricultural burning [41], which cannot be modeled using just the variables within Mesogeos. Moreover, while the daily temporal and  $1km \times 1km$  spatial resolution of the datacube is appropriate for the applications suggested in this research, it cannot be used to address other, nowcasting-type problems related to other cycles of fire management such as fire spread, fire detection, or fire recovery related applications. For the target variable, Mesogeos considers the highly reliable final burned areas from EFFIS, but ignores intermediate temporal information of the wildfire evolution, compared to work explicitly targeting wildfire spread [20, 45, 44]. Furthermore, the dataset lacks information regarding fire suppression efforts. The interventions of firefighters and responders have the potential to influence fire dynamics both at the time of ignition, achieved through water application to weaken fire spread, as well as during winter months by means of fuel cleaning or controlled burning. It should be acknowledged that the absence of such data could influence the modeling of some of the tracks. For example, when training the model to predict fire danger, it remains unknown whether the fire would have grown larger (indicating higher danger) in the absence of any wildfire suppression measures, or the opposite. Finally, it is important to highlight that while ML can assist in wildfire modeling, an operational application in wildfire management necessitates a thorough evaluation across fire seasons and against operational baselines, including domain experts and wildfire responders in the process.
7 Availability and Maintenance
The Mesogeos datacube and the datasets utilized in this study are made publicly available. The project's website https://orion-ai-lab.github.io/mesogeos/ will hold updated links to the data and code repository, as well as a leaderboard for the ML tracks. The repository contains code for generating Mesogeos, extracting datasets for the tracks, and running the models, enabling the reproduction of the results presented in this work. We encourage the community to further contribute with more ML tracks and models and advance data-driven wildfire modeling using the Mesogeos datacube.
In conclusion, this work introduces Mesogeos, a valuable resource for data-driven wildfire modeling. By leveraging the structure of a datacube and incorporating variables that represent various fire drivers and historical wildfires, Mesogeos facilitates the extraction of diverse datasets, empowering researchers to model various fire-related tasks. In this work, we demonstrate two tracks focusing on fire danger forecasting and burned area prediction to showcase the effectiveness and potential of the dataset. Lastly, we present several alternative tracks that address a wide range of challenges and tasks associated with anticipating and understanding wildfires, thereby paving the way for new avenues of research and advancement in wildfire modeling.
This work has received funding from the European Union's Horizon 2020 Research and Innovation Projects DeepCube and TREEADS, under Grant Agreement Numbers 101004188 and 101036926 respectively.
[75] Filipe R. Cordeiro and Gustavo Carneiro. A Survey on Deep Learning with Noisy Labels: How to train your model when you cannot trust on the annotations? In 2020 33rd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), pages 9–16, Recife/Porto de Galinhas, Brazil, November 2020. IEEE. ISBN 978-1-72819-274-1. doi: 10.1109/SIBGRAPI51738.2020.00010. URL https://ieeexplore.ieee.org/document/9266015/.
[76] Mark Collier, Basil Mustafa, Efi Kokioulou, Rodolphe Jenatton, and Jesse Berent. A Simple Probabilistic Method for Deep Classification under Input-Dependent Label Noise, November 2020. URL http://arxiv.org/abs/2003.06778. arXiv:2003.06778 [cs, stat].