Identifying Spatio-Temporal Drivers of Extreme Events
Authors: Mohamad Hakam Shams Eddin
Juergen Gall
Venue: NeurIPS 2024 poster (2024)
Keywords: anomaly detection
weakly supervised learning
Earth science
climate science
remote sensing
deep learning
Primary area: machine_learning_for_physical_sciences
TL;DR: We present a deep learning model, designed to leverage climate data to identify the drivers of extreme events impacts.

Abstract: The spatio-temporal relations of impacts of extreme events and their drivers in climate data are not fully understood and there is a need of machine learning approaches to identify such spatio-temporal relations from data. The task, however, is very challenging since there are time delays between extremes and their drivers, and the spatial response of such drivers is inhomogeneous. In this work, we propose a first approach and benchmarks to tackle this challenge. Our approach is trained end-to-end to predict spatio-temporally extremes and spatio-temporally drivers in the physical input variables jointly. By enforcing the network to predict extremes from spatio-temporal binary masks of identified drivers, the network successfully identifies drivers that are correlated with extremes. We evaluate our approach on three newly created synthetic benchmarks, where two of them are based on remote sensing or reanalysis climate data, and on two real-world reanalysis datasets. The source code and datasets are publicly available at the project page https://hakamshams.github.io/IDE.

Identifying
Drivers of Extreme Events
Mohamad Hakam Shams Eddin
Juergen Gall
Institute of Computer Science, University of Bonn Lamarr Institute for Machine Learning and Artificial Intelligence {shams, gall)@iai. uni.uonn.de
Abstract
The spatio-temporal relations of impacts of extreme events and their drivers in climate data not fully understood and there is need of machine learning approaches to identify such spatio-temporal relations from data. The task, however, is very challenging since there are time delays between extremes and their drivers, and the spatial response of such drivers is inhomogeneous. In this work, we propose a first approach and benchmarks to tackle this challenge. Our approach is trained end-to-end to predict spatio-temporaa temporally extremes and spatio-temporalt drivers in the physical input variables jointly. By enforcing the network to predict extremes from spatio-temporal binary masks of identified drivers, the network successfully identifies drivers that are correlated with extremes. We evaluate our approach on three newly created synthetic benchmarks, where two of them are based on remote sensing or reanalysis climate data, and on two real-world reanalysis datasets. The source code and datasets are publicly available the project page htti::://haaammhamss github.io/IDE
1 Introduction
A frontier research challenge is to understand the affectt of global change the magnitude and probability of extreme weather events [ Overall, the evolution of extreme events such as agri- cultural droughts results from stochastic processes (2] conditions ecosystem scales B 4. and the interaction between the Earth land and atmospheric variables as a part of a complex system of the event [2]. The time delays between extremes and their drivers vary seasonally [6H2], and the spatial response of these drivers is inhomogeneous 4 A major challenge is therefore to model the spatio-temporal relations between extremes and their drivers during the development of these events 4 The overarching goal of this modelling to improve our understanding of the patterns and impacts of such events. This would improve our ability to project duration and intensity of extreme events and hence assisting in adaptation planning [10
feedbacks [3]. However, the relative impacts of these factors differ depending
In this work, we propose an approach that identifies spatio-temporal drivers multivariate climate data that correlated with the impact of extreme events. For the extreme events, focus on agricultural droughts as an example, which can be measured by extremely low values of the vegetation health index (VHI). As drivers for such measurable extreme events, consider anomalies in atmospheric and hydrological state variables like temperature or soil moisture, as well as land- atmosphere fluxes like evaporation. The task of identifying end-to-end spatio-temporal drivers for measurable impacts of extremes has not been addressed before, and it is very challenging since the drivers can occur earlier in time and a different location than the measured extreme event as illustrated in Fig.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

To address this challenging task, we propose network that is trained to predict opatiottmpporaly extremes. Instead of simply predicting the extremes, the network quantizes the apatio-tempore imput variables into binary states and predicts the extremes only from the spatio-temporal binary maps for each time series of input variables. this way, the network is enforced to identify only drivers in the input variables that are spatio-temporally correlated with extreme events. While the network is trained using annotations of impacts of extreme events, which can be derived from remote sensing or reported data, we do not have any annotations of drivers or anomalies in the input variables.
Since drivers of extreme events are not fully understood, we cannot quantitatively measure the accuracy of the identified drivers on real-world data. We therefore propose a framework for generating synthetic data that can be used to assess the performance of our model as well as other baselines quantitatively. We evaluate our approach on three synthetic datasets where two of them are based on remote sensing or reanalysis climate data. Our evaluation shows that our approach outperforms approaches for interpretable forecasting, spatio-temporal anomaly detection, out-of-distributio detection, and multiple instance learning. Furthermore, we conduct empirical studies on two real- world reanalysis climate data. Our contributions can be summarized as follow:
We introduce the new task of identifying spatio-temporal drivers of extreme events and three benchmarks for evaluating this highly important task.
We propose novel approach for identifying spatio-temporal drivers in climate data that spatio-temporally correlated with the impacts of extreme events.
We further verify our approach on two long-term real-world reanalysis datasets including various physical variables from five biogeographical diverse regions.
2
Related works
Anomalies and extremes detection in climate data. The identification of climatic changes and extreme weather has been a subject of many studies [12 [3] Typical algorithms for extreme events detection are built upon domain knowledge in setting usually empirical thresholds for the physical variables through sensitivity experiments 66 [4]. Many works applied multivariate and statistical methods to detect extreme events such droughts (6] 9 [5/77]. However, individual events difficult to generalized across multiple events [2] and predefined indicators become less effective with changing climate [18]. Thus, machine and deep learning methods have been proposed as
alternative to classical methods, i.e., for supervised anomaly detection 19 20
and for the detection
of extremes in climate data [14]]
While methods for forecasting vegetation indices 25 28 do not focus on extremes, future impacts of extremes like agricultural droughts can be derived from forecast vegetation indices like the vegetation health index (VHI). For instance, the work (8 uses a climate simulation as input and forecasts the vegetation health index. Since predicting VHI directly is difficult, the approach predicts the normalized difference vegetation index and the brightness temperature instead. Both indices are then normalized and used to estimate VHI. Although we obtain the impacts of extreme events in study from vegetation indices, our approach is not limited to such extremes. Since we use a binary representation of extremes, our approach can also be applied to other extremes that cannot be derived from satellite products, but that are stored in a binary format in databases.
While we aim to learn the relations between the impacts of extreme events and their relevant drivers from a data-driven perspective, spatio-temporal relations within the Earth system can also be inferred by causal inference and causal representation learning [69] In contrast to statistical methods [37], data-driven methods do not require a prior hypothesis about drivers for extremes. Instead, they generate hypotheses that can be verified by statistical methods in a second step. We believe that this is an important direction since climate reanalysis provides huge amount of data and it is infeasible to test all combinations. This is also known curse of dimensionality in causal discovery problems (88 and data-driven approaches are therefore needed to generate potential candidates.
Anomaly detection algorithms. Since we focus on anomalies in land and meteorological data as drivers, we give an overview of approaches for anomaly detection and discuss their applicability to our task, which has not been previously addressed. One-class: The main stream in one-class anomaly detection to model the distribution of the normal data during training and consider the deviation from the learned features as anomalies. This includes distance-based [1) patch-based

---

State variable
State variable IV
Atmospheric and land state variables
Time
Time
Impact of extremes
State variable
State variable V
Time
Time
State variable III
State variable VI
Figure 1: Overview of the objective of this work. We are interested identifying spatio-tempone relations between the measurable impacts of extremes like the vegetation health index M and their drivers As drivers, we focus on anomalies in state variables of the land- atmosphee and hydro- logical cycle. The task is very challenging since the drivers can occur at different region than the extreme event and earlier in time.
general, the alignment between the anomaly type and the assumptions of the methods is the critical factor for their performance [52] One the limiting factors to apply these methods to climate data is that they assume priori knowledge about what is considered normal. Furthermore, not all detected anomalies are drivers of an extreme event. Reconstruction-baat
41,43,, student-teacher 44),47), and embedding- baaed approaches [43 48H51).
These methods assume that a trained model to reconstruct normal data will be unsuccessful in reconstructing anomalies, while it will reconstruct normal data well. Despite being widely applied for anomaly detection problems (53 (4), these methods face the same problem the one-class methods. In addition, many studies showed that anomalies can still be reconstructed by the trained model [65]. Self-supervised learning: These methods rely on the hypothesis that a model trained for pretext task normal data will be successful only on similar normal data during inference [66|69]. In addition to the above discussed limitations, finding suitable pretext task for anomaly detection is challenging. For instance, common tasks such as solving a jigsaw puzzle [69] will fail in homogeneous regions. Pseudo- intuition in pseudo-based anomaly detection is convert the problem of unsupervised learning into a supervised one by synthesizing abnormal data during training (70) Since these methods depend partially on the degree to which the proxy anomalies correspond to the unknown true anomalies [ applying these approaches to our task would require some knowledge about the coupling between the variables and extremes. Multivariate anomaly detection: Multivariate approaches detect anomalies simultaneously in multiple data streams (78-82) The main difference to our task is that we aim to detect drivers across multiple data streams that do not necessary occur simultaneously. Multiple instance learning: Multiple instance learning (MIL) has been proposed for weakly supervised anomaly detection [3391). In MIL-based algorithms, the model is provided with labeled positive and negative bags where each bag includes set of instances. The model is then trained to classify the instances inside the bags giving only the high level supervision, i.c., label of the bag [92]. A weakly supervised approach has been also applied for hyperspectral anomaly detection [93]. Most algorithms such as 97 choose the top-k potentially anomalous snippets within each video. This makes it challenging to apply them since the abnormality ratio varies in real-world applications 98
The
Furthermore, the MIL. detector can be biased toward a specific class depending on the context 99
3 Method
Our aim is to design a model that is capable of identifying spatio- temporal drivers of extremes in multivariate climate data, i.e., Earth observations or climate reanalysis. In particular, we want to identify anomalies that are spatio-temporally related to extreme events like agricultural droughts. This is different to standard anomaly detection since we are not interested all anomalies, but in spatio temporal configurations of variables that potentially cause an extreme event with some time
3

---

Feature embeddings
Quantization
moistue
1x Trensformer Tempora Attention
-6%
Extreme Agricultural Droughts
2% Video Swetn-
Anomalies Drivers of extremes
dts
Transformer Temporal wwrra---teeeiie Attention
LFQ.
-6.
30 CNN
Tempestur
2x: Video Swiin- Transformer
-34
tention
winn
mmeer
-06
Figure 2: An overview of the proposed model to identify the neatio-eempoal relations between extreme agricultural droughts and their drivers. The input variables are first encoded into features. In subsequent step, lockup free quantization layer (LFQ) takes the extracted features and classifies the variables into a binary representation of drivers, where we consider the drivers as anomalous events in the input variables. Finally, classifier is used predict impacts of extreme events from the identified drivers.
delay and potentially different location illustrated in Fig. To achieve this, we propose network that is trained end-to-end on observed extremes where we focus on agricultural droughts. The network classifies the input variables before extreme event occurs into spatio-tempont drivers without additional annotations besides the annotated extremes. The network then aims to predict future extremes based the identified drivers. It needs be noted that we are interested in input variables that do not define an extreme event, but we aim to find anomalies in input variables that correlated with the occurrence of an extreme event. We will thus denote potential drivers as anomalies.
An overview of our approach is shown in Fig. 2 The input weekly climate variables sequential time steps (At-7,-... Atg) and the model is composed three main parts. First, a feature extractor extracts relevant features from each input variable independently. The second component is quantization layer that takes the extracted features as input and classifies the input variables into drivers. The role of the quantization layer is to transform the input variable into binary representation (1 = drivers and otherwise). This ensures that no additional information is encoded as an input to the subsequent classifier except that if the input variable a specific location and time is driver not. The third component is a classifier that takes as input the variables, location, and time where drivers have been identified and predicts where extreme droughts occur at the time step Åtg. All model components are trained jointly.
We denote the input data X IR TRVXXTXXatxLxn where V is the input variables, C is the channel dimension for each variable (i.e., mean and standard deviation of the week), T is the temporal resolution (At-Tt1..... Ato), and Lat and Lon the spatial extensions. The model has two outputs, z ZXXTxx Lon representing the binary classification of the input variables into potential drivers of the extreme events, and probabilities E e L Lon to predict extreme events at the time step Atg. In the following, we describe the model components:
are extracted independently for each input variable E V with K embedding dimensions and parameters The rationale behind this independence is to prevent that drivers leak into other variables. We use the Video Swin Transformer model [100] as backbone to capture long-range interrelations across time and space. The input X is projected into a higher feature dimension K and followed by two Video Swin Transformer layers. The first layer has two consecutive shifted window blocks for spatio-temporal feature extraction. The second layer consists of one block for temporal feature
Feature extraction. First, embedded features fo X - Z Xx

---

extraction. The later is useful to focus only on the temporal evolution of the variables. An ablation study regarding the backbone is provided Sec. C.4
Quantization layer. The role of this layer is to map Z. from an embedded space into a compact
binary representation Q suitable for detecting drivers. Using vector quantization (VQ) [011 , each
embedded feature vector = E Z is assigned into learnable codebook feature vector 2g E Zy based on the Euclidean distance:
VQ: :2:2 - Zg. where g = arg min ||2 2q 12
(1)
o ()....
where Q is the size of the codebook. Recently, lookup-free quantization (LFQ) 102 substitutes the learnable codebook with set of integers Q with [Q| = Q and represents the embedding space as a Cartesian product of binary numbers. This omits the need for distance metric to do the nearest vector assignment and simplifies the quantization. Based on experimental results, we built the vector quantizer on LFQ with two integers Q - 2 (q for drivers and 0 otherwise). Given a feature vector 2, LFQ first maps into scalar value z Z R XxxxxxTxx X For multi-modality, we use two sequential 3D CNNs on each input variable followed by shared linear layer that maps to 2 and reduces the dimensions from K to The quantization is then given by the sign of 21:
2 Linear Linear(sign( 2t) = Linear( - 1 (0) + 1l (x(20)), and q = (=,20)
(2)
where q E Q represents the class (g- = 1 or q = 0), and Linear is linear layer that converts sign(2)) back to the dimension K' of the input after the quantization. Note that Zy has only two unique vectors 2x=1 for a driver and 2==0 otherwise.
Prediction of extreme events. We use a classifier that predicts the probably of extreme events E at the time step Ale from the identified drivers Zu. We use 3D CNN classifier instead of transformer to reduce the computations. For training, we only know the ground truth of extremes at time step Ao denoted by È E _LattLLn, While we could compute the cross-entropy between E and E, we found that single 3D CNN is insufficient to detect all drivers that are correlated with an extreme event. Instead, we use V+1 3D CNNs where each predicts E.. While the first V 3D CNNs take the identified drivers for single variable all variables as input. The multiple CNNs are only used for training. During inference, E is only predicted by the multivariate CNN where all variables are jointly used. The loss is thus given by
input, the last one takes the identified drivers of
V+1
(eatreme) = - Σ (Elog(E.) + (1 E) log(1 -E,)SS
where S E 277 Latx
TTaxxaxxx Lon is mask for the valid regions. We actually utilize a weighted version of ((ettreme) to mitigate the class imbalance issue (Sec. C.3 While the loss Lieztreme) ensures that extreme events can be predicted from the identified drivers, need to add standard loss terms for the quantization to ensure that the learned codes and thus drivers are compact:
(guantise) (ccommitt Iz sg ssignn (Z) + (F(inn))(
A(div) H (sign (Zi)
The commitment loss 1z 1 sg(iig (Z)) prevents the outputs of the encoder from growing and
encourages Z4 to commit to the codes [101] where sg stands for the stopgradient operator with zero
partial derivative. The term FT sigg(Zz)) encourages that the entropy per quantized code is low
102 [03] meaning that it provides more confident assignments. Whereas the term H E (iig (Z))] increases the entropy inside the batch to encourage the utilization of all codes 102 103 The last important ingredient is loss that ensures that only spatio-temporal regions are identified that correlate with an extreme event. To this end, we look at regions and intervals where no extreme event
occurred and use these examples without drivers. Formally, we use É, E Z z as the union of extreme ground truth at all time steps (At (At-T+1,.... Ata) and compute the loss by
Cidriver) (driver) Iz, sg( o)) E,)S,
(5)
where =0=0 is the quantization code for normal data without drivers. The model is trained end-to-end with the joint optimization of the loss function:
min ((entreme) 0.0.V
É) +
L(quentize)
(Zi
((driver) (Zy, E,, S, Zy=
(6)
prodicts extremes from drivers encourages confident quantization
assigns drivers to the same code the codebook
where 0, de 1 are the learnable parameters, and A(committ) ((ent) A(dse), and A(driver) are weighting
parameters. Ablation studies are provided in Sec. 5.1 and in Appendix Sec. C
5

---

4 Dataset
4.1 Defining extreme agricultural droughts from remote sensing
We are interested in a specific impact of extreme events namely extreme agricultural drought. To define such extreme event, we rely on the observational satellite-based vegetation health index (VHI) obtained from NOAA [04]. This remote sensing product cannot be directly derived from the input reanalysis, which makes the task very challenging. VHI approximates the vegetation state based on a combination of the brightness temperature and normalized difference vegetation index (VHI = 0 for unfavorable condition and VHI 100 for favorable condition). Extreme agricultural droughts are usually defined as VHI < 26 [104]. The dataset has a temporal coverage of -onward and is provided globally on a weekly basis. We mapped this dataset into the same domains of the reanalysis data as described in Sec. 4.2 and used this dataset as ground truth for extreme events. Note that VHI a general vegetation index and should be interpreted carefully. Details about this index, the dataset and pre- proccssccssng are provided in the Appendix Sec. [
4.2 Climate reanalysis
Reanalysis data aim to provide a coherent and complete reconstruction of the historical Earth system
state as close to reality as possible. During reanalysis, short-term forecasts from numerical climate
models are refined with observations within the so called data assimilation framework [105]. We
conducted the experiments on two real-world reanalysis datasets; CERRA reanalysis [106] and
ERAS-Land [07]. ERA5-Land is widely used for global climate research and it is provided hourly
at 0.1% 0.1% on the regular latitude longitude grid. CERRA is regional reanalysis for Europe and is provided originally at 5.5km x 5.5km on its Lambert conformal conical grid with a 3-hourly temporal resolution. We aggregated these
datasets on weekly basis and selected the years within the period overlapping with the remote sensing data. In addition, we mapped ERA5-Land into 6 CORDEX domains [108] over the globe and conduct experiments each region separately. We do the experiments with 6 common variables from FRAS-Land and CERRA based on their connections to agricultural droughts. For each variable and week, we computed the mean and standard deviation separately. More details regarding the variables and the domains along with the trainingvvalidationneest splits are provided in the Appendix Sec. and Tables and 21
4.3 Synthetic dataset
Although ground truth for extreme droughts can be obtained from remote sensing, an important methodological question remains as how to reliably have meaningful quantitative evaluation of the identified drivers and their relations to the extreme events. To solve this critical issue, we introduce a new synthetic dataset that mimics the properties of Earth observations including drivers and anomalies. We are aware that the dynamic of the synthetic data are simplified compared to real Earth observations. However, we rely on this generated dataset to perform the quantitative evaluation of the proposed approach. In a first step, we generate the normal data. For instance to generate synthetic data of 2m temperature from CERRA reanalysis, the normal signal based on the typical value of 2m temperature at that time and location (i.e., the mean or median value from a long-term climatology). The second step is to generate anomalies conditioned on the occurrence of extremes. To achieve this, we assign binary spatio-temporally connected flags as extreme events randomly within the datacube and track their precise spatio-temporal locations. Then based on a predefined coupling matrix between the variables and the extreme event, we generate anomalous events only for the variables that are defined to be correlated with the extremes. We consider these anomalies as the drivers for the extreme events. Finally, we add additional random anomalous events for all variables. We synthesize overall 46 years of data; 34 years for training, subsequent years for validation and the last 6 years for testing. The challenge is to identify the drivers, i.e., the anomalous events that are correlated with extreme events. Examples of the synthetic data are shown in Fig. 3 and in Appendix in Figs. I2 Technical details are explained in Appendix Sec. A
specific time and location is generated

---

Experimental results
First, we conducted experiments and ablation studies on the synthetic datasets (Sec. 4.3) We also empirically verified the effectiivnness of the proposed design compared to baselines on this synthetic dataset. Then, we validate the model on two real-world datasets over five continents See. 5.2
Setup and implementation details. We set the hidden dimension K to 16 by default. The temporal resolution T for the synthetic data and T 8 for real-world data. Since, seasonal cycles are typical in climate data, we deseasonalize locally by subtracting the median seasonal cycle and normalizing by the seasonal variance for each pixel. Details regarding the model and implementation setup are given in Appendix Sec. G For evaluation, we use the F1 Fl-score, intersection over union (IoU), and overall accuracy on both classes (OA).
5.1 Experiments on the synthetic datasets
We show the results on the Synthetic CERRA described in Sec. 4.3 and in Appendix Table The generated dataset mimics a set of variables (V = 6) using statistics from the real-world CERRA reanalysis [066.. We artificially correlated four variables with extremes (2m temperature, total cloud cover, total precipitation, and volumetric soil moisture) and kept two variables uncorrelated (albedo and relative humidity).
Comparison to the baselines. We compare the new approach to interpretable forecasting approaches using integrated gradients [109] and to 8 baselines from 3 different categories of anomaly detection approaches; one-class unsupervised 33 110 reconstruction-De
(59 655, and multiple instance learning 96). We also compare naive baseline which labels all variables drivers for pixels where extreme events occur. The implementation details of these baselines are given in Appendix Sec.
The quantitative results are shown in Table The naive baseline is impacted by two main issues; first by the time delay between drivers and extreme events, and second not all variables are correlated with the extremes. The second issue affects the one-class and reconstruction- based baselines where
Table 1:
Driver detection results on the synthetic CERRA reanalysis. The best performance on each metric highlighted in a bold text. (#) denotes the standard deviation for 3 runs.
Validation
Testing
Algorithm Fl-score (1) IoU (f)
OA (f)
Fl-score (1) IoU (f)
OA (f)
Naive 47.93
31.52
98.61
51.24
34.45
98.37
Integrated Gradients
109
24.15
14.12:6..1
92.18+:2.94
23.14=7.005
13.271:.65
91.58
Integrated Gradients II 1099
18.58
3.05
95.26
1.03
1.03
30.34
17.95
94.19 1.22
One- Class
OCSVM
39
28.21 16.44=1.67
95.64.1..16
2..9
1..6 94.91
IF
110
34.99-0.556 21.28:t0.42
97.16++0.02 37.16=:0667
22.84+0.51
96.61
SimpleNet
S1
7..01 60.39 10 99.20-10.01 73.50.L0.24
1..30
98.91-10.02
Rec.
STEALNet 59
55.98 38.87--0.86
98.47-40.03
57.74
40.60
98.22 -8.02
UniAD
65
47.53+H0.17
31.18--0.14 14 97.44 49.23-H0.41
32.65-H0336 97.18-H0.00
DeepMILL 94
70.68=11.61 54.68
99.22-t0.03
71.54+ 1.60
55.72 .99 99.09-+0.04
MIL
ARNet
95
72.92.10.85
57.39
99.26.10.01 73.68.10.86
58.34 1.08 99.13.10.02
RTFM
96
60.09=0.31 42.95=t0.31
98.34-10.33
61.88=0.28
44.81=0.30 98.12-00.02
Ours"
8..78
70.63
99.51-10.02
00.74
67.28 1.997
L:.04
Ourst
83.45=0.377
71.60 99.52-10.01
80.65
1.33 99.29-10.00
VVidoo Swin Transformer backbone 1100
Mamba backbone
7

---

Synthelic
Ground
integaated
Integrated
emmppettture abedo
Figure 3: Qualitative results on the synthetic CERRA reanalysis from the test set at time step 2160. the prediction, is the ground truth, and the false positive. Albedo and relative humidity are not correlated with extremes, meaning that they do not contain drivers, but only random anomalies.
they suffer mostly from false positives. In fact, both integrated gradients models achieve high F1- scores for detecting extremes (93.32 for Integrated Gradients I and 93.80 for Integrated Gradients II), but they have worse performance on identifying the drivers. SimpleNet is trained with our model as a feature extractor which explains its good performance. However, SimpleNet showed drop of performance when tested on other datasets (see Appendix Sec. [ for results on two more synthetic datasets). Among the reconstruction-bass approach, STEALNet outperforms UniAD, This is probability because STEALNet exploits more weakly supervision information during training by maximizing the reconstruction loss for locations with extreme flags. MIL-based baselines more suitable for the task. Finally, our model consistently outperforms the baselines on all metrics. Qualitative samples in comparison with base- lines are presented in Fig. 3] The qualitative examples indicate that model and the MIL- based baselines except RTFM are capable of learning which variables correlated with the extremes. The main weakness of RTFM is the reliance on feature magnitudes and the cross attention module (see Appendix Sec. E and Ta- ble D). which make more prone produce false positives. Other baselines predict incorrect relations between the variables and extremes. Regarding the explainable AI methods, when we add more interactions between the variables (Integrated Gradients II), the gradients tend to omit some variables (soil moisture). Both in- tegrated gradients models have also difficulties with the synthetic t2m, which includes red noise by design. These results demonstrate that networks that predict the extremes directly from the input variables utilize much more information even when it is not correlated with an extreme. It is thus beneficial to introduce a bottleneck into the network that enforces the network to explicitly identify drivers extremes.
100
80
Flccora
30
STEAUhi
AnMs RIPS
10
OCSWM
UsAD
Dansiil..
0um
Number veriab cornelated
Figure 4: Fl-score with different correlation set- tings between the input variables and extremes.
Performance easy-to-hard correlation settings. We conduct an additional experiment to assess the model performance relation to the correlation setup between the variables and extremes. We generate synthetic CERRA dataset starting with only one correlated variable with the target extreme. We then generate different versions of the dataset by increasing the number of correlated variables with the extremes up 100%. This analysis allows us to point out the strengths and weaknesses of

---

Table 2: Ablation studies from the validation set. The metric is Fl on the driver/extreme detection.
(a) Loss function
(b) Key model architecture (c) T dimension
Liquantize) L(driver) ((ettreme) Fl Fc-re (1)
Temporal- Cross- Shared- Fl-score (1) T Fi-score (1)
multi-head
attention attention (fa)
x
X
29.99 / 48.94 94 /
X
V
69.59 99..42 1 83,38 / 69.25
XX
/
02.51 / 92.88 X
/
X
67.18 / 99.778 4 81.36 90.68
31.23 / 66.40
X
82.39 /91.97 6 82.78 / 92.45
7
82.78 / 92.45 /
x
82.78 92.45 77.33 / 90.30
the comparative models for different scenarios and where our model becomes more effective, as well as where it could struggle most. The results are shown in Fig- 4 One-class, reconstruction-baeel and RTFM baselines benefit with increasing the number of correlated variables. case 6/6, the task reduces to an anomaly detection task. The performance of our model and MIIL-based baselines generally decreases when the number of correlated variables increases as the task of finding all correlated anomalies becomes harder. Nevertheless, our approach performs best in all settings.
Ablation study. We conducted a set of ablation studies. This includes three main experiments:
Loss functions. As shown in Table 2 (a). Liovantize; and Lidriver) are essential for training. As other quantization models, ours not be trained without L(quanitae) which ensures that the outputs of fo do not grow and commit to the binary embedding. L(driver) unifies the representation of drivers for all variable q 1, which boosts the extreme detection. Moreover, the results demonstrate the impact of using V+1 3D CNNs (multi-head) instead of one for L(eztreme)- If a single 3D CNN is used, drivers are only identified in small subset of variables. We discuss this more in detail in Sec. CC7
Feature extractor. In Table (b), show the benefit of having independent feature extractors for driver detection. In a first experiment, we share the feature extractor fo among the variables. The performance is worst. Second, we replaced the temporal attention by cross attention between the variables similar to [12 and [TS]] where each variable performs a cross attention with the other variables. We can
drop of performance for the second experiment. We noticed that anomalies propagate between variables when adding connections in the feature extraction stage. This also explains the poor performance of RTFM compared to other MIL. baselines. The best performance is shown for the proposed setup (last row), which also shows the benefit for the temporal attention.
Temporal resolution T. Table (c) evaluates the impact of the temporal resolution on driver and extreme detection. T'=6 provides a good balance between driver and extreme detection. More ablation studies on other aspects of the model design can be found in Appendix Sec.
5.2 Experiments on real-world datasets
We evaluate our model on two reanalysis data with diverse geographical and climate regions (Sec. 4.2) The input for the experiment is the normalized mean and standard deviation of each week. We exclude pixels over water surfaces, desert, and snow. In addition to the quantitative evaluation on the synthetic data, we aim to verify our method considering the following aspects:
Quantitative results. We expect that the developed model can identify drivers in real-world scenarios. We demonstrate this by measuring how well the model can predict extreme agricultural droughts from the identified drivers. The results verify that the model can predict the droughts across different regions and datasets (see Appendix Sec. [aaad Table Note that compared to the synthetic dataset, the real-world drought prediction is much more difficult.
Extreme detection without anomaly detection. We trained the model without the quantization step, meaning without driver detection. This can be considered detection accuracy since there is no information reduction by the quantization. We found that when trained the synthetic and real-world EUR-11 data, the F1-score for detecting extreme events increased only by 0.96% and 1.93%, respectively, compared to the model with quantization (see Appendix Table 12). This verifies that the detected drivers are highly correlated with the extremes.
upper bound the extreme
Qualitative results and spatial distribution. In Fig. (a), we show the spatial distribution of the identified drivers at specific time over EUR-11. Shown are the identified drivers up to weeks

---

(a) 20a5- week-16
(b) EUR -Ummer, weeks (25-37), 2018-2023
Figure 5: (a) Qualitative results on ERA5-Lame over the EUR-11 domain. Shown are the identified drivers localized spatio-temporally 7 weeks before the extreme agricultural drought events. (b) Temporal evolution of drivers during the extremes.
(Al-7) before the extreme agricultural droughts at time Alg- We can see that the prediction of drivers and extremes are spatially correlated with the ground truth.
Physical consistency. In Fig. 5 (b), we show the relation between the input reanalysis data, extreme droughts, and identified drivers. For this experiment, we selected pixels with extreme events during summer (weeks 25-38) and visualize the average distribution of drivers with time. The red line at At, indicates the beginning of the extreme droughts. Zecore in the underneath curve represents the deviation from the mean computed from the climatology. It is expected that evaporation reduces soil moisture, which dries out the soil and vegetation [4. Our model indicates that over Europe, the evaporation and soil moisture are the most informative variables to detect drivers related to extreme droughts. All pixels experienced a pronounced decline in soil moisture and an increase in evaporation the events evolved. Please Sec. D33 for more discussion on the scientific validity.
6 Discussions and conclusions
We introduced a model that can identify the spatio-temporee relations between impacts of extreme events and their drivers. For this, we assumed that there exist precursor drivers, primarily as anomalies in assimilated land surface and atmospheric data, for every observable impact of extremes. We demonstrated the effectiveness of our approach by measuring to which degree the identified drivers can be used to predict extreme agricultural droughts. Apart from experiments world datasets, we also presented a new framework to generate synthetic datasets that can be used for spatio-temporal anomaly detection and climate research. The results on the synthetic datasets show that the approach is not limited to droughts and can be applied to other extremes. While have shown that our approach outperforms other approaches, the study has some limitations. First, evaluating ability to handle a very large number of climate variables in a unified model needs further examination. Similarly, performing accumulating precipitation over many weeks might also improve the results. Second, modelling the temporal relations is limited by the time window T. Moreover, teleconnections of climatic anomalies can occur in distant regions on Earth, e.g., affects of EI Niño and La Niña variability on drought and flood [12]. Modelling and disentangle such large spatio-temporal relations across the globe is an open research problem. Third, it would be appealing to provide scores for drivers instead of a binary classification. This could be achieved by measuring the distance to the nearest code in the LFQ. Forth, the prediction of the model depends on the capacity of reanalysis data to accurately represent the local environmental factors and land-atmin atmopphriii feedbacks. Most importantly. drawing conclusions on drivers from weak predictive models may lead to unreliable interpretations. Finally, our model does not identify causal relationships.
two real-
additional pre-pre pre-processing of specific variables like
Despite these limitations, our approach demonstrates a clear capability in identifying drivers and anomalies in climate data which would allow a more timely event attributions during and right after extreme events. The identified spatio-temporal relations between extreme events and their drivers could support the understanding and forecasting of extremes.
10

---

7 Acknowledgments and Disclosure of Funding
We thank Petra Friederichs, Till Fohrmann, Sebastian Buschow, and Svenja Szemkus for the insightful discussions on detection and attribution of weather and climate extremes. We would also like to thank Olga Zatsarynna and Emad Bahrami for the technical discussion related feature extraction. Finally, we thank the four anonymous reviewers for their comments and suggestions which improved the quality of this paper.
This work was supported by the Deutsche Forschungsgemeinschatt (DFG, German Research Foun- dation) SFB 1502/1-2022 project no. 450058266 within the Collaborative Research Center (CRC) for the project Regional Climate Change: Disentangling the Role of I Land Use and Water Management (DETECT) and by the Federal Ministry of Education and Research (BMBF) under grant no. 011S24075C RAINA.
We acknowledge EuroHPC Joint Undertaking for awarding access to Leonardo at CINECA, Italy, through EuroHPC Regular Access Call - proposal No. SHPPC-REG-2024R011 The authors also gratefully acknowledge the granted access to the Marvin cluster hosted by the University of Bonn.
References
[1] Stephanic C. Herring, Nikolaos Christidis, Andrew Hoell, and Peter A. Stott. Explaining extreme events of 2020 from a climate perspective. Bulletin of the American Meteorological Society, 103(3):S1 S129, 2022.
[2] Jana Sillmann, Thordis Thorarinsdotii., Noel Keenlyside, Nathalie Schaller, Lisa Alexander. Gabriele Hegerl, Sonia I. Seneviratne, Robert Vautard, Xuebin Zhang. and Francis W. Zwiers. Understanding. modeling and predicting weather and climate extremes: Challenges and opportunities. Weather and Climate Extremes, 18:65-74, 2017.
[3] M. B. Ek and A. A. M. Holtslag. Influence of soil moisture on boundary layer cloud development. Journal of Hydrometeorology, 5(1) - 99, 2004.
[4] Diego G. Miralles, Pierre Gentine, Sonia I. Seneviratne, and Adriaan J. Teuling. Land-atmospheric feedhacks during droughts and heatwaves: state of the science and current challenges. Annals of the New York Academy of Sciences, 1436(1): (436(1):19-35, 2019.
[5] JL Geirinhas, AC Russo, R Libonati, DG Miralles, AM Ramos, L Gimeno, and RM Trigo. Combined large-scale tropical and subtropical foreing on the severe 2019-200000000 -02022 drought in south america. npj Climate and Atmospheric Science, 6(1):185, 2023.
[6] Shengzhi Huang, Qiang Huang, Jianxia Chang, Guoyong Leng, and Li Xing. The response of agricultural drought to meteorological drought and the influencing factors: A case study the wei river basin, china. Agricultural Water Management, 159:45- 54, 2015.
[7] Fei Wang, Hexin Lai, Yanbin Li. Kai Feng, Zezhong Zhang, Qingqing Tian, Xiaomeng Zhu, and Haibo Yang. Dynamic variation of meteorological drought and its relationships with agricultural drought across china. Agricultural Water Management, 261:107301, 2022.
[8] Meng Dai, Shengzhi Huang, Qiang Huang, Xudong Zheng, Xiaoling Su, Guoyong Leng, Ziyan Li. Yi Guo, Wei Fang, and Yongjia Liu. Propagation characteristies and mechanism from meteorological agricultural drought in various seasons. Journal of Hydrology, 610:127897, 2022.
[9] Shengpeng Cao. Lifeng Zhang, Yi He. Yali Zhang. Yi Chen, Sheng Yao, Wang Yang, and Qiang Sun. Effects and contributions of meteorological drought on agricultural drought under different climatic zones and vegetation types in northwest china. Science of The Total Environment, 821:153270, 2022.
[10] Peter A. Stott, Nikolaos Christidis, Friederike F. L. Otto, Ying Sun, Jean-Paul Vanderlinden, Geert Jan van Oldenborgh, Robert Vautard, Hans von Storch, Peter Walton, Pascal Yiou, and Francis W. Zwiers. Attribution of extreme weather and climate-related events. WIREs Climate Change, 7(1):23-41, 2016.
[11] Zengchao Hao, Vijay P. Singh, and Youlong Xia. Seasonal drought prediction: Advances, challenges, and future prospects. Reviews of Geophysics, 108 141, 2018
[12] Mahashweta Das Srinivasan Parthasarathy, Anomaly detection and spatio-tempores analysis global climate system. In Proceedings of the Third International Workshop on Knowledge Discovery from Sensor Data SensorKDD '09, page 42-150, New York, NY, USA, 2009. Association for Computing Machinery.

---

[13] Sancho Salcedo-Sanz, Jorge Pérez-Aracil, Guido Ascenso, Javier Del Ser. David Casillas-Pérees Pérez, Christo- pher Kadow, Dušan Fister, David Barriopedro, Ricardo García-Hlerrera Matteo Giuliani, al. Analysis, characterization, prediction, and attribution extreme atmospheric events with machine learning and deep learning techniques: a review. Theoretical and Applied Climatology, 155(1):1 44, 2024.
[14) Yunjie Liu, Evan Racah, Joaquin Correa, Amir Khosrowshahi, David Lavers, Kenneth Kunkel, Michael Wehner, William Collins, et al. Application of deep convolutional neural networks for detecting extreme weather in climate datasets. arXiv preprint arXiv:/605. 100501120 105.011566 156, 2016.
[15] G. Sepulere- Caputo,e-Canto, S. Horion, A. Singlcton, H. Carrao, and J. Vogt. Development of a combined drought indicator to detect agricultural drought in europe. Natural Hazards and Earth System Sciences, 12(1 12(11)331,,3331, 2012.
[16] Yu Zhang, Zengchao Hao, Sifang Feng, Xuan Zhang, Yang Xu, and Fanghua Hao. Agricultural drought prediction in china based on drought propagation and large-scale drivers. Agricultural Water Management, 255:107028, 2021.
[17] Xiang Zhang, Nengcheng Chen, Jizhen Li, Zhihong Chen, and Dev Niyogi. Multi-sensor integrated framework and index for agricultural drought monitoring. Remote Sensing of Environment, 188:141-163; 2017.
[18] Yohannes Yihdego, Babak Vaheddoost, and Radwan Al- Wesshhhh Drought indices and indicators revisited. Arabian Journal of Geoxciences, 12:1-12, 2019.
[19] Mohamed Dahoui. Use of machine learning for the detection and classification of observation anomalies, 01/2023 2023.
[20] Peter Düben, Umberto Modigliani, Alan Geer, Stephan Siemen, Florian Pappenberger. Peter Bauer, Andy Brown, Martin Palkovic, Baudouin Raoult, Nils Wedi, and Vasileios Baousis. Machine learning ecmwf: roadmap for the next years, 01/2021 2021.
[21] M. Flach, F. Gans, A. Brenning, J. Denzler, M. Reichstein, E. Rodner, S. Bathiany, Bodesheim, Y. Guanche, S. Sippel. and M. D. Mahecha. Multivariate anomaly detection for earth observations: a comparison of algorithms and feature extraction techniques. Earth System Dynamics, 8(3):677 696.66676666 2017.
[22] Evan Racah, Christopher Beckham, Tegan Maharaj, Samira Ebrahimi Kahou, Mr Prabhat, and Chris Pal. Extremewcather: A large-scale climate dataset for semi-supervised detection, localization, and understanding of extreme weather events. Advances in neural information processing systems, 30, 2017.
[23] Prabhat, K. Kashinath, M. Mudigonda, Kim, L. Kapp-SShwoeeer A. Graubner, E. Karaismailoglu, I. von Kleist, T. Kurth, A. Greiner, A. Mahesh, K. Yang. C. Lewis, J. Chen, A. Lou, S. Chandran, B. Toms, W. Chapman. K. Dagon, C. A. Shields. T. O' Brieen, M. Wehner, and W. Collins. Climatenet: expert-labeled open dataset and deep learning architecture for enabling high-precision analyses of extreme weather. Geoscientific Model Development, 14(1):107-124, 2021.
[24] Elizabeth Weirich-Benet, Maria Pyrina, Bernat Jiménez-Esteen Emest Fraenkel, Judah Cohen, and Daniela I. V. Domeisen. Subscasonal prediction of central european summer heatwaves with lincar and random forest machine learning models. Artificial Intelligence for the Earth Systems, 2(2)x, 2023.
[25] Adam B. Barrett, Steven Duivenvoorden, Edward E. Salakpi, James M. Muthoka, John Mwangi. Seb Oliver, and Pedram Rowhani. Forecasting vegetation condition for drought early warning systems in pastoral communities in kenya. Remote Sensing of Environment, 248:1 1 1886, 2020.
[26] Christian Requena-Mesa Vitus Benson, Markus Reichstein, Jakob Runge, and Joachim Denzler. Earth- net2021: A large-scale dataset and challenge for earth surface forecasting as a guided video prediction task. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pages 1132-1 142, June 2021.
[27] Vitus Benson, Claire Robin, Christian Requena-Mesa. Lazaro Alonso, Nuno Carvalhais, José Cortés. Zhihan Gao, Nora Linscheid, Mélanie Weynants, and Markus Reichstein. Multi-modal learning for geospatial vegetation forecasting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages
June 2024.
[28] M. H. Shams Eddin and J. Gall. Focal-tsmp: deep learning for vegetation health prediction and agricultural drought assessment from a regional climate simulation. Geoscientific Model Development, 17(7):2987- 3023, 2024.
12

---

[29] Jakob Runge, Sebastian Bathiany, Erik Bollt, Gustau Camps-Vail Valls, Dim Coumou, Ethan Deyle, Clark Glymour, Marlene Kretschmer, Miguel D Mahecha. Jordi Muñoz-Marí,, et al. Inferring causation from time series earth system sciences. Nature communications, 10(1):2553, 2019.
[30] Bernhard Schölkopf. Francesco Locatello, Stefan Bauer, Nan Rosemary Ke, Nal Kalchbrenner, Anirudh Goyal, and Yoshua Bengio. Toward causal representation learning. Proceedings of the IEEE, 109(5)612- 634, 2021.
[31] Johann Brehmer. Pim De Haan, Phillip Lippe, and Taco S Cohen. Weakly supervised causal representation learning, Advances in Neural Information Processing Systems, 04383319-33333 2022.
[32] Xavier-Andoni Tibau, Christian Reimers, Andreas Gerhardus, Joachim Denzler, Veronika Eyring, and Jakob Runge. A spatiotemporal stochastic climate model for benchmarking causal discovery methods for teleconnections. Environmental Data Science, 1:e 12, 2022.
[33] Julien Boussard, Chandni Nagda, Julia Kaltenborn, Charlotte Emilie Elektra Lange, Philippe Brouillard, Yaniv Gurwiez, Peer Nowack, and David Rolnick. Towards causal representations of climate model data. arXiv preprint arXiv:2312.0288 2023.
[34] Jakob Runge, Andreas Gerhardus, Gherardo Varando, Veronika Eyring, and Gustau Camps-Vall Causal inference for time series. Nature Reviews Earth Environment, 4(7):487- 505, 2023.
[35] Raanan Yehezkel Rohekar, Shami Nisimov, Yaniv Gurwicz, and Gal Novik. From temporal to contem- porancous iterative causal discovery in the presence latent confounders. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, Proceedings of the 40th International Conference Machine Learning. volume 202 of Proceedings of Machine Learning Research, pages 39939-39950. PMLR, 23-29 Jul 2023.
[36] Jose María Tárraga, Eva Sevillano- Marco, Jordi Muñoz-Mar, María Piles, Vasileios Sitokonstantinou, Michele Ronco, María Teresa Miranda, Jordi Cerdà, and Gustau Camps- Valls. Causal discovery reveals complex patterns of drought-induced displacement. iScience, 27(9): 10628, 2024.
[37] J. Vogel, P. Rivoire, C. Deidda, L. Rahimi, C. A. Sauter, E. Tschumi, K. van der Wiel, T. Zhang, and J. Zscheischler. Identifying meteorological drivers of extreme impacts: application to simulated crop yields. Earth System Dynamics, 12(():151-172, 2021.
[38] Jakob Runge, Peer Nowack, Marlene Kretschmer, Seth Flaxman, and Dino Sejdinovic. Detecting and quantifying causal associations large nonlinear time series datasets. Science Advances, 5(1 xxaauu999999 2019.
[39] Bernhard Schölkopf, Robert C Williamson, Alex Smola, John Shawe-Taylor, and John Platt. Support vector method for novelty detection. Advances in neural information processing systems, 12, 1999.
[40] Philipp Liznerski, Lukas Ruff, Robert A. Vandermeulen, Billy Joe Franks, Marius Kloft, and Klaus- Robert Müller. Explainable deep one-class classification. In International Conference on Learning Representations, 2021.
[41] Jihun Yi and Sungroh Yoon. Patch svdd: Patch-level svdd for anomaly detection and segmentation. In Proceedings of the Asian Conference on Computer Vision (ACCV), November 2020.
[42] Shenzhi Wang, Liwei Wu, Lei Cui, and Yujun Shen. Glancing at the patch: Anomaly localization with global and local feature comparison. In 2021 IRFF/CVP Conference Computer Vision and Pattern Recognition (CVPR), pages 254-263, 2021.
[43] Thomas Defard, Aleksandr Setkov, Angelique Loesch, and Romaric Audigier. Padim: A patch distribution modeling framework for anomaly detection and localization. Alberto Del Bimbo, Rita Cucchiara, Stan Sclaroff, Giovanni Maria Farinella, Tao Mei, Marco Bertini, Hugo Jair Escalante, and Roberto Vezzani, editors. Pattern Recognition. ICPR International Workshops and Challenges, pages 475-489, Cham, 2021. Springer International Publishing.
[44] Hanqiu Deng and Xingyu Li. Anomaly detection via reverse distillation from one-class embedding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pages 1737-97466 June 2022.
[45] Mohammadreza Salehi, Niousha Sadjadi, Soroosh Baselizadeh, Mohammad H Rohban, and Hamid R Rabiee. Multiresolution knowledge distillation for anomaly detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14902 14912, 2021.
13

---

[46] Paul Bergmann, Michael Fauser, David Sattlegee, and Carsten Steger. Uninformed students: Student- teacher anomaly detection with discriminative latent embeddings. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4183-4192, 2020.
[47] Guodong Wang. Shumin Han, Errui Ding, and Di Huang. teacher feature pyramid matching for anomaly detection. arXiv preprint arXiv:2103. 042577 2021.
[48] Tal Reiss, Niv Cohen, Liron Bergman, and Yedid Hoshen. Panda: Adapting pretrained features for anomaly detection and segmentation. In Proceedingy of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2806-2814 June 2021.
[49] Oliver Rippel, Patrick Mertens, Eike König, and Dorit Merhof. Gaussian anomaly detection by modeling the distribution of normal data in pretrained deep features. IEEE Transactions on Instrumentation and Measurement, 70:1 10:1-13, 2021.
[50] Karsten Roth, Latha Pemula, Joaquin Zepeda, Bernhard Schölkopf. Thomas Brox, and Peter Gehler. Towards total recall in industrial anomaly detection. In 2022 IEEE/CVF Conference on Computer Vision and Pottern Recognition (CVPR), pages 14298 14308, 2022.
[51] Zhikang Liu, Yiming Zhou, Yuansheng Xu, and Zilei Wang. Simplenet: A simple network for image anomaly detection and localization. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 20402-20411 2023.
[52] Songqiao Han, Xiyang Hu, Hailiang Huang, Minqi Jiang, and Yue Zhao. ADBench: Anomaly detec- tion benchmark. In Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2022.
[53] Laurenz Strothmann, Uwe Rascher, and Ribana Roscher. Detection of anomalous grapevine berries using all-convolutional autoencoders. In IGARSS 2019 2019 IEEF International Geoscience and Remote Sensing Symposium, pages 3701-3704, 2019.
[54] Pankaj Mishra, Riccardo Verk, Danicle Fornasier, Claudio Piciarelli, and Gian Luca Foresti. VT-ADL: vision transformer network for image anomaly detection and localization. In 30th JEEE/IES International Symposium on Industrial Electronics (ISIE), June 2021.
[55] Yunscung Lee and Pilsung Kang. Anovit: Unsupervised anomaly detection and localization with vision transformer-hased encodler-dccoder. IEEE Access, 10:46717-46724, 2022.
[56] Miro Miranda, Laura Zabawa, Anna Kicherer, Laurenz Strothmann, Uwe Rascher, and Ribana Roscher. Detection of anomalous grapevine berries using variational autocncoders. Frontiers in Plant Science, 13, 2022.
[57] Mohammad Sabokrou, Masoud Pourreza, Mohsen Fayyaz, Rahim Entezari, Mahmood Fathy, Jürgen Gall, and Ehsan Adeli. Avid: Adversarial visual irregularity detection. In C.V. Jawahar, Hongdong Li, Greg Mori, and Konrad Schindler, editors, Computer Vision 2018, pages 488-505, Cham, 2019. Springer International Publishing.
[58] Shashanka Venkataramanan, Kuan-Chuan Peng, Rajat Vikram Singh, and Abhijit Mahalanobis. Attention guided anomaly localization in images. In European Conference on Computer Vision, pages 485-503. Springer, 2020.
[59] Marcella Astrid, Muhammad Zaigham Zaheer, and Seung-Ik Lee. Synthetic temporal anomaly guided end-to-end video anomaly detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 207-214, 2021.
[60] Wennui Liu, Hong Chang, Bingpeng Ma, Shiguang Shan, and Xilin Chen. meearsty anomaly detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12147-12156, 2023.
[61] Zhiwei Yang, Jing Liu, Zhaoyang Wu. Peng Wu, and Xiaotao Liu. Video event restoration based on keyframes for video anomaly detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14592-14601, 2023.
[62] Neelu Madan, Nicolae- Cataaee-CCnlinn Ristea, Radu Tudor Ionescu, Kamal Nasrollahi, Fahad Shahbaz Khan, Thomas B. Moeslund, and Mubarak Shah. Self-supervised masked convolutional transformer block for anomaly detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(1):525 542, 2024.
14

---

[63] Julian Wyatt, Adam Leach, Sebastian M. Schmon, and Chris G. Willcocks. Anoddpm: Anomaly detection with denoising diffusion probabilistic models using simplex noise. In 2022 IEEE/CVF Conference on Computer Vision and Puttern Recognition Workshops (CVPRW), pages 649-655, 2022.
[64] Haoyang He, Yuhu Bai, Jiangning Zhang, Qingdong He, Hongxu Chen, Zhenye Gan, Chengjie Wang. Xiangtai Li, Guanzhong Tian, and Lei Xie. Mambaad: Exploring state space models for multi-class unsupervised anomaly detection, xrviv preprint
(tf::2404.66664 2024.
[65] Zhiyuan You, Lei Cui, Yujun Shen, Kai Yang, Xin Lu, Yu Zheng, and Xinyi Le. A unified model for mulli-class, anomaly detection. Advances in Neural Information Processing Systems, 35:4571 14844, 2022.
[66] Chaoqin Huang, Qinwei Xu, Yanfeng Wang, Yu Wang, and Ya Zhang. Self-supervised masking for unsupervised anomaly detection and localization. IEEE Transactions on Multimedia, 25:4426-4438, 2023.
[67] Dezhao Luo, Chang Liu, Zhou, Dongbao Yang. Can Ma, Qixiang Ye, and Weiping Wang. Video cloze procedure for self-superrised spatio-temporal learning. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 11 1701-11708 2020.
[68] Hannah M. Schlüter, Jeremy Tan, Benjamin Hou, and Bernhard Kainz. Natural synthetic anomalics for self-supervised anomaly detection and localization. In Shai Avidan, Gabriel Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassner, editors, Computer Vision ECCV 2022, pages 474-489, Cham, 2022. Springer Nature Switzerland.
[69] Guodong Wang, Yunhong Wang, Jie Qin, Dongming Zhang, Xiuguo Bao, and Di Huang. Video anomaly detection by solving decoupled neatool jigsaw puzzles. In European Conference Computer Vision, pages 494 511. Springer. 2022.
[70] Vitjan Zavrtanik, Matej Kristan, and Danijel Skočaj. Draem-a discriminatively trained reconstruction embedding for surface anomaly detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8330-8339, 2021.
[71] Anne-Sophie Collin and Christophe De Vleeschouwer. Improved anomaly detection training an au- toencoder with skip connections on images corrupted with shaped noise. 2020 25th International Conference on Pattern Recognition (ICPR), pages 7915-7922. IEEE, 2021.
[72] Chun-Liang Li, Kihyuk Sohn, Jinsung Yoon, and Tomas Pfister. Cutpaste: Self-supervised learning for anomaly detection and localization. In 2021 IEFE//VV Conference on Computer Vision and Pattern Recognition (CVPR), pages 9659-9666 2021.
[73] Masoud Pourreza, Bahram Mohammadi, Mostafa Khaki, Samir Bouindour, Hichem Snoussi, and Moham- mad Sabokrou. G2d: Generate to detect anomaly. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 2003-2011 2021.
[74] Muhammad Zaigham Zaheer, Jin-ha Lec, Marcella Astrid, and Scung- -k Lec. Old is gold: Redefining the adversarially learned one-class classifier training paradigm. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 14 14183-14193, 2020.
[75] Kamalakar Vijay Thakare, Yash Raghuwanshi, Debi Prosad Dogra, Heeseung Choi, and Ig-Jae Kim. Dyannet: A scene dynamicity guided self- video anomaly detection network. In Proceedings of the IEEE/CVF Winter conference on applications of computer vision, pages 5541-5550, 2023.
[76] Jingtao Li, Xinyu Wang, Hengwei Zhao, Liangpei Zhang, and Yanfei Zhong. A unified remote sensing anomaly detector across modalities and scenes via deviation relationship learning. arXiv preprint arXiv:210.005511 1, 2023.
[77] Ximiao Zhang, Min Xu, and Xiuzhuang Zhou. Realnet: A feature selection network with realistic synthetic anomaly for anomaly detection. arXiv preprint arXiv:2403.0588 0.0807.05977.1 2024.
[78] Jiehui Xu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Anomaly transformer:
Time series anomaly detection with association discrepancy. In International Conference on Learning Representations, 2022.
[79] Dongha Lcc, Sehun Yu, Hyunjun Ju, and Hwanjo Yu. Weakly supervised temporal anomaly segmentation with dynamic time warping. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7355-7364, 2021.
[80] Zhijie Zhong, Zhiwen Yu. Yiyuan Yang, Weizheng Wang. and Kaixiang Yang. Patchad: Patch- mip-mixer for time series anomaly detection. arXiv preprint axXiv:2401.09793 2024.
15

---

[81] Shreshth Tuli, Giuliano Casale. and Nicholas R Jennings. TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Data. Proceedings of VLDB, 15(6):100111214, 2022.
[82] Yu Zheng. Huan Yee Koh, Ming Jin, Lianhua Chi, Khoa T. Phan, Shirui Pan, Yi-Ping Phoebe Chen, and Wei Xiang. Correlation-aware seapirlllempora graph learning for multivariate time-series anomaly detection. IEFF Transactions on Neural Newworks and Learning Systems, 35(9):1 1802-11816, 2024.
[83] Jia-Xing Zhong, Nannan Li, Weijie Kong, Shan Liu, Thomas H. Li, and Ge Li. Graph convolutional label noise cleaner: Train a plug-and-play action classifier for anomaly detection. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2019.
[84] Muhammad Zaigham Zaheer, Arif Mahmood, Marcella Astrid, and Scung- Ik Lec. Claws: Clustering assisted weakly supervised leamning with normalcy suppression for anomalous event detection. Com- puter Vision ECCV 2020: 16th European Conference, Glasgow, UK, August 23 28, 2020, Proceedings, Part XXII 16, pages 358-376. Springer, 2020.
[85] Hui Lv, Chuanwei Zhou, Zhen Cui, Chunyan Xu, Yong Li, and Jian Yang. Localizing anomalies from weakly-labeled videos. IEEE Transactions on Image Processing, 30:4505-4515, 2021.
[86] Jia- Pa-Chang Feng, FinTing Hong. and Wei-Sh Shi Zheng. Mist: Multiple instance self traintri framework for video anomaly detection. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 14009-14018, 2021.
[87] Seongheon Park, Hanjae Kim, Minsu Kim, Dahye Kim, and Kwanghoon Sohn. Normality guided multiple instance learning for weakly supervised video anomaly detection. In Proceedings of the IEEF/CVF Winter Conference on Applications of Computer Vision (WACV), pages
January 2023.
[88] Hang Zhou, Junqing Yu, and Wei Yang. Dual memory units with uncertainty regulation for weakly supervised video anomaly detection. In Proceedings the AAAI Conference on Artificial Intelligence, volume 37, pages 3769-37777777 00079377772777 2023.
[89] Zhenting Qi, Ruike Zhu, Zheyu Fu, Wenhao Chai, and Volodymyr Kindratenko. Weakly supervised two- stage training scheme for deep video fight detection model. In 2022 IEEE 34th International Conference on Tools with Artificial Intelligence (ICTAI). pages 677-685. IEEE, 2022.
[90] Jhih-Cine Ciang Wu, Yen Hsieh, Ding-Jie Jie Chen, Chiou- Fuh, and Tyng-Luh Liu. Sulf-super sparse representation for video anomaly detection. In Shai Avidan, Gabriel Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassner, editors, Computer Vision ECCV 2022, pages 729-745, Cham, 2022. Springer Nature Switzerland.
[91] Hitesh Sapkota and Qi Yu. Bayesian nonparametric submodular video partition for robust anomaly detection, Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2212-3221, 2022.
[92] Minqi Jiang, Chaochuan Hou, Ao Zheng, Xiyang Hu, Songqiao Han, Hailiang Huang, Xiangnan He, Philip S Yu, and Yue Zhao. Weakly supervised anomaly detection: A survey, arXiv preprint arXiv:2302.0455 2023.
[93] Tao Jiang, Weiying Xie, Yunsong Li. Jie Lei, and Qian Du. Weakly supervised discriminative learning with spectral constrained generative adversarial network for hyperspectral anomaly detection. IEEE Transactions on Neural Networks and Learning Systems, 33(1 I 1):6504-6517, 2022.
[94] Waqas Sultani, Chen Chen, and Muharak Shah. Real-world anomaly detection in surveillance videos. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6479-6488, 2018.
[95] Boyang Wan, Yuming Fang, Xue Xia, and Jiajie Mei. Weakly supervised video anomaly detection via center-guided discriminative learning. In 2020 IEEE International Conference on Multimedia and Expo (ICME), pages 1-6, 2020.
[96] Yu Tian, Guansong Pang, Yuanhong Chen, Rajvinder Singh, Johan W. Verjans, and Gustavo Carneiro. Wcakly-supervised vidco anomaly detection with robust temporal feature magnitude lcarning. In 202/ IEEE/CVF International Conference on Computer Vision (ICCV), pages
2021.
[97] Yingxian Chen, Zhengzhe Liu, Baoheng Zhang, Wilton Fok, Xiaojuan Qi, and Yik-Chung Wu. Mgfn: Magtaivuee alance-and-focuu network for weakly-supervised video anomaly detection. In Proceedings the AAAÍ Conference on Artificial Intelligence, volume 37, pages 387-395, 2023.
[98] Yixuan Zhou, Yi Qu, Xing Xu, Fumin Shen, Jingkuan Song, and Hengtao Shen. Batchnorm batchnorm-based weakly supervised video anomaly detection. arXiv preprint arXiv:2311.15367 2023.
16

---

[99] Hui Lv. Zhonggi Yue, Qianru Sun, Bin Luo, Zhen Cui, and Hanwang Zhang. Unbiased multiple instance learning for weakly supervised video anomaly detection. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pages 8022-8031, 2023.
[100] Ze Liu, Jia Ning. Yue Cao. Yixuan Wei, Zheng Zhang. Stephen Lin, and Han Hu. Video swin transformer. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3202- 3211, 2022.
[101] Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in neural information processing systems, 30, 2017.
[102] Lijun Yu, José Lezama. Nitesh B. Gundavarapu, Luca Versari, Kihyuk Sohn, David Minnen, Yong Cheng, Agrim Gupta, Xiuye Gu, Alexander G. Hauptmann, Boqing Gong. Ming-Hsuan Yang, Irfan Essa, David A. Ross, and Lu Jiang. Language model beats diffusion tokenizer is key to visual generation, 2023.
[103] Aren Jansen, Daniel PW Bllis, Shawn Hershey, R Channing Moore, Manoj Plakal, Ashok C Popat, and Rif A Saurous. Coincidence, categorization, and consolidation: Learning to recognize sounds with minimal supervision. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 121-125. IEEE, 2020.
[104] Wenze Yang. Felix Kogan, and Wei Guo. An ongoing blended long-term vegetation health product for monitoring global food security. Agronomy, 10 12), 2020.
[105] Hans Hersbach, Bill Bell, Paul Berrisford, Shoji Hirahara, András Horányi, Joaquín Muñoz-Sabater, Julien Nicolas, Carole Peubey, Raluca Radu, Dinand Schepers, Adrian Simmons, Cornel Soci, Salch Abdalla, Xavier Abellan, Gianpaolo Balsamo, Peter Bechtold, Gionata Biavati, Jean Bidlot, Massimo Bonavita, Giovanna De Chiara, Per Dahlgren, Dick Dee, Michail Diamantakis, Rossana Dragani, Johannes Flemming, Richard Forhes, Manuel Fuentes, Alan Geer, Leo Haimberger, Sean Healy, Robin Hogan, Elías Hólm, Marta Jamisková, Sarah Keeley, Patrick Laloyaux, Philippe Lopez, Cristina Lupu, Gahor Radnoti, Patricia de Rosnay, Tryna Rozum, Freja Vamborg, Schastien Villaume, and Jean-Noël Thépaut. The era5 global reanalysis. Quarterly Journal of the Royal Meteorological Society, 146(730):1999-2444 2049, 2020.
[106] Schimanke, M. Ridal, P. Le Moigne, L. Berggren, Undén, R. Randriamampianina, U. Andrea, E. Bazile, A. Bertelsen, P. Brousseau, P. Dahlgren, L. Edvinsson, A. EI Said, M. Glinton, S. Hopsch, L. Isaksson, R. Mladek, E. Olsson, A. Verrelle, and Z.Q. Wang. Cerra sub-daily regional reanalysis data for europe single levels from 1984 present, 2021.
[107] J. Muñoz Sabater, E. Dutra, A. Agustíi-Panarede C. Albergel, G. Arduini, G. Balsamo, S. Boussetta, M. Choulga, Harrigan, Hersbach, B. Martens, G. Miralles, M. Piles, N. Rodriguez-Femáádde. Fernáddez, E. Zsoter, C. Buontempo, and J.-N. Thépaut. Era5-land: a statc-of-the-art global reanalysis dataset for land applications. Earth System Science Data, 13(9))4399 43383, 2021.
[108] F. Giorgi, C. Jones, and G. R. Asrar. Addressing climate information needs at the regional level: the cordex framework. Bulletin World Meteomolgicaa Organization, 175-133,,,183, 2009.
[109] Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In Proceedings the 34th International Conference Machine Learning Volume 70, ICML' 17, page 3319-3328. JogR. 2017.
[110] Fei Tony Liu, Kai Ming Ting, and Zhi-Hua Zhou. Isolation- anomaly detection, ACM Transactions on Knowledge Discovery from Data (TKDD), 6(1): 2012.
[111] Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.0075: 2023.
[112] Tung Nguyen, Johannes Brandstetter, Ashish Kapoor, Jayesh K Gupta, and Aditya Grover. Climax: A foundation model for weather and climate. arXiv preprint 1rXiv:2301.10343, 2023.
[113] Christian Lessig, Ilaria Luise, Bing Gong. Michael Langguth, Scarlet Stadler, and Martin Schultz. Atmorep: stochastic model of atmosphere dynamics using large scale representation learning. arXiv preprint arXiv:2308. 13280, 2023.
[114] Shung-Cheng Chiu, James Qin, Yu Zhang. Jiahui Yu, and Yonghui Wu. supervied learning with random-pres projection quantizer for speech recognition. In International Conference on Machine Learning, pages 3915-391 39244. PMLR, 2022.
17

---

[115] Jiahui Yu, Xin Li. Jing Yu Koh, Han Zhang, Ruoming Pang, James Qin, Alexander Ku, Yuanzhong Xu, Jason Baldridge, and Yonghui Wu. Vector-quantized image modeling with improved VQGAN. In International Conference on Learning Representations, 2022.
[116] Woncheol Shin, Gyubok Lee, Jiyoung Lee, Eunyi Lyu,, Joonseok Lee, and Edward Choi, Exploration into translation- tranlation-equivariant equivariant image quantization. In ICASSP 2023-2023 JEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1-5. JEEE, 2023.
[117] Fabian Mentzer, David Minnen, Eirikur Agustsson, and Michael Tschannen. Finite scalar quantization: VQ-VAE made simple. In The Twelfth International Conference on Learning Representations, 2024.
[118] Lianghui Zhu, Bencheng Liao, Qian Zhang, Xinlong Wang, Wenyu Liu, and Xinggang Wang. Vision mamha: Efficient visual representation learning with bidirectional state space model. arXiv preprint (XXi:2401.094111 2024.
[119] Xiaohuan Pci, Tao Huang, and Chang Xu. Efficientvmamba: Atrous selective scan for light weight visual mamba. arXiv preprint arXiv:2403.0900 2024.
[120] Guo Chen, Yifei Huang, Jilan Xu, Baoqi Pei, Zhe Chen, Zhiqi Li, Jiahao Wang. Kunchang Li, Tong Lu, and Limin Wang. Video mamba suite: State space model versatile altemnatie for video understanding. arXiv preprint arXiv:2403.09622 2024.
[121] Kunchang Li, Xinhao Li, Yi Wang, Yinan Hc, Yali Wang, Limin Wang, and Qiao. Vidcomamba: State space model for efficient video understanding. arXiv preprint arXiv:2403.06977 2024.
[122] Tao Huang, Xiaohuan Pei. Shan You, Fei Wang, Chen Qian, and Chang Xu. Localmamba: Visual state space model with windowed selective scan. arXiv preprint arXi::4403.0933. 2024.
[123] Jinyoung Park, Hee-Seon Kim, Kangwook Ko. Minbeom Kim, and Changick Kim. Videomamba: Spatio-temporal selective state space model. arXiv preprint arXiv:2407.08476, 2024.
[124] Vincent Tao Hu, Stefan Andreas Baumann, Ming Gui, Olga Grebenkova, Pingchuan Ma, Johannes Fischer, and Björn Ommer. Zigma: A dit-style zigzag mamba diffusion model. In Arxiv, 2024.
[125] Almudena Garcia, Francisco José Cuesta Valero, Diego G Miralles, Miguel D Mahecha, Johannes Quaas, Markus Reichstein, Jakob Zscheischler, and Jian Peng. Soil heat extremes can outpace air temperature extremes. Nature Climate Change, 13(1 13(1111227712411 2023.
[126] Preet Lal. Gurjeet Singh, Narendra N. Das, Andreas Colliander, and Dara Entekhabi. Assessment of cra5-land volumetric soil water layer product using situ and smap soil moisture observations. IEEE Geoscience and Remote Sensing Letters, 19:1-5 5, 2022.
[127] Dara Entekhabi, Eni G. Njoku, Peggy E. O' Neill,, Kent H. Kellogg, Wade T. Crow, Wendy N. Edelstein, Jared K. Entin, Shawn D. Goodman, Thomas J. Jackson, Joel Johnson. John Kimball, Jeffrey R. Piepmeier. Randal D. Koster, Neil Martin, Kyle C. McDonald, Mahta Moghaddam, Susan Moran, Rolf Reichle, C. Shi, Michacl W. Spencer, Samuel W. Thurman, Leung Tsang, and Jakob Van Zyl. The soil moisture active passive (smap) mission. Proceedings of the IEEE, 98(5):044 716, 2010.
[128] Narine Kokhlikyan, Vivek Miglani, Miguel Martin, Edward Wang, Bilal Alsallakh, Jonathan Reynolds, Alexander Melnikov, Natalia Kliushkina, Carlos Araya, Siqi Yan, et al. Captum: A unified and generic model interpretability library for pytorch. arXiv preprint arXiv:2009.07896 2020.
[129] Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 1412,694,,, 2014.
[130] Xianfeng Liu, Xiufang Zhu, Yaozhong Pan, Shuangshuang Yanxu Liu, and Yuqi Ma. Agricultural drought monitoring: Progress, challenges, and prospects. Journal of Geographical Sciences, 26:750-766 2016.
[131] I. Meza, S. Siebert, P. Döll. J. Kusche, C. Herbert, E. Eyshi Rezaei, H. Nouri, H. Gerdener, E. Popat. J. Frischen, G. Naumann, J. V. Vogt. Y. Walz, Z. Sebesvari, and M. Hagenlocher. Global-scale drought risk assessment for agricultural systems. Natural Hazards and Earth System Sciences, 20(2):655-712. 2020.
[132] María Pedro-Monzome Abel Solera, Javier Ferrer, Teodoro Estrela, and Javier Paredes-Arquoola. A review of water scarcity and drought indexes in water resources planning and management. Journal of Hydrology, 527:482 493, 2015.
18

---

[133] Compton J. Tucker. Red and photographic infrared linear combinations for monitoring vegetation. Remote Sensing of Environment, 8(2):127-150, 1979.
[134] F.N. Kogan. Application of vegetation index and brightness temperature for drought detection. Advances Space Research, (5(11):11-10) 1995, Natural Hazards: Monitoring and Assessment Using Remote Sensing Technique.
[135] Felix Kogan, Wei Guo, and Aleksandar Jelenak. Global vegetation health: Long-term data records. In Felix Kogan, Alfred Powell, and Oleg Fedorov, editors, Use of Satellite and In-Situ Data to Improve Sustainability, pages 247 255, Dordrecht, 2011. Springer Netherlands.
[136] Kogan. Remote sensing of weather impacts on vegetation in non-homogeneous areas. International Journal of Remote Sensing, 11(8): 1400-1409,, 1990.
[137] Anne Van Loon, Tom Gleeson, Julian Clark, Albert IJ Van Dijk, Kerstin Stahl, Jamie Hannaford, Giuliano Di Baldassarre, Adriaan J Teuling, Lena M Tallaksen, Remko Uijlenhoet, et al. Drought in the anthropocene. Nature Geoscience, 92): 91. 2016.
[138] Jiawei Zhuang, raphael dussin, André Jüling, and Stephan Rasp. JaaweiZZuuang/EESSM: v0.3.0 Adding ESMF.LocStream capabilities, March 2020.
[139] Mohamad Hakam Shams Eddin and Juergen Gall. Identifying spatio-temporal drivers of extreme events [data set]. 2024.
19

---

A Synthetic data
To the best of our knowledge, ground truth labels of drivers or anomalies which are correlated with extreme events impacts barely exist. This is especially the case when it comes drivers or anomalous events which are related to a specific definition of an extreme event within the Earth system (i.e., drought flood). For this reason, we designed framework generate artificial datasets which can be adopted to the task. Our framework is inspired by and related to Flach, et aspects:
al. (TT]. However, (I) First, we aim to generate multivariate anomalous that are correlated with a specific extreme event, while their aim is to generate anomalous that can occur simultaneously in multivariate data streams similar to [78-8]] Think of an tncreasing/decreasse are interested in temperature anomalies regardless of the subsequent extreme events they might cause, while we focus on temperature anomalies that can cause particular extreme events in the near future. (2) Second, we generate the synthetic data based on real-world data stream signals, while they use trigonometric functions (i.e., sine function) to mimic Earth observations across spacetime.
differ in the following
of temperature, existing approaches
In the following, we explain our overall framework in more details:
(I) First, we generate the normal base signals B R x for a of different variables V. where T is the temporal extension, and Lat and Lon are the spatial extensions. For instance, synthesize CERRA [106] B signals, we take the mean values from the CERRA climatologoy pixel-wise. This represents the typical value of B at specific time (week) and location (lat, lon). By definition, B inherits the intrinsic properties of the simulated variables including the existence of seasonality and correlations among variables.
(2) In the next step. we induce binary extreme events E Σ, Latx Lon within the datacube and track the precise spatio-temporal location of these events. Similar to Flach, et. al [2]] the type and duration of extreme events vary within the datacube. For instance, we alter the duration of the events between long and short extreme events. The spatial distribution of the extreme event vary also between a local event at one pixel (LocalEvent), a rectangular event (CubeEvent),, a Gaussian shape (GaussianEvent), an onset event that starts at specific time and lasts until the end of the series (OnsetEvent)), and a random walk event that starts specific pixel and affecting neighboring pixels with time (Random WalkEvent).
nartral event
2nommabuu event
oxtrema evant
2m
Simulsted vorsibes
total cloud cover
totel precisttation ion
reliive humidity
mesture
-Bas
+2hg
+d0bg
Figure 6: An example of the randomly generated coupling matrix between the synthetic variables and extremes for the synthetic CERRA dataset.
(3) We randomly define a coupling matrix M between the variables and the extreme event. An example is shown in Fig. 6 i.e., the anomalous events for 2m temperature start time steps (-AT,) before the extreme and last for 4 time steps afterwards (+AT,). While albedo and relative humidity are not coupled with the extremes. Based on M and Eer we generate the binary anomalous events matrix E" 2 LotxLon
(4) Similar steps (2)-(3), we generate binary random anomalous events E Z Lat
However, these events are uncorrelated with the extremes and generated randomlies for all variables. (5) We sample noise signals N I Laxxx for each variable. The noise signals could be sampled from a normal Gaussian distribution (GaussianNoise), standard Cauchy distribution
20

---

(CauchyNooise), double exponential distribution (LaplaceNoise). or from a spatiotemporal correlated noise across the datacube (RedNoise). (6) Using the indices E {I...., V). E {1..... T). lat E {,..... Lal), and lon E Lon), the synthetic signal + (v.t. is generated with the following formula:
(e,t,lat,ion)
(oo) (v,,,lat,,on)) O (e, (e,t,lat,lon)
B bat (2 ( (Eosstatttt))) V V ddatstoo)) -1)
(o.tiat.ion)
N (a,,,latt,on)
(o,t,lat, ks. (E
V Eer (on)) .NN
(8)
if
E (o,s,attoo) =
(o,,,lat,oon)
8
(9)
+1,
otherwise.
-1, if (1,t,lat, S 0,
A (vntdattoon)
(10)
+1, otherwise,
where (v,t,lat,too)) is the induced anomaly, kb, kn, and ks are control parameters for the events magnitudes, oN is the standard deviation of the noise signal, E 1 is the predefined coupling sign with extremes from M, and (e...lat,ion) controls the sign of the induced anomaly for extremes
(i.e., a deficiency in soil moisture (6 = -1) and an increased in temperature (ō +1) during an extreme drought event).
Using this framework, we generated types of datasets; (I) synthetic CERRA reanalysis (Tables (2) synthetic NOAA remote sensing (Tables D. and (3) synthetic artificial data (Tables
We generate the NOAA base signals from [104]. For the NOAA and artificial datasets, we also considered artificial linear (LinearCoupling) and non-linear (QuadraticCoupling) dependencies among the variables as additional data properties. For instance to generate a new dependent base signal from independent bases:
ΣV
((t))B(t))
W(T) B (n)
if LinearCoupling, 1),
(ic,dur,oo)
(11)
w (v) (B2 (v)
otherwise,
E (i,,att,on))
((t,,tfat,Ion)
(12)
where W() RRV are weighted coefficients sampled either from normal (NormWeight) a Lablace (LaplaceWeight) distribution. In addition, we add an option to generate disturbed weights where weights vary spatio-temporally based the locations of anomalies. Anomalous events that occur in one of the independent base signals propagate to the new generated dependent signals.
The third dataset (Artificial) does not depend on real-world data but rather consists of basis signals with trigonometric functions where also add linear latitudinal gradient (latGrad). Finally, we mask out some regions to exhibit no anomalies i.e., pixels over water surface. Each generated dataset consists 52 x 46 time steps corresponding Examples of the synthesized dataset can be found in Figs. 7 The results the synthetic CERRA reanalysis are reported in Table and the results on synthetic NOAA and the artificial data are shown Sec. Bi Tables and
46 years simulated data in 7-day intervals.
21

---

Table 3: Configurations of the synthetic CERRA reanalysis.
Symchetic CERRA reanalysis
Ind. variables
Coupled variahles Dimencion with
Extrome events
-TT +AT, Ty Si Extreme & Corrolated anomalous
variables
110==200
(Cabebvestoo-20 sx-35, syw35, sz=25),
1.69
KandomWalkLvemtasIJ10 s-125).
time-52 x 46) LocatEventto 2600, s2-17).
Randoan anomalous 1.32
Comsonneeemii 540, speds, syw35, a2w23).
OnsedVeeuin=25, sx-17, sy-17, 05801188)
Base
Dependency Weights
Noise
Random events
kb
km
ks
Albedo
WhiteNoiso
(CaheB s==15, 8y33, sr=25).
0.30
0.20
0.50
meu-f.
RandmnWalkPeemtg-toy 4-125).
sigma=0.01)
LeclE sx=17)
Gumsinn sx:15, w0=5 y:35. (:25)
2m
RedNoisee
(OnerfSeento-18
41
0.01
0
0.50
Temperatare
meuzO,
Rembm/KikEEvm66:58001111 s=125)
<igma=0.90)
1eclE sr=17)
Tocal cloud
LaplaveNoiset
(CabeB>m/000000aa sx:35 o=:3 wx=25)
0.03
0,08
0.50
sigma=0.70 70,
LecilEvemts=2200xxe11)
lambd==1) I)
Total
WhiteNoist
0.07
0.20
0.50
precipitation
meu=D,
sigma=0.00)
ComsinEneemass4000 Sunssatioomiiii...sssl........., sx22)
Relative humedity
Naichy
(OnsetEvemin=18. (t=.::/..=01..:/f.sd00000. oz20))))
0.06
0.06
0.50
meu=f,
sigma=0.7)
LecalEvento=1100 sz=17)
ComssinExemla015000 150 sx:35, x=25)
Volumetric
WhineNoiso
(CateEreminn sx:35 a=35 2=225)
0.10
0,10
0.50
moisture
meu=D.,
RanlmnWakkvemtt=-0000,
s=125).
sigmu=017)
LecalEeetoss2000eee11
GomssnnFeem(:=000 sx:15, sy:35 s=25)
Table Configurations of the synthetic NOAA remote sensing.
Synthetic NOAA
Dep.
Coupled Dimension with extreme
ATT +T,
variables variables
anomalous
(lnn~200
[Cubetiventia 200, spudd, sysa5, sow25),
0.79
1.02
las-200,
SandorraWlllBvveeni 1000 1100 s=125),
time=52
46) Decal=eem/==2600,, scal7), 7)),
% Rundom anamalous 1.76
Gaussimbventtreei ax435, syeas, s2=25),
OnsetEventis=25. 25, sx-17, sy=17, 0520.98))
Base
Dependency Weights
Noise
Ramdom evenis
km
WhiteNeisef
[CuleFient(=:30) sc-35, sx=55, s2=25).
0.25
0.10
0.50
mes-0.
FildoomWillFFeentto--l 10000011155) s-125).
sigma=0.30)
LocilEvesta s2-17),
EamosianFventnes sx=38 sy-35, se:25)
WhiteNoisef
[CuleFientto-50 300, 5x-35, sy-15, s2-25).
0.01
0.50
mes=O)
Walbbmi..EEee s=12))
sigms=0.5)
LacalExam(n==2000 sm=17),
Guussin 3x-38, 5y-35, s2-25)
Quidratic CouplingO Weight)
Norm
WhiteNeisef
0.01
0.01
0.50
mos=O)
(igme-0.65)
Tincar Laplace WeightO
WhiteNoisef
0.50
Couplings)
meg=0.
sigms=003 (65)
Linear Laplace
WhiteNoisel
0.01
0.01
0.50
Coupling
Weightg
mos=O.
(i9ms-(66)
22

---

Table Configurations of the synthetic artificial data.
Synthotic artificiial
Dep.
Coupled with extreme
event
A., +AT,,
variables
variables
anomolous
(lon=200)
[Cuhebventt==200, ax-35, sywdd, s2w25).
1.24
1.81
lat-200,
KundonWalkEvessn--ll 100, s+125)
time:32
LecalEvem(s==2500, sz=17).
Random anomalous 2.93
Gaussanbvensioo44t. sx-35, sy-35, azw25),
Onsetiventn=25, sx=17,, sy=17, oz-0189)
Buse
Dependency Weights
Nuise
Random events
kb
kn
ks
SineBase
RedNNise((
[CuheEwetin:1201.m 35=35 sy-35, sz=25)
0.35
0.35
0.35
shift-0,
men-l.
amg-3.
LecalEvent(a=4400, s2-17).
nDess46.
GussianEvenr(e=t0 sx:35, sy=35 sm:25)
IatGrad-True)
CosineBBase
LaplaueNeise( meuzt)
[OnseEEventiz 18, sx=17, sy=17, 0s:0.98).
+1
0.35
0.35
0.35
shifteD,
RankmmWalFFemmaee x=125). LecilEven(s=-10,
lambda=1)
GuusianEve(n=130)) su=35, 3y:35 sx=25))
IoGGrdd-Troe) True))
ConstrerBaso
Whineaaiser
(CuheEventt--iMM. ax-35, sy-55 s2-25)
1
0.90
0.90
0.50
RankmmNilkEmmmn220000 s=125)
IuGrnddTrue))
sigmsme6.07)
Leatliven((= u=17).
GousianExem/n2290 su=35, 3y535 sx=25)
Quadratic CouplingO
Norm Weightr)
WhiteNoiscl
0.35
0.35
0.35
sigma=00.65) 65)
Lineer Laplace Weightt)
WhiteNoiser
0.35
0.35
0.35
CouplingO
sigmes 065)
Lineur CouplingO
Laplace Weight
WhiteNoisel meuzt).
-1
0.35
0.35
0.3
sigma= 065)
23

---

<by
abedo
-02
-04
-0F4
-61
.84
-bt,
Imh tomperauuee
Mo
M:
-0tA
-bs
-MG
-bh
totba cloud COUet
Mo
-Mr
-Mh>
-Mr.
-Mr
-Ms
-Ab
Kode pracc prec-tttion gitottioo
relatvv fumidlty
-0%
-61
-64
vOL mammmee soll mont
Time
Figure 7: Examples of the synthetic CERRA reanalysis data described in Table 3 The drivers/anomalies are visualized under each variable directly. Here, albedo and relative humidity are not correlated with the extremes.
24

---

NOWI
.Mr
.22
.M3
-M4
.M3
.Mr
variabll 03
poriaall d4
.1s
-Bf
.bt
iasiaatl 05
Time
Figure 8: Examples of the synthetic NOAA remote sensing described in Table The drivers/anomalies are visualized under each variable directly. Here, NDVI and variables 05 are not correlated with the extremes.
25

---

Att
TO0000L 01
-At.
-Ah>
-At
-b5
-0%
-At
Ar
.M
-A1
At
-Ms
-Mg
-Mb
A1
-Mh
-Al
AA
-Mg
<Mg
-A1
Calieoe 50
-6G
-BL.
-bls
-ble
Time
Figure 9: Examples of the synthetic artificial data described in Table 5
The drivers/anomalies
visualized under each variable directly. Here, variables 01 and 05 are not correlated with the extremes.
26

---

Zen temperature sture
2000
1090
DO00
1500
1590
530
2000
3590
2009
Smauboodd aigial
IITOTT owent alted autrerrie
randdom AITOTTE CUR
Figure 10: Visualization of the generated signals T for 6 different variables from the synthetic CERRA reanalysis described in Table The time series are shown for the location (lat = 50, lon = 50). the drivers/anomalies which are correlated with extremes, and are random anomalies.
10000
1
Voviieble OS
300
DNVeed siggnni
STTOITI MAMSS CHEREE atted WiTh CuTIeNTie
FRANOOTE DIVOUTES OUT
Figure 11: Visualization of the generated signals for 5 different variables from the synthetic NOAA data described in Table The time series are shown for the location (lat = 50, lon = 50). are the drivers/anomalies which correlated with extremes, and are random anomalies.
27

---

KINCE
Figure 12: Visualization of the generated signals & for 6 different variables from the synthetic artificial data described in Table The time series are shown for the location (lat = 50, lon = 50). are the drivers/anomalies which are correlated with extremes, and are random anomalies.
28

---

B Additional results on the synthetic data
In Tables 6 and we report the results on the synthetic NOAA remote sensing and artificial data described in Sec. A We noticed a drop of performance for all models for the synthetic artificial data (Table This explained as the ratio of anomalies is higher than the other two datasets. Second, we used for this artificial dataset red noise and quadratic coupling to generate dependent base signals. This makes this dataset harder for training. SimpleNet exhibits dramatic dropped out performance when it is tested on these two synthetic datasets. This illustrates that the model performance is highly dependent on the dataset and the backbone the model was trained on. Our model still outperforms all baselines on these datasets.
Table 6: Driver detection results on the synthetic NOAA remote sensing. The best performance on each metric is highlighted in bold text. (=) denotes the standard deviation for 3 runs.
Validation
Testing
Algorithm Fl-score (f)
IoU (1)
OA (1)
Fl-score (1)
IoU (t)
OA (1)
Naive 47.47
31.12
99.17
51.07
34.29
98.95
Rec. One- Class
OCSVM 39
33.94
24.21
98.60-0.30
12.55
25.17-19.67
98.26
IF
110
44.14::1.18 28.34
98.68 #0.01
45.13=+1.30
29.15=t1009
98 08.09
SimpleNet
[1]
56.94.10.20 39.80.10.19
99.29.10.02
57.24
40.10.10.30
99.08.10.0
STEALNet
59
56.97:10.86
39.8410.84
99.06=00.01
58.65=10.88
41.50:0.88
98.83=10.02
UniAD 65
45.24413.8
29.30=2.94
98.65=0.21
46.99:
30.81+3.54
98.36-10229
DrepMMIL
94
71.77+00.38
55.97==0.46
99.55-00.01
72.02-+0.31
56.28
9.042+-0.01
MIL
ARNet
95
71.06=00.46
55.11=0..56
99. 99.53=0001
71.43:t0.51
55.56=+0.22
99.40==0.0
RTFM 96
60.30
43.16:10.27
98.92.10.02
61.91:10.21
44.83.10.23
9..70
Ours"
81.93
69.40
99.69=0.01
82.55:10.25
70.28:10.36
99.61=#0011
Ours
81.44+0..77
68.70+0.66
99.69=0.01
82.07=0.445
09.59
99.60=0001
"Video Swin Transformer backbone
100
Mamba backbone
|||
Table 7: Driver detection results on the synthetic artificial data. The best performance on each metric is highlighted in a bold text. (+) denotes the standard deviation for 3 runs.
Validation
Testing
Algorithm Fl-score (1)
IoU (f)
OA (f)
Fl-score (1)
IoU (1)
OA (t)
Naive 46.89
30.63
98.49
51.09
34.31
98.28
Rec. One-Clase Che-Class
OCSVM 39
16.27=1.20 1..96=1..76
97.77=0.111
33.05+1.229
19.80
97.42=t0.12
110
28.55#1.21 16.66::082
97.631:002
34.15
20.60
97.19
SimpleNet
51
34.57.10.54 20.90
97.56.10.04
41.18.10.35
25.93.10.28
97.94
STEALNet
59
56.40=1.788 39.78
98.33110.33
58.23
41.0910.92
98.11
UniAD
65
19.0
1..4
97.66:1011
1..4
1..60
97.57=1009
DeepMIL
X
20.18:223.67
13.38 16.45
71.33++20.53
18.75+21.34
14.04
71.08++20.48
MIL.
ARNet
95
48.98:15.30 32.59
98.82:10.06
44.75
28.98
98.53
RTFM
96
59.90=0.31
42.75=00.32
98.27+00.4
61.41
44.31
98.07=00.04
Ours*
70.40
54.08
98.9010.03
70.33
L0.84
98.74
Ourst
66.63-5..4
50.19-5.91
98.81+0.12
61.39:1..00 67.64 65.95
98.71=10.18
VVideo Swin Transformer backbone
100
Mamba backbone
|||
29

---

C
Ablation studies
In this section, we do ablation analyses on different aspects of the proposed model. All experiments are done on the validation set of synthetic CERRA reanalysis and evaluated with the F1-scores for drivers anomalies and extreme events detection.
C.1 Quantization layer
In T we study the performance of our model with different vector quantization algorithms. The first row Threshold (Tanh) represents a simple straight through estimator. For this quantization, we first map the input into scalar value followed by Tanh activation. Then we set positive values to be anomalies (q- = 1). In Random Quantization (RQ) [T4 the input is projected with a randomly initialized weights and then compared with randomly initialized codes. Vector Quantization (VQ) the standard quantizer which uses an Euclidean distance [[O]] or a cosine similarity (115). We further add an orthogonality loss for VQ similar to [16).. Finite Scalar Quantization (FSQ) maps
the imput into a bounded scalar value. The code is then assigned based on the rounded value in the discrete space. As seen, Lockup free quantization (LFQ) [102] performs the best. We speculate that LFQ does not need to learn the code vectors which simplifies the task of quantization.
Table 8: Ablation studies on the quantization layer. The metric is Fl-score on the driver/extreme detection.
Quantization Layer
Fl-score
Threshold (Tanh)
70.07 84.98
RQ (Fuclidean distance)
114
71.52/ / 87.23
VQ (Cosine similarity) LTS
78.83 88 88.98
VQ (Cosine similarity Orthogonality) [115
116
78.54 / 86.83
VQ (Euclidean distance) [001]
77.42 87.27
VQ (Euclidean distance Orthogonality) 101 16
79.26 / 88.28
FSQ T17
76.57
88.65
LFQ 102
82.78 / 92.445
C.2 Objective function for quantization
This part studies the loss function L(quantize) used in Eq. We denote the term 17: (Z 12 L(ommiit). H [E (sign( [T) L(div), and E H (sign(Z.)) as ((ent)- As seen from Table (cromist) is essential for training to prevent the input for the quantization layer from growing. While [ient) and (dtiv) (rows 4, 5, and 7) improve the results compared to the model with ((commiit) (second row).
Table Ablation studies on the loss function Liquantise) in Eq. The metric is Fl Fi-score on the anomaly/extreme detection.
Lient)
L (coommitt
L(dév)
F1-score (f)
X
X
00.00 00.00
XXX
X
81.86, 99.18
X
/
00.00 00.00
XX
V
X
81.87 19.48
V
82.26 92.30
V
X
00.00 00.00
V
V
82.78 92.45 V
C33
Objective function for extreme events prediction.
Due to class imbalanced, we add class weighting applied to the binary cross entropy loss ((extreme) to predict extremes. The weighting is based on the logarithm of the inverse square roots of the relative
30

---

class frequencies in the batch. The weighted loss function achieves better results as shown in Table
10
Table 10: Ablation studies on the weighted loss function L (extreme) in Eq. 33. on the driver/extreme detection.
The metric Fi-score
(extreme)
Fl-score (t)
unweighted (eezrreme)
81.14 90.91
weighted (eatreme)
82.78 / 92.45
C.4 Feature extractor (fo)
We evaluate three types of feature extractors, 3D CNN, Video Swin Transformer [100], and SSM Vision Mamba [TT]] with local scans. All models are trained from scratch. To build the 3D CNN model, we replaces the attention blocks in Swin Transformer with 3D convolutions and keep the overall architecture. For Mamba, we use local scans. Vision Mamba backbones replace the attention module with linear selective state space model which provides an efficient alternative for Video Swin Transformer [18H]224] From Table [1 we notice that Swin Transformer achieves better results compared to 3D CNN. The parameters are also less compared to 3D CNN. The results also indicate that using Mamba instead of Swin Transformer achieves similar or better results when less parameters used. In contrast to Swin Transformer, Mamba can be scaled to the global scale. However, have used Swin Transformer in all experiments to have a fair comparison with the baselines due to its commonality backbone.
Table 11: Ablation study on the backbone fo used for feature extraction. The metric is Fl-score the driver/extreme detection.
Backbone fo
Hidden dimension (K)
Parameters
Fl-score (1)
3D CNN
63k
57.15 / 91.21
Video Swin Transformer
8
19k
81.22 / 91.16
Mamba
8
15k
82.15 / 90.18
3D CNN
16
250k
70.93 / 93.75
Video Swin Transformer
16
62k
82.78 / 92.45
Mamba
16
56k
83.45 / 93.12
3D CNN
32
998k
84.95 / 93.43
Video Swin Transformer
32
230k
84.14 / 93.12
Mamba
32
214k
84.00 / 93.43
Table
shows that increasing the model parameters still does not show a sign of overfitting.
C.5
Lossy lossless driver detection
We trained the model without the quantization layer. In other words, we remove the driver detection step and trained the model to predict extreme droughts directly from the extracted features with one classification head. The reason behind this experiment is to check the information loss through the quantization/ddiver detection step. Table [2 shows the improvement in performance both the synthetic and real-world datasets. Adding the driver detection step identifies the related drivers to the events. However, the cost of slight decrease in accuracy on extreme events prediction.
Table 12: Ablation study on the driver detection step. The metric is F1-score extreme detection
where (AF1 = F1 (without quantization)
Fl, with guantization).
Dataset
A Fl--score (L)
Synthetic CERRA Reanalysis
+0.96%
ERA5-Land (EUR-11)
+1.93%
31

---

C.6 Weighting parameters in the main objective function
Table 13: Sensitivity studies the weighting parameters used in the main loss function in Eq. (6)
The metric is mean Fl-score on the driver/extreme detection for 3 random seeds.
(a)
(b)
(c)
(d)
(drrver))
Fl-score (f)
(commit))
Fl-score (1)
(eent)
Fl-score (t)
Addi))
Fl-score (1)
1
02.45 / 58.07
1.0
79.81 / 92.02
0.01
82.52 / 92.61
0.01
77.51
/91.71
10
03.34 75.73
1.5
81.43 / 99.96
0.1
82.78 / 92.45
0.1
82.78 / 92.45
100
82.78 / 92.45
2.0
82.27 / 92.11
1.0
82.72/
0.5
82.48 / 92.21
200
80.85 99.69
3.0
82.78 / 92.45
1000
79.94 86.44
4.0
82.76 / 91.89
We study the role of the weighting parameters in the main loss function in Eq. ( We can observe from Table 13 that A(driner) plays crucial role in anomaly detection. Having A(driwer)) small makes the model less constrained and reduces supervision on where extremes were reported the training data. Bigger values of A(driver) make the model more constrained to identify drivers near in overlap with the extremes and thus reduce the overall performance. We can also observe that the model is less sensitive to A(ent) compared to other parameters. We selected the final default weighting parameters the experiments based on the average performance on both driver and extreme events prediction.
C.7
Objective functions
Var 02
Var 03
Var 02
Var 03
Var 02
Var 03
Prediction
Prediction
Prediction
Ground Truth
Ground Truth
Ground Truth
L(rrtver)
[(extreme)
L(rrier))
((extreme)
x
L(ariver)
f(extreme) multi-heads
multi-heads
multi-heads
Figure 13: Supplementary to the ablation study Table
Without the loss (ddiver)) the detection of drivers/anomalies is not reliable since pixels regions and intervals where no extreme event occurred can be assigned to =g=1 (driver) as well. In case of C(estreme) multi-heads, we observe that anomalies are identified
small subset of variables
because the network omits some variables if there is correlation with other variables. Please see Fig. 13 In case of single head such flips occur less often, but they can occur. If the loss L(driver) is used, such flips cannot occur and the multi-head improves both F1-scores by large margin. When comparing rows and Table (a), there is slight decrease extreme prediction but large improvement in anomaly detection. Note that there is always trade-off between extreme and anomaly detection. The anomalies generate bottleneck of information. When more information goes through the bottleneck the better the extreme prediction gets. Without any anomaly detection, extreme prediction is best as shown in Table 12 but the increase in score only moderate. This is also visible in rows 2 and in Table 2 (b). Cross-attention improves extreme prediction, but it hurts the detection anomalies since the information propagated between the variables.
D Results on the real-world reanalysis data
D.1 Quantitative results
is difficult to quantify the quality the predicted drivers and anomalies on real-world data. Therefore, we hypothesize that the model can predict extreme agricultural droughts from the identified
32

---

drivers and anomalies, only if those drivers and anomalies are causally correlated with the extreme events. To verify this, we report in Table [4 prediction accuracy on extreme agricultural droughts. The results show that using the identify drivers as input, the model can predict extreme agricultural droughts across different regions and datasets.
Table 14: Quantitative results for extreme droughts detection on real-world data. (+) denotes the standard deviation for runs.
Validation
Testing
Dataset
Region
F1-score (t) IoU (t)
OA (1)
F1-score (1)
IoU (1)
OA (1)
CERRA
Europe
22.31:10.74
12.56:10.47
90.45:1.005
28.63:1 1.35::1445
16.71:10.98
89.13 1.44
ERAS-Land Europe
31.87:10.39
18.96:10.28
95.45:10.25
21.52:10.86
12.06:10.54
95.84-1.221
Affica
22,49:10.42
12.67::0.27
85,59=1.48
18.53+0.48
10.21 78.63=1.43
North America
27.39:10.63
15.87:10.42
93.84=0.13
31.7411..48
18.8610.34 89.95=0.42
South America
29.30::1.22
17.17:11004
89.31=0.33
28.96:1.67
16.94::1113 83,99=0.23
Central Asia
20.99:+0.43
11.73++0.27
95.38+0.04
25.01++0.11
14.29++0.07
94.51+0.04
East Asia
18.82++0.2
10,39+4044 93.10++.42 25.35++0.00
12.58+0.12 93.73+0038
The performance depends on the type and ratio of extremes, spatio-temporal resolution, the quality and consistency between the remote sensing and the reanalysis data. F1 scores substantially increase when the threshold on VHI is increased (see Table. [6]in Sec. (D3) Note that it is not required to predict all extremes in order to learn specific relations from the predicted events.
D.2 Robustness
To further check the model robustness, we train the model on the same EUR-11 region with different combinations of physical variables. We anticipate that if variable is relevant to extremes over specific region i.e., Europe it should appear in all identified sets of variables i.e., we expect soil moisture (swvll) and evaporation (e) to be always presented as explanatory variables for extremes over Europe. Our experimental results in Table 15 confirms this assumption.
Table 15: Quantitative results from ERA5-Land EUR-11 data for different combination of physical variables. The metric is Fl-score on the extreme droughts detection for the validation/test sets.
Input variables
Selected variables
F1-score(t)
(12m, fal, e, tp, stll, swvll) (d2m, 12m, sp, e, tp, stI)
(e, swvll]
31.87 / 21.52
(e)
32.72/ 23.44
[d2m, fal, sp, e, skt, swvll) (12m, sp, e, skt, stll, swvll) [12m, fal, sp, skt, stll, swvll}
(e, swvll]
32.92 / 22.41
(e, swvll]
/
(swvl1)
24.34 / 16.05
(t2m, e, tp, swvll)
(c. swvll]
30.38 / 21.59
D.3 Scientific validity
Surely, soil temperature is a key factor the drought processing and soil moisture-temperature feedback [225. State variable of the land surface such as albedo (fal/al) and soil temperature (stl 1/oot) should be very related to reflectance on the ground and consequently to VHI from remote sensing. However, our approach indicates that these variables are not informative enough for the model to identify drivers and predict extremes. To investigate this issue, we conducted 4 more experiments on both CERRA and ERAS- where we trained models that take only one variable al/fal or stl as input and predict the extreme events directly without the driver/anomaly detection step (similar very low. In the next experiment,
to Sec. C.S). In all of these experiments, the F1-score increased the threshold for VHI and trained new models to predict extremes directly. The results for the validation set are shown below in Table [6
The first potential reason to consider is that some land surface variables might deviate from the reality. ERA5-Land does not use data assimilation directly. The evolution and the simulated land fields
33

---

Table 16: The relation between the definition of extremes from VHI and the model prediction. The metric is Fl-score (1) on the extreme droughts detection for the validation sets.
Input variables
Dataset
Domain
VHI < 26
VHI < 40
VHI < <0
[stl]] [t2m, fal, e, ip, stll, swvll) ERA5-Land
ERAS-Land
EUR-11
05.67
31.53 46.72
58.36
EUR-11
33.80
68.71
are controlled by the ERA5 atmospheric forcing. Another reason might be that when training only on extremes (VHI 26), there are not enough samples learn the relations. Note that VHI is combination of both TCI and VCI. Most extremes (VHI < 26) might result from a deficiency in both stl/t2m and vsw. This might also explain why stl and albedo cannot be that informative to predict very extreme events. Last row in Table 15 shows the result when we discard albedo and soil temperature as input variables from ERA5-Land.
Moreover, state variables of the hydrological cycle in ERAS-land like volumetric soil water variable (swvll/vsw) has biases [66]. One solution to improve the validity of investigation is use satellite observations for the top layer [17]] However, the experiments showed that the model relates soil moisture anomalies with the extremes in VHI and provides reasonable predictions.
D.4 Spatial Resolution
Table 17: extreme droughts detection for the validation sets.
The impact of spatial resolution on the model prediction. The metric is F1-score the
Dataset
Domain
Spatial resolution
Fl-score(t))
ER Ladd ERA5-Land
EUR-11
0.1º
31.87
EUR-11
0.2º
30.09
E
Baselines
We evaluate our approach with an interpretable forecasting approach using integrated gradients
109 and baselines from 3 main related categories anomaly detection; one-class unsupervised
39 110 51, reconstruction-baee [59] 65]. and multiple instance learning (94. Note that these task we addressed this paper. We modified and trained all baselines from scratch. For this, we relied on the officially released codes and started from the default hyperparameters.
baselines are not directly applicable
Integrated Gradients [109] is an axiomatic attribution method which can be used as post-hoc method to explain the model prediction. We trained two models that predict extreme events directly from the input variables and then we applied a post-hoe integrated gradients from Pytorch Captum [28.. Both models use the same Swin Transformer backbone our model but without the anomaly detection step. For Integrated Gradients II, we added cross attention. For these baselines, we compute the gradient only with respect to predicted extremes and computed a different threshold for each variable separately.
Isolated Forest (IF) [110] an ensemble of random trees that isolate input instances by randomly selecting abounded splits for features. A shorter averaged path for a recursive partitioning implies an anomalous input. We empirically set the number of estimators 100.
OCSVM [39]
a one-class support vector machine solved using stochastic gradient descent. We radial basis kernel with 100 components.
For IF and OCSVM, we extracted multivariate features as the distances between the input variables and then train model on each input variable separately. We sampled 400k normal data points (locations without extreme flags) for training and defined the expected ratio of anomalous to be roughly equivalent the ratio of extreme events in the data.
34

---

SimpleNet [51] is categorized as an embedding basedd based one-class algorithm for anomaly detection. In SimpleNet, feature extractor first used extract local features from normal data followed by a feature adaptor. Then an anomaly feature generator induces anomalous features by adding Gaussian noise in the feature space. Finally, a discriminator is trained to distinguish between the normal and anomalous generated features. During inference, the feature generator is discarded and the trained discriminator is expected to separate anomalous from normal input features. During inference, we compute the median anomaly score for both normal pixels and pixels with extreme flags. Then, we set a threshold for anomalous pixels based on the average of the two former median anomaly scores. We trained SimpleNet with our pretrained model as feature extractor and set the feature dimension for feature adaptor to 512. The extracted features are scaled by 10 and the anomalous feature generator uses a Gaussian noise e ~ -N(0, o 1.5). The discriminator composes of two linear layers with a hidden dimension of 96. We set th and th to 1.0 and trained with Adam W optimizer for 48 epochs with batch size of 2 and learning rate of x 10-1 with 1 x 10-5 weight decay.
STEALNet [59] or synthetic temporal anomaly guided end-to-end video anomaly detection. This is reconstruction-base algorithm. STEALNet was trained to maximize the reconstruction loss for locations with extreme flags and to minimize the reconstruction loss otherwise. We set the dimensions for the auto-encoder 96, 128, and 256. We trained with a batch size of 8 for 100 epochs using Adam optimizer with x 10 -3 weight decay and a learning rate of x 10 10-4.
For STEALNet and UniAD, we compute during inference the mean reconstruction loss for both normal pixels and pixels with extreme flags. Then, we set threshold for anomalous pixels based on the average of these two values.
(4] is multiple instance learning for anomaly detection in surveillance videos. In MIL,, each video is represented as a bag of snippets (instances). We define positive instances as pixels with extreme event flags where the exact information (which variable is anomalous) within the positive instances is unknown. In the original implementation. the ranking is enforced only on the top instance with the highest anomaly score in each bag. We modified the ranking loss top-k and K 100.
MIL
or anomaly regression net is another MIL-based approach. The ranking loss is based on top-k binary cross entropy. In addition, there is center loss to reduce the intra-class discriminative features of normal instance. We set a = 400 and Acenter = 20.
ARNet 955
RTFM 96 or robust temporal feature magnitude learning is MIL--based algorithm which learns to
distinguish between the normal and anomalous scores by selecting the top-k snippets with the highest feature magnitudes. We modified the multi-scale temporal network (MTN) to capture the local and global spatial dependencies between pixels. We set the dimension for MTN to 32, K 100, the margin separate features to x 10 and 1 x 10-4
We use the same Video Swin Transformer backbone as our model to train DeepMIL, ARNet, and RTFM. During training, we noticed that the ranking loss becomes biases toward one variable when the ranking is computed among all variables, so we computed loss for each variable and then average the losses. RTFM only worked when we added cross attention between the variables. We suppose this to be related to the feature magnitude learning. We trained with batch size of 2 using Adam optimizer and a learning rate of 6 x 10-4 for 100 epochs with weight decay of 1 x 10 10-3 We also added an instance dropout of 0.5.
F Computational efficiency
In Table 18 we report the inference time with a fixed input of 6 variables, days and 200 x 200 spatial resolution. STEALNet is based on 3D CNN auto-encoder without self-attentions which explains its efficiency but on the cost of accuracy. In the last row, we show the estimated time when we discard the classification head to detect extreme events and only use the model for anomaly detection.
The training on the real-world data for EUR-11 1 took about ~ 21 hours with a Swin Transformer model, K 16, and NVIDIA A100 GPUs Table 19 gives rough estimation for training on the synthetic CERRA for 1 epoch. SimpleNet was trained with pretrained backbone. The training time includes some postprocessing to compute metrics on the training set. The time might also differ depending on the I/O during training and the number of available workers.
35

---

Table 18: Inference time in seconds for our model and other DL baselines.
Algorithm
GPU (sec)
Parameters (M)
SimpleNet
0.156 +0.003
0.203
STEALNet
0.003 +:0.00
6.005
UniAD
3.733 +00011
3.674
DeepMIL
0.193 #0.008
0.285
ARNet
0.191 #0.003
0.285
RTFM
0.257 #0.001
0.319
Ours
0.132 #0.000
0,479
Ours
0.122 +0.000
0.145
NVIDIA GeForce RTX 3090 GPU
2 Our model is used only for driver/anomaly detection
Table 19: Training time the synthetic CERRA for I epoch.
Algorithm
Time (min)
GPU
SimpleNet
~ 2
A100
STEALNet
~
A100
UniAD
11
4 x A100
DeepMIL
13
A40
ARNet
13
A40
RTFM
20
A40
Ours
~ 8
A40
G Implementation details
The training done mainly clusters with NVIDIA A100 80GB and NVIDIA A40 48GB GPUs. In the following, we highlight the main technical implementations and hyperparameters for training:
Synthetic data For the synthetic data we set the embedding dimension K = 16. We use one layer Video Swin Transformer with (depth=[2, 1], heads=[2, 2), window size=[(2, 4, 4], [6, 1, 1]]).
For the quantization layer, we use two sequential 3D CNN layers on each variable with: {[kernel=33, 3, 3), stride=(1, 1), padding=(1, 1)]) and followed by a shared linear layer.
The classifier 93: consists of the following layers: ([3D CNN, kernel=(3, 3, 3), stride=(3, 1, 1), padding=(0,, 1, 1)]. [3D CNN, kernel=(2, 3, 3), stride=(2, 1, 1), padding=(O, 1, 1)]] We set A(en)) A 0.1, A(onomaty) = 100, and A(comeitt) = 3. The models were trained with Adam optimizer [29 for 100 epochs with batch size of 4. We use linear warm up of 2 epochs and cosine decay with an initial learning rate of 2 x 10-3 and a weight decay of 3 x 10-3
Reanalysis data We set the embedding dimension K 24 for CERRA and CAS-11 datasets and = 16 for the rest of ERAS-Land datasets. We use one layer Video Swin Transformer with [depth=[2, 1]. heads=[2, 2]. window size=1[2, 4, 4], [8, 1. 1]]]. The quantization layer is similar to the one for the synthetic data. The classifier So consists of the layers each has [[D CNN, kernel=(2. 3, 3), stride=(2, 1, 1), padding=(0.
100 by defaults and (commir) = 1.0 for CAS-11. For CERRA, we set A(ent) = A = 0.01. Due to the high resolution the reanalysis data, use gradient checkpoint during training. For CERRA reanalysis, we also cut the boundaries between low and high latitudes focusing on the central region.
1)). We set Aient) A(div) 0.1, and (anomally)
This results in final grid with 512x832 cells.
To handle missing data and temporal gaps in the input reanalysis data, we first normalize the data using the pre-computed statistics and then replace the invalid pixels with zero values.
36

---

H Reanalysis data
The raw reanalysis data were provided by the Climate Data Store (CDS) 100 106 Technical details regarding the reanalysis datasets are provided in Tables 20 and 21 CORDEX regions [08] used in this study are shown in Fig. 14 For all regions on ERAS-Land, we used the following variables: ("t2m", "fal". "e", "tp". "swvll", "stl1"). For CERRA, did the experiments with: |"t2m", "al", "tec", "tp", "vsw", "r2").
Table 20: Datasets used in the experiments on real-world data. CORDEX domains are defined based [08].
Dataset
Region
CORDEX
Resolution
Train
Val
Test
CERRA
Europe
1069x 1069
1984-2015
2016-2018
2019-2021
ERA5-Land
Europe
EUR- 11
412x424
1981-2017
2018-2020
2021-2024
ERA5-LLand
Africa
AFR-11
804x776
1981-2017
2018-2020
2021-2024
ERA.5-Land
North America
NAM-11
520x620
1981-2017
2018-2020
2021-2024
ERAS-Land
South America
SAM-11
668x584
1981-2017
2018-2020
2021-2024
ERA5-Land
Central Asia
CAS-11
400x612
1981-2017
2018-2020
2021-2024
ERA5-Land
East Asia
EAS-11
668x812
1981-2017
2018-2020
2021-2024
NAM-11
EUR-11
CAS-11
CERRA
SAM-11
AFR-11
EAS-11
Figure 14: The definition of the domains used in the study. ERAS-Land reanalysis mapped the CORDEX domains 108 CERRA has its own domain definition [106].
Agricultural drought definition and remote sensing data
L1 Satellite-derived agricultural drought
is generally challenging to define what exactly constitutes an extreme. Extremes can be categorized from the perspective of their impacts. For instance, extreme drought can be categorized into 4 types based on their impacts (130] [T]: meteorological climatological drought, agricultural drought [I3I], hydrological drought, and socioeconomic drought [132]. Meteorological drought is mainly related to the dryness and can be defined based on deficiency temperature or precipitation. Agricultural drought measures the impact of stress on vegetation and usually defined as soil water deficits. It is also widely conceived that drought originates
progresses from meteorological into agricultural drought 6 17 Hydrological drought is related to the water storage. While socioecononic drought can be measured based on supply and demand related weather and deficit in water supply. this paper, we interested in extreme agricultural droughts.
37

---

Table 21: Details regarding the processed variables from ERA5-Land [07 and CERRA 106
reanalysis.
Dataset
Variable name
Long name
Unit
Height
CERRA
al hec
albedo
surface
high cloud cover
above 5000m
lcc
low cloud cover
%
surface-2500m
mcc
medium cloud cover
%
2500m-5000m
liqusm
liquid volumetric moisture
+/m"
top layer of soil
msl
mean sea level pressure
Pa
surface
12
metre relative humidity
%
2m
si10
10 metre wind speed
m/s
10m
skt
skin temperature soil temperature surface pressure
surface
sot
top layer of soil
Pa
surface
surface roughness
surface
12m
metre lemperature
2m
lcc
total Cloud Cover
above ground
tciwv
total column integrated water vapour
kg/m"
surface
tp
total Precipitation
kg/m²
surface
vsw
volumetric soil moisture
m n//m²
top layer of soil
wdirl0
10 metre wind direction
10m
ERAS-Land d2m
2m dewpoint temperature
K
2m
12m
2m temperature forecast albedo
2m
fal
surface
skt
skin temperature soil temperature surface pressure total evaporation
surface
stll
soil layer (0 - cm)
Pa
surface
of water
above ground
equivalent
tp
total precipitation
m m m//m²
surface
swvl1
volumetric soil water
soil layer (0 7 cm)
NDVI
a
VHI
Extreme Drought V
TCI
Figure 15: An overview of the extreme agricultural droughts definition from remote sensing.
Worldwide satellite observations allow for almost real-time monitoring of drought and vegetation con- ditions. In practice, vegetation states can be estimated from land surface reflectances acquired from satellites. As result, the reflectances on the ground can be employed as agricultural drought indica- tors and as proxies for vegetation health. To define extreme agricultural drought events, we processed aatellte-basse vegetation health dataset [104] from the National Oceanic and Atmospheric Adminis-
38

---

php [last access: 22 May 2024)]). This dataset consists of long-term remote sensing data acquired from system of NOAA satellites: the Advanced Bery-HighRResoutiin
tration (NOAA) /tttps://ww... star nesdis noaa.
Radiometer (AVHRR) which starts from 1981 until 2012 and the new system the Visible Infrared Imaging Radiometer Suite (VIRS) from 2013 onwards. The dataset has global coverage with ~ 0.05 (~4km) spatial resolution. The normalized difference vegetation index (NDVI) [133 and brightness temperature (BT) [134] are the two key products of the dataset. The BT is infrared (IR)-based calibrated spectrum radiation. While the NDVI is combination of the near-infrared (NIR) and red (R) bands. To remove the effects of clouds, atmospheric disturbance, and other error sources, the data were aggregated temporally into a smoothed product on a weekly basis. The weekly temporal coverage is needed for outliers and discontinuities removal and is suitable to study the phenological phases of vegetation and conse- quently to define agricultural drought [004 135]. Based on the long-term upper and lower bounds of the ecosystem (maximum and minimum values of the NDVI and BT), agrieultural drought indicators such as vegetation condition index (VCI), thermal condition index (TCI), and vegetation health index (VHI) can be derived [134 [36]]. VHI is combination of VCI and TCI (Fig. [5] and it fluctuates between 0 (unfavourable condition) and 100 (favourable condition). Values outside the range are clipped. Based on this definition of vegetation health, extreme agricultural drought can be defined when VHI < 26. Please note that vegetation stress detected by VHI could not be necessarily caused by a drought event i.e., a change in the land cover can change the signal as well [5 137. Thus, VHI should be interpreted carefully.
1.2 Pre-proceed pre-processing of the remote sensing data
The remote sensing dataset is provided on the Plate Carrée projection (geographic latitude and longitude). The target agricultural drought data and reanalysis data have to be aligned in the same coordinate systems and over the same regions. To realize this, we mapped the remote sensing data onto the Lambert conformal conical grid for CERRA and onto the rotated coordinate systems over the different CORDEX domains for ERA5-Land. For the mapping, we use the first-order conservative mapping using the software from Zhuang ct. al [18]. To calculate the spatial averaged, we excluded coastal lines, invalid, and water body pixels. Furthermore, we combined the dataset with masks obtained from the quality assurance metadata for pixels over no vegetation and very cold areas. As mentioned in Sec. [.] temporal decomposition was conducted to remove some discontinuity and aggregate the data into a weekly product. However, some pixels will still be empty. To tackle this issue, we first checked if the pixel was covered by another satellite and averaged the measurements of the satellites. If it was not the case, we flagged the pixel as invalid and discard it from the training and evaluation. This remote sensing dataset serves as a reference of extreme agricultural drought events to train and evaluate the performance of the model. Table 22 shows the ratio of extreme events in the datasets. Please note that there is no ground truth for drivers or anomalics in our real-world dataset. We only report the ratio of extreme agricultural drought events, which can be detected using remote sensing data.
Table 22: Details regarding the ratio of extreme events in the pre- rre-prccessed NOAA remote sensing data.
Region
Domain
Extremes (%)
Val
Test
Europe
CERRA
4.34
5.32
Europe
11
3.20
2.86
Africa
AFR-11
6.41
6.87
North America
NAM-11
3.68
6.61
South America
SAM-11
5.16
6.53
Central Asia
CAS-11
3.60
4.38
East Asia
EAS-11
3.16
3.05
39

---

J
Additional results
100
total evaporation
forecast albedo
soil temperature
% of weeks with anomalous events
50
volumetric water
temperature
total precipitation
Figure 16: The averaged spatial distribution drivers and anomalies related Portugal in Europe. For this experiment, we use prediction on EUR-11 from ERAS-Land and select frames (times) within the period 2018-2024 where there were extreme drought of least 25% of the pixels in the Portugal. Then normalize the identified drivers and anomalies by the total number of frames to obtain the final map. As can seen drivers are spatially centered around where extremes were reported.
100
total. evaporation
forecast albedo
soil temperature
% of weeks with anomalous events
50
volumetric soil water
2m temperature
total precipitation
Figure 17: The averaged spatial distribution of drivers and anomalies related to a specific place in Europe (North Rhine- Westpa-aslaaalaa... For this experiment, use prediction EUR-11 from ERA5- Land and select frames (times) within the period least 25% of the pixels the North Rhine- Westh-aaas, Then normalize the identified drivers and anomalies by the total number of frames to obtain the final map. As can be seen drivers are spatially centered around where extremes were reported.
where there were extreme drought of at
40

---

EUR-11, year 2020, week 035
Observedd droughtts
A
bb
drouughts
513
15mm
Th
FAXIL
12m
Figure 18: Qualitative results on ERAS-Land for Europe (EUR-11). Shown are the identified drivers and anomalies for each variable along with the prediction extreme agricultural droughts on top left.
Figure 19: Temporal evolution drivers and anomalies related to the extremes ERA5-Land for Europe (EUR-11). For this experiments, we select pixels with extreme events during summer (weeks 25-38) for years 2018-2023 and compute the average distribution drivers and anomalies with time. The red line at ötg indicates the beginning of the extreme droughts. Zecore in the underneath curve represents the deviation from the mean computed from the ERA5-Land climatology.
41

---

AFR-11, year=2023, week=048
Observed extreme droughts
Predicted extreme droughts
Aks
E4M
-M1c
-M5
+M4
-A2>
Figure 20: Qualitative results ERA5-LLnn for Africa (AFR-11) Shown the identified drivers and anomalies for each variable along with the prediction extreme agricultural droughts on the top left.
APRLL (25-27% 381P-202
100
Figure 21: Temporal evolution of drivers and anomalies related to the extremes in ERA5-Land for Africa (AFR-11). For this experiments, we select pixels with extreme events during summer (weeks 25-38) for the years 2019-2023 and compute the average distribution drivers and anomalies with time. The red line at 5tg indicates the beginning of the extreme droughts. Zscore in the underneath curve represents the deviation from the mean computed from the ERA5-Land climatology.
42

---

SAM-11, year= week=039
Observed extrome droughts
Predicted extreme droughts
sevll
12m
Figure 22: Qualitative results ERA5-Land for South America (SAM-11). Shown are the identified drivers and anomalies for each variable along with the prediction of extreme agricultural droughts on the top left.
SAM32 Summer 2211 2018 2033
Figure 23: Temporal evolution of drivers and anomalies related the extremes in ERA5-Land for South America (SAM-11). For this experiments, we select pixels with extreme events during summer (weeks 25-38) for the years 2018-2023 and compute the average distribution of drivers and anomalies with time. The red line at öt, indicates the beginning of the extreme droughts. Zecore in the underneath curve represents the deviation from the mean computed from the ERA5-Land climatology.
43

---

NAM-11, year=2017, week=024
Observed exteeme droughts
Predicted extreme drou
12m
12m
12
12m
13
#13
1V
e1
e:3
@11
5)
ett
12m
1p
SAAII
12/
Swirl
12m
Sw/11
12Y
18
swvl
E2m1
29
Figure 24: Qualitative results ERA5-Land for North America (NAM-11). Shown are the identified drivers and anomalies for each variable along with the prediction of extreme agricultural droughts on the top left.
NAM-11-Summe mww. 22- 2313-223-2
Figure 25; Temporal evolution of drivers and anomalies related to the extremes in ERA5-Land for North America (NAM-11). For this experiments, we select pixels with extreme events during summer (weeks 25-38) for the years 2018-2023 and compute the average distribution of drivers and anomalies with time. The red line at 8t, indicates the beginning of the extreme droughts. Z Zecore in the underneath curve represents the deviation from the mean computed from the ERA5-Land climatology.
44

---

EAS-11, year=2019, week=016
Observed extreme droughts
Predicted extreme drou
A/a
-212
sovrl
17m
0V/1
13m
12m
-Als
e13
1V
e1
#1
5
d11
taxt
10mm
tp
12m
tarl
PW/1
12m
Figure 26: Qualitative results on ERA5-Land for East Asia (EAS-11). Shown are the identified drivers and anomalies for each variable along with the prediction of extreme agricultural droughts on the top left.
weed (22-22) 3850-3322
Figure 27: Temporal evolution of drivers and anomalies related to the extremes in ERAS-Land for East Asia (EAS-11), For this experiments, we select pixels with extreme events during summer (weeks 25-38) for the years 2018-2022 and compute the average distribution of drivers and anomalies with time. The red line at öt, indicates the beginning of the extreme droughts. Zecore in the underneath curve represents the deviation from the mean computed from the ERA5-Land climatology.
45

---

CAS-11, year-2021, week=031
Oeserved extrems droughtss
Predicted OAt satroma Srought
IwG
-2
Figure 28: Qualitative results on ERAS-Land for Central Asia (CAS-1I). Shown are the identified drivers and anomalies for each variable along with the prediction of extreme agricultural droughts on the top left.
0012
Figure 29: Temporal evolution of drivers and anomalies related to the extremes in ERAS-Land for Central Asia (CAS-11). For this experiments, select pixels with extreme events during summer (weeks 25-38) for the years 2018-2023 nd compute the average distribution of drivers and anomalies with time. The red line 84, indicates the beginning of the extreme droughts. Zecore in the underneath curve represents the deviation from the mean computed from the ERAS-Land climatology.
46

---

CERRA, 2020- week 029
Diserveed extreme droughts
Preskcted oxtrome droughts
Mt
Figure 30: Qualitative results on CERRA reanalysis for Europe. Shown are the identified drivers and anomalies for each variable along with the prediction of extreme agricultural droughts on the top left.
Figure Temporal evolution drivers and anomalies related the extremes CERRA reanalysis for Europe. For this experiments, we select pixels with extreme events during summer (weeks 25-38) for the years 2016-2020 and compute the average distribution of drivers and anomalies with time. The red line at ôto indicates the beginning of the extreme droughts. Zscore in the underneath curve represents the deviation from the mean computed from the CERRA climatology.
K
Code and data availability
The source code reproduce the results available on GitHub https://github. com/ HakamShams/ IDEE, The source code for the synthetic data generation is also available on The
GitHub at https: /tttps://githann www./oom/aaaaaeee pre-processed data used in this study are available at http://///...oooo///000066/////2
139
Broader impacts
There generally direct negative social impacts for conducting climate science researches. However, anomaly detection algorithms in general could be adapted for video surveillance and might infringe privacy considerations. Although this is the risk of developing anomaly detection algorithms.
47

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: See Sec.
5
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: See Sec. d
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
48

---

Justification:
The paper dose not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: See implementation details in Sec. 5 Appendix Sec.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
49

---

Answer:
[Yes]
Justification:
The code and datasets are publicly available. See Sec. K
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https guwdes/ddeeeeee Cooeeeoodeesssseeeeee 7/nips.com /ttc: ssiniiiiccy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification:
See implementation details in Sec. 5 Appendix Sec. G
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We report the mean and standard deviation of the results for 3 different random seed runs. See Tables 14, and [8.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
50

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
See Appendix Sec. and Sec.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
The research conducted in the paper conforms with the NeurIPS Code of Ethics.
Justification:
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: See Sec, and Appendix Sec.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
51

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification: We think the paper does not pose such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
We cite the original publications for the raw data and provide URLs when applicable.
Justification:
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the Datasets.papersstt paperswi thcode. cww/ datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
52

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: See Sec. and Appendix Sec.
A
H and
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
53


Reviews:
[{"review_id": "fic6pgmMaq", "reviewer": "Reviewer_fRrQ", "summary": "This paper proposes an approach to learning the spatio-temporal relationships between events with spatial differences and temporal delays. Specifically, they propose a method that identifies spatial-temporal anomalies in multivariate climate data that are correlated with extremes. The authors conduct experiments on both synthetic data and climate reanalysis data.", "questions": "1. Anamly detection is a classification problem with severe class imbalance problems, how did the authors tackle this problem? What is the ratio of extreme events to ordinary events?\n2. What is the difference between anomaly and extreme in this paper? Could you provide some examples to illustrate this?\n3. When I read spatio-temporal relations, I thought this paper would build spatio-temporal graphs to describe the relations. Have you considered using graphs to tackle this problem?", "limitations": "The authors adequately addressed the limitations.", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The problem of anomaly detection and learning their relations is crucial. \n2. The summary of the relevant literature is relatively complete.\n3. The authors conduct experiments on both synthetic data and real-world data.", "weaknesses": "1. The motivation of model design is not clear. For example, why do you need to detect the anomaly and then classify the extreme events instead of detecting extreme events directly? Such a designed pipeline will lead to more accumulated errors. \n2. The writing can be improved. For example, there are typos, such as 'MIL Is a weakly ...'. And the difference between anomaly and extreme in this paper is not clear. \n3. The title is somehow misleading, actually what the paper does is more about extreme event prediction instead of learning spatial-temporal relations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your time and for reviewing our work. We are glad that you found the task and the problem we address in this work important. In the following, we answer your questions.\n\n> Difference between anomaly and extreme in this paper is not clear. What is the difference between anomaly and extreme in this paper? Could you provide some examples to illustrate this?\n\nWe will define the terms more precisely in the revision. Please see the author rebuttal and the global response above for clarification.\n\n> The title is somehow misleading, actually what the paper does is more about extreme event prediction instead of learning spatial-temporal relations. The motivation of model design is not clear. For example, why do you need to detect the anomaly and then classify the extreme events instead of detecting extreme events directly? Such a designed pipeline will lead to more accumulated errors.\n\nThere seems to be some misunderstanding about the objective of this work, probably caused by the terms \"anomaly\" and \"extreme\". Our work is not about extreme event prediction in the first place but about identifying the anomalous drivers of these extreme events like droughts. Note that droughts can be observed, but it is unclear what are the anomalies in the atmospheric or hydrological state variables that are spatio-temporally connected with a drought. Note that the anomalies in the atmospheric or hydrological state variables can occur earlier than the drought and at a different spatial location. \n\nWe aim to identify these anomalies that are spatio-temporally connected with an observed extreme event. Since we only observe droughts and are only interested in atmospheric or hydrological state variables that are spatio-temporally connected with droughts, we design a network (Figure 1) that spatio-temporally identifies anomalies in the atmospheric or hydrological state variables and predicts from these the droughts in an end-to-end fashion. In other words, we force the network to reduce the input variables to spatio-temporal anomalies (quantization) that are sufficient to predict the drought. Due to the quantization, the prediction of a drought is lower compared to predicting droughts without identifying anomalies in the input as reported in lines 314-319 and in Table 12 (Appendix C.5, page 29). The drop of the F1-score on drought detection, however, is relatively small. Nevertheless, we are interested in detecting the driving anomalies that are spatio-temporally connected to a drought and not the drought itself. We will clarify this and revise the title if necessary.\n\n> Anomaly detection is a classification problem with severe class imbalance problems, how did the authors tackle this problem?\n\nTo solve the class imbalance issue, we utilized a weighted binary cross entropy for $\\mathcal{L}_{(extreme)}$. Please see Appendix C.3 (page 28) and Table 10 (page 29). We will make this clearer in the method section in the revised version.\n\n> What is the ratio of extreme events to ordinary events?\n\nThe ratios of extremes are reported in Tables 3-5 (second last column, pages 20-21) in the Appendix. For the synthetic CERRA reanalysis (Table 3, page 20), the ratio is $1.16\\%$ while the ratio of the correlated anomalies with these extremes is $1.69\\%$ and the ratio of the random uncorrelated anomalies with extremes is $1.32\\%$. For convenience, we summarize the ratios below:\n\n| Dataset | extremes (\\%) | correlated anomalies (\\%)   |  random anomalies (\\%)  |\n|-|-|-|-|\n| Synthetic CERRA | 1.16 | 1.69 | 1.32 |\n| Synthetic NOAA | 0.79 | 1.02 | 1.76 |\n| Synthetic artificial | 1.24 | 1.81 | 2.93 |\n\nPlease note that there is no ground truth for anomalies in the real-world dataset. We only report the ratio of extreme events, which can be detected using remote sensing data:\n\n| Dataset | Region | extremes (\\%) ||\n|-|-|-|-|\n| | | Val | Test |\n| CERRA | Europe | 4.34 | 5.32 |\n| ERA5-Land | Europe  | 3.20 | 2.86 |\n| ERA5-Land | Africa | 6.41 | 6.87 |\n| ERA5-Land | North America | 3.68 | 6.61 |\n| ERA5-Land | South America | 5.16 | 6.53 |\n| ERA5-Land | Central Asia | 3.60 | 4.38 |\n| ERA5-Land | East Asia | 3.16 | 3.05 |\n\n> When I read spatio-temporal relations, I thought this paper would build spatio-temporal graphs to describe the relations. Have you considered using graphs to tackle this problem?\n\nNote that our real-world data like the CERRA dataset has a spatial resolution of $1069\\times1069$ (Table 17 in Appendix, page 33) and we consider 8 time steps. A spatio-temporal graph would consists of $1069\\times1069\\times8=9,142,088$ nodes, which would be computationally very expensive. Even for visualizing the results shown in Figures 16, 18, 20, 22, 24, 26 and 28 (pages 37-43), a spatio-temporal graph would not be suitable.     \n\n> The writing can be improved. For example, there are typos, such as 'MIL Is a weakly ...'.\n\nThank you for the careful reading of our paper. We will fix this typo and check the paper for any other typos.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KKc4JYY080", "reviewer": "Reviewer_5qbR", "summary": "The paper proposes a novel approach to identifying spatio-temporal anomalies correlated to extremes such as drought. Using neural network, to predict extreme events by learning spatio-temporal binary masks of anomalies identified in climate data. The network is trained end-to-end to predict both anomalies and extremes from physical input variables, focusing on the spatio-temporal relations between them.", "questions": "Have you tried the method on other extreme events other than droughts? And what if there is discontinuity in terms of temporal data?", "limitations": "If possible, please add the reasoning behind feature represetations and extremes.", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "Introduces a new method for identifying spatio-temporal anomalies that are correlated with extreme events.", "weaknesses": "The model is dependent on temporal resolution, which might not be well documented in all parts of the world.\nThe method only shows results on droughts.\nBinary masks tend to oversimplify real-world events.\nThe method seems to bluntly connect anomalies with extremes without specific theoretical reasoning.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer’s feedback for this work and that the reviewer recognized the novelty of this work. We respond to the questions below.\n\n> The method only shows results on droughts. Have you tried the method on other extreme events other than droughts?\n\nWe tested the algorithm on 9 types of extreme events; real-world agricultural droughts (Table 14 in Appendix, page 30), synthetic CERRA extreme events (Tables 1 and 3, pages 6 and 20), 5 variations of the synthetic CERRA extreme events (please see Fig. 3 where we changed the coupling between the climate variables and consequently changed the type of the events), synthetic NOAA extreme events (Tables 4 and 6, pages 20 and 27), and synthetic artificial extreme events (Tables 5 and 7, pages 21 and 27). The synthetic datasets show that the approach is not limited to droughts and can be applied to other extremes. The main difficulty is that suitable datasets for extreme events are rare, i.e., the dataset should have a high resolution and a long-term and large-scale coverage. Building a dataset from existing sources is thus a major effort (see Section H and I in the Appendix, pages 32-35). In the future, we aim to test the model on more types of events like flood, but this requires to prepare the data first.  \n\n> The model is dependent on temporal resolution, which might not be well documented in all parts of the world. What if there is discontinuity in terms of temporal data?\n\nWe assume that you are referring to the temporal gaps in the reanalysis and remote sensing data. In fact, this is an issue for real-world data. To tackle this issue for the remote sensing data, a temporal decomposition was conducted to remove some discontinuity and aggregate the data into a weekly product. However, some pixels will still be empty. To tackle this issue, we first check if the pixel was covered by another satellite. If it was not the case, we flag the pixel as invalid and we discard it from the training and evaluation.\nRegarding the input reanalysis data, we first normalize the data using the pre-computed statistics and then replace the invalid pixels with zero values. We will add more details to the Appendix.  \n\n\n> Binary masks tend to oversimplify real-world events.\n\nPlease note that we only use the binary masks as flags where anomalies or extremes are detected. Using the binary approach allows to use the approach in the future for other extremes that cannot be derived from satellite products but that are stored in a binary format in databases. We agree that an analysis regarding the physical implication requires the continuous values where binary masks only indicate the existence of anomalies or extremes, see paragraph \"Physical consistency\" in Section 5.2, page 9.  \n\n> The method seems to bluntly connect anomalies with extremes without specific theoretical reasoning.\n\nOur aim is to investigate the relations between extreme events and their drivers from a data-driven perspective. The synthetic examples demonstrate that our proposed approach is able to achieve this. In contrast to statistical methods, our method does not require a prior hypothesis about drivers for extremes; instead, it generates hypotheses that can be verified by statistical methods in a second step. We believe that this is an important direction since climate reanalysis provides huge amount of data and it is infeasible to test all combinations. Data-driven approaches are therefore needed to generate potential candidates. Please also see a related discussion in the first paragraph \"Anomalies and extreme events detection in climate data\" in Section 2.\n\n> If possible, please add the reasoning behind feature representations and extremes.\n\nWe are not sure if the previous answers already answered the question.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2JZdNws4mo", "reviewer": "Reviewer_TRJw", "summary": "This work aims to identify the atmospheric drivers of extreme droughts. For this, they assume that for every impact of extreme droughts measurable with remote sensing, there is a precursor signal in assimilated land surface and meteorological data. The work proposes to identify these precursor signals with inherently interpretable machine learning: a computer vision model is trained to map input data into a binary latent space. These binary encodings are subsequently used to predict the future occurance of drought impacts. In that way, the binary encodings are assumed to be interpretable as \"anomaly in atmospheric data\" and \"normal atmospheric data\". Models trained in this way achieve good prediction skill (F1 score ~0.9) of drought impacts on synthetic data. Also, the binary encodings match anomalies in the synthetic input data reasonably well (F1 score ~0.8). The prediction skill on real world data is rather low: F1 Score ~0.2 for drought impacts.", "questions": "Climate data has extremely high spatio-temporal autocorrelation. How do you ensure your models are not overfitting?", "limitations": "The authors mention a variety of limitations in section 6. However, I believe as is the work has more fundamental flaws that I mention in the Weaknesses section, which, unless fixed, should definitely be named as limitations.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. This work introduces a potentially novel variant of anomaly detection: Detecting only those Anomalies that are predictive for correlated impacts. This variant is relevant for studying the drivers of extreme drought impacts.\n2. The work compares a wide array of baselines and performs many ablation studies.\n3. Synthetic experiments are conducted to study the proposed method before shifting to real-world data\n4. The main text of the paper is reasonably concisely written, with many additional details supplied in the appendix.", "weaknesses": "Major points:\n1. Confusing terminology: The authors speak about \"anomalies\" and \"extreme events\" without properly defining what is meant with which term. Furthermore I believe the used terminology is non-standard in the field, and propose the authors to instead use:\n    - Land surface Impacts of extreme events: These are what you call \"extreme events\", i.e. the VHI below a certain threshold. I would argue what you mean is the impact on state variables representing the land surface state (the ecosystem health) of extreme events. In your case droughts, but this could be any type of extreme events.\n    - Atmospheric drivers of extreme events: These are what you call \"anomalies\", but both the VHI below a certain threshold or the surface temperature above a certain level could be considered anomalous. Hence I recommend you rather focus on drivers here, these could be atmospheric or hydrological state variables (e.g. temperature or soil moisture) or land-atmosphere fluxes (e.g. evaporation).\n    - Luckily, you should be able to resolve this issue through simply rewriting your article.\n2. Scientific validity of experiment design: I find a few choices of the authors a bit odd in the experiment design:\n    - Albedo \"fal\" / \"al\" & Soil temperature \"stl1\" are state variables of the land surface that should be very related to reflectance. In fact, I believe it is not unlikely that remotely sensed brightness and brightness temperature has been assimilated to obtain these variables. This is not too different from VHI, which is created from similare remote sensing products. Thus I would say it should be if anything an Output of your approach. And even if no satellite products have been used to assimilate these variables, this would not solve the issue, but rather raise another issue: then the variables would entirely depend on prescribed schemes in the land surface model of IFS, which means your whole approach is limited by how well IFS reproduces these variables, which I assume is pretty poorly, so any \"anomalies\" you detect in these variables could be considered spurious.\n    -  Soil moisture \"swvl1\" / \"vsw\" is a state variable of the hydrological cycle, and thus should be highly correlated with VHI. However, as far as I know, its representation in ECMWFs land surface reanalysis is relatively poor (e.g. https://ieeexplore.ieee.org/document/9957057)\n    - The anomalies in precipitation that would drive drought are **no precipitation for many weeks**. So if you produce a binary encoding for every single time step, this should not be very predictive. In fact, precipitation is somewhat exponentially distributed: in most regions many days observe 0 precipitation, even under no drought conditions. One way you could potentially circumvent this issue is by using accumulated precipitation over many weeks, e.g. through an exponential moving average. Another way could be to actually implement a simple water balance model. Than again, if you use Soil moisture as inputs, it is essentially coming from such a water balance model...\n   - In other words: I think scientifically most interesting would be if you could connect anomalies in atmospheric variables like temperature, humidity and precipitation to the anomalies in land surface states (VHI). Then you could find the primary drivers for the impacts on vegetation and their time lag, and their spatio-temporal variability, which could be super interesting to study.\n3. Low predictive skill: The performance on the real-world data is pretty bad if I understand correctly (F1 of 0.2 - 0.3, Table 14). In addition the synthetic experiments revealed there seemingly is typically a lower skill on the latent binary variables compared to the outputs, so this makes we wonder if the predicted anomalies for extreme events mean anything at all?\n4. Many baselines for anomaly detection, but none for interpretable forecasting. You compare with a lot of baselines, which is generally great. But all of these perform some sort of anomaly detection on the inputs, which you then assume to be predictive features for your VHI labels. To me what would be a more interesting baseline is one that directly predicts the VHI label from the inputs, and then uses some post-hoc method to try to explain the predictions (e.g. shap, integrated gradients, ...). Because then you are comparing predictions of the drivers directly, and not just general anomalies. \n\nMinor Points:\n1. This work seems to focus a lot on the spatial aspect of things. However, arguably what matters most for drought at a particular pixel is the water balance at that pixel. And that is primarily driven by precipitation and evapotranspiration at that pixel, with only runoff introducing some sort of spatial component.\n2. VHI < 26 may be the result of not just drought. A heatwave could have a similar effect.  Also, VHI is a general vegetation condition index, not just for agricultural areas, but also for natural land cover. You may wish to rephrase your framing of this work studying the \"drivers of impacts of agricultural droughts\" into the \"drivers of impacts of extreme events on vegetation\".\n3. An alternative approach could be to not just predict a binary label, but rather the exact value of VHI. This would be similar to vegetation forecasting (e.g. https://www.sciencedirect.com/science/article/abs/pii/S003442572030256X , https://openaccess.thecvf.com/content/CVPR2024/html/Benson_Multi-modal_Learning_for_Geospatial_Vegetation_Forecasting_CVPR_2024_paper.html , https://gmd.copernicus.org/articles/17/2987/2024/ ). Probably it would be relevant to mention this related stream of literature also in the related works section. Also you may want to consider adding a comment on why you directly predict the (VHI < 26) label instead of the raw VHI, and afterwards apply detection. \n4. Quite some typos, e.g. l.22 \"very\" instead of \"vary\"", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the detailed review and the thoughtful feedback. We are glad that you found our work interesting and important for future research.\n\n> Terminology\n\nThank you for this suggestion. We will follow your suggestion and define the terminology in the introduction section of the paper in the revised version. Please see the discussion in the global response above.\n\n> Albedo & Soil temperature\n\nPlease note that ERA5-Land does not use data assimilation directly. The evolution and the simulated land fields are controlled by the ERA5 atmospheric forcing. We conducted 4 more experiments on both CERRA and ERA5-Land where we trained models that take only one variable al/fal or stl as input and predict the extreme events directly without the anomaly detection step. In all of these experiments, the F1-score was very low. In the next experiment, we increased the threshold for VHI and trained new models to predict extremes directly. The results for the validation set are shown below:\n\n|Dataset|Region| Variable |VHI<26|VHI<40|VHI<50|\n|-|-|-|-|-|-|\n|ERA5-Land|EUR-11|stl1|05.67|31.53|58.36|\n|ERA5-Land|EUR-11|t2m, fal, e, tp, stl1, swvl1|33.80|46.72|68.71|\n\nThe first potential reason to consider is that some land surface variables might deviate from the reality. Another reason might be that when training only on extremes (VHI < 26), there are not enough samples to learn the relations.\nPlease note that VHI is a combination of both TCI and VCI. Most extremes (VHI < 26) might result from a deficiency in both stl/t2m and vsw. This might also explain why stl and albedo cannot be that informative to predict very extreme events. We will discuss this issue.\n\n> Soil moisture in ERA5-Land\n\nIt is true that the volumetric soil water variable in ERA5-land has some biases. One solution is to use satellite observations for the top layer. However, our experiments showed that the model relates vsw anomalies with the extremes in VHI and provides reasonable predictions. Although we do not consider this as a major issue, we will mention it in the revised version.\n\n> Anomalies in precipitation\n\nThank you for this notice. At the moment, we treat each input variable in the same way and do not apply any pre-processing that is specific to a single variable. The proposed approach to pre-process precipitation is indeed an interesting direction. We will mention this in the discussion in the revised version.\n\n> Connecting anomalies in atmospheric variables to the anomalies in land surface states (VHI)\n    \nWe agree that this is the mid-term goal, but this is beyond the scope of the paper. The purpose of the paper is to present a novel approach that addresses this very important problem and a benchmark that allows to systematically evaluate approaches for this new task. The impact will be two-fold. First, methods can be further developed that improve the detection of drivers on the synthetic datasets. Second, the anomalies that are detected by the method in atmospheric variables can be further investigated by statistically approaches.        \n\n> The performance on the real-world data\n\nWe respectfully disagree regarding this point. The performance depends on the type and ratio of extremes, spatio-temporal resolution, the quality and consistency between the remote sensing and the reanalysis data. The performance is consistent with other recent works for predicting extremes on real-world data (Nearing et al. \"Global prediction of extreme floods in ungauged watersheds\", Nature, 2024). Note that the F1 scores substantially increase when the threshold on VHI is increased (see previous answer regarding stl1). We include a figure in the rebuttal PDF (Fig. 1), which shows the predictions. Given the quantitative and qualitative results, we think that the model provides reasonable predictions. Note that it is not required to predict all extremes in order to learn some relations from the predicted events.\n\n> Baseline as an interpretable forecasting\n\nPlease see the general author rebuttal and response above for the requested experiment.\n\n> The spatial aspect\n\nWe present a general approach that is not limited to droughts and specific input variables. We used droughts only as a real-world example. For instance, variables over sea regions can impact variables over land regions.         \n\n> VHI is a general vegetation condition index\n\nThank you for this suggestion. We mention this issue in lines 987-989.\nWe followed the general association of VCI and VHI with agricultural droughts, see i.e., (Hao et al. \"Seasonal drought prediction: Advances, challenges, and future prospects.\", Reviews of Geophysics, 2018). We will discuss this in the revision.\n\n> Predicting the exact values of VHI\n\nSince we will include the new baselines based on forecasting and integrated gradients (see previous answer), we will briefly discuss the mentioned methods on vegetation forecasting. The mentioned work by Shams Eddin et al. (\"Focal-TSMP: deep learning for vegetation health prediction and agricultural drought assessment from a regional climate simulation\", GMD 2024) has shown that it is hard to predict VHI directly. Instead, they predict NDVI and BT and then normalize the predicted values to estimate VHI. Another reason is that some extremes cannot be derived from satellite products but are stored as binary or discrete variables in databases. Having such scenarios in mind as well, we decided to present extremes as binary variables. Extending the approach to continuous variables is a potential future direction.\n\n> Typos\n\nThank you for the careful reading. We will fix this typo and check the paper for any other typos.\n\n> How do you ensure your models are not overfitting?\n\nWe follow the common practice in climate science where we define different time periods for the training/validation/test sets (Table 17 in the Appendix, page 33). As also shown in the Appendix (Table 11, page 29), increasing the model parameters still does not show a sign of overfitting.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NlSTzwt4AI", "reviewer": "Reviewer_wHsZ", "summary": "In this paper, the author investigates a novel, significant, and practical problem: how to efficiently identify extreme anomaly events from climate data. To address the temporal delays between anomalies and extremes and the spatially uneven response of anomaly events, the author first innovatively constructs three comprehensive datasets, including synthetic and real-world datasets. Next, the author proposes an end-to-end spatio-temporal anomaly detection network model, with key concepts including independent spatio-temporal encoders and a compact quantization layer adapted for anomaly detection. Finally, the author conducts detailed experiments on extreme anomaly event detection tasks using two types of datasets to demonstrate the effectiveness of the proposed datasets and methods.", "questions": "+ 1. The second contribution point in Section 1 should be swapped with the first to reflect the paper's primary contribution more logically. The main contribution appears to be the introduction of a novel anomaly detection problem in Earth sciences, supported by detailed benchmark experiments, while the methodological innovation is limited and should be de-emphasized.\n\n---\n\n+ 2. The anomaly detection part in Section 2 is overly redundant. It should focus on the works most relevant to the new anomaly detection approach proposed in this paper, emphasizing the connections and main differences with existing work, as well as the motivations.\n\n---\n\n+ 3. In line 113, the phrase \"but in spatio-temporal configurations of variables that potentially cause an extreme event with some time delay and at a potentially different location\" lacks clarity about how these variables differ from standard anomaly variables. I suggest the author provide a simple illustrative example figure to help readers understand.\n\n---\n\n+ 4. Considering the large parameters of the video Swin-Transformer, is it suitable as a backbone network for global-level spatiotemporal anomaly detection? There are many lightweight alternative spatiotemporal prediction backbone networks. The author could consider adding experiments with different backbone networks. Although this is not mandatory, it would be commendable.\n\n---\n\n+ 5. Regarding the dataset, what is the spatial resolution? Specifically, what is the actual area of one pixel? How do different resolutions impact the final detection results? Are there any experiments addressing this? \n\n---\n\n+ 6. In Table 2(a), the comparison between the first and second rows shows that while the $\\mathcal{L}_{(extreme)}$ loss term improves extreme detection, it harms anomaly prediction. Ideally, it should benefit both. Why is this the case? Can you provide an explanation? Similarly, adding the $\\mathcal{L}(anomal)$ loss term (comparing the third and first rows) shows it helps both types of detection, but the comparison between the fourth and second rows indicates it harms extreme detection, which seems contradictory. I hope the author can provide reasonable supplementary experiments and explanations. \n\n---\n\n+ 7. The text in the figures is too small and not reader-friendly. It should be just one size smaller than the main text for better readability.", "limitations": "I have already listed them in the question section.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 1, "contribution": 3, "strengths": "+ 1. This paper addresses an important but overlooked issue in practical scenarios: identifying extreme anomaly events from climate data. The motivation for this research is intuitive and substantial, as the task of recognizing extreme anomaly events is crucial for understanding climate patterns and can be applied in significant areas such as agricultural production and social activities.\n\n---\n\n+ 2. Unlike traditional anomaly detection, anomaly events typically exhibit temporal delay characteristics and spatially uneven responses. Therefore, the author first constructs an anomaly event detection dataset approximating the historical Earth system based on reanalysis and remote sensing datasets. However, there are reliability issues with extreme drought events obtained from remote sensing data as ground truth. Consequently, the author further develops multiple synthetic datasets to facilitate more convenient and reasonable experimental analysis and guidance. I greatly admire and appreciate the immense effort behind this work.\n\n---\n\n+ 3. Furthermore, the author presents an end-to-end neural network solution. Despite the limited technical innovations, the performance achieved is still impressive. In particular, the thorough experimental comparisons and analyses are commendable.", "weaknesses": "+ 1. This paper involves extensive preprocessing and adaptation of datasets. Although the author has not released the related code and data details, I believe a thorough introduction to the datasets or code is necessary for future use, which is crucial for subsequent work. This raises another issue: I find the contributions from the problem and dataset introduction to be overwhelming compared to those from the method itself. Therefore, it seems more suitable for the datasets and benchmarks track rather than the research track.\n\n---\n\n+ 2. This paper appears to remain limited to regional-level anomaly event detection. However, we know that the entire Earth system is dynamically interconnected and evolving; changes in one corner of the Earth can significantly impact distant regions (e.g., the butterfly effect). To model the spatiotemporal dynamics at the global level, the current backbone network seems inadequate for such scalability.\n\n---\n\n+ 3. Although the paper is structurally clear, there is still much room for improvement in writing and presentation. Particularly for interdisciplinary papers, the author should not assume that readers have backgrounds in multiple disciplines. Necessary background knowledge and related work should be supplemented as thoroughly as possible.\n\n---\n\n+ 4. I have also listed several confusions and suggestions in the paper. If the author can address my concerns, I would be willing to support its publication here and recommend an increase in its score, even though it is more suitable for the DB track.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate and thank the reviewer for recognizing and appreciating the efforts behind this work. Please see our responses to the questions below.\n\n> Code and datasets\n\nPlease note that we will release the code and datasets along with the documentations upon publication. We describe the datasets and benchmarks in the Appendix in Sections A, H, and I (pages 18-21, and 32-35). Please let us know if more information is required. We are happy to clarify any open questions and provide more details.\n\n> The contribution of the benchmark in comparison to the method\n\nWe appreciate that the addressed problem and proposed benchmarks are considered as an important contribution. Since the proposed problem has not been addressed before, we also present a novel approach that is not a simple adaptation of existing works to the problem. As you have acknowledged, the proposed approach outperforms adaptation of one-class unsupervised [25; 107; 37], reconstruction-based [44; 49], and multiple instance learning [92; 93; 94] approaches to the problem. We also included additional comparisons in the global response. We thus argue that both the benchmarks and the proposed method are a valuable and important contributions. However, we will change the order of the contributions in Section 1 as suggested.\n\n> The backbone and the global level analysis\n\nWe already compare 6 different backbones which differ in the number of parameters in the Appendix (Table 11, page 29). We agree that using Video Swin Transformer as backbone for the global level would be very expensive. However, the backbone can be replaced by more efficient backbones. We conducted an additional experiment where we replaced the attention block with a linear selective state space model (Gu et al. \"Mamba: Linear-time sequence modeling with selective state spaces\", 2023). The results are reported below for F1 scores on the validation set for anomalies/extremes detection:\n\n|Backbone $f_{\\theta}$|Hidden dimension ($K$)|Params.|F1-score|\n|-|-|-|-|\n|3D CNN|8|63k|57.15 / 91.21|\n|Video Swin Transformer|8|19k|81.22 / 91.16|\n|Mamba|8|15k|82.15 / 90.18|\n|3D CNN|16|250k|70.93 / 93.75|\n|Video Swin Transformer|16|62k|82.78 / 92.45|\n|Mamba|16|56k|83.29 / 92.01|\n|3D CNN|32|998k|84.95 / 93.43|\n|Video Swin Transformer|32|230k|84.14 / 93.12|\n|Mamba|32|214k|84.00 / 93.43|\n\nPlease note that we did not yet fully train the network with MAMBA as backbone, but the results already indicate that using Mamba instead of Swin Transformer achieves similar or better results when less parameters are used. In contrast to Swin Transformer, Mamba can be scaled to the global scale. Please note we already cover large areas at the continental scale (Table 17, page 33) and we think that identifying spatio-temporal relations at the continental scale is at the moment already challenging enough. We also mention this limitation already lines 345-348.\n\n> I have also listed several confusions and suggestions in the paper. If the author can address my concerns, I would be willing to support its publication here and recommend an increase in its score, even though it is more suitable for the DB track.\n\nWe appreciate your careful reading of our work and your suggestions. Please find below our responses to your questions that have not been already answered.\n\n> Line 113 and a simple illustrative example figure\n\nWe will revise the sentence. Please see the author rebuttal and the general response above. We have also included a figure in the PDF (Fig. 2).\n\n> The spatial resolution\n\nWe provide the spatial resolution in Section 4.2, page 5 and in Appendix Section I.I., page 34.\nPlease note that depending on the coordinate system, the area on the Earth surface changes with respect to the location (i.e., it decreases toward the pole).\nThe spatial resolution for ERA5-Land is $0.1^\\circ \\times 0.1^\\circ$ on the regular latitude longitude grid. CERRA Reanalysis has a spatial resolution of $5.5$ km $\\times~ 5.5$ km on its Lambert conformal conical grid.\nWhile the remote sensing data has a high resolution of $0.05^\\circ \\times 0.05^\\circ$. We will make it clearer in the revision. We conducted an additional experiment regarding the impact of the spatial resolution:\n\n|Dataset|Region|Spatial resolution|F1-score anomalies/extremes|\n|-|-|-|-|\n|ERA5-Land|Europe|$0.1^\\circ$|- / 31.87|\n|ERA5-Land|Europe|$0.2^\\circ$|- / 30.09|\n|Synthetic CERRA|-|$5.5$ km|82.78 / 92.45|\n|Synthetic CERRA|-|$11$ km|68.42 / 79.77|\n\nThis shows that the spatial resolution matters as expected.\n\n> Loss functions in Table 2(a)\n\nWithout the $\\mathcal{L}\\_{(anomaly)}$ loss, the detection of anomalies is not reliable since pixels at regions and intervals where no extreme event occurred can be assigned to $z_{q=1}$ (anomaly) as well. The other loss functions actually do not prevent that this is happening.\n\nIn case of $\\mathcal{L}\\_{(extreme)}$ multi-heads, we observe that anomalies are identified in a small subset of variables because the network omits some variables if there is a correlation with other variables. Please see Fig. 4 in the rebuttal PDF.\nIn case of a single head such flips occur less often, but they can occur. If the $\\mathcal{L}\\_{(anomaly)}$ loss is used, such flips cannot occur and the multi-head improves both F1-scores by a large margin. When comparing row 2 and 4, there is a slight decrease in extreme prediction but a large improvement in anomaly detection. Note that there is always a trade-off between extreme and anomaly detection. The anomalies generate a bottleneck of information. When more information goes through the bottleneck the better the extreme prediction gets. Without any anomaly detection, the extreme prediction is best as shown in Table 12 (page 29), but the increase in F1 score is only moderate. This is also visible in row 2 and 3 in Table 2(b). Cross-attention improves extreme prediction, but it hurts the detection of anomalies since the information is propagated between the variables.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank all reviewers for their efforts and the valuable comments. We appreciate the positive and encouraging comments by the reviewers that we briefly summarize:
* Reviewer fRrQ acknowledges that the proposed task is crucial for climate science and acknowledges the experiments on real and synthetic data.
* Reviewer 5qpR acknowledges the novelty of our method for identifying spatio-temporal anomalies that are correlated with extreme events.
* Reviewer TRJw acknowledges the novel variant of anomaly detection which is relevant for studying the drivers of extreme events, and appreciates the experimental evaluation, including the wide array of baselines, ablation studies, and the evaluation on both synthetic and real-world data.
* Reviewer wHsZ acknowledges that the work addresses a crucial, practical, and overlooked task. wHsZ also appreciates the immense effort behind this work, the thorough experimental analysis, and the impressive performance of the porposed method.

The reviewers fRrQ, TRJw, and wHsZ raised issues regarding the presentation. The main presentation issue are the used terms "anomalies" and "extreme events". We agree that this terms need to rephrased and more clearly defined since an extreme event is an anomaly as well. We believe that this also resulted in a misunderstanding of our task and contribution by reviewer fRrQ. In the following, we give a brief definition of the terms "anomalies" and "extreme events" as they were used in the paper:

Extreme events: Examples of extreme events are extreme droughts, floods, or heatwaves. We represent these events by the impact on state variables. For instance, we use extremely low values in vegetation health index (VHI) as an indicator for extreme droughts.
We assume that extreme events are reported or can be derived from state variables, i.e., they are observed.   

Anomalies: We consider anomalies in atmospheric/hydrological state variables (e.g. temperature or soil moisture) or land-atmosphere fluxes (e.g. evaporation) that are the drivers of extreme events. In other words, we are looking for anomalies in a) other variables than the variables that is used to define a particular extreme event of interest, b) might occur earlier in time and at a different location than the extreme event and c) that are drivers or directly related to the extreme event. This means that not all anomalies that might occur in the atmospheric/hydrological state variables are related to an extreme event. Figure 3 in the PDF of the rebuttal illustrates this. 

We agree that the term "Anomalies" is confusing and we will rephrase it as suggested by Reviewer TRJw, who also points out that this issue can be simply addressed. The other presentation issues are minor and we explain in the comments to the individual reviewer how we will address them.       

We hope that this response also resolves the misunderstanding of Reviewer fRrQ, who struggled to understand the difference between "anomaly" and "extreme" and consequently misunderstood our work as an approach for extreme event prediction (and thus the motivation of the model design), although our focus is on identifying the drivers of extreme events.  

Reviewer 5qpR has some concern about the dependency of the model on the temporal resolution and the applicability of the model to other types of extremes, which we address in our response to 5qpR.

Reviewer TRJw asks for a comparison to interpretable forecasting approaches using integrated gradients. As suggested, we conducted this additional comparison. To this end, we trained two models that predict extreme events directly from the input variables and then we applied a post-hoc integrated gradients. Both models use the same backbone as our model but without the anomaly detection step. For (Integrated Gradients V2), we added a cross attention. For this experiment, we compute the gradient only with respect to predicted extremes and computed a different threshold for each variable separately. The models achieved F1-scores for detecting extremes of 93.09 for (Integrated Gradients V1) and 93.97 for (Integrated Gradients V2). The F1-score for "anomalies" (we use the term here for consistency with the submission) on the synthetic data are:

|Model|Val|Test|
|-|-|-|
|Integrated Gradients V1|38.14|33.11|
|Integrated Gradients V2|35.39|34.87|
|Ours|82.78|80.44|

Qualitative results are provided in Figure 2 in the PDF of the rebuttal. When we add more interactions between the variables (Integrated Gradients V2), the gradients tend to omit some variables (soil moisture). Both models have also difficulties with the synthetic t2m, which includes red noise by design. These results demonstrate that networks that predict the extremes directly from the input variables utilize much more information even it is not correlated with an extreme. It is thus beneficial to introduce a bottleneck into the network that enforces the network to explicitly identify drivers for extremes. We will include these additional baselines.

The other concerns of reviewer TRJw are addressed in our response to TRJw.

Reviewer wHsZ has a concern about the backbone and its applicability to the global scale. We already compare 6 different backbones which differ in the number of parameters in the Appendix (Table 11, page 29) and provide results for additional backbones in the response to wHsZ. Reviewer wHsZ also rates the contribution of the benchmark higher than the proposed method, but we do not think that this is a major issue and we address this point as well as the other raised questions in the response to wHsZ.

The PDF in the rebuttal contains additional figures. Figure 1 shows examples of predicted extreme events on real data and Figure 4 shows examples for the ablation study in Table 2.

We hope that the responses to the reviewers resolve their concerns. We would appreciate if the reviewers reply and give us feedback if the questions have been answered. We are happy to answer further questions.

Meta-review: This paper proposes a novel approach for detecting spatio-temporal anomalies associated with extreme weather events which gives a solid technical contribution particularly in identifying the drivers of extreme events in climate science.
The authors have adequately addressed the reviewers' concerns by clarifying the terminology and conducting additional experiments, which improved the overall quality of this paper.
Decision: Accept (poster)