Towards predicting dynamic stability of power grids with Graph Neural Networks
Authors: Christian Nauck
Michael Lindner
Konstantin Schürholt
Frank Hellmann
Venue: Submitted to ICLR 2023 (2023)
Keywords: Power grids
dynamic stability
Graph Neural Networks

Abstract: To mitigate climate change, the share of renewable energies in power production needs to be increased. Renewables introduce new challenges to power grids regarding the dynamic stability due to decentralization, reduced inertia and volatility in production. However, dynamic stability simulations are intractable and exceedingly expensive for large grids. Graph Neural Networks (GNNs) are a promising method to reduce the computational effort of analyzing dynamic stability of power grids. We provide new datasets of dynamic stability of synthetic power grids and find that GNNs are surprisingly effective at predicting highly non-linear targets from topological information only. We show that large GNNs outperform GNNs from previous work as well as as handcrafted graph features and semi-analytic approximations. Further, we demonstrate GNNs can accurately identify \emph{trouble maker}-nodes in the power grids. Lastly, we show that GNNs trained on small grids can perform accurately on a large synthetic Texan power grid model, which illustrates the potential of our approach.

TOWARDS PREDICTING DYNAMIC STABILITY OF POWER GRIDS WITH GRAPH NEURAL NETWORKS
Anonymous authors Paper under double-blind review
ABSTRACT
To mitigate climate change, the share of renewable energies in power production needs be increased. Renewables introduce new challenges power grids re- garding the dynamic stability due to decentralization, reduced inertia and volatility in production. However, dynamic stability simulations are intractable and exceed- ingly expensive for large grids. Graph Neural Networks (GNNs) are promising method to reduce the computational effort of analyzing dynamic stability of power grids. We provide new datasets of dynamic stability of synthetic power grids and find that GNNs are surprisingly effective at predicting highly non-linear targets from topological information only. We show that large GNNs trained dataset outperform GNNs from previous work, as well as several baseline models based on handcrafted features. Furthermore, accurate identification of particularly vulnerable nodes in power grids, so called troublemakers. Lastly, we find that GNNs trained on small grids generate accu- rate predictions for on large synthetic model of the Texan power grid, which illustrates the potential real-world applications of the presented approach.
rich
GNNs demonstrate the
INTRODUCTION
Adaption to and mitigation of climate change jointly influence the future of power grids: 1) Mitigation of climate change requires power grids to be carbon-neutral, with the bulk of power supplied by solar and wind generators. These are more decentralized and as opposed to conventional turbine generators they have no intrinsic ability to respond to power imbalances and frequency deviations. Furthermore, the production of renewables more volatile. Renewable energies will have to start contributing to the dynamical stability of the system (Milano et al.) 2018 Christensen cet al. 2020) in the future, requiring a new understanding of the complex synchronisation dynamics of power grids. 2) A higher global mean temperature increases the likelihood as well as the intensity of extreme weather events such as hurricanes or heatwaves (Field et al.) 2012 Pörtner et al. 2022) which result in great challenges power grids. Building sustainable grids well as increasing the resilience of existing power grids towards novel threats are challenging tasks on their own. Tackling climate change in the power grid sector calls for solution both at the same time and requires new methods to investigate aspects of dynamic stability.
Power grids are complex networks, consisting of nodes that represent different producers and consumers, as well as edges that represent power lines and power transformers. In contrast to many other networks, the interaction of nodes through the edges is governed by physical equations, the power flow. Their emergent properties can be highly unintuitive. For example, the Braess paradox describes the phenomenon that adding lines to a power grid may reduce its stability ((Witthaut & Timme 2012 (cchäfer et al., 2022)). Such effects can be non-local, i.e. the parts of the grid with decreased stability might far away from the added line. Similarly, failures of a line one part of the network lead to overloads far away. Our work deals with the challenge of predicting the ability of the grid to dynamically recover after localized faults perturbed the system.
Classically, the dynamical actors are connected to the highest voltage level, the transmission grid. Transmission grid operators routinely simulate potential faults in the current state of the power grid, to assess its real time dynamic resilience. The possible faults are called contingencies in this context. Even though such simulations do not explicitly model the lower voltage layers of the grid, they are already compute bound. Hence, not all contingencies of interest be tested for. As distributed

---

newable generation is typically connected at lower grid levels, this problem will become more acute as renewables start playing a larger role in the grids dynamics. Conducting high-fidelin simulations of the whole hierarchy of the power grid and exploring all states will not be feasible (Liemann al. 2021]. For future power grids, knowledge of how to design robust dynamics is required. This has led renewed interdisciplinary interest in understanding the collective dynamics of power grids [Brummitt et al., 2013). with a particular focus on the robustness of the self-organized synchro- nization mechanism underpinning the stable power flow (Rohden et al. 2012 Motter et al. 2013 Dörteer et al. 2013) by physicists and control mathematicians Witthaut et al. (2022).
Synchronization refers to the fact that stable power flow requires all generators to establish a joint frequency. It is self- organized in the sense that this is achieved without further communication or an external signal. To understand which structural features impact the self-organized synchronization mechanism, it has proven fruitful to take a probabilistic view (Menck et al. 2014; [ellmann et al.) 2016). Probabilistic approaches are well established in the context of static power flow analysis [Borkowska] 1974). In the dynamic context, considering the probability of systemic failure follow- ing a random fault effectively averages over the various contingencies. Such probabilities are thus well suited to reveal structural features that enhance the system robustness vulnerability. This ap- proach has been highly successful in identifying particularly vulnerable grid regions (Menck et al.] 2014 (Schultz al. [0144a] Nitzbon al. (017) and revealing general mechanisms of desynchro- nization (Hellmann et al. 2020). Probabilistic stability assessments recently gained more attention
in the engincering community as well (Liu & Zhang 2017 Liu et al. 2019 Liemann et al. 2021}.
Given the need for probabilistic analysis, and the computational cost of explicit simulations, we apply Graph Neural Networks (GNNs) to directly predict probabilistic measures from the system structure. Such GNNs could be used to select critical configurations for which a more detailed assessment should be carried out. Moreover, the analysis of the decision process of ML-models might lead to new unknown relations between dynamical properties and the topology of grids. Such insights may ultimately inform the design and development of power grids. Since datasets of probabilistic stability in power grids of sufficient size do not exist yet, we introduce new datasets, that consist of synthetic models of power grids and statistical results of dynamical simulations. We simulated datasets of increasing complexity to get closer to reality step by step. There are 10,000 small grids, 10,000 medium-sizee grids and for evaluation purposes one large grid based on a synthetic Texan power grid model.
Related work on power grid property prediction Since power grids have an underlying graph
structure, the recent development of graph representation learning (Bronstein et al. 2021 Hamilton
2020j introduces promising methods to use machine learning in the context of power grids. There
is number of applications using GNNs for different power flow-relate tasks (Donon et al.
2019
Kim et al., 2019 Bolz et al. 2019 Retiére et al., 2020
Wang et al.) 2020 Owerko et al.
2020
Jhun ctt al. 2022) and to predict transient dynamics in microgrids (Yu et al. 2022). In (Nauck et al. 2022)
Gama et al. 2020 Misyris et al. 2020 Liu et al.
2021 Bush et al. 2021 Liu et al., 2020:
small GNNs are used to predict the dynamic stability on small datasets. The authors demonstrate the general feasibility of the approach, but do not compare to conventional baselines. We add such baselines and introduce datasets that have ten times many grids. This allows us to train GNNs with much higher capacity to achieve higher predictive power.
Our main contributions are: We introduce new datasets of probabilistic dynamic stability of synthetic power grids. The new datasets have 10 times the size of previously published ones and include Texan power grid model to map the path towards real-world applications. We also observe a relevant new class of nodes in the dataset: So-called troublemakers, at which perturbations are strongly amplified. Such nodes may be dangerous to hardware and the overall grid stability. Their identification constitutes an additional task. We train strong baselines and benchmark models to evaluate the difficulty of all tasks. Our results demonstrate i) that the larger dataset allows train more powerful GNNs, (ii) which outperform the baselines, and (iii) transfer from the new datasets to real-sized power grid. The general approach is visualized in Figure T
2

---

A) Generation of synthetic grids
B) Monte-Carlo simulations to compute dynamic stability
Training of GNN models
Evaluation on large Texan power grid
dynamic stability
dynamic stability
GNN
GNN
stable
unstable
Figure 1: We generate new datasets of the dynamic stability of power grids, based on synthetic
power grids (A) and statistics of dynamical simulations (B). Then, GNN models are trained to pre- dict the dynamic stability of the synthetic grids (C) and evaluated Texan power grid model (D).
GENERATION OF THE DATASETS
Structure and modeling of power grids Viewed networks power grids have distinctive topo- logical properties. Power grids are sparsely connected. The degree distribution has a local maximum at very small degrees (c.g. R= 2.3), an exponentially decaying tail and mean of x 2.8 (Schultz et al., 2014b). Hence, most nodes are only connected to a small number of neighbours though a few highly connected nodes typically do exist.
The study of non-linear self-organizing processes in power grids, specifically the interplay of syn- chrony and active power flows has been an active topic in the research of complex power grids (Witthaut et al. 2022). Synchrony is crucial for power grids as a stable power flow is only possible if all generators operate at the same frequency. At the same time the ever present fluctuations in demand have to be compensated by locally varying the frequency. The interplay between energy balance and frequency throughout the grid is self-organized as it happens without further communi- cation between the nodes or some central entity. The relevant timescale of this process is seconds, and we will analyze the capability of the grid to regain synchrony following a localized fault.
Power grids as dynamical systems A full scale analysis that can treat high-fidelity models of real systems is currently out of reach for several reasons. These include that real world data does not exist not accessible, synthetically generating large numbers of realistic grids is challenging, and that large dynamical models can not be simulated fast enough with current software (Liemann al. 202T). These problems force trade-offs on us, most notably reducing the details of the intrinsic behaviors of dynamical actors to that of inertial oscillators. This leads to the Kuramoto model Kuramoto (1975), a paradigmatic model for synchronization studies (Kuramoto) [005] Rodrigues et al. 2016),, which was introduced for power grids in (Bergen & Hill (181)) This model strikes a careful balance of capturing the key dynamics that govern synchronization in real power grids while remaining computationally and to some degree analytically tractable. It is thus highly useful for understanding the relationship between grid structure and synchronization, but it should not be taken to be fully adequate model of the grid by itself. Any future treatment of dynamic stability based on more accurate models will also have to solve the challenging sub-problem of the impact of topology on synchrony that we consider here, thus we see this work as a first step.
In an ideal AC grid, the nodes voltage is given by a 50 or 60Hz sine curve. Writing o; for the deviation of the phase of the local AC voltage from an arbitrary reference signal, the instantancous power flow on line i, j (Piz) is given in terms of the line parameter K by P.j- K sin( Φj)- The normalized time derivative 2r ój provides the local frequency deviation from the reference frequency. The two most important dynamical processes that establish synchrony and a stable power flow are inertia, i.e. the change of local frequency result of absorbing a local power imbalance MØ; AP, and II) droop control, the local change of injected power due frequency deviation pdroop -aò,.
In order to extract the impact of topology as cleanly as possible we assume homogeneous inertia constant M, droop parameter a and line parameters K. The dynamical equations for the self- organized synchronization and stabilization of the active power flow are then given by conservation of energy. Let pd denote the power injected/consumed at node and A. the adjacency matrix:
AP; =Pd
pdroop
AAyP.
M = pMd ag. K Σ
Atysin(bi - 95)
(2)
)
3

---

Synchronous operation requires 0 Ø. e5 The fixed point equation P K E, Asysin(0} Φj) is the power flow equation grid with topology given by the adjacency matrix A.
Quantifying dynamic stability power grids We quantify dynamic stability with the single- node basin stability (SNBS) (Menck et al. 2014). This measure is widely used in the study of synchronization phenomena. As mentioned in the introduction, is probabilistic, i.e. defined as the probability that the system recovers following perturbation by random fault.
In the dynamical systems community this perturbation is typically modeled by a distribution of initial conditions of the post fault system. For every node i we choose distribution of initial conditions Pj corresponding to contingencies localized at that node. Then the SNBS at node the probability that the entire system returns to the fixed point from an initial condition drawn from Pi. Call p(y(æ) the probability of y given the initial condition r. For a distribution of initial conditions p(x:) we write ply x
p) = p(yl=))(x)) for the compound distribution. Then we have:
SNBS; p (0(00) o- (((()))(0)) p:)
This probability can be estimated as the outcome of a Bernoulli experiment. Here we simulate 10,000 trajectories per node to minimize statistical errors. The underlying simulations to generate the datasets are feasible for anyone with some domain knowledge, but the composition of entire datasets require significant amounts of computational resources. For the datasets with 20,000 grids and the Texan power grid, the simulations take roughly 700,000 CPU hours. Thus an important contribution to publish a full dataset to enable groups that have less computational resources to work on this important problem. The goal of our work is the usage of GNNs to replace these expensive simulations. To that end, train GNNs to learn the graph function (P + SNBS where P. is featurized version of Pa
2.1 NEW TROUBLEMAKER DEFINITION
The condition in the SNBS is not sufficient to ensure stable grid operation at all times. If the transient trajectory after the perturbation violates operational bounds, machines in the system switch off to protect themselves, potentially triggering failure cascades and large blackouts. This motivates the definition of survivability (Hellmann al. 2016). the probability that the system stays within these bounds after a fault. In our dataset, we observed that there are some nodes for which the transient reaction is vastly larger than the initial perturbation. We define the maximum frequency deviation (MFD) of the whole system from an initial condition as
MFD, (d(0), max 6.
(,(()
We consider nodes as safe if they have a very small probability 7 that the MFD is larger than a critical threshold 3 following fault, other nodes are troublemakers (TM)
TM, = {o o if MFD; < (0(0), 0(0) ~p:) > 11
(5)
otherwise.
The minimum size that can be chosen for the rare event rate 7 is determined by the statistical error of the estimator of the probability, see Appendix A.5
For choosing the critical threshold consider, that the outer limits for frequency deviations for the European grid are +2Hz -3Hz. For our model we chose the symmetric critical threshold
2.4Hz, corresponding to maximum deviation of the angular velocity of [0) < 15 =: 3, both for simplicity and comparability to previous work. We tune the distribution of initial conditions to focus on the worst offenders, settling As confined to +0.4H corresponding an amplification by a factor 2.48= 0.4H2 6. More details are given in Appendix A.5 The observation of large ampli- fications and the target function are both of practical importance and novel far are aware.
4

---

2.2 MODELING OF THE TEXAN POWER GRID
To take a further step towards real-world applications, we evaluate the performance of our GNN models by analyzing the dynamic stability of a real-sized synthetic power grid. Real power grid data are not available due to security reasons and calculating an entire SNBS assessment of the fully parameterized synthetic model by Birchfield appears not to be feasible due the computational effort (Liemann et al.] 2021). We chose a synthetic model derived from the Texan power grid topology, introduced by [Birchfield et al. (2017). The synthetic Texan power grid model consists of 1,910 nodes after removing 90 nodes that are not relevant for the dispatching.
We use the same modelling approach as for the other grids, i. we use only the topological properties. As consequence, we only investigate the potential applicability of GNNs to real-sized grids and can not make any statements about the real-world Texan power grid. Even after applying the simplifications, the simulations are already very expensive due to the large number of nodes. To manage the computational cost of simulating dynamic stability of such a large grid, we reduce the number of simulated perturbations from 10,000 to 1,000. Nonetheless, the simulation of that grid takes 127, 17,000 CPU hours. Computing less perturbations results in an increased standard error of approximately 0:1.31 for the SNBS estimates.
2.3 STRUCTURE OF THE DATASETS
To generate the datasets, we closely follow the methods in Nauck et al. (2022) and extend their work by computing 10 times many grids. To investigate different topological properties of differently sized grids, we generate two datasets with either 20 or 100 nodes per grid, referred to as dataset20 and d0taset100. To enable the training of complex models, both datasets consist of 10,000 graphs. Additionally, one large synthetic Texan grid is provided for testing out-of-sample performance, see Section 2.2
For every grid two input features are given, namely the adjacency matrix A E {0, N represent- ing the topology and a binary feature vector Pyd E 11 specifying the power injection/demand at the nodes, cf. Equation (T). Here N is the number of nodes. Likewise, for every grid two target vectors given, SNBS E [o. 11", see Section 2 and the TM 10, The TM class is derived
from the maximum frequency deviation MFD [0, oo which also available in the dataset, see Section 2. Details regarding the modeling strategy are provided in Appendix AA.4 Examples of the grids of dataset20, 10t and the Texan power grid as well as the distributions of SNBS (characterized by multiple modes) and the troublemaker nodes TM (imbalanced binary classification task derived from the MFD) are given in Figure 2 Even though the same modelling approach is used, there significant variations that are entirely caused by different topologies and grid sizes. Interestingly, the SNBS distribution of Texan power grid has third mode which is challenging for prediction tasks. Overall, the power grid datasets consist of the adjacency matrix encoding the topology, the binary injected power P per node as input features, and nodal SNBS and MFD.
PREDICTING DYNAMIC STABILITY OF POWER GRIDS USING GNNS
In this section, we predict the dynamic stability on the new datasets using GNNs. We start by introducing the experimental setup, followed by the prediction of SNBS on the two grid sizes as well as analyzing out-of-distribution capabilities from small to large grids. Subsequently, we establish the advantages of our large dataset in comparison to previous work. Afterwards, we evaluate the GNNs trained on our datasets on the Texan power grid as larger, more realistic test-case. Lastly, we identify troublemakers.
3.1 EXPERIMENTAL SETUP
We train GNNs on a nodal prediction tasks, using regression for SNBS, and regression with thresh- olding as well as classification for TM (visualized in Figure 3h. As input the GNNs are given an adjacency matrix and the power injection/demand at the nodes, cf. Section 2.3 We split the datasets in training, validation and testing sets 170:15:15). The validation set is used for the hyperparameter optimization, we report the performance on the test set. To minimize the effect of initializations we use 5 different initializations per model and compute average performances using the three best.
5

---

dataset20
dataset100
Texan grid
density
10
demsit
10
density
10
1.0
0.8
SNBS
SNBS
SNBS
10a
density
density
density
10
10
10
10
20
frequency deviation
max frequency deviation
frequency deviation
stable: 97.19%
stable: 94.56%
stable: 97.33%
20 40 60 90
100
40 60 60 100
20 40 60 80 100
Figure 2: Examples of power grids in the datasets with 20 nodes (top left) and 100 nodes (top center) and the Texan power grid model (top right). Blue color denotes sources and the orange sinks. Below, the histograms of SNBS and the maximum frequency deviations (logarithmic scale) are shown. At the bottom, the share of troublemakers (TM) shown, where green represent stable nodes and purple troublemakers (TM).
To evaluate the robustness of the GNNs, we analyze the performance of different models based on several GGNN-acchitcctures: GNNs with ARMA filters by Bianchi et al. (2021), Graph Convolutional Networks (GCN) by Kipf Welling (2017), SAmple and aggreGatt (SAGE) by Haamllto et al.) (2017) and Topology Adaptive Graph Convolution (TAG) by Du et al. (2017). refer to the models by ArmaNet, GCNNet, SAGENet and TAGNet. We conduct hyperparameter studies to optimize the model structure regarding number of layers, number of channels and layer-specific parameters using dataset20. Afterwards, we optimize learning rate, batch size, scheduler for dataset20 and dataset separately. Details the hyperparameter study and the models are given in Appendix A.7
Baseline models To better assess the GNN performance, we set up several baseline models. Schultz et al. (2014a) were the first to attempt predicting nodal dynamic stability of power grids us- ing a logistic regression of common network measures and hand-crafted features. For the first base- lines, we set up similar linear regression models with the following input features: degree, average- neigbor-degree,
current-flow www.bbetteeeeseeeeeeeeeeeeee the injected power pd, Additionally, we use more complex Multilayer perceptrons (MLPs) trained on the same features as baselines. We investigate the performance of two differently sized MIPs. MLPI has one hidden layer with 35 units per hidden layer, resulting in 1,541 parameters and MLP2 consists of hidden layers and 500 hidden units per layer leading to 1,507,001 parameters. We conducted hyperparameter studies optimize the batch sizes and learning rates.
and
the coefficient of determination (R2- score) for regression and F2-score for classification. F2-score a modified F-score giving more weight recall and less the precision in the calculation of the score. In case of identifying vulnerabilities of power grids it is more important to identify all critical states, even if this increases the number of false positives. The details of computing the metrics are provided Appendix A.6
Metrics for evaluation To analyze the performance,
3.2 PREDICTION OF SNBS
GNNs can accurately predict SNBS In our first set of benchmark experiments, the goal to predict SNBS with high accuracy, as measured by the coefficient of determination R². The key

---

A (RAMMM=
GNN
PE PEt0,
SNBS [0, I TM (0. 1
Figure 3: Prediction of nodal outputs SNBS and TM using GNNs. The inputs are the adjacency matrix A and the injected power pd The prediction is purely based on the structure and topology of the grid and does not consider individual faults.
result is the surprisingly high performance of GNNs across all datasets, the first two columns in Table [ R² reaches values above 82 % for dataset20 and above 88 % for dataset100. SNBS highly nonlinear property and the obtained performance exceeds expectations. The predictive performance captures not only the general trend but the modalities in the data as well (cff. Figure Interestingly the previously published Ar-bench (Nauck et 2022 performs worse than MLPs, but the larger GNNs outperform all baselines.
Table Results of predicting SNBS represented by R² score in %. Each column represents the eval- uation on different test set, e.g. tr20ev20 denotes that the models are trained and evaluated using dataset20. Additionally, we analyze the out-o--distiiuution capabilities by evaluating the models on different datasets without retraining, e.g. we train model on dataset20 and evaluate it dataset100 and refer to this by rr20ev100. Besides the performance of the GNNs, we show the performance of linear regression and Multilayer perceptrons using hand-crafted features baselines.
model
tr20ev20
tr 1000ee 100
tr20ev 100
tr20evTexas
tr 100eeTexxs 55.22 +: 6.349
Ar-bench
51.82 88
60.34 0.299
38.80 1.327
38.55 5.99
ArmaNet
80.63 +
87.47
66.75
60.54 +
73.47 :: -46.02 + 1.575
GCNNet
70.64 + 02262
75.49 = 0.276
59.46 0,400
2.27 1600
SAGENet
65.46 + 0.208
75.57 + 0.228
52.27 0,784
33.47 + 0.509
55.38 + 1.290
TAGNet
82.49 0.455
88.22 0.135
66.10 0.508
62.69 + 2.029
84.41 + 0.47
linreg
41.75
36.29
5.98
-1 -11.39
-22.62
MLP1
58.47 + 0.149
63.59 0.051
28.49 1,495
-34.52 17.934
19.79 + 8669
MLP2
58.20 0.4222
65.52
19.65 2.109
5.81
58.46
tr20ev20
tr100ev100
tr20ev100
1.0
1.0
1.0
0.8
0.8
0.8
true SNBS
true SNBS
0.6
0.6
0.6
true
0.4
0.4
0.4
0.2
0.2
0.2
0.2
0.4
0.6
0.8
1.0
0.2
0.4
0.6
0.8
1.0
0.2
0.4
0.6
0.8
1.0
model prediction
model prediction
model prediction
Figure 4: SNBS over predicted output of the ArmaNet model for tr20ev20 (trained on dataset20 and evaluated on dataser20), trI (0ev1100 and m22ev100. The diagonal represents perfect model (R2 = 1), the banded region indicates predictions which are accurate up to an error of +0.1.
GNNS generalize from dataset20 to dataset100 Note that, as reviewed the introduction, the dynamical properties of the power grid are non-linear and non-local. Our perturbations are localized, but the probabilistic measures look at the whole system response. Thus is priori unclear how well we can expect the models to generalize from smaller grids to larger ones. Training small grids without loss generalization and predictive power would huge advantage to scale to real power grids. To evaluate the potential of our datasets and GNN models that end, apply

---

out-of distribution task by training the models on dataset20 and evaluating the performance without any further training on dataset100. The third column in Table [ (tr20ev100) show that all GNNs generalize well and are able to predict SNBS with R exceeding 66 %. We would like to emphasize the significance of that finding. Given sufficient size and complexity in the source dataset, GNNs robustly predict highly nonlinear stability metrics for grids several times larger than the source. We did not expect grids of size 20 to be large enough to contain enough relevant structures to generalize larger grids. Generalizing from small, numerically solvable grids to large grids is key for real world application. The computational cost of the dynamic simulations grows faster than linearly with the size of the grids, so computational time can be saved when training models on smaller networks or sections of real-sized grids. In comparison to the baselines, the generalization capabilities of new GNN models are much better.
Training on more data increases the performance of all models General Machine Learning convention assumes that larger dataset size allows to train larger models to higher performance. In this section, we investigate the influence of the size the training set to show the relevance of the larger datasets. We train the models the smaller dataset introduced in Nauck et al.] (2022) after specifically optimizing the learning rate for the analysis of the smaller training set. Our experiments show that training on less data results in lower performance, see Figure [5] Instead of peak values of R² of 82.49 %, we only obtain 74.77 % for dataset20 and only 83.92 % instead of 88.22 % for 100 The results of all models are given in Appendix A.9 Comparing the performance differences on dataset20 and dataset100. the improvements are larger for dataset20. A reasonable explanation is the total number of nodes used for the training.
dataset20
dataset100
90
90
80
80
Ar-bench
%
ArmaNet
in
GCNNet
60
SAGENet
50
x 50-
TAGNet
40
40
800
7000
800
7000
Number of grids in training set
Number of grids in training set
Figure 5: Comparison of the performance based on the size of the training set using 800 or 7,000 grids. Training our larger dataset improves performance all models. The 800 grids used for the training follow Nauck et al. 2022) and all models are evaluated on the newly introduced test set.
Predicting SNBS on a Texan power grid model using the previously trained models Using GNNs for SNBS prediction becomes feasible, if they can be trained on relatively simple datasets and still perform well on large, complex grids. As an example of large and complex grid, we use Texan power grid model and evaluate the models previously trained on dataset20 and d0t The benchmark models achieve surprisingly high performance with R² values (above 84 %) and GCN as sole exception, see the colums tr20ev Texas and tr 0r00eeeexas in Table [ Details on the poor performance of the GCN is given in Appendix A0 Hence, the approach of training models on grids which are smaller by more than one order of magnitude feasible. We want to emphasize that successful attempt of real-sized power grid should illustrate the general potential of this approach, but we still consider the hard evidence to be the generalization from 20 to 100. The performance is significantly better for the models trained 10t We hypothesize that the repetition of geometrical structures more prevalent in 1 is useful for even larger grids. Grids of size 20 might still be small to generalize large grids, but the size 100 might actually be sufficient for many applications.
3.3 IDENTIHICATION OF TROUBLEMAKERS
In this section we introduce further benchmark tasks, namely classify nodes into categories, stable nodes or troublemakers as defined in Section [2.] As noted in Equation (5). this target is essentially thresholded version of the maximum frequency deviation (MFD). Thus, we can either
8

---

datasets can directly train the classification task, we regress the MFD estimator and then threshold. Both strategies work and depending on the model different approaches seem to be best, see Table 2 The overall performance is very high, the prediction of TM is feasible. In the main section, we only show the F2-score and the better performing baselines. Appendix [A.11 contains the full results including different performance indicators. The GNNs ArmaNet and TAGNet outperform the baselines and especially TAGNet achieves a good performance.
Table 2: F2-score in % for TM prediction. The column type shows if classification (C) or regression
(R) is used for training. For regression, thresholding at is applied to compute the F2-score. For MLP2, training on d0taset100 was successful with only one of the five seeds. We report that result as over-estimation of a strong baseline.
model
type
1r20ev20
tr tr100ev100
tr20ev100
Texass
[rl tr100ev Texas
ArmaNet
C
86.85 1: 0.503
95.75 1 0.099
84.94 1: 4,050
82.43 +: 1,47
85.14 : 1,335
TAGNet
C
89.78 1: 0.079
96.36 :: 0.063
88.77
:: 0,309
82.78 2.12
92,00 1,585
ArmaNet
R
81.68 0.509
88.29
5,898
91.49
0.740
87.15 2.22
91.86 1.588
TAGNet
R
84.55 +
95.13 = 0.316
90.20
= 1.624
90.18 + 1.645
95.48 0.562
linreg
R
72.69
91.51
91.32
73.22
93.75
MLP1
74.33 a: 0.074
91.60 d: 0.006
81.60
#: 2.155
75.17
4: 2.837
41.96 16.52
ML.P2
R
74.38 + 0.06
91.61
77.46 77.000
68.89 + 5.248
93.75
3.4 BENEFITS OF USING GNNS TO PREDICT DYNAMIC STABILITY
As the experiments above show, GNNs are suitable for the analysis of the dynamic stability, both in terms of computational effort and predictive power. The performance exceeded our expectations. For SNBS and the best model (TAGNet) roughly 95 % of the estimations differ only by 0.1 from the target values for tr 100ev 100 and tr tr100evTexas. This corresponds to the banded region shown in Figure The threshold of 0.1 can be motivated by considering the distributions in Figure 2 because the modes are separated by 0.1. To achieve a similar accuracy by conducting dynamical simulations with Monte-Carlo sampling. 100 perturbations per node are needed, which is significant reduction to 10,000 perturbations we used for the generation of the datasets. With 100 perturbations, the computation for one grid of size 100 takes roughly 30 minutes and for the Texan grid 530 days using one CPU. Evaluating the GNNs takes less than one second per grid. Hence using GNNs is more than 1,800 times faster for grids of size 100 and 4,6 x 10 times faster for the synthetic Texan power grid. This demonstrates the potential of analyzing large number of different configurations using GNNs. Furthermore, we show the successful prediction of TM, which can motivate future applications.
4 CONCLUSION AND OUTLOOK
In this work, we analyze the probabilistic dynamic stability of synthetic power grids using GNNs. We generate new datasets that are 10 times larger than previous ones to enable the training of high capacity GNNs. Our benchmark results significantly improve over previous work and show that highly nonlinear SNBS can be predicted at surprisingly high accuracy by using more complex models and training be used for prediction on a real-world-sized Texan power grid model. The results indicate the potential benefits of machine learning in this important domain. We expect the datasets to attract attention by research groups working on complexity science and non-linear dynamics as well. Furthermore, we introduce a new method identify troublemakers in power grids and and show their successful prediction. Besides further improving the performance, future work might try to explain the decision process of GNNs to generate new insights on the relation of topology and stability. Encouraged by our results, we will continue to extend our datasets with increasingly more complex and realistic grids, aiming at real power grids. All data and code will be published upon publication (see Appendix I. Given sufficient computational resources, the code can easily be adapted to generate more training data, or to simulate grids of different sizes. For reviewing, the SNBS training is provided in zip container at: https:: /rivee..vv/e//////////// google com/ drive/E biders/I Siders/19hhqqupp 9hHqqup 1 KypyYnSSceee aat.eeeaaaaaaauuuuuaaaaa 7 11 wwspssaarsrris sharini ing] The open access enables the community develop new methods analyze future renewable power grids.
larger dataset. We show that the models trained

---

REFERENCES
A.R. Bergen and DJJ. Hill. A Structure Preserving Model for Power System Stability Analysis. IEEE Transactions on Power Apparatus and Systems, PAS-100(1);25-35, January 1981. ISSN 0018-9510. doi: 10. 1..1100 100/1P/A...18..116885 Conference Name: IEEE Transactions on Power Apparatus and Systems.
Jeff Bezanson, Alan Edelman, Stefan Karpinski, and Viral B. Shah. Julia: A fresh approach to numerical computing. SIAM Review, 2017. ISSN 00361445. doi: 10.1137/141000671. .eprint: 1411.1607.
Filippo Maria Bianchi, Daniele Grattarola, Lorenzo Livi, and Cesare Alippi. Graph Neural Net- works with Convolutional ARMA Filters. IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1-1, 2021. ISSN 1939-3539. doi: 10.1 2000TPPMM.222..300. Name: IEEE Transactions on Pattern Analysis and Machine Intelligence.
Conference
Adam Birchfield. ACTIVSg2000: 2000-bus synthetic grid on footprint of Texas, visited on Nov. 1, wnggrtamu... atgg....aa...e///eeeeerr..o.o...
2021. URL wtt.hhttsss//eeeetrrrrr.. ases/act.vvg20000
Adam B. Birchfield, Ti Xu, Kathleen M. Gegner, Komal S. Shetye, and Thomas J. Overbye. Grid Structural Characteristics as Validation Criteria for Synthetic Networks. IEEE Transactions on Power Systems, 32(4):3258-33 3265, July 2017. ISSN 1558-0679. doi: 10.1109/7PWRS:2206.26 6385. Conference Name: IEEE Transactions on Power Systems.
Valentin Bolz, Johannes Rueß, and Andreas Zell. Power Flow Approximation Based Graph Convolutional Networks. In 2019 18th IEEE International Conference On Machine Learning And Applications (ICMLA), pp. 1686, December 2019. doi: 10.109/ICMLL.2211.000027
Barbara Borkowska. Probabilistic Load Flow. IEEE Transactions on Power Apparatus and Systems, PAS-93(3):752-759 May 1974. ISSN 0018-9510. doi: 10. 19.11097 Conference Name: IEEE Transactions on Power Apparatus and Systems.
Michael M. Bronstein, Joan Bruna, Taco Cohen, and Petar Veličković. Geometrie Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges. arXiv:2 104. 13478 [cs, stat], May 2021. URL. arXiv: 2104.13478.
http:// /ttp://arxv..o www.o/gg//aooo.. 2abb//2104.1347
Charles D. Brummitt, Paul D. H. Hines, Ian Dobson, Cristopher Moore, and Raissa M. D Souza. Transdisciplinary electric power grid science. Proceedings of the National Academy of Sciences, 110630): July 2013. doi: 10. 10.1073/pnas. 13091511100 URL(nttps: /www. pnas (org/doo/full//// 1073/pnass 11091 0110111110
Publisher: Proceedings of the National
Academy of Sciences.
Brian Bush, Yuzhou Chen, Dorcas Ofori-Boatenn and Yulia R. Gel. Topological Machine Learning Methods for Power System Responses to Contingencies. Proceedings of the AAAI Conference on Artificial Intelligence,
May 2021. ISSN 2374-3468. URL https:
//ojs. www.aaaaa.gggnaaoaaaa wwwp/iggg.........o..oo. php/ AAAI/ Matiaa/artilleeoooo /vipw/17ww//11/ Number: 17.
Peter Christensen, Gert Karmisholt Andersen, Matthias Seidel, Sigrid Bolik, Sönke Engelken, Thyge Knueppel, Athanasios Krontiris, Klaus Wuerflinger, Thorsten Bülo, Jörg Jahn, Mario Ndreko, Saeed Salehi, Toannis Theologitis, Bernd Weise, Helge Urdal, Agusti Egea Alvarez, Andrew Roscoe, and Jens Fortmann. High Penetration of Power Electronic Interfaced Power Sources and the Potential Contribution of Grid Forming Converters. Brussels, January 2020.
otent
Balthazar Donon, Benjamin Donnot, Isabelle Guyon, and Antoine Marot. Graph Neural Solver for Power Systems. In Proceedings of the International Joint Conference on Neural Networks, volume 2019-July. Institute of Electrical and Electronics Engineers Inc., July 2019. ISBN 978-1- 7281 1-985-4. doi: 10.1 201p:/INNN..011..8..
10

---

Jian Du, Shanghang Zhang, Guanhang Wu, Jose M. F. Moura, and Soummya Kar. Topology Adap- tive Graph Convolutional Networks. October 2017. URL http:://xxxi..oo// org/ abs 1710
10370
eprint: 1710.10370.
Florian Dörfler, Michael Chertkov, and Francesco Bullo. Synchronization in complex oscillator networks and smart grids. Proceedings of the National Academy of Sciences of the United States of America, 2013. ISSN 00278424. doi: 10.1073/pmas.12221344110
Matthias Fey and Jan E Lenssen. FAST GRAPH REPRESHNTATION LEARNING WITH PY- TORCH GEOMETRIC. pp. 9, 2019. htt.httt://axiv..oom
993. 02428
C.B. Field, V. Barros, T.F. Stocker, D. Qin, DJJ. Dokken, K.L. Ebi, M.D. Mastrandrea, K.J. Mach, G.-K. Plattner, S.K. Allen, M. Tignor, and P.M. Midgley. Managing the Risks of Extreme Events and Disasters to Advance Climate Change Adaptation. A Special Report of Working Groups and II of the Intergovernmental Panel Climate Chan, 2012.
Fernando Gama, Ekaterina Tolstaya, and Alejandro Ribeiro. Graph Neural Networks for Decentral- ized Controllers. March 2020. URL 2003.10280
h0tp://bb//200...222 10280
.eprint:
William L. Hamilton. Graph Representation Learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, 14(3):1-159, September 2020. ISSN 1939-4608, 1939-4616. doi: 10.2
200/501045EDII01 Y200009A URL httppppp/wwwooo..m.lllaaaaaaa.aa
com/
William L. Hamilton, Rex Ying, and Jure Leskovec. Inductive representation learning on large
graphs. In Advances in Neural Information Processing Systems, 2017. ISSN: 10495258 _eprint:
1706.02216.
Frank Hellmann, Paul Schultz, Carsten Grabow, Jobst Heitzig, and Jürgen Kurths. Survivability of Deterministic Dynamical Systems. Scientific Reports, 6(1):29654, July 2016. ISSN 2045-2322 doi: 10. 1038//srep22954. URL (ttts::::: //w////wwwwwww nature cww/arttcles//reeeeeette Number: 1 Publisher: Nature Publishing Group.
Frank Hellmann, Paul Schultz, Patrycja Jaros, Roman Levchenko, Tomasz Kapitaniak, Jürgen Kurths, and Yuri Maistrenko. Network-induced multistability through lossy coupling and ex- otic solitary states. Nature Communicationss, 11(1):592, January 2020. ISSN 2041-1723. doi:
10.1038/s4 467-0220 124---4777-177-7 URL https: www. nature. com wttaaarrccessssssss...ss C20 14417-7
Number: Publisher: Nature Publishing Group.
Bukyoung Jhun, Hoyun Choi, Yongsun Lee, Jongshin Lee, Cook Hyun Kim, and Kahng, Predic- tion and mitigation of nonlocal cascading failures using graph neural networks, July 2022. URL
http://ase http:://axvi..og/abbsssss. www./rrgss....ooooo...e 2208 00133 arXiv:2208.001 00133 [physics].
Cheolmin Kim, Kibaek Kim, Prasanna Balaprakash, and Mihai Anitescu. Graph Convolutional Neural Networks for Optimal Load Shedding under Line Contingency. In 2019 IEEE Power Energy Society General Meeting (PESGM), pp. 1-5, August 2019, doi: 10.1 1090PES0SS0 989794889468 ISSN: 1944-9933.
Thomas N. Kipf and Max Welling. Semi-Supervised Classification with Graph Convolutional Net- works. 16X9v:1609.022 02907 [cs, stat], February 2017. URL http::/tp//et http::://aax...oorgaaa////////// 02907
arXiv: 1609.02907.
Yoshiki Kuramoto. Self-entrainment of a population of coupled non-lineat oscillators. Mathematical Problems in Theoretical Physics, 39:420 422, January 1975. doi: 10. 10.1100//F60013330 URL
https: //ui adsabs hawvard. edu abs /19751/ ....................... 420K ADS Bibcode:
1975LNP... D..........9...20000 .9..KK..
Yoshiki Kuramoto. Self-entrainment of population of coupled non-linear oscillators. In Interna- tional Symposium on Mathematical Problems in Theoretical Physics. 2005. doi: 10.1007/bíb001
3365.
11

---

Sebastian Liemann, Lia Strenge, Paul Schultz, Holm Hinners, Johannis Porst, Marcel Sarstedt, and Frank Hellmann. Probabilistic Stability Assessment for Active Distribution Grids. In 2021 IEEE Madrid PowerTech, pp. 1-6, June 2021. doi: 10.1 1tL.100000weTTT124444222220.1.00 249499998.
Michael Lindner, Lucas Lincoln, Fenja Drauschke, Julia M. Koulen, Hans Würfel, Anton Plietzsch, and Frank Hellmann. NetworkDyynmmisssj--CCooo
and simulating complex networks in Julia. Chaos: An Interdisciplinary Journal of Nonlinear Science, 316)):06313,, June 2021. ISSN 1054-1500. doi: 10. ww..1066/.005550
063/5.0051380
Publisher: American Institute of Physics.
Yuxiao Liu, Ning Zhang. Dan Wu, Audun Botterud, Rui Yao, and Chongging Kang. Guiding Cas- cading Failure Search with Interpretable Graph Convolutional Network, January 2020. URL
http:://axxaa www.g//ab///000/ 11553 arXiv:2001.11553 [cs, eess].
Yuxiao Liu, Ning Zhang, Dan Wu, Audun Botterud, Rui Yao, and Chongqing Kang, Searching for Critical Power System Cascading Failures with Graph Convolutional Network. IEEE Transac- fions Control of Network Systems, pp. 1-1, 2021. ISSN 2325-5870. doi: 10. 10.1099TCNS.022 1.3063333. Conference Name: IEEE Transactions on Control of Network Systems.
Zhao Liu and Ziang Zhang. Quantifying transient stability of generators by basin stability and Kuramoto-like models. In 2017 North American Power Symposium (NAPS), pp. 1-6, September 2017. doi: 10.1109/NAPS.. SAA.PP...01.... 2017.
Zhao Liu, Xi He. Zhenhuan Ding, and Ziang Zhang. A Basin Stability Based Metric for Ranking the Transient Stability Generators. IEEE Transactions on Industrial Informatics, 15(3): 1450- 1459, March 2019. ISSN 1941-0050. doi: 10. 11109/11.001.28466777 Conference Name: IEEE Transactions on Industrial Informatics.
Peter J. Menck, Jobst Heitzig, Jürgen Kurths, and Hans Joachim Schellnhuber. How dead ends undermine power grid stability. Nature Communications, 5(1):3969, June 2014. ISSN 2041- 1723. doi: 10. 10.1038/ccomms496) URLiattss: wwwp///ww..../ www.nnatree.... com wwt.m//arccleee/cc.oo.oo ms4969 Number: Publisher: Nature Publishing Group.
Federico Milano, Florian Dörfler, Gabriela Hug, David Hill, and Gregor Verbič. Foundations and Challenges of Low-Inertia Systems (Invited Paper). In 2018 Power Systems Computation Conference (PSCC), pp. 1-25, June 2018.
doi: 1229999/CC...018...8888
George S. Misyris, Andreas Venzke, and Spyros Chatzivasileiadis. Physics- Neural Net- works for Power Systems. 2020 IEEE Power Energy Society General Meeting (PESGM), pp- 1-5, August 2020. doi: 10. 101109/FSSCM44554220000000
ISSN: 1944-9933.
Philipp Moritz, Robert Nishihara, Stephanie Wang, Alexey Tumanov, Richard Liaw, Eric Liang, Melih Elibol, Zongheng Yang, William Paul, Michael I. Jordan, and Ion Stoica. Ray: A Dis- tributed Framework for Emerging Al Applications. arXiv:1712.05888 [cs, stat], September 2018.
URL http:://axix org/
05889
arXiv: 1712.05889.
Adilson E. Motter, Seth A. Myers, Marian Anghel, and Takashi Nishikawa. Spontaneous synchrony in power-grid networks. Nature Physics, 131):191-197, March 2013. ISSN 1745-2481. doi: 10. 1..0088/phhs25355. URL https: //t////wwww nat mature.... com/ http//artceesssnssseeeee Number: 3 Publisher: Nature Publishing Group.
Christian Nauck, Michael Lindner, Konstantin Schürholt, Haoming Zhang, Paul Schultz, Jürgen Kurths, Ingrid Isenhardt, and Frank Hellmann. Predicting basin stability of power grids using graph neural networks. New Journal of Physics, 2022. ISSN 1367-2630. doi: 10. 10.1088/1367-26 0/ac54c9. URL a954c9
J. Nitzbon, Schultz, J. Heitzig, J. Kurths, and F. Hellmann. Deciphering the imprint of topology nonlinear dynamical network stability. New Journal of Physics, 1933)033302, March 2017. ISSN 1367-2630. doi: 10. 10.100881367-200 1110///166-2633//662222.1 URL //ottp htt.ttpps//oll.oo//11.....0 wtt.://.og/...11////////// 67 67-2630/aa633 Publisher: IOP Publishing.
12

---

Damian Owerko, Fernando Gama, and Alejandro Ribeiro. Optimal Power Flow Using Graph Neural Networks. In ICASSP 2020 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5930-9900000 9930-5934, May 2020. doi: 10.1009ICAASSPP0777.2000009 3140. ISSN: 2379-190X.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary Vetoto, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang. Junjie Bai, and Soumith Chintala. PyTorch: An Imperative Style, High-Performance
Deep Learning Library. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d' d'teextuuoesiinle
Alché-Buc, E. Fox, and R. Garnett (eds.), Advances Neural Information Processing Systems 32. pp. 8024-8035, Curran Associates, Inc., 2019, URL http: wwwpppp...uuuaa.....p.p....pppae per/ 9015 pytorch an imperative style high performance deep- lea ening-libron pdf.
Anton Plietzsch, Raphael Kogler, Sabine Auer, Julia Merino, Asier Gil-de Muro, Jan Liße, Christina Vogel, and Frank Hellmann. PowerDynamies,j] An experimentally validated open-source pack- age for the dynamical analysis of power grids. vrXiv:2101.02100 [cs, eess], January 2021. URL
aattp://arxis www.abs/////ooooooo.ot...tt..a.r / 2101. 02103
arXiv: 2101.02103.
Hans-Otto Pörtner, D.C. Roberts, M. Tignor, E.S. Poloczanska, K. Mintenbeck, A. Alegría, M. Craig, S. Langdorf, S. Löschke, V. Möller, A. Okem, and B. Rama. IPCC, 2022: Climate Change 2022: Impacts, Adaptation, and Vulnerability, Contribution of Working Group II the Sixth Assessment Report of the Intergovernmental Panel on Climate Change, 2022.
Christopher Rackauckas and Qing Nie. Differentiallqquationsss A Performant and Feature-Rich Ecosystem for Solving Differential Equations in Julia. Journal of Open Research Software 5(1): 15, May 2017. ISSN 2049-9647. doi: 10.5334/jors.
URL http: http:///ppenn http://ppp//eaaaeeeeeeeeeeeeee/e
software www.are.eettae jnl com awtaaaatiiclseee http:///eeaaaaaaaoooooaaraaaaaa. 5334/ 151/ Ubiquity Press.
Number: Publisher:
Nicolas Retiére, Dinh True Ha, and Jea Caputo. Spectral Graph Analysis of the Geometry of Power Flows in Transmission Networks. IEEE Systems Journal, 14(2):2736-2747, June 2020. Conference
ISSN 1937-9234. doi: 10.1 00.1109//YSST2221992222838
Name: IEEE Systems Journal.
Francisco A. Rodrigues, Thomas K. DM. Peron, Peng Ji, and Jürgen Kurths. The Kuramoto model in complex networks. Physics Reports, 610:1-98, January 2016. ISSN 0370-1573, doi: 10.1016/ 20.5. URL http:::tp////mm www. httpc//dereee..ccco////////aaaaa 6//11//0077015731500040
Martin Rohden, Andreas Sorge, Mare Timme, and Dirk Witthaut. Self-Organized Synchronization
in Decentralized Power Grids. Physical Review Letters, 109(6)006410,, August 2012. doi: 10.1
103/PhysRevLett. 10w..ee.........001011100... 064101. WRRLhttps. //link/lnnkk aps. wttp://oo/////....1/.... 1103///
109.tt..19.0041100
Publisher: American Physical Society.
Paul Schultz. http:://wttiaoooo/////k/oooossse
2020. URL https://gith
ub. wwt.ccomPPIK.. Mwlle-CCNNNe//yye hwt.c////...---.ceeo//yyyyttttte
Paul Schultz, Jobst Heitzig, and Jürgen Kurths. Detours around basin stability in power networks.
New Journal of Physics, 16(12):125001, December 2014a. ISSN 1367-2630. doi: 10. 1088/1367
-2630/1//2/1150011 URL htt.h:tttt/stoottt wtt.htts:tttthtttttte 7-2630/16/12/12551
ence wtt..///....cceeeeeeeeeeeo 10888/10.111111110 088
Paul Schultz, Jobst Heitzig, and Jürgen Kurths. A random growth model for power grids and other
spatially embedded infrastructure networks. The European Physical Journal Special Topics, 223 (12):2593-2610, October 2014b. ISSN 1951-6401, doi: 10.1 00011440/ppxte2210022229 URL 022799-6
https://doii .ttp//oorgggg0 10..
Benjamin Schäfer, Thiemo Pesch, Debsankha Manik, Julian Gollenstede, Guosong Lin, Hans- Peter Beck, Dirk Witthaut, and Mare Timme. Understanding Braess' Paradox in power grids. Nature Communications. 13(1):5396, September 2022. ISSN 2041-1723. doi: 10.1038/s41467-02 -32917-6. URL tatpps:/ www... nature com ott.a//e...///tiiieee/////oo11-. Number: 1 Publisher: Nature Publishing Group.
13

---

Dawei Wang, Kedi Zheng. Qixin Chen, Gang Luo, and Xuan Zhang. Probabilistic Power Flow Solu- tion with Graph Convolutional Network. 2020 IEEE PES Innovative Smart Grid Technologies Europe (ISGT-Enal JSGT-EErope), pp. 650,654, October 2020. doi: 10.1009ISCTT-E100447291..108 48786.
Dirk Witthaut and Mare Timme. Braess's paradox in oscillator networks, desynchronization and power outage. New Journal of Physies, 2012. ISSN 13672630. doi: 10.1088/1367--2500444888 83036.
Dirk Witthaut, Frank Hellmann, Jürgen Kurths, Stefan Kettemann, Hildegard Meyer-Ortmanns. and Mare Timme. Collective nonlinear dynamics and self-organization in decentralized power grids. Reviews of Modern Physics, 94(1):015005, February 2022. doi: 10.1 10.1005/ 5005. URL
/tt.///in....// aps. American Physical Society.
Publisher:
Yin Yu, Xinyuan Jiang, Daning Huang, and Yan Li. PIDGeuN: Graph Neural Network-Enabled Transient Dynamics Prediction of Networked Microgrids Through Full-Field Measurement. arXiv:2204. 08557 [cs, eess]. April 2022. URL http:/ /wtp///aivvvv//aa////i//eeeeaaav org/ abs 2ww./220...00 08557 arXiv: 2204.08557.
14

---

A APPENDIX
This section includes additional information to generate the datasets, reproduce the presented results and additional results that are not already shown the main section.
We start by providing information the availability of the datasets and the used software, before providing more details regarding the modeling for the dataset generations. Subsequently, details regarding the training, the evaluation well as the hyperparameter study are provided. Afterwards additional results are shown.
A.1 AVAILABILITY OF THE DATASETS
The new datasets and full code for the training, evaluation and generation of the figures will be published upon publication on Zenodo and GitHub and it will be licensed under CC-BY 4.0 to enable the community to contribute to this challenge.
A.2 SOFTWARE FOR GENERATING THE DATASETS
Julia is used for the simulations Bezanson et al. 2017) and the dynamic simulations rely the package DifferentialEquations/] jI Rackauckas & Nie (2017). For simulating more realistic power
grids in future work we recommend the additional use of NetworkDynamies] Lindner et al. (2021)
and PowerDYynamics.j Plietzsch et al. (002))
A.3 SOFTWARE FOR TRAINING
The training is implemented in Pytorch (Paszke et al. 2019). For the graph handling and graph convolutional layers we rely on the additional library Porch Geometric (Fey Lenssen 2019). We use the SGDDoptimizer and as loss function we use the mean squared error Furthermore ray (Mrrizz et al. 2018 is used for parallelizing the hyperparameter study.
A.4 MODELLING DETAILS OF GENERATING THE DATASETES
In the detail of the modeling and the size of the dataset we attempt to strike a balance between relevance real-world applications, computational tractability and conceptual simplicity. Therefore we employ the following criteria: (i) generate synthetic network topologies that mimic real world power grids; (ii) model the main dynamics of self-organised power flow and synchronization; (iii) minimize statistical and numerical errors with highly accurate simulations; (iv) to study out-of distribution tasks and scale effects, consider grid sizes of different orders of magnitude.
The most important simplifications in comparison to real- real-world power grids are homogeneous edges, fixed magnitudes of sources/sinks and modeling all nodes by the swing equation. contrast to our modeling, real power grid lines have different properties and there are more complex models for generators and loads. However, previous studies have shown that many interesting observations are still possible under our assumptions Nitzbon et al. (2017).
To investigate different topological properties of differently sized grids, we generate two datasets with either 20 100 nodes per grid, referred to as dataset20 and dataset100. To enable the training of complex models, both datasets consist of 10,000 graphs. Additionally, probabilistic dynamic stability values of synthetic model of the Texan power grids are provided for evaluation purposes.
Modeling of the power grids This section covers the precise modeling of power grids used for the dataset generation and may be skipped. To generate realistic topologies we use the package et al.) [014b) [Schultz] 2020). The sources and sinks are assigned randomly. For the dynamic simulations all nodes are represented by the 2nd gud-order-Kuramoo model (Kuramoto| 2005) Rodrigues et 2016), which is also called swing equation, see eq. { Using homogeneous coupling strength (K) can be interpreted as considering power grids that only have one type of power line and comparable distances between all nodes. This assumption is justifiable,
Syntheti ciet workNettorrssSShhulii
correspponds MSELoss in Pytorch
15

---

Figure 6: Identification of troublemakers based on trajectories. There three spaces: blue for the space of perturbations, green for the safe-space and purple for the trouble-space. The dashed trajectory stays within the initial space of the perturbations, the dotted line leaves that space, but stays below the trouble-thresolld and the solid line represents rrouble-maker-trajcctooyy
because in real power grids longer lines are built stronger, the actual coupling does not scale as much with the length.
To estimate SNBS rely the approach presented in Nauck et al. (2022): "[F]or every node in a graph, M = 10, 000 samples of perturbations per node are constructed by sampling phase
and frequency deviation from a uniform distribution with (ø) σ) E [-x,, , π] x [-15, 15) and adding them the synchronized state. Each such single-node perturbation serves as an initial condition of a dynamic simulation of our power grid model, [cf. Equation [ At t = 500 the integration
terminated and the outcome of the Bernoulli trial is derived from the final state. simulation
outcome is referred to as stable if at all nodes Φ; < 0.1. Otherwise it is referred to unstable. The classification threshold of 0.1 is chosen accounting for minor deviations due numerical noise and slow convergence rates within finite time-horizon."
To ensure the reliability of our results we try to minimize numerical and statistical errors: For the dynamical simulations a higher order Runge-Kurte Kutta methods with adaptive time stepping and low error tolerances used. For the Monte-Carlo sampling, 10,000 simulations per node result in standard errors for our probabilistic measure SNBS of less than +0.01.
Furthermore, we want to provide some numbers regarding the simulation time. The computation of single perturbation in of dataset20 takes .056 seconds, in case of dataset100 0. 189 seconds and in case of the Texan power grid 239.97 seconds.
A.5 IDENTIFICATION OF TROUBLAMAKERS
The idea illustrated in Figure d The critical threshold is motivated by the real-world power grids. When certain operational bounds are violated, machines in the system switch of, potentially triggering failure cascades and large blackouts. Thus, there is a discontinuous difference between perturbations that attain maximum deviation above the critical threshold and those that don't. The maximum limits for frequency deviations for the European grid are - -Hz or -3Hz. For our model we chose the symmetric critical threshold ~ 2.4Hz, corresponding
maximum deviation of angular velocity of (0) 15, both for simplicity and comparability previous work.
Selection of pertur for the identification of troublemakers If perturbations are amplified by more than a factor of 6, the desired space is left and we consider the trajectories troublesome. that occurs for at least one perturbation applied at node, we consider that node a troublemaker, because such strong amplification is danger for safe grid operation. For the analysis of TM, only consider perturbations within (Ø) E 2.5.. 2.5]. This reduces the minimum number of analyzed perturbations per node to 1,595 for dataset20, 1,569 for 1 and 129 for the Texan power grid. As mentioned in Section 2.1 the probability of rare events depends on the estimator and on the number of samples used for the estimation. As a consequence for dataset20 and dataset (0) the rare even is 7 0.0013 and for the Texan grid 7 0.015. These values derived from the Agresti-Cone Coull approximation of binomial confidence interval.
A.6 EVALUATION THE PERFORMANCE USING DIFFERENT METRICS
mse(tm mse(tmeen..)) where mse denotes the mean squared error, y the output of the model, t the target value and mean the mean of all considered targets of the test dataset. R² captures the mean square error relative to a null model that predicts the mean of the test-dataset for all points. The -core is used to measure the portion
For the regression, we use the coefficient of determination: R²
msc(y.4)
16

---

of explained variance in dataset. By design, model that predicts the mean of SNBS per grids has R 0.
For the classification, use F2-score which is based recall and precision. The recall is defined
by: Recall TP4FN where TP denotes true positives and FN false negatives. The precision
TP
TP is defined by: precision TP4FP'
This leads to the F2-score which is defined by: F-
5xprecision Xrecall
4xprecision-Treall"
A.7 HYPERPARAMETER OPTIMIZATION
We conduct hyperparameter studies in two steps. First, we optimize model properties such as the number of layers and channels as well as layer-specific parameters e.g. the number of stacks and internal layers case of ArmaNets. For this optimization we use dataset20 and the SNBS task only. For all models we investigated the influence of different numbers of layers and the numbers of channel between multiple layers. We limit the model size to just above four million parameters, so we did not investigate the full presented space, but limited for example the number of channels when adding more layers. The resulting models have the following properties: ArmaNet has 3 layers and 189, .488 parameters. GCNNet has layers and 523,020 parameters. SAGENet has layers and 728,869 parameters. TAGNet has 13 layers and 415,320 parameters.
Afterwards optimize the learning rate, batch size and scheduler of the best models for dataset20 and 100 and the tasks SNBS/TM separately. Hence, our models are not optimized to perform well the out-of distribution task. The best model from (Nauck al. (022) referred bench, is used as another baseline. It is a GNN model consisting of 1,050 parameters and based on 2 Arma-layers. The only adjustment to that model is the removal of the fully connected layer after the second Arma-Convolution and before applying the Sigmoid-layer, which improves the training.
Ar-
A.8 DETAILS OF THE TRAINING OF THE BENCHMARK MODELS
To reproduce the obtained results, more information regarding the training is provided in this section. Detailed information on the training as well as the computation time is shown in Table 3 case of dataset20, a scheduler is not applied, in case of dataset schedulers are used for Ar-bench The default validation and test set batch size 150.
(s GCNNet Cicducel.OOnPlateauu The validation and test batchsize for Ar-bench and ArmaNet3 is 500 in case of dataset20 and 100 for 000 The number of trained epochs differs, because the training is terminated in case of significant overfitting. Furthermore, different batch sizes have significant impact the simulation time. Most of the training success occurs within the first 100 epochs, afterwards the improvements relatively small.
Table 3: Properties of training models and regarding the training time, we train 5 seeds in parallel using one Vidia V100.
name dataset
number of epochs
training time
train batch size
learning rate
20
100
20
100
20
100
20
100
(hours)
(days)
Ar-
1,000
800
26
4
200
12
0.914
.300
bench
ArmaNet
1,500
1,000
46
6
228
27
3.00
3.00
GCNNet
1,000
1000
29
5
19
79
.307
.286
SAGENet 300
1000
9
5
19
16
1.10
1.23
TAGNet 400
800
11
4
52
52
0.193
.483
A.9 DETAILED RESULTS OF TRAINING ON A SMALLER DATASET
To investigate the influence available training data and to connect with previous work, we train all models on only 800 grids, from Nauck et al. (2022) The results shown in Table 4
17

---

Table 4: Performance after training on smaller training set. All models are trained on the same 800 grids as in Nauck et al. 22022) but evaluated on the newly introduced test set. The results are represented by R2 score
model
tr20ev20
tr100ev100
1120ev100
Ar-bench 46.54 2.788
59.73
31.75 1.204
ArmaNet
70.35 + 1,226
83,92 1 0,263
55.84 1 0.598
GCNNet
49.59
61.18
36.08
SAGENet
50.15 + 0.255
60.98 0.279
39.89 0,089
TAGNet
74.77
82.21
60.31
A. POOR PERFORMANCE OF GCN WHEN APPLYING TO THE TEXAN POWER GRID
The GCN model able predict the dynamic stability for the Texan power grid. To understand this behaviour, we compare it to the ArmaNet model at the out-of offtriibibbtutionn task from dataset20 and dataset100 to predict SNBS of the Texan power grid. The seatter plots are shown in Figure 7 We can clearly see that the model is not able to predict lower values of SNBS correctly. The limited output of the GCNNet results in bad performance in case of the distribution of the Texan power grid that has three modes. As consequence, model that predicts the mean of the distribution would achieve better performance. Furthermore, we provide the scatter plots of the GCNNet for the three tasks dataset20, d0taset100 and tr20ev 100 in Figure g that can be compared to Figure
Arma trx0svTexas
Arma 11100evTexas
1.0
true 5NB5
true SNBS
0.4
0.2
0.2
0.2
0.8
1.0
0.6
0.8
1.0
model prediction
model prediction
GCN tr20ev Texas
GCN tr100evTeexas
1.0
1.0
0.8
true SNB5
true SNBS
0.2
0.8
model prediction
model prediction
Figure 7: SNBS over predicted output of the Arma and GCN models for the out-of-distributiin task to predict SNBS of the Texan power grid. The diagonal represents perfect model (R² 1), the banded region indicates prediction errors < 0.1. To account for the small number of nodes, lower transparency is used comparison Figures and
A.11 PERFORMANCE IDENTIFYING TROUBLEMAKERS
In the following sections we show the full results of predicting the troublemakers using classification and regression setup. We use the previously introduced GNN models (see section 3.1). For each regression and classification task we conduct another hyperparameter study to optimize the learning rates. The same inputs are used: adjacency matrix A, nodal power input P.
18

---

tr20ev20
tr100ev100
tr20ev100
1.0
1.0
true SNBS
true SNBS
SNBS
0.2
0.4
0.8
1.0
0.2
0.4
0.6
1.0
0.2
0.6
0.8
1.0
model prediction
model prediction
model prediction
SNBS over predicted output of the GCNNet model for tr20ev20, 1r 100 and The diagonal represents perfect model (R² = 1), the band indicates the region for accurate predictions based on a threshold of 0.1.
Figure 8:
tr20ev 100.
A.11.1 CLASSIFICATION SETUP
For the classification setup, we report the F2-score in Table 3able and the recall Table 6 The GNNs outperform the baselines in all tasks and especially TAGNet achieves good performance. TAGNet achieves recalls of more than 93 % throughout all tasks, including the nut-of-distribution generaliza- tions, while still keeping F2-score above 82 %. For identifying troublemakers, thresholded variant of the "semi-anallyic lower bound for survivability (Hellmann et al. 2016 is used as further baseline. This bound can be directly computed from the input data and hence requires no training.
Table TM prediction: F2-score %. For MLP2, training on 1 was successful with only one of the five seeds. We report that result over-estimation of strong baseline. (*) Since the semi-analytic baseline does not require training, its performance is directly evaluated on the test set.
model
tr20ev20
tr100ev100
tr20ev
Texass
tr 1r100evTexas
Ar-bench
74.31 = 0330
88.29 = 0.311
77.08 0.999
78.49 = 0790
83.33 + 1856
ArmaNet
86.85 503
95.75 0,09
84.94 4.000
82.43 1.747
85.14 + 1.335
TAGNet
89.78
96.36
88.77
82.78
92.00
log
44.74
73.47
5.59
21.66
55.81
MLP1
58.06 +:
79.61 = 0.001
72.68 6.142
49.52 :: 9.990
86.02 :: 0.000
MLP2
69.43 +: 0724
91.60
41.11 7.200
32.16 #: 12:338
73.22
semi-analytic
22.86*
38.53
38.53
15.73*
15.73*
Table 6: TM prediction: recall in %. case of MLP2, the training on dataset100 was successful with only two of the five seeds. We only report that result over-estimation of strong baseline.
model
tr20
tr0
ev20
ev100
CV evTexas
cv 100
evTexas
Ar-bench
86.30 0.727
92.56 :: 0.798
83.33 1: 0.933
98.69 : 0.654
98.04 0.000
ArmaNet
88,5 38,85... 1.019
96.85
:: 0.272
86.74
1: 5455
90.20
:: 2.61
98.69 1: 0.651
TAGNet
93.84 0.039
97.16 +: 0.120
96.10
1: 0.500
99.35
:: 0.654
100.00
0.000
logreg
86.69
91.82
95.32
89.04
94.12
MLP1
76.44 0,312
90.70
68.98 6,874
44,44 9.628
94.12 0.000
MLP2
75.85 0,513
89.86
36.08 + 6.02
28, 10 11.53
68.63
semi-analytic
92.70*
97.77*
98,04
97.77*
98,04*
A.11.2 REGRESSION SETUP
Besides introducing two classes for predicting troublemakers, it is also possible directly predict the maximum frequency deviation per node (fig. ( The results are shown in table The predictions of the regression be complemented by applying thresholding afterwards categorize nodes
19

---

troublemakers. After applying the thresholding, the results can again be evaluated using F2-score (table and recall table
Table 7: Results of predicting maxiumum frequency devations represented by R² score in %. In case of MIP2, the training d0taset100 was successful with only two of the five seeds. We only report the best result as over-estimation of a strong baseline.
model
tr20cv20
tr 1r100cv100
tr20cv
tr20cv Texas
tr 1000eeeeexss 82.45 + 1.30
Ar-bench
87.45+ 0.12
92.73 + 0.132
89.21 0.226
77.76: 0614
ArmaNet
96.59 + 0074
97.70# 0,066
95.70 0.150
86.73 1482
90.70 12266
TAGNet
96.30
97.99
93.68#
81.88
84.81 0.90
linreg
83.87
87.77
86.60
80.85
78.12
MLP1
90.55
92.54
82.21
54,64
$2.71 +
MLP2
89.95 # 0.144
92.30
81,85 1.444
60.83 # 7.764
83.82
Table 8: Results of predicting troublemakers using regression and thresholding represented by F2- score in %. In case of MLP2, the training 00 was successful with only of the five seeds. We only report the best result as over-estimation of strong baseline.
model
tr20ev20
tr100cv100
tr20cv 100
1r20evTexaa
tr t00eexTesxas
Ar-bench
44.49 + 0018
78.66 = 1.69
58.28 1,302
855.24 # 0.176
89.15 11.10
ArmaNet
81.68 0.9
88.29 5.00
91.49 0,740
87.15 2922
91.86 + 1.068
TAGNet
84.55
95.13
90.20
90.18
95.48
linreg
72.69
91.51
91.32
73.22
93.75
MIP1
74.33 +
91.60
81,60
75.17 +
41.96 :
MI.P2
74.38 + 0.066
91.61
77.46 27.020
68.89 + 5.248
93.75
Table 9: Results of predicting troublemakers using regression and thresholding represented by recall in %. In case of MLP2, the training on dataset100 was successful with only two of the five seeds. We only report the best result over-estimation of strong baseline.
model
tr20ev20
tr100ev100
tr20ev 100
tr20ev
tree0aeTexas
Ar-bench
39.85 1.792
75.28 1.892
53.29 1.476
87.58 4 0.54
93.46 1207
ArmaNet
78.80 0557
86. 37 16
91.60 1.033
88.89 3,458
95.42 0644
TAGNet
82.65 +: 1.337
94.47
91.66
94.77 + 1.729
96.73 0.54
linreg
70.55
89.75
68.63
89.54
94.12
MLP1
69.89 +: 0.079
89.87 :: 0.011
79.80 3,995
86.93 17.
47.06 24.49
MLP2
69.93 1: 0.039
89.88
73.68 7.778
64.05 1: 5.584
92
20


Reviews:
[{"review_id": "Vvro8eEPoL-", "reviewer": "Reviewer_wgck", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents a dataset containing synthetic models of power grids combined with the statistical results of dynamic simulations (quantifying single-node basin stability and survivability - abbreviated SNBS and SURV). The authors train a baseline GNN to show initial efficacy of the task of predicting SNBS and SURV, as well as of transferring the learned model to larger power grids.", "summary_of_the_review": "The proposed dataset characterizes an important problem. However, the writeup does not clearly define the dataset/benchmark task or the baseline method. In addition, it would likely make more sense to present this as a software package for generating data, _alongside_ a fixed dataset, rather than just as a fixed dataset. \n\nIf the authors are able to make significant revisions to the writeup during the revision period, I would potentially be willing to significantly increase my score, given that my issues are (likely) with the presentation rather than the content of the work (at least, based on what I'm able to assess from the current writeup). In general, I think dataset papers are extremely valuable, particularly those dealing with climate/energy problems, but unfortunately the current writeup is not of high enough quality for this reviewer to recommend acceptance without such significant revisions.", "strength_and_weaknesses": "Strengths:\n* The proposed dataset characterizes an important problem. Specifically, the problem of dynamic stability is indeed increasingly important to address as power grids integrate larger proportions of time-varying renewable energy.\n\nWeaknesses:\n* The dataset structure is not clearly described, and the benchmark task (what are the inputs, what are the outputs, and what is the metric of success) is not cleanly defined. This information is implicit within the text and citations, but needs to be laid out much more clearly and explicitly to enable understanding of the specific benchmark presented - e.g., I would have expected to see some of the relevant equations laid out in the paper.\n* The setup of the baseline GNNs is also not clearly enough described. For instance, the text describes that the GNNs are given \"an adjacency matrix representing the topology of power grids and a binary feature vector representing sources and sinks,\" without additional detail - however, for a dataset paper, those inputs should be clearly spelled out.\n* Given that this is a machine learning audience as opposed to a power systems audience, the introduction needs to be made much more accessible. For instance, terms/phrases such as \"have less inertia,\" \"contingencies,\" and \"self-organized synchronization mechanism\" may not be immediately accessible to an ML audience. In addition, the word \"transformers\" should likely be clarified as \"power transformers,\" due to the different default meaning of this term in the deep learning literature.\n* Since all the data is synthetically generated, it is not clear to me why this is presented as only a static dataset, rather than the data generation code also being shared. See, e.g., [1] for an example of a dataset paper that does it this way.\n\n[1] Joswig-Jones, Trager, Kyri Baker, and Ahmed S. Zamzam. \"OPF-Learn: An open-source framework for creating representative AC optimal power flow datasets.\" 2022 IEEE Power & Energy Society Innovative Smart Grid Technologies Conference (ISGT). IEEE, 2022.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "OCaotjw6HB", "reviewer": "Reviewer_iZuM", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper introduces a new dataset of probabilistic dynamic stability of synthetic power grids which is much larger than the existing datasets. Moreover, it also includes a Texan power grid model alongside with the dataset. Second, this paper also proposes a new method to identify the nodes which could be the one leading to the unstability. Finally, the papers evaluate several GNN baselines on the proposed benchmark and made a complete analysis. In the experimental results, the paper shows the potential to use the GNN model trained with the offline data to conduct the prediction on the real-world power grid.", "summary_of_the_review": "This paper provides a new benchmark dataset about a significant real-world problem and it has shown the evaluation with several baselines which could provide some lessons to the following people. The background of the problem is sufficient. The description of the identification of trouble-maker nodes needs to be clarified, which may change my decision.", "strength_and_weaknesses": "## Strength:\n1. This paper is generally well written and the background is sufficient to let the people who is not an expert in power systems understand the task.\n2. The details of generating data are provided.\n3. The figures are concise to show the ideas.\n4. The anlysis of experimental results are sufficient and solid.\n\n## Weaknesses:\n1. No significant novelty appeared in this paper.\n2. The method to identify the trouble-maker nodes is not clearly described at least from the main part of paper. I suggest the authors can merge the related contents in the appendix and rewrite it using compact words.\n3. The authors claim that the trouble-maker identification is a classification task. However, in my view it is just a criterion rather than a classification task that need the ML model to conduct.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "6eyPXZVaMg", "reviewer": "Reviewer_SqnB", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors introduce new datasets of dynamic stability of synthetic power grids. \nThey show that large Graph Neural Networks GNNs outperform GNNs from previous work at predicting single-node basin stability.They demonstrate that GNNs can be used to identify trouble maker-nodes in the power grids and show that GNNs trained on small grids can perform accurately on a large synthetic Texan power grid model.", "summary_of_the_review": "I am not very confident, and overall enjoyed the paper. \nThe new dataset is a big contribution, but I would like some clarification on the tasks that are used to evaluate the model (trouble node classification, SNBS prediction) as I'm not sure they fully support the claims made in the paper. ", "strength_and_weaknesses": "I am unfortunately not super knowledgeable about power-grids but enjoyed reading this interesting paper. \n\nI believe that the strength of this paper lies in the introduction of a new dataset of higher complexity than previous existing datasets. I think this is very important for evaluating and training models.\nThe other strength of the proposed model is the computational efficiency and scalability (1,800 times faster for grids of size 100 and 4,6 ×107 times faster for the synthetic Texan power grid). \n\nThe fact that GNNs can generalize from a small dataset to a bigger one is very compelling, but the datasets are very similar, and generated using the same modeling strategy. I am not surprised that a GNN can generalize well in this setting, since what it learns from dataset20 will hold in dataset100. I think that this result is overstated by the authors. I might not understand the differences between dataset20 and dataset100 well enough to understand the depth of this statement. I'd appreciate if the authors could explain this to me in more details. \n\nI'm also not sure that it makes sense to flag a node as trouble maker if small disturbances are amplified by a factor of at least 6. This seems a bit arbitrary. Is there any reason coming from analysis of real datasets for this? \nThis is then used for a classification task whereas I think a regression task would be more appropriate. For example, predicting the amplification factor rather than predicting if the factor is above or below 6. Also, reporting the uncertainty should be interesting. Is the model more confident on nodes with higher amplification factor? To use in a real setting, we would want the model to have a good uncertainty representation. \n\nOne thing I didn't understand is why, in figure 3, the texan grid has 97% of stable nodes although the distribution of SNBS seems to give much less probability for high SNBS (around 1). I'd appreciate some clarification here.\n\nIn figure 5, the model seems to work very well for low SNBS but have skewed predictions for high SNBS. Would it be possible to fix this using some non-linearity? \n\nOverall, there are lots of typos that make the paper a bit hard to read. For example in the abstract: \nWe show that large GNNs outperform GNNs from previous work as well as as handcrafted graph features and semi-analytic approximations.\n\n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject