Training speedups via batching for geometric learning: an analysis of static and dynamic algorithms
Authors: Daniel T. Speckhard
Tim Bechtel
Sebastian Kehl
Jonathan Godwin
Claudia Draxl
Venue: Accepted by TMLR ()

Abstract: Graph neural networks (GNN) have shown promising results for several domains such as materials science, chemistry, and the social sciences. GNN models often contain millions of parameters, and like other neural network (NN) models, are often fed only a fraction of the graphs that make up the training dataset in batches to update model parameters. The effect of batching algorithms on training time and model performance has been thoroughly explored for NNs but not yet for GNNs. We analyze two different batching algorithms for graph-based models, namely static and dynamic batching for two datasets, the QM9 dataset of small molecules and the AFLOW materials database. Our experiments show that changing the batching algorithm can provide up to a 2.7x speedup, but the fastest algorithm depends on the data, model, batch size, hardware, and number of training steps run. Experiments show that for a select number of combinations of batch size, dataset, and model, significant differences in model learning metrics are observed between static and dynamic batching algorithms.

Published in Transactions on Machine Learning Research (3/2026)
Training Speedups via Batching for Geometric Learning: An Analysis of Static and Dynamic Algorithms
Daniel T. Speckhard
Tim Bechtel
Sebastian Kehl
Jonathan Godwin
Claudia 1rax
Chimmblldt-nnveessitee zu Berlin
Max Planck Institute for Solid State Research
Max Planck Computing and Data Facility
4Orbital Materials
Soodphysik.uu-berline
Reviewed on OpenReview: http:://opppaeeeee..eee////rrrrs
Abstract
Graph neural networks (GNN) have shown promising results for several domains such as materials science, chemistry, and the social sciences. GNN models often contain millions of parameters, and like other neural network (NN) models, are often fed only a fraction of the graphs that make up the training dataset in batches to update model parameters. The effect of batching algorithms on training time and model performance has been thoroughly explored for NNs but not yet for GNNs. We analyze two different batching algorithms for graph-based models, namely static and dynamic batching for two datasets, the QM9 dataset of small molecules and the AFLOW materials database. Our experiments show that changing the batching algorithm can provide up to a 2.7x speedup, but the fastest algorithm depends on the data, model, batch size, hardware, and number of training steps run. Experiments show that for a select number of combinations of batch size, dataset, and model, significant differences in model learning metrics are observed between static and dynamic batching algorithms.
1 Introduction
Graph neural network (GNN) models have recently shown great promise regression and classification tasks, where the input data can be represented as graphs (21; 43). These methods have been applied to predict molecular and material behavior from nanometer millimeter scale (37; 20; 34; 30). These tasks are of utmost importance to society, since they have opened up research path towards exploring new molecules for drug design and carbon capture (9) and finding new materials for energy storage and generation (35).
Similar to neural network (NN) models, GNN models typically contain millions of parameters and require large training datasets to achieve sufficient predictive power. In order to effectively train GNNs on large datasets, the graph structured data needs to be batched, otherwise each update over the entire dataset takes too long for the model parameters converge in a reasonable amount of time. In this paper, we examine two different batching techniques for graph networks, static and dynamic batching. The two algorithms differ in that static batching always grabs the same number of graphs whereas dynamic batching adds graphs to the batch conditionally and seeks to ensure that the memory occupied by the batch of graphs is constant. For static-batching, we look at three algorithms, the static-64, static-2 and static-constant algorithms.
With the advent of neural architecture searches in different fields, including with graph structured data, it is often the case that researchers train model candidate architectures several thousand times (45; 16; 40). End-users are likely re-train the resulting model and re-tune parae-paraeeeers on new datasets. The total costs involved with the GNN model search and re-training pose a significant computational, financial and
1

---

Published in Transactions on Machine Learning Research (3/2026)
environmental burden (24; 38; 31). Therefore, any savings in training time offered by the batching algorithm are significant.
We examine the effect of the algorithms on training time and metrics. For different batch sizes, we compare the computation time required to assemble the graphs in a batch. We also analyze the time to update the model parameter weights using batch of graphs. Finally, we monitor the effect of the batching techniques the test metrics of the models. We do this for different batch sizes, models, and datasets. Our main contributions presented in this paper are:
We formally introduce the static and dynamic batching algorithms and several variants thereof.
The majority of experiments show no statistically significant differences in model learning between the algorithms, except for a single combination of model, batch size and dataset.
We find the static-constant algorithm to be the slowest algorithm and recommend not using it unless the goal is to maximize the robustness of the training pipeline. Across our experiments, we observe speedups of up to 12.5x when switching from the static-constant algorithm to another algorithm.
Excluding the static-constant batching algorithm, we observe at most 2.7x speedup in mean time per training step when switching from the slowest algorithm to the fastest.
We demonstrate that the performance of the algorithm is dependent on the graph distribution in the dataset, model used, batch size, and number of training steps run.
The static-64 and dynamic algorithms generally outperform the static-2 algorithms.
We recommend using the dynamic algorithm, whose performance is similar to the static-64 algorithm, but is faster for fewer number of training steps.
2
Related Work
Several different batching methods, i.e., full-batching, mini-batching (3), stochastic learning (5), have been thoroughly analyzed for NNs in the literature (4). For instance, NN learning performance when using mini- batching has been examined as a function of the batch size. Computation time as function of batch size GPUs (22). In Ref. (7), large batch sizes were found to typically slow down the
has also been explored convergence of the model parameters. In practice, researchers typically set the batch size as a hyperparameter that is found via cross validation (CV).
For NNs, updating the model weights can take up the bulk of training time through backpropagation (27). Typically, NNs operate on numeric data (or data that has been transformed into numeric values). When training the NN with a fixed mini-batch size of numeric data, batches have constant shape and memory requirements. Using JAX, this enables highly efficient training, since the gradient-update step can be compiled once at the start of the training. For convolutional neural networks (CNNs), this is not the case, and images of different pixel dimensions are often padded before being fed into the network (41). Similarly, for graph neural networks, the graphs in the dataset typically contain a wide variety of number of nodes and edges. To this end, batching algorithms have emerged, which pad batches of graphs to constant shapes. In this way, the gradient-update step for GNNs on batches can be compiled for execution on a hardware accelerator (e.g., a GPU). Two such methods have become popular, static batching and dynamic batching. The static batching methods always collect a fixed number of graphs while dynamic methods add graphs incrementally to batch until some padding budget/targe is reached.
That said, not all GNN models and libraries pad their data. M3GNet (8), GNN trained for interatomic potentials (IAP) is written in TensorFlow and performs batching in a similar way to NNs. Its batching procedure collects a number of graphs corresponding to the batch size, and concatenates the atoms, bonds, states and indices into larger lists for the batch. However, it does not perform any padding after batching data. The Torch GNN (ptgun) library implements a dynamic batching algorithm that also does not use padding (1). The algorithm adds graphs until either the batch has the desired batch size (i.e., number of
2

---

Published in Transactions on Machine Learning Research (3/2026)
graphs) or some safety limit on the number of nodes in the batch has been reached. This safety limit ensures that the batch will fit into memory.
The pyTorch geometric library (15) implements a similar dynamic batching algorithm. The user specifies whether to use either (but not both) a total number of nodes as the batch cutoff/limit or total number of edges. The algorithm then adds graphs incrementally to the batch until the target number of graphs are in the batch (i.e., the target batch size) or the cutoff has been reached. It also allows the user to skip adding single graphs to the batch that would by themselves exceed the node/edge cutoff.
The Jraph library, which is built on JAX (6), implements dynamic batching algorithm (17). Given batch size, it performs a one-time estimate of the next largest multiple of 64 to which the sum of the nodes/ eddee in the batch should be padded. We can think of these estimates as the node/edge padding targets for each batch (i.e., the number of nodes/edges in a batch after padding). This estimation is done by sampling a random subset of the data. It then iteratively adds one graph at a time to the batch and stops if adding another graph would exceed the node/edge padding targets or the maximum number of graphs (i.e., the batch size) is already in the batch.
The Tensorflow GNN library (14) implements static and dynamic batching algorithm. The static batching adds a fixed number of graphs to the batch and then pads to a constant padding value. The dynamic batching method, similar to Jraph, estimates a padding budget (they call a size constraint) for the batch based on a random subset of data, and then adds graphs incrementally to the batch as long as they do not break this budget.
The static and dynamic algorithms have, to our knowledge, not been described in the literature, but solely within code repositories. Experiments to measure the training time as function of the algorithm, batch size, model, or dataset are also to our knowledge not found in the literature. Our work seeks to describe the static and dynamic batching algorithms in sufficient detail and perform the aforementioned timing experiments using an implementation based on the Jraph library.
3 Preliminaries
Graph representation and notation. We consider dataset of graphs g {G1, G2,..., GN}. Each graph G; (V,, E,) consists of set of nodes V, and edges & (28; 23; 39). In sparse matrix formats the graph topology is defined by two integer arrays of length [& the senders s and receivers r, where an edge exists from node to Tk-
Graph neural networks. We evaluate three GNN models to get insight into any model dependent effects on our profiling results. Specifically, the SchNet model (37), a MPEU model (message passing with edge updates) (20; 2), and the PaiNN model (36) which uses equivariant features. The three models were chosen since the SchNet model is often used as benchmark, the MPEU has shown better results the AFLOW dataset in a relevant benchmark study (2), and the PaiNN model is widely used (25). These models vary in size. The SchNet model is the smallest with 84,768 trainable parameters. The next largest is the PaiNN model with 177,568 trainable parameters, and the MPEU model contains 2,553,472 trainable parameters.
We describe the three GNN models used to evaluate the batching algorithms in greater detail. GNN models take in graph as input to the model. The nodes in the graph are typically represented by feature vectors hg, where the subscript n represents the specific node, and the superscript t represents the number of times the vector has been updated. The SchNet model (37) is graph convolutional NN. The node feature vectors are initialized with an embedding matrix and are updated with node update equation (or convolution) that convolves the feature vector of the node, hi, and feature vectors in neighborhood, Nr, of the node, i. It uses convolutional kernel, W(rj -i), which depends on the euclidean distance between nodes:
h2 O (
(1)
JEN,
3

---

Published in Transactions on Machine Learning Research (3/2026)
We also look at message passing NN model with edge updates (MPEU) (20; 2). Message passing NNs make use of message function Mij, which is defined for sender, i, and receiver node, j, and their corresponding edge c'--
Mij fm hz.
(2)
Here, the messages are aggregated for each node with a permutation-Invariant operator. The MPEU aggre- gates the messages with a sum,
m ++1
Σ Mij.
(3)
JEN,
In the MPEU, the edges between any two connected nodes, i and j, are represented by edge vectors ejj. The message function is a function of the edge vector,
MiJ = (hty, (Wfh²) o o o
(4)
where, the feature vectors, hi, are updated in the MPEU as:
h Sh ) h + (W1m
(5)
O denotes element-wise multiplication and a represents the shifted soft plus function (44).
Finally, we evaluate the algorithms on the equivariant PaiNN model (36). This model differs from the MPEU and SchNet by learning not only scalar features, 8, E R Fæ1 for the representation each atom but also equivariant features, 0 E R23 The two representations are used update each other. The scalar representation is updated with the following equation:
As a,s (si, [[Vo]) + a.v (st, (V Vo),
(si,
(6)
where ass and agv represent neural networks with non-linear activation functions and V and U weight matrices that need to be learned. The update function for the equivariant representation is:
AU, = avv (ss (VVV
(7)
For all of these GNN models, the inputs is a graph. The memory requirements store single graph depend on the number of nodes in the graph and the number of edges. The different batching methods we will explore try to ensure that the memory requirements required to store a batch of graphs is constant, which will allow the graphs to be compiled on GPU.
4 Batching Algorithms
In static batching, the method grabs a fixed number of graphs for each batch. It then checks how many modes/edges are present in the batch. It then adds a dummy graph and adds dummy edges/ bages/nodes to this graph to serve as padding. Typically, the number of graphs included the batch is the batch size minus one in order leave space for the dummy graph the batch. The algorithm pads the number of nodes nodes/edges to try to ensure that most batches have similar number of nodes/edges, and thereby similar memory footprints.
Static-constant batching. In the static-constant algorithm, implemented in TensorFlow's GNN (TF- GNN) library, the algorithm first finds the graph with the most nodes modes/edges in the dataset and multiplies this number by the batch size to get a padding target for the nodes and edges in the batch. The padding target (or budget) is defined as T (T,, Te, Tg), representing the maximum allowable number of nodes, edges, and graphs in a single batch, respectively. For example, for batch size 32, the static-constant algorithm grabs 31 graphs (T₃), and then adds dummy graph with enough dummy nodes deddes//eeess//ee so that the batch has the
4

---

Published in Transactions on Machine Learning Research (3 (3/2026)
Static-constant batching
Go
G₁
G2
G3
Padding
Go
G1'
G2
G3
Padding
Batch size minus one graphs added. Padding target fixed by the graph with largest number of nodes/edges in training data.
Static-2N batching
Go
G1
G2
G3
Padding
Go'
Gi'
G2
G3
Padding
Batch size minus one graphs added. Padding added to bring sum of node/edges to the next power of 2.
Dynamic batching
Memory required to store graph determined by the number nodes/edges.
Go
G₁
G2
Padding
Go'
Gj'
G2'
G3'
Padding
Graphs added iteratively long the padding target is not reached (budget for nodes, edges, and graphs).
Figure 1: Visualization of the different batching algorithms for two batches each. A graph, G,, is represented by a block of memory which proportional to the number nodes and edges in the graph. To simplify the comparison we visualize the algorithms as sampling two batches from two sets of graphs, and G', that differ in the size of the graphs contained within them. Each algorithm adds padding (dummy nodes podes/edges and graphs) to try to keep the tensor shape of the batches constant or relatively constant. For visual clarity, the batch size for each algorithm is set to five, meaning up to four real graphs are included per batch by the algorithms. The dynamic algorithm (bottom) adds graphs to the batch iteratively only if adding the graph would not cause the number of modes/edges, hedge//grapppps graphs to be larger than the padding target; consequently, the number of non-dummy graphs may vary. The static-2 algorithm (middle) grabs a fixed number graphs and pads the total nodes/edge pades/edges to the nearest power of two. The static-64 algorithm not shown here for visual clarity but differs from the static-2 in that it pads the batch to the nearest multiple of 64. The static-constant algorithm (top) pads to fixed target determined by the largest graph in the training set, often resulting in significant padding overhead.
same nodes and edges their respective padding targets (Tv, Te). In this way, each static-constant batch occupies the same memory footprint. This means that the gradient-updant step, which updates the model parameters, can always expect a batch size of a constant memory input, and can be compiled for execution on a hardware accelerator (e.g. GPU/TPU for a fixed input tensor shape. The downside to this approach, is that for some datasets and batch sizes the target is extremely large since it based off the largest graph in the dataset. A visualization of the algorithm can be seen in Fig.
Static-2N and static-64 algorithms. In the static-2N algorithm, which we introduce here, the algorithm grabs batch size minus one number of graphs into batch, and then pads the batch's total number of nodes/edges to the next largest power of two. Over the course of training, one might encounter batch that requires a larger power of two than previously used in order to contain all of the nodes and edges in the batch. This requires the gradient-update function to recompile for a different, i.e., larger, input size. The static-2 algorithm is shown in the Appendix in Algorithm 5. Similarly, in the static-64 algorithm, which we also introduce here, the total number of nodes/ed /ddge in the batch are padded the next largest multiple of 64.
5

---

Published in Transactions on Machine Learning Research (3/2026)
Padding targets estimation. Unlike the static-constant algorithm, the padding targets in the dynamic algorithm are not based solely off the largest graph in the training dataset. The dynamic batching algorithm, in a pre-processing step, as implemented in Jraph and TF-GNN, samples a random subset of the data (e.g., 1000 graphs) to estimate padding targets for the nodes and edges. Note, this step can be done offline, and the time required to get the padding target is not included in our timing analysis of the algorithm. The padding target estimation works by calculating the mean nodes per graph Hy and mean edges per graph He from the randomly sampled subset of graphs. The edge (node) padding target Te (T.) is then created by multiplying the mean number of edges (nodes) by the batch size, N, and then rounding to the next largest multiple of 64. The cost of this step is independent of the dataset size, however, as an upper bound, the full dataset could be used to estimate the padding target. Then, the estimation equates to a summation of node and edge counts, which is still insignificant in comparison to training time, where usually each graph is iterated over many times. The algorithm to estimate the padding target is shown in Algorithm 1, where K graphs are randomly sampled, and NextMult66 is a function to get the next multiple of 64 for an input integer. Tg, the graph padding target is set to the batch size minus one, which is done so that at least one dummy graph can be added later with padded nodes/edges The padding of the batch to the padding target is discussed in more detail in the following paragraphs.
the batch can padded to the padding targets.
Algorithm 1 Estimate padding target (budget)
Input: Training set of graphs G, sample size K, target batch size N S < SAMPLEUNIFORM({ K) (Randomly sample K graphs}
Σ, 4 0,- Σg 4 0
for G. S do
Ev f E. + |Vi]
(Accumulate node counts}
Ee f Ee + E (Accumulate edge counts}
end for
Ho + Eg/K {Calculate mean node-count}
Me + -Ee/K {Calculate mean edge-count}
(Multiply means by batch size and round up to nearest 64)
To f NEXTMULT64(4+) x N)
Te f NEXTMULT64(He x N)
Tg - N 1 (Reserve one slot for a padded graph}
return T (To, Te, Tg))
Dynamic batching. The dynamic algorithm, given the padding target, appends one graph at a time to the initially empty batch. Before the addition of each graph, it verifies whether a single graph contains more nodes/edges than the padding target. If so, the program terminates. The algorithm must be restarted with a larger node/edge padding target. This issue can be mitigated by looking at a larger sample of graphs when estimating the targets. Alternatively, the user can loop through the entire dataset of graphs offline and find the graph with the largest number of nodes/edges to determine suitable budgets. Provided the individual graph fits within the constraints of the padding target, the dynamic algorithm then checks if adding the graph would put the batch over the node, edge or graph padding target. If it does, the graph is not added to the batch. Instead, the algorithm adds a dummy graph with padding nodes/ bodes/edges, the number of which is determined so as to pad the number of modes/edges to the padding target. It then adds, if necessary, dummy graphs with no nodes/ nodes/edges to ensure that the number of graphs in the batch is equal to the batch size. The padding function is shown in Algorithm 2. A pseudocode representation of the dynamic batching method is shown in Algorithm The batch method called in the algorithm concatenates all of the nodes/ /edges in the list of graphs and creates a single disconnected super-graph out of the list of graphs (Pseudocode for the method is found in the Appendix, Section A).
Note that the algorithm we present here is quite different from the whose implementation does not perform a budget estimation (has to be entered by the user) and it does not perform a check on the number of edges, but only the nodes. We find the Jraph/TF-Gr implementation, for which a simplified version shown in Algorithm 3, to be the more sophisticated method, and so it was used for our tests.
implementation,
6

---

Published in Transactions on Machine Learning Research (3 (3/2026)
Algorithm PadgeoTaree
Input: List graphs B, Padding target = (Tv, Te, Tg)
E, 0, Eg
for G. E B do
Ex E, + [V]
Ee - E. + Ei
end for {Calculate deficits for nodes, edges, and graph )ount)
P. - T2 -v
Pe f T. -c
Pg + -B)
{1. Add one dummy graph containing all missing nodes/edger
if Pg >0 then
Gpad + GRAPH(Po,, Pc)
B BU
Pg + Pg 1
end if
{2. Fill remaining slots with empty graphs to reach }
while Pg > 0 do
Gempty &- GRAPH(O,00 0) B x- UU U {Gempty)
P -1
end while return B
Algorithm Dynamic batching
Input: Training set of graphs 9, Padding Target = (Tv, Te, Tg) x (Current batch of graphs}
(co, Ce; 0g) + (0, 0, 0) {Current counts: nodes, edges, graphs}
for G2 E G do
(u,,e)) + (lVil, (E,))
{1. Safety Check: Verify graph fits in budget}
if (v; > To) V (e; > Te) then
raise Error("Graphh G, exceeds memory budget")
end
(2. Check adding G, overflows the current batch}
if (c. + 02 TT) (ce + Te) V (c, + 1 Tg) then (Yield current batch (padded) and start new one}
yield BATCH(PADTOTAAGEETBB...
B+ {G+} (Co, Ce, Cg) & (vi, ei, 1)
else
{Accumulate graph into current batch}
B+ BU
(cu, Ce, cg) t- (c. Cs es, Cg + end if
end for {3. Yield any remaining graphs}
B # 0 then
yield (ATCH(PAATTOTAARGCE(B.... end if

---

Published in Transactions on Machine Learning Research (3 (3/2026)
0.8
0.06
0.6
Density
Density
0.04
0.4
0.02
0.2
0-
0
20
40
100
500
520
540
580
600
620
Number of nodes
Nodes in batch before padding
0.08
0.003
AFLOW
dynamic static
0.06
QM9
Density
Density
0.002
0.04
0.001
0.02
0
8000
9000
10000 11000
12000
200
400
600
800
1000
Edges in batch before padding
Number of edges
Figure 2: Left: Histograms of the number of nodes (top) and edges (bottom) in the AFLOW and QM9 datasets. Right: Histograms of the number of nodes (top) and edges (bottom) in a batch of size 32 before padding for the static-2 and dynamic batching algorithms running on the QM9 dataset.
5 Datasets
We run profiling experiments on two different datasets. The QM9 dataset (32) contains chemical properties (e.g., internal energies, molecular orbital energy levels) of small organic molecules. The AFLOW dataset (10) is a collection of relevant material properties (e.g., formation energies, band gaps, elastic properties). We choose these datasets for two reasons. First, they have served benchmark datasets for GNN models (37; 26; 2). Second, they have real world applications. For instance, learning the QM9 dataset may help models better perform targeted drug discovery, while AFLOW data may help models discover more efficient solar cell semiconductors. For QM9 we target the molecular internal energy and for AFLOW we focus on learning the enthalpy of the crystal.
The datasets are not provided in graph format. To convert them to graphs, each atom represented as node in the graph. For QM9 the edges are added by fully connecting the nodes. For AFLOW, the edges are added with the K-nearest neighbor algorithm based on pairwise distance and setting K equal to 24. This KNN value duplicates and keep only calculations that contain both the enthalpy and the band structure, following the procedure in (2). We visualize the variety in the graph structures in the datasets in Fig. 2. From the figure we can see that the QM9 dataset node distribution appears Gaussian and centered around 17 nodes per graph. AFLOW's node distribution has long tails despite the mean of the distribution being smaller than QM9's. The distribution of edges for AFLOW and QM9 are reflective of the distribution of the nodes, this is because the number of edges are dependent on the number of nodes when using the KNN algorithm or fully connecting the graphs.
chosen from CV studies in the literature (20). From the AFLOW dataset, remove
8

---

Published in Transactions on Machine Learning Research (3/2026)
250
0.7
dynamic
QM9
AFLOW
Number of recompilations
static-2"
0.6
QM9
200
static-64
0.5
150
Density
0.4
100
0.3
0.2
50
0.1
16
20
64
128
5
10
15
20
25
30
35
Batch size
Graphs in batch before padding
Figure 3: Left: Number of recompilations on the QM9 dataset after two million training steps in the gradient- update step function of batch size for different batching algorithms. Note, that these numbers are model and hardware independent. Right: Histogram of the number of graphs from the dynamic algorithm before padding in a batch size of 32 for the AFLOW and QM9 dataset.
6 Implementation
There are several open-source software libraries choose from to perform these experiments. We opt to use the Jraph library (17), which is built using the JAX library (6). The JAX library traces out computations into computation graph that it uses to optimize and compile the code. This compiled auto-differentiation code is optimized to run fast, especially on GPU. To run our experiments, we extend Jraph to run the static-2 static-64, and static-constant algorithms, log necessary profiling information and run the models we have selected for our tests.
7 Test setup and compute hardware
As mentioned, the benefit of dynamic batching and static-constant batching is that the memory allocated to a batch of graphs is always the same for each batch. This allows the user to compile the update functions only once and optimize the compiled binary to run on GPU. Note that with static-64 and static-2" batching, the update function can also be compiled, but needs be recompiled for different multiples of 64 powers of two (the static-constant algorithm does not have this problem). We time the training of models for different batching algorithms.
Batching and weight update timing experimental setup. We perform timing experiments on both algorithms for four batch sizes (16, 32, 64, 128) using both datasets and the three models for two million training steps. This number of training steps was chosen since resulted in converged models (2). Experi- ments with fewer steps are run and the results can be found the Appendix. We run each combination of batch size, algorithm, model, and dataset times to limit noise effects from the hardware the profiling results. Each experiment reports the mean batching time, mean gradient-update step time (update time), and the sum of the two (combined time).
Hardware used. We run experiments on a cluster that comprises two Intel Xeon IceLake Platinum 8360Y CPUs and 4 NVIDIA A100 GPUs connected via NVLINK. We ran two sets of experiments, one using a single CPU, and the other using a single CPU and a single GPU. This design choice allows us to isolate the algorithmic overhead of the batching strategies from hardware-specific confounding variables, such as interconnect latency and multi-device synchronization costs. More discussion on porting this algorithms multi-GPU setup is found in the Appendix F.

---

Published in Transactions on Machine Learning Research (3/2026)
8 Profiling results
Batch statistics. To understand how the batching algorithms work in practice, analyze the statistics of the number of nodes in a batch before padding occurs. The histograms for both batching algorithms are shown for the QM9 dataset in Fig. 2 for 10,000 batches with batch size of 32. As stated above, the dynamic batching algorithm checks the node and edge padding target, as well as the number of graphs already in the batch, before adding a graph to the batch. For this dataset and batch size, the dynamic algorithm's node padding target is 576. We can see that all dynamic batches have fewer than 576 nodes before padding, This results in a one-sided distribution, roughly a truncated Gaussian distribution, where the right-hand side is cutoff near the budget. For static batching, however, no such check exists, and observe roughly Gaussian distribution of nodes. Similar effects are seen in the figure for the number of edges before padding.
Recompilation (mean vs. median). For fewer training steps, the number of recompilations required in the gradient-update step can be the dominant factor defining the ranking of the algorithms. The number of recompilations is shown in Fig. 3 for the QM9 data for two million training steps. The number of recompilations is model and hardware independent. We see that the static-64 algorithm performs several orders of magnitude more recompilations than the static-2 and dynamic algorithms. This is to be expected, since the dynamic algorithm compiles only once, and if it encounters graph, which is bigger than the budget, the program terminates and needs to be restarted with a larger padding target. Moreover, for static-64 batching, JAX recompiles the gradient-update step function every time a new nearest multiple of 64 is encountered when padding, and for static-2N batching every time a new nearest power of two is encountered, which is less likely for larger powers of two. For example, for the batch size of 32, the static-2 algorithm recompiles four times while the static-64 algorithm recompiles 89 times. This is expected behavior for JAX's Just-In-Time (JIT) compilation system, which caches compiled kernels for reuse but triggers new compilation whenever it encounters tensor shape not present in its cache (6).
The recompilation overhead explains the discrep- ancy between mean and median timing results after 100,000 training steps presented in the Appendix Section C (Figs. 7 and 8). In the figures, the static-64 algorithm has the lowest median update times while the static-2N has the lowest mean up- date times. The mean is affected by outliers while the median is not. This tells us that when removing the effect of outliers, i.e., slow gradient-update steps which are caused by recompilations, the static-64 al- gorithm is fastest to update the gradients. This also tells us that the algorithm training time rankings are dependent on the dataset, where more variability in graph sizes causes more recompilations.
8
dynamic static-64 static-2"
Combined time (ms)
6-
4
X
X
XXXXXXX
2
0
Fig. 4 shows the running average of the combined batching and update times per training step for QM9 data and the MPEU model with a batch size of 32. The static-64 algorithm the slowest ini- tially but is the fastest algorithm when training for 800,000 training steps or more. Most of the recompi- lations for the QM9 dataset happen in the first hun- dred thousand training steps. From training step one hundred thousand to two million, the static-2 algorithm does not recompile again and the static- 64 algorithm recompiles only an additional twenty times. Fig. 11 in the appendix displays the same model and batch size for the AFLOW dataset and shows the dynamic (static-64) algorithm to be the fastest (slowest) initially. After 300,000 training steps the effect of the recompilations has been reduced and two algorithms are equally fast and quicker than the static-2
0.25 0.5 0.75
1
1.25 1.5
1.75
2.0
2.25
Training steps (millions)
Figure 4: The running average of the combined time (sum of the batching step and gradient-update step) required per training step as a function of the total number of training steps run. Here only a single iter- ation is run for the batch size 32, MPEU model and QM9 dataset for the dynamic, static-64 and static-2 algorithms.
10

---

Published in Transactions on Machine Learning Research (3/2026)
SchNet
MPEU
PaINN
AFLOW
QM9
AFLOW
QM9
AFLOW
QM9
Batching time (ms)
10
*
dynamic
static-2"
6-
static-64
2
10
40
Update time (ms)
8
30
6
20
4
2-
x
10
Combined time (ms)
10
40
8
30
6-
20
4
10
*
16 32 Batch size
128 16 32
16 32 64
16 32 64
32
128 16 32
128
Batch size
Batch size
Batch size
Batch size
Batch size
Figure 5: Timing measurements while varying the batch size for SchNet (left two columns), MPEU (middle columns) and PaiNN (right two columns). Results for the AFLOW and QM9 dataset shown for each model. For each datapoint, ten iterations of two million training steps are run.
method. These results demonstrate that the fastest algorithm in terms of training time dependent on the number of training steps run.
Static-constant batching results. The static-constant algorithm also evaluated for subset of data, models and batch sizes. The results are shown in the Appendix Section H. The static-constant algorithm performs poorly compared to the other static algorithms. As shown in Fig. 2, the AFLOW dataset contains a long-tailed distribution in terms of the mode/edges (the estimated excess kurtosis of the two histograms are greater than one (11)). Moreover, the mean number of nodes/edges is 10-20 times smaller than the maximum number. As a result, for AFLOW, the static-constant model uses large padding target that slows down the batching and update step. For QM9, the histograms of the dataset show that the distribution of nodes and edges resemble more closely normal distribution. The number of edges varies from roughly one hundred to six hundred edges, and as result, for the QM9 data, we see smaller yet still significant slowdown using the static-constant batching algorithm compared to other methods.
Static-64, static-2N, and dynamic algorithm results. Fig. 5 displays the breakdown batching time per training step. We see the batching times appear roughly model and dataset independent and scale linearly with the batch size. The static algorithms loop over the number of graphs in the batch, and the dynamic algorithm does the same but may exit the loop early based on the padding target checks. Therefore it is not surprising that the batching steps scale O(N) where N is the batch size. The dynamic batching is slower than the static-2 and static-64 algorithms due to the added overhead from the bookkeeping of the padding targets.
For the SchNet and MPEU models, the timing results of the gradient-update step at small batch sizes are similar for all algorithms. For the PaiNN model, the static-2N algorithm has slower gradient update for all batch sizes. For larger batch sizes, across all models and datasets, the dynamic batching algorithm performs the gradient update the fastest. The dynamic batching algorithm is the fastest, since the gradient update
11

---

Published in Transactions on Machine Learning Research (3/2026)
code is never recompiled. As seen in the histograms in Fig. 2, the number of nodes in a batch before padding is typically smaller for the dynamic algorithm. This is also true after padding. GNNs like the models we examine here, typically have a node update equation, which means that inference, and therefore the gradient-update step, scales roughly linearly with the number of nodes. When the difference between the number of nodes is small between the dynamic and static-64 algorithm we do not expect much difference in the gradient-update step time. The static-2N algorithm, however, typically adds large number of padded nodes, which explains why the algorithm is typically slower than the dynamic algorithm in the gradient- update step. The static-64 and dynamic algorithms show roughly linear scaling in the update time with the batch size. The static-2 algorithm shows poor relative performance especially for larger batch sizes. More work needs to be done to understand the static-2N scaling behavior.
The sum of the batching and gradient-update step mean timings, i.e., the mean combined time, gives us the mean time per training step. The speedup is defined the ratio between the slowest algo- rithm combined time per step, T', and the faster algorithm combined time per step, Tz, as T. (19). For the PaiNN model, QM9 dataset, and batch size 128, the dynamic batching and static-64 algorithm have a speedup of over 2.7x when compared to the static-2" algorithm. Similarly, for the PaiNN model for AFLOW and batch sizes of 32, 64 and 128, the dynamic algorithm has a speedup of roughly 1.5x when compared to the static-2 algorithm. For the MPEU model on the QM9 dataset, we observe 1.5x speedup when using the static-64 algorithm com- pared the static-2 algorithm. These observa- tions over different datasets and models point to the large effect that changing the batching algorithm can have on the training time. The speedup from switching algorithms for the PaiNN model on the QM9 and AFLOW datasets can be seen in the Fig. 6. If we include the static-constant algorithm, which is always the slowest algorithm for the parameter space we explored here, the largest speedup 12.5x for the PaiNN model on AFLOW with batch size of 16 when switching to the dynamic algorithm from the static-constant algorithm.
3.0
AFLOW PaiNN
dynamic static-2"
2.5
static-64
T.
2.0
Speedup
1.5
1.0
0.5
0.0
3.0
QM9 PaiNN
2.5
2.0
Speedup
1.5
1.0
0.5
CPU-only performance. We also benchmarked performance on CPU-only setup (detailed results in Appendix Section E). Unlike the GPU setting, the gradient-update step is significantly slower on CPU and becomes the bottleneck. We find that the dynamic batching algorithm generally provides the fastest mean combined time because it avoids the compilation overhead, which is particularly expen- sive on CPU. However, similar to the GPU results, the static-64 algorithm often achieves the best me- dian times once the compilation phase stabilizes.
0.0
16
2
128
Batch size
Figure 6: Speedup (GPU) when switching from the slowest amgorithm in terms of combined training time per step for the PaiNN model (not including the static- constant model) for the AFLOW (top) and QM9 (bot- tom) datasets. For both datasets the slowest algorithm is the s algorithm. the static-constant algo- rithm is included the speedup increases to a maximum of 12.5x
Effect on learning. The batching algorithm may also affect the learning of the model. Dynamic batching does not always include the same number of graphs in batch, since one or more dummy graphs are added when the node or edge budget is reached.
12

---

Published in Transactions on Machine Learning Research (3/2026)
These padded graphs do not contribute to the gradient in the update step. The histogram in Fig. 3 depicts the distribution of the number of graphs before padding for the AFLOW and QM9 dataset after running 10,000 iterations with the dynamic algorithm. While most batches contain 31 graphs for a batch size of 32, some batches for the AFLOW dataset contain as few as ten graphs from the training data. This is likely due to the graph node and edge distributions (see Fig. 2). QM9 has significantly shorter tails than AFLOW, which explains why the distribution of the number of graphs before padding more consistent for QM9.
To discern the effect of the algorithm on learning, we perform pairwise Mann-Whitnee U test (29) the final RMSE test loss. This non-parametric test ranks samples from both distributions and uses these rankings to assess whether the underlying distributions are identical. A key advantage of the Mann-Winne Whitney U test over the standard Student's t-test (18) is that it does not assume the data is normally distributed, an assumption that neural network training outcomes often violate (12). To account for the risk of false positives when testing across many configurations, we apply the Bonferroni correction (13) to the p-values within each model and dataset family. This correction adjusts the p-values in proportion to the number of pairwise comparisons performed. We consider results to be statistically significant only if the adjusted p-value is less than 0.05 (42).
Under this testing framework, we find that for the majority of combinations of dataset and batch size, the choice of batching algorithm has no statistically significant impact on model performance. For the QM9 dataset, we find no significant difference in test RMSE between dynamic and static batching for any model or batch size. Similarly, for the MPEU and PaiNN models on the AFLOW dataset, the results were statistically insignificant. However, significant difference was identified for the SchNet model the AFLOW dataset for batch size of 16 (adjusted p R 0.0022). This suggests that for heavy-tailed datasets, the dynamic algorithm's 's tendency to use smaller batches of training graphs can introduce gradient noise when the batch size is small. For completeness, we also provide the unadjusted Student's 's t-test results in Appendix Section I.
9 Discussion
We find the static-constant algorithm to perform the worst in terms of timing performance per training step. We do not recommend using the algorithm unless the user has a dataset where the maximum graph size is very similar to the mean graph size, and/or the priority is to have robustness in the training pipeline. We see in some cases a speedup of up to 12.5x when switching from the static-constant algorithm to the other algorithms. If we discard the static-constant algorithm, choosing one algorithm versus another can result in more than a 2.7x speedup in training time without observable differences in learning metrics. For larger hyperparameter searches or neural architecture searches (NAS), this speedup can save a very large amount of computational resources. Across the parameter space explored in this work, the static-64 and dynamic algorithm appear the fastest. In general, we would recommend the dynamic algorithm since it performs faster than the static-64 algorithm for fewer number of training steps, which can be important for NAS.
Diving deeper into the timing performance, the static algorithms perform the batching step faster. This is due to the added overhead from the bookkeeping of the padding targets in the dynamic algorithm. All of the algorithms show linear scaling in the batching time with respect to the batch size. The static-64 and dynamic algorithms show linear scaling in the gradient-update step, while the static-2 algorithm shows exponential scaling. This is due to the exponentially larger memory sizes required when using powers of two for padding. The gradient-update step is performed fastest by the dynamic algorithm. This is partially because the algorithm is never recompiled. Moreover, the static-2" algorithm typically has larger number of nodes/edges in the batch, which slows down the gradient-update step.
The model's ability to learn is statistically indistinguishable across nearly all experimental settings. We observe a significant divergence in only one combination, the AFLOW dataset, batch size of 16 and the SchNet model. This isolated difference is likely due to the dynamic algorithm containing fewer labeled graphs per batch, particularly for long-tailed distributions in the small-batch regime.
We hope that this work serves to aide users to reduce the computational cost of training graph based models. Further work might focus on predicting the fastest algorithm as a function of the dataset, batch size, and
13

---

Published in Transactions on Machine Learning Research (3/2026)
model. Finally, this work could in the future be extended to combine the batching algorithms with batching schemes that partition large graph networks across GPUs.
Software and data availability
The software to perform the batching, profiling experiments, parsing of experiments, and plotting is found in httptub...../epccarr////g.aa bepcchtard/mmm..atcciin.////// bat.batchhin//.... The complete set of pairwise Mann-W Hann-Whitney U test and Student's t-test. heatmaps can be found at https://drive www.d//iwewggo....ooooooooo googlee.coo//rvvv//toleeess
References
[1] ALLAMANIS, M., MIR, A., AND PATI, S. ptgnn: A pytorch gnn library, 2022.
(2] BECHTEL, T., SPECKHARD, T., GODWIN, J., AND DRAXL, C. Band-gap regression with architecture-optimized message-paste passing neural networks. arXiv preprint arXiv:2309.063/6 (2023).
3 BISHOP, C. M. Pattern recognition and machine learning, vol. 4. Springer, 2006.
[4] BOTTOU, L., AND BOUSQUET, O. The tradeoffs of large scale learning. Advances in neural information processing systems 20 (2007).
[5]
BOTTOU, L., AL. Stochastic gradient learning in neural networks. Proceedings of Neuro-Nmmes 91, 8 (1991), 12.
[6] BRADBURY, J., FROSTIG, R., HAWKINS, P., JOHNSON, M. J., LEARY, C., MACLAURIN, D., NECULA, G., PASZKE, A., VANDERPLAS, J., WANDERMAN-MILNE, S., AND ZHANG, Q. JAX: composable transformations of Python+ -NumP programs, 2018.
[7] BYRD, R. H., CHIN, G. M., NOCEDAL, J., AND Wu, Y. Sample size selection in optimization methods for machine learning. Mathematical programming 134, (2012), 127-155.
88] CHEN, C., AND ONG, S. P. universal graph deep learning interatomic potential for the periodic table. Nature Computational Science 2, 11 (2022), 718-728.
[9] CHOUDHARY, K., YILDIRIM, T., SIDERIUS, D. W., KUSNE, A. G., McDANNALD, A., AND ORTIZ- MONTALVO, D. L. Graph neural network predictions of metal organic framework co2 adsorption properties. Computational Materials Science 210 (2022), 111388.
[10] CURTAROLO, S., SETYAWAN, W., HART, G. L., JAHNATEK, M., CHEPULSKII, R. V., TAYLOR, R. H., WANG, S., XUE, J., YANG, K., LEVY, O., ET AL. Aflow: An automatic framework for high-throughput materials discovery. Computational Materials Science 58 (2012), 218-226.
[11] DECARLO, L. T. On the meaning and use of kurtosis. Psychological methods 2, 3 (1997), 292.
[12] DEMŠAR, J. Statistical comparisons of classifiers over multiple data sets. Journal of Machine learning
research 7, Jan (2006), 1-30.
[13] DUNN, O. J. Multiple comparisons among means. Journal of the American statistical association 56, 293 (1961), 52-64.
[14] FERLUDIN, O., EIGENWILLIG, A., BLAIS, M., ZELLE, D., PFEIFER, J., SANCHEZ-GONZALEZZ A., LI,
W. L. S., ABU-EL-HAIJA, S., BATTAGLIA, P., BULUT, N., HALCROW, J., DE ALMEIDA, F. M. G., GONNET, P., JIANG, L., KOTHARI, P., LATTANZI, S., LINHARES, A., MAYER, B., MIRROKNI, V., PALOWITCH, J., PARADKAR, M., SHE, J., TSITSULIN, A., VILLELA, K., WANG, L., WONG, D., AND PEROZZI, B. TF-GNN: graph neural networks in tensorflow. CoRR ////2007.0552 (2023).
14

---

Published in Transactions on Machine Learning Research (3/2026)
[15] FEY, M., AND LENSSEN, J. E. Fast graph representation learning with pytorch geometric. arXiv preprint ar arXiv:1903.02423 02428 (2019).
[16] GAO, Y., YANG, H., ZHANG, P., ZHOU, C., AND HU, Y. Graph neural architecture search. In Inter- national joint conference on artificial intelligence (2021), International Joint Conference Artificial Intelligence.
[17] GODWIN* J., KECK* T., BATTAGLIA, P., BAPST, V., KIPF, T., LI, Y., STACHENFELD, K., VELIČKOVIĆ, P., AND SANCHEZ-GONZALER A. Jraph: A library for graph neural networks in jax., 2020.
[18] HARTMANN, K., KROIS, J., AND RUDOLPH, A. Statistics and geodata analysis using r, 2023.
[19] HENNESSY, J. L., AND PATTERSON, D. A. Computer architeclure: a quantitative approach. Elsevier, 2011.
[20] JØRGENSEN, P. B., JACOBSEN, K. W., AND SCHMIDT, M. N. Neural message passing with edge updates for predicting properties of molecules and materials. arXiv preprint arXiv:1806.0311 (2018).
[21] KIPF, T. N., AND WELLING, M. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.0900 (2016).
[22] KOCHURA, Y., GORDIENKO, Y., TARAN, V., GORDIENKO, N., ROKOVYI, A., ALIENIN, O., AND STIRENKO, S. Batch size influence on performance of graphic and tensor processing units during training and inference phases. In International Conference on Computer Science, Engineering and Education Applications (2019), Springer, pp. 558-668.
[23] KOLLER, D., AND FRIEDMAN, N. Probabilistic graphical models: principles and techniques. MIT press, 2009.
[24] KOROLEV, V., AND MITROFANOV, A. The carbon footprint of predicting co2 storage capacity in metal-organic frameworks within neural networks. Iscience 27, 5 (2024).
[25] KUBEčKA, J., AYOUBI, D., TANG, Z., KNATTRUP, Y., ENGSVANG, M., WU, H., AND ELM, J. Accurate modeling of the potential energy surface of atmospheric molecular clusters boosted by neural networks.
Environmental Science: Advances 3, 10 (2024), 1438-1455 1451.
[26] LI, S.-C., WU, H., MENON, A., SPIEKERMANN, K. A., LI, Y.-P., AND GREEN, W. H. When do quantum mechanical descriptors help graph neural networks to predict chemical properties? Journal of the American Chemical Society 146, 33 (2024),
[27] LISTER, R., AND STONE, J. V. An empirical study of the time complexity of various error functions with conjugate gradient backpropagation. In Proceedings of ICNN'95-Internationn Conference on Neural Networks (1995), vol. 1, IEEE, pp. 237-241.
[28] LIU, Z., AND ZHOU, J. Introduction to graph neural networks. Springer Nature, 2022.
[29] MANN, H. B., AND WHITNEY, D. R. On a test of whether one two random variables is stochastically larger than the other. The annals of mathematical statistics (1947), 50-60.
[30] NEUMANN, M., GIN, J., RHODES, B., BENNETT, S., LI, Z., CHOUBISA, H., HUSSEY, A., AND GODWIN, J. Orb: A fast, scalable neural network potential. arXiv preprint 2xix22410.2257 (2024).
[31] PATTERSON, D., GONZALEZ, J., LE, Q., LIANG, C., MUNGUIA, L.-M., ROTHCHILD, D., So, D., TEXIER, M., AND DEAN, J. Carbon emissions and large neural network training. arXiv preprint arXiv:2104.10550 (2021).
[32] RAMAKRISHNAN, R., DRAL, P. O., RUPP, M., AND VON LILIENFELD, O. A. Quantum chemistry structures and properties of 134 kilo molecules. Scientific data 1, 1 (2014), 1-7.
[33] Ross, S. M. Introductory statistics. Academic Press, 2017.
15

---

Published in Transactions on Machine Learning Research (3/2026)
[34] GONHALEZ,,, A., GODWIN, J., PFAFF, T., YING, R., LESKOVEC, J., AND BATTAGLIA, P. Learning to simulate complex physics with graph networks. In International conference on machine learning (2020), PMLR, pp. 8459 8468.
[35] SCHAARSCHMIDT, M., RIVIERE, M., GANOSE, A. M., SPENCER, J. S.S,, GAUNT, A. L., KIRKPATRICK,
J., AXELROD, S., BATTAGLIA, P. W., AND GODWIN, J. Learned force fields ready for ground state catalyst discovery. arXiv preprint arXiv: 12466 (2022).
[36] SCHÜTT, K., UNKE, O., AND GASTEGGER, M. Equivariant message passing for the prediction of tensorial properties and molecular spectra. In International Conference on Machine Learning (2021), PMLR, pp. 9377-9388.
[37] SCHÜTT, K. T., SAUCEDA, H. E., KINDERMANS, P.-J., TKATCHENKO, A., AND MÜLLER, K.-R. Schnet-a a deep learning architecture for molecules and materials. The Journal of Chemical Physics 148, 24 (2018).
[38] SPECKHARD, D., BECHTEL, T., GHIRINGHELLI, L. M., KUBAN, M., RIGAMONTI, S., AND DRAXL,
C. How big is big data? Faraday Discussions (2025).
[39] SPECKHARD, D. T. Graph topology estimation of power grids using pairwise mutual information of time series data, arXiv preprint ar #xXi:2505.1151/1 (2025).
[40] SPECKHARD, D. T., MISIUNAS, K., PEREL, S., ZhU, T., CARLILE, S., AND SLANEY, M. Neu- ral architecture search for energy-effccient always-on audio machine learning. Neural Computing and Applications 35, 16 (2023),
[41] TANG, H., ORTIS, A., AND BATTIATO, pre-trained convolutional neural networks. In Image Analysis and Poncessing-CCAA 2019: 20th In- ternational Conference, Trento, Italy, September 9-13, 2019, Proceedings, Part II 20 (2019), Springer, pp. 337-344.
S. The impact of padding on image classification by using
[42] WASSERSTEIN, R. L., AND LAZAR, N. A. The asa statement on p-values: context, process, and purpose. The American Statistician 70, 2 (2016), 129-133.
[43] XU, K., HU, W., LESKOVEC, J., AND JEGELKA, S. How powerful are graph neural networks? arXiv preprint arXiv:1810.00826 (2018).
[44] ZHAO, H., LIU, F., LI, L., AND LUO, C. A novel softplus linear unit for deep convolutional neural networks. Applied Intelligence 48 (2018), 1707-1720.
(55 ZOPH, B., AND LE, Q. V. Neural architecture search with reinforcement learning. arXiv preprint 11110:1611.01578 (2016).
A Batch and static batching algorithms
This section provides the static batching algorithm and the BATCH algorithm mentioned in Section 3. The BATCH method shown in Algorithm 4. The pseudocode provided is a simplified version of the method found in Jraph, although the TF-GNN, ptgun, and pyTorrch libraries all perform something similar. The basic principle is to create single disconnected super-graph that contains all of the information of the individual graphs in the batch. It initializes lists to collect information about the individual graphs. The lists collect node features, edge features, global features, connectivity indices geener/rrecivvr pairs) and graph metadata (node/edge counts). It then loops over the input graphs list, and concatenates the graph's 'ss node features, edge features, number of nodes/edges and sender/receine indices into the respective batch lists. Finally, the method creates a new graph object where the constructor fed the concatenated lists. Thereby, from list of graphs, we have created one super-graph containing all of the information about the smaller graphs. The individual graphs can be rebuilt from the super-graph if desired. Note that the sender
16

---

Published in Transactions on Machine Learning Research (3/2026)
Algorithm 4 Batch construction (disjoint super-graph)
Input: List of graphs B [Gi,..., Gm}
Xalt, Eail, Vall (Nodes, Edges, Globals} Sall, Fall 0 {Connectivity)
Dy Dg t- 0 {Counts}
8 4-0 (Node offset}
for G; E B do
X,, 0 u; f Features(G.))
St, ri Topology(C.) (G:)
V; 6 [Vil, [
{Concatenate features}
Xalt +- CONCAT (Xill x;)
Eau + CONCAT Ball; e;)
Uall < CONCAT (ulll
(ual, uz) {Globals (no offset needed))
{Shift indices to create block-diagonal adjacency)
Sall f CONCAT(Salll, 8; + 6)
Fall (TallT(Tall:s ri + 6)
n, APPEND m v4)
ne <- APPEND Ci)
8 - 8 +
end for
return GRAPH Eall, Wall, Salt, Fall, Bp De)
and receiver node indices need to be offset by a running counter of the number nodes already added into the batch.
The static-constant batching algorithm first finds the graph with the largest number of nodes and the graph with the largest number of edges. These values are saved and when rounded to the nearest multiple of 64 serve as targets for padding. The static batching algorithm is shown in Algorithm 5. The algorithm grabs a set of batch size minus one graphs, and then pads the set of graphs to the next multiple of 64 (static-64) or next power of two (static-2*).. Note that Algorithm 5 shows the static-2 implementation.
Algorithm Static batching
Input: Dataset 9, Start index s, Batch size N B 5-o 0 (Initialize batch container)
E, 0, Σ. +- 0
{Collect N - real graphs (leaving 1 slot for padding))
for = to N - do
G + G B Bu {
Σ, - Σ, + V
E. E. + E,
end for
[Pad to nearest power of 2 (or multiple of 64)}
T. < NEXTPOWER2(E,,
T. + NEXTPOWER2 Σe) Tg < N
T + Te,TT
return BAYCH(PADTOOTAREE(((.1))
17

---

Published in Transactions on Machine Learning Research (3/2026)
B Timing experiments setup
The timing experiments made use of python's time library. Timing statements were executed before batching, before the gradient-updats step and after the gradient-update step. Block until ready commands were executed to ensure operations on the GPU had finished before the timing measurements were taken. We also experimented with placing timing measurements before the training loop and after the training loop (i.e., after million steps in some cases). We then averaged this time by the number of training steps executed. We found this number be within a standard deviation of the sum of our batching and gradient-updatt step timing results. This points the fact that the library runs batch creation and update- kernel execution consecutively. It is possible to these steps synchronously but this optimization was not done in this study. More details on the implementation can be found in the code repository.
C Mean versus median in timing measurements
For each timing experiment ten experiments are run and either the mean or median is taken. The median timing results for the MPEU model after running one hundred thousand steps is seen in Fig. The mean results for the same experiments are shown in Fig. 8. The difference in the two figures shows the importance of the number of recompilations in the gradient-update method. This is because the mean is affected by outlier measurements, such as recompilations, while the median is not. The fact that for one hundred thousand training steps, the static-64 algorithm has the best median performance but not the best mean performance is due to the number of recompilations which is highest for this algorithm.
The mean and median timing results for two million steps are shown for the MPEU model in Fig. 9 and in Fig. 10 respectively. We see the mean static-64 gradient-update step times are shifted higher than the median results, showing that the recompilations still affect the mean results. However, the rankings of the mean combined times are the same for the median combined times across algorithms, suggesting that for longer training time the effect of recompilations is no longer significant as we saw earlier for one hundred thousand steps.
D Profiling behavior as a function of the number of training steps run
The running average combined time per step is shown in Fig. 11 on the AFLOW dataset for the MPEU model using a batch size of 32. The dynamic algorithm is initially the fastest. The static-64 algorithm is initially significantly slower due to the large number of recompilations that happen in the first 100,000 training steps. After 300,000 training steps, the effect of the recompilations on the static-64 algorithm's performance is reduced and the static-64 and dynamic methods are roughly equally as fast.
E
CPU--nly timing results
This section provides supplementary details regarding the CPU only timing experiments. The algorithm performances are different when running only on CPU. The batching (inclusive of padding) performance is the same when running on a system that has both GPU and a CPU, which indicates that the batching is executed only CPU. We experimented with trying to run part of the batching and padding code GPU using JAX commands but found no speedup. The gradient-update step, however, is much slower CPU. The mean timing results, from ten iterations of one hundred thousand training steps, are shown in Fig. 12 for the MPEU model. The median is shown in Fig. 13 for the same model. We ran one hundred thousand training steps instead of two million training steps since the computer cluster we used had a time limit of twelve hours for experiments. From the mean results, we see the dynamic batching algorithm is fastest. The median results, however, show that when the effect of recompilations are reduced, the static-64 algorithm is the fastest except for batch size 128 on the AFLOW dataset. This suggests that for longer training times the
18

---

Published in Transactions on Machine Learning Research (3/2026)
AFLOW
QM9
Batching time (ms)
X
dynamic
MPEU
static-2"
median
static-64
100,000 steps
2
15
Update time (ms)
Combined time (ms)
5
1632 64
128
1632
128
Batch size
Batch size
Figure 7: Median batching (top) and median gradient-update times (middle), and median combined time (bottom) for varying batch sizes on the AFLOW (left) and QM9 (right) data using the MPEU model. For each datapoint, ten iterations of one hundred thousand training steps are run.
static-64 algorithm will be the fastest. The results for the SchNet model can be seen in the accompanying code repository.
F Multi-GPU behavior
We utilized JAX's parallel map (pmap) function assess the feasibility of the algorithms on multi-GPP setup comprising four A100 GPUs. The dynamic batching algorithm is natively compatible with distributed setups; JAX's pmap distributes the graph batches across devices without requiring changes to the batching logic itself. The static-64 and static-2" algorithms, however, did not work out of the box with pmap since the algorithm expects the batches sent to each GPU to be the same size. For the AFLOW dataset, we saw that the four batches for each GPU had different shapes after padding when using the static-2" algorithm roughly 90% the time, and close to 99% when using the static-64 algorithm. It's possible to alter the static-64 and S algorithm to pad each batch of data using the largest number of nodes and edges across all batches. The results of such tests, however, will be highly implementation dependent. To simplify the comparison we evaluate the algorithms on single-GPU setups.
19

---

Published in Transactions on Machine Learning Research (3/2026)
AFLOW
QM9
Batching time (ms)
X
dynamic
MPEU
static-2"
mean
static-64
100,000 steps
2
15
Update time (ms)
Combined time (ms)
5
1632
64
128
1632
128
Batch size
Batch size
Figure 8: Mean batching (top) and mean gradient-update times (middle), and mean combined time (bottom) for varying batch sizes on the AFLOW (left) and QM9 (right) data using the MPEU model. For each datapoint, ten iterations of one hundred training steps are run.
G Test metric results
The mean test metric (RMSE) values as a function of the number training steps are averaged across each of the ten iterations are averaged for each batch size, model and dataset combination and are shown in Fig. 14 for the MPEU and SchNet models using QM9 data. We do not see significant differences in the test performance for the static-2 or dynamic batching algorithm. The results for the AFLOW test dataset can be seen in Fig. 15. Note that we do not expect nor do we see any noticeable difference in learning between the static-64 and static-2 algorithms since the difference between the methods is the padding scheme, which does not affect the loss. Therefore, the static-64 RMSE curve left out the two figures.
Static-constant batching
The TF-GNN library tutorial suggests running the static batching algorithm with a constant padding target. As described Section 3, the algorithm first iterates through all the graphs in the dataset and saves the maximum number of nodes/edges seen in a graph and uses this as a padding target. We evaluated this method on subset of batch sizes, dataset and model combinations, and compared static-64 and dynamic algorithms. The results are shown in Table Our results show that the static-
the static-2
20

---

Published in Transactions on Machine Learning Research (3/2026)
AFLOW
QM9
10
Batching time (ms)
X
dynamic
MPEU
static-2"
static-64
10
Update time (ms)
10
Combined time (ms)
4
2
1632
64
128
1632
128
Batch size
Batch size
Figure 9: Mean batching time (upper row), mean gradient-update time (middle row), and the mean combined time (bottom row) for varying batch sizes on the AFLOW (left) and QM9 (right) data using the MPEU model. For each datapoint, ten iterations of two million training steps are run.
constant algorithm performs poorly in comparison to the other algorithms, and as a result it is not used in the main text.
Student's t-tests on test metrics
For completeness, we also provide pairwise Student's t-test metrics on the final RMSE test loss. Note, the Student's t-test works best when the underlying distributions are normal (33), which is a strong assumption. For this reason, in the main text the Mann-Whitney U test is used and this approach is included for completeness. One can either the t-test statistic (critical value approach) or the associated p-value to determine whether to reject the null hypothesis (18) that the test metrics after two million training steps from different batching methods belong to the same underlying distribution. We adopt p-value criteria from the literature (33), that argues that there is little evidence that the distribution of the test metrics differ significantly if the p-values are greater than 0.10. For p-values between 0.05 to 0.10, there is moderate evidence that the distributions are different. For lower p-values the evidence strong. Ignoring the p-values that are greater than 0.10, we can look at several interesting results. For the QM9 data, SchNet model, and batch size 64, the static-2N and dynamic methods have p-value of 0.033. For the QM9 data, SchNet model, and batch size 32, the s and static-64 algorithms have p-values with the dynamic algorithm
21

---

Published in Transactions on Machine Learning Research (3 (3/2026)
Table 1: Static-constant timing results compared with the batching algorithms.
Algorithm
Dataset
Model
Batch size
Batch time (ms)
Update time (ms)
Combined
time (ms)
static-constant
AFLOW
MPEU
16
0.63
6.34
7.30
static-2
AFLOW
MPEU
16
0.49
1.89
2.38
static-64
AFLOW
MPEU
16
0.49
1.89
2.38
dynamic
AFLOW
MPEU
16
0.60
2.41
3.01
static-constant
AFLOW
SchNet
32
0.99
6.34
7.33
static-2N
AFLOW
SchNet
32
0.71
1.38
2.01
static-64
AFLOW
SchNet
0.71
1.64
2.35
dynamic
AFLOW
SchNet
0.87
1.30
2.17
static-constant
AFLOW
PaiNN
16
0.83
46.56
47.39
static-2
AFLOW
PaiNN
16
0.45
3.82
4.27
static-64
AFLOW
PaiNN
16
0.46
3.88
4.34
dynamic
AFLOW
PaiNN
16
0.59
3.20
3.79
static-constant
QM9
SchNet
32
0.78
2.02
2.80
static-2"
QM9
SchNet
32
0.74
1.55
2.29
static-64
QM9
SchNet
0.77
1.38
2.36
dynamic
QM9
SchNet
32
1.03
1.38
2.41
static-constant
QM9
SchNet
128
2.31
5.19
7.50
static-2N
QM9
SchNet
128
2.20
3.88
6.08
static-64
QM9
SchNet
128
2.24
2.39
4.63
dynamic
QM9
SchNet
128
3.38
2.13
5.51
static-constant
QM9
PaiNN
0.94
11.67
12.61
static-2
QM9
PaiNN
0.71
6.43
7.14
static-64
QM9
PaiNN
0.50
3.30
3.80
dynamic
QM9
PaiNN
16
0.68
3.33
4.01
22

---

Published in Transactions on Machine Learning Research (3/2026)
AFLOW
QM9
10
Batching time (ms)
X
dynamic
MPEU
static-2"
median
static-64
10
Update time (ms)
X
10
Combined time (ms)
4
2
0
1632 64
128
1632
128
Batch size
Batch size
Figure 10: Median batching time (upper row), median gradient-update time (middle row), and the median combined time (bottom row) for varying batch sizes on the AFLOW (left) and QM9 (right) data using the MPEU model. For each datapoint, ten iterations of two million training steps are run.
of 0.067 and 0.053, respectively. For QM9 data, MPEU model, and batch size 64, the static-2" and dynamic methods have a p-value of 0.071. These results show that for some datasets, models, and batch sizes, the dynamic algorithm gives significantly different test metrics than the static algorithm. For the majority of combinations, however, the results are similar, which may be due to use of the Adam optimizer whose adaptive estimation of first and second moments may reduce the effect of occasionally having smaller batch sizes. Note, that unlike in the main text, Bonferroni correction is not used in this analysis, which means we are not scaling the p-value criteria based on the number of pairwise tests that are run.
Fig. 16 shows a heatmap of pairwise t-test values and Fig. 17 shows the associated p- across batching methods for the SchNet model, batch size 64, and QM9 dataset. Fig. 18 and Fig. 19 show a pairwise t-test heatmap of values and associated p-values respectively for the SchNet model, batch size 64 and AFLOW dataset. The complete set of plots for the other combinations of datasets, models and batch sizes can be found in the Data Availability section.
J Mann-Whitney U-Statistic.
We show here pairwise heatmap for the statistic values in Fig. 20 and associated p-values in Fig 21. The complete set of plots for the other combinations of datasets, models and batch sizes can be found in the Data Availability section.
23

---

Published in Transactions on Machine Learning Research (3 (0/2026)
8
dynamic static-64
static-2"
Combined time (ms)
4
2-
0.25
0.5
1.25 Training steps (millions)
0.75
1.5
1.75
2.0
Figure 11: The running average of the combined time (sum of the batching step and gradient-update step) required per training step as a function the total number of training steps run. Here only a single iteration is run for the batch size 32, MPEU model and AFLOW dataset for the dynamic, static-64 and static-2" algorithms.
24

---

Published in Transactions on Machine Learning Research (3 (022026)
AFLOW
QM9
8
Batching time (ms)
X
dynamic
CPU only
6
static-2"
MPEU
static-64
mean
4
100,000 steps
2
200
Update time (ms)
150
100
50
200
Combined (ms)
150
100
50
0
1632
64
128
1632
64
128
Batch size
Batch size
Figure 12: Mean batching time (upper row), mean gradient-update time (middle row), and the mean com- bined time (bottom row) on CPU for varying batch sizes on the AFLOW (left) and QM9 (right) data using the MPEU model. The experiments were run for one hundred thousand training steps.
25

---

Published in Transactions on Machine Learning Research (3 (022
AFLOW
QM9
8
Batching time (ms)
X
dynamic
CPU only MPEU
6
static-2"
static-64
median
4
100,000 steps
2
200
Update time (ms)
150
100
50
0
200
Combined time (ms)
150
100
50
0
1632
64
128
1632
64
128
Batch size
Batch size
Figure 13: Median batching time (upper row), median gradient-update time (middle row), and the median combined time (bottom row) on CPU for varying batch sizes on the AFLOW (left) and QM9 (right) data using the MPEU model. The experiments were run for one hundred thousand training steps.
26

---

Published in Transactions on Machine Learning Research (3 (3/2026)
MPEU
SchNet
0.8
dynamic
Batch size: 16 QM9
static-2"
RMSE (eV)
0.2
0-
0.8
Batch size: 32
0.6
RMSE (eV)
0.4
0-
Batch size: 64
0.6
RMSE (eV)
0.4
0.2
0.8
Batch size: 128
RMSE (eV)
0.4
0.2
0
0
0.5
1.0
1.5
2.0
0
0.5
1.0
1.5
2.0
Steps (millions)
Steps (millions)
Figure 14: Mean RMSE values as a function of the number of training steps for the MPEU model (left) and SchNet (right) on QM9 test data for batch sizes of 16, 32, 64 and 128 (from top to bottom).
27

---

Published in Transactions on Machine Learning Research (3 (3/2026)
MPEU
SchNet
0.3
dynamic
RMSE (eV/atom)
static-2
Batch size: 16 AFLOW
0.2
0
0.3
RMSE (eV/atom)
Batch size: 32
0.1
0
RMSE (eV/atom)
Batch size: 64
0.2
0.1
0.3
RMSE (eV/atom)
Batch size: 128
0.2
0.1
0
0.5
1.0
1.5
2.0
0
0.5
1.0 1.5
2.0
Steps (millions)
Steps (millions)
Figure 15: Mean RMSE values as a function of the number of training steps for the MPEU model (left) and SchNet (right) on AFLOW test data for batch sizes of 16, 32, and 128 (from top to bottom).
28

---

Published in Transactions on Machine Learning Research (3/ (3/2026)
dynamic
1.5
2.3
Batching method
static-2" 2
-1.5
1.2
static-6a 66
-2.3
-1.2
0
-3
dynamic
static-2" Batching method
static-64
Figure 16: Heatmap of pairwise Student's t-test values on the distribution of test RMSE values for the SchNet model, QM9 data and a batch size of 64.
1.0
dynamic
1
0.14
0.033
0.8
Batching method
static-2" 2
0.6
0.14
0.23
0.4
static-64
0.033
0.23
1
0.2
0.0
dynamic
static-2" 2
static-64
Batching method
Figure 17: Heatmap of p-values from the pairwise Student's t-test on the distribution of test RMSE values for the SchNet model, QM9 data and batch size of 64.
29

---

Published in Transactions on Machine Learning Research (3/ (3/2026)
dynamic
1.1
1
Batching method
static-2 2
-1.1
0.14
static-6i 64
-1
-0.14
0
-3
dynamic
static-2"
static-64
Batching method
Figure 18: Heatmap of pairwise Student's t-test values on the distribution of test RMSE values for the SchNet model, AFLOW data and a batch size of 64.
1.0
dynamic
1
0.3
0.32
0.8
Batching method
static-2" 2
0.6
0.3
0.89
0.4
static-64
0.32
0.89
1
0.2
0.0
dynamic
static-2" 2
static-64
Batching method
Figure 19: Heatmap of p-values from the pairwise Student's 's t-test on the distribution of test RMSE values for the SchNet model, AFLOW data and batch size of 64.
30

---

Published in Transactions on Machine Learning Research (3/ (3/2026)
100
dynamic
0.0
100.0
100.0
80
Batching method
static-2 22
60
100.0
0.0
46.0
40
static-64 64
100.0
46.0
0.0
20
-0
dynamic
static-2" 2"
static-64
Batching method
Figure 20: Heatmap of pairwise Mann- Whin-Wy U test values on the distribution of test RMSE values for the SchNet model, AFLOW data and batch size of 16.
1.00
dynamic
1.000
0.002
0.002
0.80
Batching method
static-2" 2
0.60
0.002
1.000
1.000
0.40
static-6A 64
0.002
1.000
1.000
0.20
0.05
0.00
dynamic
static-2"
static-64
Batching method
Figure 21: Heatmap of pairwise Mann-Wiin Whitney U test associated p-values with Bonferroni correction on the distribution of test RMSE values for the SchNet model, AFLOW data and batch size of 16.
31


Reviews:
[{"review_id": "IuajNXRjS6", "reviewer": "Reviewer_YPEB", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "**Summary:** This paper empirically studies the effect of different batching algorithms in training the graph neural network on training time and model performance. Specifically, they conduct the experiments over 4 batching algorithms over 3 different GNN methods and two datasets. The paper suggests using the dynamic batching algorithms as it shows good performance with fewer training steps. \n\n**Strength:**\n\n- Overall, the paper studies an interesting topic and provides evidence and suggestions about which batching method we should use in training GNNs.\n\n**Weakness:**\n\n- The presentation can be improved. For example, in section 3, before presenting batching algorithms, providing more information about GNN and its input could help the audience easier to understand batching algorithms.\n\n- Some typos and inconsistencies between the text and figures.", "claims_and_evidence_yn": "No", "claims_explanation": "- The paper presents an extensive set of experiments across 4 distinct batching algorithms with 3 popular GNN methods and benchmarks. Overall, the results indicate that the static-constants batching algorithm is the slowest and suggest that using dynamics batching would have good performance while resulting in faster training. \n\n- However, the performances of the methods and the statistical test in AFLOW are not included in the paper.  \n\n- Furthermore, the results with a bigger batch size, like 256 or 512, should strengthen the claims in the paper.", "audience_yn": "Yes", "audience_explanation": "I believe this paper presents compelling claims about the effect of the batching algorithms on training time and performance of GNNs. Specifically, the experiments in the paper could provide useful empirical evidence about how to choose the right batching algorithm for the other works in GNN. The suggestion about using dynamic batching is also good.", "requested_changes": "**Major comments:**\n\n- It could be easier for the readers to introduce the normal form of the input for the graph neural network first, before introducing the batching algorithms in Section 3.\n- Could you include the training performance of different methods in AFLOW dataset. \n\n\n\n**Minor comments:**\n\n- In Algorithm 3, the batch size B is introduced but is not used.\n\n- In Algorithm 5, are N graphs sampled randomly from the list of training graphs?\n\n- Fig. 2 is not discussed in the main text of the paper. \n\n- Figure 3 and the main text do not seem to match. Specifically, the paper said that “for the batch size of 32, …,  the static-64 algorithm recompiles 89 times”, but Figure 3 left shows that the recomplications of static-64 are higher than 100. \n\n- Some typos, e.g “static-2N” should be “static-$2^N$” in page 8.\n\n- Could you conduct more experiments with a bigger batch size like 256 or 512?", "broader_impact_concerns": "No concerns.", "additional_comments": "No.", "extra_scores": {}, "extra_text": {}}, {"review_id": "Vp7wgF4MdJ", "reviewer": "Reviewer_vC23", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work is an empirical and implementation‑oriented study of batching strategies for GNNs. It defines static and dynamic batching on GNN and empirically measure each configuration on two datasets.", "claims_and_evidence_yn": "Yes", "claims_explanation": "It has clear algorithmic description for both static and dynamic batching for GNNs, and experimental setup is clear. However, I notice a few issues.\n\n1. Multi-GPU is incomplete.\n2. T-test is used for significance testing, but they did not examine normality or variance equality nor adjust for multiple testing when running pairwise t‑tests across many configs.\n3. Figures are not sized and aligned properly.", "audience_yn": "No", "audience_explanation": "The paper only tested two dataset to make the conclusion, and did not adjust multiple testing. This makes its results somehow optimistic without correction.", "requested_changes": "More datasets shall be explored and significance testing shall be carefully examined and used with justification.", "broader_impact_concerns": "The paper only examined two datasets, so conclusion may not be convincing.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "bWR6vosvh0", "reviewer": "Reviewer_ZLAd", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "**Summary**\n\nThis paper presents a systematic empirical comparison of four batching algorithms for graph neural networks: static-constant, static-$2^n$, static-64, and dynamic batching. The authors benchmark these approaches across QM9 and AFLOW datasets, three GNN architectures (SchNet, MPEU, PaiNN), and multiple batch sizes. They measure batching time, update time, combined time, and recompilation counts, and conclude that static-constant is generally impractical and that dynamic batching tends to be the fastest.\n\n**Strengths**\n\n1. Practical and application-relevant investigation:  This work investigates a practical problem in GNN training pipelines: how the batching strategy affects runtime efficiency. The paper focuses on batching algorithms that are used in widely deployed GNN libraries (Jraph...), making the study directly relevant to practitioners.\n\n2. Comprehensive evaluation: the authors evaluate four batching algorithms, three GNNs used molecular/material (SchNet, MPEU, PaiNN), two datasets with different size distributions (QM9, AFLOW), multiple batch sizes, and multiple training setups (single GPU, multiple GPUs, CPU only).\n\n**Weaknesses**\n\n1. Algorithms are informal, with missing/unclear definitions: e.g., (1) Algorithm 1 is introduced before the paper has defined the basic notations required to understand it. (2) Terms such as senders, receivers, and padding targets have not yet been introduced clearly, and their semantics only appear later in the appendix. Presenting an algorithm before defining the objects it operates on makes the exposition non-self-contained and difficult to follow, especially for readers not already familiar with Jraph’s implementation. A clearer notation setup in the main text would substantially improve the rigor and accessibility of the algorithmic presentation.\n\n2. Method description lacks clarity in critical points (e.g. preprocessing timing). Several methodological elements that affect experiment interpretation are underspecified: The preprocessing step for dynamic batching (sampling ~1000 graphs) is described in text, but its cost is not reported, and whether it is included in timing. Also, how does the number of graphs in the subset affect batching time and test metrics?\n\n\n\n3. The paper documents that lots of differences in batching strategies occur but does not provide a causal explanation or identify the underlying dataset/model properties responsible. As a result, while the observations are empirically correct, the broader claims about differences “in some combinations” lack deeper justification or practical takeaways, e.g., \"The majority of experiments show no significant difference in model learning ... but for some combinations ..., the dynamic batching algorithm yields significantly different test metrics than the static algorithms\". It would strengthen the contribution to (1) identify common factors behind the combinations where significant differences occur (e.g., dataset size, model, batchsize, ...), and (2) turn these into concrete, condition-based guidance, for example, simple rules or a summary table indicating when dynamic vs. static batching is likely to behave differently in terms of learning metrics.", "claims_and_evidence_yn": "No", "claims_explanation": "Most of the claims are well supported by the experimental results. The timing and learning-performance claims are backed by extensive experiments across datasets, models, and batch sizes.\n\nThe minor concern is that dynamic batching includes a preprocessing step (sampling graphs to estimate node/edge budgets), but the paper does not report this cost. It remains unclear whether this overhead is included in the timing results, and how significant it is in practice. Clarifying this point would provide a more complete picture of the cost of dynamic batching.", "audience_yn": "Yes", "audience_explanation": "This paper will be of interest to: Practitioners and researchers using JAX-based GNN frameworks (Jraph, TF-GNN), where batching and padding behavior directly affects training speed.", "requested_changes": "1. Clarify algorithm notation: define all key terms/notations before algs. Ensure all algorithms are self-contained with consistent notation and explicit inputs/outputs.\n\n2. Integrate key timing analyses from the appendix into the main text: Mean/median timing differences, CPU only timing, ... These are important to interpret main results and worth to be placed in the main text.\n\n3. Clarify preprocessing cost of dynamic batching.\n\n4. Optional but helpful. Explain the “some combinations” where learning metrics diverge. Identify what dataset/model characteristics cause the significant differences between dynamic and static batching. Provide insight into why these cases occur and summarize a short “recommendation table” or conditional rules of thumb.", "broader_impact_concerns": "na", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)