Towards Efficient Training of Graph Neural Networks: A Multiscale Approach
Authors: Eshed Gal
Moshe Eliasof
Carola-Bibiane Schönlieb
Ivan Kyrchei
Eldad Haber
Eran Treister
Venue: Accepted by TMLR ()

Abstract: Graph Neural Networks (GNNs) have become powerful tools for learning from graph-structured data, finding applications across diverse domains. However, as graph sizes and connectivity increase, standard GNN training methods face significant computational and memory challenges, limiting their scalability and efficiency.
In this paper, we present a novel framework for efficient multiscale training of GNNs. Our approach leverages hierarchical graph representations and subgraphs, enabling the integration of information across multiple scales and resolutions. By utilizing coarser graph abstractions and subgraphs, each with fewer nodes and edges, we significantly reduce computational overhead during training. Building on this framework, we propose a suite of scalable training strategies, including coarse-to-fine learning, subgraph-to-full-graph transfer, and multiscale gradient computation.
We also provide some theoretical analysis of our methods and demonstrate their effectiveness across various datasets and learning tasks. Our results show that multiscale training can substantially accelerate GNN training for large scale problems while maintaining, or even improving, predictive performance.

Published in Transactions on Machine Learning Research (11/2025)
Towards Efficient Training of Graph Neural Networks: A Multiscale Approach
Eshed Gal
eshedg@post.bgu.co
Faculty of Computer and Information Science
Ben- Gurion University of the Negev
Moshe Eliasof
mc-5320cam.cce
Department of Applied Mathematics and Theoretical Physics University of Combridge
Carola-Bibiand Bibiane Schönlieb
cbs31 @com.ac. uk
Department of Applied Mathematics and Theoretical Physics University of Combridge
Ivan I. Kyrchei
beankyychei226@@@mallloom
Pidstryhach Instátute for Applied Problems of Mechanics and Mathematics NAS of Ukraine, viv, Ukraine
Eldad Haber
chaber Glober@ecos.utee
Department of Earth, Ocean and Atmospheric Sciences University of British Columbia
Eran Treister
eranl-bguu.c.
Facully of Compuler and Information Science
Ben- Gurion University of the Negev
Reviewed on OpenReview: http:///orrrriiteeeeer////eeeted
Abstract
Graph Neural Networks (GNNs) have become powerful tools for learning from graph- structured data, finding applications across diverse domains. However, as graph sizes and connectivity increase, standard GNN training methods face significant computational and memory challenges, limiting their scalability and efficiency. this paper, we present novel framework for efficient multiscale training of GNNs. Our approach leverages hierarchical graph representations and subgraphs, enabling the integration of information across multiple scales and resolutions. By utilizing coarser graph abstractions and subgraphs, each with fewer nodes and edges, we significantly reduce computational overhead during training. Building on this framework, we propose suite of scalable training strategies, including transfer, and multiscale gradient computation.
coarse-to-fine learning, subgraph-to-ffll--gae We also provide some theoretical analysis of our methods and demonstrate their effectiveness across various datasets and learning tasks. Our results show that multiscale training can substantially accelerate GNN training for large-scale problems while maintaining, even improving, predictive performance.
1 Introduction
Largo-scale graph networks arise diverse fields such as social network analysis (Kipf Welling 2016
Defferrard et al. 2016), recommendation systems (Tang al.) 2018). and bioinformatics (Jumper
2021
where relationships between entities are crucial to understanding system behavior (Abadal et. al. 2021
These

---

Published in Transactions on Machine Learning Research (11/2025)
(a) Random: Fine
(b) Random: Coarse
(c) Topk: Fine
(d) Topk: Coarse
Figure 1; Graph coarsening examples. Left two: random pooling; Right two: Topk pooling (highest-degree
nodes selected). Orange nodes indicate selected coarse nodes.
problems involve processing large graph datasets, necessitating scalable solutions. Some examples of those problems include identifying community structures social networks (Hamilton et al. 2017 optimizing traffic flow in smart cities (Geng et al. 2019), or studying protein-protein interactions in biology (Xu et al, 2024. In all these problems, elficient processing of graph data is required for practical reasons.
In recent years, many Graph Neural Networks (GNNs) algorithms have been proposed to solve large graph problems, such as GraphSage (Hamilton et al. 2017 and FastGCN (Chen et al. 2018). However, despite their success, when the involved graph is of a large scale, with many nodes and edges, it takes a long time and many resources to train the network Gaalmé et al. 2021
In this paper, we study and develop multiscale training methodologies, to improve the training efficiency of GNNs. To achieve that, we focus the standard approach for implementing GNNs, that is, the Message- Passing (MP) scheme [Gilmer et al. 2017). Mathematically, this approach is equivalent to the multiplication of the graph adjacency matrix A with node feature maps ((), which is then multiplied by a learned channel- mixing weight matrix W()), followed by an application of an element-win wise nonlinear activation function follows:
= =(XX
(1)
Assuming that the adjacency matrix is sparse with average sparsity of k neighbors per node, and channel-mixing matrix of shape c,
focus on the total multiplications in the computation of Equation ( to which we refer to as 'work'. In this case, the work required in a single MP layer is of order O(n k- e²), where is the number of nodes in the graph, and is the number of channels. For large or dense graphs, i.c., n or are large, the MP in Equation [ requires a high computational cost. Any algorithm that works with the entire graph at once therefore bound to face the cost of this malrix-vector product.
A possible solution to this high computational cost involves the utilization of multiscale framework, which requires the generation of surrogate problems to be solved [Kolaczyk & Nowak 2003). If the so- lution of the surrogate problem approximates the solution the fine (original) scale problem, then it can be used as an initial starting point, thus reducing the work required to solve the optimization problem. In the context of graphs, such multiscale represen- tation of the original problem graph coarsening. Graph coarsening algorithms are techniques that. reduce the size of graphs while pre- serving their essential properties (Cai et al. 2021). These methods aim to create smaller, more compu- tationally manageable graphs by choosing subset of the nodes the edges, based certain policies,
be achieved using
(a) Fine original graph
(b) Coarse graph
Figure 2: Coarse graph using subgraph pooling, Or- ange nodes indicate selected coarse nodes; the red node is the root for the ego-network.
2

---

Published in Transactions on Machine Learning Research (11/2025)
by using subgraph of the original graph, and may be implemented in various ways.
We demonstrate the generation of a coarse graph using random pooling and Topk pooling (which is based on the nodes with high degrees in the graph) in Figure [ As for choosing a subgraph, this can be implemented using node-based selection methods, such as ego-net (Gupta et al. 2014) or node marking and deletion (Frasca et al. 2022), which offer straightforward ways to extract subgraphs based on specific nodes of interest. We show an example of subgraph coarsening in Figure 2
Contribution. In this paper, we propose novel graph multiscale optimization algorithms to obtain compu- lationally efficient GNN training methods that retain the performance of standard gradient-based training of GNNs. Our approach is based on a multiscale framework that utilizes various graph coarsening techniques. While coarsening and subgraph methods have been in use for various purposes, from improved architectures Gao & Ji 2019), to enhanced expressiveness (Bevilacqua et al., 2022 2024), to the best of our knowledge, they were not considered in the context of efficient. training of GNNs. Therefore, in this work, we explore three novel training mechanisms to design surrogate processes that approximate the original training problem with fraction of the computational cost, using a multiscale approach. All the methods attempt to use the full graph with nodes as little as possible, and focus on training the network using coarse-scale graphs thus roducing computational cost. The three methods, detailed in Section are as follows:
Coarsc-to-Fine. coarsc-to-fine algorithm approximates the optimization problem on smaller, pooled graphs. the coarse-scale solution is similar to the fine-scale one, it can serve as an effective initialization for the fine-scale optimization process. Since gradient-based methods couverge faster when initialized near the solution (Arora al., 2018 Boyd & Vandenberghe 2004), this approach reduces the overall number
of iterations required.
Sub-to-Full. This approach leverages a sequence of subgraphs that progressively grow the original fine- scale graph. Like coarsening, each subgraph is significantly smaller than the original graph, enabling faster optinization process. Subgraphs can be generated based on node distances (if available) ego-network methods (Gupta et al. 2014).
Multiscale Gradients Computation. Building the previous methods and inspired by Multiscale Monte Carlo Giies 2001 which is widely used for solving stochastic differential equations, we introduce mechanism to approximate fine-scale gradients at fraction of the computational cost, achieved by combining gradients across scales to efficiently approximate the fine-scale gradient.
To demonstrate their effectiveness, we benchmark the obtained performance and reduce computational costs on multiple graph learning tasks, from transductive node classification to inductive tasks. Our code is available here: hutpss//giihub..cm//eeeeee (eahhegalll/GaaaMMllliiee /ruaphhuliisclee..
2
Related Work
We discuss related topies to our paper, from hierarchical graph pooling to multiscale methods.
Graph Pooling. Graph pooling can be performed in various ways, as reviewed in Liu et (2022). Popular pooling techniques include DifiPool (Ying et al. 2018) MaxCutPool (Abate & Bianchi 2024 graph wavelet.
compression (Eliasof et al. 2023), well as Independent-Set Pooling (Stanovic et al. 2024h. all aiming to
produce coarsened graph that bears similar properties to the original graph. Wang et al. (2020) and Wang et al. (2024) introduce the concept of subgraph pooling to GNNs. Bianchi & Lachi (2023) study the expressive power of pooling in GNNs. Here, we use graph pooling mechanisms, as illustrated in Figure Figure as core components for efficient training.
General Multiscale Methods. The idea of Coarso-to-Fine for the solution of optimization problems has been proposed in Brandt (1984), while a more detailed analysis of the idea in the context of optimization problems was introduced in Nash (2000). Multiscale approaches are also useful in the context. of Partial
3

---

Published in Transactions on Machine Learning Research (11/2025)
Li et al. 2024b, well as for various applications, including flood Lastly, Monte- Carlo methods (Hastings 1970 have also been adapted to a multiscale framework (Giles 2001]. 1, which can be used for various applications such as electrolytes and medical simulations LLangg et al. 2015 Klapproth et al., 2021]. In this paper, we draw inspiration from multiscale methods to introduce a novel and elficient GNN training framework.
Differential Equations [Guo & Li
2024
modeling (Bentivoglio et al. 2024
Multiscale Learning. In the context of graphs, the recent work in Shi et al. matching problem can be solved using coarse-to-fie strategy, and Cai et al. coarsening strategies. Other innovations include the recent work of Li et al. (2024a). whose goal is to mitigate the lack of labels in semi-supervised and unsupervised settings. We note that it is important to distinguish between algorithms that use a multiscale structure as a port of the network, such as Unet (Ronneberger et al.) 2015) and Graph-Unet (Gao & Ji. 2019), and malliscale training that utilizes mullisonle for training, such as the recent work on convolutional neural networks in Ahamed al. (2025). In particular, the latter, multiscale training, which is the focus of this paper, can be potentially applied to any GNN architecture.
(022 has shown that
2021 proposed different
3 Preliminaries and Notations
We now provide related graph-learning notations and preliminaries that we will use throughout this paper. Let us consider a graph defined by G = E) where V is a set of nodes and & is a set of m edges. Alternatively, a graph can be represented by its adjacency matrix A, where the (i,j)-th entry is if an edge between nodes and j exists, and otherwise. We assume that each node is associated with a feature vector with e channels, x, RR and X = ....... [x E Raxe the node features matrix. We denote by y; R the i-th node label, and by Y [Yo. -
V vn- Raxd the label matrix.
In typical graph learning problem, we are given graph data (G, X, Y), and our goal is to learn function f(X, 0) with learnable parameters O such that
Y= f(X,9:0)
(2)
For instance, in the well-known Graph Convolution Network (GCN) (Kipf & Welling 2016), each layer is defined using Equation [1, such that ((+1) the output of the I-th layer and serves as the input for the (1 + 1)-th layer. For a network with L layers, we denote by 0 (W()). ...., w(D), the collection of all learnable weight matrices within the network.
It is important to note that the matrix W(O) is independent of the size of the adjacency and node fcature matrices, and its dimensions only correspond to the feature space dimensions of the inpul and oulput layer. In terms of computational costs, a the I-th GCN layer requires the multiplication of the three matrices A, x and W
We note that the number of multiplications computed in the I-th GCN layer is directly affected by the size of the node features matrix ( and the graph adjacency matrix A. This means that for a very large graph G, each layer requires significant computational resources, both terms of time and memory usage. These computational costs come into effect both during the training process and at the inference stage. In this paper, we directly address these associated costs by leveraging GNN layers with smaller matricos A, and x(") yielding trained networks whose downstream performance is similar to those that would have been achieved had we used the original input graph and features, while significantly reducing the computational costs.
Lastly, we note that, as shown in Section D architectures.
approach is general and applicable to various GNN
4 Efficient Training Framework for GNNs
We consider the graph-learning problem in Equation (2) defined on a graph with n nodes. To reduce the cost of the learning process, we envision a sequence of R surrogate problems, defined by the set of data tuples (G,,, X,, Y.) and problems of the form
Overarching goal.
Y, f(X,, Gr:0), r= 1,....,, R
(3)
4

---

Published in Transactions on Machine Learning Research (11/2025)
Algorithm 1 Multiscale algorithu
# R total number of levels. Initialize weights 0 and network f. for r= == R, R 1.......,1 do
Compute the coarse graph G, that has ny nodes. Compute the coarse node features & labels (X,, Y,).
Solve the optimization problem in Equation Update the weights 0.
end for
where f is a GNN and Gr is a graph with n, nodes, where n = ny > n22 > ... > nu. Each graph G, is obtained by pooling the fine graph 9, using a smaller set of nodes compared to the previous (finer) graph Gr-1. X, is the node features matrix of the graph Gr, and Y, is the corresponding label matrix. In what follows, we explore different ways to obtain 9r, X, and Y, from the original resolution variables.
An important feature of GNNs is that the weights 0 of the network are independent of the input and output size. Each weight matrix ((() is of dimensions Can X Cout, for X E Rexei node feature matrix of the layer I, and ((+1) E no for the corresponding dimensions of the next layer 1+ This means that changing the number of nodes and edges in the graph will not affect the weight matrix dimensions while reducing the number of computational operations conducted in the learning process. That is, the set parameters 0 can be used on any of the R problems in Equation Let us reformulate the learning problem on the r-th level
as follows:
(4) where is the loss function that measures the error in Equation 3 and the expectation is on the data (X,, Y,). Our goal is to use the optimization problem in Equation (4) as a cheap surrogate problem
0, argming
Gr; Y,),,
obtain an approximate solution (i.e., weights 0) of the original problem in Equation (2). We describe three
methodologies to achieve this goal.
4.1 Coarse-to-Fine Training
One seemingly simple idea is to solve the r-th surrogate problem in Equation 4 and the obtained weights to solve the problem the (r 1)-th graph, that is Gr-1, which is larger. This approach can be beneficial only if 0,+1 = 0r, that is, the obtained optimal weights are similar under different scales, and if the total amount of work the coarse scale is smaller than the fine one, such that it facilitates computational cost. roduction. The multiscale framework is summarized in Algorithm
While Algorithm [ is relatively simple, it requires obtaining the r-th surrogate problem by coarsening the graph, as well as a stopping criterion for the r-th optimization problem. Generally, in our coarse-to-fine method, we use the following graph coarsening scheme:
A, P A'P,
(5)
where P, E Raxxn is a binary injection matrix that maps the fine graph with n nodes to a smaller graph with ny nodes. That is, we choose a submatrix of the p-th power of the adjacency matrix A. We use X, = P.Tx as the coarse graph node features matrix and Y, = P!Y as the label matrix of the coarse graph.
Choosing the non-zero entries of the matrix Pr, which are the nodes that remain in the graph after pooling, can be implemented in various ways. For example, P, can be obtained by sampling random nodes of the fine graph G, illustrated in Figure [ (left), or with some desired pooling logic, such as choosing nodes of the highest degree in the graph, which is method often described as Topk pooling (Huang et al. 2015) and we present an raluson of it in Figure [ (right). The power in Equation (5) controls the connectivity of the graph, where p = 1 is the original graph adjacency matrix, and using a value p > 1 will enhance the connectivity of the graph prior the coarsening step. We note that the range of applicable choices for p is subject to the given data, A, has to contain fewer edges than the original matrix in order for the resulting graph to be coarse.
5

---

Published in Transactions on Machine Learning Research (11/2025)
In Algorithm 1 apply GNN layers such in Equation L using the coarsened feature matrix X, and its corresponding adjacency matrix Ar, that is based desired pooling ratio. We note that there is no re-initialization of the weight matrix W when transferring from coarse grids to finer ones, allowing a continuous learning process along the steps of the algorithm. As we show later in our experiments in Section g our coarse-to-fine training approach, summarized in Algorithm 1 is not. restricted a specific GNN architecture or a pooling technique.
4.2 Sub-to-Full Training
We now describe an additional method for efficient training on GNNs, that relies on utilizing subgraphs, called Sub-to-Full. At its core, the method is similar to Coarse-Lo-Fine, as summarized in Algorithm | using the same coarsening in Equation such that the nodes X, form a subgraph of the original graph G.
5 However, its main difference is that we now choose the indices of P,
A subgraph can be defined in few possible manners. One is by choosing a root node j, and generating subgraph centered around that node. If the graph g has some geometric features, meaning each input node corresponds to point in the Euclidean space Rd one might consider constructing a subgraph with n, nodes to be the n, nodes that are closest to the chosen center node in terms of Euclidean distance norm. Otherwise, we can consider a subgraph created using an ego-network (Gupta et al., 2014) of k- hop steps, such that we consider all neighbors that are steps away from the central node. An illustration of this method is presented in Figure 2 The number of hops, k is a hyperparameter of the algorithm process, and we increase as we go from coarse levels fine ones within the series of graph {9r)
4.3 Multiscale Gradients Computation
In Section 4.1 Section 4.2 we presented two possible methods to facilitate the efficient training of GNNs via a multiscale approach. In what follows, we present a complementary method that builds on these idcas, as well as draws inspiration from Monte-Carke optimization techniques GGiles 2001 to further reduce the cost of GNN training.
We start with the observation that optimization in the context of deep learning dominated by stochastic gradient-based methods (Kingma & Ba 2014 Ruder 2016). At the base of such methods, stands the gradient
descent method, where at optimization step
we have:
0.+1 0, M.,-,.EVVVLE E.E(
However, when large datasets are considered, becomes computationally infeasible to take the expectation of the gradient with respect to the complete dataset. This leads the stochastic gradient descent (SGD) method, where the gradient is approximated using smaller batch of examples of size B, oblaining the following weight update equation:
B VL ([(x))))((0)))) .))
Hs B
0 0,
where (0) ( denote the node ures and graph structure of the b-th element in the batch.
To compute the update Equation 7 one is required to evaluate the loss L for B times on the finest graph G1 g, which can be computationally demanding. To reduce this computational cost, we consider the following telescopic identity:
EL
(0) + + (C! (L) .... + +.. +E +
(0)
(8)
where ( (f(X,, 0), Y,), represents the loss of the r-th scale the s-th optimization step. Equa- tion induces a hierarchical stochastic gradient descent computation based on the set of multiscale graph (g, 1=1-

---

Published in Transactions on Machine Learning Research (11/2025)
Algorithm 2 Multiscale Gradients Computation
# R total number of levels.
Initialize L = 0 for r = 2,...,R. R do
Sample M, node subsets Compute 1, Y,, y
Compute the graphs Gr and 9--1
Update L t C TT, (C(r-1) ((r)
end for
Compute ((n) and update L t L + 1 L(R) Mu
The main idea of the hierarchical stochastic gradient descent method in Equation ( in the telescopic sum can use a different number of realizations for its approximation. The assumption is that can choose hierarchies such that
is that each expectation
chr- (o) - S 5. (0)
(9)
for some 1> >/- > 71-1 >...>>>>>>> 71 .1 = 0. To understand why this concept is beneficial, let us consider the discretization of the fine-scale problem (i.e., for 91) in Equation 4 using . samples. Suppose that the error in approximating the expected loss at the first, finest scale B[c!"] by the samples behaves as e(1) = c Now, let consider 2-scale process, with M2 samples the finest resolution graph. The error in evaluating the loss on the second scale B[C!2) with M2 examples given by e(2) = VMz while the error in evaluating B[(CE)) - o is e112) Cn Since the assumption is that 7 < 1, we are able to choose M2 and Mi VM
V
such that e()) = e + (1.2). Hence, rather than using M finest graph resolution observations, we use only M1 < M such observations and sample the coarser problems instead.
More broadly, we can generalize the case of a 2-scale optimization process to an R multiscale process, such that an approximation of the fine-scale loss c(1) is obtained by:
E (0)
J. Σ c
R2 (0)
MR M Σ (c. (R-1) (0) c
c. (0)
+
1 Σ () -2. (c) (10) M,
where MR > MR-1 > M1. This implies that for the computation of the loss on the fine resolution 91 G, only a few fine graph approximations are needed, and most of the computations are done on coarse, small graphs. Our method is summarized in Algorithm 2 and we provide an illustration of the computation process in Figure Naturally, this method suits inductive learning lasks, where multiple graphs are available.
4.4 A Motivating Example
For multiscale training methods to adhere to the original learning problem, the condition in Equation 0 needs to be satisfied. This condition is dependent on the chosen pooling strategy. To study the impact of different pooling strategies, we explore a simplified model problem. We construct a dataset, illustrated in Figure 4 that consists of structured synthetic graphs representing rods with three distinct types: (1) rods with both endpoints colored blue, (2) rods with both endpoints colored yellow, and (3) rods with mismatchod endpoint colors. We wish to classify each node either part of rod background and to identify the rod type.
A key challenge in this task is ensuring that the model can effectively capture global structure. To accurately classify rod, a graph neural net net work (GNN) must aggregate information from both ends of the structure. A shallow GNN, such as one with a single or two-layer message-pasee passing scheme, is insufficient, as its receptive field does not extend across the entire rod. Additionally, an effective coarsening strategy must preserve key structural coommation-reaanninn nodes from both rod endpoints, central regions, and the background maintain an informative representation.
7

---

Published in Transactions on Machine Learning Research (11/2025)
1
(0)
I(C
M
M2
M3
(
Figure 3: Illustration of the Multiscale Gradient Computation algorithu introduced Section 4.3
Table 1: Experiments for different coarsening strategies, creating coarse graph with 1 of the original nodes. Loss discrepancy between the loss computed the original graph, and the pooled graph (Aloss) is calculated using in Equation (9) and p denotes the power in Equation
Coarsening type
# edges
Aloss (initial)
Aloss (final)
No coarsening
41,358
0
Random(p=1)
3,987
7.72e-01
3.45e-01
Random(p=22
6,673
5.62e-02
1.25c-02
Topk(p=1)
8,731
1.41e-01
2.53e-02
Topk(p=2)
20,432
3.12e-03
8.64c-03
Subgraph 9,494
1.25e-01
1.93e-01
dataset of 100 rods in varying orientations and evaluate different coarsening methods for constructing reduced graph. Table presents the loss values for various coarsening strategies along with the corresponding edge counts. The results highlight another aspect of our approach: methods that do not increase connectivity prior to pooling (i.e., with 1) may, in certain situations, lead to significant variance in loss across different coarsened graphs. For this dataset, introducing additional connectivity before pooling successfully mitigates this issue while still reducing the graph size in terms of both nodes and edges.
To explore this further, train four-layer GCN Kipf & Welling 2016)
To further evaluate the capabilitics of our multiscale methods this synthetic dataset, Appendix presents the full multiscale training results and compares them with standard fine-grid training. Additionally, we construct second synthetic dataset derived from the MNIST image dataset (LeCun et al. 1998). Results on both datasets demonstrate that our methods effectively capture global graph features while enabling efficient. training.
4.5 Theoretical Properties of Reduced Graphs
At the base of all the disvosdd techniques, the core the ability of the coarse graph to represent the fine graph. To this end, consider the least. squares model problem, in which aim to minimize over the
8

---

Published in Transactions on Machine Learning Research (11/2025)
Figure 4: The Q-tips data set. The data contains three types of "q-tip" and the goal is to label the stick type 1,2 or 3.
following loss function:
(())= = 2N O (AX YII
(11)
and assume that a is the identity mapping. One way to analyze the solution of the coarse-scale problem is through sketching (see Woodruf] (2014)) In sketching, the original problem is replaced by
1 |P AXO - Y
(0)
(12)
2N
where PT is sketching matrix. We establish the following result with its proof provided in Appendix A which applies to Sub-to-Full training, and we note is only limited to the linear case a = Td:
Theorem 4.1. Let 0" be the solution of equation i1 and let 0e be the solution of equation 12 Denote
<> O, and let PT be sporse subspace embedding with O( rows, where is the size of the feature vectors
(number of graph channels), that injects any vector into subset of indices, which we denote by C. According
to this subset, denote A Acc Acr
Xc x
and X =
Then if a = Id, there exists |||| < ||AcrX
AFC
AFF]
such that with probability .99:
1
1
(0
PT
APP
X0 - P 2 < 2N
(IAX0 Y
(13)
2N
5 Computational Complexity
As noted in Section the computational cost of GNN message-passing layers directly influenced by the size of both the node feature matrix and the adjacency matrix. This observation suggests that reducing the number of nodes and edges can lead to improved computational efficiency during training.
To illustrate this, consider Graph Convolutional Networks (GCNs) (Kipf & Welling 2016), where each layer follows the update rule defined in Equation [ When analyzing floating point--oottttoo operations (FLOPs), we account for both the input and output dimensions of each layer, as well as the sparsity of the adjacency matrix, mcaning that only nonzero entries contribute to computation. The total number of FLOPs required for a single GCN layer is given by:
(14) where & represents the set of odges, V the set of nodes, Con the input feature dimension, and Cout the output feature dimension of the layer. From this expression, follows that reducing the number of nodes and edges leads decrease in computational cost. For instance, if the number of nodes is halved, i.e., |el V) and the number of edges satisfies |&el < [8] for any pooling strategy, then the FLOPs per layer are reduced accordingly. This reduction translates into significant theoretical improvement in efficiency.
2 |8| Cin + |V] Cin Couts
Importantly, this effect is not limited to GCNs. A similar computational advantage is observed for other GNN architoctures, like Graph Isomorphism Networks (GIN) (Xu et al. 2018) and Graph Attention Networks (GAT) (Velickovic al. 2017). all cases, reducing the graph size results fewer computations per layer, demonstrating a generalizable advantage across different GNN architectures.
9

---

Published in Transactions on Machine Learning Research (11/2025)
6 Experiments
Objectives. We demonstrate that our approach achieves comparable superior downstream performance to standard GNN training on the original graph while reducing computational costs, across the training techniques outlined in Section f and the pooling strategies discussed below. We demonstrate our methods on both mid-size and large-scale datasets, showing that given a need to solve a certain graph problem, our methods will prove to be efficient given existing computational resources.
Pooling Strategies. We incorporate various coarsening techniques to enhance GNN training efficiency:
(i) Random Coarsening: Selects ny random nodes from the graph and constructs the new adjacency matrix A, via Equation 5
(ii) Topk Coarsening: Selects nt nodes with the highest degrees before computing A, using Equation (5
This approach may be computationally expensive for large graphs, as requires full degree information.
(iii Subgraph Coarsening:
Samples a random node j and selects its nearest ny neighbors. If geometric positions are available, the selection is based on Euclidean distance; otherwise, k hop ego network is used. This method caplures local graph structures but may overlook global properties.
(iv) Hybrid Coarsening: Alternates between Random and Subgraph Coarsening to leverage both local and global components of the graph.
Further details the coarsening methods are provided in Appendix
Experimental Settings. We evaluate multiscale training with 2, 3, and 4 levels of coarsening, reducing the graph size by factor of 2 at each level. Training epochs are doubled at each coarsening step, while fine-grid epochs remain fewer than in standard training. For multiscale gradient computation, we coarsen the graph to retain 75% of the nodes, perform half of the training using the coarsened graph, and then transition to fine-grid training. We validate our methods on node-level tasks, demonstrating that incorporating coarse levels during training enhances inference performance.
6.1 Transductive Learning
We evaluate our approach using the OGEN-Arxiv dataset Hu et al. 2020) with GCN (Kipf & Welling 2016), GIN Xu et al. (2018), and GAT (Velickovic et al. 2017). This dataset consists of 169,343 nodes and 1,166,243 edges, where nodes represent individual computer science papers, edges correspond to citation relationships where one paper ciles another, and each node is labeled with the paper's primary subject area. We report test accuracy across three runs with different random seeds and include standard deviation assess robustness, and present results in Table [ demonstrating that multiscale framework achieves comparable or superior performance relative to standard training. Notably, random pooling proves to be the most effective approach for this dataset, offering both efficiency and strong performance. Given the large number of edges in this dataset, we use p 1 and report the corresponding edge counts in Table
The results highlight the substantial reduction in edges at each pooling level, significantly improving training efficiency. In Table 2 we also provide a timing analysis, including both theoretical FLOP estimates (as claborated in Section 5) and empirical runtime measurements, confirming that our multiscale approach enhances efficiency while maintaining strong predictive performance. To show the efficiency of our methods, demonstrate comparison of the convergence of the training loss between the multiscale training and fine grid training in Appendix D and we report memory consumption measurements in Appendix E
We demonstrate our results using the OGBN-MAG dataset Hu el al. 2020), which is a heterogeneous network composed of a subset of the Microsoft Academic Graph (MAG). This dataset includes papers, authors, institutions, and fields of study, of which the papers are labeled. We focus on the 'paper' nodes and the 'citing' relation, using homogeneous graph dataset of 736,389 nodes and 5,416,271 edges. We summarize our results in Table 4 where we show that using GCN (Kipf & Welling 2016), our methods achieve results
10

---

Published in Transactions on Machine Learning Research (11/2025)
Table 2: Comparison of different coarsening methods on the OGBN--rxiv dataset. The lowest (best) FLOPs count for each GNN appears in blue. Time represents the training epoch time in milliseconds. The lowest (best) time for given network and level is highlighted in green. The highest (best) performing approach in terms of test accuracy (%) is highlighted in black.
L=2
L
= 4
Method
FLOPs(M) Time(us)
Test Acc(%)
(LOPs(M) Time(ms)
Test Acc(56)
(L) Tine(uus)
Test Acc(%)
GCN
Fine Grid (I m
1.96 10a
39.70
71.76% (0.1%)
Random
9.37
26.65
72.07%
(0.1%)
4.59 x 10a
18.52
71.88%
(0.0%)
2:27 x 10
15.53
72.03%
(0.0%)
Topk
1.01 ln
31.96
71.68% (0.0%)
5.16 10a
22.01
71.44% (0.2%)
2.58 10
17.01
70.99% (0.2%)
Subgraphs
161 10a
53.71
71.06% (0.6%)
8.06 10a
53.27
71.18 (0.2%)
4.05 10a
54.46
71.13% (0.0%)
Rand. & Sub.
5.49 102
37.44
70.71% (0.2%)
2.70 102
34.50
70.93%
(0.0%)
1.34 x 10
33.77
70.53%
(0.2%)
GIN
Fine Grid (L = 1)
3.75 10
61.06
68 39% (0.6%)
Random
1.86 10*
28.
69.21% (1.0%)
9.24 10a
17:21
69.19% (1.8%)
4.61 x 10
12:23
69.30%
(0.5%)
Topk
1.89 10'
32.53
67 67%82%
(3. 2%)
9.50 10a
19.84
68. 10% (1.0%)
4.76 x 10a
13:68
67 67.90% (1.0%)
Subgraphs
2.52 x 10a
17.47
67. 67.13% (1.3%)
1.26 x 10a
17.10
66. 16% (1.6%)
6.30 x 102
17.69
66.03% (6.1%)
Rand. Sub.
1.06
39.56
65.87%
(6.6%)
5.25
31.16
64.79%
(3.0%)
2.62
32.36
67.46% (1.2%)
GAT
Fine Grid (L =
1.13 10
54.71
71.07%
(0.2%)
Random
5.32 10
30.32
71.70%
(0.5%)
2.59
10a
19.39
71.78%
(0.0%)
1.28 x 10
16.48
71.12%
(0.0%)
Topk
5.86
13.90
71.341%
(0.3%)
3.01
10a
29.72
70.68% (0.1%)
1.50 x 10a
21.26
70.633 (0.1%)
Subgraphs
5.04 10a
52.96
(1.0%)
2.52
10a
50.72
70.10% (0.6%)
10
55.15
(0.7%)
Rand. & Sub.
5,18 10
43.05
70.09% (0.2%)
2.56
10P
35.79
09.62% (0.2%)
1.27 x 10
35.33
70.06% (0.2%)
Table 3: Edge number for coarsening strategies for the coarse-to-fine method, with the OGBN-Arxiv dataset.
Fine grid edge number is in red.
Method 1st Coarsening
2nd Coarsening 3rd Coarsening
Fine Grid: 1,166,243
Random
280,009
76,559
17,978
Topk
815,023
480,404
238,368
comparable to the baseline results. We demonstrate our results for 2, 3, and 4 levels of multiscale training, and note that even though GCN is widely used and may be applied to this dataset, other more advanced architectures may yield better results than the presented baseline. However, changing a chosen model will not harm the effectiveness of our proposed methods.
To emphasize that our method is general and will prove to be effective over various datasets and methods, we use additional datasets of various sizes and show that for every one of them, our methods prove to be efficient while yielding comparable or even better results compared to the original baseline. In Appendix F we show results and details for the well-known Cora, SiteCeer, and PubMed datasets (Sen et al. 2008 We
further test our methods on Flickr (Zeng et al. 2019). WikiCS (Mernyei Cangea 2020). DBLP (Bojchovski & Günnemanng 2017), the transductive versions of Facebook, BlogCatalog, and PPI Yang al. 20200)
datasets, detailed in Appendix G We present results for Flickr and PPI, along with timing evaluations, well as full results for all those datasets and additional experimental details. Additionally, in Appendix J provide ablation studies, where we study various coarsening ratios and epoch distribution.
we
6.2 Inductive Learning
We evaluate our method on the ShapeNet dataset (Chang el al. 2015), a point cloud dataset with multiple categories, for a node segmentation task. Specifically, we present results on the 'Airplane' category, which contains 2,690 point clouds, each consisting of 2,048 points. Since this dataset contains multiple graphs, its overall size is large, and achieving an improvement in training efficiency for each batch will yield significant impact training efficiency.
11

---

Published in Transactions on Machine Learning Research (11/2025)
Table 4: Comparison of different methods for OGBN-MAG dataset. We present the test accuracy for multiscale training using GCN. Results for fine grid appear in red. L represents the number of levels.
Method +
/ Levels +
2
L L= 3
L=
Fine grid: 35.73% (0.4%)
Random
36.54% (0.5%)
36.92% (0.1%)
37.07% (0.2%)
Topk
35.46% (0.4%)
34.82% (0.3%)
34.69% (0.2%)
Subgraphs
34.19% (0.3%)
34.31% (0.4%)
34.31% (1.2%)
Rand. & Sub.
34.64% (0.3%)
34.43% (0.2%)
34.05% (0.2%)
Table Results on Shapenet 'Airplanc' datasct using DGCNN. Results represent the Mcan test IOU. Results for fine grid appear in red.
Training using Multiscale Training
Graph Density (kNN)
Fine Grid
Random
Subgraphs
Rend, & Sub.
k=6
78.36% (0.28%)
79.86% (0.87%) 79.31% (0.20%) 80.01% (0.21%)
k=10
79.71% (0. 23%)
80.41% (0.14%) 80.65% (0.15%) 80.97% (0.10%)
k=20
81.06% (0.08%) 81.37% (0.05%) 81.18% (0.32%) 81.9% (0.58%)
Training using Multiscale Gradient Computation Method
Graph Density (kNN)
Fine Grid
Random
Subgraphs
Rand. & Sub.
k=6
78.36% (0.28%) 79.42% (1.40%) 79.35% (0.12%) 79.62% (0.93%)
k=10
79.71% (0.23%) 80.00% (0.33%) 80.15% (0.71%) 80.19% (1.63%)
k=20
81.06% (0.08%) 81.30% (0.41%) 81.17% (0.25%) 81.12% (0.24%)
In this dataset, nodes are classified into four categories: fuselage, wings, tail, and engine. For training, we use Dynamic Graph Convolutional Neural Network (DGCNN) from Wang et al. (2018). which dynamically constructs graphs at each layer using the k-nearest neighbors algorithm, with k as tunable hyperparameler. We test our multiscale approach with random pooling, subgraph pooling, and a combination of both. In subgraph pooling, given that the data resides in R3, we construct subgraphs by selecting a random node and its nj nearest neighbors. Experiments are conducted using three random seeds, and we report the mean Intersectionnove---niio Our results show that for all tested values of k, applying either random or subgraph pooling improves performance compared to standard training while reducing computational cost. Furthermore, combining both pooling techniques consistently yields the best results across connectivity settings, suggesting that this approach captures both high- and low-frequency components of the graph while maintaining training efficiency. Additional results for multiscale training on the full ShapeNet dataset are provided in Appendix
(IoU) scores along with standard deviations in the top section of Table 5
We apply the Multiscale Gradients Computation method to the ShapeNet 'Airplane' category, evaluating various pooling techniques using Algorithm / with two levels. As shown in the bottom scction of Table 5 our approach reduces training costs while achieving performance comparable to fine-grid training. These results show that the early stages of optimization can be efficiently performed using an approximate loss, we propose, with finc-grid calculations needed only once the loss function nears its minimum. Additional results, timing analyses and experimental details for the ShapeNet dataset are provided in Appendix H
We further evaluate our Multiscale Gradients Computation method on the protein-protein interaction (PPI) dataset (Zitnik & Leskoved 2017), consisting of graphs representing different human tissues. The results, with additional details, are provided in Appendix]] [ Furthermore, Appendix also presents results on the NCT1 (Wale et al. 2008) and ogbg-molhiv (Hu et al. 2020) datasets.
12

---

Published in Transactions on Machine Learning Research (11/2025)
7 Limitations
While our methods are broadly applicable across various models, datasets, and architectures, several limitations remain. Although our multiscale approach successfully avoids using the fine graph for most of the training process, a few iterations must still be conducted on the original fine grid, which requires it to fit in memory. Additionally, while the different variants of our method generally yield comparable performance, we currently lack a principled, deterministic criterion for selecting the optimal variant given setting, and specific cases may cause relatively high variance in certain datasets and methods. However, in this paper, we focus the framework and the overall multiscale concept. Lastly, the theoretical analysis presented in this paper is restricted to the linear case (i.e., = Id). We believe further theoretical investigation may be performed to extend the analysis to nonlinear settings.
8 Conclusions
In this work, we propose an efficient approach for training GNNs and evaluate its effectiveness across multiple datasets, architectures, and pooling strategies. We demonstrate that all proposed approaches, Coarse-to-Finee Sub-to-Full, and Multiscale Gradient Computation, improve performance while reducing the training costs.
Our method adaptable any architecture, dataset, or pooling technique, making a versatile tool for GNN training. Our experiments reveal that no single setting universally optimal; performance varies depending on the dataset and architecture. However, our approach consistently yields improvements across all considered settings for each dataset and network, least coarsening method led to improved performance, while the differences between methods were generally minor in practice.
9 Acknowledgements
This research was supported by the Ministry of Innovation, Science and Technology, Israel, and the Ministry of Education and Science of Ukraine as the Joint Ukrainian-Israel research project (under contract M/38-2022 2024, No. 0124U003312).. Also, this research was partially supported by Grant No. 2023771 from the United States-Israel Binational Science Foundation (BSF) and by Grant No. 2411264 from the United States National Science Foundation (NSF).
13

---

Published in Transactions on Machine Learning Research (11/2025)
References
Abadal, S., Jain, A., Guirado, R., López- Alonso, J., and Alarcón, E. Computing graph neural networks: A survey from algorithms to accelerators. ACM Computing Surveys (CSUR), 54(9):1-38, 2021.
Abate, C. and Bianchi, F. M. Maxcutpool: Differcntiable fcaturc-aware maxcut for pooling in graph ncural networks. In ar Xivivoor 2024. URL httpp::/aaaxxv.orrpppftt
Ahamed, S., Zakariaei, N., Haber, E., and Eliasof, M. Multiscale training of convolutional neural networks. arXiv e-prints, pp. ar 2501, 2025.
Arora, S., Cohen, N., Golowich, N., and Hu, W. A convergence analysis of gradient descent for deep linear neural networks. arXi preprint 1810 180.0228, 2018. URL httppss///a.ww..orr/////////////
02281
Bentivoglio, R., Isufi, E., Jonkman, S., and Taormina, R. Multi-scale hydraulic graph neural networks for flood modelling. In UUUpperee, 2024. URL bttps::/eguspherss cowerni wttp://r/ppppppp///////.// egusphere 2024 2022-0022222 10221//guupheeeeee 2024:ppheee2224--10 2004-0022222202 2621. pdf
Bevilacqua, B., Frasca, F., Lim, D., Srinivasan, B., Cai, C., Balamurugan, G., Bronstein, M. M., and Maron, H. Equivariant subgraph aggregation networks. In International Conference on Learning Representations, 2022. URL https://openreviiet www.:eeooroddddooooooaoa
Bevilacqua, B., Eliasof, M., Meirom, E., Ribeiro, B., and Maron, H. Efficient subgraph GNNs by learning effective selection policies. The Twelfth International Conference on Learning Representations, 2024.
Bianchi, F. and Lachi, V. The expressive power of pooling in graph neural networks. In Neural Information Processing Systems, 2023.
Bojchevski, A. and Günnemanng, S. Doep gaussian embedding of graphs Unsupervised inductive learning via ranking. In International Conference on Learning Representations, 2017. URL https://arxiv.org 2dff/177.008815
Boyd, S. and Vandenberghe, L. Conver optimization. Cambridge University press, 2004.
Brandt, A. Multigrid techniques: 1984 Guide with applications to fluid Dynamics. The Weizmann Institute of Science, Rehovot, Israel, 1984.
Cai, C.,, Wang, D., and Wang, Y. Graph coarsening with neural networks. In International Conference on Learning Representations (ICLR), 2021. URL httpp://arrav..or//aabb////
Chang, A. X., Funkhouser, T., Guibas, L., Hanrahan, P., Huang, Q.-.,, Li, Z., Savarese, S., Savva, M., Song, S., Su, H., Xiao, J., Yi, L., and Yu, F. Shapenet: An information-rich 3d model repository. In arXiv.org, 2015.
Chen, J., Ma, T., Xiao, C., and Li, J. Fastgen: Fast learning with graph convolutional networks via importance sampling. In International Conference on Learning Representations (ICLR), 2018. URL /ttr/orrrn/ddxytU-----0-
https://openrevieet
Chen, M., Wei, 7.,, Huang, 7.,, Ding, B., and Li, Y. Simple and deep graph convolutional networks. Tn International Conference on Machine Learning, 2020. URL http:///aaxii.ooo/pt//////////// 02133
Chiang, Wi-L,, Liu, X., Si, S., Li, Y., Bengio, S., and Hsieh, C.-J. Cluster-gen: An efficient algorithm for training deep and large graph convolutional networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery 6 Dala Mining (KDD), pp. 257 266, 2019. doi: 10.1145/32925000 /2022222200.30995.....
Coupette, C., Wayland, J., Simons, E., Rieck, B., Günnemann, S., Veldt, N., Benson, A. R., and Li, P. No metric to rule them all: Toward principled evaluations of graph-learning datascts. In Proceedings of the 42nd International Conference on Machine Learning (ICML). PMLR, 2025. URL abs abb/2502..0 02379 ar 02X3v.2502.02379
14

---

Published in Transactions on Machine Learning Research (11/2025)
Defferrard, M., Bresson, X., and Vandergheynst, P. Convolutional neural networks graphs with fast localized spectral filtering. In Neural Information Processing Systems, 2016.
Eliasof, M., Bodner, B. J., and Treister, E. Haar wavelet feature compression for quantized graph convolutional networks. IEEE Transactions on Neural Networks and Learning Systems, 2023.
Frasca, F., Bevilacqua, B., Bronstein, M. M., and Maron, H. Understanding and extending subgraph gnns by rethinking their symmetries. In Neural Information Processing Systems, 2022. URL https: Shaxiv.orr//ee /20.6.
Galmés, M. F., Suárcz-Varde Varcla, J., Rusck, K., Barlct-Ros, P., and Cabellos-Aparicio, A. Scaling graph-basod deep learning models to larger networks. In arxiv.org, 2021.
Gao and Ji. Graph u-nets. In IEEE, 2019.
Geng, X., Li, Y., Wang, L., Zhang, L., Yang, Q, Ye, J., and Liu, Y. Spatiotemporal multi-graph convolution network for ride-hailing demand forecasting. In AAAT, 2019.
Giles, M. B. Multilevel monte carlo methods. In Large-Scale Scientific Computing, 2001.
Gilmer, J., Schoenholz, S. S., Riley, P. F., Vinyals, O., and Dahl, G. E. Neural message passing for quantum chemistry. In Proceedings of the 34th International Conference on Machine Learning- Volume 70, pp- 1263 1272. JMLR. org, 2017.
Glorot, X. and Bengio, Y. Understanding the difficulty of training deep feedforward neural networks. International Conference on Artificial Intelligence and Statistics, 2010.
Guo, Z.-H. and Li, H.-B. Mgfno: Multi-grid architecture fourier neural operator for parametric partial differential equations. In arXiv.org, 2024. URL http://txxw.ooogptt///// .06615
Gupta, S., Yan, X., and Lerman, K. Structural propertios of ego networks. In Social Computing, Behavioral Modeling, and Prediction, 2014.
Hamilton, W. L., Ying, Z., and Leskovec, J. Inductive Induetive representation learning large graphs. In Newral Information Processing Systems, 2017. URL http::////xxvv.o.//ppooo
02216
Hastings, W. Monte carlo sampling methods using markov chains and their applications. In Biometrika,
1970. URL https:// Dasmpppangk.OOmttoooee..o..ss
sun. ac www..........a......e hatp://toaaatee//torrrrr///////p
.pdf.
Hu, W., Fey, M., Zitník, M., Dong, Y., Ren, H., Liu, B., Catasta, M., and Leskovee, J. Open graph benchmark: Datasets for machine learning on graphs. In Neural Information Processing Systems, 2020.
Huang, X., Cheng, H., Li, R.-H., Qin, L., and Yu, J. X. Top-k structural diversity search in large networks. The VLDB Journal, 24(3):319 343,343,
343, 2015.
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvuakool, K., Bates, R., Zidek, A., Potapenko, A., et al. Highly accurate protein structure prediction with alphafold. Nature, 596(7873):583 589, 2021.
Kingma, D. P. and Ba, J. Adam: A method for stochastic oplimization. In International Conference on Learning Representations, 2014. URL
Kipf, T. N. and Welling, M. Semi-supervised classification with graph convolutional networks. In ICLR, 2016.
Klapproth, A., Schuemann, J., Stangl, S., Xic, T., Li, W. B., and Multhoff, G. Multi-scale monte carlo simulations of gold nanoparticlo-indun dna damages for kilovoltage x-ray irradiation in xenograft mouse model using topas-nbio. In Cancer nanotechnology, 2021.
15

---

Published in Transactions on Machine Learning Research (11/2025)
Kolaczyk, E. and Nowak, R. Multiscale statistical models. In Denison, D., Hansen, M., Holmes, C., Mallick, B., and Yu, B. (eds.), Nonlinear Estimation and Classification, volume 171 of Lecture Noles in Stalisties, pp. 293 308. Springer, New York, NY, 2003, doi: 10.1007797000-38-2257722 14.
LeCun, Y., Bottou, L.,, Bengio, Y., and Haffner, P. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278 2324, 1998.
Li, S., Zhang, B., Song, J., Ruan, G., Wang, C., and Xie, J. Graph neural networks with coarsc-and grai-er division for mitigating label sparsity and noise. arviv preprint arXiv:2411.08774, 2024a.
Li, Z., Song, H., Xiao, D., Lai, Z., and Wang, W. Harnessing scale and physics: A multi-graph neural operator framework for pdes on arbitrary geometries. In or Xiv.or,, 2024b. URL
15178
Liang, Y., Xu, Z., and Xing, X. A multi-scale monte carlo method for electrolytes. In New Journal of Plyysics 2015. URL
Liu, Zhan, and et al. Graph pooling for graph neural networks: Progress, challenges, and opportunities. In International Joint Conference on Artificial Intelligence, 2022. URL
07321
Mernyei, P. and Cangea, C. Wiki-cs: A wikipedia-bassed benchmark for graph neural networks. In arXiv.org. 2020. URL https://arxiv. otg/pdf/2000 02901
Nash, S. A multigrid approach to discretized optimization problems. Optimization Methods and Software, 14: 99 116, 2000.
Ronneberger, Fischer, and Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Compuler- Assistee Intervention, 2015.
Ruder, S. An overview of gradient descent optimization algorithms. arXiv preprint arXie:1609.0477 2016.
Sen, P., Namata, G., Bilgic, M., Getoor, L., Gallagher, B., and Eliassi-Raas Rad, T. Collective classification in network data. In Computer Science, 2008.
Shi, Y., Cai, J.-X., Shavit, Y., Mu, T.-J., Feng, W., and Zhang, K. Clustergun: Cluster-based coarse-to-fine graph neural network for efficient feature matching. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 12517-12522 12526, 2022.
Stanovic, S., Gaüzère, B., and Brun, L. Graph neural networks with maximal independent set- set-based pooling: Mitigating over-smoothing and over-squashing In Pattern Recognition Letters, 2024.
Tang, S. W. Y., Zhu, Y., Wang, L., Xie, X., and Tan, T. based recommendation with graph neural networks. In AAAI, 2018. URL hittps://arxii: htttps///aaxiv.oor///t////1///1
Velickovie, Cueurull, Casanova, Romero, Lio, and Bengio. Graph attentin networks. Tn ICLR, 2017.
Wale, N., Watson, A., and Karypis, G. Comparison descriptor spaces for chemical compound retrieval and classification. Knowledge and Information Systems, 14(3)3347-375, 2008
Wang, Y., Sun, Y., Liu, Z., Sarma, S., Bronstein, M., and Solomon, J. Dynamic graph enn for learning on point clouds. In ACM Transactions on Graphics, 2018.
Wang, Y., Wang, W., Liang, Y., Cai, Y., and Hooi, B. Graphcrop: Subgraph cropping for graph classification. In org, 2020. URL https: wttps//.rg/ppp///o/..///// 10564
Wang, 7., Zhang, Z., Zhang, C., and Ye, Y. Subgraph pooling: Tackling negative transfer on graphs.
In International Joint Conference Artificial Intelligence, 2024. URL httpss://aariv..rg///d///// 08907v2
Woodruff, D. P. Sketching as a tool for numerical linear algebra. Foundations and Trends in Theoretical Computer Science, 1017-2):1-157, 2014.
16

---

Published in Transactions on Machine Learning Research (11/2025)
Xu, K., Hu, W., Leskovec, J., and Jegelka, S. How powerful are graph neural networks? In ICLR, 2018. Xu, M., Qian, P., Zhao, Z., Zeng, Z., Chen, J., Liu, W., and Yang, X. Graph neural networks for protein-protein interactions short survey. In orXiv.org, 2024.
Yang, R., Shi, J., Xiao, X., Yang, Y., Bhowmick, S., and Liu, J. Pane: scalable and effective attributed network embedding. In The VLDB journal, 2020. URL
Yang, Z., Cohen, W. W., and Salakhutdinov, R. Revisiting semi-supervised learning with graph embeddings. In ICML, 2016.
Ying, You, Morris, Ren, Hamilton, and Leskovee. Hierarchical graph representation learning with differentiable pooling. In Neural Information Processing Systems, 2018. URL https://arxiv.org/ wwtpaxiiii.ooo/ptt..e /1806. 08804
Zeng, H., Zhou, H., Srivastava, A., Kannan, R., and Prasanna, V. Graphsaint: Graph sampling based inductive learning method. In International Conference on Learning Representations, 2019. URL https
//axxiv.orz www...rg/pppe..... wttpf////.00./0 1907.04937
Zeng, H., Zhou, H., Srivastava, A., Kannan, R., and Prasanna, V. Graphsaint: Graph sampling based inductive learning method. In International Conference on Learning Representations (ICLR), 2020. URL
https://openreviiee net//forummmddddspppphh
Zitnik, M. and Leskovec, J. Predicting multicellular function through multi-layer tissue networks. In Bioinform, 2017. URL bttts::/arxx:: htt////wwww.r//////////////////.
17

---

Published in Transactions on Machine Learning Research (11/2025)
A
Theorem 4.1 Proof
Let us consider the model problem in Equation
11
for o Id
1
(())
AXO Y
(15)
2N
and the sketching problem from Woodruff 2014
Cc 2N
P
- A YI?
(16)
where PT is a sketching matrix.
Then the following theorem can be proved:
Theorem A.1 (Woodruff (2014). Thm. 23). Let PT be sparse subspace embedding with o rows, and
let 0r be the solulion of Equation 15 and O be the solulion of Equalion 16 Then, with probabilily 0.99, it holds that:
1 2N
AXOC - -P 1 2N
Co(Oc)
AXX - Y11
(17)
In the sketching theory, the network applied first, and only then is the graph size reduced. Here, we consider the operator PT that chooses a subset of nodes. Unlike sketching, in our multiscale training, we reduce the number of nodes first. We thus have that choosing subset of nodes has slightly different form, that is, our subgraph problem obeys
1 |P APP XO
1 ATAXO - P + R1I2
1.(0%))
PT 2
(18)
2N
2N
- 2 [R] = 1
$ 2N ( (I
AXE
2N (IAX6) YI [|(?).
Here used norm properties and Theorem A.1 and R is the residual.
Acc ACF AFC AFF
Now we focus on the case where P chooses subset of nodes. Assume that we reorder A
Xc
such that C denotes the chosen node indices, and denote: X =
Then, PAP = Acc, and PT X Xo.
X
We obtain that:
P ATAPP'X AceXe
while
P = Ace ACEXp.
Hence, have
P A (AcoXo Ac
and
P ATAX Ace ACFX ATAPP XO- + Ac
This means that |R|| < 1|AcFXp0ccl. We conclude statement with the following:
Theorem A.2. Let 0" be the solution of equation 15 and let 0% be the solution of equation 16 Demote
E>0, O, and let PT be sporse subspace embedding with O(5) rows, where e is the size of the feature vectors (nammeer of graph channels), that injects any vector inlo subset of indices, which denote by C. According
to this subset, denote A Acc ACF
X
and X
Then if Id, there exists ||||| < ||AceXFOcl
AFC AFF
X
such that with probability .99:
1 1
C.(0)) = 2N
P APPP X0
- 2 <
(|AXX Y
R
(19)
2N
18

---

Published in Transactions on Machine Learning Research (11/2025)
The approximation quality depends the residual matrix R. For separable graphs, there are no edges that are omitted and therefore R = O. As the number of edges that are omitted increases, the solution of the coarse-scale problem diverges from the solution of the fine-scale one. While theoretically the residual may become large in certain scenarios, our empirical results show that in practical settings R remains modcrate.
When combining both coarsening and subgraphs, we have a diverse sampling of the graph, which may yield better approximation on average, can be shown for certain datasets. We present such datasets in our numerical experiments presented in B
19

---

Published in Transactions on Machine Learning Research (11/2025)
B Qtips and MNIST Datasets
We evaluate our methods on the Qtips datasct by constructing a training set of 100 rods and a test set of 1,000 rods. We employ a 4-layer GCN with 256 hidden channels and compare the performance of our approach across three levels of coarsening. The baseline is trained for 500 epochs, while the multiscale methods are trained for [400, 200, 100] epochs across levels. Results, shown in Table indicate that the combination of random coarsening and subgraph pooling achieves the best performance, particularly when using p
We further test our approach a graph dataset derived from the MNIST image dataset (LeCun et al. 1998). The original MNIST images consist of 28 x 28 grayscale pixels, each depicting digit from 0 to 9. We convert each image into graph by treating each pixel as a node and connecting nodes using their spatial positions and a --nearest-neighbol algorithm. The task is to assign each node either to background class or to the digit class it belongs to. We train a 4-layer GCN with 256 hidden channels on 1,000 graphs and evaluate on 100 test graphs. Results are presented in Table 7 where we again observe that the combination of random coarsening and subgraph pooling achieves the best performance among the methods evaluated.
Table 6: Qtips dataset results, comparing the bascline with 3 levels of coarscning.
Random
Topk
Subgraphs
Random and Subgraphs
Fine Grid:
73.03% (0.1%)
p=1 1
71.47%
(0.2%)
71.46% (0.1%)
72.97% (0.1%)
73.20% (0.0%)
p=2
71.40% (0.1%)
71.30% (0.1%)
72.82% (0.1%)
73.02% (0.1%)
p-3
71.31% (0.1%)
71.30% (0.1%) 72.72% (0.1%) 73.30% (0.1%)
Table 7: MNIST dataset results, comparing the baseline with 3 levels of coarsening.
Random
Topk
Subgraphs
Random and Subgraphs
Fine Grid:
86.28% (0.1%)
p=1 1
85.78% (0.1%)
85.85% (0.0%)
85.98% (0.0%)
85.96% (0.0%)
p=2
85.77% (0.1%)
85.88% (0.1%
85.97% (0.1%)
85.99% (0.0%)
p
3
85.85% (0.1%)
85.90% (0.1%) 86.03% (0.0%)
86.04% (0.1%)
20

---

Published in Transactions on Machine Learning Research (11/2025)
C
Coarsening Methods
Additional implementation details of our coarsening procedures are provided. Algorithm 3 presents the random coarsening algorithm, Algorithm 4 outlines the Top-k coarsening method, and Algorithm describes the subgraph generation process.
Algorithm Random Coarsening m coursining rabio.
Initialize Adj- A for i = ......P 1 do
A = Adj
end for Randomly choose m nodes Use chosen nodes to calculate the injection matrix P
Use to calculate A, as in Equation
Algorithm Topk Coarsening m coarsining ratio. Initialize Adj- A for i 0, .......... p 1 do
A - Ax Adj
end for
Select m nodes with the largest degrees Use chosen nodes to calculate the injection matrix P
Use calculate A, as in Equation
Algorithm Subgraph Coarsening
# m coarsining ratio.
kr k -hop porameter for each level
Initialize Adj- = A
for i = ..... . 1 do
A - Ax Adj
end for
Choose a random node
Generate subgraph using k, hops
Use chosen nodes to calculate the injection matrix P
Use P to calculate A, in Equation 5
21

---

Published in Transactions on Machine Learning Research (11/2025)
D Loss Comparison between Multilevel Training and Single-Level Training
To further demonstrate the efficiency of our methods, we show in Figure 5 comparison between the loss during training, both using single-level training and multiscale training. We show that both training approaches converge to similar loss after a similar number training epochs, while as demonstrated Table 2 the cost of a single iteration on the coarse graph lower than on the original fine graph. The plot shows results for the OGBN-Arxiv dataset (Hu pooling.
2020). using GCN (Kipf & Welling 2016) and random
Table presents the "r values obtained during multiscale training, computed according to Equation 9 We note that "/r increases as the value of r grows.
Multiscale Training
1-Level Only (Fine Scale)
1500
Iterations
Figure 5: Comparison between losses using multiscale and single-level training. Coarse levels are generated using random coarsening.
Table 8: Demonstration of %r values, as described in Equation
7r
r =
Random 0
0.060
0.076
Topk 0
0.046
0.175
Subgraph 0.067
0.170
22

---

Published in Transactions on Machine Learning Research (11/2025)
E Memory Consumption
We show memory consumptions for OGBN-Arxiv (Hu ct al. 2020 function of coarsening layer and
network. The results are summarized in Table and they show that coarse grids consume less memory than the original fine scale data.
Table 9: Memory consumption across layers and models.
L
L = 3
L = 4
GCN
Fine Grid (L = 1)
1610.30
Random
926.62
528.33
332.99
Topk
994.36
582.51
366.58
Subgraphs
641.59
491.47
221.85
Rand. & Sub.
923.39
528.28 28
333.20
GIN
Fine Grid (L (6=00 1)
2188.64
Random
1227.56
719.56
431.26
Topk
1259.72
739.62
443.28
Subgraphs
938.48
498.18
224.91
Rand. & Sub.
1228.66
718.55
431 26
GAT
Fine Grid (L = 1)
4059.53
Random
1415.95
588.60
324.92
Topk
2787.29
1655.82
896.41
Subgraphs
1108.56
466.31
220.81
Rand. & Sub.
1423.09
601.40
322.41
23

---

Published in Transactions on Machine Learning Research (11/2025)
F Cora, CiteSeer and PubMed Datasets
We evaluate our method on the Cora, Citeseer, and PubMed datasets Sen ct al. 2008 using the standard split from Yang et al. (2016), with 20 nodes per class for training, 500 validation nodes, and 1,000 testing nodes. Cora contains machine learning research papers classified into topics, CiteSeer includes scientific publications from various computer science fields, and PubMed consists of biomedical papers related to diabetes, where nodes represent documents and edges represent citation links. Our experiments cover standard architectures GCN (Kipf
& Welling) 2016), GIN (Xu et al. 2018); and GAT (Velickovic et. al., 2017 and are summarized in Table 10 where we achieve performance that is either superior to on par with the fine-grid baseline, which trains directly on the original graphs.
We explore two pooling strategies- random and Top-k along with subgraph generation via k-hop neigh- borhoods and a hybrid approach combining both. Each method is tested under two connoctivity scttings: the original adjacency (p = 1) and an enhanced version (p 2), where we square the adjacency matrix. We report test accuracy across three runs with different random seeds and include standard deviation assess robustness. Our results indicate that enhanced connectivity improves accuracy in some settings. Table [O] further presents three levels of multiscale training alongside thcoretical FLOP analyses, demonstrating the efficiency of our approach. The details of the FLOPs analysis are provided in Section 5 and additional results
for two and four levels of multiscale training are provided in Table 11 and Table 12 We provide further experimental details Table 18 and additional information about the datascts appears Table
24

---

Published in Transactions on Machine Learning Research (11/2025)
Table 10: Comparison of different coarsening methods on the Cora, CiteSeer, and PubMed datasets. The
lowest (best) FLOPs count for each GNN appears in blue. The highest (best) performing approach in terms of test accuracy (%) highlighted black.
Cora
CiteSeer
PubMed
Method L Dataset
FLOPs(M)
Test Acc(%)
FLOPs(M)
Test Acc(%)
FL.P&sMM
Test Acc(%)
GON
Fine grid
9.91 10a
82.9% (1.1%)
2.69 10
69.6% (1.0%)
3.55 10a
77.5% (0.4%)
Random(p=1)
2.40 x
82.0% (0.5%)
6.58 x 10
70.4% (0.2%)
8.51
10a
77.5% (0.6%)
Random(p=2)
2.63 10a
83.0% (0.5%)
6.82 x 10
66.0% (0.8%)
9.99 10a
73.8% (0.4%)
Topk(p=1)
x 10a
79.9% (4.4%)
6.83 102
68.1% (0.1%)
9.33 10a
78.2% (0.3%)
Topk(p=2)
3.09 x 10a
79.6% (2.4%)
8.08 x 102
63.6% (2.7%)
1.87 10a
74.5% (0.4%)
Subgraphs(( (p=1)
2.43 10a
83.5% (0.4%)
6.56 102
71.2% (0.4%)
8.63 10a
78.0% (0.3%)
Subgraphs(p=2))
3.26 x
83.1% (0.5%)
6.82 x 102
70.9% (0.2%)
1.19 10a
76.1% (0.3%)
Rand. & (ub.(p=1)
2.41 x 10a
81.5% (0.3%)
6.57 x 10a
70.0% (0.0%)
8,57 10a
77.8% (0.8%)
Rand. Sub.(p=2)
x 10a
82.6% (0.5%)
6.83 102
70.9% (0.2%)
10a
73.9% (0.4%)
GIN
Fine grid
2.37 x
76.4% (1.6%)
6.79 x 103
63.8% (2.6%)
7.75 10a
75.4% (0.4%)
Random (p=1)
5.89 x 10a
76.2% (2.7%)
1.69 x 103
66.5% (2.2%)
1.92 10a
75.7% (2.0%)
Random(p-22
10a
77.2% (0.6%)
1.70 10
60.5% (0.8%)
10a
75.1% (0.9%)
Topk(p=1)
5.94 x 10a
77.6% (5.6%)
1.70 x 10
64.8% (3.9%)
1.96 x 10a
75.6% (1.6%)
Topk(p=2)
6.22 10a
74.6% (3.0%)
1.76 103
60.5% (2.0%)
2.40 10a
76.5% (1.5%)
Subgraphs(p=11 p==
5.91 x
78.7% (1.1%)
1.69 x 10a
68.8%
(1.6%)
1.93 10a
77.2%
(1.2%)
Subgraphs((-
6.31 x 10a
76.9% (1.72%)
1.70 x 10
65.0% (1.8%)
2.08 10a
75.2%
(1.5%)
Rand. (ub.(p=1)
102
76.9% (1.0%)
1.69 10a
65.3%
(1.2%)
10a
76.1% (2.8%)
Rand. & (ub.(p=2)
6.14 x 10a
75.9% (0.8%)
1.70 x 10
66.0% (0.2%)
2.02 x 103
75.9% (0.5%)
GAT
Fine grid
111 x 10a
79.6%
(2.05%)
3.35 x 10a
67.2%
(6.5%)
3.08 10a
75.5% (0.5%)
Random(p=1)
10a
80.3% (0.2%)
8.11 102
72.9%
(1.5%)
7,29 102
77.5% (1.5%)
Random(p=22
3.01 x 10a
79.5% (0.6%)
8.55 x 102
70.7% (1.0%)
9.01 X 102
77.4% (1.2%)
Topk(p=1) 1)
2.79 10a
79.8% (0.6%)
8.56 102
66.8% (0.6%)
8.24 102
76.1% (0.9%)
Topk(p=2)
3.72 x
76.8% (6.7%)
1.08 x 10a
65.2% (0.7%)
1.92 103
71.9% (0.7%)
Subgraphs(p=1)
2.70 10a
78.5% (0.8%)
8.08 x 102
70.2% (0.6%)
7.43 10a
75.4% (4.6%)
Subgraphs(p-22)
102
80.7% (1.2%)
8.54 102
68.6% (0.3%)
103
77.3%
(1.1%)
Rand. & Sub.(p=1)
2.67 x 10a
79.9% (1.4%)
8.10 x 102
71.2% (0.8%)
7.36 x 102
76.8% (0.5%)
Rand. & Sub.(p=2)
3.46 10a
80.2% (1.1%)
8.57 10?
70.8% (0.4%)
9.87 102
77.5% (1.1%)
25

---

Published in Transactions on Machine Learning Research (11/2025)
Table 11: Comparison of different coarsening methods on the Cora and CiteSeer datasets. The highest (best) performing approach in terms of test accuracy (%) highlighted in black.
Method
Dataset
Cora
CiteSeer
L 2
L 3
L 2
L 3
GON
Fine Grid (I
82.9% (1.1%)
69.6% (1.0%)
Random(p=1)
81.7% (0.6%)
82.0% (0.5%)
81.7% (0.4%)
70.3% (0.3%)
70.4% (0.2%)
70.7% (0.2%)
Random(p=2)
83.2% (0.6%)
83.0% (0.5%)
81.5% (0.3%)
66.6% (2.6%)
66.0% (0.8%)
66.2% (2.2%)
Topk(p=1)
79.5% (0.7%)
79.9% (4.4%)
79.8% (2.2%)
68.5% (1.1%)
68.1% (0.1%)
67.4% (0.3%)
Topk(p-2)
80.5% (2.5%)
79.6% (2.4%)
80.8% (0.8%)
66.4% (0.8%)
63.6% (2.7%)
65.8% (2.1%)
Subgraphs(p=1)
82.7% (0.7%)
83.5% (0.4%)
81.3% (0.1%)
71.6% (0.5%
71.2% (0.4%)
71.3% (0.4%)
Subgraphs(p=22)
82.8% (0.4%)
83.1% (0.5%)
81.8% (0.7%)
71.5% (0.5%)
70.9% (0.2%)
70.9% (0.4%)
Rand. & Sub.(p=1)
81.7% (0.4%)
81.5% (0.3%)
81.5% (0.3%)
71.4% (0.5%)
70.0% (0.0%)
70.7% (0.3%)
Rand. & (p=.
82.7% (0.3%)
82.6% (0.5%)
80.8% (0.1%)
70.8% (0.3%)
70.9% (0.2%)
70.7% (0.1%)
GIN
Fine Grid (I=
76.4% (1.6%)
63.8% (2.6%)
Random(p=1)
76.6% (1.5%)
76.2% (2.7%)
77.6% (2.2%)
66.3% (7.0%)
66.5% (2.2%)
66.4% (1.1%)
Random(p=2)
73.1% (0.3%)
77.2% (0.6%)
71.6% (1.6%)
60.6% (2.6%)
60.5% (0.8%)
62.8% (1.3%)
Topk(p=1)
77.5% (1.3%)
77.6% (5.6%)
79.4% (1.9%)
67.3% (1.3%)
64.8% (3.9%)
66.5% (1.5%)
Topk(p-2)
76.2% (1.6%)
74.6% (3.0%)
75.6% (2.3%)
61.1% (1.1%)
60.5% (2.0%)
60.0% (2.9%)
Subgraphs(p-
77.5% (0.9%)
78.7% (1.1%)
78.2% (1.4%)
68.2% (0.5%)
68.8% (1.6%)
68.7% (1.4%)
Subgraphs(p=22)
76.2% (8.57%)
76.9% (1.72%)
77.3% (8.55%)
60.1% (0.29%)
65.0% (1.8%)
66.6% (1.0%)
Rand. & (p=1(p=1)
80.2% (2.5%)
76.9% (1.0%)
76.8% (2. 1%))
66.0% (0.9%)
65.3% (1.2%)
65.5% (1.1%)
Rand. & (p=2
76.5% (0.7%)
75.9% (0.8%)
74.6% (22.5%)
64.1% (2.2%)
66.0% (0.2%)
68.7% (1.0%)
GAT
Fine Grid (L 1)
79.6% (2.05%)
67.2% (6.5%)
Random(p=1)
80.1% (0.8%)
80.3% (0.2%)
79.8%
(0.8%)
71.1% (0.4%)
72.9% (1.5%)
71.4% (0.3%)
Random(p=2)
82.0% (0.5%
79.5% (0.6%)
79.5% (1.9%)
71.6% (1.1%)
70.7% (1.0%)
70.6% (1.5%)
'lopk(==1)
77.3% (1.8%)
79.8% (0.6%)
77.5% (2.9%)
67.2% (1.5%)
66.8% (0.6%)
70.8% (1.3%)
Topk(p=2)
79.6% (2.3%)
76.8% (6.7%)
79.0% (1.6%)
67.4% (2.2%)
65.2% (0.7%)
67.0% (0.2%)
Subgraphs(p=1)
80.2% (2.6%)
78.5% (0.8%)
79.5% (0.8%)
69.1% (0.6%)
70.2% (0.6%)
68.4% (2.7%)
Subgraphs(p=2)
80.5% (1.5%)
80.7% (1.2%)
79.9% (0.9%)
70.1% (0.5%)
68.6% (0.3%)
66.0% (0.2%)
Rand. & (p=1)
80.6% (0.2%)
79.9% (1.4%)
80.3% (0.9%
70.8% (0.1%)
71.2% (0.8%)
71.2% (0.3%)
Rand.
(ub.(p-2)
81.1% (1.1%)
80.2% (1.1%)
78.7% (0.4%)
70.3% (0.4%)
70.8% (0.4%)
71.1% (0.3%)
26

---

Published in Transactions on Machine Learning Research (11/2025)
Table 12: Comparison of different coarsening methods on the PubMed dataset. The highest (best) performing approach in terms of test accuracy (%) is highlighted in black.
Method / Levels
L == 2
L ==3 3
L = 4
GCN
Fine Grid (L 1)
77.5% (0.4%)
Random(p=1)
77.6% (0.8%)
77.5% (0.6%)
77.5% (0.4%)
Random(p=2)
74.2% (0.4%)
73.8% (0.4%)
74.2% (0.7%)
Topk(p=1)
77.8% (0.9%)
78.2% (0.3%)
78.2% (0.6%)
(opk(p-2)
75.4% (0.4%)
74.5% (0.4%)
74.2% (0.6%)
Subgraphs(p-2 (p=1)
77.7% (0.2%)
78.0% (0.3%)
78.7% (1.3%)
Subgraphs(p-22
74.8% (1.2%)
76.1% (0.3%)
76.5% (1.1%)
Rand. & (ub.(p=1)
77.4% (0.5%)
77.8% (0.8%)
76.7% (0.3%)
Rand. & (u-2(
74.1% (0.7%)
73.9% (0.4%)
75.6% (0.9%)
GIN
Fine Grid (L 1)
75.4% (0.4%)
(andom(p=1) Random(p=2)
75.6% (1.7%)
75.7% (2.0%)
76.2% (1.3%)
75.6% (2.9%)
75.1% (0.9%)
74.3% (1.2%)
Topk(p=1)
76.6% (0.8%)
75.6% (1.6%)
75.5% (1.6%)
(o-k(p=2)
76.0% (1.8%)
76.5% (1.5%)
66.1% (6.4%)
(pbgraphs(p=1)
75.8% (1.2%)
77.2% (1.2%)
77.2% (0.7%)
Subgraphs(p=2)
75.6% (2.8%)
75.2% (1.5%)
55.4% (4.7%)
Rand. & (pb.(p=1)
75.2% (0.8%)
76.1% (2.8%)
75.0% (0.7%)
Rand. & (p-2(p-2)
77.8% (5.6%)
75.9% (0.5%)
76.4% (2.4%)
GAT
Fine Grid (L = 1)
75.5% (0.5%)
Random (p=1)
76.5% (0.3%)
77.5% (1.5%)
76.4% (0.8%)
Random (p=2)
75.2% (1.0%)
77.4% (1.2%)
78.6% (2.5%)
Topk(p=1)
76.1% (0.2%)
76.1% (0.9%)
75.6% (2.5%)
Topk(p=2)
74.2% (1.0%)
74.9% (0.7%)
73.3% (1.0%)
Subgraphs (p=1)
77.0% (1.0%)
75.4% (4.6%)
75.8% (1.6%)
Subgraphs (p=2)
75.1% (0.85%)
77.3% (1.1%)
78.2% (3.0%)
Rand. & Sub. (p=1)
77.3% (0.8%)
76.8% (0.5%)
76.0% (1.9%)
Rand. & (p-2
75.7% (1.8%)
77.5% (1.1%)
76.4% (1.9%)
27

---

Published in Transactions on Machine Learning Research (11/2025)
G Additional Transductive Learning Datasets
We further test our methods using additional datasets. We provide results for Flickr Zeng et al. (019 and PPT (transductive) Yang et. al. (2020), showing both timing analysis, in Table 13 and full multiscale training results in Table [5 Flickr is a social network where nodes represent images and edges represent mutual user connections or shared tags, focusing on image-based community detection, while PPI (transductive) is
protein-protein interaction network where nodes represent proteins and edges denote biological interactions, with the goal of predicting protein functions within a single connected graph. While pooling does not produce a clearly interpretable or meaningful molecule the molecular setting, this does not affect the applicability or performance of our methods. Our results show that while achieving comparable accuracy the original fine grid, methods show significant. timing improvement both in theoretical manner well as in FLOPs analysis, which we calculate explained in Section
In addition, we test our methods and show results for Facebook, BlogCatalog Yang et al. 2020 DBLP
[ojchhvski & Günnemanng 2017) and WikiCS (Mernyei & Cangea 2020), in Table 16 and Table 17 Facebook represents a social network whore nodes are users and edges represent friendships, BlogCatalog is a social network of bloggers connected based on shared interests, DBLP is a co-authorship network built. from academic publications computer science, and WikiCS consists of Wikipedia articles about computer science topics connected by hyperlinks. This further illustrates that our methods are not restricted to specific dataset network architecture, baseline results. Details of our experiments are provided in Table 18. and we list additional details of the datasets in Table
results are consistently comparable to the original training
We provide additional comparison between our results and sampling-basse based efficient GNN training methods, which include GraphSAINT (Zeng et al., 2020), FastGCN (Chen al., 2018)) and ClusterGCN (Chiang et al. 2019). Table shows comparison with our methods, for the Flickr (Zeng et al. 2019) dataset. We observe that our efficient methods have yielded better results for this dataset in comparison to the above methods.
Table 13: Comparison of different coarsening methods on the Flickr and PPT (transductive) datasets. The lowest (best) FLOPs count for each GNN appears in blue. Time represents the training epoch time milliseconds. The lowest (best) time for a given network and level is highlighted green. The highest, (best)
performing approach terms of test accuracy (%) highlighted in black.
Method + Dataset
Flickr
PPI (transductive)
FLOPs(M)
Time(ma)
Test Acc(%)
FLOPs(M)
Time(ms)
Test Acc(%)
GCN
Fine grid Random
1.72 10
68.90
54.77% (0.2%)
8.09 x 103
64.66
70.74% (0.2%)
3.94 10a
27.73
55.30% (1.6%)
1.65 x 10a
25.60
71.10% (0.2%)
Topk
4.28 103
36.31
55.14%
(0.1%)
2.46 10
46.14
71.37% (0.4%)
Subgraphs
4.92 x 103
73.16
54.61%
(0.8%)
1.61 x 103
113.90
71.63% (0.3%)
Rand. & Sub.
4.44 x 10
49.74
55.85%
(1.6%)
1.63 x 10
68,06
71.50% (0.0%)
GIN
Fine grid Random
3.58 x 10a
76.29
52.05%
(1.0%)
1.34 x 10a
48.76
85.67% (7.2%)
8.77 x 10a
27.93
52.16%
(3.9%)
3.17 x 10a
21.47
89.24% (0.4%)
Topk
8.93 10a
34.47
51.99% (5.1%)
3.54 x 10a
33.98
84.92% (0.8%)
Subgraphs
9.23 103
75.88
53.41%
(1.0%)
3.15
103
117.70
87.40% (8.1%)
Rand. & Sub.
9.01 10a
53.51
52.00%
(1.8%)
3.16 x 10a
68.52
90.83% (12.3%)
GAT
Fine grid Random
1.53 104
103.54
52.71%
(0.9%)
4.57 X 10a
192.20
82.37% (3.1%)
3.40 x 10a
29.57
52.91%
(0.3%)
9.32 x 102
32.30
82.93% (1.3%)
Topk
3.80 x 103
44.35
52.79%
(0.4%)
1.39 x 103
99.79
73.13% (1.6%)
Subgraphs
4.55 10a
94.21
53.37%
(0.1%)
9.10 x 10a
132.67
87.71% (1.8%)
Rand. & Sub.
3.99 10a
62.38
53.26%
(0.2%)
9.21 10a
79.21
87.03% (0.8%)
28

---

Published in Transactions on Machine Learning Research (11/2025)
Table 14: Datasets statistics.
Dataset
Nodes
Edges
Features
Classes
Cora Sen et al.
(2008)
2,708
10,556
1,433
7
Citeseer Sen et al.
(0088)
3,327
9,104
3,703
6
Pubmed Sen el al.
2008
19,717
88,648
500
3
Flickr Zeng et al.
2019)
89,250
899,756
500
7
WikiCS Mernyei & Cangea 2020)
11,701
216,123
300
10
DBLP Bojchevski & Günnemanng (2017)
17,716
105,734
1,639
Facobook Yang ct al. (2020)
4,039
88,234
1,283
193
GlogCatale Yang el al. (2020))
5,196
343,486
8,189
6
PPI (transcuctive) Yang et al. (2020)
56,944
1,612,348
50
121
Table 15: Comparison of different coarsening methods on the Flickr and PPT (transductive) datasets. The highest (best) performing approach in terms of test accuracy (%) is highlighted in black.
Method / Dataset +
Flickr
PPI (transductive)
L =
= 3
L = 1
L 2
3
L = 1
GCN
Fine Grid (I. =1=11)
54.77% (0.296)
70.74% (0.2%)
Random
35.01% (0.6%)
55.30% (1.6%)
54.17% (0.3%)
70.97% (0.2%)
71.10% (0.2%)
71.44% (0.1%)
Topk
35.28% (0.2%)
55.14% (0.1%)
55.15% (0.1%)
70.62% (0.0%)
71.37% (0.4%)
71.09% (0.3%)
Subgraphs
35.25% (0.1%)
51.61% (0.8%)
55.17% (0.3%)
70.96% (0.1%)
71.63% (0.3%)
73.07% (0.4%%
Rand. & Sub.s
55.65% (0.1%)
55.85% (1.6%)
55.08%
(0.2%)
70.88% (0.1%)
71.50% (0.0%)
72.89% (0.1%)
GIN
Fine Grid (1. (I==1)
52.05% (1.0%)
85.67% (7.2%)
Random
32.24% (0.6%)
52.16% (3.9%)
53.74%
(4.2%)
84.85% (2.6%)
89.24% (0.4%)
89. 43% (0.6%)
Topk
53.14% (3.5%)
51.99% (5.1%)
52.19% (5.3%)
81.74% (1.2%)
84.92% (0.8%)
83.70% (1.2%)
Subgraphs
33.09% (1.4%)
53.41% (1.0%)
53.33% (2.5%)
91.27% (8.8%)
10%0 (8.1%)
88. (0.2%)
Rand. Sub.
52. 74% (1.0%)
52.00% (1.8%)
52.50% (2.0%)
81.25% (4.1%)
90.83% (12.3%)
93.31% (12.1%)
GAT
Fine Grid (L
52.71%
(0.9%)
82.37% (3.1%)
Raudom
32.98% (0.3%)
52.91% (0.3%)
52.18%
(1.5%)
81.23% (1.4%)
82.93% (1.3%)
82.97% (3.0%)
Topk
52.98% (0.2%)
52.79% (0.415)
52.72% (0.2%)
77.30% (1.4%)
73. 13% (1.6%)
70.37% (0.4%)
Subgraphs
32, 96% (0.3%)
53.37% (0.1%)
53.33% (0.3%)
84.99% (0.9%)
87.71% (1.8%)
86.26% (1.9%)
Rand. & Suh.
52. 70% (0.0%)
53.26% (0.296)
52.97% (0.2%)
84.48% (1.0%)
87.03% (0.8%)
87.15%
(1.3%)
29

---

Published in Transactions on Machine Learning Research (11/2025)
Table 17: Comparison of different coarsening methods on the BlogCatalog and WikiCS datasets. The highest (best) performing approach in terms of test accuracy (%) highlighted in black.
Mothod 4 Dataset +
BlogCotalog
WikiCS
L ==
L=
L =
L 2
L
L
GCN
Fine Grid (L 1)
75.87% (0.4%)
78.07% (0.0%)
Random
76.54% (0.3%)
76.63% (0.2%)
76.54% (0.3%)
79.55% (0.1%)
78.33% (0.4%)
78.96% (0.1%)
Topk
76.44% (0.1%)
76.25% (0.3%)
75.96% (0.3%)
77.95% (0.4%)
78.06% (0.2%)
77.83% (0.2%)
Subgraphs
76.73% (0.1%)
77.88% (0.3%)
77.40% (0.3%)
78.36% (0.3%)
78. 18.28% (0.1%)
78.42% (0.1%)
Rand. & Sub.
76.44% (0.1%)
77.12% (0.3%)
76.92% (0.8%)
79. 29% (0.1%)
78.59%
(0.5%)
78.28% (0.6%)
GIN
Fine Grid (L -
80.58% (4.0%)
71.73% (0.4%)
Random
80.19% (5.4%)
81.25% (11.6%)
81.83% (1.6%)
75.01% (1.0%)
76.96% (1.1%)
76.18%
(0.9%)
Topk
80.96% (8.8%)
83.08% (23.9%)
80.35% (15.5%)
72.62% (1.7%)
71.19% (1.1%)
71.83% (0.2%)
Subgraphs
80.19% (27.6%6
82.88% (24.5%)
82.88% (7.7%)
71.71% (1.2%)
73.03% (0.7%)
73.01% (1.7%)
Rand. & Sub.
80.58% (14.7%)
82.12% (3.8%)
81.35% (3.0%)
72.64% (0.4%)
73.71% (0.8%)
74.16% (1.3%)
GAT
Fine Grid (L 1)
71.44% (3.9%)
79.09%
(1.1%)
Random
79.04% (3.9%)
77.98% (2.1%)
72.98% (3.0%)
79.10%
(0.5%)
80.28% (0.8%)
79.19% (0.4%)
Topk
71.83% (1.2%)
72.31% (1.5%)
72.31% (0.1%)
79.00% (0.4%)
77.90% (0.9%)
76.09% (0.9%)
Subgraphs
70.67% (2.0%)
69.52% (0.6%)
71.73% (3.0%)
78.45% (2.3%)
77.94% (0.6%)
76.81% (0.8%)
Rand. & Sub.
72.98% (3.9%)
74.13% (1.9%)
76.15% (3.1%)
78.93% (0.6%)
79.20% (0.2%)
79.60% (0.9%)
Table 16: Comparison of different coarsening methods on the DBLP and Facebook datasets. The highest (best) performing approach in terms of test accuracy (%) is highlighted in black.
Mothod L Dataset
+
DBLP
Facebook
==
L ==3 3
L = 4
L
L =
=
GCN
Fine Grid (L
86.48% (0.4%)
74.88% (0.6%)
Random(p==)
87.13% (0.1%)
87.05% (0.1%)
87.08% (0.1%)
75.87% (0.4%)
75.25% (0.3%)
75.25% (0.7%)
Random(p=2)
85.58% (0.0%)
85.30% (0.3%)
85.47% (0.5%)
75.50% (0.4%)
74.38% (0.0%)
73.76% (0.3%)
Topk(p=1)
86.79% (0.2%)
86.82% (0.1%)
86.03% (0.2%)
75.25% (0.5%)
74.75% (0.4%)
74.75% (0.4%)
TTppk(p-2)
85.75% (0.1%)
85.78% (0.196)
85.86% (0.0%)
74.38% (0.3%)
74.63% (0.2%)
74.88% (0.8%)
Subgraphs(p=1)
86.63% (0.1%)
86.88 (0.3%)
86.77% (0.0%)
74.63% (0.6%)
74.75% (0.3%)
74.88% (0.3%)
Subgraphs(p=2)
85.86% (0.2%)
83.75% (0.19))
85.78% (0.1%)
7488% (0.3%)
75.25% (0.6%)
75.12% (0.3%)
Rand. & (p=.(p=1)
86.68% (0.1%)
86.71% (0.2%)
86.77% (0.2%)
74.63% (0.3%)
75.37% (0.4%)
75.00% (0.3%)
Rand. & (pb.(p-2)
85.64% (0.2%)
85.50% (0.256
85.84% (0.2%)
74.75% (0.2%)
74.75%
(0.5%)
75.12% (0.3%
GIN
Fine Grid (L=1)
84.82% (0.4%)
73.89%
(0.3%)
Random(p=1)
84.65% (0.3%)
84.20% (0.2%)
84.68% (0.3%)
75.62%
(0.3%)
74.38% (3.8%)
74.88% (2.7%)
(p-dom(p=2)
85.27% (0.3%)
16. (0.39E)
85.55% (0.3%)
78%8 (0.4%)
75.62% (0.8%)
76.11% (1.0%)
Topk(p==)
84.98% (0.7%)
84.71% (0.1%)
84.09% (0.3%)
74.75% (0.7%)
74.01% (3.3%)
79.89% (6.0%)
Topk(p=2)
84.26% (0.1%%
84 44.99% (0.456)
85.47% (0.1%)
74.50% (0.7%)
74.13% (20.3%)
71.78% (16.9%)
Subgraphas(==1)
84.85% (0.2%)
84.00% (0.2%)
83.80% (0.1%)
74.88% (0.7%)
74.75% (0.3%)
75.25% (1.0%)
Subgraphs(p-2)
83.01% (0.6%)
85.21% (1.2%)
81.8%% (1.0%)
74.13% (0.39)
73.14% (0.1%)
72.77% (1.0%)
Rand. & (p=1(p=1)
84.73% (0.5%)
84.45% (0.456)
83.86% (0.1%)
75.25% (0.5%)
75.87% (0.7%)
75. 62% (1.5%)
Rand. & (p=2
84.51% (1.1%)
85.16% (0.1%)
84. 48% (0.1%)
74.75% (0.7%)
74.63% (0.2%)
70.79% (19.8%)
GAT
Fine Grid (T==1)
85.16% (0.2%)
72.65%
(0.79)
Random(p=1)
84.62% (0.2%)
84.71% (0.3%)
84.59% (0.3%)
74.38% (0.4%)
73.76%
(0.7%)
73.02% (0.5%)
Random(p-22
85.58% (0.3%)
85.05% (0.35E)
85.21% (0.5%)
73.64% (0.6%)
71.91%
(0.4%)
71.53% (0.4%))
Topk(p=1)
85.61% (0.3%)
85.07% (0.4%)
85.78% (0.7%)
73.27% (1.3%)
72.15% (0.8%)
72.15% (1.1%)
Topk(p=2)
85.86% (0.1%)
85.05% (0.2%)
81.12% (0.5%)
72. 40% (0.7%)
73.14% (1.8%)
72.90% (1.2%)
Subgraphs(p=11)
84.76% (0.1%)
84.71% (0.256)
84.59% (0.4%)
71.91% (3.1%))
71.16% (0.8%)
71.66% (0.7%)
Subgraphs(p=22)
85.84% (0.3%)
85.16% (0.2%)
85.30% (0.1%)
72.15% (2.2%)
71.16% (1.9%)
71.66% (2.0%)
Rand. & Sub.(p=1)
85.86% (0.3%)
85.30% (0.2%E)
85.17% (0.4%)
74.38% (1.3%)
72.63% (0.2%)
73.51% (0.7%)
Rand. & (p=.(p=2)
85.86% (0.3%)
85.30% (0.2%)
85.47% (0.4%)
72.28% (1.6%)
72.40% (0.1%)
70.54% (1.0%)
30

---

Published in Transactions on Machine Learning Research (11/2025)
Table 18: Experimental setup for transductive learning datasets.
Component
Details
Network architecture
GCN Kipf & Welling (2016) with 4 layers and 192 hidden channels. GIN Xu et al (2018) with layers and 256 hidden channels. GAT Velickovie et al. (2017) with 3 layers and
hidden channels, using 2 heads
Loss function
Negative Log Likelihood Loss except for PPI (transductive) and Facebook Yang et al. 20200) which Cross Entropy
Loss
Optimizer
Adam Kingma & Ba (2014)
Learning rate
1 1 10
Baseline training
2000 epochs
Mulstiscale gradients level
2, 3, and 4 levels
Coarse-to-fine strategy
Each coarsening reduces the number of nodes by half com-
pared to the previous level
Sub-to-Full strategy
Using ego-networks Gupta et al. (2014). hops on level 2,
4 on level 3, and on level 4
Multiscale training stralegy
Using [1000, 2000] epochs for 2 levels, [800, 1600, 3200] epochs for levels, and [600, 1200, 2400, 4800] epochs for levels (the first number the fine grid epoch number) Accuracy
Evaluation metric
Timing computation
GPU performance. An average of 100 epochs after training was stabled
Table 19: Comparisson with results as showes in Zeng et al. 2020 in Micro Fl score
Flickr
GCN
0.492
GraphSAGE
0.501
FastGCN
0.504
S-GCN
0.482
AS-GCN
0.504
ClusterGCN
0.481
GraphSALNT--Node
0.507
GraphSAINT- Edge
0.510
GraphSAINT--RE
0.511
GraphSAINT-
0.510
Ours
0.538
31

---

Published in Transactions on Machine Learning Research (11/2025)
H Additional Results for Shapenet Dataset
We test out methods the full Shapenet dataset Chang et al. (2015) using DGCNN Wang et al. (2018) and demonstrate the results of our multiscale training process in Table 20 We observe that our method is not limited to the 'Arpplane' category of this dataset, and we achieved comparable results across various connectivity values and pooling methods while using a siguificantly cheaper training process.
To emphasize the efficiency of our methods, we further provide timing analysis of Shapanet 'Airplanes' data computations. Timing was calculated using GPU computations, and averaged across epochs after the training loss stabilized. Results appear in Table 21 for multiscale training and in Table 22 for multiscale gradient compulalions, where we show that our training process was indeed faster than the original process, which only took into consideration the fine grid. We provide additional experimental details in Table 23
Table 20: Results full Shapenet dataset using DGCNN. Results represent Mean test ToU, using multiscale training.
Graph Density (kNN)
Fine Grid
Random
Subgraphs
Rand. & Sub.
k-6
77.86% (0.07%)
79.14% (0.17%)
77.92% (0.41%)
77.96% (0.07%)
k=10
79.70% (0.44%)
79.78% (0.32%)
79.41% (0.12%)
79.42% (0.25%)
k-20
80.16% (0.30%)
80.75% (0.18%)
80.51% (0.23%)
80.67% (0.22%)
Table 21: Average epoch time for Shapenet 'Airplane' dataset using DGCNN, using multiscale training. Time is calculated in mili seconds.
Graph Density (kNN)
Fine Grid
Random
Subgraphs
Rand. & Sub.
k=6
18,596
13,981
15,551
14,760
k=10
20,729
15,409
17,015
16,288
k=20
29,979
21,081
22,646
21,891
Table 22: Average epoch time for Shapenet 'Airplane' datasel using DGCNN, using multiscale gradient computation training. Time is calculated in mili seconds.
Graph Density (kNN)
Fine Grid
Random
Subgraphs
Rand. & Sub.
k=6
44,849
28,666
30,265
29,465
k= 10
54,064
34,528
36,094
35,299
k=20
89,923
58,840
60,660
59,814
32

---

Published in Transactions on Machine Learning Research (11/2025)
Table 23: Experimental setup for ShapeNet dataset.
Component
Details
Dataset
ShapeNet Chang et al. (2015), 'Airplanes' category
Network architecture
DGCNN Wang et al. 2018 with 3 layers and 64 hidden
channels
Loss function
Negative Log Likelihood Loss
Optimizer
Adam Kingma & Ba (0014)
Learning rate
1 x 10
Batch size
for multiscale training, 12 for multiscale gradient compu- tation
Multiscale levels
3, each time reducing graph size by half
Epochs per level
Fine training used 100 epochs. Multiscalc training used [40,
80, 160] epochs for the 3 levels.
Mulstiscale gradients level
2, coarse graph reduces graph size by 0.75%
Multiscale gradients strategy
50 epochs using multilevels gradients, 50 epochs using origi-
nal fine data
Evaluation metric
Mean loU (Intersection over Union)
33

---

Published in Transactions on Machine Learning Research (11/2025)
Additional Inductive Learning Datasets
The PPT dataset Zitnik & Leskoved (2017): is a biological graph dataset where nodes represent, proteins, edges indicate interactions between them, and each protein is associated with multiple labels describing its biological functions. includes 20 graphs for training, 2 for validation, and 2 for testing, with an average of 2,372 nodes per graph. Each node has 50 features and is assigned to one or more of 121 possible labels. We use the preprocessed data from Hamilton et al. (0177) and present our results in Table 24 In this experiment, we use the Graph Convolutional Network with Initial Residual and Identity Mapping (GCNII) Chen et al,
(2020). The details of the network architecture and the experiment are available in Table 25 Our results
demonstrate that the Multiscale Gradients Computation method achieves comparable accuracy across various pooling techniques while significantly reducing training costs.
We further evaluate our methods on the NCI1 (Wale et al. 2008) and ogbg-molhiv (Hu et al. 2020) datasets, which are inductive learning benchmarks consisting of multiple graphs (4,110 for NCI and 41,127 for ogbg-molhiv). NCI1 consists of chemical compounds represented as graphs, where nodes are atoms and edges are chemical bonds, with the goal of predicting whet her a compound active against specific cancer cell lines, while MolHIV is a molecular property prediction dataset, where the task is to predict whether a molecule inhibits HIV replication based on its molecular structure. The task for both datasets is graph classification, and we note that as stated in Coupetle et al. 2025 those dalasets are considered to be informative datasets. Our results, presented in Table 26 demonstrate that on these multi- graph datasets and for this type of task, our methods achieve performance that is comparable to, and in some cases surpasses, the baseline results.
Table 24: Results on PPI (inductive) dataset using GCNII and multiscale gradient computation method.
Method
F1 Score
Fine Grid
99.27% (0.11%)
Random
99.18%
(0.02%
Topk
99.23%
(0.04%)
Subgraph
99.21%
(1.40%)
Rand. & Sub.
99.20% (0.01%)
Table 25; Experimental setup for PPI dataset.
Component
Details
Dataset
PPI Zitnik & Leskovec (2017)
Network architecture
Chen Chen et al. (2020) with 9 layers and 2048 hidden
channels
Loss function
Binary Cross Entropy with logits
Optimizer
Adam Kingma & Ba (014
Learning rate
5 10
Wight initialization
Xavier Glorot & Bengio (010))
Batch size
Baseline training
1000 epochs
Mulstiscale gradients level
2, coarse graph reduces graph size by 0.75%
Multiscale gradients strategy
500 epochs using multilevels gradients, 500 epochs using
original fine dala
Evaluation metric
F1
34

---

Published in Transactions on Machine Learning Research (11/2025)
Table 26: Results on NCII and MolHIV datasets using GCN and multiscale training with levels.
Method NCI1 (Acc.)
MolHIV (AUC-ROC)
Fine Grid
69.37% (0.42%)
70.39% (0.76%)
Random 70.45% (2.14%)
70.41% (1.52%)
Topk 69.37% (0.83%)
70.40% (1.62%)
Subgraph
69.73% (1.70%)
70.94% (1.59%)
Rand. & Sub.
68.56% (0.88%)
71.01% (1.35%)
35

---

Published in Transactions on Machine Learning Research (11/2025)
J
Ablation Study
We conduct an ablation study on the PubMed dataset Sen et al. 2008). In Table 27 we evaluate the impact of different coarsening ratios m between levels. The results show that performance remains relatively stable across a range of values. For our main experiments, we choose m = } as it balances two trade-offs: minimal coarsening may not yield significant training efficiency gains, while excessive coarsening especially with multiple levels-can degrade performance.
In Table 28 we analyze how the allocation of training epochs across levels affects results. We compare our default strategy, which doubles the number of epochs at each coarsening level, with two alternatives: using constant number of epochs per level, and adding fixed number of epochs at each level. All strategies yield similar accuracy, but we adopt the exponential schedule, as it emphasizes learning on coarser levels, where training is more efficient.
Table 27: Various coarsening ratios for using PubMed dataset with GCN and 3 levels of coarsening.
4
Random
77.8% (0.4%)
77.7% (0.2%)
77.5% (0.6%)
77.3% (0.5%
77.4% (0.3%)
Topk
76.4% (0.5%)
77.5% (0.8%)
78.2% (0.3%)
76.9% (0.6%)
77.5% (0.8%)
Subgraphs
78.0% (0.3%)
77.9% (0.5%)
78.0% (0.3%)
78.0% (0.7%)
77.8% (0.4%)
Rand. & Sub.
78.2% (0.3%)
78.0% (0.5%)
77.8% (0.8%)
78.1% (0.3%)
77.7% (0.4%)
Table 28: Various epoch distributions using the PubMed dataset with GCN and 3 levels of coarsening.
Default
Constant
Additive
Random
77.5% (0.6%)
77.20% (0.3%)
78.10% (0.6%)
Topk
78.2% (0.3%)
76.80% (0.4%)
76.70% (0.3%)
Subgraphs
78.0% (0.3%)
78.50% (0.8%)
77.60% (0.4%)
Rand. & Sub.
77.8% (0.8%) 77.40% (0.3%)
78.00% (0.6%)
36


Reviews:
[{"review_id": "QFD2IkO7HT", "reviewer": "Reviewer_mBZE", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\n - First paper to apply multiscale optimization ideas to GNN training, demonstrating applicability across different pooling methods and model architectures.\n\n - The paper is clearly written and generally easy to follow.\n\nWeaknesses\n\n - No anonymized code repository is provided, which limits reproducibility.\n\n - Comparisons are restricted to standard GCN/GIN/GAT baselines; stronger scalable GNN methods (e.g., GraphSAINT, Cluster-GCN) are not included.\n\n - Experiments cover a limited range of datasets and should be expanded to strengthen the evaluation.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes a multiscale training framework for GNNs to reduce computational and memory costs while maintaining similar accuracy. It introduces three strategies: (1) training on coarsened graphs followed by finer graphs, (2) training on progressively larger subgraphs until reaching the full graph, and (3) combining coarse and fine gradient estimates to approximate fine-scale gradients. Extensive experiments are conducted on both transductive and inductive graph tasks.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Provide an anonymized implementation to ensure reproducibility and allow others to verify the results. \n\n- Include stronger scalable GNN training methods.\n\n- Add experiments on larger and more diverse real-world datasets.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "39DZqHZsi9", "reviewer": "Reviewer_BQ5m", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths of the paper:\n\n1. The authors started with a good, natural idea - multiscale training of neural networks\n2. Experimental evidence provided is very solid.\n3. Authors provided an explicit theoretical condition when their approach may work [Eqn. (9)].\n4. Authors connected their idea to a simple optimization problem in linear algebra (Section 4.5).\n5. Many (but not all) parts of the paper are clearly written.\n6. Authors are well-versed with state-of-the art research.\n\nWeaknesses of the paper\n1. No anonymized repository was provided, so it is not possible to rerun their experiments.\n2. The connection with sketching in linear algebra is poorly motivated and poorly explained. \n3. It is unclear whether condition the authors stated for applicability of their method [Eqn. (9)] has relevance in practice.\n4. The paper is too long -- 29 pages. Sufficient information could be conveyed more concisely.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "Contributions:\n\n1. Designing several approaches for multiscale training of graph neural networks.\n2. Providing considerable experimental validation of approaches.\n3. Connecting multiscale training to so-called sketching in linear algebra.\n4. Formulating a condition [Eqn. (9)] when multiscale training yields improvement.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "**Critical**\n\n1. Better explain and motivate connection with sketching problem -- or remove it altogether.\n\nExample problems:\n- it is not clear whether there is an optimization problem in equation (11). \n- notation for loss function differs from both (1) and Section 4.3, and thus it is hard to see any connection to the paper\n- statement of Theorem 4.1 uses O() notation but is not clear what is the underlying value that changes ($\\epsilon$ ??)\n- why is $\\pm$ symbol used in (13)\n- in the proof in the appendix R is not defined. If one argues that you defined it in the statement of Theorem 4.1, I would point out that it was far removed from actual use, and actual definition of R in that theorem's statement is a bit confusing.\n\n2. Better explain whether condition (9) has relevance in practice. If it has, you should provide values of $\\gamma_r$ used in your experiments.\n\n3. Shorten your experimental evidence to minimum necessary. Note that the numbers listed in tables are likely not reproducible exactly, anyways.\n\n4. Make your notation uniform, or provide an explanation why is that not a good idea. I am concerned about the notation for loss function (\"l\" in various fonts), and parameters ($\\boldsymbol{W}$, $\\boldsymbol{\\theta}$, $\\boldsymbol{\\Theta}$, $\\theta$.\n\n5. Datasets cleanup. \n- If you explain meaning of one dataset (OGBN-MAG), you should do it for all (e.g. OGBN-Arxiv). \n\n6. Please provide an anonymous code repository so that readers can rerun your experiments.\n\n7. Equation  (5) was not clear to me. First, you use term \"injection matrix\" which I am not familiar with. Secondly, the $p$-th power of A has entries which are NOT zeros and ones. Multiplying by a binary matrix (i.e. a matrix with zeros and ones) will not yield an incidence matrix of a subgraph, since generally speaking the entries of $A^p$ will be bigger than 1. What am I not getting here?\n\n8. It is unclear what are high-and low frequency components of a graph on page 9.\n\n**Would strengthen the work**\n\n9. A recent paper cast doubt on various benchmark datasets used for graph neural networks (https://arxiv.org/pdf/2502.02379). Please provide an example for at least one of the benchmark datasets they identified as good (MolHIV, NCI1, Peptides).", "broader_impact_concerns": "I see no broader impact concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "roPhDDjdHy", "reviewer": "Reviewer_PVUu", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n+ The paper addresses an important problem, dealing with the large computational cost of training GNNs on large graphs.\n+ The proposed method leads to significant speedups with no loss (or even small increase) in accuracy, which is nice since the gains in efficiency do not come at the cost of degraded model performance.\n+ Experiments are performed on both transductive and inductive learning tasks, demonstrating potential broad applicability of the proposed methods.\n\nWeaknesses:\n+ The intuition behind the method is unclear. As a result, it is difficult to judge what types of graphs/tasks the proposed methods can be applied to.\n+ While different subgraph selection methods are introduced, it is unclear how the optimal method can be determined for a given task or graph. If different selection methods need to be tried out and the best one is chosen, then the reported training times should also include the this overhead, and not just the time taken with the optimal method.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "A new method for efficient training of GNNs is introduced, that uses concepts from multiscale learning to train on smaller subgraphs that mimic the characteristics of the full large graph. Techniques for creating subgraphs and combing gradients across different scales are also introduced.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Critical for acceptance:\n+ Clarify the types of graph structures and/or tasks for which the proposed method can be used. While creating subgraphs makes sense for citation graphs used in the paper, what does subgraph creation mean for molecular graphs for example? How can some nodes be removed from such graphs without altering the meaning of the underlying graph? \n+ Clearly describe how the optimal training configuration is chosen for a given graph and task. (1) Should the different subgraph selection strategies (random, topk, etc.) be tried out and only the best result taken? If so, the training time will need to consider the amount of time taken to try out the different strategies also, so the reported speedups may not hold in practical scenarios. (2) Is there a way to identify the best strategy without this expensive process based on graph topology? (3) When should coarse-to-fine and sub-to-full strategies be used? Or should both always be used in combination with each other?\n+ An ablation with the different proposed strategies turned on/off in multiple combinations would help understand the sensitivity of the proposed method to hyperparemeters. \n+ Add a discussion on the limitations and scope of the proposed methods.\n\nWould strengthen the paper:\n+ Code release, along with the Q-tips dataset, would help reproducibility, In particular, the Q-tips dataset seems to be an interesting synthetic dataset that can help evaluate ability of GNNs to capture long-range dependencies.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "I1SahfRmnn", "reviewer": "Reviewer_yG2D", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n\n1. **Practical Impact**: Addresses a critical scalability challenge in GNN training, making the work highly relevant for real-world applications with large graphs.\n\n2. **Comprehensive Framework**: Provides three complementary methods that can be applied to any GNN architecture, demonstrating versatility.\n\n3. **Extensive Experimental Validation**: \n   - Tests on 15+ datasets covering both transductive and inductive settings\n   - Evaluates multiple GNN architectures and pooling strategies\n   - Shows consistent computational savings (often 2-4x reduction in FLOPs)\n\n4. **Theoretical Foundation**: Provides theoretical analysis connecting the approach to sketching theory, though limited to the linear case.\n\n5. **Clear Presentation**: Well-written paper with effective visualizations (Figures 1-3) that clearly illustrate the core concepts.\n\n**Weaknesses**\n\n1. **Theoretical Analysis**:\n   - The assumption might be too strong. Theorem 4.1 only covers the linear case (σ = Id), not realistic nonlinear GNNs\n   - The approximation bound depends on residual R = ||A_CF X_F Θ*_C||, which can be large\n\n2. **Inconsistent Performance**:\n   - Some methods show performance degradation (e.g., Table 4: OGBN-MAG results)\n   - High variance in some results (e.g., GCN with Topk on Cora: 79.9% ± 4.4%)\n   - No clear guidance on when each method works best", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents a multiscale training framework for GNNs to address computational challenges when training on large graphs. The authors propose three novel training strategies: (1) Coarse-to-Fine Training: Solves optimization problems on progressively finer graph resolutions, using coarse solutions as initialization for finer scales\n(2) Sub-to-Full Training: Leverages sequences of growing subgraphs that progressively expand to the original graph\n(3) Multiscale Gradients Computation: Inspired by Monte Carlo methods, approximates fine-scale gradients by combining gradients across multiple scales\n\nThe paper provides theoretical analysis (Theorem 4.1) showing how coarse graphs can approximate fine-scale solutions under certain conditions. Extensive experiments demonstrate that these methods reduce computational costs (measured in FLOPs and training time) while maintaining or improving performance across various datasets (OGBN-Arxiv, OGBN-MAG, Cora, CiteSeer, PubMed, ShapeNet, PPI) and GNN architectures (GCN, GIN, GAT, DGCNN, GCNII).", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "### Critical for Acceptance:\n\n1. **Add Comparisons with Existing Methods**: Include empirical comparisons with sampling-based efficient GNN training methods (GraphSAINT, FastGCN, ClusterGCN). This is essential to position the work properly in the literature.\n\n2. **Improve Theoretical Analysis**: \n   - Extend analysis beyond the linear case or clearly state limitations\n   - Provide theoretical justification for the choice of p in equation (5)\n   - Add analysis for Sub-to-Full and Multiscale Gradients methods\n\n3. **Add Method Selection Guidelines**: Provide clear recommendations on when to use each method based on graph properties (size, density, task type). Include a decision tree or flowchart.\n\n4. **Memory Usage Analysis**: Include memory consumption comparisons alongside FLOPs and timing analysis, as memory is often the bottleneck for large graphs.\n\n### Would Strengthen the Work:\n\n\n1. **Ablation Studies**: \n   - Study the effect of different coarsening ratios\n   - Analyze sensitivity to the power p parameter\n   - Investigate the impact of epoch distribution across levels\n\n\n2. **Implementation Details**: \n   - Provide pseudocode for all three methods", "broader_impact_concerns": "The paper does not raise significant ethical concerns. The work focuses on improving computational efficiency of GNN training, which could have positive environmental impact by reducing energy consumption.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)