Original: G RAPH C OARSENING WITH N EURAL N ETWORKS
Revised: G RAPH C OARSENING WITH N EURAL N ETWORKS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: As large-scale graphs become increasingly more prevalent, it poses signiﬁcant computational challenges to process, extract and analyze large graph data.
Revised: As large-scale graphs become increasingly more prevalent, it poses signiﬁcant computational challenges to process, extract and analyze large graph data.

Original: Graph coarsening is one popular technique to reduce the size of a graph while maintaining essential properties.
Revised: Graph coarsening is one popular technique to reduce the size of a graph while maintaining essential properties.

Original: Despite rich graph coarsening literature, there is only limited exploration of data-driven methods in the ﬁeld.
Revised: Despite rich graph coarsening literature, there is only limited exploration of data-driven methods in the ﬁeld.

Original: In this work, we leverage the recent progress of deep learning on graphs for graph coarsening.
Revised: In this work, we leverage the recent progress of deep learning on graphs for graph coarsening.

Original: We ﬁrst propose a framework for measuring the quality of coarsening algorithm and show that depending on the goal, we need to carefully choose the Laplace operator on the coarse graph and associated projection/lift operators.
Revised: We ﬁrst propose a framework for measuring the quality of coarsening algorithm and show that depending on the goal, we need to carefully choose the Laplace operator on the coarse graph and associated projection/lift operators.

Original: Motivated by the observation that the current choice of edge weight for the coarse graph may be suboptimal, we parametrize the weight assignment map with graph neural networks and train it to improve the coarsening quality in an unsupervised way.
Revised: Motivated by the observation that the current choice of edge weight for the coarse graph may be suboptimal, we parametrize the weight assignment map with graph neural networks and train it to improve the coarsening quality in an unsupervised way.

Original: Through extensive experiments on both synthetic and real networks, we demonstrate that our method signiﬁcantly improves common graph coarsening methods under various metrics, reduction ratios, graph sizes, and graph types.
Revised: Through extensive experiments on both synthetic and real networks, we demonstrate that our method signiﬁcantly improves common graph coarsening methods under various metrics, reduction ratios, graph sizes, and graph types.

Original: It generalizes to graphs of larger size ( 25 × of training graphs),  adaptive to different losses (differentiable and non-differentiable), and scales to much larger graphs than previous work.
Revised: It generalizes to graphs of larger size ( 25 × of training graphs), is adaptive to different losses (differentiable and non-differentiable), and scales to much larger graphs than previous work.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Many complex structures can be modeled by graphs, such as social networks, molecular graphs, biological protein-protein interaction networks, knowledge graphs, and recommender systems.
Revised: Many complex structures can be modeled by graphs, such as social networks, molecular graphs, biological protein-protein interaction networks, knowledge graphs, and recommender systems.

Original: As large scale-graphs become increasingly ubiquitous in various applications, they pose signiﬁcant computational challenges to process, extract and analyze information.
Revised: As large scale-graphs become increasingly ubiquitous in various applications, they pose signiﬁcant computational challenges to process, extract and analyze information.

Original: It is therefore natural to look for ways to simplify the graph while preserving the properties of interest.
Revised: It is therefore natural to look for ways to simplify the graph while preserving the properties of interest.

Original: There are two major ways to simplify graphs.
Revised: There are two major ways to simplify graphs.

Original: First, one may reduce the number of edges, known as graph edge sparsiﬁcation.
Revised: First, one may reduce the number of edges, known as graph edge sparsiﬁcation.

Original: It is known that pairwise distance (spanner), graph cut (cut sparsiﬁer), eigenvalues (spectral sparsiﬁer) can be approximately maintained via removing edges.
Revised: It is known that pairwise distance (spanner), graph cut (cut sparsiﬁer), eigenvalues (spectral sparsiﬁer) can be approximately maintained via removing edges.

Original: A key result (Spielman & Teng, 2004) in the spectral sparsiﬁcation is that any dense graph of size N can be sparsiﬁed to O
Revised: A key result (Spielman & Teng, 2004) in the spectral sparsiﬁcation is that any dense graph of size N can be sparsiﬁed to O

Original: ( Nlog c N/(cid:15) 2 ) edges in nearly linear time using a simple randomized algorithm based on the effective resistance.
Revised: ( Nlog c N/(cid:15) 2 ) edges in nearly linear time using a simple randomized algorithm based on the effective resistance.

Original: Alternatively, one could also reduce the number of nodes to a subset of the original node set.
Revised: Alternatively, one could also reduce the number of nodes to a subset of the original node set.

Original: The ﬁrst challenge here is how to choose the topology (edge set) of the smaller graph spanned by the sparsiﬁed node set.
Revised: The ﬁrst challenge here is how to choose the topology (edge set) of the smaller graph spanned by the sparsiﬁed node set.

Original: On the extreme, one can take the complete graph spanned by the sampled nodes.
Revised: On the extreme, one can take the complete graph spanned by the sampled nodes.

Original: However, its dense structure prohibits easy interpretation and poses computational overhead for setting the Θ( n 2 ) weights of edges.
Revised: However, its dense structure prohibits easy interpretation and poses computational overhead for setting the Θ( n 2 ) weights of edges.

Original: This paper focuses on graph coarsening , which reduces the number of nodes by contracting disjoint sets of connected vertices.
Revised: This paper focuses on graph coarsening , which reduces the number of nodes by contracting disjoint sets of connected vertices.

Original: The original idea dates back to the algebraic multigrid literature (Ruge & St¨uben, 1987) and has found various applications in graph partitioning (Hendrickson & Leland, 1995; Karypis & Kumar, 1998; Kushnir et al., 2006), visualization (Harel & Koren, 2000; Hu, 2005; Walshaw, 2000) and machine learning (Lafon &
Revised: The original idea dates back to the algebraic multigrid literature (Ruge & St¨uben, 1987) and has found various applications in graph partitioning (Hendrickson & Leland, 1995; Karypis & Kumar, 1998; Kushnir et al., 2006), visualization (Harel & Koren, 2000; Hu, 2005; Walshaw, 2000) and machine learning (Lafon &

Original: Lee, 2006; Gavish et al., 2010; Shuman et al., 2015).
Revised: Lee, 2006; Gavish et al., 2010; Shuman et al., 2015).

Original: However, most existing graph coarsening algorithms come with two restrictions.
Revised: However, most existing graph coarsening algorithms come with two restrictions.

Original: First, they are prespeciﬁed and not adapted to speciﬁc data nor different goals.
Revised: First, they are prespeciﬁed and not adapted to speciﬁc data nor different goals.

Original: Second, most coarsening algorithms set the edge weights of the coarse graph equal to the sum of weights of crossing edges in the original graph.
Revised: Second, most coarsening algorithms set the edge weights of the coarse graph equal to the sum of weights of crossing edges in the original graph.

Original: This means the weights of the coarse graph is determined by the coarsening algorithm (of the vertex set), leaving no room for adjustment.
Revised: This means the weights of the coarse graph is determined by the coarsening algorithm (of the vertex set), leaving no room for adjustment.

Original: With the two observations above we aim to develop a data-driven approach to better assigning weights for the coarse graph depending on speciﬁc goals at hand.
Revised: With the two observations above we aim to develop a data-driven approach to better assigning weights for the coarse graph depending on speciﬁc goals at hand.

Original: We will leverage the recent progress of deep learning on graphs to develop a framework to learn to assign edge weights in an unsupervised manner from a collection of input (small) graphs.
Revised: We will leverage the recent progress of deep learning on graphs to develop a framework to learn to assign edge weights in an unsupervised manner from a collection of input (small) graphs.

Original: This learned weight-assignment map can then be applied to new graphs (of potentially much larger sizes).
Revised: This learned weight-assignment map can then be applied to new graphs (of potentially much larger sizes).

Original: In particular, our contributions are threefold.
Revised: In particular, our contributions are threefold.

Original: • First, depending on the quantity of interest F (such as the quadratic form w.r.t.
Revised: • First, depending on the quantity of interest F (such as the quadratic form w.r.t.

Original: Laplace operator), one has to carefully choose projection/lift operator to relate quantities deﬁned on graphs of different sizes.
Revised: Laplace operator), one has to carefully choose projection/lift operator to relate quantities deﬁned on graphs of different sizes.

Original: We formulate this as the invariance of F under lift map, and provide three cases of projection/lift map as well as the corresponding operators on the coarse graph.
Revised: We formulate this as the invariance of F under lift map, and provide three cases of projection/lift map as well as the corresponding operators on the coarse graph.

Original: Interestingly, those operators all can be seen as the special cases of doubly-weighted Laplace operators on coarse graphs (Horak & Jost, 2013).
Revised: Interestingly, those operators all can be seen as the special cases of doubly-weighted Laplace operators on coarse graphs (Horak & Jost, 2013).

Original: Second, we are the ﬁrst to propose and develop a framework to learn the edge weights of the coarse graphs via graph neural networks (GNN) in an unsupervised manner.
Revised: Second, we are the ﬁrst to propose and develop a framework to learn the edge weights of the coarse graphs via graph neural networks (GNN) in an unsupervised manner.

Original: We show convincing results both theoretically and empirically that changing the weights is crucial to improve the quality of coarse graphs.
Revised: We show convincing results both theoretically and empirically that changing the weights is crucial to improve the quality of coarse graphs.

Original: • Third, through extensive experiments on both synthetic graphs and real networks, we demonstrate that our method GOREN signiﬁcantly improves common graph coarsening methods under different evaluation metrics, reduction ratios, graph sizes, and graph types.
Revised: • Third, through extensive experiments on both synthetic graphs and real networks, we demonstrate that our method GOREN signiﬁcantly improves common graph coarsening methods under different evaluation metrics, reduction ratios, graph sizes, and graph types.

Original: It generalizes to graphs of larger size (than the training graphs), adapts to different losses (so as to preserve different properties of original graphs), and scales to much larger graphs than what previous work can handle.
Revised: It generalizes to graphs of larger size (than the training graphs), adapts to different losses (so as to preserve different properties of original graphs), and scales to much larger graphs than what previous work can handle.

Original: Even for losses that are not differentiable w.r.t the weights of the coarse graph, we show training networks with a differentiable auxiliary loss still improves the result.
Revised: Even for losses that are not differentiable w.r.t the weights of the coarse graph, we show training networks with a differentiable auxiliary loss still improves the result.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Graph sparsiﬁcation .
Revised: Graph sparsiﬁcation .

Original: Graph sparsiﬁcation is ﬁrstly proposed to solve linear systems involving combinatorial graph Laplacian efﬁciently.
Revised: Graph sparsiﬁcation is ﬁrstly proposed to solve linear systems involving combinatorial graph Laplacian efﬁciently.

Original: Spielman & Teng (2011); Spielman & Srivastava (2011) showed that for any undirected graph G of N vertices, a spectral sparsiﬁer of G with only
Revised: Spielman & Teng (2011); Spielman & Srivastava (2011) showed that for any undirected graph G of N vertices, a spectral sparsiﬁer of G with only

Original: O ( Nlog c N/(cid:15) 2 ) edges can be constructed in nearly-linear time.
Revised: O ( Nlog c N/(cid:15) 2 ) edges can be constructed in nearly-linear time.

Original: 1 Later on, the time complexity and the dependency on the number of the edges are reduced by various researchers (Batson et al., 2012;
Revised: 1 Later on, the time complexity and the dependency on the number of the edges are reduced by various researchers (Batson et al., 2012;

Original: Allen-Zhu et al., 2015; Lee & Sun, 2018; 2017).
Revised: Allen-Zhu et al., 2015; Lee & Sun, 2018; 2017).

Original: Graph coarsening .
Revised: Graph coarsening .

Original: Previous work on graph coarsening focuses on preserving different properties, usually related to the spectrum of the original graph and coarse graph.
Revised: Previous work on graph coarsening focuses on preserving different properties, usually related to the spectrum of the original graph and coarse graph.

Original: Loukas & Vandergheynst (2018); Loukas (2019) focus on the restricted spectral approximation, a modiﬁcation of the spectral similarity measure used for graph sparsiﬁcation.
Revised: Loukas & Vandergheynst (2018); Loukas (2019) focus on the restricted spectral approximation, a modiﬁcation of the spectral similarity measure used for graph sparsiﬁcation.

Original: Hermsdorff & Gunderson (2019) develop a probabilistic framework to preserve inverse Laplacian.
Revised: Hermsdorff & Gunderson (2019) develop a probabilistic framework to preserve inverse Laplacian.

Original: Deep learning on graphs .
Revised: Deep learning on graphs .

Original: As an effort of generalizing convolution neural network to the graphs and manifolds, graph neural networks is proposed to analyze graph-structured data.
Revised: As an effort of generalizing convolution neural network to the graphs and manifolds, graph neural networks is proposed to analyze graph-structured data.

Original: They have achieved state-of-the-art performance in node classiﬁcation (Kipf & Welling, 2016), knowledge graph completion (Schlichtkrull et al., 2018), link prediction (Dettmers et al., 2018), combinatorial optimization (Li et al., 2018;
Revised: They have achieved state-of-the-art performance in node classiﬁcation (Kipf & Welling, 2016), knowledge graph completion (Schlichtkrull et al., 2018), link prediction (Dettmers et al., 2018), combinatorial optimization (Li et al., 2018b; Khalil et al., 2017), property prediction (Duvenaud et al., 2015; Xie & Grossman, 2018) and physics simulation (Sanchez-Gonzalez et al., 2020).

Original: Khalil et al., 2017), property prediction (Duvenaud et al., 2015; Xie &
Revised: Deep generative model for graphs . To generative realistic graphs such as molecules and parse trees, various approaches have been taken to model complex distributions over structures and attributes, such as variational autoencoder (Simonovsky & Komodakis, 2018; Ma et al., 2018), generative adversarial networks (GAN) (De Cao & Kipf, 2018; Zhou et al., 2019), deep autoregressive model (Liao et al., 2019; You et al., 2018b; Li et al., 2018a), and reinforcement learning type approach (You et al., 2018a).

Original: Grossman, 2018) and physics simulation (Sanchez-Gonzalez et al., 2020).
Revised: Zhou et al. (2019) proposes a GAN-based framework to preserve the hierarchical community structure via algebraic multigrid method during the generation process. However, different from our approach, the coarse graphs in Zhou et al. (2019) are not learned.

Original:  P ROPOSED A PPROACH : L EARNING E DGE W EIGHT WITH GNN 3. H IGH - LEVEL OVERVIEW
Revised:  P ROPOSED A PPROACH : L EARNING E DGE W EIGHT WITH GNN 3. H IGH - LEVEL OVERVIEW

Original: Minimize
Revised: Minimize

Original: Our input is a non-attributed (weighted or unweighted) graph G =
Revised: Our input is a non-attributed (weighted or unweighted) graph G =

Original: ( V, E ) .
Revised: ( V, E ) .

Original: Our goal is to construct an appropriate “coarser” graph (cid:98) G =
Revised: Our goal is to construct an appropriate “coarser” graph (cid:98) G =

Original: ( (cid:98) V , (cid:98) E ) that preserves certain properties of G .
Revised: ( (cid:98) V , (cid:98) E ) that preserves certain properties of G .

Original: Here, by a “coarser” graph, we assume that | (cid:98) V | << | V | and there is a surjective map π : V → (cid:98) V that we call the vertex map .
Revised: Here, by a “coarser” graph, we assume that | (cid:98) V | << | V | and there is a surjective map π : V → (cid:98) V that we call the vertex map .

Original: Intuitively, (see ﬁgure on the right), for any node ˆ v ∈ (cid:98) V , all nodes π − 1 (ˆ v ) ⊂
Revised: Intuitively, (see ﬁgure on the right), for any node ˆ v ∈ (cid:98) V , all nodes π − 1 (ˆ v ) ⊂

Original: V are mapped to this super-node ˆ v in the coarser graph (cid:98) G .
Revised: V are mapped to this super-node ˆ v in the coarser graph (cid:98) G .

Original: We will later propose a GNN based framework that can be trained using a collection of existing graphs in an unsupervised manner , so as to construct such a coarse graph (cid:98) G for a future input graph G (presumably coming from the same family as training graphs) that can preserve properties of G effectively.
Revised: We will later propose a GNN based framework that can be trained using a collection of existing graphs in an unsupervised manner , so as to construct such a coarse graph (cid:98) G for a future input graph G (presumably coming from the same family as training graphs) that can preserve properties of G effectively.

Original: We will in particular focus on preserving properties of the Laplace operator O G of G , which is by far the most common operator associated to graphs, and forms the foundation for spectral methods.
Revised: We will in particular focus on preserving properties of the Laplace operator O G of G , which is by far the most common operator associated to graphs, and forms the foundation for spectral methods.

Original: Speciﬁcally, given G =
Revised: Speciﬁcally, given G =

Original: ( V = { v 1 , . . .
Revised: ( V = { v 1 , . . .

Original: , v N } , E ) with w : E → R being the weight function for
Revised: , v N } , E ) with w : E → R being the weight function for

Original: G (all edges have weight 1 if G is unweighted), let W the corresponding N × N edge-weight matrix where W [ i ][ j ] = w ( v i , v j ) if edge ( v i , v j ) ∈ E and 0 otherwise.
Revised: G (all edges have weight 1 if G is unweighted), let W the corresponding N × N edge-weight matrix where W [ i ][ j ] = w ( v i , v j ) if edge ( v i , v j ) ∈ E and 0 otherwise.

Original: Set D to be the N × N diagonal matrix with D [ i ][ i ] equal to the sum of weights of all edges incident to v i .
Revised: Set D to be the N × N diagonal matrix with D [ i ][ i ] equal to the sum of weights of all edges incident to v i .

Original: The standard (unnormalized) combinatorial Laplace operator of G is then deﬁned as L = D − W .
Revised: The standard (unnormalized) combinatorial Laplace operator of G is then deﬁned as L = D − W .

Original: The normalized
Revised: The normalized

Original: Laplacian is deﬁned as L = D 
Revised: Laplacian is deﬁned as L = D −

Original: [Equation]
Revised: [Equation]

Original: However, to make this problem as well as our proposed approach concrete, various components need to be built appropriately.
Revised: However, to make this problem as well as our proposed approach concrete, various components need to be built appropriately.

Original: We provide an overview here, and they will be detailed in the remainder of this section.
Revised: We provide an overview here and they will be detailed in the remainder of this section.

Original: • Assuming that the set of super-nodes (cid:98) V as well as the map π : V → (cid:98) V are given, one still need to decide how to set up the connectivity (i e, edge set (cid:98) E ) for the coarse graph (cid:98) G =
Revised: • Assuming that the set of super-nodes (cid:98) V as well as the map π : V → (cid:98) V are given, one still need to decide how to set up the connectivity (i.e, edge set (cid:98) E ) for the coarse graph (cid:98) G =

Original: ( (cid:98) V , (cid:98) E ) .
Revised: ( (cid:98) V , (cid:98) E ) .

Original: We introduce a natural choice in Section 3.2, and provide some justiﬁcation for this choice.
Revised: We introduce a natural choice in Section 3.2, and provide some justiﬁcation for this choice.

Original: • As the graph G and the coarse graph (cid:98) G have the different number of nodes, their Laplace operators O G and O (cid:98) G of two graphs are not directly comparable.
Revised: • As the graph G and the coarse graph (cid:98) G have the different number of nodes, their Laplace operators O G and O (cid:98) G of two graphs are not directly comparable.

Original: Instead, we will compare F ( O G , f ) and F ( O (cid:98) G , ˆ f ) , where F is a functional intrinsic to the graph at hand (invariant to the permutation of vertices), such as the quadratic form or Rayleigh quotient.
Revised: Instead, we will compare F ( O G , f ) and F ( O (cid:98) G , ˆ f ) , where F is a functional intrinsic to the graph at hand (invariant to the permutation of vertices), such as the quadratic form or Rayleigh quotient.

Original: However, it turns out that depending on the choice of F , we need to choose the precise form of the Laplacian O (cid:98) G , as well as the (so-called lifting and projection) maps relating these two objects, carefully, so as they are comparable.
Revised: However, it turns out that depending on the choice of F , we need to choose the precise form of the Laplacian O (cid:98) G , as well as the (so-called lifting and projection) maps relating these two objects, carefully, so as they are comparable.

Original: We describe these in detail in Section 3.3.
Revised: We describe these in detail in Section 3.3.

Original: • In Section 3.4 we show that adjusting the weights of the coarse graph (cid:98) G can signiﬁcantly improve the quality of (cid:98) G .
Revised: • In Section 3.4 we show that adjusting the weights of the coarse graph (cid:98) G can signiﬁcantly improve the quality of (cid:98) G .

Original: This motivates a learning approach to learn a strategy (a map) to assign these weights from a collection of given graphs.
Revised: This motivates a learning approach to learn a strategy (a map) to assign these weights from a collection of given graphs.

Original: We then propose a GNN-based framework to do so in an unsupervised manner.
Revised: We then propose a GNN-based framework to do so in an unsupervised manner.

Original: Extensive experimental studies will be presented in Section 
Revised: Extensive experimental studies will be presented in Section 4.

Original: 3. C ONSTRUCTION OF C OARSE GRAPH
Revised: 3. C ONSTRUCTION OF C OARSE GRAPH

Original: Assume that we are already given the set of super-nodes (cid:98) V
Revised: Assume that we are already given the set of super-nodes (cid:98) V

Original: = { ˆ v 1 , . . .
Revised: = { ˆ v 1 , . . .

Original: , ˆ v n } for the coarse graph (cid:98) G together with the vertex map π : V → (cid:98) V – There has been much prior work on computing the sparsiﬁed set (cid:98) V ⊂ V and π (Loukas & Vandergheynst, 2018; Loukas, 2019); and if the vertex map π is not given, then we can simply deﬁne it by setting π ( v ) for each v ∈ V to be the nearest neighbor of v in (cid:98) V in terms of graph shortest path distance in G (Dey et al., 2013).
Revised: , ˆ v n } for the coarse graph (cid:98) G together with the vertex map π : V → (cid:98) V – There has been much prior work on computing the sparsiﬁed set (cid:98) V ⊂ V and π (Loukas & Vandergheynst, 2018; Loukas, 2019); and if the vertex map π is not given, then we can simply deﬁne it by setting π ( v ) for each v ∈ V to be the nearest neighbor of v in (cid:98) V in terms of graph shortest path distance in G (Dey et al., 2013).

Original: To construct edges for the coarse graph (cid:98) G = ( (cid:98) V , (cid:98) E ) together with the edge weight function ˆ w : (cid:98) E →
Revised: To construct edges for the coarse graph (cid:98) G = ( (cid:98) V , (cid:98) E ) together with the edge weight function ˆ w : (cid:98) E →

Original: R , instead of using a complete weighted graph over (cid:98) V , which is too dense and expensive, we set (cid:98) E to be those edges “induced” from G when collapsing each cluster π − 1 (ˆ v ) to its corresponding supernode ˆ v ∈ (cid:98) V :
Revised: R , instead of using a complete weighted graph over (cid:98) V , which is too dense and expensive, we set (cid:98) E to be those edges “induced” from G when collapsing each cluster π − 1 (ˆ v ) to its corresponding supernode ˆ v ∈ (cid:98) V :

Original: Speciﬁcally, (ˆ v, ˆ v (cid:48) )
Revised: Speciﬁcally, (ˆ v, ˆ v (cid:48) )

Original: ∈ (cid:98) E
Revised: ∈ (cid:98) E

Original: if and only if there is an edge ( v, v (cid:48) )
Revised: if and only if there is an edge ( v, v (cid:48) )

Original: ∈ E such that π ( v ) =
Revised: ∈ E such that π ( v ) =

Original: ˆ v and π ( v (cid:48) )
Revised: ˆ v and π ( v (cid:48) )

Original: = ˆ v (cid:48) .
Revised: = ˆ v (cid:48) .

Original: The weight of this edge is ˆ w (ˆ v, ˆ v (cid:48) ) :
Revised: The weight of this edge is ˆ w (ˆ v, ˆ v (cid:48) ) :

Original: = (cid:80) ( v,v (cid:48) )
Revised: = (cid:80) ( v,v (cid:48) )

Original: ∈ E (cid:0) π − 1 (ˆ v ) ,π − 1 (ˆ v (cid:48) ) (cid:1) w ( v, v (cid:48) )
Revised: ∈ E (cid:0) π − 1 (ˆ v ) ,π − 1 (ˆ v (cid:48) ) (cid:1) w ( v, v (cid:48) )

Original: where
Revised: where

Original: E ( A, B ) ⊆
Revised: E ( A, B ) ⊆

Original: E stands for the set of edges crossing sets A, B ⊆ V ; i.e., ˆ w (ˆ v, ˆ v (cid:48) ) is the total weights of all crossing edges in G between clusters π − 1 (ˆ v ) and π − 1 (ˆ v (cid:48) ) in V .
Revised: E stands for the set of edges crossing sets A, B ⊆ V ; i.e., ˆ w (ˆ v, ˆ v (cid:48) ) is the total weights of all crossing edges in G between clusters π − 1 (ˆ v ) and π − 1 (ˆ v (cid:48) ) in V .

Original: We refer to (cid:98) G constructed this way the (cid:98) V -induced coarse graph .
Revised: We refer to (cid:98) G constructed this way the (cid:98) V -induced coarse graph .

Original: As shown in Dey et al.
Revised: As shown in Dey et al.

Original: (2013), if the original graph G is the 1 -skeleton of a hidden space X , then this induced graph captures the topological of X at a coarser level if (cid:98) V is a so-called δ -net of the original vertex set V w.r.t.
Revised: (2013), if the original graph G is the 1 -skeleton of a hidden space X , then this induced graph captures the topological of X at a coarser level if (cid:98) V is a so-called δ -net of the original vertex set V w.r.t.

Original: the graph shortest path metric.
Revised: the graph shortest path metric.

Original: Let (cid:99) W be the edge weight matrix, and (cid:98) D be the diagonal matrix encoding the sum of edge weights incident to each vertex as before.
Revised: Let (cid:99) W be the edge weight matrix, and (cid:98) D be the diagonal matrix encoding the sum of edge weights incident to each vertex as before.

Original: Then the standard combinatorial Laplace operator w.r.t.
Revised: Then the standard combinatorial Laplace operator w.r.t.

Original: (cid:98) G is simply (cid:98) L
Revised: (cid:98) G is simply (cid:98) L

Original: = (cid:98) D − (cid:99) W .
Revised: = (cid:98) D − (cid:99) W .

Original: Relation to the operator of (Loukas, 2019).
Revised: Relation to the operator of (Loukas, 2019).

Original: Interestingly, this construction of the coarse graph (cid:98) G coincides with the coarse Laplace operator for a sparsiﬁed vertex set (cid:98) V constructed by Loukas (2019).
Revised: Interestingly, this construction of the coarse graph (cid:98) G coincides with the coarse Laplace operator for a sparsiﬁed vertex set (cid:98) V constructed by Loukas (2019).

Original: We will use this view of the Laplace operator later; hence we brieﬂy introduce the construction of Loukas (2019) (adapted to our setting):
Revised: We will use this view of the Laplace operator later; hence we brieﬂy introduce the construction of Loukas (2019) (adapted to our setting):

Original: Given the vertex map π : V → (cid:98) V , we set a n × N matrix P by P [ r, i ] = (cid:26) 1 | π − 1 (ˆ v r ) | if v i ∈ π − 1 (ˆ v r ) otherwise .
Revised: Given the vertex map π : V → (cid:98) V , we set a n × N matrix P by P [ r, i ] = (cid:26) 1 | π − 1 (ˆ v r ) | if v i ∈ π − 1 (ˆ v r ) otherwise .

Original: In what follows, we denote γ r := (cid:12)(cid:12) π − 1 (ˆ v r ) (cid:12)(cid:12) for any r ∈ [1 , n ] , which is the size of the cluster of ˆ v r in V .
Revised: In what follows, we denote γ r := (cid:12)(cid:12) π − 1 (ˆ v r ) (cid:12)(cid:12) for any r ∈ [1 , n ] , which is the size of the cluster of ˆ v r in V .

Original: P can be considered as the weighted projection matrix of the vertex set from V to (cid:98) V .
Revised: P can be considered as the weighted projection matrix of the vertex set from V to (cid:98) V .

Original: Let P + denote the Moore-Penrose pseudoinverse of P , which can be intuitively viewed as a way to lift a function on (cid:98) V (a vector in R n ) to a function over V (a vector in R N ).
Revised: Let P + denote the Moore-Penrose pseudoinverse of P , which can be intuitively viewed as a way to lift a function on (cid:98) V (a vector in R n ) to a function over V (a vector in R N ).

Original: As shown in Loukas (2019), P + is the N × n matrix
Revised: As shown in Loukas (2019), P + is the N × n matrix

Original: where P + [ i, r ] =
Revised: where P + [ i, r ] =

Original: if and only if π ( v i ) =
Revised: if and only if π ( v i ) =

Original: ˆ v r .
Revised: ˆ v r .

Original: See Appendix A.2 for a toy example.
Revised: See Appendix A.2 for a toy example.

Original: Finally, Loukas (2019) deﬁnes an operator for the coarsened vertex set (cid:98) V to be ˜ L (cid:98) V
Revised: Finally, Loukas (2019) deﬁnes an operator for the coarsened vertex set (cid:98) V to be ˜ L (cid:98) V

Original: =
Revised: =

Original: ( P + ) T LP + .
Revised: ( P + ) T LP + .

Original: Intuitively, (cid:98) L operators on n -vectors.
Revised: Intuitively, (cid:98) L operators on n -vectors.

Original: For any n -vector ˆ f ∈ R n , ˜ L (cid:98) V ˆ f ﬁrst lifts ˆ f to a N -vector f = P + ˆ f , and then perform L on f , and then project it down to n -dimensional via ( P + ) T .
Revised: For any n -vector ˆ f ∈ R n , ˜ L (cid:98) V ˆ f ﬁrst lifts ˆ f to a N -vector f = P + ˆ f , and then perform L on f , and then project it down to n -dimensional via ( P + ) T .

Original: Proposition 3.1.
Revised: Proposition 3.1.

Original: (Loukas, 2019)
Revised: (Loukas, 2019)

Original: The combinatorial graph Laplace operator (cid:98) L = (cid:98) D − (cid:99) W for the (cid:98) V -induced coarse graph (cid:98) G constructed above equals to the operator ˜ L (cid:98) V =
Revised: The combinatorial graph Laplace operator (cid:98) L = (cid:98) D − (cid:99) W for the (cid:98) V -induced coarse graph (cid:98) G constructed above equals to the operator ˜ L (cid:98) V =

Original: ( P + ) T LP + .
Revised: ( P + ) T LP + .

Original: 3. L APLACE OPERATOR FOR THE COARSE GRAPH
Revised: L APLACE OPERATOR FOR THE COARSE GRAPH

Original: We now have an input graph G =
Revised: We now have an input graph G =

Original: ( V, E ) and a coarse graph (cid:98) G induced from the sparsiﬁed node set (cid:98) V , and we wish to compare their corresponding Laplace operators.
Revised: ( V, E ) and a coarse graph (cid:98) G induced from the sparsiﬁed node set (cid:98) V , and we wish to compare their corresponding Laplace operators.

Original: However, as O G operates on R N (i.e, functions on the vertex set V of G ) and O (cid:98) G operates on R n , we will compare them by their effects on “corresponding” objects.
Revised: However, as O G operates on R N (i.e, functions on the vertex set V of G ) and O (cid:98) G operates on R n , we will compare them by their effects on “corresponding” objects.

Original: Loukas & Vandergheynst (2018); Loukas (2019) proposed to use the quadratic form to measure the similarity between the two linear operators.
Revised: Loukas & Vandergheynst (2018); Loukas (2019) proposed to use the quadratic form to measure the similarity between the two linear operators.

Original: In particular, given a linear operator A on R N and any x ∈ R N , Q A ( x ) = x T Ax .
Revised: In particular, given a linear operator A on R N and any x ∈ R N , Q A ( x ) = x T Ax .

Original: The quadratic form has also been used for measuring spectral approximation under edge sparsiﬁcation.
Revised: The quadratic form has also been used for measuring spectral approximation under edge sparsiﬁcation.

Original: The proof of the following result is in Appendix A.2.
Revised: The proof of the following result is in Appendix A.2.

Original: Proposition 3.2.
Revised: Proposition 3.2.

Original: For any vector ˆ x ∈ R n , we have that Q (cid:98) L (ˆ x ) =
Revised: For any vector ˆ x ∈ R n , we have that Q (cid:98) L (ˆ x ) =

Original: Q L ( P + ˆ x ) , where (cid:98) L is the combinatorial Laplace operator for the (cid:98) V -induced coarse graph (cid:98) G constructed above.
Revised: Q L ( P + ˆ x ) , where (cid:98) L is the combinatorial Laplace operator for the (cid:98) V -induced coarse graph (cid:98) G constructed above.

Original: That is, set x := P + ˆ x as the lift of ˆ x in R N , then ˆ x T (cid:98)
Revised: That is, set x := P + ˆ x as the lift of ˆ x in R N , then ˆ x T (cid:98)

Original: L ˆ x = x T Lx .
Revised: L ˆ x = x T Lx .

Original: Intuitively, this suggests that if later, we measure the similarity between L and some Laplace operator for the coarse graph (cid:98) G based on a loss from quadratic form difference, then we should choose the
Revised: Intuitively, this suggests that if later, we measure the similarity between L and some Laplace operator for the coarse graph (cid:98) G based on a loss from quadratic form difference, then we should choose the

Original: Laplace operator O (cid:98) G to be (cid:98) L and compare
Revised: Laplace operator O (cid:98) G to be (cid:98) L and compare

Original: Q (cid:98) L ( Px ) with Q L ( x ) .
Revised: Q (cid:98) L ( Px ) with Q L ( x ) .

Original: We further formalize this by considering the lifting map U : R n → R N as well as a projection map P : R N → R n , where
Revised: We further formalize this by considering the lifting map U : R n → R N as well as a projection map P : R N → R n , where

Original: P · U = Id n .
Revised: P · U = Id n .

Original: Proposition 3.2 suggests that for quadratic form-based similarity, the choices are
Revised: Proposition 3.2 suggests that for quadratic form-based similarity, the choices are

Original: U =
Revised: U =

Original: P + , P = P , and O (cid:98)
Revised: P + , P = P , and O (cid:98)

Original: G = (cid:98) L .
Revised: G = (cid:98) L .

Original: See the ﬁrst row in Table 1.
Revised: See the ﬁrst row in Table 1.

Original: [Table]
Revised: [Table]

Original: On the other hand, eigenvectors and eigenvalues of a linear operator A are more directly related, via Courant-Fischer Min-Max Theorem, to its Rayleigh quotient R A ( x ) =
Revised: On the other hand, eigenvectors and eigenvalues of a linear operator A are more directly related, via Courant-Fischer Min-Max Theorem, to its Rayleigh quotient R A ( x ) =

Original: x T Ax x T x .
Revised: x T Ax x T x .

Original: Interestingly, in this case, to preserve the Rayleigh quotient, we should change the choice of O (cid:98) G to be the following doubly-weighted Laplace operator for a graph that is both edge and vertex weighted.
Revised: Interestingly, in this case, to preserve the Rayleigh quotient, we should change the choice of O (cid:98) G to be the following doubly-weighted Laplace operator for a graph that is both edge and vertex weighted.

Original: Speciﬁcally, for the coarse graph (cid:98) G , we assume that each vertex ˆ v ∈ (cid:98) V is weighted by γ ˆ v :=
Revised: Speciﬁcally, for the coarse graph (cid:98) G , we assume that each vertex ˆ v ∈ (cid:98) V is weighted by γ ˆ v :=

Original: | π − 1 (ˆ v ) | , the size of the cluster from G that got collapsed into ˆ v .
Revised: | π − 1 (ˆ v ) | , the size of the cluster from G that got collapsed into ˆ v .

Original: Let Γ be the vertex matrix, which is the n × n diagonal matrix with Γ[ r ][ r ] = γ ˆ v r .
Revised: Let Γ be the vertex matrix, which is the n × n diagonal matrix with Γ[ r ][ r ] = γ ˆ v r .

Original: The doubly-weighted Laplace operator for a vertexand edge-weighted graph (cid:98) G is then deﬁned as:
Revised: The doubly-weighted Laplace operator for a vertexand edge-weighted graph (cid:98) G is then deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: The concept of doubly-weighted Laplace for a vertex- and edge-weighted graph is not new, see e.g
Revised: The concept of doubly-weighted Laplace for a vertex- and edge-weighted graph is not new, see e.g

Original: Chung & Langlands (1996); Horak & Jost (2013);
Revised: Chung & Langlands (1996); Horak & Jost (2013);

Original: Xu et al.
Revised: Xu et al.

Original: In particular, Horak & Jost (2013) proposes a general form of combinatorial Laplace operator for a simplicial complex where all simplices are weighted, and our doubly-weighted Laplace has the same eigenstructure as their
Revised: In particular, Horak & Jost (2013) proposes a general form of combinatorial Laplace operator for a simplicial complex where all simplices are weighted, and our doubly-weighted Laplace has the same eigenstructure as their

Original: Laplacian when restricted to graphs.
Revised: Laplacian when restricted to graphs.

Original: See Appendix A.1 for details.
Revised: See Appendix A.1 for details.

Original: Using the doubly-weighted
Revised: Using the doubly-weighted

Original: Laplacian for Rayleigh quotient based similarity measurement between the original graph and the coarse graph is justiﬁed by the following result (proof in Appendix A.1).
Revised: Laplacian for Rayleigh quotient based similarity measurement between the original graph and the coarse graph is justiﬁed by the following result (proof in Appendix A.1).

Original: Proposition 3.3.
Revised: Proposition 3.3.

Original: For any vector x ∈ R n , we have that R (cid:98) L (ˆ x ) =
Revised: For any vector x ∈ R n , we have that R (cid:98) L (ˆ x ) =

Original: R L ( P + Γ [Equation] [Equation] 2 ˆ x ) .
Revised: R L ( P + Γ − 1 / 2 ˆ x ) .

Original: That is, set the lift of ˆ x in R N to be x = P + Γ − 1 / 2 ˆ x , then we have that 
Revised: That is, set the lift of ˆ x in R N to be x = P + Γ − 1 / 2 ˆ x , then we have that ˆ

Original: [Equation]
Revised: [Equation]

Original: Finally, if using the normalized Laplace L for the original graph G , then the appropriate Laplace operator for the coarse graph and corresponding projection/lift maps are listed in the last row of
Revised: Finally, if using the normalized Laplace L for the original graph G , then the appropriate Laplace operator for the coarse graph and corresponding projection/lift maps are listed in the last row of

Original: Table 1, with proofs in Appendix A.2.
Revised: Table 1, with proofs in Appendix A.2.

Original: 3. A GNN- BASED FRAMEWORK FOR LEARNING FOR CONSTRUCTING THE COARSE GRAPH
Revised: 3. A GNN- BASED FRAMEWORK FOR LEARNING FOR CONSTRUCTING THE COARSE GRAPH

Original: [Figure]
Revised: [Figure]

Original: In the previous section, we argued that depending on what similarity measures we use, appropriate
Revised: In the previous section, we argued that depending on what similarity measures we use, appropriate

Original: Laplace operator O (cid:98) G for the coarse graph (cid:98) G should be used.
Revised: Laplace operator O (cid:98) G for the coarse graph (cid:98) G should be used.

Original: Now consider the speciﬁc case of
Revised: Now consider the speciﬁc case of

Original: Rayleigh quotient, which can be thought of as a proxy to measure similarities between the lowfrequency eigenvalues of the original graph Laplacian and the one for the coarse graph.
Revised: Rayleigh quotient, which can be thought of as a proxy to measure similarities between the lowfrequency eigenvalues of the original graph Laplacian and the one for the coarse graph.

Original: As described above, here we set O (cid:98) G as the doubly-weighted Laplacian (cid:98) L 
Revised: As described above, here we set O (cid:98) G as the doubly-weighted Laplacian (cid:98) L =

Original: =
Revised: [Equation]

Original: Γ − 1 / 2 ( (cid:98) D − (cid:99) W )Γ − 1 / 2 . 0 250 500 750 1000 1250    minnesota G.eGc.eAfter Opt
Revised: [Figure]

Original: The effect of weight adjustments.
Revised: The effect of weight adjustments.

Original: We develop an iterative algorithm with convergence guarantee (to KKT point in F.4) for optimizing over edge weights of (cid:98) G for better spectrum alignment.
Revised: We develop an iterative algorithm with convergence guarantee (to KKT point in F.4) for optimizing over edge weights of (cid:98) G for better spectrum alignment.

Original: As shown in the ﬁgure on the right, after changing the edge weight of the coarse graph, the resulting graph Laplacian has eigenvalues much closer (almost identical) to the ﬁrst n eigenvalues of the original graph Laplacian.
Revised: As shown in the ﬁgure on the right, after changing the edge weight of the coarse graph, the resulting graph Laplacian has eigenvalues much closer (almost identical) to the ﬁrst n eigenvalues of the original graph Laplacian.

Original: More speciﬁcally, in this ﬁgure, G.e and Gc.e stand for the eigenvalues of the original graph G and coarse graph (cid:98) G constructed by the so-called Variation-Edge coarsening algorithm (Loukas, 2019).
Revised: More speciﬁcally, in this ﬁgure, G.e and Gc.e stand for the eigenvalues of the original graph G and coarse graph (cid:98) G constructed by the so-called Variation-Edge coarsening algorithm (Loukas, 2019).

Original: “AfterOpt” stands for the eigenvalues of coarse graphs when weights are optimized by our iterative algorithm.
Revised: “AfterOpt” stands for the eigenvalues of coarse graphs when weights are optimized by our iterative algorithm.

Original: See Appendix F for the description of our iterative algorithm, its convergence results, and full experiment results.
Revised: See Appendix F for the description of our iterative algorithm, its convergence results, and full experiment results.

Original: A GNN-based framework for learning weight assignment map.
Revised: A GNN-based framework for learning weight assignment map.

Original: The discussions above indicate that we can obtain better Laplace operators for the coarse graph by using better-informed weights than simply summing up the weights of crossing edges from the two clusters.
Revised: The discussions above indicate that we can obtain better Laplace operators for the coarse graph by using better-informed weights than simply summing up the weights of crossing edges from the two clusters.

Original: More speciﬁcally, suppose we have a ﬁxed strategy to generate (cid:98) V from an input graph G =
Revised: More speciﬁcally, suppose we have a ﬁxed strategy to generate (cid:98) V from an input graph G =

Original: ( V, E ) .
Revised: ( V, E ) .

Original: Now given an edge (ˆ v, ˆ v (cid:48) ) ∈ (cid:98) E in the induced coarse graph (cid:98) G =
Revised: Now given an edge (ˆ v, ˆ v (cid:48) ) ∈ (cid:98) E in the induced coarse graph (cid:98) G =

Original: ( (cid:98) V , (cid:98) E ) , we model its weight ˆ w (ˆ v, ˆ v (cid:48) ) by a weight-assignment function µ ( G | π − 1 (ˆ v ) ∪ π − 1 (ˆ v (cid:48) ) ) , where G | A is the subgraph of G induced by a subset of vertices A .
Revised: ( (cid:98) V , (cid:98) E ) , we model its weight ˆ w (ˆ v, ˆ v (cid:48) ) by a weight-assignment function µ ( G | π − 1 (ˆ v ) ∪ π − 1 (ˆ v (cid:48) ) ) , where G | A is the subgraph of G induced by a subset of vertices A .

Original: However, it is not clear how to setup this function µ .
Revised: However, it is not clear how to setup this function µ .

Original: Instead, we will learn it from a collection of input graphs in an unsupervised manner.
Revised: Instead, we will learn it from a collection of input graphs in an unsupervised manner.

Original: Speciﬁcally, we will parametrize the weight-assignment map µ by a learnable neural network M θ .
Revised: Speciﬁcally, we will parametrize the weight-assignment map µ by a learnable neural network M θ .

Original: See Figure 1 for an illustration.
Revised: See Figure 1 for an illustration.

Original: In particular, we use Graph Isomorphism Network (GIN)
Revised: In particular, we use Graph Isomorphism Network (GIN)

Original: (Xu et al., 2018) to represent M θ .
Revised: (Xu et al., 2018) to represent M θ .

Original: We initialize the model by setting the edge attribute of the coarse graph to be 1.
Revised: We initialize the model by setting the edge attribute of the coarse graph to be 1.

Original: Our node feature is set to be a 5-dimensional vector based on LDP (Local Degree Proﬁle)
Revised: Our node feature is set to be a 5-dimensional vector based on LDP (Local Degree Proﬁle)

Original: (Cai & Wang, 2018).
Revised: (Cai & Wang, 2018).

Original: We enforce the learned weight of the coarse graph to be positive by applying one extra ReLU layer to the ﬁnal output.
Revised: We enforce the learned weight of the coarse graph to be positive by applying one extra ReLU layer to the ﬁnal output.

Original: All models are trained with Adam optimizer with a learning rate of 0.001.
Revised: All models are trained with Adam optimizer with a learning rate of 0.001.

Original: See Appendix E for more details.
Revised: See Appendix E for more details.

Original: We name our model as G raph c
Revised: We name our model as G raph c

Original: O arsening R eﬁnem E nt N etwork ( GOREN ).
Revised: O arsening R eﬁnem E nt N etwork ( GOREN ).

Original: Given a graph G and a coarsening algorithm A , the general form of loss is
Revised: Given a graph G and a coarsening algorithm A , the general form of loss is

Original: [Equation]
Revised: [Equation]

Original: where f i is signal on the original graph (such as eigenvectors) and
Revised: where f i is signal on the original graph (such as eigenvectors) and

Original: P f
Revised: P f

Original: i is its projection.
Revised: i is its projection.

Original: We use
Revised: We use

Original: O (cid:99) G t to denote the operator of the coarse graph during training , while O (cid:98) G standing for the operator deﬁned w.r.t.
Revised: O (cid:99) G t to denote the operator of the coarse graph during training , while O (cid:98) G standing for the operator deﬁned w.r.t.

Original: the coarse graph output by coarsening algorithm A .
Revised: the coarse graph output by coarsening algorithm A .

Original: That is, we will start with O (cid:98) G and modify it to O (cid:99) G t during the training.
Revised: That is, we will start with O (cid:98) G and modify it to O (cid:99) G t during the training.

Original: The loss can be instantiated for different cases in Table 1.
Revised: The loss can be instantiated for different cases in Table 1.

Original: For example, a loss based on quadratic form means that we choose O G , O (cid:99) G t to be the combinatorial
Revised: For example, a loss based on quadratic form means that we choose O G , O (cid:99) G t to be the combinatorial

Original: Laplacian of G and (cid:99) G t , and the resulting quadratic loss has the form:
Revised: Laplacian of G and (cid:99) G t , and the resulting quadratic loss has the form:

Original: [Equation]
Revised: [Equation]

Original: It can be seen as a natural analog of the loss for spectral sparsiﬁcation in the context of graph coarsening, which is also adopted in Loukas (2019).
Revised: It can be seen as a natural analog of the loss for spectral sparsiﬁcation in the context of graph coarsening, which is also adopted in Loukas (2019).

Original: Similarly, one can use a loss based on the Rayleigh quotient, by choosing F from the second row of Table 1.
Revised: Similarly, one can use a loss based on the Rayleigh quotient, by choosing F from the second row of Table 1.

Original: Our framework for graph coarsening is ﬂexible.
Revised: Our framework for graph coarsening is ﬂexible.

Original: Many different loss functions can be used as long as it is differentiable in the weights of the coarse graph.
Revised: Many different loss functions can be used as long as it is differentiable in the weights of the coarse graph.

Original: we will demonstrate this point in Section 4.4.
Revised: we will demonstrate this point in Section 4.4.

Original: Finally, given a collection of training graphs G 1 , . . .
Revised: Finally, given a collection of training graphs G 1 , . . .

Original: , G m , we will train for parameters in the module
Revised: , G m , we will train for parameters in the module

Original: M θ to minimize the total loss on training graphs.
Revised: M θ to minimize the total loss on training graphs.

Original: When a test graph G test is given, we simply apply M θ to set up weight for each edge in (cid:91) G test , obtaining a new graph (cid:92) G test,t .
Revised: When a test graph G test is given, we simply apply M θ to set up weight for each edge in (cid:91) G test , obtaining a new graph (cid:92) G test,t .

Original: We compare
Revised: We compare

Original: Loss ( O G test , O (cid:92) G test,t ) against Loss ( O G test , O (cid:92) G test ) and expect the former loss is smaller.
Revised: Loss ( O G test , O (cid:92) G test,t ) against Loss ( O G test , O (cid:92) G test ) and expect the former loss is smaller.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: In the following experiments, we apply six existing coarsening algorithms to obtain the coarsened vertex set (cid:98) V , which are Afﬁnity (Livne & Brandt, 2012), Algebraic Distance (Chen & Safro, 2011),
Revised: In the following experiments, we apply six existing coarsening algorithms to obtain the coarsened vertex set (cid:98) V , which are Afﬁnity (Livne & Brandt, 2012), Algebraic Distance (Chen & Safro, 2011),

Original: Heavy edge matching (Dhillon et al., 2007; Ron et al., 2011), as well as two local variation methods based on edge and neighborhood respectively (Loukas, 2019), and a simple baseline (BL); See
Revised: Heavy edge matching (Dhillon et al., 2007; Ron et al., 2011), as well as two local variation methods based on edge and neighborhood respectively (Loukas, 2019), and a simple baseline (BL); See

Original: Appendix D for detailed descriptions.
Revised: Appendix D for detailed descriptions.

Original: The two local variation methods are considered to be stateof-the-art graph coarsening algorithms Loukas (2019).
Revised: The two local variation methods are considered to be stateof-the-art graph coarsening algorithms Loukas (2019).

Original: We show that our GOREN framework can improve the qualities of coarse graphs produced by these methods.
Revised: We show that our GOREN framework can improve the qualities of coarse graphs produced by these methods.

Original: 4. P ROOF OF C ONCEPT
Revised: 4. P ROOF OF C ONCEPT

Original: [Table]
Revised: [Table]

Original: As proof of concept, we show that GOREN can improve common coarsening methods on multiple graphs (see C.2 for details).
Revised: As proof of concept, we show that GOREN can improve common coarsening methods on multiple graphs (see C.2 for details).

Original: Following the same setting as Loukas (2019), we use the relative eigenvalue error as evaluation metric.
Revised: Following the same setting as Loukas (2019), we use the relative eigenvalue error as evaluation metric.

Original: It is deﬁned as 1 k (cid:80) ki =1 | (cid:98) λ i − λ
Revised: It is deﬁned as 1 k (cid:80) ki =1 | (cid:98) λ i − λ

Original: i | λ i , where λ i , (cid:98) λ i denotes eigenvalues of combinatorial Laplacian L for G and doubly-weighted Laplacian (cid:98) L for (cid:98) G respectively, and k is set to be 40.
Revised: i | λ i , where λ i , (cid:98) λ i denotes eigenvalues of combinatorial Laplacian L for G and doubly-weighted Laplacian (cid:98) L for (cid:98) G respectively, and k is set to be 40.

Original: For simplicity,  this error is denoted as Eigenerror in the remainder of the paper.
Revised: For simplicity, [Table] this error is denoted as Eigenerror in the remainder of the paper.

Original: Denote the Eigenerror of graph coarsening method as l 1 and Eigenerror obtained by GOREN as l 2 .
Revised: Denote the Eigenerror of graph coarsening method as l 1 and Eigenerror obtained by GOREN as l 2 .

Original: In Table 2, we show the errorreduction ratio , deﬁned as l 1 − l 2 l 1 .
Revised: In Table 2, we show the errorreduction ratio , deﬁned as l 1 − l 2 l 1 .

Original: The ratio is upper bounded by 100% in the case of improvement (and the larger the value is, the better); but it is not lower bounded.
Revised: The ratio is upper bounded by 100% in the case of improvement (and the larger the value is, the better); but it is not lower bounded.

Original: Since it is hard to directly optimize Eigenerror, the loss function we use in our GOREN set to be the Rayleigh loss Loss ( O G , O (cid:99) G t )
Revised: Since it is hard to directly optimize Eigenerror, the loss function we use in our GOREN set to be the Rayleigh loss Loss ( O G , O (cid:99) G t )

Original: = 1 k (cid:80) ki =1 |F
Revised: = 1 k (cid:80) ki =1 |F

Original: ( O G , f i ) − F
Revised: ( O G , f i ) − F

Original: ( O (cid:99) G t , P f i ) | where F is Rayleigh quotient, P =
Revised: ( O (cid:99) G t , P f i ) | where F is Rayleigh quotient, P =

Original: Γ − 1 / 2 ( P + ) T and O (cid:98) G t being doubly-weighted Laplacian (cid:98) L t .
Revised: Γ − 1 / 2 ( P + ) T and O (cid:98) G t being doubly-weighted Laplacian (cid:98) L t .

Original: In other words, We[Table] use Rayleigh loss as a differentiable proxy for the Eigenerror.
Revised: In other words, We use Rayleigh loss as a differentiable proxy for the Eigenerror.

Original: As we can see in Table 2, GOREN reduces the Eigenerror by a large margin for training graphs, which serves as a sanity check for our framework, as well as for using Rayleigh loss as a proxy for Eigenerror.
Revised: As we can see in Table 2, GOREN reduces the Eigenerror by a large margin for training graphs, which serves as a sanity check for our framework, as well as for using Rayleigh loss as a proxy for Eigenerror.

Original: Due to space limit, see
Revised: Due to space limit, see

Original: Table G.1 for full results where we reproduce the results in Loukas (2019) up to small differences.
Revised: Table G.1 for full results where we reproduce the results in Loukas (2019) up to small differences.

Original: In Table 5, we will demonstrate this training strategy also generalizes well to unseen graphs.
Revised: In Table 5, we will demonstrate this training strategy also generalizes well to unseen graphs.

Original: 4. S YNTHETIC G RAPHS
Revised: 4. S YNTHETIC G RAPHS

Original: We train the GOREN on synthetic graphs from common graph generative models and test on larger unseen graphs from the same model.
Revised: We train the GOREN on synthetic graphs from common graph generative models and test on larger unseen graphs from the same model.

Original: We randomly sample 25 graphs of size { 512 , 612 , 712 , ..., 2912 } from different generative models.
Revised: We randomly sample 25 graphs of size { 512 , 612 , 712 , ..., 2912 } from different generative models.

Original: If the graph is disconnected, we keep the largest component.
Revised: If the graph is disconnected, we keep the largest component.

Original: We train GOREN on the ﬁrst 5 graphs, use the 5 graphs from the restgraphs as the validation set and the remaining 15 as test graphs.
Revised: We train GOREN on the ﬁrst 5 graphs, use the 5 graphs from the restgraphs as the validation set and the remaining 15 as test graphs.

Original: We use the following synthetic graphs: Erd ˝ os–R ´ enyi graphs (ER), Barabasi-Albert Graph (BA), Watts-Strogatz Graph (WS), random geometric graphs (GEO).
Revised: We use the following synthetic graphs: Erd ˝ os–R ´ enyi graphs (ER), Barabasi-Albert Graph (BA), Watts-Strogatz Graph (WS), random geometric graphs (GEO).

Original: See Appendix C.1 for datasets details.
Revised: See Appendix C.1 for datasets details.

Original: For simplicity, we only report experiment results for the reduction ratio 0.5.
Revised: For simplicity, we only report experiment results for the reduction ratio 0.5.

Original: For complete results of all reduction ratios (0.3, 0.5, 0.7), see Appendix G.
Revised: For complete results of all reduction ratios (0.3, 0.5, 0.7), see Appendix G.

Original: We report both the loss Loss ( L, (cid:98) L ) of different algorithms (w/o learning) and the relative improvement percentage deﬁned as Loss ( L, (cid:98) L ) − Loss ( L, (cid:99) [Table] when GOREN is applied, shown in parenthesis.
Revised: We report both the loss Loss ( L, (cid:98) L ) of different algorithms (w/o learning) and the relative improvement percentage deﬁned as Loss ( L, (cid:98) L ) − Loss ( L, (cid:99) L t ) Loss ( L, (cid:98) L ) when GOREN is applied, shown in parenthesis.

Original: As we can see in Table 3, for most methods, trained on small graphs, GOREN also performs well on test graphs of larger size across different algorithms and datasets – Again, the larger improvement percentage is, the larger the improvement by our algorithm is, and a negative value means that our algorithm makes the loss worse.
Revised: As we can see in Table 3, for most methods, trained on small graphs, GOREN also performs well on test graphs of larger size across different algorithms and datasets – Again, the larger improvement percentage is, the larger the improvement by our algorithm is, and a negative value means that our algorithm makes the loss worse.

Original: Note the size of test graphs are on average 2 .
Revised: Note the size of test graphs are on average 2 .

Original: 6 × the size of training graphs.
Revised: 6 × the size of training graphs.

Original: For ER and BA graphs, the improvement is relatively smaller compared to GEO and WS graphs.
Revised: For ER and BA graphs, the improvement is relatively smaller compared to GEO and WS graphs.

Original: This makes sense since ER and BA graphs are rather homogenous graphs, leaving less room for further improvement.
Revised: This makes sense since ER and BA graphs are rather homogenous graphs, leaving less room for further improvement.

Original: 4. R EAL N ETWORKS
Revised: 4. R EAL N ETWORKS

Original: We test on ﬁve real networks: Shape, PubMed, Coauthor-CS (CS), Coauthor-Physics (Physics), and
Revised: We test on ﬁve real networks: Shape, PubMed, Coauthor-CS (CS), Coauthor-Physics (Physics), and

Original: Flickr (largest one with 89k vertices), which are much larger than datasets used in Hermsdorff &
Revised: Flickr (largest one with 89k vertices), which are much larger than datasets used in Hermsdorff &

Original: Gunderson (2019) ( ≤ 1.5k) and Loukas (2019) ( ≤ 4k).
Revised: Gunderson (2019) ( ≤ 1.5k) and Loukas (2019) ( ≤ 4k).

Original: Since it is hard to obtain multiple large graphs (except for the Shape dataset, which contains meshes from different surface models) coming from similar distribution, we bootstrap the training data in the following way.
Revised: Since it is hard to obtain multiple large graphs (except for the Shape dataset, which contains meshes from different surface models) coming from similar distribution, we bootstrap the training data in the following way.

Original: For the given graph, we randomly sample a collection of landmark vertices and take a random walk of length l starting from selected vertices.
Revised: For the given graph, we randomly sample a collection of landmark vertices and take a random walk of length l starting from selected vertices.

Original: We take subgraphs spanned by vertices of random walks as training and validation graphs and the original graph as the test graph.
Revised: We take subgraphs spanned by vertices of random walks as training and validation graphs and the original graph as the test graph.

Original: See Appendix C.3 for dataset details.
Revised: See Appendix C.3 for dataset details.

Original:  As shown in the bottom half of Table 3, across all six different algorithms, GOREN signiﬁcantly improves the result among all ﬁve datasets in most cases.
Revised: [Table] [Table] As shown in the bottom half of Table 3, across all six different algorithms, GOREN signiﬁcantly improves the result among all ﬁve datasets in most cases.

Original: For the largest graph Flickr, the size of test graphs is more than 25 × of the training graphs, which further demonstrates the strong generalization.
Revised: For the largest graph Flickr, the size of test graphs is more than 25 × of the training graphs, which further demonstrates the strong generalization.

Original: 4. O THER LOSSES
Revised: 4. O THER LOSSES

Original: Other differentiable loss.
Revised: Other differentiable loss.

Original: To demonstrate that our framework is ﬂexible, we adapt GOREN to the following two losses.
Revised: To demonstrate that our framework is ﬂexible, we adapt GOREN to the following two losses.

Original: The two losses are both differentiable w.r.t the weights of coarse graph.
Revised: The two losses are both differentiable w.r.t the weights of coarse graph.

Original: [Table]
Revised: ) Loss based

Original: [Table]
Revised: on normalized graph Laplacian:

Original: [Equation]
Revised: [Equation]

Original: Here { f i } are the set of ﬁrst k eigenvectors of the normalized Laplacian L of original grpah G , and P = (cid:98) D 1 / 2 PD − 1 / 2 .
Revised: Here { f i } are the set of ﬁrst k eigenvectors of the normalized Laplacian L of original grpah G , and P = (cid:98) D 1 / 2 PD − 1 / 2 .

Original: (2) Conductance difference  graph and coarse graph.
Revised: (2) Conductance difference between original graph and coarse graph.

Original: Loss = 1 k (cid:80) ki =1 | ϕ ( S i ) − ϕ
Revised: Loss = 1 k (cid:80) ki =1 | ϕ ( S i ) − ϕ

Original: ( π ( S i )) | .
Revised: ( π ( S i )) | .

Original: ϕ ( S ) is the conductance ϕ
Revised: ϕ ( S ) is the conductance ϕ

Original: ( S ) :
Revised: ( S ) :

Original: = (cid:80)

Original: [Equation] j [Equation] ¯ S a ij min( a ( S ) ,a (¯ S )) where a ( S ) :
Revised: = (cid:80) i ∈ S,j ∈ ¯ S a ij min( a ( S ) ,a (¯ S )) where a ( S ) :

Original: = (cid:80)
Revised: = (cid:80)

Original: i ∈ S (cid:80) j ∈ V a ij .
Revised: i ∈ S (cid:80) j ∈ V a ij .

Original: We randomly sample k subsets of nodes S 0 , ..., S k ⊂ V where | S i | is set to be a random number sampled from the uniform distribution U ( | V | / 4 , | V | / 2) .
Revised: We randomly sample k subsets of nodes S 0 , ..., S k ⊂ V where | S i | is set to be a random number sampled from the uniform distribution U ( | V | / 4 , | V | / 2) .

Original: Due to space limits, we present the result for conductance in Appendix G.3.
Revised: Due to space limits, we present the result for conductance in Appendix G.3.

Original: Following the same setting as before, we perform experiments to minimize two different losses.
Revised: Following the same setting as before, we perform experiments to minimize two different losses.

Original: As shown in Table 4 and Appendix G.3, for most graphs and methods, GOREN still shows good generalization capacity and improvement for both losses.
Revised: As shown in Table 4 and Appendix G.3, for most graphs and methods, GOREN still shows good generalization capacity and improvement for both losses.

Original: Apart from that, we also observe the initial loss for normalized Laplacian is much smaller than that for standard Laplacian, which might be due to that the fact that eigenvalues of normalized Laplacian are in [0 , 2] .
Revised: Apart from that, we also observe the initial loss for normalized Laplacian is much smaller than that for standard Laplacian, which might be due to that the fact that eigenvalues of normalized Laplacian are in [0 , 2] .

Original: Non-differentiable loss.
Revised: Non-differentiable loss.

Original: In Section 4.1, we use Rayleigh loss as a proxy for training but the Eigenerror for validation and test.
Revised: In Section 4.1, we use Rayleigh loss as a proxy for training but the Eigenerror for validation and test.

Original: Here we train GOREN with Rayleigh loss but evaluate Eigenerror on test graphs, which is more challenging.
Revised: Here we train GOREN with Rayleigh loss but evaluate Eigenerror on test graphs, which is more challenging.

Original: Number of vectors k is 40 for synthetic graphs and 200 for real networks.
Revised: Number of vectors k is 40 for synthetic graphs and 200 for real networks.

Original: As shown in Table 5, our training strategy via Rayleigh loss can improve the eigenvalue alignment between original graphs and coarse graphs in most cases.
Revised: As shown in Table 5, our training strategy via Rayleigh loss can improve the eigenvalue alignment between original graphs and coarse graphs in most cases.

Original: Reducing Eigenerror is more challenging than other losses, possibly because we are minimizing a differentiable proxy (the
Revised: Reducing Eigenerror is more challenging than other losses, possibly because we are minimizing a differentiable proxy (the

Original: Rayleigh loss).
Revised: Rayleigh loss).

Original: Nevertheless, improvement is achieved in most cases.
Revised: Nevertheless, improvement is achieved in most cases.

Original: C ONCLUSION
Revised: [Table] 4. O N THE USE OF GNN AS WEIGHT - ASSIGNMENT MAP . Recall that we use GNN to represent a edge-weight assignment map for an edge (ˆ u, ˆ v ) between two super-nodes ˆ u, ˆ v in the coarse graph (cid:98) G . The input will be the subgraph G ˆ u, ˆ v in the original graph G spanning the clusters π − 1 (ˆ u ) , π − 1 (ˆ v ) , and the crossing edges among them; while the goal is to compute the weight of edge (ˆ u, ˆ v ) based on this subgraph G ˆ u, ˆ v .

Revised: Given that the input is a local graph G ˆ u, ˆ v , a GNN will be a natural choice to parameterize this edge-weight assignment map.

Revised: Nevertheless, in principle, any architecture applicable to graph regression can be used for this purpose.

Revised: To better understand if it is necessary to use the power of GNN, we replace GNN with the following baseline for graph regression.

Revised: In particular, the baseline is a composition of mean pooling of node features in the original graph and a 4-layer MLP with embedding dimension 50 and ReLU nonlinearity.

Revised: We use mean-pooling as the graph regression component needs to be permutation invariant over the set of node features.

Revised: However, this baseline ignores the detailed graph structure which GNN will leverage.

Revised: The results for different reduction ratios are presented in the table 6.

Revised: We have also implemented another baseline where the MLP module is replaced by a simpler linear regression modue.

Revised: The results are worse than those of MLP (and thus also GNN) as expected, and therefore omitted from this paper.

Revised: As we can see, MLP works reasonably well in most cases, indicating that learning the edge weights is indeed useful for improvement.

Revised: On the other hand, we see using GNN to parametrize the map generally yields a larger improvement over the MLP, which ignores the topology of subgraphs in the original graph.

Revised: A systematic understanding of how different models such as various graph kernels (Kriege et al., 2020; Vishwanathan et al., 2010) and graph neural networks affect the performance is an interesting question that we will leave for future work.

Revised:  C ONCLUSION

Original: We present a framework to compare original graph and the coarse one via the properly chosen
Revised: We present a framework to compare original graph and the coarse one via the properly chosen

Original: Laplace operators and projection/lift map.
Revised: Laplace operators and projection/lift map.

Original: Observing the beneﬁts of optimizing over edge weights, we propose a GNN-based framework to learn the edge weights of coarse graph to further improve the existing coarsening algorithms.
Revised: Observing the beneﬁts of optimizing over edge weights, we propose a GNN-based framework to learn the edge weights of coarse graph to further improve the existing coarsening algorithms.

Original: Through extensive experiments, we demonstrate that our method
Revised: Through extensive experiments, we demonstrate that our method

Original: GOREN signiﬁcantly improves common graph coarsening methods under different metrics, reduction ratios, graph sizes, and graph types.
Revised: GOREN signiﬁcantly improves common graph coarsening methods under different metrics, reduction ratios, graph sizes, and graph types.

Revised: For future directions, ﬁrst, the topology of the coarse graph is currently determined by the coarsening algorithm.

Revised: It would be desirable to parametrize existing methods by neural networks so that the entire process can be trained end-to-end.

Revised: Second, extending to other losses (maybe non-differentiable) such as Maretic et al.

Revised: (2019) which involves inverse Laplacian remains interesting and challenging.

Revised: Third, as there is no consensus on what speciﬁc properties should be preserved, understanding the relationship between different metrics and the downstream task is important.

Original: A C HOICE OF L APLACE O PERATOR A. L APLACE O PERATOR ON WEIGHTED SIMPLICIAL COMPLEX
Revised: A C HOICE OF L APLACE O PERATOR A. L APLACE O PERATOR ON WEIGHTED SIMPLICIAL COMPLEX

Original: Its most general form in the discrete case, presented as the operators on weighted simplicial complexes, is:
Revised: Its most general form in the discrete case, presented as the operators on weighted simplicial complexes, is:

Original: [Equation]
Revised: [Equation]

Original: where B i is the matrix corresponding to the coboundary operator δ i , and W i is the diagonal matrix representing the weights of i -th dimensional simplices.
Revised: where B i is the matrix corresponding to the coboundary operator δ i , and W i is the diagonal matrix representing the weights of i -th dimensional simplices.

Original: See (Horak & Jost, 2013) for details.
Revised: See (Horak & Jost, 2013) for details.

Original: When restricted to the graph (1 simplicial complex), we recover the most common graph Laplacians as special case of L up 0 .
Revised: When restricted to the graph (1 simplicial complex), we recover the most common graph Laplacians as special case of L up 0 .

Original: Note that although the L upi and L downi is not symmetric, we can always symmetrize them by multiple a properly chosen diagonal matrix and its inverse from left and right without altering the spectrum.
Revised: Note that although the L upi and L downi is not symmetric, we can always symmetrize them by multiple a properly chosen diagonal matrix and its inverse from left and right without altering the spectrum.

Original: A. MISSING PROOFS
Revised: A. MISSING PROOFS

Original: Minimize
Revised: Minimize

Original: We provide the missing proofs regarding the properties of the projection/lift map and the resulting operators on the coarse graph.
Revised: We provide the missing proofs regarding the properties of the projection/lift map and the resulting operators on the coarse graph.

Original: Recall as an toy example, a coarsening algorithm will take graph on the left in ﬁgure A.2 and generate a coarse graph on the right, with coarsening matrix P =
Revised: Recall as an toy example, a coarsening algorithm will take graph on the left in ﬁgure A.2 and generate a coarse graph on the right, with coarsening matrix P =

Original: (cid:20) 1 / 3 1 / 3 1 /      1 / 3 1 / 3 1 /(cid:21) ,
Revised: (cid:20) 1 / 3 1 / 3 1 /      1 / 3 1 / 3 1 /(cid:21) ,

Original: P +1 =
Revised: P +1 =

Original:   111000  , Γ = (cid:20) 30(cid:21) , Π =   1 / 3 1 / 3 1 /   1 / 3 1 / 3 1 /   1 / 3 1 / 3 1 /      1 / 3 1 / 3 1 /   1 / 3 1 / 3 1 /   1 / 3 1 / 3 1 /
Revised:   111000  , Γ = (cid:20) 30(cid:21) , Π =   1 / 3 1 / 3 1 /   1 / 3 1 / 3 1 /   1 / 3 1 / 3 1 /      1 / 3 1 / 3 1 /   1 / 3 1 / 3 1 /   1 / 3 1 / 3 1 /

Original:  .
Revised:  .

Original: Π in general is a
Revised: Π in general is a

Original: N × N block matrix of rank n .
Revised: N × N block matrix of rank n .

Original: All entries in each block Π j is equal to 1 γ j where γ j = (cid:12)(cid:12) π − 1 (ˆ v j ) (cid:12)(cid:12) .
Revised: All entries in each block Π j is equal to 1 γ j where γ j = (cid:12)(cid:12) π − 1 (ˆ v j ) (cid:12)(cid:12) .

Original: [Table]
Revised: [Table]

Original: We ﬁrst make an observation about projection and lift operator, P and U .
Revised: We ﬁrst make an observation about projection and lift operator, P and U .

Original: Lemma A.1.
Revised: Lemma A.1.

Original: P ◦ U =
Revised: P ◦ U =

Original: I.
Revised: I.

Original: U ◦ P = Π .
Revised: U ◦ P = Π .

Original: Proof.
Revised: Proof.

Original: For the ﬁrst case, it’s easy to see P ◦ U = PP + =
Revised: For the ﬁrst case, it’s easy to see P ◦ U = PP + =

Original: I and U ◦ P = P + P = Π .
Revised: I and U ◦ P = P + P = Π .

Original: For the second case, P ◦ U
Revised: For the second case, P ◦ U

Original: = Γ − 1 / 2 ( P + ) T P + Γ − 1 /= Γ − 1 / 2 ΠΓ − 1 /= I.
Revised: [Equation]

Original: U ◦ P = 
Revised: = P [Equation]

Original: P + Γ − 1 ( P + ) T

Original: = I .
Revised: I .

Original: For the third case,
Revised: For the third case,

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: Now we prove the three lemmas in the main paper.
Revised: Now we prove the three lemmas in the main paper.

Original: Proposition A.2.
Revised: Proposition A.2.

Original: For any vector ˆ x ∈ R n , we have that Q (cid:98) L (ˆ x ) =
Revised: For any vector ˆ x ∈ R n , we have that Q (cid:98) L (ˆ x ) =

Original: Q L ( P + ˆ x ) .
Revised: Q L ( P + ˆ x ) .

Original: In other words, set x := P + ˆ x as the lift of ˆ x in R N , then ˆ x T (cid:98)
Revised: In other words, set x := P + ˆ x as the lift of ˆ x in R N , then ˆ x T (cid:98)

Original: L ˆ x = x T Lx .
Revised: L ˆ x = x T Lx .

Revised: Proof.

Revised: Q L ( U ˆ x ) =

Original: [Equation]
Revised: ( U ˆ x ) T L U ˆ x = ˆ x ( P + ) T LP + ˆ x T = ˆ x T (cid:98) L ˆ x = Q (cid:98) L (ˆ x )

Original: Proposition A.3.
Revised: Proposition A.3.

Original: For any vector x ∈ R n , we have that R (cid:98) L (ˆ x ) =
Revised: For any vector x ∈ R n , we have that R (cid:98) L (ˆ x ) =

Original: R L ( P + Γ − 1 / 2 ˆ x ) .
Revised: R L ( P + Γ − 1 / 2 ˆ x ) .

Original: That is, set the lift of ˆ x in R N to be x = P + Γ − 1 / 2 ˆ x , then we have that ˆ x T (cid:98)
Revised: That is, set the lift of ˆ x in R N to be x = P + Γ − 1 / 2 ˆ x , then we have that ˆ x T (cid:98)

Original: L ˆ x ˆ x T ˆ x = xT Lx x T x .
Revised: L ˆ x ˆ x T ˆ x = x

Original: Proof.
Revised: Proof.

Original: By deﬁnition R L ( U ˆ x ) =
Revised: By deﬁnition R L ( U ˆ x ) =

Original: Q L ( U ˆ x ) ||U ˆ x || 22 , R L ( x ) = Q (cid:98) L ( x ) || x
Revised: Q L ( U ˆ x ) ||U ˆ x || 22 , R L ( x ) = Q (cid:98) L ( x ) || x

Original: || 22 .
Revised: || 22 .

Original: We will prove the lemma by showing
Revised: We will prove the lemma by showing

Original: Q L ( U ˆ x ) =
Revised: Q L ( U ˆ x ) =

Original: Q L ( x ) and ||U
Revised: Q L ( x ) and ||U

Original: ˆ x || 

Revised: ˆ x ||

Original: = ||
Revised: = ||

Original: x || 22 .
Revised: x || 22 .

Original: [Equation]
Revised: [Equation]

Original: [-(ˆ x ) ||U ˆ x || 
-]
Revised: [-(ˆ x ) ||U ˆ x || 
-]

Original: = ˆ x T Γ − 1 / 2 ( P + ) T P + Γ − 1 / 2 ˆ x = ˆ x T

Original: ˆ x =

Original: ||ˆ x || 22 .
Revised: || 22 .

Original: Since both numerator and denominator stay the same under the action of U , we conclude R L ( U ˆ x )
Revised: Since both numerator and denominator stay the same under the action of U , we conclude R L ( U ˆ x )

Original: = R (cid:98) L (ˆ x ) .
Revised: = R (cid:98) L (ˆ x ) .

Original: Proposition A.4.
Revised: Proposition A.4.

Original: For any vector x ∈ R n , we have that Q (cid:98) L ( x ) =
Revised: For any vector x ∈ R n , we have that Q (cid:98) L ( x ) =

Original: Q L ( D 1 / 2 P + (cid:98) D 1 / 2 x ) .
Revised: Q L ( D 1 / 2 P + (cid:98) D 1 / 2 x ) .

Original: That is, set the lift of ˆ x in R N to be x := D 1 / 2 P + (cid:98) D 1 / 2 x , then we have that ˆ x T (cid:98)
Revised: That is, set the lift of ˆ x in R N to be x := D 1 / 2 P + (cid:98) D 1 / 2 x , then we have that ˆ x T (cid:98)

Original: L ˆ x = x T L x .
Revised: L ˆ x = x T L x .

Original: Proof.
Revised: Proof.

Original: [Equation]
Revised: [Equation]

Original: B M ORE R ELATED W ORK
Revised: B M ORE R ELATED W ORK

Original: Graph pooling .
Revised: Graph pooling .

Original: Graph pooling (Lee et al., 2019) is proposed in the context of the hierarchical graph representation learning.
Revised: Graph pooling (Lee et al., 2019) is proposed in the context of the hierarchical graph representation learning.

Original: DiffPool (Ying et al., 2018) is proposed to use graph neural networks to parametrize the soft clustering of nodes.
Revised: DiffPool (Ying et al., 2018) is proposed to use graph neural networks to parametrize the soft clustering of nodes.

Original: Its limitation in quadratic memory is later improved by (Gao & Ji, 2019; Cangea et al., 2018).
Revised: Its limitation in quadratic memory is later improved by (Gao & Ji, 2019; Cangea et al., 2018).

Original: All those methods are supervised and tested for the graph classiﬁcation task.
Revised: All those methods are supervised and tested for the graph classiﬁcation task.

Original: Optimal transportation theory .
Revised: Optimal transportation theory .

Original: Several recent works adapt the concepts from optimal transportation theory to compare graphs of different sizes.
Revised: Several recent works adapt the concepts from optimal transportation theory to compare graphs of different sizes.

Original: Garg & Jaakkola (2019) aims to minimize optimal transport distance between probability measures on the original graph and coarse graph.
Revised: Garg & Jaakkola (2019) aims to minimize optimal transport distance between probability measures on the original graph and coarse graph.

Original: Maretic et al.
Revised: Maretic et al.

Original: (2019) proposes a framework based on Wasserstein distance between graph signal distributions in terms of their graph Laplacian matrices.
Revised: (2019) proposes a framework based on Wasserstein distance between graph signal distributions in terms of their graph Laplacian matrices.

Original: Ma & Chen (2019) replaces supervised loss tailored for speciﬁc downstream tasks with unsupervised ones based on Wasserstein distance.
Revised: Ma & Chen (2019) replaces supervised loss tailored for speciﬁc downstream tasks with unsupervised ones based on Wasserstein distance.

Original: C D ATASET C. S YNTHETIC GRAPHS
Revised: C D ATASET C. S YNTHETIC GRAPHS

Original: Erd˝os–R´enyi graphs (ER).
Revised: Erd˝os–R´enyi graphs (ER).

Original: G ( n, p ) where p = 0 .
Revised: G ( n, p ) where p = 0 .

Original: 1 ∗ 512 n
Revised: 1 ∗ 512 n

Original: Random geometric graphs (GEO).
Revised: Random geometric graphs (GEO).

Original: The random geometric graph model places n nodes uniformly at random in the unit cube.
Revised: The random geometric graph model places n nodes uniformly at random in the unit cube.

Original: Two nodes are joined by an edge if the distance between the nodes is at most radius r .
Revised: Two nodes are joined by an edge if the distance between the nodes is at most radius r .

Original: We set r = 5 .
Revised: We set r = 5 .

Original: 12 √ n .
Revised: 12 √ n .

Original: Barabasi-Albert Graph (BA).
Revised: Barabasi-Albert Graph (BA).

Original: A graph of n nodes is grown by attaching new nodes each with m edges that are preferentially attached to existing nodes with high degrees.
Revised: A graph of n nodes is grown by attaching new nodes each with m edges that are preferentially attached to existing nodes with high degrees.

Original: We set m to be 4 .
Revised: We set m to be 4 .

Original: Watts-Strogatz Graph (WS).
Revised: Watts-Strogatz Graph (WS).

Original: It is ﬁrst created from a ring over n nodes.
Revised: It is ﬁrst created from a ring over n nodes.

Original: Then each node in the ring is joined to its k nearest neighbors (or k − 1 neighbors if k is odd).
Revised: Then each node in the ring is joined to its k nearest neighbors (or k − 1 neighbors if k is odd).

Original: Then shortcuts are created by replacing some edges as follows: for each edge ( u, v ) in the underlying ” n -ring with k nearest neighbors” with probability p replace it with a new edge ( u, w ) with a uniformly random choice of existing node w .
Revised: Then shortcuts are created by replacing some edges as follows: for each edge ( u, v ) in the underlying ” n -ring with k nearest neighbors” with probability p replace it with a new edge ( u, w ) with a uniformly random choice of existing node w .

Original: We set k, p to be 10 and 0 . 1 .
Revised: We set k, p to be 10 and 0 . 1 .

Original: C. D ATASET FROM L OUKAS ’ S PAPER
Revised: C. D ATASET FROM L OUKAS ’ S PAPER

Original: Yeast.
Revised: Yeast.

Original: Protein-to-protein interaction network in budding yeast, analyzed by (Jeong et al., 2001).
Revised: Protein-to-protein interaction network in budding yeast, analyzed by (Jeong et al., 2001).

Original: The network has N = 1458 vertices and M = 1948 edges.
Revised: The network has N = 1458 vertices and M = 1948 edges.

Original: Airfoil.
Revised: Airfoil.

Original: Finite-element graph obtained by airow simulation (Preis & Diekmann, 1997), consisting of N = 4000 vertices and M = 11 , 490 edges.
Revised: Finite-element graph obtained by airow simulation (Preis & Diekmann, 1997), consisting of N = 4000 vertices and M = 11 , 490 edges.

Original: Minnesota (Gleich, 2008).
Revised: Minnesota (Gleich, 2008).

Original: Road network with N = 2642 vertices and M = 3304 edges.
Revised: Road network with N = 2642 vertices and M = 3304 edges.

Original: Bunny (Turk & Levoy, 1994).
Revised: Bunny (Turk & Levoy, 1994).

Original: Point cloud consisting of N = 2503 vertices and M = 65 , 490 edges.
Revised: Point cloud consisting of N = 2503 vertices and M = 65 , 490 edges.

Original: The point cloud has been sub-sampled from its original size.
Revised: The point cloud has been sub-sampled from its original size.

Original: C. R EAL NETWORKS
Revised: C. R EAL NETWORKS

Original: Shape graphs (Shape).
Revised: Shape graphs (Shape).

Original: Each graph is KNN graph formed by 1024 points sampled from shapes from
Revised: Each graph is KNN graph formed by 1024 points sampled from shapes from

Original: ShapeNet where each node is connected 10 nearest neighbors.
Revised: ShapeNet where each node is connected 10 nearest neighbors.

Original: Coauthor-CS (CS) and Coauthor-Physics (Physics) are co-authorship graphs based on the Microsoft
Revised: Coauthor-CS (CS) and Coauthor-Physics (Physics) are co-authorship graphs based on the Microsoft

Original: Academic Graph from the KDD Cup 2016 challenge.
Revised: Academic Graph from the KDD Cup 2016 challenge.

Original: Coauthor CS has N = 18 , 333 nodes and
Revised: Coauthor CS has N = 18 , 333 nodes and

Original: M = 81 , 894 edges.
Revised: M = 81 , 894 edges.

Original: Coauthor Physics has N = 34 , 493 nodes and M = 247 , 962 edges.
Revised: Coauthor Physics has N = 34 , 493 nodes and M = 247 , 962 edges.

Original: PubMed (Sen et al., 2008) has N = 19 , 717 nodes and M = 44 , 324 edges.
Revised: PubMed (Sen et al., 2008) has N = 19 , 717 nodes and M = 44 , 324 edges.

Original: Nodes are documents and edges are citation links.
Revised: Nodes are documents and edges are citation links.

Original: Flickr (Zeng et al., 2019) has N = 89 , 250 nodes and M = 899 , 756 edges.
Revised: Flickr (Zeng et al., 2019) has N = 89 , 250 nodes and M = 899 , 756 edges.

Original: One node in the graph represents one image uploaded to Flickr.
Revised: One node in the graph represents one image uploaded to Flickr.

Original: If two images share some common properties (e.g., same geographic location, same gallery, comments by the same user, etc.)
Revised: If two images share some common properties (e.g., same geographic location, same gallery, comments by the same user, etc.)

Original: , there is an edge between the nodes of these two images.
Revised: , there is an edge between the nodes of these two images.

Original: D E XISTING GRAPH COARSENING METHODS
Revised: D E XISTING GRAPH COARSENING METHODS

Original: Heavy Edge Matching .
Revised: Heavy Edge Matching .

Original: At each level of the scheme, the contraction family is obtained by computing a maximum-weight matching with the weight of each contraction set ( v i , v j ) calculated as w ij / max { d i , d j } .
Revised: At each level of the scheme, the contraction family is obtained by computing a maximum-weight matching with the weight of each contraction set ( v i , v j ) calculated as w ij / max { d i , d j } .

Original: In this manner, heavier edges connecting vertices that are well separated from the rest of the graph are contracted ﬁrst.
Revised: In this manner, heavier edges connecting vertices that are well separated from the rest of the graph are contracted ﬁrst.

Original: Algebraic Distance .
Revised: Algebraic Distance .

Original: This method differs from heavy edge matching in that the weight of each candidate set ( v i , v
Revised: This method differs from heavy edge matching in that the weight of each candidate set ( v i , v

Original: j ) ∈
Revised: j ) ∈

Original: E is calculated as (cid:16)(cid:80) Qq =1 ( x q ( i ) − x q ( j )) 2 (cid:17) 1 / 2 , where x k is an N -dimensional test vector computed by successive sweeps of Jacobi relaxation.
Revised: E is calculated as (cid:16)(cid:80) Q q =1 ( x q ( i ) − x q ( j )) 2 (cid:17) 1 / 2 , where x k is an N -dimensional test vector computed by successive sweeps of Jacobi relaxation.

Original: The complete method is described by Ron et al.
Revised: The complete method is described by Ron et al.

Original: (2011), see also Chen & Safro (2011).
Revised: (2011), see also Chen & Safro (2011).

Original: Afﬁnity .
Revised: Afﬁnity .

Original: This is a vertex proximity heuristic in the spirit of the algebraic distance that was proposed by Livne & Brandt (2012) in the context of their work on the lean algebraic multigrid.
Revised: This is a vertex proximity heuristic in the spirit of the algebraic distance that was proposed by Livne & Brandt (2012) in the context of their work on the lean algebraic multigrid.

Original: As per the author suggests, the Q = k test vectors are here computed by a single sweep of a Gauss-Seidel iteration.
Revised: As per the author suggests, the Q = k test vectors are here computed by a single sweep of a Gauss-Seidel iteration.

Original: Local Variation .
Revised: Local Variation .

Original: There are two variations of local variation methods, edge-based local variation, and neighborhood-based local variation.
Revised: There are two variations of local variation methods, edge-based local variation, and neighborhood-based local variation.

Original: They differ in how the contraction set is chosen.
Revised: They differ in how the contraction set is chosen.

Original: Edgebased variation is constructed for each edge, while the neighborhood-based variant takes every vertex and its neighbors as contraction set.
Revised: Edgebased variation is constructed for each edge, while the neighborhood-based variant takes every vertex and its neighbors as contraction set.

Original: What two methods have common is that they both optimize an upper bound of the restricted spectral approximation objective.
Revised: What two methods have common is that they both optimize an upper bound of the restricted spectral approximation objective.

Original: In each step, they greedily pick the sets whose local variation is the smallest.
Revised: In each step, they greedily pick the sets whose local variation is the smallest.

Original: See Loukas (2019) for more details.
Revised: See Loukas (2019) for more details.

Original: Baseline .
Revised: Baseline .

Original: We also implement a simple baseline that randomly chooses a collection of nodes in the original graph as landmarks and contract other nodes to the nearest landmarks.
Revised: We also implement a simple baseline that randomly chooses a collection of nodes in the original graph as landmarks and contract other nodes to the nearest landmarks.

Original: If there are multiple nearest landmarks, we randomly break the tie.
Revised: If there are multiple nearest landmarks, we randomly break the tie.

Original: The weight of the coarse graph is set to be the sum of the weights of the crossing edges.
Revised: The weight of the coarse graph is set to be the sum of the weights of the crossing edges.

Original: E D ETAILS OF THE EXPERIMENTAL SETUP
Revised: E D ETAILS OF THE EXPERIMENTAL SETUP

Original: Feature Initialization.
Revised: Feature Initialization.

Original: We initialize the the node feature of subgraphs as a 5 dimensional feature based on a simple heuristics local degree proﬁle (LDP) (Cai & Wang, 2018).
Revised: We initialize the the node feature of subgraphs as a 5 dimensional feature based on a simple heuristics local degree proﬁle (LDP) (Cai & Wang, 2018).

Original: For each node v ∈ G ( V ) , let DN ( v ) denote the multiset of the degree of all the neighboring nodes of v , i.e.,
Revised: For each node v ∈ G ( V ) , let DN ( v ) denote the multiset of the degree of all the neighboring nodes of v , i.e.,

Original: DN ( v ) = { degree ( u ) | ( u, v ) ∈ E } .
Revised: DN ( v ) = { degree ( u ) | ( u, v ) ∈ E } .

Original: We take ﬁve node features, which are (degree( v ), min(DN( v )), max(DN( v )),mean(DN( v )), std(DN( v ))).
Revised: We take ﬁve node features, which are (degree( v ), min(DN( v )), max(DN( v )),mean(DN( v )), std(DN( v ))).

Original: In other words, each node feature summarizes the degree information of this node and its 1- neighborhood.
Revised: In other words, each node feature summarizes the degree information of this node and its 1- neighborhood.

Original: We use the edge weight as 1 dimensional edge feature.
Revised: We use the edge weight as 1 dimensional edge feature.

Original: Optimization.
Revised: Optimization.

Original: All models are trained with Adam optimizer (Kingma & Ba, 2014) with a learning rate of 0.001 and batch size 600.
Revised: All models are trained with Adam optimizer (Kingma & Ba, 2014) with a learning rate of 0.001 and batch size 600.

Original: We use Pytorch (Paszke et al., 2017) and Pytorch Geometric (Fey & Lenssen, 2019) for all of our implementation.
Revised: We use Pytorch (Paszke et al., 2017) and Pytorch Geometric (Fey & Lenssen, 2019) for all of our implementation.

Original: We train graphs one by one where for each graph we train the model to minimize the loss for certain epochs (see hyper-parameters for details) before moving to the next graph.
Revised: We train graphs one by one where for each graph we train the model to minimize the loss for certain epochs (see hyper-parameters for details) before moving to the next graph.

Original: We save the model that performs best on the validation graphs and test it on the test graphs.
Revised: We save the model that performs best on the validation graphs and test it on the test graphs.

Original: Model Architecture.
Revised: Model Architecture.

Original: The building block of our graph neural networks is based on the modiﬁcation of Graph Isomorphism Network (GIN) that can handle both node and edge features.
Revised: The building block of our graph neural networks is based on the modiﬁcation of Graph Isomorphism Network (GIN) that can handle both node and edge features.

Original: In particular, we ﬁrst linear transform both node feature and edge feature to be vectors of the same dimension.
Revised: In particular, we ﬁrst linear transform both node feature and edge feature to be vectors of the same dimension.

Original: At the k -th layer, GNNs update node representations by
Revised: At the k -th layer, GNNs update node representations by

Original: [Equation]
Revised: [Equation]

Original:  where N ( v ) is a set of nodes adjacent to v , and e = ( v ; v ) represents the self-loop edge.
Revised:   (3) where N ( v ) is a set of nodes adjacent to v , and e = ( v ; v ) represents the self-loop edge.

Original: Edge features h ( k − 1) e is the same across the layers.
Revised: Edge features h ( k − 1) e is the same across the layers.

Original: We use average graph pooling to obtained the graph representation from node embeddings, i.e.,
Revised: We use average graph pooling to obtained the graph representation from node embeddings, i.e.,

Original: h G = MEAN (cid:16)(cid:110) h ( K ) v | v ∈ G (cid:111)(cid:17) .
Revised: h G = MEAN (cid:16)(cid:110) h ( K ) v | v ∈ G (cid:111)(cid:17) .

Original: The ﬁnal prediction of weight is 1 + ReLu(Φ( h G )) where Φ is a linear layer.
Revised: The ﬁnal prediction of weight is 1 + ReLu(Φ( h G )) where Φ is a linear layer.

Original: We set the number of layers to be 3 and the embedding dimension to be 50.
Revised: We set the number of layers to be 3 and the embedding dimension to be 50.

Original: Time Complexity.
Revised: Time Complexity.

Original: In the preprocessing step, we need to compute the ﬁrst k eigenvectors of Laplacian (either combinatorial or normalized one) of the original graph as test vectors.
Revised: In the preprocessing step, we need to compute the ﬁrst k eigenvectors of Laplacian (either combinatorial or normalized one) of the original graph as test vectors.

Original: Those can be efﬁciently computed by Restarted Lanczos Method (Lehoucq et al., 1998) to ﬁnd the eigenvalues and eigenvectors.
Revised: Those can be efﬁciently computed by Restarted Lanczos Method (Lehoucq et al., 1998) to ﬁnd the eigenvalues and eigenvectors.

Original: In the training time, our model needs to recompute the term in the loss involving the coarse graph to update the weights of the graph neural networks for each batch.
Revised: In the training time, our model needs to recompute the term in the loss involving the coarse graph to update the weights of the graph neural networks for each batch.

Original: For loss involving Laplacian (either combinatorial or normalized Laplacian), the time complexity to compute the x T Lx is O
Revised: For loss involving Laplacian (either combinatorial or normalized Laplacian), the time complexity to compute the x T Lx is O

Original: ( | E | k ) where | E | is the number of edges in the coarse graph and k is the number of test vectors.
Revised: ( | E | k ) where | E | is the number of edges in the coarse graph and k is the number of test vectors.

Original: For loss involving conductance, computing the conductance of one subset S ⊂ E is still O
Revised: For loss involving conductance, computing the conductance of one subset S ⊂ E is still O

Original: ( | E | ) so in total the time complexity is also O ( | E | k ) .
Revised: ( | E | ) so in total the time complexity is also O ( | E | k ) .

Original: In summary, the time complexity for each batch is linear in the number of edges of training graphs.
Revised: In summary, the time complexity for each batch is linear in the number of edges of training graphs.

Revised: All experiments are performed on a single Intel Xeon CPU

Original: Hyperparameters.
Revised: E5-2630 v4@ 2.20GHz × 40 and 64GB RAM machine. More concretely, for synthetic graphs, it takes a few minutes to train the model. For real graphs like CS, Physics, PubMed, it takes around 1 hour.

Revised: For the largest network Flickr of 89k nodes and 899k edges, it takes about 5 hours for most coarsening algorithms and reduction ratios.

Original:  We list the major hyperparameters of GOREN below.
Revised: Hyperparameters. We list the major hyperparameters of GOREN below.

Original: • epoch: 50 for synthetic graphs and 30 for real networks.
Revised: • epoch: 50 for synthetic graphs and 30 for real networks.

Original: • walk length: 5000 for real networks.
Revised: • walk length: 5000 for real networks.

Original: Note the size of the subgraph is usually aroundsince the random walk visits some nodes more than once.
Revised: Note the size of the subgraph is usually aroundsince the random walk visits some nodes more than once.

Original: • number of eigenvectors k : 40 for synthetic graphs and 200 for real networks.
Revised: • number of eigenvectors k : 40 for synthetic graphs and 200 for real networks.

Original: • embedding dimension:• batch size:• learning rate: 0.
Revised: • embedding dimension:• batch size:• learning rate: 0.

Original: F I TERATIVE A LGORITHM FOR S PECTRUM A LIGNMENT F. P ROBLEM S TATEMENT
Revised: F I TERATIVE A LGORITHM FOR S PECTRUM A LIGNMENT F. P ROBLEM S TATEMENT

Original: Given a graph G and its coarse graph (cid:98) G output by existing algorithm A , ideally we would like to set edge weight of (cid:98) G so that spectrum of (cid:98) L (denoted as L w below) has prespeciﬁed eigenvalues λ , i.e, 
Revised: Given a graph G and its coarse graph (cid:98) G output by existing algorithm A , ideally we would like to set edge weight of (cid:98) G so that spectrum of (cid:98) L (denoted as L w below) has prespeciﬁed eigenvalues λ , i.e, L w = U Diag( λ ) U T subject to w ≥ 0 , U T U

Original: [Equation]
Revised: = I (4)

Original: We would like to make an important note that in general, given a sequence of non decreasing numbers 0 = λ 1 ≤ λ 2 , ..., λ n and a coarse graph (cid:98) G , it is not always possible to set the edge weights (always positive) so that the resulting eigenvalues of graph Laplacian of (cid:98) G is {
Revised: We would like to make an important note that in general, given a sequence of non decreasing numbers 0 = λ 1 ≤ λ 2 , ..., λ n and a coarse graph (cid:98) G , it is not always possible to set the edge weights (always positive) so that the resulting eigenvalues of graph Laplacian of (cid:98) G is {

Original: 0 = λ 1 , λ 2 , ..., λ n } .
Revised: 0 = λ 1 , λ 2 , ..., λ n } .

Original: We introduce some notations before we present the theorem.
Revised: We introduce some notations before we present the theorem.

Original: The theorem is developed in the context of inverse eigenvalue problem for graphs (Barioli & Fallat, 2004; Hogben, 2005; Fallat et al., 2020), which aims to characterize the all possible sets of eigenvalues that can be realized by symmetric matrices whose sparsity pattern is related to the topology of a given graph.
Revised: The theorem is developed in the context of inverse eigenvalue problem for graphs (Barioli & Fallat, 2004; Hogben, 2005; Fallat et al., 2020), which aims to characterize the all possible sets of eigenvalues that can be realized by symmetric matrices whose sparsity pattern is related to the topology of a given graph.

Original: For a symmetric real n × n matrix M , the graph of M is the graph with vertices { 1 , ..., n } and edges {{ i, j } | b ij (cid:54)
Revised: For a symmetric real n × n matrix M , the graph of M is the graph with vertices { 1 , ..., n } and edges {{ i, j } | b ij (cid:54)

Original: = 

Revised: =

Original: and i (cid:54) = j } .
Revised: and i (cid:54) = j } .

Original: Note that the diagonal of M is ignored in determining G ( M ) .
Revised: Note that the diagonal of M is ignored in determining G ( M ) .

Original: Let S n be the set of real symmetric n × n matrices.
Revised: Let S n be the set of real symmetric n × n matrices.

Original: For a graph (cid:98) G with n nodes, deﬁne S ( (cid:98) G ) =
Revised: For a graph (cid:98) G with n nodes, deﬁne S ( (cid:98) G ) =

Original: (cid:110)
Revised: (cid:110)

Original: M ∈ S n | G ( M )
Revised: M ∈ S n | G ( M )

Original: = (cid:98) G (cid:111) .
Revised: = (cid:98) G (cid:111) .

Original: Theorem F.1.
Revised: Theorem F.1.

Original: (Barioli & Fallat, 2004; Hogben, 2005)
Revised: (Barioli & Fallat, 2004; Hogben, 2005)

Original: If T is a tree, for any M ∈ S ( T ) , the diameter of T is less than the number of distinct eigenvalues of M .
Revised: If T is a tree, for any M ∈ S ( T ) , the diameter of T is less than the number of distinct eigenvalues of M .

Original: For any graph (cid:98) G , its Laplacian (both combinatorial and normalized Laplacian) belongs to S ( (cid:98) G ) , the above theorem therefore applies.
Revised: For any graph (cid:98) G , its Laplacian (both combinatorial and normalized Laplacian) belongs to S ( (cid:98) G ) , the above theorem therefore applies.

Original: In other words, given a tree T and given a sequence of nondecreasing numbers 0 = λ 1 ≤ λ 2 , ...λ n , as long as the number of distinct values in sequences is less than the diameter of T , then this sequence can not be realized as the eigenvalues of graph Laplacian of T , no matter how we set the edge weights.
Revised: In other words, given a tree T and given a sequence of nondecreasing numbers 0 = λ 1 ≤ λ 2 , ...λ n , as long as the number of distinct values in sequences is less than the diameter of T , then this sequence can not be realized as the eigenvalues of graph Laplacian of T , no matter how we set the edge weights.

Original: Therefore Instead of looking for the a graph with exact spectral alignment with original graph, which is impossible for some nondecreasing sequences as illustrated by the theorem F.1, we relax the equality in equation 4 by instead minimizing the || L w − U Diag( λ ) U T || 2 F .
Revised: Therefore Instead of looking for the a graph with exact spectral alignment with original graph, which is impossible for some nondecreasing sequences as illustrated by the theorem F.1, we relax the equality in equation 4 by instead minimizing the || L w − U Diag( λ ) U T || 2 F .

Original: We ﬁrst present an algorithm for the complete graph (cid:98) G of size n .
Revised: We ﬁrst present an algorithm for the complete graph (cid:98) G of size n .

Original: This algorithm is essentially the special case of (Kumar et al., 2019).
Revised: This algorithm is essentially the special case of (Kumar et al., 2019).

Original: We then show relaxing (cid:98) G from the complete graph to the arbitrary graph will not change the convergence result.
Revised: We then show relaxing (cid:98) G from the complete graph to the arbitrary graph will not change the convergence result.

Original: Before that, we introduce some notations.
Revised: Before that, we introduce some notations.

Original: F. N OTATION
Revised: F. N OTATION

Original: Deﬁnition 1.
Revised: Deﬁnition 1.

Original: The linear operator L : w ∈ R n ( n − 1) 2 + → L w ∈ R n × n is deﬁned as 
Revised: The linear operator L : w ∈ R n ( n − 1) 2 + → L w ∈ R n × n is deﬁned as [ L w [Equation] =  

Original: [Equation]
Revised: [Equation]

Revised: >

Original: j
Revised: j

Revised: [Equation]

Revised: L w

Original: [Equation]
Revised: [Equation]

Original: ji
Revised: ji

Original: [Equation]
Revised: [Equation]

Revised: >

Revised: [Equation]

Revised: (cid:80) i (cid:54) =

Revised: [Equation]

Revised: [ L w ] ij i =

Original:  where d j = − j + j − 12 (2 n − j )
Revised: j where d j = − j + j − 12 (2 n − j )

Original: A toy example is given to illustrate the operators, Consider a weight vector w =
Revised: A toy example is given to illustrate the operators, Consider a weight vector w =

Original: [ w 1 , w 2 , w 3 , w 4 , w 5 , w 6 ] T , The Laplacian operator L on w gives 
Revised: [ w 1 , w 2 , w 3 , w 4 , w 5 , w 6 ] T , The Laplacian operator L on w gives L w = 

Revised:  (cid:80) i =1 , 2 , 3 w i − w− w− w− w(cid:80) i =1 , 4 , 5 w i

Original: [Equation]
Revised: [Equation]

Revised: −

Revised: [Equation]

Original: (cid:80)
Revised: (cid:80)

Revised: [Equation]

Revised: (cid:80)

Original: [Equation]
Revised: [Equation]

Original:  
Revised:  

Original: Adjoint operator L ∗ is deﬁned to satisfy (cid:104)
Revised: Adjoint operator L ∗ is deﬁned to satisfy (cid:104)

Original: L w , Y (cid:105) =
Revised: L w , Y (cid:105) =

Original: (cid:104) w , L ∗ Y (cid:105) . 
Revised: (cid:104) w , L ∗ Y (cid:105) . F [Equation]

Original: F. C OMPLETE G RAPH CASE
Revised: C [Equation] G RAPH C

Original:  Recall our goal is tominimize w ,U (cid:13)
Revised: [Equation] Recall our goal is to

Original: [Equation]
Revised: [Equation]

Original: subject to
Revised: subject to

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Input: coarse graph (cid:98) G , error tolerance (cid:15) , iteration limit T
Revised: Input: coarse graph (cid:98) G , error tolerance (cid:15) , iteration limit T

Original: Output: coarse graph with new edge weights 1 Initialize U  element  group O ( n, R ) and t = 0 .
Revised: Output: coarse graph with new edge weights 1 Initialize U as random element in orthogonal group O ( n, R ) and t = 0 .

Original:  (cid:15)  threshold or t > T dow to 8 and
Revised: 2 while (cid:15) is smaller than the threshold or t > T do

Original:  Lemma F.
Revised: Update w t +1 , U t +1 according to 8 and Lemma F.

Original: Compute  (cid:15) t = t + w  where λ is the desired eigenvalues of the smaller graph.
Revised: Compute Error (cid:15) t = t +6 From w t , output coarse graph with new edge weights. where λ is the desired eigenvalues of the smaller graph.

Original: One choice of λ can be the ﬁrst n eigenvalues of the original graph of size N .
Revised: One choice of λ can be the ﬁrst n eigenvalues of the original graph of size N .

Original: w and U are variables of size n ( n − 1) / 2 and n × n .
Revised: w and U are variables of size n ( n − 1) / 2 and n × n .

Original: The algorithm proceeds by iteratively updating U and w while ﬁxing the other one.
Revised: The algorithm proceeds by iteratively updating U and w while ﬁxing the other one.

Original: Update for w : It can be seen when U is ﬁxed, minimizing w is equivalent to a non-negative quadratic problem minimize
Revised: Update for w : It can be seen when U is ﬁxed, minimizing w is equivalent to a non-negative quadratic problem minimize

Original: [Equation]
Revised: [Equation]

Original:  which is strictly convex where c
Revised: (6) which is strictly convex where c

Original: =
Revised: =

Original: L ∗ ( U Diag( λ ) U T ) .
Revised: L ∗ ( U Diag( λ ) U T ) .

Original: It is easy to see that the problem is strictly convex.
Revised: It is easy to see that the problem is strictly convex.

Original: However, due the the non-negativity constraint for w , there is no closed form solution.
Revised: However, due the the non-negativity constraint for w , there is no closed form solution.

Original: Thus we derive a majorization function via the following lemma.
Revised: Thus we derive a majorization function via the following lemma.

Original: Lemma F.2.
Revised: Lemma F.2.

Original: The function f ( w ) is majorized at w t by the function
Revised: The function f ( w ) is majorized at w t by the function

Original: [Equation]
Revised: [Equation]

Original: where w t is the update from previous iteration an L 1 = (cid:107) L (cid:107) 22 = 2 n .
Revised: where w t is the update from previous iteration an L 1 = (cid:107) L (cid:107) 22 = 2 n .

Original: After ignoring the constant terms in 7, the majorized problem of 6 at w t is given minimize
Revised: After ignoring the constant terms in 7, the majorized problem of 6 at w t is given minimize

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: where a = w t − 1 L 1 ∇ f ( w t ) and ∇ f ( w t )
Revised: where a = w t − 1 L 1 ∇ f ( w t ) and ∇ f ( w t )

Original: =
Revised: =

Original: L ∗ ( L w t ) − c
Revised: L ∗ ( L w t ) − c

Original: Lemma F.3.
Revised: Lemma F.3.

Original: From the KKT optimality conditions we can easily obtain the optimal solution to 7 as
Revised: From the KKT optimality conditions we can easily obtain the optimal solution to 7 as

Original: [Equation]
Revised: [Equation]

Original: where ( x ) + := max( x, 0) .
Revised: where ( x ) + := max( x, 0) .

Original: Update for U : When w is ﬁxed, the problem of optimizing U is equivalent to minimize Utr
Revised: Update for U : When w is ﬁxed, the problem of optimizing U is equivalent to minimize U

Original: [Equation]
Revised: [Equation]

Original: subject to
Revised: subject to

Original: U

Original: [Equation]
Revised: [Equation]

Original: It can be shown that the optimal U at iteration t is achieved by U t +

Revised: It can be shown that the optimal U at iteration t is achieved by U t +

Original: =
Revised: =

Original: eigenvectors ( L w ) .
Revised: eigenvectors ( L w ) .

Original: Lemma F.4.
Revised: Lemma F.4.

Original: From KKT optimality condition, the solution to 9 is given by U t + = eigenvectors ( L w ) .
Revised: From KKT optimality condition, the solution to 9 is given by U t + = eigenvectors ( L w ) .

Original: The following theorem is proved at (Kumar et al., 2019).
Revised: The following theorem is proved at (Kumar et al., 2019).

Original: Theorem F.5.
Revised: Theorem F.5.

Original: The sequence ( w t , U t ) generated by Algorithm 1 converges to the set of KKT points of 5.
Revised: The sequence ( w t , U t ) generated by Algorithm 1 converges to the set of KKT points of 5.

Original: F. N ON - COMPLETE G RAPH C ASE
Revised: F. N ON - COMPLETE G RAPH C ASE

Original: The only complication in the case of the non-complete graph is that w has only | E | number of free variables instead of n ( n − 1)variables as the case of the complete graph.
Revised: The only complication in the case of the non-complete graph is that w has only | E | number of free variables instead of n ( n − 1)variables as the case of the complete graph.

Original: We will argue that w will stay at the subspace of dimension | E | during the iteration.
Revised: We will argue that w will stay at the subspace of dimension | E | during the iteration.

Original: For simplicity, given a non-compete graph (cid:98) G =
Revised: For simplicity, given a non-compete graph (cid:98) G =

Original: ( (cid:98) V , (cid:98) E )
Revised: ( (cid:98) V , (cid:98) E )

Original: , let us denote ˆ v = [ n ] = { 1 , 2 , ..., n } and each edge will be represented as ( i, j ) where i > j , and i, j ∈ [ n ] .
Revised: , let us denote ˆ v = [ n ] = { 1 , 2 , ..., n } and each edge will be represented as ( i, j ) where i > j , and i, j ∈ [ n ] .

Original: It is easy to see that we can map each edge ( i, j ) ( i > j ) to k -th coordinate of w via k =
Revised: It is easy to see that we can map each edge ( i, j ) ( i > j ) to k -th coordinate of w via k =

Original: Φ( i, j ) =
Revised: Φ( i, j ) =

Original: i − j + ( j − 1)(2 p − j ).
Revised: i − j + ( j − 1)(2 p − j ).

Original: Let us denote w (to emphasize its dependence on (cid:98) G , it is also denoted as w (cid:98) G later.)
Revised: Let us denote w (to emphasize its dependence on (cid:98) G , it is also denoted as w (cid:98) G later.)

Original: to be the same as w on coordinates that corresponds to edges in G and 0 for the rest entries.
Revised: to be the same as w on coordinates that corresponds to edges in G and 0 for the rest entries.

Original: In other words, w [ k ] = (cid:26) w [ k ] if Φ − 1 ( k ) ∈ E o.w.
Revised: In other words, [Equation]

Original: Similarly, for any symmetric matrix A of size n × n
Revised: Similarly, for any symmetric matrix A of size n × n

Original: A [ i, j ] = (cid:26)
Revised: [Equation]

Original: A [ i, j ] if ( i, j ) ∈ E or ( j, i ) ∈
Revised: i, [Equation] [Equation]

Original: E o.w.
Revised: o.w.

Original: Let us also deﬁne a (cid:98) G -subspace of w (denoted as (cid:98) G -subspace when there is no ambiguity) as { w | w ∈ R n ( n − 1) / 2 + } .
Revised: Let us also deﬁne a (cid:98) G -subspace of w (denoted as (cid:98) G -subspace when there is no ambiguity) as { w | w ∈ R n ( n − 1) / 2 + } .

Original: What we need to prove is that if we initialize the algorithm with w (cid:98) G instead of w , w t (cid:98) G will remain in the (cid:98) G -subspace of w for any t ∈ Z + .
Revised: What we need to prove is that if we initialize the algorithm with w (cid:98) G instead of w , w t (cid:98) G will remain in the (cid:98) G -subspace of w for any t ∈ Z + .

Original: First, we have the following lemma.
Revised: First, we have the following lemma.

Original: Lemma F.6.
Revised: Lemma F.6.

Original: We have
Revised: We have

Original: L w =
Revised: L w =

Original: L w .
Revised: L w .

Original: (cid:104)
Revised: (cid:104)

Original: w 1 , w 2 (cid:105) =
Revised: w 1 , w 2 (cid:105) =

Original: (cid:104) w 1 , w 2 (cid:105)
Revised: (cid:104) w 1 , w 2 (cid:105)

Original: = (cid:104) w 1 , w 2 (cid:105) .
Revised: = (cid:104) w 1 , w 2 (cid:105) .

Original: L ∗ Y =
Revised: L ∗ Y =

Original: L ∗ Y
Revised: L ∗ Y

Original: Proof.
Revised: Proof.

Original: Lemma 1 and 2 can be proved by deﬁnition.
Revised: Lemma 1 and 2 can be proved by deﬁnition.

Original: Now we prove the last lemma.
Revised: Now we prove the last lemma.

Original: For any w ∈ R n ( n − 1) 2 + and Y ∈ R n × n
Revised: For any w ∈ R n ( n − 1) 2 + and Y ∈ R n × n

Original: [Equation]
Revised: [Equation]

Original: [Figure]

Original: where the fourth equation follows from the deﬁnition of L ∗ and the other equations directly follows from the previous two lemmas.
Revised: where the fourth equation follows from the deﬁnition of L ∗ and the other equations directly follows from the previous two lemmas.

Original: Therefore L ∗ Y =
Revised: Therefore L ∗ Y =

Original: L ∗ Y .
Revised: L ∗ Y .

Original: Recall that we minimize the following objective when updating w (cid:98) G
Revised: Recall that we minimize the following objective when updating w (cid:98) G

Original: [Equation]
Revised: [Equation]

Original: which is equivalent to be 
Revised: which is equivalent to be minimize

Original: [Equation]
Revised: [Equation]

Revised: [Figure]

Original: Now following the same process for the case of complete graph.
Revised: Now following the same process for the case of complete graph.

Original: Equation 10 is equivalent to minimize
Revised: Equation 10 is equivalent to minimize

Original: [Equation]
Revised: [Equation]

Original: where c =
Revised: where c =

Original: L ∗ ( U Diag( λ ) U T ) .
Revised: L ∗ ( U Diag( λ ) U T ) .

Original: Use the same majorization function as the case of complete graph, we can get the following update rule
Revised: Use the same majorization function as the case of complete graph, we can get the following update rule

Original: Lemma F.7.
Revised: Lemma F.7.

Original: From the KKT optimality conditions we can easily obtain the optimal solution to as
Revised: From the KKT optimality conditions we can easily obtain the optimal solution to as

Original: [Equation]
Revised: [Equation]

Original: where ( x ) + := max( x, 0) and ∇ f ( w t (cid:98) G )
Revised: where ( x ) + := max( x, 0) and ∇ f ( w t (cid:98) G )

Original: =
Revised: =

Original: L ∗ ( L w t (cid:98) G − U Diag( λ ) U T ) .
Revised: L ∗ ( L w t (cid:98) G − U Diag( λ ) U T ) .

Original: Since ∇ f ( w t (cid:98) G )
Revised: Since ∇ f ( w t (cid:98) G )

Original: =
Revised: =

Original: L ∗ ( L w t ) − A ) =
Revised: L ∗ ( L w t ) − A ) =

Original: L ∗ ( L w t − A ) =
Revised: L ∗ ( L w t − A ) =

Original: L ∗ ( L w t − A ) where A = U Diag( λ ) U T , therefore w t +1 (cid:98) G will remain in the (cid:98) G -subspace if w t (cid:98) G is in the (cid:98) G -subspace.
Revised: L ∗ ( L w t − A ) where A = U Diag( λ ) U T , therefore w t +1 (cid:98) G will remain in the (cid:98) G -subspace if w t (cid:98) G is in the (cid:98) G -subspace.

Original: Since w 0 (cid:98) G is initialized inside (cid:98) G -subspace, by induction w t (cid:98) G stays in the (cid:98) G -subspace for any t ∈ Z + .
Revised: Since w 0 (cid:98) G is initialized inside (cid:98) G -subspace, by induction w t (cid:98) G stays in the (cid:98) G -subspace for any t ∈ Z + .

Original: Therefore, we conclude
Revised: Therefore, we conclude

Original: Theorem F.8.
Revised: Theorem F.8.

Original: In the case of non-complete graph, the sequence ( w t , U t ) generated by Algorithmconverges to the set of KKT points of 5.
Revised: In the case of non-complete graph, the sequence ( w t , U t ) generated by Algorithmconverges to the set of KKT points of 5.

Original: Remark : since for each iteration a full eigendecomposition is conducted, the computational complexity is O ( n 3 ) for each iteration, which is certainly prohibitive for large scale application.
Revised: Remark : since for each iteration a full eigendecomposition is conducted, the computational complexity is O ( n 3 ) for each iteration, which is certainly prohibitive for large scale application.

Original: Another drawback is that the algorithm is not adaptive to the data so we have to run the same algorithm for graphs from the same generative distribution.
Revised: Another drawback is that the algorithm is not adaptive to the data so we have to run the same algorithm for graphs from the same generative distribution.

Original: The main takeaway of this algorithm is that it is possible to improve the spectral alignment of the original graph and coarse graph by optimizing over edge weights, as shown in Figure 3.[Table]
Revised: The main takeaway of this algorithm is that it is possible to improve the spectral alignment of the original graph and coarse graph by optimizing over edge weights, as shown in Figure 3.

Original: G M ORE RESULTS
Revised: G M ORE RESULTS

Original: We list the full results from Section 4.4 for loss involving normalized Laplacian and conductance.
Revised: We list the full results from Section 4.4 for loss involving normalized Laplacian and conductance.

Original: G. D ETAILS ABOUT S ECTION 4. 
Revised: G. D ETAILS ABOUT S ECTION 4.

Original: We list the full details of Section 4.1.
Revised: We list the full details of Section 4.1.

Original: G. D ETAILS ABOUT S ECTION 4.2 AND 4. 
Revised: G. D ETAILS ABOUT S ECTION 4.2 AND 4.

Original: We list the full details of Section 4.2 and 4.3.
Revised: We list the full details of Section 4.2 and 4.3.

Original: G.
Revised: [Table] [Table] [Table]

Original:  D ETAILS ABOUT S ECTION 4.4.
Revised: G. D ETAILS ABOUT S ECTION 4.4.

Original: We list the full details of Section 4.4.
Revised: We list the full details of Section 4.4.

Original: G. 
Revised: G. D ETAILS ABOUT E IGENERROR

Original: D ETAILS ABOUT E IGENERROR

Original: We list the Eigenerror for all datasets when the objective function is Loss ( L, (cid:98) L ) .
Revised: We list the Eigenerror for all datasets when the objective function is Loss ( L, (cid:98) L ) .

Original: H V ISUALIZATION
Revised: H V ISUALIZATION

Original: We visualize the subgraphs corresponding to randomly sampled edges of coarse graphs.
Revised: We visualize the subgraphs corresponding to randomly sampled edges of coarse graphs.

Original: For example, in WS graphs, some subgraphs have only a few nodes and edges, while other subgraphs have some common patterns such as the dumbbell shape graph.
Revised: For example, in WS graphs, some subgraphs have only a few nodes and edges, while other subgraphs have some common patterns such as the dumbbell shape graph.

Original: For PubMed, most subgraphs have tree-like structures, possibly due to the edge sparsity in the citation network.
Revised: For PubMed, most subgraphs have tree-like structures, possibly due to the edge sparsity in the citation network.

Original: In Figure H, we visualize the weight difference between coarsening algorithms with and without learning.
Revised: In Figure H, we visualize the weight difference between coarsening algorithms with and without learning.

Original: We also plot the eigenvalues of coarse graphs, where the ﬁrst 40 eigenvalues of the original graph are smaller than the coarse ones.
Revised: We also plot the eigenvalues of coarse graphs, where the ﬁrst 40 eigenvalues of the original graph are smaller than the coarse ones.

Original: After optimizing edge weights via GOREN , we see both methods produce graphs with eigenvalues closer to the eigenvalues of the original graphs.
Revised: After optimizing edge weights via GOREN , we see both methods produce graphs with eigenvalues closer to the eigenvalues of the original graphs.

Original: [Table]
Revised: [Table]

Original: Each entry x ( y ) is: x = loss w/o learning, and y = improvement percentage.

Original: BL stands for the baseline

Original: [Table]

Original: [Table]

Original: Physics
Revised: Physics

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]