Original: G RAPH Z OOM : A M ULTI - LEVEL S PECTRAL A PPROACH FOR A CCURATE AND S CALABLE G RAPH E MBEDDING
Revised: G RAPH Z OOM : A M ULTI - LEVEL S PECTRAL A PPROACH FOR A CCURATE AND S CALABLE G RAPH E MBEDDING

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Graph embedding techniques have been increasingly deployed in a multitude of different applications that involve learning on non-Euclidean data.
Revised: Graph embedding techniques have been increasingly deployed in a multitude of different applications that involve learning on non-Euclidean data.

Original: However, existing graph embedding models either fail to incorporate node attribute information during training or suffer from node attribute noise, which compromises the accuracy.
Revised: However, existing graph embedding models either fail to incorporate node attribute information during training or suffer from node attribute noise, which compromises the accuracy.

Original: Moreover, very few of them scale to large graphs due to their high computational complexity and memory usage.
Revised: Moreover, very few of them scale to large graphs due to their high computational complexity and memory usage.

Original: In this paper we propose GraphZoom, a multi-level framework for improving both accuracy and scalability of unsupervised graph embedding algorithms.
Revised: In this paper we propose GraphZoom, a multi-level framework for improving both accuracy and scalability of unsupervised graph embedding algorithms.

Original: GraphZoom ﬁrst performs graph fusion to generate a new graph that effectively encodes the topology of the original graph and the node attribute information.
Revised: GraphZoom ﬁrst performs graph fusion to generate a new graph that effectively encodes the topology of the original graph and the node attribute information.

Original: This fused graph is then repeatedly coarsened into a much smaller graph by merging nodes with high spectral similarities.
Revised: This fused graph is then repeatedly coarsened into a much smaller graph by merging nodes with high spectral similarities.

Original: GraphZoom allows any existing embedding methods to be applied to the coarsened graph, before it progressively reﬁne the embeddings obtained at the coarsest level to increasingly ﬁner graphs.
Revised: GraphZoom allows any existing embedding methods to be applied to the coarsened graph, before it progressively reﬁne the embeddings obtained at the coarsest level to increasingly ﬁner graphs.

Original: We have evaluated our approach on a number of popular graph datasets for both transductive and inductive tasks.
Revised: We have evaluated our approach on a number of popular graph datasets for both transductive and inductive tasks.

Original: Our experiments show that GraphZoom increases the classiﬁcation accuracy and signiﬁcantly reduces the run time compared to state-of-the-art unsupervised embedding methods.
Revised: Our experiments show that GraphZoom increases the classiﬁcation accuracy and signiﬁcantly reduces the run time compared to state-of-the-art unsupervised embedding methods.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Recent years have seen a surge of interest in graph embedding, which aims to encode nodes, edges, or (sub)graphs into low dimensional vectors that maximally preserve graph structural information.
Revised: Recent years have seen a surge of interest in graph embedding, which aims to encode nodes, edges, or (sub)graphs into low dimensional vectors that maximally preserve graph structural information.

Original: Graph embedding techniques have shown promising results for various applications such as vertex classiﬁcation, link prediction, and community detection (Zhou et al., 2018); (Cai et al., 2018); (Goyal & Ferrara, 2018).
Revised: Graph embedding techniques have shown promising results for various applications such as vertex classiﬁcation, link prediction, and community detection (Zhou et al., 2018); (Cai et al., 2018); (Goyal & Ferrara, 2018).

Original: However, current graph embedding methods have several drawbacks.
Revised: However, current graph embedding methods have several drawbacks.

Original: On the one hand, random-walk based embedding algorithms, such as DeepWalk (Perozzi et al., 2014) and node2vec (Grover & Leskovec, 2016), attempt to embed a graph based on its topology without incorporating node attribute information, which limits their embedding power.
Revised: On the one hand, random-walk based embedding algorithms, such as DeepWalk (Perozzi et al., 2014) and node2vec (Grover & Leskovec, 2016), attempt to embed a graph based on its topology without incorporating node attribute information, which limits their embedding power.

Original: Later, graph convolutional networks (GCN) are developed with the basic notion that node embeddings should be smooth over the graph (Kipf & Welling, 2016).
Revised: Later, graph convolutional networks (GCN) are developed with the basic notion that node embeddings should be smooth over the graph (Kipf & Welling, 2016).

Original: While GCN leverages both topology and node attribute information for simpliﬁed graph convolution in each layer, it may suffer from high frequency noise in the initial node features, which compromises the embedding quality (Maehara, 2019).
Revised: While GCN leverages both topology and node attribute information for simpliﬁed graph convolution in each layer, it may suffer from high frequency noise in the initial node features, which compromises the embedding quality (Maehara, 2019).

Original: On the other hand, few embedding algorithms can scale well to large graphs with millions of nodes due to their high computation and storage cost (Zhang et al., 2018a).
Revised: On the other hand, few embedding algorithms can scale well to large graphs with millions of nodes due to their high computation and storage cost (Zhang et al., 2018a).

Original: For example, graph neural networks (GNNs) such as GraphSAGE (Hamilton et al., 2017) collectively aggregate feature information from the neighborhood.
Revised: For example, graph neural networks (GNNs) such as GraphSAGE (Hamilton et al., 2017) collectively aggregate feature information from the neighborhood.

Original: When stacking multiple GNN layers, the ﬁnal embedding vector of a node involves the computation of a large number of intermediate embeddings from its neighbors.
Revised: When stacking multiple GNN layers, the ﬁnal embedding vector of a node involves the computation of a large number of intermediate embeddings from its neighbors.

Original: This will not only drastically increase the number of computations among nodes but also lead to high memory usage for storing the intermediate results.
Revised: This will not only drastically increase the number of computations among nodes but also lead to high memory usage for storing the intermediate results.

Original: In literature, increasing the accuracy and improving the scalability of graph embedding methods are largely viewed as two orthogonal problems.
Revised: In literature, increasing the accuracy and improving the scalability of graph embedding methods are largely viewed as two orthogonal problems.

Original: Hence most research efforts are devoted to addressing only one of the problems.
Revised: Hence most research efforts are devoted to addressing only one of the problems.

Original: For instance, Chen et al.
Revised: For instance, Chen et al.

Original: (2018) and Fu et al.
Revised: (2018) and Fu et al.

Original: (2019) proposed multi-level methods to obtain high-quality embeddings by training unsupervised models at every level; but their techniques do not improve scalability due to the additional training overhead.
Revised: (2019) proposed multi-level methods to obtain high-quality embeddings by training unsupervised models at every level; but their techniques do not improve scalability due to the additional training overhead.

Original: Liang et al.
Revised: Liang et al.

Original: (2018) developed a heuristic algorithm to coarsen the graph by merging nodes with similar local structures.
Revised: (2018) developed a heuristic algorithm to coarsen the graph by merging nodes with similar local structures.

Original: They use GCN to reﬁne the embedding results on the coarsened graphs, which not only is time- consuming to train but may also degrade accuracy when multiple GCN layers are stacked together.
Revised: They use GCN to reﬁne the embedding results on the coarsened graphs, which not only is timeconsuming to train but may also degrade accuracy when multiple GCN layers are stacked together.

Original: More recently, Akbas & Aktas (2019) proposed a similar strategy to coarsen the graph, where certain properties of the graph structure are preserved.
Revised: More recently, Akbas & Aktas (2019) proposed a similar strategy to coarsen the graph, where certain properties of the graph structure are preserved.

Original: However, this work lacks proper reﬁnement methods to improve the embedding quality.
Revised: However, this work lacks proper reﬁnement methods to improve the embedding quality.

Original: In this paper we propose GraphZoom, a multi-level spectral approach to enhancing the quality and scalability of unsupervised graph embedding methods.
Revised: In this paper we propose GraphZoom, a multi-level spectral approach to enhancing the quality and scalability of unsupervised graph embedding methods.

Original: Speciﬁcally, GraphZoom consists of four kernels: (1) graph fusion, (2) spectral graph coarsening, (3) graph embedding, and (4) embedding reﬁnement.
Revised: Speciﬁcally, GraphZoom consists of four kernels: (1) graph fusion, (2) spectral graph coarsening, (3) graph embedding, and (4) embedding reﬁnement.

Original: More concretely, graph fusion ﬁrst converts the node feature matrix into a feature graph and then fuses it with the original topology graph.
Revised: More concretely, graph fusion ﬁrst converts the node feature matrix into a feature graph and then fuses it with the original topology graph.

Original: The fused graph provides richer information to the ensuing graph embedding step to achieve a higher accuracy.
Revised: The fused graph provides richer information to the ensuing graph embedding step to achieve a higher accuracy.

Original: Spectral graph coarsening produces a series of successively coarsened graphs by merging nodes based on their spectral similarities.
Revised: Spectral graph coarsening produces a series of successively coarsened graphs by merging nodes based on their spectral similarities.

Original: We show that our coarsening algorithm can efﬁciently and effectively retain the ﬁrst few eigenvectors of the graph Laplacian matrix, which is critical for preserving the key graph structures.
Revised: We show that our coarsening algorithm can efﬁciently and effectively retain the ﬁrst few eigenvectors of the graph Laplacian matrix, which is critical for preserving the key graph structures.

Original: During the graph embedding step, any of the existing unsupervised graph embedding techniques can be applied to obtain node embeddings for the graph at the coarsest level.
Revised: During the graph embedding step, any of the existing unsupervised graph embedding techniques can be applied to obtain node embeddings for the graph at the coarsest level.

Original: 1 Embedding reﬁnement is then employed to reﬁne the embeddings back to the original graph by applying a proper graph ﬁlter to ensure embeddings are smoothed over the graph.
Revised: 1 Embedding reﬁnement is then employed to reﬁne the embeddings back to the original graph by applying a proper graph ﬁlter to ensure embeddings are smoothed over the graph.

Original: We validate the proposed GraphZoom framework on three transductive benchmarks: Cora, Citeseer and Pubmed citation networks as well as two inductive dataset: PPI and Reddit for vertex classiﬁcation task.
Revised: We validate the proposed GraphZoom framework on three transductive benchmarks: Cora, Citeseer and Pubmed citation networks as well as two inductive dataset: PPI and Reddit for vertex classiﬁcation task.

Original: We further test on friendster dataset which contains 8 million nodes and 400 million edges to show the scalability of GraphZoom.
Revised: We further test on friendster dataset which contains 8 million nodes and 400 million edges to show the scalability of GraphZoom.

Original: Our experiments show that GraphZoom can improve the classiﬁcation accuracy over all baseline embedding methods for both transductive and inductive tasks.
Revised: Our experiments show that GraphZoom can improve the classiﬁcation accuracy over all baseline embedding methods for both transductive and inductive tasks.

Original: Our main technical contributions are summarized as follows:
Revised: Our main technical contributions are summarized as follows:

Original: • GraphZoom generates high-quality embeddings.
Revised: • GraphZoom generates high-quality embeddings.

Original: We propose novel algorithms to encode graph structures and node attribute information in a fused graph and exploit graph ﬁltering during reﬁnement to remove high frequency noise.
Revised: We propose novel algorithms to encode graph structures and node attribute information in a fused graph and exploit graph ﬁltering during reﬁnement to remove high frequency noise.

Original: This results in an increase of the embedding accuracy over the prior arts by up to 19.4%.
Revised: This results in an increase of the embedding accuracy over the prior arts by up to 19.4%.

Original: • GraphZoom improves scalability.
Revised: • GraphZoom improves scalability.

Original: Our approach can signiﬁcantly reduce the embedding run time by effectively coarsening the graph without losing the key spectral properties.
Revised: Our approach can signiﬁcantly reduce the embedding run time by effectively coarsening the graph without losing the key spectral properties.

Original: Experiments show that GraphZoom can accelerate the entire embedding process by up to 40.8x while producing a similar or better accuracy than state-of-the-art techniques.
Revised: Experiments show that GraphZoom can accelerate the entire embedding process by up to 40.8x while producing a similar or better accuracy than state-of-the-art techniques.

Original: • GraphZoom is highly composable.
Revised: • GraphZoom is highly composable.

Original: Our framework is agnostic to underlying graph embedding techniques.
Revised: Our framework is agnostic to underlying graph embedding techniques.

Original: Any of the existing unsupervised embedding methods, either transductive or inductive, can be incorporated by GraphZoom in a plug-and-play manner.
Revised: Any of the existing unsupervised embedding methods, either transductive or inductive, can be incorporated by GraphZoom in a plug-and-play manner.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: GraphZoom draws inspiration from multi-level graph embedding and graph ﬁltering to boost the performance and speed of unsupervised embedding methods.
Revised: GraphZoom draws inspiration from multi-level graph embedding and graph ﬁltering to boost the performance and speed of unsupervised embedding methods.

Original: Multi-level graph embedding.
Revised: Multi-level graph embedding.

Original: Multi-level graph embedding attempts to coarsen the graph in a series of levels where graph embedding techniques can be applied on those coarsened graphs with decreasing size.
Revised: Multi-level graph embedding attempts to coarsen the graph in a series of levels where graph embedding techniques can be applied on those coarsened graphs with decreasing size.

Original: Chen et al.
Revised: Chen et al.

Original: Lin et al.
Revised: Lin et al.

Original: (2019) coarsen the graph into several levels and then perform embedding on the hierarchy of graphs from the coarsest to the original one.
Revised: (2019) coarsen the graph into several levels and then perform embedding on the hierarchy of graphs from the coarsest to the original one.

Original: Fu et al.
Revised: Fu et al.

Original: (2019) adopt a similar idea by hierarchical sampling original graph into multi-level graphs whose embedding vectors are concatenated to obtain the ﬁnal node embeddings of the original graph.
Revised: (2019) adopt a similar idea by hierarchical sampling original graph into multi-level graphs whose embedding vectors are concatenated to obtain the ﬁnal node embeddings of the original graph.

Original: Both of these works only focus on improving embedding quality without improving the scalability.
Revised: Both of these works only focus on improving embedding quality without improving the scalability.

Original: Later,
Revised: Later,

Original: Zhang et al.
Revised: Zhang et al.

Original: (2018b)
Revised: (2018b)

Original: ; Akbas & Aktas (2019) attempt to improve graph embedding scalability by only embedding on the coarsest graph.
Revised: ; Akbas & Aktas (2019) attempt to improve graph embedding scalability by only embedding on the coarsest graph.

Original: However, their approaches lack proper reﬁnement methods to generate high-quality embeddings of the original graph.
Revised: However, their approaches lack proper reﬁnement methods to generate high-quality embeddings of the original graph.

Original: Liang et al.
Revised: Liang et al.

Original: (2018) propose MILE, which only trains the coarsest graph to obtain coarse embeddings, and leverages GCN as embeddings reﬁnement method to improve embedding quality.
Revised: (2018) propose MILE, which only trains the coarsest graph to obtain coarse embeddings, and leverages GCN as embeddings reﬁnement method to improve embedding quality.

Original: Nevertheless, MILE requires to train a GCN model which is very time consuming for large graphs and cannot support inductive embedding models due to the transductive property of GCN.
Revised: Nevertheless, MILE requires to train a GCN model which is very time consuming for large graphs and cannot support inductive embedding models due to the transductive property of GCN.

Original: Graph ﬁltering.
Revised: Graph ﬁltering.

Original: Graph ﬁlters are direct analogs of classical ﬁlters in signal processing ﬁeld, but intended for signals deﬁned on graphs.
Revised: Graph ﬁlters are direct analogs of classical ﬁlters in signal processing ﬁeld, but intended for signals deﬁned on graphs.

Original: Shuman et al.
Revised: Shuman et al.

Original: (2013) deﬁned graph ﬁlters in both vertex and spectral domains, and applies graph ﬁlter in image denoising and reconstruction tasks.
Revised: (2013) deﬁned graph ﬁlters in both vertex and spectral domains, and applies graph ﬁlter in image denoising and reconstruction tasks.

Original: Recently,
Revised: Recently,

Original: Maehara (2019) showed the fundamental link between graph embedding and ﬁltering by proving that GCN model implicitly exploits graph ﬁlter to remove high frequency noise from the node feature matrix; a ﬁlter neural network (gfNN) is then proposed to derive a stronger graph ﬁlter to improve the embedding results.
Revised: Maehara (2019) showed the fundamental link between graph embedding and ﬁltering by proving that GCN model implicitly exploits graph ﬁlter to remove high frequency noise from the node feature matrix; a ﬁlter neural network (gfNN) is then proposed to derive a stronger graph ﬁlter to improve the embedding results.

Original: Li et al.
Revised: Li et al.

Original: (2019) further derived two generalized graph ﬁlters and apply them on graph embedding models to improve their embedding quality for various classiﬁcation tasks.
Revised: (2019) further derived two generalized graph ﬁlters and apply them on graph embedding models to improve their embedding quality for various classiﬁcation tasks.

Original: G RAPH Z OOM F RAMEWORK
Revised: [Equation]

Original: Figure 1 shows the proposed GraphZoom framework which consists of four key phases: Phase (1) is graph fusion, which constructs a weighted graph that fuses the information of both the graph topology and node attributes; In Phase (2), a spectral graph coarsening process is applied to form a hierarchy of coarsened fused graphs with decreasing size; In Phase (3), any of the prior graph embedding methods can be applied to the fused graph at the coarsest level; In Phase (4), the embedding vectors obtained at the coarsest level are mapped onto a ﬁner graph using the mapping operators determined during the coarsening phase.
Revised: Figure 1 shows the proposed GraphZoom framework which consists of four key phases: Phase (1) is graph fusion, which constructs a weighted graph that fuses the information of both the graph topology and node attributes; In Phase (2), a spectral graph coarsening process is applied to form a hierarchy of coarsened fused graphs with decreasing size; In Phase (3), any of the prior graph embedding methods can be applied to the fused graph at the coarsest level; In Phase (4), the embedding vectors obtained at the coarsest level are mapped onto a ﬁner graph using the mapping operators determined during the coarsening phase.

Original: This is followed by a reﬁnement (smoothing) procedure; by iteratively applying Phase (4) to increasingly ﬁner graphs, the embedding vectors for the original graph can be eventually obtained.
Revised: This is followed by a reﬁnement (smoothing) procedure; by iteratively applying Phase (4) to increasingly ﬁner graphs, the embedding vectors for the original graph can be eventually obtained.

Original: In the rest of this section, we describe each of these four phases in more detail.
Revised: In the rest of this section, we describe each of these four phases in more detail.

Original: [Figure]
Revised: [Figure]

Original: [Figure] 1: G RAPH F USION
Revised: 3. P HASE 1: G RAPH F USION

Original: Graph fusion aims to construct a weighted graph that has the same number of nodes as the original graph but potentially different set of edges (weights) that encapsulate the original graph topology as well as node attribute information.
Revised: Graph fusion aims to construct a weighted graph that has the same number of nodes as the original graph but potentially different set of edges (weights) that encapsulate the original graph topology as well as node attribute information.

Original: Speciﬁcally, given an undirected graph G =
Revised: Speciﬁcally, given an undirected graph G =

Original: ( V , E ) with N nodes, its adjacency matrix A topo ∈ R N × N and its node attribute (feature) matrix X ∈ R N × K , where K corresponds to the dimension of node attribute vector, graph fusion can be interpreted as a function f ( · ) that outputs a weighted graph G fusion =
Revised: ( V , E ) with N nodes, its adjacency matrix A topo ∈ R N × N and its node attribute (feature) matrix X ∈ R N × K , where K corresponds to the dimension of node attribute vector, graph fusion can be interpreted as a function f ( · ) that outputs a weighted graph G fusion =

Original: ( V , E fusion ) represented by its adjacency matrix A fusion ∈ R N × N , namely, A fusion = f
Revised: ( V , E fusion ) represented by its adjacency matrix A fusion ∈ R N × N , namely, A fusion = f

Original: ( A topo , X ) .
Revised: ( A topo , X ) .

Original: Graph fusion ﬁrst converts the initial attribute matrix X into a weighted node attribute graph
Revised: Graph fusion ﬁrst converts the initial attribute matrix X into a weighted node attribute graph

Original: G feat =
Revised: G feat =

Original: ( V , E feat ) by generating a k-nearest-neighbor (kNN) graph based on the l 2 - norm distance between the attribute vectors of each node pair.
Revised: ( V , E feat ) by generating a k-nearest-neighbor (kNN) graph based on the l 2 - norm distance between the attribute vectors of each node pair.

Original: Note that a straightforward implementation requires comparing all possible node pairs and then selecting top- k nearest neighbors.
Revised: Note that a straightforward implementation requires comparing all possible node pairs and then selecting top- k nearest neighbors.

Original: However, such a na¨ıve approach has a worst-case time complexity of O ( N 2 ) , which certainly does not scale to large graphs.
Revised: However, such a na¨ıve approach has a worst-case time complexity of O ( N 2 ) , which certainly does not scale to large graphs.

Original: To allow constructing the attribute graph in linear time, we leverage our O ( |E| ) complexity spectral graph coarsening scheme described with details in Section 3.2.
Revised: To allow constructing the attribute graph in linear time, we leverage our O ( |E| ) complexity spectral graph coarsening scheme described with details in Section 3.2.

Original: More specifically, our approach starts with coarsening the original graph G to obtain a substantially reduced graph that has much fewer nodes.
Revised: More specifically, our approach starts with coarsening the original graph G to obtain a substantially reduced graph that has much fewer nodes.

Original: Note that such a procedure is very similar to spectral graph clustering, which aims to group nodes into clusters of high conductance (Peng et al., 2015).
Revised: Note that such a procedure is very similar to spectral graph clustering, which aims to group nodes into clusters of high conductance (Peng et al., 2015).

Original: Once such node clusters are formed through spectral coarsening, selecting the top- k nearest neighbors within each cluster can be accomplished in O ( M 2 ) , where M is the averaged node count within the same cluster.
Revised: Once such node clusters are formed through spectral coarsening, selecting the top- k nearest neighbors within each cluster can be accomplished in O ( M 2 ) , where M is the averaged node count within the same cluster.

Original: Since we have roughly N/M clusters, the total run time for constructing the approximate kNN graph becomes O ( MN ) .
Revised: Since we have roughly N/M clusters, the total run time for constructing the ap- proximate kNN graph becomes O ( MN ) .

Original: When a proper coarsening ratio ( M (cid:28) N ) is chosen, say
Revised: When a proper coarsening ratio ( M (cid:28) N ) is chosen, say

Original: M = 50 , the overall run time complexity will become almost linear.
Revised: M = 50 , the overall run time complexity will become almost linear.

Original: For each edge in the attribute graph, we assign its weight w i,j according to the cosine similarity of two nodes’ attribute vectors: w i,j = ( X i, : · X j, : ) /
Revised: For each edge in the attribute graph, we assign its weight w i,j according to the cosine similarity of two nodes’ attribute vectors: w i,j = ( X i, : · X j, : ) /

Original: ( (cid:107) X i, : (cid:107)(cid:107) X j, : (cid:107) ) , where X i, : and X j, : are the attribute vectors of node i and j .
Revised: ( (cid:107) X i, : (cid:107)(cid:107) X j, : (cid:107) ) , where X i, : and X j, : are the attribute vectors of node i and j .

Original: Finally, we can construct the fused graph by combining the topological graph and the attribute graph: A fusion =
Revised: Finally, we can construct the fused graph by combining the topological graph and the attribute graph: A fusion =

Original: A topo + βA feat , where β allows us to balance the graph topological and node attribute information in the fusion process.
Revised: A topo + βA feat , where β allows us to balance the graph topological and node attribute information in the fusion process.

Original: The fused graph will enable the underlying graph embedding model to utilize both graph topological and node attribute information, and thus can be fed into any downstream graph embedding procedures to further improve embedding quality.
Revised: The fused graph will enable the underlying graph embedding model to utilize both graph topological and node attribute information, and thus can be fed into any downstream graph embedding procedures to further improve embedding quality.

Original: 3. P HASE 2: S PECTRAL C OARSENING
Revised: 3. P HASE 2: S PECTRAL C OARSENING

Original: Graph coarsening via global spectral embedding.
Revised: Graph coarsening via global spectral embedding.

Original: To reduce the size of the original graph while preserving important spectral properties (e.g., the ﬁrst few eigenvalues and eigenvectors of the graph
Revised: To reduce the size of the original graph while preserving important spectral properties (e.g., the ﬁrst few eigenvalues and eigenvectors of the graph

Original: Laplacian matrix 2 ), a straightforward way is to ﬁrst embed the graph into a k -dimensional space using the ﬁrst k eigenvectors of the graph Laplacian matrix, which is also known as the spectral graph embedding technique (Belkin & Niyogi, 2003; Peng et al., 2015).
Revised: Laplacian matrix 2 ), a straightforward way is to ﬁrst embed the graph into a k -dimensional space using the ﬁrst k eigenvectors of the graph Laplacian matrix, which is also known as the spectral graph embedding technique (Belkin & Niyogi, 2003; Peng et al., 2015).

Original: Next, the graph nodes that are close to each other in the low-dimensional embedding space can be aggregatedtogether to form the coarse-level nodes and subsequently the reduced graph.
Revised: Next, the graph nodes that are close to each other in the low-dimensional embedding space can be aggregated to form the coarse-level nodes and subsequently the reduced graph.

Original: However, it will be very costly to calculate the eigenvectors of the original graph Laplacian, especially for very large graphs.
Revised: However, it will be very costly to calculate the eigenvectors of the original graph Laplacian, especially for very large graphs.

Original: Graph coarsening via local spectral embedding.
Revised: Graph coarsening via local spectral embedding.

Original: In this work, we leverage an efﬁcient yet effective local spectral embedding scheme to identify node clusters based on emerging graph signal processing techniques (Shuman et al., 2013).
Revised: In this work, we leverage an efﬁcient yet effective local spectral embedding scheme to identify node clusters based on emerging graph signal processing techniques (Shuman et al., 2013).

Original: There are obvious analogies between the traditional signal processing (Fourier analysis) and graph signal processing: (1) The signals at different time points in classical Fourier analysis correspond to the signals at different nodes in an undirected graph; (2) The more slowly oscillating functions in time domain correspond to the graph Laplacian eigenvectors associated with lower eigenvalues or the more slowly varying (smoother) components across the graph.
Revised: There are obvious analogies between the traditional signal processing (Fourier analysis) and graph signal processing: (1) The signals at different time points in classical Fourier analysis correspond to the signals at different nodes in an undirected graph; (2) The more slowly oscillating functions in time domain correspond to the graph Laplacian eigenvectors associated with lower eigenvalues or the more slowly varying (smoother) components across the graph.

Original: Instead of directly using the ﬁrst few eigenvectors of the original graph Laplacian, we apply the simple smoothing (low-pass graph ﬁltering) function to k random vectors to obtain smoothed vectors for k -dimensional graph embedding, which can be achieved in linear time.
Revised: Instead of directly using the ﬁrst few eigenvectors of the original graph Laplacian, we apply the simple smoothing (low-pass graph ﬁltering) function to k random vectors to obtain smoothed vectors for k -dimensional graph embedding, which can be achieved in linear time.

Original: Consider a random vector (graph signal) x that can be expressed with a linear combination of eigenvectors u of the graph Laplacian.
Revised: Consider a random vector (graph signal) x that can be expressed with a linear combination of eigenvectors u of the graph Laplacian.

Original: Low-pass graph ﬁlters can be adopted to quickly ﬁlter out the “high-frequency” components of the random graph signal or the eigenvectors corresponding to high eigenvalues of the graph Laplacian.
Revised: Low-pass graph ﬁlters can be adopted to quickly ﬁlter out the “high-frequency” components of the random graph signal or the eigenvectors corresponding to high eigenvalues of the graph Laplacian.

Original: By applying the smoothing function on x , a smoothed vector ˜ x can be obtained, which can be considered as a linear combination of the ﬁrst few eigenvectors:
Revised: By applying the smoothing function on x , a smoothed vector ˜ x can be obtained, which can be considered as a linear combination of the ﬁrst few eigenvectors:

Original: [Equation]
Revised: [Equation]

Original: More speciﬁcally, we apply a few (e.g. ﬁve to ten) Gauss-Seidel iterations for solving the linear system of equations L G x ( i )
Revised: More speciﬁcally, we apply a few (e.g. ﬁve to ten) Gauss-Seidel iterations for solving the linear system of equations L G x ( i )

Original: = 

Revised: =

Original: to a set of t initial random vectors T =
Revised: to a set of t initial random vectors T =

Original: ( x (1) , . . .
Revised: ( x (1) , . . .

Original: , x ( t ) ) that are orthogonal to the all-one vector 1 satisfying 1 (cid:62) x
Revised: , x ( t ) ) that are orthogonal to the all-one vector 1 satisfying 1 (cid:62) x

Original: ( i ) = 0 , and L G is the Laplacian matrix of graph G or G fusion .
Revised: ( i ) = 0 , and L G is the Laplacian matrix of graph G or G fusion .

Original: Based on the smoothed vectors in T , each node is embedded into a t -dimensional space such that nodes p and q are considered spectrally similar if their low-dimensional embedding vectors x p ∈ R t and x q ∈ R t are highly correlated.
Revised: Based on the smoothed vectors in T , each node is embedded into a t -dimensional space such that nodes p and q are considered spectrally similar if their low-dimensional embedding vectors x p ∈ R t and x q ∈ R t are highly correlated.

Original: Here the node distance is measured by the spectral node afﬁnity a p,q for neighboring nodes p and q (Livne & Brandt, 2012; Chen & Safro, 2011):
Revised: Here the node distance is measured by the spectral node afﬁnity a p,q for neighboring nodes p and q (Livne & Brandt, 2012; Chen & Safro, 2011):

Original: [Equation]
Revised: [Equation]

Original: Once the node aggregation schemes are determined, the graph mapping operators on each level ( H 10 , H 21 , · · · , H mm − 1 ) can be obtained and leveraged for constructing a series of spectrally-reduced graphs G 1 , G 2 , · · · , G m , where G 0 ( G fusion ) is the original fused graph, and V 

Revised: Once the node aggregation schemes are determined, the graph mapping operators on each level ( H 10 , H 21 , · · · , H mm − 1 ) can be obtained and leveraged for constructing a series of spectrally-reduced graphs G 1 , G 2 , · · · , G m , where G 0 ( G fusion ) is the original fused graph, and V

Original: =
Revised: =

Original: N > V 1 > · · · >
Revised: N > V 1 > · · · >

Original: V m .
Revised: V m .

Original: For example, the coarser graph Laplacian L G i +1 can be computed by
Revised: For example, the coarser graph Laplacian L G i [Equation] 1 can be computed by

Original: [Equation]
Revised: [Equation]

Original: We emphasize that the aggregation scheme based on the above spectral node afﬁnity calculations will have a (linear) complexity of O ( |E fusion | ) and thus allow preserving the spectral (global or structural) properties of the original graph in a highly efﬁcient and effective way.
Revised: We emphasize that the aggregation scheme based on the above spectral node afﬁnity calculations will have a (linear) complexity of O ( |E fusion | ) and thus allow preserving the spectral (global or structural) properties of the original graph in a highly efﬁcient and effective way.

Original: As suggested in (Zhao & Feng, 2019; Loukas, 2019), a spectral sparsiﬁcation procedure can be applied to effectively control densities of coarse level graphs.
Revised: As suggested in (Zhao & Feng, 2019; Loukas, 2019), a spectral sparsiﬁcation procedure can be applied to effectively control densities of coarse level graphs.

Original: In this work, a similarity-aware spectral sparsiﬁcation tool “GRASS” (Feng, 2018) has been adopted for achieving a desired graph sparsity at the coarsest level.
Revised: In this work, a similarity-aware spectral sparsiﬁcation tool “GRASS” (Feng, 2018) has been adopted for achieving a desired graph sparsity at the coarsest level.

Original: 3. P HASE 3: G RAPH E MBEDDING
Revised: 3. P HASE 3: G RAPH E MBEDDING

Original: Embedding the Coarsest Graph.
Revised: Embedding the Coarsest Graph.

Original: Once the coarsest graph G m is constructed, node embeddings E m on G m can be obtained by E m =
Revised: Once the coarsest graph G m is constructed, node embeddings E m on G m can be obtained by E m =

Original: l ( G m ) , where l ( · ) can be any unsupervised embedding methods.
Revised: l ( G m ) , where l ( · ) can be any unsupervised embedding methods.

Original: 3. P HASE 4: E MBEDDING R EFINEMENT
Revised: 3. P HASE 4: E MBEDDING R EFINEMENT

Original: Once the base node embedding results are available, we can easily project the node embeddings from graph G i +1 to the ﬁne-grained graph G i with the corresponding projection operator H ii +1 :
Revised: Once the base node embedding results are available, we can easily project the node embeddings from graph G i +1 to the ﬁne-grained graph G i with the corresponding projection operator H ii +1 :

Original: [Equation]
Revised: [Equation]

Original: Due to the property of the projection operator, embedding of the node in coarse-grained graph will be directly copied to the nodes of the same aggregation set in the ﬁne-grained graph.
Revised: Due to the property of the projection operator, embedding of the node in coarse-grained graph will be directly copied to the nodes of the same aggregation set in the ﬁne-grained graph.

Original: In this case, spectrally-similar nodes in the ﬁne-grained graph will have the same embedding results if they are aggregated into a single node during the coarsening phase.
Revised: In this case, spectrally-similar nodes in the ﬁne-grained graph will have the same embedding results if they are aggregated into a single node during the coarsening phase.

Original: To further improve the quality of the mapped embeddings, we apply a local reﬁnement process motivated by Tikhonov regularization to smooth the node embeddings over the graph by minimizing the following objective:
Revised: To further improve the quality of the mapped embeddings, we apply a local reﬁnement process motivated by Tikhonov regularization to smooth the node embeddings over the graph by minimizing the following objective:

Original: [Equation]
Revised: [Equation]

Original: where L i and E i are the normalized Laplacian matrix and mapped embedding matrix of the graph at the i -th coarsening level, respectively.
Revised: where L i and E i are the normalized Laplacian matrix and mapped embedding matrix of the graph at the i -th coarsening level, respectively.

Original: The reﬁned embedding matrix ˜
Revised: The reﬁned embedding matrix ˜

Original: E i is obtained by solving
Revised: E i is obtained by solving

Original: Eq.
Revised: Eq.

Original: (5), whose ﬁrst term enforces the reﬁned embeddings to agree with mapped embeddings while the second term employs Laplacian smoothing to smooth ˜ E i over the graph.
Revised: (5), whose ﬁrst term enforces the reﬁned embeddings to agree with mapped embeddings while the second term employs Laplacian smoothing to smooth ˜ E i over the graph.

Original: By taking the derivative of the objective function in Eq.
Revised: By taking the derivative of the objective function in Eq.

Original: (5) and setting it to zero, we have:
Revised: (5) and setting it to zero, we have:

Original: [Equation]
Revised: [Equation]

Original: (6) where I is the identity matrix.
Revised: (6) where I is the identity matrix.

Original: However, obtaining reﬁned embeddings in this way is very time consuming since it involves matrix inversion whose time complexity is O ( N 3 ) .
Revised: However, obtaining reﬁned embeddings in this way is very time consuming since it involves matrix inversion whose time complexity is O ( N 3 ) .

Original: Instead, we exploit a more efﬁcient graph ﬁlter to smooth the embeddings.
Revised: Instead, we exploit a more efﬁcient graph ﬁlter to smooth the embeddings.

Original: Let the term ( I + L ) − 1 denoted by h ( L ) , then its corresponding graph ﬁlter in spectral domain is h ( λ ) =
Revised: Let the term ( I + L ) − 1 denoted by h ( L ) , then its corresponding graph ﬁlter in spectral domain is h ( λ ) =

Original: (1 + λ ) − 1 .
Revised: (1 + λ ) − 1 .

Original: To avoid the inversion term, we approximate h ( λ ) by its ﬁrst-order Taylor expansion, namely, ˜ h
Revised: To avoid the inversion term, we approximate h ( λ ) by its ﬁrst-order Taylor expansion, namely, ˜ h

Original: ( λ ) =
Revised: ( λ ) =

Original: 1 − λ .
Revised: 1 − λ .

Original: We then generalize ˜ h
Revised: We then generalize ˜ h

Original: ( λ ) to ˜ h
Revised: ( λ ) to ˜ h

Original: k ( λ ) =
Revised: k ( λ ) =

Original: (1 − λ ) k , where k controls the power of graph ﬁlter.
Revised: (1 − λ ) k , where k controls the power of graph ﬁlter.

Original: After transforming ˜ h k ( λ ) into spatial domain, we have: ˜ h
Revised: After transforming ˜ h k ( λ ) into spatial domain, we have: ˜ h

Original: k
Revised: k

Original: ( L ) =
Revised: ( L ) =

Original: ( I − L ) k
Revised: ( I − L ) k

Original: = ( D − 12 AD − 12 ) k , where A is the adjacency matrix and
Revised: = ( D − 12 AD − 12 ) k , where A is the adjacency matrix and

Original: D is the degree matrix.
Revised: D is the degree matrix.

Original: It can be proved that adding a proper self-loop for every node in the graph can enable ˜
Revised: It can be proved that adding a proper self-loop for every node in the graph can enable ˜

Original: h
Revised: h

Original: k
Revised: k

Original: ( L ) to more effectively ﬁlter out high-frequency noise components (Maehara, 2019)
Revised: ( L ) to more effectively ﬁlter out high-frequency noise components (Maehara, 2019)

Original: (more details are available in Appendix G).
Revised: (more details are available in Appendix G).

Original: Thus, we modify the adjacency matrix as ˜ A
Revised: Thus, we modify the adjacency matrix as ˜ A

Original: =
Revised: =

Original: A + σI , where σ is a small value to ensure every node has its own self-loop.
Revised: A + σI , where σ is a small value to ensure every node has its own self-loop.

Original: Finally, the low-pass graph ﬁlter can be utilized to smooth the mapped embedding matrix, as shown in (7).
Revised: Finally, the low-pass graph ﬁlter can be utilized to smooth the mapped embedding matrix, as shown in (7).

Original: [Equation]
Revised: [Equation]

Original: We iteratively apply Eq.
Revised: We iteratively apply Eq.

Original: (7) to obtain the embeddings of the original graph (i.e., E 0 ).
Revised: (7) to obtain the embeddings of the original graph (i.e., E 0 ).

Original: Note that our reﬁnement stage does not involve training and can be simply considered as several (sparse) matrix multiplications, which can be computed efﬁciently.
Revised: Note that our reﬁnement stage does not involve training and can be simply considered as several (sparse) matrix multiplications, which can be computed efﬁciently.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: We have performed comparative evaluation of GraphZoom framework against several existing stateof-the-art unsupervised graph embedding techniques and multi-
Revised: We have performed comparative evaluation of GraphZoom framework against several existing stateof-the-art unsupervised graph embedding techniques and multi-

Original: level embedding frameworks on ﬁve standard graph-based dataset (transductive as well as inductive).
Revised: level embedding frameworks on ﬁve standard graph-based dataset (transductive as well as inductive).

Original: In addition, we evaluate the scalability of GraphZoom on Friendster dataset that contains 8 million nodes and 400 million edges.
Revised: In addition, we evaluate the scalability of GraphZoom on Friendster dataset that contains 8 million nodes and 400 million edges.

Original: Finally, we further analyze GraphZoom kernels separately to show their effectiveness.
Revised: Finally, we further analyze GraphZoom kernels separately to show their effectiveness.

Original: 4. E XPERIMENTAL S ETUP
Revised: 4. E XPERIMENTAL S ETUP

Original: [Table]

Original: Datasets.
Revised: Datasets.

Original: The statistics of datasets used in our experiments are demonstrated in Table 1.
Revised: The statistics of datasets used in our experiments are demonstrated in Table 1.

Original: We use
Revised: We use

Original: Cora, Citeseer, Pubmed, Friendster for transductive task and PPI, Reddit for inductive task.
Revised: Cora, Citeseer, Pubmed, Friendster for transductive task and PPI, Reddit for inductive task.

Original: We choose the same training and testing size used in Kipf & Welling (2016); Hamilton et al. 
Revised: We choose the same training and testing size used in Kipf & Welling (2016); Hamilton et al. [Table]

Original: Transductive baseline models.
Revised: Transductive baseline models.

Original: Many existing graph embedding techniques are essentially transductive learning methods, which require all nodes in the graph be present during training, and their embedding models have to be retrained whenever a new node is added.
Revised: Many existing graph embedding techniques are essentially transductive learning methods, which require all nodes in the graph be present during training, and their embedding models have to be retrained whenever a new node is added.

Original: We compare GraphZoom with transductive models DeepWalk and node2vec that have shown the state-of-the-art unsupervised embedding results on the datasets used in our experiments.
Revised: We compare GraphZoom with transductive models DeepWalk, node2vec, and Deep Graph Infomax (DGI) (Velikovi et al., 2019) 3 that have shown the state-of-the-art unsupervised embedding results on the datasets used in our experiments.

Original: We further compare GraphZoom with two multi-level frameworks: HARP (Chen et al., 2018) and MILE (Liang et al., 2018), which have shown improvement upon DeepWalk and node2vec in either embedding quality or scalability.
Revised: We further compare GraphZoom with two multi-level frameworks: HARP (Chen et al., 2018) and MILE (Liang et al., 2018), which have shown improvement upon DeepWalk and node2vec in either embedding quality or scalability.

Original: Inductive baseline models.
Revised: Inductive baseline models.

Original: Inductive graph embedding models can be trained without seeing the whole graph structure and their trained models can be applied on new nodes added to graph.
Revised: Inductive graph embedding models can be trained without seeing the whole graph structure and their trained models can be applied on new nodes added to graph.

Original: To show
Revised: To show

Original: GraphZoom can also enhance inductive learning, we compare it against GraphSAGE (Hamilton et al., 2017) using four different aggregation functions.
Revised: GraphZoom can also enhance inductive learning, we compare it against GraphSAGE (Hamilton et al., 2017) using four different aggregation functions.

Original: More details of datasets and baselines are available in Appendix A and B.
Revised: More details of datasets and baselines are available in Appendix A and B.

Original: We optimize hyperparameters of DeepWalk, node2vec and GraphSAGE on original datasets as embedding baseline, and then we choose the same hyper-parameters to embed coarsened graph in HARP, MILE and our GraphZoom framework.
Revised: We optimize hyperparameters of DeepWalk, node2vec, DGI (Velikovi et al., 2019) and GraphSAGE on original datasets as embedding baseline, and then we choose the same hyper-parameters to embed coarsened graph in HARP, MILE and our GraphZoom framework.

Original: We run all the experiments on a machine running Linux with an Intel Xeon Gold 6242 CPU (32 cores, 2.40GHz) and 384 GB of RAM.
Revised: We run all the experiments on a machine running Linux with an Intel Xeon Gold 6242 CPU (32 cores, 2.40GHz) and 384 GB of RAM.

Original: 4. P ERFORMANCE AND S CALABILITY OF G RAPH Z OOM
Revised: 4. P ERFORMANCE AND S CALABILITY OF G RAPH Z OOM

Original:  Since HARP and MILE only support transductive learning, we compare them with GraphZoom for transductive task with DeepWalk and node2vec as embedding kernels.
Revised: [Table] [Table] [Table] Since HARP and MILE only support transductive learning, we compare them with GraphZoom for transductive task with DeepWalk, node2vec, and DGI (Velikovi et al., 2019) as embedding kernels.

Original: For inductive task, we compare GraphZoom with GraphSAGE using four different aggregation functions.
Revised: For inductive task, we compare GraphZoom with GraphSAGE using four different aggregation functions.

Original: Results of both transductive learning task and inductive learning task are summarized in Tables 2 and 3, respectively.
Revised: Results of both transductive learning task and inductive learning task are summarized in Tables 2 and 3, respectively.

Original: We report the mean classiﬁcation accuracy for transductive task and micro-averaged F1 score for inductive task as well as CPU time after 10 runs for all the baselines and GraphZoom.
Revised: We report the mean classiﬁcation accuracy for transductive task and micro-averaged F1 score for inductive task as well as CPU time after 10 runs for all the baselines and GraphZoom.

Original: We measure the CPU time for graph embedding as the total run time of DeepWalk, node2vec,  and GraphSAGE.
Revised: We measure the CPU time for graph embedding as the total run time of DeepWalk, node2vec, DGI, and GraphSAGE.

Original: We use the sum of CPU time for graph coarsening, graph embedding, and embedding reﬁnement as total run time of HARP and MILE.
Revised: We use the sum of CPU time for graph coarsening, graph embedding, and embedding reﬁnement as total run time of HARP and MILE.

Original: Similarly, we sum up the CPU time for graph fusion, graph coarsening, graph embedding, and embedding reﬁnement as total run time of GraphZoom.
Revised: Similarly, we sum up the CPU time for graph fusion, graph coarsening, graph embedding, and embedding reﬁnement as total run time of GraphZoom.

Original: We also perform ﬁne-tuning on the hyper-parameters.
Revised: We also perform ﬁne-tuning on the hyper-parameters.

Original: For both DeepWalk and node2vec, we use 10 walks with a walk length of 80, a window size of 10, and an embedding dimension of 128; we further set p = 1 and q = 0 . 5 in node2vec.
Revised: For both DeepWalk and node2vec, we use 10 walks with a walk length of 80, a window size of 10, and an embedding dimension of 128; we further set p = 1 and q = 0 . 5 in node2vec.

Revised: For DGI, we choose early stopping strategy with a learning rate of 0.001, an embedding dimension of 512.

Original: For GraphSAGE, we train a two-layer model for one epoch, with a learning rate of 0.00001, an embedding dimension of 128, and a batch size of 256. 
Revised: For GraphSAGE, we train a two-layer model for one epoch, with a learning rate of 0.00001, an embedding dimension of 128, and a batch size of 256. [Figure] [Table] [Figure]

Original: Comparing GraphZoom with baseline embedding methods.
Revised: Comparing GraphZoom with baseline embedding methods.

Original: We show the results of GraphZoom with coarsening level varying 1 to 3 for transductive learning and 1 to 2 for inductive learning.
Revised: We show the results of GraphZoom with coarsening level varying 1 to 3 for transductive learning and 1 to 2 for inductive learning.

Original: Results with larger coarsening level are available in Figure 3 (blue curve) and
Revised: Results with larger coarsening level are available in Figure 3 (blue curve) and

Original: the Appendix I. Our results demonstrate that GraphZoom is agnostic to underlying embedding methods and capable of [Table] [Table] [Figure]
Revised: the Appendix I. Our results demonstrate that GraphZoom is agnostic to underlying embedding methods and capable of boosting the accuracy and speed of state-of-the-art unsupervised embedding methods on various datasets.

Original: [Table] [Figure] [Table] DW
Revised: More speciﬁcally, for transductive learning task, GraphZoom improves classiﬁcation accuracy upon both DeepWalk and node2vec by a margin of 8.3%, 19.4%, and 10.4% on Cora, Citeseer, and Pubmed, respectively, while achieving up to 40.8x run-time reduction.

Original: [Figure] boosting the accuracy and speed of state-of-the-art unsupervised embedding methods on various datasets. More speciﬁcally, for transductive learning task, GraphZoom improves classiﬁcation accuracy upon all the baseline embedding methods by a margin of 8.3%, 19.4%, and 10.4% on Cora, Citeseer, and Pubmed, respectively, while achieving up to 40.8x run time reduction.
Revised: In regards to comparing with DGI, GraphZoom achieves comparable or better accuracy with speedup up to 11.2 × .

Original: Similarly, GraphZoom outperforms all the baselines by a margin of 3.4% and 3.3% on PPI and Reddit for inductive learning task, respectively, with speedup up to 7.6x.
Revised: Similarly, GraphZoom outperforms all the baselines by a margin of 3.4% and 3.3% on PPI and Reddit for inductive learning task, respectively, with speedup up to 7.6x.

Original: Our results indicate that reducing graph size while properly retaining the key spectral properties of graph Laplacian and smoothing embeddings will not only boost the embedding speed but also lead to high embedding quality.
Revised: Our results indicate that reducing graph size while properly retaining the key spectral properties of graph Laplacian and smoothing embeddings will not only boost the embedding speed but also lead to high embedding quality.

Original: Comparing GraphZoom with multi-level frameworks.
Revised: Comparing GraphZoom with multi-level frameworks.

Original: As shown in Table 2, HARP only slightly improves and sometimes even worsens the classiﬁcation accuracy while signiﬁcantly increasing the
Revised: As shown in Table 2, HARP only slightly improves and sometimes even worsens the classiﬁcation accuracy while signiﬁcantly increasing the

Original: CPU time.
Revised: CPU time.

Original: Although MILE improves both accuracy and CPU time compared to baseline embedding methods in some cases, the performance of MILE becomes worse with increasing coarsening levels (e.g., the classiﬁcation accuracy of MILE drops from 0.708 to 0.618 on Pubmed dataset with node2vec as the embedding kernel).
Revised: Although MILE improves both accuracy and CPU time compared to baseline embedding methods in some cases, the performance of MILE becomes worse with increasing coarsening levels (e.g., the classiﬁcation accuracy of MILE drops from 0.708 to 0.618 on Pubmed dataset with node2vec as the embedding kernel).

Original: GraphZoom achieves a better accuracy and speedup compared to MILE with the same coarsening level across all datasets.
Revised: GraphZoom achieves a better accuracy and speedup compared to MILE with the same coarsening level across all datasets.

Original: Moreover, when increasing coarsening levels, namely, decreasing number of nodes on the coarsened graph, GraphZoom still produces com- parable or even a better embedding accuracy with much shorter CPU times.
Revised: Moreover, when increasing coarsening levels, namely, decreasing number of nodes on the coarsened graph, GraphZoom still produces comparable or even a better embedding accuracy with much shorter CPU times.

Original: This further conﬁrms
Revised: This further conﬁrms

Original: GraphZoom can retain the key graph structure information to be utilized by underlying embedding models to generate high-quality node embeddings.
Revised: GraphZoom can retain the key graph structure information to be utilized by underlying embedding models to generate high-quality node embeddings.

Original: GraphZoom for large graph embedding.
Revised: GraphZoom for large graph embedding.

Original: To show GraphZoom can signiﬁcantly improve performance and scalability of underlying embedding model on large graph, we test GraphZoom and
Revised: To show GraphZoom can signiﬁcantly improve performance and scalability of underlying embedding model on large graph, we test GraphZoom and

Original: MILE on Friendster dataset, which contains 8 million nodes and 400 million edges, using DeepWalk as the embedding kernel.
Revised: MILE on Friendster dataset, which contains 8 million nodes and 400 million edges, using DeepWalk as the embedding kernel.

Original: As shown in Figure 2, GraphZoom drastically boosts the Micro-F1 score up to 47 .
Revised: As shown in Figure 2, GraphZoom drastically boosts the Micro-F1 score up to 47 .

Original: 6% compared to MILE and 49 .
Revised: 6% compared to MILE and 49 .

Original: 9% compared to DeepWalk with speedup up to 119 . 8 × .
Revised: 9% compared to DeepWalk with speedup up to 119 . 8 × .

Original: When increasing coarsening level, GraphZoom achieves a higher speedup while the embedding accuracy decreases gracefully.
Revised: When increasing coarsening level, GraphZoom achieves a higher speedup while the embedding accuracy decreases gracefully.

Original: 4. A NALYSIS ON G RAPH Z OOM K ERNELS
Revised: 4. A NALYSIS ON G RAPH Z OOM K ERNELS

Original: [Figure]
Revised: [Figure]

Original: MILE MILE coarsening and reﬁnement kernels in MILE, respectively; The blue curve is basically GraphZoom and the yellow one is MILE.
Revised: and the yellow one is MILE.

Original: To study the effectiveness of our proposed GraphZoom kernels separately, we compare each of them against the corresponding kernel in MILE with other kernels ﬁxed.
Revised: To study the effectiveness of our proposed GraphZoom kernels separately, we compare each of them against the corresponding kernel in MILE with other kernels ﬁxed.

Original: As shown in Figure 3, when ﬁxing coarsening kernel and comparing reﬁnement kernel of GraphZoom with that of MILE (shown in purple curve and yellow curve), GraphZoom reﬁnement kernel can improve embedding results upon MILE reﬁnement kernel, especially when the coarsening level is large, which indicates that our proposed graph ﬁlter in reﬁnement kernel can successfully ﬁlter out high frequency noise from graph to improve embedding quality.
Revised: As shown in Figure 3, when ﬁxing coarsening kernel and comparing reﬁnement kernel of GraphZoom with that of MILE (shown in purple curve and yellow curve), GraphZoom reﬁnement kernel can improve embedding results upon MILE reﬁnement kernel, especially when the coarsening level is large, which indicates that our proposed graph ﬁlter in reﬁnement kernel can successfully ﬁlter out high frequency noise from graph to improve embedding quality.

Original: Similarly, when comparing coarsening kernels in GraphZoom and MILE with reﬁnement kernel ﬁxed (shown in light blue curve and yellow curve), GraphZoom coarsening kernel can also improve embedding quality upon MILE coarsening kernel, which shows that our spectral graph coarsening algorithm can indeed retain key graph structure for underlying graph embedding models to exploit.
Revised: Similarly, when comparing coarsening kernels in GraphZoom and MILE with reﬁnement kernel ﬁxed (shown in light blue curve and yellow curve), GraphZoom coarsening kernel can also improve embedding quality upon MILE coarsening kernel, which shows that our spectral graph coarsening algorithm can indeed retain key graph structure for underlying graph embedding models to exploit.

Original: When combining GraphZoom coarsening kernel and reﬁnement kernel (green curve), we can achieve better classiﬁcation accuracy compared with the ones using any kernel in MILE (i.e., light blue curve, purple curve and yellow curve), which means that GraphZoom coarsening kernel and reﬁnement kernel play different roles to boost embedding performance and their combination can further improve embedding result.
Revised: When combining GraphZoom coarsening kernel and reﬁnement kernel (green curve), we can achieve better classiﬁcation accuracy compared with the ones using any kernel in MILE (i.e., light blue curve, purple curve and yellow curve), which means that GraphZoom coarsening kernel and reﬁnement kernel play different roles to boost embedding performance and their combination can further improve embedding result.

Original: Moreover, when adding graph fusion kernel with the combination of GraphZoom coarsening and reﬁnement kernels (blue curve, which is our GraphZoom framework), it improves classiﬁcation accuracy by a large margin, which betokens that graph fusion can properly incorporate both graph topology and node attribute information and lifts the embedding quality of downstream embedding models.
Revised: Moreover, when adding graph fusion kernel with the combination of GraphZoom coarsening and reﬁnement kernels (blue curve, which is our GraphZoom framework), it improves classiﬁcation accuracy by a large margin, which betokens that graph fusion can properly incorporate both graph topology and node attribute information and lifts the embedding quality of downstream embedding models.

Original: Results of each kernel CPU time and speedup comparison are available in Appendix F and Appendix H.
Revised: Results of each kernel CPU time and speedup comparison are available in Appendix F and Appendix H.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this work we propose GraphZoom, a multi-level framework to improve embedding quality and scalability of underlying unsupervised graph embedding techniques.
Revised: In this work we propose GraphZoom, a multi-level framework to improve embedding quality and scalability of underlying unsupervised graph embedding techniques.

Original: GraphZoom encodes graph structure and node attribute in a single graph and exploiting spectral coarsening and reﬁnement methods to remove high frequency noise from the graph.
Revised: GraphZoom encodes graph structure and node attribute in a single graph and exploiting spectral coarsening and reﬁnement methods to remove high frequency noise from the graph.

Original: Experiments show that GraphZoom improves both classiﬁcation accuracy and embedding speed on a number of popular datasets.
Revised: Experiments show that GraphZoom improves both classiﬁcation accuracy and embedding speed on a number of popular datasets.

Original: An interesting direction for future work is to derive a proper way to propagate node labels to the coarsest graph, which would allow GraphZoom to support supervised graph embedding models
Revised: An interesting direction for future work is to derive a proper way to propagate node labels to the coarsest graph, which would allow GraphZoom to support supervised graph embedding models.

Original: A PPENDIX A D ETAILS OF DATASETS
Revised: A PPENDIX A D ETAILS OF DATASETS

Original: Transductive task We follow the experiments setup in Yang et al.
Revised: Transductive task. We follow the experiments setup in Yang et al.

Original: (2016) for three standard citation network benchmark datasets: Cora, Citeseer, and Pubmed.
Revised: (2016) for three standard citation network benchmark datasets: Cora, Citeseer, and Pubmed.

Original: In all these three citation networks, nodes represent documents and edges correspond to citations.
Revised: In all these three citation networks, nodes represent documents and edges correspond to citations.

Original: Each node has a sparse bag-of-word feature vector and a class label.
Revised: Each node has a sparse bag-of-word feature vector and a class label.

Original: We allow only 20 labels per class for training and 1,000 labeled nodes for testing.
Revised: We allow only 20 labels per class for training and 1 , 000 labeled nodes for testing.

Original: In addition, we further evaluate on Friendster dataset (Yang & Leskovec, 2015), which contains 8 million nodes and 400 million edges, with 2.5% of the nodes used for training and 0.3% nodes for testing.
Revised: In addition, we further evaluate on Friendster dataset (Yang & Leskovec, 2015), which contains 8 million nodes and 400 million edges, with 2 . 5% of the nodes used for training and 0 . 3% nodes for testing.

Original: In Friendster, nodes represent users and a pair of nodes are linked if they are friends; each node has a class label but is not associated with a feature vector.
Revised: In Friendster, nodes represent users and a pair of nodes are linked if they are friends; each node has a class label but is not associated with a feature vector.

Original: Inductive task We follow Hamilton et al.
Revised: Inductive task. We follow Hamilton et al.

Original: (2017) for setting up experiments on both protein-protein interaction (PPI)
Revised: (2017) for setting up experiments on both proteinprotein interaction (PPI)

Original: and Reddit dataset.
Revised: and Reddit dataset.

Original: PPI dataset consists of graphs corresponding to human tissues, where nodes are proteins and edges represent interaction effects between proteins.
Revised: PPI dataset consists of graphs corresponding to human tissues, where nodes are proteins and edges represent interaction effects between proteins.

Original: Reddit dataset contains nodes corresponding to users’ posts and two nodes are connected  if the same users comment on both posts.
Revised: Reddit dataset contains nodes corresponding to users’ posts: two nodes are connected through an edge if the same users comment on both posts.

Original: We use 60% nodes for training, 40% for testing on PPI and 65% for training and 35% for testing on Reddit.
Revised: We use 60% nodes for training, 40% for testing on PPI and 65% for training and 35% for testing on Reddit.

Original: A PPENDIX B D ETAILS OF BASELINES
Revised: A PPENDIX B D ETAILS OF BASELINES

Original: DeepWalk ﬁrst generates random walks based on graph structure.
Revised: DeepWalk ﬁrst generates random walks based on graph structure.

Original: Then, walks are treated as sentences in a language model and Skip-Gram model is exploited to obtain node embeddings.
Revised: Then, walks are treated as sentences in a language model and Skip-Gram model is exploited to obtain node embeddings.

Original: node2vec is different from DeepWalk in terms of generating random walks by introducing the return parameter p and the in-out parameter q, which can combine DFS-like and BFS-like neighborhood exploration.
Revised: node2vec is different from DeepWalk in terms of generating random walks by introducing the return parameter p and the in-out parameter q , which can combine DFS-like and BFS-like neighborhood exploration.

Revised: Deep Graph Infomax (DGI) is an unsupervised approach that generates node embeddings by maximizing mutual information between patch representations (local information) and corresponding high-level summaries (global information) of graphs .

Original: GraphSAGE embeds nodes in an inductive way by learning an aggregation function that aggregates node features to obtain embeddings.
Revised: GraphSAGE embeds nodes in an inductive way by learning an aggregation function that aggregates node features to obtain embeddings.

Original: GraphSAGE supports four different aggregation functions:
Revised: GraphSAGE supports four different aggregation functions:

Original: GraphSAGE-GCN, GraphSAGE-mean, GraphSAGE-LSTM and GraphSAGE-pool.
Revised: GraphSAGE-GCN, GraphSAGE-mean, GraphSAGE-LSTM and GraphSAGE-pool.

Original: HARP coarsens the original graph into severalcoarsening levels and apply underlying embedding model to train the coarsened graph at eachcoarsening level sequentially to obtain the ﬁnal embeddings on original graph.
Revised: HARP coarsens the original graph into several levels and apply underlying embedding model to train the coarsened graph at each level sequentially to obtain the ﬁnal embeddings on original graph.

Original: Since  coarsening level is ﬁxed in their implementation, we run HARP in our experiments without changing the coarsening level.
Revised: Since the coarsening level is ﬁxed in their implementation, we run HARP in our experiments without changing the coarsening level.

Original: MILE is the state-of-the-art multi-level unsupervised graph embedding framework andvery similar to our GraphZoom framework since it also contains graph coarsening and embedding reﬁnement kernels.
Revised: MILE is the state-of-the-art multi-level unsupervised graph embedding framework and similar to our GraphZoom framework since it also contains graph coarsening and embedding reﬁnement kernels.

Original: More speciﬁcally, MILE ﬁrst uses its  coarsening kernel to reduce the graph size and trains underlying unsupervised graph embedding model on coarsest graph.
Revised: More speciﬁcally, MILE ﬁrst uses its heuristic-based coarsening kernel to reduce the graph size and trains underlying unsupervised graph embedding model on coarsest graph.

Original: Then, its reﬁnement kernel employs Graph Convolutional Network (GCN) to reﬁne embeddings back to the original graph.
Revised: Then, its reﬁnement kernel employs Graph Convolutional Network (GCN) to reﬁne embeddings on the original graph.

Original: We compare GraphZoom with MILE on various datasets, including Friendster that contains 8 million nodes and 4 million edges (shown in Table 2 and Figure 2).
Revised: We compare GraphZoom with MILE on various datasets, including Friendster that contains 8 million nodes and 400 million edges (shown in Table 2 and Figure 2).

Original: Moreover, we further compare each kernel in GraphZoom and MILE in Figure 3.
Revised: Moreover, we further compare each kernel in GraphZoom and MILE in Figure 3.

Original: A PPENDIX C G RAPH SIZE AT DIFFERENT COARSENING LEVEL
Revised: A PPENDIX C G RAPH SIZE AT DIFFERENT COARSENING LEVEL

Original: The details of graph size at different coarsening level on all six datasets are shown in Table 4.Dataset GZoomGZoom- GZoomGZoom-
Revised: The details of graph size at different coarsening level on all six datasets are shown in Table 4.

Original: GZoomGZoom-

Original: Cora 2, 1,

Original: Citeseer 3, 1, Pubmed 19, 7, 3, 1, PPI 14, 5, 1, Reddit 232, 84, 30, 11, 4, 2,
Revised: [Table]

Original: Friendster 7,944, 2,734, 1,048, 409, 134, 44,

Original: A PPENDIX D
Revised: A PPENDIX D

Original: G RAPH Z OOM ALGORITHM
Revised: G RAPH Z OOM ALGORITHM

Original: Algorithm 1: GraphZoom algorithm
Revised: Algorithm 1: GraphZoom algorithm

Original: Input: Adjacency matrix A topo ∈ R N × N ; node feature matrix X ∈ R N × K ; base embedding function l ( · ) ; coarsening level m
Revised: Input: Adjacency matrix A topo ∈ R N × N ; node feature matrix X ∈ R N × K ; base embedding function l ( · ) ; coarsening level m

Original: Output: Node embedding matrix E ∈ R N × D 1 A 0 = graph fusion ( A topo , X ); 2 for i = 1 ...m do
Revised: Output: Node embedding matrix E ∈ R N × D 1 A 0 = graph fusion ( A topo , X ); 2 for i = 1 ...m do

Original: A i , H ii − 1 = spectral coarsening ( A i − 1 ) ; 4 end 5 E L = l ( A m ) ; 6 for i = m... 1 do ˆ E i − 

Revised: A i , H ii − 1 = spectral coarsening ( A i − 1 ) ; 4 end 5 E L = l ( A m ) ; 6 for i = m... 1 do ˆ E i −

Original: =
Revised: =

Original: ( H ii − 1 ) T E i ;
Revised: ( H ii − 1 ) T E i ;

Original: E i − 1 =
Revised: E i − 1 =

Original: refinement
Revised: refinement

Original: ( ˆ E i − 1 ) ; 9 end 10 E = E 0 ;
Revised: ( ˆ E i − 1 ) ; 9 end 10 E = E 0 ;

Original: A PPENDIX E
Revised: A PPENDIX E

Original: S PECTRAL COARSENING
Revised: S PECTRAL COARSENING

Original: Notice that  mapping operator H i +1 i ∈ { 0 , 1 }|V i +1 | × |V i | is acoarsening matrix containing onlyand 1.
Revised: Note that the mapping operator H i +1 i ∈ { 0 , 1 }|V i +1 | × |V i | is a matrix containing only 0s and 1s.

Original: It has following properties:
Revised: It has following properties:

Original: • Row (column) index of H
Revised: • The row (column) index of H

Original: i [-+
-] i correspond to the node index in graph[-G i +
-]
Revised: i [-+
-]+1 i corresponds to the node index in graph[-G i +
-]

Original:  ( G i ).
Revised: G i +1 ( G i ).

Original: • It is a surjective mapping of the node set, where ( H i +1 i ) p,q = 1 if node q in graph G i is aggregated to super-node p in graph G i +1 , and ( H i +1 i ) p (cid:48) ,q = 0 for all nodes p (cid:48) ∈ { v ∈ V
Revised: • It is a surjective mapping of the node set, where ( H i +1 i ) p,q = 1 if node q in graph G i is aggregated to super-node p in graph G i +1 , and ( H i +1 i ) p (cid:48) ,q = 0 for all nodes p (cid:48) ∈ { v ∈ V

Original: i +1 : v (cid:54) = p } .
Revised: i +1 : v (cid:54) = p } .

Original: • It is a locality preserving operator, where the subgraph of G i induced by the non-zero entries of ( H i +1 i ) p, : is connected for each p ∈ V
Revised: • It is a locality-preserving operator, where the coarsened version of G i induced by the nonzero entries of ( H i +1 i ) p, : is connected for each p ∈ V

Original: i +1 .
Revised: i +1 .

Original: Input: Adjacency matrix A
Revised: Input: Adjacency matrix A

Original: i ∈ R |V
Revised: i ∈ R |V

Original: i | × |V i |
Revised: i | × |V i |

Original: Output: Adjacency matrix A
Revised: Output: Adjacency matrix A

Original: i +1 ∈ R |V
Revised: i +1 ∈ R |V

Original: i +1 | × |V
Revised: i +1 | × |V

Original: i +1 | of the reduced graph G i +1 , mapping operator H
Revised: i +1 | of the reduced graph G i +1 , mapping operator H

Original: i +1 i ∈
Revised: i +1 i ∈

Original: R |V
Revised: R |V

Original: i +1 | × |V i | 11 n =
Revised: i +1 | × |V i | 11 n =

Original: |V i | , n c = n ; 12 [graph reduction ratio] γ max = 1 . 8 , δ = 0 . 9 ; 13 for each edge ( p, q )
Revised: |V i | , n c = n ; 12 [graph reduction ratio] γ max = 1 . 8 , δ = 0 . 9 ; 13 for each edge ( p, q )

Original: ∈
Revised: ∈

Original: E i do [spectral node afﬁnity set] C ← a p,q deﬁned in Eq. 2 ; 15 end 16 for each node p ∈ V i do d ( p ) =
Revised: E i do [spectral node afﬁnity set] C ← a p,q deﬁned in Eq. 2 ; 15 end 16 for each node p ∈ V i do d ( p ) =

Original: (cid:12)(cid:12)(cid:12) ( A i ) p, : (cid:12)(cid:12)(cid:12) , d m ( p ) =
Revised: (cid:12)(cid:12)(cid:12) ( A i ) p, : (cid:12)(cid:12)(cid:12) , d m ( p ) =

Original: median (cid:16)(cid:12)(cid:12)(cid:12)
Revised: median (cid:16)(cid:12)(cid:12)(cid:12)

Original: ( A i ) q, : (cid:12)(cid:12)(cid:12) for all q ∈ { q | ( p, q )
Revised: ( A i ) q, : (cid:12)(cid:12)(cid:12) for all q ∈ { q | ( p, q )

Original: ∈
Revised: ∈

Original: E i } (cid:17) ; if d ( p ) ≥ 8 · d m ( p ) then [node aggregation ﬂag] z ( p )
Revised: E i } (cid:17) ; if d ( p ) ≥ 8 · d m ( p ) then [node aggregation ﬂag] z ( p )

Original: = 0 ; else [node aggregation ﬂag]
Revised: = 0 ; else [node aggregation ﬂag]

Original: z ( p )
Revised: z ( p )

Original: =
Revised: =

Original: − 1 ; end 23 end 24 γ = 1 ; 25 while γ < γ max do
Revised: − 1 ; end 23 end 24 γ = 1 ; 25 while γ < γ max do

Original: S = ∅ , U = ∅ ; [unaggregated node set]
Revised: S = ∅ , U = ∅ ; [unaggregated node set]

Original: U ← p ∈ { p | z ( p ) =
Revised: U ← p ∈ { p | z ( p ) =

Original: = − 1 ∀ p ∈
Revised: = − 1 ∀ p ∈

Original: V i } ;
Revised: V i } ;

Original: S ← p ∈ { p | a p,q ≥ δ · max s (cid:54) = p,q (cid:18) max (
Revised: S ← p ∈ { p | a p,q ≥ δ · max s (cid:54) = p,q (cid:18) max (

Original: p,s ) ∈
Revised: p,s ) ∈

Original: E i (cid:0) a p,s (cid:1) , max ( q,s ) ∈
Revised: E i (cid:0) a p,s (cid:1) , max ( q,s ) ∈

Original: E i (cid:0) a q,s (cid:1)(cid:19)∀ p ∈ V i } ; for each node p in S ∩ U do if z ( p ) =
Revised: E i (cid:0) a q,s (cid:1)(cid:19) ; for each node p in S ∩ U do if z ( p ) =

Original: = − 1 then q = arg max ( p,q ) ∈
Revised: = − 1 then q = arg max ( p, ) ∈

Original: E i (cid:18) a p,q (cid:19) ;  if z ( q ) =
Revised: E i (cid:18) a p,q (cid:19) ;  if z ( q ) =

Original: = − 1 then z ( q ) = 0 , z ( p ) =
Revised: = − 1 then z ( q ) = 0 , z ( p ) =

Original: q , ˆ q = q ; else if z ( q ) =
Revised: q , ˆ q = q ; else if z ( q ) =

Original: = 0 then z
Revised: = 0 then z

Original: ( p ) =
Revised: ( p ) =

Original: q , ˆ q = q ; else z ( p) =
Revised: q , ˆ q = q ; else z ( p

Original: z ( q ) , ˆ q = z ( q ) ; end update smoothed vectors in T with x ( k ) p 
Revised: ; end update smoothed vectors in T with x ( k ) p =

Original: = x ( k ) ˆ q for k = 1 , · · · , t nc = n c − 1 ; end end γ = n/n c , δ = 0 . 7 · δ ; z ( p ) =
Revised: ) ˆ q for k = 1 , · · · , t n

Original:  p for node p∈ p | z ( p ) =
Revised: ; end end ; z ( p p for node p

Original: = 0 } 45 end 46 form H i +1 i and A
Revised: } 45 end 46 form H i +1 i and A

Original: i +1 based on z
Revised: i +1 based on z

Original: A PPENDIX F CPU TIME OF EACH G RAPH Z OOM KERNEL
Revised: A PPENDIX F CPU TIME OF EACH G RAPH Z OOM KERNEL

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: FusionCoarseningEmbeddingRefinement

Original: [Figure]
Revised: [Figure]

Original: m= m= m= m= m=5 m-th coarsening level 

Revised: m= m= m= m= m=5 m-th coarsening level

Original: S i n g l e
Revised: S i n g l e

Original: C P U t i m e ( s ) 827.
Revised: C P U t i m e ( s ) 827.

Original: CPU time of GraphZoom kernels on Reddit
Revised: [Figure] GraphZoom kernels [Figure]

Original: FusionCoarseningEmbeddingRefinement
Revised: FusionCoarseningEmbeddingRefinement

Original: [Figure]
Revised: [Figure]

Original: As shown in Figure 4 (note that the y axis is in logarithmic scale), the GraphZoom embedding kernel dominates the total CPU time, which can be  effectively reduced when increaing coarsening level L .
Revised: As shown in Figure 4 (note that the y axis is in logarithmic scale), the GraphZoom embedding kernel dominates the total CPU time, which can be more effectively reduced with a greater coarsening level L .

Original: All other kernels in GraphZoom are very efﬁcient, especially when the coarsening level is small. Thus, the GraphZoom framework can drastically reduce the total run time to generate ﬁnal node embeddings.
Revised: All other kernels in GraphZoom are very efﬁcient, which enable the GraphZoom framework to drastically reduce the total graph embedding time.

Original: A PPENDIX G G RAPH FILTERS AND LAPLACIAN EIGENVALUES
Revised: A PPENDIX G G RAPH FILTERS AND L APLACIAN EIGENVALUES

Original: Figure 5a shows the original distribution of graph Laplacian eigenvalues, which  can be interpreted as frequencies in graph spectral domain (small eigenvalue means low frequency).
Revised: Figure 5a shows the original distribution of graph Laplacian eigenvalues which also can be interpreted as frequencies in graph spectral domain (smaller eigenvalue means lower frequency).

Original: Since the graph ﬁlter, shown in Figure 5e, has large magnitude on both lateral sides and has small magnitude in the middle, it can be considered as band-stop ﬁlter that passes all frequencies with the exception of those within the middle stop band which are greatly attenuated.
Revised: The proposed graph ﬁlter for embedding reﬁnement (as shown in Figure 5e) can be considered as a bandstop ﬁlter that passes all frequencies with the exception of those within the middle stop band that is greatly attenuated.

Original: The band-stop ﬁlter is therefore unable to ﬁlter out high frequency noise from the graph.
Revised: Therefore, the band-stop ﬁlter may not be very effective for removing highfrequency noises from the graph signals.

Original: However, if we add self-loops  in the graph by ˜
Revised: Fortunately, it has been shown that by adding self-loops to each node in the graph as follows ˜

Original: A =
Revised: A =

Original: A + σI (shown in Figure 5b, 5c, 5d, where σ = 0 .
Revised: A + σI (shown in Figure 5b, 5c, 5d, where σ = 0 .

Original: 5 , 1 . 0 , 2 . 0 ), the distribution of  eigenvalues will be squeezed toward small values (proved in (Maehara, 2019)).
Revised: 5 , 1 . 0 , 2 . 0 ), the distribution of Laplacian eigenvalues can be squeezed to the left (towards zero) (Maehara, 2019).

Original: In other words, the maximum eigenvalues are effectively reduced by adding self-loop. If we properly set σ such that large eigenvalues  lie in the stop bandregion (e.g., σ = 1 . 0 , 2 . 0 shown in Figure 5c and 5d),then the high frequency components (corresponding to large eigenvalues) will be  ﬁltered out while low frequency components are retained, which makes the graph ﬁlter become a low-pass ﬁlter, shown in Figure 5f.
Revised: By properly choosing σ such that large eigenvalues will mostly lie in the stop band (e.g., σ = 1 . 0 , 2 .shown in Figure 5c and 5d), the graph ﬁlter will be able to effectively ﬁltered out high-frequency components (corresponding to high eigenvalues) while retaining low-frequency components, which is similar to a low-pass graph ﬁlter as shown in Figure 5f.

Original: It is worth noting that ifthe σ is too large, then the maximum eigenvalue will be squeezed to 0, which makes the graph ﬁlter unable to remove high frequence components. Thus, setting an appropriate σ is critical to make the proposed graph ﬁlter remove high frequency noise.
Revised: It is worth noting that if σ is too large, then most eigenvalues will be very close to zero, which makes the graph ﬁlter less effective for removing noises.

Original: We choose σ = 2 . 0 for all our experiments.
Revised: In this work, we choose σ = 2 . 0 for all our experiments.

Original: A PPENDIX H S PEEDUP OF G RAPH Z OOM KERNELS COMPARED TO MILE
Revised: A PPENDIX H S PEEDUP OF G RAPH Z OOM KERNELS COMPARED TO MILE

Original: As shown in Figure 6, the combination of GraphZoom coarsening and reﬁnement kernels can  achieve the largest speedup (green curve). When adding GraphZoom fusion kernel (blue curve), it reduces the speedup by a small margin, which shows the trade-off between embedding quality and 
Revised: As shown in Figure 6, the combination of GraphZoom coarsening and reﬁnement kernels can always achieve the greatest speedups (green curves); adding GraphZoom fusion kernel (blue curves) will lower the speedups by a small margin but further boost the embedding quality, showing a clear tradeoff between embedding quality and runtime efﬁciency: to achieve the highest graph embedding quality, the graph fusion kernel should always be included.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure]
Revised: [Figure]

Original: speedup: if the quality is preferred, GraphZoom fusion kernel should be used; if speedup is more important, then fusion kernel should be skipped.

Original: A PPENDIX I M ORE RESULTS ON PPI DATASETS
Revised: A PPENDIX I M ORE RESULTS ON PPI DATASETS

Original: As shown in Figure 7,when increasing coarsenine level, theGraphZoom embedding results  are always better than  GraphSAGE with different aggregation functions.
Revised: As shown in Figure 7, the embedding results by GraphZoom with increasing number of coarsening levels are still better than the ones by GraphSAGE with different aggregation functions.

Original: It is worth noting that when
Revised: It is worth noting that although the level- 5 (coarsened) graph contains only 120 nodes (as shown in C), GraphZoom embedding results can still beat the GraphSAGE baseline obtained with the original graph containing 14 , 755 nodes.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: coarsening level is 5, the coarest graph contains only 120 nodes (shown in C), GraphZoom can still beat the GraphSAGE baseline that trains on the original graph with 14,755 nodes.
Revised: [Figure] [Figure] [Figure]