Original: G RAPH

Original: A UTOENCODER

Original:  FOR GRAPH COMPRESSION AND REPRESENTATION LEARNING
Revised: G RAPH A UTOENCODER FOR GRAPH COMPRESSION AND REPRESENTATION LEARNING

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: We consider the problem of graph data compression and representation.
Revised: We consider the problem of graph data compression and representation.

Original: Recent developments in graph neural networks (GNNs) focus on generalizing convolutional neural networks (CNNs) to graph data, which includes redesigning convolution and pooling operations for graphs.
Revised: Recent developments in graph neural networks (GNNs) focus on generalizing convolutional neural networks (CNNs) to graph data, which includes redesigning convolution and pooling operations for graphs.

Original: However, few methods focus on effective graph compression to obtain a smaller graph, which can reconstruct the original full graph with less storage and can provide useful latent representations to improve downstream task performance.
Revised: However, few methods focus on effective graph compression to obtain a smaller graph, which can reconstruct the original full graph with less storage and can provide useful latent representations to improve downstream task performance.

Original: To ﬁll this gap, we propose Multi-kernel Inductive Attention Graph Autoencoder (MIAGAE), which, instead of compressing nodes/edges separately, utilizes the node similarity and graph structure to compress all nodes and edges as a whole.
Revised: To ﬁll this gap, we propose Multi-kernel Inductive Attention Graph Autoencoder (MIAGAE), which, instead of compressing nodes/edges separately, utilizes the node similarity and graph structure to compress all nodes and edges as a whole.

Original: Similarity attention graph pooling selects the most representative nodes with the most information by using the similarity and topology among nodes.
Revised: Similarity attention graph pooling selects the most representative nodes with the most information by using the similarity and topology among nodes.

Original: Our multi-kernel Inductive-Convolution layer can focus on different aspects and learn more general node representations in evolving graphs.
Revised: Our multi-kernel Inductive-Convolution layer can focus on different aspects and learn more general node representations in evolving graphs.

Original: We demonstrate that MIAGAE outperforms state-of-the-art methods for graph compression and few-shot graph classiﬁcation, with superior graph representation learning.
Revised: We demonstrate that MIAGAE outperforms state-of-the-art methods for graph compression and few-shot graph classiﬁcation, with superior graph representation learning.

Revised: (The code is released at this URL † )

Original:  I NTRODUCTION AND RELATED WORKSGraph
Revised:  I NTRODUCTION AND RELATED WORKS

Original:  Neural Networks (GNNs) (Scarselli et al., 2009; Kipf & Welling, 2016a; Gilmer et al., 2017) extend the idea of using convolution on grid-structure data (images) in Convolutional neural networks (CNNs) (LeCun et al., 2012) to graph data.
Revised: Graph Neural Networks (GNNs) (Scarselli et al., 2009; Kipf & Welling, 2016a; Gilmer et al., 2017) extend the idea of using convolution on grid-structure data (images) in Convolutional neural networks (CNNs) (LeCun et al., 2012) to graph data.

Original: They achieve great success on many tasks, such as node classiﬁcation and link prediction (Veliˇckovi´c et al., 2017; Hamilton et al., 2017a).
Revised: They achieve great success on many tasks, such as node classiﬁcation and link prediction (Veli ˇ ckovi ´ c et al., 2017; Hamilton et al., 2017a).

Original: The size of graphs presents a big obstacle to understanding the essential information they contain.
Revised: The size of graphs presents a big obstacle to understanding the essential information they contain.

Original: Graph compression (Navlakha et al., 2008) can solve this problem and be used to improve visualization, to understand the graph’s high-level structure, or as a pre-processing step for other data mining methods (Zhou, 2015).
Revised: Graph compression (Navlakha et al., 2008) can solve this problem and be used to improve visualization, to understand the graph’s high-level structure, or as a pre-processing step for other data mining methods (Zhou, 2015).

Original: However, few methods focus on graph compression and reconstruction with GNNs.
Revised: However, few methods focus on graph compression and reconstruction with GNNs.

Original: Although CNN autoencoders are an efﬁcient compression framework for images, research on Graph autoencoder (GAE) structure methods is rare.
Revised: Although CNN autoencoders are an efﬁcient compression framework for images, research on Graph autoencoder (GAE) structure methods is rare.

Original: Variational Graph Autoencoder (Kipf & Welling, 2016b) focuses on link representation while it cannot obtain a compressed graph.
Revised: Variational Graph Autoencoder (Kipf & Welling, 2016b) focuses on link representation while it cannot obtain a compressed graph.

Original: Graph Unet (Gao & Ji, 2019) achieves a graph autoencoder structure with gPool but needs extra skip feature connections between encoder and decoder, which have high storage cost.
Revised: Graph Unet (Gao & Ji, 2019) achieves a graph autoencoder structure with gPool but needs extra skip feature connections between encoder and decoder, which have high storage cost.

Original: Here, we ﬁll this gap and propose Multi-kernel Inductive
Revised: Here, we ﬁll this gap and propose Multi-kernel Inductive

Original: Attention Graph Autoencoder (MIAGAE), a GAE structure with GNN for graph compression and graph representation learning (Hamilton et al., 2017b; Goyal & Ferrara, 2018)
Revised: Attention Graph Autoencoder (MIAGAE), a GAE structure with GNN for graph compression and graph representation learning (Hamilton et al., 2017b; Goyal & Ferrara, 2018)

Original: (Fig. 1).
Revised: (Fig. 1).

Original: MIAGAE can be used in graph compression and latent graph inference, high-level structure mining, and other unsupervised learning or self-supervised learning (Table 1).
Revised: MIAGAE can be used in graph compression and latent graph inference, high-level structure mining, and other unsupervised learning or self-supervised learning (Table 1).

Original: The biggest challenge for GAE design is graph pooling (Dhillon et al., 2007; Defferrard et al., 2016).
Revised: The biggest challenge for GAE design is graph pooling (Dhillon et al., 2007; Defferrard et al., 2016).

Original: DiffPool (Ying et al., 2019) is a differentiable pooling method by learning an assignment matrix.
Revised: DiffPool (Ying et al., 2019) is a differentiable pooling method by learning an assignment matrix.

Original: SAGPool (Lee et al., 2019) uses self-attention to achieve SOTA performance on various tasks.
Revised: SAGPool (Lee et al., 2019) uses self-attention to achieve SOTA performance on various tasks.

Original: However, these pooling methods need extra trainable parameters and are not designed for graph compression tasks.
Revised: However, these pooling methods need extra trainable parameters and are not designed for graph compression tasks.

Original: Our contributions are: We propose Multi-kernel Inductive Attention Graph Autoencoder (MIAGAE), with (1) Similarity Attention Graph Pooling (SimAGPool) keeps representative nodes with a high contribution for graph compression and needs no extra trainable parameter.
Revised: Our contributions are: We propose Multi-kernel Inductive Attention Graph Autoencoder (MIAGAE), with (1) Similarity Attention Graph Pooling (SimAGPool) keeps representative nodes with a high contribution for graph compression and needs no extra trainable parameter.

Original: (2) Multi-kernel Inductive
Revised: (2) Multi-kernel Inductive

Original: Graph convolution layer (MI-Conv) to suitable for learning in evolving graphs; (3) State-of-the-art graph compression and representation learning performance, with best storage-efﬁcient; (4) A new image to scene graph pipeline  and synthesized scene graph dataset, Img2SceneGraph-ACSG 1 , which allow rapid idea prototyping for graph compression and representation learning [Table]
Revised: Graph convolution layer (MI-Conv) to suitable for learning in evolving graphs; (3) State-of-the-art graph compression and representation learning performance, with best storage-efﬁcient; (4) A new image to scene graph pipeline * and synthesized scene graph dataset, Img2SceneGraph-ACSG, which allow rapid idea prototyping for graph compression and representation learning.

Original: [Table]
Revised: [Table]

Original:  M ULTI -K ERNEL I NDUCTIVE A TTENTION G RAPH A UTOENCODER 2. M ULTI - KERNEL I NDUCTIVE C ONVOLUTIONAL LAYER
Revised:  M ULTI -K ERNEL I NDUCTIVE A TTENTION G RAPH A UTOENCODER 2. M ULTI - KERNEL I NDUCTIVE C ONVOLUTIONAL LAYER

Original: Our MIAGAE consists of an encoder network E and a decoder network D (Fig. 1).
Revised: Our MIAGAE consists of an encoder network E and a decoder network D (Fig. 1).

Original: To obtain graph compression and reconstruction ability, E ﬁrst learns to eliminate graph nodes and edges to get a compressed smaller graph, then D learns to add new nodes and reconstruct the original graph.
Revised: To obtain graph compression and reconstruction ability, E ﬁrst learns to eliminate graph nodes and edges to get a compressed smaller graph, then D learns to add new nodes and reconstruct the original graph.

Original: Since, during learning, compressed graphs assume dynamic node and edge numbers, we use inductive GCN layers (Hamilton et al., 2017a) instead of transductive layers (Kipf & Welling, 2016a).
Revised: Since, during learning, compressed graphs assume dynamic node and edge numbers, we use inductive GCN layers (Hamilton et al., 2017a) instead of transductive layers (Kipf & Welling, 2016a).

Original: On the one hand, the node representations are inﬂuenced when adding and eliminating nodes, which makes it hard to learn individual representations for each node; on the other hand, learning an inductive method to summarize the representation of new nodes by aggregating their neighbors’ feature is more general and easier to generalize across graphs and even datasets (Sec. 3.2).
Revised: On the one hand, the node representations are inﬂuenced when adding and eliminating nodes, which makes it hard to learn individual representations for each node; on the other hand, learning an inductive method to summarize the representation of new nodes by aggregating their neighbors’ feature is more general and easier to generalize across graphs and even datasets (Sec. 3.2).

Original: Our single-kernel Inductive-Conv layer is inspired by GraphSAGE (Hamilton et al., 2017a), which is:
Revised: Our single-kernel Inductive-Conv layer is inspired by GraphSAGE (Hamilton et al., 2017a), which is:

Original: [Equation]
Revised: [Equation]

Original: where mean is the aggregation function, f ik denotes the feature of node v i in layer k , W 1 and
Revised: where mean is the aggregation function, f ik denotes the feature of node v i in layer k , W 1 and

Original: W 2 denote the shared linear transformation parameter for center node v i and neighbor node v j respectively, N ( i ) denotes the neighboring node sets connected to node i .
Revised: W 2 denote the shared linear transformation parameter for center node v i and neighbor node v j respectively, N ( i ) denotes the neighboring node sets connected to node i .

Original: Inspired by CNN layers in the image domain, which have multiple convolution kernels, and each kernel may extract different aspects of a feature, we propose the Multi-kernel Inductive-Conv layer (MI-Conv).
Revised: Inspired by CNN layers in the image domain, which have multiple convolution kernels, and each kernel may extract different aspects of a feature, we propose the Multi-kernel Inductive-Conv layer (MI-Conv).

Original: Instead of using a single set of transformation weights W 1 and W 2 , MI-Conv uses multiple kernels, and each kernel has its corresponding W 1 and W 2 .
Revised: Instead of using a single set of transformation weights W 1 and W 2 , MI-Conv uses multiple kernels, and each kernel has its corresponding W 1 and W 2 .

Original: Features extracted by the ﬁrst kernel and ﬁnal features aggregating all m kernels become:
Revised: Features extracted by the ﬁrst kernel and ﬁnal features aggregating all m kernels become:

Original: [Equation]
Revised: [Equation]

Original: where m is the number of kernels; σ represents a non-linear activation function (we use ReLU);
Revised: where m is the number of kernels; σ represents a non-linear activation function (we use ReLU);

Original: Aggre represents an aggregate function to combine the multi-kernel results (we use addition).
Revised: Aggre represents an aggregate function to combine the multi-kernel results (we use addition)

Original:  2. S IMILARITY A TTENTION G RAPH P OOLING
Revised: . 2. S IMILARITY A TTENTION G RAPH P OOLING

Original: We propose a Similarity Attention Graph Pooling (SimAGPool) layer (Fig. 2) to achieve downsampling on graph data and to adaptively select a subset of nodes/edges to form a new but smaller graph through E .
Revised: We propose a Similarity Attention Graph Pooling (SimAGPool) layer (Fig. 2) to achieve downsampling on graph data and to adaptively select a subset of nodes/edges to form a new but smaller graph through E .

Original: To this end, we deﬁne Representativeness and Contribution Score (RCS) to help SimAGPool select the most representative (higher RCS) nodes which contain the majority of the graph’s information, while eliminating the less representative (lower RCS) nodes and their corresponding edges.
Revised: To this end, we deﬁne Representativeness and Contribution Score (RCS) to help SimAGPool select the most representative (higher RCS) nodes which contain the majority of the graph’s information, while eliminating the less representative (lower RCS) nodes and their corresponding edges.

Original: RCS is related to both the similarity between node features and the structure of the graph.
Revised: RCS is related to both the similarity between node features and the structure of the graph.

Original: To calculate RCS, we ﬁrst calculate the inter product v Ti v j as similarity score, which captures the similarity of node features between nodes n i and n j .
Revised: To calculate RCS, we ﬁrst calculate the inter product v Ti v j as similarity score, which captures the similarity of node features between nodes n i and n j .

Original: For each node n i , the value of RCS is the sum of all similarity scores between n i and its neighbor nodes n j , if there exist a directed edge e ij form n i to n j .
Revised: For each node n i , the value of RCS is the sum of all similarity scores between n i and its neighbor nodes n j , if there exist a directed edge e ij form n i to n j .

Original: [Equation]
Revised: [Equation]

Original: The intuition behind SimAGPool and RCS is to keep the nodes with majority graph information, according to two properties: (1) More representative: higher RCS means node n i is more similar to all the neighbors which represent the majority feature representation in a local subgraph.
Revised: The intuition behind SimAGPool and RCS is to keep the nodes with majority graph information, according to two properties: (1) More representative: higher RCS means node n i is more similar to all the neighbors which represent the majority feature representation in a local subgraph.

Original: (2) More contribution: the higher the RCS that n i has, the more information n i contributes to its neighbor nodes during message passing in MI-Conv through directed edges e ij .
Revised: (2) More contribution: the higher the RCS that n i has, the more information n i contributes to its neighbor nodes during message passing in MI-Conv through directed edges e ij .

Original: The pooling ratio p ∈ (0; 1] is a hyperparameter that determines the number of nodes to keep (keep top p percent of nodes by RCS).
Revised: The pooling ratio p ∈ (0; 1] is a hyperparameter that determines the number of nodes to keep (keep top p percent of nodes by RCS).

Original: Un-Pooling layer forms D , aiming to reconstruct the original graph given the connection information of missing nodes.
Revised: Un-Pooling layer forms D , aiming to reconstruct the original graph given the connection information of missing nodes.

Original: D and E have a symmetric structure where the number of Inductive Un-pooling layers is the same as the number of SimAGPool layers.
Revised: D and E have a symmetric structure where the number of Inductive Un-pooling layers is the same as the number of SimAGPool layers.

Original: The inductive Un-pooling layer has two inputs: the output graph from the previous layer and the edges information of new nodes, which is the same as the eliminated edges in the corresponding pooling layer in E .
Revised: The inductive Un-pooling layer has two inputs: the output graph from the previous layer and the edges information of new nodes, which is the same as the eliminated edges in the corresponding pooling layer in E .

Original: The inductive Un-pooling layer has the same parameter settings as the Inductive-Conv layer.
Revised: The inductive Un-pooling layer has the same parameter settings as the Inductive-Conv layer.

Original: (More architecture details in Appendix).
Revised: (More architecture details in Appendix).

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: We evaluate our SimAGPool and MI-Conv layers based on the MIAGAE proposed in Sec.
Revised: We evaluate our SimAGPool and MI-Conv layers based on the MIAGAE proposed in Sec.

Original: 2 on two tasks: (1) Graph compression and reconstruction; (2) Few-shot graph classiﬁcation to test graph representation learning performance.
Revised: 2 on two tasks: (1) Graph compression and reconstruction; (2) Few-shot graph classiﬁcation to test graph representation learning performance.

Original: We compare our networks with previous state-of-the-art GAE models and graph pooling methods that GAE models can use.
Revised: We compare our networks with previous state-of-the-art GAE models and graph pooling methods that GAE models can use.

Original: Table.
Revised: Table.

Original: 2 summarizes the datasets used for envaluation: COLORS-3 (Knyazev et al., 2019), FRANKENSTEIN (Orsini et al., 2015),
Revised: 2 summarizes the datasets used for envaluation: COLORS-3 (Knyazev et al., 2019), FRANKENSTEIN (Orsini et al., 2015),

Original: Visual Genome (kri) and Img2SceneGraph-ACSG which is a dataset we contribute.
Revised: Visual Genome (kri) and Img2SceneGraph-ACSG which is a dataset we contribute.

Original: (More details in Appendix).
Revised: (More details in Appendix).

Original: [Table]
Revised: [Table]

Original: 3. G RAPH COMPRESSION AND RECONSTRUCTION
Revised: 3. G RAPH COMPRESSION AND RECONSTRUCTION

Original: We use the graphs from all four datasets in Table.
Revised: We use the graphs from all four datasets in Table.

Original: 2 to evaluate graph compression and reconstruction performance.
Revised: 2 to evaluate graph compression and reconstruction performance.

Original: Table.
Revised: Table.

Original: 3 compares our MIAGAE and ﬁve baseline methods on three main dimensions:
Revised: 3 compares our MIAGAE and ﬁve baseline methods on three main dimensions:

Original: Compression ratio represents the ratio between the uncompressed size and compressed size.
Revised: Compression ratio represents the ratio between the uncompressed size and compressed size.

Original: A higher ratio represents a more powerful compression ability.
Revised: A higher ratio represents a more powerful compression ability.

Original: MSE represents the reconstruction mean square error between the original graph and the reconstructed graph by GAE.
Revised: MSE represents the reconstruction mean square error between the original graph and the reconstructed graph by GAE.

Original: A smaller value means less information loss during compression.
Revised: A smaller value means less information loss during compression.

Original: Storage represents the storage size that reconstruction needs (except for the latent space, which is the same for all methods), including the decoder network’s size and extra information (skip connection features, edge information); smaller is better.
Revised: Storage represents the storage size that reconstruction needs (except for the latent space, which is the same for all methods), including the decoder network’s size and extra information (skip connection features, edge information); smaller is better.

Original: Compared with the baseline method Graph Unet under the same compression ratio setting, we obtain better reconstruction MSE while using much less storage for reconstruction, likely mostly because we do not need skip connection feature communication between E and D .
Revised: Compared with the baseline method Graph Unet under the same compression ratio setting, we obtain better reconstruction MSE while using much less storage for reconstruction, likely mostly because we do not need skip connection feature communication between E and D .

Original: Compared with the other four baselines (which can be treated as ablation studies), our MIAGAE outperforms all of them in terms of requiring the smallest storage; On MSE, we perform better than gPool-GAE, GCN-GAE, and
Revised: Compared with the other four baselines (which can be treated as ablation studies), our MIAGAE outperforms all of them in terms of requiring the smallest storage; On MSE, we perform better than gPool-GAE, GCN-GAE, and

Original: GAT-GAE on all four datasets.
Revised: GAT-GAE on all four datasets.

Original: For SAGPool-GAE, we have smaller MSE on most datasets except
Revised: For SAGPool-GAE, we have smaller MSE on most datasets except

Original: COLORS-3 (0.026 vs. 0.028).
Revised: COLORS-3 (0.026 vs. 0.028).

Original: (More training and architecture details in Appendix).
Revised: (More training and architecture details in Appendix).

Revised: [Table]

Original: 3. G RAPH REPRESENTATION LEARNING AND F EW - SHOT G RAPH CLASSIFICATION
Revised: 3. G RAPH REPRESENTATION LEARNING AND F EW - SHOT G RAPH CLASSIFICATION

Original: In this section, we evaluate graph representation learning performance.
Revised: In this section [Table] we evaluate graph representation learning performance.

Original: GAE structure networks can be used as an unsupervised or self-supervised learning framework to learn useful graph representations on an unlabeled dataset, and then transfer the representation knowledge to downstream tasks. We
Revised: GAE structure networks can be used as an unsupervised or self-supervised learning framework to learn useful graph representations on an unlabeled dataset

Original: [Table]
Revised: [Table]

Revised: and then transfer the representation knowledge to downstream tasks.

Original:  use a few-shot graph classiﬁcation task as a downstream task which is a challenging task with only a small amount of labeled data.
Revised: We use a few-shot graph classiﬁcation task as a downstream task which is a challenging task with only a small amount of labeled data.

Original: Especially when the graph size to be classiﬁed is large, a conﬂict occurs because the large graph sample requires a large classiﬁer with more parameters.
Revised: Especially when the graph size to be classiﬁed is large, a conﬂict occurs because the large graph sample requires a large classiﬁer with more parameters.

Original: At the same time, limited labeled data can not support the training.
Revised: At the same time, limited labeled data can not support the training.

Original: So a compressed latent graph representation may alleviate this conﬂict.
Revised: So a compressed latent graph representation may alleviate this conﬂict.

Original: We ﬁrst randomly divide the FRANKENSTEIN dataset into two datasets: unlabeled dataset U to pretrain the graph compression model and labeled dataset L for a few-shot graph classiﬁcation task ( L consists of 100 training and 100 test graphs).
Revised: We ﬁrst randomly divide the FRANKENSTEIN dataset into two datasets: unlabeled dataset U to pretrain the graph compression model and labeled dataset L for a few-shot graph classiﬁcation task ( L consists of 100 training and 100 test graphs).

Original: After training the graph compression models with
Revised: After training the graph compression models with

Original: U , we use them to compress the graphs in L , and then train a GCN classiﬁer on 100 training graphs and test it on 100 test graphs.
Revised: U , we use them to compress the graphs in L , and then train a GCN classiﬁer on 100 training graphs and test it on 100 test graphs.

Original: One of our baselines is directly using the original training graphs of L to train a GCN classiﬁer without compression.
Revised: One of our baselines is directly using the original training graphs of L to train a GCN classiﬁer without compression.

Original: The other baselines have the same settings as
Revised: The other baselines have the same settings as

Original: Sec. 3.1.
Revised: Sec. 3.1.

Original: Table.
Revised: Table.

Original: 4 shows classiﬁcation accuracy for the downstream GCNs.
Revised: 4 shows classiﬁcation accuracy for the downstream GCNs.

Original: (More training details in
Revised: (More training details in

Original: Appendix).
Revised: Appendix).

Original: Our method outperforms all baselines.
Revised: Our method outperforms all baselines.

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: Generalizing across datasets .
Revised: Generalizing across datasets .

Original: To evaluate the graph representation learning performance across datasets, we pretrain the graph compression model on an unlabeled scene graph dataset
Revised: To evaluate the graph representation learning performance across datasets, we pretrain the graph compression model on an unlabeled scene graph dataset

Original: (Visual
Revised: (Visual

Original: Genome), and then directly use the pretrained graph compression model to compress the graphs in our contributed Img2SceneGraph-ACSG (where we use 1000 graphs for training and 1000 for test).
Revised: Genome), and then directly use the pretrained graph compression model to compress the graphs in our contributed Img2SceneGraph-ACSG (where we use 1000 graphs for training and 1000 for test).

Original: Table.
Revised: Table.

Original: 5 shows classiﬁcation accuracy for the downstream GCNs.
Revised: 5 shows classiﬁcation accuracy for the downstream GCNs.

Original: (More training details in
Revised: (More training details in

Original: Appendix).
Revised: Appendix).

Original: Our method outperforms all baselines.
Revised: Our method outperforms all baselines.

Original: [Table]
Revised: [Table]

Original: [Figure]

Original: [Table]
Revised: [Table]

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: We proposed a novel Graph Autoencoder structure, Multi-kernel Inductive Attention Graph Autoencoder (MIAGAE), which achieves state-of-the-art performance on graph compression and representation learning tasks.
Revised: We proposed a novel Graph Autoencoder structure, Multi-kernel Inductive Attention Graph Autoencoder (MIAGAE), which achieves state-of-the-art performance on graph compression and representation learning tasks.

Original: The proposed pooling method, SimAGPool, uses nodes similarity and topology to select the most representative nodes with no extra trainable parameters.
Revised: The proposed pooling method, SimAGPool, uses nodes similarity and topology to select the most representative nodes with no extra trainable parameters.

Original: Multi-kernel Inductive-Conv helps to learn general node representation in an autoencoder structure.
Revised: Multi-kernel Inductive-Conv helps to learn general node representation in an autoencoder structure.

Original: We show that MIAGAE can be a general graph representation learning framework for downstream tasks.
Revised: We show that MIAGAE can be a general graph representation learning framework for downstream tasks.

Original: We hope that researchers ﬁnd our Graph autoencoder useful and extend it for their applications.
Revised: We hope that researchers ﬁnd our Graph autoencoder useful and extend it for their applications [Table]

Revised: A CKNOWLEDGMENTS

Revised: This work was supported by C-BRIC (one of six centers in JUMP, a Semiconductor Research

Revised: Corporation (SRC) program sponsored by DARPA), the Army Research Ofﬁce (W911NF2020053), and the Intel and CISCO Corporations.

Revised: The authors afﬁrm that the views expressed herein are solely their own, and do not represent the views of the United States government or any agency thereof.

Original: APPENDIX A D ATASET A. I MAGE TO S CENE G RAPH P IPELINE
Revised: APPENDIX A D ATASET A. I MAGE TO S CENE G RAPH P IPELINE

Original: Img2SceneGraph provides a pipeline that transfers images to scene graphs with node attributes.
Revised: Img2SceneGraph provides a pipeline that transfers images to scene graphs with node attributes.

Original: Here is a typical workﬂow:
Revised: Here is a typical workﬂow:

Original: For each image, we used the pre-trained model from Scene-Graph-Benchmark
Revised: For each image, we used the pre-trained model from Scene-Graph-Benchmark

Original: (Tang, 2020; Tang et al., 2019; 2020) to synthesis the following outputs: 79 bounding boxes (b-boxes) labeled by a single word and over 6,000 relationship pairs (rel-pairs) between b-boxes, both of them are sorted by their corresponding conﬁdence scores.
Revised: (Tang, 2020; Tang et al., 2019; 2020) to synthesis the following outputs: 79 bounding boxes (b-boxes) labeled by a single word and over 6,000 relationship pairs (rel-pairs) between b-boxes, both of them are sorted by their corresponding conﬁdence scores.

Original: To form a scene graph, we provide multiple methods to select edges and nodes.
Revised: To form a scene graph, we provide multiple methods to select edges and nodes.

Original: Select edges ﬁrst (a) Select the top n % rel-paris as edges and corresponding b-boxes as nodes.
Revised: Select edges ﬁrst (a) Select the top n % rel-pairs as edges and corresponding b-boxes as nodes.

Original: (b) Select rel-pairs with conﬁdence score higher than k as edges and corresponding b-boxes as nodes.
Revised: (b) Select rel-pairs with conﬁdence score higher than k as edges and corresponding b-boxes as nodes.

Original: (c) Select top m rel-pairs as the edges and corresponding b-boxes as nodes.
Revised: (c) Select top m rel-pairs as the edges and corresponding b-boxes as nodes.

Original: Select nodes ﬁrst (d) Select the top n % b-boxes as nodes and corresponding rel-pairs as edges.
Revised: Select nodes ﬁrst (d) Select the top n % b-boxes as nodes and corresponding rel-pairs as edges.

Original: (e) Select the b-boxes with conﬁdence score higher than k as nodes and corresponding rel-pairs as edges.
Revised: (e) Select the b-boxes with conﬁdence score higher than k as nodes and corresponding rel-pairs as edges.

Original: (f) Select the top m b-boxes as nodes and corresponding rel-pairs as edges.
Revised: (f) Select the top m b-boxes as nodes and corresponding rel-pairs as edges.

Original: For each node, we generate a d -dimension word embedding using (Mikolov et al., 2013) from the label of b-box, which is considered as our initial node feature.
Revised: For each node, we generate a d -dimension word embedding using (Mikolov et al., 2013) from the label of b-box, which is considered as our initial node feature.

Original: For each edge, we can use a one-hot vector or word embedding to obtain the edge label/feature.
Revised: For each edge, we can use a one-hot vector or word embedding to obtain the edge label/feature.

Original: If the image has a label, it will be the graph label as well.
Revised: If the image has a label, it will be the graph label as well.

Original: Our primary motive for creating Img2SceneGraph and corresponding datasets (described below) is that it allows rapid idea prototyping for graph compression and representation learning.
Revised: Our primary motive for creating Img2SceneGraph and corresponding datasets (described below) is that it allows rapid idea prototyping for graph compression and representation learning.

Original: A. I MG 2S CENE G RAPH -ACSG
Revised: A. I MG 2S CENE G RAPH -ACSG

Original: Img2SceneGraph-ACSG is a scene graph dataset with graph labels.
Revised: Img2SceneGraph-ACSG is a scene graph dataset with graph labels.

Original: We created it by using our
Revised: We created it by using our

Original: Img2SceneGraph pipeline on AI Challenger Scene Graph dataset (Jean-Claude).
Revised: Img2SceneGraph pipeline on AI Challenger Scene Graph dataset (Jean-Claude).

Original: It is a labeled image dataset that contains 7,120 images with 80 classes.
Revised: It is a labeled image dataset that contains 7,120 images with 80 classes.

Original: We used method (a) described above with n = 10.
Revised: We used method (a) described above with n = 10.

Original: The word embedding dimension d is 500.
Revised: The word embedding dimension d is 500.

Original: A. V ISUAL G ENOME
Revised: A. V ISUAL G ENOME

Original: Visual Genome (kri) is an image dataset that contains over 108K images without labels.
Revised: Visual Genome (kri) is an image dataset that contains over 108K images without labels.

Original: We chose a small split of it (9987) and use Img2Scenegraph with the same setting (described in Sec.
Revised: We chose a small split of it (9987) and use Img2Scenegraph with the same setting (described in Sec.

Original: A.2) to generate the scene graph dataset (Img2Scenegraph-VisualGenome).
Revised: A.2) to generate the scene graph dataset (Img2Scenegraph-VisualGenome).

Original: It is used for both graph compression and compression models pretrained to evaluate the graph representation learning performance across datasets.
Revised: It is used for both graph compression and compression models pretrained to evaluate the graph representation learning performance across datasets.

Original: You can download the source code of Img2SceneGraph pipeline and synthesized dataset (Image2Scenegraph-ACSG and Image2Scenegraph-VisualGenome) from: http://anonymous/ url/for/blind/review , which we plan to keep up-to-date with contributions from ourselves and the community.
Revised: You can download the source code of Img2SceneGraph pipeline and synthesized dataset (Image2Scenegraph-ACSG and Image2Scenegraph-VisualGenome) from: http://ilab.usc. edu/datasets/i2sg , which we plan to keep up-to-date with contributions from ourselves and the community.

Original: B N ETWORK A RCHITECTURE
Revised: B N ETWORK A RCHITECTURE

Original: The network architectures of our Multi-kernel Inductive Attention Graph Autoencoder (MIAGAE) is shown in Table.
Revised: The network architectures of our Multi-kernel Inductive Attention Graph Autoencoder (MIAGAE) is shown in Table.

Original: 6, which is used in the experiments of two main tasks: graph compression (main paper Sec. 3.1) and few shot graph classiﬁcation (main paper Sec. 3.2).
Revised: 6, which is used in the experiments of two main tasks: graph compression (main paper Sec. 3.1) and few shot graph classiﬁcation (main paper Sec. 3.2).

Original: For compression task, we chose Graph Unet (Gao & Ji, 2019) as the baseline model.

Original: Besides, we create 4 other baseline models serving for our ablation study.

Original: Here are the settings: • gPool-GAE: we replaced our pooling method with gPool (Gao & Ji, 2019) • SAGPool-GAE: we replaced our pooling method with SAG Pool (Lee et al., 2019); • GCN-GAE: we used GCN (Kipf & Welling, 2016a) to substitute our MI-Conv; • GAT-GAE: we used GAT (Veli ˇ ckovi ´ c et al., 2017) to substitute our MI-Conv;
Revised: • gPool-GAE: we replaced our pooling method with gPool (Gao & Ji, 2019); • SAGPool-GAE: we replaced our pooling method with SAG Pool (Lee et al., 2019); • GCN-GAE: we used GCN (Kipf & Welling, 2016a) to substitute our MI-Conv; • GAT-GAE: we used GAT (Veli ˇ ckovi ´ c et al., 2017) to substitute our MI-Conv

Original: GCN classiﬁcation models used in the main paper Sec. 3.2 is composed of two graph convolutional layers with node feature dimension 64, followed by l layers MLP to output the ﬁnal classiﬁcation result.
Revised: GCN classiﬁcation models used in the main paper Sec. 3.2 is composed of two graph convolutional layers with node feature dimension 64, followed by l layers MLP to output the ﬁnal classiﬁcation result.

Original: Speciﬁcally, the GCN classiﬁers used in the FRANKENSTEIN dataset use l =2 without batch normalization.
Revised: Speciﬁcally, the GCN classiﬁers used in the FRANKENSTEIN dataset use l =2 without batch normalization.

Original: The GCN classiﬁers used in the Img2SceneGraph dataset use [Table] [Table] [Table]
Revised: The GCN classiﬁers used in the Img2SceneGraph-ACSG dataset use l =4 with batch normalization.

Original: → ( n 1 , 64)

Original: [Table]

Original: [Table]

Original: [Table]

Original: → ( n

Original: [Table]

Original: [Table]

Original: [Table]

Original: → ( n

Original: [Table]

Original: [Table]

Original: ( n 2 , 64)

Original: → ( n 3 , 64)

Original: [Table]

Original: Decoder ( n 3 , 64) → ( n 2 , 64)

Original: [Table]

Original: ( n

Original: [Table]

Original: → ( n 1 , 64)

Original: [Table]

Original: ( n 1 , 64)

Original: → ( n 0 , d 0 )

Original: [Table]

Original: C T RAINING DETAILS
Revised: C T RAINING DETAILS

Original: We mentioned the high-level training instructions in the main paper Sec. 3.
Revised: We mentioned the high-level training instructions in the main paper Sec. 3.

Original: Here are the details.
Revised: Here are the details.

Original: C. G RAPH C OMPRESSION
Revised: C. G RAPH C OMPRESSION

Original: We trained MIAGAE and all the baselines on all the datasets using Adam (Kingma & Ba, 2017) with β 1 =0.9 and β 2 =0.999, learning rate=0.001, for 500 epochs.
Revised: We trained MIAGAE and all the baselines on all the datasets using Adam (Kingma & Ba, 2017) with β 1 =0.9 and β 2 =0.999, learning rate=0.001, for 500 epochs.

Original: For COLORS-3, we chose the data labeled from 0 to 7 as the training set, which contains 7,996 samples.
Revised: For COLORS-3, we chose the data labeled from 0 to 7 as the training set, which contains 7,996 samples.

Original: For FRANKENSTEIN, the size of the training set is 4,000.
Revised: For FRANKENSTEIN, the size of the training set is 4,000.

Original: For Img2SceneGraph-ACSG and Visual Genome, the size of the training set is 2,000 and 5,000, respectively.
Revised: For Img2SceneGraph-ACSG and Visual Genome, the size of the training set is 2,000 and 5,000, respectively.

Revised: [Table]

Original: C. F EW SHOT GRAPH CLASSIFICATION C.2. FRANKENSTEIN
Revised: C. F EW SHOT GRAPH CLASSIFICATION C.2. FRANKENSTEIN

Original: We chose 100 samples and compressed them with the pre-trained compression model to train the classiﬁer using Adam β 1 =0.9 and β 2 =0.999, batch size 200, learning rate 0.01 for 1,000 epochs.
Revised: We chose 100 samples and compressed them with the pre-trained compression model to train the classiﬁer using Adam β 1 =0.9 and β 2 =0.999, batch size 200, learning rate 0.01 for 1,000 epochs.

Original: For the non-compression method, the input is the uncompressed graphs.
Revised: For the non-compression method, the input is the uncompressed graphs.

Original: Then we evaluated the performance on a test set with another 100 samples.
Revised: Then we evaluated the performance on a test set with another 100 samples.

Original: C.2. V ISUAL G ENOME & I MG 2S CENE G RAPH -ACSG
Revised: C.2. V ISUAL G ENOME & I MG 2S CENE G RAPH -ACSG

Original: For the compression part, we ﬁrst trained MIAGAE and all the baselines on the Visual Genome dataset using Adam β 1 =0.9 and β 2 =0.999, batch size 500, learning rate 0.001 for 100 epochs.
Revised: For the compression part, we ﬁrst trained MIAGAE and all the baselines on the Visual Genome dataset using Adam β 1 =0.9 and β 2 =0.999, batch size 500, learning rate 0.001 for 100 epochs.

Original: For the classiﬁcation part, we chose 1,000 samples from the Img2SceneGraph-ACSG dataset and compressed them using the pre-trained model.
Revised: For the classiﬁcation part, we chose 1,000 samples from the Img2SceneGraph-ACSG dataset and compressed them using the pre-trained model.

Original: The compressed graphs will train the classiﬁer using bath size 200, learning rate 0.01, weight decay 0.01 for 1,000 epochs.
Revised: The compressed graphs will train the classiﬁer using bath size 200, learning rate 0.01, weight decay 0.01 for 1,000 epochs.

Original: We evaluated the performance on a test set with another 1,000 samples from Img2SceneGraph-ACSG.
Revised: We evaluated the performance on a test set with another 1,000 samples from Img2SceneGraph-ACSG.