E GO -CNN: A N E GO N ETWORK - BASED R EPRESENTATION OF G RAPHS D ETECTING C RITICAL S TRUCTURES
Anonymous authors
Paper under double-blind review
A BSTRACT
While existing graph embedding models can generate useful embedding vectors that perform well on graph-related tasks, what valuable information can be jointly learned by a graph embedding model is less discussed.
In this paper, we consider the possibility of detecting critical structures by a graph embedding model.
We propose Ego-CNN to embed graph, which works in a local-to-global manner to take advantages of CNNs that gradually expanding the detectable local regions on the graph as the network depth increases.
Critical structures can be detected if
Ego-CNN is combined with a supervised task model.
We show that Ego-CNN is (1) competitive to state-of-the-art graph embeddings models, (2) can nicely work with CNNs visualization techniques to show the detected structures, and (3) is efﬁcient and can incorporate with scale-free priors, which commonly occurs in social network datasets, to further improve the training efﬁciency.
I NTRODUCTION
A graph embedding model embeds graphs from structural representation to ﬁxed-dimensional vectors.
It is typically trained in a unsupervised manner for general learning tasks but recently, deep learning approaches such as Structure2vec (Dai et al. (2016)) and Diffusion Convolution Neural
Network (Atwood & Towsley (2016)) are trained in a supervised manner and shown superior results against unsupervised approaches on many tasks such as the graph classiﬁcation.
While many algorithms perform well on graph-related tasks, what valuable information can be jointly learned from the graph embedding is less discussed.
In this paper, we aim to develop a graph embedding model that jointly discovers the critical structures , i.e., partial graphs that are dominant to a learning task (e.g., graph classiﬁcation) where the embedding is applied to.
This helps people running the learning task understand the reason behind the task results, and is particularly useful in certain domains such as the bioinformatics and social network analysis.
For example, people in bioinformatics may wish to tell whether a protein is an enzyme or not and why , as shown in Figure 1(a).
If a model can backtrack the most critical parts like the ones shown in Figure 1(b), it may help people understand that it is the circular structures (the “holes”) that distinguish enzyme from other substances.
New knowledge may then be discovered by investigating the functionality of these holes.
[Figure]
Unfortunately, identifying critical structures is a challenging task.
The ﬁrst challenge is that critical structures are task-speciﬁc —the shape and location of critical structures may vary from task to task.
This means that the graph embedding model needs to be learned together with the task model (e.g.,
[Figure]
(asymmetric isomer) 5­methylnoane (symmetric isomer) ...
Alkane
Alcohol ...
2­methylnonane (asymmetric isomer) 5­methylnoane (symmetric isomer) ...
Alkane
Alcohol ...
[Figure] [Figure]
a classiﬁer or regressor).
Another challenge is that critical structures may be found at the globalscale , as shown in Figure 2.
To discriminant Alcohols from Alkanes (Figure 2(a)), one can check if there exists an OH-base and if the OH-base is at the end of the compound.
In this task, the critical structure (OH-base) is relative small and can be identiﬁed at the local-scale.
But for the other task (Figure 2(b)), to identify if a methyl-nonane is symmetric or not, one must check the entire graph to know if the methyl is branched at the center position of the long carbon chain.
In this task, the critical structure is the symmetric hydrocarbon at the two sides of the methyl branch, which can only be found at the global-scale.
Finding out all matches of substructures in a graph is known as subgraph isomorphism and proven to be an NP-complete problem by Cook (1971).
To the best of our knowledge, there is no existing graph embedding algorithm that can identify critical structures up to the global-scale in an efﬁcient way.
Most existing approaches has only limited ability in identifying critical structures.
For example, Structure2vec (Dai et al.
and
Spectrum GCN (Defferrard et al.
(2016)) can only ﬁnd critical structures of very simple shape.
Patchy-San (Niepert et al.
(2016)) can only ﬁnd critical structures within a small area around each node (i.e., at the local-scale).
The only work that is able to identify critical structures at the globalscale is Spatial GCN (Bruna et al.
, but it has the complexity O ( N 2 ) , where N is the number of nodes, in both time and space, therefore is inefﬁcient for large graphs.
In this paper, we propose the Ego-CNN that embeds a graph into distributed (multi-layer), ﬁxeddimensional tensors.
The Ego-CNN can be jointly learned with a supervised task model (e.g., fully-connected layers for graph classiﬁcation) and help identify the task-speciﬁc critical structure at the global-scale.
Some existing studies like DCNN (Atwood & Towsley (2016)) and Patchy-San (Niepert et al.
(2016)) have borrowed the concept of CNN to embed graphs.
The idea is to model the ﬁlters / kernels that scan through different parts of the graph (which we call the neighborhood s) to learn patterns most helpful to the learning task.
Unlike images where a neighborhood can be clearly deﬁned as the nearby pixels across channels, graphs have no standard notion of neighborhoods.
Currently, the neighborhoods deﬁned by above works have some limitations in that (1) a neighborhood does not represent a local region of the graph so the embedding model cannot take advantage of the location invariant patterns to speed up the learning process, or (2) a neighborhood represents a local region of the graph but cannot be recursively deﬁned over the hidden layers, making the embedding model shallow and incapable of detecting critical structure at the global-scale.
Different from most existing approaches, Ego-CNN deﬁnes a neighborhood at layer l as the ( l + 1) -hop ego network centered at a speciﬁc node.
1 This makes the Ego-CNN efﬁcient since it learns the location invariant patterns.
Also, a neighborhood at layer l enlarges the receptive ﬁeld of the neighborhood centered at the same node at layer l  1 .
Ego-CNN can exploit the multi-layer architecture to detect the critical structure at the global-scale in the deeper layers.
Furthermore, Ego-CNN works nicely with some common visualization techniques for CNNs (e.g., deconvolution proposed by Zeiler et al.
and can output the critical structures behind each prediction made by the trained task model.
Our contributions are
• We propose the Ego-CNN and show that an graph embedding based on ego networks can perform as well as (if not better than) existing approaches.
We conduct experiments on
many graph classiﬁcation tasks and the results show that Ego-CNN can lead to the stateof-the-art performance.
• By letting a neighborhood (an ego network) at a deeper layer share the same centering node with a neighborhood at the preceding layer (which we call the egocentric design ), EgoCNN supports a multi-layer architecture with enlarged receptive ﬁelds at deeper layers, thus can learn the critical structures at the global-scale.
• Ego-CNN, which learns location invariant patterns recursively, is efﬁcient.
Each convolution takes only O ( Nkd ) in both space and time for a ﬁlter to scan a graph with N nodes.
We also show that Ego-CNN can readily incorporate the scale-free prior, which commonly exists in large (social) graphs, to further improve the training efﬁciency.
To the best of our knowledge, Ego-CNN is the ﬁrst embedding model that can efﬁciently detect task-dependent critical structures at the global-scale.
We hope it helps practitioners better explain the learning results and discover new knowledge in the future.
R ELATED W ORK
We compare existing graph embedding models in Table 1 on the aspects of our goal [Table] i [Table]
e. whether it jointly detects global-scale critical structure and its computational complexity.
[Table]
Defferrard et al.
; Kipf & Welling (2017))
O ( dn f | E | )
DCNN (Atwood & Towsley (2016))
O ( hn f | V | 2 ) Patchy-San
(Niepert et al.
O ( k 2 n f | V | )
Neural Fingerprint (Duvenaud et al.
O ( kn f | V | ) Ego-Convolution
O ( kdn f L | V | )
Graph Kernels All graph kernels have a common drawback in that the embeddings are generated in a unsupervised manner, which means critical structure cannot be jointly detected at the generation of embeddings.
Here we introduce three state-of-the-art graph kernels.
The Weisfeiler-Lehman kernel (Shervashidze et al.
(2011)) grows the coverage of each node by collecting information from neighbors, which is conceptually similar to our method, but differs in that WL kernel collects only node labels, while our method collects the complete labeled neighborhood graphs from neighbors.
Deep
Graph Kernels (Yanardag & Vishwanathan (2015)) and Subgraph2vec (Narayanan et al.
(2016)) were inspired by word2vec (Mikolov et al.
models that embed structures by predicting neighbors’ structures.
Multiscale Laplacian Graph Kernels (Kondor & Pan (2016)) can compare graphs at multiple scales by recursively comparing graphs on the comparison of subgraphs
However, it’s very inefﬁcient in that comparing two graphs takes O ( L | V | 5 ) , where L represents the number of comparing scales.
Graphical Models Graph can also express the conditional dependence (edge) between random variables (node) in graphical models.
models.
Structure2vec (Dai et al.
(2016)) introduced a novel layer that makes the optimization procedures of approximation inference directly trainable by SGD.
It’s efﬁcient on large graphs with time complexity linear to number of nodes.
However, it’s weak at identifying critical structures since approximation inference makes too much simpliﬁcation on structures.
For example, mean-ﬁeld approximation assumes variables are independent of each other.
As a result, Structure2vec can only identify critical structures of very simple shape.
Convolution-based Methods
Recently, many works are proposed by borrowing the concept of CNN to embed graphs.
Figure 3 summarizes their deﬁnition of ﬁlters and neighborhoods.
[Figure]
The Spatial Graph Convolutional Network (GCN) was proposed by Bruna et al.
The design of Spatial GCN (Figure 3(a)) is very different from other convolution-based methods since its goal is to perform hierarchical clustering of nodes.
A neighborhood is deﬁned as a cluster.
However, the ﬁlter is not aim to scan for local patterns but to learn the connectivity of all clusters.
This means each ﬁlter is of size O ( N 2 ) if there’re N clusters.
Also, a ﬁlter requires the global-scale information, i.e. the features of all clusters to train, so it’s very inefﬁcient on large graphs.
Hence, in the same paper, Bruna et al.
(2013) proposed another version, the Spectrum GCN, to perform hierarchical clustering in the spectrum domain.
Although the efﬁciency is improved and many recent work such as Defferrard et al.
; Kipf & Welling (2017) further improve the training efﬁciency, the major drawback is that spectrums is weak at identifying structures since only special families of graphs such as complete graphs or star-like trees can be precisely described by spectrums.
Diffusion Convolutional Neural Networks (Atwood & Towsley (2016)) embed graphs by detecting the diffusion.
The neighborhood is deﬁned as the diffusion, which are paths starting from a node to other nodes in h hops.
The diffusion can be represented by a diffusion matrix.
They use ﬁlters to scan through each node’s diffusion matrix, so DCNN is possible to detect useful diffusion patterns.
But it cannot detect critical structures since the diffusion patterns cannot precisely describe the location and the shape of structures.
DCNN reported impressive results on node classiﬁcation.
But it’s inefﬁcient on graph classiﬁcation tasks since their notion of neighborhood is at global-scale, which takes O ( hN 2 ) to embed a graph with N nodes.
Also, their deﬁnition of neighborhood makes the embedding model shallow, which is only a single layer in the neural network.
Patchy-San (Niepert et al.
(2016)) detects local neighborhoods by ﬁlters.
The deﬁnition of local neighborhood is based on the adjacency matrix of the graph, which is deﬁned as the local graphs formed by its k nearest neighbors.
The ﬁlter scans on the adjacency matrix of local neighborhoods.
Since expressing graphs by adjacency matrices is not invariant under different vertex ordering, they proposed neighborhood normalization to generate similar adjacency matrices for isomorphic neighborhoods.
Patchy-San can detect precise structures but it is inefﬁcient to scan for large local neighborhood, which limits it from detecting global-scale critical structures.
Also, the embedding model of Patchy-San is only a single layer, a question to ask is that is it possible to improve the efﬁciency by generalizing its deﬁnition of neighborhood into multiple layers?
If the adjacency matrix of neighborhoods is generated (e.g. the similarity of all embedding vectors of all neighborhoods), the neighborhood at next layer can be deﬁned as the k most similar neighborhoods.
However, this naive generalization don’t work for two reasons.
First, the concept of local neighborhood is lost since two neighborhoods that have similar embedding vectors are not necessary to be adjacent on the graph.
Second, the structure of neighborhood is dynamic since the k most similar neighborhoods changes during training.
Therefore, the idea of Patchy-San cannot be readily extended to multiple layers.
To the best of our knowledge, the only work that scans on local neighborhoods and supports multiple layers is Neural Fingerprints (Duvenaud et al.
But the neighborhood at layer l represents only the approximation (specially, the summation) of the l -hop neighbors.
They use the approximation to avoid the vertex-ordering problem of adjacency matrix.
However, at the same time, Neural
Fingerprint loses the ability of detecting precise critical structures.
We know that CNN has two advantages: (1) detecting location invariant patterns and (2) overcoming the curse of dimensionality by multi-layer representations.
To keep these advantages on graphs, we have to rethink the deﬁnition of neighborhood from scratch.
E GO -CNN
The reason why the idea of Patchy-San fails to generalize into multiple layers is that its deﬁnition of neighborhood, which is based on adjacency matrix, is not static and may not corresponding to local regions in the graph.
Our main idea is to use the egocentric design , i.e. the neighborhood at next layer is deﬁned on the same node .
The overview of our method is depicted in Figure 4.
[Figure]
Given a graph G of N nodes, (1) To prepare the input X 2 R N ⇥ d for Ego-CNN, we have to select the ﬁxed-size neighborhood and k neighbors for each node in the graph, and convert neighborhoods into d -dimensional vectors.
Any existing graph embedding models can be used, but we recommend to use Patchy-San since our Ego-CNN can be directly stacked on top of it.
(2) The Ego-CNN is composed of several Ego-Convolution layers.
To enlarge neighborhoods layerby-layer, the trick is to deﬁne neighborhood on the same node across all layers, which we called the egocentric design.
As shown in Figure 5, we deﬁne the neighborhood of a node at layer l as the node and its k neighbors’ neighborhoods at layer l  1 (which we abbreviated as the neighboring neighborhoods).
In this way, the deﬁnition of neighborhoods is static and results in larger neighborhoods at next level.
The idea of egocentric design is implemented in Ego-Convolution, whose procedure is listed in
Algorithm 1.
Algorithm
Require:
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
The l -th Ego-Convolution layer takes H l  1 2 R N ⇥ d l  1 , i.e. embeddings of neighborhoods generated by layer l  1 , as input.
To generate embeddings of neighborhood H l 2 R N ⇥ d l . for each node i , we lookup its k neighboring neighborhoods and concate together with its neighborhood embedding to form into a ( k + 1) ⇥
d l  1 matrix.
The goal of concating each node’s neighborhood embedding is to let next level neighborhood know the structure between its k neighbors.
However, a problem is that it is possible for two isomorphic neighborhoods to have different concatenated matrix if the embeddings of k neighborhood are put in non-consistent order.
This problem is similar to the vertex reordering problem encounter by Niepert et al.
and can be solved by their neighborhood normalization procedure, whose idea is simply to assigned a global ordering of all nodes and then neighbors are sorted in non-increasing/decreasing global order.
(3) The graph embedding H L 2 R N ⇥ d L generated by Ego-CNN can be used by task models such as SVM or classiﬁcation layers to trained for a speciﬁc learning task.
[Figure]
Effective Receptive Field on Ambient Graph A receptive ﬁeld in Ego-CNN has the ambient coverage on graphs.
The coverage of a receptive ﬁeld is enlarged as network depth increases (Figure 4.2).
Suppose the input (i.e. embeddings of neighborhoods) to the Ego-CNN represents 1-hop ego networks (Figure 4.2(a)) and k neighbors of each node are selected from 1-hop neighbors.
The neighborhood deﬁned at 1st Ego-Convolution layer includes k 1-hop ego networks of its 1-hop neighbors, which approximately represents a 2-hop ego network (Figure 4.2(b)).
It is an approximation of 2-hop ego network since edges across two neighbor’s ego networks are not captured by receptive ﬁelds.
As the depth of Ego-CNN increases, a receptive ﬁeld of the l -th Ego-Convolution layer approximates covers a ( l + 1) -hop ego network.
As shown in Figure 4.2(e), it’s possible to cover the entire graph by a single receptive ﬁeld of Ego-CNN.
Therefore, Ego-CNN can detect global-scale structures and can ﬁnd critical structures by combining with a task model (e.g. the classiﬁcation layers) to trained for learning task.
Also, the different coverage of receptive ﬁelds reﬂects the position of nodes (see Figure 4.2(e)-(f)).
While the receptive ﬁeld shown in Figure 4.2(e) covers entire graph, the receptive ﬁeld in Figure 4.2(f) only covers part of the graph.
This difference is related to the global position of the two node.
We think it’s possible for Ego-CNN to aware of position of nodes by comparing receptive ﬁelds and take advantages of it.
A DVANTAGES OF E GO -CNN 4. D ETECTING C RITICAL S TRUCTURES
Given a trained Ego-CNN, there are many ways to visualize critical structures.
Here we consider
Ego-CNN is used with the general task model, which can be several neural network layers or can be an additional classiﬁer such as SVM.
Our approach takes in two steps.
Find out the most important neighborhoods.
The simplest way is to add an Attention layer (Itti et al.
to generate a score representing the importance of each neighborhood.
To train the weights of the Attention layer, we ﬁx weights of previous layers (i.e. Patchy-San and Ego-Convolution) and add a Dense layer to train for the task.
We can pickup important neighborhoods by setting a threshold.
Identify critical regions in those important neighborhoods
Since the critical parts is considered in the network’s perspective, we take the approach similar to Deconvolution (Zeiler et al.
, i.e. we interpolate outputs with ﬁlters to reconstruct the neighborhoods (represented by receptive ﬁelds) in a layer-by-layer fashion, from the deepest Ego-Convolution to 1st Ego-Convolution to Patchy-San layer, where the adjacency matrix of neighborhoods are interpolated.
The interpolated neighborhood graphs (in adjacency matrix) are critical structures to the task, and the edge weight in adjacency matrix corresponds to the importance of an edge.
4. E FFICIENCY AND THE S CALE -F REE P RIOR
Computation Complexity Given a graph with N nodes.
To generate neighborhood embeddings at the l -th Ego-Convolution.
It takes O ( N ) to lookup embeddings of k neighboring neighborhoods and O ( Nk log k ) to concat neighborhoods according to neighbors’ global order, and O ( N ( k + 1) d l  1 d l ) to perform convolution through each receptive ﬁelds (of size ( k + 1) ⇥
d l  1 ) by d l ﬁlters.
In total, for a Ego-CNN with L Ego-Convolution layers, it takes
O ⇣
N ( k + 1)
P
[Equation]
⌘ to embed the graph.
Scale-Free Regularizer
The scale-free property commonly exists in real social networks.
The most notable property of scale-free networks is that the degree distribution follows a power-law.
Figure 7 shows the degree distribution of the Reddit dataset, where we it benchmark model performance on classiﬁcation tasks.
We found degree distribution of graphs in Reddit datasets follows a power-law.
This motivates us to think of the possibility of introducing scale-free priors into
Ego-CNN.
Recall that ﬁlters of the l -th Ego-Convolution layer is detecting the combinations of how neighborhoods at the ( l  1) -th layer are combined into neighborhoods at the l -th layer.
In fact, one way to generate a scale-free network is to repeat the combination at each layer, which means we can simply introduce scale-free prior into the Ego-CNN by weight-tying, i.e. let the ﬁlters share across different Ego-Convolution layers.
E XPERIMENTS
We perform a set of experiments to verify (i) the performance compared to existing graph embedding models on classiﬁcation datasets, (ii) the effectiveness of scale-free regularizer, and (iii) visualization of identiﬁed critical structures.
All experiments were performed on a computer with 48-core Intel(R)
Xeon(R) E5-2690 CPU, 64 GB RAM, and NVidia Geforce GTX 1070 GPU.
We use Tensorﬂow 1.
to implement our methods.
5. G RAPH C LASSIFICATION
We benchmark on both bioinformatic and social-network datasets, which are processed by Kersting et al.
Most graphs in bioinformatic datasets are labeled graphs that contain node and edge labels or attributes, while unlabeled graphs in social network datasets.
More details about these datasets can be found in DGK (Yanardag & Vishwanathan (2015)).
The average test accuracy of 10-fold CV is reported to compared with existing methods and the results is listed in Table 2, where ﬁgures of most baselines are directly taken from their papers since experimental settings are the same.
For simplicity, we use the same network architecture for all datasets.
To embed neighborhood graphs, we choose Patchy-San since it deals with labeled graphs and works nicely with CNN visualization techniques.
The architecture of network is composed of 1 Patchy-San layer (128 ﬁlter and k=10, which is the best setting for k reported in their paper), then follows our Ego-CNN with 5 EgoConvolution layers (128 ﬁlters and k=16), and 2 Dense layers with 128 neurons as the task model.
To normalize neighbors, we follow Patchy-San’s approach to use 1-WL labels.
We train the network by Adam with learning rate 0.0001 and use Dropout with droprate 0.5 and Batch Normalization for regularization.
[Table]
[Table]
Our Ego-CNN improves the performance of Patchy-San on most datasets since the ﬁxed-size neighborhoods (each contains at most 10 neighbors) detected by Patchy-San are grown into larger neighborhoods by Ego-Convolution.
It makes sense on datasets such as MUTAG, where the task is to predict if compounds are mutagenic to DNA.
It is reported by Debnath et al.
(1991) that compounds with more than 3 benzene rings are very likely to be positive.
To detect benzene rings in compounds, it’s easier to ﬁnd for Ego-CNN by scanning larger local neighborhoods.
This shows that representing graphs by larger neighborhoods is helpful for these datasets.
5. S CALE -F REE R EGULARIZER
Since we found that most graphs in Reddit dataset are scale-free graphs, we verify the effectiveness of scale-free regularizer on it.
The results are shown in Table 4.
[Table]
To introduce scale-free priors by weight-tying, we expand 1 Ego-Convolution layer to 5 EgoConvolution layers with the same weight, so the total parameters is 1.3M, which is the same as network in 1st row but the advantage is that we improve the test accuracy by 3.5%.
Also, in the 3rd row, we compare against the network with same architecture but without scale-free regularizer, so its number of parameter is 1M more, however, the performance is not better.
This shows that introducing scale-free priors improves the learning efﬁciency, so that the same or even better performance can be obtained by fewer parameters.
5. V ISUALIZATION OF C RITICAL S TRUCTURES
As a sanity check, we generate two example compound dataset to visualize critical structures at local-scale (Alkanes vs Alcohols) and global scale (Symmetric vs Asymmetric Isomers).
The structures of compounds are generated of different length and under different vertex-orderings.
First, we’d like to know if the network considers OH-base as critical structures.
The visualization result is in 8, where critical structures are plotted in grey color, and the node/edge size are proportional to its important score.
We ﬁnd that the OH-base in Alcohols is always consider critical no matter how long it its.
[Figure]
Next, we’d like to know how the network detects the concept of symmetry.
The visualization result of Symmetric vs Asymmetric Isomers dataset is shown below.
For symmetric isomers (Figure 9(a)), the critical structure, i.e. the symmetric hydrocarbon chains, is detected.
If carefully observe nodes and edge inside the detected structures (color in grey), we ﬁnd an interesting thing that the importance of nodes/edges (i.e. which is plotted in different size proportional to the importance score for nodes and weights in interpolated adjacency for edges) also roughly to be symmetric to the methyl-base.
This symmetric phenomenon in detected structure is also observed in asymmetric isomers.
We conjecture the network learns to compare if the two long hydrocarbon chains (which are branched from the methyl-base) are symmetric by comparing node-by-node and edge-by-edge from the methyl-base all along to the end of the hydrocarbon chains, which is similar to how people check if a structure is symmetry.
[Figure]
In the last experiment, we’d like to visualize the detected structures on Reddit dataset to see if it can help us to derive more knowledge about the dataset.
In Reddit dataset, each graph represents a discussion thread.
A node represent a user, and there’s an edge if two users discuss with each other.
The task is to classify the discussion style of threads.
i.e. wether it is a discussion-based thread (e.g. threads under Atheism subreddit) or a QA-based thread (e.g. threads under AskReddit subreddit).
[Figure]
From the visualization result (Figure 10), we observe that the detected structures seemed to be the “bridges” that interconnect highly active users who has many discussion with other users.
If each active user represents an strong opinion, then from the visualization result, the network suggests us that the discussion between different opinions are the key to discriminant discussion-based threads from QA-based threads.
C ONCLUSIONS
We propose Ego-CNN with idea of the egocentric design to deﬁne neighborhood on the same node across layers. This egocentric design lets Ego-CNN to fully take advantages of CNN to perform efﬁcient learning on graphs, so that neighborhoods are enlarged as depth increases.
Also, training efﬁciency can be increased by introducing scale-free priors, and visualization techniques for CNN can be used to visualize structures detected by Ego-CNN.
We hope Ego-CNN helps people in domains such as bioinformatic, cheminformatics to understand their experiment results.
The future direction is to further reduce the space requirement of Ego-CNN.
Since receptive ﬁelds at deeper Ego-Convolution layer represents larger neighborhoods, the overlapping area among neighborhoods of different nodes may also be enlarged.
So, instead of using all neighborhoods, it might be enough to embed graphs by only part of the neighborhoods.