E GO -CNN: A N E GO N ETWORK - BASED R EPRESENTATION OF G RAPHS D ETECTING C RITICAL S TRUCTURES
Anonymous authors
Paper under double-blind review
A BSTRACT
While existing graph embedding models can generate useful embedding vectors for graph-related tasks, what valuable information can be jointly learned from a graph embedding model is less discussed.
In this paper, we consider the possibility of detecting critical structures by a graph embedding model.
We propose
Ego-CNN to embed graphs, which works in a local-to-global manner to take advantages of CNNs that gradually expands the detectable local regions on the graph as the network depth increases.
Critical structures can be detected if Ego-CNN is combined with a supervised task model.
We show that Ego-CNN is (1) competitive to state-of-the-art graph embeddings models, (2) can work nicely with CNNs visualization techniques to show the detected structures, and (3) is efﬁcient and can incorporate with scale-free priors, which commonly occurs in social network datasets, to further improve the training efﬁciency.
I NTRODUCTION
A graph embedding algorithm converts graphs from structural representation to ﬁxed-dimensional vectors.
It is typically trained in a unsupervised manner for general learning tasks but recently, deep learning approaches such as Structure2vec (Dai et al. (2016)) and Diffusion Convolution Neural
Network (Atwood & Towsley (2016)) are trained in a supervised manner and shown superior results against unsupervised approaches on many tasks such as node classiﬁcation and graph classiﬁcation.
While many algorithms perform well on graph-related tasks, what valuable information can be jointly learned from the graph embedding is less discussed.
In this paper, we aim to develop a graph embedding model that jointly discovers the critical structures , i.e., partial graphs that are dominant to a learning task (e.g., graph classiﬁcation) where the embedding is applied to.
This helps people running the learning task to understand the reason behind the task results, and is particularly useful in certain domains such as the bioinformatics and social network analysis.
For example, people in bioinformatics may wish to tell whether a protein is an enzyme or not and why .
As shown in Figure 1, if there is model that can backtrack the most critical parts, it may help people understand that it is the circular structures (the “holes”) that distinguish enzyme from other substances.
New knowledge may then be discovered by investigating the functionality of these holes.
Unfortunately, identifying critical structures is a challenging task.
The ﬁrst challenge is that critical structures are task-speciﬁc —the shape and location of critical structures may vary from task to task.
This means that the graph embedding model needs to be learned together with the task model (e.g.,
2­methylnonane (asymmetric isomer) 5­methylnoane (symmetric isomer)
C
H H
H
C ...
H H
C
H H
OH
Alkane
Alcohol
C
H H
H
C ...
H H
C
H H
H
CH
H H
C ...
H
C
H H
H
H H C H
C
H H
H
C ...
H
C
H H
H
H H C H ...
2­methylnonane (asymmetric isomer) 5­methylnoane (symmetric isomer)
C
H H
H
C ...
H H
C
H H
OH
Alkane
Alcohol
C
H H
H
C ...
H H
C
H H
H
CH
H H
C ...
H
C
H H
H
H H C H
C
H H
H
C ...
H
C
H H
H
H H C H ...
(a) (b) a classiﬁer or regressor).
Another challenge is that critical structures may be found at the globalscale , as shown in Figure 2.
To discriminant Alcohols from Alkanes (Figure 2(a)), one can check if there exists an OH-base and if the OH-base is at the end of the compound.
In this task, the critical structure (OH-base) is relative small and can be identiﬁed at the local-scale.
But for the other task (Figure 2(b)), to identify if a methyl-nonane is symmetric or not, one must check the entire graph to know if the methyl is branched at the center position of the long carbon chain.
In this task, the critical structure is the symmetric hydrocarbon at the two sides of the methyl branch, which can only be found at the global-scale.
The second challenge is that ﬁnding out all matches of substructures in a graph is known as subgraph isomorphism and proven to be an NP-complete problem by Cook (1971).
To the best of our knowledge, there is no existing graph embedding algorithm that can identify critical structures up to the global-scale in an efﬁcient way.
Most existing approaches has only limited ability in identifying critical structures.
For example, Structure2vec (Dai et al.
and
Spectrum GCN (Defferrard et al.
(2016)) can only ﬁnd critical structures of very simple shape.
Patchy-San (Niepert et al.
(2016)) can only ﬁnd critical structures within a small area around each node (i.e., at the local-scale).
The only work that is able to identify critical structures at the globalscale is Spatial GCN (Bruna et al.
, but it has the complexity O ( N 3 ) , where N is the number of nodes, in both time and space, therefore is inefﬁcient for large graphs.
In this paper, we propose the Ego-CNN that embeds a graph into distributed (multi-layer), ﬁxeddimensional tensors.
The Ego-CNN can be jointly learned with a supervised task model (e.g., fully-connected layers for graph classiﬁcation) and help identify the task-speciﬁc critical structure at the global-scale.
Some existing studies like DCNN (Atwood & Towsley (2016)) and Patchy-San (Niepert et al.
(2016)) have borrowed the concept of CNN to embed graphs.
The idea is to model the ﬁlters / kernels that scan through different parts of the graph (which we call the neighborhood s) to learn patterns most helpful to the learning task.
Unlike images where a neighborhood can be clearly deﬁned as the nearby pixels across channels, graphs have no standard notion of neighborhoods.
Currently, the neighborhoods deﬁned by above works have some limitations in that (1) a neighborhood does not represent a local region of the graph so the embedding model cannot take advantage of the location invariant patterns to speed up the learning process, or (2) a neighborhood represents a local region of the graph but cannot be recursively deﬁned over the hidden layers, making the embedding model shallow and incapable of detecting critical structure at the global-scale.
Different from most existing approaches, Ego-CNN deﬁnes a neighborhood at layer l as the ( l + 1) -hop ego network centered at a speciﬁc node.
1 This makes the Ego-CNN efﬁcient since it learns the location invariant patterns.
Also, a neighborhood at layer l enlarges the receptive ﬁeld of the neighborhood centered at the same node at layer l − 1 .
Ego-CNN can exploit the multi-layer architecture to detect the critical structure at the global-scale in the deeper layers.
Furthermore, Ego-CNN works nicely with some common visualization techniques for CNNs (e.g., deconvolution proposed by Zeiler et al.
and can output the critical structures behind each prediction made by the trained task model.
Our contributions are
• We propose the Ego-CNN and show that an graph embedding based on ego networks can perform as well as (if not better than) existing approaches.
We conduct experiments on
many graph classiﬁcation tasks and the results show that Ego-CNN can lead to the stateof-the-art performance.
• By letting a neighborhood (an ego network) at a deeper layer share the same centering node with a neighborhood at the preceding layer (which we call the egocentric design ), EgoCNN supports a multi-layer architecture with enlarged receptive ﬁelds at deeper layers, thus can learn the critical structures at the global-scale.
• Ego-CNN, which learns location invariant patterns recursively, is efﬁcient.
Each convolution takes only O ( Nkd ) in both space and time for a ﬁlter to scan a graph with N nodes.
We also show that Ego-CNN can readily incorporate the scale-free prior, which commonly exists in large (social) graphs, to further improve the training efﬁciency.
To the best of our knowledge, Ego-CNN is the ﬁrst embedding model that can efﬁciently detect task-dependent critical structures at the global-scale.
We hope it helps practitioners better explain the learning results and discover new knowledge in the future.
R ELATED W ORK
We compare existing graph embedding models in Table 1 on the aspects of our goal, i.e. whether it jointly detects the global-scale critical structure and its computational complexity.
[Table]
Spectrum GCN (Bruna et al.
Defferrard et al.
; Kipf & Welling (2017)) !
O ( dn f | E | )
DCNN (Atwood & Towsley (2016))
O ( hn f | V | 2 ) Patchy-San
(Niepert et al.
O ( k 2 n f | V | )
Neural Fingerprint (Duvenaud et al.
O ( kn f | V | )
Ego-Convolution ! !
O ( kdn f L | V | )
Graph Kernels All graph kernels have a common drawback in that the embeddings are generated in a unsupervised manner, which means critical structure cannot be jointly detected at the generation of embeddings.
Here we introduce three state-of-the-art graph kernels.
The Weisfeiler-Lehman kernel (Shervashidze et al.
(2011)) grows the coverage of each node by collecting information from neighbors, which is conceptually similar to our method, but differs in that WL kernel collects only node labels, while our method collects the complete labeled neighborhood graphs from neighbors.
Deep
Graph Kernels (Yanardag & Vishwanathan (2015)) and Subgraph2vec (Narayanan et al.
(2016)) were inspired by word2vec (Mikolov et al.
models that embed structures by predicting neighbors’ structures.
Multiscale Laplacian Graph Kernels (Kondor & Pan (2016)) can compare graphs at multiple scales by recursively comparing graphs based on the comparison of subgraphs.
However, it’s very inefﬁcient in that comparing two graphs takes O ( L | V | 5 ) , where L represents the number of comparing scales.
Graphical Models Graph can also express the conditional dependence (edge) between random variables (node) in graphical models.
models.
Structure2vec (Dai et al.
(2016)) introduced a novel layer that makes the optimization procedures of approximation inference directly trainable by SGD.
It’s efﬁcient on large graphs with time complexity linear to number of nodes.
However, it’s weak at identifying critical structures since approximation inference makes too much simpliﬁcation on structures.
For example, mean-ﬁeld approximation assumes variables are independent of each other.
As a result, Structure2vec can only identify critical structures of very simple shape.
Convolution-based Methods
Recently, many works are proposed by borrowing the concept of CNN to embed graphs.
Figure 3 summarizes their deﬁnition of ﬁlters and neighborhoods.
W
N N
N filtersbyscan
N h k
N k filtersbyscan
N d filtersbyscan k N d filtersbyscan
[Figure]
The Spatial Graph Convolutional Network (GCN) was proposed by Bruna et al.
The design of Spatial GCN (Figure 3(a)) is very different from other convolution-based methods since its goal is to perform hierarchical clustering of nodes.
A neighborhood is deﬁned as a cluster.
However, the ﬁlter is not aim to scan for local patterns but to learn the connectivity of all clusters.
This means each ﬁlter is of size O ( N 2 ) if there’re N clusters.
Also, a ﬁlter requires the global-scale information, i.e. the features of all clusters to train, so it’s very inefﬁcient on large graphs.
Hence, in the same paper, Bruna et al.
(2013) proposed another version, the Spectrum GCN, to perform hierarchical clustering in the spectrum domain.
Although the efﬁciency is improved and many recent work such as Defferrard et al.
; Kipf & Welling (2017) further improve the training efﬁciency, the major drawback is that spectrums is weak at identifying structures since only special families of graphs such as complete graphs or star-like trees can be precisely described by spectrums.
Diffusion Convolutional Neural Networks (Atwood & Towsley (2016)) embed graphs by detecting the diffusion.
The neighborhood is deﬁned as the diffusion, which are paths starting from a node to other nodes in h hops.
The diffusion can be represented by a diffusion matrix.
They use ﬁlters to scan through each node’s diffusion matrix, so DCNN is possible to detect useful diffusion patterns.
But it cannot detect critical structures since the diffusion patterns cannot precisely describe the location and the shape of structures.
DCNN reported impressive results on node classiﬁcation.
But it’s inefﬁcient on graph classiﬁcation tasks since their notion of neighborhood is at the global-scale, which takes O ( hN 2 ) to embed a graph with N nodes.
Also, their deﬁnition of neighborhood makes the embedding model shallow, which is only a single layer in the neural network.
Patchy-San (Niepert et al.
(2016)) detects local neighborhoods by ﬁlters.
The deﬁnition of local neighborhood is based on the adjacency matrix of the graph, which is deﬁned as the local graphs formed by its k nearest neighbors.
The ﬁlter scans on the adjacency matrix of local neighborhoods.
Since expressing graphs by adjacency matrices is not invariant under different vertex ordering, they proposed neighborhood normalization to generate similar adjacency matrices for isomorphic neighborhoods.
Patchy-San can detect precise structures but it is inefﬁcient to scan for large local neighborhood, which limits it from detecting global-scale critical structures.
Also, the embedding model of Patchy-San is only a single layer, a question to ask is that is it possible to improve the efﬁciency by generalizing its deﬁnition of neighborhood into multiple layers?
If the adjacency matrix of neighborhoods is generated (e.g. the similarity of all embedding vectors of all neighborhoods), the neighborhood at next layer can be deﬁned as the k most similar neighborhoods.
However, this naive generalization don’t work for two reasons.
First, the concept of local neighborhood is lost since two neighborhoods that have similar embedding vectors are not necessary to be adjacent on the graph.
Second, the structure of neighborhood is dynamic since the k most similar neighborhoods changes during training.
Thus, this generalization (based on adjacency matrix) does not give neighborhoods corresponding to local regions
Therefore, the idea of Patchy-San cannot be readily extended to multiple layers.
To the best of our knowledge, the only work that scans on local neighborhoods and supports multiple layers is Neural Fingerprints (Duvenaud et al.
But the neighborhood at layer l represents only the approximation (specially, the summation) of the l -hop neighbors.
They use the approximation to avoid the vertex-ordering problem of adjacency matrix.
However, at the same time, Neural
Fingerprint loses the ability of detecting precise critical structures.
We know that CNN has two advantages: (1) detecting location invariant patterns and (2) overcoming the curse of dimensionality by multi-layer representations.
To keep these advantages on graphs, we have to rethink the deﬁnition of neighborhood from scratch.
E GO -CNN
The reason why the idea of Patchy-San fails to generalize into multiple layers is that its deﬁnition of neighborhood (which is based on adjacency matrix) makes the composition of neighborhoods at deeper layer not static and may not correspond to local regions in the graph.
Our main idea is to use the egocentric design , i.e. the neighborhood is deﬁned on the same node as all the preceeding layers.
The overview of our method is depicted in Figure 4.
[Figure]
Given a graph G of N nodes,
(1) Preprocess step: generate the neighborhood embeddings h (0) ∈ R N × d
and select the k nearest neighbors Nbr for all nodes
For each node i , we select its k nearest neighbors Nbr ( i ) , take its neighborhood as the subgraph formed by including node i itself and its k neighbors, and then embed this neighborhood to a ddim vector h (0) i ∈ R d .
In general, any existing graph embedding models can be used to embed the neighborhood graphs, but we recommend to use Patchy-San since our Ego-CNN can be directly stacked on top of it.
(2) Ego-CNN: output embeddings of the enlarged neighborhoods
The Ego-CNN is composed of several Ego-Convolution layers.
Our egocentric design is to deﬁne a neighborhood as its and its k neighbors’ neighborhoods at previous layer, which results in a larger neighborhood centered on the same node.
Figure 5 shows how this deﬁnition leads to the enlarged neighborhoods.
Suppose neighborhoods are originally 1 -hop ego networks.
Consider the bold black node in Figure 5, aggregating its neighbors’ neighborhoods (centered in black nodes), its neighborhood is stretched out by 1 -hop, becoming a 2 -hop ego network.
Speciﬁcally, we give the formal deﬁnition of our Ego-Convolution. A ∗ B denotes
M (cid:88) i =
N (cid:88) j =
A ij B ij for two matrices A, B of the same size.
The l -th Ego-Convolution layer generates node i ’s neighborhood embedding h ( l ) i ∈
R d l by h ( l ) ij = σ (cid:16) X ( l ) i ∗ W ( l ) j + b ( l ) j (cid:17) , ∀ j = 1 , · · · , d l , where σ is the activation function, X ( l )
i
[Equation]
 , n
[Equation]
[Equation]
, n
[Equation]
Nbr ( i ) are node i ’s neighbors and W ( l ) 1 , ·
· · , W ( l ) d l ∈ R ( k +1) × d l − 1 are ﬁlters at l -th layer, and b ( l ) 1 , · · · , b ( l ) d l ∈ R are bias terms.
That is, X ( l ) i is the concatenated matrix of its and its k neighbors’ neighborhood embeddings at previous layer.
The ordering of its neighbors n 1 · · · n k is determined using the neighborhood normalization proposed by Niepert et al.
(2016) in the the preprocessing step, where the goal of using neighborhood normalization is to ensure two isomorphic neighborhoods to have the same concatenated matrix.
It is important to note that X ( l ) i keeps the precise structure of node i ’s neighborhood.
As
[Equation]
preserve its neighbors’ neighborhood graphs, and the relative information about how its k neighbors’ neighborhoods are connected is kept in h ( l − 1) i and in the ordering of k neighbors.
Hence, by scanning through each X ( l ) i , the ﬁlters can capture precise neighborhood structures.
By contrast, previous works such as GCN(Kipf & Welling (2017)) and Neural Fingerprints(Duvenaud et al.
(2015)) fail to learn precise neighborhood structure as they generate node embeddings equivalently by
[Equation]
where A ( l )
[Equation]
R d l − 1 and c 0 , · · · , c k are constants.
Because the ﬁlters only learn the approximated neighborhood A ( l ) i (i.e. the summation of neighbors’ node embeddings).
The relative information of neighbors’ neighborhoods is lost due to the summation.
Task model: generate predictions
Given graph embedding h ( L ) ∈ R N × d L generated by Ego-CNN task models such as SVM or classiﬁcation layers can be used to generate predictions for the learning task.
Effective Receptive Field on Ambient Graph A receptive ﬁeld in Ego-CNN has the ambient coverage on graph that is effectively enlarged as network depth increases.
Figure 6 shows the coverage of a node’s receptive ﬁeld across layers.
Originally, this node’s neighborhood is a 1-hop ego networks (Figure 6(a)).
The coverage of its corresponding receptive ﬁeld is expanded into a 2-hop ego network (Figure 6(b)) by passed through 1 Ego-Convolution.
Note that it is not exactly its 2-hop ego network since edges across two neighbor’s ego networks are not captured.
As the depth of Ego-CNN increases, a receptive ﬁeld at the l -th Ego-Convolution layer approximates covers a ( l + 1) -hop ego network.
As shown in Figure 6(e), it’s possible to cover the entire graph by a single receptive ﬁeld.
The effectiveness of ambient coverage on graphs enables Ego-CNN to detect global-scale structures.
Besides, the effective coverage can be quite different for different receptive ﬁelds, which may reﬂect the positional information of the nodes.
For example, the receptive ﬁeld shown in Figure 6(e) can cover the entire graph, while the one in Figure 6(f) only covers a small part of the graph.
This
[Figure]
difference is related to the global position of these nodes.
By comparing receptive ﬁelds, we think it is possible for Ego-CNN to be aware of the positional information of nodes and then take advantages of it.
A DVANTAGES OF E GO -CNN 4. D ETECTING C RITICAL S TRUCTURES
Given a trained Ego-CNN, there are many ways to visualize critical structures.
Here we consider
Ego-CNN is used with the general task model, which can be several neural network layers or can be an additional classiﬁer such as SVM.
Our approach takes in two steps.
Find out the most important neighborhoods.
The simplest way is to add an Attention layer (Itti et al.
to generate a score representing the importance of each neighborhood.
To train the weights of the Attention layer, we ﬁx weights of previous layers (i.e. Patchy-San and Ego-Convolution) and add a Dense layer to train for the task.
We can pickup important neighborhoods by setting a threshold.
Identify critical regions in those important neighborhoods.
Since the critical parts is considered in the network’s perspective, we take the approach similar to Deconvolution (Zeiler et al.
For simplicity, assume we use ReLU as the activation function.
Given the node embedding of an important neighborhood (found by step 1) h ( l ) i ∈ R d l at the l -th layer, we can interpolate its receptive ﬁeld (which contains its and its k neighbors’ neighborhoods at previous layer) by d l (cid:88)
j = h ( l ) ij W ( l ) j =    h
( l − 1)
i h ( l − 1) n... h ( l − 1) n k 
  , where n 1 . . .
n k ∈ Nbr ( i ) .
Note that the interpolated values belonging to the same position at the ( l − 1) -th layer are summed up, which is the same as Deconvolution on traditional
CNNs.
Then, for each of the interpolated neighborhoods, we iteratively repeat the Deconvolution to interpolate their neighborhoods at the preceding layer until reaching the
Patchy-San layer getting many neighborhoods in the form of adjacency matrices.
These adjacency matrix are critical structures to the task, and the edge weight in adjacency matrix corresponds to the importance of an edge.
4. E FFICIENCY AND THE S CALE -F REE P RIOR
Computation Complexity Given a graph with N nodes.
To generate neighborhood embeddings at the l -th Ego-Convolution.
It takes O ( N ) to lookup embeddings of k neighbors’ neighborhoods and O ( Nk log k ) to concat neighborhoods according to neighbors’ global order, and
O ( N ( k + 1) d l − 1 d l ) to perform convolution through each receptive ﬁelds (of size ( k +1)
×
d l − 1 ) by d l ﬁlters.
In total, for a Ego-CNN with L Ego-Convolution layers, it takes O (cid:32)
N ( k + 1)
L (cid:88) l = d l − 1 d l (cid:33) to embed the graph.
Scale-Free Regularizer Scale-free networks (Li et al.
(2005)) are networks with self-similarity, which means the same patterns can be observed when zooming at different scales.
Speciﬁcally, scale-free networks can be determined by RS box-covering(Kim et al.
if it satisﬁes
N B ( l B ) ∼ l s B B , i.e. to cover the entire network by ﬁxed-length boxes, the box length l B and the required number of boxes N B ( l B ) follows a power-law.
In practice, the power-law degree distribution is usually a common indicator for scale-free networks.
Figure 7 shows the degree distribution of the Reddit dataset, where we use it to benchmark model performance on classiﬁcation tasks.
We found degree distribution of graphs in Reddit datasets follows a power-law.
This motivates us to think of the possibility of introducing scale-free priors into
Ego-CNN.
Recall that ﬁlters of the l -th Ego-Convolution layer is detecting the combinations of how neighborhoods at the ( l − 1) -th layer are combined into neighborhoods at the l -th layer.
Because of the self-similarity in the deﬁnition, one way to generate the scale-free network is to repeat the combination at each layer.
This means we can simply introduce scale-free prior into the Ego-CNN by weight-tying, i.e. let the ﬁlters share across different Ego-Convolution layers.
E XPERIMENTS
We perform a set of experiments to verify (i) the performance compared to existing graph embedding models on classiﬁcation datasets, (ii) the effectiveness of scale-free regularizer, and (iii) visualization of identiﬁed critical structures.
All experiments were performed on a computer with 48-core Intel(R)
Xeon(R) E5-2690 CPU, 64 GB RAM, and NVidia Geforce GTX 1070 GPU.
We use Tensorﬂow 1.
to implement our methods.
5. G RAPH C LASSIFICATION
We benchmark on both bioinformatic and social-network datasets, which are processed by Kersting et al.
Most graphs in bioinformatic datasets are labeled graphs that contain node and edge labels or attributes, while graphs are unlabeled (i.e. only structure of graph is given) in social network datasets.
More details about these datasets can be found in DGK (Yanardag & Vishwanathan (2015)).
The average test accuracy of 10-fold CV is reported to compared with existing methods.
The results is listed in Table 2, where ﬁgures of most baselines are directly taken from their papers since experimental settings are the same.
For simplicity, we use the same network architecture for all datasets.
To embed neighborhood graphs, we choose Patchy-San since it deals with labeled graphs and works nicely with CNN visualization techniques.
The architecture of network is composed of
Patchy-San layer (128 ﬁlter and k=10, which is the best setting for k reported in their paper), then follows our Ego-CNN with 5 Ego-Convolution layers (128 ﬁlters and k=16), and 2 Dense layers with 128 neurons as the task model.
To normalize neighbors, we follow Patchy-San’s approach to use 1-WL labels.
We train the network by Adam with learning rate 0.0001 and use Dropout with droprate 0.5 and Batch Normalization for regularization
[Table]
Our Ego-CNN improves the performance of Patchy-San on most datasets since the ﬁxed-size neighborhoods (each contains at most 10 neighbors) detected by Patchy-San are grown into larger neighborhoods by Ego-Convolution.
It makes sense on datasets such as MUTAG, where the task is to predict if compounds are mutagenic to DNA.
It is reported by Debnath et al.
(1991) that compounds with more than 3 benzene rings are very likely to be positive.
To detect benzene rings in compounds, it’s easier for Ego-CNN since we detect much larger local neighborhoods.
This shows that representing graphs by larger neighborhoods is helpful on these datasets.
5. S CALE -F REE R EGULARIZER
Since we found most graphs in Reddit dataset are scale-free graphs, we verify the effectiveness of scale-free regularizer on it.
The results are shown in Table 3.
[Table]
To introduce scale-free priors into network by weight-tying, we expand 1 Ego-Convolution layer to 5 Ego-Convolution layers with the same weight, so the total parameters is 1.3M, which is the same as network in 1st row but the advantage is that we improve the test accuracy by 3.5%.
Also, in the 3rd row, we compare against the network with same architecture but without using the scale-free regularizer.
The number of parameter is 1M more but does not give better performance.
This shows that introducing scale-free priors improves the learning efﬁciency, so that the same (or even better) performance can be obtained by training with fewer parameters.
5. V ISUALIZATION OF C RITICAL S TRUCTURES
As a sanity check, we generate two example compound datasets to visualize critical structures at the local-scale (Alkanes vs Alcohols) and global scale (Symmetric vs Asymmetric Isomers).
The structures of compounds are generated of different size and under different vertex-orderings.
First, we’d like to know if the network considers OH-base as critical structures.
The visualization result is shown in 8, where critical structures are plotted in grey color, and the node/edge size are proportional to its important score.
We ﬁnd that the OH-base in Alcohols is always consider critical no matter how long it its.
[Figure]
Next, we’d like to know how the network detects the concept of symmetry.
The visualization result of Symmetric vs Asymmetric Isomers dataset is shown below.
For symmetric isomers (Figure 9(a)), the critical structure, i.e. the symmetric hydrocarbon chains, is detected.
By carefully observing the nodes and edge inside the detected structures (color in grey), we ﬁnd an interesting thing that the importance of nodes/edges (i.e. which is plotted in different size proportional to the importance score for nodes and weights in interpolated adjacency for edges) are also roughly to be symmetric to the methyl-base.
This symmetric phenomenon of detected structure can also be observed in asymmetric isomers (Figure 9(b)).
We conjecture the network learns to compare if the two long hydrocarbon chains (which are branched from the methyl-base) are symmetric or not by starting comparing nodes and edges from the methyl-base all the way to the end of the hydrocarbon chains, which is similar to how people check if a structure is symmetric.
[Figure]
In the last experiment, we’d like to visualize the detected structures on Reddit dataset to see if it can help us to derive more knowledge about the dataset.
In Reddit dataset, each graph represents a discussion thread.
A node represent a user, and there’s an edge if two users discuss with each other.
The task is to classify the discussion style of threads.
i.e. wether it is a discussion-based thread (e.g. threads under Atheism subreddit) or a QA-based thread (e.g. threads under AskReddit subreddit).
From the visualization result (Figure 10), we observe that the detected structures seemed to be the “bridges” that interconnect highly active users who has many discussions with other users.
If each active user represents an strong opinion in the thread, the visualization result suggests that the discussion of different opinions are the key to discriminant discussion-based threads from QA-based threads.
[Figure]
C ONCLUSIONS
We propose Ego-CNN with idea of the egocentric design to deﬁne neighborhood on the same node across layers.
The egocentric design enables Ego-CNN to fully take advantages of CNN to perform efﬁcient learning on graphs, so that neighborhoods are enlarged as depth increases.
Also, training efﬁciency can be increased by introducing scale-free priors and visualization techniques for CNN can be used to visualize structures detected by Ego-CNN.
We hope Ego-CNN helps people in domains such as bioinformatic, cheminformatics to understand their experiment results.
The future direction is to further reduce the space requirement of Ego-CNN.
Since receptive ﬁelds at deeper Ego-Convolution layer represents larger neighborhoods, the overlapping area among neighborhoods of different nodes may also be enlarged.
So, instead of using all neighborhoods, it might be enough to embed graphs by only part of the neighborhoods.