E GO -CNN: A N E GO N ETWORK - BASED R EPRESENTATION OF G RAPHS D ETECTING C RITICAL S TRUCTURES
Anonymous authors
Paper under double-blind review
A BSTRACT
While existing graph embedding models can generate useful embedding vectors for graph-related tasks, what valuable information can be jointly learned from a graph embedding model is less discussed.
In this paper, we consider the possibility of detecting critical structures by a graph embedding model.
We propose
Ego-CNN to embed graphs, which works in a local-to-global manner to take advantages of CNNs that gradually expands the detectable local regions on the graph as the network depth increases.
Critical structures can be detected if Ego-CNN is combined with a supervised task model.
We show that Ego-CNN is (1) competitive to state-of-the-art graph embeddings models, (2) can work nicely with CNNs visualization techniques to show the detected structures, and (3) is efﬁcient and can incorporate with scale-free priors, which commonly occurs in social network datasets, to further improve the training efﬁciency.
I NTRODUCTION
A graph embedding algorithm converts graphs from structural representation to ﬁxed-dimensional vectors.
It is typically trained in a unsupervised manner for general learning tasks but recently, deep learning approaches such as Structure2vec (Dai et al. (2016)) and Diffusion Convolution Neural
Network (Atwood & Towsley (2016)) are trained in a supervised manner and shown superior results against unsupervised approaches on many tasks such as node classiﬁcation and graph classiﬁcation.
While many algorithms perform well on graph-related tasks, what valuable information can be jointly learned from the graph embedding is less discussed.
In this paper, we aim to develop a graph embedding model that jointly discovers the critical structures , i.e., partial graphs that are dominant to a learning task (e.g., graph classiﬁcation) where the embedding is applied to.
This helps people running the learning task to understand the reason behind the task results, and is particularly useful in certain domains such as the bioinformatics and social network analysis.
For example, people in bioinformatics may wish to tell whether a protein is an enzyme or not and why , as shown in Figure 1(a).
If a model can backtrack the most critical parts like the ones shown in Figure 1(b), it may help people understand that it is the circular structures (the “holes”) that distinguish enzyme from other substances.
New knowledge may then be discovered by investigating the functionality of these holes.
[Figure]
Unfortunately, identifying critical structures is a challenging task.
The ﬁrst challenge is that critical structures are task-speciﬁc —the shape and location of critical structures may vary from task to task.
This means that the graph embedding model needs to be learned together with the task model (e.g.,
2­methylnonane (asymmetric isomer) 5­methylnoane (symmetric isomer)
C
H H
H
C ...
H H
C
H H
OH
Alkane
Alcohol
C
H H
H
C ...
H H
C
H H
H
CH
H H
C ...
H
C
H H
H
H H C H
C
H H
H
C ...
H
C
H H
H
H H C H ...
2­methylnonane (asymmetric isomer) 5­methylnoane (symmetric isomer)
C
H H
H
C ...
H H
C
H H
OH
Alkane
Alcohol
C
H H
H
C ...
H H
C
H H
H
CH
H H
C ...
H
C
H H
H
H H C H
C
H H
H
C ...
H
C
H H
H
H H C H ...
(a) (b) a classiﬁer or regressor).
Another challenge is that critical structures may be found at the globalscale , as shown in Figure 2.
To discriminant Alcohols from Alkanes (Figure 2(a)), one can check if there exists an OH-base and if the OH-base is at the end of the compound.
In this task, the critical structure (OH-base) is relative small and can be identiﬁed at the local-scale.
But for the other task (Figure 2(b)), to identify if a methyl-nonane is symmetric or not, one must check the entire graph to know if the methyl is branched at the center position of the long carbon chain.
In this task, the critical structure is the symmetric hydrocarbon at the two sides of the methyl branch, which can only be found at the global-scale.
The second challenge is that ﬁnding out all matches of substructures in a graph is known as subgraph isomorphism and proven to be an NP-complete problem by Cook (1971).
To the best of our knowledge, there is no existing graph embedding algorithm that can identify critical structures up to the global-scale in an efﬁcient way.
Most existing approaches has only limited ability in identifying critical structures.
For example, Structure2vec (Dai et al.
and
Spectrum GCN (Defferrard et al.
(2016)) can only ﬁnd critical structures of very simple shape.
Patchy-San (Niepert et al.
(2016)) can only ﬁnd critical structures within a small area around each node (i.e., at the local-scale).
The only work that is able to identify critical structures at the globalscale is Spatial GCN (Bruna et al.
, but it has the complexity O ( N 3 ) , where N is the number of nodes, in both time and space, therefore is inefﬁcient for large graphs.
In this paper, we propose the Ego-CNN that embeds a graph into distributed (multi-layer), ﬁxeddimensional tensors.
The Ego-CNN can be jointly learned with a supervised task model (e.g., fully-connected layers for graph classiﬁcation) and help identify the task-speciﬁc critical structure at the global-scale.
Some existing studies like DCNN (Atwood & Towsley (2016)) and Patchy-San (Niepert et al.
(2016)) have borrowed the concept of CNN to embed graphs.
The idea is to model the ﬁlters / kernels that scan through different parts of the graph (which we call the neighborhood s) to learn patterns most helpful to the learning task.
Unlike images where a neighborhood can be clearly deﬁned as the nearby pixels across channels, graphs have no standard notion of neighborhoods.
Currently, the neighborhoods deﬁned by above works have some limitations in that (1) a neighborhood does not represent a local region of the graph so the embedding model cannot take advantage of the location invariant patterns to speed up the learning process, or (2) a neighborhood represents a local region of the graph but cannot be recursively deﬁned over the hidden layers, making the embedding model shallow and incapable of detecting critical structure at the global-scale.
Different from most existing approaches, Ego-CNN deﬁnes a neighborhood at layer l as the ( l + 1) -hop ego network centered at a speciﬁc node.
1 This makes the Ego-CNN efﬁcient since it learns the location invariant patterns.
Also, a neighborhood at layer l enlarges the receptive ﬁeld of the neighborhood centered at the same node at layer l − 1 .
Ego-CNN can exploit the multi-layer architecture to detect the critical structure at the global-scale in the deeper layers.
Furthermore, Ego-CNN works nicely with some common visualization techniques for CNNs (e.g., deconvolution proposed by Zeiler et al.
and can output the critical structures behind each prediction made by the trained task model.
Our contributions are
• We propose the Ego-CNN and show that an graph embedding based on ego networks can perform as well as (if not better than) existing approaches.
We conduct experiments on
many graph classiﬁcation tasks and the results show that Ego-CNN can lead to the stateof-the-art performance.
• By letting a neighborhood (an ego network) at a deeper layer share the same centering node with a neighborhood at the preceding layer (which we call the egocentric design ), EgoCNN supports a multi-layer architecture with enlarged receptive ﬁelds at deeper layers, thus can learn the critical structures at the global-scale.
• Ego-CNN, which learns location invariant patterns recursively, is efﬁcient.
Each convolution takes only O ( Nkd ) in both space and time for a ﬁlter to scan a graph with N nodes.
We also show that Ego-CNN can readily incorporate the scale-free prior, which commonly exists in large (social) graphs, to further improve the training efﬁciency.
To the best of our knowledge, Ego-CNN is the ﬁrst embedding model that can efﬁciently detect task-dependent critical structures at the global-scale.
We hope it helps practitioners better explain the learning results and discover new knowledge in the future.
R ELATED W ORK
We compare existing graph embedding models in Table 1 on the aspects of our goal, i.e. whether it jointly detects the global-scale critical structure and its computational complexity.
[Table]
Spectrum GCN (Bruna et al.
Defferrard et al.
; Kipf & Welling (2017)) !
O ( dn f | E | )
DCNN (Atwood & Towsley (2016))
O ( hn f | V | 2 ) Patchy-San
(Niepert et al.
O ( k 2 n f | V | )
Neural Fingerprint (Duvenaud et al.
O ( kn f | V | )
Ego-Convolution ! !
O ( kdn f L | V | )
Graph Kernels All graph kernels have a common drawback in that the embeddings are generated in a unsupervised manner, which means critical structure cannot be jointly detected at the generation of embeddings.
Here we introduce three state-of-the-art graph kernels.
The Weisfeiler-Lehman kernel (Shervashidze et al.
(2011)) grows the coverage of each node by collecting information from neighbors, which is conceptually similar to our method, but differs in that WL kernel collects only node labels, while our method collects the complete labeled neighborhood graphs from neighbors.
Deep
Graph Kernels (Yanardag & Vishwanathan (2015)) and Subgraph2vec (Narayanan et al.
(2016)) were inspired by word2vec (Mikolov et al.
models that embed structures by predicting neighbors’ structures.
Multiscale Laplacian Graph Kernels (Kondor & Pan (2016)) can compare graphs at multiple scales by recursively comparing graphs based on the comparison of subgraphs.
However, it’s very inefﬁcient in that comparing two graphs takes O ( L | V | 5 ) , where L represents the number of comparing scales.
Graphical Models Graph can also express the conditional dependence (edge) between random variables (node) in graphical models.
models.
Structure2vec (Dai et al.
(2016)) introduced a novel layer that makes the optimization procedures of approximation inference directly trainable by SGD.
It’s efﬁcient on large graphs with time complexity linear to number of nodes.
However, it’s weak at identifying critical structures since approximation inference makes too much simpliﬁcation on structures.
For example, mean-ﬁeld approximation assumes variables are independent of each other.
As a result, Structure2vec can only identify critical structures of very simple shape.
Convolution-based Methods
Recently, many works are proposed by borrowing the concept of CNN to embed graphs.
Figure 3 summarizes their deﬁnition of ﬁlters and neighborhoods.
W
N N
N filtersbyscan
N h k
N k filtersbyscan
N d filtersbyscan k N d filtersbyscan
[Figure]
The Spatial Graph Convolutional Network (GCN) was proposed by Bruna et al.
The design of Spatial GCN (Figure 3(a)) is very different from other convolution-based methods since its goal is to perform hierarchical clustering of nodes.
A neighborhood is deﬁned as a cluster.
However, the ﬁlter is not aim to scan for local patterns but to learn the connectivity of all clusters.
This means each ﬁlter is of size O ( N 2 ) if there’re N clusters.
Also, a ﬁlter requires the global-scale information, i.e. the features of all clusters to train, so it’s very inefﬁcient on large graphs.
Hence, in the same paper, Bruna et al.
(2013) proposed another version, the Spectrum GCN, to perform hierarchical clustering in the spectrum domain.
Although the efﬁciency is improved and many recent work such as Defferrard et al.
; Kipf & Welling (2017) further improve the training efﬁciency, the major drawback is that spectrums is weak at identifying structures since only special families of graphs such as complete graphs or star-like trees can be precisely described by spectrums.
Diffusion Convolutional Neural Networks (Atwood & Towsley (2016)) embed graphs by detecting the diffusion.
The neighborhood is deﬁned as the diffusion, which are paths starting from a node to other nodes in h hops.
The diffusion can be represented by a diffusion matrix.
They use ﬁlters to scan through each node’s diffusion matrix, so DCNN is possible to detect useful diffusion patterns.
But it cannot detect critical structures since the diffusion patterns cannot precisely describe the location and the shape of structures.
DCNN reported impressive results on node classiﬁcation.
But it’s inefﬁcient on graph classiﬁcation tasks since their notion of neighborhood is at the global-scale, which takes O ( hN 2 ) to embed a graph with N nodes.
Also, their deﬁnition of neighborhood makes the embedding model shallow, which is only a single layer in the neural network.
Patchy-San (Niepert et al.
whose embedding model is also a single layer network that detects ﬁxed-size local neighborhoods.
Its deﬁnition of local neighborhood is the subgraph formed by k nearest neighbors and is represented in k × k adjacency matrix.
The ﬁlter learns useful patterns by scanning through the k × k adjacency matrices.
Since expressing graphs by adjacency matrices is not invariant under different vertex ordering, they proposed neighborhood normalization to generate similar adjacency matrices for isomorphic neighborhoods.
Although Patchy-San can detect precise structures, its detectable local regions are limited to neighborhoods with at most k modes, making it inefﬁcient in detecting large local structures and also limits it from detecting global scale critical structures.
A question to ask is whether it is possible to stack multiple Patchy-San into layers to enlarge detectable local regions.
Unfortunately, it is not possible to directly stack multiple Patchy-San layers since its deﬁnition of neighborhood is for nodes but the outputs of Patchy-San represent neighborhoods.
To make it possible, a distance metric for neighborhoods must be deﬁned, so that the two necessary pieces in Patchy-San’s algorithm (i.e. k nearest neighbors and k x k adjacency matrices) can be generated.
A naive generalization is to use the similarity of neighborhood embeddings to select nearest neighborhoods and generate the adjacency matrices.
In this way, the detectable regions can be enlarged as depth increases.
However, this naive generalization don’t work for two reasons.
First, the concept of local neighborhood is lost since two neighborhoods that have similar embed- ding vectors are not necessary to be adjacent on the graph.
Second, the structure of neighborhood is dynamic since the k most similar neighborhoods changes during training.
Therefore, the algorithm of Patchy-San which is based on adjacency matrix cannot be readily extended to multiple layers.
To the best of our knowledge, the only work that scans on local neighborhoods and supports multiple layers is Neural Fingerprints (Duvenaud et al.
But the neighborhood at layer l represents only the approximation (specially, the summation) of the l -hop neighbors.
They use the approximation to avoid the vertex-ordering problem of adjacency matrix.
However, at the same time, Neural
Fingerprint loses the ability of detecting precise critical structures.
We know that CNN has two advantages: (1) detecting location invariant patterns and (2) overcoming the curse of dimensionality by multi-layer representations.
To keep these advantages on graphs, we have to rethink the deﬁnition of neighborhood from scratch.
E GO -CNN
The reason why the naive generalization of Patchy-San (in Section 2) fails is that neighborhoods deﬁned by similarity are not local neighborhoods and their composition are not static during training.
To overcome the problems, our main idea is to use the egocentric design .
We propose EgoConvolution that is egocentric in two aspects.
First, the embeddings generated by Ego-Convolution are egocentric graphs in that the i -th embedding represents the i -th node’s local neighborhood.
Second, neighborhoods are grown in an egocentric view in that they are enlarged by 1 -hop after each layer by collecting neighbors’ neighborhoods.
The overview of our method is depicted in Figure 4.
[Table]
[Figure]
[Table]
[Figure] [Table] [Figure] [Table]
H L d L
N
Given a graph G of N nodes,
(1) Preprocess step: generate neighborhood embeddings X ∈ R N × d and select all node’s k nearest neighbors Nbr For each node, a ﬁxed-size neighborhood and its k nearest neighbors are selected and the neighborhood is converted to a d -dimensional vector.
In general, any graph embedding algorithms can be used to convert neighborhood graphs. However, we recommend Patchy-San since our Ego-CNN can be directly stacked on top of it.
(2) Ego-CNN: output embeddings of the enlarged neighborhoods
The Ego-CNN is composed of multiple Ego-Convolution layers.
Our deﬁnition of neighborhood a node’s neighborhood and its k neighbors’ neighborhoods (which abbreviated as neighboring neighborhoods) at previous layer.
Figure 5 shows how this deﬁnition results in enlarged neighborhoods.
Suppose the given neighborhoods are all 1 -hop ego-network. Let’s consider the bold black node in Figure 5, whose neighborhood is originally 1 -hop. After aggregating its neighbors’ 1 -hop ego-network, its neighborhood is stretched out by 1 -hop becoming a 2 -hop ego-network.
Algorithm 1 shows the detail steps of Ego-Convolution. Algorithm 1 l - th Ego-Convolution layer
Require:
[Equation] [Equation] [Equation] [Equation]
[Equation]
For each node, its output embedding is generated by looking-up and concatenating the embeddings of its k neighboring neighborhoods with its own embeddings, which forms into a ( k + 1) ×
d l −matrix.
The goal of concatenating each node’s embedding is to preserve the structure between neighbors.
However, a problem is that it is possible for two isomorphic neighborhoods to have different concatenated matrix if the embeddings of k neighborhoods are put in non-consistent order.
This problem is similar to the vertex reordering problem in Niepert et al.
(2016), which can also be solved by their neighborhood normalization, whose idea is simply to assigned a global ordering of all nodes and then neighbors are sorted in non-increasing/decreasing global order.
(3) Task model: generate predictions
Given graph embedding H L ∈ R N × d L generated by Ego-CNN, task models such as SVM or classiﬁcation layer can be used to generate predictions for the learning task.
[Figure]
Effective Receptive Field on Ambient Graph A receptive ﬁeld in Ego-CNN has the ambient coverage on graphs that is effectively enlarged as network depth increases.
Figure 6 shows the coverage of a node’s receptive ﬁeld across layers.
The node’s neighborhood is originally a 1 -hop ego-network (Figure 6(a)). Its coverage grows into a approximate 2 -hop egonetwork (Figure 6(b)) by passed through 1 Ego-Convolution.
It is not exactly its 2-hop ego-network because edges across two neighbor’s ego networks are not captured.
As the depth of Ego-CNN increases, a receptive ﬁeld at l -th Ego-Convolution layer approximately covers a ( l + 1) -hop egonetwork.
As shown in Figure 6(e), it is possible to cover the entire graph by a single receptive ﬁeld.
The effectiveness of ambient coverage in graph enables Ego-CNN to detect global-scale structures.
Besides, the effective coverage by receptive ﬁelds can be quite different. The difference can reﬂect the positional information of the nodes.
For example, the receptive ﬁeld shown in Figure 6(e) can cover the entire graph, while the one in Figure 6(f) only covers a small part of the graph.
This difference is related the global position of these nodes.
By comparing receptive ﬁelds, we think it is possible for Ego-CNN to be aware of the positional information of nodes and then take advantages of it.
A DVANTAGES OF E GO -CNN 4. D ETECTING C RITICAL S TRUCTURES
Given a trained Ego-CNN, there are many ways to visualize critical structures.
Here we consider
Ego-CNN is used with the general task model, which can be several neural network layers or can be an additional classiﬁer such as SVM.
Our approach takes in two steps.
Find out the most important neighborhoods.
The simplest way is to add an Attention layer (Itti et al.
to generate a score representing the importance of each neighborhood.
To train the weights of the Attention layer, we ﬁx weights of previous layers (i.e. Patchy-San and Ego-Convolution) and add a Dense layer to train for the task.
We can pickup important neighborhoods by setting a threshold.
Identify critical regions in those important neighborhoods.
Since the critical parts is considered in the network’s perspective, we take the approach similar to Deconvolution (Zeiler et al.
, i.e. we interpolate outputs with ﬁlters to reconstruct the neighborhoods (represented by receptive ﬁelds) in a layer-by-layer fashion, from the deepest Ego-Convolution to 1st Ego-Convolution to Patchy-San layer, where the adjacency matrix of neighborhoods are interpolated.
The interpolated neighborhood graphs (in adjacency matrix) are critical structures to the task, and the edge weight in adjacency matrix corresponds to the importance of an edge.
4. E FFICIENCY AND THE S CALE -F REE P RIOR
Computation Complexity Given a graph with N nodes.
To generate neighborhood embeddings at the l -th Ego-Convolution.
It takes O ( N ) to lookup embeddings of k neighboring neighborhoods and O ( Nk log k ) to concat neighborhoods according to neighbors’ global order, and O ( N ( k + 1) d l − 1 d l ) to perform convolution through each receptive ﬁelds (of size ( k + 1) ×
d l − 1 ) by d l ﬁlters.
In total, for a Ego-CNN with L Ego-Convolution layers, it takes
O (cid:16) N ( k + 1) (cid:80) Ll =
d l − 1 d l (cid:17) to embed the graph.
Scale-Free Regularizer
The scale-free property commonly exists in real social networks.
The most notable property of scale-free networks is that the degree distribution follows a power-law.
Figure 7 shows the degree distribution of the Reddit dataset, where we use it to benchmark model performance on classiﬁcation tasks.
We found degree distribution of graphs in Reddit datasets follows a power-law.
This motivates us to think of the possibility of introducing scale-free priors into
Ego-CNN.
Recall that ﬁlters of the l -th Ego-Convolution layer is detecting the combinations of how neighborhoods at the ( l − 1) -th layer are combined into neighborhoods at the l -th layer.
In fact, one way to generate a scale-free network is to repeat the combination at each layer, which means we can simply introduce scale-free prior into the Ego-CNN by weight-tying, i.e. let the ﬁlters share across different Ego-Convolution layers.
E XPERIMENTS
We perform a set of experiments to verify (i) the performance compared to existing graph embedding models on classiﬁcation datasets, (ii) the effectiveness of scale-free regularizer, and (iii) visualization of identiﬁed critical structures.
All experiments were performed on a computer with 48-core Intel(R)
Xeon(R) E5-2690 CPU, 64 GB RAM, and NVidia Geforce GTX 1070 GPU.
We use Tensorﬂow 1.
to implement our methods.
5. G RAPH C LASSIFICATION
We benchmark on both bioinformatic and social-network datasets, which are processed by Kersting et al.
Most graphs in bioinformatic datasets are labeled graphs that contain node and edge labels or attributes, while graphs are unlabeled (i.e. only structure of graph is given) in social network datasets.
More details about these datasets can be found in DGK (Yanardag & Vishwanathan (2015)).
The average test accuracy of 10-fold CV is reported to compared with existing methods.
The results is listed in Table 2, where ﬁgures of most baselines are directly taken from their papers since experimental settings are the same.
For simplicity, we use the same network architecture for all datasets.
To embed neighborhood graphs, we choose Patchy-San since it deals with labeled graphs and works nicely with CNN visualization techniques.
The architecture of network is composed of
Patchy-San layer (128 ﬁlter and k=10, which is the best setting for k reported in their paper), then follows our Ego-CNN with 5 Ego-Convolution layers (128 ﬁlters and k=16), and 2 Dense layers with 128 neurons as the task model.
To normalize neighbors, we follow Patchy-San’s approach to use 1-WL labels.
We train the network by Adam with learning rate 0.0001 and use Dropout with drop-rate 0.5 and Batch Normalization for regularization.
Our Ego-CNN improves the performance of Patchy-San on most datasets since the ﬁxed-size neighborhoods (each contains at most 10 neighbors) detected by Patchy-San are grown into larger neighborhoods by Ego-Convolution.
It makes sense on datasets such as MUTAG, where the task is to predict if compounds are mutagenic to DNA.
It is reported by Debnath et al.
(1991) that compounds with more than 3 benzene rings are very likely to be positive.
To detect benzene rings in compounds, it’s easier for Ego-CNN since we detect much larger local neighborhoods.
This shows that representing graphs by larger neighborhoods is helpful on these datasets.
5. S CALE -F REE R EGULARIZER
Since we found most graphs in Reddit dataset are scale-free graphs, we verify the effectiveness of scale-free regularizer on it.
The results are shown in Table 4.
To introduce scale-free priors into network by weight-tying, we expand 1 Ego-Convolution layer to 5 Ego-Convolution layers with the same weight, so the total parameters is 1.3M, which is the same as network in 1st row but the advantage is that we improve the test accuracy by 3.5%.
Also, in the 3rd row, we compare against the network with same architecture but without using the scale-free regularizer.
The number of parameter is 1M more but does not give better performance.
This shows [Table] [Table] that introducing scale-free priors improves the learning efﬁciency, so that the same (or even better) performance can be obtained by training with fewer parameters
. 5. V ISUALIZATION OF C RITICAL S TRUCTURES
As a sanity check [Table] we generate two example compound datasets to visualize critical structures at the local-scale (Alkanes vs Alcohols) and global scale (Symmetric vs Asymmetric Isomers).
The structures of compounds are generated of different size and under different vertex-orderings.
First, we’d like to know if the network considers OH-base as critical structures.
The visualization result is shown in 8, where critical structures are plotted in grey color, and the node/edge size are proportional to its important score.
We ﬁnd that the OH-base in Alcohols is always consider critical no matter how long it its.
Next, we’d like to know how the network detects the concept of symmetry.
The visualization result of Symmetric vs Asymmetric Isomers dataset is shown below.
For symmetric isomers (Figure 9(a)), the critical structure, i.e. the symmetric hydrocarbon chains, is detected.
By carefully observing the nodes and edge inside the detected structures (color in grey), we ﬁnd an interesting thing that the importance of nodes/edges (i.e. which is plotted in different size proportional to the importance score for nodes and weights in interpolated adjacency for edges) are also roughly to be symmetric to the methyl-base.
This symmetric phenomenon of detected structure can also be observed in asymmetric isomers (Figure 9(b)).
We conjecture the network learns to compare if the two long hydrocarbon chains (which are branched from the methyl-base) are symmetric or not by starting comparing nodes and edges from the methyl-base all the way to the end of the hydrocarbon chains, which is similar to how people check if a structure is symmetric.
In the last experiment, we’d like to visualize the detected structures on Reddit dataset to see if it can help us to derive more knowledge about the dataset.
In Reddit dataset, each graph represents a discussion thread.
A node represent a user, and there’s an edge if two users discuss with each other.
The task is to classify the discussion style of threads.
i.e. wether it is a discussion-based thread (e.g. threads under Atheism subreddit) or a QA-based thread (e.g. threads under AskReddit subreddit). Network architecture Weight-tying? 10-Fold CV
[Figure]
acc (%) Total # of parameters Patchy-San + 1 Ego-Convolution 84. 1.3M
Patchy-San + 5 Ego-Convolution ! 88. 1.3M Patchy-San + 5 Ego-Convolution 87.
2.3M
[Figure]
From the visualization result (Figure 10), we observe that the detected structures seemed to be the “bridges” that interconnect highly active users who has many discussions with other users.
If each active user represents an strong opinion in the thread, the visualization result suggests that the discussion of different opinions are the key to discriminant discussion-based threads from QA-based threads.
C ONCLUSIONS
We propose Ego-CNN with idea of the egocentric design to deﬁne neighborhood on the same node across layers.
The egocentric design enables Ego-CNN to fully take advantages of CNN to perform efﬁcient learning on graphs, so that neighborhoods are enlarged as depth increases.
Also, training efﬁciency can be increased by introducing scale-free priors, and visualization techniques for CNN can be used to visualize structures detected by Ego-CNN.
We hope Ego-CNN helps people in domains such as bioinformatic, cheminformatics to understand their experiment results.
The future direction is to further reduce the space requirement of Ego-CNN.
Since receptive ﬁelds at deeper Ego-Convolution layer represents larger neighborhoods, the overlapping area among neighborhoods of different nodes may also be enlarged.
So, instead of using all neighborhoods, it might be enough to embed graphs by only part of the neighborhoods.