Utilizing Edge Features in Graph Neural Networks via Variational Information Maximization
Anonymous authors
Paper under double-blind review
Abstract
Graph Neural Networks (GNNs) broadly follow the scheme that the representation vector of each node is updated recursively using the message from neighbor nodes, where the message of a neighbor is usually pre-processed with a parameterized transform matrix.
To make better use of edge features, we propose the Edge Information maximized Graph Neural Network (EIGNN) that maximizes the Mutual Information (MI) between edge features and message passing channels.
The MI is reformulated as a diﬀerentiable objective via a variational approach.
We theoretically show that the newly introduced objective enables the model to preserve edge information, and empirically corroborate the enhanced performance of MI-maximized models across a broad range of learning tasks including regression on molecular graphs and relation prediction in knowledge graphs.
Introduction
Many real-world datasets naturally come in the form of graphs, such as citation networks (Kipf & Welling, 2017), social networks Hamilton et al.
knowledge graphs (Schlichtkrull et al., 2018), molecular graphs (Scarselli et al., 2009; Duvenaud et al., 2015) etc., all of which consist of a number of nodes and edges equipped with their inherent features.
Recently, impressive performance has been achieved in graph learning tasks with various forms of
Graph Neural Networks (GNNs) (Zhou et al., 2018).
Compared to prior works, such as node2vec (Grover & Leskovec, 2016), GNNs learn the state of a node by recursively aggregating messages from its neighbors: combining the graph structure with node features.
Intuitively, edge features should play an important role in graph learning tasks.
For example, chemical bonds in a molecule have a high impact on chemical properties of molecules, and edge features in knowledge graphs encode important relations between concepts, data, and entities.
Our proposed method focuses on improving the usage of edge features in GNNs.
The expressive power of GNNs largely depends on how the message is passed between nodes.
A widely adopted scheme is multiplying neighbor node states with a parameterized transform matrix before aggregation (Gilmer et al., 2017; Xu et al., 2019).
Despite tremendous success of GNNs, existing models do not exhaustively exploit the full potentials of edge features on graphs.
For example, many GNNs such as GCN (Kipf & Welling, 2017), ChebyNet (Deﬀerrard et al., 2016) and GAT (Veličković et al., 2018) do not even consider categorized edge types.
To utilize edge features in multi-relational graphs, RGCN (Schlichtkrull et al., 2018) proposes to learn a diﬀerent transform matrix for each edge type, respectively.
However, it does not generalize to edge features in continuous space.
MPNN (Gilmer et al., 2017) introduces an edge network that takes edge feature vectors as input and outputs transform matrices, which are used to transform states of neighbor nodes.
In principle, the MPNN framework can handle complex edge features.
Yet, the lack of maximization of MI between edge and message channels implies that the MPNN may give an edge-independent transform matrix.
In this work, we aim to more eﬃciently exploit the full potentials of edge features from the perspective of training.
We propose the E dge I nformation maximized G raph N eural
N etwork (EIGNN) that maximizes the Mutual Information (MI) between edge features and the message passing channel which is parameterized as the transform matrix in the widely-accepted message passing framework (transformation and aggregation)
(Gilmer et al., 2017; Xu et al., 2019).
Considering the challenge of computing the MI, we adopt a variational approach to reformulate it as an diﬀerentiable objective, which can be easily applied as a regularization term.
We theoretically show that EIGNN can reduce information loss of edge features.
Apart from demonstrating the impressive performance of EIGNN on extensive benchmarks of molecular graphs and knowledge graphs, we also analyze and attribute the enhanced eﬀectiveness of EIGNN to the exploitation of edge features instead of the regularization eﬀects.
Notably, attribution analysis on molecular graphs show that EIGNN can capture domain knowledge without human interference.
Preliminaries Let G =
( V , E ) be a graph with node feature vectors x v ∈ R d for node v ∈ V and edge feature vectors e vw ∈ E for the edge connecting node v and w .
In GNNs, the state of each node is updated recursively using neighbor nodes.
Let N v be the set of neighbor nodes of v and h ( l ) v
∈ R
d l be the hidden state of v at l -th layer, where d l is the dimension of the hidden layer.
For simplicity of notation, we use a single d to denote the dimension such that h ( l ) v ∈ R d .
We also have h (0) v = x v at the input layer.
Related works 2. Relational Modeling in Graph Neural Networks
Single-relational modeling.
Many variants such as GCN (Kipf & Welling, 2017),
GAT (Veličković et al., 2018), ChebyNet (Deﬀerrard et al., 2016), GraphSAGE (Hamilton et al., 2017) focus on learning node states.
These models can assign weight to neighbors, but they can not handle various edge features.
A typical neighborhood aggregation scheme is
[Equation]
where σ denotes an activation function, α vw can be a normalization constant or a learned attention coeﬃcient (Veličković et al., 2018).
States of all neighbors are multiplied by the same trainable transform matrix W ( l ) 1 .
Sometimes the self-connection is also treated in the same way, s.t., W ( l )
= W ( l ) 1 .
Multi-relational modeling.
A simple strategy to handle multi-relational graphs is assigning each edge type with a separate transform matrix as presented in RGCN (Schlichtkrull et al., 2018) and adopted by GGNN (Li et al., 2016) and
LNet (Liao et al., 2019).
RGCN updates node states according to the following scheme
[Equation]
where N rv is the collection of neighboring nodes of v with relation r ∈ R and α vw,r is a normalization constant similar as α vw in Eq.
Such a scheme faces challenge in handling edge features of continuous space.
GGNN and LNet do not focus on the improvement of edge expressibility.
GGNN introduces Gated Recurrent Unit (GRU) (Cho et al., 2014) and
LNet focus on handling multi-scale connections.
Complex-relational modeling.
The relation in a graph can be quite complex, expressed as a general feature vector e .
MPNN (Gilmer et al., 2017) introduces an edge network which takes edge feature vectors as input and outputs transform matrices.
A single edge network is shared in a MPNN model.
The forward propagation is formalized as
[Equation]
where f : e → W denotes the edge network.
Recently, some research works treat a multirelational problem as the complex-relational one by introducing a continuous edge embedding vector for each edge type, so as to handle increasing number of relations (Nathani et al., 2019).
Although the MPNN architecture allows the usage of arbitrary edge features, this advantage is not utilized in practice.
MPNN can actually learn an edge-independent transform matrix.
A GNN model that eﬃciently utilizes edge features is yet to emerge.
2. Readout functions
After several forward propagations, GNNs yield ﬁnal states of all nodes, which are suitable for node/edge classiﬁcation or regression.
For graph classiﬁcation or regression tasks, we can apply a readout function (Ying et al., 2018;
Vinyals et al., 2015) such that
[Equation]
where h Lv is the state of v at the last layer, R is the readout function that outputs a graph-level representation y , e.g., summing up the ﬁnal node states, applying hierarchical pooling (Ying et al., 2018) or using the set2set model (Vinyals et al., 2015).
Our method 3. The Usage of Mutual Information
In probability theory and information theory, MI is a measure of mutual dependence between two random variables.
Our method proposes to preserve edge information in GNNs, which is important in many real-world graph structures such as molecules - apart from node (atom) features, attributes of edges (bonds) are equally important for predicting properties of molecules.
To this end, we maximize I ( e ; W ) - the MI between the edge feature vector e and the message passing channel, i.e., the transform matrix W which is used to transform neighbor node states in the forward propagation.
Our method can be easily generalized to directly maximize the MI between edge features and the message itself in methods that do not explicitly have the transform matrix, e.g., the message from node w to node v can be expressed as f ( h v , e vw , h w ) rather than f ( e vw ) h w , which is shown in Section 4.3.
An general principle of maximum MI is described for unsupervised learning task by (Linsker, 1988) and MI inspired objective functions have long been adopted in unsupervised learning (Bridle et al., 1992; Barber & Agakov, 2006; Veličković et al., 2019;
Hjelm et al., 2019), semi-supervised learning (Krause et al., 2010) and generative adversarial networks (Chen et al., 2016).
Speciﬁcally, DGI (Veličković et al., 2019) also applies MI to GNNs.
DGI proposes to learn node-wise representations in an unsupervised manner by maximizing the MI between node representations and corresponding high-level summaries of graphs, using adversarial learning and negative sampling.
The node representations may then be retrieved and used for downstream tasks, such as node classiﬁcation.
DGI can be used to pre-train GNNs, as demonstrated in Hu et al.
Our EIGNN also proposes information maximization but targets a completely diﬀerent objective and adopts a quite diﬀerent approach.
3. A Variational Approach to Maximize Mutual Information
Computing I ( e ; W ) itself is intractable in practice, needless to say that training a model requires the derivative.
Thus, we adopt a variational approach (Agakov, 2004) to reformulate
I ( e ; W ) as a diﬀerentiable objective.
We show that our objective is an approximated lower bound of I ( e ; W ) and notably, optimizing our objective does lead to maximizing I ( e ; W ) .
Following MPNN (Gilmer et al., 2017), we use an edge network to parameterize the transform matrix W and relate it to edge features.
Therefore, the prior p ( W | e ) is
[Equation]
where δ ( · ) is the Dirac delta function.
The posterior p ( e | W ) is intractable, so we deﬁne a variational distribution q ( e | W ) , which can be obtained by deﬁning a neural network g : W → e .
Speciﬁcally, q ( e | W ) substitutes to some distribution (such as Gaussian distribution) with parameter g ( W ) .
In this way, f and g are similar to the probabilistic encoder and decoder in the Variational Auto-Encoder (VAE) (Kingma & Welling, 2013).
Then we can approximate
I ( e ; W ) with a diﬀerentiable objective L I ( f, g ; e ) as follows.
Theorem 1.
Let e be the edge feature vector, W be the transform matrix with conditional distribution p
( W | e ) speciﬁed by the probabilistic encoder f as shown in Eq.
(5) and q ( e | W ) be the variational distribution speciﬁed by the probabilistic decoder g , then we have
[Equation]
where L I ( f, g ; e ) = log q ( e | f ( e )) and H ( · ) denotes the entropy.
Proof.
Let D KL ( · (cid:107) · ) denote the KL-divergence, which should be nonnegative, then we have
[Equation]
where the equality ( a ) follows from Eq.
According to Theorem
[Equation]
we can maximize the variational lower bound for I ( e ; W ) .
The bound becomes tight when the variational distribution q ( e | W ) approaches the true posterior p ( e | W ) .
Moreover, H ( e ) is a constant because the distribution of edge feature vector e is ﬁxed for given graphs, hence we can equivalently maximize L I ( f, g ; e ) .
We choose the widely accepted Gaussian distribution as the prior distribution for the probabilistic decoder g ,
[Equation]
Then we have
[Equation]
where λ > 0 is a constant determined by σ and the dimension of e , taken as a tunable parameter.
The following Theorem 2 shows that maximizing the objective in Eq.
(8) does lead to the maximization of I ( e ; W ) , hence enables the model to preserve edge information.
Theorem 2.
Assume the optimal solution of maximizing L I ( f, g ; e ) is f (cid:63) and g (cid:63) , then f (cid:63) also maximizes I ( e ; W ) .
Proof.
Note that H ( e ) is a constant when the graphs are given.
In information theory, we have
[Equation]
I ( e ; W ) is upper bounded by H ( e ) ,
[Equation]
Since f (cid:63) and g (cid:63) is the optimal solution of maximizing L I ( f, g ; e ) presented in Eq.
(8), it is not diﬃcult to see that e = g (cid:63) ( f (cid:63) ( e )) , ∀ e ∈ E .
In this case, the inequalities in Eq.
9 become equalities, i.e.,
[Equation]
Therefore, we have I ( e ; W ) =
H ( e ) , i.e., the maximum is attained in this case.
3. Edge Information Maximized Graph Neural Networks
Our EIGNN is derived by implementing our MI objective in GNNs.
As a concrete example, the forward propagation of our model follows the formulation in Eq.
(3), where the dege network f : e → W is expressed as a multi-layer perceptron (MLP).
According to theoretical analysis presented in Sec. 3.2 , we introduce another MLP g : W → e as the decoder.
For graph regression or classiﬁcation tasks, the model outputs a prediction y for each graph
G , which has label ˆ y .
Without MI maximization, we denote the vanilla loss as L 0 (ˆ y, y ; G ) .
Common choice of L 0 includes Mean Square Error (MSE), Mean Absolute Error (MAE) and
Cross Entropy (CE).
For a graph G =
( V , E )
, EIGNN maximizes L I ( f, g ; e ) and minimizes
L 0 (ˆ y, y ; G ) using the following loss function
[Equation]
where E e ∈E [ · ] denotes taking the mean over all edges in G =
( V , E ) and λ is the regularization parameter.
When EIGNN is trained using mini-batches, L 0 (ˆ y, y ; G ) is averaged over all graphs in the batch while L I ( f, g ; e ) is averaged over all edges of all graphs in the batch.
Similarly, for relational prediction tasks in knowledge graphs, EIGNN directly yields nodelevel representations h v for each node v ∈ V and edge-level representations e for each relationship.
The objective function of EIGNN can be derived from the translational scoring function Bordes et al.
(2013), which learns embedding such that for a given valid triple t vw =
( h v , e vw , h w ) from the valid set S , the condition d t vw =
h v + e vw − h w ≈ 0 holds.
Let
L 0 = E t vw ∈ S E t (cid:48) vw ∈ S (cid:48) max { d t (cid:48) vw − d t vw + γ, 0 } , where S (cid:48) denotes a set of invalid triples and γ is a margin hyper-parameter, EIGNN can be trained by minimizing the following loss,
[Equation]
Experiments
In this section, we ﬁrst conduct experiments on a large quantum chemistry benchmark QM9, which is challenging for most baselines.
Then we evaluate EIGNN on several useful molecule benchmarks and use attribution analysis to show that EIGNN increases the impact of edges and captures domain knowledge without human interference.
Finally, we adopt our method to large-scale knowledge graphs and evaluate the performance on challenging relation prediction tasks using a wide variety of real-world datasets.
All experimental results demonstrate a clear and substantial improvement of EIGNN over the state-of-the-art methods.
4. Quantum Chemistry
QM9 (Ramakrishnan et al., 2014) is a large benchmark containing 134k molecules withquantum chemistry regression properties, which have been show to be quite challenging for many GNNs (Gilmer et al., 2017).
Feature engineering of nodes and edges exactly follows (Gilmer et al., 2017) such that molecules are preprocessed as graphs according to atom features and bond features.
We compare our EIGNN with nine state-of-the-art baselines which can be categorized into three groups according to the ability of handling edge features: i) GCN, ChebyNet, GAT and GIN (Xu et al., 2019) which simply use binary edge features to indicate the existence of a bond without any other edge features; ii) RGCN, GGNN, LNet and simpliﬁed MPNN (sMPNN) which consider bond types (no bond, single, double, triple, or aromatic); iii) MPNN and our EIGNN which use edge feature vectors to indicate both edge types and pairwise distance between atoms.
For a fair comparison, we repeat all experiments 3 times with diﬀerent random seeds while during each run, all methods share the same random seed.
We randomly choose 10k molecules for validation, 10k molecules for testing, and keep the rest for training.
Each target property is normalized to zero mean and unit variance for training.
Each model is trained to predict the 12 target properties simultaneously.
λ is naively set to 1 for EIGNN.
We use mean square error (MSE) loss to train the models for at most 300 epochs till convergence, and the performance is measured by mean absolute error (MAE).
For LNet and GGNN, implementation of the readout function follows the original paper.
While for all other models,
[Table]
mu 0.
0.097 alpha 0.
0.294 HOMO (10 − 3 ) 5.
2.230 LUMO (10 − 3 ) 6.
2.593 gap (10 − 3 ) 8.
3.275 R 53.
5.646 ZPVE (10 − 3 ) 2.
0.612 U 2.
0.357 U 2.
0.357 H 2.
0.357 G 2.
[Table]
[Figure]
we use the same set2set (Vinyals et al., 2015) readout, which has been demonstrated to work well in (Gilmer et al., 2017).
In Table 1, we list regression results for all methods.
We report individual MAE for each target in their original scale, averaged MAE (Avg. MAE) over 12 properties, and averaged normalized MAE (Avg.
nMAE; averaged over normalized target properties since diﬀerent targets have diﬀerent units and ranges).
Our EIGNN achieves the best performance for each metric and each target.
Now we are ready to answer the following research questions.
i) Are edge features important?
Yes.
The error has a trend of decreasing with increasing edge features.
The comparison between sMPNN (using edge types) and MPNN (using edge types and distance) directly veriﬁes the importance of edge features.
It is also consistent with the expert knowledge that distances between pairwise atoms are closely related to quantum properties.
For example, the smaller the distance between the two atoms, the stronger the bond is, and consequently a higher bond energy is associated with this atom pair.
ii)
Does the EIGNN work?
Yes.
EIGNN achieves the best performance on each target, outperforming the strong baseline MPNN.
Moreover, the advantage of EIGNN over MPNN is consistent over 3 runs and the standard deviation on this task is quite small.
Detailed results are shown in Table 4 of Appendix A. iii)
How does the EIGNN work?
Our MI objective is easily implemented on top of vanilla loss function.
We have shown that our objective enables preserving of edge information.
Fig.
1 demonstrates that regularization such as L 2 weight decay can increase training error while our objective does not.
Moreover, the validation performance veriﬁes that regularization itself does not reduce the validation error.
Thus, the eﬀectiveness of EIGNN is due to exploiting edge features rather than the regularization eﬀect.
We further run an ablation study where we concatenate edge features to node representations (i.e., MPNN+concat in Fig. 1) in message passing.
Concatenation is unable to identify correlations between edges and nodes (Gilmer et al., 2017) and our results show that it slightly reduces the mean validation error but increases the variance.
4. More Molecule Benchmarks with Potential Applications
We further evaluate EIGNN on three molecule benchmarks: Lipophilicity (Wu et al., 2018),
ESOL (Delaney, 2004) and FreeSolv (Mobley & Guthrie, 2014).
These datasets contain fewer molecules, and have potential usages in applications such as chemistry, drug discovery, and materials science.
For example, the property lipophilicity is an important feature of drug molecules that aﬀects both membrane permeability and solubility.
The dataset Lipophilicity contains 4200 compounds.
ESOL provides water solubility data for 1128 compounds.
FreeSolv contains hydration free energy of 642 small molecules in water.
We conduct graph regression experiments on these benchmarks.
All datasets are split into training, validation and test according to a proportion of 0.8/0.1/0.1.
MPNN and our EIGNN share the same architecture with 3 layers of message passing and 3 steps of set2set.
We repeat each experiment 3 times
[Table]
[Figure]
with diﬀerent random seeds.
Results of testing root mean square error (RMSE) in Tableverify the eﬀectiveness of our method.
Our EIGNN outperforms MPNN on each dataset and each run.
Detailed results for each run are presented in Appendix B.
Attribution analysis.
To understand how our EIGNN reduces the regression error, we conduct attribution analysis, i.e., attributing the prediction of a deep network to its input features, which usually builds up on the standard gradient operator (Sundararajan et al., 2017).
For an output y , i.e., the prediction of a GNN, we deﬁne its sensitivity to an edge with features e as S e = | ∂y∂e | , where |·| denotes the L 1 norm.
Similarly, the sensitivity to an atom with features x is S x = | ∂y∂x | .
Then S e and S x are used as the metrics of attribution in our experiments.
As an example, we show in Fig.
2 the attribution for two molecules in Lipophilicity: (a) Nc1nonc1C(=NO)Nc2ccc(F)c(Cl)c2 and (b) Oc 1 c 2 ncc 3 ccccc 3 c 2 nn 1 c 4 ccccc 4 .
Molecule in (a) has the potential to be used to treat, prevent and/or diagnose cancer Prinz et al.
Compared with MPNN, we can observe an increasing of overall edge attribution under our EIGNN and a decreasing of prediction error in both cases.
Interestingly, the attribution under EIGNN is similar to the expert knowledge of chemists: halogen atoms such as {Cl, Br, I} and their bond with the carbon atom greatly eﬀect the lipophilicity of a molecule Wilcken et al.
(2013), while atoms {O, N} also have high impact on the lipophilicity but usually in a negative way (Augustijns & Brewster, 2007).
In Fig.
2 (a), the attribution of the halogen bond C-Cl and the pair {O, N} under our EIGNN is much higher than the one under MPNN, which is consistent with the expert knowledge.
In Fig.
2 (b), similarly, the attribution of atoms {O, N} and the bond C-O under EIGNN is much higher.
More examples are presented in Appendix C.
4. Predicting Relations in Knowledge Graphs
In this subsection, we adopt EIGNN to tackle the problem of relation prediction in knowledge graphs (KGs), which entails predicting whether a given triple is valid or not.
For example, a triple (London, capital of, United Kingdom) should be classiﬁed as valid or London should be predicted as the capital of United Kingdom .
KGs represent human knowledge as a directed graph, and have been widely used in practical applications, such as semantic search, dialogue generation, question answering etc.
Recovering missing relations in KGs have been a major task for practical usages of KGs.
We evaluate our methods on three benchmark datasets,
WN18RR (Dettmers et al., 2018),
FB15k-237 (Toutanova et al., 2015) and NELL-995 (Xiong et al., 2017).
Without the reversible relation problem (Dettmers et al., 2018), WN18RR includes 11 relations scraped from WordNet for 40 , 943 synsets.
FB15k-237 is a subset of
Freebase, and contains 14 , 541 entities associated with 237 types of edge.
NELL-995 is constructed from the 995 th iteration of NELL system, containing 75 , 492 entities andtypes of edge.
[Table]
A critical issue of applying Eq.
(3) to KGs is that high-dimensional embedding vectors are required to distinguish massive amount of entities and relations, leading to a rapid growth in number of parameters in EIGNN.
To address this issue, we adopt the architecture of (Nathani et al., 2019) and learn graph attention based embeddings that target relation prediction on
KGs as follows,
[Equation]
Compared with Eq.
(3), where m vw = f
( e vw ) h w , the above equation absorbs the transform matrix f ( e vw ) into f ( h ( l ) v , e vw , h ( l ) w ) and reduces the model parameters.
In the following experiments, we implement f using a MLP as in previous experiments, and maximizes the MI between e vw and m vw by introducing another MLP with λ = 0 . 01 .
Multi-head attention is further introduced to stabilize the learning process and encapsulate more information about neighbors according to (Veličković et al., 2018).
After training EIGNN, ConvKB (Nguyen et al., 2018) is adopted as a regression function for a given triple by analyzing the global embedding properties across each dimension.
In the relation prediction task, the aim is to predict a triple ( v, e vw , w ) with v or w missing.
We can generate a set of candidate triples for each missing entity v by randomly replacing it with an arbitrary one.
Scores can be calculated by ConvKB for all triples, and we ﬁnd the rank of a correct triple by sorting all scores in ascending order.
Thus, the performance of relation prediction task can be evaluated by mean reciprocal rank (MRR) and the proportion of correct entities in the top N ranks (Hits@N) for N = 1 , 3 , and 10 (Bordes et al., 2013).
We compare our EIGNN with seven state-of-the-art baselines focusing on this task:
DistMult (Yang et al., 2014), ComplEx (Trouillon et al., 2016), ConvE (Dettmers et al., 2018), TransE (Bordes et al., 2013), ConvKB (Nguyen et al., 2018), RGCN (Schlichtkrull et al., 2018) and KBGAT (Nathani et al., 2019).
As shown in Table 3, our EIGNN achieves the best performance for each metric on FB15K-237 and NELL-995, and achieves the best performance on WN18RR with Hit@ 3 and 10 metrics.
The results of KBGAT are reproduced following the oﬃcial implementation 1 , and the results of other methods can be found in the previous peer-reviewed publications, i.e. (Nathani et al., 2019).
Conclusions
In this work, to make better use of edge features in GNNs, we proposed the edge information maximized graph neural network (EIGNN) that maximizes the mutual information between edge feature vectors and message passing channels.
We reformulated the mutual information as a diﬀerentiable objective by adopting a variational approach.
We have theoretically proved that our proposed objective enables EIGNN to preserve edge information and empirically evaluated EIGNN’s performance on a variety of benchmarks incorporating an array of challenging molecular datasets and knowledge graphs.
These results clearly manifested a substantial improvement of EIGNN over the prior state-of-the-art methods.
Apart from demonstrating the impressive performance of EIGNN, we also showed that its eﬀectiveness is due to exploitation of edge features instead of the regularization eﬀect.
Notably, attribution analysis on molecular graphs show that EIGNN can capture domain knowledge in an end-to-end fashion.
A Full Results on QM
[Table]
We present full results of quantum property regressions on QM9 with average performance and standard deviation in Table 4.
The (10 − 3 ) in the parentheses indicates that the values in the corresponding raw of the table should multiply by 10 − 3 .
This is simply for clear presentation of the values.
We also present a detailed descriptions on the target properties in Table 5 for your reference.
In our results, we directly report MAE instead of Error Ratio [(MAE)/(Chemical Accuracy) (Gilmer et al., 2017)], because it is common to report MAE in terms of chemical unit.
This practice has been widely adopted not only in computational chemistry but also in the machine learning community working on molecular graphs (Wu et al., 2018; Morris et al., 2019).
Still, we add Table 6 which contains the Error Ratio for MPNN and our EIGNN.
In this comparison, we follow (Gilmer et al., 2017) and train models separately to predict each target.
Our EIGNN consistently outperforms MPNN.
[Table]
[Table]
B Full Results on Lipophilicity, ESOL and FreeSolv
In this section, we present experimental results on Lipophilicity, ESOL and FreeSolv with detailed results for each run.
The results in Table 7 verify that our EIGNN consistently outperforms MPNN.
C More Examples of Attribution
In this section, we present more examples of attribution analysis on Lipophilicity.
As a supplement for Fig.
2 in the main text, the observation here is similar.
Compared with
MPNN, we can observe an increasing of overall edge attribution under our EIGNN and a decreasing of prediction error in both cases.
Notably, our EIGNN is able to capture the expert knowledge.
(a)
The molecule is CN [ C @@ H ]( C ) C
(= O ) N
[ C @@ H ]( C 1 CCCCC 1) C (=
O )
N [ C @ H ]2 CCCN ( CCc 3 ccc ( F ) cc 3)
C 2 .
The attribution of {O, N} and the halogen atom F is higher under EIGNN.
(b) The molecule is CC ( C ) N 1 CCN
[ C @ H ]( C 1) C (=
O ) N 2 CCN (
CC 2) C
(= O )
Nc 3 ccc ( Cl ) c
( Cl ) c
Our EIGNN successfully captures the importance of two critical halogen atoms Cl and several atoms N. (c)
The molecule is
COc 1 ccc ( cc 1) C
(= O ) N 2 CCCC
= O .
The attribution of two atoms O at the top is
[Table] [Figure]