Property-Aware Relation Networks for Few-Shot Molecular Property Prediction
Authors: Yaqing Wang
ABULIKEMU ABUDUWEILI
quanming yao
Dejing Dou
Venue: NeurIPS 2021 Spotlight (2021)
Keywords: molecular property prediction
few-shot learning
meta-learning

Abstract: Molecular property prediction plays a fundamental role in drug discovery to identify candidate molecules with target properties. However, molecular property prediction is essentially a few-shot problem, which makes it hard to use regular machine learning models. In this paper, we propose Property-Aware Relation networks (PAR) to handle this problem. In comparison to existing works, we leverage the fact that both relevant substructures and relationships among molecules change across different molecular properties. We first introduce a property-aware embedding function to transform the generic molecular embeddings to substructure-aware space relevant to the target property.  Further, we design an adaptive relation graph learning module to jointly estimate molecular relation graph and refine molecular embeddings w.r.t. the target property, such that the limited labels can be effectively propagated among similar molecules. We adopt a meta-learning strategy where the parameters are selectively updated within tasks in order to model generic and property-aware knowledge separately. Extensive experiments on benchmark molecular property prediction datasets show that PAR consistently outperforms existing methods and can obtain property-aware molecular embeddings and model molecular relation graph properly.  

Property-Aware Relation Networks for Few-Shot Molecular Property Prediction
Yaqing Wang Abulikemu Abuduweili 422 Quanming Yao Dejing Dou'
Baidu Research, Baidu Inc., China The Robotics Institute, Carnegie Mellon University, USA Department of EE, Tsinghua University, China (wangyaqing01, v abbuduueili, doudej (nndejjnn)/baiiu.... com qyaoaaktsinghua. edu. en
Abstract
Molecular property prediction plays fundamental role in drug discovery to identify candidate molecules with target properties. However, molecular property prediction is essentially a few-shot problem, which makes it hard to use regular machine Relation networks
learning models. In this paper, we propose Property-Awee (PAR) to handle this problem. In comparison to existing works, we leverage the fact that both relevant substructures and relationships among molecules change across different molecular properties. We first introduce property function to transform generic molecular embeddings to substructure-awee space relevant to the target property. Further, we design an adaptive relation graph learning module to jointly estimate molecular relation graph and refine molecular embeddings w.r.t. the target property, such that the limited labels can be effectively propagated among similar molccules. We adopt a meta-learning strategy where the parameters are selectively updated within tasks in order to model generic and property-awand knowledge separately. Extensive experiments on benchmark molecular property prediction datasets show that PAR consistently outperforms existing methods and can obtain property- molecular embeddings and model molecular relation graph properly.
embedding
Introduction
Drug discovery is an important biomedical task, which targets at finding new potential medical compounds with desired properties such better absorption, distribution, metabolism, and excre- tion (ADME), low toxicity and active pharmacological activity [1, 2, 3]. It is recorded that drug discovery takes more than billion and at least 10 years in average while clinical success rate is around 10% [4, 5, 6]. To speed up this process, quantitative structure property/activity relation- ship (QSPR/QSAR) modeling uses machine learning methods to establish the connection between molecular structure and particular properties [7]. It usually consists of two components: molecular encoder which encodes molecular structure as fixed-lenge length molecular representation, and predictor which estimates the activity of certain property based on the molecular representation. Predictive models can be leveraged in virtual screening to discover potential molecules more efficiently [8]. However, molecular property prediction is essentially few-shot problem, which makes hard to solve. Only a small amount of candidate molecules can pass virtual screening to be evaluated the lead optimization stage of drug discovery [9]. After series of wet-lab experiments, most candidates
"Equal contribution. A. Abuduweili did his work during internship Baidu Research. Cooresponndecce to.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

SR-HSE task
SR-MMP task
Molecule
Label
SMILES
SR-HSE SR-MMP
Mol-1
elco23ce(XC3CCCCCC3)eeee
Mol-1
Mol-2
Mol-2 Ccloccc(/--NWeeccccCCCCCcccee
0
C=CIC)C@@]]CNICCH
Mol-3
0 0
([]=0)))CC)]]]CC(==0)
Mol-4
O=C(clcceecl) CICCCI
Mol-3
Mol-4
Figure I: Examples of relation graphs for the same molecules coexisting in two tasks of Tox21. Red (blue) edges mean the connected molecules are both active (inactive) on the target property.
eventually fail to be potential drug due to the lack of any desired properties [7]. These together result limited number of labeled data [10].
Few-shot learning (FSL.) [11, 12] methods target at generalizing from a limited number of labeled data. Recently, they have also been introduced into molecular property prediction [3, 8). These methods attempt to learn a predictor from set of property prediction tasks and generalize to predict new properties given few labeled molecules. As molecules can be naturally represented as graphs, graph- based molecular representation learning methods use graph neural networks (GNNs) [13, 14] to obtain graph-level representation the molecular embedding. Specifically, the pioneering IterRe[LSTM [3] adopts GNN as the molecular encoder and adapts classic FSL method [15] proposed for image classification to handle few-shot molecular prediction tasks. The recent Meta-MGNt [8] leverages
GNN pretrained from large-scale self-supervised tasks as molecular encoder and introduces additional self-supervised tasks such as bond reconstruction and atom type prediction to be jointly optimized with the molecular property prediction tasks.
However, aforementioned methods neglect two key facts in molecular property prediction. The first fact is that different molecular properties are attributed to different molecular substructures as found by previous QSPR studies [16, 17, 18]. However, IterReILSTM and Meta-MGNN use graph-based molecular encoder to encode molecules regardless of target properties whose relevant substructures are quite different. The second fact that the relationship among molecules also vary w.r.t the target property. This can be commonly observed in benchmark molecular property prediction datasets. As shown Figure 1, Mol-1 and Mol-4 from the Tox21 dataset [19] have the same activity in SR-HSE task while acting differently in SR-MMP task. However, existing works fail to leverage such relation graph among molecules.
To handle these problems, we propose Property-Aware Relation networks (PAR) which is compatible with existing graph-based molecular encoders, and is further equipped with the ability to obtain property-aware molecular embeddings and model molecular relation graph adaptively. Specifically, our contribution can be summarized as follows:
We propose property-awaid embedding function which co-adapts each molecular embedding with respect to context information of the task and further projects it to a substructure-awarc space w.r.t the target property.
We propose adaptive relation graph learning module to jointly estimate molecular relation graph and refine molecular embeddings w.r.t. the target property, such that the limited labels can be effectively propagated among similar molecules.
We propose meta-learning strategy to selectively update parameters within each task, which is particularly helpful to separately capture the generic knowledge shared across different tasks and those specific to each property prediction task.
We conduct extensive empirical studies on real molecular property prediction datasets. Results show that PAR consistently outperforms the others. Further model analysis shows PAR can obtain property-aware molecular embeddings and model molecular relation graph properly.
Notation. In the sequel, denote vectors by lowercase boldface, matrices by uppercase boldface, and sets by uppercase calligraphic font. For a vector x, [a); denotes the ith element of x. For a matrix x, [X]a denotes the vector on its ith row, [Xit denotes the (i,j)th element of X. The superscript (.) denotes the matrix transpose.

---

2 Review: Graph Neural Networks (GNNs)
graph neural network (GNN) can learn expressive node/graph representation from the topological structure and associated features of a graph via neighborhood aggregation [13, 20, 21J. Consider graph G = {V,E) with node feature h. for each node V and edge feature by for each edge evu E 3 between nodes U, u. At the lth layer, GNN updates the node embedding h!) of node v as:
h (PDATE (hg (1-1), ACGREGATE") (((h)-)) " bu-") buu)/u E ((()))):
(1)
where (v) is a of neighbors v. After iterations of aggregation, the graph-level representation for G is obtained as
g READOUT (ln!9)v v
(2)
where READOUT(-) function aggregates all node embeddings into the graph embedding [22].
Our paper is related to GNN in two aspects: (i) use graph-baset molecular encoder to obtain molecular representation, and (ii) conduct graph structure learning to model relation graph among molecules.
Graph-based Molecular Representation Learning. Representing molecules properly as fixed- length vectors is vital to the success of downstream biomedical applications [23]. Recently, graph- based molecular representation learning methods are popularly used and obtain state-of-the-art performance. A molecule x; represented an undirected graph G3 (V,&&,, where each node E V; represents an atom with feature h E IRd" and each edge evu E & represents the bond between two nodes v, u with feature beu E IR'" Graph-based molecular representation learning methods use GNNs to obtain graph-level representation gi molecular embedding. Examples include graph convolutional networks (GCN) [24]. graph attention networks (GAT) [25], message passing neural networks (MPNN) [20], graph isomorphism network (GIN) [22], pretrained GNN (Pre-GNNN) [26] and GROVER [9].
Existing two works in few-shot molecular property prediction both use graph-based molecular encoder to obtain molecular embeddings: IterReILSTM [3] uses GCN while Meta-MGNN [8] uses Pre-GNN. Using these graph-based molecular encoders cannot discover molecular substructures corresponding to the target property. There exist GNNs which handle subgraphs [27, 28, 29), which usually predefined simply K-hop neighborhood. While discovering and enumerating molecular substructures is extremely hard even for domain experts [17, 30J. In this paper, we first obtain molecular embeddings using graph-based molecular encoders. We further learn to extract relevant substructure embeddings w.r.t. the target property upon these generic molecular embeddings, which more effective and improves the performance.
Graph Structure Learning. As the provided graphs may not optimal, a number of graph
structure learning methods target at jointly learning graph structure and node embeddings [31, 1, 32, 33). In general, they iterate over two procedures: (i) estimate adjacency matrix (i.c., refining neighborhood (v))) which encodes graph structure from the current node embeddings; and (ii) apply a GNN on this updated graph to obtain new node embeddings.
There exist some FSL methods [34, 35, 36, 37, 38] which learn to construct fully connected relation graph among images
N-way K-shot few-shot image classification task. Their methods cannot work for the 2-way K K-shot property prediction tasks where choosing wrong neighbor in the different class will heavily deteriorate the quality of molecular embeddings. We share the same spirit of learning relation graph, and further design several regularizations to encourage our adaptive property-aware relation graph learning module to select correct neighbors.
Proposed Method
In this section, we present the details of PAR, whose overall architecture is shown in Figure 2. Considering few-shot molecular property prediction problem, we first use a specially designed embedding function to obtain property-aware molecular embedding for each molecule, and then adaptively learn relation graph among molecules which allows effective propagation of the limited labels. Finally, we describe meta-learning strategy to train PAR.
3

---

active
self-ttention
estimate relation graph structure
with selfattennton with
average
active
GNN
GNN
GNN
self-attention with
inactive
GNN
classifier
average
inactive
GNN
self-attention
with
self-attention
GNN
gr,
with
Pr,
h
The architecture the proposed PAR, where we plot a 2-way 2-shot task from Tox21. PAR is optimized over a set of tasks. Within each task Tr, the modules with dotted lines are fine-tuned on support set S, and those with solid lines are fixed. query molecule X,,i will first be represented as gr,i using graph-based molecular encoder, then transformed to Pr,, by property-awane embedding function. This Pr, further co-ade co-adapts with embeddings of molecules in S, on the relation graph as h which is taken as the final molecular embedding and used for class prediction.
Figure 2:
3.1 Problem Definition
Following the problem definition adopted by IterRefI.STM [3] and Meta-MGNN [8), the target is learn predictor from set of few-shot molecular property prediction tasks (T=) and generalize to predict new properties given a few labeled molecules. The rth task T, predicts whether molecule Xr,i with index i active (yr, 1) inactive (y/,i on target property, provided with a small number of K labeled samples per class. This Tr is then formulated as a 2-way K-shot classification task with a support set S, = ((x,,i Vr,)) containing the 2K labeled samples and query set N² 2 ((x..) Mra)) containing N unlabeled samples to be classified.
3.2 Property-aware Molecular Embedding
As different molecular properties are attributed different molecule substructures, we design property- embedding function transform the generic molecular embeddings to substructure- aware space relevant to the target property.
As introduced in Section 2, graph-based molecular encoders can obtain good molecular embeddings. By learning from large-scale tasks, they can capture generic information shared by molecules [26, 9]. Thus, we first use a graph-
graph-based based molecular encoder such as GIN [22] and Pre-GNN [26] to extract a molecular embedding Br, E R of length d for each X,,i- The parameter this graph-based molccular encoder is denoted W..
However, existing graph- based molecular encoders cannot capture property-aware substructures. Especially when learning across tasks, molecule can be evaluated for multiple properties. This leads to one-to-mant relationship between a molecule and properties, which makes few-shot molecular property prediction particularly hard. Thus, the embedding space w.r.t. the target property of T,. Let cf denote the class prototype for class E {0, 1}, which is computed
motivated to implicitly capture substructures in
= E(x Br,
(3)
where S = ((x,, yr (xx))((r,s Vr.i) E S, and Hr., = c). We take these class prototypes as the context information of T,, and further encode them
with CT (gr,s cf] E 12 d b = [softmax(CC.... (C.a
(4)
where [l] extracts the jth row vector which corresponds to X, Here b is computed using scaled dot-product self-attention [39], such that each gr,i can be compared with class prototypes in a dimensional wise manner. The property-awe aware molecular embedding Pr. is then obtained
Pr. MLPw, (concatig.. br.l))-
(5)

---

MLPw, denotes the multilayer perceptron (MLP) parameterized by Wp. p. which is used to find a lower dimensional space which encodes substructures that are more relevant to the target property of T,. This contextualized Pr, is property-aware which can be more predictive of the target property.
3.3 Adaptive Relation Graph Among Molecules
Apart from relevant substructures, the relationship among molecules also changes across properties. As shown in Figure two molecules with shared property be different from each other on another property [1, 40, 41J. Therefore, we further propose an adaptive relation graph learning module capture and leverage this property-aware relation graph among molecules, such that the limited labels can be efficiently propagated between similar molecules.
In this relation graph learning module, we alternately estimate the adjacency matrix of the relation graph among molecules and refine the molecular embeddings on the learned relation graph for T times.
At the tth iteration, let g denotes the relation graph where V, takes the 2K molecules in S, and (t)
a query molecule in 2, as nodes. A E [(22+1)x(2K+))) denotes the corresponding adjacency
matrix encoding the gf", where [A [A > if nodes X,,i, X7,j E V/ connected. Ideally, the similarity between property-aware molecular embeddings Pr. Pr.j of X7,i, Xr.j reveals their
relationship under the current property prediction task. Therefore, we h Pr, initially.
We first estimate A. (4) using the current molecular embeddings. The (i. element of [A, [A.1]s
records the similarity between x,,1, Xr,j which calculated as:
MLPP., (exp(_eex((11t)))) (t-1) I))
(6)
where Wa is the parameter of this MLP. The resultant A is a dense matrix, which encodes a fully connected g(
(1)
However, query molecule only has K real neighbors in g in 2-way K-shot task. For binary classification, choosing a wrong neighbor in the opposite class will heavily deteriorate the quality of molecular embeddings, especially when only interference of wrong neighbors, we further reduce gld) to a K-nearest neighbor (KNN) graph, where K is set to be exactly the same as the number of labeled molecules per class in S. The indices of the top K largest [A,
labeled molecule is provided per class. To avoid
(A 11:: 1,...,,2K 2K 1 for x,,i is recorded in A/O (x+,). Then, we set
A
[A -11A91
if Xr.j (x((()x..) otherwise
$1000
The values in [" are normalized to range between 0 and 1, which is done by applying softmax function on each row ( This normalization can also done by z-score, min-max and sigmoid normalization. Then, we co-adapt each node embedding (() with respect other node embeddings
on this updated relation graph encoded (0) Let H.") denote all node embeddings collectively where the ith row corresponds to h (t)
H (*) is updated as
H LeakyReLu(A A (())),,)
(8)
where W, is a learnable parameter.
After T iterations, we return h,, = [H as the final molecular embedding for x,,,, and A,
A (T) the final optimized relation graph.
Denote yr, as the class prediction of xT,i w.r.t. active/inactive, which is calculated as
T = softmax (Wa W. h,.))
where [softmax exp([x).)/ Σ; 2K+1 exp((x),) is applied per row, and W, is parameter.
5

---

Algorithm Meta-training procedure for PAR.
1: initialize 0 (W g: w.. W.) and d (W,, p: We) randomly; if a pretrained molecular
encoder is available, take its parameter as Wgi gi
2: while not done do
3: sample a batch of tasks T;:
4: for all T, do
5:
sample support set S, and query set 2, from Tr:
6:
obtain molecular embedding Br, for each x,, by graph-based molecular encoder; adapt Sr, to be property-avee aware Pr.i by (5); initialize node embeddings h Pr,
7:
8:
9:
for t 1, T do
estimate adjacency matrix A (1)
of relation graph among molecules using h by (6);
refine [ r. end for
on the updated relation graph A (0) by (8);
11:
13:
obtain class prediction Vr,; using h,,; = h, evaluate training loss L(S+; Jo,o) on S,; fine-tune + as 4 by (11); evaluate testing loss ((Q fo,+,) on Q+:
14:
16:
17:
end for
18:
update O and d by (12);
19:
end while
3.4 Training and Inference
We denote PAR as fo In particular, 0 (W 5: Wa. W,) denotes the collection of parameters of graph-based molecular encoder and adaptive relation graph learning module. While $ = {W, p: W.) includes the parameters of property-awar aware molecular embedding function and classifier.
We adopt the gradient- based meta-learning strategy [42]: a good initialized parameter is learned from a set of meta-training tasks [T+} which acts as starting point for each task T,. Upon this general strategy, we selectively update parameters within tasks in order to encourage the model capture generic and property-awar information separately. In detail, we keep 0 fixed while fine-tuning & as o, on S, in each T,. The training loss C(S,.fo.0 evaluated on S, takes the form:
C(Sr, - E(et,.ienesss g(x Vr.si (r.s; So.4)
(10)
with g(XT,,; Ur.i. fo,4) -yT, log($+;)) E(x, J.S., ([A*) im [Ar]im) where y+, E R a one-hot vector with all Os but single one denoting the index of the ground-truth class E (0, 1}. and A; is computed using ground-truth labels with [A = if yr, = Yr.j and 0 otherwise. The first term is the cross entropy for classification. The second term is the specially designed neighbor
alignment regularizer which penalizes the selection of wrong neighbors in the relation graph.
o is obtained by taking few gradient descent updates with learning rate
Φ a C(S,
(11)
0* and ox are learned by optimizing the following objective:
Σ T=1 ((2r:
N₂
min 0.4
(12)
where the loss C(Or, fe ) is calculated in the same form of (10) but is evaluated on O, instead. It also optimized by gradient descent [42].
The complete algorithm of PAR shown in Algorithm 1. Line 6-7 correspond to property-aware
embedding Ри,í which encodes substructure w.r.t the target property (Section 3.2). Line 8-12 correspond to adaptive relation graph learning which facilitates effective label propagation among similar molecules (Section 3.3).
For inference, the generalization ability of PAR evaluated on the query set Qnew of each new task Ta which tests on new property in meta-testing stage. Still, 0" is fixed and &* is fine-tuned on Spow.

---

Experiments
Table 1: Summary of datasets used.
We perform experiments on widely used benchmark few-shot molecular property prediction datasets (Table 1) included in MoleculeNet [43]. Details of these benchmarks are in Appendix A.
Dataset
Tox21
SIDER
MUV
ToxCast
Compounds
8014
1427
93127
8615
# Tasks
12
27
17
617
# Meta- Deeta-Taining Tasks
9
21
12
450
# Meta- Meta-Testing Tasks
3
5
167
4.1 Experimental Settings
Baselines. In the paper, we compare our PAR (Algorithm 1) with two types of baselines: (i)
FSL methods with graph-based molecular encoder learned from scratch, including Siamese [44], ProtoNet [45], MAML [42]. TPN [35], EGNN [36], and IterReILSTM [3]; and (ii) methods which
leverage pretrained graph-base based molecular encoder including Pre-GNN [26], Meta-MGNN [8], and
Pre-PAR which is our PAR equipped with Pre-GNN. We use results of Siamese and IterRefLSTT reported in [3] as the codes are not available. For the other methods, we implement them using public
codes of the respective authors. More implementation details are in Appendix B.
Generic Graph-based Molecular Representation. Following [26, 8), we use RDKit [46] to build molecular graphs from SMILES, and to extract atom features (atom number and chirality tag) and bond features (bond type and bond direction). For all methods re-implemented by us, we use GIN [22] as the graph-based molecular encoder to extract molecular embeddings. Pre-GNN, Meta-MGNN and Pre-PAR further use the pretrained GIN which also provided by the authors of [26].
Evaluation Metrics Following [26, 8), we evaluate the binary classification performance by ROC- AUC scores calculated on the query set of each meta-testing task. We run experiments for ten times with different random seeds, and report the mean and standard deviations of ROC-AUC computed over all meta-testing tasks.
4.2 Performance Comparison
Table shows the results. Results of Siamese, IterReILSTM and Meta-MGNN on ToxCast are not provided: the first
methods lack codes and not evaluated on ToxCast before, while Meta-MGNN runs out of memory as it weighs the contribution of each task among all tasks during meta-training. As can be seen, Pre-PAR consistently obtains the best performance, while PAR obtains the best performance among methods using graph-based molecular encoders learned from scratch. In terms of average improvement, PAR obtains significantly better performance than the best baseline learned from scratch (e.g. EGNN) by 1 1.59%, and Pre-PAR is better than the best baseline with pretrained molecular encoders (e.g. Meta-MGNN) by 1.49%. Pre-PAR also takes less time and episodes to converge than Meta-MGNN, which is shown in Appendix C.I. In addition, we observe that FSL methods that learn relation graphs (i.e., GNN, TPN, EGNN) obtain better performance than the classic ProtoNet and MAML.
4.3 Ablation Study
We further compare Pre-PAR and PAR with the following variants: (i) w/o P: w/o applying the property-aware embedding function; (ii) w/o context in P: w/o context by in equation (5); (iii) w/o R: w/o using the adaptive relation graph learning; (iv) w/ cos-sim in R: use cosine similarity to obtain the adjacency matrix [Ar] وj Pr.Prr// (elp:/([[=+lllleee -lleallll=>illll l12), then calculate (7) and (8) in PAR:
(v) w/o KNN in R: w/o reducing Gr to KNN graph; (vi) w/o reg: w/o using the neighbor alignment regularizer in equation (10); and (vii) tune all: fine-tune all parameters on line 15 of Algorithm 1. Note that these variants follows control variates method. They cover all components of training PAR without overlapping functionalities.
Results on 10-shot tasks are Figure 3. Again, Pre-PAR obtains better performance than PAR due to better starting point. PAR and Pre-PAR outperform their variants. The removal of any component leads to significant performance drop. In particular, the performance gain of PAR and Pre-PAR with
Codes are available at https: http//gttttb..ooo wwt. httmm//taaa611/PAR.... NeurIPS11
7

---

Table 2: ROC-AUC scores benchmark molecular property prediction datasets. The best results (according to the pairwise t-test with 95% confidence) are highlighted in gray. Methods which use pretrained graph- molecular encoder marked in green.
Tox21
SIDER
MUV
ToxCast
Method
10-shot
1-shot
10-shot
1-shot
10-shot
1-shot
10-shot
1-shot
Siamese
(0.40 65. 65.00
(1.10(4.32) (3.43
(3.9)
(0.00(o.17)
ProtoNet
(0.2)(
65.58(1.72)
(4.54(0..9) (2.50
(4.11
(8.31
63. (3.70
56.36(1.54)
MAML
(0.24
75.74(0.45)
70.43
(7.81
63.90
(3.51(
66. (0.0s) 65.97 (s.04)
TPN
(6.05 60.16(1.1s)
67.84
62.90
(5.22(5.82)
(0.00
62. (2..44(145)) 50.01
EGNN
(1.21(o.16)
79.44 22)
(2.87(0.73)
70.79(o.95)
(5.20(2.08)
65.
62.18
63.65 61.02(1.94)
IterReLLSTM
(1.10 880.77
69. 69.633g.30
71.73
49.56(5.12)
(8.54(s.12)
PAR
82 12) 80,46 13)
74.68 31) 71.87 (0.48)
66.4882.12)
64. (1.12(1.188)
69. (19.2 63) 67 (8.28 (2.90)
Pre-GNN
82.14
81.68 09)
73.96 (0)) (3.24
(7.14
(1.45(
73.68
(2.90(0.84)
Meta-MGNN 82.97 (0.11 10) 83 13)
75. 43 21) (0.3))
68.99
(5.54(2.13)
Pre-PAR 84.93 11) 83.01 (o 09)
78, 78.08 74.46 29)
(9.96 (6.94(..12)
75. (2 73.63(1.00)
respect to "w/ cos-sim in R" validates the necessity of learning a similarity function from the data rather than using the fixed cosine similarity. We also try to iterate the estimation of relation graph constructed by cosine similarity, but observe a performance drop given more iterations. Results on 1-shot is put in Appendix C.2 where the observations are consistent.
15.47
34
12.95
92.12
Store
$2.04
01.35
02
10.33
60
82-
76.43
78:45
Pre-
cottect
cossiia it
stst an
tuie 40
165
in in
cor-000 in
(a) Pre-PAR
(b) PAR
Figure 3: Ablation study 10-shot tasks from Tox21.
4.4 Using Other Graph-based Molecular Encoders
In the experiments, we use GIN and its pretrained version. However, as introduced in Section 3.2, our PAR is compatible with any existing graph-based molecular encoder introduced in Section 2. Here, we consider the following popular choices as the encoder to output gr,s: GIN [22], GCN [24], GraphSAGE [14] and GAT [25], which are either learned from scratch or pretrained. We compare the proposed PAR with simply fine-tuning the encoder on support sets (denote as GNN).
Figure 4 shows the results. As can be seen, GIN is the best graph-based molecular encoder among the four chosen GNNs. PAR outperforms the fine-tuned GNN consistently. This validates the effectiveness of the property-aware molecular embedding function and the adaptive relation graph learning module. We further notice that using pretrained encoders can improve the performance except for GAT, which is also observed in [26].
Although using pretrained graph-based molecular encoders can improve the performance in general, please note that both molecular encoders learned from scratch or pretrained are useful. Pretrained encoders contain rich generic molecular information by learning enormous unlabeled data, while encoders learned from scratch can carry some new insights. For example, the recent DimeNet [47] can model directional information such as bond angles and rotations between atoms, which has no pretrained version. As our proposed method can use any molecular encoder to obtain generic molecular embedding, it can easily accommodate newly proposed molecular encoder w/o or w/ pretraining.
8

---

00.02.0.0
91.81 01.991.0
20-3
75.00
15.00
24:4
65.72
(a) 10-shot
(b) 1-shot
Figure 4: ROC-AUC scores Tox21 using different graph-based molecular encoders.
4.5
Case Study
Finally, validate whether PAR obtain different property-awan molecular embeddings and relation graphs for tasks containing overlapping molecules but evaluating different properties.
To examine this under controlled setting, sample fixed group of 10 molecules Tox21 (Table 5 in Appendix C.3) which coexist different meta-testing tasks (i.e., the 10th, 11th and 12th tasks). Provided with the meta-learned parameters 0* and Ø', we take these 10 molecules as the support set fine-tune 4 as 4 and keep 0 fixed in each task T,, As the support set is fixed now, the ratio of active molecules to inactive molecules among the 10 molecules may not be 1:1 in the three tasks. Thus, the resultant task may not evenly contain K labeled samples per class.
Visualization of the Learned Relation Graphs. As described in Section 3.3, PAR returns A, as
the adjacency matrix encoding the optimized relation graph among molecules. Fach element [A>]ip records the pairwise similarity of the 10 molecules and random query (which is dropped then). As the number of active and inactive molecules may not be equal the support set, we no longer reduce adjacency matrices A, to A, which encodes KNN graph. Figure 5 plots the optimized adjacency matrices obtained on all three tasks. As can be observed, PAR obtains different adjacency matrices property-prediction prediction tasks. Besides, the learned adjacency matrices are visually similar to the ones computed using ground-truth labels.
for different property-pe
1.0
meL
mel
mol
mol 5-
0.4
mel
mol
mol.1
moL3
met
mel
0.6
5-
0.4
mot
10
mel1
mel.2
met
mel.6
mel.7
m
mel.9
mot.
mel.
mot
mol.3
mel
mai
moi.
mol.?
mol
mol.
mel
mel.?
mot
mel
mol
mol
mold
(a) the 10th task
(b) the 11th task
(c) the 12th task
Figure 5; Comparison between A; computed using ground-true -ruth labels (the first row) and adjacency
matrix A, returned by PAR (the second row) for the molecules. We set (A 1 if molecules
X and X have the same label and 0 otherwise.

---

1.6
trectre
16:
1.4
proto mooctive
proto, Foacciv
12.
1..-
0.8
06-
mpl
0.0
Instin active
activn
active
prote_.ractive
prote, mooliie
proto active
prote active
prote active
1.2
1.0-
"gLL mOLs
MALL
0.8
0.6
0.4-
0.2
1.8
1.5
wrtive
1.4
Jnactive
proto action
proto_active
moly
0.8
motu
0.6
ROLY
not
0.4
n2L2 mere
02
250
08-
mot
moLp
(a) the 10th task
(b) the 11th task
(c) the 12th task
Figure 6: t-SNE visualization of gr.; (the first row), Pr, (the second row), and b (the third row) of the ten molecules. Proto_active cnnto_iiattive denotes the class prototype of active (inactive) class.
Visualization of the Learned Molecular Embeddings. We also present the t-SNE visualization of gr.i (molecular embedding obtained by graph-based molecular encoders), pr, (molecular embedding obtained by property-awan embedding function), and h,, (molecular embedding returned by PAR) for these 10 molecules. For the same X+,i gr. is the same across 10th, 11th, 12th task, while pr. and h copeetyyaware Figure 6 shows the results. As shown, PAR indeed captures property-ave aware information during encoding the same molecules for different molecular property prediction tasks. From the first row to the third row in Figure molecular embeddings gradually get closer the class prototypes on all three tasks.
5 Conclusion
We propose Property-Award Relation networks (PAR) address the few-shot molecular property prediction problem. PAR contains: a graph-based molecular encoder to encode the topological structure of the molecular graph, atom features, and bond features into a molecular embedding: a property-aware embedding function to obtain property-awand embeddings encoding context infor- mation of each task; and an adaptive relation graph learning module to construct relation graph to effectively propagate information among similar molecules. Empirical results consistently show that PAR obtains state-of-the-art performance on few-shot molecular property prediction problem.
There are several directions to explore in the future. In this paper, PAR is evaluated on biophysics and physiology molecular properties which are modeled as classification tasks. While the prediction of quantum mechanics and physical chemistry properties are mainly regression tasks, it is interesting to extend PAR handle these different levels of molecular properties. In addition, although PAR targets at few-shot molecular property prediction, the proposed property- embedding function, adaptive relation graph learning module, and the neighbor alignment regularizer can be helpful to improve the performance of graph- molecular encoders in general. Finally, interpreting the substructures learned by PAR is also a meaningful direction.
10

---

Acknowledgements
We sincerely thank the anonymous reviewers for their valuable comments and suggestions. Parts of experiments were carried out on Baidu Data Federation Platform.
References
[1] Sebastian G Rohrer and Knut Baumann. Maximum unbiased validation (MUV) data sets for virtual screening based on PubChem bioactivity data. Journal of Chemical Information and Modeling, 49(2):169-1844 2009.
[2] Karim Abbasi, Antti Poso, Jahanbakhsh Ghasemi, Massoud Amanlou, and Ali Masoudi- Nejad. Deep transferable compound representation across domains and tasks for low data drug discovery. Journal of Chemical Information and Modeling, 59( 1):4528 4539999-4399 2019.
[3] Han Altae-Tran, Bharath Ramsundar, Aneesh S Pappu, and Vijay Pande. Low data drug discovery with one-shot learning. ACS Central Science, 3(4):283-293, 2017.
[4] Steven M Paul, Daniel S Mytelka, Christopher Dunwiddie, Charles C Persinger, Bernard H Munos, Stacy R Lindborg, and Aaron L Schacht. How to improve R&D productivity: The pharmaceutical industry's grand challenge. Nature Reviews Drug Discovery, 9():203-214, 2010.
[5] Sumudu Leelananda and Steffen Lindert. Computational methods in drug discovery. Beilstein journal of organic chemistry, 12(1):2694-277 2718, 2016.
[6] Alex Zhavoronkov, Yan A Ivanenkov, Alex Aliper, Mark Š Veselov, Vladimir A Aladinskiy, Anastasiya V Aladinskaya, Victor A Terentiev, Daniil A Polykovskiy, Maksim D Kuznetsov, Arip Asadulaev, et al. Deep learning enables rapid identification of potent DDRI kinase inhibitors. Nature Biotechnology, 3739) 2019.
[7] George E Dahl, Navdeep Jaitly, and Ruslan Salakhutdinov. Multi-task neural networks for QSAR predictions. arXiv preprint arXiv: /406. 1231, 2014.
[8] Zhichun Guo, Chuxu Zhang, Wenhao Yu, John Herr, Olaf Wiest, Meng Jiang, and Nitesh V Chawla. Few-shot graph learning for molecular property prediction. In The Web Conference, 2021.
[9] Yu Rong, Yatao Bian, Tingyang Xu, Weiyang Xie, Ying Wei, Wenbing Huang, and Junzhou Huang. Self-supervised graph transformer large-scale molecular data. Advances in Neural 12571, 2020.
Information Processing Systems,
[10] Cuong Q Nguyen, Constantine Kreatsoulas, and Kim M Branson. Meta-learning GNN initial- izations for low-resource molecular property prediction. arXiv preprint 2rXhv:2003.0599611 pages arXiv-2003, 2020.
[11] Li Fei-Fei, Rob Fergus, and Pietro Perona. One-shot learning of object categories. IEEE Transactions Pallern Analysis and Machine Intelligence, 28(4):594-611, 2006.
[12] Yaqing Wang, Quanming Yao, James T Kwok, and Lionel M Ni. Generalizing from a few examples: survey on few-shot learning. ACM Computing Surveys, 53(3):1-34, 2020.
[13] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. International Conference on Learning Representations, 2016.
[14] William L Hamilton, Rex Ying, and Jure Leskovec. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems, pages 1025-1035, 2017.
[15] Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Koray Kavukcuoglu, and Daan Wierstra. Matching networks for one shot learning. In Advances Neural Information Processing Systems, pages 3637-3645, 2016.

---

[16] Alexandre Varnek, Denis Fourches, Frank Hoonakker, and Vitaly P Solov' ev. Substructural fragments: An universal language to encode reactions, molecular and supramolecular structures. Journal of computer-aided molecular design, 19(9):693-7033 2005.
[17] Subhash Ajmani, Kamalakar Jadhav, and Sudhir A Kulkarni. Group-based QSAR (G-QSAR): Mitigating interpretation challenges in QSAR. QSAR Combinatorial Science, 281):36-55, 2009.
[18] Paulo Costa, Joel S Evangelista, Igor Leal, and Paulo CML Miranda. Chemical graph theory for property modeling QSAR and QSPR- QSAR & QSPR. Mathematics, (1):60, 2021.
[19] National Center for Advancing Translational Sciences. Tox21 challenge. http://tripod. nih. http/tox22//haleengeee. 2017. Accessed: 2016-11-06.
[20] Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In International Conference on Machine Learning, pages 1263-1272 2017.
[21] Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. In Advances in Neural Information Processing Systems, volume 33, pages 22118 221333,, 2020.
[22] Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanic Jegelka. How powerful are graph neural networks? In International Conference on Learning Representations, 2018.
[23] Erik Gawehn, Jan A Hiss, and Gisbert Schneider. Deep learning in drug discovery. Molecular Informatics, 35(1):3-14, 2016.
[24] David Duvenaud, Dougal Maclaurin, Jorge Aguilera-Iparragunn Rafael Gómez-Bombarele Bombarelli, Timothy Hirzel, Alán Aspuru- Guzuu,, and Ryan Adams. Convolutional networks on graphs for learning molecular fingerprints. In Advances in Neural Information Processing Systems, 2015.
[25] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. arXiv preprint arXiv: 10. 10903, 2017.
[26] Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zimik, Percy Liang, Vijay Pande, and Jure Leskovec. Strategies for pre- vre-training graph neural networks. In International Conference on Learning Representations, 2019.
[27] Federico Monti, Karl Otness, and Michael M Bronstein. MotifNet: A motif-based graph convolutional network for directed graphs. In IEEE Data Science Workshop, pages 225-228. IEEE, 2018
[28] Emily Alsentzer, Samuel Finlayson, Michelle Li, and Marinka Zitnik. Subgraph neural networks. In Advances in Neural Information Processing Systems, volume 33, pages 8017-80 2020.
[29] Xinyu Fu, Jiani Zhang, Ziqiao Meng, and Irwin King. MAGNN: Metapath aggregated graph neural network for heterogeneous graph embedding. In The Web Conference, pages 2331-2341, 2020.
[30] Wenying Yu. Hui Xiao, Jiayuh Lin, and Chenglong Li. Discovery of novel STAT3 small molecule inhibitors via in silico site- ditectie fragment-base based drug design. Journal of Medicinal Chemistry, 56(1 1):4402-4412, 2013.
[31] Yu Chen, Lingfei Wu, and Mohammed Zaki. Iterative deep graph learning for graph neural networks: Better and robust node embeddings. In Advances in Neural Information Processing Systems, pages 19314-19326 2020.
[32] Yanqiao Zhu, Weizhi Xu, Jinghao Zhang, Qiang Liu, Shu Wu, and Liang Wang. Deep graph structure learning for robust representations: A survey, arXiv preprint arXiv:2103.03036 2021.
[33] Yaqing Wang, Song Wang, Quamming Yao, and Dejing Dou. Hierarchical heterogeneous graph representation learning for short text classification. In Conference on Empirical Methods in Natural Language Processing, pages 3091
12

---

[34] Victor Garcia and Joan Bruna. Few-shot learning with graph neural networks. In International Conference on Learning Representations, 2018.
[35] Yanbin Liu, Juho Lee, Minseop Park, Sachoon Kim, Eunho Yang, Sung Ju Hwang, and Yi Yang. Learning to propagate labels: Transductive propagation network for few-shot learning. In International Conference on Learning Representations, 2018.
[36] Jongmin Kim, Taesup Kim, Sungwoong Kim, and Chang D Yoo. Edge-labeling graph neural network for few-shot learning. In IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 1-200, 2019.
[37] Ling Yang, Liangliang Li, Zilun Zhang, Xinyu Zhou, Erjin Zhou, and Yu Liu. DPGN: Distribu- tion propagation graph network for few-shot learning. In IEEE/CVF Conference on Computer Vision and Patlern Recognition, pages 13390-13399, 2020.
[38] Pau Rodríguez, Issam Laradji, Alexandre Drouin, and Alexandre Lacoste. Embedding propa- gation: Smoother manifold for few-shot classification. In European Conference on Computer Vision, pages 121-138. Springer, 2020.
[39] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Informa- tion Processing Systems, pages 6000-6010, 2017.
[40] Michael Kuhn, Ivica Letunic, Lars Juhl Jensen, and Peer Bork. The SIDER database of drugs and side effects. Nucleic Acids Research, 44(D1)D1075--1077) 2016.
[41] Ann M Richard, Richard S Judson, Keith A Houck, Christopher M Grulke, Patra Volarath, Inthirany Thillainadarajah, Chihae Yang, James Rathman, Matthew T Martin, John F Wambaugh, et al. ToxCast chemical landscape: Paving the road to 21st century toxicology. Chemical Research in Toxicology, 29(8): (288):122-1121 1251,, 2016.
[42] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adapta- tion of deep networks. In International Conference Machine Learning, pages 1126-1135, PMLR, 2017.
[43] Zhenqin Wu, Bharath Ramsundar, Evan N Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh S Pappu, Karl Leswing, and Vijay Pande. MoleculeNet: A benchmark for molecular machine learning. Chemical Science, 92):513-530, 2018.
[44] Gregory Koch, Richard Zemel, and Ruslan Salakhutdinov. Siamese neural networks for one-shot image recognition. In ICML Deep Learning Workshop, volume 2. Lille, 2015.
[45] Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. In Advances in Neural Information Processing Systems, pages 4080-4090, 2017.
[46] Greg Landrum. RDKit: A software suite for cheminformatics, computational chemistry, and predictive modeling, 2013.
[47] Johannes Klicpera, Janek Groß, and Stephan Günnemann. Directional message passing for molecular graphs. In International Conference on Learning Representations, 2019.
[48] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 112.9914 2014.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
13

---

(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes]
(c) Did you report error bars (e.g., w.r.t. the random seed after running experiments multiple times)? [Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes]
4. If you using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [No]
(c) Did you include any new assets either in the supplemental material or as a URL? [N/A]
(d) Did you discuss whether and how consent obtained from people whose data you's 're using/curating? [Yes] We use the public codes of the authors.
(e) Did you discuss whether the data you are using/curating contains personally identifiable information offensive content? [N/A] We use public benchmark molecular property prediction datasets.
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you deseribe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "zJ1Iyd8GpG", "reviewer": "Reviewer_8nqn", "summary": "This paper proposes a property-aware adaptive relation networks for few shot molecular property prediction.  They propose a graph based molecule encoder which is trained from large scale tasks to capture a generic representation of the molecules and a property aware embedding function which adapts the generic molecule embedding to be property-aware. Next, an adaptive relation graph learning module is proposed to capture the relationship between similar molecules with respect to a target property. The adaptive relation graph learning module is based on GNNs. A new training strategy, where only the parameters of property aware embedding function and the classifier module is tuned, is also proposed. The authors argue that it useful to separately capture generic molecule representations and those that are specific to the particular task.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The authors propose a new method for Few shot learning of molecular properties. The proposed method achieves state of the art performance on four classification tasks (Tox21, Sider, MUV, ToxCast). As the authors argue, few shot learning for predicting molecular properties is very important for drug discovery and related applications. Collecting a large number of labels for a particular molecular property is usually expensive. Hence the proposed problem is of great practical interest. \n\nThe proposed method is novel and the experimentation is thorough, despite the evaluation being done only a small number of tasks (see question below about extending it to other tasks). The ablation study disentangles the contribution of each variant on the  final score. The paper is well written and clear. \n\nOverall, I am quite positive about this paper. I feel it tackles an important problem in a novel way and shows good performance.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Limitations and Questions for authors:\nMany of the molecular properties of interest are regression tasks (for example, quantum mechanical properties , binding affinities and so on). Can the authors explain if it is possible to extend the methodology to regression tasks?\n\nThe evaluation is done only on Tox21, Sider, MUV and Toxcast.    Does the method transfer to other tasks from the moleculenet dataset?", "review_text": "", "extra_scores": {"ethical_concerns": "I don't see any ethical concerns with the paper.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "1.5"}, "extra_text": {}}, {"review_id": "o1sUPQu8gw-", "reviewer": "Reviewer_d5zA", "summary": "This paper proposes a novel framework to solve a few-shot molecular property prediction problem w.r.t different properties. It takes GNN as the encoder to obtain property-aware molecular embeddings and projects it to a substructure-aware space according to the target property. Moreover, a relation graph between molecules is learned adaptively to refine the embeddings. Training strategies like gradient-based meta-learning is applied to boost property-aware training. Extensive experiments are conducted over a few-shot property prediction task as well as transfer learning across datasets", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The authors design a novel graph-based property-aware framework for few-shot molecular property prediction and generate a relation graph among molecules to help further refine the molecular embeddings. However, with a closer look, the proposed model seems to be a combination of multiple existing works, e.g., property-aware attention[16], gradient-based meta-learning strategy [9]. It utilizes numbers of modules, and my concern is that how important each module is. It is hard to say all these modules/strategies work effectively since the performance improvement seems limited. The ablation study compares the performance of some variants but it only takes out one module at a time, so there is a chance that certain combinations can obtain competitive performance too.\n\nIn addition, the experiment results show that the performance of PAR does not significantly surpass other baseline models w/o pretrain models, instead the pre-GNN + PAR performs the best. The effectiveness of the proposed PAR seems unimpressive. Moreover, the details of the pretrain model implementation are not given, like what dataset is used and what experimental settings are. Also, I am curious that why the training takes so long (2000 epochs), while the original pre-GNN only trains 100 epochs. The dataset does not seem that large, it’s only a few thousand molecules. Additionally, considering such a large training epoch, it’s better to report the training time for the proposed model.\n\nThe paper seems to be finished in rush, many unfinished part, e.g., missing citation on P3 L101; no number filling out on P7 L249; the caption of Fig.5 is not consistent with the notation; the dataset description on P7 L216 is not consistent with table 1. The writing needs to be improved.     \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "See above", "review_text": "", "extra_scores": {"ethical_concerns": "n/a", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4 HOURS"}, "extra_text": {}}, {"review_id": "KqyWGmPs9wb", "reviewer": "Reviewer_VWYi", "summary": "The authors look at molecular property prediction from a few shot learning perspective. They propose an approach, closely following Meta-MGNN, to adapt part of a (optionally pre-trained) graph embedding network in MAML-style per task. A number of well-motivated novelties are introduced: (a) adapt part of the GNN encoder, called property-aware transformation (vs just task-specific attention in Meta-MGNN). (b) explicitly constructing the similarity graph between a test sample and the support set (labeled samples) and add a sparsity loss for the adjacency matrix during training. The results, focusing on 1-shot and 10-shot prediction, convincingly show gains in this regime, and extensive ablations and baselines are presented.\n", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "\nMajor strengths:\n\n\n\n* Strong motivation for property-specific adaptation + adaptive relation graph learning is presented.  This is done via examples (Fig1), comparisons to works without property-specific adaptation, and examples of how learned relation graphs differ greatly across tasks.\n* The proposed model (PAR), though complex, seems to appropriately address the problem setup.\n* Strong results (only on 1-shot and 10-shot classification) + comparison to multiple baselines, for each of PAR’s major dimensions.\n* Comprehensive ablations of contributions and analysis of their effects.\n\nWeaknesses - none are major, all are expanded upon below.\n\n\n\n* Somewhat unclear writing.\n* High overall complexity of the approach, adding elements to already-complex IterRefLSTM and Meta-MGNN\n* Results only on artificial few-shot settings \n\nOverall this is a strong paper, hence a score of 8: Top 50% of accepted papers, clear accept\n\n## Detailed review\n\n\n### Motivation and method\n\nThe paper has a strong motivation around few-shot learning and the need for property-specific adaptation of general purpose embeddings.\n\nAlso the motivation behind ensuring the relation graph (similarity  is correctly constructed for a task is well-taken, and illustrated convincingly with Fig 4,8,9. Ablations show the gain of constructing and sparsifying the similarity graph.\n\nQuestions for the authors:\n\n\n\n* Can the adjacency matrix be estimated directly from cosine similarity of embeddings, rather than doing the iterative estimation?\n\nOther minor comments: \n\n\n\n* Appendix Figure 8 row order doesn’t seem to match caption? (0-1 should be ground truth versus grayscale predicted?)\n* Figure 4 - these adjacency matrices are the same but according to Fig9 and caption they should be different. Also same comment as above.\n* Figure 3 - please label the variants with a short summary to avoid having to cross-reference the text.\n\n\n### Impact / significance / results\n\n\n\n* The ablations are strong.  The three main contributions (beyond results) are ablated, and show large degradations when removed:\n    * property-aware embedding\n    * fine-tuning of all parameters degrades wrt only final layers\n* Comparison to existing work is thorough.\n    * 10 seeds for confidence intervals.  Proper comparison to several lines of work which were the basis for PAR.\n* Overall this paper addresses a difficult task in a better way, with strong motivation and appropriate novel methods.\n\nComments:\n\n\n\n* It would be helpful to see a section on limitations.\n* It would be interesting to see results on full tox21 / other moleculenet datasets, without artificially constraining to 1 or 10-shot classification. However we recognize this is following the standard set by IterRefLSTM and Meta-MGNN.\n* See suggested ablation above: “Can the adjacency matrix be estimated directly from cosine similarity of embeddings, rather than doing the iterative estimation?”\n\n\n### Clarity of writing\n\nOverall the paper is thorough, well-motivated and contains necessary detail.\n\nThe **clarity of the paper could be improved**:\n\n\n\n* Some concepts should be high-level described (relation graph / property-aware embedding) when the terms are introduced.\n* Heavy notation can be reduced and gets in the way of understanding. Authors should look for places to provide a high level summary sentence at the start of the paragraph to summarize the paragraph.\n* It would be helpful to be more explicit where the aggregation (GNN node embeddings -> single vector) happens, versus where is task-specific adaptation. \n\nSpecific comments about equations:\n\n\n\n* Please move description of g, p, r, c superscripts to the front of [Section 3 - Proposed Method]\n    * ‘c’ superscript is used for both classifier parameters $\\theta^c$ and context embedding $e^c$, which occur in two different stages?\n* What is the $e^p_{x_{Tj}}$ initialization value for equation (2)?\n\nMinor comments and typos\n\n* Several grammatical errors (eg: “while fine-tune theta”, “taking a few gradient descents”)\n* Figure 4: (d) 12th task instead of 11th task?\n* Parameterize MLP in eq (3) by $\\theta^r$ ?\n* Also see comments about figures above", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "It would be helpful to see a section on limitations.\nNegative societal impact: N/A", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "10"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes  a framework for few shot learning for molecular properties prediction. Reviewers have agreed that the problem of few shot training is important in the molecular  space and that the  approach proposed in the paper is novel and elegant. Authors showed evidence of their method working in the few shot regime. Accept 
Decision: Accept (Spotlight)