Property-Aware Relation Networks for Few-Shot Molecular Property Prediction
Year: 2022

Abstract:
Molecular property prediction plays a fundamental role in drug discovery to identify candidate molecules with target properties. However, molecular property prediction is essentially a few-shot problem, which makes it hard to use regular machine learning models. In this paper, we propose Property-Aware Relation networks (PAR) to handle this problem. In comparison to existing works, we leverage the fact that both relevant substructures and relationships among molecules change across different molecular properties. We first introduce a property-aware embedding function to transform the generic molecular embeddings to substructure-aware space relevant to the target property. Further, we design an adaptive relation graph learning module to jointly estimate molecular relation graph and refine molecular embeddings w.r.t. the target property, such that the limited labels can be effectively propagated among similar molecules. We adopt a meta-learning strategy where the parameters are selectively updated within tasks in order to model generic and property-aware knowledge separately. Extensive experiments on benchmark molecular property prediction datasets show that PAR consistently outperforms existing methods and can obtain property-aware molecular embeddings and model molecular relation graph properly.

heading: 1 Introduction
text: Drug discovery is an important biomedical task, which targets at finding new potential medical compounds with desired properties such as better absorption, distribution, metabolism, and excretion (ADME), low toxicity and active pharmacological activity [1, 2, 3]. It is recorded that drug discovery takes more than 2 billion and at least 10 years in average while the clinical success rate is around 10% [4, 5, 6]. To speed up this process, quantitative structure property/activity relationship (QSPR/QSAR) modeling uses machine learning methods to establish the connection between molecular structure and particular properties [7]. It usually consists of two components: a molecular encoder which encodes molecular structure as a fixed-length molecular representation, and a predictor which estimates the activity of a certain property based on the molecular representation. Predictive models can be leveraged in virtual screening to discover potential molecules more efficiently [8]. However, molecular property prediction is essentially a few-shot problem, which makes it hard to solve. Only a small amount of candidate molecules can pass virtual screening to be evaluated in the lead optimization stage of drug discovery [9]. After a series of wet-lab experiments, most candidates
⇤Equal contribution. A. Abuduweili did his work during internship at Baidu Research. †Correspondence to.
35th Conference on Neural Information Processing Systems (NeurIPS 2021).
eventually fail to be a potential drug due to the lack of any desired properties [7]. These together result in a limited number of labeled data [10].
Few-shot learning (FSL) [11, 12] methods target at generalizing from a limited number of labeled data. Recently, they have also been introduced into molecular property prediction [3, 8]. These methods attempt to learn a predictor from a set of property prediction tasks and generalize to predict new properties given a few labeled molecules. As molecules can be naturally represented as graphs, graphbased molecular representation learning methods use graph neural networks (GNNs) [13, 14] to obtain graph-level representation as the molecular embedding. Specifically, the pioneering IterRefLSTM [3] adopts GNN as the molecular encoder and adapts a classic FSL method [15] proposed for image classification to handle few-shot molecular prediction tasks. The recent Meta-MGNN [8] leverages a GNN pretrained from large-scale self-supervised tasks as molecular encoder and introduces additional self-supervised tasks such as bond reconstruction and atom type prediction to be jointly optimized with the molecular property prediction tasks.
However, aforementioned methods neglect two key facts in molecular property prediction. The first fact is that different molecular properties are attributed to different molecular substructures as found by previous QSPR studies [16, 17, 18]. However, IterRefLSTM and Meta-MGNN use graph-based molecular encoder to encode molecules regardless of target properties whose relevant substructures are quite different. The second fact is that the relationship among molecules also vary w.r.t. the target property. This can be commonly observed in benchmark molecular property prediction datasets. As shown in Figure 1, Mol-1 and Mol-4 from the Tox21 dataset [19] have the same activity in SR-HSE task while acting differently in SR-MMP task. However, existing works fail to leverage such relation graph among molecules.
To handle these problems, we propose Property-Aware Relation networks (PAR) which is compatible with existing graph-based molecular encoders, and is further equipped with the ability to obtain property-aware molecular embeddings and model molecular relation graph adaptively. Specifically, our contribution can be summarized as follows:
• We propose a property-aware embedding function which co-adapts each molecular embedding with respect to context information of the task and further projects it to a substructure-aware space w.r.t. the target property.
• We propose an adaptive relation graph learning module to jointly estimate molecular relation graph and refine molecular embeddings w.r.t. the target property, such that the limited labels can be effectively propagated among similar molecules.
• We propose a meta-learning strategy to selectively update parameters within each task, which is particularly helpful to separately capture the generic knowledge shared across different tasks and those specific to each property prediction task.
• We conduct extensive empirical studies on real molecular property prediction datasets. Results show that PAR consistently outperforms the others. Further model analysis shows PAR can obtain property-aware molecular embeddings and model molecular relation graph properly.
Notation. In the sequel, we denote vectors by lowercase boldface, matrices by uppercase boldface, and sets by uppercase calligraphic font. For a vector x, [x]i denotes the ith element of x. For a matrix X, [X]i: denotes the vector on its ith row, [X]ij denotes the (i, j)th element of X. The superscript (·)> denotes the matrix transpose.

heading: 2 Review: Graph Neural Networks (GNNs)
text: A graph neural network (GNN) can learn expressive node/graph representation from the topological structure and associated features of a graph via neighborhood aggregation [13, 20, 21]. Consider a graph G = {V, E} with node feature h(0)v for each node v 2 V and edge feature bvu for each edge evu 2 E between nodes v, u. At the lth layer, GNN updates the node embedding h(l)v of node v as:
h (l) v = UPDATE
(l) ⇣ h
(l 1) v , AGGREGATE (l) ⇣ {(h(l 1)v ,h(l 1)u ,bvu)|u 2 N (v)} ⌘⌘ , (1)
where N (v) is a set of neighbors of v. After L iterations of aggregation, the graph-level representation g for G is obtained as
g = READOUT ⇣ {h(L)v |v 2 V} ⌘ , (2)
where READOUT(·) function aggregates all node embeddings into the graph embedding [22]. Our paper is related to GNN in two aspects: (i) use graph-based molecular encoder to obtain molecular representation, and (ii) conduct graph structure learning to model relation graph among molecules.
Graph-based Molecular Representation Learning. Representing molecules properly as fixedlength vectors is vital to the success of downstream biomedical applications [23]. Recently, graphbased molecular representation learning methods are popularly used and obtain state-of-the-art performance. A molecule xi is represented as an undirected graph Gi = {Vi, Ei}, where each node v 2 Vi represents an atom with feature h(0)v 2 Rd n
and each edge evu 2 Ei represents the bond between two nodes v, u with feature bvu 2 Rd e
. Graph-based molecular representation learning methods use GNNs to obtain graph-level representation gi as molecular embedding. Examples include graph convolutional networks (GCN) [24], graph attention networks (GAT) [25], message passing neural networks (MPNN) [20], graph isomorphism network (GIN) [22], pretrained GNN (Pre-GNN) [26] and GROVER [9].
Existing two works in few-shot molecular property prediction both use graph-based molecular encoder to obtain molecular embeddings: IterRefLSTM [3] uses GCN while Meta-MGNN [8] uses Pre-GNN. Using these graph-based molecular encoders cannot discover molecular substructures corresponding to the target property. There exist GNNs which handle subgraphs [27, 28, 29], which are usually predefined or simply K-hop neighborhood. While discovering and enumerating molecular substructures is extremely hard even for domain experts [17, 30]. In this paper, we first obtain molecular embeddings using graph-based molecular encoders. We further learn to extract relevant substructure embeddings w.r.t. the target property upon these generic molecular embeddings, which is more effective and improves the performance.
Graph Structure Learning. As the provided graphs may not be optimal, a number of graph structure learning methods target at jointly learning graph structure and node embeddings [31, 32, 33]. In general, they iterate over two procedures: (i) estimate adjacency matrix (i.e., refining neighborhood u 2 N (v)) which encodes graph structure from the current node embeddings; and (ii) apply a GNN on this updated graph to obtain new node embeddings.
There exist some FSL methods [34, 35, 36, 37, 38] which learn to construct fully connected relation graph among images in a N -way K-shot few-shot image classification task. Their methods cannot work for the 2-way K-shot property prediction tasks where choosing a wrong neighbor in the different class will heavily deteriorate the quality of molecular embeddings. We share the same spirit of learning relation graph, and further design several regularizations to encourage our adaptive property-aware relation graph learning module to select correct neighbors.

heading: 3 Proposed Method
text: In this section, we present the details of PAR, whose overall architecture is shown in Figure 2. Considering few-shot molecular property prediction problem, we first use a specially designed embedding function to obtain property-aware molecular embedding for each molecule, and then adaptively learn relation graph among molecules which allows effective propagation of the limited labels. Finally, we describe our meta-learning strategy to train PAR.

heading: 3.1 Problem Definition
text: Following the problem definition adopted by IterRefLSTM [3] and Meta-MGNN [8], the target is to learn a predictor from a set of few-shot molecular property prediction tasks {T⌧}Nt⌧=1 and generalize to predict new properties given a few labeled molecules. The ⌧ th task T⌧ predicts whether a molecule x⌧,i with index i is active (y⌧,i = 1) or inactive (y⌧,i = 0) on a target property, provided with a small number of K labeled samples per class. This T⌧ is then formulated as a 2-way K-shot classification task with a support set S⌧ = {(x⌧,i, y⌧,i)}2Ki=1 containing the 2K labeled samples and a query set Q⌧ = {(x⌧,j , y⌧,j)} Nq⌧ j=1 containing N q ⌧ unlabeled samples to be classified.

heading: 3.2 Property-aware Molecular Embedding
text: As different molecular properties are attributed to different molecule substructures, we design a property-aware embedding function to transform the generic molecular embeddings to substructureaware space relevant to the target property.
As introduced in Section 2, graph-based molecular encoders can obtain good molecular embeddings. By learning from large-scale tasks, they can capture generic information shared by molecules [26, 9]. Thus, we first use a graph-based molecular encoder such as GIN [22] and Pre-GNN [26] to extract a molecular embedding g⌧,i 2 Rd g
of length dg for each x⌧,i. The parameter of this graph-based molecular encoder is denoted as Wg .
However, existing graph-based molecular encoders cannot capture property-aware substructures. Especially when learning across tasks, a molecule can be evaluated for multiple properties. This leads to a one-to-many relationship between a molecule and properties, which makes few-shot molecular property prediction particularly hard. Thus, we are motivated to implicitly capture substructures in the embedding space w.r.t. the target property of T⌧ . Let cc⌧ denote the class prototype for class c 2 {0, 1}, which is computed as
c c ⌧ = 1/|S c ⌧ | X
(x⌧,i,y⌧,i)2Sc⌧ g⌧,i, (3)
where Sc⌧ = {(x⌧,i, y⌧,i)|(x⌧,i, y⌧,i) 2 S⌧ and y⌧,i = c}. We take these class prototypes as the context information of T⌧ , and further encode them as
b⌧,i = ⇥ softmax(C⌧,iC>⌧,i/ p dg)C⌧,i ⇤ 1: with C>⌧,i = [g⌧,i, c 0 ⌧ , c 1 ⌧ ] 2 Rd g⇥3, (4)
where [·]j: extracts the jth row vector which corresponds to x⌧,i. Here b⌧,i is computed using scaled dot-product self-attention [39], such that each g⌧,i can be compared with class prototypes in a dimensional wise manner. The property-aware molecular embedding p⌧,i is then obtained as
p⌧,i = MLPWp(concat[g⌧,i,b⌧,i]). (5)
MLPWp denotes the multilayer perceptron (MLP) parameterized by Wp, which is used to find a lower dimensional space which encodes substructures that are more relevant to the target property of T⌧ . This contextualized p⌧,i is property-aware which can be more predictive of the target property.

heading: 3.3 Adaptive Relation Graph Among Molecules
text: Apart from relevant substructures, the relationship among molecules also changes across properties. As shown in Figure 1, two molecules with a shared property can be different from each other on another property [1, 40, 41]. Therefore, we further propose an adaptive relation graph learning module to capture and leverage this property-aware relation graph among molecules, such that the limited labels can be efficiently propagated between similar molecules.
In this relation graph learning module, we alternately estimate the adjacency matrix of the relation graph among molecules and refine the molecular embeddings on the learned relation graph for T times.
At the tth iteration, let G(t)⌧ denotes the relation graph where V⌧ takes the 2K molecules in S⌧ and a query molecule in Q⌧ as nodes. A(t)⌧ 2 R(2K+1)⇥(2K+1) denotes the corresponding adjacency matrix encoding the G(t)⌧ , where [A(t)⌧ ]ij 0 if nodes x⌧,i,x⌧,j 2 V⌧ are connected. Ideally, the similarity between property-aware molecular embeddings p⌧,i,p⌧,j of x⌧,i,x⌧,j reveals their relationship under the current property prediction task. Therefore, we set h(0)⌧,i = p⌧,i initially.
We first estimate A(t)⌧ using the current molecular embeddings. The (i, j)th element of [A (t) ⌧ ]ij records the similarity between x⌧,i,x⌧,j which is calculated as: h A
(t) ⌧
i
ij = MLPWa(exp( |h (t 1) ⌧,i h (t 1) ⌧,j |)), (6)
where Wa is the parameter of this MLP. The resultant A (t) ⌧ is a dense matrix, which encodes a fully connected G(t)⌧ .
However, a query molecule only has K real neighbors in G(t)⌧ in a 2-way K-shot task. For binary classification, choosing a wrong neighbor in the opposite class will heavily deteriorate the quality of molecular embeddings, especially when only one labeled molecule is provided per class. To avoid the interference of wrong neighbors, we further reduce G(t)⌧ to a K-nearest neighbor (KNN) graph, where K is set to be exactly the same as the number of labeled molecules per class in S . The indices of the top K largest [A(t)⌧ ]ij , j = 1, . . . , 2K 1 for x⌧,i is recorded in N (t)(x⌧,i). Then, we set
h Â
(t) ⌧
i
ij = ( [A(t)⌧ ]ij if x⌧,j 2 N (t)(x⌧,i) 0 otherwise . (7)
The values in [Â(t)⌧ ] are normalized to range between 0 and 1, which is done by applying softmax function on each row [Â(t)⌧ ]i:. This normalization can also be done by z-score, min-max and sigmoid normalization. Then, we co-adapt each node embedding h(t) with respect to other node embeddings on this updated relation graph encoded Â(t)⌧ . Let H (t) ⌧ denote all node embeddings collectively where the ith row corresponds to h(t)⌧,i. H (t) ⌧ is updated as
H (t) ⌧ = LeakyReLu(Â (t) ⌧ H (t 1) ⌧ Wr), (8)
where Wr is a learnable parameter.
After T iterations, we return h⌧,i = [H (T ) ⌧ ]i: as the final molecular embedding for x⌧,i, and Â⌧ = Â (T ) ⌧ as the final optimized relation graph.
Denote ŷ⌧,i as the class prediction of x⌧,i w.r.t. active/inactive, which is calculated as
ŷ⌧,i = softmax Wc · h⌧,i , (9)
where [softmax(x)]i = exp([x]i)/ P2K+1 j exp([x]j) is applied per row, and Wc is a parameter.
Algorithm 1 Meta-training procedure for PAR.
1: initialize ✓ = {Wg,Wa,Wr} and = {Wp,Wc} randomly; if a pretrained molecular encoder is available, take its parameter as Wg; 2: while not done do 3: sample a batch of tasks T⌧ ; 4: for all T⌧ do 5: sample support set S⌧ and query set Q⌧ from T⌧ ; 6: obtain molecular embedding g⌧,i for each x⌧,i by a graph-based molecular encoder; 7: adapt g⌧,i to be property-aware p⌧,i by (5); 8: initialize node embeddings as h(0)⌧,i = p⌧,i; 9: for t = 1, . . . , T do 10: estimate adjacency matrix A(t)⌧ of relation graph among molecules using h (t 1) ⌧,i by (6); 11: refine h(t)⌧,i on the updated relation graph A (t) ⌧ by (8); 12: end for 13: obtain class prediction ŷ⌧,i using h⌧,i = h (T ) ⌧,i ; 14: evaluate training loss L(S⌧ , f✓, ) on S⌧ ; 15: fine-tune as ⌧ by (11); 16: evaluate testing loss L(Q⌧ , f✓, ⌧ ) on Q⌧ ; 17: end for 18: update ✓ and by (12); 19: end while

heading: 3.4 Training and Inference
text: We denote PAR as f✓, . In particular, ✓ = {Wg,Wa,Wr} denotes the collection of parameters of graph-based molecular encoder and adaptive relation graph learning module. While = {Wp,Wc} includes the parameters of property-aware molecular embedding function and classifier.
We adopt the gradient-based meta-learning strategy [42]: a good initialized parameter is learned from a set of meta-training tasks {T⌧}Nt⌧=1, which acts as starting point for each task T⌧ . Upon this general strategy, we selectively update parameters within tasks in order to encourage the model to capture generic and property-aware information separately. In detail, we keep ✓ fixed while fine-tuning as ⌧ on S⌧ in each T⌧ . The training loss L(S⌧ , f✓, ) evaluated on S⌧ takes the form:
L(S⌧ , f✓, )= X
(x⌧,i,y⌧,i)2S⌧ g(x⌧,i, y⌧,i, f✓, ) (10)
with g(x⌧,i, y⌧,i, f✓, ) = y>⌧,i · log(ŷ⌧,i)+ P (x⌧,m,y⌧,m)2S⌧ ([A ⇤ ⌧ ]im [Â⌧ ]im)2 where y⌧,i 2 R2 is a one-hot vector with all 0s but a single one denoting the index of the ground-truth class c 2 {0, 1}, and A⇤⌧ is computed using ground-truth labels with [A⇤⌧ ]ij = 1 if y⌧,i = y⌧,j and 0 otherwise. The first term is the cross entropy for classification. The second term is the specially designed neighbor alignment regularizer which penalizes the selection of wrong neighbors in the relation graph.
⌧ is obtained by taking a few gradient descent updates with learning rate ↵: ⌧ = ↵r L(S⌧ , f✓, ). (11)
✓⇤ and ⇤ are learned by optimizing the following objective:
min ✓, XNt ⌧=1 L(Q⌧ , f✓, ⌧ ), (12)
where the loss L(Q⌧ , f✓, ⌧ ) is calculated in the same form of (10) but is evaluated on Q⌧ instead. It is also optimized by gradient descent [42].
The complete algorithm of PAR is shown in Algorithm 1. Line 6-7 correspond to property-aware embedding p⌧,i which encodes substructure w.r.t the target property (Section 3.2). Line 8-12 correspond to adaptive relation graph learning which facilitates effective label propagation among similar molecules (Section 3.3).
For inference, the generalization ability of PAR is evaluated on the query set Qnew of each new task Tnew which tests on new property in meta-testing stage. Still, ✓⇤ is fixed and ⇤ is fine-tuned on Snew.

heading: 4 Experiments
text: We perform experiments on widely used benchmark few-shot molecular property prediction datasets (Table 1) included in MoleculeNet [43]. Details of these benchmarks are in Appendix A.

heading: 4.1 Experimental Settings
text: Baselines. In the paper, we compare our PAR3 (Algorithm 1) with two types of baselines: (i) FSL methods with graph-based molecular encoder learned from scratch, including Siamese [44], ProtoNet [45], MAML [42], TPN [35], EGNN [36], and IterRefLSTM [3]; and (ii) methods which leverage pretrained graph-based molecular encoder including Pre-GNN [26], Meta-MGNN [8], and Pre-PAR which is our PAR equipped with Pre-GNN. We use results of Siamese and IterRefLSTM reported in [3] as the codes are not available. For the other methods, we implement them using public codes of the respective authors. More implementation details are in Appendix B.
Generic Graph-based Molecular Representation. Following [26, 8], we use RDKit [46] to build molecular graphs from raw SMILES, and to extract atom features (atom number and chirality tag) and bond features (bond type and bond direction). For all methods re-implemented by us, we use GIN [22] as the graph-based molecular encoder to extract molecular embeddings. Pre-GNN, Meta-MGNN and Pre-PAR further use the pretrained GIN which is also provided by the authors of [26].
Evaluation Metrics. Following [26, 8], we evaluate the binary classification performance by ROCAUC scores calculated on the query set of each meta-testing task. We run experiments for ten times with different random seeds, and report the mean and standard deviations of ROC-AUC computed over all meta-testing tasks.

heading: 4.2 Performance Comparison
text: Table 2 shows the results. Results of Siamese, IterRefLSTM and Meta-MGNN on ToxCast are not provided: the first two methods lack codes and are not evaluated on ToxCast before, while Meta-MGNN runs out of memory as it weighs the contribution of each task among all tasks during meta-training. As can be seen, Pre-PAR consistently obtains the best performance, while PAR obtains the best performance among methods using graph-based molecular encoders learned from scratch. In terms of average improvement, PAR obtains significantly better performance than the best baseline learned from scratch (e.g. EGNN) by 1.59%, and Pre-PAR is better than the best baseline with pretrained molecular encoders (e.g. Meta-MGNN) by 1.49%. Pre-PAR also takes less time and episodes to converge than Meta-MGNN, which is shown in Appendix C.1. In addition, we observe that FSL methods that learn relation graphs (i.e., GNN, TPN, EGNN) obtain better performance than the classic ProtoNet and MAML.

heading: 4.3 Ablation Study
text: We further compare Pre-PAR and PAR with the following variants: (i) w/o P: w/o applying the property-aware embedding function; (ii) w/o context in P: w/o context b⌧,i in equation (5); (iii) w/o R: w/o using the adaptive relation graph learning; (iv) w/ cos-sim in R: use cosine similarity to obtain the adjacency matrix as ⇥ A⌧ ⇤ ij = p>⌧,ip⌧,j/(kp⌧,ik2kp⌧,jk2), then calculate (7) and (8) as in PAR; (v) w/o KNN in R: w/o reducing G⌧ to KNN graph; (vi) w/o reg: w/o using the neighbor alignment regularizer in equation (10); and (vii) tune all: fine-tune all parameters on line 15 of Algorithm 1. Note that these variants follows control variates method. They cover all components of training PAR without overlapping functionalities.
Results on 10-shot tasks are in Figure 3. Again, Pre-PAR obtains better performance than PAR due to a better starting point. PAR and Pre-PAR outperform their variants. The removal of any component leads to significant performance drop. In particular, the performance gain of PAR and Pre-PAR with
3Codes are available at https://github.com/tata1661/PAR-NeurIPS21.
respect to “w/ cos-sim in R" validates the necessity of learning a similarity function from the data rather than using the fixed cosine similarity. We also try to iterate the estimation of relation graph constructed by cosine similarity, but observe a performance drop given more iterations. Results on 1-shot is put in Appendix C.2 where the observations are consistent.

heading: 4.4 Using Other Graph-based Molecular Encoders
text: In the experiments, we use GIN and its pretrained version. However, as introduced in Section 3.2, our PAR is compatible with any existing graph-based molecular encoder introduced in Section 2. Here, we consider the following popular choices as the encoder to output g⌧,i: GIN [22], GCN [24], GraphSAGE [14] and GAT [25], which are either learned from scratch or pretrained. We compare the proposed PAR with simply fine-tuning the encoder on support sets (denote as GNN).
Figure 4 shows the results. As can be seen, GIN is the best graph-based molecular encoder among the four chosen GNNs. PAR outperforms the fine-tuned GNN consistently. This validates the effectiveness of the property-aware molecular embedding function and the adaptive relation graph learning module. We further notice that using pretrained encoders can improve the performance except for GAT, which is also observed in [26].
Although using pretrained graph-based molecular encoders can improve the performance in general, please note that both molecular encoders learned from scratch or pretrained are useful. Pretrained encoders contain rich generic molecular information by learning enormous unlabeled data, while encoders learned from scratch can carry some new insights. For example, the recent DimeNet [47] can model directional information such as bond angles and rotations between atoms, which has no pretrained version. As our proposed method can use any molecular encoder to obtain generic molecular embedding, it can easily accommodate newly proposed molecular encoder w/o or w/ pretraining.

heading: 4.5 Case Study
text: Finally, we validate whether PAR can obtain different property-aware molecular embeddings and relation graphs for tasks containing overlapping molecules but evaluating different properties.
To examine this under a controlled setting, we sample a fixed group of 10 molecules on Tox21 (Table 5 in Appendix C.3) which coexist in different meta-testing tasks (i.e., the 10th, 11th and 12th tasks). Provided with the meta-learned parameters ✓⇤ and ⇤, we take these 10 molecules as the support set to fine-tune ⇤ as ⇤⌧ and keep ✓⇤ fixed in each task T⌧ . As the support set is fixed now, the ratio of active molecules to inactive molecules among the 10 molecules may not be 1:1 in the three tasks. Thus, the resultant task may not evenly contain K labeled samples per class.
Visualization of the Learned Relation Graphs. As described in Section 3.3, PAR returns Â⌧ as the adjacency matrix encoding the optimized relation graph among molecules. Each element [Â⌧ ]ij records the pairwise similarity of the 10 molecules and a random query (which is dropped then). As the number of active and inactive molecules may not be equal in the support set, we no longer reduce adjacency matrices A⌧ to Â⌧ which encodes KNN graph. Figure 5 plots the optimized adjacency matrices obtained on all three tasks. As can be observed, PAR obtains different adjacency matrices for different property-prediction tasks. Besides, the learned adjacency matrices are visually similar to the ones computed using ground-truth labels.
Visualization of the Learned Molecular Embeddings. We also present the t-SNE visualization of g⌧,i (molecular embedding obtained by graph-based molecular encoders), p⌧,i (molecular embedding obtained by property-aware embedding function), and h⌧,i (molecular embedding returned by PAR) for these 10 molecules. For the same x⌧,i, g⌧,i is the same across 10th, 11th, 12th task, while p⌧,i and h⌧,i are property-aware. Figure 6 shows the results. As shown, PAR indeed captures property-aware information during encoding the same molecules for different molecular property prediction tasks. From the first row to the third row in Figure 6, molecular embeddings gradually get closer to the class prototypes on all three tasks.

heading: 5 Conclusion
text: We propose Property-Aware Relation networks (PAR) to address the few-shot molecular property prediction problem. PAR contains: a graph-based molecular encoder to encode the topological structure of the molecular graph, atom features, and bond features into a molecular embedding; a property-aware embedding function to obtain property-aware embeddings encoding context information of each task; and an adaptive relation graph learning module to construct a relation graph to effectively propagate information among similar molecules. Empirical results consistently show that PAR obtains state-of-the-art performance on few-shot molecular property prediction problem.
There are several directions to explore in the future. In this paper, PAR is evaluated on biophysics and physiology molecular properties which are modeled as classification tasks. While the prediction of quantum mechanics and physical chemistry properties are mainly regression tasks, it is interesting to extend PAR to handle these different levels of molecular properties. In addition, although PAR targets at few-shot molecular property prediction, the proposed property-aware embedding function, adaptive relation graph learning module, and the neighbor alignment regularizer can be helpful to improve the performance of graph-based molecular encoders in general. Finally, interpreting the substructures learned by PAR is also a meaningful direction.

heading: Acknowledgements
text: We sincerely thank the anonymous reviewers for their valuable comments and suggestions. Parts of experiments were carried out on Baidu Data Federation Platform.

Review summaries:
This paper proposes a novel framework to solve a few-shot molecular property prediction problem w.r.t different properties. It takes GNN as the encoder to obtain property-aware molecular embeddings and projects it to a substructure-aware space according to the target property. Moreover, a relation graph between molecules is learned adaptively to refine the embeddings. Training strategies like gradient-based meta-learning is applied to boost property-aware training. Extensive experiments are conducted over a few-shot property prediction task as well as transfer learning across datasets

The authors look at molecular property prediction from a few shot learning perspective. They propose an approach, closely following Meta-MGNN, to adapt part of a (optionally pre-trained) graph embedding network in MAML-style per task. A number of well-motivated novelties are introduced: (a) adapt part of the GNN encoder, called property-aware transformation (vs just task-specific attention in Meta-MGNN). (b) explicitly constructing the similarity graph between a test sample and the support set (labeled samples) and add a sparsity loss for the adjacency matrix during training. The results, focusing on 1-shot and 10-shot prediction, convincingly show gains in this regime, and extensive ablations and baselines are presented. 

This paper proposes a property-aware adaptive relation networks for few shot molecular property prediction.  They propose a graph based molecule encoder which is trained from large scale tasks to capture a generic representation of the molecules and a property aware embedding function which adapts the generic molecule embedding to be property-aware. Next, an adaptive relation graph learning module is proposed to capture the relationship between similar molecules with respect to a target property. The adaptive relation graph learning module is based on GNNs. A new training strategy, where only the parameters of property aware embedding function and the classifier module is tuned, is also proposed. The authors argue that it useful to separately capture generic molecule representations and those that are specific to the particular task.