Functionally Regionalized Knowledge Transfer for Low-resource Drug Discovery
Authors: Huaxiu Yao
Ying Wei
Long-Kai Huang
Ding Xue
Junzhou Huang
Zhenhui Li
Venue: NeurIPS 2021 Poster (2021)
Keywords: low-resource drug discovery
knowledge transfer
functional regions

Abstract: More recently, there has been a surge of interest in employing machine learning approaches to expedite the drug discovery process where virtual screening for hit discovery and ADMET prediction for lead optimization play essential roles. One of the main obstacles to the wide success of machine learning approaches in these two tasks is that the number of compounds labeled with activities or ADMET properties is too small to build an effective predictive model. This paper seeks to remedy the problem by transferring the knowledge from previous assays, namely in-vivo experiments, by different laboratories and against various target proteins. To accommodate these wildly different assays and capture the similarity between assays, we propose a functional rationalized meta-learning algorithm FRML for such knowledge transfer. FRML constructs the predictive model with layers of neural sub-networks or so-called functional regions. Building on this, FRML shares an initialization for the weights of the predictive model across all assays, while customizes it to each assay with a region localization network choosing the pertinent regions. The compositionality of the model improves the capacity of generalization to various and even out-of-distribution tasks. Empirical results on both virtual screening and ADMET prediction validate the superiority of FRML over state-of-the-art baselines powered with interpretability in assay relationship.

Functionally Regionalized Knowledge Transfer for Low-resource Drug Discovery
Huaxiu Yao'; Ying Wei', Long-Kai Huang Ding Xue
Junzhou Huang', Zhenhui Li
Stanford University, *City University of Hong Kong, Tencent AI Lab
University of Texas at Arlington, PPennsylvania State University
"haaxiu @cs. ess.taaoord.eeuu
Plongkai @ gmail.com, dingxue dingxue@tenccent @tencent.com "jzhuang @ uta.edu, "jessieli ist.psu.edd
Abstract
More recently, there has been surge of interest in employing machine learning approaches expedite the drug discovery process where virtual screening for hit discovery and ADMET prediction for lead optimization play essential roles. One of the main obstacles to the wide success of machine learning approaches in these two tasks is that the number of compounds labeled with activities or ADMET properties is too small to build an effective predictive model. This paper seeks remedy the problem by transferring the knowledge from previous assays, namely in-vivo experiments, by different laboratories and against various target proteins. To accommodate these wildly different assays and capture the similarity between assays, we propose a functional rationalized meta-learning algorithm FRML for such knowledge transfer. FRML constructs the predictive model with layers of neural sub-netwe networks or so-called functional regions. Building on this, FRML shares initialization for the weights of the predictive model across all assays, while customizes it to each assay with a region localization network choosing the pertinent regions. The compositionality of the model improves the capacity of generalization to various and even -uitt--distribution tasks. Empirical results on both virtual screening and ADMET prediction validate the superiority of FRML over state-of-the-art baselines powered with interpretability in assay relationship.
Introduction
Drug discovery brings candidate medications to billions of people, helping them live longer, healthier and more productive lives. One crux step in drug discovery virtual screening, which is fast and cost-effective method that computationally predicts the activity value of a compound against the target protein of a disease. As shown in Figure [(a), the hits screened out of large drug libraries of compounds by virtual screening algorithm in-vivo activities, resulting in leads. After optimizing the ADMET properties (absorption, distribution, metabolism, excretion and toxicities) of the leads, we obtain the drug candidates.
further empirically validated against their
There have been both traditional machine learning 33 and deep learning approaches [4] devoted to virtual screening, while the prediction performance (i.e., the hit rate) is far from satisfactory. The crucial challenge lies in that the number of training compounds whose activities have been tested against the target protein of focus is severely limited. Though state-of-the-aa deep learning algorithms typically rely on supervision in the form of thousands to millions of annotated data, is highly expensive and almost impossible for in-vivo experiments to collect sufficient set of drug
Part of the work done when H.Y. was student at Penn State University, correspondence Y.W.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

compounds with activity labels. In fact, virtual screening as computational pre-screening method is desired precisely because of the prohibitive costs of an in-oivo experiment (i.e., an assay). Fortunately. previous assays conducted by different laboratories around the world towards a wide variety of diseases with different biological target proteins together provide a rich repository for learning the interactions between a protein and compound. For example, as COVID-19 and SARS share high amino acid sequence identity, previous assays against SARS 3CLpro and PLpro proteases contribute a lot to learning predictive model for COVID-19 [16].
We are highly motivated to transfer the knowl- edge of interactions from this repository to ad-
dress the scarcity of labeled compounds in the assay against the target protein of our focus,
0.6
PRUE
0.6
which we name as the target assay for conve-
Land
nience. The challenges of such knowledge trans-
FIUL
fer are two-fold: (1) how to share the transfer- able knowledge but meanwhile accommodate the wide variance between assays, and (2) how to adequately identify the nearest neighbor as- says to the target assay to reduce the risk of neg- ative transfer. Since assays are from different institutions and against various target proteins,
cot caloo
0.2
0.0
OOug candicis
Doranseposss Aristy
(a)
(b)
Figure 1: (a): Workflow for discovery of drug can- didates using virtual screening. (b): Distributions of activity values for 10 randomly selected assays. the compounds tested and the distribution of activity values vary lot from assay to assay. As evidenced in Figure [(b] there exists a large discrepancy between distributions of activity values for 10 randomly selected assays. The prevalent fine-tuning strategy in transfer learning [21], trains a single model on previous assays and fine-tunes it to the target assay - it struggles in predicting accurately for each assay and confuses the most similar assays to the target with the others.
Gradient-based meta-learning 7 has been a promising practice, which learns from previous assays an initialization for shared predictive model and adapts the model from this initialization to each assay. while the initialization is learned so that the adapted model of each assay generalizes well on testing compounds, maintaining a shared initialization is still insufficient to handle wildly varying assays (37
and pinpoint the most similar assays. Recent efforts on heterogeneous meta-learnin deal with this issue by modulating the shared initialization to different assays via task embedding 20
35 37
Instead of only differentiating initializations, motivated by compositionality and brain functional specialization in neuroscience [5 26),
we aim push ahead with distinguishing neural sub-networks, or so-called finctional regions, each of which consists of a disparate set of parameters. This advancement brings at least the following two benefits. First, the similarity between assays is more accurately measured in a divide-and-coneeee in those overlapping regions between two assays are considered for comparison. Second, the reduced parameter space prevents the predictive model from overfitting to a limited set of training compounds.
manner only modulation for the initialization weights
We name the resulting meta-lcarning algorithm FRML. The predictive model of the FRML is dissected into a sequence of hierarchically organized functional regions. Provided with an assay, the contrastive assay representation network forwards the learned assay embedding to region localization network. The region localization network locates the most relevant functional regions for the assay in recurrent manner, to be consistent with the hierarchical organization of functional regions. In the stage of meta-training on previous assays, both the region localization network and the weights for initializations of all functional regions jointly learned. When it comes meta-testing, FRML quickly adapts to the target assay via easy assembly of the located regions.
We summarize major contributions as follows. (1) We propose novel meta-learnin algorithm FRML, which pushes a step forward from differentiating initializations to differentiating neural sub-networks between tasks; (2) We demonstrate the effectiveness of FRML on not only virtual screening but also the task of ADMET prediction. (3) FRML respects the key principle of machine learning models in healthcare- it is interpretable in the relationship between assays.
2 Notations and Problem Definition
In this section, we define some notations and discuss our problem. In drug prediction, we consider each task Ti an assay which refers to an in-vivo experiment group of compounds, and all tasks

---

are sampled from the distribution p(T). Note that we use either task or assay alternatingly in the remainder of this paper. Assuming that we have N historical assays (T.) meta-traimin assays, we aim to generalize a meta-learner from these meta-training assays and quickly adapt it to unseen target assays {Ta) No even with limited amount of annotated data. Here, we define the process of learning well-generalized meta-knowe meta-knowledge from the meta-traing veta-training assays as the meta-training phase and the adaption process on the target assays as the meta-testing phase.
Concretely, for each task Tr, a support set training samples DJ = (X; = {(x', y 1
and a query set of testing samples D = {X", Y?) ((x,,")).,) "")= are sampled from Tr, where vu and n represent the number of support and query samples, respectively. Denote that the feature space X' and the label space V, a predictive model (a.k.a., base learner) f - y defined to A' to its predicted value ý E y. For each task T. the base learner f is updated
map sample from the initialization 0e by minimizing the expected empirical loss L on D;, i.e., ming C(O: D;), resulting in the optimal parameters 0,. Specifically, the loss function C is defined as mean square enror (i.e., Exnee (x) or cross-entropy loss (i.e.. - Lixyeep logp(y|x, fo)) for regression and classification problems, respectively. In the meta-training phase, the query sets {D; ) of all meta-training assays are used to optimize the initialization of the base learner, so that the final initialization 0 is well-generalized. of can be further adapted to each meta-testing task Ti via the corresponding support set Dj. Formally, we define our problem as,
Y = arg max Y?
The well cell-generalized model initial weights encrypt the comprehensive knowledge learned from meta-training assays. We will detail how to learn 6% in Section
3 Methodology
In this section, we introduce the proposed framework FRML whose overview is illustrated in Figure The goal of FRML is improve the generalization ability for wide range of and out-of- distribution target assays with limited training samples via discriminating functional regions between assays. To achieve this goal, we dissect the base learner into sequence of functional regions. Given a new assay, we propose region localization network taking the learned assay representation as input locate and assemble the most relevant functional regions. Subsequently, FRML can be quickly adapted to the novel assay on the assembled functional region set. In the following subsections, we will first discuss the predictive models for virtual screening and ADMET classification the base learner and our meta-learning pipeline. Then we elaborate the details of three key components (i.c.. assay representation learning, localization strategy, and region localization network).
Region L Localization
Assay Resaessstaiion
Networlk
PoL
Base Lesmnr
fo
Input.
D2
4
Figure 2: Overview of the proposed FRMI.. In each assay Tú, the recurrent region localization network, guided by its learned representation t,, locates the most relevant functional regions (darker blocks) and assembles them (trace: input-+ oh + thg 0bs + 0%, in the dissected base learner fe-
3.1 Predictive Models for Drug Discovery and Gradient-basee Meta-Learning
We build predictive models for virtual screening and ADMET prediction, both of which are crucial for drug discovery. The input to the predictive models is drug compound represented by 1024 dimensional Morgan fingerprints (28), i.e., E R1024 For virtual screening, the output the activity value of the compound against the target protein in this assay, i.c., y E R, while the output for
3

---

ADMET prediction could be discrete category or real value. In our empirical study, we only consider those ADMET prediction tasks of classification, ie., y C, where C denotes the set of property categories. Building on these, we construct a neural network consisting of two fully connected layers as the predictive model, which also serves as the base learner f. We denote the weights for the base learner f to be 0.
With the base learner f. we introduce gradient-based meta-learning as the backbone meta-learning framework, which regards the initialization 8 for the base learner as the transferable knowledge. Apparently, enjoys the advantage of being independent of problem types. Specifically, here we illustrate the gradient-based meta- meta-learming by using model-agnostic meta-learnin meta-learning (MAML) [6] as an example. In the meta- training phase, MAML obtains the assay-specific model for each assay Ti by updating the parameters 0 via the support set D in a few gradient steps starting from Oo, i.e..
A a VoC((0 D,).
Here denotes the learning rate for assay adaptation. Though only one gradient step is presented as exemplary in Eqn. (2). it is easy to extend to several gradient steps. The crux is to evaluate the adapted assay-specific model 0, on the query set Dj and leverage the result as feedback to meta-update the initializations Ou as,
0 0 3NN NNN
Σ ((O,))), T.EP(T)
(3)
where 8 is the learning rate for meta-updating. As result of the meta-training phase, we get the well-generalizedd initialization % for the base learner. In the meta-testing phase. the specific model d for each target assay Ti with the support set D; is achieved by a few gradient steps starting from the learned initialization 0g i.e., d, = 02 (00); D)). Finally, the performance is evaluated on the query set DD of the target assay T.. Without loss of generality, we again take MAML as the backbone meta-learning framework of FRML and detail each component in the following.
3.2 Contrastive Assay Representation Learning
Learning the representation of assay Ti a prerequisite determining the functional regions that are specific to the assay. Following previous works 37), we represent the assay with a representation vector t; E Rd by aggregating all training samples of the support set D;=((x", .y > aggregator AGG is involved. The aggregator consists of a mapping function denoted as MF (e.g.. recurrent network, convolutional network) that first encodes each individual sample into dense representation vector, and a sample-level mean pooling layer to summarize all samples to generate the assay representation t;. Note that the pooling guarantees the assay representation to be invariant of the permutation of samples. Formally, we define the aggregation process as,
where an
1 = n Σ MF(F(x)) y!).
L = AGG
(4)
where F() is an embedding function that transforms the input features into a low-dimessional vector. Both the embedded input features and the label are concatenated by the operator @. We will provide more details on the definitions of F() and (-)- later in Section
The loss function to train the parameters F(-) and MF(-) could be Eqn. (3) only. Unfortunately, it is far from enough to learn robust assay representation: first, the gradients back-propagated through the base learner and the region localization network tend to be too small for training to work effectively; second, the assay representation and the region localization are interleaving, so that the objective in Eqn. (3) takes them as a whole regardless of the accuracy for each of them. To overcome this limitation, we are motivated impose another loss function on the assay representation network directly. The key intuition is that each set of samples in an assay provides a partial view of the assay, and the assay representation is expected to be consistent across views. This motivates the constrastive objective different views of the same assay have similar task representations, while the representations of views from different assays should be different. Specifically, we create different views of assay Tr by randomly splitting D; into n. sets of size n /nc. By defining c, := ((u / un///e) /ne). we obtain n. subsets of equal size, i.e., Di = Or" We can now formulate the contrastive learning objective as follows:
exp ( (AGG)F(
Lat
Σ ISuSesmo
log
(5)
ΣN exp (.

---

where ① is similarity measure function. In our experiments, we adopt the dot product, i.c., ((a, b) = b. This contrastive loss function pushes the representations of different assays apart and meantime stabilizes the assay representation.
3.3 Localization Strategy
The assays are measured by different experimenters on different equipment, that they expected to have widely distributed assay representations. Given an assay with its representation, in this section, the localization strategy sets out to locate and assemble the functional regions that are specific to this assay. Before detailing the localization strategy, we first dissect the initialization do of the base learner into K functional regions. These functional regions are dissected in layer-wiss, wise manner to maintain the hierarchical structure of the neural network. For each layer 4, we denote its corresponding functional regions as 6 = 19 yAr! where M' represents the total number of functional regions in the I-th layer and Σ. .M' K.
Following the hierarchical representation in neural networks, we locate and assemble these functional regions in hierarchical manner- each functional region at layer + 1 receives signals from the functional regions at layer I. For each assay Tr, denoting the representation of functional region m' in layer I as h." we define the representation of functional region m (+1 in layer / + 1 to be:
M'
M'
h ++
cnn't
+11
P
mm h
Σ
P
=1
(6)
where pmi't (.) represents the mapping function for functional region ml+ p," mmmmm m1 defined as
the probability of functional region m' being assembled to +1 is crucial; a value of p." "'m'm""''
suggests that functional region m' should be included for assay T. Obviously, the probability p." mm'-mm/+) ++1 varies from assay to assay, so that we model it as a function of the representation t,, i.c.,
p m m' ++1 = RG(,,),
where RC(+) represents the region localization network we detail in the next subsection.
3.4
Region Localization Network
An ideal region localization network is expected to satisfy two criteria, including high representational capacity and consistency with the hierarchical structure behind functional regions. To meet the criteria, we propose recurrent region localization network, where a recurrent neural network (GRU as exemplary) used, The input to the recurrent neural network at step + +111 is the combination of assay representation E; and the assembly probabilistic set P of layer I, where p' = (pr" umm |mm [1, " m' E [1, M' Consequently, the hidden representation at step 1 is,
F = CRU @ r'),, + by.
(8)
where W, E R='xM'''' and by E Rixartr' are learnable parameters and F!+1 (Pm +m Im E [1, [1, m/ E [1, M/++1} RIXM' The hidden representations at step I 1. in return, determine the assembly probability at layer / + Note that the assembly probability is expected to be as close to the bounds of its range (0, 1) as possible, so that only the most pertinent functional regions are located. To this end, we apply the Gumbel-softmal estimator models the categorical distribution to F., i.e.,
[0] [8] which
exp((? +m 1+1 + qt +m 1/T)
p
(9)
- exp (m/ qe mml-mm11 1/7)
where 7 the temperature and g.th Gumbel(O, 1).
141 is sampled from the Gumbel distribution, i.c., qt" -mtt
Combining the meta-learning loss in Eqn. B and the contrastive loss in Eqn. (). we arrive the overall objective function of FRML defined as:
min Cail min
Σ L+ ACat
(10)
TPP((T
where the hyperparameter À balances between two losses and θ represents all learnable parameters. For better understanding of our framework, we show the meta-training process in Algorithm T the meta-testing process in Appendix A.
and
5

---

Algorithm 1 Meta-training Process of FRML
Require: (M* ..... 3 # of functional regions of each layer; 8: learning rates; A1, A2: item factors in loss
1: Randomly initialize e
2: while not done do
3: Sample batch of assays from p(7)
4: for all T; do
5: Sample D; D from T,
6:
Get assay representation t, in Eqn. (4) and the reconstruction loss Let via Eqn. (
7:
Use Eqn.
(8) to compute {fi..... ........ Ft)
8: Calculate [pr..... } by Eqn. ( and get the assembled trace across functional regions
9: Use gradient descent update parameters based on the learned trace: ===0, VVV
10: end for
Update + B Ve ETEA(T) [(0.: + (D))
end while
4
Experiments
In this section, we empirically evaluate the effectiveness of FRML on two diverse drug discovery tasks: drug activity prediction and ADMET property prediction. We consider comparison of the proposed FRML with three categories of baselines. The first category simply using base learner without assay adaptation, including FC-Individual and FC-AIl. The second category is knowledge transfer with assay adaptation: Fine-tuning, MAMI. [6] ANII. [24), ANIL+ 3 In the second category of methods, we also compared FRML with Prototypical Network (ProtoNet) [31] and Matching Network (MatchingNet) (34) for classification tasks, i.e., ADMET property prediction. The last category is heterogeneous meta-learning methods, including MMAML (35). HSML (37), ARML [38. Detailed descriptions of all baselines are provided in Appendix B and the detailed hyperparameters for both applications are listed in Appendix D.
4.1
Drug Activity Prediction
4.1.1
Dataset Description
For drug activity prediction, use the dose-response activity assays from ChEMBL(], where 4,276 assays are selected in this problem. Here, we randomly sample 100 assays as the meta-testing set, 76 assays as the meta-vale meta-validation set, and the rest of assays for meta-training. The random splitting is repeated four times to construct four assay groups, named Assay Group I, II, III, IV, respectively. A few support and query drug compounds are available for each assay. In terms of the features for each drug compound, we use 1.024-dimensional Moragn fingerprint implemented in RDKit [12]. For each assay Tr calculated the coefficient of determination (IZ?) between the predicted value Y! and the ground truth value YÝ. The median and mean I? values of all meta-testing assays are reported. We adopt another widely used metric for evaluating whether virtual screening model is usable in practice, i.e., the number of assays with R² > 0.3. More detailed information and data statistics are summarized in Appendix C.
4.1.2 Overall Performance
The performance of FRMI. and the baselines are reported in Table T In this experiment, FRMI. incorporates ANIL and ANIL+ while all other heterogeneous meta-learning algorithms (c.g.. MMAML) incorporate ANIL. Note that ANIL+ is modified from ANIL to improve stability. From the results in Table I we obtain the key observations: (1) The performance of FC-Individual is inferior to that of other methods, indicates that involving the data from source assays benefits the performance: (2) Gradient-band based leaanlea methods (MAML, ANIL, ANIL++, heterogeneous methods, and FRML) achieve significantly better performance than Fine-tuning corrugating our motivation that Fine-tuning may confuse the most similar assays to target with the others; (3) In most cases, heterogeneous methods (MMAML--ANIL, HSML-ANIL, ARML-ANII, FRML-ANIL) achieve better performance than homogeneous meta-learning models, showing the effectiveness of integrating assay-specific knowledge transfer, (4) Our proposed FRML-ANIL++ achieves the best per-

---

Table 1: Performance of drug activity prediction (Measured by mean R², median Ht2 and # R²2 > 0.3).
Assay Group1 I
Assay Group II
Assay Group III
Assay Group IV
Model
Mcan
Med.
R
00.
Meun
Med.
R² 20.3
Mcan
Med.
R² >0.
Mcan
Mod.
R 5003
FC-Individual
0.141
0.064
16
0.114
0.060
10
0.112
0.046
10
0.118
0.047
10
FC-AIl
0.228
0.131
30
0.187
0.103
23
0.199
0.103
28
0.252
0.160
35
Fine- tuning
0.251
0.166
37
0.197
0.124
24
0.219
0.121
31
0.266
0.194
37
MAML
0.291
0.182
38
0.232
0,158
29
0.265
0.191
36
0.302
0.256
46
ANIL
0.299
0.184
41
0.226
0.143
30
0.268
0.199
37
0.304
0.282
48
ANIL+
0.367
0.299
50
0.315
0.252
43
0.335
0.289
48
0.362
0.321
51
MMAML-ANIL
0.292
0.205
42
0.231
0.154
31
0.276
0.187
37
0.306
0.260
46
HSML-ANIL
0.295
0.192
41
0.234
0.145
34
0.277
0.196
35
0.306
0.254
47
ARML-ANIL
0.299
0.204
43
0.233
0.159
32
0.270
0.191
39
0.311
0.267
46
FRML-ANIL (ours)
0.310
0.226
44
0.237
0,162
35
0.285
0.207
40
0.322
0.287
49
FRMI-ANIL++ (ours)
0.375
0.328
52
0.327
0.311
51
0.345
0.315
51
0.372
0.349
56
Table 2: Ablation study on drug activity prediction.
Assay Group 1
Assay Group
Assay Group III
Assay Group
Model
Mean
Med.
It >0.3
Meun
Med.
ft >0.3
Meun
Med.
ft
20.3
Mean
Med.
It"
200
A
0.367
0.299
50
0.315
0.252
43
0.335
0.289
48
0.362
0.324
Ablation I (wwo cl)
0.371
0.315
51
0.318
0.263
45
0.338
0.305
49
0.368
0.338
54
Ablation III (wlo localization)
0.369
0.301
50
0.317
0.263
47
0.336
0.291
49
0.368
0.329
53
Ablation (RNN > FC)
0.372
0.303
52
0.326
0.299
0.311
0.306
50
0.367
0.333
FRML-ANII++ (ours)
0.375
0.328
52
0.327 0.311
51
0.345 0.315
51
0.372 0.349
56
formance in all four assay groups This possibly results from that differentiating neural sub-networks reduces the parameter space, which improves the generalization capability and further benefits the performance. Besides, integrating FRML. with ANIL also achieves consistent improvements, showing its compatibility with different backbone meta-learning models.
4.1.3 Ablation Study
To further show the effectiveness of the proposed modules in FRML, we conduct comprehensive ablation studies by comparing FRML with three ablation models described as follows. First, we consider an ablation model (Ablation I (w/o cl)) with the contrastive loss removed. Second, we design Ablation II (w/o localization) to show that the improvements of FRML is caused by knowledge localization rather than increasing the capacity of baseline. Third, we change the recurrent structure to plain localization network and propose Ablation III (RNN->FC), where fully connected layers with softmax are utilized to learn the assembly probability set (p!.....p!! ..').
We evaluate the ablation models all four assay groups and report the performance in Table 2 Note that FRML is also included in comparison. From the results in the table, we have the following three findings: (1) removing the contrastive loss hurts the performance, which indicates the effectiveness of the contrastive loss in learning well-differentiatee assay representations; (2) the superiority of FRML over abalation II demonstrates that the improvements stem from efficient knowledge structuring rather than larger model capacity; (3) compared to the plain localization network, the performance gain of recurrent region localization network demonstrates its superiority by predicting the assembly probability in hierarchical way.
4.1.4 Analysis
0.50
Effect of the Number of Functional Re- gions. We analyze the effect of the number of functional regions and illustrate the results in Figure 3(a) In this figure, we observe that (1) if the number of functional regions too small (e.g. 1), it may be insufficient to cap- ture the structures across assays. (2) when we continually increase the number of functional regions, the results keep stable or even slightly decrease, which are consistent with our find-
3.36
2.34
0.32
PINLANIL
0:20
0.30
0.25 Naslo Sumpled Support Set
Num. ofFunctionll Rogians
(b) Figure 3: (a): Num. of functional regions w.r.t the mean R² on Assay Group I, II. III. IV. (b): Perfor-
(a)
mance w.r.t support set ratio on Assay Group
7

---

ings that the gains of FRML arise out of the effective knowledge structuring instead of the increase of the model capacity.
Besides, we also conduct an experiment using a dif- ferent number of functional regions in each layer. Here, we use two functional blocks in layer one and four functional blocks in layer two. The results are re- ported in T 3
Table 3: Performance of FRML with differ- ent knowledge blocks in each layer on drug activity prediction. (2, 4) represents that we use blocks in layer and 4 blocks in layer 2. G-I represents group I. Mean St2 is reported.
We observe that the performance is slightly worse than the strategy of using three layers for each layer. One potential reason is that the Mor- gan fingerprint features have covered enough low- level features, and it might be more useful to add more functional blocks in the first layer.
of Blocks
G-I
G-II
G-III
G-IV
(2,
0.369
0.324
0.344
0.367
(3,3)
0.375
0.327
0.345
0.372
Effect of the Ratio of the Support set. In order to show the superiority of FRML under different ratios of the support set, we analyze the performance w.r.t. the support set ratio and show the results in Figure 3(b) When we down-sample the support to contain 5%, 25%, and 50% of all compounds in an assay, FRML consistently achieves better performance than the most competitive baseline +NIL+ This marks the capability of FRML in handling the data scarcity problem in healthcare.
Analysis of Localization Strategy. We further analyze the localization strategy, where the assembled traces of six randomly selected meta-testing assays from Group II are illustrated in Figure and their corresponding biological properties are reported in the right table of Figure[4] Here, we observe that the six assays mainly located in three different traces. Besides, assays 1640791, 701282, 1639959, 302952 activate the same trace 1-21. The trace groups are consistent with their biological properties reported in the table. First, assays 1640791 and 701282 are both cell-based functional assays targeting GPCRs by evaluating the antagonistic activity of compounds to their downstream cAMP pathway. 1639959 and 302952 both cell-based functional assay of membrane transporters. All four assays are targeting membrane proteins (receptors or transporters). Thus, they share the first layer but select different traces in the second layer. Second, different from the above four assays, assays 147797 and 1520 choose a completely different path since they are single protein assays that directly evaluate the effect of compounds to their protein targets. The consistency of localization results and biological properties further verify the effectiveness of FRML for distinguishing different domains via localization strategy.
Aspy
Type
Target
Target Group
Organion
1640791
Call-bead functiorel
G-getiin cosplel neepttr 6
GPCR
Homo sipiens
Cell fundiina
Postzaoid DP mecptic
GPCR
Hono sapiezs
(al: 1640791
Ib): 701202
(ch: 1839959
1630959
Cal-based functiona
Farsprtaa
Mastinas basaararr
Hono
300953
Cal-Aused functiend
Popide trarsportor
Masriaaz basqarder
Hono
147797
Sizgle proteix
Deexit 200estor2
GPCR
Homo susicas
1530
Sirgle groteer
Servioenn In (5-Ef) hel meeglor
GPOR
Homo sapiens
302952
[el: 147797
(F::
Figure 4: Left Figure (f)-(f) show the located traces from six meta-testing assays of Group II, where their corresponding biological information are reported in the right table. Darker blocks and blue links represent located functional regions and assembled links, respectively.
4.2
ADMET Property Prediction
4.2.1
Dataset Description & Evaluation Metric
Besides the drug activity prediction, we further evaluate FRML on ADMET property prediction. The AMDET Prediction problem is constructed by combining 4 benchmark datasets from the Molecu- leNet (36] with biophysiology and physiology targets. The 4 datasets are MUV (2,, SIDER [[]. Tox21 and ToxCast [27]. Each property prediction is a binary classification task. All the properties from MUV, SIDER, Tox21, and 22 properties form ToxCast are involved in the experiment, resulting in 68 tasks. We randomly sample 42 tasks for meta-training and use the remaining 26 tasks for meta-testing. Considering the data balance, for each tasks, we randomly sample only partial instances from the majority category to match the size of minority data, together with all the minority data, to
8

---

Table 4: Performance of ADEMT property prediction (averaged accuracy with 95% confidence
interval are reported).
Model
SIDER
Tox21
MUV
ToxCast
FC-Individual
52.12 + 0.81%
51.25 0.37%
52.91 + 0.67%
62.75 + 1.27%
FC-AIl
67.13 + 0.89%
68.63 + 0.84%
55.04 + 1.06%
70.82 + 1.61%
Fine-tuning
67.60 + 0.89%
68.84 0.84%
55.41 + 1.05%
71.04 + 1.59%
MAML
67.69 + 0.81%
69.12 +: 0.84%
56.66 + 1.09%
72.53 + 1.64%
ProtoNet
68.03 + 1.16%
69.29 1.30%
55.19 +: 1.18%
72.10 +: 1.52%
MatchingNet
66.69 + 1.04%
68.72 + 1.04%
55.15 1.07%
71.39 + 1.33%
ANIL
67.92 + 0.89%
69.81 0.85%
55.13 = 1.22%
72.09 + 1.78%
ANIL+
68.04 + 0.86%
68.94 + 0.92%
56.95 + 1.13%
72.66 + 1.67%
MMAMI-ANII
68.57 + 0.82%
69.86 0.90%
58.06 + 1.21%
72.10 + 1.55%
HSML-ANIL
69.15 + 0.87%
69.98 + 0.88%
57.94 + 1.18%
71.73 + 1.46%
ARML-ANIL
68.94 + 844
70.07 0.91%
58.99 + 1.16%
72.08 + 1.56%
FRML-ANIL (ours)
69.89 + 0.87%
70.85 0.85%
59.94 + 1.00%
73.56 + 1.58%
FRMI-ANIL++ (ours)
70.01 0.86%
71.07 0.91% %
60.66 + 1.09%
74.02 + 1.57%
form the task dataset. In this experiment, following the conventional few-shot learning protocol [7]. we apply 2-way classification with 5 5-shot support samples for each task. The details of the dataset descriptions are available in Appendix C.2. As for the model performance, it is measured by averaged classification accuracy.
4.2.2 Results
We report the performance of FRML and the baselines in Table Similar findings to that of drug activity prediction experiments are observed. Therefore, we again confirm the effectiveness and importance of integrating task-specific knowledge transfer in the proposed FRML, A specific finding is that: all heterogeneous meta-learning models and FRML obtain higher gain of performance on MUV than on the three datasets. In particular, FRML achieves significant performance improvement on MUV dataset. This may be caused by the category difference of MUV from the other three datasets which we will detail in the next subsection. Besides, we conduct similar ablation studies to those for drug activity prediction and report the results in Table Similar results are observed, again demostrating the effectiveness of FRMI. differentiating different properties.
Table 5: Ablation study of ADEMT property prediction (averaged accuracy with 95% confidence interval are reported).
Model
SIDER
Tox21
MUV
ToxCast
+NIL+
68.04 1 0.86%
68.94 1 0.92%
56.95 1 1.13%
72.66 + 1.67%
Ablation (w/o cl)
68.79 0.84%
69.65 1 0.86%
58.70 = 1.18%
72.86 1.68%
Ablation II (w/ localization)
68.30 + 0.83%
69.42 + 0.92%
57.33 + 1.04%
72.70 + 1.55%
Ahlation III (RNN->FC)
69.15 + 0.80%
70.08 + 0.87%
59.54 + 1.00%
73.41 1.68%
FRII-ANTIL+ (ours)
70.01 + 0.86%
71.07 + 0.91%
60.66 + 1.09%
74.02 + 1.57%
4.2.3 Analysis of Localization Strategy
In this part, we analyze the localization strat- egy for ADMET prediction. In Figure we show the assembled traces of four meta-testing tasks sampled from different sub-datasets. In these figures, tasks from different sub- domains are located in different trace groups (i.e., the three datasets SIDER, Tox21, Tox- cast select 1 1-+21 -211+22-p4 +4 while MUV selects
(a): SIDER
(b): Tox21
(c): MUV
(d): ToxCast
Figure 5: (a)-(d) show the assembled traces (blue links) among located regions (darker blocks) from four meta-testing tasks sampled from SIDER, Tox21, MUV, ToxCast, respectively.

---

1-222+31-+4, respectively). Compared to SIDER, Tox21, Toxcast, we notice that MUV selects different trace, which matches the natural difference between MUV and the other three datasets. The category of the MUV dataset is a biophysics while that of the other three are physiology. Besides, MUV is designed for validation of virtual screening techniques, while the other three are designed for measuring different targets.
5 Related Work
The goal for meta-learning is learn set of meta-knowledge that facilitates the learning process of new tasks. There are two mainstream categories of meta-learning approaches. The first category of algorithms, called gradient-based meta-learning algorithms, regards the meta-knowledge as initial- izations for the base learner
25 30]. As for the second category, i.c., metric-based meta-learning algorithms, the aim to learn a transferable metric space for the meta-learner as well as a lazy learner [13 31 32
17-9
13
39]. However, metric-based algorithms only handle classification problems. In light of this, we consider gradient-base algorithms which are flexible and general enough to be independent of problem types. The majority of grudient-basee meta-lcarning algorithms focus on maintaining a shared set of knowledoweee (i.e., the initializations for the weights) learned from meta-training tasks. To enhance the ability of generalization to more complicated heterogeneous tasks (e.g., tasks sampled from various distributions), recent studies customize the shared model weight initializations to different tasks modulating the globally-shared weight initializations to be task-specific 20 35 37 38). However, our proposed FRML goes further than customization of weight initializations it also differentiates neural sub-networks and enhances the generalization capability for significantly different tasks.
34
Up to now, only few studies have explored the application of meta-learning to address the problem of limited labeled data in healthcare. The two representative metric- based meta-learning algorithms, i.c., MatchingNet (34] and ProtoNet [I1, have been used for protein binding prediction [2] and dematological disease diagnosis [22]. As we mentioned above, metric-based meta-learning algorithms do not work for the regression of activity values we focus on in this work. On the other hand, Zhang al. 44). . Qiu et al. 23], and Luo al. [17] applied the widely used model agnostic meta-learning (MAML) algorithm 7 to the problems of clinical risk prediction, genomic survival analysis, and protein binding, respectively. Yet, the proposed FRML accommodates a wide range of assays effectively by tailoring sub-networks for each assay.
6 Conclusion
In this paper, aim to tackle the challenge of data insufficiency in drug discovery by transferring the knowledge from historical assays. Specifically, we propose novel meta-learning framework, FRML, effectively learn the transferable knowledge and meantime adapt various assays. FRML dissects the base learner into hierarchically organized functional regions. The representation of a target assay forwarded to the recurrent region localization network locate and assemble the assay-specific functional regions. The experiments on virtual screening and ADMET prediction demonstrate the effectiveness of FRML, and the analyses onThe the localization strategy further verify its sound interpretability in capturing the similarity between assays.
The limitation of this work is that we have not investigated the robustness of the proposed FRML. If the proposed framework is easy to be attacked, it may cause negative social impacts. For example, the framework suggests misleading results, it will delay even harm drug discovery progress. We will investigate the problem in the future.
Acknowledgement
H.Y. and Z.L. are supported in part by NSF awards IIS##1522225 and IIS+#1618448. The views and conclusions contained this paper are those of the authors and should not be interpreted as representing any funding agencies. This work was also supported in part by the start-up grant from City University of Hong Kong (9610512). Besides, Y.W. would like to acknowledge the support from the Tencent Al Lab Rhino- Gift Fund.
10

---

References
Chembl. so..............................
[2] Han Altae-Tran, Bharath Ramsundar, Aneesh S Pappu, and Vijay Pande. Low data drug discovery with one-shot learning. ACS central science, 3(4):283-293, 2017.
[3] Antreas Antoniou, Harrison Edwards, and Amos Storkey. How to train your maml. In International Conference on Learning Representations, 2019.
[4] Kristy Carpenter, David S Cohen, Juliet T Jarrell, and Xudong Huang, Deep learning and virtual drug screening. Future medicinal chemistry, 10(21):2557-2567, 2018.
[5] Michael W Cole, Joset A Etzel, Jeffrey M Zacks, Walter Schneider, and Todd S Braver. Rapid transfer of abstract rules novel contexts in human lateral prefrontal cortex. Frontiers in human neuroxcience, 5: 14, 2011.
[6] Chelsea Finn and Sergey Levine. Meta-learning and universality: Deep representations and gradient descent can approximate any learning algorithm. arXiv preprint arXiv:1710.11622. 2017.
[7] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adapta- tion of deep networks. In ICML,, pages 1126-1135, 2017.
[8] Chelsea Finn, Kelvin Xu, and Sergey Levine. Probabilistic model -agel-agnostic meta-learning. arXiv preprint arXiv: 02877 2018.
[9] Erin Grant, Chelsea Finn, Sergey Levine, Trevor Darrell, and Thomas Griffiths. Recasting gradient-based meta-learning as hierarchical bayes. arXiv preprint urXiv:180I.089) 2018.
[10] Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax. In ICLR, 2017.
[11] Michael Kuhn, Ivica Letunic, Lars Juhl Jensen, and Peer Bork. The sider database of drugs and side effects. Nucleic acids research, 4101))10075--1079, 2016.
[12] Greg Landrum. Rdkit: Open-source cheminformatics software. 2016. URL https://
httin//.wwo//oktt///////////eeee 2016 _9__]
[13] Yoonho Lee and Seungjin Choi. Gradient-based meta-leariin with learned layerwise metric and subspace. In ICML, pages 2927-2936, 2018.
[14] Zhenguo Li, Fengwei Zhou, Fei Chen, and Hang Li. Meta-sgd: Learning to learn quickly for few shot learning. arXiv preprini 3XXiv:/77.009835 2017.
[15] Ming- Yu Liu, Xun Huang, Arun Mallya, Tero Karras, Timo Aila, Jaakko Lehtinen, and Jan Kautz. Few-shot unsupervised image-to-image translation. arXiv preprint arXiv: 190..01905.0172. 2019.
[16) Yuzhi Liu, Chengyuan Liang, Liang Xin, Xiaodong Ren, Lei Tian, Xingke Ju, Han Li, Yongho Wang, Qianqian Zhao, Hong Liu, et al. The development of coronavirus 3c-like protease (3clpro) inhibitors from 2010 to 2020. European journal of medicinal chemistry, page 112711, 2020.
[17] Yunan Luo, Jianzhu Ma, Xiaoming Zhao, Yufeng Su, Yang Liu, Trey Ideker, and Jian Peng. Mitigating data scarcity in protein binding prediction using meta-learning. In RECOMB, pages 305-307. Springer, 2019.
[18] Chris J Maddison. Andriy Mnih, and Yee Whye Teh. The concrete distribution: A continuous relaxation of discrete random variables. 2017.
[19] Alex Nichol and John Schulman. Reptile: a scalable metalearning algorithm. arXiv preprint ar0iv: 02999, 2018
[20] Boris Oreshkin, Pau Rodríguez López, and Alexandre Lacoste. Tadam: Task dependent adaptive metric for improved few-shot learning. In NeurIPS, pages 721-731, 2018.
2018.
11

---

[21] Sinno Jialin Pan, Qiang Yang, et al. A survey on transfer learning. IEEE Transactions on knowledge and data engineering. 22(10)) 2010.
[22] Viraj Prabhu, Anitha Kannan, Murali Ravuri, Manish Chablani, David Sontag, and Xavier Amatriain. Prototypical clustering networks for dermatological disease diagnosis. arXiv preprint arXiv:1811.03066 2018.
[23] Yeping Lina Qiu, Hong Zheng, Arnout Devos, Heather Selby, and Olivier Gevaert. A meta- learning approach for genomic survival analysis. Nature communications, 11(1):1-11, 2020. 2020.
[24] Aniruddh Raghu, Maithra Raghu, Samy Bengio, and Oriol Vinyals. Rapid learning or feature reuse? towards understanding the effectiveness of maml. In ICLR, 2020.
[25] Aravind Rajeswaran, Chelsea Finn, Sham M Kakade, and Sergey Levine. Meta-learning with implicit gradients. In NeurlPS, pages 113-124, 2019.
[26] Carlo Reverberi, Kai Görgen, and John-Dylan Haynes. Compositionality of rule representations in human prefrontal cortex. Cerebral cortex, 1277-1133 2012.
[27] Ann M Richard, Richard Judson, Keith A Houck, Christopher M Grulke, Patra Volarath, Inthirany Thillainadarajah, Chihae Yang, James Rathman, Matthew T Martin, John F Wambaugh, al. Toxcast chemical landscape: paving the road to 21 century toxicology. Chemical research in toxicology, 29(8): 10011122-1222
[28] David Rogers and Mathew Hahn. serended-connectiviy fingerprints. Journal of chemical information and modeling, 50(5):742-754, 2010.
[29] Sebastian G Rohrer and Knut Baumann. Maximum unbiased validation (muv) data sets for virtual screening based on pubchem bioactivity data. Journal of chemical information and modeling, 49(2): 1992)11691184, 2009.
[30] Andrei A Rusu, Dushyant Rao, Jakub Sygnowski, Oriol Vinyals, Razvan Pascanu, Simon Osindero, and Raia Hadsell. Meta-learning with latent embedding optimization. arXiv preprint 1rXiv:1807.05964 2018.
[31] Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. In NeurlPS, pages 4077-4087, 2017.
[32] Flood Sung. Yongxin Yang, Li Zhang, Tao Xiang, Philip HS Torr, and Timothy M Hospedales. Learning to compare: Relation network for few-shot learning. CVPR, pages 1199-1208, 2018.
[33] Vladimir Svetnik, Andy Liaw, Christopher Tong, J Christopher Culberson, Robert P Sheridan, and Bradley Feuston. Random forest: a classification and regression tool for compound classification and qsar modeling. Journal of chemical information and computer sciences, 43 (6):1947-1958, 2003.
[34] Oriol Vinyals, Charles Blundell, Timothy Lillicrap. Daan Wierstra, et al. Matching networks for one shot learning. In NeurIPS, pages 3630-3638, 2016.
[35] Risto Vuorio, Shao-Hua Sun, Hexiang Hu, and Joseph J Lim. Multimodal model-agnostic meta-learning via task-aware modulation. NeuriPS, pages 1-12, 2019.
[36] Zhenqin Wu, Bharath Ramsundar, Evan N Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh Pappu, Karl Leswing, and Vijay Pande. Moleculenet: a benchmark for molecular machine learning. Chemical science, 9()):513-530, 2018.
[37] Huaxiu Yao, Ying Wei, Junzhou Huang, and Zhenhui Li. Hierarchically structured meta-learning. In ICML, pages 7045-7054, 2019.
[38] Huaxiu Yao, Xian Wu, Zhiqiang Tao, Yaliang Li, Bolin Ding, Ruirui Li, and Zhenhui Li. Automated relational meta-learning. In ICLR, 2020.
[39] Sung Whan Yoon, Jun Seo, and Jaekyun Moon. Tapnet: Neural network augmented with task-adaptive projection for few-shot learning. In ICML, pages 7115-7123, 2019.
12

---

[40] Xi Sheryl Zhang, Fengyi Tang. Hiroko H Dodge, Jiayu Zhou, and Fei Wang. Metapred: Meta- learning for clinical risk prediction with limited patient electronic health records. In SIGKDD, pages 2487-2495, 2019.
13


Reviews:
[{"review_id": "x-tb28g83U", "reviewer": "Reviewer_wsge", "summary": "This paper presents a new meta-learning method called FRML. By analyzing functional regions of a base learner, it tries to transfer useful knowledge to a task with insufficient data. Specifically, they present the contrastive learning loss and localization strategy that seems to be the key components of FRML. It is evaluated for two drug discovery tasks, screening and ADMET prediction, which supports their hypothesis well. Several analyses highlight the various aspect of the results. ", "questions": "", "limitations": "", "rating": 7, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: The originality of this work can be found in the framework and its subunits. They proposed a new loss function and new localization strategy, which are the key ingredient of the new framework.\n\n\nQuality: They are motivated by the problem of existing models, and hypothesize functionally regionalized knowledge transfer would be useful in low-resource tasks. Accordingly, they proposed FRML, and they validate the hypothesis in the experimental results with two drug discovery benchmarks. However, it's hard to judge if those two benchmarks are low-resource tasks. Therefore, if non-meta learning SOTA models' performance of each benchmark is provided, their validation would be stronger. Just a quick reference of performance, the best score of Tox21 and Toxcast are 0.871 (AUC) and 0.777(AUC), respectively (I understand those two numbers can't be directly compared to FRML's scores).\n\n\nClarity: This is generally a well-written paper, however, it'd be better if insightful figures are included instead of an explanation.  \n\n\n\nSignificance: This work deals with one of the fundamental tasks in drug discovery, therefore it would benefit many researchers in this field.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "## Suggestions\n* compare with non-meta learning models\n* use insightful figures instead of wordy explanation\n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}, {"review_id": "vimp1zyj4yN", "reviewer": "Reviewer_BVPN", "summary": "This paper seeks to remedy the low-resource drug discovery problem by transferring the knowledge from previous assays, namely in-vivo experiments, by different laboratories and against various target proteins. The authors propose a functional rationalized meta-learning algorithm FRML for such knowledge transfer. The compositionality of the model improves the capacity of generalization to various and even out-of-distribution tasks. Empirical results on both virtual screening and ADMET prediction validate the superiority of FRML over state-of-the-art baselines powered with interpretability in assay relationships.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper aims to tackle the challenge of data insufficiency in drug discovery by transferring the knowledge from historical assays, with a novel meta-learning framework, FRML, to effectively learn the transferable knowledge and meantime adapt to various assays. \n\nPros:\n\n- Overall, the idea of this paper is very valuable for low-resource drug discovery, the originality is good and the authors also present the difference of this work with previous contributions. \n- The quality of this paper is technically sound. Experimental results on virtual screening and ADMET prediction also show the effectiveness of FRML, and the analyses on the localization strategy further verify its sound interpretability in capturing the similarity between assays.\n- The proposed FRML framework is reasonable and interesting for accommodating a wide range of assays effectively by tailoring sub-networks for each assay. This paper provides comprehensive experiments, including Drug Activity Prediction and ADMET Property Prediction, to show the effectiveness of the proposed model. The authors also carried out an ablation study to further show the effectiveness of the proposed modules in FRML.\n- The paper is well-written and the design decisions are clearly explained. The comparison of benchmark methods is also interesting to read. In general, I think this is a worthy publication.\n\n\nCons:\n\n- The choice of the baseline method. Why did the author choose these three categories of baselines? There are some other meta-learning methods, such as Non-parametric methods or Bayesian meta-learning. Why doesn't the author consider comparing these methods?\n- The paper stated the proposed method improves the capacity of generalization to various and even out-of-distribution tasks. What is the out-of-distribution task? Is this a statistically significant distribution? The author needs to provide a more detailed explanation.\n- Some minor typos: \n  - line 373: \"analyses onThe the localization strategy\" -> analyses on the localization strategy\n  - Appendix B line 12: \"Note thatall the base learners\" -> Note that all the base learners\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "As the authors presented in the paper, the limitation of this work is that they have not investigated the robustness of the proposed FRML. If the proposed framework is easy to be attacked, it may cause negative social impacts.", "review_text": "", "extra_scores": {"ethical_concerns": "N/A", "needs_ethics_review": "No", "ethics_review_area": ["I don’t know"], "time_spent_reviewing": "1.0"}, "extra_text": {}}, {"review_id": "YA0L1kYNqlD", "reviewer": "Reviewer_3Ywv", "summary": "In this paper, the authors propose a new meta-learning method for drug discovery. The proposed model transfers knowledge from source drug assays to target drug assays. To better leverage the similarity among drug assays, the proposed model disentangles the base learner to different functional regions and reassembles the selected functional regions for each drug assay. The proposed model is evaluated on virtual screening and ADMET prediction tasks and achieves better performance than other baseline methods.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths: \n\n1) This paper focuses on making accurate virtual screening and ADMET predictions with limited drug compounds, which is an important and practical problem in drug discovery. \n\n2) The motivation of the proposed FTML is clear. The authors discuss the similarity among different drug assays and use functional regions to model similarities and differences across drug assays.   \n\n3) The experiments are comprehensive. The authors evaluate the performance on both regression (virtual screening) and classification (ADMET prediction) datasets and verify the effectiveness of the proposed method. Besides, the qualitative analysis about learned traces further strengthens the proposed model's motivation and its interpretability.\n\nWeaknesses:\n\n1) It would also be interesting to see a discussion about different layers use a different number of functional regions. From my perspective, lower layers may use less functional regions since they capture more general features than higher layers.\n\n2) In the ablation study III, the authors replace RNN localization network with FC. How do you design the FC network? Do you share the weight across different layers?\n\nOverall, the paper is clearly motivated by comprehensive experiments. I vote for accepting.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper seeks to remedy the low-resource drug discovery problem by transferring the knowledge from previous assays, namely in-vivo experiments, by different laboratories and against various target proteins. The authors propose a functional rationalized meta-learning algorithm FRML for such knowledge transfer. The approach appears well motivated and the empirical results appear advance the state-of-the-art.

The reviewers provided detailed reviews providing a long list of pros and a rather short list of cons of the approach/manuscript. They are all coming to the same conclusion that this is a good paper for NeurIPS. I follow the recommendation of the reviewers to accept the manuscript. 

From the scores, it may not be in the spotlight or oral presentation range (7), however, I think it could be bumped up if one aims to increase health-related topics in the oral presentation program.

Decision: Accept (Poster)