Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: In this paper, the authors propose a new meta-learning method for drug discovery. The proposed model transfers knowledge from source drug assays to target drug assays. To better leverage the similarity among drug assays, the proposed model disentangles the base learner to different functional regions and reassembles the selected functional regions for each drug assay. The proposed model is evaluated on virtual screening and ADMET prediction tasks and achieves better performance than other baseline methods.
main_review: Strengths: 

1) This paper focuses on making accurate virtual screening and ADMET predictions with limited drug compounds, which is an important and practical problem in drug discovery. 

2) The motivation of the proposed FTML is clear. The authors discuss the similarity among different drug assays and use functional regions to model similarities and differences across drug assays.   

3) The experiments are comprehensive. The authors evaluate the performance on both regression (virtual screening) and classification (ADMET prediction) datasets and verify the effectiveness of the proposed method. Besides, the qualitative analysis about learned traces further strengthens the proposed model's motivation and its interpretability.

Weaknesses:

1) It would also be interesting to see a discussion about different layers use a different number of functional regions. From my perspective, lower layers may use less functional regions since they capture more general features than higher layers.

2) In the ablation study III, the authors replace RNN localization network with FC. How do you design the FC network? Do you share the weight across different layers?

Overall, the paper is clearly motivated by comprehensive experiments. I vote for accepting.
limitations_and_societal_impact: Yes
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper presents a new meta-learning method called FRML. By analyzing functional regions of a base learner, it tries to transfer useful knowledge to a task with insufficient data. Specifically, they present the contrastive learning loss and localization strategy that seems to be the key components of FRML. It is evaluated for two drug discovery tasks, screening and ADMET prediction, which supports their hypothesis well. Several analyses highlight the various aspect of the results. 
main_review: Originality: The originality of this work can be found in the framework and its subunits. They proposed a new loss function and new localization strategy, which are the key ingredient of the new framework.


Quality: They are motivated by the problem of existing models, and hypothesize functionally regionalized knowledge transfer would be useful in low-resource tasks. Accordingly, they proposed FRML, and they validate the hypothesis in the experimental results with two drug discovery benchmarks. However, it's hard to judge if those two benchmarks are low-resource tasks. Therefore, if non-meta learning SOTA models' performance of each benchmark is provided, their validation would be stronger. Just a quick reference of performance, the best score of Tox21 and Toxcast are 0.871 (AUC) and 0.777(AUC), respectively (I understand those two numbers can't be directly compared to FRML's scores).


Clarity: This is generally a well-written paper, however, it'd be better if insightful figures are included instead of an explanation.  



Significance: This work deals with one of the fundamental tasks in drug discovery, therefore it would benefit many researchers in this field.
limitations_and_societal_impact: ## Suggestions
* compare with non-meta learning models
* use insightful figures instead of wordy explanation
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper seeks to remedy the low-resource drug discovery problem by transferring the knowledge from previous assays, namely in-vivo experiments, by different laboratories and against various target proteins. The authors propose a functional rationalized meta-learning algorithm FRML for such knowledge transfer. The compositionality of the model improves the capacity of generalization to various and even out-of-distribution tasks. Empirical results on both virtual screening and ADMET prediction validate the superiority of FRML over state-of-the-art baselines powered with interpretability in assay relationships.
main_review: This paper aims to tackle the challenge of data insufficiency in drug discovery by transferring the knowledge from historical assays, with a novel meta-learning framework, FRML, to effectively learn the transferable knowledge and meantime adapt to various assays. 

Pros:

- Overall, the idea of this paper is very valuable for low-resource drug discovery, the originality is good and the authors also present the difference of this work with previous contributions. 
- The quality of this paper is technically sound. Experimental results on virtual screening and ADMET prediction also show the effectiveness of FRML, and the analyses on the localization strategy further verify its sound interpretability in capturing the similarity between assays.
- The proposed FRML framework is reasonable and interesting for accommodating a wide range of assays effectively by tailoring sub-networks for each assay. This paper provides comprehensive experiments, including Drug Activity Prediction and ADMET Property Prediction, to show the effectiveness of the proposed model. The authors also carried out an ablation study to further show the effectiveness of the proposed modules in FRML.
- The paper is well-written and the design decisions are clearly explained. The comparison of benchmark methods is also interesting to read. In general, I think this is a worthy publication.


Cons:

- The choice of the baseline method. Why did the author choose these three categories of baselines? There are some other meta-learning methods, such as Non-parametric methods or Bayesian meta-learning. Why doesn't the author consider comparing these methods?
- The paper stated the proposed method improves the capacity of generalization to various and even out-of-distribution tasks. What is the out-of-distribution task? Is this a statistically significant distribution? The author needs to provide a more detailed explanation.
- Some minor typos: 
  - line 373: "analyses onThe the localization strategy" -> analyses on the localization strategy
  - Appendix B line 12: "Note thatall the base learners" -> Note that all the base learners

limitations_and_societal_impact: As the authors presented in the paper, the limitation of this work is that they have not investigated the robustness of the proposed FRML. If the proposed framework is easy to be attacked, it may cause negative social impacts.
review_title: null

Meta-review: This paper seeks to remedy the low-resource drug discovery problem by transferring the knowledge from previous assays, namely in-vivo experiments, by different laboratories and against various target proteins. The authors propose a functional rationalized meta-learning algorithm FRML for such knowledge transfer. The approach appears well motivated and the empirical results appear advance the state-of-the-art.

The reviewers provided detailed reviews providing a long list of pros and a rather short list of cons of the approach/manuscript. They are all coming to the same conclusion that this is a good paper for NeurIPS. I follow the recommendation of the reviewers to accept the manuscript. 

From the scores, it may not be in the spotlight or oral presentation range (7), however, I think it could be bumped up if one aims to increase health-related topics in the oral presentation program.

Decision: Accept (Poster)