Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: The authors present a model with state-of-the-art performance for predicting protein-ligand affinity and provide a thorough set of benchmarks to illustrate the superiority of combining learned low-dimensional embedding representations  of both ligands and proteins. The authors then show that these learned representations are more powerful than handcrafted features such as circular fingerprints, etc. when combined into a model that jointly takes as input both the ligand and protein.

I originally suggested to accept the paper, but agree with the other reviewers that the novelty of the work in this paper likely doesn't meet the bar for acceptance given that the most significant contributions of this paper are around combining good ideas from other papers without much additional novelty.

Major comments

Unless I'm misunderstanding the naming conventions used in Table 1, it seems like an omission not to include the performance of DeepPCM+HP+HC in Table 1 to more fully decouple the performance due to the DeepPCM architecture versus the contribution due to using both unsupervised descriptors in combination.

I might expect the performance of the DeepPCM + HP + HC model (not currently shown unless I'm missing it) to exceed the performance of the NIB + HP + HC model based upon the reasoning given in the discussion (i.e., usefulness of joint input training), even if the individual representations were inferior to the unsupervised counterparts.

Since the primary point of the paper is to illustrate the power of combining the unsupervised representations, I'm surprised the aforementioned performance comparison is not prominent. That is, the performance comparison of DeepPCM+HP+HC versus DeepPCM+UP+UC seems like a very central quantity to present and discuss, but appears to be missing currently.


Minor comments

Table 1 was a bit confusing to me at first, because it appears that UD = UP + UC, and HD = HP + HC, but this wasn't obvious to me initially; I'd change the NIB rows to be NIB + HP + HC and NIB + UP + UC, and then you don't even need to define the UD and HD acronyms, which simplifies the table's cognitive load for me and also makes the relationship between the DeepPCM variants more obvious.

> "On the low-coverage-split, we also find that our method significantly outperforms the benchmark."

It would be good to add the %improvement inline here to parallel the other dataset splits listed just before so that you can directly compare the magnitudes.

> "using unsupervised-learned descriptors than when using handcrafted descriptors"

Would be more compelling if you added the DeepPCM + HC + HP performance to Table 1.

> "All hyperparameter optimization of our model was performed on the temporal split"

I think it would improve the paper if the extent to which various architecture choices were optimized over could be included as well; for example, which types, layer sizes, and depths of network architectures were considered in the hyperparam tuning, and any accompanying justification for these design choices.

review_title: Official Blind Review #4

final_score: rating: 3: Weak Reject
review_content: This paper tries to solve the protein-legend binding prediction problem in the computational biology field. It uses the learned embedding for protein and legend, separately, from two published papers. Then those two embeddings were inputted to another deep learning model, performing the final prediction. Tested on one dataset, it shows the proposed method can outperform the other baseline methods.

The paper should be rejected for the following reasons:
1. The idea of the paper is not interesting and novel enough. It only used the results from two published papers and then applied another deep learning model on them. The novelty of the paper is limited.
2. The experiment part is not that comprehensive. It indeed performs enough ablation studies. However, all the legendary methods in the computational biology field are not included in the comparison. 
3. The experiments are only performed on one dataset. Usually, for application papers, the experiments should be performed against at least 2 datasets to avoid bias.
4. The discussion part is not well-developed. For the paper which focuses on only one task, the more in-depth discussion is expected beyond the simple discussion of the performance. For example, the authors may try to explain the result: why the model used embedding from unsupervised learning is better than the hand-crafted features. Since they shared the same model, the unsupervised embedding should contain more information. Then, what is the additional information? 


Some further questions and comments:
1. What's the sequence similarity of the 1226 proteins?
2. Can the model generalize well to a completely new protein? 
3. What's the detailed performance of the model on proteins belonging to different families? 
4. I guess if the authors check the detailed performance, they will find nonuniform performance across different proteins. I think the authors can also further investigate that.
5. Can the authors train the embedding model as well as the classification model in an end-to-end fashion? This can be more interesting.
6. One big problem of the assay data is that it would not be able to provide the structure information of the interaction between the protein and the legend. Usually, it is a very important piece of information for the biology people. The computational methods based on the assay data will also inherit the flaw. 
review_title: Official Blind Review #2

final_score: rating: 1: Reject
review_content: This paper proposes to learn representations of protein and molecules for the prediction of protein-ligand binding prediction.

The presentation of this paper is a bit lengthy and repetitive in some cases. The long descriptions of protein/drug descriptors are a nice overview,  but it may be unnecessary as the authors in the end use other works’ embedding.

The author points out that there are interpretability issues & the inability to capture the shape of the substructures with previous ligand descriptors, however, it seems that CDDD also is not interpretable and could not capture the shape as it operates on SMILES strings, although seems to have better predictive performance.

For the protein descriptor, the author is missing several important descriptors that may not have the issues mentioned such as Protein Sequence Composition descriptor. 

The technical novelty is very limited. It seems the usage of CDDD and UniRep are its only difference from previous works such as DeepDTA, WideDTA, DeepConv-DTI, PADME and CDDD and UniRep are also from other works. It may be more suitable for a domain journal instead of ICLR which focuses on method innovation. 


The experimental setup is solid with realistic considerations. However, it is missing many baselines such as DeepDTA, WideDTA, DeepConv-DTI, PADME and more classic methods such as SimBoost and KronRLS. 
review_title: Official Blind Review #1

Meta-review: This paper uses unsupervised learning to create useful representations to improve the performance of models in predicting protein-ligand binding. After reviewers had time to consider each other's comments, there was consensus that the current work is too lacking in novelty on the modeling side to warrant publication in ICLR. Additionally, current experiments are lacking comparisons with important baselines. The work in its current form may be better suited for a domain journal. 
Decision: Reject