Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: The paper predicts DNA folding using different machine learning methods. The authors show that LSTM out performs other methods. They attribute this to the memory of sequential DNA and LSTM model structure. They also propose a weighted mean squared error that improves the performance of the proposed model.

The authors compare the LSTM model with other classical approaches showing better performance based on predictive and quality metrics, applied to Hi-C data for drosophila, for predicting TADs.

My major concern is that it is not clear if the improvement is a by-product of LSTM without the proposed new metric. A fair comparison would also consider similar loss function designs for other approaches or at least comparing to a vanilla LSTM model. 

Also the approach lacks illustration of generalizability. The definition of the loss function is also very specific (why 11?) and I wonder if this is generalizable to other Hi-C datasets or predictions based on other epigenetic features beyond ChIP-seq, e.g. ATAC-seq. 

review_title: Official Blind Review #3

final_score: rating: 3: Weak Reject
review_content: The authors utilise DNA spatial structures called Topologically-associative domains (TADs) from Hi-C data (of the Drosophila fly) and epigenetic marks (such as binding factors) from Chiq-seq data, thereby making use of physical proximity, to predict DNA folding patterns. The authors use a bidirectional LSTM RNN further emphasising that memory of the DNA states contributes to chromatin folding structures.

The paper offers for a good read. 

Below are comments for improvement and clarification.

a) There is only one equation in the paper and this is also not given clearly. What is the K being summed over? 

b) wMSE is an old concept and depending on the objective function, the equation can vary. Therefore, change the sentence to read that the authors have a ‘modified' wMSE instead. 

c) Section 3.2, Page 3 last line: what is [5]?

d) Section 4.3: 3rd paragraph, last line. Consider formalising the sentence, it will make it more readable. 

e) Figure 5: Right panel (top and bottom): correct the x-axis label

f) There is no discussion explaining why regression is better or worse than a neural network, in this application setting. 

g) Figure 6 and associated experiment is very interesting and important. The authors should elaborate why, in certain cases, there were huge negative errors in training whereas the test error was positive.
review_title: Official Blind Review #1

final_score: rating: 1: Reject
review_content: The authors compared a bidirectional LSTM to traditional machine learning models for genomic contact prediction. Although the paper presents an interesting application of ML, I vote for rejection since i)  the paper is similar to previously published works and lacks methodological novelty, ii) the description about the data and methods is not written clearly enough, and iii) the evaluation needs to be strengthened by additional baseline models and evaluation metrics.

Major comments
=============
1. As pointed out by Maria Anna Rapsomaniki, the paper is similar to previously published papers, which are not cited.

2. It is unclear which features were used as inputs. How were the features that are described in section 3.1 represented? Are these binary values or the mean of the Hi-C signal over the genomic region? What does 20kb mean? How were genomic segments of different chromosomes treated?

3. The motivation of using a biLSTM in section 4.3 is unclear. What has the choice of a biLSTM to do with the fact that DNA is double stranded? The DNA is not used as input to the model and genomic contacts can be formed between non-adjacent segments. Please compare recurrent models to non-recurrent models such as fully connected or convolutional networks.

4. The authors used the weighted MSE as evaluation metric, which was used as training loss of the biLSTM, but it is unclear if the same loss was used for linear and gradient boosting regression. To understand if the performance differences are due to the loss function or model architecture, the authors should use the same loss function for training all models, and use additional evaluation metrics such as the MSE and R^2 score.

5. The authors used linear regression weights to quantify feature importance (figure 4). It is unclear if the biLSTM assigned the same importance to features. To quantify the importance of features learned by the biLSTM, the authors could consider correlating the activations of LSTM units with input features, and also analyze if importances depend on the position of features.

6. Which hyper-parameters of the biLSTM and baseline method did the authors tune and how?

7. The authors should compare the described weighted MSE to the mean squared logarithmic error loss, which is commonly used for penalizing large errors less.

8. Why did the authors use the activation of the center LSTM hidden state instead of concatenating the last hidden state of the forward and reverse LSTM? By using the center hidden state, half of the forward and reverse LSTM activations are ignored.
review_title: Official Blind Review #2

Meta-review: The authors consider the problem of predicting DNA folding patterns.                                                               
They use a range of simple, linear models and find that a bi-LSTM architecture                                                     
yielded best performance.                                                                                                          
                                                                                                                                   
This paper is below acceptance.                                                                                                    
Reviewers pointed out strong similarity to previously published work.                                                              
Furthermore the manuscript lacked in clarity, leaving uncertain eg details about                                                   
experimental details. 
Decision: Reject