Venue: ICLR 2018 conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper proposes Densely Interactive Inference Network to solve recognizing textual entailment via extracting a semantic feature from interaction tensor end-to-end. Their results show that this model has better performance than others.

Even though the results of this paper is interesting, I have the problem with paper writing and motivation for their architecture:

- Paper pages are well beyond 8-page limits for ICLR. The paper should be 8-pages + References. This paper has 11 pages excluding the references.
-  The introduction text in the 2nd page doesn't have smooth flow and sometimes hard to follow.
-  In my view section, 3.1 is redundant and text in section 3.2 can be improved
-  Encoding layer in section 3.2 is really hard to follow in regards to equations and naming e.g p_{itr att} and why choose \alpha(a,b,w)? 
-  Encoding layer in section 3.2, there is no motivation why it needs to use fuse gate.
-  Feature Extraction Layer is very confusing again. What is FSDR or TSDR?
-  Why the paper uses Eq. 8? the intuition behind it?
-  One important thing which is missing in this paper, I didn't understand what is the motivation behind using each of these components? and how each of these components is selected?
- How long does it take to train this network? Since it needs to works with other models (GLOV+ char features + POS tagging,..), it requires lots of effort to set up this network.

Even though the paper outperforms others, it would be useful to the community by providing the motivation and intuition why each of these components was chosen. This is important especially for this paper because each layer of their architecture uses multiple components, i.e. embedding layer [Glov+ Character Features + Syntactical features].  In my view, having just good results are not enough and will not guarantee a publication in ICLR, the paper should be well-written and well-motivated in order to be useful for the future research and the other researchers.
In summary, I don't think the paper is ready yet and it needs significant revision.



---------------------------------------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------------------------------------------------------------------------
Comments after the rebuttal and revision :
I'd like thanks the authors for the revision and their answers. 
Here are my comments after reading the revised version and considering the rebuttal:
- It is fair to say that the paper presentation is much better now. That said I am still having issues with 11 pages.
- The authors imply on page 2, end of paragraph 5,  that this is the first work that shows attention weight contains rich semantic and previous works are used attention merely as a medium for alignment.  Referring to the some of the related works (cited in this paper), I am not sure this is a correct statement.
- The authors claim to introduce a new class of architectures for NLI and generability of for this problem. In my view, this is a very strong statement and unsupported in the paper, especially considering ablation studies (table 5). In order for the model to show the best performance, all these components should come together. I am not sure why this method can be considered a class of architecture and why not just a new model?

some other comments:
- In page 4, the citation is missing for highway networks
- Page 5, equation 1, the parenthesis should close after \hat{P}_j.

Since the new version has been improved, I have increased my review score.  However, I'm still not convinced that this paper would be a good fit at ICLR given novelty and contribution.
review_title: The result of this paper for recognizing textual entailment is very interesting; however, the paper is not well-written and it doesn't provide motivation and intuition for each component of their model and it needs major revision. 

final_score: rating: 6: Marginally above acceptance threshold
review_content: Thank you for this paper! It is very nice piece of work and the problem of coding the "necessary semantic information required for understanding the text" is really a very important one.

Yet, as many papers, it fails to be clear in describing what is its real novelty and the introduction does not help in focussing what is this innovation. 

The key point of the paper seems to demonstrate that the "interaction tensor contains the necessary semantic information required for understanding the text". This is a clear issue as this demostration is given only using 1) ablation studies removing gates and non capabilities; 2) analyzing the behavior of the model in the annotated subpart of the MultiNLI corpus; 3) a visual representation of the alignment produced by the model. Hence, there is not a direct analysis of what's inside the interaction tensors. This is the major limitation of the study. According to this analysis, DIIN seems to be a very good paraphrase detector and word aligner. In fact, Table 6 reports the astonishing 100% in paraphrase detection for the Mismatch examples. It seems also that examples where rules are necessary are not correctly modeled by DIIN: this is shown by the poor result on Conditional and Active Passive. Hence, DIIN seems not to be able to capture rules. 

For a better demostration, there should be a clearer analysis of these "interaction tensors". The issue of the interpretability of what is in these tensors is gaining attention and should be taken into consideration if the main claim of the paper is that: "interaction tensor contains the necessary semantic information required for understanding the text". Some interesting attempts have been made in "Harnessing Deep Neural Networks with Logic Rules", ACL 2016 and in "Can we explain natural language inference decisions taken with neural networks? Inference rules in distributed representations", IJCNN 2017.


Minor issues
======
Capital letters are used in the middle of some sentences, e.g. "On the other hand, A mul",  "powerful capability, We hypothesize"




review_title: Does interaction tensor contains the required information?

final_score: rating: 6: Marginally above acceptance threshold
review_content: Pros: 
The paper proposes a “Densely Interactive Inference Network (DIIN)” for NLI or NLI alike tasks. Although using tensors to capture high-order interaction and performing dimension reduction over that are both not novel, the paper explores them for NLI. The paper is written clearly and is very easy to follow. The ablation experiments in Table 5 give a good level of details to help observe different components' effectiveness.
Cons:
1) The differences of performances between the proposed model and the previous models are not very clear. With regard to MultiNLI, since the previous results (e.g., those in Table 2) did not use cross-sentence attention and had to represent a premise or a hypothesis as a *fixed-length* vector, is it fair to compare DIIN with them? Note that the proposed DIIN model does represent a premise or a hypothesis by variable lengths (see interaction layer in Figure 1), and tensors provide some sorts of attention between them. Can this (Table 2) really shows the advantage of the proposed models? However, when a variable-length representation is allowed (see Table 3 on SNLI), the advantage of the model is also not observed, with no improvement as a single model (compared with ESIM) and being almost same as previous models (e.g., model 18 in Table 3) in ensembling.
2) Method-wise, as discussed above, using tensors to capture high-order interaction and performing dimension reduction over that are both not novel.
3) The paper mentions the use of untied parameters for premise and hypothesis, but it doesn’t compare it with tied version in the experiment section. 
4) In Table 6, for CONDITIONAL tag, why the baseline models (lower total accuracies) have a 100% accuracy, but DIIN only has about a 60% accuracy?

review_title: The paper explores interaction tensors (or attention weights) to capture semantic information to help solve NLI or NLI alike tasks, but contributions are not clear as the claims are not convincingly supported by experiments.

Meta-review: This paper presents a marginally interesting idea -- that of an interaction tensor that compares two sentence representations word by word, and feeds the interaction tensor into a higher level feature extraction mechanism.  It produces good results on multi-NLI and SNLI datasets.  There is some criticism about comparing with several baselines for multi-NLI where there was a restriction of not using inter-sentence comparison networks, but the authors do compare with a similar approach without that restriction and shows improvements.   However, there is no solid error analysis that shows what type of examples this interaction tensor idea captures better than other strong baselines such as ESIM. Overall, the committee feels this paper will add value to the conference.
Decision: Accept (Poster)