Venue: ICLR 2017 conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: SUMMARY.

The paper proposes a gating mechanism to combine word embeddings with character-level word representations.
The gating mechanism uses features associated to a word to decided which word representation is the most useful.
The fine-grain gating is applied as part of systems which seek to solve the task of cloze-style reading comprehension question answering, and Twitter hashtag prediction.
For the question answering task, a fine-grained reformulation of gated attention for combining document words and questions is proposed.
In both tasks the fine-grain gating helps to get better accuracy, outperforming state-of-the-art methods on the CBT dataset and performing on-par with state-of-the-art approach on the SQuAD dataset.


----------

OVERALL JUDGMENT

This paper proposes a clever fine-grained extension of a scalar gate for combining word representation.
It is clear and well written. It covers all the necessary prior work and compares the proposed method with previous similar models.

I liked the ablation study that shows quite clearly the impact of individual contributions.
And I also liked the fact that some (shallow) linguistic prior knowledge e.g., pos tags ner tags, frequency etc. has been used in a clever way. 
It would be interesting to see if syntactic features can be helpful.


review_title: null

final_score: rating: null
review_content: Hi, is there a particular reason for using a mono-directional RNN as character-level encoding? Have you tried with more expressive models i.e., a bidirectional RNN?
review_title: character-level encoding

final_score: rating: null
review_content: Considering that combined representation is a nonlinear function of the features, lookup value and the char-level representations (essentially the flat concatenation), did you try a, say, deep feedforward neural network to implement that function? I think that would be a good baseline to compare against to show that the inductive bias introduced by fine-grained gating is indeed helpful compared to other means of implementing that function.
review_title: Deep baselines

final_score: rating: 6: Marginally above acceptance threshold
review_content: I think the problem here is well motivated, the approach is insightful and intuitive, and the results are convincing of the approach (although lacking in variety of applications). I like the fact that the authors use POS and NER in terms of an intermediate signal for the decision. Also they compare against a sufficient range of baselines to show the effectiveness of the proposed model.

I am also convinced by the authors' answers to my question, I think there is sufficient evidence provided in the results to show the effectiveness of the inductive bias introduced by the fine-grained gating model.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: This paper proposes a new gating mechanism to combine word and character representations. The proposed model sets a new state-of-the-art on the CBT dataset; the new gating mechanism also improves over scalar gates without linguistic features on SQuAD and a twitter classification task. 

Intuitively, the vector-based gate working better than the scalar gate is unsurprising, as it is more similar to LSTM and GRU gates. The real contribution of the paper for me is that using features such as POS tags and NER help learn better gates. The visualization in Figure 3 and examples in Table 4 effectively confirm the utility of these features, very nice! 

In sum, while the proposed gate is nothing technically groundbreaking, the paper presents a very focused contribution that I think will be useful to the NLP community. Thus, I hope it is accepted.
review_title: review

Meta-review: The consensus amongst reviewers is that this paper's proposal for combining character level information with word-level information is sound, well presented, and well evaluated. The main negative sentiment was that the approach was perhaps a little increment, although the conceptual size of the increment was sufficient to warrant publication, and will be relevant to the interests of ICLR attendees working on the intersection of deep learning and NLP. After a cursory reading of the paper, I see no reason to disagree, and recommend acceptance.
Decision: Accept (Poster)