Venue: ICLR 2017 conference

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: This paper addresses automated argumentation mining using pointer network. Although the task and the discussion is interesting, the contribution and the novelty is marginal because this is a single-task application of PN among many potential tasks.
review_title: An Application of PN Network

final_score: rating: null
review_content: Is it the case that every example in your corpora is structured as a tree, or are there examples of other structures as well (such as forests of non-connected trees)?

If it is the case that the data is guaranteed to be tree-structured, why did you only weakly/indirectly enforce this constraint? What tradeoffs do you see between a model like yours and something more directly parsing-inspired that could guarantee a tree-structured output?
review_title: Tree validity constraint

final_score: rating: null
review_content: Among many tasks that PN can handle, how do you characterize the problem, i.e., prediction AC types and links? I think this is an application of PN to some specific problme, but does the paper imply more than that?
review_title: How do you characterize your problem?

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper proposes a model for the task of argumentation mining (labeling the set of relationships between statements expressed as sentence-sized spans in a short text). The model combines a pointer network component that identifies links between statements and a classifier that predicts the roles of these statements. The resulting model works well: It outperforms strong baselines, even on datasets with fewer than 100 training examples.

I don't see any major technical issues with this paper, and the results are strong. I am concerned, though, that the paper doesn't make a substantial novel contribution to representation learning. It focuses on ways to adapt reasonably mature techniques to a novel NLP problem. I think that one of the ACL conferences would be a better fit for this work.

The choice of a pointer network for this problem seems reasonable, though (as noted by other commenters) the paper does not make any substantial comparison with other possible ways of producing trees. The paper does a solid job at breaking down the results quantitatively, but I would appreciate some examples of model output and some qualitative error analysis.

Detail notes: 

- Figure 2 appears to have an error. You report that the decoder produces a distribution over input indices only, but you show an example of the network pointing to an output index in one case.
- I don't think "Wei12" is a name.
review_title: Solid work, fit unclear

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper addresses the problem of argument mining, which consists of finding argument types and predicting the relationships between the arguments. The authors proposed a pointer network structure to recover the argument relations. They also propose modifications on pointer network to perform joint training on both type and link prediction tasks. Overall the model is reasonable, but I am not sure if ICLR is the best venue for this work.

My first concern of the paper is on the novelty of the model. Pointer network has been proposed before. The proposed multi-task learning method is interesting, but the authors only verified it on one task. This makes me feel that maybe the submission is more for a NLP conference rather than ICLR. 

The authors stated that the pointer network is less restrictive compared to some of the existing tree predicting method. However, the datasets seem to only contain single trees or forests, and the stack-based method can be used for forest prediction by adding a virtual root node to each example (as done in the dependency parsing tasks). Therefore, I think the experiments right now cannot reflect the advantages of pointer network models unfortunately. 

My second concern of the paper is on the target task. Given that the authors want to analyze the structures between sentences, is the argumentation mining the best dataset? For example, authors could verify their model by applying it to the other tasks that require tree structures such as dependency parsing. As for NLP applications, I found that the assumption that the boundaries of AC are given is a very strong constraint, and could potentially limit the usefulness of the proposed model. 

Overall, in terms of ML, I also feel that baseline methods the authors compared to are probably strong for the argument mining task, but not necessary strong enough for the general tree/forest prediction tasks (as there are other tree/forest prediction methods). In terms of NLP applications, I think the assumption of having AC boundaries is too restrictive, and maybe ICLR is not the best venture for this submission. 

review_title: Review

final_score: rating: null
review_content: 1.	Could the authors explain the reasons they choose to use the pointer network to predict tree structures? While the proposed model makes sense, there are several other ways of predicting tree structures in the NLP literatures such as using stack-based models or adding dynamic programming layers. Are there any particular advantages of using the proposed structures? 
2.	Do the baseline models (classifiers) use word embedding or brown clustering information? It is not clear to me how much of the benefit is from the pre-trained embeddings and how much is from the proposed structures.

review_title: Clarification Questions

Meta-review: The paper presents an interesting application of pointer networks to the argumentation mining task, and the reviewers found it generally solid. The reviewers generally agree (and I share their concerns) that the contribution on the machine learning side (i.e. the modification to PNs) is not sufficient to warrant publication at ICLR. Moreover, the task is not as standard and extensively studied in NLP; so strong results on the benchmark should not automatically warrant publication at ICLR (e.g., I would probably treat differently standard benchmarks in syntactic parsing of English or machine translation). If judged as an NLP paper, as pointed out by one of reviewers, the lack of qualitative evaluation / error analysis seems also problematic.
Decision: Reject