Task: Language Identification
Input (Spanish, English): - Strengths:  The paper is well-written and easy to understand. The methods and results are interesting.  - Weaknesses:  The evaluation and the obtained results might be problematic (see my comments below).  - General Discussion:  This paper proposes a system for end-to-end argumentation mining using neural networks. The authors model the problem using two approaches: (1) sequence labeling (2) dependency parsing. The paper also includes the results of experimenting with a multitask learning setting for the sequence labeling approach. The paper clearly explains the motivation behind the proposed model. Existing methods are based on ILP, manual feature engineering and manual design of ILP constraints. However, the proposed model avoids such manual effort. Moreover, the model jointly learns the subtasks in argumentation mining and therefore, avoids the error back propagation problem in pipeline methods. Except a few missing details (mentioned below), the methods are explained clearly.  The experiments are substantial, the comparisons are performed properly, and the results are interesting. My main concern about this paper is the small size of the dataset and the large capacity of the used (Bi)LSTM-based recurrent neural networks (BLC and BLCC). The dataset includes only around 320 essays for training and 80 essays for testing. The size of the development set, however, is not mentioned in the paper (and also the supplementary materials). This is worrying because very few number of essays are left for training, which is a crucial problem. The total number of tags in the training data is probably only a few thousand. Compare it to the standard sequence labeling tasks, where hundreds of thousands (sometimes millions) of tags are available. For this reason, I am not sure if the model parameters are trained properly. The paper also does not analyze the overfitting problem. It would be interesting to see the training and development "loss" values during training (after each parameter update or after each epoch). The authors have also provided some information that can be seen as the evidence for overfitting: Line 622 "Our explanation is that taggers are simpler local models, and thus need less training data and are less prone to overfitting".  For the same reason, I am not sure if the models are stable enough. Mean and standard deviation of multiple runs (different initializations of parameters) need to be included. Statistical significance tests would also provide more information about the stability of the models and the reliability of results. Without these tests, it is hard to say if the better results are because of the superiority of the proposed method or chance.  I understand that the neural networks used for modeling the tasks use their regularization techniques. However, since the size of the dataset is too small, the authors need to pay more attention to the regularization methods. The paper does not mention regularization at all and the supplementary material only mentions briefly about the regularization in LSTM-ER. This problem needs to be addressed properly in the paper.  Instead of the current hyper-parameter optimization method (described in supplementary materials) consider using Bayesian optimization methods.  Also move the information about pre-trained word embeddings and the error analysis from the supplementary material to the paper. The extra one page should be enough for this.  Please include some inter-annotator agreement scores. The paper describing the dataset has some relevant information. This information would provide some insight about the performance of the systems and the available room for improvement.  Please consider illustrating figure 1 with different colors to make the quality better for black and white prints.  Edit:  Thanks for answering my questions. I have increased the recommendation score to 4. Please do include the F1-score ranges in your paper and also report mean and variance of different settings. I am still concerned about the model stability. For example, the large variance of Kiperwasser setting needs to be analyzed properly. Even the F1 changes in the range [0.56, 0.61] is relatively large. Including these score ranges in your paper helps replicating your work.
Output (English): en