Task: Language Identification
Input (Spanish, English): - Strengths:  This paper presents a step in the direction of developing more challenging corpora for training sentence planners in data-to-text NLG, which is an important and timely direction.   - Weaknesses:  It is unclear whether the work reported in this paper represents a substantial advance over Perez-Beltrachini et al.'s (2016) method for selecting content.  The authors do not directly compare the present paper to that one. It appears that the main novelty of this paper is the additional analysis, which is however rather superficial.  It is good that the authors report a comparison of how an NNLG baseline fares on this corpus in comparison to that of Wen et al. (2016).  However, the BLEU scores in Wen et al.'s paper appear to be much much higher, suggesting that this NNLG baseline is not sufficient for an informative comparison.  - General Discussion:  The authors need to more clearly articulate why this paper should count as a substantial advance over what has been published already by Perez-Beltrachini et al, and why the NNLG baseline should be taken seriously.  In contrast to LREC, it is not so common for ACL to publish a main session paper on a corpus development methodology in the absence of some new results of a system making use of the corpus.  The paper would also be stronger if it included an analysis of the syntactic constructions in the two corpora, thereby more directly bolstering the case that the new corpus is more complex.  The exact details of how the number of different path shapes are determined should also be included, and ideally associated with the syntactic constructions.  Finally, the authors should note the limitation that their method does nothing to include richer discourse relations such as Contrast, Consequence, Background, etc., which have long been central to NLG. In this respect, the corpora described by Walker et al. JAIR-2007 and Isard LREC-2016 are more interesting and should be discussed in comparison to the method here.  References  Marilyn Walker, Amanda Stent, François Mairesse, and Rashmi Prasad. 2007. Individual and domain adaptation in sentence planning for dialogue. Journal of Artificial Intelligence Research (JAIR), 30:413–456.  Amy Isard, 2016. “The Methodius Corpus of Rhetorical Discourse Structures and Generated Texts” , Proceedings of the Tenth Conference on Language Resources and Evaluation (LREC 2016), Portorož, Slovenia, May 2016.  --- Addendum following author response:  Thank you for the informative response.  As the response offers crucial clarifications, I have raised my overall rating.  Re the comparison to Perez-Beltrachini et al.: While this is perhaps more important to the PC than to the eventual readers of the paper, it still seems to this reviewer that the advance over this paper could've been made much clearer.  While it is true that Perez-Beltrachini et al. "just" cover content selection, this is the key to how this dataset differs from that of Wen et al.  There doesn't really seem to be much to the "complete methodology" of constructing the data-to-text dataset beyond obvious crowd-sourcing steps; to the extent these steps are innovative or especially crucial, this should be highlighted.  Here it is interesting that 8.7% of the crowd-sourced texts were rejected during the verification step; related to Reviewer 1's concerns, it would be interesting to see some examples of what was rejected, and to what extent this indicates higher-quality texts than those in Wen et al.'s dataset.  Beyond that, the main point is really that collecting the crowd-sourced texts makes it possible to make the comparisons with the Wen et al. corpus at both the data and text levels (which this reviewer can see is crucial to the whole picture).  Re the NNLG baseline, the issue is that the relative difference between the performance of this baseline on the two corpora could disappear if Wen et al.'s substantially higher-scoring method were employed.  The assumption that this relative difference would remain even with fancier methods should be made explicit, e.g. by acknowledging the issue in a footnote.  Even with this limitation, the comparison does still strike this reviewer as a useful component of the overall comparison between the datasets.  Re whether a paper about dataset creation should be able to get into ACL without system results:  though this indeed not unprecedented, the key issue is perhaps how novel and important the dataset is likely to be, and here this reviewer acknowledges the importance of the dataset in comparison to existing ones (even if the key advance is in the already published content selection work).  Finally, this reviewer concurs with Reviewer 1 about the need to clarify the role of domain dependence and what it means to be "wide coverage" in the final version of the paper, if accepted.
Output (English): en