Task: Language Identification
Input (Spanish, English): The paper presents two approaches for generating English poetry. The first approach combine a neural phonetic encoder predicting the next phoneme with a phonetic-orthographic HMM decoder computing the most likely word corresponding to a sequence of phonemes. The second approach combines a character language model with a weigthed FST to impose rythm constraints on the output of the language model. For the second approach, the authors also present a heuristic approach which permit constraining the generated poem according to theme (e.g;, love) or poetic devices (e.g., alliteration). The generated poems are evaluated both instrinsically by comparing the rythm of the generated lines with a gold standard and extrinsically by asking 70 human evaluators to (i) determine whether the poem was written by a human or a machine and (ii) rate poems wrt to readability, form and evocation.  The results indicate that the second model performs best and that human evaluators find it difficult to distinguish between human written and machine generated poems.  This is an interesting, clearly written article with novel ideas (two different models for poetry generation, one based on a phonetic language model the other on a character LM) and convincing results.   For the evaluation, more precision about the evaluators and the protocol would be good. Did all evaluators evaluate all poems and if not how many judgments were collected for each poem for each task ? You mention 9 non English native speakers. Poems are notoriously hard to read. How fluent were these ?   In the second model (character based), perhaps I missed it, but do you have a mechanism to avoid generating non words ? If not, how frequent are non words in the generated poems ?  In the first model, why use an HMM to transliterate from phonetic to an orhographic representation rather than a CRF?   Since overall, you rule out the first model as a good generic model for generating poetry, it might have been more interesting to spend less space on that model and more on the evaluation of the second model. In particular, I would have been interested in a more detailed discussion of the impact of the heuristic you use to constrain theme or poetic devices. How do these impact evaluation results ? Could they be combined to jointly constrain theme and poetic devices ?   The combination of a neural mode with a WFST is reminiscent of the following paper which combine character based neural model to generate from dialog acts with an WFST to avoid generating non words. YOu should relate your work to theirs and cite them.   Natural Language Generation through Character-Based RNNs with Finite-State Prior Knowledge Goyal, Raghav and Dymetman, Marc and Gaussier, Eric and LIG, Uni COLING 2016
Output (English): en