pwc-archive/papers-with-abstracts huggingface dataset:
QTUNA: A Corpus for Understanding How Speakers Use Quantification
Authors: Guanyi Chen, Kees Van Deemter, Silvia Pagliaro, Louk Smalbil, Chenghua Lin
Date: 2019-10-01 00:00:00
Proceeding: WS 2019 10
Tasks: Text Generation

A prominent strand of work in formal semantics investigates the ways in which human languages quantify over the elements of a set, as when we say {``}\textit{All $A$ are $B$ }{''}, {``}\textit{All except two $A$ are $B$ }{''}, {``}\textit{Only a few of the $A$ are $B$ }{''} and so on. Our aim is to build Natural Language Generation algorithms that mimic humans{'} use of quantified expressions. To inform these algorithms, we conducted on a series of elicitation experiments in which human speakers were asked to perform a linguistic task that invites the use of quantified expressions. We discuss how these experiments were conducted and what corpora they gave rise to. We conduct an informal analysis of the corpora, and offer an initial assessment of the challenges that these corpora pose for Natural Language Generation. The dataset is available at:{\textasciitilde}\url{https://github.com/a-quei/qtuna}.
Sketch Me if You Can: Towards Generating Detailed Descriptions of Object Shape by Grounding in Images and Drawings
Authors: Ting Han, Sina Zarrie{\ss}
Date: 2019-10-01 00:00:00
Proceeding: WS 2019 10
Tasks: Attribute, Image Captioning, Object

A lot of recent work in Language {\&} Vision has looked at generating descriptions or referring expressions for objects in scenes of real-world images, though focusing mostly on relatively simple language like object names, color and location attributes (e.g., brown chair on the left). This paper presents work on Draw-and-Tell, a dataset of detailed descriptions for common objects in images where annotators have produced fine-grained attribute-centric expressions distinguishing a target object from a range of similar objects. Additionally, the dataset comes with hand-drawn sketches for each object. As Draw-and-Tell is medium-sized and contains a rich vocabulary, it constitutes an interesting challenge for CNN-LSTM architectures used in state-of-the-art image captioning models. We explore whether the additional modality given through sketches can help such a model to learn to accurately ground detailed language referring expressions to object shapes. Our results are encouraging.
An Encoder with non-Sequential Dependency for Neural Data-to-Text Generation
Authors: Feng Nie, Jinpeng Wang, Rong pan, Chin-Yew Lin
Date: 2019-10-01 00:00:00
Proceeding: WS 2019 10
Tasks: Data-to-Text Generation, Text Generation

Data-to-text generation aims to generate descriptions given a structured input data (i.e., a table with multiple records). Existing neural methods for encoding input data can be divided into two categories: a) pooling based encoders which ignore dependencies between input records or b) recurrent encoders which model only sequential dependencies between input records. In our investigation, although the recurrent encoder generally outperforms the pooling based encoder by learning the sequential dependencies, it is sensitive to the order of the input records (i.e., performance decreases when injecting the random shuffling noise over input data). To overcome this problem, we propose to adopt the self-attention mechanism to learn dependencies between arbitrary input records. Experimental results show the proposed method achieves comparable results and remains stable under random shuffling over input data.
Tell Me More: A Dataset of Visual Scene Description Sequences
Authors: Nikolai Ilinykh, Sina Zarrie{\ss}, David Schlangen
Date: 2019-10-01 00:00:00
Proceeding: WS 2019 10
Tasks: Image Description, Sentence

We present a dataset consisting of what we call image description sequences, which are multi-sentence descriptions of the contents of an image. These descriptions were collected in a pseudo-interactive setting, where the describer was told to describe the given image to a listener who needs to identify the image within a set of images, and who successively asks for more information. As we show, this setup produced nicely structured data that, we think, will be useful for learning models capable of planning and realising such description discourses.
A Closer Look at Recent Results of Verb Selection for Data-to-Text NLG
Authors: Guanyi Chen, Jin-Ge Yao
Date: 2019-10-01 00:00:00
Proceeding: WS 2019 10
Tasks: Data-to-Text Generation, Text Generation

Automatic natural language generation systems need to use the contextually-appropriate verbs when describing different kinds of facts or events, which has triggered research interest on verb selection for data-to-text generation. In this paper, we discuss a few limitations of the current task settings and the evaluation metrics. We also provide two simple, efficient, interpretable baseline approaches for statistical selection of trend verbs, which give a strong performance on both previously used evaluation metrics and our new evaluation.
BERT for Question Generation
Authors: Ying-Hong Chan, Yao-Chung Fan
Date: 2019-10-01 00:00:00
Proceeding: WS 2019 10
Tasks: Language Modeling, Language Modelling, Question Answering, Question Generation, Question-Generation, Text Generation

In this study, we investigate the employment of the pre-trained BERT language model to tackle question generation tasks. We introduce two neural architectures built on top of BERT for question generation tasks. The first one is a straightforward BERT employment, which reveals the defects of directly using BERT for text generation. And, the second one remedies the first one by restructuring the BERT employment into a sequential manner for taking information from previous decoded results. Our models are trained and evaluated on the question-answering dataset SQuAD. Experiment results show that our best model yields state-of-the-art performance which advances the BLEU4 score of existing best models from 16.85 to 18.91.
Visually grounded generation of entailments from premises
Authors: Somayeh Jafaritazehjani, Albert Gatt, Marc Tanti
Date: 2019-10-01 00:00:00
Proceeding: WS 2019 10
Tasks: Natural Language Inference, Sentence

Natural Language Inference (NLI) is the task of determining the semantic relationship between a premise and a hypothesis. In this paper, we focus on the generation of hypotheses from premises in a multimodal setting, to generate a sentence (hypothesis) given an image and/or its description (premise) as the input. The main goals of this paper are (a) to investigate whether it is reasonable to frame NLI as a generation task; and (b) to consider the degree to which grounding textual premises in visual information is beneficial to generation. We compare different neural architectures, showing through automatic and human evaluation that entailments can indeed be generated successfully. We also show that multimodal models outperform unimodal models in this task, albeit marginally
Neural Conversation Model Controllable by Given Dialogue Act Based on Adversarial Learning and Label-aware Objective
Authors: Seiya Kawano, Koichiro Yoshino, Satoshi Nakamura
Date: 2019-10-01 00:00:00
Proceeding: WS 2019 10

Building a controllable neural conversation model (NCM) is an important task. In this paper, we focus on controlling the responses of NCMs by using dialogue act labels of responses as conditions. We introduce an adversarial learning framework for the task of generating conditional responses with a new objective to a discriminator, which explicitly distinguishes sentences by using labels. This change strongly encourages the generation of label-conditioned sentences. We compared the proposed method with some existing methods for generating conditional responses. The experimental results show that our proposed method has higher controllability for dialogue acts even though it has higher or comparable naturalness to existing methods.
Low Level Linguistic Controls for Style Transfer and Content Preservation
Authors: Katy Gero, Chris Kedzie, Jonathan Reeve, Lydia Chilton
Date: 2019-10-01 00:00:00
Proceeding: WS 2019 10
Tasks: Decoder, Style Transfer, Text Generation

Despite the success of style transfer in image processing, it has seen limited progress in natural language generation. Part of the problem is that content is not as easily decoupled from style in the text domain. Curiously, in the field of stylometry, content does not figure prominently in practical methods of discriminating stylistic elements, such as authorship and genre. Rather, syntax and function words are the most salient features. Drawing on this work, we model style as a suite of low-level linguistic controls, such as frequency of pronouns, prepositions, and subordinate clause constructions. We train a neural encoder-decoder model to reconstruct reference sentences given only content words and the setting of the controls. We perform style transfer by keeping the content words fixed while adjusting the controls to be indicative of another style. In experiments, we show that the model reliably responds to the linguistic controls and perform both automatic and manual evaluations on style transfer. We find we can fool a style classifier 84{\%} of the time, and that our model produces highly diverse and stylistically distinctive outputs. This work introduces a formal, extendable model of style that can add control to any neural text generation system.
Efficiency Metrics for Data-Driven Models: A Text Summarization Case Study
Authors: Erion {\c{C}}ano, Ond{\v{r}}ej Bojar
Date: 2019-10-01 00:00:00
Proceeding: WS 2019 10
Tasks: Articles, Text Summarization

Using data-driven models for solving text summarization or similar tasks has become very common in the last years. Yet most of the studies report basic accuracy scores only, and nothing is known about the ability of the proposed models to improve when trained on more data. In this paper, we define and propose three data efficiency metrics: data score efficiency, data time deficiency and overall data efficiency. We also propose a simple scheme that uses those metrics and apply it for a more comprehensive evaluation of popular methods on text summarization and title generation tasks. For the latter task, we process and release a huge collection of 35 million abstract-title pairs from scientific articles. Our results reveal that among the tested models, the Transformer is the most efficient on both tasks.
An NLG System for Constituent Correspondence: Personality, Affect, and Alignment
Authors: William Kolkey, Jian Dong, Greg Bybee
Date: 2019-10-01 00:00:00
Proceeding: WS 2019 10
Tasks: Text Generation

Roughly 30{\%} of congressional staffers in the United States report spending a {``}great deal{''} of time writing responses to constituent letters. Letters often solicit an update on the status of legislation and a description of a congressman{'}s vote record or vote intention {---} structurable data that can be leveraged by a natural language generation (NLG) system to create a coherent letter response. This paper describes how PoliScribe, a pipeline-architectured NLG platform, constructs personalized responses to constituents inquiring about legislation. Emphasis will be placed on adapting NLG methodologies to the political domain, which entails special attention to affect, discursive variety, and rhetorical strategies that align a speaker with their interlocutor, even in cases of policy disagreement.
Margin Call: an Accessible Web-based Text Viewer with Generated Paragraph Summaries in the Margin
Authors: Nabah Rizvi, Sebastian Gehrmann, Franck Dernoncourt
Date: 2019-10-01 00:00:00
Proceeding: WS 2019 10
Tasks: Sentence

We present Margin Call, a web-based text viewer that automatically generates short summaries for each paragraph of the text and displays the summaries in the margin of the text next to the corresponding paragraph. On the back-end, the summarizer first identifies the most important sentence for each paragraph in the text file uploaded by the user. The selected sentence is then automatically compressed to produce the short summary. The resulting summary is a few words long. The displayed summaries can help the user understand and retrieve information faster from the text, while increasing the retention of information.
Privacy-Aware Text Rewriting
Authors: Qiongkai Xu, Lizhen Qu, Chenchen Xu, Ran Cui
Date: 2019-10-01 00:00:00
Proceeding: WS 2019 10
Tasks: Fairness, Translation

Biased decisions made by automatic systems have led to growing concerns in research communities. Recent work from the NLP community focuses on building systems that make fair decisions based on text. Instead of relying on unknown decision systems or human decision-makers, we argue that a better way to protect data providers is to remove the trails of sensitive information before publishing the data. In light of this, we propose a new privacy-aware text rewriting task and explore two privacy-aware back-translation methods for the task, based on adversarial training and approximate fairness risk. Our extensive experiments on three real-world datasets with varying demographical attributes show that our methods are effective in obfuscating sensitive attributes. We have also observed that the fairness risk method retains better semantics and fluency, while the adversarial training method tends to leak less sensitive information.
Personalized Substitution Ranking for Lexical Simplification
Authors: John Lee, Chak Yan Yeung
Date: 2019-10-01 00:00:00
Proceeding: WS 2019 10
Tasks: Lexical Simplification

A lexical simplification (LS) system substitutes difficult words in a text with simpler ones to make it easier for the user to understand. In the typical LS pipeline, the Substitution Ranking step determines the best substitution out of a set of candidates. Most current systems do not consider the user{'}s vocabulary proficiency, and always aim for the simplest candidate. This approach may overlook less-simple candidates that the user can understand, and that are semantically closer to the original word. We propose a personalized approach for Substitution Ranking to identify the candidate that is the closest synonym and is non-complex for the user. In experiments on learners of English at different proficiency levels, we show that this approach enhances the semantic faithfulness of the output, at the cost of a relatively small increase in the number of complex words.
Revisiting the Binary Linearization Technique for Surface Realization
Authors: Yevgeniy Puzikov, Claire Gardent, Ido Dagan, Iryna Gurevych
Date: 2019-10-01 00:00:00
Proceeding: WS 2019 10
Tasks: Decision Making

End-to-end neural approaches have achieved state-of-the-art performance in many natural language processing (NLP) tasks. Yet, they often lack transparency of the underlying decision-making process, hindering error analysis and certain model improvements. In this work, we revisit the binary linearization approach to surface realization, which exhibits more interpretable behavior, but was falling short in terms of prediction accuracy. We show how enriching the training data to better capture word order constraints almost doubles the performance of the system. We further demonstrate that encoding both local and global prediction contexts yields another considerable performance boost. With the proposed modifications, the system which ranked low in the latest shared task on multilingual surface realization now achieves best results in five out of ten languages, while being on par with the state-of-the-art approaches in others.
Head-First Linearization with Tree-Structured Representation
Authors: Xiang Yu, Agnieszka Falenska, Ngoc Thang Vu, Jonas Kuhn
Date: 2019-10-01 00:00:00
Proceeding: WS 2019 10
Tasks: Decoder

We present a dependency tree linearization model with two novel components: (1) a tree-structured encoder based on bidirectional Tree-LSTM that propagates information first bottom-up then top-down, which allows each token to access information from the entire tree; and (2) a linguistically motivated head-first decoder that emphasizes the central role of the head and linearizes the subtree by incrementally attaching the dependents on both sides of the head. With the new encoder and decoder, we reach state-of-the-art performance on the Surface Realization Shared Task 2018 dataset, outperforming not only the shared tasks participants, but also previous state-of-the-art systems (Bohnet et al., 2011; Puduppully et al., 2016). Furthermore, we analyze the power of the tree-structured encoder with a probing task and show that it is able to recognize the topological relation between any pair of tokens in a tree.
Generation of Hip-Hop Lyrics with Hierarchical Modeling and Conditional Templates
Authors: Enrique Manjavacas, Mike Kestemont, Folgert Karsdorp
Date: 2019-10-01 00:00:00
Proceeding: WS 2019 10
Tasks: Language Modeling, Language Modelling, Rhythm, Text Generation

This paper addresses Hip-Hop lyric generation with conditional Neural Language Models. We develop a simple yet effective mechanism to extract and apply conditional templates from text snippets, and show{---}on the basis of a large-scale crowd-sourced manual evaluation{---}that these templates significantly improve the quality and realism of the generated snippets. Importantly, the proposed approach enables end-to-end training, targeting formal properties of text such as rhythm and rhyme, which are central characteristics of rap texts. Additionally, we explore how generating text at different scales (e.g.{\textasciitilde}character-level or word-level) affects the quality of the output. We find that a hybrid form{---}a hierarchical model that aims to integrate Language Modeling at both word and character-level scales{---}yields significant improvements in text quality, yet surprisingly, cannot exploit conditional templates to their fullest extent. Our findings highlight that text generation models based on Recurrent Neural Networks (RNN) are sensitive to the modeling scale and call for further research on the observed differences in effectiveness of the conditioning mechanism at different scales.
Controlling Contents in Data-to-Document Generation with Human-Designed Topic Labels
Authors: Kasumi Aoki, Akira Miyazawa, Tatsuya Ishigaki, Tatsuya Aoki, Hiroshi Noji, Keiichi Goshima, Ichiro Kobayashi, Hiroya Takamura, Yusuke Miyao
Date: 2019-10-01 00:00:00
Proceeding: WS 2019 10
Tasks: Language Modeling, Language Modelling, Sentence, Time Series, Time Series Analysis

We propose a data-to-document generator that can easily control the contents of output texts based on a neural language model. Conventional data-to-text model is useful when a reader seeks a global summary of data because it has only to describe an important part that has been extracted beforehand. However, because depending on users, it differs what they are interested in, so it is necessary to develop a method to generate various summaries according to users{'} interests. We develop a model to generate various summaries and to control their contents by providing the explicit targets for a reference to the model as controllable factors. In the experiments, we used five-minute or one-hour charts of 9 indicators (e.g., Nikkei225), as time-series data, and daily summaries of Nikkei Quick News as textual data. We conducted comparative experiments using two pieces of information: human-designed topic labels indicating the contents of a sentence and automatically extracted keywords as the referential information for generation.
Agreement is overrated: A plea for correlation to assess human evaluation reliability
Authors: Jacopo Amidei, Paul Piwek, Alistair Willis
Date: 2019-10-01 00:00:00
Proceeding: WS 2019 10
Tasks: nlg evaluation

Inter-Annotator Agreement (IAA) is used as a means of assessing the quality of NLG evaluation data, in particular, its reliability. According to existing scales of IAA interpretation {--} see, for example, Lommel et al. (2014), Liu et al. (2016), Sedoc et al. (2018) and Amidei et al. (2018a) {--} most data collected for NLG evaluation fail the reliability test. We confirmed this trend by analysing papers published over the last 10 years in NLG-specific conferences (in total 135 papers that included some sort of human evaluation study). Following Sampson and Babarczy (2008), Lommel et al. (2014), Joshi et al. (2016) and Amidei et al. (2018b), such phenomena can be explained in terms of irreducible human language variability. Using three case studies, we show the limits of considering IAA as the only criterion for checking evaluation reliability. Given human language variability, we propose that for human evaluation of NLG, correlation coefficients and agreement coefficients should be used together to obtain a better assessment of the evaluation data reliability. This is illustrated using the three case studies.
Best practices for the human evaluation of automatically generated text
Authors: Chris van der Lee, Albert Gatt, Emiel van Miltenburg, S Wubben, er, Emiel Krahmer
Date: 2019-10-01 00:00:00
Proceeding: WS 2019 10
Tasks: Text Generation

Currently, there is little agreement as to how Natural Language Generation (NLG) systems should be evaluated. While there is some agreement regarding automatic metrics, there is a high degree of variation in the way that human evaluation is carried out. This paper provides an overview of how human evaluation is currently conducted, and presents a set of best practices, grounded in the literature. With this paper, we hope to contribute to the quality and consistency of human evaluations in NLG.
Toward a Better Story End: Collecting Human Evaluation with Reasons
Authors: Yusuke Mori, Hiroaki Yamane, Yusuke Mukuta, Tatsuya Harada
Date: 2019-10-01 00:00:00
Proceeding: WS 2019 10
Tasks: Sentence

Creativity is an essential element of human nature used for many activities, such as telling a story. Based on human creativity, researchers have attempted to teach a computer to generate stories automatically or support this creative process. In this study, we undertake the task of story ending generation. This is a relatively new task, in which the last sentence of a given incomplete story is automatically generated. This is challenging because, in order to predict an appropriate ending, the generation method should comprehend the context of events. Despite the importance of this task, no clear evaluation metric has been established thus far; hence, it has remained an open problem. Therefore, we study the various elements involved in evaluating an automatic method for generating story endings. First, we introduce a baseline hierarchical sequence-to-sequence method for story ending generation. Then, we conduct a pairwise comparison against human-written endings, in which annotators choose the preferable ending. In addition to a quantitative evaluation, we conduct a qualitative evaluation by asking annotators to specify the reason for their choice. From the collected reasons, we discuss what elements the evaluation should focus on, to thereby propose effective metrics for the task.
Hotel Scribe: Generating High Variation Hotel Descriptions
Authors: Saad Mahamood, Maciej Zembrzuski
Date: 2019-10-01 00:00:00
Proceeding: WS 2019 10
Tasks: Text Generation, Vocal Bursts Intensity Prediction

This paper describes the implementation of the Hotel Scribe system. A commercial Natural Language Generation (NLG) system which generates descriptions of hotels from accommodation metadata with a high level of content and linguistic variation in English. It has been deployed live by *Anonymised Company Name* for the purpose of improving coverage of accommodation descriptions and for Search Engine Optimisation (SEO). In this paper, we describe the motivation for building this system, the challenges faced when dealing with limited metadata, and the implementation used to generate the highly variate accommodation descriptions. Additionally, we evaluate the uniqueness of the texts generated by our system against comparable human written accommodation description texts.