pwc-archive/papers-with-abstracts huggingface dataset:
SimpleNLG-ZH: a Linguistic Realisation Engine for Mandarin
Authors: Guanyi Chen, Kees Van Deemter, Chenghua Lin
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Morphological Inflection, Text Generation

We introduce SimpleNLG-ZH, a realisation engine for Mandarin that follows the software design paradigm of SimpleNLG (Gatt and Reiter, 2009). We explain the core grammar (morphology and syntax) and the lexicon of SimpleNLG-ZH, which is very different from English and other languages for which SimpleNLG engines have been built. The system was evaluated by regenerating expressions from a body of test sentences and a corpus of human-authored expressions. Human evaluation was conducted to estimate the quality of regenerated sentences.
Adapting SimpleNLG to Galician language
Authors: Andrea Cascallar-Fuentes, Alej Ramos-Soto, ro, Alberto Bugar{\'\i}n Diz
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Text Generation

In this paper, we describe SimpleNLG-GL, an adaptation of the linguistic realisation SimpleNLG library for the Galician language. This implementation is derived from SimpleNLG-ES, the English-Spanish version of this library. It has been tested using a battery of examples which covers the most common rules for Galician.
Going Dutch: Creating SimpleNLG-NL
Authors: Ruud de Jong, Mari{\"e}t Theune
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Text Generation

This paper presents SimpleNLG-NL, an adaptation of the SimpleNLG surface realisation engine for the Dutch language. It describes a novel method for determining and testing the grammatical constructions to be implemented, using target sentences sampled from a treebank.
Learning to Flip the Bias of News Headlines
Authors: Wei-Fan Chen, Henning Wachsmuth, Khalid Al-Khatib, Benno Stein
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Articles, Text Generation

This paper introduces the task of {``}flipping{''} the bias of news articles: Given an article with a political bias (left or right), generate an article with the same topic but opposite bias. To study this task, we create a corpus with bias-labeled articles from \textit{all-sides.com}. As a first step, we analyze the corpus and discuss intrinsic characteristics of bias. They point to the main challenges of bias flipping, which in turn lead to a specific setting in the generation process. The paper in hand narrows down the general bias flipping task to focus on bias flipping for news article \textit{headlines}. A manual annotation of headlines from each side reveals that they are self-informative in general and often convey bias. We apply an autoencoder incorporating information from an article{'}s content to learn how to automatically flip the bias. From 200 generated headlines, 73 are classified as understandable by annotators, and 83 maintain the topic while having opposite bias. Insights from our analysis shed light on how to solve the main challenges of bias flipping.
Stylistically User-Specific Generation
Authors: Abdurrisyad Fikri, Hiroya Takamura, Manabu Okumura
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Machine Translation, Response Generation, Text Generation

Recent neural models for response generation show good results in terms of general responses. In real conversations, however, depending on the speaker/responder, similar utterances should require different responses. In this study, we attempt to consider individual user{'}s information in adjusting the notable sequence-to-sequence (seq2seq) model for more diverse, user-specific responses. We assume that we need user-specific features to adjust the response and we argue that some selected representative words from the users are suitable for this task. Furthermore, we prove that even for unseen or unknown users, our model can provide more diverse and interesting responses, while maintaining correlation with input utterances. Experimental results with human evaluation show that our model can generate more interesting responses than the popular seq2seqmodel and achieve higher relevance with input utterances than our baseline.
Explainable Autonomy: A Study of Explanation Styles for Building Clear Mental Models
Authors: Francisco Javier Chiyah Garcia, David A. Robb, Xingkun Liu, Atanas Laskov, Pedro Patron, Helen Hastie
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Text Generation

As unmanned vehicles become more autonomous, it is important to maintain a high level of transparency regarding their behaviour and how they operate. This is particularly important in remote locations where they cannot be directly observed. Here, we describe a method for generating explanations in natural language of autonomous system behaviour and reasoning. Our method involves deriving an interpretable model of autonomy through having an expert {`}speak aloud{'} and providing various levels of detail based on this model. Through an online evaluation study with operators, we show it is best to generate explanations with multiple possible reasons but tersely worded. This work has implications for designing interfaces for autonomy as well as for explainable AI and operator training.
Treat the system like a human student: Automatic naturalness evaluation of generated text without reference texts
Authors: Isabel Groves, Ye Tian, Ioannis Douratsos
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Image Captioning, Machine Translation, nlg evaluation, Sentence, Text Generation, Text Summarization

The current most popular method for automatic Natural Language Generation (NLG) evaluation is comparing generated text with human-written reference sentences using a metrics system, which has drawbacks around reliability and scalability. We draw inspiration from second language (L2) assessment and extract a set of linguistic features to predict human judgments of sentence naturalness. Our experiment using a small dataset showed that the feature-based approach yields promising results, with the added potential of providing interpretability into the source of the problems.
Content Aware Source Code Change Description Generation
Authors: Pablo Loyola, Edison Marrese-Taylor, Jorge Balazs, Yutaka Matsuo, Fumiko Satoh
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Machine Translation, Text Generation

We propose to study the generation of descriptions from source code changes by integrating the messages included on code commits and the intra-code documentation inside the source in the form of docstrings. Our hypothesis is that although both types of descriptions are not directly aligned in semantic terms {---}one explaining a change and the other the actual functionality of the code being modified{---} there could be certain common ground that is useful for the generation. To this end, we propose an architecture that uses the source code-docstring relationship to guide the description generation. We discuss the results of the approach comparing against a baseline based on a sequence-to-sequence model, using standard automatic natural language generation metrics as well as with a human study, thus offering a comprehensive view of the feasibility of the approach.
Generating Market Comments Referring to External Resources
Authors: Tatsuya Aoki, Akira Miyazawa, Tatsuya Ishigaki, Keiichi Goshima, Kasumi Aoki, Ichiro Kobayashi, Hiroya Takamura, Yusuke Miyao
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Text Generation

Comments on a stock market often include the reason or cause of changes in stock prices, such as {``}Nikkei turns lower as yen{'}s rise hits exporters.{''} Generating such informative sentences requires capturing the relationship between different resources, including a target stock price. In this paper, we propose a model for automatically generating such informative market comments that refer to external resources. We evaluated our model through an automatic metric in terms of BLEU and human evaluation done by an expert in finance. The results show that our model outperforms the existing model both in BLEU scores and human judgment.
SpatialVOC2K: A Multilingual Dataset of Images with Annotations and Features for Spatial Relations between Objects
Authors: Anja Belz, Adrian Muscat, Pierre Anguill, Mouhamadou Sow, Ga{\'e}tan Vincent, Yassine Zinessabah
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Image to text, Object, Text Generation

We present SpatialVOC2K, the first multilingual image dataset with spatial relation annotations and object features for image-to-text generation, built using 2,026 images from the PASCAL VOC2008 dataset. The dataset incorporates (i) the labelled object bounding boxes from VOC2008, (ii) geometrical, language and depth features for each object, and (iii) for each pair of objects in both orders, (a) the single best preposition and (b) the set of possible prepositions in the given language that describe the spatial relationship between the two objects. Compared to previous versions of the dataset, we have roughly doubled the size for French, and completely reannotated as well as increased the size of the English portion, providing single best prepositions for English for the first time. Furthermore, we have added explicit 3D depth features for objects. We are releasing our dataset for free reuse, along with evaluation tools to enable comparative evaluation.
Adding the Third Dimension to Spatial Relation Detection in 2D Images
Authors: Br Birmingham, on, Adrian Muscat, Anja Belz
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Image Description, Object, Relation, Text Generation

Detection of spatial relations between objects in images is currently a popular subject in image description research. A range of different language and geometric object features have been used in this context, but methods have not so far used explicit information about the third dimension (depth), except when manually added to annotations. The lack of such information hampers detection of spatial relations that are inherently 3D. In this paper, we use a fully automatic method for creating a depth map of an image and derive several different object-level depth features from it which we add to an existing feature set to test the effect on spatial relation detection. We show that performance increases are obtained from adding depth features in all scenarios tested.
Automatic Opinion Question Generation
Authors: Yllias Chali, Tina Baghaee
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Community Question Answering, Question Answering, Question Generation, Question-Generation, Reading Comprehension, Text Generation

We study the problem of opinion question generation from sentences with the help of community-based question answering systems. For this purpose, we use a sequence to sequence attentional model, and we adopt coverage mechanism to prevent sentences from repeating themselves. Experimental results on the Amazon question/answer dataset show an improvement in automatic evaluation metrics as well as human evaluations from the state-of-the-art question generation systems.
Modelling Pro-drop with the Rational Speech Acts Model
Authors: Guanyi Chen, Kees Van Deemter, Chenghua Lin
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Coreference Resolution, Machine Translation, Text Generation

We extend the classic Referring Expressions Generation task by considering zero pronouns in {``}pro-drop{''} languages such as Chinese, modelling their use by means of the Bayesian Rational Speech Acts model (Frank and Goodman, 2012). By assuming that highly salient referents are most likely to be referred to by zero pronouns (i.e., pro-drop is more likely for salient referents than the less salient ones), the model offers an attractive explanation of a phenomenon not previously addressed probabilistically.
Self-Learning Architecture for Natural Language Generation
Authors: Hyungtak Choi, Siddarth K.M., Haehun Yang, Heesik Jeon, Inchul Hwang, Jihie Kim
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Self-Learning, Text Generation

In this paper, we propose a self-learning architecture for generating natural language templates for conversational assistants. Generating templates to cover all the combinations of slots in an intent is time consuming and labor-intensive. We examine three different models based on our proposed architecture - Rule-based model, Sequence-to-Sequence (Seq2Seq) model and Semantically Conditioned LSTM (SC-LSTM) model for the IoT domain - to reduce the human labor required for template generation. We demonstrate the feasibility of template generation for the IoT domain using our self-learning architecture. In both automatic and human evaluation, the self-learning architecture outperforms previous works trained with a fully human-labeled dataset. This is promising for commercial conversational assistant solutions.
Enriching the WebNLG corpus
Authors: Thiago Castro Ferreira, Diego Moussallem, Emiel Krahmer, S Wubben, er
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Machine Translation, Referring Expression, Referring expression generation, Text Generation, Translation

This paper describes the enrichment of WebNLG corpus (Gardent et al., 2017a,b), with the aim to further extend its usefulness as a resource for evaluating common NLG tasks, including Discourse Ordering, Lexicalization and Referring Expression Generation. We also produce a silver-standard German translation of the corpus to enable the exploitation of NLG approaches to other languages than English. The enriched corpus is publicly available.
Towards making NLG a voice for interpretable Machine Learning
Authors: James Forrest, Somayajulu Sripada, Wei Pang, George Coghill
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: BIG-bench Machine Learning, Interpretable Machine Learning, Text Generation

This paper presents a study to understand the issues related to using NLG to humanise explanations from a popular interpretable machine learning framework called LIME. Our study shows that self-reported rating of NLG explanation was higher than that for a non-NLG explanation. However, when tested for comprehension, the results were not as clear-cut showing the need for performing more studies to uncover the factors responsible for high-quality NLG explanations.
Template-based multilingual football reports generation using Wikidata as a knowledge base
Authors: Lorenzo Gatti, Chris van der Lee, Mari{\"e}t Theune
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Machine Translation, Text Generation, Translation

This paper presents a new version of a football reports generation system called PASS. The original version generated Dutch text and relied on a limited hand-crafted knowledge base. We describe how, in a short amount of time, we extended PASS to produce English texts, exploiting machine translation and Wikidata as a large-scale source of multilingual knowledge.
Poem Machine - a Co-creative NLG Web Application for Poem Writing
Authors: Mika H{\"a}m{\"a}l{\"a}inen
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Text Generation

We present Poem Machine, an interactive online tool for co-authoring Finnish poetry with a computationally creative agent. Poem Machine can produce poetry of its own and assist the user in authoring poems. The main target group for the system is primary school children, and its use as a part of teaching is currently under study.
Japanese Advertising Slogan Generator using Case Frame and Word Vector
Authors: Kango Iwama, Yoshinobu Kano
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Machine Translation, Sentence, Text Generation

There has been many works published for automatic sentence generation of a variety of domains. However, there would be still no single method available at present that can generate sentences for all of domains. Each domain will require a suitable generation method. We focus on automatic generation of Japanese advertisement slogans in this paper. We use our advertisement slogan database, case frame information, and word vector information. We employed our system to apply for a copy competition for human copywriters, where our advertisement slogan was left as a finalist. Our system could be regarded as the world first system that generates slogans in a practical level, as an advertising agency already employs our system in their business.
Underspecified Universal Dependency Structures as Inputs for Multilingual Surface Realisation
Authors: Simon Mille, Anja Belz, Bernd Bohnet, Leo Wanner
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Natural Language Understanding, Text Generation

In this paper, we present the datasets used in the Shallow and Deep Tracks of the First Multilingual Surface Realisation Shared Task (SR{'}18). For the Shallow Track, data in ten languages has been released: Arabic, Czech, Dutch, English, Finnish, French, Italian, Portuguese, Russian and Spanish. For the Deep Track, data in three languages is made available: English, French and Spanish. We describe in detail how the datasets were derived from the Universal Dependencies V2.0, and report on an evaluation of the Deep Track input quality. In addition, we examine the motivation for, and likely usefulness of, deriving NLG inputs from annotations in resources originally developed for Natural Language Understanding (NLU), and assess whether the resulting inputs supply enough information of the right kind for the final stage in the NLG process.
LSTM Hypertagging
Authors: Reid Fu, Michael White
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Text Generation

Hypertagging, or supertagging for surface realization, is the process of assigning lexical categories to nodes in an input semantic graph. Previous work has shown that hypertagging significantly increases realization speed and quality by reducing the search space of the realizer. Building on recent work using LSTMs to improve accuracy on supertagging for parsing, we develop an LSTM hypertagging method for OpenCCG, an open source NLP toolkit for CCG. Our results show significant improvements in both hypertagging accuracy and downstream realization performance.
Generating E-Commerce Product Titles and Predicting their Quality
Authors: Jos{\'e} G. Camargo de Souza, Michael Kozielski, Prashant Mathur, Ernie Chang, Marco Guerini, Matteo Negri, Marco Turchi, Evgeny Matusov
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Text Generation

E-commerce platforms present products using titles that summarize product information. These titles cannot be created by hand, therefore an algorithmic solution is required. The task of automatically generating these titles given noisy user provided titles is one way to achieve the goal. The setting requires the generation process to be fast and the generated title to be both human-readable and concise. Furthermore, we need to understand if such generated titles are usable. As such, we propose approaches that (i) automatically generate product titles, (ii) predict their quality. Our approach scales to millions of products and both automatic and human evaluations performed on real-world data indicate our approaches are effective and applicable to existing e-commerce scenarios.
Designing and testing the messages produced by a virtual dietitian
Authors: Luca Anselma, Aless Mazzei, ro
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Data-to-Text Generation, Management, Text Generation

This paper presents a project about the automatic generation of persuasive messages in the context of the diet management. In the first part of the paper we introduce the basic mechanisms related to data interpretation and content selection for a numerical data-to-text generation architecture. In the second part of the paper we discuss a number of factors influencing the design of the messages. In particular, we consider the design of the aggregation procedure. Finally, we present the results of a human-based evaluation concerning this design factor.
Generation of Company descriptions using concept-to-text and text-to-text deep models: dataset collection and systems evaluation
Authors: Raheel Qader, Khoder Jneid, Fran{\c{c}}ois Portet, Cyril Labb{\'e}
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Text Generation

In this paper we study the performance of several state-of-the-art sequence-to-sequence models applied to generation of short company descriptions. The models are evaluated on a newly created and publicly available company dataset that has been collected from Wikipedia. The dataset consists of around 51K company descriptions that can be used for both concept-to-text and text-to-text generation tasks. Automatic metrics and human evaluation scores computed on the generated company descriptions show promising results despite the difficulty of the task as the dataset (like most available datasets) has not been originally designed for machine learning. In addition, we perform correlation analysis between automatic metrics and human evaluations and show that certain automatic metrics are more correlated to human judgments.
Automatically Generating Questions about Novel Metaphors in Literature
Authors: Natalie Parde, Rodney Nielsen
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Text Generation

The automatic generation of stimulating questions is crucial to the development of intelligent cognitive exercise applications. We developed an approach that generates appropriate \textit{Questioning the Author} queries based on novel metaphors in diverse syntactic relations in literature. We show that the generated questions are comparable to human-generated questions in terms of naturalness, sensibility, and depth, and score slightly higher than human-generated questions in terms of clarity. We also show that questions generated about novel metaphors are rated as cognitively deeper than questions generated about non- or conventional metaphors, providing evidence that metaphor novelty can be leveraged to promote cognitive exercise.
A Master-Apprentice Approach to Automatic Creation of Culturally Satirical Movie Titles
Authors: Khalid Alnajjar, Mika H{\"a}m{\"a}l{\"a}inen
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: NMT, Text Generation

Satire has played a role in indirectly expressing critique towards an authority or a person from time immemorial. We present an autonomously creative master-apprentice approach consisting of a genetic algorithm and an NMT model to produce humorous and culturally apt satire out of movie titles automatically. Furthermore, we evaluate the approach in terms of its creativity and its output. We provide a solid definition for creativity to maximize the objectiveness of the evaluation.
Evaluation methodologies in Automatic Question Generation 2013-2018
Authors: Jacopo Amidei, Paul Piwek, Alistair Willis
Date: 2018-11-01 00:00:00
Proceeding: WS 2018 11
Tasks: Question Generation, Question-Generation, Text Generation

In the last few years Automatic Question Generation (AQG) has attracted increasing interest. In this paper we survey the evaluation methodologies used in AQG. Based on a sample of 37 papers, our research shows that the systems{'} development has not been accompanied by similar developments in the methodologies used for the systems{'} evaluation. Indeed, in the papers we examine here, we find a wide variety of both intrinsic and extrinsic evaluation methodologies. Such diverse evaluation practices make it difficult to reliably compare the quality of different generation systems. Our study suggests that, given the rapidly increasing level of research in the area, a common framework is urgently needed to compare the performance of AQG systems and NLG systems more generally.