pwc-archive/papers-with-abstracts huggingface dataset:
The use of rating and Likert scales in Natural Language Generation human evaluation tasks: A review and some recommendations
Authors: Jacopo Amidei, Paul Piwek, Alistair Willis
Date: 2019-10-01 00:00:00
Proceeding: WS 2019 10
Tasks: nlg evaluation, Text Generation

Rating and Likert scales are widely used in evaluation experiments to measure the quality of Natural Language Generation (NLG) systems. We review the use of rating and Likert scales for NLG evaluation tasks published in NLG specialized conferences over the last ten years (135 papers in total). Our analysis brings to light a number of deviations from good practice in their use. We conclude with some recommendations about the use of such scales. Our aim is to encourage the appropriate use of evaluation methodologies in the NLG community.
On task effects in NLG corpus elicitation: a replication study using mixed effects modeling
Authors: Emiel van Miltenburg, Merel van de Kerkhof, Ruud Koolen, Martijn Goudbeek, Emiel Krahmer
Date: 2019-10-01 00:00:00
Proceeding: WS 2019 10

Task effects in NLG corpus elicitation recently started to receive more attention, but are usually not modeled statistically. We present a controlled replication of the study by Van Miltenburg et al. (2018b), contrasting spoken with written descriptions. We collected additional written Dutch descriptions to supplement the spoken data from the DIDEC corpus, and analyzed the descriptions using mixed effects modeling to account for variation between participants and items. Our results show that the effects of modality largely disappear in a controlled setting.
Procedural Text Generation from a Photo Sequence
Authors: Taichi Nishimura, Atsushi Hashimoto, Shinsuke Mori
Date: 2019-10-01 00:00:00
Proceeding: WS 2019 10
Tasks: Text Generation

Multimedia procedural texts, such as instructions and manuals with pictures, support people to share how-to knowledge. In this paper, we propose a method for generating a procedural text given a photo sequence allowing users to obtain a multimedia procedural text. We propose a single embedding space both for image and text enabling to interconnect them and to select appropriate words to describe a photo. We implemented our method and tested it on cooking instructions, i.e., recipes. Various experimental results showed that our method outperforms standard baselines.
SimpleNLG-DE: Adapting SimpleNLG 4 to German
Authors: Daniel Braun, Kira Klimt, Daniela Schneider, Florian Matthes
Date: 2019-10-01 00:00:00
Proceeding: WS 2019 10

SimpleNLG is a popular open source surface realiser for the English language. For German, however, the availability of open source and non-domain specific realisers is sparse, partly due to the complexity of the German language. In this paper, we present SimpleNLG-DE, an adaption of SimpleNLG to German. We discuss which parts of the German language have been implemented and how we evaluated our implementation using the TIGER Corpus and newly created data-sets.
Generating Paraphrases with Lean Vocabulary
Authors: Tadashi Nomoto
Date: 2019-10-01 00:00:00
Proceeding: WS 2019 10
Tasks: Paraphrase Generation, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In this work, we examine whether it is possible to achieve the state of the art performance in paraphrase generation with reduced vocabulary. Our approach consists of building a convolution to sequence model (Conv2Seq) partially guided by the reinforcement learning, and training it on the subword representation of the input. The experiment on the Quora dataset, which contains over 140,000 pairs of sentences and corresponding paraphrases, found that with less than 1,000 token types, we were able to achieve performance which exceeded that of the current state of the art.
A Personalized Data-to-Text Support Tool for Cancer Patients
Authors: Saar Hommes, Chris van der Lee, Felix Clouth, Jeroen Vermunt, X Verbeek, er, Emiel Krahmer
Date: 2019-10-01 00:00:00
Proceeding: WS 2019 10
Tasks: Decision Making

In this paper, we present a novel data-to-text system for cancer patients, providing information on quality of life implications after treatment, which can be embedded in the context of shared decision making. Currently, information on quality of life implications is often not discussed, partly because (until recently) data has been lacking. In our work, we rely on a newly developed prediction model, which assigns patients to scenarios. Furthermore, we use data-to-text techniques to explain these scenario-based predictions in personalized and understandable language. We highlight the possibilities of NLG for personalization, discuss ethical implications and also present the outcomes of a first evaluation with clinicians.
Using NLG for speech synthesis of mathematical sentences
Authors: Aless Mazzei, ro, Michele Monticone, Cristian Bernareggi
Date: 2019-10-01 00:00:00
Proceeding: WS 2019 10
Tasks: Sentence, Speech Synthesis, Text Generation

People with sight impairments can access to a mathematical expression by using its LaTeX source. However, this mechanisms have several drawbacks: (1) it assumes the knowledge of the LaTeX, (2) it is slow, since LaTeX is verbose and (3) it is error-prone since LATEX is a typographical language. In this paper we study the design of a natural language generation system for producing a mathematical sentence, i.e. a natural language sentence expressing the semantics of a mathematical expression. Moreover, we describe the main results of a first human based evaluation experiment of the system for Italian language.
Teaching FORGe to Verbalize DBpedia Properties in Spanish
Authors: Simon Mille, Stamatia Dasiopoulou, Beatriz Fisas, Leo Wanner
Date: 2019-10-01 00:00:00
Proceeding: WS 2019 10

Statistical generators increasingly dominate the research in NLG. However, grammar-based generators that are grounded in a solid linguistic framework remain very competitive, especially for generation from deep knowledge structures. Furthermore, if built modularly, they can be ported to other genres and languages with a limited amount of work, without the need of the annotation of a considerable amount of training data. One of these generators is FORGe, which is based on the Meaning-Text Model. In the recent WebNLG challenge (the first comprehensive task addressing the mapping of RDF triples to text) FORGe ranked first with respect to the overall quality in human evaluation. We extend the coverage of FORGE{'}s open source grammatical and lexical resources for English, so as to further improve the English texts, and port them to Spanish, to achieve a comparable quality. This confirms that, as already observed in the case of SimpleNLG, a robust universal grammar-driven framework and a systematic organization of the linguistic resources can be an adequate choice for NLG applications.
Towards Generating Math Word Problems from Equations and Topics
Authors: Qingyu Zhou, Danqing Huang
Date: 2019-10-01 00:00:00
Proceeding: WS 2019 10
Tasks: Math

A math word problem is a narrative with a specific topic that provides clues to the correct equation with numerical quantities and variables therein. In this paper, we focus on the task of generating math word problems. Previous works are mainly template-based with pre-defined rules. We propose a novel neural network model to generate math word problems from the given equations and topics. First, we design a fusion mechanism to incorporate the information of both equations and topics. Second, an entity-enforced loss is introduced to ensure the relevance between the generated math problem and the equation. Automatic evaluation results show that the proposed model significantly outperforms the baseline models. In human evaluations, the math word problems generated by our model are rated as being more relevant (in terms of solvability of the given equations and relevance to topics) and natural (i.e., grammaticality, fluency) than the baseline models.
DisSim: A Discourse-Aware Syntactic Text Simplification Framework for English and German
Authors: Christina Niklaus, Matthias Cetto, Andr{\'e} Freitas, H, Siegfried schuh
Date: 2019-10-01 00:00:00
Proceeding: WS 2019 10
Tasks: Sentence, Text Simplification

We introduce DisSim, a discourse-aware sentence splitting framework for English and German whose goal is to transform syntactically complex sentences into an intermediate representation that presents a simple and more regular structure which is easier to process for downstream semantic applications. For this purpose, we turn input sentences into a two-layered semantic hierarchy in the form of core facts and accompanying contexts, while identifying the rhetorical relations that hold between them. In that way, we preserve the coherence structure of the input and, hence, its interpretability for downstream tasks.
Real World Voice Assistant System for Cooking
Authors: Takahiko Ito, Shintaro Inuzuka, Yoshiaki Yamada, Jun Harashima
Date: 2019-10-01 00:00:00
Proceeding: WS 2019 10
Tasks: BIG-bench Machine Learning, Reading Comprehension

This study presents a voice assistant system to support cooking by utilizing smart speakers in Japan. This system not only speaks the procedures written in recipes point by point but also answers the common questions from users for the specified recipes. The system applies machine comprehension techniques to millions of recipes for answering the common questions in cooking such as {``}人参はどうしたらよいですか (How should I cook carrots?){''}. Furthermore, numerous machine-learning techniques are applied to generate better responses to users.
VAE-PGN based Abstractive Model in Multi-stage Architecture for Text Summarization
Authors: Hyungtak Choi, Lohith Ravuru, Tomasz Dryja{\'n}ski, Sunghan Rye, Dong-Hyun Lee, Hojung Lee, Inchul Hwang
Date: 2019-10-01 00:00:00
Proceeding: WS 2019 10
Tasks: Abstractive Text Summarization, Extractive Summarization, Sentence, Text Summarization

This paper describes our submission to the TL;DR challenge. Neural abstractive summarization models have been successful in generating fluent and consistent summaries with advancements like the copy (Pointer-generator) and coverage mechanisms. However, these models suffer from their extractive nature as they learn to copy words from the source text. In this paper, we propose a novel abstractive model based on Variational Autoencoder (VAE) to address this issue. We also propose a Unified Summarization Framework for the generation of summaries. Our model eliminates non-critical information at a sentence-level with an extractive summarization module and generates the summary word by word using an abstractive summarization module. To implement our framework, we combine submodules with state-of-the-art techniques including Pointer-Generator Network (PGN) and BERT while also using our new VAE-PGN abstractive model. We evaluate our model on the benchmark Reddit corpus as part of the TL;DR challenge and show that our model outperforms the baseline in ROUGE score while generating diverse summaries.
Generating Abstractive Summaries with Finetuned Language Models
Authors: Sebastian Gehrmann, Zachary Ziegler, Alex Rush, er
Date: 2019-10-01 00:00:00
Proceeding: WS 2019 10
Tasks: Document Summarization, Inductive Bias, News Summarization, Transfer Learning

Neural abstractive document summarization is commonly approached by models that exhibit a mostly extractive behavior. This behavior is facilitated by a copy-attention which allows models to copy words from a source document. While models in the mostly extractive news summarization domain benefit from this inductive bias, they commonly fail to paraphrase or compress information from the source document. Recent advances in transfer-learning from large pretrained language models give rise to alternative approaches that do not rely on copy-attention and instead learn to generate concise and abstractive summaries. In this paper, as part of the TL;DR challenge, we compare the abstractiveness of summaries from different summarization approaches and show that transfer-learning can be efficiently utilized without any changes to the model architecture. We demonstrate that the approach leads to a higher level of abstraction for a similar performance on the TL;DR challenge tasks, enabling true natural language compression.
Towards Summarization for Social Media - Results of the TL;DR Challenge
Authors: Shahbaz Syed, Michael V{\"o}lske, Nedim Lipka, Benno Stein, Hinrich Sch{\"u}tze, Martin Potthast
Date: 2019-10-01 00:00:00
Proceeding: WS 2019 10

In this paper, we report on the results of the TL;DR challenge, discussing an extensive manual evaluation of the expected properties of a good summary based on analyzing the comments provided by human annotators.
Generating Quantified Descriptions of Abstract Visual Scenes
Authors: Guanyi Chen, Kees Van Deemter, Chenghua Lin
Date: 2019-10-01 00:00:00
Proceeding: WS 2019 10
Tasks: Position, Text Generation

Quantified expressions have always taken up a central position in formal theories of meaning and language use. Yet quantified expressions have so far attracted far less attention from the Natural Language Generation community than, for example, referring expressions. In an attempt to start redressing the balance, we investigate a recently developed corpus in which quantified expressions play a crucial role; the corpus is the result of a carefully controlled elicitation experiment, in which human participants were asked to describe visually presented scenes. Informed by an analysis of this corpus, we propose algorithms that produce computer-generated descriptions of a wider class of visual scenes, and we evaluate the descriptions generated by these algorithms in terms of their correctness, completeness, and human-likeness. We discuss what this exercise can teach us about the nature of quantification and about the challenges posed by the generation of quantified expressions.
What goes into a word: generating image descriptions with top-down spatial knowledge
Authors: Mehdi Ghanimifard, Simon Dobnik
Date: 2019-10-01 00:00:00
Proceeding: WS 2019 10
Tasks: Decoder, Language Modeling, Language Modelling, Text Generation, World Knowledge

Generating grounded image descriptions requires associating linguistic units with their corresponding visual clues. A common method is to train a decoder language model with attention mechanism over convolutional visual features. Attention weights align the stratified visual features arranged by their location with tokens, most commonly words, in the target description. However, words such as spatial relations (e.g. \textit{next{\textasciitilde}to} and \textit{under}) are not directly referring to geometric arrangements of pixels but to complex geometric and conceptual representations. The aim of this paper is to evaluate what representations facilitate generating image descriptions with spatial relations and lead to better grounded language generation. In particular, we investigate the contribution of three different representational modalities in generating relational referring expressions: (i) pre-trained convolutional visual features, (ii) different top-down geometric relational knowledge between objects, and (iii) world knowledge captured by contextual embeddings in language models.
Neural Generation for Czech: Data and Baselines
Authors: Ond{\v{r}}ej Du{\v{s}}ek, Filip Jur{\v{c}}{\'\i}{\v{c}}ek
Date: 2019-10-01 00:00:00
Proceeding: WS 2019 10
Tasks: Data-to-Text Generation, Language Modeling, Language Modelling

We present the first dataset targeted at end-to-end NLG in Czech in the restaurant domain, along with several strong baseline models using the sequence-to-sequence approach. While non-English NLG is under-explored in general, Czech, as a morphologically rich language, makes the task even harder: Since Czech requires inflecting named entities, delexicalization or copy mechanisms do not work out-of-the-box and lexicalizing the generated outputs is non-trivial. In our experiments, we present two different approaches to this this problem: (1) using a neural language model to select the correct inflected form while lexicalizing, (2) a two-step generation setup: our sequence-to-sequence model generates an interleaved sequence of lemmas and morphological tags, which are then inflected by a morphological generator.
Learning from the past, predicting the statistics for the future, learning an evolving system
Authors: Daniel Levin, Terry Lyons, Hao Ni
Date: 2013-09-01 00:00:00
Tasks: Dimensionality Reduction, regression, Time Series, Time Series Analysis

We bring the theory of rough paths to the study of non-parametric statistics on streamed data. We discuss the problem of regression where the input variable is a stream of information, and the dependent response is also (potentially) a stream. A certain graded feature set of a stream, known in the rough path literature as the signature, has a universality that allows formally, linear regression to be used to characterise the functional relationship between independent explanatory variables and the conditional distribution of the dependent response. This approach, via linear regression on the signature of the stream, is almost totally general, and yet it still allows explicit computation. The grading allows truncation of the feature set and so leads to an efficient local description for streams (rough paths). In the statistical context this method offers potentially significant, even transformational dimension reduction. By way of illustration, our approach is applied to stationary time series including the familiar AR model and ARCH model. In the numerical examples we examined, our predictions achieve similar accuracy to the Gaussian Process (GP) approach with much lower computational cost especially when the sample size is large.
Attention-based View Selection Networks for Light-field Disparity Estimation
Authors: Yu-Ju Tsai, Yu-Lun Liu, Ming Ouhyoung, Yung-Yu Chuang
Date: 2020-02-07 00:00:00
Proceeding: AAAI 2020 : The Thirty-Fourth AAAI Conference on Artificial Intelligence 2020 2
Tasks: Depth Estimation, Disparity Estimation

This paper introduces a novel deep network for estimating depth maps from a light field image. For utilizing the views more effectively and reducing redundancy within views, we propose a view selection module that generates an attention map indicating the importance of each view and its potential for contributing to accurate depth estimation. By exploring the symmetric property of light field views, we enforce symmetry in the attention map and further improve accuracy. With the attention map, our architecture utilizes all views more effectively and efficiently. Experiments show that the proposed method achieves state-of-the-art performance in terms of accuracy and ranks the first on a popular benchmark for disparity estimation for light field images.
Mixtape: Breaking the Softmax Bottleneck Efficiently
Authors: Zhilin Yang, Thang Luong, Russ R. Salakhutdinov, Quoc V. Le
Date: 2019-12-01 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Language Modeling, Language Modelling, Machine Translation, Translation, Tree Decomposition

The softmax bottleneck has been shown to limit the expressiveness of neural language models. Mixture of Softmaxes (MoS) is an effective approach to address such a theoretical limitation, but are expensive compared to softmax in terms of both memory and time. We propose Mixtape, an output layer that breaks the softmax bottleneck more efficiently with three novel techniques---logit space vector gating, sigmoid tree decomposition, and gate sharing. On four benchmarks including language modeling and machine translation, the Mixtape layer substantially improves the efficiency over the MoS layer by 3.5x to 10.5x while obtaining similar or better performance. A network equipped with Mixtape is only 20% to 34% slower than a softmax-based network with 10-30K vocabulary sizes, and outperforms softmax by a large margin in perplexity and translation quality. Notably, Mixtape achieves state-of-the-art results of 29.8 BLEU on WMT'14 English-German and 43.9 BLEU on WMT'14 English-French.
Effects of truck traffic on crash injury severity on rural highways in Wyoming using Bayesian binary logit models
Authors: Mohamed M. Ahmed, Rebecca Franke, Khaled Ksaibati, Debbie S. Shinstine
Date: 2018-08-01 00:00:00
Proceeding: journal 2018 8
Tasks: Crash injury severity

Roadway safety is an integral part of a functioning infrastructure. A major use of the highway system is the
transport of goods. The United States has experienced constant growth in the amount of freight transported by
truck in the last few years. Wyoming is experiencing a large increase in truck traffic on its local and county roads
due to an increase in oil and gas production.
This study explores the involvement of heavy trucks in crashes and their significance as a predictor of crash
severity and addresses the effect that large truck traffic is having on the safety of roadways for various road
classifications. Studies have been done on the factors involved in and the causation of heavy truck crashes, but
none address the causation and effect of roadway classifications on truck crashes.
Binary Logit Models (BLM) with Bayesian inferences were utilized to classify heavy truck involvement in
severe and non-severe crashes using ten years (2002–2011) of historical crash data in the State of Wyoming.
From the final main effects model, various interactions proved to be significant in predicting the severity of
crashes and varied depending on the roadway classification. The results indicated the odds of a severe crash
increase to 2.3 and 4.5 times when a heavy truck is involved on state and interstate highways respectively. The
severity of crashes is significantly increased when road conditions were not clear, icy, and during snowy weather
conditions.
Refinements of Barndorff-Nielsen and Shephard model: an analysis of crude oil price with machine learning
Authors: Indranil SenGupta, William Nganje, Erik Hanson
Date: 2019-11-29 00:00:00
Tasks: BIG-bench Machine Learning

A commonly used stochastic model for derivative and commodity market analysis is the Barndorff-Nielsen and Shephard (BN-S) model. Though this model is very efficient and analytically tractable, it suffers from the absence of long range dependence and many other issues. For this paper, the analysis is restricted to crude oil price dynamics. A simple way of improving the BN-S model with the implementation of various machine learning algorithms is proposed. This refined BN-S model is more efficient and has fewer parameters than other models which are used in practice as improvements of the BN-S model. The procedure and the model show the application of data science for extracting a "deterministic component" out of processes that are usually considered to be completely stochastic. Empirical applications validate the efficacy of the proposed model for long range dependence.
What's Hidden in a Randomly Weighted Neural Network?
Authors: Vivek Ramanujan, Mitchell Wortsman, Aniruddha Kembhavi, Ali Farhadi, Mohammad Rastegari
Date: 2019-11-29 00:00:00
Proceeding: CVPR 2020 6
Tasks: Image Classification

Training a neural network is synonymous with learning the values of the weights. By contrast, we demonstrate that randomly weighted neural networks contain subnetworks which achieve impressive performance without ever training the weight values. Hidden in a randomly weighted Wide ResNet-50 we show that there is a subnetwork (with random weights) that is smaller than, but matches the performance of a ResNet-34 trained on ImageNet. Not only do these "untrained subnetworks" exist, but we provide an algorithm to effectively find them. We empirically show that as randomly weighted neural networks with fixed weights grow wider and deeper, an "untrained subnetwork" approaches a network with learned weights in accuracy. Our code and pretrained models are available at https://github.com/allenai/hidden-networks.