pwc-archive/papers-with-abstracts huggingface dataset:
Data-Driven News Generation for Automated Journalism
Authors: Leo Lepp{\"a}nen, Myriam Munezero, Mark Granroth-Wilding, Hannu Toivonen
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Articles, News Generation, Text Generation

Despite increasing amounts of data and ever improving natural language generation techniques, work on automated journalism is still relatively scarce. In this paper, we explore the field and challenges associated with building a journalistic natural language generation system. We present a set of requirements that should guide system design, including transparency, accuracy, modifiability and transferability. Guided by the requirements, we present a data-driven architecture for automated journalism that is largely domain and language independent. We illustrate its practical application in the production of news articles about the 2017 Finnish municipal elections in three languages, demonstrating the successfulness of the data-driven, modular approach of the design. We then draw some lessons for future automated journalism.
Data Augmentation for Visual Question Answering
Authors: Kushal Kafle, Mohammed Yousefhussien, Christopher Kanan
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Data Augmentation, General Classification, image-classification, Image Classification, Question Answering, Text Generation, Visual Question Answering, Visual Question Answering (VQA)

Data augmentation is widely used to train deep neural networks for image classification tasks. Simply flipping images can help learning tremendously by increasing the number of training images by a factor of two. However, little work has been done studying data augmentation in natural language processing. Here, we describe two methods for data augmentation for Visual Question Answering (VQA). The first uses existing semantic annotations to generate new questions. The second method is a generative approach using recurrent neural networks. Experiments show that the proposed data augmentation improves performance of both baseline and state-of-the-art VQA algorithms.
Personalized Questions, Answers and Grammars: Aiding the Search for Relevant Web Information
Authors: Marta Gatius
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Text Generation

This work proposes an organization of knowledge to facilitate the generation of personalized questions, answers and grammars from web documents. To reduce the human effort needed in the generation of the linguistic resources for a new domain, the general aspects that can be reuse across domains are separated from those more specific. The proposed approach is based on the representation of the main domain concepts as a set of attributes. These attributes are related to a syntactico-semantic taxonomy representing the general relationships between conceptual and linguistic knowledge. User models are incorporated by distinguishing different user groups and relating each group to the appropriate conceptual attributes. Then, the data is extracted from the web documents and represented as instances of the domain concepts. Questions, answers and grammars are generated from these instances.
Investigating the content and form of referring expressions in Mandarin: introducing the Mtuna corpus
Authors: Kees van Deemter, Le Sun, Rint Sybesma, Xiao Li, Bo Chen, Muyun Yang
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Form, Text Generation

East Asian languages are thought to handle reference differently from languages such as English, particularly in terms of the marking of definiteness and number. We present the first Data-Text corpus for Referring Expressions in Mandarin, and we use this corpus to test some initial hypotheses inspired by the theoretical linguistics literature. Our findings suggest that function words deserve more attention in Referring Expressions Generation than they have so far received, and they have a bearing on the debate about whether different languages make different trade-offs between clarity and brevity.
Realization of long sentences using chunking
Authors: Ewa Muszy{\'n}ska, Ann Copestake
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Chunking, Sentence, Text Generation

We propose sentence chunking as a way to reduce the time and memory costs of realization of long sentences. During chunking we divide the semantic representation of a sentence into smaller components which can be processed and recombined without loss of information. Our meaning representation of choice is the Dependency Minimal Recursion Semantics (DMRS). We show that realizing chunks of a sentence and combining the results of such realizations increases the coverage for long sentences, significantly reduces the resources required and does not affect the quality of the realization.
SaToS: Assessing and Summarising Terms of Services from German Webshops
Authors: Daniel Braun, Elena Scepankova, Patrick Holl, Florian Matthes
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Text Generation

Every time we buy something online, we are confronted with Terms of Services. However, only a few people actually read these terms, before accepting them, often to their disadvantage. In this paper, we present the SaToS browser plugin which summarises and simplifies Terms of Services from German webshops.
Textually Summarising Incomplete Data
Authors: Stephanie Inglis, Ehud Reiter, Somayajulu Sripada
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Text Generation

Many data-to-text NLG systems work with data sets which are incomplete, ie some of the data is missing. We have worked with data journalists to understand how they describe incomplete data, and are building NLG algorithms based on these insights. A pilot evaluation showed mixed results, and highlighted several areas where we need to improve our system.
Improving the generation of personalised descriptions
Authors: Thiago Castro Ferreira, Iv Paraboni, r{\'e}
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Referring Expression, Referring expression generation, Text Generation

Referring expression generation (REG) models that use speaker-dependent information require a considerable amount of training data produced by every individual speaker, or may otherwise perform poorly. In this work we propose a simple personalised method for this task, in which speakers are grouped into profiles according to their referential behaviour. Intrinsic evaluation shows that the use of speaker{'}s profiles generally outperforms the personalised method found in previous work.
Linguistic Description of Complex Phenomena with the rLDCP R Package
Authors: Jose Alonso, Patricia Conde-Clemente, Gracian Trivino
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Text Generation

Monitoring and analysis of complex phenomena attract the attention of both academy and industry. Dealing with data produced by complex phenomena requires the use of advance computational intelligence techniques. Namely, linguistic description of complex phenomena constitutes a mature research line. It is supported by the Computational Theory of Perceptions grounded on the Fuzzy Sets Theory. Its aim is the development of computational systems with the ability to generate vague descriptions of the world in a similar way how humans do. This is a human-centric and multi-disciplinary research work. Moreover, its success is a matter of careful design; thus, developers play a key role. The rLDCP R package was designed to facilitate the development of new applications. This demo introduces the use of rLDCP, for both beginners and advance developers, in practical use cases.
A demo of FORGe: the Pompeu Fabra Open Rule-based Generator
Authors: Simon Mille, Leo Wanner
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Sentence, Text Generation

This demo paper presents the multilingual deep sentence generator developed by the TALN group at Universitat Pompeu Fabra, implemented as a series of rule-based graph-transducers for the syntacticization of the input graphs, the resolution of morphological agreements, and the linearization of the trees.
Referential Success of Set Referring Expressions with Fuzzy Properties
Authors: Nicol{\'a}s Mar{\'\i}n, Gustavo Rivas-Gervilla, Daniel S{\'a}nchez
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Text Generation

We introduce the properties to be satisfied by measures of referential success of set referring expressions with fuzzy properties. We define families of measures on the basis of n-cardinality measures and we illustrate some of them with a toy example.
Neural Response Generation for Customer Service based on Personality Traits
Authors: Jonathan Herzig, Michal Shmueli-Scheuer, S, Tommy bank, David Konopnicki
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Response Generation, Text Generation

We present a neural response generation model that generates responses conditioned on a target personality. The model learns high level features based on the target personality, and uses them to update its hidden state. Our model achieves performance improvements in both perplexity and BLEU scores over a baseline sequence-to-sequence model, and is validated by human judges.
Neural Paraphrase Generation using Transfer Learning
Authors: Florin Brad, Traian Rebedea
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Machine Translation, Natural Language Inference, Paraphrase Generation, Question Answering, Text Generation, Transfer Learning, Translation

Progress in statistical paraphrase generation has been hindered for a long time by the lack of large monolingual parallel corpora. In this paper, we adapt the neural machine translation approach to paraphrase generation and perform transfer learning from the closely related task of entailment generation. We evaluate the model on the Microsoft Research Paraphrase (MSRP) corpus and show that the model is able to generate sentences that capture part of the original meaning, but fails to pick up on important words or to show large lexical variation.
Character and Subword-Based Word Representation for Neural Language Modeling Prediction
Authors: Matthieu Labeau, Alex Allauzen, re
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Language Modeling, Language Modelling, Machine Translation, Speech Recognition, Word Embeddings

Most of neural language models use different kinds of embeddings for word prediction. While word embeddings can be associated to each word in the vocabulary or derived from characters as well as factored morphological decomposition, these word representations are mainly used to parametrize the input, i.e. the context of prediction. This work investigates the effect of using subword units (character and factored morphological decomposition) to build output representations for neural language modeling. We present a case study on Czech, a morphologically-rich language, experimenting with different input and output representations. When working with the full training vocabulary, despite unstable training, our experiments show that augmenting the output word representations with character-based embeddings can significantly improve the performance of the model. Moreover, reducing the size of the output look-up table, to let the character-based embeddings represent rare words, brings further improvement.
Character Based Pattern Mining for Neology Detection
Authors: Ga{\"e}l Lejeune, Emmanuel Cartier
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: General Classification

Detecting neologisms is essential in real-time natural language processing applications. Not only can it enable to follow the lexical evolution of languages, but it is also essential for updating linguistic resources and parsers. In this paper, neology detection is considered as a classification task where a system has to assess whether a given lexical item is an actual neologism or not. We propose a combination of an unsupervised data mining technique and a supervised machine learning approach. It is inspired by current researches in stylometry and on token-level and character-level patterns. We train and evaluate our system on a manually designed reference dataset in French and Russian. We show that this approach is able to largely outperform state-of-the-art neology detection systems. Furthermore, character-level patterns exhibit good properties for multilingual extensions of the system.
Supersense Tagging with a Combination of Character, Subword, and Word-level Representations
Authors: Youhyun Shin, Sang-goo Lee
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Entity Alignment, Language Modeling, Language Modelling, Machine Translation, Named Entity Recognition (NER), Word Sense Disambiguation

Recently, there has been increased interest in utilizing characters or subwords for natural language processing (NLP) tasks. However, the effect of utilizing character, subword, and word-level information simultaneously has not been examined so far. In this paper, we propose a model to leverage various levels of input features to improve on the performance of an supersense tagging task. Detailed analysis of experimental results show that different levels of input representation offer distinct characteristics that explain performance discrepancy among different tasks.
Weakly supervised learning of allomorphy
Authors: Miikka Silfverberg, Mans Hulden
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Weakly-supervised Learning

Most NLP resources that offer annotations at the word segment level provide morphological annotation that includes features indicating tense, aspect, modality, gender, case, and other inflectional information. Such information is rarely aligned to the relevant parts of the words{---}i.e. the allomorphs, as such annotation would be very costly. These unaligned weak labelings are commonly provided by annotated NLP corpora such as treebanks in various languages. Although they lack alignment information, the presence/absence of labels at the word level is also consistent with the amount of supervision assumed to be provided to L1 and L2 learners. In this paper, we explore several methods to learn this latent alignment between parts of word forms and the grammatical information provided. All the methods under investigation favor hypotheses regarding allomorphs of morphemes that re-use a small inventory, i.e. implicitly minimize the number of allomorphs that a morpheme can be realized as. We show that the provided information offers a significant advantage for both word segmentation and the learning of allomorphy.
Character-based recurrent neural networks for morphological relational reasoning
Authors: Olof Mogren, Richard Johansson
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Decoder, Language Modeling, Language Modelling, Morphological Analysis, Morphological Inflection, Relation, Relational Reasoning, Text Categorization

We present a model for predicting word forms based on \textit{morphological relational reasoning} with analogies. While previous work has explored tasks such as morphological inflection and reinflection, these models rely on an explicit enumeration of morphological features, which may not be available in all cases. To address the task of predicting a word form given a \textit{demo relation} (a pair of word forms) and a \textit{query word}, we devise a character-based recurrent neural network architecture using three separate encoders and a decoder. We also investigate a multiclass learning setup, where the prediction of the relation type label is used as an auxiliary task. Our results show that the exact form can be predicted for English with an accuracy of 94.7{\%}. For Swedish, which has a more complex morphology with more inflectional patterns for nouns and verbs, the accuracy is 89.3{\%}. We also show that using the auxiliary task of learning the relation type speeds up convergence and improves the prediction accuracy for the word generation task.
Exploring Cross-Lingual Transfer of Morphological Knowledge In Sequence-to-Sequence Models
Authors: Huiming Jin, Katharina Kann
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Cross-Lingual Transfer, Decoder, Language Modeling, Language Modelling, Multi-Task Learning, Transfer Learning

Multi-task training is an effective method to mitigate the data sparsity problem. It has recently been applied for cross-lingual transfer learning for paradigm completion{---}the task of producing inflected forms of lemmata{---}with sequence-to-sequence networks. However, it is still vague how the model transfers knowledge across languages, as well as if and which information is shared. To investigate this, we propose a set of data-dependent experiments using an existing encoder-decoder recurrent neural network for the task. Our results show that indeed the performance gains surpass a pure regularization effect and that knowledge about language and morphology can be transferred.
Vowel and Consonant Classification through Spectral Decomposition
Authors: Patricia Thaine, Gerald Penn
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Classification, General Classification

We consider two related problems in this paper. Given an undeciphered alphabetic writing system or mono-alphabetic cipher, determine: (1) which of its letters are vowels and which are consonants; and (2) whether the writing system is a vocalic alphabet or an abjad. We are able to show that a very simple spectral decomposition based on character co-occurrences provides nearly perfect performance with respect to answering both question types.
Character-based Bidirectional LSTM-CRF with words and characters for Japanese Named Entity Recognition
Authors: Shotaro Misawa, Motoki Taniguchi, Yasuhide Miura, Tomoko Ohkuma
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, TAG

Recently, neural models have shown superior performance over conventional models in NER tasks. These models use CNN to extract sub-word information along with RNN to predict a tag for each word. However, these models have been tested almost entirely on English texts. It remains unclear whether they perform similarly in other languages. We worked on Japanese NER using neural models and discovered two obstacles of the state-of-the-art model. First, CNN is unsuitable for extracting Japanese sub-word information. Secondly, a model predicting a tag for each word cannot extract an entity when a part of a word composes an entity. The contributions of this work are (1) verifying the effectiveness of the state-of-the-art NER model for Japanese, (2) proposing a neural model for predicting a tag for each character using word and character information. Experimentally obtained results demonstrate that our model outperforms the state-of-the-art neural English NER model in Japanese.
Spell-Checking based on Syllabification and Character-level Graphs for a Peruvian Agglutinative Language
Authors: Carlo Alva, Arturo Oncevay
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9

There are several native languages in Peru which are mostly agglutinative. These languages are transmitted from generation to generation mainly in oral form, causing different forms of writing across different communities. For this reason, there are recent efforts to standardize the spelling in the written texts, and it would be beneficial to support these tasks with an automatic tool such as an spell-checker. In this way, this spelling corrector is being developed based on two steps: an automatic rule-based syllabification method and a character-level graph to detect the degree of error in a misspelled word. The experiments were realized on Shipibo-konibo, a highly agglutinative and amazonian language, and the results obtained have been promising in a dataset built for the purpose.
What do we need to know about an unknown word when parsing German
Authors: Bich-Ngoc Do, Ines Rehbein, Anette Frank
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Language Modelling, POS, Word Embeddings

We propose a new type of subword embedding designed to provide more information about unknown compounds, a major source for OOV words in German. We present an extrinsic evaluation where we use the compound embeddings as input to a neural dependency parser and compare the results to the ones obtained with other types of embeddings. Our evaluation shows that adding compound embeddings yields a significant improvement of 2{\%} LAS over using word embeddings when no POS information is available. When adding POS embeddings to the input, however, the effect levels out. This suggests that it is not the missing information about the semantics of the unknown words that causes problems for parsing German, but the lack of morphological information for unknown words. To augment our evaluation, we also test the new embeddings in a language modelling task that requires both syntactic and semantic information.
Inflection Generation for Spanish Verbs using Supervised Learning
Authors: Cristina Barros, Dimitra Gkatzia, Elena Lloret
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: LEMMA, Machine Translation, Morphological Inflection, Text Generation

We present a novel supervised approach to inflection generation for verbs in Spanish. Our system takes as input the verb{'}s lemma form and the desired features such as person, number, tense, and is able to predict the appropriate grammatical conjugation. Even though our approach learns from fewer examples comparing to previous work, it is able to deal with all the Spanish moods (indicative, subjunctive and imperative) in contrast to previous work which only focuses on indicative and subjunctive moods. We show that in an intrinsic evaluation, our system achieves 99{\%} accuracy, outperforming (although not significantly) two competitive state-of-art systems. The successful results obtained clearly indicate that our approach could be integrated into wider approaches related to text generation in Spanish.
Sub-character Neural Language Modelling in Japanese
Authors: Viet Nguyen, Julian Brooke, Timothy Baldwin
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Language Modeling, Language Modelling

In East Asian languages such as Japanese and Chinese, the semantics of a character are (somewhat) reflected in its sub-character elements. This paper examines the effect of using sub-characters for language modeling in Japanese. This is achieved by decomposing characters according to a range of character decomposition datasets, and training a neural language model over variously decomposed character representations. Our results indicate that language modelling can be improved through the inclusion of sub-characters, though this result depends on a good choice of decomposition dataset and the appropriate granularity of decomposition.
Byte-based Neural Machine Translation
Authors: Marta R. Costa-juss{\`a}, Carlos Escolano, Jos{\'e} A. R. Fonollosa
Date: 2017-09-01 00:00:00
Proceeding: WS 2017 9
Tasks: Language Modeling, Language Modelling, Machine Translation, Named Entity Recognition (NER), Speech Recognition, Translation

This paper presents experiments comparing character-based and byte-based neural machine translation systems. The main motivation of the byte-based neural machine translation system is to build multi-lingual neural machine translation systems that can share the same vocabulary. We compare the performance of both systems in several language pairs and we see that the performance in test is similar for most language pairs while the training time is slightly reduced in the case of byte-based neural machine translation.