pwc-archive/papers-with-abstracts huggingface dataset:
The Lazy Encoder: A Fine-Grained Analysis of the Role of Morphology in Neural Machine Translation
Authors: Arianna Bisazza, Clara Tump
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Machine Translation, NMT, Translation, Word Embeddings

Neural sequence-to-sequence models have proven very effective for machine translation, but at the expense of model interpretability. To shed more light into the role played by linguistic structure in the process of neural machine translation, we perform a fine-grained analysis of how various source-side morphological features are captured at different levels of the NMT encoder while varying the target language. Differently from previous work, we find no correlation between the accuracy of source morphology encoding and translation quality. We do find that morphological features are only captured in context and only to the extent that they are directly transferable to the target words.
An Encoder-Decoder Approach to the Paradigm Cell Filling Problem
Authors: Miikka Silfverberg, Mans Hulden
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Decoder, Denoising, Language Acquisition, Morphological Inflection

The Paradigm Cell Filling Problem in morphology asks to complete word inflection tables from partial ones. We implement novel neural models for this task, evaluating them on 18 data sets in 8 languages, showing performance that is comparable with previous work with far less training data. We also publish a new dataset for this task and code implementing the system described in this paper.
Estimating Marginal Probabilities of n-grams for Recurrent Neural Language Models
Authors: Thanapon Noraset, Doug Downey, Lidong Bing
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Language Modeling, Language Modelling, Machine Translation, Speech Recognition

Recurrent neural network language models (RNNLMs) are the current standard-bearer for statistical language modeling. However, RNNLMs only estimate probabilities for complete sequences of text, whereas some applications require context-independent phrase probabilities instead. In this paper, we study how to compute an RNNLM{'}s em marginal probability: the probability that the model assigns to a short sequence of text when the preceding context is not known. We introduce a simple method of altering the RNNLM training to make the model more accurate at marginal estimation. Our experiments demonstrate that the technique is effective compared to baselines including the traditional RNNLM probability and an importance sampling approach. Finally, we show how we can use the marginal estimation to improve an RNNLM by training the marginals to match n-gram probabilities from a larger corpus.
How to represent a word and predict it, too: Improving tied architectures for language modelling
Authors: Kristina Gulordava, Laura Aina, Gemma Boleda
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Language Modelling, Prediction, Representation Learning, Word Embeddings

Recent state-of-the-art neural language models share the representations of words given by the input and output mappings. We propose a simple modification to these architectures that decouples the hidden state from the word embedding prediction. Our architecture leads to comparable or better results compared to previous tied models and models without tying, with a much smaller number of parameters. We also extend our proposal to word2vec models, showing that tying is appropriate for general word prediction tasks.
Three Strategies to Improve One-to-Many Multilingual Translation
Authors: Yining Wang, Jiajun Zhang, FeiFei Zhai, Jingfang Xu, Cheng-qing Zong
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Decoder, Machine Translation, Multi-Task Learning, Translation

Due to the benefits of model compactness, multilingual translation (including many-to-one, many-to-many and one-to-many) based on a universal encoder-decoder architecture attracts more and more attention. However, previous studies show that one-to-many translation based on this framework cannot perform on par with the individually trained models. In this work, we introduce three strategies to improve one-to-many multilingual translation by balancing the shared and unique features. Within the architecture of one decoder for all target languages, we first exploit the use of unique initial states for different target languages. Then, we employ language-dependent positional embeddings. Finally and especially, we propose to divide the hidden cells of the decoder into shared and language-dependent ones. The extensive experiments demonstrate that our proposed methods can obtain remarkable improvements over the strong baselines. Moreover, our strategies can achieve comparable or even better performance than the individually trained translation models.
Fixing Translation Divergences in Parallel Corpora for Neural MT
Authors: MinhQuang Pham, Josep Crego, Jean Senellart, Fran{\c{c}}ois Yvon
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Machine Translation, Sentence, Sentence Similarity, Translation

Corpus-based approaches to machine translation rely on the availability of clean parallel corpora. Such resources are scarce, and because of the automatic processes involved in their preparation, they are often noisy. This paper describes an unsupervised method for detecting translation divergences in parallel sentences. We rely on a neural network that computes cross-lingual sentence similarity scores, which are then used to effectively filter out divergent translations. Furthermore, similarity scores predicted by the network are used to identify and fix some partial divergences, yielding additional parallel segments. We evaluate these methods for English-French and English-German machine translation tasks, and show that using filtered/corrected corpora actually improves MT performance.
Adversarial Evaluation of Multimodal Machine Translation
Authors: Desmond Elliott
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Machine Translation, Multimodal Machine Translation, text similarity, Translation

The promise of combining language and vision in multimodal machine translation is that systems will produce better translations by leveraging the image data. However, the evidence surrounding whether the images are useful is unconvincing due to inconsistencies between text-similarity metrics and human judgements. We present an adversarial evaluation to directly examine the utility of the image data in this task. Our evaluation tests whether systems perform better when paired with congruent images or incongruent images. This evaluation shows that only one out of three publicly available systems is sensitive to this perturbation of the data. We recommend that multimodal translation systems should be able to pass this sanity check in the future.
End-to-End Non-Autoregressive Neural Machine Translation with Connectionist Temporal Classification
Authors: Jind{\v{r}}ich Libovick{\'y}, Jind{\v{r}}ich Helcl
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Decoder, General Classification, Machine Translation, Translation

Autoregressive decoding is the only part of sequence-to-sequence models that prevents them from massive parallelization at inference time. Non-autoregressive models enable the decoder to generate all output symbols independently in parallel. We present a novel non-autoregressive architecture based on connectionist temporal classification and evaluate it on the task of neural machine translation. Unlike other non-autoregressive methods which operate in several steps, our model can be trained end-to-end. We conduct experiments on the WMT English-Romanian and English-German datasets. Our models achieve a significant speedup over the autoregressive models, keeping the translation quality comparable to other non-autoregressive models.
Prediction Improves Simultaneous Neural Machine Translation
Authors: Ashkan Alinejad, Maryam Siahbani, Anoop Sarkar
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Machine Translation, Prediction, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Translation

Simultaneous speech translation aims to maintain translation quality while minimizing the delay between reading input and incrementally producing the output. We propose a new general-purpose prediction action which predicts future words in the input to improve quality and minimize delay in simultaneous translation. We train this agent using reinforcement learning with a novel reward function. Our agent with prediction has better translation quality and less delay compared to an agent-based simultaneous translation system without prediction.
Context and Copying in Neural Machine Translation
Authors: Rebecca Knowles, Philipp Koehn
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Machine Translation, Translation

Neural machine translation systems with subword vocabularies are capable of translating or copying unknown words. In this work, we show that they learn to copy words based on both the context in which the words appear as well as features of the words themselves. In contexts that are particularly copy-prone, they even copy words that they have already learned they should translate. We examine the influence of context and subword features on this and other types of copying behavior.
Encoding Gated Translation Memory into Neural Machine Translation
Authors: Qian Cao, Deyi Xiong
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Decoder, Machine Translation, NMT, Semantic Textual Similarity, Sentence Embeddings, Translation

Translation memories (TM) facilitate human translators to reuse existing repetitive translation fragments. In this paper, we propose a novel method to combine the strengths of both TM and neural machine translation (NMT) for high-quality translation. We treat the target translation of a TM match as an additional reference input and encode it into NMT with an extra encoder. A gating mechanism is further used to balance the impact of the TM match on the NMT decoder. Experiment results on the UN corpus demonstrate that when fuzzy matches are higher than 50{\%}, the quality of NMT translation can be significantly improved by over 10 BLEU points.
Automatic Post-Editing of Machine Translation: A Neural Programmer-Interpreter Approach
Authors: Thuy-Trang Vu, Gholamreza Haffari
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Automatic Post-Editing, Machine Translation, Translation

Automated Post-Editing (PE) is the task of automatically correct common and repetitive errors found in machine translation (MT) output. In this paper, we present a neural programmer-interpreter approach to this task, resembling the way that human perform post-editing using discrete edit operations, wich we refer to as programs. Our model outperforms previous neural models for inducing PE programs on the WMT17 APE task for German-English up to +1 BLEU score and -0.7 TER scores.
Word Embeddings for Code-Mixed Language Processing
Authors: Adithya Pratapa, Monojit Choudhury, Sunayana Sitaram
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Machine Translation, POS, POS Tagging, Sentiment Analysis, Word Embeddings

We compare three existing bilingual word embedding approaches, and a novel approach of training skip-grams on synthetic code-mixed text generated through linguistic models of code-mixing, on two tasks - sentiment analysis and POS tagging for code-mixed text. Our results show that while CVM and CCA based embeddings perform as well as the proposed embedding technique on semantic and syntactic tasks respectively, the proposed approach provides the best performance for both tasks overall. Thus, this study demonstrates that existing bilingual embedding techniques are not ideal for code-mixed text processing and there is a need for learning multilingual word embedding from the code-mixed text.
Part-of-Speech Tagging for Code-Switched, Transliterated Texts without Explicit Language Identification
Authors: Kelsey Ball, Dan Garrette
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Language Identification, Part-Of-Speech Tagging, Transliteration, Word Embeddings

Code-switching, the use of more than one language within a single utterance, is ubiquitous in much of the world, but remains a challenge for NLP largely due to the lack of representative data for training models. In this paper, we present a novel model architecture that is trained exclusively on monolingual resources, but can be applied to unseen code-switched text at inference time. The model accomplishes this by jointly maintaining separate word representations for each of the possible languages, or scripts in the case of transliteration, allowing each to contribute to inferences without forcing the model to commit to a language. Experiments on Hindi-English part-of-speech tagging demonstrate that our approach outperforms standard models when training on monolingual text without transliteration, and testing on code-switched text with alternate scripts.
Few-Shot and Zero-Shot Multi-Label Learning for Structured Label Spaces
Authors: Anthony Rios, Ramakanth Kavuluru
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: General Classification, Multi-Label Classification, Multi-Label Learning, Multi Label Text Classification, Multi-Label Text Classification, text-classification, Text Classification

Large multi-label datasets contain labels that occur thousands of times (frequent group), those that occur only a few times (few-shot group), and labels that never appear in the training dataset (zero-shot group). Multi-label few- and zero-shot label prediction is mostly unexplored on datasets with large label spaces, especially for text classification. In this paper, we perform a fine-grained evaluation to understand how state-of-the-art methods perform on infrequent labels. Furthermore, we develop few- and zero-shot methods for multi-label text classification when there is a known structure over the label space, and evaluate them on two publicly available medical text datasets: MIMIC II and MIMIC III. For few-shot labels we achieve improvements of 6.2{\%} and 4.8{\%} in R@10 for MIMIC II and MIMIC III, respectively, over prior efforts; the corresponding R@10 improvements for zero-shot labels are 17.3{\%} and 19{\%}.
Automatic Poetry Generation with Mutual Reinforcement Learning
Authors: Xiaoyuan Yi, Maosong Sun, Ruoyu Li, Wenhao Li
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Poetry is one of the most beautiful forms of human language art. As a crucial step towards computer creativity, automatic poetry generation has drawn researchers{'} attention for decades. In recent years, some neural models have made remarkable progress in this task. However, they are all based on maximum likelihood estimation, which only learns common patterns of the corpus and results in loss-evaluation mismatch. Human experts evaluate poetry in terms of some specific criteria, instead of word-level likelihood. To handle this problem, we directly model the criteria and use them as explicit rewards to guide gradient update by reinforcement learning, so as to motivate the model to pursue higher scores. Besides, inspired by writing theories, we propose a novel mutual reinforcement learning schema. We simultaneously train two learners (generators) which learn not only from the teacher (rewarder) but also from each other to further improve performance. We experiment on Chinese poetry. Based on a strong basic model, our method achieves better results and outperforms the current state-of-the-art method.
Variational Autoregressive Decoder for Neural Response Generation
Authors: Jiachen Du, Wenjie Li, Yulan He, Ruifeng Xu, Lidong Bing, Xuan Wang
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Decoder, Diversity, Response Generation

Combining the virtues of probability graphic models and neural networks, Conditional Variational Auto-encoder (CVAE) has shown promising performance in applications such as response generation. However, existing CVAE-based models often generate responses from a single latent variable which may not be sufficient to model high variability in responses. To solve this problem, we propose a novel model that sequentially introduces a series of latent variables to condition the generation of each word in the response sequence. In addition, the approximate posteriors of these latent variables are augmented with a backward Recurrent Neural Network (RNN), which allows the latent variables to capture long-term dependencies of future tokens in generation. To facilitate training, we supplement our model with an auxiliary objective that predicts the subsequent bag of words. Empirical experiments conducted on Opensubtitle and Reddit datasets show that the proposed model leads to significant improvement on both relevance and diversity over state-of-the-art baselines.
Knowledge Graph Embedding with Hierarchical Relation Structure
Authors: Zhao Zhang, Fuzhen Zhuang, Meng Qu, Fen Lin, Qing He
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Graph Embedding, Information Retrieval, Knowledge Base Completion, Knowledge Graph Embedding, Knowledge Graphs, Question Answering, Relation

The rapid development of knowledge graphs (KGs), such as Freebase and WordNet, has changed the paradigm for AI-related applications. However, even though these KGs are impressively large, most of them are suffering from incompleteness, which leads to performance degradation of AI applications. Most existing researches are focusing on knowledge graph embedding (KGE) models. Nevertheless, those models simply embed entities and relations into latent vectors without leveraging the rich information from the relation structure. Indeed, relations in KGs conform to a three-layer hierarchical relation structure (HRS), i.e., semantically similar relations can make up relation clusters and some relations can be further split into several fine-grained sub-relations. Relation clusters, relations and sub-relations can fit in the top, the middle and the bottom layer of three-layer HRS respectively. To this end, in this paper, we extend existing KGE models TransE, TransH and DistMult, to learn knowledge representations by leveraging the information from the HRS. Particularly, our approach is capable to extend other KGE models. Finally, the experiment results clearly validate the effectiveness of the proposed approach against baselines.
Implicational Universals in Stochastic Constraint-Based Phonology
Authors: Giorgio Magri
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10

This paper focuses on the most basic implicational universals in phonological theory, called T-orders after Anttila and Andrus (2006). It shows that the T-orders predicted by stochastic (and partial order) Optimality Theory coincide with those predicted by categorical OT. Analogously, the T-orders predicted by stochastic Harmonic Grammar coincide with those predicted by categorical HG. In other words, these stochastic constraint-based frameworks do not tamper with the typological structure induced by the original categorical frameworks.
A Computational Exploration of Exaggeration
Authors: Enrica Troiano, Carlo Strapparava, G{\"o}zde {\"O}zbal, Serra Sinem Tekiro{\u{g}}lu
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10

Several NLP studies address the problem of figurative language, but among non-literal phenomena, they have neglected exaggeration. This paper presents a first computational approach to this figure of speech. We explore the possibility to automatically detect exaggerated sentences. First, we introduce HYPO, a corpus containing overstatements (or hyperboles) collected on the web and validated via crowdsourcing. Then, we evaluate a number of models trained on HYPO, and bring evidence that the task of hyperbole identification can be successfully performed based on a small set of semantic features.
Hierarchical Dirichlet Gaussian Marked Hawkes Process for Narrative Reconstruction in Continuous Time Domain
Authors: Yeon Seonwoo, Alice Oh, Sungjoon Park
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Articles

In news and discussions, many articles and posts are provided without their related previous articles or posts. Hence, it is difficult to understand the context from which the articles and posts have occurred. In this paper, we propose the Hierarchical Dirichlet Gaussian Marked Hawkes process (HD-GMHP) for reconstructing the narratives and thread structures of news articles and discussion posts. HD-GMHP unifies three modeling strategies in previous research: temporal characteristics, triggering event relations, and meta information of text in news articles and discussion threads. To show the effectiveness of the model, we perform experiments in narrative reconstruction and thread reconstruction with real world datasets: articles from the New York Times and a corpus of Wikipedia conversations. The experimental results show that HD-GMHP outperforms the baselines of LDA, HDP, and HDHP for both tasks.
Investigating the Role of Argumentation in the Rhetorical Analysis of Scientific Publications with Neural Multi-Task Learning Models
Authors: Anne Lauscher, Goran Glava{\v{s}}, Simone Paolo Ponzetto, Kai Eckert
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: General Classification, Multi-Task Learning

Exponential growth in the number of scientific publications yields the need for effective automatic analysis of rhetorical aspects of scientific writing. Acknowledging the argumentative nature of scientific text, in this work we investigate the link between the argumentative structure of scientific publications and rhetorical aspects such as discourse categories or citation contexts. To this end, we (1) augment a corpus of scientific publications annotated with four layers of rhetoric annotations with argumentation annotations and (2) investigate neural multi-task learning architectures combining argument extraction with a set of rhetorical classification tasks. By coupling rhetorical classifiers with the extraction of argumentative components in a joint multi-task learning setting, we obtain significant performance gains for different rhetorical analysis tasks.
Content Explorer: Recommending Novel Entities for a Document Writer
Authors: Michal Lukasik, Richard Zens
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Extreme Multi-Label Classification, General Classification, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION

Background research is an essential part of document writing. Search engines are great for retrieving information once we know what to look for. However, the bigger challenge is often identifying topics for further research. Automated tools could help significantly in this discovery process and increase the productivity of the writer. In this paper, we formulate the problem of recommending topics to a writer. We consider this as a supervised learning problem and run a user study to validate this approach. We propose an evaluation metric and perform an empirical comparison of state-of-the-art models for extreme multi-label classification on a large data set. We demonstrate how a simple modification of the cross-entropy loss function leads to improved results of the deep learning models.