pwc-archive/papers-with-abstracts huggingface dataset:
A Multi-lingual Multi-task Architecture for Low-resource Sequence Labeling
Authors: Ying Lin, Shengqi Yang, Veselin Stoyanov, Heng Ji
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: Abstractive Text Summarization, Machine Translation, Multi-Task Learning, Part-Of-Speech Tagging, Text Summarization, Transfer Learning

We propose a multi-lingual multi-task architecture to develop supervised models with a minimal amount of labeled data for sequence labeling. In this new architecture, we combine various transfer models using two layers of parameter sharing. On the first layer, we construct the basis of the architecture to provide universal word representation and feature extraction capability for all models. On the second level, we adopt different parameter sharing strategies for different transfer schemes. This architecture proves to be particularly effective for low-resource settings, when there are less than 200 training sentences for the target task. Using Name Tagging as a target task, our approach achieved 4.3{\%}-50.5{\%} absolute F-score gains compared to the mono-lingual single-task baseline model.
Two Methods for Domain Adaptation of Bilingual Tasks: Delightfully Simple and Broadly Applicable
Authors: Viktor Hangya, Fabienne Braune, Alex Fraser, er, Hinrich Sch{\"u}tze
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: Bilingual Lexicon Induction, Classification, Domain Adaptation, General Classification, Image Classification, Semi-Supervised Image Classification, Sentiment Analysis, Sentiment Classification, Transfer Learning, Word Embeddings

Bilingual tasks, such as bilingual lexicon induction and cross-lingual classification, are crucial for overcoming data sparsity in the target language. Resources required for such tasks are often out-of-domain, thus domain adaptation is an important problem here. We make two contributions. First, we test a delightfully simple method for domain adaptation of bilingual word embeddings. We evaluate these embeddings on two bilingual tasks involving different domains: cross-lingual twitter sentiment classification and medical bilingual lexicon induction. Second, we tailor a broadly applicable semi-supervised classification method from computer vision to these tasks. We show that this method also helps in low-resource setups. Using both methods together we achieve large improvements over our baselines, by using only additional unlabeled data.
Semantically Equivalent Adversarial Rules for Debugging NLP models
Authors: Marco Tulio Ribeiro, Sameer Singh, Carlos Guestrin
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: Data Augmentation, Question Answering, Reading Comprehension, Sentiment Analysis, Visual Question Answering, Visual Question Answering (VQA)

Complex machine learning models for NLP are often brittle, making different predictions for input instances that are extremely similar semantically. To automatically detect this behavior for individual instances, we present semantically equivalent adversaries (SEAs) {--} semantic-preserving perturbations that induce changes in the model{'}s predictions. We generalize these adversaries into semantically equivalent adversarial rules (SEARs) {--} simple, universal replacement rules that induce adversaries on many instances. We demonstrate the usefulness and flexibility of SEAs and SEARs by detecting bugs in black-box state-of-the-art models for three domains: machine comprehension, visual question-answering, and sentiment analysis. Via user studies, we demonstrate that we generate high-quality local adversaries for more instances than humans, and that SEARs induce four times as many mistakes as the bugs discovered by human experts. SEARs are also actionable: retraining models using data augmentation significantly reduces bugs, while maintaining accuracy.
Bridging Languages through Images with Deep Partial Canonical Correlation Analysis
Authors: Guy Rotman, Ivan Vuli{\'c}, Roi Reichart
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: Image Description, Image Retrieval, Question Answering, Representation Learning, Retrieval, Visual Question Answering (VQA), Word Similarity

We present a deep neural network that leverages images to improve bilingual text embeddings. Relying on bilingual image tags and descriptions, our approach conditions text embedding induction on the shared visual information for both languages, producing highly correlated bilingual embeddings. In particular, we propose a novel model based on Partial Canonical Correlation Analysis (PCCA). While the original PCCA finds linear projections of two views in order to maximize their canonical correlation conditioned on a shared third variable, we introduce a non-linear Deep PCCA (DPCCA) model, and develop a new stochastic iterative algorithm for its optimization. We evaluate PCCA and DPCCA on multilingual word similarity and cross-lingual image description retrieval. Our models outperform a large variety of previous methods, despite not having access to any visual signal during test time inference.
Illustrative Language Understanding: Large-Scale Visual Grounding with Image Search
Authors: Jamie Kiros, William Chan, Geoffrey Hinton
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: General Classification, Image Retrieval, Machine Translation, Natural Language Inference, Sentence, Topic Classification, Translation, Visual Grounding, Word Embeddings, Word Similarity

We introduce Picturebook, a large-scale lookup operation to ground language via {`}snapshots{'} of our physical world accessed through image search. For each word in a vocabulary, we extract the top-$k$ images from Google image search and feed the images through a convolutional network to extract a word embedding. We introduce a multimodal gating function to fuse our Picturebook embeddings with other word representations. We also introduce Inverse Picturebook, a mechanism to map a Picturebook embedding back into words. We experiment and report results across a wide range of tasks: word similarity, natural language inference, semantic relatedness, sentiment/topic classification, image-sentence ranking and machine translation. We also show that gate activations corresponding to Picturebook embeddings are highly correlated to human judgments of concreteness ratings.
What Action Causes This? Towards Naive Physical Action-Effect Prediction
Authors: Qiaozi Gao, Shaohua Yang, Joyce Chai, V, Lucy erwende
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7

Despite recent advances in knowledge representation, automated reasoning, and machine learning, artificial agents still lack the ability to understand basic action-effect relations regarding the physical world, for example, the action of cutting a cucumber most likely leads to the state where the cucumber is broken apart into smaller pieces. If artificial agents (e.g., robots) ever become our partners in joint tasks, it is critical to empower them with such action-effect understanding so that they can reason about the state of the world and plan for actions. Towards this goal, this paper introduces a new task on naive physical action-effect prediction, which addresses the relations between concrete actions (expressed in the form of verb-noun pairs) and their effects on the state of the physical world as depicted by images. We collected a dataset for this task and developed an approach that harnesses web image data through distant supervision to facilitate learning for action-effect prediction. Our empirical results have shown that web data can be used to complement a small number of seed examples (e.g., three examples for each action) for model learning. This opens up possibilities for agents to learn physical action-effect relations for tasks at hand through communication with humans with a few examples.
Target-Sensitive Memory Networks for Aspect Sentiment Classification
Authors: Shuai Wang, Sahisnu Mazumder, Bing Liu, Mianwei Zhou, Yi Chang
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: Aspect-Based Sentiment Analysis (ABSA), Classification, General Classification, Sentence, Sentiment Analysis, Sentiment Classification

Aspect sentiment classification (ASC) is a fundamental task in sentiment analysis. Given an aspect/target and a sentence, the task classifies the sentiment polarity expressed on the target in the sentence. Memory networks (MNs) have been used for this task recently and have achieved state-of-the-art results. In MNs, attention mechanism plays a crucial role in detecting the sentiment context for the given target. However, we found an important problem with the current MNs in performing the ASC task. Simply improving the attention mechanism will not solve it. The problem is referred to as target-sensitive sentiment, which means that the sentiment polarity of the (detected) context is dependent on the given target and it cannot be inferred from the context alone. To tackle this problem, we propose the target-sensitive memory networks (TMNs). Several alternative techniques are designed for the implementation of TMNs and their effectiveness is experimentally evaluated.
Identifying Transferable Information Across Domains for Cross-domain Sentiment Classification
Authors: Raksha Sharma, Pushpak Bhattacharyya, D, S apat, ipan, Himanshu Sharad Bhatt
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: Classification, Domain Adaptation, domain classification, General Classification, Sentiment Analysis, Sentiment Classification, Transfer Learning

Getting manually labeled data in each domain is always an expensive and a time consuming task. Cross-domain sentiment analysis has emerged as a demanding concept where a labeled source domain facilitates a sentiment classifier for an unlabeled target domain. However, polarity orientation (positive or negative) and the significance of a word to express an opinion often differ from one domain to another domain. Owing to these differences, cross-domain sentiment classification is still a challenging task. In this paper, we propose that words that do not change their polarity and significance represent the transferable (usable) information across domains for cross-domain sentiment classification. We present a novel approach based on Ï‡2 test and cosine-similarity between context vector of words to identify polarity preserving significant words across domains. Furthermore, we show that a weighted ensemble of the classifiers enhances the cross-domain classification performance.
Fluency Boost Learning and Inference for Neural Grammatical Error Correction
Authors: Tao Ge, Furu Wei, Ming Zhou
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: Grammatical Error Correction, Sentence

Most of the neural sequence-to-sequence (seq2seq) models for grammatical error correction (GEC) have two limitations: (1) a seq2seq model may not be well generalized with only limited error-corrected data; (2) a seq2seq model may fail to completely correct a sentence with multiple errors through normal seq2seq inference. We attempt to address these limitations by proposing a fluency boost learning and inference mechanism. Fluency boosting learning generates fluency-boost sentence pairs during training, enabling the error correction model to learn how to improve a sentence{'}s fluency from more instances, while fluency boosting inference allows the model to correct a sentence incrementally with multiple inference steps until the sentence{'}s fluency stops increasing. Experiments show our approaches improve the performance of seq2seq models for GEC, achieving state-of-the-art results on both CoNLL-2014 and JFLEG benchmark datasets.
A Neural Architecture for Automated ICD Coding
Authors: Pengtao Xie, Eric Xing
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: Diagnostic

The International Classification of Diseases (ICD) provides a hierarchy of diagnostic codes for classifying diseases. Medical coding {--} which assigns a subset of ICD codes to a patient visit {--} is a mandatory process that is crucial for patient care and billing. Manual coding is time-consuming, expensive, and error prone. In this paper, we build a neural architecture for automated coding. It takes the diagnosis descriptions (DDs) of a patient as inputs and selects the most relevant ICD codes. This architecture contains four major ingredients: (1) tree-of-sequences LSTM encoding of code descriptions (CDs), (2) adversarial learning for reconciling the different writing styles of DDs and CDs, (3) isotonic constraints for incorporating the importance order among the assigned codes, and (4) attentional matching for performing many-to-one and one-to-many mappings from DDs to CDs. We demonstrate the effectiveness of the proposed methods on a clinical datasets with 59K patient visits.
TDNN: A Two-stage Deep Neural Network for Prompt-independent Automated Essay Scoring
Authors: Cancan Jin, Ben He, Kai Hui, Le Sun
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: Automated Essay Scoring

Existing automated essay scoring (AES) models rely on rated essays for the target prompt as training data. Despite their successes in prompt-dependent AES, how to effectively predict essay ratings under a prompt-independent setting remains a challenge, where the rated essays for the target prompt are not available. To close this gap, a two-stage deep neural network (TDNN) is proposed. In particular, in the first stage, using the rated essays for non-target prompts as the training data, a shallow model is learned to select essays with an extreme quality for the target prompt, serving as pseudo training data; in the second stage, an end-to-end hybrid deep model is proposed to learn a prompt-dependent rating model consuming the pseudo training data from the first step. Evaluation of the proposed TDNN on the standard ASAP dataset demonstrates a promising improvement for the prompt-independent AES task.
Learning to Control the Specificity in Neural Response Generation
Authors: Ruqing Zhang, Jiafeng Guo, Yixing Fan, Yanyan Lan, Jun Xu, Xue-Qi Cheng
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: Machine Translation, Response Generation, Specificity

In conversation, a general response (e.g., {``}I don{'}t know{''}) could correspond to a large variety of input utterances. Previous generative conversational models usually employ a single model to learn the relationship between different utterance-response pairs, thus tend to favor general and trivial responses which appear frequently. To address this problem, we propose a novel controlled response generation mechanism to handle different utterance-response relationships in terms of specificity. Specifically, we introduce an explicit specificity control variable into a sequence-to-sequence model, which interacts with the usage representation of words through a Gaussian Kernel layer, to guide the model to generate responses at different specificity levels. We describe two ways to acquire distant labels for the specificity control variable in learning. Empirical studies show that our model can significantly outperform the state-of-the-art response generation models under both automatic and human evaluations.
Multi-Turn Response Selection for Chatbots with Deep Attention Matching Network
Authors: Xiangyang Zhou, Lu Li, daxiang dong, Yi Liu, Ying Chen, Wayne Xin Zhao, dianhai yu, Hua Wu
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: Chatbot, Conversational Response Selection, Translation

Human generates responses relying on semantic and functional dependencies, including coreference relation, among dialogue elements and their context. In this paper, we investigate matching a response with its multi-turn context using dependency information based entirely on attention. Our solution is inspired by the recently proposed Transformer in machine translation (Vaswani et al., 2017) and we extend the attention mechanism in two ways. First, we construct representations of text segments at different granularities solely with stacked self-attention. Second, we try to extract the truly matched segment pairs with attention across the context and response. We jointly introduce those two kinds of attention in one uniform neural network. Experiments on two large-scale multi-turn response selection tasks show that our proposed model significantly outperforms the state-of-the-art models.
A Framework for Representing Language Acquisition in a Population Setting
Authors: Jordan Kodner, Christopher Cerezo Falco
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: Language Acquisition

Language variation and change are driven both by individuals{'} internal cognitive processes and by the social structures through which language propagates. A wide range of computational frameworks have been proposed to connect these drivers. We compare the strengths and weaknesses of existing approaches and propose a new analytic framework which combines previous network models{'} ability to capture realistic social structure with practically and more elegant computational properties. The framework privileges the process of language acquisition and embeds learners in a social network but is modular so that population structure can be combined with different acquisition models. We demonstrate two applications for the framework: a test of practical concerns that arise when modeling acquisition in a population setting and an application of the framework to recent work on phonological mergers in progress.
Prefix Lexicalization of Synchronous CFGs using Synchronous TAG
Authors: Logan Born, Anoop Sarkar
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: Machine Translation, TAG

We show that an epsilon-free, chain-free synchronous context-free grammar (SCFG) can be converted into a weakly equivalent synchronous tree-adjoining grammar (STAG) which is prefix lexicalized. This transformation at most doubles the grammar{'}s rank and cubes its size, but we show that in practice the size increase is only quadratic. Our results extend Greibach normal form from CFGs to SCFGs and prove new formal properties about SCFG, a formalism with many applications in natural language processing.
Searching for the X-Factor: Exploring Corpus Subjectivity for Word Embeddings
Authors: Maksim Tkachenko, Chong Cher Chia, Hady Lauw
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: General Classification, Sentence, Sentiment Analysis, Sentiment Classification, Word Embeddings

We explore the notion of subjectivity, and hypothesize that word embeddings learnt from input corpora of varying levels of subjectivity behave differently on natural language processing tasks such as classifying a sentence by sentiment, subjectivity, or topic. Through systematic comparative analyses, we establish this to be the case indeed. Moreover, based on the discovery of the outsized role that sentiment words play on subjectivity-sensitive tasks such as sentiment classification, we develop a novel word embedding SentiVec which is infused with sentiment information from a lexical resource, and is shown to outperform baselines on such tasks.
Word Embedding and WordNet Based Metaphor Identification and Interpretation
Authors: Rui Mao, Chenghua Lin, Frank Guerin
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: Decision Making, Machine Translation, Sentence, Sentiment Analysis, Translation, Word Embeddings

Metaphoric expressions are widespread in natural language, posing a significant challenge for various natural language processing tasks such as Machine Translation. Current word embedding based metaphor identification models cannot identify the exact metaphorical words within a sentence. In this paper, we propose an unsupervised learning method that identifies and interprets metaphors at word-level without any preprocessing, outperforming strong baselines in the metaphor identification task. Our model extends to interpret the identified metaphors, paraphrasing them into their literal counterparts, so that they can be better translated by machines. We evaluated this with two popular translation systems for English to Chinese, showing that our model improved the systems significantly.
Incorporating Latent Meanings of Morphological Compositions to Enhance Word Embeddings
Authors: Yang Xu, Jiawei Liu, Wei Yang, Liusheng Huang
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: General Classification, Information Retrieval, Machine Translation, Morphological Analysis, Sentiment Analysis, text-classification, Text Classification, Word Embeddings, Word Similarity

Traditional word embedding approaches learn semantic information at word level while ignoring the meaningful internal structures of words like morphemes. Furthermore, existing morphology-based models directly incorporate morphemes to train word embeddings, but still neglect the latent meanings of morphemes. In this paper, we explore to employ the latent meanings of morphological compositions of words to train and enhance word embeddings. Based on this purpose, we propose three Latent Meaning Models (LMMs), named LMM-A, LMM-S and LMM-M respectively, which adopt different strategies to incorporate the latent meanings of morphemes during the training process. Experiments on word similarity, syntactic analogy and text classification are conducted to validate the feasibility of our models. The results demonstrate that our models outperform the baselines on five word similarity datasets. On Wordsim-353 and RG-65 datasets, our models nearly achieve 5{\%} and 7{\%} gains over the classic CBOW model, respectively. For the syntactic analogy and text classification tasks, our models also surpass all the baselines including a morphology-based model.
Forest-Based Neural Machine Translation
Authors: Chunpeng Ma, Akihiro Tamura, Masao Utiyama, Tiejun Zhao, Eiichiro Sumita
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: Machine Translation, NMT, Translation

Tree-based neural machine translation (NMT) approaches, although achieved impressive performance, suffer from a major drawback: they only use the 1-best parse tree to direct the translation, which potentially introduces translation mistakes due to parsing errors. For statistical machine translation (SMT), forest-based methods have been proven to be effective for solving this problem, while for NMT this kind of approach has not been attempted. This paper proposes a forest-based NMT method that translates a linearized packed forest under a simple sequence-to-sequence framework (i.e., a forest-to-sequence NMT model). The BLEU score of the proposed method is higher than that of the sequence-to-sequence NMT, tree-based NMT, and forest-based SMT systems.
Which Melbourne? Augmenting Geocoding with Maps
Authors: Milan Gritta, Mohammad Taher Pilehvar, Nigel Collier
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7

The purpose of text geolocation is to associate geographic information contained in a document with a set (or sets) of coordinates, either implicitly by using linguistic features and/or explicitly by using geographic metadata combined with heuristics. We introduce a geocoder (location mention disambiguator) that achieves state-of-the-art (SOTA) results on three diverse datasets by exploiting the implicit lexical clues. Moreover, we propose a new method for systematic encoding of geographic metadata to generate two distinct views of the same text. To that end, we introduce the Map Vector (MapVec), a sparse representation obtained by plotting prior geographic probabilities, derived from population figures, on a World Map. We then integrate the implicit (language) and explicit (map) features to significantly improve a range of metrics. We also introduce an open-source dataset for geoparsing of news events covering global disease outbreaks and epidemics to help future evaluation in geoparsing.