pwc-archive/papers-with-abstracts huggingface dataset:
Sequential Attention with Keyword Mask Model for Community-based Question Answering
Authors: Jianxin Yang, Wenge Rong, Libin Shi, Zhang Xiong
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6
Tasks: Answer Selection, Question Answering

In Community-based Question Answering system(CQA), Answer Selection(AS) is a critical task, which focuses on finding a suitable answer within a list of candidate answers. For neural network models, the key issue is how to model the representations of QA text pairs and calculate the interactions between them. We propose a Sequential Attention with Keyword Mask model(SAKM) for CQA to imitate human reading behavior. Question and answer text regard each other as context within keyword-mask attention when encoding the representations, and repeat multiple times(hops) in a sequential style. So the QA pairs capture features and information from both question text and answer text, interacting and improving vector representations iteratively through hops. The flexibility of the model allows to extract meaningful keywords from the sentences and enhance diverse mutual information. We perform on answer selection tasks and multi-level answer ranking tasks. Experiment results demonstrate the superiority of our proposed model on community-based QA datasets.
AttentiveChecker: A Bi-Directional Attention Flow Mechanism for Fact Verification
Authors: Santosh Tokala, Vishal G, Avirup Saha, Niloy Ganguly
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6
Tasks: Claim Verification, Fact Checking, Fact Verification, Retrieval, Sentence

The recently released FEVER dataset provided benchmark results on a fact-checking task in which given a factual claim, the system must extract textual evidence (sets of sentences from Wikipedia pages) that support or refute the claim. In this paper, we present a completely task-agnostic pipelined system, AttentiveChecker, consisting of three homogeneous Bi-Directional Attention Flow (BIDAF) networks, which are multi-layer hierarchical networks that represent the context at different levels of granularity. We are the first to apply to this task a bi-directional attention flow mechanism to obtain a query-aware context representation without early summarization. AttentiveChecker can be used to perform document retrieval, sentence selection, and claim verification. Experiments on the FEVER dataset indicate that AttentiveChecker is able to achieve the state-of-the-art results on the FEVER test set.
Practical, Efficient, and Customizable Active Learning for Named Entity Recognition in the Digital Humanities
Authors: Alex Erdmann, er, David Joseph Wrisley, Benjamin Allen, Christopher Brown, Sophie Cohen-Bod{\'e}n{\`e}s, Micha Elsner, Yukun Feng, Brian Joseph, B{\'e}atrice Joyeux-Prunel, Marie-Catherine de Marneffe
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6
Tasks: Active Learning, General Classification, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER)

Scholars in inter-disciplinary fields like the Digital Humanities are increasingly interested in semantic annotation of specialized corpora. Yet, under-resourced languages, imperfect or noisily structured data, and user-specific classification tasks make it difficult to meet their needs using off-the-shelf models. Manual annotation of large corpora from scratch, meanwhile, can be prohibitively expensive. Thus, we propose an active learning solution for named entity recognition, attempting to maximize a custom model{'}s improvement per additional unit of manual annotation. Our system robustly handles any domain or user-defined label set and requires no external resources, enabling quality named entity recognition for Humanities corpora where such resources are not available. Evaluating on typologically disparate languages and datasets, we reduce required annotation by 20-60{\%} and greatly outperform a competitive active learning baseline.
Doc2hash: Learning Discrete Latent variables for Documents Retrieval
Authors: Yifei Zhang, Hao Zhu
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6
Tasks: Retrieval

Learning to hash via generative model has become a powerful paradigm for fast similarity search in documents retrieval. To get binary representation (i.e., hash codes), the discrete distribution prior (i.e., Bernoulli Distribution) is applied to train the variational autoencoder (VAE). However, the discrete stochastic layer is usually incompatible with the backpropagation in the training stage, and thus causes a gradient flow problem because of non-differentiable operators. The reparameterization trick of sampling from a discrete distribution usually inc non-differentiable operators. In this paper, we propose a method, Doc2hash, that solves the gradient flow problem of the discrete stochastic layer by using continuous relaxation on priors, and trains the generative model in an end-to-end manner to generate hash codes. In qualitative and quantitative experiments, we show the proposed model outperforms other state-of-art methods.
Old is Gold: Linguistic Driven Approach for Entity and Relation Linking of Short Text
Authors: Ahmad Sakor, on, Isaiah o Mulang{'}, Kuldeep Singh, Saeedeh Shekarpour, Maria Esther Vidal, Jens Lehmann, S{\"o}ren Auer
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6
Tasks: Entity Linking, Implicit Relations, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Relation, Relation Linking

Short texts challenge NLP tasks such as named entity recognition, disambiguation, linking and relation inference because they do not provide sufficient context or are partially malformed (e.g. wrt. capitalization, long tail entities, implicit relations). In this work, we present the Falcon approach which effectively maps entities and relations within a short text to its mentions of a background knowledge graph. Falcon overcomes the challenges of short text using a light-weight linguistic approach relying on a background knowledge graph. Falcon performs joint entity and relation linking of a short text by leveraging several fundamental principles of English morphology (e.g. compounding, headword identification) and utilizes an extended knowledge graph created by merging entities and relations from various knowledge sources. It uses the context of entities for finding relations and does not require training data. Our empirical study using several standard benchmarks and datasets show that Falcon significantly outperforms state-of-the-art entity and relation linking for short text query inventories.
An Encoding Strategy Based Word-Character LSTM for Chinese NER
Authors: Wei Liu, Tongge Xu, Qinghua Xu, Jiayu Song, Yueran Zu
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6
Tasks: NER, Segmentation

A recently proposed lattice model has demonstrated that words in character sequence can provide rich word boundary information for character-based Chinese NER model. In this model, word information is integrated into a shortcut path between the start and the end characters of the word. However, the existence of shortcut path may cause the model to degenerate into a partial word-based model, which will suffer from word segmentation errors. Furthermore, the lattice model can not be trained in batches due to its DAG structure. In this paper, we propose a novel word-character LSTM(WC-LSTM) model to add word information into the start or the end character of the word, alleviating the influence of word segmentation errors while obtaining the word boundary information. Four different strategies are explored in our model to encode word information into a fixed-sized representation for efficient batch training. Experiments on benchmark datasets show that our proposed model outperforms other state-of-the-arts models.
Highly Effective Arabic Diacritization using Sequence to Sequence Modeling
Authors: Hamdy Mubarak, Ahmed Abdelali, Hassan Sajjad, Younes Samih, Kareem Darwish
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6
Tasks: Feature Engineering, Machine Translation, text-to-speech, Text to Speech, Translation

Arabic text is typically written without short vowels (or diacritics). However, their presence is required for properly verbalizing Arabic and is hence essential for applications such as text to speech. There are two types of diacritics, namely core-word diacritics and case-endings. Most previous works on automatic Arabic diacritic recovery rely on a large number of manually engineered features, particularly for case-endings. In this work, we present a unified character level sequence-to-sequence deep learning model that recovers both types of diacritics without the use of explicit feature engineering. Specifically, we employ a standard neural machine translation setup on overlapping windows of words (broken down into characters), and then we use voting to select the most likely diacritized form of a word. The proposed model outperforms all previous state-of-the-art systems. Our best settings achieve a word error rate (WER) of 4.49{\%} compared to the state-of-the-art of 12.25{\%} on a standard dataset.
SC-LSTM: Learning Task-Specific Representations in Multi-Task Learning for Sequence Labeling
Authors: Peng Lu, Ting Bai, Philippe Langlais
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6
Tasks: Chunking, Multi-Task Learning, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Part-Of-Speech Tagging

Multi-task learning (MTL) has been studied recently for sequence labeling. Typically, auxiliary tasks are selected specifically in order to improve the performance of a target task. Jointly learning multiple tasks in a way that benefit all of them simultaneously can increase the utility of MTL. In order to do so, we propose a new LSTM cell which contains both shared parameters that can learn from all tasks, and task-specific parameters that can learn task-specific information. We name it a Shared-Cell Long-Short Term Memory SC-LSTM. Experimental results on three sequence labeling benchmarks (named-entity recognition, text chunking, and part-of-speech tagging) demonstrate the effectiveness of our SC-LSTM cell.
A Simple and Robust Approach to Detecting Subject-Verb Agreement Errors
Authors: Simon Flachs, Oph{\'e}lie Lacroix, Marek Rei, Helen Yannakoudakis, Anders S{\o}gaard
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6

While rule-based detection of subject-verb agreement (SVA) errors is sensitive to syntactic parsing errors and irregularities and exceptions to the main rules, neural sequential labelers have a tendency to overfit their training data. We observe that rule-based error generation is less sensitive to syntactic parsing errors and irregularities than error detection and explore a simple, yet efficient approach to getting the best of both worlds: We train neural sequential labelers on the combination of large volumes of silver standard data, obtained through rule-based error generation, and gold standard data. We show that our simple protocol leads to more robust detection of SVA errors on both in-domain and out-of-domain data, as well as in the context of other errors and long-distance dependencies; and across four standard benchmarks, the induced model on average achieves a new state of the art.
Self-Discriminative Learning for Unsupervised Document Embedding
Authors: Hong-You Chen, Chin-Hua Hu, Leila Wehbe, Shou-De Lin
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6
Tasks: Document Classification, Document Embedding, General Classification, Representation Learning

Unsupervised document representation learning is an important task providing pre-trained features for NLP applications. Unlike most previous work which learn the embedding based on self-prediction of the surface of text, we explicitly exploit the inter-document information and directly model the relations of documents in embedding space with a discriminative network and a novel objective. Extensive experiments on both small and large public datasets show the competitiveness of the proposed method. In evaluations on standard document classification, our model has errors that are 5 to 13{\%} lower than state-of-the-art unsupervised embedding models. The reduction in error is even more pronounced in scarce label setting.
Adaptive Convolution for Text Classification
Authors: Byung-Ju Choi, Jun-Hyung Park, SangKeun Lee
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6
Tasks: Classification, General Classification, text-classification, Text Classification

In this paper, we present an adaptive convolution for text classification to give flexibility to convolutional neural networks (CNNs). Unlike traditional convolutions which utilize the same set of filters regardless of different inputs, the adaptive convolution employs adaptively generated convolutional filters conditioned on inputs. We achieve this by attaching filter-generating networks, which are carefully designed to generate input-specific filters, to convolution blocks in existing CNNs. We show the efficacy of our approach in existing CNNs based on the performance evaluation. Our evaluation indicates that all of our baselines achieve performance improvements with adaptive convolutions as much as up to 2.6 percentage point in seven benchmark text classification datasets.
Adversarial Category Alignment Network for Cross-domain Sentiment Classification
Authors: Xiaoye Qu, Zhikang Zou, Yu Cheng, Yang Yang, Pan Zhou
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6
Tasks: Classification, General Classification, Sentiment Analysis, Sentiment Classification

Cross-domain sentiment classification aims to predict sentiment polarity on a target domain utilizing a classifier learned from a source domain. Most existing adversarial learning methods focus on aligning the global marginal distribution by fooling a domain discriminator, without taking category-specific decision boundaries into consideration, which can lead to the mismatch of category-level features. In this work, we propose an adversarial category alignment network (ACAN), which attempts to enhance category consistency between the source domain and the target domain. Specifically, we increase the discrepancy of two polarity classifiers to provide diverse views, locating ambiguous features near the decision boundaries. Then the generator learns to create better features away from the category boundaries by minimizing this discrepancy. Experimental results on benchmark datasets show that the proposed method can achieve state-of-the-art performance and produce more discriminative features.
Target-oriented Opinion Words Extraction with Target-fused Neural Sequence Labeling
Authors: Zhifang Fan, Zhen Wu, Xin-yu Dai, Shu-Jian Huang, Jia-Jun Chen
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6
Tasks: Aspect-Based Sentiment Analysis, Aspect-Based Sentiment Analysis (ABSA), Aspect-oriented  Opinion Extraction, Opinion Summarization, Sentiment Analysis, target-oriented opinion words extraction

Opinion target extraction and opinion words extraction are two fundamental subtasks in Aspect Based Sentiment Analysis (ABSA). Recently, many methods have made progress on these two tasks. However, few works aim at extracting opinion targets and opinion words as pairs. In this paper, we propose a novel sequence labeling subtask for ABSA named TOWE (Target-oriented Opinion Words Extraction), which aims at extracting the corresponding opinion words for a given opinion target. A target-fused sequence labeling neural network model is designed to perform this task. The opinion target information is well encoded into context by an Inward-Outward LSTM. Then left and right contexts of the opinion target and the global context are combined to find the corresponding opinion words. We build four datasets for TOWE based on several popular ABSA benchmarks from laptop and restaurant reviews. The experimental results show that our proposed model outperforms the other compared methods significantly. We believe that our work may not only be helpful for downstream sentiment analysis task, but can also be used for pair-wise opinion summarization.
Data-efficient Neural Text Compression with Interactive Learning
Authors: Avinesh P. V. S, Christian M. Meyer
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6
Tasks: Active Learning, Headline Generation, Small Data Image Classification, Text Compression

Neural sequence-to-sequence models have been successfully applied to text compression. However, these models were trained on huge automatically induced parallel corpora, which are only available for a few domains and tasks. In this paper, we propose a novel interactive setup to neural text compression that enables transferring a model to new domains and compression tasks with minimal human supervision. This is achieved by employing active learning, which intelligently samples from a large pool of unlabeled data. Using this setup, we can successfully adapt a model trained on small data of 40k samples for a headline generation task to a general text compression dataset at an acceptable compression quality with just 500 sampled instances annotated by a human.
The World in My Mind: Visual Dialog with Adversarial Multi-modal Feature Encoding
Authors: Yiqun Yao, Jiaming Xu, Bo Xu
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6
Tasks: General Knowledge, Visual Dialog

Visual Dialog is a multi-modal task that requires a model to participate in a multi-turn human dialog grounded on an image, and generate correct, human-like responses. In this paper, we propose a novel Adversarial Multi-modal Feature Encoding (AMFE) framework for effective and robust auxiliary training of visual dialog systems. AMFE can force the language-encoding part of a model to generate hidden states in a distribution closely related to the distribution of real-world images, resulting in language features containing general knowledge from both modalities by nature, which can help generate both more correct and more general responses with reasonably low time cost. Experimental results show that AMFE can steadily bring performance gains to different models on different scales of data. Our method outperforms both the supervised learning baselines and other fine-tuning methods, achieving state-of-the-art results on most metrics of VisDial v0.5/v0.9 generative tasks.
Iterative Search for Weakly Supervised Semantic Parsing
Authors: Pradeep Dasigi, Matt Gardner, Shikhar Murty, Luke Zettlemoyer, Eduard Hovy
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6
Tasks: Semantic Parsing, Visual Reasoning

Training semantic parsers from question-answer pairs typically involves searching over an exponentially large space of logical forms, and an unguided search can easily be misled by spurious logical forms that coincidentally evaluate to the correct answer. We propose a novel iterative training algorithm that alternates between searching for consistent logical forms and maximizing the marginal likelihood of the retrieved ones. This training scheme lets us iteratively train models that provide guidance to subsequent ones to search for logical forms of increasing complexity, thus dealing with the problem of spuriousness. We evaluate these techniques on two hard datasets: WikiTableQuestions (WTQ) and Cornell Natural Language Visual Reasoning (NLVR), and show that our training algorithm outperforms the previous best systems, on WTQ in a comparable setting, and on NLVR with significantly less supervision.
Alignment over Heterogeneous Embeddings for Question Answering
Authors: Vikas Yadav, Steven Bethard, Mihai Surdeanu
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6
Tasks: AI2 Reasoning Challenge, ARC, Question Answering, Sentence, Sentence Embeddings

We propose a simple, fast, and mostly-unsupervised approach for non-factoid question answering (QA) called Alignment over Heterogeneous Embeddings (AHE). AHE simply aligns each word in the question and candidate answer with the most similar word in the retrieved supporting paragraph, and weighs each alignment score with the inverse document frequency of the corresponding question/answer term. AHE{'}s similarity function operates over embeddings that model the underlying text at different levels of abstraction: character (FLAIR), word (BERT and GloVe), and sentence (InferSent), where the latter is the only supervised component in the proposed approach. Despite its simplicity and lack of supervision, AHE obtains a new state-of-the-art performance on the {``}Easy{''} partition of the AI2 Reasoning Challenge (ARC) dataset (64.6{\%} accuracy), top-two performance on the {``}Challenge{''} partition of ARC (34.1{\%}), and top-three performance on the WikiQA dataset (74.08{\%} MRR), outperforming many other complex, supervised approaches. Our error analysis indicates that alignments over character, word, and sentence embeddings capture substantially different semantic information. We exploit this with a simple meta-classifier that learns how much to trust the predictions over each representation, which further improves the performance of unsupervised AHE.
Incorporating Word Attention into Character-Based Word Segmentation
Authors: Shohei Higashiyama, Masao Utiyama, Eiichiro Sumita, Masao Ideuchi, Yoshiaki Oida, Yohei Sakamoto, Isaac Okada
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6
Tasks: Feature Engineering, Japanese Word Segmentation, Segmentation

Neural network models have been actively applied to word segmentation, especially Chinese, because of the ability to minimize the effort in feature engineering. Typical segmentation models are categorized as character-based, for conducting exact inference, or word-based, for utilizing word-level information. We propose a character-based model utilizing word information to leverage the advantages of both types of models. Our model learns the importance of multiple candidate words for a character on the basis of an attention mechanism, and makes use of it for segmentation decisions. The experimental results show that our model achieves better performance than the state-of-the-art models on both Japanese and Chinese benchmark datasets.
Shrinking Japanese Morphological Analyzers With Neural Networks and Semi-supervised Learning
Authors: Arseny Tolmachev, Daisuke Kawahara, Sadao Kurohashi
Date: 2019-06-01 00:00:00
Proceeding: NAACL 2019 6
Tasks: Chinese Word Segmentation, Morphological Analysis, Part-Of-Speech Tagging, Segmentation

For languages without natural word boundaries, like Japanese and Chinese, word segmentation is a prerequisite for downstream analysis. For Japanese, segmentation is often done jointly with part of speech tagging, and this process is usually referred to as morphological analysis. Morphological analyzers are trained on data hand-annotated with segmentation boundaries and part of speech tags. A segmentation dictionary or character n-gram information is also provided as additional inputs to the model. Incorporating this extra information makes models large. Modern neural morphological analyzers can consume gigabytes of memory. We propose a compact alternative to these cumbersome approaches which do not rely on any externally provided n-gram or word representations. The model uses only unigram character embeddings, encodes them using either stacked bi-LSTM or a self-attention network, and independently infers both segmentation and part of speech information. The model is trained in an end-to-end and semi-supervised fashion, on labels produced by a state-of-the-art analyzer. We demonstrate that the proposed technique rivals performance of a previous dictionary-based state-of-the-art approach and can even surpass it when training with the combination of human-annotated and automatically-annotated data. Our model itself is significantly smaller than the dictionary-based one: it uses less than 15 megabytes of space.