pwc-archive/papers-with-abstracts huggingface dataset:
A Mixture of h - 1 Heads is Better than h Heads
Authors: Hao Peng, Roy Schwartz, Dianqi Li, Noah A. Smith
Date: 2020-07-01 00:00:00
Proceeding: ACL 2020 6
Tasks: Language Modeling, Language Modelling, Machine Translation, Mixture-of-Experts, Translation

Multi-head attentive neural architectures have achieved state-of-the-art results on a variety of natural language processing tasks. Evidence has shown that they are overparameterized; attention heads can be pruned without significant performance loss. In this work, we instead {``}reallocate{''} them{---}the model learns to activate different heads on different inputs. Drawing connections between multi-head attention and mixture of experts, we propose the mixture of attentive experts model (MAE). MAE is trained using a block coordinate descent algorithm that alternates between updating (1) the responsibilities of the experts and (2) their parameters. Experiments on machine translation and language modeling show that MAE outperforms strong baselines on both tasks. Particularly, on the WMT14 English to German translation dataset, MAE improves over {``}transformer-base{''} by 0.8 BLEU, with a comparable number of parameters. Our analysis shows that our model learns to specialize different experts to different inputs.
Dependency Graph Enhanced Dual-transformer Structure for Aspect-based Sentiment Classification
Authors: Hao Tang, Donghong Ji, Chenliang Li, Qiji Zhou
Date: 2020-07-01 00:00:00
Proceeding: ACL 2020 6
Tasks: General Classification, Representation Learning, Sentence, Sentiment Analysis, Sentiment Classification

Aspect-based sentiment classification is a popular task aimed at identifying the corresponding emotion of a specific aspect. One sentence may contain various sentiments for different aspects. Many sophisticated methods such as attention mechanism and Convolutional Neural Networks (CNN) have been widely employed for handling this challenge. Recently, semantic dependency tree implemented by Graph Convolutional Networks (GCN) is introduced to describe the inner connection between aspects and the associated emotion words. But the improvement is limited due to the noise and instability of dependency trees. To this end, we propose a dependency graph enhanced dual-transformer network (named DGEDT) by jointly considering the flat representations learnt from Transformer and graph-based representations learnt from the corresponding dependency graph in an iterative interaction manner. Specifically, a dual-transformer structure is devised in DGEDT to support mutual reinforcement between the flat representation learning and graph-based representation learning. The idea is to allow the dependency graph to guide the representation learning of the transformer encoder and vice versa. The results on five datasets demonstrate that the proposed DGEDT outperforms all state-of-the-art alternatives with a large margin.
Evaluating and Enhancing the Robustness of Neural Network-based Dependency Parsing Models with Adversarial Examples
Authors: Xiaoqing Zheng, Jiehang Zeng, Yi Zhou, Cho-Jui Hsieh, Minhao Cheng, Xuanjing Huang
Date: 2020-07-01 00:00:00
Proceeding: ACL 2020 6
Tasks: Dependency Parsing, Question Answering, Reading Comprehension, Sentence, Sentiment Analysis

Despite achieving prominent performance on many important tasks, it has been reported that neural networks are vulnerable to adversarial examples. Previously studies along this line mainly focused on semantic tasks such as sentiment analysis, question answering and reading comprehension. In this study, we show that adversarial examples also exist in dependency parsing: we propose two approaches to study where and how parsers make mistakes by searching over perturbations to existing texts at sentence and phrase levels, and design algorithms to construct such examples in both of the black-box and white-box settings. Our experiments with one of state-of-the-art parsers on the English Penn Treebank (PTB) show that up to 77{\%} of input examples admit adversarial perturbations, and we also show that the robustness of parsing models can be improved by crafting high-quality adversaries and including them in the training stage, while suffering little to no performance drop on the clean input data.
Modeling Morphological Typology for Unsupervised Learning of Language Morphology
Authors: Hongzhi Xu, Jordan Kodner, Mitchell Marcus, Charles Yang
Date: 2020-07-01 00:00:00
Proceeding: ACL 2020 6
Tasks: Morphological Analysis

This paper describes a language-independent model for fully unsupervised morphological analysis that exploits a universal framework leveraging morphological typology. By modeling morphological processes including suffixation, prefixation, infixation, and full and partial reduplication with constrained stem change rules, our system effectively constrains the search space and offers a wide coverage in terms of morphological typology. The system is tested on nine typologically and genetically diverse languages, and shows superior performance over leading systems. We also investigate the effect of an oracle that provides only a handful of bits per language to signal morphological type.
Low-Resource Generation of Multi-hop Reasoning Questions
Authors: Jianxing Yu, Wei Liu, Shuang Qiu, Qinliang Su, Kai Wang, Xiaojun Quan, Jian Yin
Date: 2020-07-01 00:00:00
Proceeding: ACL 2020 6
Tasks: Machine Reading Comprehension, Reading Comprehension, valid

This paper focuses on generating multi-hop reasoning questions from the raw text in a low resource circumstance. Such questions have to be syntactically valid and need to logically correlate with the answers by deducing over multiple relations on several sentences in the text. Specifically, we first build a multi-hop generation model and guide it to satisfy the logical rationality by the reasoning chain extracted from a given text. Since the labeled data is limited and insufficient for training, we propose to learn the model with the help of a large scale of unlabeled data that is much easier to obtain. Such data contains rich expressive forms of the questions with structural patterns on syntax and semantics. These patterns can be estimated by the neural hidden semi-Markov model using latent variables. With latent patterns as a prior, we can regularize the generation model and produce the optimal results. Experimental results on the HotpotQA data set demonstrate the effectiveness of our model. Moreover, we apply the generated results to the task of machine reading comprehension and achieve significant performance improvements.
Parsing into Variable-in-situ Logico-Semantic Graphs
Authors: Yufei Chen, Weiwei Sun
Date: 2020-07-01 00:00:00
Proceeding: ACL 2020 6
Tasks: Negation, valid

We propose variable-in-situ logico-semantic graphs to bridge the gap between semantic graph and logical form parsing. The new type of graph-based meaning representation allows us to include analysis for scope-related phenomena, such as quantification, negation and modality, in a way that is consistent with the state-of-the-art underspecification approach. Moreover, the well-formedness of such a graph is clear, since model-theoretic interpretation is available. We demonstrate the effectiveness of this new perspective by developing a new state-of-the-art semantic parser for English Resource Semantics. At the core of this parser is a novel neural graph rewriting system which combines the strengths of Hyperedge Replacement Grammar, a knowledge-intensive model, and Graph Neural Networks, a data-intensive model. Our parser achieves an accuracy of 92.39{\%} in terms of elementary dependency match, which is a 2.88 point improvement over the best data-driven model in the literature. The output of our parser is highly coherent: at least 91{\%} graphs are valid, in that they allow at least one sound scope-resolved logical form.
Semantic Parsing for English as a Second Language
Authors: Yuanyuan Zhao, Weiwei Sun, Junjie Cao, Xiaojun Wan
Date: 2020-07-01 00:00:00
Proceeding: ACL 2020 6
Tasks: Grammatical Error Correction, Language Acquisition, Reranking, Semantic Parsing

This paper is concerned with semantic parsing for English as a second language (ESL). Motivated by the theoretical emphasis on the learning challenges that occur at the syntax-semantics interface during second language acquisition, we formulate the task based on the divergence between literal and intended meanings. We combine the complementary strengths of English Resource Grammar, a linguistically-precise hand-crafted deep grammar, and TLE, an existing manually annotated ESL UD-TreeBank with a novel reranking model. Experiments demonstrate that in comparison to human annotations, our method can obtain a very promising SemBanking quality. By means of the newly created corpus, we evaluate state-of-the-art semantic parsing as well as grammatical error correction models. The evaluation profiles the performance of neural NLP techniques for handling ESL data and suggests some research directions.
Semi-Supervised Semantic Dependency Parsing Using CRF Autoencoders
Authors: Zixia Jia, Youmi Ma, Jiong Cai, Kewei Tu
Date: 2020-07-01 00:00:00
Proceeding: ACL 2020 6
Tasks: ARC, Decoder, Dependency Parsing, Semantic Dependency Parsing, Sentence

Semantic dependency parsing, which aims to find rich bi-lexical relationships, allows words to have multiple dependency heads, resulting in graph-structured representations. We propose an approach to semi-supervised learning of semantic dependency parsers based on the CRF autoencoder framework. Our encoder is a discriminative neural semantic dependency parser that predicts the latent parse graph of the input sentence. Our decoder is a generative neural model that reconstructs the input sentence conditioned on the latent parse graph. Our model is arc-factored and therefore parsing and learning are both tractable. Experiments show our model achieves significant and consistent improvement over the supervised baseline.
Document Translation vs. Query Translation for Cross-Lingual Information Retrieval in the Medical Domain
Authors: Shadi Saleh, Pavel Pecina
Date: 2020-07-01 00:00:00
Proceeding: ACL 2020 6
Tasks: Cross-Lingual Information Retrieval, Document Translation, Information Retrieval, Machine Translation, NMT, Retrieval, Translation

We present a thorough comparison of two principal approaches to Cross-Lingual Information Retrieval: document translation (DT) and query translation (QT). Our experiments are conducted using the cross-lingual test collection produced within the CLEF eHealth information retrieval tasks in 2013{--}2015 containing English documents and queries in several European languages. We exploit the Statistical Machine Translation (SMT) and Neural Machine Translation (NMT) paradigms and train several domain-specific and task-specific machine translation systems to translate the non-English queries into English (for the QT approach) and the English documents to all the query languages (for the DT approach). The results show that the quality of QT by SMT is sufficient enough to outperform the retrieval results of the DT approach for all the languages. NMT then further boosts translation quality and retrieval quality for both QT and DT for most languages, but still, QT provides generally better retrieval results than DT.
Classification-Based Self-Learning for Weakly Supervised Bilingual Lexicon Induction
Authors: Mladen Karan, Ivan Vuli{\'c}, Anna Korhonen, Goran Glava{\v{s}}
Date: 2020-07-01 00:00:00
Proceeding: ACL 2020 6
Tasks: Bilingual Lexicon Induction, Classification, General Classification, Self-Learning, Translation

Effective projection-based cross-lingual word embedding (CLWE) induction critically relies on the iterative self-learning procedure. It gradually expands the initial small seed dictionary to learn improved cross-lingual mappings. In this work, we present ClassyMap, a classification-based approach to self-learning, yielding a more robust and a more effective induction of projection-based CLWEs. Unlike prior self-learning methods, our approach allows for integration of diverse features into the iterative process. We show the benefits of ClassyMap for bilingual lexicon induction: we report consistent improvements in a weakly supervised setup (500 seed translation pairs) on a benchmark with 28 language pairs.
Uncertainty-Aware Curriculum Learning for Neural Machine Translation
Authors: Yikai Zhou, Baosong Yang, Derek F. Wong, Yu Wan, Lidia S. Chao
Date: 2020-07-01 00:00:00
Proceeding: ACL 2020 6
Tasks: Machine Translation, NMT, Translation

Neural machine translation (NMT) has proven to be facilitated by curriculum learning which presents examples in an easy-to-hard order at different training stages. The keys lie in the assessment of data difficulty and model competence. We propose uncertainty-aware curriculum learning, which is motivated by the intuition that: 1) the higher the uncertainty in a translation pair, the more complex and rarer the information it contains; and 2) the end of the decline in model uncertainty indicates the completeness of current training stage. Specifically, we serve cross-entropy of an example as its data difficulty and exploit the variance of distributions over the weights of the network to present the model uncertainty. Extensive experiments on various translation tasks reveal that our approach outperforms the strong baseline and related methods on both translation quality and convergence speed. Quantitative analyses reveal that the proposed strategy offers NMT the ability to automatically govern its learning schedule.
CorefQA: Coreference Resolution as Query-based Span Prediction
Authors: Wei Wu, Fei Wang, Arianna Yuan, Fei Wu, Jiwei Li
Date: 2020-07-01 00:00:00
Proceeding: ACL 2020 6
Tasks: coreference-resolution, Coreference Resolution, Data Augmentation, Prediction, Question Answering

In this paper, we present CorefQA, an accurate and extensible approach for the coreference resolution task. We formulate the problem as a span prediction task, like in question answering: A query is generated for each candidate mention using its surrounding context, and a span prediction module is employed to extract the text spans of the coreferences within the document using the generated query. This formulation comes with the following key advantages: (1) The span prediction strategy provides the flexibility of retrieving mentions left out at the mention proposal stage; (2) In the question answering framework, encoding the mention and its context explicitly in a query makes it possible to have a deep and thorough examination of cues embedded in the context of coreferent mentions; and (3) A plethora of existing question answering datasets can be used for data augmentation to improve the model{'}s generalization capability. Experiments demonstrate significant performance boost over previous models, with 83.1 (+3.5) F1 score on the CoNLL-2012 benchmark and 87.5 (+2.5) F1 score on the GAP benchmark.
From Zero to Hero: Human-In-The-Loop Entity Linking in Low Resource Domains
Authors: Jan-Christoph Klie, Richard Eckart de Castilho, Iryna Gurevych
Date: 2020-07-01 00:00:00
Proceeding: ACL 2020 6
Tasks: Entity Linking, text annotation

Entity linking (EL) is concerned with disambiguating entity mentions in a text against knowledge bases (KB). It is crucial in a considerable number of fields like humanities, technical writing and biomedical sciences to enrich texts with semantics and discover more knowledge. The use of EL in such domains requires handling noisy texts, low resource settings and domain-specific KBs. Existing approaches are mostly inappropriate for this, as they depend on training data. However, in the above scenario, there exists hardly annotated data, and it needs to be created from scratch. We therefore present a novel domain-agnostic Human-In-The-Loop annotation approach: we use recommenders that suggest potential concepts and adaptive candidate ranking, thereby speeding up the overall annotation process and making it less tedious for users. We evaluate our ranking approach in a simulation on difficult texts and show that it greatly outperforms a strong baseline in ranking accuracy. In a user study, the annotation speed improves by 35{\%} compared to annotating without interactive support; users report that they strongly prefer our system. An open-source and ready-to-use implementation based on the text annotation platform INCEpTION (https://inception-project.github.io) is made available.
Language to Network: Conditional Parameter Adaptation with Natural Language Descriptions
Authors: Tian Jin, Zhun Liu, Shengjia Yan, Alex Eichenberger, re, Louis-Philippe Morency
Date: 2020-07-01 00:00:00
Proceeding: ACL 2020 6
Tasks: General Classification, image-classification, Image Classification, Transfer Learning, Zero-Shot Image Classification, Zero-Shot Learning

Transfer learning using ImageNet pre-trained models has been the de facto approach in a wide range of computer vision tasks. However, fine-tuning still requires task-specific training data. In this paper, we propose \textbf{N3} (\textbf{N}eural \textbf{N}etworks from \textbf{N}atural Language) - a new paradigm of synthesizing task-specific neural networks from language descriptions and a generic pre-trained model. \textbf{N3} leverages language descriptions to generate parameter adaptations as well as a new task-specific classification layer for a pre-trained neural network, effectively {``}fine-tuning{''} the network for a new task using only language descriptions as input. To the best of our knowledge, \textbf{N3} is the first method to synthesize entire neural networks from natural language. Experimental results show that \textbf{N3} can out-perform previous natural-language based zero-shot learning methods across 4 different zero-shot image classification benchmarks. We also demonstrate a simple method to help identify keywords in language descriptions leveraged by \textbf{N3} when synthesizing model parameters.
Transition-based Semantic Dependency Parsing with Pointer Networks
Authors: Daniel Fern{\'a}ndez-Gonz{\'a}lez, Carlos G{\'o}mez-Rodr{\'\i}guez
Date: 2020-07-01 00:00:00
Proceeding: ACL 2020 6
Tasks: Dependency Parsing, Semantic Dependency Parsing, Word Embeddings

Transition-based parsers implemented with Pointer Networks have become the new state of the art in dependency parsing, excelling in producing labelled syntactic trees and outperforming graph-based models in this task. In order to further test the capabilities of these powerful neural networks on a harder NLP problem, we propose a transition system that, thanks to Pointer Networks, can straightforwardly produce labelled directed acyclic graphs and perform semantic dependency parsing. In addition, we enhance our approach with deep contextualized word embeddings extracted from BERT. The resulting system not only outperforms all existing transition-based models, but also matches the best fully-supervised accuracy to date on the SemEval 2015 Task 18 datasets among previous state-of-the-art graph-based parsers.
tBERT: Topic Models and BERT Joining Forces for Semantic Similarity Detection
Authors: Nicole Peinelt, Dong Nguyen, Maria Liakata
Date: 2020-07-01 00:00:00
Proceeding: ACL 2020 6
Tasks: Natural Language Understanding, Semantic Similarity, Semantic Textual Similarity, Topic Models

Semantic similarity detection is a fundamental task in natural language understanding. Adding topic information has been useful for previous feature-engineered semantic similarity models as well as neural models for other tasks. There is currently no standard way of combining topics with pretrained contextual representations such as BERT. We propose a novel topic-informed BERT-based architecture for pairwise semantic similarity detection and show that our model improves performance over strong neural baselines across a variety of English language datasets. We find that the addition of topics to BERT helps particularly with resolving domain-specific cases.
Exploiting Personal Characteristics of Debaters for Predicting Persuasiveness
Authors: Khalid Al Khatib, Michael V{\"o}lske, Shahbaz Syed, Nikolay Kolyada, Benno Stein
Date: 2020-07-01 00:00:00
Proceeding: ACL 2020 6
Tasks: Persuasiveness, Prediction

Predicting the persuasiveness of arguments has applications as diverse as writing assistance, essay scoring, and advertising. While clearly relevant to the task, the personal characteristics of an argument{'}s source and audience have not yet been fully exploited toward automated persuasiveness prediction. In this paper, we model debaters{'} prior beliefs, interests, and personality traits based on their previous activity, without dependence on explicit user profiles or questionnaires. Using a dataset of over 60,000 argumentative discussions, comprising more than three million individual posts collected from the subreddit r/ChangeMyView, we demonstrate that our modeling of debater{'}s characteristics enhances the prediction of argument persuasiveness as well as of debaters{'} resistance to persuasion.
Meta-Reinforced Multi-Domain State Generator for Dialogue Systems
Authors: Yi Huang, Junlan Feng, Min Hu, Xiaoting Wu, Xiaoyu Du, Shuo Ma
Date: 2020-07-01 00:00:00
Proceeding: ACL 2020 6
Tasks: Domain Adaptation, Meta-Learning, Reinforcement Learning (RL)

A Dialogue State Tracker (DST) is a core component of a modular task-oriented dialogue system. Tremendous progress has been made in recent years. However, the major challenges remain. The state-of-the-art accuracy for DST is below 50{\%} for a multi-domain dialogue task. A learnable DST for any new domain requires a large amount of labeled in-domain data and training from scratch. In this paper, we propose a Meta-Reinforced Multi-Domain State Generator (MERET). Our first contribution is to improve the DST accuracy. We enhance a neural model based DST generator with a reward manager, which is built on policy gradient reinforcement learning (RL) to fine-tune the generator. With this change, we are able to improve the joint accuracy of DST from 48.79{\%} to 50.91{\%} on the MultiWOZ corpus. Second, we explore to train a DST meta-learning model with a few domains as source domains and a new domain as target domain. We apply the model-agnostic meta-learning algorithm (MAML) to DST and the obtained meta-learning model is used for new domain adaptation. Our experimental results show this solution is able to outperform the traditional training approach with extremely less training data in target domain.
Heterogeneous Graph Transformer for Graph-to-Sequence Learning
Authors: Shaowei Yao, Tianming Wang, Xiaojun Wan
Date: 2020-07-01 00:00:00
Proceeding: ACL 2020 6
Tasks: AMR-to-Text Generation, Graph-to-Sequence, Machine Translation, Text Generation, Translation

The graph-to-sequence (Graph2Seq) learning aims to transduce graph-structured representations to word sequences for text generation. Recent studies propose various models to encode graph structure. However, most previous works ignore the indirect relations between distance nodes, or treat indirect relations and direct relations in the same way. In this paper, we propose the Heterogeneous Graph Transformer to independently model the different relations in the individual subgraphs of the original graph, including direct relations, indirect relations and multiple possible relations between nodes. Experimental results show that our model strongly outperforms the state of the art on all four standard benchmarks of AMR-to-text generation and syntax-based neural machine translation.