pwc-archive/papers-with-abstracts huggingface dataset:
Graph-to-Tree Learning for Solving Math Word Problems
Authors: Jipeng Zhang, Lei Wang, Roy Ka-Wei Lee, Yi Bin, Yan Wang, Jie Shao, Ee-Peng Lim
Date: 2020-07-01 00:00:00
Proceeding: ACL 2020 6
Tasks: Decoder, Math, Math Word Problem Solving

While the recent tree-based neural models have demonstrated promising results in generating solution expression for the math word problem (MWP), most of these models do not capture the relationships and order information among the quantities well. This results in poor quantity representations and incorrect solution expressions. In this paper, we propose Graph2Tree, a novel deep learning architecture that combines the merits of the graph-based encoder and tree-based decoder to generate better solution expressions. Included in our Graph2Tree framework are two graphs, namely the Quantity Cell Graph and Quantity Comparison Graph, which are designed to address limitations of existing methods by effectively representing the relationships and order information among the quantities in MWPs. We conduct extensive experiments on two available datasets. Our experiment results show that Graph2Tree outperforms the state-of-the-art baselines on two benchmark datasets significantly. We also discuss case studies and empirically examine Graph2Tree{'}s effectiveness in translating the MWP text into solution expressions.
Adaptive Compression of Word Embeddings
Authors: Yeachan Kim, Kang-Min Kim, SangKeun Lee
Date: 2020-07-01 00:00:00
Proceeding: ACL 2020 6
Tasks: Self-Driving Cars, Word Embeddings

Distributed representations of words have been an indispensable component for natural language processing (NLP) tasks. However, the large memory footprint of word embeddings makes it challenging to deploy NLP models to memory-constrained devices (e.g., self-driving cars, mobile devices). In this paper, we propose a novel method to adaptively compress word embeddings. We fundamentally follow a code-book approach that represents words as discrete codes such as (8, 5, 2, 4). However, unlike prior works that assign the same length of codes to all words, we adaptively assign different lengths of codes to each word by learning downstream tasks. The proposed method works in two steps. First, each word directly learns to select its code length in an end-to-end manner by applying the Gumbel-softmax tricks. After selecting the code length, each word learns discrete codes through a neural network with a binary constraint. To showcase the general applicability of the proposed method, we evaluate the performance on four different downstream tasks. Comprehensive evaluation results clearly show that our method is effective and makes the highly compressed word embeddings without hurting the task accuracy. Moreover, we show that our model assigns word to each code-book by considering the significance of tasks.
Autoencoding Keyword Correlation Graph for Document Clustering
Authors: Billy Chiu, Sunil Kumar Sahu, Derek Thomas, Neha Sengupta, Mohammady Mahdy
Date: 2020-07-01 00:00:00
Proceeding: ACL 2020 6
Tasks: Clustering, Representation Learning

Document clustering requires a deep understanding of the complex structure of long-text; in particular, the intra-sentential (local) and inter-sentential features (global). Existing representation learning models do not fully capture these features. To address this, we present a novel graph-based representation for document clustering that builds a \textit{graph autoencoder} (GAE) on a Keyword Correlation Graph. The graph is constructed with topical keywords as nodes and multiple local and global features as edges. A GAE is employed to aggregate the two sets of features by learning a latent representation which can jointly reconstruct them. Clustering is then performed on the learned representations, using vector dimensions as features for inducing document classes. Extensive experiments on two datasets show that the features learned by our approach can achieve better clustering performance than other existing features, including term frequency-inverse document frequency and average embedding.
CluBERT: A Cluster-Based Approach for Learning Sense Distributions in Multiple Languages
Authors: Tommaso Pasini, Federico Scozzafava, Bianca Scarlini
Date: 2020-07-01 00:00:00
Proceeding: ACL 2020 6
Tasks: LEMMA, Word Sense Disambiguation

Knowing the Most Frequent Sense (MFS) of a word has been proved to help Word Sense Disambiguation (WSD) models significantly. However, the scarcity of sense-annotated data makes it difficult to induce a reliable and high-coverage distribution of the meanings in a language vocabulary. To address this issue, in this paper we present CluBERT, an automatic and multilingual approach for inducing the distributions of word senses from a corpus of raw sentences. Our experiments show that CluBERT learns distributions over English senses that are of higher quality than those extracted by alternative approaches. When used to induce the MFS of a lemma, CluBERT attains state-of-the-art results on the English Word Sense Disambiguation tasks and helps to improve the disambiguation performance of two off-the-shelf WSD models. Moreover, our distributions also prove to be effective in other languages, beating all their alternatives for computing the MFS on the multilingual WSD tasks. We release our sense distributions in five different languages at https://github.com/SapienzaNLP/clubert.
Adversarial and Domain-Aware BERT for Cross-Domain Sentiment Analysis
Authors: Chunning Du, Haifeng Sun, Jingyu Wang, Qi Qi, Jianxin Liao
Date: 2020-07-01 00:00:00
Proceeding: ACL 2020 6
Tasks: Domain Adaptation, Language Modeling, Language Modelling, Sentiment Analysis, Sentiment Classification, Unsupervised Domain Adaptation

Cross-domain sentiment classification aims to address the lack of massive amounts of labeled data. It demands to predict sentiment polarity on a target domain utilizing a classifier learned from a source domain. In this paper, we investigate how to efficiently apply the pre-training language model BERT on the unsupervised domain adaptation. Due to the pre-training task and corpus, BERT is task-agnostic, which lacks domain awareness and can not distinguish the characteristic of source and target domain when transferring knowledge. To tackle these problems, we design a post-training procedure, which contains the target domain masked language model task and a novel domain-distinguish pre-training task. The post-training procedure will encourage BERT to be domain-aware and distill the domain-specific features in a self-supervised way. Based on this, we could then conduct the adversarial training to derive the enhanced domain-invariant features. Extensive experiments on Amazon dataset show that our model outperforms state-of-the-art methods by a large margin. The ablation study demonstrates that the remarkable improvement is not only from BERT but also from our method.
He said ``who's gonna take care of your children when you are at ACL?'': Reported Sexist Acts are Not Sexist
Authors: Patricia Chiril, V{\'e}ronique Moriceau, Farah Benamara, Alda Mari, Gloria Origgi, Marl{\`e}ne Coulomb-Gully
Date: 2020-07-01 00:00:00
Proceeding: ACL 2020 6
Tasks: Word Embeddings

In a context of offensive content mediation on social media now regulated by European laws, it is important not only to be able to automatically detect sexist content but also to identify if a message with a sexist content is really sexist or is a story of sexism experienced by a woman. We propose: (1) a new characterization of sexist content inspired by speech acts theory and discourse analysis studies, (2) the first French dataset annotated for sexism detection, and (3) a set of deep learning experiments trained on top of a combination of several tweet{'}s vectorial representations (word embeddings, linguistic features, and various generalization strategies). Our results are encouraging and constitute a first step towards offensive content moderation.
Enriched In-Order Linearization for Faster Sequence-to-Sequence Constituent Parsing
Authors: Daniel Fern{\'a}ndez-Gonz{\'a}lez, Carlos G{\'o}mez-Rodr{\'\i}guez
Date: 2020-07-01 00:00:00
Proceeding: ACL 2020 6

Sequence-to-sequence constituent parsing requires a linearization to represent trees as sequences. Top-down tree linearizations, which can be based on brackets or shift-reduce actions, have achieved the best accuracy to date. In this paper, we show that these results can be improved by using an in-order linearization instead. Based on this observation, we implement an enriched in-order shift-reduce linearization inspired by Vinyals et al. (2015){'}s approach, achieving the best accuracy to date on the English PTB dataset among fully-supervised single-model sequence-to-sequence constituent parsers. Finally, we apply deterministic attention mechanisms to match the speed of state-of-the-art transition-based parsers, thus showing that sequence-to-sequence models can match them, not only in accuracy, but also in speed.
Exact yet Efficient Graph Parsing, Bi-directional Locality and the Constructivist Hypothesis
Authors: Yajie Ye, Weiwei Sun
Date: 2020-07-01 00:00:00
Proceeding: ACL 2020 6

A key problem in processing graph-based meaning representations is graph parsing, i.e. computing all possible derivations of a given graph according to a (competence) grammar. We demonstrate, for the first time, that exact graph parsing can be efficient for large graphs and with large Hyperedge Replacement Grammars (HRGs). The advance is achieved by exploiting locality as terminal edge-adjacency in HRG rules. In particular, we highlight the importance of 1) a terminal edge-first parsing strategy, 2) a categorization of a subclass of HRG, i.e. what we call Weakly Regular Graph Grammar, and 3) distributing argument-structures to both lexical and phrasal rules.
Max-Margin Incremental CCG Parsing
Authors: Milo{\v{s}} Stanojevi{\'c}, Mark Steedman
Date: 2020-07-01 00:00:00
Proceeding: ACL 2020 6
Tasks: Language Modelling, Sentence

Incremental syntactic parsing has been an active research area both for cognitive scientists trying to model human sentence processing and for NLP researchers attempting to combine incremental parsing with language modelling for ASR and MT. Most effort has been directed at designing the right transition mechanism, but less has been done to answer the question of what a probabilistic model for those transition parsers should look like. A very incremental transition mechanism of a recently proposed CCG parser when trained in straightforward locally normalised discriminative fashion produces very bad results on English CCGbank. We identify three biases as the causes of this problem: label bias, exposure bias and imbalanced probabilities bias. While known techniques for tackling these biases improve results, they still do not make the parser state of the art. Instead, we tackle all of these three biases at the same time using an improved version of beam search optimisation that minimises all beam search violations instead of minimising only the biggest violation. The new incremental parser gives better results than all previously published incremental CCG parsers, and outperforms even some widely used non-incremental CCG parsers.
Neural Reranking for Dependency Parsing: An Evaluation
Authors: Bich-Ngoc Do, Ines Rehbein
Date: 2020-07-01 00:00:00
Proceeding: ACL 2020 6
Tasks: Dependency Parsing, Reranking

Recent work has shown that neural rerankers can improve results for dependency parsing over the top k trees produced by a base parser. However, all neural rerankers so far have been evaluated on English and Chinese only, both languages with a configurational word order and poor morphology. In the paper, we re-assess the potential of successful neural reranking models from the literature on English and on two morphologically rich(er) languages, German and Czech. In addition, we introduce a new variation of a discriminative reranker based on graph convolutional networks (GCNs). We show that the GCN not only outperforms previous models on English but is the only model that is able to improve results over the baselines on German and Czech. We explain the differences in reranking performance based on an analysis of a) the gold tree ratio and b) the variety in the k-best lists.
Probing for Referential Information in Language Models
Authors: Ionut-Teodor Sorodoc, Kristina Gulordava, Gemma Boleda
Date: 2020-07-01 00:00:00
Proceeding: ACL 2020 6
Tasks: Sentence

Language models keep track of complex information about the preceding context {--} including, e.g., syntactic relations in a sentence. We investigate whether they also capture information beneficial for resolving pronominal anaphora in English. We analyze two state of the art models with LSTM and Transformer architectures, via probe tasks and analysis on a coreference annotated corpus. The Transformer outperforms the LSTM in all analyses. Our results suggest that language models are more successful at learning grammatical constraints than they are at learning truly referential information, in the sense of capturing the fact that we use language to refer to entities in the world. However, we find traces of the latter aspect, too.
Tchebycheff Procedure for Multi-task Text Classification
Authors: YUREN MAO, Shuang Yun, Weiwei Liu, Bo Du
Date: 2020-07-01 00:00:00
Proceeding: ACL 2020 6
Tasks: Classification, General Classification, Multiobjective Optimization, Multi-Task Learning, text-classification, Text Classification

Multi-task Learning methods have achieved great progress in text classification. However, existing methods assume that multi-task text classification problems are convex multiobjective optimization problems, which is unrealistic in real-world applications. To address this issue, this paper presents a novel Tchebycheff procedure to optimize the multi-task classification problems without convex assumption. The extensive experiments back up our theoretical analysis and validate the superiority of our proposals.
Modeling Word Formation in English--German Neural Machine Translation
Authors: Marion Weller-Di Marco, Alex Fraser, er
Date: 2020-07-01 00:00:00
Proceeding: ACL 2020 6
Tasks: Machine Translation, Morphological Analysis, NMT, Segmentation, Translation

This paper studies strategies to model word formation in NMT using rich linguistic information, namely a word segmentation approach that goes beyond splitting into substrings by considering fusional morphology. Our linguistically sound segmentation is combined with a method for target-side inflection to accommodate modeling word formation. The best system variants employ source-side morphological analysis and model complex target-side words, improving over a standard system.
Graph Neural News Recommendation with Unsupervised Preference Disentanglement
Authors: Linmei Hu, Siyong Xu, Chen Li, Cheng Yang, Chuan Shi, Nan Duan, Xing Xie, Ming Zhou
Date: 2020-07-01 00:00:00
Proceeding: ACL 2020 6
Tasks: Disentanglement, News Recommendation

With the explosion of news information, personalized news recommendation has become very important for users to quickly find their interested contents. Most existing methods usually learn the representations of users and news from news contents for recommendation. However, they seldom consider high-order connectivity underlying the user-news interactions. Moreover, existing methods failed to disentangle a user{'}s latent preference factors which cause her clicks on different news. In this paper, we model the user-news interactions as a bipartite graph and propose a novel Graph Neural News Recommendation model with Unsupervised Preference Disentanglement, named GNUD. Our model can encode high-order relationships into user and news representations by information propagation along the graph. Furthermore, the learned representations are disentangled with latent preference factors by a neighborhood routing algorithm, which can enhance expressiveness and interpretability. A preference regularizer is also designed to force each disentangled subspace to independently reflect an isolated preference, improving the quality of the disentangled representations. Experimental results on real-world news datasets demonstrate that our proposed model can effectively improve the performance of news recommendation and outperform state-of-the-art news recommendation methods.
Identifying Principals and Accessories in a Complex Case based on the Comprehension of Fact Description
Authors: Yakun Hu, Zhunchen Luo, WenHan Chao
Date: 2020-07-01 00:00:00
Proceeding: ACL 2020 6
Tasks: Learning-To-Rank

In this paper, we study the problem of identifying the principals and accessories from the fact description with multiple defendants in a criminal case. We treat the fact descriptions as narrative texts and the defendants as roles over the narrative story. We propose to model the defendants with \textit{behavioral semantic information} and \textit{statistical characteristics}, then learning the importances of defendants within a learning-to-rank framework. Experimental results on a real-world dataset demonstrate the behavior analysis can effectively model the defendants{'} impacts in a complex case.
Programming in Natural Language with fuSE: Synthesizing Methods from Spoken Utterances Using Deep Natural Language Understanding
Authors: Sebastian Weigelt, Vanessa Steurer, Tobias Hey, Walter F. Tichy
Date: 2020-07-01 00:00:00
Proceeding: ACL 2020 6
Tasks: Information Retrieval, Natural Language Understanding, Retrieval

The key to effortless end-user programming is natural language. We examine how to teach intelligent systems new functions, expressed in natural language. As a first step, we collected 3168 samples of teaching efforts in plain English. Then we built fuSE, a novel system that translates English function descriptions into code. Our approach is three-tiered and each task is evaluated separately. We first classify whether an intent to teach new functionality is present in the utterance (accuracy: 97.7{\%} using BERT). Then we analyze the linguistic structure and construct a semantic model (accuracy: 97.6{\%} using a BiLSTM). Finally, we synthesize the signature of the method, map the intermediate steps (instructions in the method body) to API calls and inject control structures (F1: 67.0{\%} with information retrieval and knowledge-based methods). In an end-to-end evaluation on an unseen dataset fuSE synthesized 84.6{\%} of the method signatures and 79.2{\%} of the API calls correctly.
AMR Parsing with Latent Structural Information
Authors: Qiji Zhou, Yue Zhang, Donghong Ji, Hao Tang
Date: 2020-07-01 00:00:00
Proceeding: ACL 2020 6
Tasks: AMR Parsing, Sentence

Abstract Meaning Representations (AMRs) capture sentence-level semantics structural representations to broad-coverage natural sentences. We investigate parsing AMR with explicit dependency structures and interpretable latent structures. We generate the latent soft structure without additional annotations, and fuse both dependency and latent structure via an extended graph neural networks. The fused structural information helps our experiments results to achieve the best reported results on both AMR 2.0 (77.5{\%} Smatch F1 on LDC2017T10) and AMR 1.0 ((71.8{\%} Smatch F1 on LDC2014T12).
Target Inference in Argument Conclusion Generation
Authors: Milad Alshomary, Shahbaz Syed, Martin Potthast, Henning Wachsmuth
Date: 2020-07-01 00:00:00
Proceeding: ACL 2020 6
Tasks: Triplet

In argumentation, people state premises to reason towards a conclusion. The conclusion conveys a stance towards some target, such as a concept or statement. Often, the conclusion remains implicit, though, since it is self-evident in a discussion or left out for rhetorical reasons. However, the conclusion is key to understanding an argument and, hence, to any application that processes argumentation. We thus study the question to what extent an argument{'}s conclusion can be reconstructed from its premises. In particular, we argue here that a decisive step is to infer a conclusion{'}s target, and we hypothesize that this target is related to the premises{'} targets. We develop two complementary target inference approaches: one ranks premise targets and selects the top-ranked target as the conclusion target, the other finds a new conclusion target in a learned embedding space using a triplet neural network. Our evaluation on corpora from two domains indicates that a hybrid of both approaches is best, outperforming several strong baselines. According to human annotators, we infer a reasonably adequate conclusion target in 89{\%} of the cases.
Multimodal Transformer for Multimodal Machine Translation
Authors: Shaowei Yao, Xiaojun Wan
Date: 2020-07-01 00:00:00
Proceeding: ACL 2020 6
Tasks: Machine Translation, Multimodal Machine Translation, Translation

Multimodal Machine Translation (MMT) aims to introduce information from other modality, generally static images, to improve the translation quality. Previous works propose various incorporation methods, but most of them do not consider the relative importance of multiple modalities. Equally treating all modalities may encode too much useless information from less important modalities. In this paper, we introduce the multimodal self-attention in Transformer to solve the issues above in MMT. The proposed method learns the representation of images based on the text, which avoids encoding irrelevant information in images. Experiments and visualization analysis demonstrate that our model benefits from visual information and substantially outperforms previous works and competitive baselines in terms of various metrics.
Sentiment and Emotion help Sarcasm? A Multi-task Learning Framework for Multi-Modal Sarcasm, Sentiment and Emotion Analysis
Authors: Dushyant Singh Chauhan, Dhanush S R, Asif Ekbal, Pushpak Bhattacharyya
Date: 2020-07-01 00:00:00
Proceeding: ACL 2020 6
Tasks: Emotion Recognition, Multi-Task Learning, Sarcasm Detection, Sentence, Sentiment Analysis

In this paper, we hypothesize that sarcasm is closely related to sentiment and emotion, and thereby propose a multi-task deep learning framework to solve all these three problems simultaneously in a multi-modal conversational scenario. We, at first, manually annotate the recently released multi-modal MUStARD sarcasm dataset with sentiment and emotion classes, both implicit and explicit. For multi-tasking, we propose two attention mechanisms, viz. Inter-segment Inter-modal Attention (Ie-Attention) and Intra-segment Inter-modal Attention (Ia-Attention). The main motivation of Ie-Attention is to learn the relationship between the different segments of the sentence across the modalities. In contrast, Ia-Attention focuses within the same segment of the sentence across the modalities. Finally, representations from both the attentions are concatenated and shared across the five classes (i.e., sarcasm, implicit sentiment, explicit sentiment, implicit emotion, explicit emotion) for multi-tasking. Experimental results on the extended version of the MUStARD dataset show the efficacy of our proposed approach for sarcasm detection over the existing state-of-the-art systems. The evaluation also shows that the proposed multi-task framework yields better performance for the primary task, i.e., sarcasm detection, with the help of two secondary tasks, emotion and sentiment analysis.