pwc-archive/papers-with-abstracts huggingface dataset:
A Dual-Channel Framework for Sarcasm Recognition by Detecting Sentiment Conflict
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Sarcasm Detection

Sarcasm employs ambivalence, where one says something positive but actually means negative, vice versa. The essence of sarcasm, which is also a sufficient and necessary condition, is the conflict between literal and implied sentiments. However, it is difficult to recognize the sentiment conflict because more than one mixed or even implicit sentiments coexist in one text. As a result, the recognition of sophisticated and obscure sentiment brings in a great challenge to sarcasm detection.  In this paper, we propose a dual-channel framework by modeling both literal and implied sentiment separately. Based on the flexible dual-channel framework, we design Dual-Channel Net (DC-Net) to recognize the sentiment conflict. Experiments on political debates (i.e. IAC-V1 and IAC-V2) and Twitter datasets show that our proposed DC-Net achieves state-of-the-art performance on sarcasm recognition.
AbductionRules: Training Transformers to Explain Unexpected Inputs
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Common Sense Reasoning, Logical Reasoning, scientific discovery

Transformers have recently been shown to be capable of reliably performing logical reasoning over facts and rules expressed in natural language, but abductive reasoning - inference to the best explanation of an unexpected observation - has been underexplored despite significant applications to scientific discovery, common-sense reasoning, and model interpretability.

This paper presents AbductionRules, a group of natural language datasets designed to train and test generalisable abduction over natural-language knowledge bases.
We use these datasets to finetune pretrained Transformers and discuss their performance, finding that our models learned generalisable abductive techniques but also learned to exploit the structure of our data.
Finally, we discuss the viability of this approach to abductive reasoning and ways in which it may be improved in future work.
Neural Keyphrase Generation: Analysis and Evaluation
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Decoder, Keyphrase Generation, Semantic Similarity, Semantic Textual Similarity

Keyphrase generation aims at generating topical phrases from a given text either by copying from the original text (present keyphrases) or by producing new keyphrases (absent keyphrases) that capture the semantic meaning of the text. Encoder-decoder models are most widely used for this task because of their capabilities for absent keyphrase generation. However, there has been little to no analysis on the performance and behavior of such models for keyphrase generation. In this paper, we study various tendencies exhibited by two strong models: T5 (based on a pre-trained transformer) and ExHiRD (based on a recurrent neural network). We analyze prediction confidence scores, model calibration, and the effect of position on present keyphrases generation. Moreover, we motivate and propose a novel metric, SoftKeyScore, to evaluate the similarity between two sets of keyphrases by using soft-scores to account for partial matching and semantic similarity. We find that SoftKeyScore performs better than the standard F$_{1}$ metric for evaluating two sets of given keyphrases. We will release our code.
BERT is Robust! A Case Against Synonym-Based Adversarial Examples in Text Classification
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Data Augmentation, text-classification, Text Classification

In this work, we investigate the robustness of BERT using four word substitution-based attacks. We combine a human evaluation of individual word substitutions and a probabilistic analysis to show that between 96% and 99% of the analyzed attacks do not preserve semantics, indicating that their success is mainly based on feeding poor data to the model. To further confirm that, we introduce an efficient data augmentation procedure and show that many successful attacks can be prevented by including data similar to adversarial examples during training. Compared to traditional adversarial training, our data augmentation procedure requires 30x less computation time per epoch, while achieving better performance on two out of three datasets. We introduce an additional post-processing step that reduces the success rates of state-of-the-art attacks below 4%, 5%, and 8% on the three considered datasets. Finally, by looking at constraints for word substitutions that better preserve the semantics, we conclude that BERT is considerably more robust than previous research suggests.
MAML-CL: Edited Model-Agnostic Meta-Learning for Continual Learning
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Continual Learning, Meta-Learning, text-classification, Text Classification

Continual learning (CL) exhibits a learning ability to well-learn all sequentially seen tasks drawn from various domains. Yet, existing sequential training methods fail to consolidate learned knowledge from earlier tasks due to data distribution shifts, hereby leading to catastrophic forgetting. We devise an optimization-based meta learning framework for CL in accordance with MAML, where query samples are edited for generalization of learned knowledge. We conduct extensive experiments on text classification in a low resource CL setup, where we downsize training set to its 10%. The experimental results demonstrate the superiority of our method in terms of stability, fast adaptation, memory efficiency and knowledge retention across various domains.
Two-Level Supervised Contrastive Learning for Response Selection in Multi-Turn Dialogue
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Contrastive Learning, Retrieval, Sentence, STS

Selecting an appropriate response from many candidates given the utterances in a multi-turn dialogue is the key problem for a retrieval-based dialogue system.  Existing work formalizes the task as matching between the utterances and a candidate and uses the cross-entropy loss in learning of the model. This paper applies contrastive learning to the problem by using the supervised contrastive loss.  In this way, the learned representations of positive examples and representations of negative examples can be more distantly separated in the embedding space, and the performance of matching can be enhanced. We further develop a new method for supervised contrastive learning, referred to as two-level supervised contrastive learning, and employ the method in response selection in multi-turn dialogue. Our method exploits two techniques:  sentence token shuffling (STS) and sentence re-ordering (SR) for supervised contrastive learning. Experimental results on three benchmark datasets demonstrate that the proposed method significantly outperforms the contrastive learning baseline and the state-of-the-art methods for the task.
One-to-Many and Many-to-One Dialogue Learning via Sentence Semantic Segmentation Guided Conditional Variational Auto-Encoder
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Dialogue Generation, Semantic Segmentation, Sentence

Due to the complex mapping relations, one-to-many and many-to-one phenomena are huge challenges for open-domain dialogue generation task, which tend to make dialogue models generate irrelevant, incoherent or non-diverse responses. Most existing methods avoid learning such phenomena through introducing the external information, reconstructing the optimization function or manipulating data samples. However, avoiding confronting such challenges ignores valuable information in these responses, and the dialogue models cannot learn the nature of such phenomena. In this paper, we propose a Sentence Semantic Segmentation guided Conditional Variational Auto-Encoder (SegCVAE) to directly learn one-to-many and many-to-one responses. SegCVAE uses prominent semantics to replace the original semantics to learn the distribution of latent variables, which avoids the gap between latent variables and the context, thus ensuring the relevance and coherence of the generated responses. Furthermore, SegCVAE can segment multiple prominent semantics to ensure the diversity of generated responses. To evaluate the model, we first define two new tasks named one-to-many dialogue learning task and many-to-one dialogue learning task. And then provide two new dialogue datasets named One-to-Many and Many-to-One, which are extracted from the well-established dataset. Finally, we also propose the evaluation strategies based on some commonly-used metrics. The experiment results show that our model achieve better performance than the baseline models in addressing these two new tasks.
MATHion: Solving Math Word Problems with Logically Consistent Problems
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Language Modeling, Language Modelling, Math, Retrieval

Solving the math word problems (MWPs) is a challenging task. Some existing MWP solvers retrieve textually similar problems and draw on their solutions to solve the given problem. However, textually similar questions are not guaranteed to have similar solutions. And questions could share the same solution but with different descriptions. Therefore in this work, we investigate the logical consistency among different problems and propose a novel framework named MATHion which solves math word problems with logically consistent problems. Experimental results show that our method outperforms many strong baselines, including some pre-trained language model-based methods. Further analysis shows that our retrieval method can learn the logical similarity between questions and plays a key role in our model's performance.
From Stance to Concern: Adaptation of Propositional Analysis to New Tasks and Domains
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Semantic Similarity, Semantic Textual Similarity

We present a generalized paradigm for adaptation of propositional analysis (predicate-argument pairs) to new tasks and domains, leveraging an analogy between stances (belief-driven sentiment) and concerns (topical issues with moral dimensions/endorsements). A key contribution is the combination of semi-automatic resource building for extraction of domain-dependent concern types (with 2-4 hours of human labor per domain) and an entirely automatic procedure for extraction of domain-independent moral dimensions and endorsement values.  Prudent (automatic) selection of terms from propositional structures for lexical expansion (via semantic similarity) produces new moral dimension lexicons at three levels of granularity beyond a strong baseline lexicon. We develop a ground truth (GT) based on expert annotators and compare our concern detection output to GT, to yield 231% improvement in recall over baseline, with only a 10% loss in precision. F1 yields 66% improvement over baseline and 97.8% of human performance. Moreover, our lexically based approach yields large savings in terms of human labor and costly model building. Work produced herein provides to the community a newly expanded moral dimension/value lexicon, annotation guidelines, and GT.
Improving Equation Set Problems with Label Augmentation
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Decoder, Math

Math word problems solving has received considerable attention from many NLP researchers. Inspired by the encoder-decoder structure, they created a series of neural network models to solve arithmetic word problems and equation set problems. However, these encoder-decoder models used the ground truth as the only generation target, resulting in shallow heuristics to generate expressions. In this paper, we propose a simple and effective label augmentation method for equation set problems. Specifically, we transform the ground truth into several equivalent labels by normalization rules, and these new labels will be used as additional generation targets for model training. Experimental results on the English dataset DRAW1K and Chinese dataset HMWP show that the label augmentation method has at most 4.5% improvement over the state-of-the-art (SoTA) models.
Things not Written in Text: Exploring Spatial Commonsense from Visual Signals
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Image Generation, Natural Language Understanding, Position

Spatial commonsense, the knowledge about spatial position and relationship between objects (like the relative size of a lion and a girl, and the position of a boy relative to a bicycle when cycling), is an important part of commonsense knowledge. Although pretrained language models (PLMs) succeed in many NLP tasks, they are shown to be ineffective in spatial commonsense reasoning. 
Starting from the observation that images are more likely to exhibit spatial commonsense than texts, we explore whether models with visual signals learn more spatial commonsense than text-based PLMs. We propose a spatial commonsense benchmark that focuses on the relative scales of objects, and the positional relationship between people and objects under different actions.
We probe PLMs and models with visual signals, including vision-language pretrained models and image synthesis models, on this benchmark, and find that image synthesis models are more capable of learning accurate and consistent spatial knowledge than other models. The spatial knowledge from image synthesis models also helps in natural language understanding tasks that require spatial commonsense.
Utterance Rewriting with Contrastive Learning in Multi-turn Dialogue
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR September 2021 9
Tasks: Contrastive Learning, Multi-Task Learning, Sentence

Context modeling plays a significant role in building multi-turn dialogue systems. In order to make full use of context information, systems can use Incomplete Utterance Rewriting(IUR) methods to simplify the multi-turn dialogue into single-turn by merging current utterance and context information into a self-contained utterance. However, previous approaches ignore the intent consistency between the original query and rewritten query. The detection of omitted or coreferred locations in the original query can be further improved. In this paper, we introduce contrastive learning and multi-task learning to jointly model the problem. Our method benefits from carefully designed self-supervised objectives, which act as auxiliary tasks to capture semantics at both sentence-level and token-level. The experiments show that our proposed model achieves state-of-the-art performance on several public datasets.
Logic-Driven Context Extension and Data Augmentation for Logical Reasoning of Text
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Contrastive Learning, Data Augmentation, Logical Reasoning

Logical reasoning of text requires identifying critical logical structures in the text and performing inference over them. Existing methods for logical reasoning mainly focus on contextual semantics of text while struggling to explicitly model the logical inference process. In this paper, we not only put forward a logic-driven context extension framework but also propose a logic-driven data augmentation algorithm. The former follows a three-step reasoning paradigm, and each step is respectively to extract logical expressions as elementary reasoning units, symbolically infer the implicit expressions following equivalence laws and extend the context to validate the options. The latter augments literally similar but logically different instances and incorporates contrastive learning to better capture logical information, especially logical negative and conditional relationships. We conduct experiments on two benchmark datasets, ReClor and LogiQA. The results show that our method achieves state-of-the-art performance on both datasets, and even surpasses human performance on the ReClor dataset.
KNN-BERT: Fine-Tuning Pre-Trained Models with KNN Classifier
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Contrastive Learning, text-classification, Text Classification

Pre-trained models are widely used in fine-tuning downstream tasks with linear classifiers optimized by the cross entropy loss, which might face robustness and stability problems.
These problems can be improved by learning representations that focus on similarities in the same class and variance in different classes when making predictions.
In this paper, we utilize the K-Nearest Neighbors Classifier in pre-trained model fine-tuning.
For this KNN classifier, we introduce a supervised momentum contrastive learning framework to learn the clustered representations of the supervised downstream tasks.
Extensive experiments on text classification tasks and robustness tests show that by incorporating KNNs with the traditional fine-tuning process, we can obtain significant improvements on the clean accuracy in both rich-source and few-shot settings and can improve the robustness against adversarial attacks.
\footnote{all codes will be available at https://github.com//}
Learning Non-Autoregressive Models from Search for Unsupervised Sentence Summarization
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Headline Generation, Sentence, Sentence Summarization, Text Summarization, Unsupervised Sentence Summarization

Text summarization aims to generate a short summary for an input text. In this work, we propose a Non-Autoregressive Unsupervised Summarization (NAUS) approach, which does not require parallel data for training. Our NAUS first performs edit-based search towards a heuristically defined score, and generates a summary as pseudo-groundtruth. Then, we train an encoder-only non-autoregressive Transformer based on the search result. We also proposed a dynamic programming approach for length-control decoding, which is important for the summarization task. Experiments on the Gigaword headline generation and DUC2004 datasets show that NAUS achieves state-of-the-art performance for unsupervised summarization, yet largely improving inference efficiency. Further, our algorithm is able to perform length-transfer summary generation.
Cross-domain Named Entity Recognition via Graph Matching
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Cross-Domain Named Entity Recognition, Few-Shot Learning, Graph Matching, Multi-Task Learning, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Transfer Learning

Cross-domain NER is a practical yet challenging problem since the data scarcity in the real-world scenario. A common practice is first to learn a NER model in a rich-resource general domain and then adapt the model to specific domains. Due to the mismatch problem between entity types across domains, the wide knowledge in the general domain can not effectively transfer to the target domain NER model. To this end, we model the label relationship as a probability distribution and construct label graphs in both source and target label spaces. To enhance the contextual representation with label structures, we fuse the label graph into the word embedding output by BERT. By representing label relationships as graphs, we formulate cross-domain NER as a graph matching problem. Furthermore, the proposed method has good applicability with pre-training methods and is potentially capable of other cross-domain prediction tasks. Empirical results on four datasets show that our method outperforms a series of transfer learning, multi-task learning, and few-shot learning methods.
A Simple General Method for Detecting Textual Adversarial Examples
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Ensemble Learning, Representation Learning, Sentence

Although deep neural networks have achieved state-of-the-art performance in various machine learning and artificial intelligence tasks, adversarial examples, constructed by adding small non-random perturbations to correctly classified inputs, successfully fool highly expressive deep classifiers into incorrect predictions. Approaches to adversarial attacks in natural language tasks have boomed in the last five years using character-level, word-level, phrase-level, or sentence-level textual perturbations. While there is some work in NLP on defending against such attacks through proactive methods, like adversarial training, there is to our knowledge no effective reactive approaches to defence via detection of textual adversarial examples such as is found in the image processing literature. In this paper, we apply distance-based ensemble learning and semantic representations from different representation learning models based on our understanding of the reason for adversarial examples to fill this gap. Our technique, MultiDistance Representation Ensemble Method (MDRE), obtains state-of-the-art results on character-level, word-level, and phrase-level attacks on the IMDB dataset as well as on the later two with respect to the MultiNLI dataset. If this paper is accepted, we will publish our code.
Fusing Heterogeneous Factors with Triaffine Mechanism for Nested Named Entity Recognition
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Classification, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Nested Named Entity Recognition

Nested entities are observed in many domains due to their compositionality, which cannot be easily recognized by the widely-used sequence labeling framework.
A natural solution is to treat the task as a span classification problem.
To learn better span representation and increase classification performance, it is crucial to effectively integrate heterogeneous factors including inside tokens, boundaries, labels, and related spans which could be contributing to nested entities recognition.
To fuse these heterogeneous factors, we propose a novel triaffine mechanism including triaffine attention and scoring.
Triaffine attention uses boundaries and labels as queries, and uses inside tokens and related spans as keys and values for span representations.
Triaffine scoring interacts with boundaries and span representations for classification.
Experiments show that our proposed method achieves the state-of-the-art $F_1$ scores on four nested NER datasets: ACE2004,  ACE2005, GENIA, and KBP2017.
XLTime: A Cross-Lingual Knowledge Transfer Framework for Zero-Shot Low-Resource Language Temporal Expression Extraction
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Causal Inference, Information Retrieval, Multi-Task Learning, Question Answering, Retrieval, Transfer Learning

Temporal Expression Extraction (TEE) is essential for understanding time in natural language. It has applications in Natural Language Processing (NLP) tasks such as question answering, information retrieval, and causal inference.  To date, work in this area has mostly focused on English as TEE for low-resource languages is hindered by a scarcity of training data. We propose XLTime, a novel framework for zero-shot low-resource language TEE. XLTime works on top of pre-trained language models and leverages multi-task learning to prompt cross-language knowledge transfer both from English and within the low-resource languages. It alleviates the problems caused by the shortage in low-resource language training data. We apply XLTime with different language models and show that it outperforms the previous automatic SOTA methods on four low-resource languages, i.e., French, Spanish, Portuguese, and Basque, by large margins.  It also closes the gap considerably on the handcrafted HeidelTime tool.