pwc-archive/papers-with-abstracts huggingface dataset:
Seeing things or seeing scenes: Investigating the capabilities of V&L models to align scene descriptions to images
Authors: Anonymous
Date: 2021-10-16 00:00:00
Proceeding: ACL ARR October 2021 10
Tasks: Object

Images can be described in terms of the objects they contain, or in terms of the types of scene or place that they instantiate. In this paper we address to what extent pretrained Vision and Language models can learn to align descriptions of both types with images. We compare 3 state-of-the-art models, VisualBERT, LXMERT and CLIP. We find that (i) V\&L models are susceptible to stylistic biases acquired during pretraining; (ii) only CLIP performs consistently well on both object- and scene-level descriptions. A follow-up ablation study shows that CLIP uses object-level information in the visual modality to align with scene-level textual descriptions.
N-Shot Learning for Augmenting Task-Oriented Dialogue State Tracking
Authors: Anonymous
Date: 2021-10-16 00:00:00
Proceeding: ACL ARR October 2021 10
Tasks: Dialogue State Tracking, Language Modeling, Language Modelling

We introduce an augmentation framework that utilizes belief state annotations to match turns from various dialogues and forms new synthetic dialogues in a bottom-up manner. Unlike other augmentation strategies, it operates with as few as five examples. Our augmentation strategy yields significant improvements when both adapting a DST model to a new domain, and when adapting a language model to the DST task, on evaluations with TRADE and TOD-BERT models. Further analysis shows that our model performs better on seen values during training, and it is also more robust to unseen values even though we do not use any external dataset for augmentation. We conclude that exploiting belief state annotations enhances dialogue augmentation and results in improved models in $n$-shot training scenarios.
Semantic Search as Extractive Paraphrase Span Detection
Authors: Anonymous
Date: 2021-10-16 00:00:00
Proceeding: ACL ARR October 2021 10
Tasks: Extractive Question-Answering, Question Answering, Retrieval, Sentence, Sentence Embeddings, Sentence Similarity, Translation

In this paper, we approach the problem of semantic search by framing the search task as paraphrase span detection, i.e. given a segment of text as a query phrase, the task is to identify its paraphrase in a given document, the same modelling setup as typically used in extractive question answering. On the Turku Paraphrase Corpus of 100,000 manually extracted Finnish paraphrase pairs including their original document context, we find that our paraphrase span detection model outperforms two strong retrieval baselines (lexical similarity and BERT sentence embeddings) by 31.9pp and 22.4pp respectively in terms of exact match, and by 22.3pp and 12.9pp in terms of token-level F-score. This demonstrates a strong advantage of modelling the task in terms of span retrieval, rather than sentence similarity. Additionally, we introduce a method for creating artificial paraphrase data through back-translation, suitable for languages where manually annotated paraphrase resources for training the span detection model are not available.
FarFetched: An Entity-centric Approach for Reasoning on Textually Represented Environments
Authors: Anonymous
Date: 2021-10-16 00:00:00
Proceeding: ACL ARR October 2021 10
Tasks: Articles, Natural Language Inference, Semantic Similarity, Semantic Textual Similarity, STS

We address the problem of automatically acquiring knowledge from news articles and leverage it to estimate the veracity of a user's claim based on the supporting or refuting content within the accumulated evidence. We present FarFetched, an entity-centric approach for reasoning based on news, where latent connections between events, actions or statements are discovered via their identified entity mentions and are represented with the help of a knowledge graph. We propose a way of selecting specific subsets from the accumulated wealth of information based on the user hypothesis and construct relevant premises relying on the semantic similarity between them. We leverage textual entailment recognition to provide a measurable way for assessing whether the user claim is plausible based on the selected evidence. Our work is demonstrated on the less-resourced Greek language and supported by the training of state-of-the-art models for STS and NLI that are evaluated on benchmark datasets.
Graph-to-Graph Annotation Conversion Based on Pretrained Models
Authors: Anonymous
Date: 2021-10-16 00:00:00
Proceeding: ACL ARR October 2021 10

Annotation conversion is an effective way to construct datasets under new annotation guidelines based on existing datasets with little human labour. Previous work has been limited in conversion between tree-structured datasets and mainly focused on feature-based models which are not easily applicable to new conversion. In this paper, we propose two pretrained model-based graph-to-graph annotation conversion approaches, namely Label Switching and Graph2Graph Linear Transfer, which are able to deal with conversion between graph-structured annotations and require no manually designed feature. We manually construct a graph-structured parallel annotated dataset and evaluate the proposed approaches on it as well as four existing parallel annotated datasets. Experimental results show that the proposed approaches outperform two strong baselines across all the datasets. Furthermore, the combination of the two models have a better effect.
Emotion Style Transfer with a Specified Intensity Using Deep Reinforcement Learning
Authors: Anonymous
Date: 2021-10-16 00:00:00
Proceeding: ACL ARR October 2021 10
Tasks: Deep Reinforcement Learning, Diversity, reinforcement-learning, Reinforcement Learning (RL), Sentence, Style Transfer, Text Generation, Text Style Transfer

Text style transfer is a widely explored task in natural language generation which aims to change the stylistic properties of the text while retaining its style-independent content. In this work, we propose the task of emotion style transfer with a specified intensity in an unsupervised setting. The aim is to rewrite a given sentence, in any emotion, to a target emotion while also controlling the intensity of the target emotion. Emotions are gradient in nature, some words/phrases represent higher emotional intensity, while others represent lower intensity. In this task, we want to control this gradient nature of the emotion in the output. Additionally, we explore the issues with the existing datasets and address them. A novel BART-based model is proposed that is trained for the task by direct rewards. Unlike existing work, we bootstrap the BART model by training it to generate paraphrases so that it can explore lexical and syntactic diversity required for the output. Extensive automatic and human evaluations show the efficacy of our model in solving the problem.
Old BERT, New Tricks: Artificial Language Learning for Pre-Trained Language Models
Authors: Anonymous
Date: 2021-10-16 00:00:00
Proceeding: ACL ARR October 2021 10
Tasks: Sensitivity, Sentence

We extend the artificial language learning experimental paradigm from psycholinguistics and apply it to pre-trained language models -- specifically, BERT (Devlin et al., 2019). We treat a pretrained model as a subject in an artificial language learning experimental setting: in order to learn the relation between two linguistic properties A and B, we introduce a set of new, non-existent, linguistic items, give the model information about their variation along property A, then measure to what extent the model learns property B for these items as a result of training. We show this method at work for degree modifiers (expressions like *slightly*, *very*, *rather*, *extremely*) and test the hypothesis that the degree expressed by the modifier (low, medium or high degree) is related to its sensitivity to sentence polarity (whether it shows preference for affirmative or negative sentences or neither). Our experimental results are compatible with existing linguistic observations that relate degree semantics to polarity-sensitivity, including the main one: low degree semantics leads to positive polarity sensitivity (that is, to preference towards affirmative contexts). 
The method can be used in linguistic theory to elaborate on hypotheses and interpret experimental results, as well as for more insightful evaluation of linguistic representations in language models.
Exploring the Impact of Negative Samples of Contrastive Learning: A Case Study of Sentence Embedding
Authors: Anonymous
Date: 2021-10-16 00:00:00
Proceeding: ACL ARR October 2021 10
Tasks: Contrastive Learning, Sentence, Sentence Embedding, Sentence-Embedding, STS, text similarity

Contrastive learning is emerging as a powerful self-supervised technique for extracting knowledge from unlabeled image and text data. This technique requires a balanced mixture of two ingredients: positive (similar) and negative (dissimilar) samples. This is typically achieved by maintaining a queue of negative samples during training. Prior works in the area typically uses a fixed-length negative sample queue, but how the negative sample size affects the model performance remains unclear. The opaque impact of the number of negative samples on performance when employing contrastive learning aroused our in-depth exploration. This paper presents a momentum contrastive learning model with negative sample queue for sentence embedding, namely MoCoSE. We add the prediction layer to the online branch to make the model asymmetric and together with EMA update mechanism of the target branch to prevent model from collapsing. We define a maximum traceable distance metric, through which we learn to what extent the text contrastive learning benefits from the historical information of negative samples. Our experiments find that the best results are obtained when the maximum traceable distance is at a certain range, demonstrating that there is an optimal range of historical information for a negative sample queue. We evaluate the proposed unsupervised MoCoSE on the semantic text similarity (STS) task and obtain an average Spearman's correlation of $77.27\%$. Source code is available at https://anonymous.4open.science/r/mocose-3E3C.
On the Importance of Effectively Adapting Pretrained Language Models for Active Learning
Authors: Anonymous
Date: 2021-10-16 00:00:00
Proceeding: ACL ARR October 2021 10
Tasks: Active Learning

Recent active learning (AL) approaches  in Natural Language Processing (NLP) proposed using off-the-shelf pretrained language models (LMs). In this paper, we argue that these LMs are not adapted effectively to the downstream task during AL and we explore ways to address this issue. We suggest to first adapt the pretrained LM to the target task by continuing training with all the available unlabeled data and then use it for AL. We also propose a simple yet effective fine-tuning method to ensure that the adapted LM is properly trained in both low and high resource scenarios during AL. Our experiments demonstrate that our approach provides substantial data efficiency improvements compared to the standard fine-tuning approach, suggesting that a poor training strategy can be catastrophic for AL.
Large-Scale Hate Speech Detection with Cross-Domain Transfer
Authors: Anonymous
Date: 2021-10-16 00:00:00
Proceeding: ACL ARR October 2021 10
Tasks: Hate Speech Detection

Hate speech towards people with different backgrounds is a major problem observed in social media. Although there are various attempts to detect hate speech automatically via supervised learning models, the performance of such models simply rely on limited datasets on which models are trained. In this study, we construct large-scale tweet datasets for supervised hate speech detection in English and Turkish, including human-labeled 100k tweets per each. Our datasets are designed to have equal number of tweets distributed over five domains; namely religion, gender, race, politics, and sports. We analyze the performance of state-of-the-art language models on large-scale hate speech detection with a special focus on model scalability. We also examine cross-domain transfer ability of hate speech detection.
Zero-shot Cross-Language Transfer of Monolingual Entity Linking Models
Authors: Anonymous
Date: 2021-10-16 00:00:00
Proceeding: ACL ARR October 2021 10
Tasks: Cross-Lingual Transfer, Entity Linking, Zero-Shot Cross-Lingual Transfer

Most entity linking systems, whether mono or multilingual, link mentions to a single English knowledge base. Few have considered linking non-English text to a non-English KB, and therefore, transferring an English entity linking model to both a new document and new KB language. We consider the task of zero-shot cross-lingual transfer of entity linking systems to a new language and KB. We find that a system trained with multilingual representations does reasonably well, and propose improvements to system training that lead to improved recall in most datasets, often matching the in-language performance. We further conduct a detailed evaluation to elucidate the challenges of this setting.
Knowledge Inheritance for Pre-trained Language Models
Authors: Anonymous
Date: 2021-10-16 00:00:00
Proceeding: ACL ARR October 2021 10
Tasks: Lifelong learning, Self-Learning, Transfer Learning

Recent explorations of large-scale pre-trained language models (PLMs) such as GPT-3 have revealed the power of PLMs with huge amounts of parameters, setting off a wave of training ever-larger PLMs. However, training a large-scale PLM requires tremendous amounts of computational resources, which is time-consuming and expensive. In addition, existing large-scale PLMs are mainly trained from scratch individually, ignoring the availability of many existing well-trained PLMs. To this end, we explore the question that how can previously trained PLMs benefit training larger PLMs in future. Specifically, we introduce a novel pre-training framework named "knowledge inheritance" (KI), which combines both self-learning and teacher-guided learning to efficiently train larger PLMs. Experimental results demonstrate the superiority of our KI framework. We also conduct empirical analyses to explore the effects of teacher PLMs' pre-training settings, including model architecture, pre-training data, etc. Finally, we show that KI can well support lifelong learning and knowledge transfer. All source code and model parameters will be available to advance further research explorations.
It's my Job to be Repetitive! My Job! My Job! -- Linking Repetitions to In-Context Learning in Language Models
Authors: Anonymous
Date: 2021-10-16 00:00:00
Proceeding: ACL ARR October 2021 10
Tasks: In-Context Learning

Recent studies have shown that large language models can display surprising accuracy at learning tasks from few examples presented in the input context, which goes under the name of in-context learning. Other studies have shown that language models can sometimes display the undesirable behavior of falling back into loops in which an utterance is repeated infinitely often. Here, we observe that the model's capacity to produce repetitions goes well beyond frequent or well-formed utterances, and generalizes to repeating completely arbitrary sequences of tokens. Construing this as a simple form of in-context learning, we hypothesize that these two phenomena are linked through shared processing steps. With controlled experiments, we show that impairing the network from producing repetitions severely affects in-context learning, without reducing its overall predictive performance, thus supporting the proposed hypothesis.
Use of a Taxonomy of Empathetic Response Intents to Control and Interpret Empathy in Neural Chatbots
Authors: Anonymous
Date: 2021-10-16 00:00:00
Proceeding: ACL ARR October 2021 10
Tasks: Response Generation

A recent trend in the domain of open-domain conversational agents is enabling them to converse empathetically to emotional prompts. Current approaches either follow an end-to-end approach or condition the responses on similar emotion labels to generate empathetic responses. But empathy is a broad concept that refers to the cognitive and emotional reactions of an individual to the observed experiences of another and it is more complex than mere mimicry of emotion. Hence, it requires identifying complex human conversational strategies and dynamics in addition to generic emotions to control and interpret empathetic responding capabilities of chatbots. In this work, we make use of a taxonomy of eight empathetic response intents in addition to generic emotion categories in building a dialogue response generation model capable of generating empathetic responses in a controllable and interpretable manner. It consists of two modules: 1) a response emotion/intent prediction module; and 2) a response generation module. We propose several rule-based and neural approaches to predict the next response's emotion/intent and generate responses conditioned on these predicted emotions/intents. Automatic and human evaluation results emphasize the importance of the use of the taxonomy of empathetic response intents in producing more diverse and empathetically more appropriate responses than end-to-end models.
Compositional Generalization Requires Compositional Parsers
Authors: Anonymous
Date: 2021-10-16 00:00:00
Proceeding: ACL ARR October 2021 10

A growing body of research has focused on the task of \textit{compositional generalization}, the ability of a semantic parser to dynamically combine known linguistic elements in novel structures. We analyze the accuracy of different parsers on the recent COGS corpus (Kim and Linzen, 2020). While lexical generalization tasks are solvable by almost all existing models, tasks involving changes to the linguistic structure are hard for even the best sequence-to-sequence models. Structural generalization tasks can be solved with models that have compositionality built in; we present new results confirming this from the AM parser (Groschwitz et al., 2021). We further analyze the role of syntactic generalization in compositional generalization, and we discuss ramifications for the design of both semantic parsers and compositional generalization datasets.
Learning Meta Word Embeddings by Unsupervised Weighted Concatenation of Source Embeddings
Authors: Anonymous
Date: 2021-10-16 00:00:00
Proceeding: ACL ARR October 2021 10
Tasks: Representation Learning, Word Embeddings

We propose a method to protect the privacy of search engine users by decomposing the queries using
semantically \emph{related} and unrelated \emph{distractor} terms. Instead of a single query, the search engine
receives multiple decomposed query terms. Next, we reconstruct the search results relevant to the original
query term by aggregating the search results retrieved for the decomposed query terms.
We show that the word embeddings learnt using a distributed representation learning method can be used to find semantically related and distractor query terms.
We derive the relationship between the \emph{obfuscity} achieved through the proposed query anonymisation method and the \emph{reconstructability} of the original search results using the decomposed queries.
We analytically study the risk of discovering the search engine users' information intents under the proposed
query obfuscation method, and empirically evaluate its robustness against clustering-based attacks.
Our experimental results show that the proposed method can accurately reconstruct the search results for user queries, without compromising the privacy of the search engine users.
Benchmarking Biomedical Nested NER and Relation Extraction Models
Authors: Anonymous
Date: 2021-10-16 00:00:00
Proceeding: ACL ARR October 2021 10
Tasks: Benchmarking, NER, Relation, Relation Extraction

The Open EPPI corpus comprises $151$ full-text papers annotated by domain experts for entity mentions, protein-protein interactions (PPIs), and normalisation of entities to publicly available ontologies.
The corpus is publicly available at [ANON].
We benchmark recent nested NER and relation extraction models.
Results show that, although existing nested NER models achieve good performance on outermost and innermost entity mentions, they struggle with other types of nested mentions.
Benchmark results for relation extraction show substantial room for improvement with precision under $70$ and recall around $40$ to $52$.
A Two-Stage Curriculum Training Framework for NMT
Authors: Anonymous
Date: 2021-10-16 00:00:00
Proceeding: ACL ARR October 2021 10
Tasks: Machine Translation, NMT, Translation, Vocal Bursts Valence Prediction

Neural Machine Translation (NMT) models are typically trained on heterogeneous data that are concatenated and randomly shuffled. Curriculum training aims to present the data to the NMT systems in a meaningful order. In this work, we introduce a two-stage curriculum training framework for NMT where we fine-tune a base NMT model on subsets of data, selected by both deterministic scoring using pre-trained methods and online scoring that consider prediction scores of the emerging NMT model. Through extensive experiments on six language pairs comprising low- and high-resource languages from WMT'21, we have shown that our curriculum strategies consistently demonstrate better quality (up to +2.2 BLEU improvement) and faster convergence (approximately 50% fewer updates).
On the current state of reproducibility and reporting of uncertainty for Aspect-based Sentiment Analysis
Authors: Anonymous
Date: 2021-10-16 00:00:00
Proceeding: ACL ARR October 2021 10
Tasks: Articles, Aspect-Based Sentiment Analysis, Aspect-Based Sentiment Analysis (ABSA), Sentiment Analysis

For the latter part of the past decade, Aspect-Based Sentiment Analysis has been a field of great interest within Natural Language Processing. Supported by the Semantic Evaluation Conferences in 2014 -- 2016, a variety of methods has been developed competing in improving performances on benchmark data sets. Exploiting the transformer architecture behind BERT, results improved rapidly and efforts in this direction still continue today. Our contribution to this body of research is a holistic comparison of six different architectures which achieved (near) state-of-the-art results at some point in time. We utilize a broad spectrum of five benchmark data sets and introduce a fixed setting with respect to the pre-processing, the train/validation splits, the performance measures and the quantification of uncertainty. Overall, our findings are two-fold: First, we find that the results reported in the scientific articles are hardly reproducible, since in our experiments the observed performance (most of the time) fell short of the reported one. Second, the results are burdened with notable uncertainty (depending on the data splits) which is why a reporting of uncertainty measures is crucial.
Measuring the Language of Self-Disclosure across Corpora
Authors: Anonymous
Date: 2021-10-16 00:00:00
Proceeding: ACL ARR October 2021 10

Being able to reliably estimate self-disclosure -- a key component of friendship and intimacy -- from language is important for many psychology studies. We build single-task models on five self-disclosure corpora, but find that these models generalize poorly; the within-domain accuracy of predicted message-level self-disclosure of the best-performing model (mean Pearson's r=0.69) is much higher than the respective across data set accuracy (mean Pearson's r=0.32), due to both variations in the corpora (e.g., medical vs. general topics) and labeling instructions (target variables: self-disclosure, emotional disclosure, intimacy). However, some lexical features, such as expression of negative emotions and use of first person personal pronouns such as 'I' reliably predict self-disclosure across corpora. We develop a multi-task model that yields better results, with an average Pearson's r of 0.37 for out-of-corpora prediction.