pwc-archive/papers-with-abstracts huggingface dataset:
“Find Me a Dataset”: Scientific Dataset Recommendation from Method Descriptions
Authors: Anonymous
Date: 2021-12-17 00:00:00
Proceeding: ACL ARR December 2022 12

Much of modern science relies on public datasets to develop research ideas. Finding a dataset for a given task can be difficult, particularly for new researchers. We aim to improve the process of dataset discovery by introducing a system called DatasetFinder which recommends relevant datasets given a short natural language description of a research idea. For the new task of dataset recommendation, we construct an English-language dataset that leverages existing annotations and compare several ranking models on this dataset. We also compare our proposed models against existing commercial search engines and find evidence that leveraging natural language descriptions improves search relevance. To encourage development on this new task, we release our constructed dataset and models to the public.
ME-GCN: Multi-dimensional Edge-Enhanced Graph Convolutional Networks for Semi-supervised Text Classification
Authors: Anonymous
Date: 2021-12-17 00:00:00
Proceeding: ACL ARR December 2022 12
Tasks: Semi-Supervised Text Classification, text-classification, Text Classification

Compared to sequential learning models, graph-based neural networks exhibit excellent ability in capturing global information and have been used for semi-supervised learning tasks, including citation network analysis or text classification. Most Graph Convolutional Networks are designed with the single-dimensional edge feature and failed to utilise the rich edge information about graphs. In this paper, we introduce the ME-GCN (Multi-dimensional Edge-enhanced Graph Convolutional Networks) for semi-supervised text classification. A text graph for an entire corpus is firstly constructed to describe the undirected and multi-dimensional relationship of word-to-word, document-document, and word-to-document. The graph is initialised with corpus-trained multi-dimensional word and document node representation, and the relations are represented according to the distance of those words/documents nodes. Then, the generated graph is trained with ME-GCN, which considers the edge features as multi-stream signals, and each stream performs a separate graph convolutional operation. Our ME-GCN can integrate a rich source of graph edge information of the entire text corpus. The results have demonstrated that our proposed model has significantly outperformed the state-of-the-art methods across eight benchmark datasets.
HyEnA: A Hybrid Method for Extracting Arguments from Opinions
Authors: Anonymous
Date: 2021-12-17 00:00:00
Proceeding: ACL ARR December 2022 12

The key arguments underlying a large and noisy set of opinions help understand the opinions quickly and accurately. Fully automated methods can extract arguments but (1) require large labeled datasets and (2) work well for known viewpoints, but not for novel points of view. We propose HyEnA, a hybrid (human + AI) method for extracting arguments from opinionated texts, combining the speed of automated processing with the understanding and reasoning capabilities of humans. We evaluate HyEnA on three feedback corpora on COVID-19 relaxation measures. We find that, on the one hand, HyEnA achieves higher coverage and precision than a state-of-the-art automated method, when compared on a common set of diverse opinions, justifying the need for human insight. On the other hand, HyEnA requires less human effort and does not compromise quality compared to (fully manual) expert analysis, demonstrating the benefit of combining human and machine intelligence.
Representation of ambiguity in pretrained models and the problem of domain specificity
Authors: Anonymous
Date: 2021-12-17 00:00:00
Proceeding: ACL ARR December 2022 12
Tasks: Specificity

Recent developments in pretrained language models have led to many advances in NLP. These models have excelled at learning powerful contextual representations from very large corpora. Fine-tuning these models for downstream tasks has been one of the most used (and successful) approaches to solving a plethora of NLP problems. But how capable are these models in capturing subtle linguistic traits like ambiguity in their representations? We present results from a probing task designed to test the capability of the models to identify ambiguous sentences under different experimental settings. The results show how different pretrained models fare against each other in the same task. We also explore how domain specificity limits the representational capabilities of the probes.
Enhancing Cross-lingual Prompting with Two-level Augmentation
Authors: Anonymous
Date: 2021-12-17 00:00:00
Proceeding: ACL ARR December 2022 12
Tasks: Cross-Lingual Transfer, Sensitivity, Vocal Bursts Valence Prediction

Prompting approaches show promising results in few-shot scenarios. However, its strength for multilingual/cross-lingual problems has not been fully exploited. Zhao and Schütze (2021) made initial explorations in this direction by presenting that cross-lingual prompting outperforms cross-lingual finetuning. In this paper, we first conduct sensitivity analysis on the effect of each component in cross-lingual prompting and derive Universal Prompting across languages. Based on this, we propose a two-level augmentation framework to further improve the performance of prompt-based cross-lingual transfer. Notably, for XNLI, our method achieves 46.54% with only 16 English training examples per class, significantly better than 34.99% of finetuning.
Cross-modal Contrastive Learning for Speech Translation
Authors: Anonymous
Date: 2021-12-17 00:00:00
Proceeding: ACL ARR December 2022 12
Tasks: Contrastive Learning, Retrieval, Speech-to-Text, Speech-to-Text Translation, Text Retrieval, Translation

How to learn similar representations for spoken utterances and their written text? We believe a unified and aligned representation of speech and text will lead to improvement in speech translation. To this end, we propose ConST, a cross-modal contrastive learning method for end-to-end speech-to-text translation. We evaluate ConST and a variety of previous baselines on multiple language directions (En-De/Fr/Ru) of a popular benchmark MuST-C. Experiments show that the proposed ConST consistently outperforms all previous methods, and achieves the state-of-the-art average BLEU of 28.5. The analysis further verifies that ConST indeed closes the representation gap of different modalities --- its learned representation improves the accuracy of cross-modal text retrieval from 4% to 88%.
HiURE: Hierarchical Exemplar Contrastive Learning for Unsupervised Relation Extraction
Authors: Anonymous
Date: 2021-12-17 00:00:00
Proceeding: ACL ARR December 2022 12
Tasks: Clustering, Contrastive Learning, Relation, Relation Extraction, Sentence

Unsupervised relation extraction aims to extract the relationship between entities from natural language sentences without prior information on relational scope or distribution. Existing works either utilize self-supervised schemes to refine relational feature signals by iteratively leveraging adaptive clustering and classification that provoke gradual drift problems, or adopt instance-wise contrastive learning which unreasonably pushes apart those sentence pairs that are semantically similar. To overcome these defects, we propose a novel contrastive learning framework named HiURE, which has the capability to derive hierarchical signals from relational feature space using cross hierarchy attention and effectively optimize relation representation of sentences under exemplar-wise contrastive learning. Experimental results on two public datasets demonstrate the advanced effectiveness and robustness of HiURE on unsupervised relation extraction when compared with state-of-the-art models.
Training Dynamics for Curriculum Learning: A Study on Monolingual and Cross-lingual NLU
Authors: Anonymous
Date: 2021-12-17 00:00:00
Proceeding: ACL ARR December 2022 12
Tasks: Cross-Lingual Transfer, Natural Language Understanding, Zero-Shot Cross-Lingual Transfer

Curriculum Learning (CL) is a technique of training models via ranking examples in a typically increasing difficulty trend with the aim of accelerating convergence and improving generalisability. However, current approaches for Natural Language Understanding (NLU) tasks use CL to improve in-domain model performance often via metrics that are detached from the model one aims to improve. In this work, instead, we employ CL for NLU by taking advantage of training dynamics as difficulty metrics, i.e. statistics that measure the behavior of the model at hand on data instances during training. In addition, we propose two modifications of existing CL schedulers based on these statistics. Differently from existing works, we focus on evaluating models on out-of-distribution data as well as languages other than English via zero-shot cross-lingual transfer. We show across four XNLU tasks that CL with training dynamics in both monolingual and cross-lingual settings can achieve significant speedups up to 58%. We also find that performance can be improved on challenging tasks, with OOD generalisation up by 8\% and zero-shot cross-lingual transfer up by 1%. Overall, experiments indicate that training dynamics can lead to better performing models and smoother training compared to other difficulty metrics.
Pretrained Language Models Are All You Need For Text-to-SQL Schema Linking
Authors: Anonymous
Date: 2021-12-17 00:00:00
Proceeding: ACL ARR December 2022 12
Tasks: All, Language Modeling, Language Modelling, Text to SQL, Text-To-SQL

The use of Exact Match based  Schema  Linking (EMSL) has become standard in text-to-SQL: many state-of-the-art text-to-SQL models employ EMSL, and their performance drops significantly when the EMSL component is removed. In this work, however, we demonstrate that EMSL reduces robustness, rendering models vulnerable to synonym substitution and typos. Instead of relying on EMSL to make up for deficiencies in question-schema encoding, we show that by utilizing the pre-trained language model as the encoder, we can improve the performance without using EMSL, and thus the model is more robust. Our experiments suggest that EMSL is not the icing on the cake, but it is the one that introduces the vulnerability, and it can be replaced by better input encoding.
A High-Precision Health-relatedness Score for Phrases to Mine Cause-Effect Statements from the Web
Authors: Anonymous
Date: 2021-12-17 00:00:00
Proceeding: ACL ARR December 2022 12

The measurement of the health-relatedness of a phrase is important when mining the web at scale for health information, e.g., when building a search engine or when carrying out health-sociological analyses. We propose a new termhood scoring scheme that allows for the prediction of the health-relatedness of phrases at high precision. An evaluation on several corpora of cause--effect statements (heuristically and professionally labeled) yields about 60\%~recall at over 90\%~precision, outperforming state-of-the-art vocabulary-based approaches and performing on par with BERT while being less resource-demanding. A new resource of over 4~million health-related cause--effect statements is compiled, such as ``Studies show that stress induces insomnia.'', which explicitly connect symptoms (`stress') as claimed causes for conditions (`insomnia'). It consists of over 4~million sentences from more than 2~million unique web pages and 234,000 unique websites.
Challenging America: Modeling language in longer time scales
Authors: Anonymous
Date: 2021-12-17 00:00:00
Proceeding: ACL ARR December 2022 12
Tasks: Cloze Test, Optical Character Recognition (OCR)

The aim of the paper is to apply, for historical texts,  the methodology used commonly to solve various NLP tasks defined for contemporary data, i.e. pre-train and fine-tune large Transformer models. This paper introduces an ML challenge, named Challenging America (ChallAm), based on OCR-ed excerpts from historical newspapers collected from the Chronicling America portal. ChallAm provides a dataset of clippings, labeled with metadata on their origin, and paired with their textual contents retrieved by an OCR tool. Three, publicly available, ML tasks are defined in the challenge: to determine the article date, to detect the location of the issue, and to deduce a word in a text gap (cloze test). Strong baselines are provided for all three ChallAm tasks. In particular, we pre-trained a RoBERTa model from scratch from the historical texts. We also discuss the issues of discrimination and hate-speech present in the historical American texts.
Cloze Evaluation for Deeper Understanding of Commonsense Stories in Indonesian
Authors: Anonymous
Date: 2021-12-17 00:00:00
Proceeding: ACL ARR December 2022 12
Tasks: Cloze Test, Cross-Lingual Transfer, Sentence, Zero-Shot Cross-Lingual Transfer

Story comprehension that involves complex causal and temporal relations is imperative in NLP, but previous studies have focused on English, leaving open the question of how the findings generalize to other languages, such as Indonesian. In this paper, we follow the Story Cloze Test framework of Mostafazadeh et al. (2016) in evaluating story understanding in Indonesian, by constructing a four-sentence story with one correct ending and one incorrect ending. To investigate commonsense knowledge acquisition in language models, we experimented with: (1) a classification task to predict the correct ending; and (2) a generation task to complete the story with a single sentence. We investigate these tasks in two settings: (i) monolingual training and (ii) zero-shot cross-lingual transfer between Indonesian and English.
TASA: Twin Answer Sentences Attack for Adversarial Context Generation in Question Answering
Authors: Anonymous
Date: 2021-12-17 00:00:00
Proceeding: ACL ARR December 2022 12
Tasks: Adversarial Attack, Question Answering, Sentence

We present Twin Answer Sentences Attack (TASA), a novel question answering (QA) adversarial attack method that produces fluent and grammatical adversarial contexts while maintaining its gold answers. Despite phenomenal progresses on general adversarial attacks, few works have investigated the vulnerability and adversarial attack specifically for QA. In this work, we first investigate the biases in the existing models and discover that they heavily rely on keyword matching and ignore the relevant entities from the question. TASA explores the two biases above and attacks the target model in two folds: (1) lowering the model's confidence on the gold answer with a perturbed answer sentence; (2) misguiding the model towards a wrong answer with a distracting answer sentence. Equipped with designed beam search and filtering methods, TASA is able to attack the target model efficiently while sustaining the quality of contexts. Extensive experiments on four QA datasets and human evaluations demonstrate that TASA generates substantial-high-quality attacks than existing textual adversarial attack methods.
Diagnosing Vision-and-Language Navigation: What Really Matters
Authors: Anonymous
Date: 2021-12-17 00:00:00
Proceeding: ACL ARR December 2022 12
Tasks: Diagnostic, Object, Vision and Language Navigation

Vision-and-language navigation (VLN) is a multimodal task where an agent follows natural language instructions and navigates in visual environments. Multiple setups have been proposed, and researchers apply new model architectures or training techniques to boost navigation performance. However, there still exist non-negligible gaps between machines' performance and human benchmarks. Moreover, the agents' inner mechanisms for navigation decisions remain unclear. To the best of our knowledge, how the agents perceive the multimodal input is under-studied and needs investigation. In this work, we conduct a series of diagnostic experiments to unveil agents' focus during navigation. Results show that indoor navigation agents refer to both object and direction tokens when making decisions. In contrast, outdoor navigation agents heavily rely on direction tokens and poorly understand the object tokens. The differences in dataset designs and the visual features lead to distinct behaviors on visual environment understanding. Many models claim that they can align object tokens with specific visual targets when it comes to vision-and-language alignments. We find unbalanced attention on the vision and text input and doubt the reliability of such cross-modal alignments.
AWS-EP: A Multi-Task Prediction Approach for MBTI/Big5 Personality Tests
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Sociology

Personality and preferences are essential variables in computational sociology and social science. They describe differences between people at both individual and group levels. In recent years, automated approaches to detect personality traits have received much attention due to the massive availability of individuals' digital footprints. Furthermore, researchers have demonstrated a strong link between personality traits and various downstream tasks such as personalized filtering, profile categorization, and profile embedding. Therefore, the detection of individuals' personality traits has become a critical process for improving the performance of different tasks. In this paper, we build on the importance of the individual personality and propose a novel multitask modeling approach that understands and models the user personality based on its textual posts and comments within a multimedia framework. Experiments and results demonstrate that our model outperforms state-of-the-art performances across multiple famous personality datasets.
Plot Writing From Pre-Trained Language Models
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Story Generation

Pre-trained language models (PLMs) fail to generate long-form narrative text because they do not consider global structure. As a result, the generated texts are often incohesive, repetitive, or lack content. Recent work in story generation reintroduced explicit content planning in the form of prompts, keywords, or semantic frames. Trained on large parallel corpora, these models can generate more logical event sequences and thus more contentful stories. However, these intermediate representations are often not in natural language and cannot be utilized by PLMs without fine-tuning. We propose generating story plots using off-the-shelf PLMs while maintaining the benefit of content planning to generate cohesive and contentful stories. Our proposed method, ScratchPlot, first prompts a PLM to compose a content plan. Then, we generate the story's body and ending conditioned on the content plan. Furthermore, we take a generate-and-rank approach by using additional PLMs to rank the generated (story, ending) pairs. We benchmark our method with various baselines and achieved superior results in both human and automatic evaluation.
Adaptive Transfer Learning for Multi-Label Emotion Classification
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Classification, Emotion Classification, Transfer Learning

In this study, we explore how data annotated with different taxonomies can be used to improve multi-label emotion classification. We propose a novel transfer learning framework to model the interaction between emotion categories, and introduce an adaptive aggregation mechanism to fuse the information from different taxonomies. The cross-taxonomy emotion interaction allows the source and target tasks to collaborate effectively, resulting in more accurate predictions. The experimental results on the SemEval-2018 dataset show that our approach can effectively boost the performance gain brought by transfer learning, and significantly outperforms existing methods.
Deep Continuous Prompt for Contrastive Learning of Sentence Embeddings
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Contrastive Learning, Language Modeling, Language Modelling, Sentence, Sentence Embeddings

The performance of sentence representation has been remarkably improved by the framework of contrastive learning. However, recent works still require full fine-tuning, which is quite inefficient for large-scaled pre-trained language models. To this end, we present a novel method which freezes the whole language model and only optimizes the prefix deep continuous prompts. It not only tunes around 0.1\% parameters of the original language model, but avoids the cumbersome computation of searching handcrafted prompts. Experimental results show that our proposed DCPCSE outperforms the state-of-the-art method SimCSE by a large margin. We raise the performance of unsupervised BERT$_{base}$ and supervised RoBERTa$_{large}$ by 2.24 and 1.00 points, respectively. Our code will be released at Github.
Topic Modeling with Topological Data Analysis
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Clustering, Topological Data Analysis, Word Embeddings

Recent unsupervised topic modelling approaches that use clustering techniques on word, token or document embeddings can extract coherent topics. However, a common limitation of such approaches is that they reveal nothing about inter-topic relationships which are essential in many real-world application domains. We present an unsupervised topic modelling method which harnesses Topological Data Analysis (TDA) to extract a topological skeleton of the manifold upon which contextualised word embeddings lie. We demonstrate that our approach, which performs on par with a recent baseline, is able to construct a network of coherent topics together with meaningful relationships between them.
Feasibility of BERT Embeddings For Domain-Specific Knowledge Mining
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Benchmarking, Language Modelling, Word Embeddings

Extracting information from large corpora of unstructured text using computational methods presents a challenge. Tshitoyan et al. (2019) demonstrated that unsupervised mathematical word-embeddings produced by a static language model could be utilized to uncover `latent knowledge' within a materials science corpus. The rise of contextualized and massively pre-trained language models like BERT have seen static models becoming surpassed for most NLP tasks. Nevertheless, due to innate architectural and use differences, BERT requires adaptation for knowledge mining. This study tests the suitability of BERT-derived word embeddings for knowledge mining purposes. It utilizes a variation of the approach described by Bommasani et al. (2020) for creating static-equivalent vectors from multiple contextualized word representations. It is conducted using a biomedical corpus, a biomedical BERT variation and validated using domain-specific intrinsic benchmarking tools. Novel, layer-wise BERT performance characteristics are demonstrated. A key finding is that layer-wise intrinsic performance differs for nouns and verbs. Performance also varies according to whether a word of interest belongs to BERT's native vocabulary or is built from sub-word representations: BERT-native representations perform best when extracted from earlier layers, while representations requiring multiple tokens perform best when extracted from the middle-to-latter model layers.