pwc-archive/papers-with-abstracts huggingface dataset:
Are You Really Okay? A Transfer Learning-based Approach for Identification of Underlying Mental Illnesses
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Transfer Learning

Evidence has demonstrated that there are similarities in language use across mental illnesses, and oftentimes underlying mental illnesses are also identified by healthcare providers following initial diagnosis. We review literature to identify semantic correlations and similarities among mental illnesses. We also present a novel transfer learning-based approach that learns from data associated with known mental illnesses and predicts the presence of those previously unknown by the model.  Our model achieves a predictive accuracy of 85%, providing promising evidence that language models can harness learned patterns from known mental illnesses to aid in their prediction of others that may lie latent.
End-To-End Sign Language Translation via Multitask Learning
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Decoder, Sign Language Recognition, Sign Language Translation, Translation

Sign language translation (SLT) is usually seen as a two-step process of continuous sign language recognition (CSLR) and gloss-to-text translation. We propose a novel, Transformer-based architecture to jointly perform CSLR and sign-translation in an end-to-end fashion. We extend the ordinary Transformer decoder with two channels to support multitasking, where each channel is devoted to solve a particular problem. To control the memory footprint of our model, channels are designed to share most of their parameters among each other. However, each channel still has a dedicated set of parameters which is fine-tuned with respect to the channel's task. In order to evaluate the proposed architecture, we focus on translating German signs into English sequences and use the RWTH-PHOENIX-Weather 2014 T corpus in our experiments. Evaluation results indicate that the mixture of information provided by the multitask decoder was successful and enabled us to achieve superior performance in comparison to other SLT models.
Meta-learning via Language Model In-context Tuning
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: In-Context Learning, Inductive Bias, Language Modeling, Language Modelling, Meta-Learning, model, text-classification, Text Classification

The goal of meta-learning is to learn to adapt to a new task with only a few labeled examples. Inspired by the recent progress in large language models, we propose $\textit{in-context tuning}$ (ICT), which recasts task adaptation and prediction as a simple sequence prediction problem: to form the input sequence, we concatenate the task instruction, labeled in-context examples, and the target input to predict; to meta-train the model to learn from in-context examples, we fine-tune a pre-trained language model (LM) to predict the target label given the input sequence on a collection of tasks.
We benchmark our method on two collections of text classification tasks: LAMA and BinaryClfs. Compared to MAML which adapts the model through gradient descent, our method leverages the inductive bias of pre-trained LMs to perform pattern matching, and outperforms MAML by an absolute $6\%$ average AUC-ROC score on BinaryClfs, gaining more advantage with increasing model size. Compared to non-fine-tuned in-context learning (i.e. prompting a raw LM), in-context tuning meta-trains the model to learn from in-context examples. On BinaryClfs, ICT improves the average AUC-ROC score by an absolute $10\%$, and reduces the variance due to example ordering by 6x and example choices by 2x.
LipKey: A Large-Scale News Dataset with Abstractive Keyphrases and Their Benefits for Summarization
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Document Summarization

Summaries, keyphrases, and titles are different ways of concisely capturing the content of a document. While most previous work has addressed them separately, in this work, we jointly use the three elements via multi-task training and training as joint structured inputs, in the context of document summarization. We release LipKey, the largest news corpus with human-written summaries, titles, and keyphrases, as well as being the first large-scale Indonesian keyphrase dataset. We find that including keyphrases and titles as additional context to the source document improves transformer-based summarization models.
Constructing Phrase-level Semantic Labels to Form Multi-GrainedSupervision for Image-Text Retrieval
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Form, Image-text Retrieval, Representation Learning, Retrieval, Sentence, Text Retrieval

Existing research for image text retrieval mainly relies on sentence-level supervision to distinguish matched and mismatched sentences for a query image. However, semantic mismatch between an image and sentences usually happens in finer grain, i.e., phrase level. In this paper, we explore to introduce additional phrase-level supervision for the better identification of mismatched units in the text. In practice, multi-grained semantic labels are automatically constructed for a query image in both sentence-level and phrase-level. We construct text scene graphs for the matched sentences and extract entities and triples as the phrase-level labels. In order to integrate both supervision of sentence-level and phrase-level, we propose Semantic Structure Aware Multimodal Transformer (SSAMT) for multi-modal representation learning. Inside the SSAMT, we utilize different kinds of attention mechanisms to enforce interactions of multi-grain semantic units in both sides of vision and language. For the training, we propose multi-scale matching losses from both global and local perspectives, and penalize mismatched phrases. Experimental results on MS-COCO and Flickr30K show the effectiveness of our approach compared to some state-of-the-art models.
The patient is more dead than alive: exploring the current state of the multi-document summarisation of the biomedical literature
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11

Although multi-document summarisation (MDS) of the biomedical literature is a highly valuable task that has recently attracted substantial interest, evaluation of the quality of biomedical summaries lacks consistency and transparency. In this paper, we examine the summaries generated by two current models in order to understand the deficiencies of existing evaluation approaches in the context of the challenges that arise in the MDS task. Based on this analysis, we propose a new approach to human evaluation and identify several challenges that must be overcome to develop effective biomedical MDS systems.
Knowledge Graph is in Rescue: Task Oriented Dialogue System for Response Generation without NLU and DM
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Dialogue Generation, Dialogue Management, Management, Natural Language Understanding, Response Generation

Natural language understanding (NLU) and dialogue management (DM) are the standard prerequisites for response generation in a task-oriented dialogue system. In the existing literature, NLU and DM have been tackled as two independent tasks, requiring separate labeled data. Besides this problem of additional data requirements, NLU and DM also introduce errors in the pipe-lined processing of dialogue. Direct generation of responses from the user utterances without using any intermediate NLU and DM modules is, thus, a worthy goal. To accomplish this, the model should be able to (implicitly) understand the intent of the user, fetch the appropriate data from the knowledge base, (implicitly) decide an action by looking at the fetched data and conversation history, and finally, generate the response. In this work, we build an end-to-end dialogue generation system that does not require NLU and DM components or their associated labels in the data. We propose an effective technique based on the pre-trained GPT-2 and Graph Convolution Network (GCN) that takes conversation history and knowledge graph as input and produces appropriate responses. Experiments on three benchmark datasets viz. MultiWOZ, InCar Assist, and CamRest show that our proposed model achieves performance comparable to state-of-the-art systems without using the NLU and DM labels. Human evaluation conducted on the outputs also confirms that our proposed model generates highly fluent and contextually relevant responses.
Right for the Right Reason: Evidence Extraction for Trustworthy Tabular Reasoning
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11

When pre-trained contextualized embeddings-based models developed for unstructured data are adapted for structured tabular data, they perform admirably. However, recent probing studies show that these models use spurious correlations and often ignore or focus on wrong evidence to predict labels. To study this issue, we introduce the task of Trustworthy Tabular Reasoning, where a model needs to extract evidence to be used for reasoning, in addition to predicting the label. As a case study, we propose a two-stage sequential prediction approach, which includes an evidence extraction and an inference stage. To begin, we crowdsource evidence row labels and develop several unsupervised and supervised evidence extraction strategies for InfoTabS, a tabular NLI benchmark. Our evidence extraction strategy outperforms earlier baselines. On the downstream tabular inference task, using the automatically extracted evidence as the only premise, our approach outperforms prior benchmarks.
Data Contamination: From Memorization to Exploitation
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Memorization

It is common nowadays to train NLP models on massive web-based datasets. Previous works have shown that these datasets sometimes contain downstream test sets, a phenomenon typically referred to as "data contamination". It is not clear however to what extent models exploit the contaminated data for downstream tasks. In this paper we present a principled method to study this question. We pretrain BERT models on joint corpora of Wikipedia and labeled downstream datasets, and fine-tune them on the relevant task. 
Comparing performance between samples seen and unseen during pretraining enables us to define and quantify levels of memorization and exploitation.
Our experiments with two models and three downstream tasks indicate that exploitation exists in some cases, but in others the models  memorize the contaminated data, but do not exploit it. We show these two measures are affected by different factors such as contaminated data occurrences, model size, and random seeds. Our results highlight the importance of analyzing massive web-scale datasets to verify that progress in NLP is obtained by better language understanding and not better data exploitation.
PESTO: A Post-User Fusion Network for Rumour Detection on Social Media
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Data Augmentation, Misinformation, Rumour Detection

Rumour detection on social media is an important topic due to the challenges of misinformation propagation and slow verification of misleading information.
Most previous work focus on the response posts on social media, ignoring the useful characteristics of involved users and their relations.
In this paper, we propose a novel framework, Post-User Fusion Network (PESTO), which models the patterns of rumours from both post diffusion and user social networks. Specifically, we propose a novel Chronologically-masked Transformer architecture to model both temporal sequence and diffusion structure of rumours, and apply a Relational Graph Convolutional Network to model the social relations of involved users, with a fusion network based on self-attention mechanism to incorporate the two aspects. Additionally, two data augmentation techniques are leveraged to improve the robustness and accuracy of our models. Empirical results on several benchmarks show the superiority of the proposed method.
LOPS: Learning Order Inspired Pseudo-Label Selection for Weakly Supervised Text Classification
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Memorization, Pseudo Label, text-classification, Text Classification

Iterative self-training is a popular framework in weakly supervised text classification that involves bootstrapping a deep neural classifier from heuristic pseudo-labels. The quality of pseudo-labels, especially the initial ones, is crucial to final performance but they are inevitably noisy due to their heuristic nature, so selecting the correct ones has a huge potential for performance boost. One straightforward solution is to select samples based on the softmax probability scores corresponding to their pseudo-labels. However, we show through our experiments that such methods are ineffective and unstable due to the erroneously high-confidence predictions from poorly calibrated models. Recent studies on the memorization effects of deep neural models suggest that these models first memorize training samples with clean labels and then those with noisy labels. Inspired by this observation, we propose a novel pseudo-label selection method LOPS that takes learning order of samples into consideration. We hypothesize that the learning order reflects the probability of wrong annotation in terms of ranking, and therefore, select the top samples that are learnt earlier. LOPS can be viewed as a strong performance-boost plug-in to most of existing weakly-supervised text classification methods, as confirmed in extensive experiments on six real-world datasets.
Do Pre-trained Models Benefit Knowledge Graph Completion? A Reliable Evaluation and a Reasonable Approach
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Knowledge Graph Completion, Link Prediction

In recent years, pre-trained language models (PLMs) have been shown to capture factual knowledge from massive texts, which encourages the proposal of PLM-based knowledge graph completion (KGC) models. However, these models are still quite behind the SOTA KGC models in terms of performance. In this work, we find two main reasons for the weak performance: (1) Inaccurate evaluation setting. The evaluation setting under the closed-world assumption (CWA) may underestimate the PLM-based KGC models since they introduce more external knowledge; (2) Inappropriate utilization of PLMs. Most PLM-based KGC models simply splice the labels of entities and relations as inputs, leading to incoherent sentences that do not take full advantage of the implicit knowledge in PLMs. To alleviate these problems, we highlight a more accurate evaluation setting under the open-world assumption (OWA), which manual checks the correctness of knowledge that is not in KGs. Moreover, motivated by prompt tuning, we propose a novel PLM-based KGC model named PKGC. The basic idea is to convert each triple and its support information into natural prompt sentences, which is further fed into PLMs for classification. Experiment results on two KGC datasets demonstrate OWA is more reliable for evaluating KGC, especially on the link prediction, and the effectiveness of our PKCG model on both CWA and OWA settings.
Debiased Contrastive Learning of Unsupervised Sentence Representations
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Contrastive Learning, Semantic Textual Similarity, Sentence

Recently, contrastive learning has shown effectiveness in fine-tuning pre-trained language models (PLM) to derive sentence representations, which pulls augmented positive examples together to improve the alignment while pushing apart irrelevant negatives for the uniformity of the whole representation space. However, previous works mostly sample negatives from the batch or training data at random. It may cause sampling bias that improper negatives (\eg false negatives and anisotropy representations) will be learned by sentence representations, and hurt the uniformity of the representation space. To solve it, we present a new framework \textbf{DCLR} to alleviate the influence of sampling bias. In DCLR, we design an instance weighting method to punish false negatives and generate noise-based negatives to guarantee the uniformity of the representation space. Experiments on 7 semantic textual similarity tasks show that our approach is more effective than competitive baselines. Our codes and data will be released to reproduce all the experiments.
Improving Neural Models for Radiology Report Retrieval with Lexicon-based Automated Annotation
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Information Retrieval, Retrieval, Sentence, Triplet

Many clinical informatics tasks that are based on electronic health records need relevant patient cohorts to be selected based on findings, symptoms, and diseases. Frequently, these conditions are described in radiology reports which can be retrieved using information retrieval (IR) methods. The latest of these techniques utilize neural IR models such as BERT trained on clinical text. However, these methods still lack semantic understanding of the underlying clinical conditions as well as ruled out findings, resulting in poor precision during retrieval. In this paper we combine clinical finding detection with supervised query match learning. Specifically, we use lexicon-driven concept detection to detect relevant findings in sentences. These findings are used as queries to train a Sentence-BERT (SBERT) model using triplet loss on matched and unmatched query-sentence pairs. We show that the proposed supervised training task remarkably improves the retrieval performance of SBERT. The trained model generalizes well to unseen queries and reports from different collections.
Generated Knowledge Prompting for Commonsense Reasoning
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Language Modeling, Language Modelling, Open-Ended Question Answering

It remains an open question whether incorporating external knowledge benefits commonsense reasoning while maintaining the flexibility of pretrained sequence models. To investigate this question, we develop generated knowledge prompting, which consists of generating knowledge from a language model, then providing the knowledge as additional input when answering a question. Our method does not require task-specific supervision for knowledge integration, or access to a structured knowledge base, yet it improves performance of large-scale, state-of-the-art models on four commonsense reasoning tasks, achieving state-of-the-art results on numerical commonsense (NumerSense), general commonsense (CommonsenseQA 2.0), and scientific commonsense (QASC) benchmarks. Generated knowledge prompting highlights large-scale language models as flexible sources of external knowledge for improving commonsense reasoning.
Our code is available at \url{github.com/anonymous_repo}.
Extreme Multi-label Text Classification with Multi-layer Experts
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Classification, Multi Label Text Classification, Multi-Label Text Classification, text-classification, Text Classification

Extreme multi-label text classification (XMTC) is the task of tagging each document with the relevant labels from a very large space of predefined categories, which presents an open challenge in the recent development of neural classifiers.  Popular Transformer-based XMTC methods typically use the last-layer features to represent the document and to match it against candidate labels.  We argue that the last-layer features may not be sufficient for predicting labels at different levels of semantic granularity, and that multi-layer features may offer a better choice instead.  Based on this insight we propose a novel multi-expert model, namely ME-XML (Multiple Experts for XMTC), which combines multi-layer embeddings in Transformer for improving the prediction power of the model.
Empirical Evaluation of Topic Zero- and Few-Shot Learning for Stance Dissonance Detection
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Few-Shot Learning, Stance Detection

We address stance dissonance detection, the task of detecting conflicting stance between two input statements. Computational models for traditional stance detection have typically been trained to indicate pro/cons for a given target topic  (e.g. gun control)  and thus do not generalize well to new topics. In this paper, we systematically evaluate the generalizability of this task to situations where examples of the topic have not been seen at all (zero-shot) or only a few times (few-shot). We first build a large-scale dataset of stance dissonance detection from an online debate platform, consisting of 23.8k pairs of statements from  34 diverse topics. We show that stance dissonance detection models trained only on a small number of non-target topics already perform as well as those trained on a target topic. We also show that adding more non-target topics further boosts the performance, indicating the generalizability of non-target topics to a target topic in the stance dissonance detection task.
Probing Schema Linking Information from Pre-trained Language Models for Text-to-SQL Parsing
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: SQL Parsing, Text to SQL, Text-To-SQL

The importance of building text-to-SQL parsers which can be applied to new databases has long been acknowledged, and a critical step to achieve this goal is schema linking, i.e., properly recognizing mentions of unseen columns or tables when generating SQLs. In this work, we propose a novel framework to elicit relational structures from large-scale pre-trained language models (PLMs) via a probing procedure based on Poincaré distance metric, and use the induced relations to augment current graph-based parsers for better schema linking. Compared with commonly-used rule-based methods for schema linking, we found that probing relations can robustly capture semantic correspondences, even when surface forms of mentions and entities differ. Moreover, our probing procedure is entirely unsupervised and requires no additional parameters. Extensive experiments show that our framework outperforms strong baselines on three benchmarks, and sets new state-of-the-art performance on two of them. We empirically verify that our probing procedure can indeed find desired relational structures through qualitative analysis.  For reproducibility, we will release our code and data upon the publication of this paper.