pwc-archive/papers-with-abstracts huggingface dataset:
CONQRR: Conversational Query Rewriting for Retrieval with Reinforcement Learning
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Conversational Question Answering, Passage Retrieval, Question Answering, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Retrieval

Compared to standard retrieval tasks, passage retrieval for conversational question answering (CQA) poses new challenges in understanding the current user question, as each question needs to be interpreted within the dialogue context. Moreover, it can be expensive to re-train well-established retrievers such as search engines that are originally developed for non-conversational queries. To facilitate their use, we develop a query rewriting model CONQRR that rewrites a conversational question in the context into a standalone question. It is trained with a novel reward function to directly optimize towards retrieval using reinforcement learning and can be adapted to any fixed retriever. We show that CONQRR achieves state-of-the-art results on a recent open-domain CQA dataset containing conversations from three different sources, and is effective for two different fixed retrievers. Our extensive analysis also shows the robustness of CONQRR to out-of-domain dialogues as well as to limited query rewriting supervision.
Logical Story Representations via FrameNet + Semantic Parsing
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Formal Logic, Question Answering, Semantic Parsing

We present a means of obtaining rich semantic representations of stories by combining neural FrameNet identification, a formal logic-based semantic parser, and a hierarchical event schema representation. The final schematic representation of the story abstracts constants to variables, preserving their types and relationships to other individuals in the story. All identified FrameNet frames are incorporated as temporally bound ``episodes'' and related to one another in time. The semantic role information from the frames is also incorporated into the final schema's type constraints. We describe this system as well as its possible applications to question answering and open-domain event schema learning.
Boosted Dense Retriever
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Quantization, Retrieval

We propose DrBoost, a dense retrieval ensemble inspired by boosting. DrBoost is trained in stages: each component model is learned sequentially and specialized by focusing only on retrieval mistakes made by the current ensemble. The final representation is the concatenation of the output vectors of all the component models, making it a drop-in replacement for standard dense retrievers at test time. DrBoost enjoys several advantages compared to standard dense retrieval models. It produces representations which are 4x more compact, while delivering comparable retrieval results. It also performs surprisingly well under approximate search with coarse quantization, reducing latency and bandwidth needs by another 4x. In practice, this can make the difference between serving indices from disk versus from memory, paving the way for much cheaper deployments.
Analyzing Modality Robustness in Multimodal Sentiment Analysis
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Diagnostic, Multimodal Sentiment Analysis, Sentiment Analysis

Building robust multimodal models are crucial to achieving reliable deployment in the wild. Despite its importance, less attention has been paid to identifying and improving the robustness of Multimodal Sentiment Analysis (MSA) models. In this work, we hope to address that by (i) Proposing simple diagnostic checks for modality robustness in a trained multimodal model. Using these checks, we find MSA models to be highly sensitive to a single modality, which creates issues in their robustness; (ii) We analyze well-known robust training strategies to alleviate the issues. Critically, we observe that robustness can be achieved without compromising on the original performance. We hope our extensive study--performed across five models and two benchmark datasets--and proposed procedures would make robustness an integral component in MSA research. Our diagnostic checks and robust training solutions are simple to implement and shall be released at https://github.com/XXXX.
CheckDST: Measuring Real-World Generalization of Dialogue State Tracking Performance
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Dialogue State Tracking

Neural models that extend the pretrain-then-finetune paradigm continue to achieve new state-of-the-art results in dialogue state tracking (DST) benchmarks on joint goal accuracy (JGA). However, motivated by CheckList (Ribeiro et al. 2020), we argue for a holistic assessment of DST models since JGA is unable to capture robustness to the inevitable test-time distribution shifts. To this end, we build on recent work on robustness testing in task-oriented dialogue and introduce CheckDST, an instantiation of CheckList for DST that quantifies robustness with test set augmentations and new metrics that measure consistency. Using CheckDST, we are able to extensively compare state-of-the-art DST models, finding that, although span-based classification models achieve slightly better JGA on the original test set than generation models, they are significantly less robust to distribution shift. Secondly, we observe that while stopping training early, e.g. at the first epoch, hurts JGA, the resulting models are significantly more robust to distribution shift. Lastly, guided by the weaknesses exposed by CheckDST, we explore training DST models that simultaneously boost JGA and CheckDST metrics and report preliminary success with PrefineDST, a simple generation model pretrained with non-target datasets to internalize reasoning skills relevant to dialogue state tracking.
Masked Measurement Prediction: Learning to Jointly Predict Quantities and Units from Textual Context
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1

Physical measurements constitute a large portion of numbers in academic papers, engineering reports, and web tables. Current benchmarks fall short of properly evaluating numeracy of pretrained language models on measurements, hindering research on developing new methods and applying them to numerical tasks. To that end, we introduce a novel task, Masked Measurement Prediction (MMP), where a model learns to reconstruct a number together with its associated unit given masked text. MMP is useful for both training new numerically informed models as well as evaluating numeracy of existing systems. To address this task, we introduce a new Generative Masked Measurement (GeMM) model that jointly learns to predict numbers along with their units. We perform fine-grained analyses comparing our model with various ablations and baselines. We use linear probing of traditional pretrained transformer models (RoBERTa) to show that they significantly underperform jointly trained number-unit models, highlighting the difficulty of this new task and the benefits of our proposed pretraining approach. We hope this framework accelerates the progress towards building more robust numerical reasoning systems in the future.
Crossmodal-3600: A Massively Multilingual Multimodal Evaluation Dataset
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Image Captioning, Model Selection, Translation

Research in massively multilingual image captioning has been severely hampered by a lack of high-quality evaluation datasets. In this paper we present and make available the Crossmodal-3600 dataset, a geographically diverse set of 3600 images each of them annotated with human-generated reference captions in 36 languages. We select a representative set of images from across the world for this dataset, and annotate it with captions that achieve consistency in terms of style across all languages, while avoiding annotation artifacts due to direct translation. We apply this benchmark to model selection for massively multilingual image captioning models, and show superior correlation results with human evaluations when using the Crossmodal-3600 dataset as golden references for automatic metrics.
Guiding Topic Flows in the Generative Chatbot by Enhancing the ConceptNet with the Conversation Corpora
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Chatbot, Response Generation, World Knowledge

Human conversations consist of reasonable and natural topic flows, which are observed as the shifts of the mentioned concepts across utterances.Previous chatbots that incorporate the external commonsense knowledge graph prove that modeling the concept shifts can effectively alleviate the dull and uninformative response dilemma.However, there still exists a gap between the concept relations in the natural conversation and those in the external commonsense knowledge graph. Specifically, the concept relations in the external commonsense knowledge graph are not intuitively built from the conversational scenario but the world knowledge, which makes them insufficient for the chatbot construction.To bridge the above gap, we propose the method to supply more concept relations extracted from the conversational corpora and build an enhanced concept graph for the chatbot construction. We then introduce the enhanced graph to the response generation process with a designed network.Experimental results on the Reddit conversation dataset indicate our proposed method significantly outperforms strong baseline systems and achieves new SOTA results.Further analysis individually proves the effectiveness of the enhanced concept graph.
Enhanced Protein-Protein Interactions Extraction from the Literature using Entity Type- and Position-aware Representation
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Position, Relation, Relation Classification, Relation Extraction

Since protein-protein interactions (PPIs) are crucial to understanding living systems, harvesting these data is essential to probe the development of diseases and to understand gene/protein functions and biological processes. Some curated datasets exist containing PPI data derived from the literature and other sources (e.g., IntAct, BioGrid, DIP and HPRD), but these are far from exhaustive and their maintenance is a labor intensive process.On the other hand, machine learning (ML) methods to automate PPI knowledge extraction from the scientific literature have been limited by a shortage of appropriate annotated data.In this work, we create a unified multi-source PPI corpora with vetted interaction definitions, and augmented by binary interaction type labels.We also present a Transformer-based deep learning method, exploiting entity type and positional information for relation representation to improve relation classification performance.We evaluated our model's performance on three widely studied relation extraction datasets from biology and computer science domains as well as our work's target PPI datasets to observe the effectiveness of the representation to relation extraction tasks in various domains, and found it to outperform prior state-of-the-art (SOTA) models.
MetaICL: Learning to Learn In Context
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Few-Shot Learning, In-Context Learning, Language Modeling, Language Modelling, Multi-Task Learning, Natural Language Inference, Question Answering

We introduce MetaICL (Meta-training for In-Context Learning), a new meta-training framework for few-shot learning where a pretrained language model is tuned to do in-context learning on a large set of training tasks. This meta-training enables the model to more effectively learn a new task in context at test time, by simply conditioning on a few training examples with no parameter updates or task-specific templates. We experiment on a large, diverse collection of tasks consisting of 142 NLP datasets including classification, question answering, natural language inference, paraphrase detection and more, across seven different meta-training/target splits. MetaICL outperforms a range of baselines including in-context learning without meta-training and multi-task learning followed by zero-shot transfer. We find that the gains are particularly significant for target tasks that have domain shifts from the meta-training tasks, and that using a diverse set of the meta-training tasks is key to improvements. We also show that MetaICL approaches (and sometimes beats) the performance of models fully finetuned on the target task training data, and outperforms much bigger models with nearly 8x parameters.
Language Models for Code-switch Detection of te reo Māori and English in a Low-resource Setting
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Word Embeddings

Te reo Māori, New Zealand's only indigenous language, is code-switched with English. Most Māori speakers are bilingual, and the use of  Māori is increasing in New Zealand English. Unfortunately, due to the minimal availability of resources, including digital data, Māori is under-represented in technological advances. Cloud-based systems such as Google and Azure support Māori language detection. However, we provide experimental evidence to show that the accuracy of such systems is low when detecting Māori. Hence, with the support of Māori community, we collect Māori i and bilingual data to use natural language processing (NLP) to improve Māori language detection. We train bilingual sub-word embeddings and provide evidence to show that our bilingual embeddings improve overall accuracy compared to the publicly-available monolingual embeddings. This improvement has been verified for various NLP tasks using three bilingual databases containing formal transcripts and informal social media data. We also show that BiLSTM with bilingual sub-word embeddings outperforms large-scale contextual language models such as BERT on down streaming tasks of detecting Māori language. The best accuracy of 87% was obtained using BiLSTM with bilingual embeddings for detecting code-switch points of bilingual sentences.
Tree Knowledge Distillation for Compressing Transformer-Based Language Models
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Knowledge Distillation

Knowledge distillation has emerged as a promising technique for compressing neural language models. However, most knowledge distillation methods focus on extracting the ``knowledge'' from a teacher network to guide the training of a student network, ignoring the ``requirements'' of the student.  In this paper, we introduce Tree Knowledge Distillation for Transformer-based teacher and student models, which allows student to actively extract its ``requirements'' via a tree of tokens. In specific, we first choose the |[CLS]| token at the output layer of Transformer in student as the root of the tree.  We choose tokens with the highest values in the row for |[CLS]| of the attention feature map at the second last layer as the children of the root.  Then we choose children of these nodes in their corresponding rows of the attention feature map at the next layer, respectively.  Later, we connect layers of Transformer in student to corresponding layers in teacher by skipping every $t$ layers. At last, we improve the loss function by adding the summed mean squared errors between the embeddings of the tokens in the tree. The experiments show that tree knowledge distillation achieves competitive performance for compressing BERT among other knowledge distillation methods in GLUE benchmark.
Quantifying Adaptability in Pre-trained Language Models with 500 Tasks
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Language Modeling, Language Modelling, Logical Reasoning, Memorization, World Knowledge

When a neural language model (LM) is adapted to perform a new task, what aspects of the task predict the eventual performance of the model? In NLP, systematic features of LM generalization to individual examples are well characterized, but systematic aspects of LM adaptability to new tasks are not nearly as well understood. We present a large-scale empirical study of the features and limits of LM adaptability using a new benchmark, TaskBench500, built from 500 procedurally generated sequence modeling tasks. These tasks combine core aspects of language processing, including lexical semantics, sequence processing, memorization, logical reasoning, and world knowledge. Using TaskBench500, we evaluate three facets of adaptability, finding that: (1) adaptation procedures differ dramatically in their ability to memorize small datasets; (2) within a subset of task types, adaptation procedures exhibit compositional adaptability to complex tasks; and (3) failure to match training label distributions is explained by mismatches in the intrinsic difficulty of predicting individual labels. Our experiments show that adaptability to new tasks, like generalization to new examples, can be systematically described and understood, and we conclude with a discussion of additional aspects of adaptability that could be studied using the new benchmark.
Visual Commonsense in Pretrained Unimodal and Multimodal Models
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Attribute, World Knowledge

Our commonsense knowledge about objects includes their typical visual attributes; we know that bananas are typically yellow or green, and not purple. Text and image corpora, being subject to reporting bias, represent this world-knowledge to varying degrees of faithfulness. In this paper, we investigate to what degree unimodal (language-only) and multimodal (image and language) models capture a broad range of visually salient attributes. To that end, we automatically extract a visually-grounded commonsense dataset covering 5 property types (color, shape, material, size, and visual co-occurrence) for over 5000 subjects. We validate this dataset by showing that our grounded color data correlates much better than ungrounded text-only data with crowdsourced color judgments provided by Paik et al. (2021). We then use our dataset to evaluate pre-trained unimodal models and multimodal models. Our results show that multimodal models better reconstruct attribute distributions, but are still subject to reporting bias. Moreover, increasing model size does not enhance performance, suggesting that the key to visual commonsense lies in the data.
KCD: Knowledge Walks and Textual Cues Enhanced Political Perspective Detection in News Media
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Articles, Knowledge Graphs, Representation Learning

    Political perspective detection has become an increasingly important task that can help combat echo chambers and political polarization. Previous approaches generally focus on leveraging textual content to identify stances, while they fail to reason with background knowledge or leverage the rich semantic and syntactic textual labels in news articles. In light of these limitations, we propose KCD, a political perspective detection approach to enable multi-hop knowledge reasoning and incorporate textual cues as paragraph-level labels. Specifically, we firstly generate random walks on external knowledge graphs and infuse them with news text representations. We then construct a heterogeneous information network to jointly model news content as well as semantic, syntactic and entity cues in news articles. Finally, we adopt relational graph neural networks for graph-level representation learning and conduct political perspective detection. Extensive experiments demonstrate that our approach outperforms state-of-the-art methods on two benchmark datasets. We further examine the effect of knowledge walks and textual cues and how they contribute to our approach's data efficiency.
Competition Dynamics in the Meme Ecosystem
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Diversity

The creation and sharing of memes is a common modality of online social interactions. The goal of the present work is to better understand the collective dynamics of memes in this accelerating and competitive environment. By taking an ecological perspective and tracking the meme-text from 352 popular memes over the entirety of Reddit, we are able to show that the frequency of memes has scaled almost exactly with the total amount of content created over the past decade. One consequence of limited human attention in the face of a growing number of memes is that the diversity of these memes has decreased at the community level, albeit slightly, in the same period. Another consequence is that the average lifespan of a meme has decreased dramatically, which is further evidence of an increase in competition among memes and a decreasing collective attention span.
Cross-lingual Lifelong Learning
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Continual Learning, Lifelong learning, Transfer Learning

The longstanding goal of multi-lingual learning has been to develop a universal cross-lingual model that can withstand the changes in multi-lingual data distributions. However, most existing models assume full access to the target languages in advance, whereas in realistic scenarios this is not often the case, as new languages can be incorporated later on. In this paper, we present the Cross-lingual Lifelong Learning (CLL) challenge, where a model is continually fine-tuned to adapt to emerging data from different languages. We provide insights into what makes multilingual sequential learning particularly challenging. To surmount such challenges, we benchmark a representative set of cross-lingual continual learning algorithms and analyze their knowledge preservation, accumulation, and generalization capabilities compared to baselines on carefully curated datastreams. The implications of this analysis include a recipe for how to measure and balance between different cross-lingual continual learning desiderata, which goes beyond conventional transfer learning.
Integrating Empirical Knowledge into Multi-View Feature Attention Network for Disease Diagnosis
Authors: Anonymous
Date: 2022-01-16 00:00:00
Proceeding: ACL ARR January 2022 1
Tasks: Diagnostic

As one of the currently significant problems in AI-enabled healthcare research, disease diagnosis based on the medical text has made substantial progress. However, the length of the diagnostic evidences is different, leading to the difficulty of capturing multi-scale features of each disease. And recent studies have discovered that structural knowledge from medical text is critical for disease diagnosis. This paper proposes integrating empirical knowledge of disease into a multi-view feature attention network to address these issues. The multi-view feature attention network employs multi encoders to capture segment information of diagnostic evidences of each illness. Meanwhile, we used an abductive causal graph constructed from medical text to extract the empirical knowledge representation of diseases by graph convolutional network. The evaluation conducted on the MIMIC-III-50 dataset and Chinese dataset demonstrates that the proposed method outperforms the structural knowledge-based state-of-the-art models.