pwc-archive/papers-with-abstracts huggingface dataset:
Code Synonyms Do Matter: Multiple Synonyms Matching Network for Automatic ICD Coding
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Code Classification, Representation Learning

Automatic ICD coding is defined as assigning disease codes to electronic medical records (EMRs).
Existing methods apply label attention with code representations to match related text snippets for coding.
Unlike these works that model the label with the code hierarchy or description, we argue that the code synonyms can provide more comprehensive knowledge based on the observation that the code expressions in EMRs vary from their descriptions in ICD. 
By aligning codes to concepts in UMLS, we collect synonyms of every code in ICD. Then, we propose a multiple synonyms matching network to leverage synonyms for better code representation learning, and finally help the code classification. 
Experiments on two settings of the MIMIC-III dataset show that our proposed method outperforms previous state-of-the-art methods.
Semi-supervised New Event Type Induction and Description via Contrastive Loss-Enforced Batch Attention
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Event Extraction

Most event extraction methods have traditionally relied on an annotated set of event types. However, creating event ontologies and annotating supervised training data are expensive and time-consuming. Previous work has proposed semi-supervised approaches which leverage seen (annotated) types to learn how to automatically discover new event types. State-of-the-art methods, both semi-supervised or fully unsupervised, use a form of reconstruction loss on specific tokens in a context. In contrast, we present a novel approach to semi-supervised new event type induction using a masked contrastive loss which learns similarities between event mentions by enforcing an attention mechanism over the data minibatch. We further disentangle the discovered clusters by approximating the underlying manifolds in the data, which allows us to increase normalized mutual information and Fowlkes-Mallows scores by over 20% absolute. Building on these clustering results, we extend our approach to two new tasks: predicting the type name of the discovered clusters and linking them to FrameNet frames.
IDPG: An Instance-Dependent Prompt Generation Method
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Language Modeling, Language Modelling, Natural Language Understanding, Sentence, Transfer Learning

Prompt tuning is a new, efficient NLP transfer learning paradigm that adds a task-specific prompt in each input instance during the model training stage. It freezes the pre-trained language model and only optimizes a few task-specific prompts. In this paper, we propose a conditional prompt generation method to generate prompts for each input instance, referred to as the Instance-Dependent Prompt Generation (IDPG). Unlike traditional prompt tuning methods that use a fixed prompt, IDPG introduces a lightweight and trainable component to generate prompts based on each input sentence. Empirical experiments on ten natural language understanding (NLU) tasks show that our proposed method consistently outperforms various prompt tuning methods and other efficient transfer learning methods such as Compacter while tuning far fewer model parameters.
Situated Dialogue Learning through Procedural Environment Generation
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11

We teach goal-driven agents to interactively act and speak in situated environments by training on generated curriculums. Our agents operate in LIGHT (Urbanek et al. 2019)---a large-scale crowd-sourced fantasy text adventure game wherein an agent perceives and interacts with the world through textual natural language. Goals in this environment take the form of character-based quests, consisting of personas and motivations. We augment LIGHT by learning to procedurally generate additional novel textual worlds and quests to create a curriculum of steadily increasing difficulty for training agents to achieve such goals. In particular, we measure curriculum difficulty in terms of the rarity of the quest in the original training distribution---an easier environment is one that is more likely to have been found in the unaugmented dataset. An ablation study shows that this method of learning from the tail of a distribution results in significantly higher generalization abilities as measured by zero-shot performance on never-before-seen quests.
Legal Fairness Analysis via Treatment Effect Estimation
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Fairness

Legal fairness is one of the most important principles pursued by modern legal systems. Unfortunately, unfairness may be inevitably introduced in real-world cases due to both objective and subjective uncertainty, such as ambiguity in the law or practical bias in judgments. Existing works for fairness analysis mainly rely on labor-intensive element annotation for cases, which suffer from limited generalization ability. To address this issue, we propose to utilize large-scale textual data to perform quantitative legal fairness analysis via our Causal-based Legal Fairness Measuring Framework (CaLF). To verify its effectiveness, we construct a legal-fairness dataset, and experimental results show that CaLF can accurately characterize the unfairness. Further, we adopt CaLF on a large-scale real-world dataset and come to several interesting experimental observations from the perspective of gender, age, and region.
Towards Few-shot Entity Recognition in Document Images: A Label-aware Sequence-to-Sequence Framework
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11

Entity recognition is a fundamental task in understanding document images. Traditional sequence labeling framework requires extensive datasets and high-quality annotations, which are typically expensive in practice. In this paper, we aim to build an entity recognition model based on only a few shots of annotated document images. To overcome the data limitation, we propose to leverage the label surface names to better inform the model of the target entity semantics. Specifically, we go beyond sequence labeling and develop a novel label-aware seq2seq framework, LASER. We design a new labeling scheme that generates the label surface names word-by-word explicitly after generating the entities. Moreover, we design special layout identifiers to capture the spatial correspondence between regions and labels. During training, LASER refines the label semantics by updating the label surface name representations and also strengthens the label-region correlation. In this way, LASER recognizes the entities from document images through both semantic and layout correspondence. Extensive experiments on two benchmark datasets demonstrate the superiority of LASER under the few-shot setting.
Spot the Difference: A Cooperative Object-Referring Game in Non-Perfectly Co-Observable Scene
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11

Visual dialogue has witnessed great progress after introducing various vision-oriented goals into the conversation, especially such as GuessWhich and GuessWhat, where the only image is visible by either and both of the questioner and the answerer, respectively. Researchers explore more on visual dialogue tasks in such kind of single- or perfectly co-observable visual scene, while somewhat neglect the exploration on tasks of non-perfectly co-observable visual scene, where the images accessed by two agents may not be exactly the same, often occurred in practice. Although building common ground in non-perfectly co-observable visual scene through conversation is significant for advanced dialogue agents, the lack of such dialogue task and corresponding large-scale dataset makes it impossible to carry out in-depth research. To break this limitation, we propose an object-referring game in non-perfectly co-observable visual scene, where the goal is to spot the difference between the similar visual scenes through conversing in natural language. The task addresses challenges of the dialogue strategy in non-perfectly co-observable visual scene and the ability of categorizing objects. Correspondingly, we construct a large-scale multi-modal dataset, named \textit{SpotDiff}, which contains 49k Virtual Reality images and 97k dialogues generated by self-play. Finally, we give benchmark models for this task, and conduct extensive experiments to evaluate its performance as well as analyze its main challenges.
DGMED: A Novel Document-Level Graph Convolution Network for Multi-Event Detection
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Event Detection, Event Extraction, Sentence

Online news documents can contain thousands of characters and tens of events. To detect events in these documents, it is important to construct long-range context information. Such information, however, is not effectively created in existing event detection methods including DMBERT, MOGANED. As a result, these methods show poor event detection accuracy in production where long documents are common. To address this, this paper proposes a Document-level Graph convolution network for Multi-event Detection (DGMED). DGMED represents each sentence in a long document as a graph, and it interconnects these graphs using novel cross-sentence global neural network nodes. These nodes allow DGMED further construct accurate document-level contextual information, thus accurately extracting multiple events as required. We evaluate DGMED using a public event extraction dataset (i.e., Maven) and a production large-scale dataset (named AML). Evaluation results show that DGMED can out-perform state-of-the-art methods BERT+CRF and BiLSTM+CRF up to 0.7% in Maven and 5.7% in AML.
Personality Prediction of Narrative Characters from Movie Scripts
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Prediction, text-classification, Text Classification

An NLP model that understands stories should be able to understand the characters in them. To support the development of neural models for this purpose, we construct a benchmark, Story2Personality. The task is to predict a movie character’s personality based on the narratives of the character in the movie script. Experiments show that our task is challenging for the existing text classification models, as none is able to largely outperform random guesses. We further proposed a multi-view model to use both verbal and non-verbal descriptions for personality prediction, which gives improvement compared to using only verbal descriptions. The uniqueness and challenges in our dataset call for the development of narrative comprehension techniques from the perspective of understanding characters.
Divide-and-Conquer Text Simplification by Scalable Data Enhancement
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Text Simplification

Text simplification, whose aim is to reduce reading difficulty, can be decomposed into four discrete rewriting operations: substitution, deletion, reordering, and splitting. However, due to a large distribution discrepancy between existing training data and human-annotated data, models may learn improper operations, thus lead to poor generalization capabilities. In order to bridge this gap, we propose a novel data enhancement method, Simsim, that generates training pairs by simulating specific simplification operations. Experiments show that the models trained with Simsim outperform multiple strong baselines and achieve the better SARI on the Turk and Asset datasets. The newly constructed dataset Simsim is available at *.
Detection, Disambiguation, Re-ranking: Autoregressive Entity Linking as a Multi-Task Problem
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Entity Linking, Re-Ranking

We propose an autoregressive entity linking model, that is trained with two auxiliary tasks, and learns to re-rank generated samples at inference time. Our proposed novelties address two weaknesses in the literature. First, as recent improvements in entity linking suggest learning mention detection explicitly could increase performance, we train mention detection as an auxiliary task. Second, previous work suggests that re-ranking could help correct prediction errors. We add a new, auxiliary task, match prediction, to learn re-ranking. Without the use of a knowledge base or candidate sets, our model sets a new state of the art in two benchmark datasets of entity linking: COMETA in the biomedical domain, and AIDA-CoNLL in the news domain. We show through ablation studies that each of the two auxiliary tasks increases performance, and that re-ranking is an important factor to the increase. Finally, our low-resource experimental results suggest that performance on the main task benefits from the knowledge learned by the auxiliary tasks, and not just from the additional training data.
DAML: Chinese Named Entity Recognition with a fusion method of data-augmentation and meta-learning
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Chinese Named Entity Recognition, Data Augmentation, Meta-Learning, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Transfer Learning

Overfitting is still a common problem in NER with insufficient data. Latest methods such as Transfer Learning, which focuses on storing knowledge gained while solving one task and applying it to a different but related task, or Model-Agnostic Meta-Learning (MAML), which learns a model parameter initialization that generalizes better to similar tasks. However, these methods still need rich resources to pre-train. In this work, we present new perspectives on how to make the most of in-domain and out-domain information. By introducing a fusion method of data augmentation and MAML, we first use data augmentation to mine more information. With the augmented resources, we directly utilize out-domain and in-domain data with MAML, while avoiding performance degradation after domain transfer. To further improve the model’s generalization ability, we proposed a new data augmentation method based on a generative approach. We conduct experiments on six open Chinese NER datasets (MSRANER, PeopleDairyNER, CLUENER, WeiboNER, Resume NER, and BOSONNER). The results show that our method significantly reduces the impact of insufficient data and outperforms the state-of-the-art.
Interpretable embeddings to understand computing careers
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Metric Learning, Word Embeddings

We propose an approach for analyzing and comparing curricula of study programs in higher education. Pre-trained word embeddings are fine-tuned in a study program classification task, where each curriculum is represented by the names and content of its courses. By combining metric learning with a novel course-guided attention mechanism, our method obtains more accurate curriculum representations than strong baselines. Experiments on a new dataset containing curricula of computing programs demonstrate the interpretability power of our approach via attention weights, topic modeling, and embeddings visualizations. We also present a use case that compares computing study programs in the US and Latin America and showcase the capabilities of our method for identifying similarities and differences in topics of study in curricula from different countries.
S$^4$-Tuning: A Simple Cross-lingual Sub-network Tuning Method
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Transfer Learning

The emergence of multilingual pre-trained language models makes it possible to adapt to target languages with only few labeled examples.
However, vanilla fine-tuning tends to achieve degenerated and unstable results, owing to the  Language Interference among different languages, and Parameter Overload under the few-sample transfer learning scenarios.
To address two problems elegantly, we propose S$^4$-Tuning, a Simple Cross-lingual Sub-network Tuning method. 
S$^4$-Tuning first detects the most essential sub-network for each target language, and only updates it during fine-tuning.
In this way, the language sub-networks lower the scale of trainable parameters, and hence better suit the low-resource scenarios.
Meanwhile, the commonality and characteristics across languages are modeled by the overlapping and non-overlapping parts to ease the interference among languages.
Simple but effective, S$^4$-Tuning gains consistent improvements over vanilla fine-tuning on three multi-lingual tasks involving 37 different languages in total (XNLI, PAWS-X, and Tatoeba).
Open Domain Question Answering over Virtual Documents: A Unified Approach for Data and Text
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Data-to-Text Generation, Natural Questions, Open-Domain Question Answering, Question Answering, Text Generation

Due to its potential for a universal interface over both data and text, data-to-text generation is becoming increasingly popular.
However, few prior work has focused on its application to downstream tasks, e.g. using the converted data for grounding or reasoning. 
In this work, we bridge this gap and use the data-to-text method as a means for encoding structured knowledge for knowledge-intensive applications, i.e. open-domain question answering (ODQA).
Specifically, we propose a verbalizer-retriever-reader framework for ODQA over data and text where verbalized tables from Wikipedia and graphs from Wikidata are used as augmented knowledge sources.
We show that our Unified Data and Text QA, UDT-QA.
can effectively benefit from the expanded knowledge index, leading to large gains over text-only baselines.
Notably, our approach sets the single-model state-of-the-art on Natural Questions.
Furthermore, our analyses indicate that verbalized knowledge is preferred for answer reasoning for both adapted and hot-swap settings.
Augmenting Document Representations for Dense Retrieval with Interpolation and Perturbation
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Retrieval

Dense retrieval models, which aim at retrieving the most relevant document for an input query on a dense representation space, have gained considerable attention for their remarkable success. Yet, dense models require a vast amount of labeled training data for notable performance, whereas it is often challenging to acquire query-document pairs annotated by humans. To tackle this problem, we propose a simple but effective Document Augmentation for dense Retrieval (DAR) framework, which augments the representations of documents with their interpolation and perturbation. We validate the performance of DAR on retrieval tasks with two benchmark datasets, showing that the proposed DAR significantly outperforms relevant baselines on the dense retrieval of both the labeled and unlabeled documents.
Compressing Sentence Representation via Homomorphic Projective Distillation
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Language Modeling, Language Modelling, Retrieval, Semantic Retrieval, Semantic Textual Similarity, Sentence, Sentence Embeddings, STS

How to learn highly compact yet effective sentence representation? Pre-trained language models have been effective in many NLP tasks. However, these models are often huge and produce large sentence embeddings. Moreover, there is a big performance gap between large and small models. In this paper, we propose Homomorphic Projective Distillation (HPD) to learn compressed sentence embeddings. Our method augments a small Transformer encoder model with learnable projection layers to produce compact representations while mimicking a large pre-trained language model to retain the sentence representation quality. We evaluate our method with different model sizes on both semantic textual similarity (STS) and semantic retrieval (SR) tasks. Experiments show that our method achieves 2.7-4.5 points performance gain on STS tasks compared with previous best representations of the same size. In SR tasks, our method improves retrieval speed (8.2×) and memory usage (8.0×) compared with state-of-the-art large models.
Towards Better Citation Intent Classification
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Articles, Citation Intent Classification, Classification, Data Augmentation, intent-classification, Intent Classification, Model Selection

Accurate classification of citation intents in a scientific article provides deeper contextual understanding of and better quantifies the contributions of cited articles. This improves scientific literature platform capabilities such as search relevance, ranking and more. To our knowledge, we present the most comprehensive survey of Transformer-based language models performance on the citation intent classification task using SciCite dataset. Here, we make three recommendations. Firstly, we propose to report model performance as a distribution in contrast to a single averaged performance value. This arises from our observation that model performance is sensitive to the random seed choice resulting in wide performance variations from multiple finetuning runs. Secondly, this provides practical insights for model selection, showing the model's best possible performance. Thus, we propose that practitioners perform multiple finetuning runs before selecting the best performing model. Thirdly, we propose a simple data augmentation to improve the distribution of model performance overall. Moving forward, we suggest exploring improvements to the finetuning and model selection process as promising future directions.
A Graph-to-Sequence Model for Joint Intent Detection and Slot Filling in Task-Oriented Dialogue Systems
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Decoder, global-optimization, Graph-to-Sequence, Intent Detection, slot-filling, Slot Filling, Spoken Language Understanding, Task-Oriented Dialogue Systems

Effectively decoding semantic frames in task-oriented dialogue systems remains a challenge, which typically includes intent detection and slot filling. Although RNN-based neural models show promising results by jointly learning of these two tasks, dominant RNNs are primarily focusing on modeling sequential dependencies. Rich graph structure information hidden in the dialogue context is seldomly explored. In this paper, we propose a novel Graph-to-Sequence model to tackle the spoken language understanding problem by modeling both temporal dependencies and structural information in a conversation. We introduce a new Graph Convolutional LSTM (GC-LSTM) encoder to learn the semantics contained in the dialogue dependency graph by incorporating a powerful graph convolutional operator. Our proposed GC-LSTM can not only capture the spatio-temporal semantic features in a dialogue, but also learn the co-occurrence relationship between intent detection and slot filling. Furthermore, a LSTM decoder is utilized to perform final decoding of both slot filling and intent detection, which mutually improves both tasks through global optimization. Experiments on benchmark ATIS and Snips datasets show that our model achieves state-of-the-art performance and outperforms existing models.