pwc-archive/papers-with-abstracts huggingface dataset:
Modeling Hierarchical Syntax Structure with Triplet Position for Source Code Summarization
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Code Summarization, Graph Neural Network, Position, Source Code Summarization, Triplet

Automatic code summarization, which aims to describe the source code in natural language, has become an essential task in software maintenance. Our fellow researchers have attempted to achieve such a purpose through various machine learning-based approaches. One key challenge keeping these approaches from being practical lies in the lacking of retaining the semantic structure of source code, which has unfortunately been overlooked by the state-of-the-art. Existing approaches resort to representing the syntax structure of code by modeling the Abstract Syntax Trees (ASTs). However, the hierarchical structures of ASTs have not been well explored. In this paper, we propose CODESCRIBE to model the hierarchical syntax structure of code by introducing a novel triplet position for code summarization. Specifically, CODESCRIBE leverages the graph neural network and Transformer to preserve the structural and sequential information of code, respectively. In addition, we propose a pointer-generator network that pays attention to both the structure and sequential tokens of code for a better summary generation. Experiments on two real-world datasets in Java and Python demonstrate the effectiveness of our proposed approach when compared with several state-of-the-art baselines.
MINER: Improving Out-of-Vocabulary Named Entity Recognition from an Information Theoretic Perspective
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER

NER model has achieved promising performance on standard NER benchmarks. However, recent studies show that previous approaches may over-rely on entity mention information, resulting in poor performance on out-of-vocabulary(OOV) entity recognition.  In this work, we propose MINER, a novel NER learning framework, to remedy this issue from an information-theoretic perspective. The proposed approach contains two mutual information based training objectives: i) generalizing information maximization, which enhances representation via deep understanding of context and entity surface forms; ii) superfluous information minimization, which discourages representation from rotate memorizing entity names or exploiting biased cues in data. Experiments on various settings and datasets demonstrate that it achieves better performance in predicting OOV entities.
Towards Asking Clarification Questions in Task-Oriented Dialogue
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Task-Oriented Dialogue Systems

Task-oriented dialogues aim at providing users with task-specific services. To provide satisfactory services, two major challenges exist: 1) users are not able to fully describe their complex needs due to lack of task knowledge, and; 2) systems need to personalize the service to their users since different users have different profiles and preferences. In order to solve these challenges, systems need to be able to ask questions so as to clarify the user's profile and needs. However, existing task-oriented dialogue systems ignore this aspect. In this paper, we formulate the problem of asking clarification questions in task-oriented dialogue systems. To this end, we propose a dialogue-based user simulator to collect a dataset, called TaskClariQ\footnote{To foster research in this area, the dataset and code will be made public upon paper’s acceptance.}. We further propose a new System Ask paradigm and a Multi-Attention Seq2Seq Networks (MAS2S) that implements it. Experimental results on TaskClariQ show that MAS2S outperforms competitive baselines.
Hierarchical Inductive Transfer for Continual Dialogue Learning
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: General Knowledge

Pre-trained models have achieved excellent performance on the dialogue task. 
However, for the continual increase of online chit-chat scenarios, directly fine-tuning these models for each of the new tasks not only explodes the capacity of the dialogue system on the embedded devices but also causes knowledge forgetting on pre-trained models and knowledge interference between diverse dialogue tasks. 
In this work, we propose a hierarchical inductive transfer framework to learn and deploy the dialogue skills continually and efficiently. 
First, we introduce the adapter module into pre-trained models for learning new dialogue tasks. As the only trainable module, it is beneficial for the dialogue system on the embedded devices to acquire new dialogue skills with negligible additional parameters. 
Then, for alleviating knowledge interference between tasks yet benefiting the regularization between them, we further design hierarchical inductive transfer that enables new tasks to use general knowledge in the base adapter without being misled by diverse knowledge in task-specific adapters. 
Empirical evaluation and analysis indicate that our framework obtains comparable performance under deployment-friendly model capacity.
Probing and Generalization of Metaphorical Knowledge in Pre-Trained Language Models
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11

Human languages are full of metaphorical expressions. Metaphors help people understand the world by connecting new concepts and domains to more familiar ones. Large pre-trained language models (PLMs) are therefore assumed to encode metaphorical knowledge useful for NLP systems. In this paper, we investigate this hypothesis for PLMs, by probing metaphoricity information in their encodings, and by measuring the cross-lingual and cross-dataset generalization of this information. We present studies in multiple metaphor detection datasets and in four languages (i.e., English, Spanish, Russian, and Farsi). Our extensive experiments suggest that contextual representations in PLMs do encode metaphorical knowledge, and mostly in their middle layers. The knowledge is transferable between languages and datasets, especially when the annotation is consistent across training and testing sets. Our findings give helpful insights for both cognitive and NLP scientists.
Diversifying Neural Dialogue Generation via Negative Distillation
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Dialogue Generation

Generative dialogue models suffer from serious generic response problems, limiting their applications to a few toy scenarios.  Recently, an interesting approach, namely negative training, has been proposed to alleviate this problem by reminding the model not to generate high-frequency responses during training. However, its performance is hindered by two issues, ignoring low-frequency but generic responses and bringing low-frequency but meaningless responses. In this paper, we propose a novel negative training paradigm, called negative distillation, to keep the model away from the undesirable generic responses while avoiding the above problems.  First, we introduce a negative teacher model that can produce query-wise generic responses, and then the student model is required to maximize the distance with multi-level negative knowledge. Empirical results show that our method outperforms previous negative training methods significantly.
Learning to learn STEM courses
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Astronomy

We curate a new dataset from MIT EECS (Course 6), Physics (Course 8), Economics (Course 14), Mathematics (Course 18), Harvard Statistics, and Columbia Computer Science course questions, transform them into programming tasks using OpenAI Codex, and solve them by executing programs. We curate, transform, and solve ten courses: (i) MIT EECS 6.003 Signal Processing, (ii) MIT EECS 6.036 Introduction to Machine Learning, (iii) MIT EECS 6.042 Mathematics for Computer Science, (iv) MIT Physics 8.282 Introduction to Astronomy, (v) MIT Economics 14.01 Principles of Microeconomics, (vi) MIT Mathematics 18.05 Introduction to Probability and Statistics, (vii) MIT Mathematics 18.06 Linear Algebra, (viii) MIT Mathematics 18.781 Theory of Numbers, (ix) Harvard Statistics STATS110 Probability, and (x) Columbia University COMS3251 Computational Linear Algebra. Our approach works surprisingly well since question solutions and programs share an underlying tree representation. We are able to use Codex to correctly solve all questions by specifying both question and programming contexts such as which mathematical rules to use or which programming packages to load. In addition to generating code which solves problems the resulting code generates plots which are useful for understanding the solutions. We interactively transform the original course questions until they are solved correctly and measure the similarity between the original and transformed questions. Finally, we automatically generate novel questions for each course, providing a way to rapidly synthesize new course content. Our approach is the first scalable solution towards automatically learning to learn all university STEM courses by machine.
Inverse is Better! Fast and Accurate Prompt for Slot Tagging
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Few-Shot Learning, Prediction, Sentence

Prompting methods recently achieve impressive success in few-shot learning. These methods embed input samples with prompt sentence pieces and decode label-related tokens to map samples to the label. However, such a paradigm is very inefficient for the task of slot tagging. Because the slot tagging samples are multiple consecutive words in a sentence, the prompting methods have to enumerate all n-grams token span to find all the possible slots, which greatly slows down the prediction. To tackle this, we introduce an inverse paradigm for prompting. Different from the classic prompts map tokens to labels, we reversely predict slot values given slot types. Such inverse prompting only requires a one-turn prediction for each slot type and greatly speeds up the prediction. Besides, we propose a novel Iterative Prediction Strategy, from which the model learns to refine predictions by considering the relations between different slot types. We find, somewhat surprisingly, the proposed method not only predicts faster, but also significantly improves the effect (improve over 6.1 F1-scores on 10-shot setting) and achieves new state-of-the-art performance.
Fair comparison of knowledge graphs for question answering
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Knowledge Graphs, Question Answering

Knowledge graphs are commonly used as sources of information in question answering. Models often combine pre-trained text encoders with a graph encoder to use this information to increase accuracy. However, the way that these two types of model interact is not clear. Here we show that, when provided with graph information for a random question, two recent models exhibit no significant change in performance. These models cannot therefore be used to obtain graph-structured explanations, or to compare the relevance of a particular knowledge graph to a dataset. We perform two model ablations and show that the resulting model is more responsive to variation in graph input, and so can be used for gathering explanations and measuring KG-dataset fit. We also show that uncontrollable nondeterminism can cause significant changes in results, and highlight the importance of statistical testing of these models.
Towards Building Automatic Medical Consultation System: Framework, Task and Dataset
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Dialogue Act Classification, Dialogue Understanding, Medical Named Entity Recognition, Medical Report Generation, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER)

In this paper, we propose two frameworks to support automatic medical consultation, namely doctor-patient dialogue understanding and diagnosis-oriented interaction. A new medical dialogue dataset with multi-level fine-grained annotations is introduced and five evaluation tasks are established, including medical named entity recognition, dialogue act classification, symptom recognition, medical report generation and diagnosis-oriented dialogue system. We report a set of benchmark results for each track, which shows the usability of the dataset and sets a baseline for future studies.
A Probabilistic Framework for Analyzing Moral Perspectives in the COVID-19 Vaccine Debate
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Sentiment Analysis

The Covid-19 pandemic has led to infodemic of low quality information leading to poor health decisions. Combating the outcomes of this infodemic is not only a question of identifying false claims, it requires understanding the reasoning behind the decisions individuals make.
In this work we propose a holistic analysis framework connecting stance and reason analysis and fine-grained entity level moral sentiment analysis. We study how to model the dependencies between the different level of analysis and incorporate human insights into the learning process. Our experiments show that our framework can provide reliable predictions even in the low-supervision settings.
KART: Parameterization of Privacy Leakage Scenarios from Pre-trained Language Models
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11

For the safe sharing pre-trained language models, no guidelines exist at present owing to the difficulty in estimating the upper bound of the risk of privacy leakage. One problem is that previous studies have assessed the risk for different real-world privacy leakage scenarios and attack methods, which reduces the portability of the findings.
To tackle this problem, we represent complex real-world privacy leakage scenarios under a universal parameterization, \textit{Knowledge, Anonymization, Resource, and Target} (KART). KART parameterization has two merits: (i) it clarifies the definition of privacy leakage in each experiment and (ii) it improves the comparability of the findings of risk assessments. We show that previous studies can be simply reviewed by parameterizing the scenarios with KART. We also demonstrate privacy risk assessments in different scenarios under the same attack method, which suggests that KART helps approximate the upper bound of risk under a specific attack or scenario. We believe that KART helps integrate past and future findings on privacy risk and will contribute to a standard for sharing language models.
Incorporating Dual-Aware with Hierarchical Interactive Memory Networks for Task-Oriented Dialogue
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Task-Oriented Dialogue Systems

Recent years, end-to-end task-oriented dialogue systems have made a remarkable breakthrough. However, existing dialogue models tend to equally summarize all the history as the context representation and apply memory networks to incorporate external knowledge. They neglect to highlight the latest request of users, which will cause the unexpected responses to be generated. In addition, it is insufficient for the original memory networks to interact between memories only at hop-level and difficult to extract more useful knowledge information. To address this issue, we propose a novel neural model which incorporates Dual-Aware with Hierarchical Interactive Memory Networks (DA-HIMN). The dual-aware constituting static  request-aware and dynamic KB-aware is responsible for capturing the latest request of users and collecting related knowledge information. Furthermore, we design a hierarchical interaction mechanism to augment the memory networks at layer-level to more adequately learn the knowledge representation. Our experimental results demonstrate that our model outperforms the baseline model on two task-oriented dialogue datasets in several evaluation metrics.
When Chosen Wisely, More Data Is What You Need: A Universal Sample-Efficient Strategy For Data Augmentation
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Data Augmentation, HellaSwag, Knowledge Distillation

Data Augmentation (DA) is vital in deep learning to improve the generalizability of neural networks. Most of existing DA techniques in NLP naively add a certain number of augmented samples without paying attention to the quality and added computational cost of these samples. Furthermore, state-of-the-art DA techniques in the literature usually learn to generate or re-weight augmented samples more specific to the main task; however, these learning-based DA techniques are not sample-efficient and they are computationally  expensive.  In this work, we propose a universal DA technique, called Glitter, for NLP which aims at efficiency and performance at the same time. In other words, Glitter can be applied to any existing DA technique to improve its training efficiency and sample efficiency and maintain its competitive performance. We evaluate Glitter on several downstream tasks such as the GLUE benchmark, SQuAD, and HellaSwag in a variety of scenarios including general single network, consistency training, self-distillation and knowledge distillation (KD) setups.
Distantly Supervised Named Entity Recognition with Category-Oriented Confidence Calibration
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER

In this work, we study the noisy-labeled named entity recognition under distant supervision setting. Considering that most NER systems based on confidence estimation deal with noisy labels ignoring the fact that model has different levels of confidence towards different categories, we propose a category-oriented confidence calibration(Coca) strategy with an automatically confidence threshold calculation module. We integrate our method into a teacher-student framework to improve the model performance. Our proposed approach achieves promising performance among advanced baseline models, setting new state-of-the-art performance on three existing distantly supervised NER benchmarks.
Word2Box: Capturing Set-Theoretic Semantics of Words using BoxEmbeddings
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Word Similarity

Learning representations of words in a continuous space is perhaps the most fundamental task in NLP, a prerequisite for nearly all modern machine-learning techniques. Often the objective is to capture distributional similarity via vector dot product, however this is just one relation between word meanings we may wish to capture.  It is natural to consider words as (soft) equivalence classes based on similarity, it is natural to expect the ability to perform set-theoretic operations (intersection, union, difference) on these representations. This is particularly relevant for words which are homographs- for example, “tongue”∩“body” should be similar to “mouth”, while “tongue”∩“language” should  be  similar  to  “dialect”. Box embeddings are a novel region-based representation which provide the capability to perform these set-theoretic operations. In this work, we provide a fuzzy-set interpretation of box embeddings, and train box embeddings with a CBOW objective where contexts are represented using intersection. We demonstrate improved performance on various word similarity tasks, particularly on less common words, and perform a quantitative and qualitative analysis exploring the additional unique expressivity provided by Word2Box.
Robust and Effective Grammatical Error Correction with Simple Cycle Self-Augmenting
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Adversarial Attack, Grammatical Error Correction

Recent studies have revealed that grammatical error correction methods in the sequence-to-sequence paradigm are vulnerable to adversarial attack, and simply utilizing adversarial examples in the pre-training or post-training process can significantly enhance the robustness of GEC models to certain types of attack without suffering too much performance loss on clean data. In this paper, we further conduct a thorough robustness evaluation of cutting-edge GEC methods to four different types of adversarial attacks and propose a simple yet very effective Cycle Self-Augmenting (CSA) method accordingly. By leveraging the augmenting data from the GEC models themselves in the post-training process and introducing regularization data for cycle training, our proposed method can effectively improve model robustness of well-trained GEC models with only a few more training epochs as the extra cost. Experiments on four benchmark datasets and seven strong models indicate that our proposed training method can significantly enhance the robustness to four types of attacks without using purposely built adversarial examples in training. Evaluation results on clean data further confirm that our proposed CSA method significantly improves the performance of four baselines and yields nearly comparable results with other state-of-the-art models. Our code is available in the supplementary .zip file, which will be released after the anonymous period.
DEMix Layers: Disentangling Domains for Modular Language Modeling
Authors: Anonymous
Date: 2021-10-16 00:00:00
Proceeding: ACL ARR October 2021 10
Tasks: Language Modeling, Language Modelling

We introduce a new domain expert mixture (DEMix) layer that enables conditioning a language model (LM) on the domain of the input text.  A DEMix layer is a collection of expert feedforward networks, each specialized to a domain, that makes the LM modular: experts can be mixed, added, or removed after initial training. Extensive experiments with autoregressive transformer LMs (up to 1.3B parameters) show that DEMix layers reduce perplexity, increase training efficiency, and enable rapid adaptation. Mixing experts during inference, using a parameter-free weighted ensemble, enables better generalization to heterogeneous or unseen domains. Adding experts incorporates new domains without forgetting older ones, and removing experts restricts access to unwanted domains without additional training. Overall, these results demonstrate benefits of explicitly conditioning on textual domains during language modeling.
DoTAT: A Domain-oriented Text Annotation Tool
Authors: Anonymous
Date: 2021-10-16 00:00:00
Proceeding: ACL ARR October 2021 10
Tasks: text annotation

We propose DoTAT, a domain-oriented text annotation tool. The tool designs and implements functions heavily in need in domain-oriented information extraction. Firstly, the tool supports a multi-person collaborative process with automatically merging and review, which can greatly improve the annotation accuracy. Secondly, the tool provides annotation of event, nested event, and nested entity, which are frequently required in domain-related text structuring tasks. Finally, DoTAT provides visualized annotation specification definition, automatic batch annotation, and iterative annotation to improve annotation efficiency. Experiments on the ACE2005 dataset show that DoTAT can reduce the event annotation time by 19.7% compared with existing annotation tools. The accuracy without review is 84.09%, 1.35% higher than Brat, and 2.59% higher than Webanno. The accuracy of DoTAT even reaches 93.76% with the review. The demonstration video can be accessed from https://ecust-nlp-docker.oss-cn-shanghai.aliyuncs.com/dotat_demo.mp4. A live demo website is available at https://github.com/FXLP/MarkTool.