pwc-archive/papers-with-abstracts huggingface dataset:
End-to-end Task-oriented Dialog Policy Learning based on Pre-trained Language Model
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: dialog state tracking, Domain Adaptation, Language Modeling, Language Modelling

This paper presents our approach to dialog policy learning (DPL), which aims to determine the next system’s action based on the current dialog state maintained by a dialog state tracking module. Different from previous stage-wise DPL, we propose an end-to-end DPL system to avoid error accumulation between the dialogue turns. The DPL system is deployed from two perspectives. Firstly, we consider turn-level DPL that selects the best dialog action from a predefined action set. Specifically, we proposed a dialog action-oriented BERT (DA-BERT), which integrates a new pre-training procedure named masked last action task (MLA) that encourages BERT to be dialog-aware and distill action-specific features. Secondly, we propose a word-level DPL that directly generates the dialog action. We creatively model DPL as a sequence generation model conditioned on the dialog action structure. Then GPT-2 equipped with an action structure parser module (termed as DA-GPT-2) is applied to learn the word level DPL. The effectiveness and different characteristics of the proposed models are demonstrated with the in-domain tasks and domain adaptation tasks on MultiWOZ with both simulator evaluation and human evaluation.
Redistributing Low-Frequency Words: Making the Most of Monolingual Data in Non-Autoregressive Translation
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Knowledge Distillation, Translation, Word Translation

Knowledge distillation (KD) is the preliminary step for training non-autoregressive translation (NAT) models, which eases the training of NAT models at the cost of losing important information for translating low-frequency words. In this work, we provide an appealing alternative for NAT -- monolingual KD, which trains NAT student on external monolingual data with AT teacher trained on the original bilingual data. Monolingual KD is able to transfer both the knowledge of the original bilingual data (implicitly encoded in the trained AT teacher model) and that of the new monolingual data to the NAT student model. Extensive experiments on eight WMT benchmarks over two advanced NAT models show that monolingual KD consistently outperforms the standard KD by improving low-frequency word translation, without introducing any computational cost. Monolingual KD enjoys desirable expandability, which can be further enhanced (when given more computational budget) by combining with the standard KD, a reverse monolingual KD, or enlarging the scale of monolingual data. Extensive analyses demonstrate that these techniques can be used together profitably to further recall the useful information lost in the standard KD. Encouragingly, combining with standard KD, our approach achieves 30.4 and 34.1 BLEU points on the WMT14 English-German and German-English datasets, respectively. Code, data, and models will be released.
Identifying Corporate Credit Risk Sentiments from Financial News
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Management, text-classification, Text Classification

Credit risk management is one major practice for financial institutions, that helps them measure and understand the inherent risk within their portfolios. Historically, they relied on the assessment of default probabilities (via structural or default intensity models) and used the press as one tool to gather insights on the latest credit event developments of an entity. However, because the current news volume and coverage for companies is generally heavy, analyzing news manually by financial experts is considered a highly laborious task. To this end, we propose a novel deep learning-powered approach to automate news analysis and credit adverse events detection, with the aim of scoring the credit sentiment associated with a company in order to assist credit risk management efficiently. The result is a complete system leveraging news extraction and data enrichment (with targeted sentiment entity recognition to detect companies and text classification to identify credit events), as well as a custom scoring mechanism designed to provide the company's credit sentiment, called Credit Sentiment Score™ (CSS). Additionally, studies are shown to illustrate how CSS helps to gain knowledge about the company's credit profile but also discriminates between defaulters and non-defaulters.
Distill and Calibrate: Denoising Inconsistent Labeling Instances for Chinese Named Entity Recognition
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Chinese Named Entity Recognition, Denoising, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER

Data-driving supervised models for named entity recognition (NER) have made significant improvements on standard benchmarks. However, such models often have severe performance degradation on large-scale noisy data. Thus, a practical and challenging question arises: Can we leverage only a small amount of relatively clean data to guide the NER model learning from large-scale noisy data? To answer this question, we focus on the inconsistent labeling instances problem. We observe that inconsistent labeling instances can be classified into five types of noise, each of which will largely hinder the model performance in our experiments. Based on the above observation, we propose a simple yet effective denoising framework named Distillation and Calibration for Chinese NER (DCNER). DCNER consists: (1) a Dual-stream Label Distillation mechanism for distilling five types of inconsistent labeling instances from the noisy data; and (2) a Consistency-aware Label Calibration network for calibrating inconsistent labeling instances based on relatively clean data.  Additionally, we propose the first benchmark towards validating the ability of Chinese NER to resist inconsistent labeling instances. Finally, detailed experiments show that our method consistently and significantly outperforms previous methods on the proposed benchmark.
Improving Document-level Relation Extraction via Context Guided Mention Integration and Inter-pair Reasoning
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Document-level Relation Extraction, Relation, Relation Extraction, Sentence

Document-level Relation Extraction (DRE) aims to recognize the relations between two entities.
The entity may correspond to multiple mentions that span beyond sentence boundary.
Few previous studies have investigated the mention integration, which may be problematic because coreferential mentions do not equally contribute to a specific relation.
Moreover, prior efforts mainly focus on reasoning at entity-level rather than capturing the global interactions between entity pairs.
In this paper, we propose two novel techniques, Context Guided Mention Integration and Inter-pair Reasoning (CGM2IR), to improve the DRE.
Instead of simply applying average pooling, the contexts are utilized to guide the integration of coreferential mentions in a weighted sum manner.
Additionally, inter-pair reasoning executes an iterative algorithm on the entity pair graph, so as to model the interdependency of relations.
We evaluate our CGM2IR model on three widely used benchmark datasets, namely DocRED, CDR, and GDA. 
Experimental results show that our model outperforms previous state-of-the-art models.
Post-processing Networks: A Method for Optimizing Pipeline Task-oriented Dialogue Systems using Reinforcement Learning
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Task-Oriented Dialogue Systems

Many studies have proposed methods for optimizing the dialogue performance of an entire pipeline system by jointly training modules in the system using reinforcement learning. However, these methods are limited in that they can only be applied to modules implemented using trainable neural-based methods. To address this problem, we propose a method for optimizing a pipeline system composed of modules implemented with arbitrary methods for dialogue performance. In our method, neural-based components called post-processing networks (PPNs) are installed inside the system to post-process the output of each module. All PPNs are updated to improve the overall dialogue performance of the system by using reinforcement learning, not necessitating each module to be updated. Through dialogue simulation experiments on the MultiWOZ dataset, we show that PPNs can improve the dialogue performance of pipeline systems consisting of various modules.
Graph Pre-training for AMR Parsing and Generation
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Abstract Meaning Representation, AMR Parsing, AMR-to-Text Generation, Text Generation

Abstract meaning representation (AMR) highlights the core semantic information of text in a graph structure.
Recently, pre-trained language models (PLMs) have advanced tasks of AMR parsing and AMR-to-text generation.
However, PLMs are typically pre-trained on textual data, thus are sub-optimal for modeling structural knowledge.
To this end, we investigate graph self-supervised training to improve the structure awareness of PLMs over AMR graphs.
In particular, we introduce two graph auto-encoding strategies for graph-to-graph pre-training and four tasks to integrate text and graph information during pre-training.
We further design a unified framework to bridge the gap between pre-training and fine-tuning tasks.
Experimental results on both AMR parsing and AMR-to-text generation tasks show the superiority of our model.
To our knowledge, we are the first to consider pre-training on AMR graphs.
SalesBot: Transitioning from Chit-Chat to Task-Oriented Dialogues
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Dialogue Generation

Dialogue systems are usually categorized into two types, open-domain and task-oriented. The first one focuses on chatting with users and making them engage in the conversations, where selecting a proper topic to fit the dialogue context is essential for a successful dialogue.  The other one focuses on a specific task instead of casual talks, e.g., finding a movie on Friday night, playing a song. These two directions have been studied separately due to their different purposes. However, how to smoothly transition from social chatting to task-oriented dialogues is important for triggering the business opportunities, and there is no any public data focusing on such scenarios. Hence, this paper focuses on investigating the conversations starting from open-domain social chatting and then gradually transitioning to task-oriented purposes, and releases a large-scale dataset with detailed annotations for encouraging this research direction. To achieve this goal, this paper proposes a framework to automatically generate many dialogues without human involvement, in which any powerful open-domain dialogue generation model can be easily leveraged. The human evaluation shows that our generated dialogue data has a natural flow at a reasonable quality, showing that our released data has a great potential of guiding future research directions and commercial activities. Furthermore, the released models allow researchers to automatically generate unlimited dialogues in the target scenarios, which can greatly benefit semi-supervised and unsupervised approaches.
AutoMin: A Novel Dataset for Automatic Minuting from Multi-Party Meetings in English and Czech
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11

Taking minutes is an essential component of every meeting, although the goals, style, and procedure  of this activity (``minuting'' for short) can vary. Minuting is a rather unstructured writing activity and is affected by who is taking the minutes and for whom the intended minutes are. With the rise of online meetings, automatic minuting would be an important benefit for the meeting participants as well as for those who might have missed the meeting. However, automatically generating meeting minutes is a challenging problem due to a variety of factors including the quality of automatic speech recorders (ASRs), availability of public meeting data, subjective knowledge of the minuter, etc. In this work, we present the first of its kind dataset on Automatic Minuting. We develop a dataset of English and Czech technical project meetings which consists of transcripts generated from ASRs, manually corrected, and minuted by several annotators. Our dataset, AutoMin, consists of 113 (English) and 53 (Czech) meetings, covering more than 160 hours of meeting content. The meeting sessions are recorded, automatically transcribed, corrected, equipped with human-generated minutes, and finally de-identified. We would publicly release (aaa.bbb.ccc) the dataset as a set of meeting transcripts and minutes, excluding the recordings for privacy reasons. 
A unique feature of our dataset is that most meetings are equipped with more than one minute, each created independently. Our corpus thus allows studying differences in what people find important while taking the minutes. We also provide baseline experiments for the community to explore this novel problem further. To the best of our knowledge AutoMin is probably the first resource on minuting in English and also in a language other than English (Czech).
Automatic Generation of Electromyogram Diagnosis Report: Task and Dataset
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Diagnostic

Report-writing of electromyogram can be problematic for under-experienced physicians and time-consuming for experienced physicians. In this paper, we explore to generate textual report from tabular diagnostic records of electromyogram. We construct the first dataset for this task and demonstrate results of some baseline approaches.
DAML-ST5: Low Resource Style Transfer via Domain Adaptive Meta Learning
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: General Knowledge, Language Modeling, Language Modelling, Meta-Learning, Style Transfer, Text Style Transfer, Unsupervised Text Style Transfer

Text style transfer (TST) without parallel data has achieved some practical success. However, most of the existing unsupervised text style transfer methods suffer from (i) requiring massive amounts of nonparallel data to guide the transferring of different text styles. (ii) huge performance degradation when fine-tuning the model in new domains. In this work, we propose DAML-ST5, which consists of two parts, DAML and ST5. DAML is a domain adaptive meta-learning approach to refine  general knowledge in multi-heterogeneous source domains, which is capable of adapting to new unseen domains with a small amount data. Moreover, we propose a new  unsupervised TST model Style-T5 (ST5), which is composed of a sequence-to-sequence pre-trained language model T5 and uses style adversarial training for better content preservation and style transfer. Results on multi-domain datasets demonstrate that  our  approach  generalize well on unseen low-resource domains, achieving state of the art results against ten strong baselines.
Effective Token Graph Modeling using a Novel Labeling Strategy for Structured Sentiment Analysis
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Dependency Parsing, Graph Attention, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION, Prediction, Relation Prediction, Sentiment Analysis, Structured Prediction

The state-of-the-art model for structured  sentiment analysis casts the task as a dependency parsing problem, which has some limitations: (1) The label proportions for span prediction and span relation prediction are imbalanced; (2) Two nodes in a dependency graph cannot have multiple arcs, which are necessary for this task; (3) The losses of predicting the imbalanced labels are directly applied in the prediction layer, which further exacerbate the imbalance problem. In this work, we propose nichetargeting solutions for this issues. First, we introduce a novel labeling strategy, which contains two sets of token pair labels, namely essential labels and whole labels. The essential label set consists of the minimum labels for this task, which are relatively balanced and applied in the prediction layer. The whole label set includes rich labels to help our model capture various token relations, which is imbalanced but merely applied in the hidden layer to softly influence our model. Moreover, we also propose an effective model to well collaborate with our labeling strategy, which is equipped with the graph attention network to iteratively refine token representations, and the adaptive multi-label classification to dynamically predict multiple relations between token pairs. We perform extensive experiments on 5 benchmark datasets in four languages. Experimental results show that our model outperforms previous SOTA models by a large margin. We believe that our labeling strategy and model can be well extended to other structured prediction tasks.
Self-training with Modeling Ambiguous Data for Low-Resource Relation Extraction
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Relation, Relation Extraction

We present a simple yet effective approach to improve the performance of self-training relation extraction in a low-resource scenario. The approach first classifies the auto-annotated instances into two groups: confident instances and uncertain instances, according to the probabilities predicted by a teacher model. In contrast to most previous studies, which mainly only use the confident instances for self-training, we make use of the uncertain instances. We propose a method to identify some ambiguous but useful instances from the uncertain instances. Then, we propose to utilize negative training for the ambiguous instances and positive training for the confident instances. Finally, they are combined in a joint-training manner to build a relation extraction system. Experimental results on two widely used datasets with low-resource settings demonstrate that this new approach indeed achieves significant and consistent improvements when compared to several competitive self-training systems.
Exploring Topic-Metadata Relationships with the STM: A Bayesian Approach
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: regression, Topic Models

The initial purpose of topic models was to identify latent topical clusters within unstructured text. Meanwhile, the focus of advanced studies  has changed primarily to estimating the relationship between the discovered topical structure and theoretically relevant metadata. Methods used to estimate such relationships must take into account that the topical structure is not directly observed, but instead being estimated itself in an unsupervised fashion. In the Structural Topic Model (STM;Roberts et al., 2016), for instance, multiple repeated linear regressions of sampled topic proportions on metadata covariates are performed. This is done by using a Monte Carlo sampling technique known as the \textit{method of composition}. In this paper, we propose two modifications of this approach: First, we implement a substantial correction to the model by replacing linear regression with the more appropriate Beta regression. Second, we provide a fundamental enhancement of the entire estimation framework by substituting the current blending of frequentist and Bayesian methods with a fully Bayesian approach instead. This allows for a more appropriate quantification of uncertainty. We illustrate our improved methodology by investigating relationships between Twitter posts by German parliamentarians and different metadata covariates related to their electoral districts.
Deep Learning for Fitness
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Deep Learning, Pose Estimation

We present Fitness tutor, an application for maintaining correct posture during workout exercises or doing yoga. Current work on fitness focuses on suggesting food supplements, accessing workouts, workout wearables does a great job in improving the fitness. Meanwhile, the current situation is making difficult to monitor workouts by trainee. Inspired by healthcare innovations like robotic surgery, we design a novel application Fitness tutor which can guide the workouts using pose estimation. Pose estimation can be deployed on the reference image for gathering data and guide the user with the data. This allow Fitness tutor to guide the workouts (both exercise and yoga) in remote conditions with a single reference posture as image. We use posenet model in tensorflow with p5js for developing skeleton. Fitness tutor is an application of pose estimation model in bringing a realtime teaching experience in fitness. Our experiments shows that it can leverage potential of pose estimation models by providing guidance in realtime.
Enabling Multimodal Generation on CLIP via Vision-Language Knowledge Distillation
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Image Captioning, Knowledge Distillation, Language Modeling, Language Modelling, multimodal generation, Question Answering, Visual Question Answering, Visual Question Answering (VQA)

The recent large-scale vision-language pre-training (VLP) of dual-stream architectures (e.g., CLIP) with a tremendous amount of image-text pair data, has shown its superiority on various multimodal alignment tasks. Despite its success, the resulting models are not capable of generative multimodal tasks due to the weak text encoder. To tackle this problem, we propose to augment the dual-stream VLP model with a textual pre-trained language model (PLM) via vision-language knowledge distillation (VLKD), enabling the capability for multimodal generation. VLKD is pretty data- and computation-efficient compared to the pre-training from scratch. Experimental results show that the resulting model has strong zero-shot performance on multimodal generation tasks, such as open-ended visual question answering and image captioning. For example, it achieves 39.7% zero-shot accuracy on the VQA 2.0 dataset, surpassing the previous state-of-the-art zero-shot model with 14x fewer parameters. Furthermore, the original text processing ability of the PLM is maintained after VLKD, which makes our model versatile for both multimodal and unimodal tasks.
Hansel: A Chinese Few-Shot and Zero-Shot Entity Linking Benchmark
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR September 2021 9
Tasks: Articles, Entity Linking

Modern Entity Linking (EL) systems entrench a popularity bias. However, there is no dataset focusing on tail and emerging entities in languages other than English. We present Hansel, a new benchmark in Chinese that fills the vacancy of non-English few-shot and zero-shot EL challenges. Hansel is human annotated and reviewed, with a novel method for collecting zero-shot EL datasets. It is a diverse dataset covering 8.2K documents in news, social media posts and other web articles, with Wikidata as its target Knowledge Base. We demonstrate that the existing state-of-the-art EL system performs poorly on Hansel (R@1 of 35.8% on Few-Shot). We then establish a strong baseline that scores a R@1 of 43.2% on Few-Shot and 76.6% on Zero-Shot on our dataset. We also show that our baseline achieves competitive results on TAC-KBP2015 Chinese Entity Linking task.