pwc-archive/papers-with-abstracts huggingface dataset:
Focus on the Target’s Vocabulary: Masked Label Smoothing for Machine Translation
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Machine Translation, Translation

Label smoothing and vocabulary sharing are two widely used techniques in neural machine translation models. However, we argue that jointly adopting these two techniques can be conflicting and even leads to sub-optimal performance, since the soft label produced by label smoothing still considers the source-side words that would not appear at the target side. To address this issue, we propose Masked Label Smoothing (MLS), a new mechanism that masks the soft label probability of source-side words to zero. Simple yet effective, MLS manages to better integrate label smoothing with vocabulary sharing and hence improves the quality of the translation. Our extensive experiments show that MLS consistently yields improvement over original label smoothing on different datasets, including bilingual and multilingual translation in both BLEU and calibration scores.
Prompt-Learning for Fine-Grained Entity Typing
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Entity Typing, Knowledge Probing, Language Modeling, Language Modelling, Masked Language Modeling, Natural Language Inference, Prompt Learning, Sentiment Analysis, Sentiment Classification

As an effective approach to tune pre-trained language models (PLMs) for specific tasks, prompt-learning has recently attracted much attention from researchers. By using cloze-style language prompts to stimulate the versatile knowledge of PLMs, prompt-learning can achieve promising results on a series of NLP tasks, such as natural language inference, sentiment classification, and knowledge probing. In this work, we investigate the application of prompt-learning on fine-grained entity typing in fully supervised, few-shot, and zero-shot scenarios. 
We first develop a simple and effective prompt-learning pipeline by constructing entity-oriented verbalizer and templates and conducting masked language modeling. Further, to tackle the zero-shot regime, we propose a self-supervised strategy that carries out distribution-level optimization in prompt-learning to automatically summarize the information of entity types. Extensive experiments on three fine-grained entity typing benchmarks (with up to 86 classes) under fully supervised, few-shot and zero-shot settings show that prompt-learning methods significantly outperform fine-tuning baselines, especially when the training data is insufficient.
Progressive Down-Sampling for Acoustic Encoding
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), speech-recognition, Speech Recognition, Translation

In acoustic encoding, the fine-grained frame-level features are not suited for capturing global dependencies. But condensing them into a semantically complete representation by stacked down-sampling does not work well. We find that the condensation leads to the degraded correlation of the representations in adjacent positions, which poses the risk of information loss in the stacked method. In this work, we propose a new method, progressive down-sampling (PDS), for encoding the context sufficiently before each condensation. Also, we develop a representation fusion method to alleviate information loss by combining the multi-scale representations. Experimental results on the 960h LibriSpeech automatic speech recognition task show that, for a strong Conformer-based system, our method down-samples the input speech features to 1/32 of the initial length, while yielding an improvement of 0.47 WER with a speedup of 1.42$\times$. It also achieves the state-of-the-art BLEU score (25.8) on the MuST-C En-De speech translation benchmark with no additional training data.
SANCL: Multimodal Review Helpfulness Prediction with Selective Attention and Natural Contrastive Learning
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Contrastive Learning

With the boom of e-commerce, Multimodal Review Helpfulness Prediction (MRHP) that identifies the helpfulness score of multimodal product reviews has become a research hotspot. Previous work on this task focuses on attention-based modality fusion, information integration, and relation modeling, which primarily exposes the following drawbacks: 1) the model may fail to capture the real essential information due to the indiscriminate attention formulation; 2) analysis on relations and proper formulation are missing, generating noise samples and degenerating the modeling quality. In this paper, we propose SANCL: Selective Attention and Natural Contrastive Learning for MRHP. SANCL adopts a probe-based strategy to enforce high attention weights on the regions of greater significance. It also constructs a contrastive learning framework based on natural matching properties in the dataset. Experimental results on two benchmark datasets with three categories show that SANCL achieves state-of-the-art baseline performance with lower memory consumption.
LogInsights: Understanding and Extracting Information from Logs for Fault Classification at run-time
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Management

Software monitoring is the most critical part in any software management life cycle. One of the ways to detect the health of the program and the software is to monitor the logs efficiently. In this paper, we describe a method to process a stream of logs for identifying any fault being mentioned in the log at runtime. At first, we extract meaningful features for detecting the erroneous ones from the stream of logs. Next, we categorize the erroneous logs into the pre-defined categories of commonly occurring faults, using the proposed two-step framework. We propose efficient, fast and intelligent rule-based systems with the domain knowledge being incorporated using the word embedding model. We have built a domain specific corpus and trained a word embedding model for this purpose. The methods described here have shown improved results in the existing product pipeline. Experiments on logs obtained from various applications also show the efficacy of our proposed method.
Tracking Satisfaction States for Customer Satisfaction Prediction in E-commerce Service Chatbots
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Chatbot

Due to the increasing use of service chatbots in E-commerce platforms in recent years, customer satisfaction prediction (CSP) is gaining more and more attention. CSP is dedicated to evaluating subjective customer satisfaction in conversational service and thus helps improve customer service experience. However, previous methods focus on modeling customer-chatbot interaction at different single turns, neglecting the important dynamic satisfaction states throughout the customer journey. In this work, we investigate the problem of satisfaction states tracking and its effects on CSP in E-commerce service chatbots. To this end, we propose a dialogue-level classification model named DialogueCSP to track satisfaction states for CSP. In particular, we explore a novel two-step interaction module to represent the dynamic satisfaction states at each turn. In order to capture dialogue-level satisfaction states for CSP, we further introduce dialogue-aware attentions to integrate historical informative cues into the interaction module. To evaluate the proposed approach, we also build a Chinese E-commerce dataset for CSP. Experiment results demonstrate that our model significantly outperforms multiple baselines, illustrating the benefits of satisfaction states tracking on CSP.
Coreference Resolution as Span Boundary Alignment
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: coreference-resolution, Coreference Resolution

We propose a new fast and accurate solution for coreference resolution, in which the task is formulated as a span boundary alignment problem. In this solution, a mention is linked to another one via two edges modeling how likely two linked mentions point to the same entity. Specifically, for each mention, its head word (left boundary) needs to be well aligned with the head words of all other mentions that refer to the same entity, so does its tail word (right boundary).  Such a ``head-to-head'' and ``tail-to-tail'' alignment strategy greatly reduces the computational complexity of coreference decisions on any pair of mentions, mitigates the error propagation problem caused by mention pruning, and encourages the sharing of features across all mentions that refer to the same entity. Experimental results show that our solution achieves close to state-of-the-art performance on the CoNLL-2012 and GAP benchmarks with much less computational cost.
Enhancing Robustness of Pre-trained Language Model with Lexical Simplification
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Diversity, Language Modeling, Language Modelling, Lexical Simplification, Sentence, text-classification, Text Classification

For both human readers and pre-trained language models (PrLMs), lexical diversity may lead to confusion and inaccuracy when understanding the underlying semantic meanings of given sentences. By substituting complex words with simple alternatives, lexical simplification (LS) is a recognized method to reduce such lexical diversity. In this paper, we leverage a novel improved LS approach which can enhance robustness of PrLMs, resulting in improved performances in downstream tasks. A rule-based simplification process is applied to a given sentence. PrLMs are encouraged to predict the real label of the given sentence with auxiliary inputs from the simplified version. Using strong PrLMs (BERT and ELECTRA) as baselines, our approach can still further improve the performance in various text classification tasks.
Topic-Aware Response Generation in Task-Oriented Dialogue with Unstructured Knowledge Access
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Response Generation, Sentence, Task-Oriented Dialogue Systems

To alleviate the problem of structured databases' limited coverage, recent task-oriented dialogue systems incorporate external unstructured knowledge to guide the generation of system responses. However, these usually use word or sentence level similarities to detect the relevant knowledge context, which only partially captures the topical level relevance. In this paper, we examine how to better integrate topical information in knowledge grounded task-oriented dialogue and propose ``Topic-Aware Response Generation'' (TARG), an end-to-end response generation model. TARG incorporates multiple topic-aware attention mechanisms to derive the importance weighting scheme over dialogue utterances and external knowledge sources towards a better understanding of the dialogue history. Experimental results indicate that TARG achieves state-of-the-art performance in knowledge selection and response generation, outperforming previous state-of-the-art by 3.2, 3.6, and 4.2 points in EM, F1 and BLEU-4 respectively on Doc2Dial, and performing comparably with previous work on DSTC9; both being knowledge-grounded task-oriented dialogue datasets.\footnote{Code will be made public on the paper's acceptance.}
End-to-end Reference-free Single-document Summary Quality Assessment
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Language Modeling, Language Modelling

Canonical automatic summary evaluation metrics, such as ROUGE, require the presence of reference summaries, which are often expensive or impractical to obtain, especially in industry applications. Such metrics do not capture linguistic qualities effectively either. To holistically address the limitations, we introduce a reference-free, weakly supervised approach to summary quality assessment. Two negative sampling methods are proposed to generate training samples from massively available document-summary pairs and our approach is as simple as fine-tuning a pre-trained language model using the generated samples on a straightforward task.  Although simple, our strategy outperforms reference-free baselines with substantial improvements nearly all the time at the summary level, and oftentimes at the system level, on TAC2010, RealSumm and  Newsroom datasets. It also outperforms many reference-based metrics including ROUGE on linguistic aspects. 
Our code and models are open-sourced at \url{https://anonymous.4open.science/r/37CF}.
Better Language Model with Hypernym Class Prediction
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Language Modeling, Language Modelling, model, Prediction

Class-based language models (LMs) have been long devised to address context sparsity in $n$-gram LMs. In this study, we revisit this approach in the context of neural LMs. We hypothesize that class-based prediction leads to an implicit context aggregation for similar words and thus can improve generalization for rare words. We map words that have a common WordNet hypernym to the same class and train large neural LMs by gradually annealing from predicting the class to token prediction during training. Empirically, this curriculum learning strategy consistently improves perplexity over various large, highly-performant state-of-the-art Transformer-based models on two datasets, WikiText-103 and Arxiv. Our analysis shows that the performance improvement is achieved without sacrificing performance on rare words. Finally, we document other attempts that failed to yield empirical gains, and discuss future directions for the adoption of class-based LMs on a larger scale.
EntSUM: A Data Set for Entity-Centric Extractive Summarization
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Extractive Summarization

Controllable summarization aims to provide summaries that take into account user-specified aspects and preferences to better assist them with their information need, as opposed to the standard summarization setup which build a single generic summary of a document.
We introduce a human-annotated data set EntSUM for controllable summarization with a focus on named entities as the aspects to control.
We conduct an extensive quantitative analysis to motivate the task of entity-centric summarization and show that existing methods for controllable summarization fail to generate entity-centric summaries. We propose extensions to state-of-the-art summarization approaches that achieve substantially better results on our data set. Our analysis and results show the challenging nature of this task and of the proposed data set.
When Does Translation Require Context? A Data-driven, Multilingual Exploration
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Machine Translation, Translation

Although proper handling of discourse phenomena significantly contributes to the quality of machine translation (MT), improvements on these phenomena are not adequately measured in common translation quality metrics. Recent works in context-aware MT attempt to target a small set of these phenomena during evaluation. In this paper, we propose a methodology to identify translations that require context systematically, and use this methodology to both confirm the difficulty of previously studied phenomena as well as uncover new ones that have not been addressed in previous work. We then develop the \textbf{Mu}ltilingual \textbf{D}iscourse-\textbf{A}ware (MuDA) benchmark, a series of taggers for these phenomena in 14 different language pairs, which we use to evaluate context-aware MT. We find that state-of-the-art context-aware MT models make marginal improvements over context-agnostic models, which suggests current models do not handle these ambiguities effectively. We will release code and data to invite the MT research community to increase efforts on context-aware translation on discourse phenomena and languages that are currently overlooked.
On the Origin of Hallucinations in Conversational Models: Is it the Datasets or the Models?
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Hallucination

Although knowledge-grounded conversational models are able to generate fluent responses that are indistinguishable from human-generated ones, they are known to suffer from producing factually invalid statements, a phenomenon commonly called hallucination.  In this work, we investigate the underlying causes of this phenomenon: is hallucination due to the training data, or to the models? We conduct a comprehensive human study on both existing knowledge-grounded dialogue datasets and several state-of-the-art models. Our study reveals that the standard benchmarks consist of more than 60% hallucinated responses, leading to models that not only hallucinate but even amplify hallucinations. Moreover, we qualitatively analyze the nature of hallucinations, and identify key response strategies used by humans and models that lead to hallucinations. We hope these insights will show the way forward towards building hallucination-free conversational models.
Speaker Profiling in Multi-party Conversations
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: All, Speaker Profiling

In a conversation, individual speakers respond uniquely. Consequently, a `one size fits all' technique is not the best way for a dialog agent to generate responses. While many studies design personalized dialog agents with the help of persona information of speakers, all of them assume that speaker persona is supplied beforehand. However, this assumption does not hold for all conversational agents. This paper attempts to address this gap by exploring the task -- Speaker Profiling in Conversations (SPC). The aim of SPC is to produce a summary of the persona characteristics for individual speakers present in a dialog. We divide this task into two subtasks -- persona discovery and persona-type identification. Given a dialog, the former subtask aims to find all its utterances that carry persona information. The latter subtask focuses on evaluating these utterances to identify the type of persona information present in them. We present SPICE, a novel dataset that is specifically curated with labels to tackle the task of SPC. We show the performance of various baselines on SPICE and benchmark it using SPOT, a neural model based on GRU and Transformer. SPOT shows an improvement of ~7% for persona discovery and ~3% for persona-type identification over the best baseline. We further present a thorough analysis of SPOT to diagnose individual modules and their limitations, both quantitatively and qualitatively.
Context-Aware Language Modeling for Goal-Oriented Dialogue Systems
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Data Augmentation, Goal-Oriented Dialogue Systems, Language Modeling, Language Modelling, Open-Ended Question Answering, Reinforcement Learning (RL), Text Generation

Goal-oriented dialogue systems has long faced the trade-off between fluent language generation and task-specific control. While supervised learning with large language models are capable of producing realistic responses, how to steer such responses towards completing a specific task without sacrificing language quality remains an open question. In this work, by viewing a goal-oriented dialogue system as a reinforcement learning (RL) problem, we turn a supervised language model into a dynamics model and a behavioral cloning policy in a partially observable Markov decision process. This view allows RL techniques such as task relabeling and goal-conditioned policy to be naturally adopted as a form of data augmentation and task-specific fintuning of language models. We evaluate our method, Context-Aware Language Models (\method), on a practical flight-booking task using AirDialogue. Empirically, \method outperforms the previous state-of-the-art method by more than 10\% in terms of task success, achieving human-level task performance on this dataset.
Relevant CommonSense Subgraphs for "What if..." Procedural Reasoning
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11

This work deals with the challenge of learning causal reasoning over procedural text to answer "What if..." questions when external commonsense knowledge is required. We propose a novel multi-hop graph reasoning model to 1) efficiently extract a commonsense subgraph with the most relevant information from a large knowledge graph; 2) predict the causal answer by reasoning over the representations obtained from the commonsense subgraph and the contextual interactions between the questions and context. We evaluate our model on WIQA dataset and achieve state-of-the-art performance compared to the recent models.
Do Current Natural Language Inference Models Truly Understand Sentences? Insights from Simple Sentences
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Natural Language Inference, Sentence

Natural language inference (NLI) is a task to infer the relationship between a premise and a hypothesis (e.g. entailment, neutral, or contradiction), and transformer-based models perform well on current NLI datasets such as MNLI and SNLI. Nevertheless, given the complexity of the task, especially the complexity of the sentences used for model evaluations, it remains controversial whether these models can truly infer the meaning of sentences or they simply guess the answer via non-humanlike heuristics. Here, we reduce the complexity of the task using two approaches. The first approach simplifies the relationship between the premise and hypothesis by making them unrelated. A test set, referred to as Random Pair, is constructed by randomly pairing premises and hypotheses in MNLI/SNLI. Models fine-tuned on MNLI/SNLI identify a large proportion (up to 77.6%) of these unrelated statements as being contradictory. Models fine-tuned on SICK, a dataset that included unrelated premise-hypothesis pairs, perform well on Random Pair. The second approach simplifies the task by constraining the premises/hypotheses to be syntactically/semantically simple sentences. A new test set, referred to as Simple Pair, is constructed using simple sentences, such as short SVO sentences, and basic conjunction sentences. We find that models fine-tuned on MNLI/SNLI generally fail to understand these simple sentences, but their performance can be boosted by re-fine-tuning the models using only a few hundreds of samples from SICK. All models tested here, however, fail to understand the fundamental compositional binding relation between a subject and a predicate (up to ~100% error rate) for basic conjunction sentences. Taken together, the results show that models achieving high accuracy on mainstream datasets can still lack basic sentence comprehension capacity, and datasets discouraging non-humanlike heuristics are required to build more robust NLI models.
Learning to Ignore Adversarial Attacks
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Data Augmentation

Despite the strong performance of current NLP models, they can be brittle against adversarial attacks. To enable effective learning against adversarial inputs, we introduce the use of rationale models that can explicitly learn to ignore attack tokens. We find that the rationale models can ignore over 90\% of attack tokens. This approach leads to consistent sizable improvements ($\sim$8\%) over baseline models in robustness, for both BERT and RoBERTa, on MultiRC and FEVER, and also reliably outperforms data augmentation with adversarial examples alone. In many cases, we find that our method is able to close the gap between model performance on a clean test set and an attacked test set, eliminating the effect of adversarial attacks.