pwc-archive/papers-with-abstracts huggingface dataset:
Enjoy the Salience: Towards Better Transformer-based Faithful Explanations with Word Salience
Authors: George Chrysostomou, Nikolaos Aletras
Date: 2021-08-31 00:00:00
Proceeding: EMNLP 2021 11

Pretrained transformer-based models such as BERT have demonstrated state-of-the-art predictive performance when adapted into a range of natural language processing tasks. An open problem is how to improve the faithfulness of explanations (rationales) for the predictions of these models. In this paper, we hypothesize that salient information extracted a priori from the training data can complement the task-specific information learned by the model during fine-tuning on a downstream task. In this way, we aim to help BERT not to forget assigning importance to informative input tokens when making predictions by proposing SaLoss; an auxiliary loss function for guiding the multi-head attention mechanism during training to be close to salient information extracted a priori using TextRank. Experiments for explanation faithfulness across five datasets, show that models trained with SaLoss consistently provide more faithful explanations across four different feature attribution methods compared to vanilla BERT. Using the rationales extracted from vanilla BERT and SaLoss models to train inherently faithful classifiers, we further show that the latter result in higher predictive performance in downstream tasks.
A Search Engine for Discovery of Scientific Challenges and Directions
Authors: Dan Lahav, Jon Saad Falcon, Bailey Kuehl, Sophie Johnson, Sravanthi Parasa, Noam Shomron, Duen Horng Chau, Diyi Yang, Eric Horvitz, Daniel S. Weld, Tom Hope
Date: 2021-08-31 00:00:00
Proceeding: NeurIPS Workshop AI4Scien 2021 12

Keeping track of scientific challenges, advances and emerging directions is a fundamental part of research. However, researchers face a flood of papers that hinders discovery of important knowledge. In biomedicine, this directly impacts human lives. To address this problem, we present a novel task of extraction and search of scientific challenges and directions, to facilitate rapid knowledge discovery. We construct and release an expert-annotated corpus of texts sampled from full-length papers, labeled with novel semantic categories that generalize across many types of challenges and directions. We focus on a large corpus of interdisciplinary work relating to the COVID-19 pandemic, ranging from biomedicine to areas such as AI and economics. We apply a model trained on our data to identify challenges and directions across the corpus and build a dedicated search engine. In experiments with 19 researchers and clinicians using our system, we outperform a popular scientific search engine in assisting knowledge discovery. Finally, we show that models trained on our resource generalize to the wider biomedical domain and to AI papers, highlighting its broad utility. We make our data, model and search engine publicly available. https://challenges.apps.allenai.org/
Plan-then-Generate: Controlled Data-to-Text Generation via Planning
Authors: Yixuan Su, David Vandyke, Sihui Wang, Yimai Fang, Nigel Collier
Date: 2021-08-31 00:00:00
Proceeding: Findings (EMNLP) 2021 11
Tasks: Data-to-Text Generation, Diversity, Sentence, Text Generation

Recent developments in neural networks have led to the advance in data-to-text generation. However, the lack of ability of neural models to control the structure of generated output can be limiting in certain real-world applications. In this study, we propose a novel Plan-then-Generate (PlanGen) framework to improve the controllability of neural data-to-text models. Extensive experiments and analyses are conducted on two benchmark datasets, ToTTo and WebNLG. The results show that our model is able to control both the intra-sentence and inter-sentence structure of the generated output. Furthermore, empirical comparisons against previous state-of-the-art methods show that our model improves the generation quality as well as the output diversity as judged by human and automatic evaluations.
Knowledge-Grounded Dialogue with Reward-Driven Knowledge Selection
Authors: Shilei Liu, Xiaofeng Zhao, Bochao Li, Feiliang Ren
Date: 2021-08-31 00:00:00
Tasks: Response Generation

Knowledge-grounded dialogue is a task of generating a fluent and informative response based on both conversation context and a collection of external knowledge, in which knowledge selection plays an important role and attracts more and more research interest. However, most existing models either select only one knowledge or use all knowledge for responses generation. The former may lose valuable information in discarded knowledge, while the latter may bring a lot of noise. At the same time, many approaches need to train the knowledge selector with knowledge labels that indicate ground-truth knowledge, but these labels are difficult to obtain and require a large number of manual annotations. Motivated by these issues, we propose Knoformer, a dialogue response generation model based on reinforcement learning, which can automatically select one or more related knowledge from the knowledge pool and does not need knowledge labels during training. Knoformer is evaluated on two knowledge-guided conversation datasets, and achieves state-of-the-art performance.
Faithful or Extractive? On Mitigating the Faithfulness-Abstractiveness Trade-off in Abstractive Summarization
Authors: Faisal Ladhak, Esin Durmus, He He, Claire Cardie, Kathleen McKeown
Date: 2021-08-31 00:00:00
Proceeding: ACL 2022 5
Tasks: Abstractive Text Summarization

Despite recent progress in abstractive summarization, systems still suffer from faithfulness errors. While prior work has proposed models that improve faithfulness, it is unclear whether the improvement comes from an increased level of extractiveness of the model outputs as one naive way to improve faithfulness is to make summarization models more extractive. In this work, we present a framework for evaluating the effective faithfulness of summarization systems, by generating a faithfulnessabstractiveness trade-off curve that serves as a control at different operating points on the abstractiveness spectrum. We then show that the Maximum Likelihood Estimation (MLE) baseline as well as a recently proposed method for improving faithfulness, are both worse than the control at the same level of abstractiveness. Finally, we learn a selector to identify the most faithful and abstractive summary for a given document, and show that this system can attain higher faithfulness scores in human evaluations while being more abstractive than the baseline system on two datasets. Moreover, we show that our system is able to achieve a better faithfulness-abstractiveness trade-off than the control at the same level of abstractiveness.
Gray Cycles of Maximum Length Related to k-Character Substitutions
Authors: Jean Néraud
Date: 2021-08-31 00:00:00
Tasks: Relation

Given a word binary relation $\tau$ we define a $\tau$-Gray cycle over a finite language X to be a permutation w [i] 0$\le$i$\le$|X|--1 of X such that each word wi is an image of the previous word wi--1 by $\tau$. In that framework, we introduce the complexity measure $\lambda$(n), equal to the largest cardinality of a language X having words of length at most n, and such that a $\tau$-Gray cycle over X exists. The present paper is concerned with the relation $\tau$ = $\sigma$ k , the so-called k-character substitution, where (u, v) belongs to $\sigma$ k if, and only if, the Hamming distance of u and v is k. We compute the bound $\lambda$(n) for all cases of the alphabet cardinality and the argument n.
Automatic Rule Generation for Time Expression Normalization
Authors: Wentao Ding, Jianhao Chen, Jinmao Li, Yuzhong Qu
Date: 2021-08-31 00:00:00
Proceeding: Findings (EMNLP) 2021 11

The understanding of time expressions includes two sub-tasks: recognition and normalization. In recent years, significant progress has been made in the recognition of time expressions while research on normalization has lagged behind. Existing SOTA normalization methods highly rely on rules or grammars designed by experts, which limits their performance on emerging corpora, such as social media texts. In this paper, we model time expression normalization as a sequence of operations to construct the normalized temporal value, and we present a novel method called ARTime, which can automatically generate normalization rules from training data without expert interventions. Specifically, ARTime automatically captures possible operation sequences from annotated data and generates normalization rules on time expressions with common surface forms. The experimental results show that ARTime can significantly surpass SOTA methods on the Tweets benchmark, and achieves competitive results with existing expert-engineered rule methods on the TempEval-3 benchmark.
MELM: Data Augmentation with Masked Entity Language Modeling for Low-Resource NER
Authors: Ran Zhou, Xin Li, Ruidan He, Lidong Bing, Erik Cambria, Luo Si, Chunyan Miao
Date: 2021-08-31 00:00:00
Proceeding: ACL 2022 5
Tasks: Cross-Lingual NER, Data Augmentation, Language Modeling, Language Modelling, Masked Language Modeling, NER, Sentence

Data augmentation is an effective solution to data scarcity in low-resource scenarios. However, when applied to token-level tasks such as NER, data augmentation methods often suffer from token-label misalignment, which leads to unsatsifactory performance. In this work, we propose Masked Entity Language Modeling (MELM) as a novel data augmentation framework for low-resource NER. To alleviate the token-label misalignment issue, we explicitly inject NER labels into sentence context, and thus the fine-tuned MELM is able to predict masked entity tokens by explicitly conditioning on their labels. Thereby, MELM generates high-quality augmented data with novel entities, which provides rich entity regularity knowledge and boosts NER performance. When training data from multiple languages are available, we also integrate MELM with code-mixing for further improvement. We demonstrate the effectiveness of MELM on monolingual, cross-lingual and multilingual NER across various low-resource levels. Experimental results show that our MELM presents substantial improvement over the baseline methods.
Discretized Integrated Gradients for Explaining Language Models
Authors: Soumya Sanyal, Xiang Ren
Date: 2021-08-31 00:00:00
Proceeding: EMNLP 2021 11
Tasks: Feature Importance, Sentiment Analysis, Sentiment Classification

As a prominent attribution-based explanation algorithm, Integrated Gradients (IG) is widely adopted due to its desirable explanation axioms and the ease of gradient computation. It measures feature importance by averaging the model's output gradient interpolated along a straight-line path in the input data space. However, such straight-line interpolated points are not representative of text data due to the inherent discreteness of the word embedding space. This questions the faithfulness of the gradients computed at the interpolated points and consequently, the quality of the generated explanations. Here we propose Discretized Integrated Gradients (DIG), which allows effective attribution along non-linear interpolation paths. We develop two interpolation strategies for the discrete word embedding space that generates interpolation points that lie close to actual words in the embedding space, yielding more faithful gradient computation. We demonstrate the effectiveness of DIG over IG through experimental and human evaluations on multiple sentiment classification datasets. We provide the source code of DIG to encourage reproducible research.
Dynamic Sliding Window for Meeting Summarization
Authors: Zhengyuan Liu, Nancy F. Chen
Date: 2021-08-31 00:00:00
Tasks: Meeting Summarization

Recently abstractive spoken language summarization raises emerging research interest, and neural sequence-to-sequence approaches have brought significant performance improvement. However, summarizing long meeting transcripts remains challenging. Due to the large length of source contents and targeted summaries, neural models are prone to be distracted on the context, and produce summaries with degraded quality. Moreover, pre-trained language models with input length limitations cannot be readily applied to long sequences. In this work, we first analyze the linguistic characteristics of meeting transcripts on a representative corpus, and find that the sentences comprising the summary correlate with the meeting agenda. Based on this observation, we propose a dynamic sliding window strategy for meeting summarization. Experimental results show that performance benefit from the proposed method, and outputs obtain higher factual consistency than the base model.
T3-Vis: a visual analytic framework for Training and fine-Tuning Transformers in NLP
Authors: Raymond Li, Wen Xiao, Lanjun Wang, Hyeju Jang, Giuseppe Carenini
Date: 2021-08-31 00:00:00

Transformers are the dominant architecture in NLP, but their training and fine-tuning is still very challenging. In this paper, we present the design and implementation of a visual analytic framework for assisting researchers in such process, by providing them with valuable insights about the model's intrinsic properties and behaviours. Our framework offers an intuitive overview that allows the user to explore different facets of the model (e.g., hidden states, attention) through interactive visualization, and allows a suite of built-in algorithms that compute the importance of model components and different parts of the input sequence. Case studies and feedback from a user focus group indicate that the framework is useful, and suggest several improvements.
Linguistic Characterization of Divisive Topics Online: Case Studies on Contentiousness in Abortion, Climate Change, and Gun Control
Authors: Jacob Beel, Tong Xiang, Sandeep Soni, Diyi Yang
Date: 2021-08-30 00:00:00

As public discourse continues to move and grow online, conversations about divisive topics on social media platforms have also increased. These divisive topics prompt both contentious and non-contentious conversations. Although what distinguishes these conversations, often framed as what makes these conversations contentious, is known in broad strokes, much less is known about the linguistic signature of these conversations. Prior work has shown that contentious content and structure can be a predictor for this task, however, most of them have been focused on conversation in general, very specific events, or complex structural analysis. Additionally, many models used in prior work have lacked interpret-ability, a key factor in online moderation. Our work fills these gaps by focusing on conversations from highly divisive topics (abortion, climate change, and gun control), operationalizing a set of novel linguistic and conversational characteristics and user factors, and incorporating them to build interpretable models. We demonstrate that such characteristics can largely improve the performance of prediction on this task, and also enable nuanced interpretability. Our case studies on these three contentious topics suggest that certain generic linguistic characteristics are highly correlated with contentiousness in conversations while others demonstrate significant contextual influences on specific divisive topics.
Towards Consistent Document-level Entity Linking: Joint Models for Entity Linking and Coreference Resolution
Authors: Klim Zaporojets, Johannes Deleu, Yiwei Jiang, Thomas Demeester, Chris Develder
Date: 2021-08-30 00:00:00
Proceeding: ACL 2022 5
Tasks: coreference-resolution, Coreference Resolution, Entity Linking, Structured Prediction

We consider the task of document-level entity linking (EL), where it is important to make consistent decisions for entity mentions over the full document jointly. We aim to leverage explicit "connections" among mentions within the document itself: we propose to join the EL task with that of coreference resolution (coref). This is complementary to related works that exploit either (i) implicit document information (e.g., latent relations among entity mentions, or general language models) or (ii) connections between the candidate links (e.g, as inferred from the external knowledge base). Specifically, we cluster mentions that are linked via coreference, and enforce a single EL for all of the clustered mentions together. The latter constraint has the added benefit of increased coverage by joining EL candidate lists for the thus clustered mentions. We formulate the coref+EL problem as a structured prediction task over directed trees and use a globally normalized model to solve it. Experimental results on two datasets show a boost of up to +5% F1-score on both coref and EL tasks, compared to their standalone counterparts. For a subset of hard cases, with individual mentions lacking the correct EL in their candidate entity list, we obtain a +50% increase in accuracy.
ConVIScope: Visual Analytics for Exploring Patient Conversations
Authors: Raymond Li, Enamul Hoque, Giuseppe Carenini, Richard Lester, Raymond Chau
Date: 2021-08-30 00:00:00

The proliferation of text messaging for mobile health is generating a large amount of patient-doctor conversations that can be extremely valuable to health care professionals. We present ConVIScope, a visual text analytic system that tightly integrates interactive visualization with natural language processing in analyzing patient-doctor conversations. ConVIScope was developed in collaboration with healthcare professionals following a user-centered iterative design. Case studies with six domain experts suggest the potential utility of ConVIScope and reveal lessons for further developments.
Uniform Consistency in Nonparametric Mixture Models
Authors: Bryon Aragam, Ruiyi Yang
Date: 2021-08-31 00:00:00
Tasks: regression

We study uniform consistency in nonparametric mixture models as well as closely related mixture of regression (also known as mixed regression) models, where the regression functions are allowed to be nonparametric and the error distributions are assumed to be convolutions of a Gaussian density. We construct uniformly consistent estimators under general conditions while simultaneously highlighting several pain points in extending existing pointwise consistency results to uniform results. The resulting analysis turns out to be nontrivial, and several novel technical tools are developed along the way. In the case of mixed regression, we prove $L^1$ convergence of the regression functions while allowing for the component regression functions to intersect arbitrarily often, which presents additional technical challenges. We also consider generalizations to general (i.e. non-convolutional) nonparametric mixtures.
Double Machine Learning for Partially Linear Mixed-Effects Models with Repeated Measurements
Authors: Corinne Emmenegger, Peter Bühlmann
Date: 2021-08-31 00:00:00
Tasks: BIG-bench Machine Learning

Traditionally, spline or kernel approaches in combination with parametric estimation are used to infer the linear coefficient (fixed effects) in a partially linear mixed-effects model for repeated measurements. Using machine learning algorithms allows us to incorporate complex interaction structures and high-dimensional variables. We employ double machine learning to cope with the nonparametric part of the partially linear mixed-effects model: the nonlinear variables are regressed out nonparametrically from both the linear variables and the response. This adjustment can be performed with any machine learning algorithm, for instance random forests, which allows to take complex interaction terms and nonsmooth structures into account. The adjusted variables satisfy a linear mixed-effects model, where the linear coefficient can be estimated with standard linear mixed-effects techniques. We prove that the estimated fixed effects coefficient converges at the parametric rate, is asymptotically Gaussian distributed, and semiparametrically efficient. Two simulation studies demonstrate that our method outperforms a penalized regression spline approach in terms of coverage. We also illustrate our proposed approach on a longitudinal dataset with HIV-infected individuals. Software code for our method is available in the R-package dmlalg.
An Analysis Of Entire Space Multi-Task Models For Post-Click Conversion Prediction
Authors: Conor O'Brien, Kin Sum Liu, James Neufeld, Rafael Barreto, Jonathan J Hunt
Date: 2021-08-18 00:00:00
Tasks: Multi-Task Learning, Recommendation Systems

Industrial recommender systems are frequently tasked with approximating probabilities for multiple, often closely related, user actions. For example, predicting if a user will click on an advertisement and if they will then purchase the advertised product. The conceptual similarity between these tasks has promoted the use of multi-task learning: a class of algorithms that aim to bring positive inductive transfer from related tasks. Here, we empirically evaluate multi-task learning approaches with neural networks for an online advertising task. Specifically, we consider approximating the probability of post-click conversion events (installs) (CVR) for mobile app advertising on a large-scale advertising platform, using the related click events (CTR) as an auxiliary task. We use an ablation approach to systematically study recent approaches that incorporate both multitask learning and "entire space modeling" which train the CVR on all logged examples rather than learning a conditional likelihood of conversion given clicked. Based on these results we show that several different approaches result in similar levels of positive transfer from the data-abundant CTR task to the CVR task and offer some insight into how the multi-task design choices address the two primary problems affecting the CVR task: data sparsity and data bias. Our findings add to the growing body of evidence suggesting that standard multi-task learning is a sensible approach to modelling related events in real-world large-scale applications and suggest the specific multitask approach can be guided by ease of implementation in an existing system.
RRU-Net: The Ringed Residual U-Net for Image Splicing Forgery Detection
Authors: Xiuli Bi, Yang Wei, Bin Xiao, Weisheng Li
Date: 2019-10-01 00:00:00
Proceeding: cvpr 2019 workshop 2019 10
Tasks: Attribute

Detecting a splicing forgery image and then locating the forgery regions is a challenging task. Some traditional feature extraction methods and convolutional neural network (CNN)-based detection methods have been proposed to ﬁnish this task by exploring the differences of image attributes between the un-tampered and tampered regions in an image. However, the performance of the existing detection methods is unsatisfactory. In this paper, we propose a ringed residual U-Net (RRU-Net) for image splicing forgery detection. The proposed RRU-Net is an end-to-end image essence attribute segmentation network, which is independent of human visual system, it can accomplish the forgery detection without any preprocessing and post-processing. The core idea of the RRU-Net is to strengthen the learning way of CNN, which is inspired by the recall and the consolidation mechanism of the human brain and implemented by the propagation and the feedback process of the residual in CNN. The residual propagation recalls the input feature information to solve the gradient degradation problem in the deeper network; the residual feedback consolidates the input feature information to make the differences of image attributes between the un-tampered and tampered regions be more obvious. Experimental results show that the proposed detection method can achieve a promising result compared with the state-of-the-art splicing forgery detection methods.