pwc-archive/papers-with-abstracts huggingface dataset:
A Splicing Approach to Best Subset of Groups Selection
Authors: Yanhang Zhang, Junxian Zhu, Jin Zhu, Xueqin Wang
Date: 2021-04-23 00:00:00
Tasks: Variable Selection

Best subset of groups selection (BSGS) is the process of selecting a small part of non-overlapping groups to achieve the best interpretability on the response variable. It has attracted increasing attention and has far-reaching applications in practice. However, due to the computational intractability of BSGS in high-dimensional settings, developing efficient algorithms for solving BSGS remains a research hotspot. In this paper,we propose a group-splicing algorithm that iteratively detects the relevant groups and excludes the irrelevant ones. Moreover, coupled with a novel group information criterion, we develop an adaptive algorithm to determine the optimal model size. Under mild conditions, it is certifiable that our algorithm can identify the optimal subset of groups in polynomial time with high probability. Finally, we demonstrate the efficiency and accuracy of our methods by comparing them with several state-of-the-art algorithms on both synthetic and real-world datasets.
MAQ-CaF: A Modular Air Quality Calibration and Forecasting method for cross-sensitive pollutants
Authors: Yousuf Hashmy, ZillUllah Khan, Rehan Hafiz, Usman Younis, Tausif Tauqeer
Date: 2021-04-22 00:00:00

The climatic challenges are rising across the globe in general and in worst hit under-developed countries in particular. The need for accurate measurements and forecasting of pollutants with low-cost deployment is more pertinent today than ever before. Low-cost air quality monitoring sensors are prone to erroneous measurements, frequent downtimes, and uncertain operational conditions. Such a situation demands a prudent approach to ensure an effective and flexible calibration scheme. We propose MAQ-CaF, a modular air quality calibration, and forecasting methodology, that side-steps the challenges of unreliability through its modular machine learning-based design which leverages the potential of IoT framework. It stores the calibrated data both locally and remotely with an added feature of future predictions. Our specially designed validation process helps to establish the proposed solution's applicability and flexibility without compromising accuracy. CO, SO2, NO2, O3, PM1.0, PM2.5 and PM10 were calibrated and monitored with reasonable accuracy. Such an attempt is a step toward addressing climate change's global challenge through appropriate monitoring and air quality tracking across a wider geographical region via affordable monitoring.
Focused Attention Improves Document-Grounded Generation
Authors: Shrimai Prabhumoye, Kazuma Hashimoto, Yingbo Zhou, Alan W Black, Ruslan Salakhutdinov
Date: 2021-04-26 00:00:00
Proceeding: NAACL 2021 4
Tasks: Decoder, Response Generation, Text Generation

Document grounded generation is the task of using the information provided in a document to improve text generation. This work focuses on two different document grounded generation tasks: Wikipedia Update Generation task and Dialogue response generation. Our work introduces two novel adaptations of large scale pre-trained encoder-decoder models focusing on building context driven representation of the document and enabling specific attention to the information in the document. Additionally, we provide a stronger BART baseline for these tasks. Our proposed techniques outperform existing methods on both automated (at least 48% increase in BLEU-4 points) and human evaluation for closeness to reference and relevance to the document. Furthermore, we perform comprehensive manual inspection of the generated output and categorize errors to provide insights into future directions in modeling these tasks.
Non-Parametric Few-Shot Learning for Word Sense Disambiguation
Authors: Howard Chen, Mengzhou Xia, Danqi Chen
Date: 2021-04-26 00:00:00
Proceeding: NAACL 2021 4
Tasks: Few-Shot Learning, Word Sense Disambiguation

Word sense disambiguation (WSD) is a long-standing problem in natural language processing. One significant challenge in supervised all-words WSD is to classify among senses for a majority of words that lie in the long-tail distribution. For instance, 84% of the annotated words have less than 10 examples in the SemCor training data. This issue is more pronounced as the imbalance occurs in both word and sense distributions. In this work, we propose MetricWSD, a non-parametric few-shot learning approach to mitigate this data imbalance issue. By learning to compute distances among the senses of a given word through episodic training, MetricWSD transfers knowledge (a learned metric space) from high-frequency words to infrequent ones. MetricWSD constructs the training episodes tailored to word frequencies and explicitly addresses the problem of the skewed distribution, as opposed to mixing all the words trained with parametric models in previous work. Without resorting to any lexical resources, MetricWSD obtains strong performance against parametric alternatives, achieving a 75.1 F1 score on the unified WSD evaluation benchmark (Raganato et al., 2017b). Our analysis further validates that infrequent words and senses enjoy significant improvement.
Evaluating the Values of Sources in Transfer Learning
Authors: Md Rizwan Parvez, Kai-Wei Chang
Date: 2021-04-26 00:00:00
Proceeding: NAACL 2021 4
Tasks: Cross-Lingual POS Tagging, Transfer Learning

Transfer learning that adapts a model trained on data-rich sources to low-resource targets has been widely applied in natural language processing (NLP). However, when training a transfer model over multiple sources, not every source is equally useful for the target. To better transfer a model, it is essential to understand the values of the sources. In this paper, we develop SEAL-Shap, an efficient source valuation framework for quantifying the usefulness of the sources (e.g., domains/languages) in transfer learning based on the Shapley value method. Experiments and comprehensive analyses on both cross-domain and cross-lingual transfers demonstrate that our framework is not only effective in choosing useful transfer sources but also the source values match the intuitive source-target similarity.
What Makes a Message Persuasive? Identifying Adaptations Towards Persuasiveness in Nine Exploratory Case Studies
Authors: Sebastian Duerr, Krystian Teodor Lange, Peter A. Gloor
Date: 2021-04-26 00:00:00
Tasks: Persuasiveness, Text Generation

The ability to persuade others is critical to professional and personal success. However, crafting persuasive messages is demanding and poses various challenges. We conducted nine exploratory case studies to identify adaptations that professional and non-professional writers make in written scenarios to increase their subjective persuasiveness. Furthermore, we identified challenges that those writers faced and identified strategies to resolve them with persuasive natural language generation, i.e., artificial intelligence. Our findings show that humans can achieve high degrees of persuasiveness (more so for professional-level writers), and artificial intelligence can complement them to achieve increased celerity and alignment in the process.
Attention vs non-attention for a Shapley-based explanation method
Authors: Tom Kersten, Hugh Mee Wong, Jaap Jumelet, Dieuwke Hupkes
Date: 2021-04-26 00:00:00
Proceeding: NAACL (DeeLIO) 2021 6

The field of explainable AI has recently seen an explosion in the number of explanation methods for highly non-linear deep neural networks. The extent to which such methods -- that are often proposed and tested in the domain of computer vision -- are appropriate to address the explainability challenges in NLP is yet relatively unexplored. In this work, we consider Contextual Decomposition (CD) -- a Shapley-based input feature attribution method that has been shown to work well for recurrent NLP models -- and we test the extent to which it is useful for models that contain attention operations. To this end, we extend CD to cover the operations necessary for attention-based models. We then compare how long distance subject-verb relationships are processed by models with and without attention, considering a number of different syntactic structures in two different languages: English and Dutch. Our experiments confirm that CD can successfully be applied for attention-based models as well, providing an alternative Shapley-based attribution method for modern neural networks. In particular, using CD, we show that the English and Dutch models demonstrate similar processing behaviour, but that under the hood there are consistent differences between our attention and non-attention models.
Teaching NLP with Bracelets and Restaurant Menus: An Interactive Workshop for Italian Students
Authors: Ludovica Pannitto, Lucia Busso, Claudia Roberta Combei, Lucio Messina, Alessio Miaschi, Gabriele Sarti, Malvina Nissim
Date: 2021-04-26 00:00:00
Proceeding: NAACL (TeachingNLP) 2021 6

Although Natural Language Processing (NLP) is at the core of many tools young people use in their everyday life, high school curricula (in Italy) do not include any computational linguistics education. This lack of exposure makes the use of such tools less responsible than it could be and makes choosing computational linguistics as a university degree unlikely. To raise awareness, curiosity, and longer-term interest in young people, we have developed an interactive workshop designed to illustrate the basic principles of NLP and computational linguistics to high school Italian students aged between 13 and 18 years. The workshop takes the form of a game in which participants play the role of machines needing to solve some of the most common problems a computer faces in understanding language: from voice recognition to Markov chains to syntactic parsing. Participants are guided through the workshop with the help of instructors, who present the activities and explain core concepts from computational linguistics. The workshop was presented at numerous outlets in Italy between 2019 and 2021, both face-to-face and online.
A dissemination workshop for introducing young Italian students to NLP
Authors: Lucio Messina, Lucia Busso, Claudia Roberta Combei, Ludovica Pannitto, Alessio Miaschi, Gabriele Sarti, Malvina Nissim
Date: 2021-04-26 00:00:00
Proceeding: NAACL (TeachingNLP) 2021 6

We describe and make available the game-based material developed for a laboratory run at several Italian science festivals to popularize NLP among young students.
DADgraph: A Discourse-aware Dialogue Graph Neural Network for Multiparty Dialogue Machine Reading Comprehension
Authors: Jiaqi Li, Ming Liu, Zihao Zheng, Heng Zhang, Bing Qin, Min-Yen Kan, Ting Liu
Date: 2021-04-26 00:00:00
Tasks: Graph Neural Network, Machine Reading Comprehension, Question Answering, Reading Comprehension

Multiparty Dialogue Machine Reading Comprehension (MRC) differs from traditional MRC as models must handle the complex dialogue discourse structure, previously unconsidered in traditional MRC. To fully exploit such discourse structure in multiparty dialogue, we present a discourse-aware dialogue graph neural network, DADgraph, which explicitly constructs the dialogue graph using discourse dependency links and discourse relations. To validate our model, we perform experiments on the Molweni corpus, a large-scale MRC dataset built over multiparty dialogue annotated with discourse structure. Experiments on Molweni show that our discourse-aware model achieves statistically significant improvements compared against strong neural network MRC baselines.
PanGu-$α$: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computation
Authors: Wei Zeng, Xiaozhe Ren, Teng Su, Hui Wang, Yi Liao, Zhiwei Wang, Xin Jiang, ZhenZhang Yang, Kaisheng Wang, Xiaoda Zhang, Chen Li, Ziyan Gong, Yifan Yao, Xinjing Huang, Jun Wang, Jianfeng Yu, Qi Guo, Yue Yu, Yan Zhang, Jin Wang, Hengtao Tao, Dasen Yan, Zexuan Yi, Fang Peng, Fangqing Jiang, Han Zhang, Lingfeng Deng, Yehong Zhang, Zhe Lin, Chao Zhang, Shaojie Zhang, Mingyue Guo, Shanzhi Gu, Gaojun Fan, YaoWei Wang, Xuefeng Jin, Qun Liu, Yonghong Tian
Date: 2021-04-26 00:00:00
Tasks: Cloze (multi-choices) (Few-Shot), Cloze (multi-choices) (One-Shot), Cloze (multi-choices) (Zero-Shot), Common Sense Reasoning (Few-Shot), Common Sense Reasoning (One-Shot), Common Sense Reasoning (Zero-Shot), Dialogue Generation, Few-Shot Image Classification, In-Context Learning, Natural Language Inference, Natural Language Inference (Few-Shot), Natural Language Inference (One-Shot), Natural Language Inference (Zero-Shot), Natural Language Understanding, Question Answering, Reading Comprehension, Reading Comprehension (Few-Shot), Reading Comprehension (One-Shot), Reading Comprehension (Zero-Shot), Text Classification, Text Summarization

Large-scale Pretrained Language Models (PLMs) have become the new paradigm for Natural Language Processing (NLP). PLMs with hundreds of billions parameters such as GPT-3 have demonstrated strong performances on natural language understanding and generation with \textit{few-shot in-context} learning. In this work, we present our practice on training large-scale autoregressive language models named PanGu-$\alpha$, with up to 200 billion parameters. PanGu-$\alpha$ is developed under the MindSpore and trained on a cluster of 2048 Ascend 910 AI processors. The training parallelism strategy is implemented based on MindSpore Auto-parallel, which composes five parallelism dimensions to scale the training task to 2048 processors efficiently, including data parallelism, op-level model parallelism, pipeline model parallelism, optimizer model parallelism and rematerialization. To enhance the generalization ability of PanGu-$\alpha$, we collect 1.1TB high-quality Chinese data from a wide range of domains to pretrain the model. We empirically test the generation ability of PanGu-$\alpha$ in various scenarios including text summarization, question answering, dialogue generation, etc. Moreover, we investigate the effect of model scales on the few-shot performances across a broad range of Chinese NLP tasks. The experimental results demonstrate the superior capabilities of PanGu-$\alpha$ in performing various tasks under few-shot or zero-shot settings.
A Sliding-Window Approach to Automatic Creation of Meeting Minutes
Authors: Jia Jin Koay, Alexander Roustai, Xiaojin Dai, Fei Liu
Date: 2021-04-26 00:00:00
Proceeding: NAACL 2021 4
Tasks: Navigate

Meeting minutes record any subject matters discussed, decisions reached and actions taken at meetings. The importance of minuting cannot be overemphasized in a time when a significant number of meetings take place in the virtual space. In this paper, we present a sliding window approach to automatic generation of meeting minutes. It aims to tackle issues associated with the nature of spoken text, including lengthy transcripts and lack of document structure, which make it difficult to identify salient content to be included in the meeting minutes. Our approach combines a sliding window and a neural abstractive summarizer to navigate through the transcripts to find salient content. The approach is evaluated on transcripts of natural meeting conversations, where we compare results obtained for human transcripts and two versions of automatic transcripts and discuss how and to what extent the summarizer succeeds at capturing salient content.
Reranking Machine Translation Hypotheses with Structured and Web-based Language Models
Authors: Wen Wang, Andreas Stolcke, Jing Zheng
Date: 2021-04-25 00:00:00
Tasks: Language Modeling, Language Modelling, Machine Translation, Reranking, Re-Ranking, Translation

In this paper, we investigate the use of linguistically motivated and computationally efficient structured language models for reranking N-best hypotheses in a statistical machine translation system. These language models, developed from Constraint Dependency Grammar parses, tightly integrate knowledge of words, morphological and lexical features, and syntactic dependency constraints. Two structured language models are applied for N-best rescoring, one is an almost-parsing language model, and the other utilizes more syntactic features by explicitly modeling syntactic dependencies between words. We also investigate effective and efficient language modeling methods to use N-grams extracted from up to 1 teraword of web documents. We apply all these language models for N-best re-ranking on the NIST and DARPA GALE program 2006 and 2007 machine translation evaluation tasks and find that the combination of these language models increases the BLEU score up to 1.6% absolutely on blind test sets.
Contextual-Lexicon Approach for Abusive Language Detection
Authors: Francielle Vargas, Fabiana Rodrigues de Góes, Isabelle Carvalho, Fabrício Benevenuto, Thiago Alexandre Salgueiro Pardo
Date: 2021-04-25 00:00:00
Proceeding: RANLP 2021 9
Tasks: Abusive Language, Hate Speech Detection

Since a lexicon-based approach is more elegant scientifically, explaining the solution components and being easier to generalize to other applications, this paper provides a new approach for offensive language and hate speech detection on social media. Our approach embodies a lexicon of implicit and explicit offensive and swearing expressions annotated with contextual information. Due to the severity of the social media abusive comments in Brazil, and the lack of research in Portuguese, Brazilian Portuguese is the language used to validate the models. Nevertheless, our method may be applied to any other language. The conducted experiments show the effectiveness of the proposed approach, outperforming the current baseline methods for the Portuguese language.
User Preference-aware Fake News Detection
Authors: Yingtong Dou, Kai Shu, Congying Xia, Philip S. Yu, Lichao Sun
Date: 2021-04-25 00:00:00
Tasks: Fact Checking, Fake News Detection, Graph Classification, Misinformation

Disinformation and fake news have posed detrimental effects on individuals and society in recent years, attracting broad attention to fake news detection. The majority of existing fake news detection algorithms focus on mining news content and/or the surrounding exogenous context for discovering deceptive signals; while the endogenous preference of a user when he/she decides to spread a piece of fake news or not is ignored. The confirmation bias theory has indicated that a user is more likely to spread a piece of fake news when it confirms his/her existing beliefs/preferences. Users' historical, social engagements such as posts provide rich information about users' preferences toward news and have great potential to advance fake news detection. However, the work on exploring user preference for fake news detection is somewhat limited. Therefore, in this paper, we study the novel problem of exploiting user preference for fake news detection. We propose a new framework, UPFD, which simultaneously captures various signals from user preferences by joint content and graph modeling. Experimental results on real-world datasets demonstrate the effectiveness of the proposed framework. We release our code and data as a benchmark for GNN-based fake news detection: https://github.com/safe-graph/GNN-FakeNews.
XLM-T: Multilingual Language Models in Twitter for Sentiment Analysis and Beyond
Authors: Francesco Barbieri, Luis Espinosa Anke, Jose Camacho-Collados
Date: 2021-04-25 00:00:00
Proceeding: LREC 2022 6
Tasks: Language Modelling, Sentiment Analysis, XLM-R

Language models are ubiquitous in current NLP, and their multilingual capacity has recently attracted considerable attention. However, current analyses have almost exclusively focused on (multilingual variants of) standard benchmarks, and have relied on clean pre-training and task-specific corpora as multilingual signals. In this paper, we introduce XLM-T, a model to train and evaluate multilingual language models in Twitter. In this paper we provide: (1) a new strong multilingual baseline consisting of an XLM-R (Conneau et al. 2020) model pre-trained on millions of tweets in over thirty languages, alongside starter code to subsequently fine-tune on a target task; and (2) a set of unified sentiment analysis Twitter datasets in eight different languages and a XLM-T model fine-tuned on them.
Identifying Offensive Expressions of Opinion in Context
Authors: Francielle Alves Vargas, Isabelle Carvalho, Fabiana Rodrigues de Góes
Date: 2021-04-25 00:00:00

Classic information extraction techniques consist in building questions and answers about the facts. Indeed, it is still a challenge to subjective information extraction systems to identify opinions and feelings in context. In sentiment-based NLP tasks, there are few resources to information extraction, above all offensive or hateful opinions in context. To fill this important gap, this short paper provides a new cross-lingual and contextual offensive lexicon, which consists of explicit and implicit offensive and swearing expressions of opinion, which were annotated in two different classes: context dependent and context-independent offensive. In addition, we provide markers to identify hate speech. Annotation approach was evaluated at the expression-level and achieves high human inter-annotator agreement. The provided offensive lexicon is available in Portuguese and English languages.
Automatic Post-Editing for Vietnamese
Authors: Thanh Vu, Dai Quoc Nguyen
Date: 2021-04-25 00:00:00
Proceeding: ALTA 2021 12
Tasks: Automatic Post-Editing, Machine Translation, Sentence, Translation

Automatic post-editing (APE) is an important remedy for reducing errors of raw translated texts that are produced by machine translation (MT) systems or software-aided translation. In this paper, we present a systematic approach to tackle the APE task for Vietnamese. Specifically, we construct the first large-scale dataset of 5M Vietnamese translated and corrected sentence pairs. We then apply strong neural MT models to handle the APE task, using our constructed dataset. Experimental results from both automatic and human evaluations show the effectiveness of the neural MT models in handling the Vietnamese APE task.
Open Intent Discovery through Unsupervised Semantic Clustering and Dependency Parsing
Authors: PengFei Liu, Youzhang Ning, King Keung Wu, Kun Li, Helen Meng
Date: 2021-04-25 00:00:00
Tasks: Clustering, Dependency Parsing, Intent Discovery, Open Intent Discovery, Sentence, Sentence Embeddings

Intent understanding plays an important role in dialog systems, and is typically formulated as a supervised learning problem. However, it is challenging and time-consuming to design the intents for a new domain from scratch, which usually requires a lot of manual effort of domain experts. This paper presents an unsupervised two-stage approach to discover intents and generate meaningful intent labels automatically from a collection of unlabeled utterances in a domain. In the first stage, we aim to generate a set of semantically coherent clusters where the utterances within each cluster convey the same intent. We obtain the utterance representation from various pre-trained sentence embeddings and present a metric of balanced score to determine the optimal number of clusters in K-means clustering for balanced datasets. In the second stage, the objective is to generate an intent label automatically for each cluster. We extract the ACTION-OBJECT pair from each utterance using a dependency parser and take the most frequent pair within each cluster, e.g., book-restaurant, as the generated intent label. We empirically show that the proposed unsupervised approach can generate meaningful intent labels automatically and achieve high precision and recall in utterance clustering and intent discovery.