pwc-archive/papers-with-abstracts huggingface dataset:
The Current Chinese Global Supply Chain Monopoly and the Covid-19 Pandemic
Authors: George Rapciewicz Jr., Dr. Donald Buresh
Date: 2021-03-23 00:00:00
Tasks: Decision Making, Management

Because of the ongoing Covid-19 crisis, supply chain management performance seems to be struggling. The purpose of this paper is to examine a variety of critical factors related to the application of contingency theory to determine its feasibility in preventing future supply chain bottlenecks. The study reviewed current online news reports, previous research on contingency theory, as well as strategic and structural contingency theories. This paper also systematically reviewed several global supply chain management and strategic decision-making studies in an effort to promote a new strategy. The findings indicated that the need for mass production of products within the United States, as well as within trading partners, is necessary to prevent additional Covid-19 related supply chain gaps. The paper noted that in many instances, the United States has become dependent on foreign products, where the prevention of future supply chain gaps requires the United States restore its manufacturing prowess.
Low-Resource Machine Translation Training Curriculum Fit for Low-Resource Languages
Authors: Garry Kuwanto, Afra Feyza Akyürek, Isidora Chara Tourni, Siyang Li, Alexander Gregory Jones, Derry Wijaya
Date: 2021-03-24 00:00:00
Tasks: Cross-Lingual Bitext Mining, Language Modelling, Low-Resource Neural Machine Translation, Machine Translation, NMT, Translation

We conduct an empirical study of neural machine translation (NMT) for truly low-resource languages, and propose a training curriculum fit for cases when both parallel training data and compute resource are lacking, reflecting the reality of most of the world's languages and the researchers working on these languages. Previously, unsupervised NMT, which employs back-translation (BT) and auto-encoding (AE) tasks has been shown barren for low-resource languages. We demonstrate that leveraging comparable data and code-switching as weak supervision, combined with BT and AE objectives, result in remarkable improvements for low-resource languages even when using only modest compute resources. The training curriculum proposed in this work achieves BLEU scores that improve over supervised NMT trained on the same backbone architecture by +12.2 BLEU for English to Gujarati and +3.7 BLEU for English to Kazakh, showcasing the potential of weakly-supervised NMT for the low-resource languages. When trained on supervised data, our training curriculum achieves a new state-of-the-art result on the Somali dataset (BLEU of 29.3 for Somali to English). We also observe that adding more time and GPUs to training can further improve performance, which underscores the importance of reporting compute resource usage in MT research.
Relay Selection with Imperfect SIC for FD/HD NOMA Cooperative Networks over Nakagami-$m$ Fades
Authors: Efendi Fidan, Eray Güven, Mehmet Akif Durmaz, Güneş Karabulut Kurt, Oğuz Kucur
Date: 2021-03-23 00:00:00
Tasks: valid

In this work, a non-orthogonal multiple access (NOMA) based transmission between two sources and two end-users is examined over independent non-identically distributed (i.n.i.d.) slow Nakagami-$m$ fading channels, where a single relay using decode-and-forward (DF) protocol is selected out of a set of full-duplex/half-duplex (FD/HD) multiple relays in accordance with the quality of service criterion. Two relay selection (RS) strategies, selecting a relay to maximize data rate of user 1 at selected relay and selecting a relay out of a set of relays providing service quality for user 1 to maximize data rate of user 2, are analyzed. Additionally, not only perfect successive interference cancellation (pSIC) but also imperfect SIC (ipSIC) is considered. The exact and asymptotic outage probability (OP) expressions are derived and validated via Monte Carlo simulation technique. Unlike existing works, our expressions are unique and valid for all cases such as FD and HD together with pSIC and ipSIC, i.e. expressions are not given separately but in a single compact form. Effect of each component such as pSIC, ipSIC, and self-interference (SI) for FD transmission on error floor of OP is demonstrated. Moreover, the optimum relay location is illustrated for a plenty of scenarios consisting of combination of different power allocations, data rates, pSIC/ipSIC, and total transmitted powers.
Joint framework with deep feature distillation and adaptive focal loss for weakly supervised audio tagging and acoustic event detection
Authors: Yunhao Liang, Yanhua Long, Yijie Li, Jiaen Liang, Yuping Wang
Date: 2021-03-23 00:00:00
Tasks: Audio Tagging, Event Detection

A good joint training framework is very helpful to improve the performances of weakly supervised audio tagging (AT) and acoustic event detection (AED) simultaneously. In this study, we propose three methods to improve the best teacher-student framework in the IEEE AASP Challenge on Detection and Classification of Acoustic Scenes and Events (DCASE) 2019 Task 4 for both audio tagging and acoustic events detection tasks. A frame-level target-events based deep feature distillation is first proposed, which aims to leverage the potential of limited strong-labeled data in weakly supervised framework to learn better intermediate feature maps. Then, we propose an adaptive focal loss and two-stage training strategy to enable an effective and more accurate model training, where the contribution of hard and easy acoustic events to the total cost function can be automatically adjusted. Furthermore, an event-specific post processing is designed to improve the prediction of target event time-stamps. Our experiments are performed on the public DCASE 2019 Task 4 dataset, results show that our approach achieves competitive performances in both AT (81.2\% F1-score) and AED (49.8\% F1-score) tasks.
Rule-Based Safety-Critical Control Design using Control Barrier Functions with Application to Autonomous Lane Change
Authors: Suiyi He, Jun Zeng, Bike Zhang, Koushil Sreenath
Date: 2021-03-23 00:00:00

This paper develops a new control design for guaranteeing a vehicle's safety during lane change maneuvers in a complex traffic environment. The proposed method uses a finite state machine (FSM), where a quadratic program based optimization problem using control Lyapunov functions and control barrier functions (CLF-CBF-QP) is used to calculate the system's optimal inputs via rule-based control strategies. The FSM can make switches between different states automatically according to the command of driver and traffic environment, which makes the ego vehicle find a safe opportunity to do a collision-free lane change maneuver. By using a convex quadratic program, the controller can guarantee the system's safety at a high update frequency. A set of pre-designed typical lane change scenarios as well as randomly generated driving scenarios are simulated to show the performance of our controller.
What Do We Get from Two-Way Fixed Effects Regressions? Implications from Numerical Equivalence
Authors: Shoya Ishimaru
Date: 2021-03-23 00:00:00
Tasks: regression

In any multiperiod panel, a two-way fixed effects (TWFE) regression is numerically equivalent to a first-difference (FD) regression that pools all possible between-period gaps. Building on this observation, this paper develops numerical and causal interpretations of the TWFE coefficient. At the sample level, the TWFE coefficient is a weighted average of FD coefficients with different between-period gaps. This decomposition improves transparency by revealing the sources of variation that the TWFE coefficient captures. At the population level, causal interpretation of the TWFE coefficient relies on a common trends assumption for any between-period gap, conditional on changes, not levels, of time-varying covariates. I propose a simple modification to the TWFE approach that naturally eases this assumption.
Unsupervised Contextual Paraphrase Generation using Lexical Control and Reinforcement Learning
Authors: Sonal Garg, Sumanth Prabhu, Hemant Misra, G. Srinivasaraghavan
Date: 2021-03-23 00:00:00
Tasks: Diversity, Natural Language Inference, Paraphrase Generation, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Semantic Similarity, Semantic Textual Similarity

Customer support via chat requires agents to resolve customer queries with minimum wait time and maximum customer satisfaction. Given that the agents as well as the customers can have varying levels of literacy, the overall quality of responses provided by the agents tend to be poor if they are not predefined. But using only static responses can lead to customer detraction as the customers tend to feel that they are no longer interacting with a human. Hence, it is vital to have variations of the static responses to reduce monotonicity of the responses. However, maintaining a list of such variations can be expensive. Given the conversation context and the agent response, we propose an unsupervised frame-work to generate contextual paraphrases using autoregressive models. We also propose an automated metric based on Semantic Similarity, Textual Entailment, Expression Diversity and Fluency to evaluate the quality of contextual paraphrases and demonstrate performance improvement with Reinforcement Learning (RL) fine-tuning using the automated metric as the reward function.
Repairing Pronouns in Translation with BERT-Based Post-Editing
Authors: Reid Pryzant
Date: 2021-03-23 00:00:00
Tasks: Machine Translation, NMT, Translation

Pronouns are important determinants of a text's meaning but difficult to translate. This is because pronoun choice can depend on entities described in previous sentences, and in some languages pronouns may be dropped when the referent is inferrable from the context. These issues can lead Neural Machine Translation (NMT) systems to make critical errors on pronouns that impair intelligibility and even reinforce gender bias. We investigate the severity of this pronoun issue, showing that (1) in some domains, pronoun choice can account for more than half of a NMT systems' errors, and (2) pronouns have a disproportionately large impact on perceived translation quality. We then investigate a possible solution: fine-tuning BERT on a pronoun prediction task using chunks of source-side sentences, then using the resulting classifier to repair the translations of an existing NMT model. We offer an initial case study of this approach for the Japanese-English language pair, observing that a small number of translations are significantly improved according to human evaluators.
Towards a Formal Model of Narratives
Authors: Louis Castricato, Stella Biderman, Rogelio E. Cardona-Rivera, David Thue
Date: 2021-03-23 00:00:00
Tasks: model

In this paper, we propose the beginnings of a formal framework for modeling narrative \textit{qua} narrative. Our framework affords the ability to discuss key qualities of stories and their communication, including the flow of information from a Narrator to a Reader, the evolution of a Reader's story model over time, and Reader uncertainty. We demonstrate its applicability to computational narratology by giving explicit algorithms for measuring the accuracy with which information was conveyed to the Reader and two novel measurements of story coherence.
Complex Factoid Question Answering with a Free-Text Knowledge Graph
Authors: Chen Zhao, Chenyan Xiong, Xin Qian, Jordan Boyd-Graber
Date: 2021-03-23 00:00:00
Tasks: Graph Neural Network, Graph Question Answering, Question Answering, Reading Comprehension

We introduce DELFT, a factoid question answering system which combines the nuance and depth of knowledge graph question answering approaches with the broader coverage of free-text. DELFT builds a free-text knowledge graph from Wikipedia, with entities as nodes and sentences in which entities co-occur as edges. For each question, DELFT finds the subgraph linking question entity nodes to candidates using text sentences as edges, creating a dense and high coverage semantic graph. A novel graph neural network reasons over the free-text graph-combining evidence on the nodes via information along edge sentences-to select a final answer. Experiments on three question answering datasets show DELFT can answer entity-rich questions better than machine reading based models, bert-based answer ranking and memory networks. DELFT's advantage comes from both the high coverage of its free-text knowledge graph-more than double that of dbpedia relations-and the novel graph neural network which reasons on the rich but noisy free-text evidence.
TeCoMiner: Topic Discovery Through Term Community Detection
Authors: Andreas Hamm, Jana Thelen, Rasmus Beckmann, Simon Odrowski
Date: 2021-03-23 00:00:00
Tasks: Community Detection

This note is a short description of TeCoMiner, an interactive tool for exploring the topic content of text collections. Unlike other topic modeling tools, TeCoMiner is not based on some generative probabilistic model but on topological considerations about co-occurrence networks of terms. We outline the methods used for identifying topics, describe the features of the tool, and sketch an application, using a corpus of policy related scientific news on environmental issues published by the European Commission over the last decade.
UNICORN on RAINBOW: A Universal Commonsense Reasoning Model on a New Multitask Benchmark
Authors: Nicholas Lourie, Ronan Le Bras, Chandra Bhagavatula, Yejin Choi
Date: 2021-03-24 00:00:00
Tasks: Common Sense Reasoning, HellaSwag, Knowledge Graphs, Question Answering, Sentence Completion, Transfer Learning, Winogrande

Commonsense AI has long been seen as a near impossible goal -- until recently. Now, research interest has sharply increased with an influx of new benchmarks and models. We propose two new ways to evaluate commonsense models, emphasizing their generality on new tasks and building on diverse, recently introduced benchmarks. First, we propose a new multitask benchmark, RAINBOW, to promote research on commonsense models that generalize well over multiple tasks and datasets. Second, we propose a novel evaluation, the cost equivalent curve, that sheds new insight on how the choice of source datasets, pretrained language models, and transfer learning methods impacts performance and data efficiency. We perform extensive experiments -- over 200 experiments encompassing 4800 models -- and report multiple valuable and sometimes surprising findings, e.g., that transfer almost always leads to better or equivalent performance if following a particular recipe, that QA-based commonsense datasets transfer well with each other, while commonsense knowledge graphs do not, and that perhaps counter-intuitively, larger models benefit more from transfer than smaller ones. Last but not least, we introduce a new universal commonsense reasoning model, UNICORN, that establishes new state-of-the-art performance across 8 popular commonsense benchmarks, aNLI (87.3%), CosmosQA (91.8%), HellaSWAG (93.9%), PIQA (90.1%), SocialIQa (83.2%), WinoGrande (86.6%), CycIC (94.0%) and CommonsenseQA (79.3%).
Topic Modeling Genre: An Exploration of French Classical and Enlightenment Drama
Authors: Christof Schöch
Date: 2021-03-24 00:00:00
Tasks: Clustering

The concept of literary genre is a highly complex one: not only are different genres frequently defined on several, but not necessarily the same levels of description, but consideration of genres as cognitive, social, or scholarly constructs with a rich history further complicate the matter. This contribution focuses on thematic aspects of genre with a quantitative approach, namely Topic Modeling. Topic Modeling has proven to be useful to discover thematic patterns and trends in large collections of texts, with a view to class or browse them on the basis of their dominant themes. It has rarely if ever, however, been applied to collections of dramatic texts. In this contribution, Topic Modeling is used to analyze a collection of French Drama of the Classical Age and the Enlightenment. The general aim of this contribution is to discover what semantic types of topics are found in this collection, whether different dramatic subgenres have distinctive dominant topics and plot-related topic patterns, and inversely, to what extent clustering methods based on topic scores per play produce groupings of texts which agree with more conventional genre distinctions. This contribution shows that interesting topic patterns can be detected which provide new insights into the thematic, subgenre-related structure of French drama as well as into the history of French drama of the Classical Age and the Enlightenment.
Czert -- Czech BERT-like Model for Language Representation
Authors: Jakub Sido, Ondřej Pražák, Pavel Přibáň, Jan Pašek, Michal Seják, Miloslav Konopík
Date: 2021-03-24 00:00:00
Tasks: model

This paper describes the training process of the first Czech monolingual language representation models based on BERT and ALBERT architectures. We pre-train our models on more than 340K of sentences, which is 50 times more than multilingual models that include Czech data. We outperform the multilingual models on 9 out of 11 datasets. In addition, we establish the new state-of-the-art results on nine datasets. At the end, we discuss properties of monolingual and multilingual models based upon our results. We publish all the pre-trained and fine-tuned models freely for the research community.
Thinking Aloud: Dynamic Context Generation Improves Zero-Shot Reasoning Performance of GPT-2
Authors: Gregor Betz, Kyle Richardson, Christian Voigt
Date: 2021-03-24 00:00:00
Tasks: Language Modeling, Language Modelling, Natural Language Inference

Thinking aloud is an effective meta-cognitive strategy human reasoners apply to solve difficult problems. We suggest to improve the reasoning ability of pre-trained neural language models in a similar way, namely by expanding a task's context with problem elaborations that are dynamically generated by the language model itself. Our main result is that dynamic problem elaboration significantly improves the zero-shot performance of GPT-2 in a deductive reasoning and natural language inference task: While the model uses a syntactic heuristic for predicting an answer, it is capable (to some degree) of generating reasoned additional context which facilitates the successful application of its heuristic. We explore different ways of generating elaborations, including fewshot learning, and find that their relative performance varies with the specific problem characteristics (such as problem difficulty). Moreover, the effectiveness of an elaboration can be explained in terms of the degree to which the elaboration semantically coheres with the corresponding problem. In particular, elaborations that are most faithful to the original problem description may boost accuracy by up to 24%.
Finetuning Pretrained Transformers into RNNs
Authors: Jungo Kasai, Hao Peng, Yizhe Zhang, Dani Yogatama, Gabriel Ilharco, Nikolaos Pappas, Yi Mao, Weizhu Chen, Noah A. Smith
Date: 2021-03-24 00:00:00
Proceeding: EMNLP 2021 11
Tasks: Language Modelling, Machine Translation, Text Generation

Transformers have outperformed recurrent neural networks (RNNs) in natural language generation. But this comes with a significant computational cost, as the attention mechanism's complexity scales quadratically with sequence length. Efficient transformer variants have received increasing interest in recent works. Among them, a linear-complexity recurrent variant has proven well suited for autoregressive generation. It approximates the softmax attention with randomized or heuristic feature maps, but can be difficult to train and may yield suboptimal accuracy. This work aims to convert a pretrained transformer into its efficient recurrent counterpart, improving efficiency while maintaining accuracy. Specifically, we propose a swap-then-finetune procedure: in an off-the-shelf pretrained transformer, we replace the softmax attention with its linear-complexity recurrent alternative and then finetune. With a learned feature map, our approach provides an improved tradeoff between efficiency and accuracy over the standard transformer and other recurrent variants. We also show that the finetuning process has lower training cost relative to training these recurrent variants from scratch. As many models for natural language tasks are increasingly dependent on large-scale pretrained transformers, this work presents a viable approach to improving inference efficiency without repeating the expensive pretraining process.
Paragraph-level Rationale Extraction through Regularization: A case study on European Court of Human Rights Cases
Authors: Ilias Chalkidis, Manos Fergadiotis, Dimitrios Tsarapatsanis, Nikolaos Aletras, Ion Androutsopoulos, Prodromos Malakasiotis
Date: 2021-03-24 00:00:00
Proceeding: NAACL 2021 4

Interpretability or explainability is an emerging research field in NLP. From a user-centric point of view, the goal is to build models that provide proper justification for their decisions, similar to those of humans, by requiring the models to satisfy additional constraints. To this end, we introduce a new application on legal text where, contrary to mainstream literature targeting word-level rationales, we conceive rationales as selected paragraphs in multi-paragraph structured court cases. We also release a new dataset comprising European Court of Human Rights cases, including annotations for paragraph-level rationales. We use this dataset to study the effect of already proposed rationale constraints, i.e., sparsity, continuity, and comprehensiveness, formulated as regularizers. Our findings indicate that some of these constraints are not beneficial in paragraph-level rationale extraction, while others need re-formulation to better handle the multi-label nature of the task we consider. We also introduce a new constraint, singularity, which further improves the quality of rationales, even compared with noisy rationale supervision. Experimental results indicate that the newly introduced task is very challenging and there is a large scope for further research.
Finnish Paraphrase Corpus
Authors: Jenna Kanerva, Filip Ginter, Li-Hsin Chang, Iiro Rastas, Valtteri Skantsi, Jemina Kilpeläinen, Hanna-Mari Kupari, Jenna Saarni, Maija Sevón, Otto Tarkka
Date: 2021-03-24 00:00:00
Proceeding: NoDaLiDa 2021 5
Tasks: All

In this paper, we introduce the first fully manually annotated paraphrase corpus for Finnish containing 53,572 paraphrase pairs harvested from alternative subtitles and news headings. Out of all paraphrase pairs in our corpus 98% are manually classified to be paraphrases at least in their given context, if not in all contexts. Additionally, we establish a manual candidate selection method and demonstrate its feasibility in high quality paraphrase selection in terms of both cost and quality.
Representing Numbers in NLP: a Survey and a Vision
Authors: Avijit Thawani, Jay Pujara, Pedro A. Szekely, Filip Ilievski
Date: 2021-03-24 00:00:00
Proceeding: NAACL 2021 4
Tasks: Survey

NLP systems rarely give special consideration to numbers found in text. This starkly contrasts with the consensus in neuroscience that, in the brain, numbers are represented differently from words. We arrange recent NLP work on numeracy into a comprehensive taxonomy of tasks and methods. We break down the subjective notion of numeracy into 7 subtasks, arranged along two dimensions: granularity (exact vs approximate) and units (abstract vs grounded). We analyze the myriad representational choices made by 18 previously published number encoders and decoders. We synthesize best practices for representing numbers in text and articulate a vision for holistic numeracy in NLP, comprised of design trade-offs and a unified evaluation.