pwc-archive/papers-with-abstracts huggingface dataset:
Evaluation of soccer team defense based on prediction models of ball recovery and being attacked: A pilot study
Authors: Kosuke Toda, Masakiyo Teranishi, Keisuke Kushiro, Keisuke Fujii
Date: 2021-03-17 00:00:00

With the development of measurement technology, data on the movements of actual games in various sports can be obtained and used for planning and evaluating the tactics and strategy. Defense in team sports is generally difficult to be evaluated because of the lack of statistical data. Conventional evaluation methods based on predictions of scores are considered unreliable because they predict rare events throughout the game. Besides, it is difficult to evaluate various plays leading up to a score. In this study, we propose a method to evaluate team defense from a comprehensive perspective related to team performance by predicting ball recovery and being attacked, which occur more frequently than goals, using player actions and positional data of all players and the ball. Using data from 45 soccer matches, we examined the relationship between the proposed index and team performance in actual matches and throughout a season. Results show that the proposed classifiers predicted the true events (mean F1 score $>$ 0.483) better than the existing classifiers which were based on rare events or goals (mean F1 score $<$ 0.201). Also, the proposed index had a moderate correlation with the long-term outcomes of the season ($r =$ 0.397). These results suggest that the proposed index might be a more reliable indicator rather than winning or losing with the inclusion of accidental factors.
Code Word Detection in Fraud Investigations using a Deep-Learning Approach
Authors: Youri van der Zee, Jan C. Scholtes, Marcel Westerhoud, Julien Rossi
Date: 2021-03-17 00:00:00
Tasks: BIG-bench Machine Learning

In modern litigation, fraud investigators often face an overwhelming number of documents that must be reviewed throughout a matter. In the majority of legal cases, fraud investigators do not know beforehand, exactly what they are looking for, nor where to find it. In addition, fraudsters may use deception to hide their behaviour and intentions by using code words. Effectively, this means fraud investigators are looking for a needle in the haystack without knowing what the needle looks like. As part of a larger research program, we use a framework to expedite the investigation process applying text-mining and machine learning techniques. We structure this framework using three well-known methods in fraud investigations: (i) the fraud triangle (ii) the golden ("W") investigation questions, and (iii) the analysis of competing hypotheses. With this framework, it is possible to automatically organize investigative data, so it is easier for investigators to find answers to typical investigative questions. In this research, we focus on one of the components of this framework: the identification of the usage of code words by fraudsters. Here for, a novel (annotated) synthetic data set is created containing such code words, hidden in normal email communication. Subsequently, a range of machine learning techniques are employed to detect such code words. We show that the state-of-the-art BERT model significantly outperforms other methods on this task. With this result, we demonstrate that deep neural language models can reliably (F1 score of 0.9) be applied in fraud investigations for the detection of code words.
DoubleML -- An Object-Oriented Implementation of Double Machine Learning in R
Authors: Philipp Bach, Victor Chernozhukov, Malte S. Kurz, Martin Spindler, Sven Klaassen
Date: 2021-03-17 00:00:00
Tasks: BIG-bench Machine Learning, valid

The R package DoubleML implements the double/debiased machine learning framework of Chernozhukov et al. (2018). It provides functionalities to estimate parameters in causal models based on machine learning methods. The double machine learning framework consist of three key ingredients: Neyman orthogonality, high-quality machine learning estimation and sample splitting. Estimation of nuisance components can be performed by various state-of-the-art machine learning methods that are available in the mlr3 ecosystem. DoubleML makes it possible to perform inference in a variety of causal models, including partially linear and interactive regression models and their extensions to instrumental variable estimation. The object-oriented implementation of DoubleML enables a high flexibility for the model specification and makes it easily extendable. This paper serves as an introduction to the double machine learning framework and the R package DoubleML. In reproducible code examples with simulated and real data sets, we demonstrate how DoubleML users can perform valid inference based on machine learning methods.
Code-Mixing on Sesame Street: Dawn of the Adversarial Polyglots
Authors: Samson Tan, Shafiq Joty
Date: 2021-03-17 00:00:00
Proceeding: NAACL (CALCS) 2021 6
Tasks: Cross-Lingual Transfer, XLM-R

Multilingual models have demonstrated impressive cross-lingual transfer performance. However, test sets like XNLI are monolingual at the example level. In multilingual communities, it is common for polyglots to code-mix when conversing with each other. Inspired by this phenomenon, we present two strong black-box adversarial attacks (one word-level, one phrase-level) for multilingual models that push their ability to handle code-mixed sentences to the limit. The former uses bilingual dictionaries to propose perturbations and translations of the clean example for sense disambiguation. The latter directly aligns the clean example with its translations before extracting phrases as perturbations. Our phrase-level attack has a success rate of 89.75% against XLM-R-large, bringing its average accuracy of 79.85 down to 8.18 on XNLI. Finally, we propose an efficient adversarial training scheme that trains in the same number of steps as the original model and show that it improves model accuracy.
Automatic Generation of Contrast Sets from Scene Graphs: Probing the Compositional Consistency of GQA
Authors: Yonatan Bitton, Gabriel Stanovsky, Roy Schwartz, Michael Elhadad
Date: 2021-03-17 00:00:00
Proceeding: NAACL 2021 4
Tasks: Question Answering, Relational Reasoning, Visual Question Answering, Visual Question Answering (VQA)

Recent works have shown that supervised models often exploit data artifacts to achieve good test scores while their performance severely degrades on samples outside their training distribution. Contrast sets (Gardneret al., 2020) quantify this phenomenon by perturbing test samples in a minimal way such that the output label is modified. While most contrast sets were created manually, requiring intensive annotation effort, we present a novel method which leverages rich semantic input representation to automatically generate contrast sets for the visual question answering task. Our method computes the answer of perturbed questions, thus vastly reducing annotation cost and enabling thorough evaluation of models' performance on various semantic aspects (e.g., spatial or relational reasoning). We demonstrate the effectiveness of our approach on the GQA dataset and its semantic scene graph image representation. We find that, despite GQA's compositionality and carefully balanced label distribution, two high-performing models drop 13-17% in accuracy compared to the original test set. Finally, we show that our automatic perturbation can be applied to the training set to mitigate the degradation in performance, opening the door to more robust models.
An Efficient Method for the Classification of Croplands in Scarce-Label Regions
Authors: Houtan Ghaffari
Date: 2021-03-17 00:00:00
Tasks: Classification, Domain Adaptation, General Classification, Open-Ended Question Answering, Time Series Analysis, Unsupervised Domain Adaptation

Two of the main challenges for cropland classification by satellite time-series images are insufficient ground-truth data and inaccessibility of high-quality hyperspectral images for under-developed areas. Unlabeled medium-resolution satellite images are abundant, but how to benefit from them is an open question. We will show how to leverage their potential for cropland classification using self-supervised tasks. Self-supervision is an approach where we provide simple training signals for the samples, which are apparent from the data's structure. Hence, they are cheap to acquire and explain a simple concept about the data. We introduce three self-supervised tasks for cropland classification. They reduce epistemic uncertainty, and the resulting model shows superior accuracy in a wide range of settings compared to SVM and Random Forest. Subsequently, we use the self-supervised tasks to perform unsupervised domain adaptation and benefit from the labeled samples in other regions. It is crucial to know what information to transfer to avoid degrading the performance. We show how to automate the information selection and transfer process in cropland classification even when the source and target areas have a very different feature distribution. We improved the model by about 24% compared to a baseline architecture without any labeled sample in the target domain. Our method is amenable to gradual improvement, works with medium-resolution satellite images, and does not require complicated models. Code and data are available.
Theoretical bounds on data requirements for the ray-based classification
Authors: Brian J. Weber, Sandesh S. Kalantre, Thomas McJunkin, Jacob M. Taylor, Justyna P. Zwolak
Date: 2021-03-17 00:00:00
Tasks: Classification, General Classification

The problem of classifying high-dimensional shapes in real-world data grows in complexity as the dimension of the space increases. For the case of identifying convex shapes of different geometries, a new classification framework has recently been proposed in which the intersections of a set of one-dimensional representations, called rays, with the boundaries of the shape are used to identify the specific geometry. This ray-based classification (RBC) has been empirically verified using a synthetic dataset of two- and three-dimensional shapes (Zwolak et al. in Proceedings of Third Workshop on Machine Learning and the Physical Sciences (NeurIPS 2020), Vancouver, Canada [December 11, 2020], arXiv:2010.00500, 2020) and, more recently, has also been validated experimentally (Zwolak et al., PRX Quantum 2:020335, 2021). Here, we establish a bound on the number of rays necessary for shape classification, defined by key angular metrics, for arbitrary convex shapes. For two dimensions, we derive a lower bound on the number of rays in terms of the shape's length, diameter, and exterior angles. For convex polytopes in $\mathbb{R}^N$, we generalize this result to a similar bound given as a function of the dihedral angle and the geometrical parameters of polygonal faces. This result enables a different approach for estimating high-dimensional shapes using substantially fewer data elements than volumetric or surface-based approaches.
The U-Net based GLOW for Optical-Flow-free Video Interframe Generation
Authors: Saem Park, Donghoon Han, Nojun Kwak
Date: 2021-03-17 00:00:00
Tasks: Occlusion Handling, Optical Flow Estimation, Video Frame Interpolation

Video frame interpolation is the task of creating an interframe between two adjacent frames along the time axis. So, instead of simply averaging two adjacent frames to create an intermediate image, this operation should maintain semantic continuity with the adjacent frames. Most conventional methods use optical flow, and various tools such as occlusion handling and object smoothing are indispensable. Since the use of these various tools leads to complex problems, we tried to tackle the video interframe generation problem without using problematic optical flow . To enable this , we have tried to use a deep neural network with an invertible structure, and developed an U-Net based Generative Flow which is a modified normalizing flow. In addition, we propose a learning method with a new consistency loss in the latent space to maintain semantic temporal consistency between frames. The resolution of the generated image is guaranteed to be identical to that of the original images by using an invertible network. Furthermore, as it is not a random image like the ones by generative models, our network guarantees stable outputs without flicker. Through experiments, we \sam {confirmed the feasibility of the proposed algorithm and would like to suggest the U-Net based Generative Flow as a new possibility for baseline in video frame interpolation. This paper is meaningful in that it is the world's first attempt to use invertible networks instead of optical flows for video interpolation.
Regularized Behavior Value Estimation
Authors: Caglar Gulcehre, Sergio Gómez Colmenarejo, Ziyu Wang, Jakub Sygnowski, Thomas Paine, Konrad Zolna, Yutian Chen, Matthew Hoffman, Razvan Pascanu, Nando de Freitas
Date: 2021-03-17 00:00:00
Tasks: Offline RL

Offline reinforcement learning restricts the learning process to rely only on logged-data without access to an environment. While this enables real-world applications, it also poses unique challenges. One important challenge is dealing with errors caused by the overestimation of values for state-action pairs not well-covered by the training data. Due to bootstrapping, these errors get amplified during training and can lead to divergence, thereby crippling learning. To overcome this challenge, we introduce Regularized Behavior Value Estimation (R-BVE). Unlike most approaches, which use policy improvement during training, R-BVE estimates the value of the behavior policy during training and only performs policy improvement at deployment time. Further, R-BVE uses a ranking regularisation term that favours actions in the dataset that lead to successful outcomes. We provide ample empirical evidence of R-BVE's effectiveness, including state-of-the-art performance on the RL Unplugged ATARI dataset. We also test R-BVE on new datasets, from bsuite and a challenging DeepMind Lab task, and show that R-BVE outperforms other state-of-the-art discrete control offline RL methods.
Modeling differential rates of aging using routine laboratory data; Implications for morbidity and health care expenditure
Authors: Alix Jean Santos, Xavier Eugenio Asuncion, Camille Rivero-Co, Maria Eloisa Ventura, Reynaldo Geronia II, Lauren Bangerter, Natalie E. Sheils
Date: 2021-03-17 00:00:00

Aging is a multidimensional process where phenotypes change at varying rates. Longitudinal studies of aging typically involve following a cohort of individuals over the course of several years. This design is hindered by cost, attrition, and subsequently small sample size. Alternative methodologies are therefore warranted. In this study, we used a variational autoencoder to estimate rates of aging from cross-sectional data from routine laboratory tests of 1.4 million individuals collected from 2016 to 2019. By incorporating metrics that would ensure model's stability and distinctness of the dimensions, we uncovered four aging dimensions that represent the following bodily functions: 1) kidney, 2) thyroid, 3) white blood cells, and 4) liver and heart. We then examined the relationship between rates of aging on morbidity and health care expenditure. In general, faster agers along these dimensions are more likely to develop chronic diseases that are related to these bodily functions. They also had higher health care expenditures compared to the slower agers. K-means clustering of individuals based on rate of aging revealed that clusters with higher odds of developing morbidity had the highest cost across all types of health care services. Results suggest that cross-sectional laboratory data can be leveraged as an alternative methodology to understand age along the different dimensions. Moreover, rates of aging are differentially related to future costs, which can aid in the development of interventions to delay disease progression.
A Practical Guide to Multi-Objective Reinforcement Learning and Planning
Authors: Conor F. Hayes, Roxana Rădulescu, Eugenio Bargiacchi, Johan Källström, Matthew Macfarlane, Mathieu Reymond, Timothy Verstraeten, Luisa M. Zintgraf, Richard Dazeley, Fredrik Heintz, Enda Howley, Athirai A. Irissappane, Patrick Mannion, Ann Nowé, Gabriel Ramos, Marcello Restelli, Peter Vamplew, Diederik M. Roijers
Date: 2021-03-17 00:00:00
Tasks: Decision Making, Multi-Objective Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Real-world decision-making tasks are generally complex, requiring trade-offs between multiple, often conflicting, objectives. Despite this, the majority of research in reinforcement learning and decision-theoretic planning either assumes only a single objective, or that multiple objectives can be adequately handled via a simple linear combination. Such approaches may oversimplify the underlying problem and hence produce suboptimal results. This paper serves as a guide to the application of multi-objective methods to difficult problems, and is aimed at researchers who are already familiar with single-objective reinforcement learning and planning methods who wish to adopt a multi-objective perspective on their research, as well as practitioners who encounter multi-objective decision problems in practice. It identifies the factors that may influence the nature of the desired solution, and illustrates by example how these influence the design of multi-objective decision-making systems for complex problems.
Endangered Languages are not Low-Resourced!
Authors: Mika Hämäläinen
Date: 2021-03-17 00:00:00

The term low-resourced has been tossed around in the field of natural language processing to a degree that almost any language that is not English can be called "low-resourced"; sometimes even just for the sake of making a mundane or mediocre paper appear more interesting and insightful. In a field where English is a synonym for language and low-resourced is a synonym for anything not English, calling endangered languages low-resourced is a bit of an overstatement. In this paper, I inspect the relation of the endangered with the low-resourced from my own experiences.
Hierarchical Random Walker Segmentation for Large Volumetric Biomedical Images
Authors: Dominik Drees, Florian Eilers, Xiaoyi Jiang
Date: 2021-03-17 00:00:00
Tasks: Image Segmentation, Interactive Segmentation, Segmentation, Semantic Segmentation

The random walker method for image segmentation is a popular tool for semi-automatic image segmentation, especially in the biomedical field. However, its linear asymptotic run time and memory requirements make application to 3D datasets of increasing sizes impractical. We propose a hierarchical framework that, to the best of our knowledge, is the first attempt to overcome these restrictions for the random walker algorithm and achieves sublinear run time and constant memory complexity. The goal of this framework is -- rather than improving the segmentation quality compared to the baseline method -- to make interactive segmentation on out-of-core datasets possible. The method is evaluated quantitavely on synthetic data and the CT-ORG dataset where the expected improvements in algorithm run time while maintaining high segmentation quality are confirmed. The incremental (i.e., interaction update) run time is demonstrated to be in seconds on a standard PC even for volumes of hundreds of gigabytes in size. In a small case study the applicability to large real world from current biomedical research is demonstrated. An implementation of the presented method is publicly available in version 5.2 of the widely used volume rendering and processing software Voreen (https://www.uni-muenster.de/Voreen/).
Big Plastic Masses Detection using Sentinel 2 Images
Authors: Fernando Martin-Rodriguez
Date: 2021-03-17 00:00:00
Tasks: Earth Observation

This communication describes a preliminary research on detection of big masses of plastic (marine litter) on the oceans and seas using EO (Earth Observation) satellite systems. Free images from the Sentinel 2 (Copernicus Project) platform are used. To develop a plastic recognizer, we start with an image where we can find a big accumulation of "nonfloating" plastic: Almer\'ia greenhouses. We made a test using remote sensing differential indexes, but we got much better results using all available wavelengths (thirteen frequency bands) and applying Neural Networks to that feature vector.
Multi-channel Deep Supervision for Crowd Counting
Authors: Bo Wei, Mulin Chen, Qi Wang, Xuelong Li
Date: 2021-03-17 00:00:00
Tasks: Crowd Counting, Decoder

Crowd counting is a task worth exploring in modern society because of its wide applications such as public safety and video monitoring. Many CNN-based approaches have been proposed to improve the accuracy of estimation, but there are some inherent issues affect the performance, such as overfitting and details lost caused by pooling layers. To tackle these problems, in this paper, we propose an effective network called MDSNet, which introduces a novel supervision framework called Multi-channel Deep Supervision (MDS). The MDS conducts channel-wise supervision on the decoder of the estimation model to help generate the density maps. To obtain the accurate supervision information of different channels, the MDSNet employs an auxiliary network called SupervisionNet (SN) to generate abundant supervision maps based on existing groundtruth. Besides the traditional density map supervision, we also use the SN to convert the dot annotations into continuous supervision information and conduct dot supervision in the MDSNet. Extensive experiments on several mainstream benchmarks show that the proposed MDSNet achieves competitive results and the MDS significantly improves the performance without changing the network structure.
ENCONTER: Entity Constrained Progressive Sequence Generation via Insertion-based Transformer
Authors: Lee-Hsun Hsieh, Yang-Yin Lee, Ee-Peng Lim
Date: 2021-03-17 00:00:00
Proceeding: EACL 2021 2
Tasks: Text Generation

Pretrained using large amount of data, autoregressive language models are able to generate high quality sequences. However, these models do not perform well under hard lexical constraints as they lack fine control of content generation process. Progressive insertion-based transformers can overcome the above limitation and efficiently generate a sequence in parallel given some input tokens as constraint. These transformers however may fail to support hard lexical constraints as their generation process is more likely to terminate prematurely. The paper analyses such early termination problems and proposes the Entity-constrained insertion transformer (ENCONTER), a new insertion transformer that addresses the above pitfall without compromising much generation efficiency. We introduce a new training strategy that considers predefined hard lexical constraints (e.g., entities to be included in the generated sequence). Our experiments show that ENCONTER outperforms other baseline models in several performance metrics rendering it more suitable in practical applications. Our code is available at https://github.com/LARC-CMU-SMU/Enconter