pwc-archive/papers-with-abstracts huggingface dataset:
Population Density and Spreading of COVID-19 in England and Wales
Authors: Jack Sutton, Golnaz Shahtahmassebi, Haroldo V. Ribeiro, Quentin S. Hanley
Date: 2021-02-25 00:00:00

We investigated daily COVID-19 cases and deaths in the 337 lower tier local authority regions in England and Wales to better understand how the disease propagated over a 15-month period. Population density scaling models revealed residual variance and skewness to be sensitive indicators of the dynamics of propagation. Lockdowns and schools reopening triggered increased variance indicative of outbreaks with local impact and country scale heterogeneity. University reopening and December holidays triggered reduced variance indicative of country scale homogenisation which reached a minimum in mid-January 2021. Homogeneous propagation was associated with better correspondence with normally distributed residuals while heterogeneous propagation was more consistent with skewed models. Skewness varied from strongly negative to strongly positive revealing an unappreciated feature of community propagation. Hot spots and super-spreading events are well understood descriptors of regional disease dynamics that would be expected to be associated with positively skewed distributions. Positively skewed behaviour was observed; however, negative skewness indicative of cold-spots and super-isolation dominated for approximately 8 months during the period of study. In contrast, death metrics showed near constant behaviour in scaling, variance, and skewness metrics over the full period with rural regions preferentially affected, an observation consistent with regional age demographics in England and Wales. Regional positions relative to density scaling laws were remarkably persistent after the first 5-9 days of the available data set. The determinants of this persistent behaviour probably precede the pandemic and remain unchanged.
Twin actor twin delayed deep deterministic policy gradient (TATD3) learning for batch process control
Authors: Tanuja Joshi, Shikhar Makker, Hariprasad Kodamana, Harikumar Kandath
Date: 2021-02-25 00:00:00
Tasks: continuous-control, Continuous Control, Reinforcement Learning (RL)

Control of batch processes is a difficult task due to their complex nonlinear dynamics and unsteady-state operating conditions within batch and batch-to-batch. It is expected that some of these challenges can be addressed by developing control strategies that directly interact with the process and learning from experiences. Recent studies in the literature have indicated the advantage of having an ensemble of actors in actor-critic Reinforcement Learning (RL) frameworks for improving the policy. The present study proposes an actor-critic RL algorithm, namely, twin actor twin delayed deep deterministic policy gradient (TATD3), by incorporating twin actor networks in the existing twin-delayed deep deterministic policy gradient (TD3) algorithm for the continuous control. In addition, two types of novel reward functions are also proposed for TATD3 controller. We showcase the efficacy of the TATD3 based controller for various batch process examples by comparing it with some of the existing RL algorithms presented in the literature.
Learning for Unconstrained Space-Time Video Super-Resolution
Authors: Zhihao Shi, Xiaohong Liu, Chengqi Li, Linhui Dai, Jun Chen, Timothy N. Davidson, Jiying Zhao
Date: 2021-02-25 00:00:00
Tasks: Optical Flow Estimation, Space-time Video Super-resolution, Super-Resolution, Video Enhancement, Video Super-Resolution

Recent years have seen considerable research activities devoted to video enhancement that simultaneously increases temporal frame rate and spatial resolution. However, the existing methods either fail to explore the intrinsic relationship between temporal and spatial information or lack flexibility in the choice of final temporal/spatial resolution. In this work, we propose an unconstrained space-time video super-resolution network, which can effectively exploit space-time correlation to boost performance. Moreover, it has complete freedom in adjusting the temporal frame rate and spatial resolution through the use of the optical flow technique and a generalized pixelshuffle operation. Our extensive experiments demonstrate that the proposed method not only outperforms the state-of-the-art, but also requires far fewer parameters and less running time.
t-SNE, Forceful Colorings and Mean Field Limits
Authors: Yulan Zhang, Stefan Steinerberger
Date: 2021-02-25 00:00:00
Tasks: Dimensionality Reduction

t-SNE is one of the most commonly used force-based nonlinear dimensionality reduction methods. This paper has two contributions: the first is forceful colorings, an idea that is also applicable to other force-based methods (UMAP, ForceAtlas2,...). In every equilibrium, the attractive and repulsive forces acting on a particle cancel out: however, both the size and the direction of the attractive (or repulsive) forces acting on a particle are related to its properties: the force vector can serve as an additional feature. Secondly, we analyze the case of t-SNE acting on a single homogeneous cluster (modeled by affinities coming from the adjacency matrix of a random k-regular graph); we derive a mean-field model that leads to interesting questions in classical calculus of variations. The model predicts that, in the limit, the t-SNE embedding of a single perfectly homogeneous cluster is not a point but a thin annulus of diameter $\sim k^{-1/4} n^{-1/4}$. This is supported by numerical results. The mean field ansatz extends to other force-based dimensionality reduction methods.
Imitation Learning with Human Eye Gaze via Multi-Objective Prediction
Authors: Ravi Kumar Thakur, MD-Nazmus Samin Sunbeam, Vinicius G. Goecks, Ellen Novoseller, Ritwik Bera, Vernon J. Lawhern, Gregory M. Gremillion, John Valasek, Nicholas R. Waytowich
Date: 2021-02-25 00:00:00
Tasks: Continuous Control, Imitation Learning, Navigate, Robot Manipulation, Self-Driving Cars, Visual Navigation

Approaches for teaching learning agents via human demonstrations have been widely studied and successfully applied to multiple domains. However, the majority of imitation learning work utilizes only behavioral information from the demonstrator, i.e. which actions were taken, and ignores other useful information. In particular, eye gaze information can give valuable insight towards where the demonstrator is allocating visual attention, and holds the potential to improve agent performance and generalization. In this work, we propose Gaze Regularized Imitation Learning (GRIL), a novel context-aware, imitation learning architecture that learns concurrently from both human demonstrations and eye gaze to solve tasks where visual attention provides important context. We apply GRIL to a visual navigation task, in which an unmanned quadrotor is trained to search for and navigate to a target vehicle in a photorealistic simulated environment. We show that GRIL outperforms several state-of-the-art gaze-based imitation learning algorithms, simultaneously learns to predict human visual attention, and generalizes to scenarios not present in the training data. Supplemental videos and code can be found at https://sites.google.com/view/gaze-regularized-il/.
Towards Unbiased and Accurate Deferral to Multiple Experts
Authors: Vijay Keswani, Matthew Lease, Krishnaram Kenthapadi
Date: 2021-02-25 00:00:00
Tasks: BIG-bench Machine Learning, Fairness

Machine learning models are often implemented in cohort with humans in the pipeline, with the model having an option to defer to a domain expert in cases where it has low confidence in its inference. Our goal is to design mechanisms for ensuring accuracy and fairness in such prediction systems that combine machine learning model inferences and domain expert predictions. Prior work on "deferral systems" in classification settings has focused on the setting of a pipeline with a single expert and aimed to accommodate the inaccuracies and biases of this expert to simultaneously learn an inference model and a deferral system. Our work extends this framework to settings where multiple experts are available, with each expert having their own domain of expertise and biases. We propose a framework that simultaneously learns a classifier and a deferral system, with the deferral system choosing to defer to one or more human experts in cases of input where the classifier has low confidence. We test our framework on a synthetic dataset and a content moderation dataset with biased synthetic experts, and show that it significantly improves the accuracy and fairness of the final predictions, compared to the baselines. We also collect crowdsourced labels for the content moderation task to construct a real-world dataset for the evaluation of hybrid machine-human frameworks and show that our proposed learning framework outperforms baselines on this real-world dataset as well.
An introduction to distributed training of deep neural networks for segmentation tasks with large seismic datasets
Authors: Claire Birnie, Haithem Jarraya, Fredrik Hansteen
Date: 2021-02-25 00:00:00

Deep learning applications are drastically progressing in seismic processing and interpretation tasks. However, the majority of approaches subsample data volumes and restrict model sizes to minimise computational requirements. Subsampling the data risks losing vital spatio-temporal information which could aid training whilst restricting model sizes can impact model performance, or in some extreme cases, renders more complicated tasks such as segmentation impossible. This paper illustrates how to tackle the two main issues of training of large neural networks: memory limitations and impracticably large training times. Typically, training data is preloaded into memory prior to training, a particular challenge for seismic applications where data is typically four times larger than that used for standard image processing tasks (float32 vs. uint8). Using a microseismic use case, we illustrate how over 750GB of data can be used to train a model by using a data generator approach which only stores in memory the data required for that training batch. Furthermore, efficient training over large models is illustrated through the training of a 7-layer UNet with input data dimensions of 4096X4096. Through a batch-splitting distributed training approach, training times are reduced by a factor of four. The combination of data generators and distributed training removes any necessity of data 1 subsampling or restriction of neural network sizes, offering the opportunity of utilisation of larger networks, higher-resolution input data or moving from 2D to 3D problem spaces.
Maximizing Cosine Similarity Between Spatial Features for Unsupervised Domain Adaptation in Semantic Segmentation
Authors: Inseop Chung, Daesik Kim, Nojun Kwak
Date: 2021-02-25 00:00:00
Tasks: Domain Adaptation, Segmentation, Semantic Segmentation, Unsupervised Domain Adaptation

We propose a novel method that tackles the problem of unsupervised domain adaptation for semantic segmentation by maximizing the cosine similarity between the source and the target domain at the feature level. A segmentation network mainly consists of two parts, a feature extractor and a classification head. We expect that if we can make the two domains have small domain gap at the feature level, they would also have small domain discrepancy at the classification head. Our method computes a cosine similarity matrix between the source feature map and the target feature map, then we maximize the elements exceeding a threshold to guide the target features to have high similarity with the most similar source feature. Moreover, we use a class-wise source feature dictionary which stores the latest features of the source domain to prevent the unmatching problem when computing the cosine similarity matrix and be able to compare a target feature with various source features from various images. Through extensive experiments, we verify that our method gains performance on two unsupervised domain adaptation tasks (GTA5$\to$ Cityscaspes and SYNTHIA$\to$ Cityscapes).
Data-Driven Incident Detection in Power Distribution Systems
Authors: Nayara Aguiar, Vijay Gupta, Rodrigo D. Trevizan, Babu R. Chalamala, Raymond H. Byrne
Date: 2021-02-25 00:00:00
Tasks: Data Augmentation

In a power distribution network with energy storage systems (ESS) and advanced controls, traditional monitoring and protection schemes are not well suited for detecting anomalies such as malfunction of controllable devices. In this work, we propose a data-driven technique for the detection of incidents relevant to the operation of ESS in distribution grids. This approach leverages the causal relationship observed among sensor data streams, and does not require prior knowledge of the system model or parameters. Our methodology includes a data augmentation step which allows for the detection of incidents even when sensing is scarce. The effectiveness of our technique is illustrated through case studies which consider active power dispatch and reactive power control of ESS.
Metal-Oxide Sensor Array for Selective Gas Detection in Mixtures
Authors: Noureddine Tayebi, Varvara Kollia, Pradyumna S. Singh
Date: 2021-02-25 00:00:00
Tasks: BIG-bench Machine Learning

We present a monolithic, microfabricated, metal-oxide semiconductor (MOS) sensor array in conjunction with a machine learning algorithm to determine unique fingerprints of individual gases within homogenous mixtures. The array comprises four different metal oxides, and is engineered for independent temperature control and readout from each individual pixel in a multiplexed fashion. The sensor pixels are designed on a very thin membrane to minimize heat dissipation, thereby significantly lowering the overall power consumption ($<$30 $\mu$W average power). The high dimensional data obtained by running the pixels at different temperatures, is used to train our machine learning algorithm with an average accuracy $\sim$ 88$\%$ for high resolution detection and estimation of concentration of individual constituents in a homogenous mixture. While the response of MOS sensors to various gases has been demonstrated, very few studies have investigated the response of these sensors to homogeneous mixtures of gases comprising several gases. We demonstrate this principle for a binary homogeneous mixture of ozone and carbon monoxide, both of which are criteria pollutant gases. Our findings indicate that a multiplicity of MOS elements together with the ability to vary and measure at various temperatures are essential in predicting concentration of individual gases within mixtures, thereby overcoming a key limitation of MOS sensors - poor selectivity. The small form-factor and microfabrication approach of our sensor array also lends itself to CMOS integration paving the way for a platform for wearable and portable applications.
A Primer on Contrastive Pretraining in Language Processing: Methods, Lessons Learned and Perspectives
Authors: Nils Rethmeier, Isabelle Augenstein
Date: 2021-02-25 00:00:00
Tasks: Contrastive Learning, Language Modeling, Language Modelling, Masked Language Modeling, Metric Learning, Sentence, Zero-Shot Learning

Modern natural language processing (NLP) methods employ self-supervised pretraining objectives such as masked language modeling to boost the performance of various application tasks. These pretraining methods are frequently extended with recurrence, adversarial or linguistic property masking, and more recently with contrastive learning objectives. Contrastive self-supervised training objectives enabled recent successes in image representation pretraining by learning to contrast input-input pairs of augmented images as either similar or dissimilar. However, in NLP, automated creation of text input augmentations is still very challenging because a single token can invert the meaning of a sentence. For this reason, some contrastive NLP pretraining methods contrast over input-label pairs, rather than over input-input pairs, using methods from Metric Learning and Energy Based Models. In this survey, we summarize recent self-supervised and supervised contrastive NLP pretraining methods and describe where they are used to improve language modeling, few or zero-shot learning, pretraining data-efficiency and specific NLP end-tasks. We introduce key contrastive learning concepts with lessons learned from prior research and structure works by applications and cross-field relations. Finally, we point to open challenges and future directions for contrastive NLP to encourage bringing contrastive NLP pretraining closer to recent successes in image representation pretraining.
Non-invasive Cognitive-level Human Interfacing for the Robotic Restoration of Reaching & Grasping
Authors: Ali Shafti, A. Aldo Faisal
Date: 2021-02-25 00:00:00

Assistive and Wearable Robotics have the potential to support humans with different types of motor impairments to become independent and fulfil their activities of daily living successfully. The success of these robot systems, however, relies on the ability to meaningfully decode human action intentions and carry them out appropriately. Neural interfaces have been explored for use in such system with several successes, however, they tend to be invasive and require training periods in the order of months. We present a robotic system for human augmentation, capable of actuating the user's arm and fingers for them, effectively restoring the capability of reaching, grasping and manipulating objects; controlled solely through the user's eye movements. We combine wearable eye tracking, the visual context of the environment and the structural grammar of human actions to create a cognitive-level assistive robotic setup that enables the users in fulfilling activities of daily living, while conserving interpretability, and the agency of the user. The interface is worn, calibrated and ready to use within 5 minutes. Users learn to control and make successful use of the system with an additional 5 minutes of interaction. The system is tested with 5 healthy participants, showing an average success rate of $96.6\%$ on first attempt across 6 tasks.
Are pre-trained text representations useful for multilingual and multi-dimensional language proficiency modeling?
Authors: Taraka Rama, Sowmya Vajjala
Date: 2021-02-25 00:00:00

Development of language proficiency models for non-native learners has been an active area of interest in NLP research for the past few years. Although language proficiency is multidimensional in nature, existing research typically considers a single "overall proficiency" while building models. Further, existing approaches also considers only one language at a time. This paper describes our experiments and observations about the role of pre-trained and fine-tuned multilingual embeddings in performing multi-dimensional, multilingual language proficiency classification. We report experiments with three languages -- German, Italian, and Czech -- and model seven dimensions of proficiency ranging from vocabulary control to sociolinguistic appropriateness. Our results indicate that while fine-tuned embeddings are useful for multilingual proficiency modeling, none of the features achieve consistently best performance for all dimensions of language proficiency. All code, data and related supplementary material can be found at: https://github.com/nishkalavallabhi/MultidimCEFRScoring.
A statistical framework for efficient out of distribution detection in deep neural networks
Authors: Matan Haroush, Tzviel Frostig, Ruth Heller, Daniel Soudry
Date: 2021-02-25 00:00:00
Proceeding: ICLR 2022 4
Tasks: Autonomous Vehicles, Out-of-Distribution Detection, Out of Distribution (OOD) Detection

Background. Commonly, Deep Neural Networks (DNNs) generalize well on samples drawn from a distribution similar to that of the training set. However, DNNs' predictions are brittle and unreliable when the test samples are drawn from a dissimilar distribution. This is a major concern for deployment in real-world applications, where such behavior may come at a considerable cost, such as industrial production lines, autonomous vehicles, or healthcare applications. Contributions. We frame Out Of Distribution (OOD) detection in DNNs as a statistical hypothesis testing problem. Tests generated within our proposed framework combine evidence from the entire network. Unlike previous OOD detection heuristics, this framework returns a $p$-value for each test sample. It is guaranteed to maintain the Type I Error (T1E - incorrectly predicting OOD for an actual in-distribution sample) for test data. Moreover, this allows to combine several detectors while maintaining the T1E. Building on this framework, we suggest a novel OOD procedure based on low-order statistics. Our method achieves comparable or better results than state-of-the-art methods on well-accepted OOD benchmarks, without retraining the network parameters or assuming prior knowledge on the test distribution -- and at a fraction of the computational cost.
Event-Driven Receding Horizon Control of Energy-Aware Dynamic Agents For Distributed Persistent Monitoring
Authors: Shirantha Welikala, Christos G. Cassandras
Date: 2021-02-25 00:00:00

This paper addresses the persistent monitoring problem defined on a network where a set of nodes (targets) needs to be monitored by a team of dynamic energy-aware agents. The objective is to control the agents' motion to jointly optimize the overall agent energy consumption and a measure of overall node state uncertainty, evaluated over a finite period of interest. To achieve these objectives, we extend an established event-driven Receding Horizon Control (RHC) solution by adding an optimal controller to account for agent motion dynamics and associated energy consumption. The resulting RHC solution is computationally efficient, distributed and on-line. Finally, numerical results are provided highlighting improvements compared to an existing RHC solution that uses energy-agnostic first-order agents.
On regret bounds for continual single-index learning
Authors: The Tien Mai
Date: 2021-02-25 00:00:00
Tasks: Continual Learning

In this paper, we generalize the problem of single-index model to the context of continual learning in which a learner is challenged with a sequence of tasks one by one and the dataset of each task is revealed in an online fashion. We propose a randomized strategy that is able to learn a common single-index (meta-parameter) for all tasks and a specific link function for each task. The common single-index allows to transfer the information gained from the previous tasks to a new one. We provide a rigorous theoretical analysis of our proposed strategy by proving some regret bounds under different assumption on the loss function.
Stein Variational Gradient Descent: many-particle and long-time asymptotics
Authors: Nikolas Nüsken, D. R. Michiel Renger
Date: 2021-02-25 00:00:00
Tasks: Bayesian Inference, Variational Inference

Stein variational gradient descent (SVGD) refers to a class of methods for Bayesian inference based on interacting particle systems. In this paper, we consider the originally proposed deterministic dynamics as well as a stochastic variant, each of which represent one of the two main paradigms in Bayesian computational statistics: variational inference and Markov chain Monte Carlo. As it turns out, these are tightly linked through a correspondence between gradient flow structures and large-deviation principles rooted in statistical physics. To expose this relationship, we develop the cotangent space construction for the Stein geometry, prove its basic properties, and determine the large-deviation functional governing the many-particle limit for the empirical measure. Moreover, we identify the Stein-Fisher information (or kernelised Stein discrepancy) as its leading order contribution in the long-time and many-particle regime in the sense of $\Gamma$-convergence, shedding some light on the finite-particle properties of SVGD. Finally, we establish a comparison principle between the Stein-Fisher information and RKHS-norms that might be of independent interest.