pwc-archive/papers-with-abstracts huggingface dataset:
Adversarial Turing Patterns from Cellular Automata
Authors: Nurislam Tursynbek, Ilya Vilkoviskiy, Maria Sindeeva, Ivan Oseledets
Date: 2020-11-18 00:00:00

State-of-the-art deep classifiers are intriguingly vulnerable to universal adversarial perturbations: single disturbances of small magnitude that lead to misclassification of most in-puts. This phenomena may potentially result in a serious security problem. Despite the extensive research in this area,there is a lack of theoretical understanding of the structure of these perturbations. In image domain, there is a certain visual similarity between patterns, that represent these perturbations, and classical Turing patterns, which appear as a solution of non-linear partial differential equations and are underlying concept of many processes in nature. In this paper,we provide a theoretical bridge between these two different theories, by mapping a simplified algorithm for crafting universal perturbations to (inhomogeneous) cellular automata,the latter is known to generate Turing patterns. Furthermore,we propose to use Turing patterns, generated by cellular automata, as universal perturbations, and experimentally show that they significantly degrade the performance of deep learning models. We found this method to be a fast and efficient way to create a data-agnostic quasi-imperceptible perturbation in the black-box scenario. The source code is available at https://github.com/NurislamT/advTuring.
Diverse Plausible Shape Completions from Ambiguous Depth Images
Authors: Brad Saund, Dmitry Berenson
Date: 2020-11-18 00:00:00
Tasks: Decoder, Diversity

We propose PSSNet, a network architecture for generating diverse plausible 3D reconstructions from a single 2.5D depth image. Existing methods tend to produce only small variations on a single shape, even when multiple shapes are consistent with an observation. To obtain diversity we alter a Variational Auto Encoder by providing a learned shape bounding box feature as side information during training. Since these features are known during training, we are able to add a supervised loss to the encoder and noiseless values to the decoder. To evaluate, we sample a set of completions from a network, construct a set of plausible shape matches for each test observation, and compare using our plausible diversity metric defined over sets of shapes. We perform experiments using Shapenet mugs and partially-occluded YCB objects and find that our method performs comparably in datasets with little ambiguity, and outperforms existing methods when many shapes plausibly fit an observed depth image. We demonstrate one use for PSSNet on a physical robot when grasping objects in occlusion and clutter.
Plug-And-Play Learned Gaussian-mixture Approximate Message Passing
Authors: Osman Musa, Peter Jung, Giuseppe Caire
Date: 2020-11-18 00:00:00
Tasks: compressed sensing, Denoising

Deep unfolding showed to be a very successful approach for accelerating and tuning classical signal processing algorithms. In this paper, we propose learned Gaussian-mixture AMP (L-GM-AMP) - a plug-and-play compressed sensing (CS) recovery algorithm suitable for any i.i.d. source prior. Our algorithm builds upon Borgerding's learned AMP (LAMP), yet significantly improves it by adopting a universal denoising function within the algorithm. The robust and flexible denoiser is a byproduct of modelling source prior with a Gaussian-mixture (GM), which can well approximate continuous, discrete, as well as mixture distributions. Its parameters are learned using standard backpropagation algorithm. To demonstrate robustness of the proposed algorithm, we conduct Monte-Carlo (MC) simulations for both mixture and discrete distributions. Numerical evaluation shows that the L-GM-AMP algorithm achieves state-of-the-art performance without any knowledge of the source prior.
Introduction to Core-sets: an Updated Survey
Authors: Dan Feldman
Date: 2020-11-18 00:00:00
Tasks: Clustering, Data Summarization, Survey

In optimization or machine learning problems we are given a set of items, usually points in some metric space, and the goal is to minimize or maximize an objective function over some space of candidate solutions. For example, in clustering problems, the input is a set of points in some metric space, and a common goal is to compute a set of centers in some other space (points, lines) that will minimize the sum of distances to these points. In database queries, we may need to compute such a some for a specific query set of $k$ centers. However, traditional algorithms cannot handle modern systems that require parallel real-time computations of infinite distributed streams from sensors such as GPS, audio or video that arrive to a cloud, or networks of weaker devices such as smartphones or robots. Core-set is a "small data" summarization of the input "big data", where every possible query has approximately the same answer on both data sets. Generic techniques enable efficient coreset \changed{maintenance} of streaming, distributed and dynamic data. Traditional algorithms can then be applied on these coresets to maintain the approximated optimal solutions. The challenge is to design coresets with provable tradeoff between their size and approximation error. This survey summarizes such constructions in a retrospective way, that aims to unified and simplify the state-of-the-art.
Bio-plausible Unsupervised Delay Learning for Extracting Temporal Features in Spiking Neural Networks
Authors: Alireza Nadafian, Mohammad Ganjtabesh
Date: 2020-11-18 00:00:00
Tasks: Mathematical Proofs

The plasticity of the conduction delay between neurons plays a fundamental role in learning. However, the exact underlying mechanisms in the brain for this modulation is still an open problem. Understanding the precise adjustment of synaptic delays could help us in developing effective brain-inspired computational models in providing aligned insights with the experimental evidence. In this paper, we propose an unsupervised biologically plausible learning rule for adjusting the synaptic delays in spiking neural networks. Then, we provided some mathematical proofs to show that our learning rule gives a neuron the ability to learn repeating spatio-temporal patterns. Furthermore, the experimental results of applying an STDP-based spiking neural network equipped with our proposed delay learning rule on Random Dot Kinematogram indicate the efficacy of the proposed delay learning rule in extracting temporal features.
LAVA: Latent Action Spaces via Variational Auto-encoding for Dialogue Policy Optimization
Authors: Nurul Lubis, Christian Geishauser, Michael Heck, Hsien-Chin Lin, Marco Moresi, Carel van Niekerk, Milica Gašić
Date: 2020-11-18 00:00:00
Proceeding: COLING 2020 8
Tasks: Decision Making, Reinforcement Learning (RL), Sequential Decision Making, Task-Oriented Dialogue Systems

Reinforcement learning (RL) can enable task-oriented dialogue systems to steer the conversation towards successful task completion. In an end-to-end setting, a response can be constructed in a word-level sequential decision making process with the entire system vocabulary as action space. Policies trained in such a fashion do not require expert-defined action spaces, but they have to deal with large action spaces and long trajectories, making RL impractical. Using the latent space of a variational model as action space alleviates this problem. However, current approaches use an uninformed prior for training and optimize the latent distribution solely on the context. It is therefore unclear whether the latent representation truly encodes the characteristics of different actions. In this paper, we explore three ways of leveraging an auxiliary task to shape the latent variable distribution: via pre-training, to obtain an informed prior, and via multitask learning. We choose response auto-encoding as the auxiliary task, as this captures the generative factors of dialogue responses while requiring low computational cost and neither additional data nor labels. Our approach yields a more action-characterized latent representations which support end-to-end dialogue policy optimization and achieves state-of-the-art success rates. These results warrant a more wide-spread use of RL in end-to-end dialogue models.
Out-of-Task Training for Dialog State Tracking Models
Authors: Michael Heck, Carel van Niekerk, Nurul Lubis, Christian Geishauser, Hsien-Chin Lin, Marco Moresi, Milica Gašić
Date: 2020-11-18 00:00:00
Proceeding: COLING 2020 8
Tasks: dialog state tracking, Multi-Task Learning, Specificity, Transfer Learning

Dialog state tracking (DST) suffers from severe data sparsity. While many natural language processing (NLP) tasks benefit from transfer learning and multi-task learning, in dialog these methods are limited by the amount of available data and by the specificity of dialog applications. In this work, we successfully utilize non-dialog data from unrelated NLP tasks to train dialog state trackers. This opens the door to the abundance of unrelated NLP corpora to mitigate the data sparsity issue inherent to DST.
Cycle-to-Cycle Queue Length Estimation from Connected Vehicles with Filtering on Primary Parameters
Authors: Gurcan Comert, Negash Begashaw
Date: 2020-11-18 00:00:00

Estimation models from connected vehicles often assume low level parameters such as arrival rates and market penetration rates as known or estimate them in real-time. At low market penetration rates, such parameter estimators produce large errors making estimated queue lengths inefficient for control or operations applications. In order to improve accuracy of low level parameter estimations, this study investigates the impact of connected vehicles information filtering on queue length estimation models. Filters are used as multilevel real-time estimators. Accuracy is tested against known arrival rate and market penetration rate scenarios using microsimulations. To understand the effectiveness for short-term or for dynamic processes, arrival rates, and market penetration rates are changed every 15 minutes. The results show that with Kalman and Particle filters, parameter estimators are able to find the true values within 15 minutes and meet and surpass the accuracy of known parameter scenarios especially for low market penetration rates. In addition, using last known estimated queue lengths when no connected vehicle is present performs better than inputting average estimated values. Moreover, the study shows that both filtering algorithms are suitable for real-time applications that require less than 0.1 second computational time.
Attentional Separation-and-Aggregation Network for Self-supervised Depth-Pose Learning in Dynamic Scenes
Authors: Feng Gao, Jincheng Yu, Hao Shen, Yu Wang, Huazhong Yang
Date: 2020-11-18 00:00:00

Learning depth and ego-motion from unlabeled videos via self-supervision from epipolar projection can improve the robustness and accuracy of the 3D perception and localization of vision-based robots. However, the rigid projection computed by ego-motion cannot represent all scene points, such as points on moving objects, leading to false guidance in these regions. To address this problem, we propose an Attentional Separation-and-Aggregation Network (ASANet), which can learn to distinguish and extract the scene's static and dynamic characteristics via the attention mechanism. We further propose a novel MotionNet with an ASANet as the encoder, followed by two separate decoders, to estimate the camera's ego-motion and the scene's dynamic motion field. Then, we introduce an auto-selecting approach to detect the moving objects for dynamic-aware learning automatically. Empirical experiments demonstrate that our method can achieve the state-of-the-art performance on the KITTI benchmark.
Self-Gradient Networks
Authors: Hossein Aboutalebi, Mohammad Javad Shafiee Alexander Wong
Date: 2020-11-18 00:00:00
Tasks: Adversarial Defense

The incredible effectiveness of adversarial attacks on fooling deep neural networks poses a tremendous hurdle in the widespread adoption of deep learning in safety and security-critical domains. While adversarial defense mechanisms have been proposed since the discovery of the adversarial vulnerability issue of deep neural networks, there is a long path to fully understand and address this issue. In this study, we hypothesize that part of the reason for the incredible effectiveness of adversarial attacks is their ability to implicitly tap into and exploit the gradient flow of a deep neural network. This innate ability to exploit gradient flow makes defending against such attacks quite challenging. Motivated by this hypothesis we argue that if a deep neural network architecture can explicitly tap into its own gradient flow during the training, it can boost its defense capability significantly. Inspired by this fact, we introduce the concept of self-gradient networks, a novel deep neural network architecture designed to be more robust against adversarial perturbations. Gradient flow information is leveraged within self-gradient networks to achieve greater perturbation stability beyond what can be achieved in the standard training process. We conduct a theoretical analysis to gain better insights into the behaviour of the proposed self-gradient networks to illustrate the efficacy of leverage this additional gradient flow information. The proposed self-gradient network architecture enables much more efficient and effective adversarial training, leading to faster convergence towards an adversarially robust solution by at least 10X. Experimental results demonstrate the effectiveness of self-gradient networks when compared with state-of-the-art adversarial learning strategies, with 10% improvement on the CIFAR10 dataset under PGD and CW adversarial perturbations.
Neural network approximation and estimation of classifiers with classification boundary in a Barron class
Authors: Andrei Caragea, Philipp Petersen, Felix Voigtlaender
Date: 2020-11-18 00:00:00
Tasks: Binary Classification, General Classification

We prove bounds for the approximation and estimation of certain binary classification functions using ReLU neural networks. Our estimation bounds provide a priori performance guarantees for empirical risk minimization using networks of a suitable size, depending on the number of training samples available. The obtained approximation and estimation rates are independent of the dimension of the input, showing that the curse of dimensionality can be overcome in this setting; in fact, the input dimension only enters in the form of a polynomial factor. Regarding the regularity of the target classification function, we assume the interfaces between the different classes to be locally of Barron-type. We complement our results by studying the relations between various Barron-type spaces that have been proposed in the literature. These spaces differ substantially more from each other than the current literature suggests.
A Discussion on Practical Considerations with Sparse Regression Methodologies
Authors: Owais Sarwar, Benjamin Sauk, Nikolaos V. Sahinidis
Date: 2020-11-18 00:00:00
Tasks: regression

Sparse linear regression is a vast field and there are many different algorithms available to build models. Two new papers published in Statistical Science study the comparative performance of several sparse regression methodologies, including the lasso and subset selection. Comprehensive empirical analyses allow the researchers to demonstrate the relative merits of each estimator and provide guidance to practitioners. In this discussion, we summarize and compare the two studies and we examine points of agreement and divergence, aiming to provide clarity and value to users. The authors have started a highly constructive dialogue, our goal is to continue it.
A Knowledge Distillation Ensemble Framework for Predicting Short and Long-term Hospitalisation Outcomes from Electronic Health Records Data
Authors: Zina M Ibrahim, Daniel Bean, Thomas Searle, Honghan Wu, Anthony Shek, Zeljko Kraljevic, James Galloway, Sam Norton, James T Teo, Richard JB Dobson
Date: 2020-11-18 00:00:00
Tasks: Decision Making, ICU Admission, Knowledge Distillation, Management, Outlier Detection, outlier ensembles, Prediction, Prognosis, Time Series, Time Series Analysis

The ability to perform accurate prognosis of patients is crucial for proactive clinical decision making, informed resource management and personalised care. Existing outcome prediction models suffer from a low recall of infrequent positive outcomes. We present a highly-scalable and robust machine learning framework to automatically predict adversity represented by mortality and ICU admission from time-series vital signs and laboratory results obtained within the first 24 hours of hospital admission. The stacked platform comprises two components: a) an unsupervised LSTM Autoencoder that learns an optimal representation of the time-series, using it to differentiate the less frequent patterns which conclude with an adverse event from the majority patterns that do not, and b) a gradient boosting model, which relies on the constructed representation to refine prediction, incorporating static features of demographics, admission details and clinical summaries. The model is used to assess a patient's risk of adversity over time and provides visual justifications of its prediction based on the patient's static features and dynamic signals. Results of three case studies for predicting mortality and ICU admission show that the model outperforms all existing outcome prediction models, achieving PR-AUC of 0.891 (95$%$ CI: 0.878 - 0.969) in predicting mortality in ICU and general ward settings and 0.908 (95$%$ CI: 0.870-0.935) in predicting ICU admission.
FLaaS: Federated Learning as a Service
Authors: Nicolas Kourtellis, Kleomenis Katevas, Diego Perino
Date: 2020-11-18 00:00:00
Tasks: CPU, Federated Learning, Management, object-detection, Object Detection, Privacy Preserving

Federated Learning (FL) is emerging as a promising technology to build machine learning models in a decentralized, privacy-preserving fashion. Indeed, FL enables local training on user devices, avoiding user data to be transferred to centralized servers, and can be enhanced with differential privacy mechanisms. Although FL has been recently deployed in real systems, the possibility of collaborative modeling across different 3rd-party applications has not yet been explored. In this paper, we tackle this problem and present Federated Learning as a Service (FLaaS), a system enabling different scenarios of 3rd-party application collaborative model building and addressing the consequent challenges of permission and privacy management, usability, and hierarchical model training. FLaaS can be deployed in different operational environments. As a proof of concept, we implement it on a mobile phone setting and discuss practical implications of results on simulated and real devices with respect to on-device training CPU cost, memory footprint and power consumed per FL model round. Therefore, we demonstrate FLaaS's feasibility in building unique or joint FL models across applications for image object detection in a few hours, across 100 devices.
Asymmetric Private Set Intersection with Applications to Contact Tracing and Private Vertical Federated Machine Learning
Authors: Nick Angelou, Ayoub Benaissa, Bogdan Cebere, William Clark, Adam James Hall, Michael A. Hoeh, Daniel Liu, Pavlos Papadopoulos, Robin Roehm, Robert Sandmann, Phillipp Schoppmann, Tom Titcombe
Date: 2020-11-18 00:00:00
Tasks: BIG-bench Machine Learning, Privacy Preserving

We present a multi-language, cross-platform, open-source library for asymmetric private set intersection (PSI) and PSI-Cardinality (PSI-C). Our protocol combines traditional DDH-based PSI and PSI-C protocols with compression based on Bloom filters that helps reduce communication in the asymmetric setting. Currently, our library supports C++, C, Go, WebAssembly, JavaScript, Python, and Rust, and runs on both traditional hardware (x86) and browser targets. We further apply our library to two use cases: (i) a privacy-preserving contact tracing protocol that is compatible with existing approaches, but improves their privacy guarantees, and (ii) privacy-preserving machine learning on vertically partitioned data.
Deep Empirical Risk Minimization in finance: looking into the future
Authors: A. Max Reppen, H. Mete Soner
Date: 2020-11-18 00:00:00
Tasks: Synthetic Data Generation

Many modern computational approaches to classical problems in quantitative finance are formulated as empirical loss minimization (ERM), allowing direct applications of classical results from statistical machine learning. These methods, designed to directly construct the optimal feedback representation of hedging or investment decisions, are analyzed in this framework demonstrating their effectiveness as well as their susceptibility to generalization error. Use of classical techniques shows that over-training renders trained investment decisions to become anticipative, and proves overlearning for large hypothesis spaces. On the other hand, non-asymptotic estimates based on Rademacher complexity show the convergence for sufficiently large training sets. These results emphasize the importance of synthetic data generation and the appropriate calibration of complex models to market data. A numerically studied stylized example illustrates these possibilities, including the importance of problem dimension in the degree of overlearning, and the effectiveness of this approach.
A Tunnel Gaussian Process Model for Learning Interpretable Flight's Landing Parameters
Authors: Sim Kuan Goh, Narendra Pratap Singh, Zhi Jun Lim, Sameer Alam
Date: 2020-11-18 00:00:00

Approach and landing accidents have resulted in a significant number of hull losses worldwide. Technologies (e.g., instrument landing system) and procedures (e.g., stabilized approach criteria) have been developed to reduce the risks. In this paper, we propose a data-driven method to learn and interpret flight's approach and landing parameters to facilitate comprehensible and actionable insights into flight dynamics. Specifically, we develop two variants of tunnel Gaussian process (TGP) models to elucidate aircraft's approach and landing dynamics using advanced surface movement guidance and control system (A-SMGCS) data, which then indicates the stability of flight. TGP hybridizes the strengths of sparse variational Gaussian process and polar Gaussian process to learn from a large amount of data in cylindrical coordinates. We examine TGP qualitatively and quantitatively by synthesizing three complex trajectory datasets and compared TGP against existing methods on trajectory learning. Empirically, TGP demonstrates superior modeling performance. When applied to operational A-SMGCS data, TGP provides the generative probabilistic description of landing dynamics and interpretable tunnel views of approach and landing parameters. These probabilistic tunnel models can facilitate the analysis of procedure adherence and augment existing aircrew and air traffic controllers' displays during the approach and landing procedures, enabling necessary corrective actions.