pwc-archive/papers-with-abstracts huggingface dataset:
Learning while Respecting Privacy and Robustness to Distributional Uncertainties and Adversarial Data
Authors: Alireza Sadeghi, Gang Wang, Meng Ma, Georgios B. Giannakis
Date: 2020-07-07 00:00:00
Tasks: Federated Learning

Data used to train machine learning models can be adversarial--maliciously constructed by adversaries to fool the model. Challenge also arises by privacy, confidentiality, or due to legal constraints when data are geographically gathered and stored across multiple learners, some of which may hold even an "anonymized" or unreliable dataset. In this context, the distributionally robust optimization framework is considered for training a parametric model, both in centralized and federated learning settings. The objective is to endow the trained model with robustness against adversarially manipulated input data, or, distributional uncertainties, such as mismatches between training and testing data distributions, or among datasets stored at different workers. To this aim, the data distribution is assumed unknown, and lies within a Wasserstein ball centered around the empirical data distribution. This robust learning task entails an infinite-dimensional optimization problem, which is challenging. Leveraging a strong duality result, a surrogate is obtained, for which three stochastic primal-dual algorithms are developed: i) stochastic proximal gradient descent with an $\epsilon$-accurate oracle, which invokes an oracle to solve the convex sub-problems; ii) stochastic proximal gradient descent-ascent, which approximates the solution of the convex sub-problems via a single gradient ascent step; and, iii) a distributionally robust federated learning algorithm, which solves the sub-problems locally at different workers where data are stored. Compared to the empirical risk minimization and federated learning methods, the proposed algorithms offer robustness with little computation overhead. Numerical tests using image datasets showcase the merits of the proposed algorithms under several existing adversarial attacks and distributional uncertainties.
Non-parametric Models for Non-negative Functions
Authors: Ulysse Marteau-Ferey, Francis Bach, Alessandro Rudi
Date: 2020-07-08 00:00:00
Proceeding: NeurIPS 2020 12
Tasks: Density Estimation, quantile regression, regression

Linear models have shown great effectiveness and flexibility in many fields such as machine learning, signal processing and statistics. They can represent rich spaces of functions while preserving the convexity of the optimization problems where they are used, and are simple to evaluate, differentiate and integrate. However, for modeling non-negative functions, which are crucial for unsupervised learning, density estimation, or non-parametric Bayesian methods, linear models are not applicable directly. Moreover, current state-of-the-art models like generalized linear models either lead to non-convex optimization problems, or cannot be easily integrated. In this paper we provide the first model for non-negative functions which benefits from the same good properties of linear models. In particular, we prove that it admits a representer theorem and provide an efficient dual formulation for convex problems. We study its representation power, showing that the resulting space of functions is strictly richer than that of generalized linear models. Finally we extend the model and the theoretical results to functions with outputs in convex cones. The paper is complemented by an experimental evaluation of the model showing its effectiveness in terms of formulation, algorithmic derivation and practical results on the problems of density estimation, regression with heteroscedastic errors, and multiple quantile regression.
Machine Learning Classification of Kuiper Belt Populations
Authors: Rachel A. Smullen, Kathryn Volk
Date: 2020-07-07 00:00:00
Tasks: BIG-bench Machine Learning, Classification, General Classification

In the outer solar system, the Kuiper Belt contains dynamical sub-populations sculpted by a combination of planet formation and migration and gravitational perturbations from the present-day giant planet configuration. The subdivision of observed Kuiper Belt objects (KBOs) into different dynamical classes is based on their current orbital evolution in numerical integrations of their orbits. Here we demonstrate that machine learning algorithms are a promising tool for reducing both the computational time and human effort required for this classification. Using a Gradient Boosting Classifier, a type of machine learning regression tree classifier trained on features derived from short numerical simulations, we sort observed KBOs into four broad, dynamically distinct populations - classical, resonant, detached, and scattering - with a >97 per cent accuracy for the testing set of 542 securely classified KBOs. Over 80 per cent of these objects have a $>3\sigma$ probability of class membership, indicating that the machine learning method is classifying based on the fundamental dynamical features of each population. We also demonstrate how, by using computational savings over traditional methods, we can quickly derive a distribution of class membership by examining an ensemble of object clones drawn from the observational errors. We find two major reasons for misclassification: inherent ambiguity in the orbit of the object - for instance, an object that is on the edge of resonance - and a lack of representative examples in the training set. This work provides a promising avenue to explore for fast and accurate classification of the thousands of new KBOs expected to be found by surveys in the coming decade.
Detection as Regression: Certified Object Detection by Median Smoothing
Authors: Ping-Yeh Chiang, Michael J. Curry, Ahmed Abdelkader, Aounon Kumar, John Dickerson, Tom Goldstein
Date: 2020-07-07 00:00:00
Tasks: Object, object-detection, Object Detection, regression

Despite the vulnerability of object detectors to adversarial attacks, very few defenses are known to date. While adversarial training can improve the empirical robustness of image classifiers, a direct extension to object detection is very expensive. This work is motivated by recent progress on certified classification by randomized smoothing. We start by presenting a reduction from object detection to a regression problem. Then, to enable certified regression, where standard mean smoothing fails, we propose median smoothing, which is of independent interest. We obtain the first model-agnostic, training-free, and certified defense for object detection against $\ell_2$-bounded attacks. The code for all experiments in the paper is available at http://github.com/Ping-C/CertifiedObjectDetection .
Trajectories, bifurcations and pseudotime in large clinical datasets: applications to myocardial infarction and diabetes data
Authors: Sergey E. Golovenkin, Jonathan Bac, Alexander Chervov, Evgeny M. Mirkes, Yuliya V. Orlova, Emmanuel Barillot, Alexander N. Gorban, Andrei Zinovyev
Date: 2020-07-07 00:00:00
Tasks: Clustering, Data Visualization, Dimensionality Reduction, feature selection, Missing Values, Prognosis

Large observational clinical datasets become increasingly available for mining associations between various disease traits and administered therapy. These datasets can be considered as representations of the landscape of all possible disease conditions, in which a concrete pathology develops through a number of stereotypical routes, characterized by `points of no return' and `final states' (such as lethal or recovery states). Extracting this information directly from the data remains challenging, especially in the case of synchronic (with a short-term follow up) observations. Here we suggest a semi-supervised methodology for the analysis of large clinical datasets, characterized by mixed data types and missing values, through modeling the geometrical data structure as a bouquet of bifurcating clinical trajectories. The methodology is based on application of elastic principal graphs which can address simultaneously the tasks of dimensionality reduction, data visualization, clustering, feature selection and quantifying the geodesic distances (pseudotime) in partially ordered sequences of observations. The methodology allows positioning a patient on a particular clinical trajectory (pathological scenario) and characterizing the degree of progression along it with a qualitative estimate of the uncertainty of the prognosis. Overall, our pseudo-time quantification-based approach gives a possibility to apply the methods developed for dynamical disease phenotyping and illness trajectory analysis (diachronic data analysis) to synchronic observational data. We developed a tool $ClinTrajan$ for clinical trajectory analysis implemented in Python programming language. We test the methodology in two large publicly available datasets: myocardial infarction complications and readmission of diabetic patients data.
Making Adversarial Examples More Transferable and Indistinguishable
Authors: Junhua Zou, Yexin Duan, Boyu Li, Wu Zhang, Yu Pan, Zhisong Pan
Date: 2020-07-08 00:00:00

Fast gradient sign attack series are popular methods that are used to generate adversarial examples. However, most of the approaches based on fast gradient sign attack series cannot balance the indistinguishability and transferability due to the limitations of the basic sign structure. To address this problem, we propose a method, called Adam Iterative Fast Gradient Tanh Method (AI-FGTM), to generate indistinguishable adversarial examples with high transferability. Besides, smaller kernels and dynamic step size are also applied to generate adversarial examples for further increasing the attack success rates. Extensive experiments on an ImageNet-compatible dataset show that our method generates more indistinguishable adversarial examples and achieves higher attack success rates without extra running time and resource. Our best transfer-based attack NI-TI-DI-AITM can fool six classic defense models with an average success rate of 89.3% and three advanced defense models with an average success rate of 82.7%, which are higher than the state-of-the-art gradient-based attacks. Additionally, our method can also reduce nearly 20% mean perturbation. We expect that our method will serve as a new baseline for generating adversarial examples with better transferability and indistinguishability.
SiENet: Siamese Expansion Network for Image Extrapolation
Authors: Xiaofeng Zhang, Feng Chen, Cailing Wang, Songsong Wu, Ming Tao, Guoping Jiang
Date: 2020-07-08 00:00:00
Tasks: Decoder, Image Inpainting, Image Outpainting

Different from image inpainting, image outpainting has relative less context in the image center to capture and more content at the image border to predict. Therefore, classical encoder-decoder pipeline of existing methods may not predict the outstretched unknown content perfectly. In this paper, a novel two-stage siamese adversarial model for image extrapolation, named Siamese Expansion Network (SiENet) is proposed. In two stages, a novel border sensitive convolution named adaptive filling convolution is designed for allowing encoder to predict the unknown content, alleviating the burden of decoder. Besides, to introduce prior knowledge to network and reinforce the inferring ability of encoder, siamese adversarial mechanism is designed to enable our network to model the distribution of covered long range feature for that of uncovered image feature. The results on four datasets has demonstrated that our method outperforms existing state-of-the-arts and could produce realistic results.
Enable an Open Software Defined Mobility Ecosystem through VEC-OF
Authors: Sanchu Han, Yong He, Yin Ding
Date: 2020-07-08 00:00:00
Tasks: Autonomous Driving

OEMs and new entrants can take the Mobility as a Service market (MaaS) as the entry point, upgrade its E/E (Electric and Electronic) architecture to be C/C (Computing and Communication) architecture, build one open software defined and data driven software platform for its production and service model, use efficient and collaborative ways of vehicles, roads, cloud and network to continuously improve core technologies such as autonomous driving, provide MaaS operators with an affordable and agile platform. In this paper we present one new framework, VEC-OF (Vehicle-Edge-Cloud Open Framework), which is a new data and AI centric vehicle software framework enabling a much safer, more efficient, connected and trusted MaaS through cooperative vehicle, infrastructure and cloud capabilities and intelligence
Remix: Rebalanced Mixup
Authors: Hsin-Ping Chou, Shih-Chieh Chang, Jia-Yu Pan, Wei Wei, Da-Cheng Juan
Date: 2020-07-08 00:00:00

Deep image classifiers often perform poorly when training data are heavily class-imbalanced. In this work, we propose a new regularization technique, Remix, that relaxes Mixup's formulation and enables the mixing factors of features and labels to be disentangled. Specifically, when mixing two samples, while features are mixed in the same fashion as Mixup, Remix assigns the label in favor of the minority class by providing a disproportionately higher weight to the minority class. By doing so, the classifier learns to push the decision boundaries towards the majority classes and balance the generalization error between majority and minority classes. We have studied the state-of-the art regularization techniques such as Mixup, Manifold Mixup and CutMix under class-imbalanced regime, and shown that the proposed Remix significantly outperforms these state-of-the-arts and several re-weighting and re-sampling techniques, on the imbalanced datasets constructed by CIFAR-10, CIFAR-100, and CINIC-10. We have also evaluated Remix on a real-world large-scale imbalanced dataset, iNaturalist 2018. The experimental results confirmed that Remix provides consistent and significant improvements over the previous methods.
Responsive Safety in Reinforcement Learning by PID Lagrangian Methods
Authors: Adam Stooke, Joshua Achiam, Pieter Abbeel
Date: 2020-07-08 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Safe Reinforcement Learning

Lagrangian methods are widely used algorithms for constrained optimization problems, but their learning dynamics exhibit oscillations and overshoot which, when applied to safe reinforcement learning, leads to constraint-violating behavior during agent training. We address this shortcoming by proposing a novel Lagrange multiplier update method that utilizes derivatives of the constraint function. We take a controls perspective, wherein the traditional Lagrange multiplier update behaves as \emph{integral} control; our terms introduce \emph{proportional} and \emph{derivative} control, achieving favorable learning dynamics through damping and predictive measures. We apply our PID Lagrangian methods in deep RL, setting a new state of the art in Safety Gym, a safe RL benchmark. Lastly, we introduce a new method to ease controller tuning by providing invariance to the relative numerical scales of reward and cost. Our extensive experiments demonstrate improved performance and hyperparameter robustness, while our algorithms remain nearly as simple to derive and implement as the traditional Lagrangian approach.
Distributed Training of Deep Learning Models: A Taxonomic Perspective
Authors: Matthias Langer, Zhen He, Wenny Rahayu, Yanbo Xue
Date: 2020-07-08 00:00:00
Tasks: Deep Learning, GPU

Distributed deep learning systems (DDLS) train deep neural network models by utilizing the distributed resources of a cluster. Developers of DDLS are required to make many decisions to process their particular workloads in their chosen environment efficiently. The advent of GPU-based deep learning, the ever-increasing size of datasets and deep neural network models, in combination with the bandwidth constraints that exist in cluster environments require developers of DDLS to be innovative in order to train high quality models quickly. Comparing DDLS side-by-side is difficult due to their extensive feature lists and architectural deviations. We aim to shine some light on the fundamental principles that are at work when training deep neural networks in a cluster of independent machines by analyzing the general properties associated with training deep learning models and how such workloads can be distributed in a cluster to achieve collaborative model training. Thereby we provide an overview of the different techniques that are used by contemporary DDLS and discuss their influence and implications on the training process. To conceptualize and compare DDLS, we group different techniques into categories, thus establishing a taxonomy of distributed deep learning systems.
MPC Protocol for G-module and its Application in Secure Compare and ReLU
Authors: Qizhi Zhang, Lichun Li, Shan Yin, Juanjuan Sun
Date: 2020-07-08 00:00:00
Tasks: Privacy Preserving, Privacy Preserving Deep Learning

Secure comparison and secure selection are two fundamental MPC (secure Multi-Party Computation) protocols. One important application of these protocols is the secure ReLU and DReLU computation in privacy preserving deep learning. In this paper, we introduce G-module, a mathematics tool, to re-design such protocols. In mathematics, given a group G, a G-module is an abelian group M on which G acts compatibly with the abelian group structure on M. We design three secure protocols for three G-module operations. i.e. "G-module action", "Cross G-module action" and "G-module recover". As far as we know, this is the first work on secure G-module operations. Based on them, we design secure comparison, selection, ReLU and DReLU protocols, which improve communication efficiency by 2X to 10X compared with state of arts. Our protocols are very computation efficient too. They do not require public key operations or any other expensive operations.
Dynamic social learning under graph constraints
Authors: Konstantin Avrachenkov, Vivek S. Borkar, Sharayu Moharir, Suhail M. Shah
Date: 2020-07-08 00:00:00

We introduce a model of graph-constrained dynamic choice with reinforcement modeled by positively $\alpha$-homogeneous rewards. We show that its empirical process, which can be written as a stochastic approximation recursion with Markov noise, has the same probability law as a certain vertex reinforced random walk. We use this equivalence to show that for $\alpha > 0$, the asymptotic outcome concentrates around the optimum in a certain limiting sense when `annealed' by letting $\alpha\uparrow\infty$ slowly.
Attacking Split Manufacturing from a Deep Learning Perspective
Authors: Haocheng Li, Satwik Patnaik, Abhrajit Sengupta, Hao-Yu Yang, Johann Knechtel, Bei Yu, Evangeline F. Y. Young, Ozgur Sinanoglu
Date: 2020-07-08 00:00:00
Tasks: Deep Learning

The notion of integrated circuit split manufacturing which delegates the front-end-of-line (FEOL) and back-end-of-line (BEOL) parts to different foundries, is to prevent overproduction, piracy of the intellectual property (IP), or targeted insertion of hardware Trojans by adversaries in the FEOL facility. In this work, we challenge the security promise of split manufacturing by formulating various layout-level placement and routing hints as vector- and image-based features. We construct a sophisticated deep neural network which can infer the missing BEOL connections with high accuracy. Compared with the publicly available network-flow attack [1], for the same set of ISCAS-85 benchmarks, we achieve 1.21X accuracy when splitting on M1 and 1.12X accuracy when splitting on M3 with less than 1% running time.
Simultaneous Estimation of X-ray Back-Scatter and Forward-Scatter using Multi-Task Learning
Authors: Philipp Roser, Xia Zhong, Annette Birkhold, Alexander Preuhs, Christopher Syben, Elisabeth Hoppe, Norbert Strobel, Markus Kowarschik, Rebecca Fahrig, Andreas Maier
Date: 2020-07-08 00:00:00
Tasks: Blocking, Multi-Task Learning

Scattered radiation is a major concern impacting X-ray image-guided procedures in two ways. First, back-scatter significantly contributes to patient (skin) dose during complicated interventions. Second, forward-scattered radiation reduces contrast in projection images and introduces artifacts in 3-D reconstructions. While conventionally employed anti-scatter grids improve image quality by blocking X-rays, the additional attenuation due to the anti-scatter grid at the detector needs to be compensated for by a higher patient entrance dose. This also increases the room dose affecting the staff caring for the patient. For skin dose quantification, back-scatter is usually accounted for by applying pre-determined scalar back-scatter factors or linear point spread functions to a primary kerma forward projection onto a patient surface point. However, as patients come in different shapes, the generalization of conventional methods is limited. Here, we propose a novel approach combining conventional techniques with learning-based methods to simultaneously estimate the forward-scatter reaching the detector as well as the back-scatter affecting the patient skin dose. Knowing the forward-scatter, we can correct X-ray projections, while a good estimate of the back-scatter component facilitates an improved skin dose assessment. To simultaneously estimate forward-scatter as well as back-scatter, we propose a multi-task approach for joint back- and forward-scatter estimation by combining X-ray physics with neural networks. We show that, in theory, highly accurate scatter estimation in both cases is possible. In addition, we identify research directions for our multi-task framework and learning-based scatter estimation in general.
Machine learning and data analytics for the IoT
Authors: Erwin Adi, Adnan Anwar, Zubair Baig, Sherali Zeadally
Date: 2020-06-30 00:00:00
Tasks: BIG-bench Machine Learning

The Internet of Things (IoT) applications have grown in exorbitant numbers, generating a large amount of data required for intelligent data processing. However, the varying IoT infrastructures (i.e., cloud, edge, fog) and the limitations of the IoT application layer protocols in transmitting/receiving messages become the barriers in creating intelligent IoT applications. These barriers prevent current intelligent IoT applications to adaptively learn from other IoT applications. In this paper, we critically review how IoT-generated data are processed for machine learning analysis and highlight the current challenges in furthering intelligent solutions in the IoT environment. Furthermore, we propose a framework to enable IoT applications to adaptively learn from other IoT applications and present a case study in how the framework can be applied to the real studies in the literature. Finally, we discuss the key factors that have an impact on future intelligent applications for the IoT.
A Survey of Real-Time Social-Based Traffic Detection
Authors: Hashim Abu-gellban
Date: 2020-07-07 00:00:00
Tasks: BIG-bench Machine Learning, Event Detection, Survey

Online traffic news web sites do not always announce traffic events in areas in real-time. There is a capability to employ text mining and machine learning techniques on the twitter stream to perform event detection, in order to develop a real-time traffic detection system. In this present survey paper, we will deliberate the current state-of-art techniques in detecting traffic events in real-time focusing on five papers [1, 2, 3, 4, 5]. Lastly, applying text mining techniques and SVM classifiers in paper [2] gave the best results (i.e. 95.75% accuracy and 95.8% F1-score).