pwc-archive/papers-with-abstracts huggingface dataset:
Stochastic sparse adversarial attacks
Authors: Manon Césaire, Lucas Schott, Hatem Hajri, Sylvain Lamprier, Patrick Gallinari
Date: 2020-11-24 00:00:00

This paper introduces stochastic sparse adversarial attacks (SSAA), standing as simple, fast and purely noise-based targeted and untargeted attacks of neural network classifiers (NNC). SSAA offer new examples of sparse (or $L_0$) attacks for which only few methods have been proposed previously. These attacks are devised by exploiting a small-time expansion idea widely used for Markov processes. Experiments on small and large datasets (CIFAR-10 and ImageNet) illustrate several advantages of SSAA in comparison with the-state-of-the-art methods. For instance, in the untargeted case, our method called Voting Folded Gaussian Attack (VFGA) scales efficiently to ImageNet and achieves a significantly lower $L_0$ score than SparseFool (up to $\frac{2}{5}$) while being faster. Moreover, VFGA achieves better $L_0$ scores on ImageNet than Sparse-RS when both attacks are fully successful on a large number of samples.
Wide-band butterfly network: stable and efficient inversion via multi-frequency neural networks
Authors: Matthew Li, Laurent Demanet, Leonardo Zepeda-Núñez
Date: 2020-11-24 00:00:00

We introduce an end-to-end deep learning architecture called the wide-band butterfly network (WideBNet) for approximating the inverse scattering map from wide-band scattering data. This architecture incorporates tools from computational harmonic analysis, such as the butterfly factorization, and traditional multi-scale methods, such as the Cooley-Tukey FFT algorithm, to drastically reduce the number of trainable parameters to match the inherent complexity of the problem. As a result WideBNet is efficient: it requires fewer training points than off-the-shelf architectures, and has stable training dynamics, thus it can rely on standard weight initialization strategies. The architecture automatically adapts to the dimensions of the data with only a few hyper-parameters that the user must specify. WideBNet is able to produce images that are competitive with optimization-based approaches, but at a fraction of the cost, and we also demonstrate numerically that it learns to super-resolve scatterers in the full aperture scattering setup.
Online Domain Adaptation for Continuous Cross-Subject Liver Viability Evaluation Based on Irregular Thermal Data
Authors: Sahand Hajifar, Hongyue Sun
Date: 2020-11-24 00:00:00
Tasks: Domain Adaptation, Online Domain Adaptation

Accurate evaluation of liver viability during its procurement is a challenging issue and has traditionally been addressed by taking invasive biopsy on liver. Recently, people have started to investigate on the non-invasive evaluation of liver viability during its procurement using the liver surface thermal images. However, existing works include the background noise in the thermal images and do not consider the cross-subject heterogeneity of livers, thus the viability evaluation accuracy can be affected. In this paper, we propose to use the irregular thermal data of the pure liver region, and the cross-subject liver evaluation information (i.e., the available viability label information in cross-subject livers), for the real-time evaluation of a new liver's viability. To achieve this objective, we extract features of irregular thermal data based on tools from graph signal processing (GSP), and propose an online domain adaptation (DA) and classification framework using the GSP features of cross-subject livers. A multiconvex block coordinate descent based algorithm is designed to jointly learn the domain-invariant features during online DA and learn the classifier. Our proposed framework is applied to the liver procurement data, and classifies the liver viability accurately.
Distribution Conditional Denoising: A Flexible Discriminative Image Denoiser
Authors: Anthony Kelly
Date: 2020-11-24 00:00:00
Tasks: Denoising, Multi-Task Learning

A flexible discriminative image denoiser is introduced in which multi-task learning methods are applied to a densoising FCN based on U-Net. The activations of the U-Net model are modified by affine transforms that are a learned function of conditioning inputs. The learning procedure for multiple noise types and levels involves applying a distribution of noise parameters during training to the conditioning inputs, with the same noise parameters applied to a noise generating layer at the input (similar to the approach taken in a denoising autoencoder). It is shown that this flexible denoising model achieves state of the art performance on images corrupted with Gaussian and Poisson noise. It has also been shown that this conditional training method can generalise a fixed noise level U-Net denoiser to a variety of noise levels.
Geom-SPIDER-EM: Faster Variance Reduced Stochastic Expectation Maximization for Nonconvex Finite-Sum Optimization
Authors: Gersende Fort, Eric Moulines, Hoi-To Wai
Date: 2020-11-24 00:00:00

The Expectation Maximization (EM) algorithm is a key reference for inference in latent variable models; unfortunately, its computational cost is prohibitive in the large scale learning setting. In this paper, we propose an extension of the Stochastic Path-Integrated Differential EstimatoR EM (SPIDER-EM) and derive complexity bounds for this novel algorithm, designed to solve smooth nonconvex finite-sum optimization problems. We show that it reaches the same state of the art complexity bounds as SPIDER-EM; and provide conditions for a linear rate of convergence. Numerical results support our findings.
A3D: Adaptive 3D Networks for Video Action Recognition
Authors: Sijie Zhu, Taojiannan Yang, Matias Mendieta, Chen Chen
Date: 2020-11-24 00:00:00
Tasks: Action Recognition, Temporal Action Localization

This paper presents A3D, an adaptive 3D network that can infer at a wide range of computational constraints with one-time training. Instead of training multiple models in a grid-search manner, it generates good configurations by trading off between network width and spatio-temporal resolution. Furthermore, the computation cost can be adapted after the model is deployed to meet variable constraints, for example, on edge devices. Even under the same computational constraints, the performance of our adaptive networks can be significantly boosted over the baseline counterparts by the mutual training along three dimensions. When a multiple pathway framework, e.g. SlowFast, is adopted, our adaptive method encourages a better trade-off between pathways than manual designs. Extensive experiments on the Kinetics dataset show the effectiveness of the proposed framework. The performance gain is also verified to transfer well between datasets and tasks. Code will be made available.
Reinforced optimal control
Authors: Christian Bayer, Denis Belomestny, Paul Hager, Paolo Pigato, John Schoenmakers, Vladimir Spokoiny
Date: 2020-11-24 00:00:00
Tasks: Math, regression

Least squares Monte Carlo methods are a popular numerical approximation method for solving stochastic control problems. Based on dynamic programming, their key feature is the approximation of the conditional expectation of future rewards by linear least squares regression. Hence, the choice of basis functions is crucial for the accuracy of the method. Earlier work by some of us [Belomestny, Schoenmakers, Spokoiny, Zharkynbay. Commun.~Math.~Sci., 18(1):109-121, 2020](arXiv:1808.02341) proposes to reinforce the basis functions in the case of optimal stopping problems by already computed value functions for later times, thereby considerably improving the accuracy with limited additional computational cost. We extend the reinforced regression method to a general class of stochastic control problems, while considerably improving the method's efficiency, as demonstrated by substantial numerical examples as well as theoretical analysis.
Invariant Representation Learning for Treatment Effect Estimation
Authors: Claudia Shi, Victor Veitch, David Blei
Date: 2020-11-24 00:00:00
Tasks: Causal Identification, Causal Inference, Representation Learning, valid

The defining challenge for causal inference from observational data is the presence of `confounders', covariates that affect both treatment assignment and the outcome. To address this challenge, practitioners collect and adjust for the covariates, hoping that they adequately correct for confounding. However, including every observed covariate in the adjustment runs the risk of including `bad controls', variables that induce bias when they are conditioned on. The problem is that we do not always know which variables in the covariate set are safe to adjust for and which are not. To address this problem, we develop Nearly Invariant Causal Estimation (NICE). NICE uses invariant risk minimization (IRM) [Arj19] to learn a representation of the covariates that, under some assumptions, strips out bad controls but preserves sufficient information to adjust for confounding. Adjusting for the learned representation, rather than the covariates themselves, avoids the induced bias and provides valid causal inferences. We evaluate NICE on both synthetic and semi-synthetic data. When the covariates contain unknown collider variables and other bad controls, NICE performs better than adjusting for all the covariates.
A Non-linear Function-on-Function Model for Regression with Time Series Data
Authors: Qiyao Wang, HaiYan Wang, Chetan Gupta, Aniruddha Rajendra Rao, Hamed Khorasgani
Date: 2020-11-24 00:00:00
Tasks: regression, Time Series, Time Series Analysis, Time Series Regression

In the last few decades, building regression models for non-scalar variables, including time series, text, image, and video, has attracted increasing interests of researchers from the data analytic community. In this paper, we focus on a multivariate time series regression problem. Specifically, we aim to learn mathematical mappings from multiple chronologically measured numerical variables within a certain time interval S to multiple numerical variables of interest over time interval T. Prior arts, including the multivariate regression model, the Seq2Seq model, and the functional linear models, suffer from several limitations. The first two types of models can only handle regularly observed time series. Besides, the conventional multivariate regression models tend to be biased and inefficient, as they are incapable of encoding the temporal dependencies among observations from the same time series. The sequential learning models explicitly use the same set of parameters along time, which has negative impacts on accuracy. The function-on-function linear model in functional data analysis (a branch of statistics) is insufficient to capture complex correlations among the considered time series and suffer from underfitting easily. In this paper, we propose a general functional mapping that embraces the function-on-function linear model as a special case. We then propose a non-linear function-on-function model using the fully connected neural network to learn the mapping from data, which addresses the aforementioned concerns in the existing approaches. For the proposed model, we describe in detail the corresponding numerical implementation procedures. The effectiveness of the proposed model is demonstrated through the application to two real-world problems.
Play Fair: Frame Attributions in Video Models
Authors: Will Price, Dima Damen
Date: 2020-11-24 00:00:00
Tasks: Action Recognition, Relational Reasoning

In this paper, we introduce an attribution method for explaining action recognition models. Such models fuse information from multiple frames within a video, through score aggregation or relational reasoning. We break down a model's class score into the sum of contributions from each frame, fairly. Our method adapts an axiomatic solution to fair reward distribution in cooperative games, known as the Shapley value, for elements in a variable-length sequence, which we call the Element Shapley Value (ESV). Critically, we propose a tractable approximation of ESV that scales linearly with the number of frames in the sequence. We employ ESV to explain two action recognition models (TRN and TSN) on the fine-grained dataset Something-Something. We offer detailed analysis of supporting/distracting frames, and the relationships of ESVs to the frame's position, class prediction, and sequence length. We compare ESV to naive baselines and two commonly used feature attribution methods: Grad-CAM and Integrated-Gradients.
C-Learning: Horizon-Aware Cumulative Accessibility Estimation
Authors: Panteha Naderian, Gabriel Loaiza-Ganem, Harry J. Braviner, Anthony L. Caterini, Jesse C. Cresswell, Tong Li, Animesh Garg
Date: 2020-11-24 00:00:00
Proceeding: ICLR 2021 1
Tasks: continuous-control, Continuous Control, Motion Planning

Multi-goal reaching is an important problem in reinforcement learning needed to achieve algorithmic generalization. Despite recent advances in this field, current algorithms suffer from three major challenges: high sample complexity, learning only a single way of reaching the goals, and difficulties in solving complex motion planning tasks. In order to address these limitations, we introduce the concept of cumulative accessibility functions, which measure the reachability of a goal from a given state within a specified horizon. We show that these functions obey a recurrence relation, which enables learning from offline interactions. We also prove that optimal cumulative accessibility functions are monotonic in the planning horizon. Additionally, our method can trade off speed and reliability in goal-reaching by suggesting multiple paths to a single goal depending on the provided horizon. We evaluate our approach on a set of multi-goal discrete and continuous control tasks. We show that our method outperforms state-of-the-art goal-reaching algorithms in success rate, sample complexity, and path optimality. Our code is available at https://github.com/layer6ai-labs/CAE, and additional visualizations can be found at https://sites.google.com/view/learning-cae/.
FireSRnet: Geoscience-Driven Super-Resolution of Future Fire Risk from Climate Change
Authors: Tristan Ballard, Gopal Erinjippurath
Date: 2020-11-24 00:00:00
Tasks: Super-Resolution

With fires becoming increasingly frequent and severe across the globe in recent years, understanding climate change's role in fire behavior is critical for quantifying current and future fire risk. However, global climate models typically simulate fire behavior at spatial scales too coarse for local risk assessments. Therefore, we propose a novel approach towards super-resolution (SR) enhancement of fire risk exposure maps that incorporates not only 2000 to 2020 monthly satellite observations of active fires but also local information on land cover and temperature. Inspired by SR architectures, we propose an efficient deep learning model trained for SR on fire risk exposure maps. We evaluate this model on resolution enhancement and find it outperforms standard image interpolation techniques at both 4x and 8x enhancement while having comparable performance at 2x enhancement. We then demonstrate the generalizability of this SR model over northern California and New South Wales, Australia. We conclude with a discussion and application of our proposed model to climate model simulations of fire risk in 2040 and 2100, illustrating the potential for SR enhancement of fire risk maps from the latest state-of-the-art climate models.
Hindsight Network Credit Assignment
Authors: Kenny Young
Date: 2020-11-24 00:00:00

We present Hindsight Network Credit Assignment (HNCA), a novel learning method for stochastic neural networks, which works by assigning credit to each neuron's stochastic output based on how it influences the output of its immediate children in the network. We prove that HNCA provides unbiased gradient estimates while reducing variance compared to the REINFORCE estimator. We also experimentally demonstrate the advantage of HNCA over REINFORCE in a contextual bandit version of MNIST. The computational complexity of HNCA is similar to that of backpropagation. We believe that HNCA can help stimulate new ways of thinking about credit assignment in stochastic compute graphs.
Improving Clinical Outcome Predictions Using Convolution over Medical Entities with Multimodal Learning
Authors: Batuhan Bardak, Mehmet Tan
Date: 2020-11-24 00:00:00
Tasks: Management, Time Series, Time Series Analysis

Early prediction of mortality and length of stay(LOS) of a patient is vital for saving a patient's life and management of hospital resources. Availability of electronic health records(EHR) makes a huge impact on the healthcare domain and there has seen several works on predicting clinical problems. However, many studies did not benefit from the clinical notes because of the sparse, and high dimensional nature. In this work, we extract medical entities from clinical notes and use them as additional features besides time-series features to improve our predictions. We propose a convolution based multimodal architecture, which not only learns effectively combining medical entities and time-series ICU signals of patients, but also allows us to compare the effect of different embedding techniques such as Word2vec, FastText on medical entities. In the experiments, our proposed method robustly outperforms all other baseline models including different multimodal architectures for all clinical tasks. The code for the proposed method is available at https://github.com/tanlab/ConvolutionMedicalNer.
The Application of Data Mining in the Production Processes
Authors: Hamza Saad
Date: 2020-11-24 00:00:00

Traditional statistical and measurements are unable to solve all industrial data in the right way and appropriate time. Open markets mean the customers are increased, and production must increase to provide all customer requirements. Nowadays, large data generated daily from different production processes and traditional statistical or limited measurements are not enough to handle all daily data. Improve production and quality need to analyze data and extract the important information about the process how to improve. Data mining applied successfully in the industrial processes and some algorithms such as mining association rules, and decision tree recorded high professional results in different industrial and production fields. The study applied seven algorithms to analyze production data and extract the best result and algorithm in the industry field. KNN, Tree, SVM, Random Forests, ANN, Na\"ive Bayes, and AdaBoost applied to classify data based on three attributes without neglect any variables whether this variable is numerical or categorical. The best results of accuracy and area under the curve (ROC) obtained from Decision tree and its ensemble algorithms (Random Forest and AdaBoost). Thus, a decision tree is an appropriate algorithm to handle manufacturing and production data especially this algorithm can handle numerical and categorical data.
Trust but Verify: Assigning Prediction Credibility by Counterfactual Constrained Learning
Authors: Luiz F. O. Chamon, Santiago Paternain, Alejandro Ribeiro
Date: 2020-11-24 00:00:00
Tasks: counterfactual, Graph Neural Network

Prediction credibility measures, in the form of confidence intervals or probability distributions, are fundamental in statistics and machine learning to characterize model robustness, detect out-of-distribution samples (outliers), and protect against adversarial attacks. To be effective, these measures should (i) account for the wide variety of models used in practice, (ii) be computable for trained models or at least avoid modifying established training procedures, (iii) forgo the use of data, which can expose them to the same robustness issues and attacks as the underlying model, and (iv) be followed by theoretical guarantees. These principles underly the framework developed in this work, which expresses the credibility as a risk-fit trade-off, i.e., a compromise between how much can fit be improved by perturbing the model input and the magnitude of this perturbation (risk). Using a constrained optimization formulation and duality theory, we analyze this compromise and show that this balance can be determined counterfactually, without having to test multiple perturbations. This results in an unsupervised, a posteriori method of assigning prediction credibility for any (possibly non-convex) differentiable model, from RKHS-based solutions to any architecture of (feedforward, convolutional, graph) neural network. Its use is illustrated in data filtering and defense against adversarial attacks.
Sequential convergence of AdaGrad algorithm for smooth convex optimization
Authors: Cheik Traoré, Edouard Pauwels
Date: 2020-11-24 00:00:00

We prove that the iterates produced by, either the scalar step size variant, or the coordinatewise variant of AdaGrad algorithm, are convergent sequences when applied to convex objective functions with Lipschitz gradient. The key insight is to remark that such AdaGrad sequences satisfy a variable metric quasi-Fej\'er monotonicity property, which allows to prove convergence.
mForms : Multimodal Form-Filling with Question Answering
Authors: Larry Heck, Simon Heck, Anirudh Sundar
Date: 2020-11-24 00:00:00
Tasks: Form, Question Answering, slot-filling, Slot Filling, Transfer Learning, Zero-shot Slot Filling

This paper presents a new approach to form-filling by reformulating the task as multimodal natural language Question Answering (QA). The reformulation is achieved by first translating the elements on the GUI form (text fields, buttons, icons, etc.) to natural language questions, where these questions capture the element's multimodal semantics. After a match is determined between the form element (Question) and the user utterance (Answer), the form element is filled through a pre-trained extractive QA system. By leveraging pre-trained QA models and not requiring form-specific training, this approach to form-filling is zero-shot. The paper also presents an approach to further refine the form-filling by using multi-task training to incorporate a potentially large number of successive tasks. Finally, the paper introduces a multimodal natural language form-filling dataset Multimodal Forms (mForms), as well as a multimodal extension of the popular ATIS dataset to support future research and experimentation. Results show the new approach not only maintains robust accuracy for sparse training conditions but achieves state-of-the-art F1 of 0.97 on ATIS with approximately 1/10th of the training data.
PeleNet: A Reservoir Computing Framework for Loihi
Authors: Carlo Michaelis
Date: 2020-11-24 00:00:00

High-level frameworks for spiking neural networks are a key factor for fast prototyping and efficient development of complex algorithms. Such frameworks have emerged in the last years for traditional computers, but programming neuromorphic hardware is still a challenge. Often low level programming with knowledge about the hardware of the neuromorphic chip is required. The PeleNet framework aims to simplify reservoir computing for the neuromorphic hardware Loihi. It is build on top of the NxSDK from Intel and is written in Python. The framework manages weight matrices, parameters and probes. In particular, it provides an automatic and efficient distribution of networks over several cores and chips. With this, the user is not confronted with technical details and can concentrate on experiments.