pwc-archive/papers-with-abstracts huggingface dataset:
NAOMI: Non-Autoregressive Multiresolution Sequence Imputation
Authors: Yukai Liu, Rose Yu, Stephan Zheng, Eric Zhan, Yisong Yue
Date: 2019-01-30 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Imitation Learning, Imputation, Multivariate Time Series Imputation

Missing value imputation is a fundamental problem in spatiotemporal modeling, from motion tracking to the dynamics of physical systems. Deep autoregressive models suffer from error propagation which becomes catastrophic for imputing long-range sequences. In this paper, we take a non-autoregressive approach and propose a novel deep generative model: Non-AutOregressive Multiresolution Imputation (NAOMI) to impute long-range sequences given arbitrary missing patterns. NAOMI exploits the multiresolution structure of spatiotemporal data and decodes recursively from coarse to fine-grained resolutions using a divide-and-conquer strategy. We further enhance our model with adversarial training. When evaluated extensively on benchmark datasets from systems of both deterministic and stochastic dynamics. NAOMI demonstrates significant improvement in imputation accuracy (reducing average prediction error by 60% compared to autoregressive counterparts) and generalization for long range sequences.
Securing Fog-to-Things Environment Using Intrusion Detection System Based On Ensemble Learning
Authors: Poulmanogo Illy, Georges Kaddoum, Christian Miranda Moreira, Kuljeet Kaur, Sahil Garg
Date: 2019-01-30 00:00:00
Tasks: Anomaly Detection, Ensemble Learning, Intrusion Detection

The growing interest in the Internet of Things (IoT) applications is
associated with an augmented volume of security threats. In this vein, the
Intrusion detection systems (IDS) have emerged as a viable solution for the
detection and prevention of malicious activities. Unlike the signature-based
detection approaches, machine learning-based solutions are a promising means
for detecting unknown attacks. However, the machine learning models need to be
accurate enough to reduce the number of false alarms. More importantly, they
need to be trained and evaluated on realistic datasets such that their efficacy
can be validated on real-time deployments. Many solutions proposed in the
literature are reported to have high accuracy but are ineffective in real
applications due to the non-representativity of the dataset used for training
and evaluation of the underlying models. On the other hand, some of the
existing solutions overcome these challenges but yield low accuracy which
hampers their implementation for commercial tools. These solutions are majorly
based on single learners and are therefore directly affected by the intrinsic
limitations of each learning algorithm. The novelty of this paper is to use the
most realistic dataset available for intrusion detection called NSL-KDD, and
combine multiple learners to build ensemble learners that increase the accuracy
of the detection. Furthermore, a deployment architecture in a fog-to-things
environment that employs two levels of classifications is proposed. In such
architecture, the first level performs an anomaly detection which reduces the
latency of the classification substantially, while the second level, executes
attack classifications, enabling precise prevention measures. Finally, the
experimental results demonstrate the effectiveness of the proposed IDS in
comparison with the other state-of-the-arts on the NSL-KDD dataset.
Benchmarking Classic and Learned Navigation in Complex 3D Environments
Authors: Dmytro Mishkin, Alexey Dosovitskiy, Vladlen Koltun
Date: 2019-01-30 00:00:00
Tasks: Benchmarking

Navigation research is attracting renewed interest with the advent of
learning-based methods. However, this new line of work is largely disconnected
from well-established classic navigation approaches. In this paper, we take a
step towards coordinating these two directions of research. We set up classic
and learning-based navigation systems in common simulated environments and
thoroughly evaluate them in indoor spaces of varying complexity, with access to
different sensory modalities. Additionally, we measure human performance in the
same environments. We find that a classic pipeline, when properly tuned, can
perform very well in complex cluttered environments. On the other hand, learned
systems can operate more robustly with a limited sensor suite. Overall, both
approaches are still far from human-level performance.
A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms
Authors: Yoshua Bengio, Tristan Deleu, Nasim Rahaman, Rosemary Ke, Sébastien Lachapelle, Olexa Bilaniuk, Anirudh Goyal, Christopher Pal
Date: 2019-01-30 00:00:00
Proceeding: ICLR 2020 1
Tasks: Meta-Learning

We propose to meta-learn causal structures based on how fast a learner adapts
to new distributions arising from sparse distributional changes, e.g. due to
interventions, actions of agents and other sources of non-stationarities. We
show that under this assumption, the correct causal structural choices lead to
faster adaptation to modified distributions because the changes are
concentrated in one or just a few mechanisms when the learned knowledge is
modularized appropriately. This leads to sparse expected gradients and a lower
effective number of degrees of freedom needing to be relearned while adapting
to the change. It motivates using the speed of adaptation to a modified
distribution as a meta-learning objective. We demonstrate how this can be used
to determine the cause-effect relationship between two observed variables. The
distributional changes do not need to correspond to standard interventions
(clamping a variable), and the learner has no direct knowledge of these
interventions. We show that causal structures can be parameterized via
continuous variables and learned end-to-end. We then explore how these ideas
could be used to also learn an encoder that would map low-level observed
variables to unobserved causal variables leading to faster adaptation
out-of-distribution, learning a representation space where one can satisfy the
assumptions of independent mechanisms and of small and sparse changes in these
mechanisms due to actions and non-stationarities.
Characterization of migrated seismic volumes using texture attributes: a comparative study
Authors: Zhiling Long, Yazeed Alaudah, Muhammad Ali Qureshi, Motaz Al Farraj, Zhen Wang, Asjad Amin, Mohamed Deriche, Ghassan AlRegib
Date: 2019-01-30 00:00:00
Tasks: Image Retrieval, Retrieval, Seismic Interpretation, Texture Classification

In this paper, we examine several typical texture attributes developed in the
image processing community in recent years with respect to their capability of
characterizing a migrated seismic volume. These attributes are generated in
either frequency or space domain, including steerable pyramid, curvelet, local
binary pattern, and local radius index. The comparative study is performed
within an image retrieval framework. We evaluate these attributes in terms of
retrieval accuracy. It is our hope that this comparative study will help
acquaint the seismic interpretation community with the many available powerful
image texture analysis techniques, providing more alternative attributes for
their seismic exploration.
InfoBot: Transfer and Exploration via the Information Bottleneck
Authors: Anirudh Goyal, Riashat Islam, Daniel Strouse, Zafarali Ahmed, Matthew Botvinick, Hugo Larochelle, Yoshua Bengio, Sergey Levine
Date: 2019-01-30 00:00:00
Tasks: Reinforcement Learning

A central challenge in reinforcement learning is discovering effective policies for tasks where rewards are sparsely distributed. We postulate that in the absence of useful reward signals, an effective exploration strategy should seek out {\it decision states}. These states lie at critical junctions in the state space from where the agent can transition to new, potentially unexplored regions. We propose to learn about decision states from prior experience. By training a goal-conditioned policy with an information bottleneck, we can identify decision states by examining where the model actually leverages the goal state. We find that this simple mechanism effectively identifies decision states, even in partially observed settings. In effect, the model learns the sensory cues that correlate with potential subgoals. In new environments, this model can then identify novel subgoals for further exploration, guiding the agent through a sequence of potential decision states and through new regions of the state space.
On Correlation of Features Extracted by Deep Neural Networks
Authors: Babajide O. Ayinde, Tamer Inanc, Jacek M. Zurada
Date: 2019-01-30 00:00:00

Redundancy in deep neural network (DNN) models has always been one of their
most intriguing and important properties. DNNs have been shown to
overparameterize, or extract a lot of redundant features. In this work, we
explore the impact of size (both width and depth), activation function, and
weight initialization on the susceptibility of deep neural network models to
extract redundant features. To estimate the number of redundant features in
each layer, all the features of a given layer are hierarchically clustered
according to their relative cosine distances in feature space and a set
threshold. It is shown that both network size and activation function are the
two most important components that foster the tendency of DNNs to extract
redundant features. The concept is illustrated using deep multilayer perceptron
and convolutional neural networks on MNIST digits recognition and CIFAR-10
dataset, respectively.
Span Model for Open Information Extraction on Accurate Corpus
Authors: Junlang Zhan, Hai Zhao
Date: 2019-01-30 00:00:00
Tasks: Open Information Extraction

Open information extraction (Open IE) is a challenging task especially due to its brittle data basis. Most of Open IE systems have to be trained on automatically built corpus and evaluated on inaccurate test set. In this work, we first alleviate this difficulty from both sides of training and test sets. For the former, we propose an improved model design to more sufficiently exploit training dataset. For the latter, we present our accurately re-annotated benchmark test set (Re-OIE6) according to a series of linguistic observation and analysis. Then, we introduce a span model instead of previous adopted sequence labeling formulization for n-ary Open IE. Our newly introduced model achieves new state-of-the-art performance on both benchmark evaluation datasets.
Learning to Project in Multi-Objective Binary Linear Programming
Authors: Alvaro Sierra-Altamiranda, Hadi Charkhgard, Iman Dayarian, Ali Eshragh, Sorna Javadi
Date: 2019-01-30 00:00:00
Tasks: BIG-bench Machine Learning

In this paper, we investigate the possibility of improving the performance of
multi-objective optimization solution approaches using machine learning
techniques. Specifically, we focus on multi-objective binary linear programs
and employ one of the most effective and recently developed criterion space
search algorithms, the so-called KSA, during our study. This algorithm computes
all nondominated points of a problem with p objectives by searching on a
projected criterion space, i.e., a (p-1)-dimensional criterion apace. We
present an effective and fast learning approach to identify on which projected
space the KSA should work. We also present several generic features/variables
that can be used in machine learning techniques for identifying the best
projected space. Finally, we present an effective bi-objective optimization
based heuristic for selecting the best subset of the features to overcome the
issue of overfitting in learning. Through an extensive computational study over
2000 instances of tri-objective Knapsack and Assignment problems, we
demonstrate that an improvement of up to 12% in time can be achieved by the
proposed learning method compared to a random selection of the projected space.
Benefits and Pitfalls of the Exponential Mechanism with Applications to Hilbert Spaces and Functional PCA
Authors: Jordan Awan, Ana Kenney, Matthew Reimherr, Aleksandra Slavković
Date: 2019-01-30 00:00:00

The exponential mechanism is a fundamental tool of Differential Privacy (DP)
due to its strong privacy guarantees and flexibility. We study its extension to
settings with summaries based on infinite dimensional outputs such as with
functional data analysis, shape analysis, and nonparametric statistics. We show
that one can design the mechanism with respect to a specific base measure over
the output space, such as a Guassian process. We provide a positive result that
establishes a Central Limit Theorem for the exponential mechanism quite
broadly. We also provide an apparent negative result, showing that the
magnitude of the noise introduced for privacy is asymptotically non-negligible
relative to the statistical estimation error. We develop an \ep-DP mechanism
for functional principal component analysis, applicable in separable Hilbert
spaces. We demonstrate its performance via simulations and applications to two
datasets.
A Simple Explanation for the Existence of Adversarial Examples with Small Hamming Distance
Authors: Adi Shamir, Itay Safran, Eyal Ronen, Orr Dunkelman
Date: 2019-01-30 00:00:00

The existence of adversarial examples in which an imperceptible change in the
input can fool well trained neural networks was experimentally discovered by
Szegedy et al in 2013, who called them "Intriguing properties of neural
networks". Since then, this topic had become one of the hottest research areas
within machine learning, but the ease with which we can switch between any two
decisions in targeted attacks is still far from being understood, and in
particular it is not clear which parameters determine the number of input
coordinates we have to change in order to mislead the network. In this paper we
develop a simple mathematical framework which enables us to think about this
baffling phenomenon from a fresh perspective, turning it into a natural
consequence of the geometry of $\mathbb{R}^n$ with the $L_0$ (Hamming) metric,
which can be quantitatively analyzed. In particular, we explain why we should
expect to find targeted adversarial examples with Hamming distance of roughly
$m$ in arbitrarily deep neural networks which are designed to distinguish
between $m$ input classes.
View Invariant 3D Human Pose Estimation
Authors: Guoqiang Wei, Cuiling Lan, Wen-Jun Zeng, Zhibo Chen
Date: 2019-01-30 00:00:00
Tasks: 3D Human Pose Estimation, 3D Pose Estimation, Diversity, Pose Estimation

The recent success of deep networks has significantly advanced 3D human pose
estimation from 2D images. The diversity of capturing viewpoints and the
flexibility of the human poses, however, remain some significant challenges. In
this paper, we propose a view invariant 3D human pose estimation module to
alleviate the effects of viewpoint diversity. The framework consists of a base
network, which provides an initial estimation of a 3D pose, a view-invariant
hierarchical correction network (VI-HC) on top of that to learn the 3D pose
refinement under consistent views, and a view-invariant discriminative network
(VID) to enforce high-level constraints over body configurations. In VI-HC, the
initial 3D pose inputs are automatically transformed to consistent views for
further refinements at the global body and local body parts level,
respectively. For the VID, under consistent viewpoints, we use adversarial
learning to differentiate between estimated poses and real poses to avoid
implausible 3D poses. Experimental results demonstrate that the consistent
viewpoints can dramatically enhance the performance. Our module shows
robustness for different 3D pose base networks and achieves a significant
improvement (about 9%) over a powerful baseline on the public 3D pose
estimation benchmark Human3.6M.
Noise-tolerant fair classification
Authors: Alexandre Louis Lamy, Ziyuan Zhong, Aditya Krishna Menon, Nakul Verma
Date: 2019-01-30 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Classification, Fairness, General Classification

Fairness-aware learning involves designing algorithms that do not discriminate with respect to some sensitive feature (e.g., race or gender). Existing work on the problem operates under the assumption that the sensitive feature available in one's training sample is perfectly reliable. This assumption may be violated in many real-world cases: for example, respondents to a survey may choose to conceal or obfuscate their group identity out of fear of potential discrimination. This poses the question of whether one can still learn fair classifiers given noisy sensitive features. In this paper, we answer the question in the affirmative: we show that if one measures fairness using the mean-difference score, and sensitive features are subject to noise from the mutually contaminated learning model, then owing to a simple identity we only need to change the desired fairness-tolerance. The requisite tolerance can be estimated by leveraging existing noise-rate estimators from the label noise literature. We finally show that our procedure is empirically effective on two case-studies involving sensitive feature censoring.
Diversity Regularized Adversarial Learning
Authors: Babajide O. Ayinde, Keishin Nishihama, Jacek M. Zurada
Date: 2019-01-30 00:00:00
Tasks: Diversity, Image Generation

The two key players in Generative Adversarial Networks (GANs), the
discriminator and generator, are usually parameterized as deep neural networks
(DNNs). On many generative tasks, GANs achieve state-of-the-art performance but
are often unstable to train and sometimes miss modes. A typical failure mode is
the collapse of the generator to a single parameter configuration where its
outputs are identical. When this collapse occurs, the gradient of the
discriminator may point in similar directions for many similar points. We
hypothesize that some of these shortcomings are in part due to primitive and
redundant features extracted by discriminator and this can easily make the
training stuck. We present a novel approach for regularizing adversarial models
by enforcing diverse feature learning. In order to do this, both generator and
discriminator are regularized by penalizing both negatively and positively
correlated features according to their differentiation and based on their
relative cosine distances. In addition to the gradient information from the
adversarial loss made available by the discriminator, diversity regularization
also ensures that a more stable gradient is provided to update both the
generator and discriminator. Results indicate our regularizer enforces diverse
features, stabilizes training, and improves image synthesis.
Automated Skin Lesion Classification Using Ensemble of Deep Neural Networks in ISIC 2018: Skin Lesion Analysis Towards Melanoma Detection Challenge
Authors: Md Ashraful Alam Milton
Date: 2019-01-30 00:00:00
Tasks: General Classification, Lesion Classification, Skin Lesion Classification

In this paper, we studied extensively on different deep learning based
methods to detect melanoma and skin lesion cancers. Melanoma, a form of
malignant skin cancer is very threatening to health. Proper diagnosis of
melanoma at an earlier stage is crucial for the success rate of complete cure.
Dermoscopic images with Benign and malignant forms of skin cancer can be
analyzed by computer vision system to streamline the process of skin cancer
detection. In this study, we experimented with various neural networks which
employ recent deep learning based models like PNASNet-5-Large,
InceptionResNetV2, SENet154, InceptionV4. Dermoscopic images are properly
processed and augmented before feeding them into the network. We tested our
methods on International Skin Imaging Collaboration (ISIC) 2018 challenge
dataset. Our system has achieved best validation score of 0.76 for
PNASNet-5-Large model. Further improvement and optimization of the proposed
methods with a bigger training dataset and carefully chosen hyper-parameter
could improve the performances. The code available for download at
https://github.com/miltonbd/ISIC_2018_classification
Generalized Tensor Models for Recurrent Neural Networks
Authors: Valentin Khrulkov, Oleksii Hrinchuk, Ivan Oseledets
Date: 2019-01-30 00:00:00
Proceeding: ICLR 2019 5

Recurrent Neural Networks (RNNs) are very successful at solving challenging
problems with sequential data. However, this observed efficiency is not yet
entirely explained by theory. It is known that a certain class of
multiplicative RNNs enjoys the property of depth efficiency --- a shallow
network of exponentially large width is necessary to realize the same score
function as computed by such an RNN. Such networks, however, are not very often
applied to real life tasks. In this work, we attempt to reduce the gap between
theory and practice by extending the theoretical analysis to RNNs which employ
various nonlinearities, such as Rectified Linear Unit (ReLU), and show that
they also benefit from properties of universality and depth efficiency. Our
theoretical results are verified by a series of extensive computational
experiments.
Deep Archetypal Analysis
Authors: Sebastian Mathias Keller, Maxim Samarin, Mario Wieser, Volker Roth
Date: 2019-01-30 00:00:00
Tasks: Representation Learning

"Deep Archetypal Analysis" generates latent representations of high-dimensional datasets in terms of fractions of intuitively understandable basic entities called archetypes. The proposed method is an extension of linear "Archetypal Analysis" (AA), an unsupervised method to represent multivariate data points as sparse convex combinations of extremal elements of the dataset. Unlike the original formulation of AA, "Deep AA" can also handle side information and provides the ability for data-driven representation learning which reduces the dependence on expert knowledge. Our method is motivated by studies of evolutionary trade-offs in biology where archetypes are species highly adapted to a single task. Along these lines, we demonstrate that "Deep AA" also lends itself to the supervised exploration of chemical space, marking a distinct starting point for de novo molecular design. In the unsupervised setting we show how "Deep AA" is used on CelebA to identify archetypal faces. These can then be superimposed in order to generate new faces which inherit dominant traits of the archetypes they are based on.
Recurrent Neural Networks for P300-based BCI
Authors: Ori Tal, Doron Friedman
Date: 2019-01-30 00:00:00
Tasks: Brain Computer Interface, EEG, Electroencephalogram (EEG)

P300-based spellers are one of the main methods for EEG-based brain-computer
interface, and the detection of the P300 target event with high accuracy is an
important prerequisite. The rapid serial visual presentation (RSVP) protocol is
of high interest because it can be used by patients who have lost control over
their eyes. In this study we wish to explore the suitability of recurrent
neural networks (RNNs) as a machine learning method for identifying the P300
signal in RSVP data. We systematically compare RNN with alternative methods
such as linear discriminant analysis (LDA) and convolutional neural network
(CNN). Our results indicate that LDA performs as well as the neural network
models or better on single subject data, but a network combining CNN and RNN
has advantages when transferring learning among subejcts, and is significantly
more resilient to temporal noise than other methods.