pwc-archive/papers-with-abstracts huggingface dataset:
Sequential Outlier Detection based on Incremental Decision Trees
Authors: Mohammadreza Mohaghegh Neyshabouri, Suleyman Serdar Kozat
Date: 2018-03-09 00:00:00
Tasks: Outlier Detection

We introduce an online outlier detection algorithm to detect outliers in a
sequentially observed data stream. For this purpose, we use a two-stage
filtering and hedging approach. In the first stage, we construct a multi-modal
probability density function to model the normal samples. In the second stage,
given a new observation, we label it as an anomaly if the value of
aforementioned density function is below a specified threshold at the newly
observed point. In order to construct our multi-modal density function, we use
an incremental decision tree to construct a set of subspaces of the observation
space. We train a single component density function of the exponential family
using the observations, which fall inside each subspace represented on the
tree. These single component density functions are then adaptively combined to
produce our multi-modal density function, which is shown to achieve the
performance of the best convex combination of the density functions defined on
the subspaces. As we observe more samples, our tree grows and produces more
subspaces. As a result, our modeling power increases in time, while mitigating
overfitting issues. In order to choose our threshold level to label the
observations, we use an adaptive thresholding scheme. We show that our adaptive
threshold level achieves the performance of the optimal pre-fixed threshold
level, which knows the observation labels in hindsight. Our algorithm provides
significant performance improvements over the state of the art in our wide set
of experiments involving both synthetic as well as real data.
Competitive Machine Learning: Best Theoretical Prediction vs Optimization
Authors: Amin Khajehnejad, Shima Hajimirza
Date: 2018-03-09 00:00:00
Tasks: BIG-bench Machine Learning, Prediction

Machine learning is often used in competitive scenarios: Participants learn
and fit static models, and those models compete in a shared platform. The
common assumption is that in order to win a competition one has to have the
best predictive model, i.e., the model with the smallest out-sample error. Is
that necessarily true? Does the best theoretical predictive model for a target
always yield the best reward in a competition? If not, can one take the best
model and purposefully change it into a theoretically inferior model which in
practice results in a higher competitive edge? How does that modification look
like? And finally, if all participants modify their prediction models towards
the best practical performance, who benefits the most? players with inferior
models, or those with theoretical superiority? The main theme of this paper is
to raise these important questions and propose a theoretical model to answer
them. We consider a study case where two linear predictive models compete over
a shared target. The model with the closest estimate gets the whole reward,
which is equal to the absolute value of the target. We characterize the reward
function of each model, and using a basic game theoretic approach, demonstrate
that the inferior competitor can significantly improve his performance by
choosing optimal model coefficients that are different from the best
theoretical prediction. This is a preliminary study that emphasizes the fact
that in many applications where predictive machine learning is at the service
of competition, much can be gained from practical (back-testing) optimization
of the model compared to static prediction improvement.
Provably robust estimation of modulo 1 samples of a smooth function with applications to phase unwrapping
Authors: Mihai Cucuringu, Hemant Tyagi
Date: 2018-03-09 00:00:00
Tasks: Denoising, Riemannian optimization

Consider an unknown smooth function $f: [0,1]^d \rightarrow \mathbb{R}$, and say we are given $n$ noisy mod 1 samples of $f$, i.e., $y_i = (f(x_i) + \eta_i)\mod 1$, for $x_i \in [0,1]^d$, where $\eta_i$ denotes the noise. Given the samples $(x_i,y_i)_{i=1}^{n}$, our goal is to recover smooth, robust estimates of the clean samples $f(x_i) \bmod 1$. We formulate a natural approach for solving this problem, which works with angular embeddings of the noisy mod 1 samples over the unit circle, inspired by the angular synchronization framework. This amounts to solving a smoothness regularized least-squares problem -- a quadratically constrained quadratic program (QCQP) -- where the variables are constrained to lie on the unit circle. Our approach is based on solving its relaxation, which is a trust-region sub-problem and hence solvable efficiently. We provide theoretical guarantees demonstrating its robustness to noise for adversarial, and random Gaussian and Bernoulli noise models. To the best of our knowledge, these are the first such theoretical results for this problem. We demonstrate the robustness and efficiency of our approach via extensive numerical simulations on synthetic data, along with a simple least-squares solution for the unwrapping stage, that recovers the original samples of $f$ (up to a global shift). It is shown to perform well at high levels of noise, when taking as input the denoised modulo $1$ samples. Finally, we also consider two other approaches for denoising the modulo 1 samples that leverage tools from Riemannian optimization on manifolds, including a Burer-Monteiro approach for a semidefinite programming relaxation of our formulation. For the two-dimensional version of the problem, which has applications in radar interferometry, we are able to solve instances of real-world data with a million sample points in under 10 seconds, on a personal laptop.
Patchwork Kriging for Large-scale Gaussian Process Regression
Authors: Chiwoo Park, Daniel Apley
Date: 2017-01-23 00:00:00
Tasks: regression, Uncertainty Quantification

This paper presents a new approach for Gaussian process (GP) regression for
large datasets. The approach involves partitioning the regression input domain
into multiple local regions with a different local GP model fitted in each
region. Unlike existing local partitioned GP approaches, we introduce a
technique for patching together the local GP models nearly seamlessly to ensure
that the local GP models for two neighboring regions produce nearly the same
response prediction and prediction error variance on the boundary between the
two regions. This largely mitigates the well-known discontinuity problem that
degrades the boundary accuracy of existing local partitioned GP methods. Our
main innovation is to represent the continuity conditions as additional
pseudo-observations that the differences between neighboring GP responses are
identically zero at an appropriately chosen set of boundary input locations. To
predict the response at any input location, we simply augment the actual
response observations with the pseudo-observations and apply standard GP
prediction methods to the augmented data. In contrast to heuristic continuity
adjustments, this has an advantage of working within a formal GP framework, so
that the GP-based predictive uncertainty quantification remains valid. Our
approach also inherits a sparse block-like structure for the sample covariance
matrix, which results in computationally efficient closed-form expressions for
the predictive mean and variance. In addition, we provide a new spatial
partitioning scheme based on a recursive space partitioning along local
principal component directions, which makes the proposed approach applicable
for regression domains having more than two dimensions. Using three spatial
datasets and three higher dimensional datasets, we investigate the numerical
performance of the approach and compare it to several state-of-the-art
approaches.
Hourly-Similarity Based Solar Forecasting Using Multi-Model Machine Learning Blending
Authors: Cong Feng, Jie Zhang
Date: 2018-03-09 00:00:00
Tasks: BIG-bench Machine Learning

With the increasing penetration of solar power into power systems,
forecasting becomes critical in power system operations. In this paper, an
hourly-similarity (HS) based method is developed for 1-hour-ahead (1HA) global
horizontal irradiance (GHI) forecasting. This developed method utilizes diurnal
patterns, statistical distinctions between different hours, and hourly
similarities in solar data to improve the forecasting accuracy. The HS-based
method is built by training multiple two-layer multi-model forecasting
framework (MMFF) models independently with the same-hour subsets. The final
optimal model is a combination of MMFF models with the best-performed blending
algorithm at every hour. At the forecasting stage, the most suitable model is
selected to perform the forecasting subtask of a certain hour. The HS-based
method is validated by 1-year data with six solar features collected by the
National Renewable Energy Laboratory (NREL). Results show that the HS-based
method outperforms the non-HS (all-in-one) method significantly with the same
MMFF architecture, wherein the optimal HS- based method outperforms the best
all-in-one method by 10.94% and 7.74% based on the normalized mean absolute
error and normalized root mean square error, respectively.
Representation Learning using Event-based STDP
Authors: Amirhossein Tavanaei, Timothee Masquelier, Anthony Maida
Date: 2017-06-20 00:00:00
Tasks: Quantization, Representation Learning

Although representation learning methods developed within the framework of
traditional neural networks are relatively mature, developing a spiking
representation model remains a challenging problem. This paper proposes an
event-based method to train a feedforward spiking neural network (SNN) layer
for extracting visual features. The method introduces a novel
spike-timing-dependent plasticity (STDP) learning rule and a threshold
adjustment rule both derived from a vector quantization-like objective function
subject to a sparsity constraint. The STDP rule is obtained by the gradient of
a vector quantization criterion that is converted to spike-based,
spatio-temporally local update rules in a spiking network of leaky,
integrate-and-fire (LIF) neurons. Independence and sparsity of the model are
achieved by the threshold adjustment rule and by a softmax function
implementing inhibition in the representation layer consisting of
WTA-thresholded spiking neurons. Together, these mechanisms implement a form of
spike-based, competitive learning. Two sets of experiments are performed on the
MNIST and natural image datasets. The results demonstrate a sparse spiking
visual representation model with low reconstruction loss comparable with
state-of-the-art visual coding approaches, yet our rule is local in both time
and space, thus biologically plausible and hardware friendly.
Dropout Feature Ranking for Deep Learning Models
Authors: Chun-Hao Chang, Ladislav Rampasek, Anna Goldenberg
Date: 2017-12-22 00:00:00
Tasks: Deep Learning, Time Series, Time Series Analysis

Deep neural networks (DNNs) achieve state-of-the-art results in a variety of
domains. Unfortunately, DNNs are notorious for their non-interpretability, and
thus limit their applicability in hypothesis-driven domains such as biology and
healthcare. Moreover, in the resource-constraint setting, it is critical to
design tests relying on fewer more informative features leading to high
accuracy performance within reasonable budget. We aim to close this gap by
proposing a new general feature ranking method for deep learning. We show that
our simple yet effective method performs on par or compares favorably to eight
strawman, classical and deep-learning feature ranking methods in two
simulations and five very different datasets on tasks ranging from
classification to regression, in both static and time series scenarios. We also
illustrate the use of our method on a drug response dataset and show that it
identifies genes relevant to the drug-response.
BP-STDP: Approximating Backpropagation using Spike Timing Dependent Plasticity
Authors: Amirhossein Tavanaei, Anthony S. Maida
Date: 2017-11-12 00:00:00

The problem of training spiking neural networks (SNNs) is a necessary
precondition to understanding computations within the brain, a field still in
its infancy. Previous work has shown that supervised learning in multi-layer
SNNs enables bio-inspired networks to recognize patterns of stimuli through
hierarchical feature acquisition. Although gradient descent has shown
impressive performance in multi-layer (and deep) SNNs, it is generally not
considered biologically plausible and is also computationally expensive. This
paper proposes a novel supervised learning approach based on an event-based
spike-timing-dependent plasticity (STDP) rule embedded in a network of
integrate-and-fire (IF) neurons. The proposed temporally local learning rule
follows the backpropagation weight change updates applied at each time step.
This approach enjoys benefits of both accurate gradient descent and temporally
local, efficient STDP. Thus, this method is able to address some open questions
regarding accurate and efficient computations that occur in the brain. The
experimental results on the XOR problem, the Iris data, and the MNIST dataset
demonstrate that the proposed SNN performs as successfully as the traditional
NNs. Our approach also compares favorably with the state-of-the-art multi-layer
SNNs.
The Importance of Being Recurrent for Modeling Hierarchical Structure
Authors: Ke Tran, Arianna Bisazza, Christof Monz
Date: 2018-03-09 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Language Modeling, Language Modelling, Machine Translation, Translation

Recent work has shown that recurrent neural networks (RNNs) can implicitly
capture and exploit hierarchical information when trained to solve common
natural language processing tasks such as language modeling (Linzen et al.,
2016) and neural machine translation (Shi et al., 2016). In contrast, the
ability to model structured data with non-recurrent neural networks has
received little attention despite their success in many NLP tasks (Gehring et
al., 2017; Vaswani et al., 2017). In this work, we compare the two
architectures---recurrent versus non-recurrent---with respect to their ability
to model hierarchical structure and find that recurrency is indeed important
for this purpose.
Intentions of Vulnerable Road Users - Detection and Forecasting by Means of Machine Learning
Authors: Michael Goldhammer, Sebastian Köhler, Stefan Zernetsch, Konrad Doll, Bernhard Sick, Klaus Dietmayer
Date: 2018-03-09 00:00:00
Tasks: BIG-bench Machine Learning, General Classification, Trajectory Prediction

Avoiding collisions with vulnerable road users (VRUs) using sensor-based
early recognition of critical situations is one of the manifold opportunities
provided by the current development in the field of intelligent vehicles. As
especially pedestrians and cyclists are very agile and have a variety of
movement options, modeling their behavior in traffic scenes is a challenging
task. In this article we propose movement models based on machine learning
methods, in particular artificial neural networks, in order to classify the
current motion state and to predict the future trajectory of VRUs. Both model
types are also combined to enable the application of specifically trained
motion predictors based on a continuously updated pseudo probabilistic state
classification. Furthermore, the architecture is used to evaluate
motion-specific physical models for starting and stopping and video-based
pedestrian motion classification. A comprehensive dataset consisting of 1068
pedestrian and 494 cyclist scenes acquired at an urban intersection is used for
optimization, training, and evaluation of the different models. The results
show substantial higher classification rates and the ability to earlier
recognize motion state changes with the machine learning approaches compared to
interacting multiple model (IMM) Kalman Filtering. The trajectory prediction
quality is also improved for all kinds of test scenes, especially when starting
and stopping motions are included. Here, 37\% and 41\% lower position errors
were achieved on average, respectively.
City-wide Analysis of Electronic Health Records Reveals Gender and Age Biases in the Administration of Known Drug-Drug Interactions
Authors: Rion Brattig Correia, Luciana P. de Araújo, Mauro M. Mattos, Luis M. Rocha
Date: 2018-03-09 00:00:00
Tasks: Management

The occurrence of drug-drug-interactions (DDI) from multiple drug dispensations is a serious problem, both for individuals and health-care systems, since patients with complications due to DDI are likely to reenter the system at a costlier level. We present a large-scale longitudinal study (18 months) of the DDI phenomenon at the primary- and secondary-care level using electronic health records (EHR) from the city of Blumenau in Southern Brazil (pop. $\approx 340,000$). We found that 181 distinct drug pairs known to interact were dispensed concomitantly to 12\% of the patients in the city's public health-care system. Further, 4\% of the patients were dispensed drug pairs that are likely to result in major adverse drug reactions (ADR)---with costs estimated to be much larger than previously reported in smaller studies. The large-scale analysis reveals that women have a 60\% increased risk of DDI as compared to men; the increase becomes 90\% when considering only DDI known to lead to major ADR. Furthermore, DDI risk increases substantially with age; patients aged 70-79 years have a 34\% risk of DDI when they are dispensed two or more drugs concomitantly. Interestingly, a statistical null model demonstrates that age- and female-specific risks from increased polypharmacy fail by far to explain the observed DDI risks in those populations, suggesting unknown social or biological causes. We also provide a network visualization of drugs and demographic factors that characterize the DDI phenomenon and demonstrate that accurate DDI prediction can be included in healthcare and public-health management, to reduce DDI-related ADR and costs.
An Incremental Self-Organizing Architecture for Sensorimotor Learning and Prediction
Authors: Luiza Mici, German I. Parisi, Stefan Wermter
Date: 2017-12-22 00:00:00

During visuomotor tasks, robots must compensate for temporal delays inherent
in their sensorimotor processing systems. Delay compensation becomes crucial in
a dynamic environment where the visual input is constantly changing, e.g.,
during the interacting with a human demonstrator. For this purpose, the robot
must be equipped with a prediction mechanism for using the acquired perceptual
experience to estimate possible future motor commands. In this paper, we
present a novel neural network architecture that learns prototypical visuomotor
representations and provides reliable predictions on the basis of the visual
input. These predictions are used to compensate for the delayed motor behavior
in an online manner. We investigate the performance of our method with a set of
experiments comprising a humanoid robot that has to learn and generate visually
perceived arm motion trajectories. We evaluate the accuracy in terms of mean
prediction error and analyze the response of the network to novel movement
demonstrations. Additionally, we report experiments with incomplete data
sequences, showing the robustness of the proposed architecture in the case of a
noisy and faulty visual sensor.
Fast cosmic web simulations with generative adversarial networks
Authors: Andres C. Rodriguez, Tomasz Kacprzak, Aurelien Lucchi, Adam Amara, Raphael Sgier, Janis Fluri, Thomas Hofmann, Alexandre Réfrégier
Date: 2018-01-27 00:00:00

Dark matter in the universe evolves through gravity to form a complex network
of halos, filaments, sheets and voids, that is known as the cosmic web.
Computational models of the underlying physical processes, such as classical
N-body simulations, are extremely resource intensive, as they track the action
of gravity in an expanding universe using billions of particles as tracers of
the cosmic matter distribution. Therefore, upcoming cosmology experiments will
face a computational bottleneck that may limit the exploitation of their full
scientific potential. To address this challenge, we demonstrate the application
of a machine learning technique called Generative Adversarial Networks (GAN) to
learn models that can efficiently generate new, physically realistic
realizations of the cosmic web. Our training set is a small, representative
sample of 2D image snapshots from N-body simulations of size 500 and 100 Mpc.
We show that the GAN-generated samples are qualitatively and quantitatively
very similar to the originals. For the larger boxes of size 500 Mpc, it is very
difficult to distinguish them visually. The agreement of the power spectrum
$P_k$ is 1-2\% for most of the range, between $k=0.06$ and $k=0.4$. An
important advantage of generating cosmic web realizations with a GAN is the
considerable gains in terms of computation time. Each new sample generated by a
GAN takes a fraction of a second, compared to the many hours needed by
traditional N-body techniques. We anticipate that the use of generative models
such as GANs will therefore play an important role in providing extremely fast
and precise simulations of cosmic web in the era of large cosmological surveys,
such as Euclid and Large Synoptic Survey Telescope (LSST).
Explaining Black-box Android Malware Detection
Authors: Marco Melis, Davide Maiorca, Battista Biggio, Giorgio Giacinto, Fabio Roli
Date: 2018-03-09 00:00:00
Tasks: Android Malware Detection, BIG-bench Machine Learning, Malware Detection

Machine-learning models have been recently used for detecting malicious
Android applications, reporting impressive performances on benchmark datasets,
even when trained only on features statically extracted from the application,
such as system calls and permissions. However, recent findings have highlighted
the fragility of such in-vitro evaluations with benchmark datasets, showing
that very few changes to the content of Android malware may suffice to evade
detection. How can we thus trust that a malware detector performing well on
benchmark data will continue to do so when deployed in an operating
environment? To mitigate this issue, the most popular Android malware detectors
use linear, explainable machine-learning models to easily identify the most
influential features contributing to each decision. In this work, we generalize
this approach to any black-box machine- learning model, by leveraging a
gradient-based approach to identify the most influential local features. This
enables using nonlinear models to potentially increase accuracy without
sacrificing interpretability of decisions. Our approach also highlights the
global characteristics learned by the model to discriminate between benign and
malware applications. Finally, as shown by our empirical analysis on a popular
Android malware detection task, it also helps identifying potential
vulnerabilities of linear and nonlinear models against adversarial
manipulations.
A concentration inequality for the excess risk in least-squares regression with random design and heteroscedastic noise
Authors: Adrien Saumard
Date: 2017-02-16 00:00:00
Tasks: regression

We prove a new and general concentration inequality for the excess risk in
least-squares regression with random design and heteroscedastic noise. No
specific structure is required on the model, except the existence of a suitable
function that controls the local suprema of the empirical process. So far, only
the case of linear contrast estimation was tackled in the literature with this
level of generality on the model. We solve here the case of a quadratic
contrast, by separating the behavior of a linearized empirical process and the
empirical process driven by the squares of functions of models.