pwc-archive/papers-with-abstracts huggingface dataset:
Multi-Stage Fault Warning for Large Electric Grids Using Anomaly Detection and Machine Learning
Authors: Sanjeev Raja, Ernest Fokoué
Date: 2019-03-15 00:00:00
Tasks: Anomaly Detection, BIG-bench Machine Learning, Classification, Clustering, Dimensionality Reduction, Dynamic Time Warping, Fault Detection, General Classification, Subgroup Discovery, Time Series, Time Series Analysis

In the monitoring of a complex electric grid, it is of paramount importance
to provide operators with early warnings of anomalies detected on the network,
along with a precise classification and diagnosis of the specific fault type.
In this paper, we propose a novel multi-stage early warning system prototype
for electric grid fault detection, classification, subgroup discovery, and
visualization. In the first stage, a computationally efficient anomaly
detection method based on quartiles detects the presence of a fault in real
time. In the second stage, the fault is classified into one of nine pre-defined
disaster scenarios. The time series data are first mapped to highly
discriminative features by applying dimensionality reduction based on temporal
autocorrelation. The features are then mapped through one of three
classification techniques: support vector machine, random forest, and
artificial neural network. Finally in the third stage, intra-class clustering
based on dynamic time warping is used to characterize the fault with further
granularity. Results on the Bonneville Power Administration electric grid data
show that i) the proposed anomaly detector is both fast and accurate; ii)
dimensionality reduction leads to dramatic improvement in classification
accuracy and speed; iii) the random forest method offers the most accurate,
consistent, and robust fault classification; and iv) time series within a given
class naturally separate into five distinct clusters which correspond closely
to the geographical distribution of electric grid buses.
Tuning Hyperparameters without Grad Students: Scalable and Robust Bayesian Optimisation with Dragonfly
Authors: Kirthevasan Kandasamy, Karun Raju Vysyaraju, Willie Neiswanger, Biswajit Paria, Christopher R. Collins, Jeff Schneider, Barnabas Poczos, Eric P. Xing
Date: 2019-03-15 00:00:00
Tasks: Bayesian Optimisation

Bayesian Optimisation (BO) refers to a suite of techniques for global optimisation of expensive black box functions, which use introspective Bayesian models of the function to efficiently search for the optimum. While BO has been applied successfully in many applications, modern optimisation tasks usher in new challenges where conventional methods fail spectacularly. In this work, we present Dragonfly, an open source Python library for scalable and robust BO. Dragonfly incorporates multiple recently developed methods that allow BO to be applied in challenging real world settings; these include better methods for handling higher dimensional domains, methods for handling multi-fidelity evaluations when cheap approximations of an expensive function are available, methods for optimising over structured combinatorial spaces, such as the space of neural network architectures, and methods for handling parallel evaluations. Additionally, we develop new methodological improvements in BO for selecting the Bayesian model, selecting the acquisition function, and optimising over complex domains with different variable types and additional constraints. We compare Dragonfly to a suite of other packages and algorithms for global optimisation and demonstrate that when the above methods are integrated, they enable significant improvements in the performance of BO. The Dragonfly library is available at dragonfly.github.io.
Improving Strong-Scaling of CNN Training by Exploiting Finer-Grained Parallelism
Authors: Nikoli Dryden, Naoya Maruyama, Tom Benson, Tim Moon, Marc Snir, Brian Van Essen
Date: 2019-03-15 00:00:00
Tasks: image-classification, Image Classification

Scaling CNN training is necessary to keep up with growing datasets and reduce
training time. We also see an emerging need to handle datasets with very large
samples, where memory requirements for training are large. Existing training
frameworks use a data-parallel approach that partitions samples within a
mini-batch, but limits to scaling the mini-batch size and memory consumption
makes this untenable for large samples. We describe and implement new
approaches to convolution, which parallelize using spatial decomposition or a
combination of sample and spatial decomposition. This introduces many
performance knobs for a network, so we develop a performance model for CNNs and
present a method for using it to automatically determine efficient
parallelization strategies.
  We evaluate our algorithms with microbenchmarks and image classification with
ResNet-50. Our algorithms allow us to prototype a model for a mesh-tangling
dataset, where sample sizes are very large. We show that our parallelization
achieves excellent strong and weak scaling and enables training for previously
unreachable datasets.
GEE: A Gradient-based Explainable Variational Autoencoder for Network Anomaly Detection
Authors: Quoc Phong Nguyen, Kar Wai Lim, Dinil Mon Divakaran, Kian Hsiang Low, Mun Choon Chan
Date: 2019-03-15 00:00:00
Tasks: Anomaly Detection

This paper looks into the problem of detecting network anomalies by analyzing
NetFlow records. While many previous works have used statistical models and
machine learning techniques in a supervised way, such solutions have the
limitations that they require large amount of labeled data for training and are
unlikely to detect zero-day attacks. Existing anomaly detection solutions also
do not provide an easy way to explain or identify attacks in the anomalous
traffic. To address these limitations, we develop and present GEE, a framework
for detecting and explaining anomalies in network traffic. GEE comprises of two
components: (i) Variational Autoencoder (VAE) - an unsupervised deep-learning
technique for detecting anomalies, and (ii) a gradient-based fingerprinting
technique for explaining anomalies. Evaluation of GEE on the recent UGR dataset
demonstrates that our approach is effective in detecting different anomalies as
well as identifying fingerprints that are good representations of these various
attacks.
Rectified deep neural networks overcome the curse of dimensionality for nonsmooth value functions in zero-sum games of nonlinear stiff systems
Authors: Christoph Reisinger, Yufei Zhang
Date: 2019-03-15 00:00:00

In this paper, we establish that for a wide class of controlled stochastic differential equations (SDEs) with stiff coefficients, the value functions of corresponding zero-sum games can be represented by a deep artificial neural network (DNN), whose complexity grows at most polynomially in both the dimension of the state equation and the reciprocal of the required accuracy. Such nonlinear stiff systems may arise, for example, from Galerkin approximations of controlled stochastic partial differential equations (SPDEs), or controlled PDEs with uncertain initial conditions and source terms. This implies that DNNs can break the curse of dimensionality in numerical approximations and optimal control of PDEs and SPDEs. The main ingredient of our proof is to construct a suitable discrete-time system to effectively approximate the evolution of the underlying stochastic dynamics. Similar ideas can also be applied to obtain expression rates of DNNs for value functions induced by stiff systems with regime switching coefficients and driven by general L\'{e}vy noise.
Adversarial Networks for Camera Pose Regression and Refinement
Authors: Mai Bui, Christoph Baur, Nassir Navab, Slobodan Ilic, Shadi Albarqouni
Date: 2019-03-15 00:00:00
Tasks: Pose Estimation, regression

Despite recent advances on the topic of direct camera pose regression using neural networks, accurately estimating the camera pose of a single RGB image still remains a challenging task. To address this problem, we introduce a novel framework based, in its core, on the idea of implicitly learning the joint distribution of RGB images and their corresponding camera poses using a discriminator network and adversarial learning. Our method allows not only to regress the camera pose from a single image, however, also offers a solely RGB-based solution for camera pose refinement using the discriminator network. Further, we show that our method can effectively be used to optimize the predicted camera poses and thus improve the localization accuracy. To this end, we validate our proposed method on the publicly available 7-Scenes dataset improving upon the results of direct camera pose regression methods.
On Evaluation of Adversarial Perturbations for Sequence-to-Sequence Models
Authors: Paul Michel, Xi-An Li, Graham Neubig, Juan Miguel Pino
Date: 2019-03-15 00:00:00
Proceeding: NAACL 2019 6
Tasks: Adversarial Robustness, Machine Translation

Adversarial examples --- perturbations to the input of a model that elicit
large changes in the output --- have been shown to be an effective way of
assessing the robustness of sequence-to-sequence (seq2seq) models. However,
these perturbations only indicate weaknesses in the model if they do not change
the input so significantly that it legitimately results in changes in the
expected output. This fact has largely been ignored in the evaluations of the
growing body of related literature. Using the example of untargeted attacks on
machine translation (MT), we propose a new evaluation framework for adversarial
attacks on seq2seq models that takes the semantic equivalence of the pre- and
post-perturbation input into account. Using this framework, we demonstrate that
existing methods may not preserve meaning in general, breaking the
aforementioned assumption that source side perturbations should not result in
changes in the expected output. We further use this framework to demonstrate
that adding additional constraints on attacks allows for adversarial
perturbations that are more meaning-preserving, but nonetheless largely change
the output sequence. Finally, we show that performing untargeted adversarial
training with meaning-preserving attacks is beneficial to the model in terms of
adversarial robustness, without hurting test performance. A toolkit
implementing our evaluation framework is released at
https://github.com/pmichel31415/teapot-nlp.
Matching Entities Across Different Knowledge Graphs with Graph Embeddings
Authors: Michael Azmy, Peng Shi, Jimmy Lin, Ihab F. Ilyas
Date: 2019-03-15 00:00:00
Tasks: General Classification, Knowledge Graphs

This paper explores the problem of matching entities across different
knowledge graphs. Given a query entity in one knowledge graph, we wish to find
the corresponding real-world entity in another knowledge graph. We formalize
this problem and present two large-scale datasets for this task based on
exiting cross-ontology links between DBpedia and Wikidata, focused on several
hundred thousand ambiguous entities. Using a classification-based approach, we
find that a simple multi-layered perceptron based on representations derived
from RDF2Vec graph embeddings of entities in each knowledge graph is sufficient
to achieve high accuracy, with only small amounts of training data. The
contributions of our work are datasets for examining this problem and strong
baselines on which future work can be based.
On Certifying Non-uniform Bound against Adversarial Attacks
Authors: Chen Liu, Ryota Tomioka, Volkan Cevher
Date: 2019-03-15 00:00:00

This work studies the robustness certification problem of neural network models, which aims to find certified adversary-free regions as large as possible around data points. In contrast to the existing approaches that seek regions bounded uniformly along all input features, we consider non-uniform bounds and use it to study the decision boundary of neural network models. We formulate our target as an optimization problem with nonlinear constraints. Then, a framework applicable for general feedforward neural networks is proposed to bound the output logits so that the relaxed problem can be solved by the augmented Lagrangian method. Our experiments show the non-uniform bounds have larger volumes than uniform ones and the geometric similarity of the non-uniform bounds gives a quantitative, data-agnostic metric of input features' robustness. Further, compared with normal models, the robust models have even larger non-uniform bounds and better interpretability.
Deep Neural Network Ensembles for Time Series Classification
Authors: Hassan Ismail Fawaz, Germain Forestier, Jonathan Weber, Lhassane Idoumghar, Pierre-Alain Muller
Date: 2019-03-15 00:00:00
Tasks: Attribute, Classification, Deep Learning, General Classification, Time Series, Time Series Analysis, Time Series Classification

Deep neural networks have revolutionized many fields such as computer vision
and natural language processing. Inspired by this recent success, deep learning
started to show promising results for Time Series Classification (TSC).
However, neural networks are still behind the state-of-the-art TSC algorithms,
that are currently composed of ensembles of 37 non deep learning based
classifiers. We attribute this gap in performance due to the lack of neural
network ensembles for TSC. Therefore in this paper, we show how an ensemble of
60 deep learning models can significantly improve upon the current
state-of-the-art performance of neural networks for TSC, when evaluated over
the UCR/UEA archive: the largest publicly available benchmark for time series
analysis. Finally, we show how our proposed Neural Network Ensemble (NNE) is
the first time series classifier to outperform COTE while reaching similar
performance to the current state-of-the-art ensemble HIVE-COTE.
PifPaf: Composite Fields for Human Pose Estimation
Authors: Sven Kreiss, Lorenzo Bertoni, Alexandre Alahi
Date: 2019-03-15 00:00:00
Proceeding: CVPR 2019 6
Tasks: 2D Human Pose Estimation, Keypoint Detection, Multi-Person Pose Estimation, Pose Estimation, Self-Driving Cars

We propose a new bottom-up method for multi-person 2D human pose estimation
that is particularly well suited for urban mobility such as self-driving cars
and delivery robots. The new method, PifPaf, uses a Part Intensity Field (PIF)
to localize body parts and a Part Association Field (PAF) to associate body
parts with each other to form full human poses. Our method outperforms previous
methods at low resolution and in crowded, cluttered and occluded scenes thanks
to (i) our new composite field PAF encoding fine-grained information and (ii)
the choice of Laplace loss for regressions which incorporates a notion of
uncertainty. Our architecture is based on a fully convolutional, single-shot,
box-free design. We perform on par with the existing state-of-the-art bottom-up
method on the standard COCO keypoint task and produce state-of-the-art results
on a modified COCO keypoint task for the transportation domain.
Policy Distillation and Value Matching in Multiagent Reinforcement Learning
Authors: Samir Wadhwania, Dong-Ki Kim, Shayegan Omidshafiei, Jonathan P. How
Date: 2019-03-15 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Multiagent reinforcement learning algorithms (MARL) have been demonstrated on
complex tasks that require the coordination of a team of multiple agents to
complete. Existing works have focused on sharing information between agents via
centralized critics to stabilize learning or through communication to increase
performance, but do not generally look at how information can be shared between
agents to address the curse of dimensionality in MARL. We posit that a
multiagent problem can be decomposed into a multi-task problem where each agent
explores a subset of the state space instead of exploring the entire state
space. This paper introduces a multiagent actor-critic algorithm and method for
combining knowledge from homogeneous agents through distillation and
value-matching that outperforms policy distillation alone and allows further
learning in both discrete and continuous action spaces.
A nonasymptotic law of iterated logarithm for general M-estimators
Authors: Victor-Emmanuel Brunel, Arnak S. Dalalyan, Nicolas Schreuder
Date: 2019-03-15 00:00:00
Tasks: BIG-bench Machine Learning, Learning Theory

M-estimators are ubiquitous in machine learning and statistical learning theory. They are used both for defining prediction strategies and for evaluating their precision. In this paper, we propose the first non-asymptotic "any-time" deviation bounds for general M-estimators, where "any-time" means that the bound holds with a prescribed probability for every sample size. These bounds are nonasymptotic versions of the law of iterated logarithm. They are established under general assumptions such as Lipschitz continuity of the loss function and (local) curvature of the population risk. These conditions are satisfied for most examples used in machine learning, including those ensuring robustness to outliers and to heavy tailed distributions. As an example of application, we consider the problem of best arm identification in a parametric stochastic multi-arm bandit setting. We show that the established bound can be converted into a new algorithm, with provably optimal theoretical guarantees. Numerical experiments illustrating the validity of the algorithm are reported.
Multi-label Cloud Segmentation Using a Deep Network
Authors: Soumyabrata Dev, Shilpa Manandhar, Yee Hui Lee, Stefan Winkler
Date: 2019-03-15 00:00:00
Tasks: Cloud Detection, Image Segmentation, Segmentation, Semantic Segmentation

Different empirical models have been developed for cloud detection. There is
a growing interest in using the ground-based sky/cloud images for this purpose.
Several methods exist that perform binary segmentation of clouds. In this
paper, we propose to use a deep learning architecture (U-Net) to perform
multi-label sky/cloud image segmentation. The proposed approach outperforms
recent literature by a large margin.
GolfDB: A Video Database for Golf Swing Sequencing
Authors: William McNally, Kanav Vats, Tyler Pinto, Chris Dulhanty, John McPhee, Alexander Wong
Date: 2019-03-15 00:00:00

The golf swing is a complex movement requiring considerable full-body
coordination to execute proficiently. As such, it is the subject of frequent
scrutiny and extensive biomechanical analyses. In this paper, we introduce the
notion of golf swing sequencing for detecting key events in the golf swing and
facilitating golf swing analysis. To enable consistent evaluation of golf swing
sequencing performance, we also introduce the benchmark database GolfDB,
consisting of 1400 high-quality golf swing videos, each labeled with event
frames, bounding box, player name and sex, club type, and view type.
Furthermore, to act as a reference baseline for evaluating golf swing
sequencing performance on GolfDB, we propose a lightweight deep neural network
called SwingNet, which possesses a hybrid deep convolutional and recurrent
neural network architecture. SwingNet correctly detects eight golf swing events
at an average rate of 76.1%, and six out of eight events at a rate of 91.8%. In
line with the proposed baseline SwingNet, we advocate the use of
computationally efficient models in future research to promote in-the-field
analysis via deployment on readily-available mobile devices.
Phenotypic Profiling of High Throughput Imaging Screens with Generic Deep Convolutional Features
Authors: Philip T. Jackson, Yinhai Wang, Sinead Knight, Hongming Chen, Thierry Dorval, Martin Brown, Claus Bendtsen, Boguslaw Obara
Date: 2019-03-15 00:00:00
Tasks: Clustering, Drug Discovery

While deep learning has seen many recent applications to drug discovery, most
have focused on predicting activity or toxicity directly from chemical
structure. Phenotypic changes exhibited in cellular images are also indications
of the mechanism of action (MoA) of chemical compounds. In this paper, we show
how pre-trained convolutional image features can be used to assist scientists
in discovering interesting chemical clusters for further investigation. Our
method reduces the dimensionality of raw fluorescent stained images from a high
throughput imaging (HTI) screen, producing an embedding space that groups
together images with similar cellular phenotypes. Running standard unsupervised
clustering on this embedding space yields a set of distinct phenotypic
clusters. This allows scientists to further select and focus on interesting
clusters for downstream analyses. We validate the consistency of our embedding
space qualitatively with t-sne visualizations, and quantitatively by measuring
embedding variance among images that are known to be similar. Results suggested
the usefulness of our proposed workflow using deep learning and clustering and
it can lead to robust HTI screening and compound triage.
A Ranking Model Motivated by Nonnegative Matrix Factorization with Applications to Tennis Tournaments
Authors: Rui Xia, Vincent Y. F. Tan, Louis Filstroff, Cédric Févotte
Date: 2019-03-15 00:00:00

We propose a novel ranking model that combines the Bradley-Terry-Luce probability model with a nonnegative matrix factorization framework to model and uncover the presence of latent variables that influence the performance of top tennis players. We derive an efficient, provably convergent, and numerically stable majorization-minimization-based algorithm to maximize the likelihood of datasets under the proposed statistical model. The model is tested on datasets involving the outcomes of matches between 20 top male and female tennis players over 14 major tournaments for men (including the Grand Slams and the ATP Masters 1000) and 16 major tournaments for women over the past 10 years. Our model automatically infers that the surface of the court (e.g., clay or hard court) is a key determinant of the performances of male players, but less so for females. Top players on various surfaces over this longitudinal period are also identified in an objective manner.
Content Differences in Syntactic and Semantic Representations
Authors: Daniel Hershcovich, Omri Abend, Ari Rappoport
Date: 2019-03-15 00:00:00
Tasks: Semantic Parsing, UCCA Parsing

Syntactic analysis plays an important role in semantic parsing, but the
nature of this role remains a topic of ongoing debate. The debate has been
constrained by the scarcity of empirical comparative studies between syntactic
and semantic schemes, which hinders the development of parsing methods informed
by the details of target schemes and constructions. We target this gap, and
take Universal Dependencies (UD) and UCCA as a test case. After abstracting
away from differences of convention or formalism, we find that most content
divergences can be ascribed to: (1) UCCA's distinction between a Scene and a
non-Scene; (2) UCCA's distinction between primary relations, secondary ones and
participants; (3) different treatment of multi-word expressions, and (4)
different treatment of inter-clause linkage. We further discuss the long tail
of cases where the two schemes take markedly different approaches. Finally, we
show that the proposed comparison methodology can be used for fine-grained
evaluation of UCCA parsing, highlighting both challenges and potential sources
for improvement. The substantial differences between the schemes suggest that
semantic parsers are likely to benefit downstream text understanding
applications beyond their syntactic counterparts.