pwc-archive/papers-with-abstracts huggingface dataset:
Coordinate Dual Averaging for Decentralized Online Optimization with Nonseparable Global Objectives
Authors: Soomin Lee, Angelia Nedić, Maxim Raginsky
Date: 2015-08-31 00:00:00

We consider a decentralized online convex optimization problem in a network
of agents, where each agent controls only a coordinate (or a part) of the
global decision vector. For such a problem, we propose two decentralized
variants (ODA-C and ODA-PS) of Nesterov's primal-dual algorithm with dual
averaging. In ODA-C, to mitigate the disagreements on the primal-vector
updates, the agents implement a generalization of the local
information-exchange dynamics recently proposed by Li and Marden over a static
undirected graph. In ODA-PS, the agents implement the broadcast-based push-sum
dynamics over a time-varying sequence of uniformly connected digraphs. We show
that the regret bounds in both cases have sublinear growth of $O(\sqrt{T})$,
with the time horizon $T$, when the stepsize is of the form $1/\sqrt{t}$ and
the objective functions are Lipschitz-continuous convex functions with
Lipschitz gradients. We also implement the proposed algorithms on a sensor
network to complement our theoretical analysis.
Linear-memory and Decomposition-invariant Linearly Convergent Conditional Gradient Algorithm for Structured Polytopes
Authors: Dan Garber, Ofer Meshi
Date: 2016-05-20 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Structured Prediction

Recently, several works have shown that natural modifications of the
classical conditional gradient method (aka Frank-Wolfe algorithm) for
constrained convex optimization, provably converge with a linear rate when: i)
the feasible set is a polytope, and ii) the objective is smooth and
strongly-convex. However, all of these results suffer from two significant
shortcomings: large memory requirement due to the need to store an explicit
convex decomposition of the current iterate, and as a consequence, large
running-time overhead per iteration, and worst case convergence rate that
depends unfavorably on the dimension.
  In this work we present a new conditional gradient variant and a
corresponding analysis that improves on both of the above shortcomings. In
particular: both memory and computation overheads are only linear in the
dimension. Moreover, in case the optimal solution is sparse, the new
convergence rate replaces a factor which is at least linear in the dimension in
previous works, with a linear dependence on the number of non-zeros in the
optimal solution.
  At the heart of our method, and corresponding analysis, is a novel way to
compute decomposition-invariant away-steps. While our theoretical guarantees do
not apply to any polytope, they apply to several important structured polytopes
that capture central concepts such as paths in graphs, perfect matchings in
bipartite graphs, marginal distributions that arise in structured prediction
tasks, and more. Our theoretical findings are complemented by empirical
evidence which shows that our method delivers state-of-the-art performance.
X-ray image separation via coupled dictionary learning
Authors: Nikos Deligiannis, João F. C. Mota, Bruno Cornelis, Miguel R. D. Rodrigues, Ingrid Daubechies
Date: 2016-05-20 00:00:00
Tasks: Dictionary Learning

In support of art investigation, we propose a new source sepa- ration method
that unmixes a single X-ray scan acquired from double-sided paintings. Unlike
prior source separation meth- ods, which are based on statistical or structural
incoherence of the sources, we use visual images taken from the front- and
back-side of the panel to drive the separation process. The coupling of the two
imaging modalities is achieved via a new multi-scale dictionary learning
method. Experimental results demonstrate that our method succeeds in the
discrimination of the sources, while state-of-the-art methods fail to do so.
Anomaly Detection in XML-Structured SOAP Messages Using Tree-Based Association Rule Mining
Authors: Reyhaneh Ghassem Esfahani, Mohammad Abadollahi Azgomi, Reza Fathi
Date: 2016-05-20 00:00:00
Tasks: Anomaly Detection

Web services are software systems designed for supporting interoperable
dynamic cross-enterprise interactions. The result of attacks to Web services
can be catastrophic and causing the disclosure of enterprises' confidential
data. As new approaches of attacking arise every day, anomaly detection systems
seem to be invaluable tools in this context. The aim of this work has been to
target the attacks that reside in the Web service layer and the extensible
markup language (XML)-structured simple object access protocol (SOAP) messages.
After studying the shortcomings of the existing solutions, a new approach for
detecting anomalies in Web services is outlined. More specifically, the
proposed technique illustrates how to identify anomalies by employing mining
methods on XML-structured SOAP messages. This technique also takes the
advantages of tree-based association rule mining to extract knowledge in the
training phase, which is used in the test phase to detect anomalies. In
addition, this novel composition of techniques brings nearly low false alarm
rate while maintaining the detection rate reasonably high, which is shown by a
case study.
Swapout: Learning an ensemble of deep architectures
Authors: Saurabh Singh, Derek Hoiem, David Forsyth
Date: 2016-05-20 00:00:00
Proceeding: NeurIPS 2016 12

We describe Swapout, a new stochastic training method, that outperforms
ResNets of identical network structure yielding impressive results on CIFAR-10
and CIFAR-100. Swapout samples from a rich set of architectures including
dropout, stochastic depth and residual architectures as special cases. When
viewed as a regularization method swapout not only inhibits co-adaptation of
units in a layer, similar to dropout, but also across network layers. We
conjecture that swapout achieves strong regularization by implicitly tying the
parameters across layers. When viewed as an ensemble training method, it
samples a much richer set of architectures than existing methods such as
dropout or stochastic depth. We propose a parameterization that reveals
connections to exiting architectures and suggests a much richer set of
architectures to be explored. We show that our formulation suggests an
efficient training method and validate our conclusions on CIFAR-10 and
CIFAR-100 matching state of the art accuracy. Remarkably, our 32 layer wider
model performs similar to a 1001 layer ResNet model.
Virtual Worlds as Proxy for Multi-Object Tracking Analysis
Authors: Adrien Gaidon, Qiao Wang, Yohann Cabon, Eleonora Vig
Date: 2016-05-20 00:00:00
Proceeding: CVPR 2016 6
Tasks: Instance Segmentation, Multi-Object Tracking, Object, object-detection, Object Detection, Object Tracking, Optical Flow Estimation, Semantic Segmentation

Modern computer vision algorithms typically require expensive data
acquisition and accurate manual labeling. In this work, we instead leverage the
recent progress in computer graphics to generate fully labeled, dynamic, and
photo-realistic proxy virtual worlds. We propose an efficient real-to-virtual
world cloning method, and validate our approach by building and publicly
releasing a new video dataset, called Virtual KITTI (see
http://www.xrce.xerox.com/Research-Development/Computer-Vision/Proxy-Virtual-Worlds),
automatically labeled with accurate ground truth for object detection,
tracking, scene and instance segmentation, depth, and optical flow. We provide
quantitative experimental evidence suggesting that (i) modern deep learning
algorithms pre-trained on real data behave similarly in real and virtual
worlds, and (ii) pre-training on virtual data improves performance. As the gap
between real and virtual worlds is small, virtual worlds enable measuring the
impact of various weather and imaging conditions on recognition performance,
all other things being equal. We show these factors may affect drastically
otherwise high-performing deep models for tracking.
Query-Efficient Imitation Learning for End-to-End Autonomous Driving
Authors: Jiakai Zhang, Kyunghyun Cho
Date: 2016-05-20 00:00:00
Tasks: Autonomous Driving, Car Racing, Imitation Learning

One way to approach end-to-end autonomous driving is to learn a policy
function that maps from a sensory input, such as an image frame from a
front-facing camera, to a driving action, by imitating an expert driver, or a
reference policy. This can be done by supervised learning, where a policy
function is tuned to minimize the difference between the predicted and
ground-truth actions. A policy function trained in this way however is known to
suffer from unexpected behaviours due to the mismatch between the states
reachable by the reference policy and trained policy functions. More advanced
algorithms for imitation learning, such as DAgger, addresses this issue by
iteratively collecting training examples from both reference and trained
policies. These algorithms often requires a large number of queries to a
reference policy, which is undesirable as the reference policy is often
expensive. In this paper, we propose an extension of the DAgger, called
SafeDAgger, that is query-efficient and more suitable for end-to-end autonomous
driving. We evaluate the proposed SafeDAgger in a car racing simulator and show
that it indeed requires less queries to a reference policy. We observe a
significant speed up in convergence, which we conjecture to be due to the
effect of automated curriculum learning.
ATD: Anomalous Topic Discovery in High Dimensional Discrete Data
Authors: Hossein Soleimani, David J. Miller
Date: 2015-12-20 00:00:00
Tasks: Anomaly Detection, Topic Models, Vocal Bursts Intensity Prediction

We propose an algorithm for detecting patterns exhibited by anomalous
clusters in high dimensional discrete data. Unlike most anomaly detection (AD)
methods, which detect individual anomalies, our proposed method detects groups
(clusters) of anomalies; i.e. sets of points which collectively exhibit
abnormal patterns. In many applications this can lead to better understanding
of the nature of the atypical behavior and to identifying the sources of the
anomalies. Moreover, we consider the case where the atypical patterns exhibit
on only a small (salient) subset of the very high dimensional feature space.
Individual AD techniques and techniques that detect anomalies using all the
features typically fail to detect such anomalies, but our method can detect
such instances collectively, discover the shared anomalous patterns exhibited
by them, and identify the subsets of salient features. In this paper, we focus
on detecting anomalous topics in a batch of text documents, developing our
algorithm based on topic models. Results of our experiments show that our
method can accurately detect anomalous topics and salient features (words)
under each such topic in a synthetic data set and two real-world text corpora
and achieves better performance compared to both standard group AD and
individual AD techniques. All required code to reproduce our experiments is
available from https://github.com/hsoleimani/ATD
Combining Adversarial Guarantees and Stochastic Fast Rates in Online Learning
Authors: Wouter M. Koolen, Peter Grünwald, Tim van Erven
Date: 2016-05-20 00:00:00
Proceeding: NeurIPS 2016 12

We consider online learning algorithms that guarantee worst-case regret rates
in adversarial environments (so they can be deployed safely and will perform
robustly), yet adapt optimally to favorable stochastic environments (so they
will perform well in a variety of settings of practical importance). We
quantify the friendliness of stochastic environments by means of the well-known
Bernstein (a.k.a. generalized Tsybakov margin) condition. For two recent
algorithms (Squint for the Hedge setting and MetaGrad for online convex
optimization) we show that the particular form of their data-dependent
individual-sequence regret guarantees implies that they adapt automatically to
the Bernstein parameters of the stochastic environment. We prove that these
algorithms attain fast rates in their respective settings both in expectation
and with high probability.
Optimal Black-Box Reductions Between Optimization Objectives
Authors: Zeyuan Allen-Zhu, Elad Hazan
Date: 2016-03-17 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: BIG-bench Machine Learning, General Classification, regression

The diverse world of machine learning applications has given rise to a
plethora of algorithms and optimization methods, finely tuned to the specific
regression or classification task at hand. We reduce the complexity of
algorithm design for machine learning by reductions: we develop reductions that
take a method developed for one setting and apply it to the entire spectrum of
smoothness and strong-convexity in applications.
  Furthermore, unlike existing results, our new reductions are OPTIMAL and more
PRACTICAL. We show how these new reductions give rise to new and faster running
times on training linear classifiers for various families of loss functions,
and conclude with experiments showing their successes also in practice.
Learning shape correspondence with anisotropic convolutional neural networks
Authors: Davide Boscaini, Jonathan Masci, Emanuele Rodolà, Michael M. Bronstein
Date: 2016-05-20 00:00:00
Proceeding: NeurIPS 2016 12

Establishing correspondence between shapes is a fundamental problem in
geometry processing, arising in a wide variety of applications. The problem is
especially difficult in the setting of non-isometric deformations, as well as
in the presence of topological noise and missing parts, mainly due to the
limited capability to model such deformations axiomatically. Several recent
works showed that invariance to complex shape transformations can be learned
from examples. In this paper, we introduce an intrinsic convolutional neural
network architecture based on anisotropic diffusion kernels, which we term
Anisotropic Convolutional Neural Network (ACNN). In our construction, we
generalize convolutions to non-Euclidean domains by constructing a set of
oriented anisotropic diffusion kernels, creating in this way a local intrinsic
polar representation of the data (`patch'), which is then correlated with a
filter. Several cascades of such filters, linear, and non-linear operators are
stacked to form a deep neural network whose parameters are learned by
minimizing a task-specific cost. We use ACNNs to effectively learn intrinsic
dense correspondences between deformable shapes in very challenging settings,
achieving state-of-the-art results on some of the most difficult recent
correspondence benchmarks.
An unsupervised spatiotemporal graphical modeling approach to anomaly detection in distributed CPS
Authors: Chao Liu, Sambuddha Ghosal, Zhanhong Jiang, Soumik Sarkar
Date: 2015-12-24 00:00:00
Tasks: Anomaly Detection

Modern distributed cyber-physical systems (CPSs) encounter a large variety of
physical faults and cyber anomalies and in many cases, they are vulnerable to
catastrophic fault propagation scenarios due to strong connectivity among the
sub-systems. This paper presents a new data-driven framework for system-wide
anomaly detection for addressing such issues. The framework is based on a
spatiotemporal feature extraction scheme built on the concept of symbolic
dynamics for discovering and representing causal interactions among the
subsystems of a CPS. The extracted spatiotemporal features are then used to
learn system-wide patterns via a Restricted Boltzmann Machine (RBM). The
results show that: (1) the RBM free energy in the off-nominal conditions is
different from that in the nominal conditions and can be used for anomaly
detection; (2) the framework can capture multiple nominal modes with one
graphical model; (3) the case studies with simulated data and an integrated
building system validate the proposed approach.
End-to-End Goal-Driven Web Navigation
Authors: Rodrigo Nogueira, Kyunghyun Cho
Date: 2016-02-06 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Decision Making, Question Answering, Sequential Decision Making

We propose a goal-driven web navigation as a benchmark task for evaluating an
agent with abilities to understand natural language and plan on partially
observed environments. In this challenging task, an agent navigates through a
website, which is represented as a graph consisting of web pages as nodes and
hyperlinks as directed edges, to find a web page in which a query appears. The
agent is required to have sophisticated high-level reasoning based on natural
languages and efficient sequential decision-making capability to succeed. We
release a software tool, called WebNav, that automatically transforms a website
into this goal-driven web navigation task, and as an example, we make WikiNav,
a dataset constructed from the English Wikipedia. We extensively evaluate
different variants of neural net based artificial agents on WikiNav and observe
that the proposed goal-driven web navigation well reflects the advances in
models, making it a suitable benchmark for evaluating future progress.
Furthermore, we extend the WikiNav with question-answer pairs from Jeopardy!
and test the proposed agent based on recurrent neural networks against strong
inverted index based search engines. The artificial agents trained on WikiNav
outperforms the engined based approaches, demonstrating the capability of the
proposed goal-driven navigation as a good proxy for measuring the progress in
real-world tasks such as focused crawling and question-answering.
Shape Recognition by Bag of Skeleton-associated Contour Parts
Authors: Wei Shen, Yuan Jiang, Wenjing Gao, Dan Zeng, Xinggang Wang
Date: 2016-05-20 00:00:00

Contour and skeleton are two complementary representations for shape
recognition. However combining them in a principal way is nontrivial, as they
are generally abstracted by different structures (closed string vs graph),
respectively. This paper aims at addressing the shape recognition problem by
combining contour and skeleton according to the correspondence between them.
The correspondence provides a straightforward way to associate skeletal
information with a shape contour. More specifically, we propose a new shape
descriptor. named Skeleton-associated Shape Context (SSC), which captures the
features of a contour fragment associated with skeletal information. Benefited
from the association, the proposed shape descriptor provides the complementary
geometric information from both contour and skeleton parts, including the
spatial distribution and the thickness change along the shape part. To form a
meaningful shape feature vector for an overall shape, the Bag of Features
framework is applied to the SSC descriptors extracted from it. Finally, the
shape feature vector is fed into a linear SVM classifier to recognize the
shape. The encouraging experimental results demonstrate that the proposed way
to combine contour and skeleton is effective for shape recognition, which
achieves the state-of-the-art performances on several standard shape
benchmarks.
Ristretto: Hardware-Oriented Approximation of Convolutional Neural Networks
Authors: Philipp Gysel
Date: 2016-05-20 00:00:00
Tasks: GPU

Convolutional neural networks (CNN) have achieved major breakthroughs in
recent years. Their performance in computer vision have matched and in some
areas even surpassed human capabilities. Deep neural networks can capture
complex non-linear features; however this ability comes at the cost of high
computational and memory requirements. State-of-art networks require billions
of arithmetic operations and millions of parameters. To enable embedded devices
such as smartphones, Google glasses and monitoring cameras with the astonishing
power of deep learning, dedicated hardware accelerators can be used to decrease
both execution time and power consumption. In applications where fast
connection to the cloud is not guaranteed or where privacy is important,
computation needs to be done locally. Many hardware accelerators for deep
neural networks have been proposed recently. A first important step of
accelerator design is hardware-oriented approximation of deep networks, which
enables energy-efficient inference. We present Ristretto, a fast and automated
framework for CNN approximation. Ristretto simulates the hardware arithmetic of
a custom hardware accelerator. The framework reduces the bit-width of network
parameters and outputs of resource-intense layers, which reduces the chip area
for multiplication units significantly. Alternatively, Ristretto can remove the
need for multipliers altogether, resulting in an adder-only arithmetic. The
tool fine-tunes trimmed networks to achieve high classification accuracy. Since
training of deep neural networks can be time-consuming, Ristretto uses highly
optimized routines which run on the GPU. This enables fast compression of any
given network. Given a maximum tolerance of 1%, Ristretto can successfully
condense CaffeNet and SqueezeNet to 8-bit. The code for Ristretto is available.
Bayesian Hyperparameter Optimization for Ensemble Learning
Authors: Julien-Charles Lévesque, Christian Gagné, Robert Sabourin
Date: 2016-05-20 00:00:00
Tasks: Bayesian Optimization, Ensemble Learning, Hyperparameter Optimization

In this paper, we bridge the gap between hyperparameter optimization and
ensemble learning by performing Bayesian optimization of an ensemble with
regards to its hyperparameters. Our method consists in building a fixed-size
ensemble, optimizing the configuration of one classifier of the ensemble at
each iteration of the hyperparameter optimization algorithm, taking into
consideration the interaction with the other models when evaluating potential
performances. We also consider the case where the ensemble is to be
reconstructed at the end of the hyperparameter optimization phase, through a
greedy selection over the pool of models generated during the optimization. We
study the performance of our proposed method on three different hyperparameter
spaces, showing that our approach is better than both the best single model and
a greedy ensemble construction over the models produced by a standard Bayesian
optimization.
Towards Automation of Knowledge Understanding: An Approach for Probabilistic Generative Classifiers
Authors: Dominik Fisch, Christian Gruhl, Edgar Kalkowski, Bernhard Sick, Seppo J. Ovaska
Date: 2016-05-20 00:00:00
Tasks: Informativeness

After data selection, pre-processing, transformation, and feature extraction,
knowledge extraction is not the final step in a data mining process. It is then
necessary to understand this knowledge in order to apply it efficiently and
effectively. Up to now, there is a lack of appropriate techniques that support
this significant step. This is partly due to the fact that the assessment of
knowledge is often highly subjective, e.g., regarding aspects such as novelty
or usefulness. These aspects depend on the specific knowledge and requirements
of the data miner. There are, however, a number of aspects that are objective
and for which it is possible to provide appropriate measures. In this article
we focus on classification problems and use probabilistic generative
classifiers based on mixture density models that are quite common in data
mining applications. We define objective measures to assess the
informativeness, uniqueness, importance, discrimination, representativity,
uncertainty, and distinguishability of rules contained in these classifiers
numerically. These measures not only support a data miner in evaluating results
of a data mining process based on such classifiers. As we will see in
illustrative case studies, they may also be used to improve the data mining
process itself or to support the later application of the extracted knowledge.