pwc-archive/papers-with-abstracts huggingface dataset:
Learning Representations in Model-Free Hierarchical Reinforcement Learning
Authors: Jacob Rafati, David C. Noelle
Date: 2018-10-23 00:00:00
Tasks: Hierarchical Reinforcement Learning, model, Montezuma's Revenge, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Common approaches to Reinforcement Learning (RL) are seriously challenged by
large-scale applications involving huge state spaces and sparse delayed reward
feedback. Hierarchical Reinforcement Learning (HRL) methods attempt to address
this scalability issue by learning action selection policies at multiple levels
of temporal abstraction. Abstraction can be had by identifying a relatively
small set of states that are likely to be useful as subgoals, in concert with
the learning of corresponding skill policies to achieve those subgoals. Many
approaches to subgoal discovery in HRL depend on the analysis of a model of the
environment, but the need to learn such a model introduces its own problems of
scale. Once subgoals are identified, skills may be learned through intrinsic
motivation, introducing an internal reward signal marking subgoal attainment.
In this paper, we present a novel model-free method for subgoal discovery using
incremental unsupervised learning over a small memory of the most recent
experiences (trajectories) of the agent. When combined with an intrinsic
motivation learning mechanism, this method learns both subgoals and skills,
based on experiences in the environment. Thus, we offer an original approach to
HRL that does not require the acquisition of a model of the environment,
suitable for large-scale applications. We demonstrate the efficiency of our
method on two RL problems with sparse delayed feedback: a variant of the rooms
environment and the first screen of the ATARI 2600 Montezuma's Revenge game.
Structured Domain Randomization: Bridging the Reality Gap by Context-Aware Synthetic Data
Authors: Aayush Prakash, Shaad Boochoon, Mark Brophy, David Acuna, Eric Cameracci, Gavriel State, Omer Shapira, Stan Birchfield
Date: 2018-10-23 00:00:00

We present structured domain randomization (SDR), a variant of domain randomization (DR) that takes into account the structure and context of the scene. In contrast to DR, which places objects and distractors randomly according to a uniform probability distribution, SDR places objects and distractors randomly according to probability distributions that arise from the specific problem at hand. In this manner, SDR-generated imagery enables the neural network to take the context around an object into consideration during detection. We demonstrate the power of SDR for the problem of 2D bounding box car detection, achieving competitive results on real data after training only on synthetic data. On the KITTI easy, moderate, and hard tasks, we show that SDR outperforms other approaches to generating synthetic data (VKITTI, Sim 200k, or DR), as well as real data collected in a different domain (BDD100K). Moreover, synthetic SDR data combined with real KITTI data outperforms real KITTI data alone.
NestDNN: Resource-Aware Multi-Tenant On-Device Deep Learning for Continuous Mobile Vision
Authors: Biyi Fang, Xiao Zeng, Mi Zhang
Date: 2018-10-23 00:00:00
Tasks: Deep Learning

Mobile vision systems such as smartphones, drones, and augmented-reality
headsets are revolutionizing our lives. These systems usually run multiple
applications concurrently and their available resources at runtime are dynamic
due to events such as starting new applications, closing existing applications,
and application priority changes. In this paper, we present NestDNN, a
framework that takes the dynamics of runtime resources into account to enable
resource-aware multi-tenant on-device deep learning for mobile vision systems.
NestDNN enables each deep learning model to offer flexible resource-accuracy
trade-offs. At runtime, it dynamically selects the optimal resource-accuracy
trade-off for each deep learning model to fit the model's resource demand to
the system's available runtime resources. In doing so, NestDNN efficiently
utilizes the limited resources in mobile vision systems to jointly maximize the
performance of all the concurrently running applications. Our experiments show
that compared to the resource-agnostic status quo approach, NestDNN achieves as
much as 4.2% increase in inference accuracy, 2.0x increase in video frame
processing rate and 1.7x reduction on energy consumption.
A Continuous-Time View of Early Stopping for Least Squares
Authors: Alnur Ali, J. Zico Kolter, Ryan J. Tibshirani
Date: 2018-10-23 00:00:00
Tasks: regression

We study the statistical properties of the iterates generated by gradient
descent, applied to the fundamental problem of least squares regression. We
take a continuous-time view, i.e., consider infinitesimal step sizes in
gradient descent, in which case the iterates form a trajectory called gradient
flow. Our primary focus is to compare the risk of gradient flow to that of
ridge regression. Under the calibration $t=1/\lambda$---where $t$ is the time
parameter in gradient flow, and $\lambda$ the tuning parameter in ridge
regression---we prove that the risk of gradient flow is no less than 1.69 times
that of ridge, along the entire path (for all $t \geq 0$). This holds in finite
samples with very weak assumptions on the data model (in particular, with no
assumptions on the features $X$). We prove that the same relative risk bound
holds for prediction risk, in an average sense over the underlying signal
$\beta_0$. Finally, we examine limiting risk expressions (under standard
Marchenko-Pastur asymptotics), and give supporting numerical experiments.
Model Selection for Nonnegative Matrix Factorization by Support Union Recovery
Authors: Zhaoqiang Liu
Date: 2018-10-23 00:00:00
Tasks: Model Selection

Nonnegative matrix factorization (NMF) has been widely used in machine
learning and signal processing because of its non-subtractive, part-based
property which enhances interpretability. It is often assumed that the latent
dimensionality (or the number of components) is given. Despite the large amount
of algorithms designed for NMF, there is little literature about automatic
model selection for NMF with theoretical guarantees. In this paper, we propose
an algorithm that first calculates an empirical second-order moment from the
empirical fourth-order cumulant tensor, and then estimates the latent
dimensionality by recovering the support union (the index set of non-zero rows)
of a matrix related to the empirical second-order moment. By assuming a
generative model of the data with additional mild conditions, our algorithm
provably detects the true latent dimensionality. We show on synthetic examples
that our proposed algorithm is able to find an approximately correct number of
components.
A Statistical Approach to Adult Census Income Level Prediction
Authors: Navoneel Chakrabarty, Sanket Biswas
Date: 2018-10-23 00:00:00
Tasks: Prediction, valid

The prominent inequality of wealth and income is a huge concern especially in
the United States. The likelihood of diminishing poverty is one valid reason to
reduce the world's surging level of economic inequality. The principle of
universal moral equality ensures sustainable development and improve the
economic stability of a nation. Governments in different countries have been
trying their best to address this problem and provide an optimal solution. This
study aims to show the usage of machine learning and data mining techniques in
providing a solution to the income equality problem. The UCI Adult Dataset has
been used for the purpose. Classification has been done to predict whether a
person's yearly income in US falls in the income category of either greater
than 50K Dollars or less equal to 50K Dollars category based on a certain set
of attributes. The Gradient Boosting Classifier Model was deployed which
clocked the highest accuracy of 88.16%, eventually breaking the benchmark
accuracy of existing works.
A Fusion Approach for Multi-Frame Optical Flow Estimation
Authors: Zhile Ren, Orazio Gallo, Deqing Sun, Ming-Hsuan Yang, Erik B. Sudderth, Jan Kautz
Date: 2018-10-23 00:00:00
Tasks: Optical Flow Estimation

To date, top-performing optical flow estimation methods only take pairs of
consecutive frames into account. While elegant and appealing, the idea of using
more than two frames has not yet produced state-of-the-art results. We present
a simple, yet effective fusion approach for multi-frame optical flow that
benefits from longer-term temporal cues. Our method first warps the optical
flow from previous frames to the current, thereby yielding multiple plausible
estimates. It then fuses the complementary information carried by these
estimates into a new optical flow field. At the time of writing, our method
ranks first among published results in the MPI Sintel and KITTI 2015
benchmarks. Our models will be available on https://github.com/NVlabs/PWC-Net.
Statistical mechanics of low-rank tensor decomposition
Authors: Jonathan Kadmon, Surya Ganguli
Date: 2018-10-23 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Tensor Decomposition

Often, large, high dimensional datasets collected across multiple modalities
can be organized as a higher order tensor. Low-rank tensor decomposition then
arises as a powerful and widely used tool to discover simple low dimensional
structures underlying such data. However, we currently lack a theoretical
understanding of the algorithmic behavior of low-rank tensor decompositions. We
derive Bayesian approximate message passing (AMP) algorithms for recovering
arbitrarily shaped low-rank tensors buried within noise, and we employ dynamic
mean field theory to precisely characterize their performance. Our theory
reveals the existence of phase transitions between easy, hard and impossible
inference regimes, and displays an excellent match with simulations. Moreover,
it reveals several qualitative surprises compared to the behavior of symmetric,
cubic tensor decomposition. Finally, we compare our AMP algorithm to the most
commonly used algorithm, alternating least squares (ALS), and demonstrate that
AMP significantly outperforms ALS in the presence of noise.
Graph Laplacian mixture model
Authors: Hermina Petric Maretic, Pascal Frossard
Date: 2018-10-23 00:00:00
Tasks: Clustering, Graph Learning, model

Graph learning methods have recently been receiving increasing interest as means to infer structure in datasets. Most of the recent approaches focus on different relationships between a graph and data sample distributions, mostly in settings where all available data relate to the same graph. This is, however, not always the case, as data is often available in mixed form, yielding the need for methods that are able to cope with mixture data and learn multiple graphs. We propose a novel generative model that represents a collection of distinct data which naturally live on different graphs. We assume the mapping of data to graphs is not known and investigate the problem of jointly clustering a set of data and learning a graph for each of the clusters. Experiments demonstrate promising performance in data clustering and multiple graph inference, and show desirable properties in terms of interpretability and coping with high dimensionality on weather and traffic data, as well as digit classification.
Language Modeling at Scale
Authors: Mostofa Patwary, Milind Chabbi, Heewoo Jun, Jiaji Huang, Gregory Diamos, Kenneth Church
Date: 2018-10-23 00:00:00
Tasks: GPU, Language Modeling, Language Modelling, Machine Translation, speech-recognition, Speech Recognition

We show how Zipf's Law can be used to scale up language modeling (LM) to take
advantage of more training data and more GPUs. LM plays a key role in many
important natural language applications such as speech recognition and machine
translation. Scaling up LM is important since it is widely accepted by the
community that there is no data like more data. Eventually, we would like to
train on terabytes (TBs) of text (trillions of words). Modern training methods
are far from this goal, because of various bottlenecks, especially memory
(within GPUs) and communication (across GPUs). This paper shows how Zipf's Law
can address these bottlenecks by grouping parameters for common words and
character sequences, because $U \ll N$, where $U$ is the number of unique words
(types) and $N$ is the size of the training set (tokens). For a local batch
size $K$ with $G$ GPUs and a $D$-dimension embedding matrix, we reduce the
original per-GPU memory and communication asymptotic complexity from
$\Theta(GKD)$ to $\Theta(GK + UD)$. Empirically, we find $U \propto
(GK)^{0.64}$ on four publicly available large datasets. When we scale up the
number of GPUs to 64, a factor of 8, training time speeds up by factors up to
6.7$\times$ (for character LMs) and 6.3$\times$ (for word LMs) with negligible
loss of accuracy. Our weak scaling on 192 GPUs on the Tieba dataset shows a
35\% improvement in LM prediction accuracy by training on 93 GB of data
(2.5$\times$ larger than publicly available SOTA dataset), but taking only
1.25$\times$ increase in training time, compared to 3 GB of the same dataset
running on 6 GPUs.
Efficiently measuring a quantum device using machine learning
Authors: D. T. Lennon, H. Moon, L. C. Camenzind, Liuqi Yu, D. M. Zumbühl, G. A. D. Briggs, M. A. Osborne, E. A. Laird, N. Ares
Date: 2018-10-23 00:00:00
Tasks: BIG-bench Machine Learning

Scalable quantum technologies will present challenges for characterizing and
tuning quantum devices. This is a time-consuming activity, and as the size of
quantum systems increases, this task will become intractable without the aid of
automation. We present measurements on a quantum dot device performed by a
machine learning algorithm. The algorithm selects the most informative
measurements to perform next using information theory and a probabilistic
deep-generative model, the latter capable of generating multiple
full-resolution reconstructions from scattered partial measurements. We
demonstrate, for two different measurement configurations, that the algorithm
outperforms standard grid scan techniques, reducing the number of measurements
required by up to 4 times and the measurement time by 3.7 times. Our
contribution goes beyond the use of machine learning for data search and
analysis, and instead presents the use of algorithms to automate measurement.
This work lays the foundation for automated control of large quantum circuits.
DeepLSR: a deep learning approach for laser speckle reduction
Authors: Taylor L. Bobrow, Faisal Mahmood, Miguel Inserni, Nicholas J. Durr
Date: 2018-10-23 00:00:00
Tasks: Deep Learning

Speckle artifacts degrade image quality in virtually all modalities that
utilize coherent energy, including optical coherence tomography, reflectance
confocal microscopy, ultrasound, and widefield imaging with laser illumination.
We present an adversarial deep learning framework for laser speckle reduction,
called DeepLSR (https://durr.jhu.edu/DeepLSR), that transforms images from a
source domain of coherent illumination to a target domain of speckle-free,
incoherent illumination. We apply this method to widefield images of objects
and tissues illuminated with a multi-wavelength laser, using light emitting
diode-illuminated images as ground truth. In images of gastrointestinal
tissues, DeepLSR reduces laser speckle noise by 6.4 dB, compared to a 2.9 dB
reduction from optimized non-local means processing, a 3.0 dB reduction from
BM3D, and a 3.7 dB reduction from an optical speckle reducer utilizing an
oscillating diffuser. Further, DeepLSR can be combined with optical speckle
reduction to reduce speckle noise by 9.4 dB. This dramatic reduction in speckle
noise may enable the use of coherent light sources in applications that require
small illumination sources and high-quality imaging, including medical
endoscopy.
Negative results for approximation using single layer and multilayer feedforward neural networks
Authors: J. M. Almira, P. E. Lopez-de-Teruel, D. J. Romero-Lopez, F. Voigtlaender
Date: 2018-10-23 00:00:00

We prove a negative result for the approximation of functions defined on compact subsets of $\mathbb{R}^d$ (where $d \geq 2$) using feedforward neural networks with one hidden layer and arbitrary continuous activation function. In a nutshell, this result claims the existence of target functions that are as difficult to approximate using these neural networks as one may want. We also demonstrate an analogous result (for general $d \in \mathbb{N}$) for neural networks with an \emph{arbitrary} number of hidden layers, for activation functions that are either rational functions or continuous splines with finitely many pieces.
Stochastic Substitute Training: A Gray-box Approach to Craft Adversarial Examples Against Gradient Obfuscation Defenses
Authors: Mohammad Hashemi, Greg Cusack, Eric Keller
Date: 2018-10-23 00:00:00
Tasks: Blocking

It has been shown that adversaries can craft example inputs to neural
networks which are similar to legitimate inputs but have been created to
purposely cause the neural network to misclassify the input. These adversarial
examples are crafted, for example, by calculating gradients of a carefully
defined loss function with respect to the input. As a countermeasure, some
researchers have tried to design robust models by blocking or obfuscating
gradients, even in white-box settings. Another line of research proposes
introducing a separate detector to attempt to detect adversarial examples. This
approach also makes use of gradient obfuscation techniques, for example, to
prevent the adversary from trying to fool the detector. In this paper, we
introduce stochastic substitute training, a gray-box approach that can craft
adversarial examples for defenses which obfuscate gradients. For those defenses
that have tried to make models more robust, with our technique, an adversary
can craft adversarial examples with no knowledge of the defense. For defenses
that attempt to detect the adversarial examples, with our technique, an
adversary only needs very limited information about the defense to craft
adversarial examples. We demonstrate our technique by applying it against two
defenses which make models more robust and two defenses which detect
adversarial examples.
Active Ranking with Subset-wise Preferences
Authors: Aadirupa Saha, Aditya Gopalan
Date: 2018-10-23 00:00:00

We consider the problem of probably approximately correct (PAC) ranking $n$
items by adaptively eliciting subset-wise preference feedback. At each round,
the learner chooses a subset of $k$ items and observes stochastic feedback
indicating preference information of the winner (most preferred) item of the
chosen subset drawn according to a Plackett-Luce (PL) subset choice model
unknown a priori. The objective is to identify an $\epsilon$-optimal ranking of
the $n$ items with probability at least $1 - \delta$. When the feedback in each
subset round is a single Plackett-Luce-sampled item, we show $(\epsilon,
\delta)$-PAC algorithms with a sample complexity of
$O\left(\frac{n}{\epsilon^2} \ln \frac{n}{\delta} \right)$ rounds, which we
establish as being order-optimal by exhibiting a matching sample complexity
lower bound of $\Omega\left(\frac{n}{\epsilon^2} \ln \frac{n}{\delta}
\right)$---this shows that there is essentially no improvement possible from
the pairwise comparisons setting ($k = 2$). When, however, it is possible to
elicit top-$m$ ($\leq k$) ranking feedback according to the PL model from each
adaptively chosen subset of size $k$, we show that an $(\epsilon, \delta)$-PAC
ranking sample complexity of $O\left(\frac{n}{m \epsilon^2} \ln
\frac{n}{\delta} \right)$ is achievable with explicit algorithms, which
represents an $m$-wise reduction in sample complexity compared to the pairwise
case. This again turns out to be order-wise unimprovable across the class of
symmetric ranking algorithms. Our algorithms rely on a novel {pivot trick} to
maintain only $n$ itemwise score estimates, unlike $O(n^2)$ pairwise score
estimates that has been used in prior work. We report results of numerical
experiments that corroborate our findings.
Time-Aware and Corpus-Specific Entity Relatedness
Authors: Nilamadhaba Mohapatra, Vasileios Iosifidis, Asif Ekbal, Stefan Dietze, Pavlos Fafalios
Date: 2018-10-23 00:00:00
Tasks: Entity Linking, Information Retrieval, Retrieval, Word Embeddings

Entity relatedness has emerged as an important feature in a plethora of
applications such as information retrieval, entity recommendation and entity
linking. Given an entity, for instance a person or an organization, entity
relatedness measures can be exploited for generating a list of highly-related
entities. However, the relation of an entity to some other entity depends on
several factors, with time and context being two of the most important ones
(where, in our case, context is determined by a particular corpus). For
example, the entities related to the International Monetary Fund are different
now compared to some years ago, while these entities also may highly differ in
the context of a USA news portal compared to a Greek news portal. In this
paper, we propose a simple but flexible model for entity relatedness which
considers time and entity aware word embeddings by exploiting the underlying
corpus. The proposed model does not require external knowledge and is language
independent, which makes it widely useful in a variety of applications.
LoGAN: Generating Logos with a Generative Adversarial Neural Network Conditioned on color
Authors: Ajkel Mino, Gerasimos Spanakis
Date: 2018-10-23 00:00:00
Tasks: Descriptive

Designing a logo is a long, complicated, and expensive process for any
designer. However, recent advancements in generative algorithms provide models
that could offer a possible solution. Logos are multi-modal, have very few
categorical properties, and do not have a continuous latent space. Yet,
conditional generative adversarial networks can be used to generate logos that
could help designers in their creative process. We propose LoGAN: an improved
auxiliary classifier Wasserstein generative adversarial neural network (with
gradient penalty) that is able to generate logos conditioned on twelve
different colors. In 768 generated instances (12 classes and 64 logos per
class), when looking at the most prominent color, the conditional generation
part of the model has an overall precision and recall of 0.8 and 0.7
respectively. LoGAN's results offer a first glance at how artificial
intelligence can be used to assist designers in their creative process and open
promising future directions, such as including more descriptive labels which
will provide a more exhaustive and easy-to-use system.
The Faults in Our Pi Stars: Security Issues and Open Challenges in Deep Reinforcement Learning
Authors: Vahid Behzadan, Arslan Munir
Date: 2018-10-23 00:00:00
Tasks: Autonomous Navigation, Deep Reinforcement Learning, Reinforcement Learning

Since the inception of Deep Reinforcement Learning (DRL) algorithms, there
has been a growing interest in both research and industrial communities in the
promising potentials of this paradigm. The list of current and envisioned
applications of deep RL ranges from autonomous navigation and robotics to
control applications in the critical infrastructure, air traffic control,
defense technologies, and cybersecurity. While the landscape of opportunities
and the advantages of deep RL algorithms are justifiably vast, the security
risks and issues in such algorithms remain largely unexplored. To facilitate
and motivate further research on these critical challenges, this paper presents
a foundational treatment of the security problem in DRL. We formulate the
security requirements of DRL, and provide a high-level threat model through the
classification and identification of vulnerabilities, attack vectors, and
adversarial capabilities. Furthermore, we present a review of current
literature on security of deep RL from both offensive and defensive
perspectives. Lastly, we enumerate critical research venues and open problems
in mitigation and prevention of intentional attacks against deep RL as a
roadmap for further research in this area.