pwc-archive/papers-with-abstracts huggingface dataset:
Primal-Dual $π$ Learning: Sample Complexity and Sublinear Run Time for Ergodic Markov Decision Problems
Authors: Mengdi Wang
Date: 2017-10-17 00:00:00
Tasks: Reinforcement Learning

Consider the problem of approximating the optimal policy of a Markov decision
process (MDP) by sampling state transitions. In contrast to existing
reinforcement learning methods that are based on successive approximations to
the nonlinear Bellman equation, we propose a Primal-Dual $\pi$ Learning method
in light of the linear duality between the value and policy. The $\pi$ learning
method is model-free and makes primal-dual updates to the policy and value
vectors as new data are revealed. For infinite-horizon undiscounted Markov
decision process with finite state space $S$ and finite action space $A$, the
$\pi$ learning method finds an $\epsilon$-optimal policy using the following
number of sample transitions $$ \tilde{O}( \frac{(\tau\cdot t^*_{mix})^2 |S|
|A| }{\epsilon^2} ),$$ where $t^*_{mix}$ is an upper bound of mixing times
across all policies and $\tau$ is a parameter characterizing the range of
stationary distributions across policies. The $\pi$ learning method also
applies to the computational problem of MDP where the transition probabilities
and rewards are explicitly given as the input. In the case where each state
transition can be sampled in $\tilde{O}(1)$ time, the $\pi$ learning method
gives a sublinear-time algorithm for solving the averaged-reward MDP.
Efficient Second Order Online Learning by Sketching
Authors: Haipeng Luo, Alekh Agarwal, Nicolo Cesa-Bianchi, John Langford
Date: 2016-02-06 00:00:00
Proceeding: NeurIPS 2016 12

We propose Sketched Online Newton (SON), an online second order learning
algorithm that enjoys substantially improved regret guarantees for
ill-conditioned data. SON is an enhanced version of the Online Newton Step,
which, via sketching techniques enjoys a running time linear in the dimension
and sketch size. We further develop sparse forms of the sketching methods (such
as Oja's rule), making the computation linear in the sparsity of features.
Together, the algorithm eliminates all computational obstacles in previous
second order online learning approaches.
Spontaneous Symmetry Breaking in Neural Networks
Authors: Ricky Fok, Aijun An, Xiaogang Wang
Date: 2017-10-17 00:00:00

We propose a framework to understand the unprecedented performance and
robustness of deep neural networks using field theory. Correlations between the
weights within the same layer can be described by symmetries in that layer, and
networks generalize better if such symmetries are broken to reduce the
redundancies of the weights. Using a two parameter field theory, we find that
the network can break such symmetries itself towards the end of training in a
process commonly known in physics as spontaneous symmetry breaking. This
corresponds to a network generalizing itself without any user input layers to
break the symmetry, but by communication with adjacent layers. In the layer
decoupling limit applicable to residual networks (He et al., 2015), we show
that the remnant symmetries that survive the non-linear layers are
spontaneously broken. The Lagrangian for the non-linear and weight layers
together has striking similarities with the one in quantum field theory of a
scalar. Using results from quantum field theory we show that our framework is
able to explain many experimentally observed phenomena,such as training on
random labels with zero error (Zhang et al., 2017), the information bottleneck,
the phase transition out of it and gradient variance explosion (Shwartz-Ziv &
Tishby, 2017), shattered gradients (Balduzzi et al., 2017), and many more.
Focusing Attention: Towards Accurate Text Recognition in Natural Images
Authors: Zhanzhan Cheng, Fan Bai, Yunlu Xu, Gang Zheng, ShiLiang Pu, Shuigeng Zhou
Date: 2017-09-07 00:00:00
Proceeding: ICCV 2017 10
Tasks: Decoder, Scene Text Recognition

Scene text recognition has been a hot research topic in computer vision due
to its various applications. The state of the art is the attention-based
encoder-decoder framework that learns the mapping between input images and
output sequences in a purely data-driven way. However, we observe that existing
attention-based methods perform poorly on complicated and/or low-quality
images. One major reason is that existing methods cannot get accurate
alignments between feature areas and targets for such images. We call this
phenomenon "attention drift". To tackle this problem, in this paper we propose
the FAN (the abbreviation of Focusing Attention Network) method that employs a
focusing attention mechanism to automatically draw back the drifted attention.
FAN consists of two major components: an attention network (AN) that is
responsible for recognizing character targets as in the existing methods, and a
focusing network (FN) that is responsible for adjusting attention by evaluating
whether AN pays attention properly on the target areas in the images.
Furthermore, different from the existing methods, we adopt a ResNet-based
network to enrich deep representations of scene text images. Extensive
experiments on various benchmarks, including the IIIT5k, SVT and ICDAR
datasets, show that the FAN method substantially outperforms the existing
methods.
Face Transfer with Generative Adversarial Network
Authors: Runze Xu, Zhiming Zhou, Wei-Nan Zhang, Yong Yu
Date: 2017-10-17 00:00:00
Tasks: Face Transfer, Generative Adversarial Network

Face transfer animates the facial performances of the character in the target
video by a source actor. Traditional methods are typically based on face
modeling. We propose an end-to-end face transfer method based on Generative
Adversarial Network. Specifically, we leverage CycleGAN to generate the face
image of the target character with the corresponding head pose and facial
expression of the source. In order to improve the quality of generated videos,
we adopt PatchGAN and explore the effect of different receptive field sizes on
generated images.
On the challenges of learning with inference networks on sparse, high-dimensional data
Authors: Rahul G. Krishnan, Dawen Liang, Matthew Hoffman
Date: 2017-10-17 00:00:00
Tasks: parameter estimation, Variational Inference

We study parameter estimation in Nonlinear Factor Analysis (NFA) where the
generative model is parameterized by a deep neural network. Recent work has
focused on learning such models using inference (or recognition) networks; we
identify a crucial problem when modeling large, sparse, high-dimensional
datasets -- underfitting. We study the extent of underfitting, highlighting
that its severity increases with the sparsity of the data. We propose methods
to tackle it via iterative optimization inspired by stochastic variational
inference \citep{hoffman2013stochastic} and improvements in the sparse data
representation used for inference. The proposed techniques drastically improve
the ability of these powerful models to fit sparse data, achieving
state-of-the-art results on a benchmark text-count dataset and excellent
results on the task of top-N recommendation.
Estimate exponential memory decay in Hidden Markov Model and its applications
Authors: Felix X. -F. Ye, Yi-An Ma, Hong Qian
Date: 2017-10-17 00:00:00

Inference in hidden Markov model has been challenging in terms of scalability due to dependencies in the observation data. In this paper, we utilize the inherent memory decay in hidden Markov models, such that the forward and backward probabilities can be carried out with subsequences, enabling efficient inference over long sequences of observations. We formulate this forward filtering process in the setting of the random dynamical system and there exist Lyapunov exponents in the i.i.d random matrices production. And the rate of the memory decay is known as $\lambda_2-\lambda_1$, the gap of the top two Lyapunov exponents almost surely. An efficient and accurate algorithm is proposed to numerically estimate the gap after the soft-max parametrization. The length of subsequences $B$ given the controlled error $\epsilon$ is $B=\log(\epsilon)/(\lambda_2-\lambda_1)$. We theoretically prove the validity of the algorithm and demonstrate the effectiveness with numerical examples. The method developed here can be applied to widely used algorithms, such as mini-batch stochastic gradient method. Moreover, the continuity of Lyapunov spectrum ensures the estimated $B$ could be reused for the nearby parameter during the inference.
PubMed 200k RCT: a Dataset for Sequential Sentence Classification in Medical Abstracts
Authors: Franck Dernoncourt, Ji Young Lee
Date: 2017-10-17 00:00:00
Proceeding: IJCNLP 2017 11
Tasks: General Classification, Sentence, Sentence Classification, Text Classification

We present PubMed 200k RCT, a new dataset based on PubMed for sequential
sentence classification. The dataset consists of approximately 200,000
abstracts of randomized controlled trials, totaling 2.3 million sentences. Each
sentence of each abstract is labeled with their role in the abstract using one
of the following classes: background, objective, method, result, or conclusion.
The purpose of releasing this dataset is twofold. First, the majority of
datasets for sequential short-text classification (i.e., classification of
short texts that appear in sequences) are small: we hope that releasing a new
large dataset will help develop more accurate algorithms for this task. Second,
from an application perspective, researchers need better tools to efficiently
skim through the literature. Automatically classifying each sentence in an
abstract would help researchers read abstracts more efficiently, especially in
fields where abstracts may be long, such as the medical field.
Hardware for Machine Learning: Challenges and Opportunities
Authors: Vivienne Sze, Yu-Hsin Chen, Joel Emer, Amr Suleiman, Zhengdong Zhang
Date: 2016-12-22 00:00:00
Tasks: BIG-bench Machine Learning, Self-Driving Cars

Machine learning plays a critical role in extracting meaningful information
out of the zetabytes of sensor data collected every day. For some applications,
the goal is to analyze and understand the data to identify trends (e.g.,
surveillance, portable/wearable electronics); in other applications, the goal
is to take immediate action based the data (e.g., robotics/drones, self-driving
cars, smart Internet of Things). For many of these applications, local embedded
processing near the sensor is preferred over the cloud due to privacy or
latency concerns, or limitations in the communication bandwidth. However, at
the sensor there are often stringent constraints on energy consumption and cost
in addition to throughput and accuracy requirements. Furthermore, flexibility
is often required such that the processing can be adapted for different
applications or environments (e.g., update the weights and model in the
classifier). In many applications, machine learning often involves transforming
the input data into a higher dimensional space, which, along with programmable
weights, increases data movement and consequently energy consumption. In this
paper, we will discuss how these challenges can be addressed at various levels
of hardware design ranging from architecture, hardware-friendly algorithms,
mixed-signal circuits, and advanced technologies (including memories and
sensors).
Reverse Curriculum Generation for Reinforcement Learning
Authors: Carlos Florensa, David Held, Markus Wulfmeier, Michael Zhang, Pieter Abbeel
Date: 2017-07-17 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Many relevant tasks require an agent to reach a certain state, or to
manipulate objects into a desired configuration. For example, we might want a
robot to align and assemble a gear onto an axle or insert and turn a key in a
lock. These goal-oriented tasks present a considerable challenge for
reinforcement learning, since their natural reward function is sparse and
prohibitive amounts of exploration are required to reach the goal and receive
some learning signal. Past approaches tackle these problems by exploiting
expert demonstrations or by manually designing a task-specific reward shaping
function to guide the learning agent. Instead, we propose a method to learn
these tasks without requiring any prior knowledge other than obtaining a single
state in which the task is achieved. The robot is trained in reverse, gradually
learning to reach the goal from a set of start states increasingly far from the
goal. Our method automatically generates a curriculum of start states that
adapts to the agent's performance, leading to efficient training on
goal-oriented tasks. We demonstrate our approach on difficult simulated
navigation and fine-grained manipulation problems, not solvable by
state-of-the-art reinforcement learning methods.
Image Crowd Counting Using Convolutional Neural Network and Markov Random Field
Authors: Kang Han, Wanggen Wan, Haiyan Yao, Li Hou
Date: 2017-06-12 00:00:00
Tasks: Crowd Counting

In this paper, we propose a method called Convolutional Neural Network-Markov
Random Field (CNN-MRF) to estimate the crowd count in a still image. We first
divide the dense crowd visible image into overlapping patches and then use a
deep convolutional neural network to extract features from each patch image,
followed by a fully connected neural network to regress the local patch crowd
count. Since the local patches have overlapping portions, the crowd count of
the adjacent patches has a high correlation. We use this correlation and the
Markov random field to smooth the counting results of the local patches.
Experiments show that our approach significantly outperforms the
state-of-the-art methods on UCF and Shanghaitech crowd counting datasets.
Intention-Net: Integrating Planning and Deep Learning for Goal-Directed Autonomous Navigation
Authors: Wei Gao, David Hsu, Wee Sun Lee, ShengMei Shen, Karthikk Subramanian
Date: 2017-10-16 00:00:00
Tasks: Autonomous Navigation, Navigate

How can a delivery robot navigate reliably to a destination in a new office
building, with minimal prior information? To tackle this challenge, this paper
introduces a two-level hierarchical approach, which integrates model-free deep
learning and model-based path planning. At the low level, a neural-network
motion controller, called the intention-net, is trained end-to-end to provide
robust local navigation. The intention-net maps images from a single monocular
camera and "intentions" directly to robot controls. At the high level, a path
planner uses a crude map, e.g., a 2-D floor plan, to compute a path from the
robot's current location to the goal. The planned path provides intentions to
the intention-net. Preliminary experiments suggest that the learned motion
controller is robust against perceptual uncertainty and by integrating with a
path planner, it generalizes effectively to new environments and goals.
Evolution in Virtual Worlds
Authors: Tim Taylor
Date: 2017-10-17 00:00:00

This chapter discusses the possibility of instilling a virtual world with
mechanisms for evolution and natural selection in order to generate rich
ecosystems of complex organisms in a process akin to biological evolution. Some
previous work in the area is described, and successes and failures are
discussed. The components of a more comprehensive framework for designing such
worlds are mapped out, including the design of the individual organisms, the
properties and dynamics of the environmental medium in which they are evolving,
and the representational relationship between organism and environment. Some of
the key issues discussed include how to allow organisms to evolve new
structures and functions with few restrictions, and how to create an
interconnectedness between organisms in order to generate drives for continuing
evolutionary activity.
A deep generative model for single-cell RNA sequencing with application to detecting differentially expressed genes
Authors: Romain Lopez, Jeffrey Regier, Michael Cole, Michael Jordan, Nir Yosef
Date: 2017-10-13 00:00:00
Tasks: Stochastic Optimization, Variational Inference

We propose a probabilistic model for interpreting gene expression levels that
are observed through single-cell RNA sequencing. In the model, each cell has a
low-dimensional latent representation. Additional latent variables account for
technical effects that may erroneously set some observations of gene expression
levels to zero. Conditional distributions are specified by neural networks,
giving the proposed model enough flexibility to fit the data well. We use
variational inference and stochastic optimization to approximate the posterior
distribution. The inference procedure scales to over one million cells, whereas
competing algorithms do not. Even for smaller datasets, for several tasks, the
proposed procedure outperforms state-of-the-art methods like ZIFA and
ZINB-WaVE. We also extend our framework to take into account batch effects and
other confounding factors and propose a natural Bayesian hypothesis framework
for differential expression that outperforms tradition DESeq2.
Proximal Quasi-Newton Methods for Regularized Convex Optimization with Linear and Accelerated Sublinear Convergence Rates
Authors: Hiva Ghanbari, Katya Scheinberg
Date: 2016-07-11 00:00:00

In [19], a general, inexact, efficient proximal quasi-Newton algorithm for
composite optimization problems has been proposed and a sublinear global
convergence rate has been established. In this paper, we analyze the
convergence properties of this method, both in the exact and inexact setting,
in the case when the objective function is strongly convex. We also investigate
a practical variant of this method by establishing a simple stopping criterion
for the subproblem optimization. Furthermore, we consider an accelerated
variant, based on FISTA [1], to the proximal quasi-Newton algorithm. A similar
accelerated method has been considered in [7], where the convergence rate
analysis relies on very strong impractical assumptions. We present a modified
analysis while relaxing these assumptions and perform a practical comparison of
the accelerated proximal quasi- Newton algorithm and the regular one. Our
analysis and computational results show that acceleration may not bring any
benefit in the quasi-Newton setting.
Non-Euclidean Conditional Expectation and Filtering
Authors: Anastasis Kratsios, Cody B. Hyndman
Date: 2017-10-16 00:00:00

A non-Euclidean generalization of conditional expectation is introduced and
characterized as the minimizer of expected intrinsic squared-distance from a
manifold-valued target. The computational tractable formulation expresses the
non-convex optimization problem as transformations of Euclidean conditional
expectation. This gives computationally tractable filtering equations for the
dynamics of the intrinsic conditional expectation of a manifold-valued signal
and is used to obtain accurate numerical forecasts of efficient portfolios by
incorporating their geometric structure into the estimates.
Deep Learning in a Generalized HJM-type Framework Through Arbitrage-Free Regularization
Authors: Anastasis Kratsios, Cody B. Hyndman
Date: 2017-10-14 00:00:00
Tasks: Model Selection

We introduce a regularization approach to arbitrage-free factor-model selection. The considered model selection problem seeks to learn the closest arbitrage-free HJM-type model to any prespecified factor-model. An asymptotic solution to this, a priori computationally intractable, problem is represented as the limit of a 1-parameter family of optimizers to computationally tractable model selection tasks. Each of these simplified model-selection tasks seeks to learn the most similar model, to the prescribed factor-model, subject to a penalty detecting when the reference measure is a local martingale-measure for the entire underlying financial market. A simple expression for the penalty terms is obtained in the bond market withing the affine-term structure setting, and it is used to formulate a deep-learning approach to arbitrage-free affine term-structure modelling. Numerical implementations are also performed to evaluate the performance in the bond market.
AI Buzzwords Explained: Multi-Agent Path Finding (MAPF)
Authors: Hang Ma, Sven Koenig
Date: 2017-10-10 00:00:00
Tasks: Multi-Agent Path Finding

Explanation of the hot topic "multi-agent path finding".
Efficiency of quantum versus classical annealing in non-convex learning problems
Authors: Carlo Baldassi, Riccardo Zecchina
Date: 2017-06-26 00:00:00

Quantum annealers aim at solving non-convex optimization problems by
exploiting cooperative tunneling effects to escape local minima. The underlying
idea consists in designing a classical energy function whose ground states are
the sought optimal solutions of the original optimization problem and add a
controllable quantum transverse field to generate tunneling processes. A key
challenge is to identify classes of non-convex optimization problems for which
quantum annealing remains efficient while thermal annealing fails. We show that
this happens for a wide class of problems which are central to machine
learning. Their energy landscapes is dominated by local minima that cause
exponential slow down of classical thermal annealers while simulated quantum
annealing converges efficiently to rare dense regions of optimal solutions.
Sequence Tutor: Conservative Fine-Tuning of Sequence Generation Models with KL-control
Authors: Natasha Jaques, Shixiang Gu, Dzmitry Bahdanau, José Miguel Hernández-Lobato, Richard E. Turner, Douglas Eck
Date: 2016-11-09 00:00:00
Proceeding: ICML 2017 8
Tasks: Diversity, Reinforcement Learning, Reinforcement Learning (RL)

This paper proposes a general method for improving the structure and quality
of sequences generated by a recurrent neural network (RNN), while maintaining
information originally learned from data, as well as sample diversity. An RNN
is first pre-trained on data using maximum likelihood estimation (MLE), and the
probability distribution over the next token in the sequence learned by this
model is treated as a prior policy. Another RNN is then trained using
reinforcement learning (RL) to generate higher-quality outputs that account for
domain-specific incentives while retaining proximity to the prior policy of the
MLE RNN. To formalize this objective, we derive novel off-policy RL methods for
RNNs from KL-control. The effectiveness of the approach is demonstrated on two
applications; 1) generating novel musical melodies, and 2) computational
molecular generation. For both problems, we show that the proposed method
improves the desired properties and structure of the generated sequences, while
maintaining information learned from data.
Dynamic Oracle for Neural Machine Translation in Decoding Phase
Authors: Zi-Yi Dou, Hao Zhou, Shu-Jian Huang, Xin-yu Dai, Jia-Jun Chen
Date: 2017-09-19 00:00:00
Proceeding: LREC 2018 5
Tasks: Machine Translation, NMT, Translation

The past several years have witnessed the rapid progress of end-to-end Neural
Machine Translation (NMT). However, there exists discrepancy between training
and inference in NMT when decoding, which may lead to serious problems since
the model might be in a part of the state space it has never seen during
training. To address the issue, Scheduled Sampling has been proposed. However,
there are certain limitations in Scheduled Sampling and we propose two dynamic
oracle-based methods to improve it. We manage to mitigate the discrepancy by
changing the training process towards a less guided scheme and meanwhile
aggregating the oracle's demonstrations. Experimental results show that the
proposed approaches improve translation quality over standard NMT system.