pwc-archive/papers-with-abstracts huggingface dataset:
Monte Carlo Q-learning for General Game Playing
Authors: Hui Wang, Michael Emmerich, Aske Plaat
Date: 2018-02-16 00:00:00
Tasks: Board Games, Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

After the recent groundbreaking results of AlphaGo, we have seen a strong
interest in reinforcement learning in game playing. General Game Playing (GGP)
provides a good testbed for reinforcement learning. In GGP, a specification of
games rules is given. GGP problems can be solved by reinforcement learning.
Q-learning is one of the canonical reinforcement learning methods, and has been
used by (Banerjee & Stone, IJCAI 2007) in GGP. In this paper we implement
Q-learning in GGP for three small-board games (Tic-Tac-Toe, Connect Four, Hex),
to allow comparison to Banerjee et al. As expected, Q-learning converges,
although much slower than MCTS. Borrowing an idea from MCTS, we enhance
Q-learning with Monte Carlo Search, to give QM-learning. This enhancement
improves the performance of pure Q-learning. We believe that QM-learning can
also be used to improve performance of reinforcement learning further for
larger games, something which we will test in future work.
A New Lower Bound for Agnostic Learning with Sample Compression Schemes
Authors: Steve Hanneke, Aryeh Kontorovich
Date: 2018-05-21 00:00:00
Tasks: Form

We establish a tight characterization of the worst-case rates for the excess
risk of agnostic learning with sample compression schemes and for uniform
convergence for agnostic sample compression schemes. In particular, we find
that the optimal rates of convergence for size-$k$ agnostic sample compression
schemes are of the form $\sqrt{\frac{k \log(n/k)}{n}}$, which contrasts with
agnostic learning with classes of VC dimension $k$, where the optimal rates are
of the form $\sqrt{\frac{k}{n}}$.
Meta-learning with differentiable closed-form solvers
Authors: Luca Bertinetto, João F. Henriques, Philip H. S. Torr, Andrea Vedaldi
Date: 2018-05-21 00:00:00
Proceeding: ICLR 2019 5
Tasks: BIG-bench Machine Learning, Few-Shot Learning, Form, Meta-Learning, regression

Adapting deep networks to new concepts from a few examples is challenging, due to the high computational requirements of standard fine-tuning procedures. Most work on few-shot learning has thus focused on simple learning techniques for adaptation, such as nearest neighbours or gradient descent. Nonetheless, the machine learning literature contains a wealth of methods that learn non-deep models very efficiently. In this paper, we propose to use these fast convergent methods as the main adaptation mechanism for few-shot learning. The main idea is to teach a deep network to use standard machine learning tools, such as ridge regression, as part of its own internal model, enabling it to quickly adapt to novel data. This requires back-propagating errors through the solver steps. While normally the cost of the matrix operations involved in such a process would be significant, by using the Woodbury identity we can make the small number of examples work to our advantage. We propose both closed-form and iterative solvers, based on ridge regression and logistic regression components. Our methods constitute a simple and novel approach to the problem of few-shot learning and achieve performance competitive with or superior to the state of the art on three benchmarks.
Overabundant Information and Learning Traps
Authors: Annie Liang, Xiaosheng Mu
Date: 2018-05-21 00:00:00

We develop a model of social learning from overabundant information:
Short-lived agents sequentially choose from a large set of (flexibly
correlated) information sources for prediction of an unknown state. Signal
realizations are public. We demonstrate two starkly different long-run
outcomes: (1) efficient information aggregation, where the community eventually
learns as fast as possible; (2) "learning traps," where the community gets
stuck observing suboptimal sources and learns inefficiently. Our main results
identify a simple property of the signal correlation structure that separates
these outcomes. In both regimes, we characterize which sources are observed in
the long run and how often.
A General Family of Robust Stochastic Operators for Reinforcement Learning
Authors: Yingdong Lu, Mark S. Squillante, Chai Wah Wu
Date: 2018-05-21 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We consider a new family of operators for reinforcement learning with the goal of alleviating the negative effects and becoming more robust to approximation or estimation errors. Various theoretical results are established, which include showing on a sample path basis that our family of operators preserve optimality and increase the action gap. Our empirical results illustrate the strong benefits of our family of operators, significantly outperforming the classical Bellman operator and recently proposed operators.
Stacked Semantic-Guided Attention Model for Fine-Grained Zero-Shot Learning
Authors: Yunlong Yu, Zhong Ji, Yanwei Fu, Jichang Guo, Yanwei Pang, Zhongfei Zhang
Date: 2018-05-21 00:00:00
Tasks: General Classification, Multi-class Classification, Retrieval, zero-shot-classification, Zero-Shot Learning

Zero-Shot Learning (ZSL) is achieved via aligning the semantic relationships
between the global image feature vector and the corresponding class semantic
descriptions. However, using the global features to represent fine-grained
images may lead to sub-optimal results since they neglect the discriminative
differences of local regions. Besides, different regions contain distinct
discriminative information. The important regions should contribute more to the
prediction. To this end, we propose a novel stacked semantics-guided attention
(S2GA) model to obtain semantic relevant features by using individual class
semantic features to progressively guide the visual features to generate an
attention map for weighting the importance of different local regions. Feeding
both the integrated visual features and the class semantic features into a
multi-class classification architecture, the proposed framework can be trained
end-to-end. Extensive experimental results on CUB and NABird datasets show that
the proposed approach has a consistent improvement on both fine-grained
zero-shot classification and retrieval tasks.
Variational online learning of neural dynamics
Authors: Yuan Zhao, Il Memming Park
Date: 2017-07-27 00:00:00
Tasks: Decision Making, Experimental Design, State Space Models, Time Series Analysis

New technologies for recording the activity of large neural populations during complex behavior provide exciting opportunities for investigating the neural computations that underlie perception, cognition, and decision-making. Nonlinear state space models provide an interpretable signal processing framework by combining an intuitive dynamical system with a probabilistic observation model, which can provide insights into neural dynamics, neural computation, and development of neural prosthetics and treatment through feedback control. It brings the challenge of learning both latent neural state and the underlying dynamical system because neither is known for neural systems a priori. We developed a flexible online learning framework for latent nonlinear state dynamics and filtered latent states. Using the stochastic gradient variational Bayes approach, our method jointly optimizes the parameters of the nonlinear dynamical system, the observation model, and the black-box recognition model. Unlike previous approaches, our framework can incorporate non-trivial distributions of observation noise and has constant time and space complexity. These features make our approach amenable to real-time applications and the potential to automate analysis and experimental design in ways that testably track and modify behavior using stimuli designed to influence learning.
Comparison of Semantic Segmentation Approaches for Horizon/Sky Line Detection
Authors: Touqeer Ahmad, Pavel Campr, Martin Čadík, George Bebis
Date: 2018-05-21 00:00:00
Tasks: geo-localization, Line Detection, Segmentation, Semantic Segmentation, Transfer Learning

Horizon or skyline detection plays a vital role towards mountainous visual
geo-localization, however most of the recently proposed visual geo-localization
approaches rely on \textbf{user-in-the-loop} skyline detection methods.
Detecting such a segmenting boundary fully autonomously would definitely be a
step forward for these localization approaches. This paper provides a
quantitative comparison of four such methods for autonomous horizon/sky line
detection on an extensive data set. Specifically, we provide the comparison
between four recently proposed segmentation methods; one explicitly targeting
the problem of horizon detection\cite{Ahmad15}, second focused on visual
geo-localization but relying on accurate detection of skyline \cite{Saurer16}
and other two proposed for general semantic segmentation -- Fully Convolutional
Networks (FCN) \cite{Long15} and SegNet\cite{Badrinarayanan15}. Each of the
first two methods is trained on a common training set \cite{Baatz12} comprised
of about 200 images while models for the third and fourth method are fine tuned
for sky segmentation problem through transfer learning using the same data set.
Each of the method is tested on an extensive test set (about 3K images)
covering various challenging geographical, weather, illumination and seasonal
conditions. We report average accuracy and average absolute pixel error for
each of the presented formulation.
The Anisotropic Noise in Stochastic Gradient Descent: Its Behavior of Escaping from Sharp Minima and Regularization Effects
Authors: Zhanxing Zhu, Jingfeng Wu, Bing Yu, Lei Wu, Jinwen Ma
Date: 2018-03-01 00:00:00
Proceeding: ICLR 2019 5

Understanding the behavior of stochastic gradient descent (SGD) in the context of deep neural networks has raised lots of concerns recently. Along this line, we study a general form of gradient based optimization dynamics with unbiased noise, which unifies SGD and standard Langevin dynamics. Through investigating this general optimization dynamics, we analyze the behavior of SGD on escaping from minima and its regularization effects. A novel indicator is derived to characterize the efficiency of escaping from minima through measuring the alignment of noise covariance and the curvature of loss function. Based on this indicator, two conditions are established to show which type of noise structure is superior to isotropic noise in term of escaping efficiency. We further show that the anisotropic noise in SGD satisfies the two conditions, and thus helps to escape from sharp and poor minima effectively, towards more stable and flat minima that typically generalize well. We systematically design various experiments to verify the benefits of the anisotropic noise, compared with full gradient descent plus isotropic diffusion (i.e. Langevin dynamics).
Computational Historical Linguistics
Authors: Gerhard Jäger
Date: 2018-05-21 00:00:00

Computational approaches to historical linguistics have been proposed since
half a century. Within the last decade, this line of research has received a
major boost, owing both to the transfer of ideas and software from
computational biology and to the release of several large electronic data
resources suitable for systematic comparative work.
  In this article, some of the central research topic of this new wave of
computational historical linguistics are introduced and discussed. These are
automatic assessment of genetic relatedness, automatic cognate detection,
phylogenetic inference and ancestral state reconstruction. They will be
demonstrated by means of a case study of automatically reconstructing a
Proto-Romance word list from lexical data of 50 modern Romance languages and
dialects.
Invariant Representations from Adversarially Censored Autoencoders
Authors: Ye Wang, Toshiaki Koike-Akino, Deniz Erdogmus
Date: 2018-05-21 00:00:00
Tasks: Decoder, Style Transfer

We combine conditional variational autoencoders (VAE) with adversarial
censoring in order to learn invariant representations that are disentangled
from nuisance/sensitive variations. In this method, an adversarial network
attempts to recover the nuisance variable from the representation, which the
VAE is trained to prevent. Conditioning the decoder on the nuisance variable
enables clean separation of the representation, since they are recombined for
model learning and data reconstruction. We show this natural approach is
theoretically well-founded with information-theoretic arguments. Experiments
demonstrate that this method achieves invariance while preserving model
learning performance, and results in visually improved performance for style
transfer and generative sampling tasks.
Understanding Self-Paced Learning under Concave Conjugacy Theory
Authors: Shiqi Liu, Zilu Ma, Deyu Meng
Date: 2018-05-21 00:00:00

By simulating the easy-to-hard learning manners of humans/animals, the
learning regimes called curriculum learning~(CL) and self-paced learning~(SPL)
have been recently investigated and invoked broad interests. However, the
intrinsic mechanism for analyzing why such learning regimes can work has not
been comprehensively investigated. To this issue, this paper proposes a concave
conjugacy theory for looking into the insight of CL/SPL. Specifically, by using
this theory, we prove the equivalence of the SPL regime and a latent concave
objective, which is closely related to the known non-convex regularized penalty
widely used in statistics and machine learning. Beyond the previous theory for
explaining CL/SPL insights, this new theoretical framework on one hand
facilitates two direct approaches for designing new SPL models for certain
tasks, and on the other hand can help conduct the latent objective of
self-paced curriculum learning, which is the advanced version of both CL/SPL
and possess advantages of both learning regimes to a certain extent. This
further facilitates a theoretical understanding for SPCL, instead of only
CL/SPL as conventional. Under this theory, we attempt to attain intrinsic
latent objectives of two curriculum forms, the partial order and group
curriculums, which easily follow the theoretical understanding of the
corresponding SPCL regimes.
Small steps and giant leaps: Minimal Newton solvers for Deep Learning
Authors: João F. Henriques, Sebastien Ehrhardt, Samuel Albanie, Andrea Vedaldi
Date: 2018-05-21 00:00:00
Proceeding: ICLR 2019 5

We propose a fast second-order method that can be used as a drop-in
replacement for current deep learning solvers. Compared to stochastic gradient
descent (SGD), it only requires two additional forward-mode automatic
differentiation operations per iteration, which has a computational cost
comparable to two standard forward passes and is easy to implement. Our method
addresses long-standing issues with current second-order solvers, which invert
an approximate Hessian matrix every iteration exactly or by conjugate-gradient
methods, a procedure that is both costly and sensitive to noise. Instead, we
propose to keep a single estimate of the gradient projected by the inverse
Hessian matrix, and update it once per iteration. This estimate has the same
size and is similar to the momentum variable that is commonly used in SGD. No
estimate of the Hessian is maintained. We first validate our method, called
CurveBall, on small problems with known closed-form solutions (noisy Rosenbrock
function and degenerate 2-layer linear networks), where current deep learning
solvers seem to struggle. We then train several large models on CIFAR and
ImageNet, including ResNet and VGG-f networks, where we demonstrate faster
convergence with no hyperparameter tuning. Code is available.
Variational based Mixed Noise Removal with CNN Deep Learning Regularization
Authors: Faqiang Wang, Hai-yang Huang, Jun Liu
Date: 2018-05-21 00:00:00
Tasks: Deep Learning, Image Reconstruction, parameter estimation

In this paper, the traditional model based variational method and learning
based algorithms are naturally integrated to address mixed noise removal
problem. To be different from single type noise (e.g. Gaussian) removal, it is
a challenge problem to accurately discriminate noise types and levels for each
pixel. We propose a variational method to iteratively estimate the noise
parameters, and then the algorithm can automatically classify the noise
according to the different statistical parameters. The proposed variational
problem can be separated into regularization, synthesis, parameter estimation
and noise classification four steps with the operator splitting scheme. Each
step is related to an optimization subproblem. To enforce the regularization,
the deep learning method is employed to learn the natural images priori.
Compared with some model based regularizations, the CNN regularizer can
significantly improve the quality of the restored images. Compared with some
learning based methods, the synthesis step can produce better reconstructions
by analyzing the recognized noise types and levels. In our method, the
convolution neutral network (CNN) can be regarded as an operator which
associated to a variational functional. From this viewpoint, the proposed
method can be extended to many image reconstruction and inverse problems.
Numerical experiments in the paper show that our method can achieve some
state-of-the-art results for mixed noise removal.
NeuralREG: An end-to-end approach to referring expression generation
Authors: Thiago Castro Ferreira, Diego Moussallem, Ákos Kádár, Sander Wubben, Emiel Krahmer
Date: 2018-05-21 00:00:00
Proceeding: ACL 2018 7
Tasks: Form, Referring Expression, Referring expression generation

Traditionally, Referring Expression Generation (REG) models first decide on
the form and then on the content of references to discourse entities in text,
typically relying on features such as salience and grammatical function. In
this paper, we present a new approach (NeuralREG), relying on deep neural
networks, which makes decisions about form and content in one go without
explicit feature extraction. Using a delexicalized version of the WebNLG
corpus, we show that the neural model substantially improves over two strong
baselines. Data and models are publicly available.
Efficient and Robust Question Answering from Minimal Context over Documents
Authors: Sewon Min, Victor Zhong, Richard Socher, Caiming Xiong
Date: 2018-05-21 00:00:00
Proceeding: ACL 2018 7
Tasks: Question Answering, Reading Comprehension, Sentence, TriviaQA

Neural models for question answering (QA) over documents have achieved
significant performance improvements. Although effective, these models do not
scale to large corpora due to their complex modeling of interactions between
the document and the question. Moreover, recent work has shown that such models
are sensitive to adversarial inputs. In this paper, we study the minimal
context required to answer the question, and find that most questions in
existing datasets can be answered with a small set of sentences. Inspired by
this observation, we propose a simple sentence selector to select the minimal
set of sentences to feed into the QA model. Our overall system achieves
significant reductions in training (up to 15 times) and inference times (up to
13 times), with accuracy comparable to or better than the state-of-the-art on
SQuAD, NewsQA, TriviaQA and SQuAD-Open. Furthermore, our experimental results
and analyses show that our approach is more robust to adversarial inputs.
Do deep reinforcement learning agents model intentions?
Authors: Tambet Matiisen, Aqeel Labash, Daniel Majoral, Jaan Aru, Raul Vicente
Date: 2018-05-15 00:00:00
Tasks: Deep Reinforcement Learning, model, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Inferring other agents' mental states such as their knowledge, beliefs and
intentions is thought to be essential for effective interactions with other
agents. Recently, multiagent systems trained via deep reinforcement learning
have been shown to succeed in solving different tasks, but it remains unclear
how each agent modeled or represented other agents in their environment. In
this work we test whether deep reinforcement learning agents explicitly
represent other agents' intentions (their specific aims or goals) during a task
in which the agents had to coordinate the covering of different spots in a 2D
environment. In particular, we tracked over time the performance of a linear
decoder trained to predict the final goal of all agents from the hidden state
of each agent's neural network controller. We observed that the hidden layers
of agents represented explicit information about other agents' goals, i.e. the
target landmark they ended up covering. We also performed a series of
experiments, in which some agents were replaced by others with fixed goals, to
test the level of generalization of the trained agents. We noticed that during
the training phase the agents developed a differential preference for each
goal, which hindered generalization. To alleviate the above problem, we propose
simple changes to the MADDPG training algorithm which leads to better
generalization against unseen agents. We believe that training protocols
promoting more active intention reading mechanisms, e.g. by preventing simple
symmetry-breaking solutions, is a promising direction towards achieving a more
robust generalization in different cooperative and competitive tasks.
GANE: A Generative Adversarial Network Embedding
Authors: Huiting Hong, Xin Li, Mingzhong Wang
Date: 2018-05-18 00:00:00
Tasks: Clustering, Generative Adversarial Network, Link Prediction, Network Embedding

Network embedding has become a hot research topic recently which can provide
low-dimensional feature representations for many machine learning applications.
Current work focuses on either (1) whether the embedding is designed as an
unsupervised learning task by explicitly preserving the structural connectivity
in the network, or (2) whether the embedding is a by-product during the
supervised learning of a specific discriminative task in a deep neural network.
In this paper, we focus on bridging the gap of the two lines of the research.
We propose to adapt the Generative Adversarial model to perform network
embedding, in which the generator is trying to generate vertex pairs, while the
discriminator tries to distinguish the generated vertex pairs from real
connections (edges) in the network. Wasserstein-1 distance is adopted to train
the generator to gain better stability. We develop three variations of models,
including GANE which applies cosine similarity, GANE-O1 which preserves the
first-order proximity, and GANE-O2 which tries to preserves the second-order
proximity of the network in the low-dimensional embedded vector space. We later
prove that GANE-O2 has the same objective function as GANE-O1 when negative
sampling is applied to simplify the training process in GANE-O2. Experiments
with real-world network datasets demonstrate that our models constantly
outperform state-of-the-art solutions with significant improvements on
precision in link prediction, as well as on visualizations and accuracy in
clustering tasks.