pwc-archive/papers-with-abstracts huggingface dataset:
A Unified Computational and Statistical Framework for Nonconvex Low-Rank Matrix Estimation
Authors: Lingxiao Wang, Xiao Zhang, Quanquan Gu
Date: 2016-10-17 00:00:00
Tasks: Matrix Completion

We propose a unified framework for estimating low-rank matrices through
nonconvex optimization based on gradient descent algorithm. Our framework is
quite general and can be applied to both noisy and noiseless observations. In
the general case with noisy observations, we show that our algorithm is
guaranteed to linearly converge to the unknown low-rank matrix up to minimax
optimal statistical error, provided an appropriate initial estimator. While in
the generic noiseless setting, our algorithm converges to the unknown low-rank
matrix at a linear rate and enables exact recovery with optimal sample
complexity. In addition, we develop a new initialization algorithm to provide a
desired initial estimator, which outperforms existing initialization algorithms
for nonconvex low-rank matrix estimation. We illustrate the superiority of our
framework through three examples: matrix regression, matrix completion, and
one-bit matrix completion. We also corroborate our theory through extensive
experiments on synthetic data.
A Bayesian Group Sparse Multi-Task Regression Model for Imaging Genetics
Authors: Keelin Greenlaw, Elena Szefer, Jinko Graham, Mary Lesperance, Farouk S. Nathoo
Date: 2016-05-07 00:00:00
Tasks: regression

Motivation: Recent advances in technology for brain imaging and
high-throughput genotyping have motivated studies examining the influence of
genetic variation on brain structure. Wang et al. (Bioinformatics, 2012) have
developed an approach for the analysis of imaging genomic studies using
penalized multi-task regression with regularization based on a novel group
$l_{2,1}$-norm penalty which encourages structured sparsity at both the gene
level and SNP level. While incorporating a number of useful features, the
proposed method only furnishes a point estimate of the regression coefficients;
techniques for conducting statistical inference are not provided. A new
Bayesian method is proposed here to overcome this limitation.
  Results: We develop a Bayesian hierarchical modeling formulation where the
posterior mode corresponds to the estimator proposed by Wang et al.
(Bioinformatics, 2012), and an approach that allows for full posterior
inference including the construction of interval estimates for the regression
parameters. We show that the proposed hierarchical model can be expressed as a
three-level Gaussian scale mixture and this representation facilitates the use
of a Gibbs sampling algorithm for posterior simulation. Simulation studies
demonstrate that the interval estimates obtained using our approach achieve
adequate coverage probabilities that outperform those obtained from the
nonparametric bootstrap. Our proposed methodology is applied to the analysis of
neuroimaging and genetic data collected as part of the Alzheimer's Disease
Neuroimaging Initiative (ADNI), and this analysis of the ADNI cohort
demonstrates clearly the value added of incorporating interval estimation
beyond only point estimation when relating SNPs to brain imaging
endophenotypes.
Black-box Importance Sampling
Authors: Qiang Liu, Jason D. Lee
Date: 2016-10-17 00:00:00
Tasks: BIG-bench Machine Learning

Importance sampling is widely used in machine learning and statistics, but
its power is limited by the restriction of using simple proposals for which the
importance weights can be tractably calculated. We address this problem by
studying black-box importance sampling methods that calculate importance
weights for samples generated from any unknown proposal or black-box mechanism.
Our method allows us to use better and richer proposals to solve difficult
problems, and (somewhat counter-intuitively) also has the additional benefit of
improving the estimation accuracy beyond typical importance sampling. Both
theoretical and empirical analyses are provided.
Pre-Translation for Neural Machine Translation
Authors: Jan Niehues, Eunah Cho, Thanh-Le Ha, Alex Waibel
Date: 2016-10-17 00:00:00
Proceeding: COLING 2016 12
Tasks: Machine Translation, NMT, Sentence, Translation

Recently, the development of neural machine translation (NMT) has
significantly improved the translation quality of automatic machine
translation. While most sentences are more accurate and fluent than
translations by statistical machine translation (SMT)-based systems, in some
cases, the NMT system produces translations that have a completely different
meaning. This is especially the case when rare words occur.
  When using statistical machine translation, it has already been shown that
significant gains can be achieved by simplifying the input in a preprocessing
step. A commonly used example is the pre-reordering approach.
  In this work, we used phrase-based machine translation to pre-translate the
input into the target language. Then a neural machine translation system
generates the final hypothesis using the pre-translation. Thereby, we use
either only the output of the phrase-based machine translation (PBMT) system or
a combination of the PBMT output and the source sentence.
  We evaluate the technique on the English to German translation task. Using
this approach we are able to outperform the PBMT system as well as the baseline
neural MT system by up to 2 BLEU points. We analyzed the influence of the
quality of the initial system on the final result.
Evolving the Structure of Evolution Strategies
Authors: Sander van Rijn, Hao Wang, Matthijs van Leeuwen, Thomas Bäck
Date: 2016-10-17 00:00:00

Various variants of the well known Covariance Matrix Adaptation Evolution
Strategy (CMA-ES) have been proposed recently, which improve the empirical
performance of the original algorithm by structural modifications. However, in
practice it is often unclear which variation is best suited to the specific
optimization problem at hand. As one approach to tackle this issue, algorithmic
mechanisms attached to CMA-ES variants are considered and extracted as
functional \emph{modules}, allowing for combinations of them. This leads to a
configuration space over ES structures, which enables the exploration of
algorithm structures and paves the way toward novel algorithm generation.
Specifically, eleven modules are incorporated in this framework with two or
three alternative configurations for each module, resulting in $4\,608$
algorithms. A self-adaptive Genetic Algorithm (GA) is used to efficiently
evolve effective ES-structures for given classes of optimization problems,
outperforming any classical CMA-ES variants from literature. The proposed
approach is evaluated on noiseless functions from BBOB suite. Furthermore, such
an observation is again confirmed on different function groups and
dimensionality, indicating the feasibility of ES configuration on real-world
problem classes.
Learning and Transfer of Modulated Locomotor Controllers
Authors: Nicolas Heess, Greg Wayne, Yuval Tassa, Timothy Lillicrap, Martin Riedmiller, David Silver
Date: 2016-10-17 00:00:00

We study a novel architecture and training procedure for locomotion tasks. A
high-frequency, low-level "spinal" network with access to proprioceptive
sensors learns sensorimotor primitives by training on simple tasks. This
pre-trained module is fixed and connected to a low-frequency, high-level
"cortical" network, with access to all sensors, which drives behavior by
modulating the inputs to the spinal network. Where a monolithic end-to-end
architecture fails completely, learning with a pre-trained spinal module
succeeds at multiple high-level tasks, and enables the effective exploration
required to learn from sparse rewards. We test our proposed architecture on
three simulated bodies: a 16-dimensional swimming snake, a 20-dimensional
quadruped, and a 54-dimensional humanoid. Our results are illustrated in the
accompanying video at https://youtu.be/sboPYvhpraQ
On the Identification and Mitigation of Weaknesses in the Knowledge Gradient Policy for Multi-Armed Bandits
Authors: James Edwards, Paul Fearnhead, Kevin Glazebrook
Date: 2016-07-20 00:00:00
Tasks: Decision Making, Multi-Armed Bandits

The Knowledge Gradient (KG) policy was originally proposed for online ranking
and selection problems but has recently been adapted for use in online decision
making in general and multi-armed bandit problems (MABs) in particular. We
study its use in a class of exponential family MABs and identify weaknesses,
including a propensity to take actions which are dominated with respect to both
exploitation and exploration. We propose variants of KG which avoid such
errors. These new policies include an index heuristic which deploys a KG
approach to develop an approximation to the Gittins index. A numerical study
shows this policy to perform well over a range of MABs including those for
which index policies are not optimal. While KG does not make dominated actions
when bandits are Gaussian, it fails to be index consistent and appears not to
enjoy a performance advantage over competitor policies when arms are correlated
to compensate for its greater computational demands.
Spatio-temporal Gaussian processes modeling of dynamical systems in systems biology
Authors: Mu Niu, Zhenwen Dai, Neil Lawrence, Kolja Becker
Date: 2016-10-17 00:00:00
Tasks: Bayesian Inference, Gaussian Processes

Quantitative modeling of post-transcriptional regulation process is a
challenging problem in systems biology. A mechanical model of the regulatory
process needs to be able to describe the available spatio-temporal protein
concentration and mRNA expression data and recover the continuous
spatio-temporal fields. Rigorous methods are required to identify model
parameters. A promising approach to deal with these difficulties is proposed
using Gaussian process as a prior distribution over the latent function of
protein concentration and mRNA expression. In this study, we consider a partial
differential equation mechanical model with differential operators and latent
function. Since the operators at stake are linear, the information from the
physical model can be encoded into the kernel function. Hybrid Monte Carlo
methods are employed to carry out Bayesian inference of the partial
differential equation parameters and Gaussian process kernel parameters. The
spatio-temporal field of protein concentration and mRNA expression are
reconstructed without explicitly solving the partial differential equation.
The Peaking Phenomenon in Semi-supervised Learning
Authors: Jesse H. Krijthe, Marco Loog
Date: 2016-10-17 00:00:00

For the supervised least squares classifier, when the number of training
objects is smaller than the dimensionality of the data, adding more data to the
training set may first increase the error rate before decreasing it. This,
possibly counterintuitive, phenomenon is known as peaking. In this work, we
observe that a similar but more pronounced version of this phenomenon also
occurs in the semi-supervised setting, where instead of labeled objects,
unlabeled objects are added to the training set. We explain why the learning
curve has a more steep incline and a more gradual decline in this setting
through simulation studies and by applying an approximation of the learning
curve based on the work by Raudys & Duin.
X-CNN: Cross-modal Convolutional Neural Networks for Sparse Datasets
Authors: Petar Veličković, Duo Wang, Nicholas D. Lane, Pietro Liò
Date: 2016-10-01 00:00:00
Tasks: Data Augmentation

In this paper we propose cross-modal convolutional neural networks (X-CNNs),
a novel biologically inspired type of CNN architectures, treating gradient
descent-specialised CNNs as individual units of processing in a larger-scale
network topology, while allowing for unconstrained information flow and/or
weight sharing between analogous hidden layers of the network---thus
generalising the already well-established concept of neural network ensembles
(where information typically may flow only between the output layers of the
individual networks). The constituent networks are individually designed to
learn the output function on their own subset of the input data, after which
cross-connections between them are introduced after each pooling operation to
periodically allow for information exchange between them. This injection of
knowledge into a model (by prior partition of the input data through domain
knowledge or unsupervised methods) is expected to yield greatest returns in
sparse data environments, which are typically less suitable for training CNNs.
For evaluation purposes, we have compared a standard four-layer CNN as well as
a sophisticated FitNet4 architecture against their cross-modal variants on the
CIFAR-10 and CIFAR-100 datasets with differing percentages of the training data
being removed, and find that at lower levels of data availability, the X-CNNs
significantly outperform their baselines (typically providing a 2--6% benefit,
depending on the dataset size and whether data augmentation is used), while
still maintaining an edge on all of the full dataset tests.
Risk-Aware Algorithms for Adversarial Contextual Bandits
Authors: Wen Sun, Debadeepta Dey, Ashish Kapoor
Date: 2016-10-17 00:00:00
Tasks: Multi-Armed Bandits

In this work we consider adversarial contextual bandits with risk
constraints. At each round, nature prepares a context, a cost for each arm, and
additionally a risk for each arm. The learner leverages the context to pull an
arm and then receives the corresponding cost and risk associated with the
pulled arm. In addition to minimizing the cumulative cost, the learner also
needs to satisfy long-term risk constraints -- the average of the cumulative
risk from all pulled arms should not be larger than a pre-defined threshold. To
address this problem, we first study the full information setting where in each
round the learner receives an adversarial convex loss and a convex constraint.
We develop a meta algorithm leveraging online mirror descent for the full
information setting and extend it to contextual bandit with risk constraints
setting using expert advice. Our algorithms can achieve near-optimal regret in
terms of minimizing the total cost, while successfully maintaining a sublinear
growth of cumulative risk constraint violation.
A General Framework for Content-enhanced Network Representation Learning
Authors: Xiaofei Sun, Jiang Guo, Xiao Ding, Ting Liu
Date: 2016-10-10 00:00:00
Tasks: Network Embedding, Node Classification, Representation Learning

This paper investigates the problem of network embedding, which aims at
learning low-dimensional vector representation of nodes in networks. Most
existing network embedding methods rely solely on the network structure, i.e.,
the linkage relationships between nodes, but ignore the rich content
information associated with it, which is common in real world networks and
beneficial to describing the characteristics of a node. In this paper, we
propose content-enhanced network embedding (CENE), which is capable of jointly
leveraging the network structure and the content information. Our approach
integrates text modeling and structure modeling in a general framework by
treating the content information as a special kind of node. Experiments on
several real world net- works with application to node classification show that
our models outperform all existing network embedding methods, demonstrating the
merits of content information and joint learning.
Identifying collusion groups using spectral clustering
Authors: Suneel Sarswat, Kandathil Mathew Abraham, Subir Kumar Ghosh
Date: 2015-09-22 00:00:00
Tasks: Clustering

In an illiquid stock, traders can collude and place orders on a predetermined
price and quantity at a fixed schedule. This is usually done to manipulate the
price of the stock or to create artificial liquidity in the stock, which may
mislead genuine investors. Here, the problem is to identify such group of
colluding traders. We modeled the problem instance as a graph, where each
trader corresponds to a vertex of the graph and trade corresponds to edges of
the graph. Further, we assign weights on edges depending on total volume, total
number of trades, maximum change in the price and commonality between two
vertices. Spectral clustering algorithms are used on the constructed graph to
identify colluding group(s). We have compared our results with simulated data
to show the effectiveness of spectral clustering to detecting colluding groups.
Moreover, we also have used parameters of real data to test the effectiveness
of our algorithm.
Encoding the Local Connectivity Patterns of fMRI for Cognitive State Classification
Authors: Itir Onal Ertugrul, Mete Ozay, Fatos T. Yarman Vural
Date: 2016-10-17 00:00:00
Tasks: ARC, Clustering, General Classification

In this work, we propose a novel framework to encode the local connectivity
patterns of brain, using Fisher Vectors (FV), Vector of Locally Aggregated
Descriptors (VLAD) and Bag-of-Words (BoW) methods. We first obtain local
descriptors, called Mesh Arc Descriptors (MADs) from fMRI data, by forming
local meshes around anatomical regions, and estimating their relationship
within a neighborhood. Then, we extract a dictionary of relationships, called
\textit{brain connectivity dictionary} by fitting a generative Gaussian mixture
model (GMM) to a set of MADs, and selecting the codewords at the mean of each
component of the mixture. Codewords represent the connectivity patterns among
anatomical regions. We also encode MADs by VLAD and BoW methods using the
k-Means clustering.
  We classify the cognitive states of Human Connectome Project (HCP) task fMRI
dataset, where we train support vector machines (SVM) by the encoded MADs.
Results demonstrate that, FV encoding of MADs can be successfully employed for
classification of cognitive tasks, and outperform the VLAD and BoW
representations. Moreover, we identify the significant Gaussians in mixture
models by computing energy of their corresponding FV parts, and analyze their
effect on classification accuracy. Finally, we suggest a new method to
visualize the codewords of brain connectivity dictionary.
SR-Clustering: Semantic Regularized Clustering for Egocentric Photo Streams Segmentation
Authors: Mariella Dimiccoli, Marc Bolaños, Estefania Talavera, Maedeh Aghaei, Stavri G. Nikolov, Petia Radeva
Date: 2015-12-22 00:00:00
Tasks: Clustering

While wearable cameras are becoming increasingly popular, locating relevant
information in large unstructured collections of egocentric images is still a
tedious and time consuming processes. This paper addresses the problem of
organizing egocentric photo streams acquired by a wearable camera into
semantically meaningful segments. First, contextual and semantic information is
extracted for each image by employing a Convolutional Neural Networks approach.
Later, by integrating language processing, a vocabulary of concepts is defined
in a semantic space. Finally, by exploiting the temporal coherence in photo
streams, images which share contextual and semantic attributes are grouped
together. The resulting temporal segmentation is particularly suited for
further analysis, ranging from activity and event recognition to semantic
indexing and summarization. Experiments over egocentric sets of nearly 17,000
images, show that the proposed approach outperforms state-of-the-art methods.
New Ideas for Brain Modelling
Authors: Kieran Greer
Date: 2014-03-05 00:00:00

This paper describes some biologically-inspired processes that could be used
to build the sort of networks that we associate with the human brain. New to
this paper, a 'refined' neuron will be proposed. This is a group of neurons
that by joining together can produce a more analogue system, but with the same
level of control and reliability that a binary neuron would have. With this new
structure, it will be possible to think of an essentially binary system in
terms of a more variable set of values. The paper also shows how recent
research associated with the new model, can be combined with established
theories, to produce a more complete picture. The propositions are largely in
line with conventional thinking, but possibly with one or two more radical
suggestions. An earlier cognitive model can be filled in with more specific
details, based on the new research results, where the components appear to fit
together almost seamlessly. The intention of the research has been to describe
plausible 'mechanical' processes that can produce the appropriate brain
structures and mechanisms, but that could be used without the magical
'intelligence' part that is still not fully understood. There are also some
important updates from an earlier version of this paper.
Weekly maintenance scheduling using exact and genetic methods
Authors: Andrew W. Palmer, Robin Vujanic, Andrew J. Hill, Steven J. Scheding
Date: 2016-10-17 00:00:00
Tasks: Scheduling

The weekly maintenance schedule specifies when maintenance activities should
be performed on the equipment, taking into account the availability of workers
and maintenance bays, and other operational constraints. The current approach
to generating this schedule is labour intensive and requires coordination
between the maintenance schedulers and operations staff to minimise its impact
on the operation of the mine. This paper presents methods for automatically
generating this schedule from the list of maintenance tasks to be performed,
the availability roster of the maintenance staff, and time windows in which
each piece of equipment is available for maintenance. Both Mixed-Integer Linear
Programming (MILP) and genetic algorithms are evaluated, with the genetic
algorithm shown to significantly outperform the MILP. Two fitness functions for
the genetic algorithm are also examined, with a linear fitness function
outperforming an inverse fitness function by up to 5% for the same calculation
time. The genetic algorithm approach is computationally fast, allowing the
schedule to be rapidly recalculated in response to unexpected delays and
breakdowns.
Interactive Attention for Neural Machine Translation
Authors: Fandong Meng, Zhengdong Lu, Hang Li, Qun Liu
Date: 2016-10-17 00:00:00
Proceeding: COLING 2016 12
Tasks: Decoder, Machine Translation, NMT, Sentence, Translation

Conventional attention-based Neural Machine Translation (NMT) conducts
dynamic alignment in generating the target sentence. By repeatedly reading the
representation of source sentence, which keeps fixed after generated by the
encoder (Bahdanau et al., 2015), the attention mechanism has greatly enhanced
state-of-the-art NMT. In this paper, we propose a new attention mechanism,
called INTERACTIVE ATTENTION, which models the interaction between the decoder
and the representation of source sentence during translation by both reading
and writing operations. INTERACTIVE ATTENTION can keep track of the interaction
history and therefore improve the translation performance. Experiments on NIST
Chinese-English translation task show that INTERACTIVE ATTENTION can achieve
significant improvements over both the previous attention-based NMT baseline
and some state-of-the-art variants of attention-based NMT (i.e., coverage
models (Tu et al., 2016)). And neural machine translator with our INTERACTIVE
ATTENTION can outperform the open source attention-based NMT system Groundhog
by 4.22 BLEU points and the open source phrase-based system Moses by 3.94 BLEU
points averagely on multiple test sets.