pwc-archive/papers-with-abstracts huggingface dataset:
Encoding Source Language with Convolutional Neural Network for Machine Translation
Authors: Fandong Meng, Zhengdong Lu, Mingxuan Wang, Hang Li, Wenbin Jiang, Qun Liu
Date: 2015-03-06 00:00:00
Proceeding: IJCNLP 2015 7
Tasks: Language Modeling, Language Modelling, Machine Translation, Sentence, Translation

The recently proposed neural network joint model (NNJM) (Devlin et al., 2014)
augments the n-gram target language model with a heuristically chosen source
context window, achieving state-of-the-art performance in SMT. In this paper,
we give a more systematic treatment by summarizing the relevant source
information through a convolutional architecture guided by the target
information. With different guiding signals during decoding, our specifically
designed convolution+gating architectures can pinpoint the parts of a source
sentence that are relevant to predicting a target word, and fuse them with the
context of entire source sentence to form a unified representation. This
representation, together with target language words, are fed to a deep neural
network (DNN) to form a stronger NNJM. Experiments on two NIST Chinese-English
translation tasks show that the proposed model can achieve significant
improvements over the previous NNJM by up to +1.08 BLEU points on average
A Tensor-Based Dictionary Learning Approach to Tomographic Image Reconstruction
Authors: Sara Soltani, Misha E. Kilmer, Per Christian Hansen
Date: 2015-06-08 00:00:00
Tasks: Dictionary Learning, Image Reconstruction

We consider tomographic reconstruction using priors in the form of a
dictionary learned from training images. The reconstruction has two stages:
first we construct a tensor dictionary prior from our training data, and then
we pose the reconstruction problem in terms of recovering the expansion
coefficients in that dictionary. Our approach differs from past approaches in
that a) we use a third-order tensor representation for our images and b) we
recast the reconstruction problem using the tensor formulation. The dictionary
learning problem is presented as a non-negative tensor factorization problem
with sparsity constraints. The reconstruction problem is formulated in a convex
optimization framework by looking for a solution with a sparse representation
in the tensor dictionary. Numerical results show that our tensor formulation
leads to very sparse representations of both the training images and the
reconstructions due to the ability of representing repeated features compactly
in the dictionary.
LOCO: Distributing Ridge Regression with Random Projections
Authors: Christina Heinze, Brian McWilliams, Nicolai Meinshausen, Gabriel Krummenacher
Date: 2014-06-13 00:00:00
Tasks: regression

We propose LOCO, an algorithm for large-scale ridge regression which
distributes the features across workers on a cluster. Important dependencies
between variables are preserved using structured random projections which are
cheap to compute and must only be communicated once. We show that LOCO obtains
a solution which is close to the exact ridge regression solution in the fixed
design setting. We verify this experimentally in a simulation study as well as
an application to climate prediction. Furthermore, we show that LOCO achieves
significant speedups compared with a state-of-the-art distributed algorithm on
a large-scale regression problem.
Video Inpainting of Complex Scenes
Authors: Alasdair Newson, Andrés Almansa, Matthieu Fradet, Yann Gousseau, Patrick Pérez
Date: 2015-03-18 00:00:00
Tasks: Image Inpainting, Video Editing, Video Inpainting

We propose an automatic video inpainting algorithm which relies on the
optimisation of a global, patch-based functional. Our algorithm is able to deal
with a variety of challenging situations which naturally arise in video
inpainting, such as the correct reconstruction of dynamic textures, multiple
moving objects and moving background. Furthermore, we achieve this in an order
of magnitude less execution time with respect to the state-of-the-art. We are
also able to achieve good quality results on high definition videos. Finally,
we provide specific algorithmic details to make implementation of our algorithm
as easy as possible. The resulting algorithm requires no segmentation or manual
input other than the definition of the inpainting mask, and can deal with a
wider variety of situations than is handled by previous work. 1. Introduction.
Advanced image and video editing techniques are increasingly common in the
image processing and computer vision world, and are also starting to be used in
media entertainment. One common and difficult task closely linked to the world
of video editing is image and video " inpainting ". Generally speaking, this is
the task of replacing the content of an image or video with some other content
which is visually pleasing. This subject has been extensively studied in the
case of images, to such an extent that commercial image inpainting products
destined for the general public are available, such as Photoshop's " Content
Aware fill " [1]. However, while some impressive results have been obtained in
the case of videos, the subject has been studied far less extensively than
image inpainting. This relative lack of research can largely be attributed to
high time complexity due to the added temporal dimension. Indeed, it has only
very recently become possible to produce good quality inpainting results on
high definition videos, and this only in a semi-automatic manner. Nevertheless,
high-quality video inpainting has many important and useful applications such
as film restoration, professional post-production in cinema and video editing
for personal use. For this reason, we believe that an automatic, generic video
inpainting algorithm would be extremely useful for both academic and
professional communities.
Microscopic approach of a time elapsed neural model
Authors: Julien Chevallier, Maria J. Caceres, Marie Doumic, Patricia Reynaud-Bouret
Date: 2015-06-08 00:00:00
Tasks: model, Point Processes

The spike trains are the main components of the information processing in the
brain. To model spike trains several point processes have been investigated in
the literature. And more macroscopic approaches have also been studied, using
partial differential equation models. The main aim of the present article is to
build a bridge between several point processes models (Poisson, Wold, Hawkes)
that have been proved to statistically fit real spike trains data and
age-structured partial differential equations as introduced by Pakdaman,
Perthame and Salort.
Policy Gradient for Coherent Risk Measures
Authors: Aviv Tamar, Yin-Lam Chow, Mohammad Ghavamzadeh, Shie Mannor
Date: 2015-02-13 00:00:00
Proceeding: NeurIPS 2015 12
Tasks: Policy Gradient Methods, Reinforcement Learning

Several authors have recently developed risk-sensitive policy gradient
methods that augment the standard expected cost minimization problem with a
measure of variability in cost. These studies have focused on specific
risk-measures, such as the variance or conditional value at risk (CVaR). In
this work, we extend the policy gradient method to the whole class of coherent
risk measures, which is widely accepted in finance and operations research,
among other fields. We consider both static and time-consistent dynamic risk
measures. For static risk measures, our approach is in the spirit of policy
gradient algorithms and combines a standard sampling approach with convex
programming. For dynamic risk measures, our approach is actor-critic style and
involves explicit approximation of value function. Most importantly, our
contribution presents a unified approach to risk-sensitive reinforcement
learning that generalizes and extends previous results.
Path Finding under Uncertainty through Probabilistic Inference
Authors: David Tolpin, Brooks Paige, Jan Willem van de Meent, Frank Wood
Date: 2015-02-25 00:00:00

We introduce a new approach to solving path-finding problems under
uncertainty by representing them as probabilistic models and applying
domain-independent inference algorithms to the models. This approach separates
problem representation from the inference algorithm and provides a framework
for efficient learning of path-finding policies. We evaluate the new approach
on the Canadian Traveler Problem, which we formulate as a probabilistic model,
and show how probabilistic inference allows high performance stochastic
policies to be obtained for this problem.
Convergence Rates of Active Learning for Maximum Likelihood Estimation
Authors: Kamalika Chaudhuri, Sham Kakade, Praneeth Netrapalli, Sujay Sanghavi
Date: 2015-06-08 00:00:00
Proceeding: NeurIPS 2015 12
Tasks: Active Learning, Binary Classification, General Classification, Multi-class Classification, regression

An active learner is given a class of models, a large set of unlabeled
examples, and the ability to interactively query labels of a subset of these
examples; the goal of the learner is to learn a model in the class that fits
the data well.
  Previous theoretical work has rigorously characterized label complexity of
active learning, but most of this work has focused on the PAC or the agnostic
PAC model. In this paper, we shift our attention to a more general setting --
maximum likelihood estimation. Provided certain conditions hold on the model
class, we provide a two-stage active learning algorithm for this problem. The
conditions we require are fairly general, and cover the widely popular class of
Generalized Linear Models, which in turn, include models for binary and
multi-class classification, regression, and conditional random fields.
  We provide an upper bound on the label requirement of our algorithm, and a
lower bound that matches it up to lower order terms. Our analysis shows that
unlike binary classification in the realizable case, just a single extra round
of interaction is sufficient to achieve near-optimal performance in maximum
likelihood estimation. On the empirical side, the recent work in
~\cite{Zhang12} and~\cite{Zhang14} (on active linear and logistic regression)
shows the promise of this approach.
Wavelets and continuous wavelet transform for autostereoscopic multiview images
Authors: Vladimir Saveljev
Date: 2015-06-08 00:00:00

Recently, the reference functions for the synthesis and analysis of the
autostereoscopic multiview and integral images in three-dimensional displays we
introduced. In the current paper, we propose the wavelets to analyze such
images. The wavelets are built on the reference functions as on the scaling
functions of the wavelet analysis. The continuous wavelet transform was
successfully applied to the testing wireframe binary objects. The restored
locations correspond to the structure of the testing wireframe binary objects.
A Multi-layered Acoustic Tokenizing Deep Neural Network (MAT-DNN) for Unsupervised Discovery of Linguistic Units and Generation of High Quality Features
Authors: Cheng-Tao Chung, Cheng-Yu Tsai, Hsiang-Hung Lu, Yuan-ming Liou, Yen-chen Wu, Yen-Ju Lu, Hung-Yi Lee, Lin-shan Lee
Date: 2015-06-07 00:00:00

This paper summarizes the work done by the authors for the Zero Resource
Speech Challenge organized in the technical program of Interspeech 2015. The
goal of the challenge is to discover linguistic units directly from unlabeled
speech data. The Multi-layered Acoustic Tokenizer (MAT) proposed in this work
automatically discovers multiple sets of acoustic tokens from the given corpus.
Each acoustic token set is specified by a set of hyperparameters that describe
the model configuration. These sets of acoustic tokens carry different
characteristics of the given corpus and the language behind thus can be
mutually reinforced. The multiple sets of token labels are then used as the
targets of a Multi-target DNN (MDNN) trained on low-level acoustic features.
Bottleneck features extracted from the MDNN are used as feedback for the MAT
and the MDNN itself. We call this iterative system the Multi-layered Acoustic
Tokenizing Deep Neural Network (MAT-DNN) which generates high quality features
for track 1 of the challenge and acoustic tokens for track 2 of the challenge.
A Framework for Constrained and Adaptive Behavior-Based Agents
Authors: Renato de Pontes Pereira, Paulo Martins Engel
Date: 2015-06-07 00:00:00
Tasks: Hierarchical Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Behavior Trees are commonly used to model agents for robotics and games,
where constrained behaviors must be designed by human experts in order to
guarantee that these agents will execute a specific chain of actions given a
specific set of perceptions. In such application areas, learning is a desirable
feature to provide agents with the ability to adapt and improve interactions
with humans and environment, but often discarded due to its unreliability. In
this paper, we propose a framework that uses Reinforcement Learning nodes as
part of Behavior Trees to address the problem of adding learning capabilities
in constrained agents. We show how this framework relates to Options in
Hierarchical Reinforcement Learning, ensuring convergence of nested learning
nodes, and we empirically show that the learning nodes do not affect the
execution of other nodes in the tree.
Winner-Take-All Autoencoders
Authors: Alireza Makhzani, Brendan Frey
Date: 2014-09-09 00:00:00
Proceeding: NeurIPS 2015 12
Tasks: All

In this paper, we propose a winner-take-all method for learning hierarchical
sparse representations in an unsupervised fashion. We first introduce
fully-connected winner-take-all autoencoders which use mini-batch statistics to
directly enforce a lifetime sparsity in the activations of the hidden units. We
then propose the convolutional winner-take-all autoencoder which combines the
benefits of convolutional architectures and autoencoders for learning
shift-invariant sparse representations. We describe a way to train
convolutional autoencoders layer by layer, where in addition to lifetime
sparsity, a spatial sparsity within each feature map is achieved using
winner-take-all activation functions. We will show that winner-take-all
autoencoders can be used to to learn deep sparse representations from the
MNIST, CIFAR-10, ImageNet, Street View House Numbers and Toronto Face datasets,
and achieve competitive classification performance.
Optimal Ridge Detection using Coverage Risk
Authors: Yen-Chi Chen, Christopher R. Genovese, Shirley Ho, Larry Wasserman
Date: 2015-06-07 00:00:00
Proceeding: NeurIPS 2015 12

We introduce the concept of coverage risk as an error measure for density
ridge estimation. The coverage risk generalizes the mean integrated square
error to set estimation. We propose two risk estimators for the coverage risk
and we show that we can select tuning parameters by minimizing the estimated
risk. We study the rate of convergence for coverage risk and prove consistency
of the risk estimators. We apply our method to three simulated datasets and to
cosmology data. In all the examples, the proposed method successfully recover
the underlying density structure.
Randomized Structural Sparsity based Support Identification with Applications to Locating Activated or Discriminative Brain Areas: A Multi-center Reproducibility Study
Authors: Yi-Lun Wang, Sheng Zhang, Junjie Zheng, Heng Chen, Huafu Chen
Date: 2015-06-07 00:00:00
Tasks: feature selection

In this paper, we focus on how to locate the relevant or discriminative brain
regions related with external stimulus or certain mental decease, which is also
called support identification, based on the neuroimaging data. The main
difficulty lies in the extremely high dimensional voxel space and relatively
few training samples, easily resulting in an unstable brain region discovery
(or called feature selection in context of pattern recognition). When the
training samples are from different centers and have betweencenter variations,
it will be even harder to obtain a reliable and consistent result.
Corresponding, we revisit our recently proposed algorithm based on stability
selection and structural sparsity. It is applied to the multi-center MRI data
analysis for the first time. A consistent and stable result is achieved across
different centers despite the between-center data variation while many other
state-of-the-art methods such as two sample t-test fail. Moreover, we have
empirically showed that the performance of this algorithm is robust and
insensitive to several of its key parameters. In addition, the support
identification results on both functional MRI and structural MRI are
interpretable and can be the potential biomarkers.
Knowledge Transfer Pre-training
Authors: Zhiyuan Tang, Dong Wang, Yiqiao Pan, Zhiyong Zhang
Date: 2015-06-07 00:00:00
Tasks: speech-recognition, Speech Recognition, Transfer Learning

Pre-training is crucial for learning deep neural networks. Most of existing
pre-training methods train simple models (e.g., restricted Boltzmann machines)
and then stack them layer by layer to form the deep structure. This layer-wise
pre-training has found strong theoretical foundation and broad empirical
support. However, it is not easy to employ such method to pre-train models
without a clear multi-layer structure,e.g., recurrent neural networks (RNNs).
This paper presents a new pre-training approach based on knowledge transfer
learning. In contrast to the layer-wise approach which trains model components
incrementally, the new approach trains the entire model as a whole but with an
easier objective function. This is achieved by utilizing soft targets produced
by a prior trained model (teacher model). Compared to the conventional
layer-wise methods, this new method does not care about the model structure, so
can be used to pre-train very complex models. Experiments on a speech
recognition task demonstrated that with this approach, complex RNNs can be well
trained with a weaker deep neural network (DNN) model. Furthermore, the new
method can be combined with conventional layer-wise pre-training to deliver
additional gains.
Reconstruction of recurrent synaptic connectivity of thousands of neurons from simulated spiking activity
Authors: Yury V. Zaytsev, Abigail Morrison, Moritz Deger
Date: 2015-02-17 00:00:00
Tasks: Clustering

Dynamics and function of neuronal networks are determined by their synaptic
connectivity. Current experimental methods to analyze synaptic network
structure on the cellular level, however, cover only small fractions of
functional neuronal circuits, typically without a simultaneous record of
neuronal spiking activity. Here we present a method for the reconstruction of
large recurrent neuronal networks from thousands of parallel spike train
recordings. We employ maximum likelihood estimation of a generalized linear
model of the spiking activity in continuous time. For this model the point
process likelihood is concave, such that a global optimum of the parameters can
be obtained by gradient ascent. Previous methods, including those of the same
class, did not allow recurrent networks of that order of magnitude to be
reconstructed due to prohibitive computational cost and numerical
instabilities. We describe a minimal model that is optimized for large networks
and an efficient scheme for its parallelized numerical optimization on generic
computing clusters. For a simulated balanced random network of 1000 neurons,
synaptic connectivity is recovered with a misclassification error rate of less
than 1% under ideal conditions. We show that the error rate remains low in a
series of example cases under progressively less ideal conditions. Finally, we
successfully reconstruct the connectivity of a hidden synfire chain that is
embedded in a random network, which requires clustering of the network
connectivity to reveal the synfire groups. Our results demonstrate how synaptic
connectivity could potentially be inferred from large-scale parallel spike
train recordings.
String Gaussian Process Kernels
Authors: Yves-Laurent Kom Samo, Stephen Roberts
Date: 2015-06-07 00:00:00
Tasks: Gaussian Processes

We introduce a new class of nonstationary kernels, which we derive as
covariance functions of a novel family of stochastic processes we refer to as
string Gaussian processes (string GPs). We construct string GPs to allow for
multiple types of local patterns in the data, while ensuring a mild global
regularity condition. In this paper, we illustrate the efficacy of the approach
using synthetic data and demonstrate that the model outperforms competing
approaches on well studied, real-life datasets that exhibit nonstationary
features.
Primal Method for ERM with Flexible Mini-batching Schemes and Non-convex Losses
Authors: Dominik Csiba, Peter Richtárik
Date: 2015-06-07 00:00:00

In this work we develop a new algorithm for regularized empirical risk
minimization. Our method extends recent techniques of Shalev-Shwartz [02/2015],
which enable a dual-free analysis of SDCA, to arbitrary mini-batching schemes.
Moreover, our method is able to better utilize the information in the data
defining the ERM problem. For convex loss functions, our complexity results
match those of QUARTZ, which is a primal-dual method also allowing for
arbitrary mini-batching schemes. The advantage of a dual-free analysis comes
from the fact that it guarantees convergence even for non-convex loss
functions, as long as the average loss is convex. We illustrate through
experiments the utility of being able to design arbitrary mini-batching
schemes.
Randomized Structural Sparsity via Constrained Block Subsampling for Improved Sensitivity of Discriminative Voxel Identification
Authors: Yi-Lun Wang, Junjie Zheng, Sheng Zhang, Xujun Duan, Huafu Chen
Date: 2014-10-17 00:00:00
Tasks: feature selection, Sensitivity

In this paper, we consider voxel selection for functional Magnetic Resonance
Imaging (fMRI) brain data with the aim of finding a more complete set of
probably correlated discriminative voxels, thus improving interpretation of the
discovered potential biomarkers. The main difficulty in doing this is an
extremely high dimensional voxel space and few training samples, resulting in
unreliable feature selection. In order to deal with the difficulty, stability
selection has received a great deal of attention lately, especially due to its
finite sample control of false discoveries and transparent principle for
choosing a proper amount of regularization. However, it fails to make explicit
use of the correlation property or structural information of these
discriminative features and leads to large false negative rates. In other
words, many relevant but probably correlated discriminative voxels are missed.
Thus, we propose a new variant on stability selection "randomized structural
sparsity", which incorporates the idea of structural sparsity. Numerical
experiments demonstrate that our method can be superior in controlling for
false negatives while also keeping the control of false positives inherited
from stability selection.
Boosting Optical Character Recognition: A Super-Resolution Approach
Authors: Chao Dong, Ximei Zhu, Yubin Deng, Chen Change Loy, Yu Qiao
Date: 2015-06-07 00:00:00
Tasks: Image Super-Resolution, Optical Character Recognition, Optical Character Recognition (OCR), Super-Resolution

Text image super-resolution is a challenging yet open research problem in the
computer vision community. In particular, low-resolution images hamper the
performance of typical optical character recognition (OCR) systems. In this
article, we summarize our entry to the ICDAR2015 Competition on Text Image
Super-Resolution. Experiments are based on the provided ICDAR2015 TextSR
dataset and the released Tesseract-OCR 3.02 system. We report that our winning
entry of text image super-resolution framework has largely improved the OCR
performance with low-resolution images used as input, reaching an OCR accuracy
score of 77.19%, which is comparable with that of using the original
high-resolution images 78.80%.