pwc-archive/papers-with-abstracts huggingface dataset:
Crowdsourcing with Sparsely Interacting Workers
Authors: Yao Ma, Alex Olshevsky, Venkatesh Saligrama, Csaba Szepesvari
Date: 2017-06-20 00:00:00
Tasks: Binary Classification, Matrix Completion

We consider estimation of worker skills from worker-task interaction data
(with unknown labels) for the single-coin crowd-sourcing binary classification
model in symmetric noise. We define the (worker) interaction graph whose nodes
are workers and an edge between two nodes indicates whether or not the two
workers participated in a common task. We show that skills are asymptotically
identifiable if and only if an appropriate limiting version of the interaction
graph is irreducible and has odd-cycles. We then formulate a weighted rank-one
optimization problem to estimate skills based on observations on an
irreducible, aperiodic interaction graph. We propose a gradient descent scheme
and show that for such interaction graphs estimates converge asymptotically to
the global minimum. We characterize noise robustness of the gradient scheme in
terms of spectral properties of signless Laplacians of the interaction graph.
We then demonstrate that a plug-in estimator based on the estimated skills
achieves state-of-art performance on a number of real-world datasets. Our
results have implications for rank-one matrix completion problem in that
gradient descent can provably recover $W \times W$ rank-one matrices based on
$W+1$ off-diagonal observations of a connected graph with a single odd-cycle.
Spectral learning of dynamic systems from nonequilibrium data
Authors: Hao Wu, Frank Noé
Date: 2016-09-04 00:00:00
Proceeding: NeurIPS 2016 12

Observable operator models (OOMs) and related models are one of the most
important and powerful tools for modeling and analyzing stochastic systems.
They exactly describe dynamics of finite-rank systems and can be efficiently
and consistently estimated through spectral learning under the assumption of
identically distributed data. In this paper, we investigate the properties of
spectral learning without this assumption due to the requirements of analyzing
large-time scale systems, and show that the equilibrium dynamics of a system
can be extracted from nonequilibrium observation data by imposing an
equilibrium constraint. In addition, we propose a binless extension of spectral
learning for continuous data. In comparison with the other continuous-valued
spectral algorithms, the binless algorithm can achieve consistent estimation of
equilibrium dynamics with only linear complexity.
Passive Classification of Source Printer using Text-line-level Geometric Distortion Signatures from Scanned Images of Printed Documents
Authors: Hardik Jain, Gaurav Gupta, Sharad Joshi, Nitin Khanna
Date: 2017-06-20 00:00:00
Tasks: General Classification

In this digital era, one thing that still holds the convention is a printed
archive. Printed documents find their use in many critical domains such as
contract papers, legal tenders and proof of identity documents. As more
advanced printing, scanning and image editing techniques are becoming
available, forgeries on these legal tenders pose a serious threat. Ability to
easily and reliably identify source printer of a printed document can help a
lot in reducing this menace. During printing procedure, printer hardware
introduces certain distortions in printed characters' locations and shapes
which are invisible to naked eyes. These distortions are referred as geometric
distortions, their profile (or signature) is generally unique for each printer
and can be used for printer classification purpose. This paper proposes a set
of features for characterizing text-line-level geometric distortions, referred
as geometric distortion signatures and presents a novel system to use them for
identification of the origin of a printed document. Detailed experiments
performed on a set of thirteen printers demonstrate that the proposed system
achieves state of the art performance and gives much higher accuracy under
small training size constraint. For four training and six test pages of three
different fonts, the proposed method gives 99\% classification accuracy.
Policy Gradient Methods for Reinforcement Learning with Function Approximation and Action-Dependent Baselines
Authors: Philip S. Thomas, Emma Brunskill
Date: 2017-06-20 00:00:00
Proceeding: NeurIPS 1999
Tasks: Policy Gradient Methods, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We show how an action-dependent baseline can be used by the policy gradient
theorem using function approximation, originally presented with
action-independent baselines by (Sutton et al. 2000).
Word-Entity Duet Representations for Document Ranking
Authors: Chenyan Xiong, Jamie Callan, Tie-Yan Liu
Date: 2017-06-20 00:00:00
Tasks: Document Ranking, Learning-To-Rank, Retrieval

This paper presents a word-entity duet framework for utilizing knowledge
bases in ad-hoc retrieval. In this work, the query and documents are modeled by
word-based representations and entity-based representations. Ranking features
are generated by the interactions between the two representations,
incorporating information from the word space, the entity space, and the
cross-space connections through the knowledge graph. To handle the
uncertainties from the automatically constructed entity representations, an
attention-based ranking model AttR-Duet is developed. With back-propagation
from ranking labels, the model learns simultaneously how to demote noisy
entities and how to rank documents with the word-entity duet. Evaluation
results on TREC Web Track ad-hoc task demonstrate that all of the four-way
interactions in the duet are useful, the attention mechanism successfully
steers the model away from noisy entities, and together they significantly
outperform both word-based and entity-based learning to rank systems.
Co-Fusion: Real-time Segmentation, Tracking and Fusion of Multiple Objects
Authors: Martin Rünz, Lourdes Agapito
Date: 2017-06-20 00:00:00
Tasks: Instance Segmentation, Object, Object SLAM, Semantic SLAM, Simultaneous Localization and Mapping

In this paper we introduce Co-Fusion, a dense SLAM system that takes a live
stream of RGB-D images as input and segments the scene into different objects
(using either motion or semantic cues) while simultaneously tracking and
reconstructing their 3D shape in real time. We use a multiple model fitting
approach where each object can move independently from the background and still
be effectively tracked and its shape fused over time using only the information
from pixels associated with that object label. Previous attempts to deal with
dynamic scenes have typically considered moving regions as outliers, and
consequently do not model their shape or track their motion over time. In
contrast, we enable the robot to maintain 3D models for each of the segmented
objects and to improve them over time through fusion. As a result, our system
can enable a robot to maintain a scene description at the object level which
has the potential to allow interactions with its working environment; even in
the case of dynamic scenes.
Analog CMOS-based Resistive Processing Unit for Deep Neural Network Training
Authors: Seyoung Kim, Tayfun Gokmen, Hyung-Min Lee, Wilfried E. Haensch
Date: 2017-06-20 00:00:00
Tasks: CPU, GPU

Recently we have shown that an architecture based on resistive processing
unit (RPU) devices has potential to achieve significant acceleration in deep
neural network (DNN) training compared to today's software-based DNN
implementations running on CPU/GPU. However, currently available device
candidates based on non-volatile memory technologies do not satisfy all the
requirements to realize the RPU concept. Here, we propose an analog CMOS-based
RPU design (CMOS RPU) which can store and process data locally and can be
operated in a massively parallel manner. We analyze various properties of the
CMOS RPU to evaluate the functionality and feasibility for acceleration of DNN
training.
Observational Learning by Reinforcement Learning
Authors: Diana Borsa, Bilal Piot, Rémi Munos, Olivier Pietquin
Date: 2017-06-20 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Observational learning is a type of learning that occurs as a function of
observing, retaining and possibly replicating or imitating the behaviour of
another agent. It is a core mechanism appearing in various instances of social
learning and has been found to be employed in several intelligent species,
including humans. In this paper, we investigate to what extent the explicit
modelling of other agents is necessary to achieve observational learning
through machine learning. Especially, we argue that observational learning can
emerge from pure Reinforcement Learning (RL), potentially coupled with memory.
Through simple scenarios, we demonstrate that an RL agent can leverage the
information provided by the observations of an other agent performing a task in
a shared environment. The other agent is only observed through the effect of
its actions on the environment and never explicitly modeled. Two key aspects
are borrowed from observational learning: i) the observer behaviour needs to
change as a result of viewing a 'teacher' (another agent) and ii) the observer
needs to be motivated somehow to engage in making use of the other agent's
behaviour. The later is naturally modeled by RL, by correlating the learning
agent's reward with the teacher agent's behaviour.
End-to-End Neural Ad-hoc Ranking with Kernel Pooling
Authors: Chenyan Xiong, Zhuyun Dai, Jamie Callan, Zhiyuan Liu, Russell Power
Date: 2017-06-20 00:00:00
Tasks: Ad-Hoc Information Retrieval, Document Ranking, Learning-To-Rank, Translation, Word Embeddings

This paper proposes K-NRM, a kernel based neural model for document ranking.
Given a query and a set of documents, K-NRM uses a translation matrix that
models word-level similarities via word embeddings, a new kernel-pooling
technique that uses kernels to extract multi-level soft match features, and a
learning-to-rank layer that combines those features into the final ranking
score. The whole model is trained end-to-end. The ranking layer learns desired
feature patterns from the pairwise ranking loss. The kernels transfer the
feature patterns into soft-match targets at each similarity level and enforce
them on the translation matrix. The word embeddings are tuned accordingly so
that they can produce the desired soft matches. Experiments on a commercial
search engine's query log demonstrate the improvements of K-NRM over prior
feature-based and neural-based states-of-the-art, and explain the source of
K-NRM's advantage: Its kernel-guided embedding encodes a similarity metric
tailored for matching query words to document words, and provides effective
multi-level soft matches.
A Unified Approach to Adaptive Regularization in Online and Stochastic Optimization
Authors: Vineet Gupta, Tomer Koren, Yoram Singer
Date: 2017-06-20 00:00:00
Tasks: Stochastic Optimization

We describe a framework for deriving and analyzing online optimization
algorithms that incorporate adaptive, data-dependent regularization, also
termed preconditioning. Such algorithms have been proven useful in stochastic
optimization by reshaping the gradients according to the geometry of the data.
Our framework captures and unifies much of the existing literature on adaptive
online methods, including the AdaGrad and Online Newton Step algorithms as well
as their diagonal versions. As a result, we obtain new convergence proofs for
these algorithms that are substantially simpler than previous analyses. Our
framework also exposes the rationale for the different preconditioned updates
used in common stochastic optimization methods.
Inference in Deep Networks in High Dimensions
Authors: Alyson K. Fletcher, Sundeep Rangan
Date: 2017-06-20 00:00:00
Tasks: Vocal Bursts Intensity Prediction

Deep generative networks provide a powerful tool for modeling complex data in
a wide range of applications. In inverse problems that use these networks as
generative priors on data, one must often perform inference of the inputs of
the networks from the outputs. Inference is also required for sampling during
stochastic training on these generative models. This paper considers inference
in a deep stochastic neural network where the parameters (e.g., weights, biases
and activation functions) are known and the problem is to estimate the values
of the input and hidden units from the output. While several approximate
algorithms have been proposed for this task, there are few analytic tools that
can provide rigorous guarantees in the reconstruction error. This work presents
a novel and computationally tractable output-to-input inference method called
Multi-Layer Vector Approximate Message Passing (ML-VAMP). The proposed
algorithm, derived from expectation propagation, extends earlier AMP methods
that are known to achieve the replica predictions for optimality in simple
linear inverse problems. Our main contribution shows that the mean-squared
error (MSE) of ML-VAMP can be exactly predicted in a certain large system limit
(LSL) where the numbers of layers is fixed and weight matrices are random and
orthogonally-invariant with dimensions that grow to infinity. ML-VAMP is thus a
principled method for output-to-input inference in deep networks with a
rigorous and precise performance achievability result in high dimensions.
Fast Learning with Nonconvex L1-2 Regularization
Authors: Quanming Yao, James T. Kwok, Xiawei Guo
Date: 2016-10-29 00:00:00
Tasks: compressed sensing, Sparse Learning

Convex regularizers are often used for sparse learning. They are easy to
optimize, but can lead to inferior prediction performance. The difference of
$\ell_1$ and $\ell_2$ ($\ell_{1-2}$) regularizer has been recently proposed as
a nonconvex regularizer. It yields better recovery than both $\ell_0$ and
$\ell_1$ regularizers on compressed sensing. However, how to efficiently
optimize its learning problem is still challenging. The main difficulty is that
both the $\ell_1$ and $\ell_2$ norms in $\ell_{1-2}$ are not differentiable,
and existing optimization algorithms cannot be applied. In this paper, we show
that a closed-form solution can be derived for the proximal step associated
with this regularizer. We further extend the result for low-rank matrix
learning and the total variation model. Experiments on both synthetic and real
data sets show that the resultant accelerated proximal gradient algorithm is
more efficient than other noncovex optimization algorithms.
A comparative study of breast surface reconstruction for aesthetic outcome assessment
Authors: Rene Lacher, Francisco Vasconcelos, David Bishop, Norman Williams, Mohammed Keshtgar, David Hawkes, John Hipwell, Danail Stoyanov
Date: 2017-06-20 00:00:00
Tasks: 3D Reconstruction, Surface Reconstruction

Breast cancer is the most prevalent cancer type in women, and while its
survival rate is generally high the aesthetic outcome is an increasingly
important factor when evaluating different treatment alternatives. 3D scanning
and reconstruction techniques offer a flexible tool for building detailed and
accurate 3D breast models that can be used both pre-operatively for surgical
planning and post-operatively for aesthetic evaluation. This paper aims at
comparing the accuracy of low-cost 3D scanning technologies with the
significantly more expensive state-of-the-art 3D commercial scanners in the
context of breast 3D reconstruction. We present results from 28 synthetic and
clinical RGBD sequences, including 12 unique patients and an anthropomorphic
phantom demonstrating the applicability of low-cost RGBD sensors to real
clinical cases. Body deformation and homogeneous skin texture pose challenges
to the studied reconstruction systems. Although these should be addressed
appropriately if higher model quality is warranted, we observe that low-cost
sensors are able to obtain valuable reconstructions comparable to the
state-of-the-art within an error margin of 3 mm.
A Divergence Bound for Hybrids of MCMC and Variational Inference and an Application to Langevin Dynamics and SGVI
Authors: Justin Domke
Date: 2017-06-20 00:00:00
Proceeding: ICML 2017 8
Tasks: Variational Inference

Two popular classes of methods for approximate inference are Markov chain
Monte Carlo (MCMC) and variational inference. MCMC tends to be accurate if run
for a long enough time, while variational inference tends to give better
approximations at shorter time horizons. However, the amount of time needed for
MCMC to exceed the performance of variational methods can be quite high,
motivating more fine-grained tradeoffs. This paper derives a distribution over
variational parameters, designed to minimize a bound on the divergence between
the resulting marginal distribution and the target, and gives an example of how
to sample from this distribution in a way that interpolates between the
behavior of existing methods based on Langevin dynamics and stochastic gradient
variational inference (SGVI).
A Hybrid Approach with Multi-channel I-Vectors and Convolutional Neural Networks for Acoustic Scene Classification
Authors: Hamid Eghbal-zadeh, Bernhard Lehner, Matthias Dorfer, Gerhard Widmer
Date: 2017-06-20 00:00:00
Tasks: Acoustic Scene Classification, General Classification, Scene Classification

In Acoustic Scene Classification (ASC) two major approaches have been
followed . While one utilizes engineered features such as
mel-frequency-cepstral-coefficients (MFCCs), the other uses learned features
that are the outcome of an optimization algorithm. I-vectors are the result of
a modeling technique that usually takes engineered features as input. It has
been shown that standard MFCCs extracted from monaural audio signals lead to
i-vectors that exhibit poor performance, especially on indoor acoustic scenes.
At the same time, Convolutional Neural Networks (CNNs) are well known for their
ability to learn features by optimizing their filters. They have been applied
on ASC and have shown promising results. In this paper, we first propose a
novel multi-channel i-vector extraction and scoring scheme for ASC, improving
their performance on indoor and outdoor scenes. Second, we propose a CNN
architecture that achieves promising ASC results. Further, we show that
i-vectors and CNNs capture complementary information from acoustic scenes.
Finally, we propose a hybrid system for ASC using multi-channel i-vectors and
CNNs by utilizing a score fusion technique. Using our method, we participated
in the ASC task of the DCASE-2016 challenge. Our hybrid approach achieved 1 st
rank among 49 submissions, substantially improving the previous state of the
art.
Recovering Dense Tissue Multispectral Signal from in vivo RGB Images
Authors: Jianyu Lin, Neil T. Clancy, Daniel S. Elson
Date: 2017-06-20 00:00:00
Tasks: Decision Making, GPU

Hyperspectral/multispectral imaging (HSI/MSI) contains rich information
clinical applications, such as 1) narrow band imaging for vascular
visualisation; 2) oxygen saturation for intraoperative perfusion monitoring and
clinical decision making [1]; 3) tissue classification and identification of
pathology [2]. The current systems which provide pixel-level HSI/MSI signal can
be generally divided into two types: spatial scanning and spectral scanning.
However, the trade-off between spatial/spectral resolution, the acquisition
time, and the hardware complexity hampers implementation in real-world
applications, especially intra-operatively. Acquiring high resolution images in
real-time is important for HSI/MSI in intra-operative imaging, to alleviate the
side effect caused by breathing, heartbeat, and other sources of motion.
Therefore, we developed an algorithm to recover a pixel-level MSI stack using
only the captured snapshot RGB images from a normal camera. We refer to this
technique as "super-spectral-resolution". The proposed method enables recovery
of pixel-level-dense MSI signals with 24 spectral bands at ~11 frames per
second (FPS) on a GPU. Multispectral data captured from porcine bowel and
sheep/rabbit uteri in vivo has been used for training, and the algorithm has
been validated using unseen in vivo animal experiments.
Unperturbed: spectral analysis beyond Davis-Kahan
Authors: Justin Eldridge, Mikhail Belkin, Yusu Wang
Date: 2017-06-20 00:00:00
Tasks: Clustering

Classical matrix perturbation results, such as Weyl's theorem for eigenvalues
and the Davis-Kahan theorem for eigenvectors, are general purpose. These
classical bounds are tight in the worst case, but in many settings sub-optimal
in the typical case. In this paper, we present perturbation bounds which
consider the nature of the perturbation and its interaction with the
unperturbed structure in order to obtain significant improvements over the
classical theory in many scenarios, such as when the perturbation is random. We
demonstrate the utility of these new results by analyzing perturbations in the
stochastic blockmodel where we derive much tighter bounds than provided by the
classical theory. We use our new perturbation theory to show that a very simple
and natural clustering algorithm -- whose analysis was difficult using the
classical tools -- nevertheless recovers the communities of the blockmodel
exactly even in very sparse graphs.
Two New Approaches to Compressed Sensing Exhibiting Both Robust Sparse Recovery and the Grouping Effect
Authors: Mehmet Eren Ahsen, Niharika Challapalli, Mathukumalli Vidyasagar
Date: 2014-10-30 00:00:00
Tasks: compressed sensing

In this paper we introduce a new optimization formulation for sparse
regression and compressed sensing, called CLOT (Combined L-One and Two),
wherein the regularizer is a convex combination of the $\ell_1$- and
$\ell_2$-norms. This formulation differs from the Elastic Net (EN) formulation,
in which the regularizer is a convex combination of the $\ell_1$- and
$\ell_2$-norm squared. It is shown that, in the context of compressed sensing,
the EN formulation does not achieve robust recovery of sparse vectors, whereas
the new CLOT formulation achieves robust recovery. Also, like EN but unlike
LASSO, the CLOT formulation achieves the grouping effect, wherein coefficients
of highly correlated columns of the measurement (or design) matrix are assigned
roughly comparable values. It is already known LASSO does not have the grouping
effect. Therefore the CLOT formulation combines the best features of both LASSO
(robust sparse recovery) and EN (grouping effect).
  The CLOT formulation is a special case of another one called SGL (Sparse
Group LASSO) which was introduced into the literature previously, but without
any analysis of either the grouping effect or robust sparse recovery. It is
shown here that SGL achieves robust sparse recovery, and also achieves a
version of the grouping effect in that coefficients of highly correlated
columns belonging to the same group of the measurement (or design) matrix are
assigned roughly comparable values.