pwc-archive/papers-with-abstracts huggingface dataset:
Asynchronous Distributed Variational Gaussian Processes for Regression
Authors: Hao Peng, Shandian Zhe, Yuan Qi
Date: 2017-04-22 00:00:00
Proceeding: ICML 2017
Tasks: Gaussian Processes, regression

Gaussian processes (GPs) are powerful non-parametric function estimators.
However, their applications are largely limited by the expensive computational
cost of the inference procedures. Existing stochastic or distributed
synchronous variational inferences, although have alleviated this issue by
scaling up GPs to millions of samples, are still far from satisfactory for
real-world large applications, where the data sizes are often orders of
magnitudes larger, say, billions. To solve this problem, we propose ADVGP, the
first Asynchronous Distributed Variational Gaussian Process inference for
regression, on the recent large-scale machine learning platform,
PARAMETERSERVER. ADVGP uses a novel, flexible variational framework based on a
weight space augmentation, and implements the highly efficient, asynchronous
proximal gradient optimization. While maintaining comparable or better
predictive performance, ADVGP greatly improves upon the efficiency of the
existing variational methods. With ADVGP, we effortlessly scale up GP
regression to a real-world application with billions of samples and demonstrate
an excellent, superior prediction accuracy to the popular linear models.
The loss surface of deep and wide neural networks
Authors: Quynh Nguyen, Matthias Hein
Date: 2017-04-26 00:00:00
Proceeding: ICML 2017 8

While the optimization problem behind deep neural networks is highly
non-convex, it is frequently observed in practice that training deep networks
seems possible without getting stuck in suboptimal points. It has been argued
that this is the case as all local minima are close to being globally optimal.
We show that this is (almost) true, in fact almost all local minima are
globally optimal, for a fully connected network with squared loss and analytic
activation function given that the number of hidden units of one layer of the
network is larger than the number of training points and the network structure
from this layer on is pyramidal.
Query-by-Example Search with Discriminative Neural Acoustic Word Embeddings
Authors: Shane Settle, Keith Levin, Herman Kamper, Karen Livescu
Date: 2017-06-12 00:00:00
Tasks: Dynamic Time Warping, Word Embeddings

Query-by-example search often uses dynamic time warping (DTW) for comparing
queries and proposed matching segments. Recent work has shown that comparing
speech segments by representing them as fixed-dimensional vectors --- acoustic
word embeddings --- and measuring their vector distance (e.g., cosine distance)
can discriminate between words more accurately than DTW-based approaches. We
consider an approach to query-by-example search that embeds both the query and
database segments according to a neural model, followed by nearest-neighbor
search to find the matching segments. Earlier work on embedding-based
query-by-example, using template-based acoustic word embeddings, achieved
competitive performance. We find that our embeddings, based on recurrent neural
networks trained to optimize word discrimination, achieve substantial
improvements in performance and run-time efficiency over the previous
approaches.
Acoustic data-driven lexicon learning based on a greedy pronunciation selection framework
Authors: Xiaohui Zhang, Vimal Manohar, Daniel Povey, Sanjeev Khudanpur
Date: 2017-06-12 00:00:00
Tasks: speech-recognition, Speech Recognition

Speech recognition systems for irregularly-spelled languages like English
normally require hand-written pronunciations. In this paper, we describe a
system for automatically obtaining pronunciations of words for which
pronunciations are not available, but for which transcribed data exists. Our
method integrates information from the letter sequence and from the acoustic
evidence. The novel aspect of the problem that we address is the problem of how
to prune entries from such a lexicon (since, empirically, lexicons with too
many entries do not tend to be good for ASR performance). Experiments on
various ASR tasks show that, with the proposed framework, starting with an
initial lexicon of several thousand words, we are able to learn a lexicon which
performs close to a full expert lexicon in terms of WER performance on test
data, and is better than lexicons built using G2P alone or with a pruning
criterion based on pronunciation probability.
Portable Trust: biometric-based authentication and blockchain storage for self-sovereign identity systems
Authors: J. S. Hammudoglu, J. Sparreboom, J. I. Rauhamaa, J. K. Faber, L. C. Guerchi, I. P. Samiotis, S. P. Rao, J. A. Pouwelse
Date: 2017-06-12 00:00:00

We devised a mobile biometric-based authentication system only relying on
local processing. Our Android open source solution explores the capability of
current smartphones to acquire, process and match fingerprints using only its
built-in hardware. Our architecture is specifically designed to run completely
locally and autonomously, not requiring any cloud service, server, or
permissioned access to fingerprint reader hardware. It involves three main
stages, starting with the fingerprint acquisition using the smartphone camera,
followed by a processing pipeline to obtain minutiae features and a final step
for matching against other locally stored fingerprints, based on Oriented FAST
and Rotated BRIEF (ORB) descriptors. We obtained a mean matching accuracy of
55%, with the highest value of 67% for thumb fingers. Our ability to capture
and process a finger fingerprint in mere seconds using a smartphone makes this
work usable in a wide range of scenarios, for instance, offline remote regions.
This work is specifically designed to be a key building block for a
self-sovereign identity solution and integrate with our permissionless
blockchain for identity and key attestation.
Large-Scale Plant Classification with Deep Neural Networks
Authors: Ignacio Heredia
Date: 2017-06-12 00:00:00
Tasks: General Classification

This paper discusses the potential of applying deep learning techniques for
plant classification and its usage for citizen science in large-scale
biodiversity monitoring. We show that plant classification using near
state-of-the-art convolutional network architectures like ResNet50 achieves
significant improvements in accuracy compared to the most widespread plant
classification application in test sets composed of thousands of different
species labels. We find that the predictions can be confidently used as a
baseline classification in citizen science communities like iNaturalist (or its
Spanish fork, Natusfera) which in turn can share their data with biodiversity
portals like GBIF.
Transferring a Semantic Representation for Person Re-Identification and Search
Authors: Zhiyuan Shi, Timothy M. Hospedales, Tao Xiang
Date: 2017-06-12 00:00:00
Proceeding: CVPR 2015 6
Tasks: Attribute, Person Re-Identification, Person Search

Learning semantic attributes for person re-identification and
description-based person search has gained increasing interest due to
attributes' great potential as a pose and view-invariant representation.
However, existing attribute-centric approaches have thus far underperformed
state-of-the-art conventional approaches. This is due to their non-scalable
need for extensive domain (camera) specific annotation. In this paper we
present a new semantic attribute learning approach for person re-identification
and search. Our model is trained on existing fashion photography datasets --
either weakly or strongly labelled. It can then be transferred and adapted to
provide a powerful semantic description of surveillance person detections,
without requiring any surveillance domain supervision. The resulting
representation is useful for both unsupervised and supervised person
re-identification, achieving state-of-the-art and near state-of-the-art
performance respectively. Furthermore, as a semantic representation it allows
description-based person search to be integrated within the same framework.
Incorporating Pass-Phrase Dependent Background Models for Text-Dependent Speaker Verification
Authors: A. K. Sarkar, Zheng-Hua Tan
Date: 2016-11-19 00:00:00
Tasks: Speaker Verification, Text-Dependent Speaker Verification

In this paper, we propose pass-phrase dependent background models (PBMs) for
text-dependent (TD) speaker verification (SV) to integrate the pass-phrase
identification process into the conventional TD-SV system, where a PBM is
derived from a text-independent background model through adaptation using the
utterances of a particular pass-phrase. During training, pass-phrase specific
target speaker models are derived from the particular PBM using the training
data for the respective target model. While testing, the best PBM is first
selected for the test utterance in the maximum likelihood (ML) sense and the
selected PBM is then used for the log likelihood ratio (LLR) calculation with
respect to the claimant model. The proposed method incorporates the pass-phrase
identification step in the LLR calculation, which is not considered in
conventional standalone TD-SV systems. The performance of the proposed method
is compared to conventional text-independent background model based TD-SV
systems using either Gaussian mixture model (GMM)-universal background model
(UBM) or Hidden Markov model (HMM)-UBM or i-vector paradigms. In addition, we
consider two approaches to build PBMs: speaker-independent and
speaker-dependent. We show that the proposed method significantly reduces the
error rates of text-dependent speaker verification for the non-target types:
target-wrong and imposter-wrong while it maintains comparable TD-SV performance
when imposters speak a correct utterance with respect to the conventional
system. Experiments are conducted on the RedDots challenge and the RSR2015
databases that consist of short utterances.
Neuromorphic Silicon Photonic Networks
Authors: Alexander N. Tait, Thomas Ferreira de Lima, Ellen Zhou, Allie X. Wu, Mitchell A. Nahmias, Bhavin J. Shastri, Paul R. Prucnal
Date: 2016-11-05 00:00:00

Photonic systems for high-performance information processing have attracted
renewed interest. Neuromorphic silicon photonics has the potential to integrate
processing functions that vastly exceed the capabilities of electronics. We
report first observations of a recurrent silicon photonic neural network, in
which connections are configured by microring weight banks. A mathematical
isomorphism between the silicon photonic circuit and a continuous neural
network model is demonstrated through dynamical bifurcation analysis.
Exploiting this isomorphism, a simulated 24-node silicon photonic neural
network is programmed using "neural compiler" to solve a differential system
emulation task. A 294-fold acceleration against a conventional benchmark is
predicted. We also propose and derive power consumption analysis for
modulator-class neurons that, as opposed to laser-class neurons, are compatible
with silicon photonic platforms. At increased scale, Neuromorphic silicon
photonics could access new regimes of ultrafast information processing for
radio, control, and scientific computing.
Progressive and Multi-Path Holistically Nested Neural Networks for Pathological Lung Segmentation from CT Images
Authors: Adam P. Harrison, Ziyue Xu, Kevin George, Le Lu, Ronald M. Summers, Daniel J. Mollura
Date: 2017-06-12 00:00:00

Pathological lung segmentation (PLS) is an important, yet challenging,
medical image application due to the wide variability of pathological lung
appearance and shape. Because PLS is often a pre-requisite for other imaging
analytics, methodological simplicity and generality are key factors in
usability. Along those lines, we present a bottom-up deep-learning based
approach that is expressive enough to handle variations in appearance, while
remaining unaffected by any variations in shape. We incorporate the deeply
supervised learning framework, but enhance it with a simple, yet effective,
progressive multi-path scheme, which more reliably merges outputs from
different network stages. The result is a deep model able to produce finer
detailed masks, which we call progressive holistically-nested networks
(P-HNNs). Using extensive cross-validation, our method is tested on
multi-institutional datasets comprising 929 CT scans (848 publicly available),
of pathological lungs, reporting mean dice scores of 0.985 and demonstrating
significant qualitative and quantitative improvements over state-of-the art
approaches.
A New Method to Visualize Deep Neural Networks
Authors: Luisa M. Zintgraf, Taco S. Cohen, Max Welling
Date: 2016-03-08 00:00:00
Tasks: Decision Making

We present a method for visualising the response of a deep neural network to
a specific input. For image data for instance our method will highlight areas
that provide evidence in favor of, and against choosing a certain class. The
method overcomes several shortcomings of previous methods and provides great
additional insight into the decision making process of convolutional networks,
which is important both to improve models and to accelerate the adoption of
such methods in e.g. medicine. In experiments on ImageNet data, we illustrate
how the method works and can be applied in different ways to understand deep
neural nets.
Optimal resampling for the noisy OneMax problem
Authors: Jialin Liu, Michael Fairbank, Diego Pérez-Liébana, Simon M. Lucas
Date: 2016-07-22 00:00:00

The OneMax problem is a standard benchmark optimisation problem for a binary
search space. Recent work on applying a Bandit-Based Random Mutation
Hill-Climbing algorithm to the noisy OneMax Problem showed that it is important
to choose a good value for the resampling number to make a careful trade off
between taking more samples in order to reduce noise, and taking fewer samples
to reduce the total computational cost. This paper extends that observation, by
deriving an analytical expression for the running time of the RMHC algorithm
with resampling applied to the noisy OneMax problem, and showing both
theoretically and empirically that the optimal resampling number increases with
the number of dimensions in the search space.
Lifelong Metric Learning
Authors: Gan Sun, Yang Cong, Ji Liu, Xiaowei Xu
Date: 2017-05-03 00:00:00
Tasks: Lifelong learning, Metric Learning, Model Optimization

The state-of-the-art online learning approaches are only capable of learning
the metric for predefined tasks. In this paper, we consider lifelong learning
problem to mimic "human learning", i.e., endowing a new capability to the
learned metric for a new task from new online samples and incorporating
previous experiences and knowledge. Therefore, we propose a new metric learning
framework: lifelong metric learning (LML), which only utilizes the data of the
new task to train the metric model while preserving the original capabilities.
More specifically, the proposed LML maintains a common subspace for all learned
metrics, named lifelong dictionary, transfers knowledge from the common
subspace to each new metric task with task-specific idiosyncrasy, and redefines
the common subspace over time to maximize performance across all metric tasks.
For model optimization, we apply online passive aggressive optimization
algorithm to solve the proposed LML framework, where the lifelong dictionary
and task-specific partition are optimized alternatively and consecutively.
Finally, we evaluate our approach by analyzing several multi-task metric
learning datasets. Extensive experimental results demonstrate effectiveness and
efficiency of the proposed framework.
A Richer Theory of Convex Constrained Optimization with Reduced Projections and Improved Rates
Authors: Tianbao Yang, Qihang Lin, Lijun Zhang
Date: 2016-08-11 00:00:00
Proceeding: ICML 2017 8
Tasks: Metric Learning

This paper focuses on convex constrained optimization problems, where the
solution is subject to a convex inequality constraint. In particular, we aim at
challenging problems for which both projection into the constrained domain and
a linear optimization under the inequality constraint are time-consuming, which
render both projected gradient methods and conditional gradient methods (a.k.a.
the Frank-Wolfe algorithm) expensive. In this paper, we develop projection
reduced optimization algorithms for both smooth and non-smooth optimization
with improved convergence rates under a certain regularity condition of the
constraint function. We first present a general theory of optimization with
only one projection. Its application to smooth optimization with only one
projection yields $O(1/\epsilon)$ iteration complexity, which improves over the
$O(1/\epsilon^2)$ iteration complexity established before for non-smooth
optimization and can be further reduced under strong convexity. Then we
introduce a local error bound condition and develop faster algorithms for
non-strongly convex optimization at the price of a logarithmic number of
projections. In particular, we achieve an iteration complexity of $\widetilde
O(1/\epsilon^{2(1-\theta)})$ for non-smooth optimization and $\widetilde
O(1/\epsilon^{1-\theta})$ for smooth optimization, where $\theta\in(0,1]$
appearing the local error bound condition characterizes the functional local
growth rate around the optimal solutions. Novel applications in solving the
constrained $\ell_1$ minimization problem and a positive semi-definite
constrained distance metric learning problem demonstrate that the proposed
algorithms achieve significant speed-up compared with previous algorithms.
Fractional Langevin Monte Carlo: Exploring Lévy Driven Stochastic Differential Equations for Markov Chain Monte Carlo
Authors: Umut Şimşekli
Date: 2017-06-12 00:00:00
Tasks: Efficient Exploration

Along with the recent advances in scalable Markov Chain Monte Carlo methods,
sampling techniques that are based on Langevin diffusions have started
receiving increasing attention. These so called Langevin Monte Carlo (LMC)
methods are based on diffusions driven by a Brownian motion, which gives rise
to Gaussian proposal distributions in the resulting algorithms. Even though
these approaches have proven successful in many applications, their performance
can be limited by the light-tailed nature of the Gaussian proposals. In this
study, we extend classical LMC and develop a novel Fractional LMC (FLMC)
framework that is based on a family of heavy-tailed distributions, called
$\alpha$-stable L\'{e}vy distributions. As opposed to classical approaches, the
proposed approach can possess large jumps while targeting the correct
distribution, which would be beneficial for efficient exploration of the state
space. We develop novel computational methods that can scale up to large-scale
problems and we provide formal convergence analysis of the proposed scheme. Our
experiments support our theory: FLMC can provide superior performance in
multi-modal settings, improved convergence rates, and robustness to algorithm
parameters.
Sequence-to-Sequence Models Can Directly Translate Foreign Speech
Authors: Ron J. Weiss, Jan Chorowski, Navdeep Jaitly, Yonghui Wu, Zhifeng Chen
Date: 2017-03-24 00:00:00
Tasks: Decoder, Machine Translation, Sequence-To-Sequence Speech Recognition, speech-recognition, Speech Recognition, Translation

We present a recurrent encoder-decoder deep neural network architecture that
directly translates speech in one language into text in another. The model does
not explicitly transcribe the speech into text in the source language, nor does
it require supervision from the ground truth source language transcription
during training. We apply a slightly modified sequence-to-sequence with
attention architecture that has previously been used for speech recognition and
show that it can be repurposed for this more complex task, illustrating the
power of attention-based models. A single model trained end-to-end obtains
state-of-the-art performance on the Fisher Callhome Spanish-English speech
translation task, outperforming a cascade of independently trained
sequence-to-sequence speech recognition and machine translation models by 1.8
BLEU points on the Fisher test set. In addition, we find that making use of the
training data in both languages by multi-task training sequence-to-sequence
speech translation and recognition models with a shared encoder network can
improve performance by a further 1.4 BLEU points.
Characterization of an inconsistency ranking for pairwise comparison matrices
Authors: László Csató
Date: 2016-10-24 00:00:00

Pairwise comparisons between alternatives are a well-known method for
measuring preferences of a decision-maker. Since these often do not exhibit
consistency, a number of inconsistency indices has been introduced in order to
measure the deviation from this ideal case. We axiomatically characterize the
inconsistency ranking induced by the Koczkodaj inconsistency index: six
independent properties are presented such that they determine a unique linear
order on the set of all pairwise comparison matrices.
Automatic Selection of Context Configurations for Improved Class-Specific Word Representations
Authors: Ivan Vulić, Roy Schwartz, Ari Rappoport, Roi Reichart, Anna Korhonen
Date: 2016-08-19 00:00:00
Proceeding: CONLL 2017 8
Tasks: Word Similarity

This paper is concerned with identifying contexts useful for training word
representation models for different word classes such as adjectives (A), verbs
(V), and nouns (N). We introduce a simple yet effective framework for an
automatic selection of class-specific context configurations. We construct a
context configuration space based on universal dependency relations between
words, and efficiently search this space with an adapted beam search algorithm.
In word similarity tasks for each word class, we show that our framework is
both effective and efficient. Particularly, it improves the Spearman's rho
correlation with human scores on SimLex-999 over the best previously proposed
class-specific contexts by 6 (A), 6 (V) and 5 (N) rho points. With our selected
context configurations, we train on only 14% (A), 26.2% (V), and 33.6% (N) of
all dependency-based contexts, resulting in a reduced training time. Our
results generalise: we show that the configurations our algorithm learns for
one English training setup outperform previously proposed context types in
another training setup for English. Moreover, basing the configuration space on
universal dependencies, it is possible to transfer the learned configurations
to German and Italian. We also demonstrate improved per-class results over
other context types in these two languages.
Reflectance Adaptive Filtering Improves Intrinsic Image Estimation
Authors: Thomas Nestmeyer, Peter V. Gehler
Date: 2016-12-15 00:00:00
Proceeding: CVPR 2017 7

Separating an image into reflectance and shading layers poses a challenge for
learning approaches because no large corpus of precise and realistic ground
truth decompositions exists. The Intrinsic Images in the Wild~(IIW) dataset
provides a sparse set of relative human reflectance judgments, which serves as
a standard benchmark for intrinsic images. A number of methods use IIW to learn
statistical dependencies between the images and their reflectance layer.
Although learning plays an important role for high performance, we show that a
standard signal processing technique achieves performance on par with current
state-of-the-art. We propose a loss function for CNN learning of dense
reflectance predictions. Our results show a simple pixel-wise decision, without
any context or prior knowledge, is sufficient to provide a strong baseline on
IIW. This sets a competitive baseline which only two other approaches surpass.
We then develop a joint bilateral filtering method that implements strong prior
knowledge about reflectance constancy. This filtering operation can be applied
to any intrinsic image algorithm and we improve several previous results
achieving a new state-of-the-art on IIW. Our findings suggest that the effect
of learning-based approaches may have been over-estimated so far. Explicit
prior knowledge is still at least as important to obtain high performance in
intrinsic image decompositions.