pwc-archive/papers-with-abstracts huggingface dataset:
Discrepancy-based Evolutionary Diversity Optimization
Authors: Aneta Neumann, Wanru Gao, Carola Doerr, Frank Neumann, Markus Wagner
Date: 2018-02-15 00:00:00
Tasks: Combinatorial Optimization, Diversity, Evolutionary Algorithms

Diversity plays a crucial role in evolutionary computation. While diversity
has been mainly used to prevent the population of an evolutionary algorithm
from premature convergence, the use of evolutionary algorithms to obtain a
diverse set of solutions has gained increasing attention in recent years.
Diversity optimization in terms of features on the underlying problem allows to
obtain a better understanding of possible solutions to the problem at hand and
can be used for algorithm selection when dealing with combinatorial
optimization problems such as the Traveling Salesperson Problem. We explore the
use of the star-discrepancy measure to guide the diversity optimization process
of an evolutionary algorithm.
  In our experimental investigations, we consider our discrepancy-based
diversity optimization approaches for evolving diverse sets of images as well
as instances of the Traveling Salesperson problem where a local search is not
able to find near optimal solutions. Our experimental investigations comparing
three diversity optimization approaches show that a discrepancy-based diversity
optimization approach using a tie-breaking rule based on weighted differences
to surrounding feature points provides the best results in terms of the star
discrepancy measure.
History PCA: A New Algorithm for Streaming PCA
Authors: Puyudi Yang, Cho-Jui Hsieh, Jane-Ling Wang
Date: 2018-02-15 00:00:00

In this paper we propose a new algorithm for streaming principal component
analysis. With limited memory, small devices cannot store all the samples in
the high-dimensional regime. Streaming principal component analysis aims to
find the $k$-dimensional subspace which can explain the most variation of the
$d$-dimensional data points that come into memory sequentially. In order to
deal with large $d$ and large $N$ (number of samples), most streaming PCA
algorithms update the current model using only the incoming sample and then
dump the information right away to save memory. However the information
contained in previously streamed data could be useful. Motivated by this idea,
we develop a new streaming PCA algorithm called History PCA that achieves this
goal. By using $O(Bd)$ memory with $B\approx 10$ being the block size, our
algorithm converges much faster than existing streaming PCA algorithms. By
changing the number of inner iterations, the memory usage can be further
reduced to $O(d)$ while maintaining a comparable convergence speed. We provide
theoretical guarantees for the convergence of our algorithm along with the rate
of convergence. We also demonstrate on synthetic and real world data sets that
our algorithm compares favorably with other state-of-the-art streaming PCA
methods in terms of the convergence speed and performance.
Multilingual Speech Recognition With A Single End-To-End Model
Authors: Shubham Toshniwal, Tara N. Sainath, Ron J. Weiss, Bo Li, Pedro Moreno, Eugene Weinstein, Kanishka Rao
Date: 2017-11-06 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Language Modeling, Language Modelling, speech-recognition, Speech Recognition

Training a conventional automatic speech recognition (ASR) system to support
multiple languages is challenging because the sub-word unit, lexicon and word
inventories are typically language specific. In contrast, sequence-to-sequence
models are well suited for multilingual ASR because they encapsulate an
acoustic, pronunciation and language model jointly in a single network. In this
work we present a single sequence-to-sequence ASR model trained on 9 different
Indian languages, which have very little overlap in their scripts.
Specifically, we take a union of language-specific grapheme sets and train a
grapheme-based sequence-to-sequence model jointly on data from all languages.
We find that this model, which is not explicitly given any information about
language identity, improves recognition performance by 21% relative compared to
analogous sequence-to-sequence models trained on each language individually. By
modifying the model to accept a language identifier as an additional input
feature, we further improve performance by an additional 7% relative and
eliminate confusion between different languages.
On the Theory of Variance Reduction for Stochastic Gradient Monte Carlo
Authors: Niladri S. Chatterji, Nicolas Flammarion, Yi-An Ma, Peter L. Bartlett, Michael. I. Jordan
Date: 2018-02-15 00:00:00
Proceeding: ICML 2018 7

We provide convergence guarantees in Wasserstein distance for a variety of
variance-reduction methods: SAGA Langevin diffusion, SVRG Langevin diffusion
and control-variate underdamped Langevin diffusion. We analyze these methods
under a uniform set of assumptions on the log-posterior distribution, assuming
it to be smooth, strongly convex and Hessian Lipschitz. This is achieved by a
new proof technique combining ideas from finite-sum optimization and the
analysis of sampling methods. Our sharp theoretical bounds allow us to identify
regimes of interest where each method performs better than the others. Our
theory is verified with experiments on real-world and synthetic datasets.
Blind Source Separation with Optimal Transport Non-negative Matrix Factorization
Authors: Antoine Rolet, Vivien Seguy, Mathieu Blondel, Hiroshi Sawada
Date: 2018-02-15 00:00:00
Tasks: blind source separation

Optimal transport as a loss for machine learning optimization problems has
recently gained a lot of attention. Building upon recent advances in
computational optimal transport, we develop an optimal transport non-negative
matrix factorization (NMF) algorithm for supervised speech blind source
separation (BSS). Optimal transport allows us to design and leverage a cost
between short-time Fourier transform (STFT) spectrogram frequencies, which
takes into account how humans perceive sound. We give empirical evidence that
using our proposed optimal transport NMF leads to perceptually better results
than Euclidean NMF, for both isolated voice reconstruction and BSS tasks.
Finally, we demonstrate how to use optimal transport for cross domain sound
processing tasks, where frequencies represented in the input spectrograms may
be different from one spectrogram to another.
A Very Low Resource Language Speech Corpus for Computational Language Documentation Experiments
Authors: P. Godard, G. Adda, M. Adda-Decker, J. Benjumea, L. Besacier, J. Cooper-Leavitt, G-N. Kouarata, L. Lamel, H. Maynard, M. Mueller, A. Rialland, S. Stueker, F. Yvon, M. Zanon-Boito
Date: 2017-10-10 00:00:00
Proceeding: LREC 2018 5

Most speech and language technologies are trained with massive amounts of
speech and text information. However, most of the world languages do not have
such resources or stable orthography. Systems constructed under these almost
zero resource conditions are not only promising for speech technology but also
for computational language documentation. The goal of computational language
documentation is to help field linguists to (semi-)automatically analyze and
annotate audio recordings of endangered and unwritten languages. Example tasks
are automatic phoneme discovery or lexicon discovery from the speech signal.
This paper presents a speech corpus collected during a realistic language
documentation process. It is made up of 5k speech utterances in Mboshi (Bantu
C25) aligned to French text translations. Speech transcriptions are also made
available: they correspond to a non-standard graphemic form close to the
language phonology. We present how the data was collected, cleaned and
processed and we illustrate its use through a zero-resource task: spoken term
discovery. The dataset is made available to the community for reproducible
computational language documentation experiments and their evaluation.
Less is More: Micro-expression Recognition from Video using Apex Frame
Authors: Sze-Teng Liong, John See, KokSheik Wong, Raphael C. -W. Phan
Date: 2016-06-06 00:00:00
Tasks: Micro Expression Recognition, Micro-Expression Recognition, Optical Flow Estimation

Despite recent interest and advances in facial micro-expression research,
there is still plenty room for improvement in terms of micro-expression
recognition. Conventional feature extraction approaches for micro-expression
video consider either the whole video sequence or a part of it, for
representation. However, with the high-speed video capture of micro-expressions
(100-200 fps), are all frames necessary to provide a sufficiently meaningful
representation? Is the luxury of data a bane to accurate recognition? A novel
proposition is presented in this paper, whereby we utilize only two images per
video: the apex frame and the onset frame. The apex frame of a video contains
the highest intensity of expression changes among all frames, while the onset
is the perfect choice of a reference frame with neutral expression. A new
feature extractor, Bi-Weighted Oriented Optical Flow (Bi-WOOF) is proposed to
encode essential expressiveness of the apex frame. We evaluated the proposed
method on five micro-expression databases: CAS(ME)$^2$, CASME II, SMIC-HS,
SMIC-NIR and SMIC-VIS. Our experiments lend credence to our hypothesis, with
our proposed technique achieving a state-of-the-art F1-score recognition
performance of 61% and 62% in the high frame rate CASME II and SMIC-HS
databases respectively.
Cloud No Longer a Silver Bullet, Edge to the Rescue
Authors: Yuhao Zhu, Gu-Yeon Wei, David Brooks
Date: 2018-02-15 00:00:00
Tasks: Position

This paper takes the position that, while cognitive computing today relies
heavily on the cloud, we will soon see a paradigm shift where cognitive
computing primarily happens on network edges. The shift toward edge devices is
fundamentally propelled both by technological constraints in data centers and
wireless network infrastructures, as well as practical considerations such as
privacy and safety. The remainder of this paper lays out our view of how these
constraints will impact future cognitive computing. Bringing cognitive
computing to edge devices opens up several new opportunities and challenges,
some of which demand new solutions and some of which require us to revisit
entrenched techniques in light of new technologies. We close the paper with a
call to action for future research.
Selecting the Best in GANs Family: a Post Selection Inference Framework
Authors: Yao-Hung Hubert Tsai, Makoto Yamada, Denny Wu, Ruslan Salakhutdinov, Ichiro Takeuchi, Kenji Fukumizu
Date: 2018-02-15 00:00:00

"Which Generative Adversarial Networks (GANs) generates the most plausible
images?" has been a frequently asked question among researchers. To address
this problem, we first propose an \emph{incomplete} U-statistics estimate of
maximum mean discrepancy $\mathrm{MMD}_{inc}$ to measure the distribution
discrepancy between generated and real images. $\mathrm{MMD}_{inc}$ enjoys the
advantages of asymptotic normality, computation efficiency, and model
agnosticity. We then propose a GANs analysis framework to select and test the
"best" member in GANs family using the Post Selection Inference (PSI) with
$\mathrm{MMD}_{inc}$. In the experiments, we adopt the proposed framework on 7
GANs variants and compare their $\mathrm{MMD}_{inc}$ scores.
"Dependency Bottleneck" in Auto-encoding Architectures: an Empirical Study
Authors: Denny Wu, Yixiu Zhao, Yao-Hung Hubert Tsai, Makoto Yamada, Ruslan Salakhutdinov
Date: 2018-02-15 00:00:00
Tasks: Density Estimation

Recent works investigated the generalization properties in deep neural
networks (DNNs) by studying the Information Bottleneck in DNNs. However, the
mea- surement of the mutual information (MI) is often inaccurate due to the
density estimation. To address this issue, we propose to measure the dependency
instead of MI between layers in DNNs. Specifically, we propose to use
Hilbert-Schmidt Independence Criterion (HSIC) as the dependency measure, which
can measure the dependence of two random variables without estimating
probability densities. Moreover, HSIC is a special case of the Squared-loss
Mutual Information (SMI). In the experiment, we empirically evaluate the
generalization property using HSIC in both the reconstruction and prediction
auto-encoding (AE) architectures.
Learning Markov Chain in Unordered Dataset
Authors: Yao-Hung Hubert Tsai, Han Zhao, Ruslan Salakhutdinov, Nebojsa Jojic
Date: 2017-11-08 00:00:00
Proceeding: ICLR 2018 1

The assumption that data samples are independently identically distributed is
the backbone of many learning algorithms. Nevertheless, datasets often exhibit
rich structure in practice, and we argue that there exist some unknown order
within the data instances. In this technical report, we introduce OrderNet that
can be used to extract the order of data instances in an unsupervised way. By
assuming that the instances are sampled from a Markov chain, our goal is to
learn the transitional operator of the underlying Markov chain, as well as the
order by maximizing the generation probability under all possible data
permutations. Specifically, we use neural network as a compact and soft lookup
table to approximate the possibly huge, but discrete transition matrix. This
strategy allows us to amortize the space complexity with a single model.
Furthermore, this simple and compact representation also provides a short
description to the dataset and generalizes to unseen instances as well. To
ensure that the learned Markov chain is ergodic, we propose a greedy batch-wise
permutation scheme that allows fast training. Empirically, we show that
OrderNet is able to discover an order among data instances. We also extend the
proposed OrderNet to one-shot recognition task and demonstrate favorable
results.
High Dimensional Bayesian Optimization Using Dropout
Authors: Cheng Li, Sunil Gupta, Santu Rana, Vu Nguyen, Svetha Venkatesh, Alistair Shilton
Date: 2018-02-15 00:00:00
Tasks: Bayesian Optimization, global-optimization, Vocal Bursts Intensity Prediction

Scaling Bayesian optimization to high dimensions is challenging task as the
global optimization of high-dimensional acquisition function can be expensive
and often infeasible. Existing methods depend either on limited active
variables or the additive form of the objective function. We propose a new
method for high-dimensional Bayesian optimization, that uses a dropout strategy
to optimize only a subset of variables at each iteration. We derive theoretical
bounds for the regret and show how it can inform the derivation of our
algorithm. We demonstrate the efficacy of our algorithms for optimization on
two benchmark functions and two real-world applications- training cascade
classifiers and optimizing alloy composition.
Competitive caching with machine learned advice
Authors: Thodoris Lykouris, Sergei Vassilvitskii
Date: 2018-02-15 00:00:00
Proceeding: ICML 2018 7
Tasks: Decision Making, Decision Making Under Uncertainty

Traditional online algorithms encapsulate decision making under uncertainty, and give ways to hedge against all possible future events, while guaranteeing a nearly optimal solution as compared to an offline optimum. On the other hand, machine learning algorithms are in the business of extrapolating patterns found in the data to predict the future, and usually come with strong guarantees on the expected generalization error. In this work we develop a framework for augmenting online algorithms with a machine learned oracle to achieve competitive ratios that provably improve upon unconditional worst case lower bounds when the oracle has low error. Our approach treats the oracle as a complete black box, and is not dependent on its inner workings, or the exact distribution of its errors. We apply this framework to the traditional caching problem -- creating an eviction strategy for a cache of size $k$. We demonstrate that naively following the oracle's recommendations may lead to very poor performance, even when the average error is quite low. Instead we show how to modify the Marker algorithm to take into account the oracle's predictions, and prove that this combined approach achieves a competitive ratio that both (i) decreases as the oracle's error decreases, and (ii) is always capped by $O(\log k)$, which can be achieved without any oracle input. We complement our results with an empirical evaluation of our algorithm on real world datasets, and show that it performs well empirically even using simple off-the-shelf predictions.
Deep Learning and Data Assimilation for Real-Time Production Prediction in Natural Gas Wells
Authors: Kelvin Loh, Pejman Shoeibi Omrani, Ruud van der Linden
Date: 2018-02-14 00:00:00
Tasks: Decision Making, Prediction

The prediction of the gas production from mature gas wells, due to their
complex end-of-life behavior, is challenging and crucial for operational
decision making. In this paper, we apply a modified deep LSTM model for
prediction of the gas flow rates in mature gas wells, including the
uncertainties in input parameters. Additionally, due to changes in the system
in time and in order to increase the accuracy and robustness of the prediction,
the Ensemble Kalman Filter (EnKF) is used to update the flow rate predictions
based on new observations. The developed approach was tested on the data from
two mature gas production wells in which their production is highly dynamic and
suffering from salt deposition. The results show that the flow predictions
using the EnKF updated model leads to better Jeffreys' J-divergences than the
predictions without the EnKF model updating scheme.
Reducing over-clustering via the powered Chinese restaurant process
Authors: Jun Lu, Meng Li, David Dunson
Date: 2018-02-15 00:00:00
Tasks: Clustering

Dirichlet process mixture (DPM) models tend to produce many small clusters
regardless of whether they are needed to accurately characterize the data -
this is particularly true for large data sets. However, interpretability,
parsimony, data storage and communication costs all are hampered by having
overly many clusters. We propose a powered Chinese restaurant process to limit
this kind of problem and penalize over clustering. The method is illustrated
using some simulation examples and data with large and small sample size
including MNIST and the Old Faithful Geyser data.
Shamap: Shape-based Manifold Learning
Authors: Fenglei Fan, Ziyu Su, Yueyang Teng, Ge Wang
Date: 2018-02-15 00:00:00
Tasks: Dimensionality Reduction

For manifold learning, it is assumed that high-dimensional sample/data points are embedded on a low-dimensional manifold. Usually, distances among samples are computed to capture an underlying data structure. Here we propose a metric according to angular changes along a geodesic line, thereby reflecting the underlying shape-oriented information or a topological similarity between high- and low-dimensional representations of a data cloud. Our results demonstrate the feasibility and merits of the proposed dimensionality reduction scheme.
Fooling OCR Systems with Adversarial Text Images
Authors: Congzheng Song, Vitaly Shmatikov
Date: 2018-02-15 00:00:00
Tasks: Adversarial Text, Optical Character Recognition, Optical Character Recognition (OCR)

We demonstrate that state-of-the-art optical character recognition (OCR)
based on deep learning is vulnerable to adversarial images. Minor modifications
to images of printed text, which do not change the meaning of the text to a
human reader, cause the OCR system to "recognize" a different text where
certain words chosen by the adversary are replaced by their semantic opposites.
This completely changes the meaning of the output produced by the OCR system
and by the NLP applications that use OCR for preprocessing their inputs.
Deep Learning Based Speech Beamforming
Authors: Kaizhi Qian, Yang Zhang, Shiyu Chang, Xuesong Yang, Dinei Florencio, Mark Hasegawa-Johnson
Date: 2018-02-15 00:00:00
Tasks: Deep Learning, Speech Enhancement

Multi-channel speech enhancement with ad-hoc sensors has been a challenging
task. Speech model guided beamforming algorithms are able to recover natural
sounding speech, but the speech models tend to be oversimplified or the
inference would otherwise be too complicated. On the other hand, deep learning
based enhancement approaches are able to learn complicated speech distributions
and perform efficient inference, but they are unable to deal with variable
number of input channels. Also, deep learning approaches introduce a lot of
errors, particularly in the presence of unseen noise types and settings. We
have therefore proposed an enhancement framework called DEEPBEAM, which
combines the two complementary classes of algorithms. DEEPBEAM introduces a
beamforming filter to produce natural sounding speech, but the filter
coefficients are determined with the help of a monaural speech enhancement
neural network. Experiments on synthetic and real-world data show that DEEPBEAM
is able to produce clean, dry and natural sounding speech, and is robust
against unseen noise.
Popularity-Aware Item Weighting for Long-Tail Recommendation
Authors: Himan Abdollahpouri, Robin Burke, Bamshad Mobasher
Date: 2018-02-15 00:00:00
Tasks: Recommendation Systems

Many recommender systems suffer from the popularity bias problem: popular
items are being recommended frequently while less popular, niche products, are
recommended rarely if not at all. However, those ignored products are exactly
the products that businesses need to find customers for and their
recommendations would be more beneficial. In this paper, we examine an item
weighting approach to improve long-tail recommendation. Our approach works as a
simple yet powerful add-on to existing recommendation algorithms for making a
tunable trade-off between accuracy and long-tail coverage.
On the Resistance of Nearest Neighbor to Random Noisy Labels
Authors: Wei Gao, Bin-Bin Yang, Zhi-Hua Zhou
Date: 2016-07-26 00:00:00

Nearest neighbor has always been one of the most appealing non-parametric
approaches in machine learning, pattern recognition, computer vision, etc.
Previous empirical studies partly shows that nearest neighbor is resistant to
noise, yet there is a lack of deep analysis. This work presents the
finite-sample and distribution-dependent bounds on the consistency of nearest
neighbor in the random noise setting. The theoretical results show that, for
asymmetric noises, k-nearest neighbor is robust enough to classify most data
correctly, except for a handful of examples, whose labels are totally misled by
random noises. For symmetric noises, however, k-nearest neighbor achieves the
same consistent rate as that of noise-free setting, which verifies the
resistance of k-nearest neighbor to random noisy labels. Motivated by the
theoretical analysis, we propose the Robust k-Nearest Neighbor (RkNN) approach
to deal with noisy labels. The basic idea is to make unilateral corrections to
examples, whose labels are totally misled by random noises, and classify the
others directly by utilizing the robustness of k-nearest neighbor. We verify
the effectiveness of the proposed algorithm both theoretically and empirically.