pwc-archive/papers-with-abstracts huggingface dataset:
The Knowledge Gradient Policy Using A Sparse Additive Belief Model
Authors: Yan Li, Han Liu, Warren Powell
Date: 2015-03-18 00:00:00
Tasks: global-optimization

We propose a sequential learning policy for noisy discrete global
optimization and ranking and selection (R\&S) problems with high dimensional
sparse belief functions, where there are hundreds or even thousands of
features, but only a small portion of these features contain explanatory power.
We aim to identify the sparsity pattern and select the best alternative before
the finite budget is exhausted. We derive a knowledge gradient policy for
sparse linear models (KGSpLin) with group Lasso penalty. This policy is a
unique and novel hybrid of Bayesian R\&S with frequentist learning.
Particularly, our method naturally combines B-spline basis expansion and
generalizes to the nonparametric additive model (KGSpAM) and functional ANOVA
model. Theoretically, we provide the estimation error bounds of the posterior
mean estimate and the functional estimate. Controlled experiments show that the
algorithm efficiently learns the correct set of nonzero parameters even when
the model is imbedded with hundreds of dummy parameters. Also it outperforms
the knowledge gradient for a linear model.
Text Segmentation based on Semantic Word Embeddings
Authors: Alexander A. Alemi, Paul Ginsparg
Date: 2015-03-18 00:00:00
Tasks: Articles, Segmentation, Text Segmentation, Word Embeddings

We explore the use of semantic word embeddings in text segmentation
algorithms, including the C99 segmentation algorithm and new algorithms
inspired by the distributed word vector representation. By developing a general
framework for discussing a class of segmentation objectives, we study the
effectiveness of greedy versus exact optimization approaches and suggest a new
iterative refinement technique for improving the performance of greedy
strategies. We compare our results to known benchmarks, using known metrics. We
demonstrate state-of-the-art performance for an untrained method with our
Content Vector Segmentation (CVS) on the Choi test set. Finally, we apply the
segmentation procedure to an in-the-wild dataset consisting of text extracted
from scholarly articles in the arXiv.org database.
On Using Very Large Target Vocabulary for Neural Machine Translation
Authors: Sébastien Jean, Kyunghyun Cho, Roland Memisevic, Yoshua Bengio
Date: 2014-12-05 00:00:00
Proceeding: IJCNLP 2015 7
Tasks: Machine Translation, Translation

Neural machine translation, a recently proposed approach to machine
translation based purely on neural networks, has shown promising results
compared to the existing approaches such as phrase-based statistical machine
translation. Despite its recent success, neural machine translation has its
limitation in handling a larger vocabulary, as training complexity as well as
decoding complexity increase proportionally to the number of target words. In
this paper, we propose a method that allows us to use a very large target
vocabulary without increasing training complexity, based on importance
sampling. We show that decoding can be efficiently done even with the model
having a very large target vocabulary by selecting only a small subset of the
whole target vocabulary. The models trained by the proposed approach are
empirically found to outperform the baseline models with a small vocabulary as
well as the LSTM-based neural machine translation models. Furthermore, when we
use the ensemble of a few models with very large target vocabularies, we
achieve the state-of-the-art translation performance (measured by BLEU) on the
English->German translation and almost as high performance as state-of-the-art
English->French translation system.
Exploration of the scalability of LocFaults
Authors: Mohammed Bekkouche
Date: 2015-03-18 00:00:00

A model checker can produce a trace of counterexample, for an erroneous
program, which is often long and difficult to understand. In general, the part
about the loops is the largest among the instructions in this trace. This makes
the location of errors in loops critical, to analyze errors in the overall
program. In this paper, we explore the scalability capabilities of LocFaults,
our error localization approach exploiting paths of CFG(Control Flow Graph)
from a counterexample to calculate the MCDs (Minimal Correction Deviations),
and MCSs (Minimal Correction Subsets) from each found MCD. We present the times
of our approach on programs with While-loops unfolded b times, and a number of
deviated conditions ranging from 0 to n. Our preliminary results show that the
times of our approach, constraint-based and flow-driven, are better compared to
BugAssist which is based on SAT and transforms the entire program to a Boolean
formula, and further the information provided by LocFaults is more expressive
for the user.
Interpretable Aircraft Engine Diagnostic via Expert Indicator Aggregation
Authors: Tsirizo Rabenoro, Jérôme Lacaille, Marie Cottrell, Fabrice Rossi
Date: 2015-03-18 00:00:00
Tasks: Anomaly Detection, Diagnostic, feature selection, Scheduling

Detecting early signs of failures (anomalies) in complex systems is one of
the main goal of preventive maintenance. It allows in particular to avoid
actual failures by (re)scheduling maintenance operations in a way that
optimizes maintenance costs. Aircraft engine health monitoring is one
representative example of a field in which anomaly detection is crucial.
Manufacturers collect large amount of engine related data during flights which
are used, among other applications, to detect anomalies. This article
introduces and studies a generic methodology that allows one to build automatic
early signs of anomaly detection in a way that builds upon human expertise and
that remains understandable by human operators who make the final maintenance
decision. The main idea of the method is to generate a very large number of
binary indicators based on parametric anomaly scores designed by experts,
complemented by simple aggregations of those scores. A feature selection method
is used to keep only the most discriminant indicators which are used as inputs
of a Naive Bayes classifier. This give an interpretable classifier based on
interpretable anomaly detectors whose parameters have been optimized indirectly
by the selection process. The proposed methodology is evaluated on simulated
data designed to reproduce some of the anomaly types observed in real world
engines.
Nonparametric Detection of Nonlinearly Mixed Pixels and Endmember Estimation in Hyperspectral Images
Authors: Tales Imbiriba, José Carlos Moreira Bermudez, Cédric Richard, Jean-Yves Tourneret
Date: 2015-03-18 00:00:00
Tasks: Hyperspectral Unmixing, regression

Mixing phenomena in hyperspectral images depend on a variety of factors such
as the resolution of observation devices, the properties of materials, and how
these materials interact with incident light in the scene. Different parametric
and nonparametric models have been considered to address hyperspectral unmixing
problems. The simplest one is the linear mixing model. Nevertheless, it has
been recognized that mixing phenomena can also be nonlinear. The corresponding
nonlinear analysis techniques are necessarily more challenging and complex than
those employed for linear unmixing. Within this context, it makes sense to
detect the nonlinearly mixed pixels in an image prior to its analysis, and then
employ the simplest possible unmixing technique to analyze each pixel. In this
paper, we propose a technique for detecting nonlinearly mixed pixels. The
detection approach is based on the comparison of the reconstruction errors
using both a Gaussian process regression model and a linear regression model.
The two errors are combined into a detection statistics for which a probability
density function can be reasonably approximated. We also propose an iterative
endmember extraction algorithm to be employed in combination with the detection
algorithm. The proposed Detect-then-Unmix strategy, which consists of
extracting endmembers, detecting nonlinearly mixed pixels and unmixing, is
tested with synthetic and real images.
Equilibrium States in Numerical Argumentation Networks
Authors: D. Gabbay, O. Rodrigues
Date: 2014-08-28 00:00:00

Given an argumentation network with initial values to the arguments, we look
for algorithms which can yield extensions compatible with such initial values.
We find that the best way of tackling this problem is to offer an iteration
formula that takes the initial values and the attack relation and iterates a
sequence of intermediate values that eventually converges leading to an
extension. The properties surrounding the application of the iteration formula
and its connection with other numerical and non-numerical techniques proposed
by others are thoroughly investigated in this paper.
Exploration of the scalability of LocFaults approach for error localization with While-loops programs
Authors: Mohammed Bekkouche
Date: 2015-03-18 00:00:00

A model checker can produce a trace of counterexample, for an erroneous
program, which is often long and difficult to understand. In general, the part
about the loops is the largest among the instructions in this trace. This makes
the location of errors in loops critical, to analyze errors in the overall
program. In this paper, we explore the scala-bility capabilities of LocFaults,
our error localization approach exploiting paths of CFG(Control Flow Graph)
from a counterexample to calculate the MCDs (Minimal Correction Deviations),
and MCSs (Minimal Correction Subsets) from each found MCD. We present the times
of our approach on programs with While-loops unfolded b times, and a number of
deviated conditions ranging from 0 to n. Our preliminary results show that the
times of our approach, constraint-based and flow-driven, are better compared to
BugAssist which is based on SAT and transforms the entire program to a Boolean
formula, and further the information provided by LocFaults is more expressive
for the user.
Probabilistic Argumentation. An Equational Approach
Authors: D. M. Gabbay, O. Rodrigues
Date: 2015-03-18 00:00:00
Tasks: Abstract Argumentation

There is a generic way to add any new feature to a system. It involves 1)
identifying the basic units which build up the system and 2) introducing the
new feature to each of these basic units.
  In the case where the system is argumentation and the feature is
probabilistic we have the following. The basic units are: a. the nature of the
arguments involved; b. the membership relation in the set S of arguments; c.
the attack relation; and d. the choice of extensions.
  Generically to add a new aspect (probabilistic, or fuzzy, or temporal, etc)
to an argumentation network <S,R> can be done by adding this feature to each
component a-d. This is a brute-force method and may yield a non-intuitive or
meaningful result.
  A better way is to meaningfully translate the object system into another
target system which does have the aspect required and then let the target
system endow the aspect on the initial system. In our case we translate
argumentation into classical propositional logic and get probabilistic
argumentation from the translation.
  Of course what we get depends on how we translate.
  In fact, in this paper we introduce probabilistic semantics to abstract
argumentation theory based on the equational approach to argumentation
networks. We then compare our semantics with existing proposals in the
literature including the approaches by M. Thimm and by A. Hunter. Our
methodology in general is discussed in the conclusion.
Homotopy based algorithms for $\ell_0$-regularized least-squares
Authors: Charles Soussen, Jérôme Idier, Junbo Duan, David Brie
Date: 2014-01-31 00:00:00
Tasks: Heuristic Search

Sparse signal restoration is usually formulated as the minimization of a
quadratic cost function $\|y-Ax\|_2^2$, where A is a dictionary and x is an
unknown sparse vector. It is well-known that imposing an $\ell_0$ constraint
leads to an NP-hard minimization problem. The convex relaxation approach has
received considerable attention, where the $\ell_0$-norm is replaced by the
$\ell_1$-norm. Among the many efficient $\ell_1$ solvers, the homotopy
algorithm minimizes $\|y-Ax\|_2^2+\lambda\|x\|_1$ with respect to x for a
continuum of $\lambda$'s. It is inspired by the piecewise regularity of the
$\ell_1$-regularization path, also referred to as the homotopy path. In this
paper, we address the minimization problem $\|y-Ax\|_2^2+\lambda\|x\|_0$ for a
continuum of $\lambda$'s and propose two heuristic search algorithms for
$\ell_0$-homotopy. Continuation Single Best Replacement is a forward-backward
greedy strategy extending the Single Best Replacement algorithm, previously
proposed for $\ell_0$-minimization at a given $\lambda$. The adaptive search of
the $\lambda$-values is inspired by $\ell_1$-homotopy. $\ell_0$ Regularization
Path Descent is a more complex algorithm exploiting the structural properties
of the $\ell_0$-regularization path, which is piecewise constant with respect
to $\lambda$. Both algorithms are empirically evaluated for difficult inverse
problems involving ill-conditioned dictionaries. Finally, we show that they can
be easily coupled with usual methods of model order selection.
Shared latent subspace modelling within Gaussian-Binary Restricted Boltzmann Machines for NIST i-Vector Challenge 2014
Authors: Danila Doroshin, Alexander Yamshinin, Nikolay Lubimov, Marina Nastasenko, Mikhail Kotov, Maxim Tkachenko
Date: 2015-03-18 00:00:00
Tasks: parameter estimation, Speaker Verification

This paper presents a novel approach to speaker subspace modelling based on
Gaussian-Binary Restricted Boltzmann Machines (GRBM). The proposed model is
based on the idea of shared factors as in the Probabilistic Linear Discriminant
Analysis (PLDA). GRBM hidden layer is divided into speaker and channel factors,
herein the speaker factor is shared over all vectors of the speaker. Then
Maximum Likelihood Parameter Estimation (MLE) for proposed model is introduced.
Various new scoring techniques for speaker verification using GRBM are
proposed. The results for NIST i-vector Challenge 2014 dataset are presented.
Variance-Constrained Actor-Critic Algorithms for Discounted and Average Reward MDPs
Authors: Prashanth L. A., Mohammad Ghavamzadeh
Date: 2014-03-25 00:00:00
Tasks: Decision Making, Sequential Decision Making, Traffic Signal Control

In many sequential decision-making problems we may want to manage risk by
minimizing some measure of variability in rewards in addition to maximizing a
standard criterion. Variance related risk measures are among the most common
risk-sensitive criteria in finance and operations research. However, optimizing
many such criteria is known to be a hard problem. In this paper, we consider
both discounted and average reward Markov decision processes. For each
formulation, we first define a measure of variability for a policy, which in
turn gives us a set of risk-sensitive criteria to optimize. For each of these
criteria, we derive a formula for computing its gradient. We then devise
actor-critic algorithms that operate on three timescales - a TD critic on the
fastest timescale, a policy gradient (actor) on the intermediate timescale, and
a dual ascent for Lagrange multipliers on the slowest timescale. In the
discounted setting, we point out the difficulty in estimating the gradient of
the variance of the return and incorporate simultaneous perturbation approaches
to alleviate this. The average setting, on the other hand, allows for an actor
update using compatible features to estimate the gradient of the variance. We
establish the convergence of our algorithms to locally risk-sensitive optimal
policies. Finally, we demonstrate the usefulness of our algorithms in a traffic
signal control application.
IT-map: an Effective Nonlinear Dimensionality Reduction Method for Interactive Clustering
Authors: Teng Qiu, Yong-Jie Li
Date: 2015-01-26 00:00:00
Tasks: Clustering, Dimensionality Reduction

Scientists in many fields have the common and basic need of dimensionality
reduction: visualizing the underlying structure of the massive multivariate
data in a low-dimensional space. However, many dimensionality reduction methods
confront the so-called "crowding problem" that clusters tend to overlap with
each other in the embedding. Previously, researchers expect to avoid that
problem and seek to make clusters maximally separated in the embedding.
However, the proposed in-tree (IT) based method, called IT-map, allows clusters
in the embedding to be locally overlapped, while seeking to make them
distinguishable by some small yet key parts. IT-map provides a simple,
effective and novel solution to cluster-preserving mapping, which makes it
possible to cluster the original data points interactively and thus should be
of general meaning in science and engineering.
Nonparametric Nearest Neighbor Descent Clustering based on Delaunay Triangulation
Authors: Teng Qiu, Yong-Jie Li
Date: 2015-02-17 00:00:00
Tasks: Clustering

In our physically inspired in-tree (IT) based clustering algorithm and the
series after it, there is only one free parameter involved in computing the
potential value of each point. In this work, based on the Delaunay
Triangulation or its dual Voronoi tessellation, we propose a nonparametric
process to compute potential values by the local information. This computation,
though nonparametric, is relatively very rough, and consequently, many local
extreme points will be generated. However, unlike those gradient-based methods,
our IT-based methods are generally insensitive to those local extremes. This
positively demonstrates the superiority of these parametric (previous) and
nonparametric (in this work) IT-based methods.
Global 6DOF Pose Estimation from Untextured 2D City Models
Authors: Clemens Arth, Christian Pirchheim, Jonathan Ventura, Vincent Lepetit
Date: 2015-03-09 00:00:00
Tasks: Pose Estimation, Semantic Segmentation, Translation

We propose a method for estimating the 3D pose for the camera of a mobile
device in outdoor conditions, using only an untextured 2D model. Previous
methods compute only a relative pose using a SLAM algorithm, or require many
registered images, which are cumbersome to acquire. By contrast, our method
returns an accurate, absolute camera pose in an absolute referential using
simple 2D+height maps, which are broadly available, to refine a first estimate
of the pose provided by the device's sensors. We show how to first estimate the
camera absolute orientation from straight line segments, and then how to
estimate the translation by aligning the 2D map with a semantic segmentation of
the input image. We demonstrate the robustness and accuracy of our approach on
a challenging dataset.
Efficient Machine Learning for Big Data: A Review
Authors: O. Y. Al-Jarrah, P. D. Yoo, S Muhaidat, G. K. Karagiannidis, K. Taha
Date: 2015-03-18 00:00:00
Tasks: BIG-bench Machine Learning, General Classification

With the emerging technologies and all associated devices, it is predicted
that massive amount of data will be created in the next few years, in fact, as
much as 90% of current data were created in the last couple of years,a trend
that will continue for the foreseeable future. Sustainable computing studies
the process by which computer engineer/scientist designs computers and
associated subsystems efficiently and effectively with minimal impact on the
environment. However, current intelligent machine-learning systems are
performance driven, the focus is on the predictive/classification accuracy,
based on known properties learned from the training samples. For instance, most
machine-learning-based nonparametric models are known to require high
computational cost in order to find the global optima. With the learning task
in a large dataset, the number of hidden nodes within the network will
therefore increase significantly, which eventually leads to an exponential rise
in computational complexity. This paper thus reviews the theoretical and
experimental data-modeling literature, in large-scale data-intensive fields,
relating to: (1) model efficiency, including computational requirements in
learning, and data-intensive areas structure and design, and introduces (2) new
algorithmic approaches with the least memory requirements and processing to
minimize computational cost, while maintaining/improving its
predictive/classification accuracy and stability.
A warped kernel improving robustness in Bayesian optimization via random embeddings
Authors: Mickaël Binois, David Ginsbourger, Olivier Roustant
Date: 2014-11-13 00:00:00
Tasks: Bayesian Optimization

This works extends the Random Embedding Bayesian Optimization approach by
integrating a warping of the high dimensional subspace within the covariance
kernel. The proposed warping, that relies on elementary geometric
considerations, allows mitigating the drawbacks of the high extrinsic
dimensionality while avoiding the algorithm to evaluate points giving redundant
information. It also alleviates constraints on bound selection for the embedded
domain, thus improving the robustness, as illustrated with a test case with 25
variables and intrinsic dimension 6.
A General Framework for Robust Testing and Confidence Regions in High-Dimensional Quantile Regression
Authors: Tianqi Zhao, Mladen Kolar, Han Liu
Date: 2014-12-30 00:00:00
Tasks: parameter estimation, quantile regression, regression

We propose a robust inferential procedure for assessing uncertainties of
parameter estimation in high-dimensional linear models, where the dimension $p$
can grow exponentially fast with the sample size $n$. Our method combines the
de-biasing technique with the composite quantile function to construct an
estimator that is asymptotically normal. Hence it can be used to construct
valid confidence intervals and conduct hypothesis tests. Our estimator is
robust and does not require the existence of first or second moment of the
noise distribution. It also preserves efficiency in the sense that the worst
case efficiency loss is less than 30\% compared to the square-loss-based
de-biased Lasso estimator. In many cases our estimator is close to or better
than the latter, especially when the noise is heavy-tailed. Our de-biasing
procedure does not require solving the $L_1$-penalized composite quantile
regression. Instead, it allows for any first-stage estimator with desired
convergence rate and empirical sparsity. The paper also provides new proof
techniques for developing theoretical guarantees of inferential procedures with
non-smooth loss functions. To establish the main results, we exploit the local
curvature of the conditional expectation of composite quantile loss and apply
empirical process theories to control the difference between empirical
quantities and their conditional expectations. Our results are established
under weaker assumptions compared to existing work on inference for
high-dimensional quantile regression. Furthermore, we consider a
high-dimensional simultaneous test for the regression parameters by applying
the Gaussian approximation and multiplier bootstrap theories. We also study
distributed learning and exploit the divide-and-conquer estimator to reduce
computation complexity when the sample size is massive. Finally, we provide
empirical results to verify the theory.