pwc-archive/papers-with-abstracts huggingface dataset:
Autoencoder Regularized Network For Driving Style Representation Learning
Authors: Weishan Dong, Ting Yuan, Kai Yang, Changsheng Li, Shilei Zhang
Date: 2017-01-05 00:00:00
Tasks: Driver Identification, Representation Learning

In this paper, we study learning generalized driving style representations
from automobile GPS trip data. We propose a novel Autoencoder Regularized deep
neural Network (ARNet) and a trip encoding framework trip2vec to learn drivers'
driving styles directly from GPS records, by combining supervised and
unsupervised feature learning in a unified architecture. Experiments on a
challenging driver number estimation problem and the driver identification
problem show that ARNet can learn a good generalized driving style
representation: It significantly outperforms existing methods and alternative
architectures by reaching the least estimation error on average (0.68, less
than one driver) and the highest identification accuracy (by at least 3%
improvement) compared with traditional supervised learning methods.
Universal, Unsupervised (Rule-Based), Uncovered Sentiment Analysis
Authors: David Vilares, Carlos Gómez-Rodríguez, Miguel A. Alonso
Date: 2016-06-17 00:00:00
Tasks: Sentiment Analysis

We present a novel unsupervised approach for multilingual sentiment analysis
driven by compositional syntax-based rules. On the one hand, we exploit some of
the main advantages of unsupervised algorithms: (1) the interpretability of
their output, in contrast with most supervised models, which behave as a black
box and (2) their robustness across different corpora and domains. On the other
hand, by introducing the concept of compositional operations and exploiting
syntactic information in the form of universal dependencies, we tackle one of
their main drawbacks: their rigidity on data that are structured differently
depending on the language concerned. Experiments show an improvement both over
existing unsupervised methods, and over state-of-the-art supervised models when
evaluating outside their corpus of origin. Experiments also show how the same
compositional operations can be shared across languages. The system is
available at http://www.grupolys.org/software/UUUSA/
Subpopulation Diversity Based Selecting Migration Moment in Distributed Evolutionary Algorithms
Authors: Cheng-Jun Li, Jia Wu
Date: 2017-01-05 00:00:00
Tasks: Diversity, Evolutionary Algorithms, Traveling Salesman Problem

In distributed evolutionary algorithms, migration interval is used to decide
migration moments. Nevertheless, migration moments predetermined by intervals
cannot match the dynamic situation of evolution. In this paper, a scheme of
setting the success rate of migration based on subpopulation diversity at each
interval is proposed. With the scheme, migration still occurs at intervals, but
the probability of immigrants entering the target subpopulation will be
determined by the diversity of this subpopulation according to a proposed
formula. An analysis shows that the time consumption of our scheme is
acceptable. In our experiments, the basement of parallelism is an evolutionary
algorithm for the traveling salesman problem. Under different value
combinations of parameters for the formula, outcomes for eight benchmark
instances of the distributed evolutionary algorithm with the proposed scheme
are compared with those of a traditional one, respectively. Results show that
the distributed evolutionary algorithm based on our scheme has a significant
advantage on solutions especially for high difficulty instances. Moreover, it
can be seen that the algorithm with the scheme has the most outstanding
performance under three value combinations of above-mentioned parameters for
the formula.
Combining Existential Rules and Transitivity: Next Steps
Authors: Jean-François Baget, Meghyn Bienvenu, Marie-Laure Mugnier, Swan Rocher
Date: 2015-04-28 00:00:00

We consider existential rules (aka Datalog+) as a formalism for specifying
ontologies. In recent years, many classes of existential rules have been
exhibited for which conjunctive query (CQ) entailment is decidable. However,
most of these classes cannot express transitivity of binary relations, a
frequently used modelling construct. In this paper, we address the issue of
whether transitivity can be safely combined with decidable classes of
existential rules.
  First, we prove that transitivity is incompatible with one of the simplest
decidable classes, namely aGRD (acyclic graph of rule dependencies), which
clarifies the landscape of `finite expansion sets' of rules.
  Second, we show that transitivity can be safely added to linear rules (a
subclass of guarded rules, which generalizes the description logic DL-Lite-R)
in the case of atomic CQs, and also for general CQs if we place a minor
syntactic restriction on the rule set. This is shown by means of a novel query
rewriting algorithm that is specially tailored to handle transitivity rules.
  Third, for the identified decidable cases, we pinpoint the combined and data
complexities of query entailment.
Synthesizing Training Images for Boosting Human 3D Pose Estimation
Authors: Wenzheng Chen, Huan Wang, Yangyan Li, Hao Su, Zhenhua Wang, Changhe Tu, Dani Lischinski, Daniel Cohen-Or, Baoquan Chen
Date: 2016-04-10 00:00:00
Tasks: 2D Pose Estimation, 3D Pose Estimation, Diversity, Domain Adaptation, Pose Estimation

Human 3D pose estimation from a single image is a challenging task with
numerous applications. Convolutional Neural Networks (CNNs) have recently
achieved superior performance on the task of 2D pose estimation from a single
image, by training on images with 2D annotations collected by crowd sourcing.
This suggests that similar success could be achieved for direct estimation of
3D poses. However, 3D poses are much harder to annotate, and the lack of
suitable annotated training images hinders attempts towards end-to-end
solutions. To address this issue, we opt to automatically synthesize training
images with ground truth pose annotations. Our work is a systematic study along
this road. We find that pose space coverage and texture diversity are the key
ingredients for the effectiveness of synthetic training data. We present a
fully automatic, scalable approach that samples the human pose space for
guiding the synthesis procedure and extracts clothing textures from real
images. Furthermore, we explore domain adaptation for bridging the gap between
our synthetic training images and real testing photos. We demonstrate that CNNs
trained with our synthetic images out-perform those trained with real photos on
3D pose estimation tasks.
Adaptive Questionnaires for Direct Identification of Optimal Product Design
Authors: Max Yi Ren, Clayton Scott
Date: 2017-01-05 00:00:00
Tasks: Marketing

We consider the problem of identifying the most profitable product design
from a finite set of candidates under unknown consumer preference. A standard
approach to this problem follows a two-step strategy: First, estimate the
preference of the consumer population, represented as a point in part-worth
space, using an adaptive discrete-choice questionnaire. Second, integrate the
estimated part-worth vector with engineering feasibility and cost models to
determine the optimal design. In this work, we (1) demonstrate that accurate
preference estimation is neither necessary nor sufficient for identifying the
optimal design, (2) introduce a novel adaptive questionnaire that leverages
knowledge about engineering feasibility and manufacturing costs to directly
determine the optimal design, and (3) interpret product design in terms of a
nonlinear segmentation of part-worth space, and use this interpretation to
illuminate the intrinsic difficulty of optimal design in the presence of noisy
questionnaire responses. We establish the superiority of the proposed approach
using a well-documented optimal product design task. This study demonstrates
how the identification of optimal product design can be accelerated by
integrating marketing and manufacturing knowledge into the adaptive
questionnaire.
Overlapping Cover Local Regression Machines
Authors: Mohamed Elhoseiny, Ahmed Elgammal
Date: 2017-01-05 00:00:00
Tasks: Gaussian Processes, GPR, Pose Estimation, regression

We present the Overlapping Domain Cover (ODC) notion for kernel machines, as
a set of overlapping subsets of the data that covers the entire training set
and optimized to be spatially cohesive as possible. We show how this notion
benefit the speed of local kernel machines for regression in terms of both
speed while achieving while minimizing the prediction error. We propose an
efficient ODC framework, which is applicable to various regression models and
in particular reduces the complexity of Twin Gaussian Processes (TGP)
regression from cubic to quadratic. Our notion is also applicable to several
kernel methods (e.g., Gaussian Process Regression(GPR) and IWTGP regression, as
shown in our experiments). We also theoretically justified the idea behind our
method to improve local prediction by the overlapping cover. We validated and
analyzed our method on three benchmark human pose estimation datasets and
interesting findings are discussed.
A Review of Neural Network Based Machine Learning Approaches for Rotor Angle Stability Control
Authors: Reza Yousefian, Sukumar Kamalasadan
Date: 2017-01-05 00:00:00
Tasks: BIG-bench Machine Learning, energy management, Management, Reinforcement Learning, Reinforcement Learning (RL)

This paper reviews the current status and challenges of Neural Networks (NNs)
based machine learning approaches for modern power grid stability control
including their design and implementation methodologies. NNs are widely
accepted as Artificial Intelligence (AI) approaches offering an alternative way
to control complex and ill-defined problems. In this paper various application
of NNs for power system rotor angle stabilization and control problem is
discussed. The main focus of this paper is on the use of Reinforcement Learning
(RL) and Supervised Learning (SL) algorithms in power system wide-area control
(WAC). Generally, these algorithms due to their capability in modeling
nonlinearities and uncertainties are used for transient classification,
neuro-control, wide-area monitoring and control, renewable energy management
and control, and so on. The works of researchers in the field of conventional
and renewable energy systems are reported and categorized. Paper concludes by
presenting, comparing and evaluating various learning techniques and
infrastructure configurations based on efficiency.
Learning Semidefinite Regularizers
Authors: Yong Sheng Soh, Venkat Chandrasekaran
Date: 2017-01-05 00:00:00
Tasks: Dictionary Learning

Regularization techniques are widely employed in optimization-based approaches for solving ill-posed inverse problems in data analysis and scientific computing. These methods are based on augmenting the objective with a penalty function, which is specified based on prior domain-specific expertise to induce a desired structure in the solution. We consider the problem of learning suitable regularization functions from data in settings in which precise domain knowledge is not directly available. Previous work under the title of `dictionary learning' or `sparse coding' may be viewed as learning a regularization function that can be computed via linear programming. We describe generalizations of these methods to learn regularizers that can be computed and optimized via semidefinite programming. Our framework for learning such semidefinite regularizers is based on obtaining structured factorizations of data matrices, and our algorithmic approach for computing these factorizations combines recent techniques for rank minimization problems along with an operator analog of Sinkhorn scaling. Under suitable conditions on the input data, our algorithm provides a locally linearly convergent method for identifying the correct regularizer that promotes the type of structure contained in the data. Our analysis is based on the stability properties of Operator Sinkhorn scaling and their relation to geometric aspects of determinantal varieties (in particular tangent spaces with respect to these varieties). The regularizers obtained using our framework can be employed effectively in semidefinite programming relaxations for solving inverse problems.
Estimation of low rank density matrices by Pauli measurements
Authors: Dong Xia
Date: 2016-10-16 00:00:00
Tasks: Quantum State Tomography

Density matrices are positively semi-definite Hermitian matrices with unit
trace that describe the states of quantum systems. Many quantum systems of
physical interest can be represented as high-dimensional low rank density
matrices. A popular problem in {\it quantum state tomography} (QST) is to
estimate the unknown low rank density matrix of a quantum system by conducting
Pauli measurements. Our main contribution is twofold. First, we establish the
minimax lower bounds in Schatten $p$-norms with $1\leq p\leq +\infty$ for low
rank density matrices estimation by Pauli measurements. In our previous paper,
these minimax lower bounds are proved under the trace regression model with
Gaussian noise and the noise is assumed to have common variance. In this paper,
we prove these bounds under the Binomial observation model which meets the
actual model in QST.
  Second, we study the Dantzig estimator (DE) for estimating the unknown low
rank density matrix under the Binomial observation model by using Pauli
measurements. In our previous papers, we studied the least squares estimator
and the projection estimator, where we proved the optimal convergence rates for
the least squares estimator in Schatten $p$-norms with $1\leq p\leq 2$ and,
under a stronger condition, the optimal convergence rates for the projection
estimator in Schatten $p$-norms with $1\leq p\leq +\infty$. In this paper, we
show that the results of these two distinct estimators can be simultaneously
obtained by the Dantzig estimator. Moreover, better convergence rates in
Schatten norm distances can be proved for Dantzig estimator under conditions
weaker than those needed in previous papers. When the objective function of DE
is replaced by the negative von Neumann entropy, we obtain sharp convergence
rate in Kullback-Leibler divergence.
Textual Entailment with Structured Attentions and Composition
Authors: Kai Zhao, Liang Huang, Mingbo Ma
Date: 2017-01-04 00:00:00
Proceeding: COLING 2016 12
Tasks: Natural Language Inference, Relation

Deep learning techniques are increasingly popular in the textual entailment
task, overcoming the fragility of traditional discrete models with hard
alignments and logics. In particular, the recently proposed attention models
(Rockt\"aschel et al., 2015; Wang and Jiang, 2015) achieves state-of-the-art
accuracy by computing soft word alignments between the premise and hypothesis
sentences. However, there remains a major limitation: this line of work
completely ignores syntax and recursion, which is helpful in many traditional
efforts. We show that it is beneficial to extend the attention model to tree
nodes between premise and hypothesis. More importantly, this subtree-level
attention reveals information about entailment relation. We study the recursive
composition of this subtree-level entailment relation, which can be viewed as a
soft version of the Natural Logic framework (MacCartney and Manning, 2009).
Experiments show that our structured attention and entailment composition model
can correctly identify and infer entailment relations from the bottom up, and
bring significant improvements in accuracy.
Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
Authors: Jiajun Wu, Chengkai Zhang, Tianfan Xue, William T. Freeman, Joshua B. Tenenbaum
Date: 2016-10-24 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: 3D Object Recognition, 3D Point Cloud Linear Classification, Generative Adversarial Network, Object, Object Recognition, Unsupervised 3D Point Cloud Linear Evaluation

We study the problem of 3D object generation. We propose a novel framework,
namely 3D Generative Adversarial Network (3D-GAN), which generates 3D objects
from a probabilistic space by leveraging recent advances in volumetric
convolutional networks and generative adversarial nets. The benefits of our
model are three-fold: first, the use of an adversarial criterion, instead of
traditional heuristic criteria, enables the generator to capture object
structure implicitly and to synthesize high-quality 3D objects; second, the
generator establishes a mapping from a low-dimensional probabilistic space to
the space of 3D objects, so that we can sample objects without a reference
image or CAD models, and explore the 3D object manifold; third, the adversarial
discriminator provides a powerful 3D shape descriptor which, learned without
supervision, has wide applications in 3D object recognition. Experiments
demonstrate that our method generates high-quality 3D objects, and our
unsupervisedly learned features achieve impressive performance on 3D object
recognition, comparable with those of supervised learning methods.
Private Incremental Regression
Authors: Shiva Prasad Kasiviswanathan, Kobbi Nissim, Hongxia Jin
Date: 2017-01-04 00:00:00
Tasks: BIG-bench Machine Learning, regression

Data is continuously generated by modern data sources, and a recent challenge
in machine learning has been to develop techniques that perform well in an
incremental (streaming) setting. In this paper, we investigate the problem of
private machine learning, where as common in practice, the data is not given at
once, but rather arrives incrementally over time.
  We introduce the problems of private incremental ERM and private incremental
regression where the general goal is to always maintain a good empirical risk
minimizer for the history observed under differential privacy. Our first
contribution is a generic transformation of private batch ERM mechanisms into
private incremental ERM mechanisms, based on a simple idea of invoking the
private batch ERM procedure at some regular time intervals. We take this
construction as a baseline for comparison. We then provide two mechanisms for
the private incremental regression problem. Our first mechanism is based on
privately constructing a noisy incremental gradient function, which is then
used in a modified projected gradient procedure at every timestep. This
mechanism has an excess empirical risk of $\approx\sqrt{d}$, where $d$ is the
dimensionality of the data. While from the results of [Bassily et al. 2014]
this bound is tight in the worst-case, we show that certain geometric
properties of the input and constraint set can be used to derive significantly
better results for certain interesting regression problems.
Reward Augmented Maximum Likelihood for Neural Structured Prediction
Authors: Mohammad Norouzi, Samy Bengio, Zhifeng Chen, Navdeep Jaitly, Mike Schuster, Yonghui Wu, Dale Schuurmans
Date: 2016-09-01 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Machine Translation, Prediction, speech-recognition, Speech Recognition, Structured Prediction, Translation

A key problem in structured output prediction is direct optimization of the
task reward function that matters for test evaluation. This paper presents a
simple and computationally efficient approach to incorporate task reward into a
maximum likelihood framework. By establishing a link between the log-likelihood
and expected reward objectives, we show that an optimal regularized expected
reward is achieved when the conditional distribution of the outputs given the
inputs is proportional to their exponentiated scaled rewards. Accordingly, we
present a framework to smooth the predictive probability of the outputs using
their corresponding rewards. We optimize the conditional log-probability of
augmented outputs that are sampled proportionally to their exponentiated scaled
rewards. Experiments on neural sequence to sequence models for speech
recognition and machine translation show notable improvements over a maximum
likelihood baseline by using reward augmented maximum likelihood (RAML), where
the rewards are defined as the negative edit distance between the outputs and
the ground truth labels.
Fitted Learning: Models with Awareness of their Limits
Authors: Navid Kardan, Kenneth O. Stanley
Date: 2016-09-07 00:00:00
Tasks: Classification, General Classification

Though deep learning has pushed the boundaries of classification forward, in
recent years hints of the limits of standard classification have begun to
emerge. Problems such as fooling, adding new classes over time, and the need to
retrain learning models only for small changes to the original problem all
point to a potential shortcoming in the classic classification regime, where a
comprehensive a priori knowledge of the possible classes or concepts is
critical. Without such knowledge, classifiers misjudge the limits of their
knowledge and overgeneralization therefore becomes a serious obstacle to
consistent performance. In response to these challenges, this paper extends the
classic regime by reframing classification instead with the assumption that
concepts present in the training set are only a sample of the hypothetical
final set of concepts. To bring learning models into this new paradigm, a novel
elaboration of standard architectures called the competitive overcomplete
output layer (COOL) neural network is introduced. Experiments demonstrate the
effectiveness of COOL by applying it to fooling, separable concept learning,
one-class neural networks, and standard classification benchmarks. The results
suggest that, unlike conventional classifiers, the amount of generalization in
COOL networks can be tuned to match the problem.
Word Emdeddings through Hellinger PCA
Authors: Rémi Lebret, Ronan Collobert
Date: 2013-12-19 00:00:00
Tasks: NER, Word Embeddings

Word embeddings resulting from neural language models have been shown to be
successful for a large variety of NLP tasks. However, such architecture might
be difficult to train and time-consuming. Instead, we propose to drastically
simplify the word embeddings computation through a Hellinger PCA of the word
co-occurence matrix. We compare those new word embeddings with some well-known
embeddings on NER and movie review tasks and show that we can reach similar or
even better performance. Although deep learning is not really necessary for
generating good word embeddings, we show that it can provide an easy way to
adapt embeddings to specific tasks.
Stochastic Planning and Lifted Inference
Authors: Roni Khardon, Scott Sanner
Date: 2017-01-04 00:00:00
Tasks: Decision Making, Sequential Decision Making

Lifted probabilistic inference (Poole, 2003) and symbolic dynamic programming
for lifted stochastic planning (Boutilier et al, 2001) were introduced around
the same time as algorithmic efforts to use abstraction in stochastic systems.
Over the years, these ideas evolved into two distinct lines of research, each
supported by a rich literature. Lifted probabilistic inference focused on
efficient arithmetic operations on template-based graphical models under a
finite domain assumption while symbolic dynamic programming focused on
supporting sequential decision-making in rich quantified logical action models
and on open domain reasoning. Given their common motivation but different focal
points, both lines of research have yielded highly complementary innovations.
In this chapter, we aim to help close the gap between these two research areas
by providing an overview of lifted stochastic planning from the perspective of
probabilistic inference, showing strong connections to other chapters in this
book. This also allows us to define Generalized Lifted Inference as a paradigm
that unifies these areas and elucidates open problems for future research that
can benefit both lifted inference and stochastic planning.