pwc-archive/papers-with-abstracts huggingface dataset:
Learning the Information Divergence
Authors: Onur Dikmen, Zhirong Yang, Erkki Oja
Date: 2014-06-05 00:00:00
Tasks: BIG-bench Machine Learning, Topic Models

Information divergence that measures the difference between two nonnegative
matrices or tensors has found its use in a variety of machine learning
problems. Examples are Nonnegative Matrix/Tensor Factorization, Stochastic
Neighbor Embedding, topic models, and Bayesian network optimization. The
success of such a learning task depends heavily on a suitable divergence. A
large variety of divergences have been suggested and analyzed, but very few
results are available for an objective choice of the optimal divergence for a
given task. Here we present a framework that facilitates automatic selection of
the best divergence among a given family, based on standard maximum likelihood
estimation. We first propose an approximated Tweedie distribution for the
beta-divergence family. Selecting the best beta then becomes a machine learning
problem solved by maximum likelihood. Next, we reformulate alpha-divergence in
terms of beta-divergence, which enables automatic selection of alpha by maximum
likelihood with reuse of the learning principle for beta-divergence.
Furthermore, we show the connections between gamma and beta-divergences as well
as R\'enyi and alpha-divergences, such that our automatic selection framework
is extended to non-separable divergences. Experiments on both synthetic and
real-world data demonstrate that our method can quite accurately select
information divergence across different learning problems and various
divergence families.
Basis Identification for Automatic Creation of Pronunciation Lexicon for Proper Names
Authors: Sunil Kumar Kopparapu, M Laxminarayana
Date: 2014-06-05 00:00:00

Development of a proper names pronunciation lexicon is usually a manual
effort which can not be avoided. Grapheme to phoneme (G2P) conversion modules,
in literature, are usually rule based and work best for non-proper names in a
particular language. Proper names are foreign to a G2P module. We follow an
optimization approach to enable automatic construction of proper names
pronunciation lexicon. The idea is to construct a small orthogonal set of words
(basis) which can span the set of names in a given database. We propose two
algorithms for the construction of this basis. The transcription lexicon of all
the proper names in a database can be produced by the manual transcription of
only the small set of basis words. We first construct a cost function and show
that the minimization of the cost function results in a basis. We derive
conditions for convergence of this cost function and validate them
experimentally on a very large proper name database. Experiments show the
transcription can be achieved by transcribing a set of small number of basis
words. The algorithms proposed are generic and independent of language; however
performance is better if the proper names have same origin, namely, same
language or geographical region.
Illusory Shapes via Phase Transition
Authors: Yoon Mo Jung, Jianhong Jackie Shen
Date: 2014-06-05 00:00:00

We propose a new variational illusory shape (VIS) model via phase fields and
phase transitions. It is inspired by the first-order variational illusory
contour (VIC) model proposed by Jung and Shen [{\em J. Visual Comm. Image
Repres.}, {\bf 19}:42-55, 2008]. Under the new VIS model, illusory shapes are
represented by phase values close to 1 while the rest by values close to 0. The
0-1 transition is achieved by an elliptic energy with a double-well potential,
as in the theory of $\Gamma$-convergence. The VIS model is non-convex, with the
zero field as its trivial global optimum. To seek visually meaningful local
optima that can induce illusory shapes, an iterative algorithm is designed and
its convergence behavior is closely studied. Several generic numerical examples
confirm the versatility of the model and the algorithm.
Image retrieval with hierarchical matching pursuit
Authors: Shasha Bu, Yu-Jin Zhang
Date: 2014-06-03 00:00:00
Tasks: Image Retrieval, Retrieval

A novel representation of images for image retrieval is introduced in this
paper, by using a new type of feature with remarkable discriminative power.
Despite the multi-scale nature of objects, most existing models perform feature
extraction on a fixed scale, which will inevitably degrade the performance of
the whole system. Motivated by this, we introduce a hierarchical sparse coding
architecture for image retrieval to explore multi-scale cues. Sparse codes
extracted on lower layers are transmitted to higher layers recursively. With
this mechanism, cues from different scales are fused. Experiments on the
Holidays dataset show that the proposed method achieves an excellent retrieval
performance with a small code length.
Shared Representation Learning for Heterogeneous Face Recognition
Authors: Dong Yi, Zhen Lei, Shengcai Liao, Stan Z. Li
Date: 2014-06-05 00:00:00
Tasks: Face Recognition, Heterogeneous Face Recognition, Representation Learning

After intensive research, heterogenous face recognition is still a challenging problem. The main difficulties are owing to the complex relationship between heterogenous face image spaces. The heterogeneity is always tightly coupled with other variations, which makes the relationship of heterogenous face images highly nonlinear. Many excellent methods have been proposed to model the nonlinear relationship, but they apt to overfit to the training set, due to limited samples. Inspired by the unsupervised algorithms in deep learning, this paper proposes an novel framework for heterogeneous face recognition. We first extract Gabor features at some localized facial points, and then use Restricted Boltzmann Machines (RBMs) to learn a shared representation locally to remove the heterogeneity around each facial point. Finally, the shared representations of local RBMs are connected together and processed by PCA. Two problems (Sketch-Photo and NIR-VIS) and three databases are selected to evaluate the proposed method. For Sketch-Photo problem, we obtain perfect results on the CUFS database. For NIR-VIS problem, we produce new state-of-the-art performance on the CASIA HFB and NIR-VIS 2.0 databases.
Towards the Safety of Human-in-the-Loop Robotics: Challenges and Opportunities for Safety Assurance of Robotic Co-Workers
Authors: Kerstin Eder, Chris Harper, Ute Leonards
Date: 2014-04-08 00:00:00

The success of the human-robot co-worker team in a flexible manufacturing
environment where robots learn from demonstration heavily relies on the correct
and safe operation of the robot. How this can be achieved is a challenge that
requires addressing both technical as well as human-centric research questions.
In this paper we discuss the state of the art in safety assurance, existing as
well as emerging standards in this area, and the need for new approaches to
safety assurance in the context of learning machines. We then focus on robotic
learning from demonstration, the challenges these techniques pose to safety
assurance and indicate opportunities to integrate safety considerations into
algorithms "by design". Finally, from a human-centric perspective, we stipulate
that, to achieve high levels of safety and ultimately trust, the robotic
co-worker must meet the innate expectations of the humans it works with. It is
our aim to stimulate a discussion focused on the safety aspects of
human-in-the-loop robotics, and to foster multidisciplinary collaboration to
address the research challenges identified.
The Best Templates Match Technique For Example Based Machine Translation
Authors: T. El-Shishtawy, A. El-Sammak
Date: 2014-06-04 00:00:00
Tasks: Machine Translation, Translation

It has been proved that large scale realistic Knowledge Based Machine
Translation applications require acquisition of huge knowledge about language
and about the world. This knowledge is encoded in computational grammars,
lexicons and domain models. Another approach which avoids the need for
collecting and analyzing massive knowledge, is the Example Based approach,
which is the topic of this paper. We show through the paper that using Example
Based in its native form is not suitable for translating into Arabic. Therefore
a modification to the basic approach is presented to improve the accuracy of
the translation process. The basic idea of the new approach is to improve the
technique by which template-based approaches select the appropriate templates.
A Geometric Method to Obtain the Generation Probability of a Sentence
Authors: Chen Lijiang
Date: 2014-06-04 00:00:00
Tasks: Sentence

"How to generate a sentence" is the most critical and difficult problem in
all the natural language processing technologies. In this paper, we present a
new approach to explain the generation process of a sentence from the
perspective of mathematics. Our method is based on the premise that in our
brain a sentence is a part of a word network which is formed by many word
nodes. Experiments show that the probability of the entire sentence can be
obtained by the probabilities of single words and the probabilities of the
co-occurrence of word pairs, which indicate that human use the synthesis method
to generate a sentence.
Multi-task Neural Networks for QSAR Predictions
Authors: George E. Dahl, Navdeep Jaitly, Ruslan Salakhutdinov
Date: 2014-06-04 00:00:00

Although artificial neural networks have occasionally been used for
Quantitative Structure-Activity/Property Relationship (QSAR/QSPR) studies in
the past, the literature has of late been dominated by other machine learning
techniques such as random forests. However, a variety of new neural net
techniques along with successful applications in other domains have renewed
interest in network approaches. In this work, inspired by the winning team's
use of neural networks in a recent QSAR competition, we used an artificial
neural network to learn a function that predicts activities of compounds for
multiple assays at the same time. We conducted experiments leveraging recent
methods for dealing with overfitting in neural networks as well as other tricks
from the neural networks literature. We compared our methods to alternative
methods reported to perform well on these tasks and found that our neural net
methods provided superior performance.
A Semantic Approach to Summarization
Authors: Divyanshu Bhartiya, Ashudeep Singh
Date: 2014-06-04 00:00:00
Tasks: Semantic Role Labeling, Sentence

Sentence extraction based summarization methods has some limitations as it
doesn't go into the semantics of the document. Also, it lacks the capability of
sentence generation which is intuitive to humans. Here we present a novel
method to summarize text documents taking the process to semantic levels with
the use of WordNet and other resources, and using a technique for sentence
generation. We involve semantic role labeling to get the semantic
representation of text and use of segmentation to form clusters of the related
pieces of text. Picking out the centroids and sentence generation completes the
task. We evaluate our system against human composed summaries and also present
an evaluation done by humans to measure the quality attributes of our
summaries.
Identifying Duplicate and Contradictory Information in Wikipedia
Authors: Sarah Weissman, Samet Ayhan, Joshua Bradley, Jimmy Lin
Date: 2014-06-04 00:00:00
Tasks: Articles

Our study identifies sentences in Wikipedia articles that are either
identical or highly similar by applying techniques for near-duplicate detection
of web pages. This is accomplished with a MapReduce implementation of minhash
to identify clusters of sentences with high Jaccard similarity. We show that
these clusters can be categorized into six different types, two of which are
particularly interesting: identical sentences quantify the extent to which
content in Wikipedia is copied and pasted, and near-duplicate sentences that
state contradictory facts point to quality issues in Wikipedia.
A self-organizing system for urban traffic control based on predictive interval microscopic model
Authors: Bartlomiej Placzek
Date: 2014-06-04 00:00:00

This paper introduces a self-organizing traffic signal system for an urban
road network. The key elements of this system are agents that control traffic
signals at intersections. Each agent uses an interval microscopic traffic model
to predict effects of its possible control actions in a short time horizon. The
executed control action is selected on the basis of predicted delay intervals.
Since the prediction results are represented by intervals, the agents can
recognize and suspend those control actions, whose positive effect on the
performance of traffic control is uncertain. Evaluation of the proposed traffic
control system was performed in a simulation environment. The simulation
experiments have shown that the proposed approach results in an improved
performance, particularly for non-uniform traffic streams.
Neural Variational Inference and Learning in Belief Networks
Authors: Andriy Mnih, Karol Gregor
Date: 2014-01-31 00:00:00
Tasks: Variational Inference

Highly expressive directed latent variable models, such as sigmoid belief
networks, are difficult to train on large datasets because exact inference in
them is intractable and none of the approximate inference methods that have
been applied to them scale well. We propose a fast non-iterative approximate
inference method that uses a feedforward network to implement efficient exact
sampling from the variational posterior. The model and this inference network
are trained jointly by maximizing a variational lower bound on the
log-likelihood. Although the naive estimator of the inference model gradient is
too high-variance to be useful, we make it practical by applying several
straightforward model-independent variance reduction techniques. Applying our
approach to training sigmoid belief networks and deep autoregressive networks,
we show that it outperforms the wake-sleep algorithm on MNIST and achieves
state-of-the-art results on the Reuters RCV1 document dataset.
PAC Learning, VC Dimension, and the Arithmetic Hierarchy
Authors: Wesley Calvert
Date: 2014-06-04 00:00:00
Tasks: All, PAC learning

We compute that the index set of PAC-learnable concept classes is
$m$-complete $\Sigma^0_3$ within the set of indices for all concept classes of
a reasonable form. All concept classes considered are computable enumerations
of computable $\Pi^0_1$ classes, in a sense made precise here. This family of
concept classes is sufficient to cover all standard examples, and also has the
property that PAC learnability is equivalent to finite VC dimension.
A variational approach to stable principal component pursuit
Authors: Aleksandr Aravkin, Stephen Becker, Volkan Cevher, Peder Olsen
Date: 2014-06-04 00:00:00

We introduce a new convex formulation for stable principal component pursuit
(SPCP) to decompose noisy signals into low-rank and sparse representations. For
numerical solutions of our SPCP formulation, we first develop a convex
variational framework and then accelerate it with quasi-Newton methods. We
show, via synthetic and real data experiments, that our approach offers
advantages over the classical SPCP formulations in scalability and practical
parameter selection.
A Methodology for Empirical Analysis of LOD Datasets
Authors: Vit Novacek
Date: 2014-06-04 00:00:00

CoCoE stands for Complexity, Coherence and Entropy, and presents an
extensible methodology for empirical analysis of Linked Open Data (i.e., RDF
graphs). CoCoE can offer answers to questions like: Is dataset A better than B
for knowledge discovery since it is more complex and informative?, Is dataset X
better than Y for simple value lookups due its flatter structure?, etc. In
order to address such questions, we introduce a set of well-founded measures
based on complementary notions from distributional semantics, network analysis
and information theory. These measures are part of a specific implementation of
the CoCoE methodology that is available for download. Last but not least, we
illustrate CoCoE by its application to selected biomedical RDF datasets.
Learning to Diversify via Weighted Kernels for Classifier Ensemble
Authors: Xu-Cheng Yin, Chun Yang, Hong-Wei Hao
Date: 2014-06-04 00:00:00
Tasks: Diversity, Ensemble Learning, Ensemble Pruning

Classifier ensemble generally should combine diverse component classifiers.
However, it is difficult to give a definitive connection between diversity
measure and ensemble accuracy. Given a list of available component classifiers,
how to adaptively and diversely ensemble classifiers becomes a big challenge in
the literature. In this paper, we argue that diversity, not direct diversity on
samples but adaptive diversity with data, is highly correlated to ensemble
accuracy, and we propose a novel technology for classifier ensemble, learning
to diversify, which learns to adaptively combine classifiers by considering
both accuracy and diversity. Specifically, our approach, Learning TO Diversify
via Weighted Kernels (L2DWK), performs classifier combination by optimizing a
direct but simple criterion: maximizing ensemble accuracy and adaptive
diversity simultaneously by minimizing a convex loss function. Given a measure
formulation, the diversity is calculated with weighted kernels (i.e., the
diversity is measured on the component classifiers' outputs which are kernelled
and weighted), and the kernel weights are automatically learned. We minimize
this loss function by estimating the kernel weights in conjunction with the
classifier weights, and propose a self-training algorithm for conducting this
convex optimization procedure iteratively. Extensive experiments on a variety
of 32 UCI classification benchmark datasets show that the proposed approach
consistently outperforms state-of-the-art ensembles such as Bagging, AdaBoost,
Random Forests, Gasen, Regularized Selective Ensemble, and Ensemble Pruning via
Semi-Definite Programming.
Optimal Data Collection For Informative Rankings Expose Well-Connected Graphs
Authors: Braxton Osting, Christoph Brune, Stanley J. Osher
Date: 2012-07-26 00:00:00
Tasks: Clustering, Experimental Design, Informativeness, Scheduling

Given a graph where vertices represent alternatives and arcs represent
pairwise comparison data, the statistical ranking problem is to find a
potential function, defined on the vertices, such that the gradient of the
potential function agrees with the pairwise comparisons. Our goal in this paper
is to develop a method for collecting data for which the least squares
estimator for the ranking problem has maximal Fisher information. Our approach,
based on experimental design, is to view data collection as a bi-level
optimization problem where the inner problem is the ranking problem and the
outer problem is to identify data which maximizes the informativeness of the
ranking. Under certain assumptions, the data collection problem decouples,
reducing to a problem of finding multigraphs with large algebraic connectivity.
This reduction of the data collection problem to graph-theoretic questions is
one of the primary contributions of this work. As an application, we study the
Yahoo! Movie user rating dataset and demonstrate that the addition of a small
number of well-chosen pairwise comparisons can significantly increase the
Fisher informativeness of the ranking. As another application, we study the
2011-12 NCAA football schedule and propose schedules with the same number of
games which are significantly more informative. Using spectral clustering
methods to identify highly-connected communities within the division, we argue
that the NCAA could improve its notoriously poor rankings by simply scheduling
more out-of-conference games.
Integration of a Predictive, Continuous Time Neural Network into Securities Market Trading Operations
Authors: Christopher S Kirk
Date: 2014-06-04 00:00:00

This paper describes recent development and test implementation of a
continuous time recurrent neural network that has been configured to predict
rates of change in securities. It presents outcomes in the context of popular
technical analysis indicators and highlights the potential impact of continuous
predictive capability on securities market trading operations.
A Game-theoretic Machine Learning Approach for Revenue Maximization in Sponsored Search
Authors: Di He, Wei Chen, Li-Wei Wang, Tie-Yan Liu
Date: 2014-06-03 00:00:00
Tasks: BIG-bench Machine Learning, Bilevel Optimization

Sponsored search is an important monetization channel for search engines, in
which an auction mechanism is used to select the ads shown to users and
determine the prices charged from advertisers. There have been several pieces
of work in the literature that investigate how to design an auction mechanism
in order to optimize the revenue of the search engine. However, due to some
unrealistic assumptions used, the practical values of these studies are not
very clear. In this paper, we propose a novel \emph{game-theoretic machine
learning} approach, which naturally combines machine learning and game theory,
and learns the auction mechanism using a bilevel optimization framework. In
particular, we first learn a Markov model from historical data to describe how
advertisers change their bids in response to an auction mechanism, and then for
any given auction mechanism, we use the learnt model to predict its
corresponding future bid sequences. Next we learn the auction mechanism through
empirical revenue maximization on the predicted bid sequences. We show that the
empirical revenue will converge when the prediction period approaches infinity,
and a Genetic Programming algorithm can effectively optimize this empirical
revenue. Our experiments indicate that the proposed approach is able to produce
a much more effective auction mechanism than several baselines.
Beyond $χ^2$ Difference: Learning Optimal Metric for Boundary Detection
Authors: Fei He, Shengjin Wang
Date: 2014-06-04 00:00:00
Tasks: Boundary Detection

This letter focuses on solving the challenging problem of detecting natural
image boundaries. A boundary usually refers to the border between two regions
with different semantic meanings. Therefore, a measurement of dissimilarity
between image regions plays a pivotal role in boundary detection of natural
images. To improve the performance of boundary detection, a Learning-based
Boundary Metric (LBM) is proposed to replace $\chi^2$ difference adopted by the
classical algorithm mPb. Compared with $\chi^2$ difference, LBM is composed of
a single layer neural network and an RBF kernel, and is fine-tuned by
supervised learning rather than human-crafted. It is more effective in
describing the dissimilarity between natural image regions while tolerating
large variance of image data. After substituting $\chi^2$ difference with LBM,
the F-measure metric of mPb on the BSDS500 benchmark is increased from 0.69 to
0.71. Moreover, when image features are computed on a single scale, the
proposed LBM algorithm still achieves competitive results compared with
\emph{mPb}, which makes use of multi-scale image features.