pwc-archive/papers-with-abstracts huggingface dataset:
Learning Parameters for Weighted Matrix Completion via Empirical Estimation
Authors: Jason Jo
Date: 2014-12-31 00:00:00
Tasks: Matrix Completion

Recently theoretical guarantees have been obtained for matrix completion in
the non-uniform sampling regime. In particular, if the sampling distribution
aligns with the underlying matrix's leverage scores, then with high probability
nuclear norm minimization will exactly recover the low rank matrix. In this
article, we analyze the scenario in which the non-uniform sampling distribution
may or may not not align with the underlying matrix's leverage scores. Here we
explore learning the parameters for weighted nuclear norm minimization in terms
of the empirical sampling distribution. We provide a sufficiency condition for
these learned weights which provide an exact recovery guarantee for weighted
nuclear norm minimization. It has been established that a specific choice of
weights in terms of the true sampling distribution not only allows for weighted
nuclear norm minimization to exactly recover the low rank matrix, but also
allows for a quantifiable relaxation in the exact recovery conditions. In this
article we extend this quantifiable relaxation in exact recovery conditions for
a specific choice of weights defined analogously in terms of the empirical
distribution as opposed to the true sampling distribution. To accomplish this
we employ a concentration of measure bound and a large deviation bound. We also
present numerical evidence for the healthy robustness of the weighted nuclear
norm minimization algorithm to the choice of empirically learned weights. These
numerical experiments show that for a variety of easily computable empirical
weights, weighted nuclear norm minimization outperforms unweighted nuclear norm
minimization in the non-uniform sampling regime.
Direct l_(2,p)-Norm Learning for Feature Selection
Authors: Hanyang Peng, Yong Fan
Date: 2015-04-02 00:00:00
Tasks: feature selection, General Classification, Sparse Learning

In this paper, we propose a novel sparse learning based feature selection
method that directly optimizes a large margin linear classification model
sparsity with l_(2,p)-norm (0 < p < 1)subject to data-fitting constraints,
rather than using the sparsity as a regularization term. To solve the direct
sparsity optimization problem that is non-smooth and non-convex when 0<p<1, we
provide an efficient iterative algorithm with proved convergence by converting
it to a convex and smooth optimization problem at every iteration step. The
proposed algorithm has been evaluated based on publicly available datasets, and
extensive comparison experiments have demonstrated that our algorithm could
achieve feature selection performance competitive to state-of-the-art
algorithms.
Decomposition-Based Domain Adaptation for Real-World Font Recognition
Authors: Zhangyang Wang, Jianchao Yang, Hailin Jin, Eli Shechtman, Aseem Agarwala, Jonathan Brandt, Thomas S. Huang
Date: 2014-12-18 00:00:00
Tasks: Domain Adaptation, Font Recognition, General Classification

We present a domain adaption framework to address a domain mismatch between
synthetic training and real-world testing data. We demonstrate our method on a
challenging fine-grain classification problem: recognizing a font style from an
image of text. In this task, it is very easy to generate lots of rendered font
examples but very hard to obtain real-world labeled images. This
real-to-synthetic domain gap caused poor generalization to new real data in
previous font recognition methods (Chen et al. (2014)). In this paper, we
introduce a Convolutional Neural Network decomposition approach, leveraging a
large training corpus of synthetic data to obtain effective features for
classification. This is done using an adaptation technique based on a Stacked
Convolutional Auto-Encoder that exploits a large collection of unlabeled
real-world text images combined with synthetic data preprocessed in a specific
way. The proposed DeepFont method achieves an accuracy of higher than 80%
(top-5) on a new large labeled real-world dataset we collected.
Signatures of Infinity: Nonergodicity and Resource Scaling in Prediction, Complexity, and Learning
Authors: James P. Crutchfield, Sarah Marzen
Date: 2015-04-01 00:00:00

We introduce a simple analysis of the structural complexity of
infinite-memory processes built from random samples of stationary, ergodic
finite-memory component processes. Such processes are familiar from the well
known multi-arm Bandit problem. We contrast our analysis with
computation-theoretic and statistical inference approaches to understanding
their complexity. The result is an alternative view of the relationship between
predictability, complexity, and learning that highlights the distinct ways in
which informational and correlational divergences arise in complex ergodic and
nonergodic processes. We draw out consequences for the resource divergences
that delineate the structural hierarchy of ergodic processes and for processes
that are themselves hierarchical.
Bayesian Clustering of Shapes of Curves
Authors: Zhengwu Zhang, Debdeep Pati, Anuj Srivastava
Date: 2015-04-01 00:00:00
Tasks: Clustering, valid

Unsupervised clustering of curves according to their shapes is an important
problem with broad scientific applications. The existing model-based clustering
techniques either rely on simple probability models (e.g., Gaussian) that are
not generally valid for shape analysis or assume the number of clusters. We
develop an efficient Bayesian method to cluster curve data using an elastic
shape metric that is based on joint registration and comparison of shapes of
curves. The elastic-inner product matrix obtained from the data is modeled
using a Wishart distribution whose parameters are assigned carefully chosen
prior distributions to allow for automatic inference on the number of clusters.
Posterior is sampled through an efficient Markov chain Monte Carlo procedure
based on the Chinese restaurant process to infer (1) the posterior distribution
on the number of clusters, and (2) clustering configuration of shapes. This
method is demonstrated on a variety of synthetic data and real data examples on
protein structure analysis, cell shape analysis in microscopy images, and
clustering of shaped from MPEG7 database.
Hierarchical Sparse and Collaborative Low-Rank Representation for Emotion Recognition
Authors: Xiang Xiang, Minh Dao, Gregory D. Hager, Trac. D. Tran
Date: 2014-10-07 00:00:00
Tasks: Emotion Recognition, General Classification, Sparse Representation-based Classification

In this paper, we design a Collaborative-Hierarchical Sparse and Low-Rank
(C-HiSLR) model that is natural for recognizing human emotion in visual data.
Previous attempts require explicit expression components, which are often
unavailable and difficult to recover. Instead, our model exploits the lowrank
property over expressive facial frames and rescue inexact sparse
representations by incorporating group sparsity. For the CK+ dataset, C-HiSLR
on raw expressive faces performs as competitive as the Sparse Representation
based Classification (SRC) applied on manually prepared emotions. C-HiSLR
performs even better than SRC in terms of true positive rate.
Representation Learning with Deep Extreme Learning Machines for Efficient Image Set Classification
Authors: Muhammad Uzair, Faisal Shafait, Bernard Ghanem, Ajmal Mian
Date: 2015-03-09 00:00:00
Tasks: Classification, General Classification, Representation Learning

Efficient and accurate joint representation of a collection of images, that
belong to the same class, is a major research challenge for practical image set
classification. Existing methods either make prior assumptions about the data
structure, or perform heavy computations to learn structure from the data
itself. In this paper, we propose an efficient image set representation that
does not make any prior assumptions about the structure of the underlying data.
We learn the non-linear structure of image sets with Deep Extreme Learning
Machines (DELM) that are very efficient and generalize well even on a limited
number of training samples. Extensive experiments on a broad range of public
datasets for image set classification (Honda/UCSD, CMU Mobo, YouTube
Celebrities, Celebrity-1000, ETH-80) show that the proposed algorithm
consistently outperforms state-of-the-art image set classification methods both
in terms of speed and accuracy.
A New Repair Operator for Multi-objective Evolutionary Algorithm in Constrained Optimization Problems
Authors: Zhun Fan, Wenji Li, Xinye Cai, Huibiao Lin, Shuxiang Xie, Erik Goodman
Date: 2015-04-01 00:00:00
Tasks: Diversity, Evolutionary Algorithms

In this paper, we design a set of multi-objective constrained optimization
problems (MCOPs) and propose a new repair operator to address them. The
proposed repair operator is used to fix the solutions that violate the box
constraints. More specifically, it employs a reversed correction strategy that
can effectively avoid the population falling into local optimum. In addition,
we integrate the proposed repair operator into two classical multi-objective
evolutionary algorithms MOEA/D and NSGA-II. The proposed repair operator is
compared with other two kinds of commonly used repair operators on benchmark
problems CTPs and MCOPs. The experiment results demonstrate that our proposed
approach is very effective in terms of convergence and diversity.
Iterative Regularization for Learning with Convex Loss Functions
Authors: Junhong Lin, Lorenzo Rosasco, Ding-Xuan Zhou
Date: 2015-03-31 00:00:00
Tasks: BIG-bench Machine Learning

We consider the problem of supervised learning with convex loss functions and
propose a new form of iterative regularization based on the subgradient method.
Unlike other regularization approaches, in iterative regularization no
constraint or penalization is considered, and generalization is achieved by
(early) stopping an empirical iteration. We consider a nonparametric setting,
in the framework of reproducing kernel Hilbert spaces, and prove finite sample
bounds on the excess risk under general regularity conditions. Our study
provides a new class of efficient regularized learning algorithms and gives
insights on the interplay between statistics and optimization in machine
learning.
Knowledge reduction of dynamic covering decision information systems with immigration of more objects
Authors: Guangming Lang
Date: 2015-04-01 00:00:00

In practical situations, it is of interest to investigate computing
approximations of sets as an important step of knowledge reduction of dynamic
covering decision information systems. In this paper, we present incremental
approaches to computing the type-1 and type-2 characteristic matrices of
dynamic coverings whose cardinalities increase with immigration of more
objects. We also present the incremental algorithms of computing the second and
sixth lower and upper approximations of sets in dynamic covering approximation
spaces.
Multidimensional Digital Smoothing Filters for Target Detection
Authors: Hugh L. Kennedy
Date: 2014-10-02 00:00:00
Tasks: regression

Recursive, causal and non-causal, multidimensional digital filters, with
infinite impulse responses and maximally flat magnitude and delay responses in
the low-frequency region, are designed to negate correlated clutter and
interference in the background and to accumulate power due to dim targets in
the foreground of a surveillance sensor. Expressions relating mean
impulse-response duration, frequency selectivity and group delay, to low-order
linear-difference-equation coefficients are derived using discrete Laguerre
polynomials and discounted least-squares regression, then verified through
simulation.
The Libra Toolkit for Probabilistic Models
Authors: Daniel Lowd, Amirmohammad Rooshenas
Date: 2015-04-01 00:00:00

The Libra Toolkit is a collection of algorithms for learning and inference
with discrete probabilistic models, including Bayesian networks, Markov
networks, dependency networks, and sum-product networks. Compared to other
toolkits, Libra places a greater emphasis on learning the structure of
tractable models in which exact inference is efficient. It also includes a
variety of algorithms for learning graphical models in which inference is
potentially intractable, and for performing exact and approximate inference.
Libra is released under a 2-clause BSD license to encourage broad use in
academia and industry.
A Weight-coded Evolutionary Algorithm for the Multidimensional Knapsack Problem
Authors: Quan Yuan, Zhi-Xin Yang
Date: 2013-02-21 00:00:00

A revised weight-coded evolutionary algorithm (RWCEA) is proposed for solving
multidimensional knapsack problems. This RWCEA uses a new decoding method and
incorporates a heuristic method in initialization. Computational results show
that the RWCEA performs better than a weight-coded evolutionary algorithm
proposed by Raidl (1999) and to some existing benchmarks, it can yield better
results than the ones reported in the OR-library.
A Theory of Feature Learning
Authors: Brendan van Rooyen, Robert C. Williamson
Date: 2015-04-01 00:00:00

Feature Learning aims to extract relevant information contained in data sets
in an automated fashion. It is driving force behind the current deep learning
trend, a set of methods that have had widespread empirical success. What is
lacking is a theoretical understanding of different feature learning schemes.
This work provides a theoretical framework for feature learning and then
characterizes when features can be learnt in an unsupervised fashion. We also
provide means to judge the quality of features via rate-distortion theory and
its generalizations.
Crowdsourcing Feature Discovery via Adaptively Chosen Comparisons
Authors: James Y. Zou, Kamalika Chaudhuri, Adam Tauman Kalai
Date: 2015-03-31 00:00:00

We introduce an unsupervised approach to efficiently discover the underlying
features in a data set via crowdsourcing. Our queries ask crowd members to
articulate a feature common to two out of three displayed examples. In addition
we also ask the crowd to provide binary labels to the remaining examples based
on the discovered features. The triples are chosen adaptively based on the
labels of the previously discovered features on the data set. In two natural
models of features, hierarchical and independent, we show that a simple
adaptive algorithm, using "two-out-of-three" similarity queries, recovers all
features with less labor than any nonadaptive algorithm. Experimental results
validate the theoretical findings.
Improved Error Bounds Based on Worst Likely Assignments
Authors: Eric Bax
Date: 2015-03-31 00:00:00

Error bounds based on worst likely assignments use permutation tests to
validate classifiers. Worst likely assignments can produce effective bounds
even for data sets with 100 or fewer training examples. This paper introduces a
statistic for use in the permutation tests of worst likely assignments that
improves error bounds, especially for accurate classifiers, which are typically
the classifiers of interest.
Weakly Supervised Learning of Objects, Attributes and their Associations
Authors: Zhiyuan Shi, Yongxin Yang, Timothy M. Hospedales, Tao Xiang
Date: 2015-03-31 00:00:00
Tasks: Attribute, Image Description, Object, Retrieval, Weakly-supervised Learning

When humans describe images they tend to use combinations of nouns and
adjectives, corresponding to objects and their associated attributes
respectively. To generate such a description automatically, one needs to model
objects, attributes and their associations. Conventional methods require strong
annotation of object and attribute locations, making them less scalable. In
this paper, we model object-attribute associations from weakly labelled images,
such as those widely available on media sharing sites (e.g. Flickr), where only
image-level labels (either object or attributes) are given, without their
locations and associations. This is achieved by introducing a novel weakly
supervised non-parametric Bayesian model. Once learned, given a new image, our
model can describe the image, including objects, attributes and their
associations, as well as their locations and segmentation. Extensive
experiments on benchmark datasets demonstrate that our weakly supervised model
performs at par with strongly supervised models on tasks such as image
description and retrieval based on object-attribute associations.
Real-World Font Recognition Using Deep Network and Domain Adaptation
Authors: Zhangyang Wang, Jianchao Yang, Hailin Jin, Eli Shechtman, Aseem Agarwala, Jonathan Brandt, Thomas S. Huang
Date: 2015-03-31 00:00:00
Tasks: Domain Adaptation, Font Recognition, General Classification

We address a challenging fine-grain classification problem: recognizing a
font style from an image of text. In this task, it is very easy to generate
lots of rendered font examples but very hard to obtain real-world labeled
images. This real-to-synthetic domain gap caused poor generalization to new
real data in previous methods (Chen et al. (2014)). In this paper, we refer to
Convolutional Neural Networks, and use an adaptation technique based on a
Stacked Convolutional Auto-Encoder that exploits unlabeled real-world images
combined with synthetic data. The proposed method achieves an accuracy of
higher than 80% (top-5) on a real-world dataset.
Example Selection For Dictionary Learning
Authors: Tomoki Tsuchida, Garrison W. Cottrell
Date: 2014-12-18 00:00:00
Tasks: Dictionary Learning

In unsupervised learning, an unbiased uniform sampling strategy is typically
used, in order that the learned features faithfully encode the statistical
structure of the training data. In this work, we explore whether active example
selection strategies - algorithms that select which examples to use, based on
the current estimate of the features - can accelerate learning. Specifically,
we investigate effects of heuristic and saliency-inspired selection algorithms
on the dictionary learning task with sparse activations. We show that some
selection algorithms do improve the speed of learning, and we speculate on why
they might work.
Towards Using Machine Translation Techniques to Induce Multilingual Lexica of Discourse Markers
Authors: António Lopes, David Martins de Matos, Vera Cabarrão, Ricardo Ribeiro, Helena Moniz, Isabel Trancoso, Ana Isabel Mata
Date: 2015-03-31 00:00:00
Tasks: Machine Translation, Sentence, Translation

Discourse markers are universal linguistic events subject to language
variation. Although an extensive literature has already reported language
specific traits of these events, little has been said on their cross-language
behavior and on building an inventory of multilingual lexica of discourse
markers. This work describes new methods and approaches for the description,
classification, and annotation of discourse markers in the specific domain of
the Europarl corpus. The study of discourse markers in the context of
translation is crucial due to the idiomatic nature of these structures.
Multilingual lexica together with the functional analysis of such structures
are useful tools for the hard task of translating discourse markers into
possible equivalents from one language to another. Using Daniel Marcu's
validated discourse markers for English, extracted from the Brown Corpus, our
purpose is to build multilingual lexica of discourse markers for other
languages, based on machine translation techniques. The major assumption in
this study is that the usage of a discourse marker is independent of the
language, i.e., the rhetorical function of a discourse marker in a sentence in
one language is equivalent to the rhetorical function of the same discourse
marker in another language.
Encoding Spike Patterns in Multilayer Spiking Neural Networks
Authors: Brian Gardner, Ioana Sporea, André Grüning
Date: 2015-03-31 00:00:00

Information encoding in the nervous system is supported through the precise
spike-timings of neurons; however, an understanding of the underlying processes
by which such representations are formed in the first place remains unclear.
Here we examine how networks of spiking neurons can learn to encode for input
patterns using a fully temporal coding scheme. To this end, we introduce a
learning rule for spiking networks containing hidden neurons which optimizes
the likelihood of generating desired output spiking patterns. We show the
proposed learning rule allows for a large number of accurate input-output spike
pattern mappings to be learnt, which outperforms other existing learning rules
for spiking neural networks: both in the number of mappings that can be learnt
as well as the complexity of spike train encodings that can be utilised. The
learning rule is successful even in the presence of input noise, is
demonstrated to solve the linearly non-separable XOR computation and
generalizes well on an example dataset. We further present a biologically
plausible implementation of backpropagated learning in multilayer spiking
networks, and discuss the neural mechanisms that might underlie its function.
Our approach contributes both to a systematic understanding of how pattern
encodings might take place in the nervous system, and a learning rule that
displays strong technical capability.
Denoising autoencoder with modulated lateral connections learns invariant representations of natural images
Authors: Antti Rasmus, Tapani Raiko, Harri Valpola
Date: 2014-12-22 00:00:00
Tasks: Decoder, Denoising

Suitable lateral connections between encoder and decoder are shown to allow
higher layers of a denoising autoencoder (dAE) to focus on invariant
representations. In regular autoencoders, detailed information needs to be
carried through the highest layers but lateral connections from encoder to
decoder relieve this pressure. It is shown that abstract invariant features can
be translated to detailed reconstructions when invariant features are allowed
to modulate the strength of the lateral connection. Three dAE structures with
modulated and additive lateral connections, and without lateral connections
were compared in experiments using real-world images. The experiments verify
that adding modulated lateral connections to the model 1) improves the accuracy
of the probability model for inputs, as measured by denoising performance; 2)
results in representations whose degree of invariance grows faster towards the
higher layers; and 3) supports the formation of diverse invariant poolings.