pwc-archive/papers-with-abstracts huggingface dataset:
Convolutional Dictionary Learning through Tensor Factorization
Authors: Furong Huang, Animashree Anandkumar
Date: 2015-06-10 00:00:00
Tasks: Dictionary Learning, parameter estimation, Tensor Decomposition, Topic Models

Tensor methods have emerged as a powerful paradigm for consistent learning of
many latent variable models such as topic models, independent component
analysis and dictionary learning. Model parameters are estimated via CP
decomposition of the observed higher order input moments. However, in many
domains, additional invariances such as shift invariances exist, enforced via
models such as convolutional dictionary learning. In this paper, we develop
novel tensor decomposition algorithms for parameter estimation of convolutional
models. Our algorithm is based on the popular alternating least squares method,
but with efficient projections onto the space of stacked circulant matrices.
Our method is embarrassingly parallel and consists of simple operations such as
fast Fourier transforms and matrix multiplications. Our algorithm converges to
the dictionary much faster and more accurately compared to the alternating
minimization over filters and activation maps.
Optimally Combining Classifiers Using Unlabeled Data
Authors: Akshay Balsubramani, Yoav Freund
Date: 2015-03-05 00:00:00
Tasks: Binary Classification, General Classification

We develop a worst-case analysis of aggregation of classifier ensembles for
binary classification. The task of predicting to minimize error is formulated
as a game played over a given set of unlabeled data (a transductive setting),
where prior label information is encoded as constraints on the game. The
minimax solution of this game identifies cases where a weighted combination of
the classifiers can perform significantly better than any single classifier.
A tree augmented naive Bayesian network experiment for breast cancer prediction
Authors: Ping Ren
Date: 2015-06-18 00:00:00
Tasks: Diagnostic

In order to investigate the breast cancer prediction problem on the aging
population with the grades of DCIS, we conduct a tree augmented naive Bayesian
network experiment trained and tested on a large clinical dataset including
consecutive diagnostic mammography examinations, consequent biopsy outcomes and
related cancer registry records in the population of women across all ages. The
aggregated results of our ten-fold cross validation method recommend a biopsy
threshold higher than 2% for the aging population.
Multi-Context Models for Reasoning under Partial Knowledge: Generative Process and Inference Grammar
Authors: Ardavan Salehi Nobandegani, Ioannis N. Psaromiligkos
Date: 2014-12-13 00:00:00

Arriving at the complete probabilistic knowledge of a domain, i.e., learning
how all variables interact, is indeed a demanding task. In reality, settings
often arise for which an individual merely possesses partial knowledge of the
domain, and yet, is expected to give adequate answers to a variety of posed
queries. That is, although precise answers to some queries, in principle,
cannot be achieved, a range of plausible answers is attainable for each query
given the available partial knowledge. In this paper, we propose the
Multi-Context Model (MCM), a new graphical model to represent the state of
partial knowledge as to a domain. MCM is a middle ground between Probabilistic
Logic, Bayesian Logic, and Probabilistic Graphical Models. For this model we
discuss: (i) the dynamics of constructing a contradiction-free MCM, i.e., to
form partial beliefs regarding a domain in a gradual and probabilistically
consistent way, and (ii) how to perform inference, i.e., to evaluate a
probability of interest involving some variables of the domain.
A Fast Incremental Gaussian Mixture Model
Authors: Rafael Pinto, Paulo Engel
Date: 2015-06-14 00:00:00
Tasks: Incremental Learning, model

This work builds upon previous efforts in online incremental learning, namely
the Incremental Gaussian Mixture Network (IGMN). The IGMN is capable of
learning from data streams in a single-pass by improving its model after
analyzing each data point and discarding it thereafter. Nevertheless, it
suffers from the scalability point-of-view, due to its asymptotic time
complexity of $\operatorname{O}\bigl(NKD^3\bigr)$ for $N$ data points, $K$
Gaussian components and $D$ dimensions, rendering it inadequate for
high-dimensional data. In this paper, we manage to reduce this complexity to
$\operatorname{O}\bigl(NKD^2\bigr)$ by deriving formulas for working directly
with precision matrices instead of covariance matrices. The final result is a
much faster and scalable algorithm which can be applied to high dimensional
tasks. This is confirmed by applying the modified algorithm to high-dimensional
classification datasets.
Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
Authors: Emily Denton, Soumith Chintala, Arthur Szlam, Rob Fergus
Date: 2015-06-18 00:00:00

In this paper we introduce a generative parametric model capable of producing
high quality samples of natural images. Our approach uses a cascade of
convolutional networks within a Laplacian pyramid framework to generate images
in a coarse-to-fine fashion. At each level of the pyramid, a separate
generative convnet model is trained using the Generative Adversarial Nets (GAN)
approach (Goodfellow et al.). Samples drawn from our model are of significantly
higher quality than alternate approaches. In a quantitative assessment by human
evaluators, our CIFAR10 samples were mistaken for real images around 40% of the
time, compared to 10% for samples drawn from a GAN baseline model. We also show
samples from models trained on the higher resolution images of the LSUN scene
dataset.
From Pixels to Torques: Policy Learning with Deep Dynamical Models
Authors: Niklas Wahlström, Thomas B. Schön, Marc Peter Deisenroth
Date: 2015-02-08 00:00:00
Tasks: Model-based Reinforcement Learning, Model Predictive Control, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Data-efficient learning in continuous state-action spaces using very
high-dimensional observations remains a key challenge in developing fully
autonomous systems. In this paper, we consider one instance of this challenge,
the pixels to torques problem, where an agent must learn a closed-loop control
policy from pixel information only. We introduce a data-efficient, model-based
reinforcement learning algorithm that learns such a closed-loop policy directly
from pixel information. The key ingredient is a deep dynamical model that uses
deep auto-encoders to learn a low-dimensional embedding of images jointly with
a predictive model in this low-dimensional feature space. Joint learning
ensures that not only static but also dynamic properties of the data are
accounted for. This is crucial for long-term predictions, which lie at the core
of the adaptive model predictive control strategy that we use for closed-loop
control. Compared to state-of-the-art reinforcement learning methods for
continuous states and actions, our approach learns quickly, scales to
high-dimensional state spaces and is an important step toward fully autonomous
learning from pixels to torques.
Minimizing Regret in Dynamic Decision Problems
Authors: Joseph Y. Halpern, Samantha Leung
Date: 2015-01-31 00:00:00

The menu-dependent nature of regret-minimization creates subtleties when it
is applied to dynamic decision problems. Firstly, it is not clear whether
\emph{forgone opportunities} should be included in the \emph{menu}, with
respect to which regrets are computed, at different points of the decision
problem. If forgone opportunities are included, however, we can characterize
when a form of dynamic consistency is guaranteed. Secondly, more subtleties
arise when sophistication is used to deal with dynamic inconsistency. In the
full version of this paper, we examine, axiomatically and by common examples,
the implications of different menu definitions for sophisticated,
regret-minimizing agents.
On the accuracy of self-normalized log-linear models
Authors: Jacob Andreas, Maxim Rabinovich, Dan Klein, Michael. I. Jordan
Date: 2015-06-12 00:00:00
Proceeding: NeurIPS 2015 12
Tasks: Generalization Bounds

Calculation of the log-normalizer is a major computational obstacle in
applications of log-linear models with large output spaces. The problem of fast
normalizer computation has therefore attracted significant attention in the
theoretical and applied machine learning literature. In this paper, we analyze
a recently proposed technique known as "self-normalization", which introduces a
regularization term in training to penalize log normalizers for deviating from
zero. This makes it possible to use unnormalized model scores as approximate
probabilities. Empirical evidence suggests that self-normalization is extremely
effective, but a theoretical understanding of why it should work, and how
generally it can be applied, is largely lacking. We prove generalization bounds
on the estimated variance of normalizers and upper bounds on the loss in
accuracy due to self-normalization, describe classes of input distributions
that self-normalize easily, and construct explicit examples of high-variance
input distributions. Our theoretical results make predictions about the
difficulty of fitting self-normalized models to several classes of
distributions, and we conclude with empirical validation of these predictions.
"The Sum of Its Parts": Joint Learning of Word and Phrase Representations with Autoencoders
Authors: Rémi Lebret, Ronan Collobert
Date: 2015-06-18 00:00:00

Recently, there has been a lot of effort to represent words in continuous
vector spaces. Those representations have been shown to capture both semantic
and syntactic information about words. However, distributed representations of
phrases remain a challenge. We introduce a novel model that jointly learns word
vector representations and their summation. Word representations are learnt
using the word co-occurrence statistical information. To embed sequences of
words (i.e. phrases) with different sizes into a common semantic space, we
propose to average word vector representations. In contrast with previous
methods which reported a posteriori some compositionality aspects by simple
summation, we simultaneously train words to sum, while keeping the maximum
information from the original vectors. We evaluate the quality of the word
representations on several classical word evaluation tasks, and we introduce a
novel task to evaluate the quality of the phrase representations. While our
distributed representations compete with other methods of learning word
representations on word evaluations, we show that they give better performance
on the phrase evaluation. Such representations of phrases could be interesting
for many tasks in natural language processing.
A hybrid algorithm for Bayesian network structure learning with application to multi-label learning
Authors: Maxime Gasse, Alex Aussem, Haytham Elghazel
Date: 2015-06-18 00:00:00
Tasks: General Classification, Multi-class Classification, Multi-Label Learning

We present a novel hybrid algorithm for Bayesian network structure learning,
called H2PC. It first reconstructs the skeleton of a Bayesian network and then
performs a Bayesian-scoring greedy hill-climbing search to orient the edges.
The algorithm is based on divide-and-conquer constraint-based subroutines to
learn the local structure around a target variable. We conduct two series of
experimental comparisons of H2PC against Max-Min Hill-Climbing (MMHC), which is
currently the most powerful state-of-the-art algorithm for Bayesian network
structure learning. First, we use eight well-known Bayesian network benchmarks
with various data sizes to assess the quality of the learned structure returned
by the algorithms. Our extensive experiments show that H2PC outperforms MMHC in
terms of goodness of fit to new data and quality of the network structure with
respect to the true dependence structure of the data. Second, we investigate
H2PC's ability to solve the multi-label learning problem. We provide
theoretical results to characterize and identify graphically the so-called
minimal label powersets that appear as irreducible factors in the joint
distribution under the faithfulness condition. The multi-label learning problem
is then decomposed into a series of multi-class classification problems, where
each multi-class variable encodes a label powerset. H2PC is shown to compare
favorably to MMHC in terms of global classification accuracy over ten
multi-label data sets covering different application domains. Overall, our
experiments support the conclusions that local structural learning with H2PC in
the form of local neighborhood induction is a theoretically well-motivated and
empirically effective learning framework that is well suited to multi-label
learning. The source code (in R) of H2PC as well as all data sets used for the
empirical tests are publicly available.
Concentric network symmetry grasps authors' styles in word adjacency networks
Authors: Diego R. Amancio, Filipi N. Silva, Luciano da F. Costa
Date: 2015-04-09 00:00:00
Tasks: Authorship Attribution, Information Retrieval, Retrieval

Several characteristics of written texts have been inferred from statistical
analysis derived from networked models. Even though many network measurements
have been adapted to study textual properties at several levels of complexity,
some textual aspects have been disregarded. In this paper, we study the
symmetry of word adjacency networks, a well-known representation of text as a
graph. A statistical analysis of the symmetry distribution performed in several
novels showed that most of the words do not display symmetric patterns of
connectivity. More specifically, the merged symmetry displayed a distribution
similar to the ubiquitous power-law distribution. Our experiments also revealed
that the studied metrics do not correlate with other traditional network
measurements, such as the degree or betweenness centrality. The effectiveness
of the symmetry measurements was verified in the authorship attribution task.
Interestingly, we found that specific authors prefer particular types of
symmetric motifs. As a consequence, the authorship of books could be accurately
identified in 82.5% of the cases, in a dataset comprising books written by 8
authors. Because the proposed measurements for text analysis are complementary
to the traditional approach, they can be used to improve the characterization
of text networks, which might be useful for related applications, such as those
relying on the identification of topical words and information retrieval.
FastMMD: Ensemble of Circular Discrepancy for Efficient Two-Sample Test
Authors: Ji Zhao, Deyu Meng
Date: 2014-05-12 00:00:00
Tasks: Vocal Bursts Valence Prediction

The maximum mean discrepancy (MMD) is a recently proposed test statistic for
two-sample test. Its quadratic time complexity, however, greatly hampers its
availability to large-scale applications. To accelerate the MMD calculation, in
this study we propose an efficient method called FastMMD. The core idea of
FastMMD is to equivalently transform the MMD with shift-invariant kernels into
the amplitude expectation of a linear combination of sinusoid components based
on Bochner's theorem and Fourier transform (Rahimi & Recht, 2007). Taking
advantage of sampling of Fourier transform, FastMMD decreases the time
complexity for MMD calculation from $O(N^2 d)$ to $O(L N d)$, where $N$ and $d$
are the size and dimension of the sample set, respectively. Here $L$ is the
number of basis functions for approximating kernels which determines the
approximation accuracy. For kernels that are spherically invariant, the
computation can be further accelerated to $O(L N \log d)$ by using the Fastfood
technique (Le et al., 2013). The uniform convergence of our method has also
been theoretically proved in both unbiased and biased estimates. We have
further provided a geometric explanation for our method, namely ensemble of
circular discrepancy, which facilitates us to understand the insight of MMD,
and is hopeful to help arouse more extensive metrics for assessing two-sample
test. Experimental results substantiate that FastMMD is with similar accuracy
as exact MMD, while with faster computation speed and lower variance than the
existing MMD approximation methods.
Point-wise Map Recovery and Refinement from Functional Correspondence
Authors: Emanuele Rodolà, Michael Moeller, Daniel Cremers
Date: 2015-06-18 00:00:00
Tasks: Image Segmentation, Semantic Segmentation

Since their introduction in the shape analysis community, functional maps
have met with considerable success due to their ability to compactly represent
dense correspondences between deformable shapes, with applications ranging from
shape matching and image segmentation, to exploration of large shape
collections. Despite the numerous advantages of such representation, however,
the problem of converting a given functional map back to a point-to-point map
has received a surprisingly limited interest. In this paper we analyze the
general problem of point-wise map recovery from arbitrary functional maps. In
doing so, we rule out many of the assumptions required by the currently
established approach -- most notably, the limiting requirement of the input
shapes being nearly-isometric. We devise an efficient recovery process based on
a simple probabilistic model. Experiments confirm that this approach achieves
remarkable accuracy improvements in very challenging cases.
Collaborative Deep Learning for Recommender Systems
Authors: Hao Wang, Naiyan Wang, Dit-yan Yeung
Date: 2014-09-10 00:00:00
Tasks: Collaborative Filtering, Deep Learning, Recommendation Systems, Representation Learning

Collaborative filtering (CF) is a successful approach commonly used by many
recommender systems. Conventional CF-based methods use the ratings given to
items by users as the sole source of information for learning to make
recommendation. However, the ratings are often very sparse in many
applications, causing CF-based methods to degrade significantly in their
recommendation performance. To address this sparsity problem, auxiliary
information such as item content information may be utilized. Collaborative
topic regression (CTR) is an appealing recent method taking this approach which
tightly couples the two components that learn from two different sources of
information. Nevertheless, the latent representation learned by CTR may not be
very effective when the auxiliary information is very sparse. To address this
problem, we generalize recent advances in deep learning from i.i.d. input to
non-i.i.d. (CF-based) input and propose in this paper a hierarchical Bayesian
model called collaborative deep learning (CDL), which jointly performs deep
representation learning for the content information and collaborative filtering
for the ratings (feedback) matrix. Extensive experiments on three real-world
datasets from different domains show that CDL can significantly advance the
state of the art.
Emergence of synchrony in an Adaptive Interaction Model
Authors: Kevin Sanlaville, Gérard Assayag, Frédéric Bevilacqua, Catherine Pelachaud
Date: 2015-06-18 00:00:00
Tasks: model

In a Human-Computer Interaction context, we aim to elaborate an adaptive and
generic interaction model in two different use cases: Embodied Conversational
Agents and Creative Musical Agents for musical improvisation. To reach this
goal, we'll try to use the concepts of adaptation and synchronization to
enhance the interactive abilities of our agents and guide the development of
our interaction model, and will try to make synchrony emerge from non-verbal
dimensions of interaction.
Comparing and evaluating extended Lambek calculi
Authors: Richard Moot
Date: 2015-06-18 00:00:00

Lambeks Syntactic Calculus, commonly referred to as the Lambek calculus, was
innovative in many ways, notably as a precursor of linear logic. But it also
showed that we could treat our grammatical framework as a logic (as opposed to
a logical theory). However, though it was successful in giving at least a basic
treatment of many linguistic phenomena, it was also clear that a slightly more
expressive logical calculus was needed for many other cases. Therefore, many
extensions and variants of the Lambek calculus have been proposed, since the
eighties and up until the present day. As a result, there is now a large class
of calculi, each with its own empirical successes and theoretical results, but
also each with its own logical primitives. This raises the question: how do we
compare and evaluate these different logical formalisms? To answer this
question, I present two unifying frameworks for these extended Lambek calculi.
Both are proof net calculi with graph contraction criteria. The first calculus
is a very general system: you specify the structure of your sequents and it
gives you the connectives and contractions which correspond to it. The calculus
can be extended with structural rules, which translate directly into graph
rewrite rules. The second calculus is first-order (multiplicative
intuitionistic) linear logic, which turns out to have several other,
independently proposed extensions of the Lambek calculus as fragments. I will
illustrate the use of each calculus in building bridges between analyses
proposed in different frameworks, in highlighting differences and in helping to
identify problems.
Learning Scale-Free Networks by Dynamic Node-Specific Degree Prior
Authors: Qingming Tang, Siqi Sun, Jinbo Xu
Date: 2015-03-07 00:00:00

Learning the network structure underlying data is an important problem in
machine learning. This paper introduces a novel prior to study the inference of
scale-free networks, which are widely used to model social and biological
networks. The prior not only favors a desirable global node degree
distribution, but also takes into consideration the relative strength of all
the possible edges adjacent to the same node and the estimated degree of each
individual node.
  To fulfill this, ranking is incorporated into the prior, which makes the
problem challenging to solve. We employ an ADMM (alternating direction method
of multipliers) framework to solve the Gaussian Graphical model regularized by
this prior. Our experiments on both synthetic and real data show that our prior
not only yields a scale-free network, but also produces many more correctly
predicted edges than the others such as the scale-free inducing prior, the
hub-inducing prior and the $l_1$ norm.
Exact Hybrid Covariance Thresholding for Joint Graphical Lasso
Authors: Qingming Tang, Chao Yang, Jian Peng, Jinbo Xu
Date: 2015-03-07 00:00:00

This paper considers the problem of estimating multiple related Gaussian
graphical models from a $p$-dimensional dataset consisting of different
classes. Our work is based upon the formulation of this problem as group
graphical lasso. This paper proposes a novel hybrid covariance thresholding
algorithm that can effectively identify zero entries in the precision matrices
and split a large joint graphical lasso problem into small subproblems. Our
hybrid covariance thresholding method is superior to existing uniform
thresholding methods in that our method can split the precision matrix of each
individual class using different partition schemes and thus split group
graphical lasso into much smaller subproblems, each of which can be solved very
fast. In addition, this paper establishes necessary and sufficient conditions
for our hybrid covariance thresholding algorithm. The superior performance of
our thresholding method is thoroughly analyzed and illustrated by a few
experiments on simulated data and real gene expression data.