pwc-archive/papers-with-abstracts huggingface dataset:
On the properties of $α$-unchaining single linkage hierarchical clustering
Authors: A. Martínez-Pérez
Date: 2014-02-18 00:00:00
Tasks: Clustering

In the election of a hierarchical clustering method, theoretic properties may
give some insight to determine which method is the most suitable to treat a
clustering problem. Herein, we study some basic properties of two hierarchical
clustering methods: $\alpha$-unchaining single linkage or $SL(\alpha)$ and a
modified version of this one, $SL^*(\alpha)$. We compare the results with the
properties satisfied by the classical linkage-based hierarchical clustering
methods.
Factorial Hidden Markov Models for Learning Representations of Natural Language
Authors: Anjan Nepal, Alexander Yates
Date: 2013-12-20 00:00:00
Tasks: Chunking, Part-Of-Speech Tagging, Representation Learning

Most representation learning algorithms for language and image processing are
local, in that they identify features for a data point based on surrounding
points. Yet in language processing, the correct meaning of a word often depends
on its global context. As a step toward incorporating global context into
representation learning, we develop a representation learning algorithm that
incorporates joint prediction into its technique for producing features for a
word. We develop efficient variational methods for learning Factorial Hidden
Markov Models from large texts, and use variational distributions to produce
features for each word that are sensitive to the entire input sequence, not
just to a local context window. Experiments on part-of-speech tagging and
chunking indicate that the features are competitive with or better than
existing state-of-the-art representation learning methods.
The Random Forest Kernel and other kernels for big data from random partitions
Authors: Alex Davies, Zoubin Ghahramani
Date: 2014-02-18 00:00:00
Tasks: Gaussian Processes

We present Random Partition Kernels, a new class of kernels derived by
demonstrating a natural connection between random partitions of objects and
kernels between those objects. We show how the construction can be used to
create kernels from methods that would not normally be viewed as random
partitions, such as Random Forest. To demonstrate the potential of this method,
we propose two new kernels, the Random Forest Kernel and the Fast Cluster
Kernel, and show that these kernels consistently outperform standard kernels on
problems involving real-world datasets. Finally, we show how the form of these
kernels lend themselves to a natural approximation that is appropriate for
certain big data problems, allowing $O(N)$ inference in methods such as
Gaussian Processes, Support Vector Machines and Kernel PCA.
Discretization of Temporal Data: A Survey
Authors: P. Chaudhari, D. P. Rana, R. G. Mehta, N. J. Mistry, M. M. Raghuwanshi
Date: 2014-02-18 00:00:00
Tasks: Survey

In real world, the huge amount of temporal data is to be processed in many
application areas such as scientific, financial, network monitoring, sensor
data analysis. Data mining techniques are primarily oriented to handle discrete
features. In the case of temporal data the time plays an important role on the
characteristics of data. To consider this effect, the data discretization
techniques have to consider the time while processing to resolve the issue by
finding the intervals of data which are more concise and precise with respect
to time. Here, this research is reviewing different data discretization
techniques used in temporal data applications according to the inclusion or
exclusion of: class label, temporal order of the data and handling of stream
data to open the research direction for temporal data discretization to improve
the performance of data mining technique.
Sparse, complex-valued representations of natural sounds learned with phase and amplitude continuity priors
Authors: Wiktor Mlynarski
Date: 2013-12-17 00:00:00

Complex-valued sparse coding is a data representation which employs a
dictionary of two-dimensional subspaces, while imposing a sparse, factorial
prior on complex amplitudes. When trained on a dataset of natural image
patches, it learns phase invariant features which closely resemble receptive
fields of complex cells in the visual cortex. Features trained on natural
sounds however, rarely reveal phase invariance and capture other aspects of the
data. This observation is a starting point of the present work. As its first
contribution, it provides an analysis of natural sound statistics by means of
learning sparse, complex representations of short speech intervals. Secondly,
it proposes priors over the basis function set, which bias them towards
phase-invariant solutions. In this way, a dictionary of complex basis functions
can be learned from the data statistics, while preserving the phase invariance
property. Finally, representations trained on speech sounds with and without
priors are compared. Prior-based basis functions reveal performance comparable
to unconstrained sparse coding, while explicitely representing phase as a
temporal shift. Such representations can find applications in many perceptual
and machine learning tasks.
Group-sparse Embeddings in Collective Matrix Factorization
Authors: Arto Klami, Guillaume Bouchard, Abhishek Tripathi
Date: 2013-12-20 00:00:00
Tasks: MULTI-VIEW LEARNING, Recommendation Systems

CMF is a technique for simultaneously learning low-rank representations based
on a collection of matrices with shared entities. A typical example is the
joint modeling of user-item, item-property, and user-feature matrices in a
recommender system. The key idea in CMF is that the embeddings are shared
across the matrices, which enables transferring information between them. The
existing solutions, however, break down when the individual matrices have
low-rank structure not shared with others. In this work we present a novel CMF
solution that allows each of the matrices to have a separate low-rank structure
that is independent of the other matrices, as well as structures that are
shared only by a subset of them. We compare MAP and variational Bayesian
solutions based on alternating optimization algorithms and show that the model
automatically infers the nature of each factor using group-wise sparsity. Our
approach supports in a principled way continuous, binary and count observations
and is efficient for sparse matrices involving missing data. We illustrate the
solution on a number of examples, focusing in particular on an interesting
use-case of augmented multi-view learning.
Extracting Networks of Characters and Places from Written Works with CHAPLIN
Authors: Roberto Marazzato, Amelia Carolina Sparavigna
Date: 2014-02-18 00:00:00

We are proposing a tool able to gather information on social networks from
narrative texts. Its name is CHAPLIN, CHAracters and PLaces Interaction
Network, implemented in VB.NET. Characters and places of the narrative works
are extracted in a list of raw words. Aided by the interface, the user selects
names out of them. After this choice, the tool allows the user to enter some
parameters, and, according to them, creates a network where the nodes are the
characters and places, and the edges their interactions. Edges are labelled by
performances. The output is a GV file, written in the DOT graph scripting
language, which is rendered by means of the free open source software Graphviz.
The More, the Merrier: the Blessing of Dimensionality for Learning Large Gaussian Mixtures
Authors: Joseph Anderson, Mikhail Belkin, Navin Goyal, Luis Rademacher, James Voss
Date: 2013-11-12 00:00:00

In this paper we show that very large mixtures of Gaussians are efficiently
learnable in high dimension. More precisely, we prove that a mixture with known
identical covariance matrices whose number of components is a polynomial of any
fixed degree in the dimension n is polynomially learnable as long as a certain
non-degeneracy condition on the means is satisfied. It turns out that this
condition is generic in the sense of smoothed complexity, as soon as the
dimensionality of the space is high enough. Moreover, we prove that no such
condition can possibly exist in low dimension and the problem of learning the
parameters is generically hard. In contrast, much of the existing work on
Gaussian Mixtures relies on low-dimensional projections and thus hits an
artificial barrier. Our main result on mixture recovery relies on a new
"Poissonization"-based technique, which transforms a mixture of Gaussians to a
linear map of a product distribution. The problem of learning this map can be
efficiently solved using some recent results on tensor decompositions and
Independent Component Analysis (ICA), thus giving an algorithm for recovering
the mixture. In addition, we combine our low-dimensional hardness results for
Gaussian mixtures with Poissonization to show how to embed difficult instances
of low-dimensional Gaussian mixtures into the ICA setting, thus establishing
exponential information-theoretic lower bounds for underdetermined ICA in low
dimension. To the best of our knowledge, this is the first such result in the
literature. In addition to contributing to the problem of Gaussian mixture
learning, we believe that this work is among the first steps toward better
understanding the rare phenomenon of the "blessing of dimensionality" in the
computational aspects of statistical inference.
One-Shot Adaptation of Supervised Deep Convolutional Models
Authors: Judy Hoffman, Eric Tzeng, Jeff Donahue, Yangqing Jia, Kate Saenko, Trevor Darrell
Date: 2013-12-21 00:00:00
Tasks: Domain Adaptation, image-classification, Image Classification

Dataset bias remains a significant barrier towards solving real world
computer vision tasks. Though deep convolutional networks have proven to be a
competitive approach for image classification, a question remains: have these
models have solved the dataset bias problem? In general, training or
fine-tuning a state-of-the-art deep model on a new domain requires a
significant amount of data, which for many applications is simply not
available. Transfer of models directly to new domains without adaptation has
historically led to poor recognition performance. In this paper, we pose the
following question: is a single image dataset, much larger than previously
explored for adaptation, comprehensive enough to learn general deep models that
may be effectively applied to new image domains? In other words, are deep CNNs
trained on large amounts of labeled data as susceptible to dataset bias as
previous methods have been shown to be? We show that a generic supervised deep
CNN model trained on a large dataset reduces, but does not remove, dataset
bias. Furthermore, we propose several methods for adaptation with deep models
that are able to operate with little (one example per category) or no labeled
domain specific data. Our experiments show that adaptation of deep models on
benchmark visual domain adaptation datasets can provide a significant
performance boost.
When Learners Surpass their Sources: Mathematical Modeling of Learning from an Inconsistent Source
Authors: Yelena Mandelshtam, Natalia Komarova
Date: 2014-02-18 00:00:00
Tasks: Language Acquisition

We present a new algorithm to model and investigate the learning process of a
learner mastering a set of grammatical rules from an inconsistent source. The
compelling interest of human language acquisition is that the learning succeeds
in virtually every case, despite the fact that the input data are formally
inadequate to explain the success of learning. Our model explains how a learner
can successfully learn from or even surpass its imperfect source without
possessing any additional biases or constraints about the types of patterns
that exist in the language. We use the data collected by Singleton and Newport
(2004) on the performance of a 7-year boy Simon, who mastered the American Sign
Language (ASL) by learning it from his parents, both of whom were imperfect
speakers of ASL. We show that the algorithm possesses a frequency-boosting
property, whereby the frequency of the most common form of the source is
increased by the learner. We also explain several key features of Simon's ASL.
Semistochastic Quadratic Bound Methods
Authors: Aleksandr Y. Aravkin, Anna Choromanska, Tony Jebara, Dimitri Kanevsky
Date: 2013-09-05 00:00:00

Partition functions arise in a variety of settings, including conditional
random fields, logistic regression, and latent gaussian models. In this paper,
we consider semistochastic quadratic bound (SQB) methods for maximum likelihood
inference based on partition function optimization. Batch methods based on the
quadratic bound were recently proposed for this class of problems, and
performed favorably in comparison to state-of-the-art techniques.
Semistochastic methods fall in between batch algorithms, which use all the
data, and stochastic gradient type methods, which use small random selections
at each iteration. We build semistochastic quadratic bound-based methods, and
prove both global convergence (to a stationary point) under very weak
assumptions, and linear convergence rate under stronger assumptions on the
objective. To make the proposed methods faster and more stable, we consider
inexact subproblem minimization and batch-size selection schemes. The efficacy
of SQB methods is demonstrated via comparison with several state-of-the-art
techniques on commonly used datasets.
Continuous Learning: Engineering Super Features With Feature Algebras
Authors: Michael Tetelman
Date: 2013-12-19 00:00:00

In this paper we consider a problem of searching a space of predictive models
for a given training data set. We propose an iterative procedure for deriving a
sequence of improving models and a corresponding sequence of sets of non-linear
features on the original input space. After a finite number of iterations N,
the non-linear features become 2^N -degree polynomials on the original space.
We show that in a limit of an infinite number of iterations derived non-linear
features must form an associative algebra: a product of two features is equal
to a linear combination of features from the same feature space for any given
input point. Because each iteration consists of solving a series of convex
problems that contain all previous solutions, the likelihood of the models in
the sequence is increasing with each iteration while the dimension of the model
parameter space is set to a limited controlled value.
Selective Sampling with Drift
Authors: Edward Moroshko, Koby Crammer
Date: 2014-02-17 00:00:00
Tasks: Active Learning

Recently there has been much work on selective sampling, an online active
learning setting, in which algorithms work in rounds. On each round an
algorithm receives an input and makes a prediction. Then, it can decide whether
to query a label, and if so to update its model, otherwise the input is
discarded. Most of this work is focused on the stationary case, where it is
assumed that there is a fixed target model, and the performance of the
algorithm is compared to a fixed model. However, in many real-world
applications, such as spam prediction, the best target function may drift over
time, or have shifts from time to time. We develop a novel selective sampling
algorithm for the drifting setting, analyze it under no assumptions on the
mechanism generating the sequence of instances, and derive new mistake bounds
that depend on the amount of drift in the problem. Simulations on synthetic and
real-world datasets demonstrate the superiority of our algorithms as a
selective sampling algorithm in the drifting setting.
Statistical Noise Analysis in SENSE Parallel MRI
Authors: Santiago Aja-Fernandez, Gonzalo Vegas-Sanchez-Ferrero, Antonio Trsitan-Vega
Date: 2014-02-17 00:00:00

A complete first and second order statistical characterization of noise in
SENSE reconstructed data is proposed. SENSE acquisitions have usually been
modeled as Rician distributed, since the data reconstruction takes place into
the spatial domain, where Gaussian noise is assumed. However, this model just
holds for the first order statistics and obviates other effects induced by
coils correlations and the reconstruction interpolation. Those effects are
properly taken into account in this study, in order to fully justify a final
SENSE noise model. As a result, some interesting features of the reconstructed
image arise: (1) There is a strong correlation between adjacent lines. (2) The
resulting distribution is non-stationary and therefore the variance of noise
will vary from point to point across the image. Closed equations for the
calculation of the variance of noise and the correlation coefficient between
lines are proposed. The proposed model is totally compatible with g-factor
formulations.
The Algebraic Approach to Phase Retrieval and Explicit Inversion at the Identifiability Threshold
Authors: Franz J. Király, Martin Ehler
Date: 2014-02-17 00:00:00
Tasks: regression, Retrieval

We study phase retrieval from magnitude measurements of an unknown signal as
an algebraic estimation problem. Indeed, phase retrieval from rank-one and more
general linear measurements can be treated in an algebraic way. It is verified
that a certain number of generic rank-one or generic linear measurements are
sufficient to enable signal reconstruction for generic signals, and slightly
more generic measurements yield reconstructability for all signals. Our results
solve a few open problems stated in the recent literature. Furthermore, we show
how the algebraic estimation problem can be solved by a closed-form algebraic
estimation technique, termed ideal regression, providing non-asymptotic success
guarantees.
Modeling correlations in spontaneous activity of visual cortex with centered Gaussian-binary deep Boltzmann machines
Authors: Nan Wang, Dirk Jancke, Laurenz Wiskott
Date: 2013-12-20 00:00:00

Spontaneous cortical activity -- the ongoing cortical activities in absence
of intentional sensory input -- is considered to play a vital role in many
aspects of both normal brain functions and mental dysfunctions. We present a
centered Gaussian-binary Deep Boltzmann Machine (GDBM) for modeling the
activity in early cortical visual areas and relate the random sampling in GDBMs
to the spontaneous cortical activity. After training the proposed model on
natural image patches, we show that the samples collected from the model's
probability distribution encompass similar activity patterns as found in the
spontaneous activity. Specifically, filters having the same orientation
preference tend to be active together during random sampling. Our work
demonstrates the centered GDBM is a meaningful model approach for basic
receptive field properties and the emergence of spontaneous activity patterns
in early cortical visual areas. Besides, we show empirically that centered
GDBMs do not suffer from the difficulties during training as GDBMs do and can
be properly trained without the layer-wise pretraining.
Revisiting Natural Gradient for Deep Networks
Authors: Razvan Pascanu, Yoshua Bengio
Date: 2013-01-16 00:00:00

We evaluate natural gradient, an algorithm originally proposed in Amari
(1997), for learning deep models. The contributions of this paper are as
follows. We show the connection between natural gradient and three other
recently proposed methods for training deep models: Hessian-Free (Martens,
2010), Krylov Subspace Descent (Vinyals and Povey, 2012) and TONGA (Le Roux et
al., 2008). We describe how one can use unlabeled data to improve the
generalization error obtained by natural gradient and empirically evaluate the
robustness of the algorithm to the ordering of the training set compared to
stochastic gradient descent. Finally we extend natural gradient to incorporate
second order information alongside the manifold information and provide a
benchmark of the new algorithm using a truncated Newton approach for inverting
the metric matrix instead of using a diagonal approximation of it.
Is Spiking Logic the Route to Memristor-Based Computers?
Authors: Ella Gale, Ben De Lacy Costello, Andrew Adamatzky
Date: 2014-02-17 00:00:00

Memristors have been suggested as a novel route to neuromorphic computing
based on the similarity between neurons (synapses and ion pumps) and
memristors. The D.C. action of the memristor is a current spike, which we think
will be fruitful for building memristor computers. In this paper, we introduce
4 different logical assignations to implement sequential logic in the memristor
and introduce the physical rules, summation, `bounce-back', directionality and
`diminishing returns', elucidated from our investigations. We then demonstrate
how memristor sequential logic works by instantiating a NOT gate, an AND gate
and a Full Adder with a single memristor. The Full Adder makes use of the
memristor's memory to add three binary values together and outputs the value,
the carry digit and even the order they were input in.
Connecting Spiking Neurons to a Spiking Memristor Network Changes the Memristor Dynamics
Authors: Deborah Gater, Attya Iqbal, Jeffrey Davey, Ella Gale
Date: 2014-02-17 00:00:00
Tasks: Translation

Memristors have been suggested as neuromorphic computing elements. Spike-time
dependent plasticity and the Hodgkin-Huxley model of the neuron have both been
modelled effectively by memristor theory. The d.c. response of the memristor is
a current spike. Based on these three facts we suggest that memristors are
well-placed to interface directly with neurons. In this paper we show that
connecting a spiking memristor network to spiking neuronal cells causes a
change in the memristor network dynamics by: removing the memristor spikes,
which we show is due to the effects of connection to aqueous medium; causing a
change in current decay rate consistent with a change in memristor state;
presenting more-linear $I-t$ dynamics; and increasing the memristor spiking
rate, as a consequence of interaction with the spiking neurons. This
demonstrates that neurons are capable of communicating directly with
memristors, without the need for computer translation.
Does the D.C. Response of Memristors Allow Robotic Short-Term Memory and a Possible Route to Artificial Time Perception?
Authors: Ella Gale, Ben de Lacy Costello, Andrew Adamatzky
Date: 2014-02-17 00:00:00

Time perception is essential for task switching, and in the mammalian brain
appears alongside other processes. Memristors are electronic components used as
synapses and as models for neurons. The d.c. response of memristors can be
considered as a type of short-term memory. Interactions of the memristor d.c.
response within networks of memristors leads to the emergence of oscillatory
dynamics and intermittent spike trains, which are similar to neural dynamics.
Based on this data, the structure of a memristor network control for a robot as
it undergoes task switching is discussed and it is suggested that these
emergent network dynamics could improve the performance of role switching and
learning in an artificial intelligence and perhaps create artificial time
perception.
Proteus: A Hierarchical Portfolio of Solvers and Transformations
Authors: Barry Hurley, Lars Kotthoff, Yuri Malitsky, Barry O'Sullivan
Date: 2013-06-24 00:00:00

In recent years, portfolio approaches to solving SAT problems and CSPs have
become increasingly common. There are also a number of different encodings for
representing CSPs as SAT instances. In this paper, we leverage advances in both
SAT and CSP solving to present a novel hierarchical portfolio-based approach to
CSP solving, which we call Proteus, that does not rely purely on CSP solvers.
Instead, it may decide that it is best to encode a CSP problem instance into
SAT, selecting an appropriate encoding and a corresponding SAT solver. Our
experimental evaluation used an instance of Proteus that involved four CSP
solvers, three SAT encodings, and six SAT solvers, evaluated on the most
challenging problem instances from the CSP solver competitions, involving
global and intensional constraints. We show that significant performance
improvements can be achieved by Proteus obtained by exploiting alternative
view-points and solvers for combinatorial problem-solving.