pwc-archive/papers-with-abstracts huggingface dataset:
Visual Sentiment Prediction with Deep Convolutional Neural Networks
Authors: Can Xu, Suleyman Cetintas, Kuang-Chih Lee, Li-Jia Li
Date: 2014-11-21 00:00:00
Tasks: Object Recognition, Prediction, Sentiment Analysis, Transfer Learning, Visual Sentiment Prediction

Images have become one of the most popular types of media through which users
convey their emotions within online social networks. Although vast amount of
research is devoted to sentiment analysis of textual data, there has been very
limited work that focuses on analyzing sentiment of image data. In this work,
we propose a novel visual sentiment prediction framework that performs image
understanding with Deep Convolutional Neural Networks (CNN). Specifically, the
proposed sentiment prediction framework performs transfer learning from a CNN
with millions of parameters, which is pre-trained on large-scale data for
object recognition. Experiments conducted on two real-world datasets from
Twitter and Tumblr demonstrate the effectiveness of the proposed visual
sentiment analysis framework.
Metric recovery from directed unweighted graphs
Authors: Tatsunori B. Hashimoto, Yi Sun, Tommi S. Jaakkola
Date: 2014-11-20 00:00:00

We analyze directed, unweighted graphs obtained from $x_i\in \mathbb{R}^d$ by
connecting vertex $i$ to $j$ iff $|x_i - x_j| < \epsilon(x_i)$. Examples of
such graphs include $k$-nearest neighbor graphs, where $\epsilon(x_i)$ varies
from point to point, and, arguably, many real world graphs such as
co-purchasing graphs. We ask whether we can recover the underlying Euclidean
metric $\epsilon(x_i)$ and the associated density $p(x_i)$ given only the
directed graph and $d$.
  We show that consistent recovery is possible up to isometric scaling when the
vertex degree is at least $\omega(n^{2/(2+d)}\log(n)^{d/(d+2)})$. Our estimator
is based on a careful characterization of a random walk over the directed graph
and the associated continuum limit. As an algorithm, it resembles the PageRank
centrality metric. We demonstrate empirically that the estimator performs well
on simulated examples as well as on real-world co-purchasing graphs even with a
small number of points and degree scaling as low as $\log(n)$.
An algorithm for improving Non-Local Means operators via low-rank approximation
Authors: Victor May, Yosi Keller, Nir Sharon, Yoel Shkolnisky
Date: 2014-11-20 00:00:00
Tasks: Denoising

We present a method for improving a Non Local Means operator by computing its
low-rank approximation. The low-rank operator is constructed by applying a
filter to the spectrum of the original Non Local Means operator. This results
in an operator which is less sensitive to noise while preserving important
properties of the original operator. The method is efficiently implemented
based on Chebyshev polynomials and is demonstrated on the application of
natural images denoising. For this application, we provide a comprehensive
comparison of our method with leading denoising methods.
Clustering evolving data using kernel-based methods
Authors: Rocco Langone
Date: 2014-11-20 00:00:00
Tasks: Clustering, Community Detection, Fault Detection, Image Segmentation, Model Selection, Semantic Segmentation, Time Series Analysis, Time Series Clustering

In this thesis, we propose several modelling strategies to tackle evolving
data in different contexts. In the framework of static clustering, we start by
introducing a soft kernel spectral clustering (SKSC) algorithm, which can
better deal with overlapping clusters with respect to kernel spectral
clustering (KSC) and provides more interpretable outcomes. Afterwards, a whole
strategy based upon KSC for community detection of static networks is proposed,
where the extraction of a high quality training sub-graph, the choice of the
kernel function, the model selection and the applicability to large-scale data
are key aspects. This paves the way for the development of a novel clustering
algorithm for the analysis of evolving networks called kernel spectral
clustering with memory effect (MKSC), where the temporal smoothness between
clustering results in successive time steps is incorporated at the level of the
primal optimization problem, by properly modifying the KSC formulation. Later
on, an application of KSC to fault detection of an industrial machine is
presented. Here, a smart pre-processing of the data by means of a proper
windowing operation is necessary to catch the ongoing degradation process
affecting the machine. In this way, in a genuinely unsupervised manner, it is
possible to raise an early warning when necessary, in an online fashion.
Finally, we propose a new algorithm called incremental kernel spectral
clustering (IKSC) for online learning of non-stationary data. This ambitious
challenge is faced by taking advantage of the out-of-sample property of kernel
spectral clustering (KSC) to adapt the initial model, in order to tackle
merging, splitting or drifting of clusters across time. Real-world applications
considered in this thesis include image segmentation, time-series clustering,
community detection of static and evolving networks.
Learning a Recurrent Visual Representation for Image Caption Generation
Authors: Xinlei Chen, C. Lawrence Zitnick
Date: 2014-11-20 00:00:00
Tasks: Caption Generation, Image Retrieval, Retrieval, Sentence, Sentence Retrieval

In this paper we explore the bi-directional mapping between images and their
sentence-based descriptions. We propose learning this mapping using a recurrent
neural network. Unlike previous approaches that map both sentences and images
to a common embedding, we enable the generation of novel sentences given an
image. Using the same model, we can also reconstruct the visual features
associated with an image given its visual description. We use a novel recurrent
visual memory that automatically learns to remember long-term visual concepts
to aid in both sentence generation and visual feature reconstruction. We
evaluate our approach on several tasks. These include sentence generation,
sentence retrieval and image retrieval. State-of-the-art results are shown for
the task of generating novel image descriptions. When compared to human
generated captions, our automatically generated captions are preferred by
humans over $19.8\%$ of the time. Results are better than or comparable to
state-of-the-art results on the image and sentence retrieval tasks for methods
using similar visual features.
N-sphere chord length distribution
Authors: Panagiotis Sidiropoulos
Date: 2014-11-20 00:00:00

This work studies the chord length distribution, in the case where both ends
lie on a $N$-dimensional hypersphere ($N \geq 2$). Actually, after connecting
this distribution to the recently estimated surface of a hyperspherical cap
\cite{SLi11}, closed-form expressions of both the probability density function
and the cumulative distribution function are straightforwardly extracted, which
are followed by a discussion on its basic properties, among which its
dependence from the hypersphere dimension. Additionally, the distribution of
the dot product of unitary vectors is estimated, a problem that is related to
the chord length.
Ontology Module Extraction via Datalog Reasoning
Authors: Ana Armas Romero, Mark Kaminski, Bernardo Cuenca Grau, Ian Horrocks
Date: 2014-11-19 00:00:00

Module extraction - the task of computing a (preferably small) fragment M of
an ontology T that preserves entailments over a signature S - has found many
applications in recent years. Extracting modules of minimal size is, however,
computationally hard, and often algorithmically infeasible. Thus, practical
techniques are based on approximations, where M provably captures the relevant
entailments, but is not guaranteed to be minimal. Existing approximations,
however, ensure that M preserves all second-order entailments of T w.r.t. S,
which is stronger than is required in many applications, and may lead to large
modules in practice. In this paper we propose a novel approach in which module
extraction is reduced to a reasoning problem in datalog. Our approach not only
generalises existing approximations in an elegant way, but it can also be
tailored to preserve only specific kinds of entailments, which allows us to
extract significantly smaller modules. An evaluation on widely-used ontologies
has shown very encouraging results.
Predicting the Future Behavior of a Time-Varying Probability Distribution
Authors: Christoph H. Lampert
Date: 2014-06-20 00:00:00
Proceeding: CVPR 2015 6
Tasks: Domain Adaptation

We study the problem of predicting the future, though only in the
probabilistic sense of estimating a future state of a time-varying probability
distribution. This is not only an interesting academic problem, but solving
this extrapolation problem also has many practical application, e.g. for
training classifiers that have to operate under time-varying conditions. Our
main contribution is a method for predicting the next step of the time-varying
distribution from a given sequence of sample sets from earlier time steps. For
this we rely on two recent machine learning techniques: embedding probability
distributions into a reproducing kernel Hilbert space, and learning operators
by vector-valued regression. We illustrate the working principles and the
practical usefulness of our method by experiments on synthetic and real data.
We also highlight an exemplary application: training a classifier in a domain
adaptation setting without having access to examples from the test time
distribution at training time.
Autonomization of Monoidal Categories
Authors: Antonin Delpeuch
Date: 2014-11-14 00:00:00

We show that contrary to common belief in the DisCoCat community, a monoidal category is all that is needed to define a categorical compositional model of natural language. This relies on a construction which freely adds adjoints to a monoidal category. In the case of distributional semantics, this broadens the range of available models, to include non-linear maps and cartesian products for instance. We illustrate the applications of this principle to various distributional models of meaning.
Vector Autoregressions with Parsimoniously Time Varying Parameters and an Application to Monetary Policy
Authors: Laurent Callot, Johannes Tang Kristensen
Date: 2014-11-04 00:00:00
Tasks: valid

This paper proposes a parsimoniously time varying parameter vector
autoregressive model (with exogenous variables, VARX) and studies the
properties of the Lasso and adaptive Lasso as estimators of this model. The
parameters of the model are assumed to follow parsimonious random walks, where
parsimony stems from the assumption that increments to the parameters have a
non-zero probability of being exactly equal to zero. By varying the degree of
parsimony our model can accommodate constant parameters, an unknown number of
structural breaks, or parameters with a high degree of variation.
  We characterize the finite sample properties of the Lasso by deriving upper
bounds on the estimation and prediction errors that are valid with high
probability; and asymptotically we show that these bounds tend to zero with
probability tending to one if the number of non zero increments grows slower
than $\sqrt{T}$.
  By simulation experiments we investigate the properties of the Lasso and the
adaptive Lasso in settings where the parameters are stable, experience
structural breaks, or follow a parsimonious random walk. We use our model to
investigate the monetary policy response to inflation and business cycle
fluctuations in the US by estimating a parsimoniously time varying parameter
Taylor rule. We document substantial changes in the policy response of the Fed
in the 1980s and since 2008.
Maximum Likelihood Directed Enumeration Method in Piecewise-Regular Object Recognition
Authors: Andrey Savchenko
Date: 2014-11-20 00:00:00
Tasks: Face Recognition, Object Recognition

We explore the problems of classification of composite object (images, speech
signals) with low number of models per class. We study the question of
improving recognition performance for medium-sized database (thousands of
classes). The key issue of fast approximate nearest-neighbor methods widely
applied in this task is their heuristic nature. It is possible to strongly
prove their efficiency by using the theory of algorithms only for simple
similarity measures and artificially generated tasks. On the contrary, in this
paper we propose an alternative, statistically optimal greedy algorithm. At
each step of this algorithm joint density (likelihood) of distances to
previously checked models is estimated for each class. The next model to check
is selected from the class with the maximal likelihood. The latter is estimated
based on the asymptotic properties of the Kullback-Leibler information
discrimination and mathematical model of piecewise-regular object with
distribution of each regular segment of exponential type. Experimental results
in face recognition for FERET dataset prove that the proposed method is much
more effective than not only brute force and the baseline (directed enumeration
method) but also approximate nearest neighbor methods from FLANN and
NonMetricSpaceLib libraries (randomized kd-tree, composite index, perm-sort).
Fast gradient descent for drifting least squares regression, with application to bandits
Authors: Nathaniel Korda, Prashanth L. A., Rémi Munos
Date: 2013-07-11 00:00:00
Tasks: News Recommendation, regression

Online learning algorithms require to often recompute least squares
regression estimates of parameters. We study improving the computational
complexity of such algorithms by using stochastic gradient descent (SGD) type
schemes in place of classic regression solvers. We show that SGD schemes
efficiently track the true solutions of the regression problems, even in the
presence of a drift. This finding coupled with an $O(d)$ improvement in
complexity, where $d$ is the dimension of the data, make them attractive for
implementation in the big data settings. In the case when strong convexity in
the regression problem is guaranteed, we provide bounds on the error both in
expectation and high probability (the latter is often needed to provide
theoretical guarantees for higher level algorithms), despite the drifting least
squares solution. As an example of this case we prove that the regret
performance of an SGD version of the PEGE linear bandit algorithm
[Rusmevichientong and Tsitsiklis 2010] is worse that that of PEGE itself only
by a factor of $O(\log^4 n)$. When strong convexity of the regression problem
cannot be guaranteed, we investigate using an adaptive regularisation. We make
an empirical study of an adaptively regularised, SGD version of LinUCB [Li et
al. 2010] in a news article recommendation application, which uses the large
scale news recommendation dataset from Yahoo! front page. These experiments
show a large gain in computational complexity, with a consistently low tracking
error and click-through-rate (CTR) performance that is $75\%$ close.
$l_1$-regularized Outlier Isolation and Regression
Authors: Sheng Han, Suzhen Wang, Xinyu Wu
Date: 2014-06-01 00:00:00
Tasks: regression

This paper proposed a new regression model called $l_1$-regularized outlier
isolation and regression (LOIRE) and a fast algorithm based on block coordinate
descent to solve this model. Besides, assuming outliers are gross errors
following a Bernoulli process, this paper also presented a Bernoulli estimate
model which, in theory, should be very accurate and robust due to its complete
elimination of affections caused by outliers. Though this Bernoulli estimate is
hard to solve, it could be approximately achieved through a process which takes
LOIRE as an important intermediate step. As a result, the approximate Bernoulli
estimate is a good combination of Bernoulli estimate's accuracy and LOIRE
regression's efficiency with several simulations conducted to strongly verify
this point. Moreover, LOIRE can be further extended to realize robust rank
factorization which is powerful in recovering low-rank component from massive
corruptions. Extensive experimental results showed that the proposed method
outperforms state-of-the-art methods like RPCA and GoDec in the aspect of
computation speed with a competitive performance.
Confidence sets for persistence diagrams
Authors: Brittany Terese Fasy, Fabrizio Lecci, Alessandro Rinaldo, Larry Wasserman, Sivaraman Balakrishnan, Aarti Singh
Date: 2013-03-28 00:00:00

Persistent homology is a method for probing topological properties of point
clouds and functions. The method involves tracking the birth and death of
topological features (2000) as one varies a tuning parameter. Features with
short lifetimes are informally considered to be "topological noise," and those
with a long lifetime are considered to be "topological signal." In this paper,
we bring some statistical ideas to persistent homology. In particular, we
derive confidence sets that allow us to separate topological signal from
topological noise.
Liquid State Machine with Dendritically Enhanced Readout for Low-power, Neuromorphic VLSI Implementations
Authors: Subhrajit Roy, Amitava Banerjee, Arindam Basu
Date: 2014-11-20 00:00:00

In this paper, we describe a new neuro-inspired, hardware-friendly readout
stage for the liquid state machine (LSM), a popular model for reservoir
computing. Compared to the parallel perceptron architecture trained by the
p-delta algorithm, which is the state of the art in terms of performance of
readout stages, our readout architecture and learning algorithm can attain
better performance with significantly less synaptic resources making it
attractive for VLSI implementation. Inspired by the nonlinear properties of
dendrites in biological neurons, our readout stage incorporates neurons having
multiple dendrites with a lumped nonlinearity. The number of synaptic
connections on each branch is significantly lower than the total number of
connections from the liquid neurons and the learning algorithm tries to find
the best 'combination' of input connections on each branch to reduce the error.
Hence, the learning involves network rewiring (NRW) of the readout network
similar to structural plasticity observed in its biological counterparts. We
show that compared to a single perceptron using analog weights, this
architecture for the readout can attain, even by using the same number of
binary valued synapses, up to 3.3 times less error for a two-class spike train
classification problem and 2.4 times less error for an input rate approximation
task. Even with 60 times larger synapses, a group of 60 parallel perceptrons
cannot attain the performance of the proposed dendritically enhanced readout.
An additional advantage of this method for hardware implementations is that the
'choice' of connectivity can be easily implemented exploiting address event
representation (AER) protocols commonly used in current neuromorphic systems
where the connection matrix is stored in memory. Also, due to the use of binary
synapses, our proposed method is more robust against statistical variations.
Deep Unfolding: Model-Based Inspiration of Novel Deep Architectures
Authors: John R. Hershey, Jonathan Le Roux, Felix Weninger
Date: 2014-09-09 00:00:00
Tasks: Speech Enhancement

Model-based methods and deep neural networks have both been tremendously
successful paradigms in machine learning. In model-based methods, problem
domain knowledge can be built into the constraints of the model, typically at
the expense of difficulties during inference. In contrast, deterministic deep
neural networks are constructed in such a way that inference is
straightforward, but their architectures are generic and it is unclear how to
incorporate knowledge. This work aims to obtain the advantages of both
approaches. To do so, we start with a model-based approach and an associated
inference algorithm, and \emph{unfold} the inference iterations as layers in a
deep network. Rather than optimizing the original model, we \emph{untie} the
model parameters across layers, in order to create a more powerful network. The
resulting architecture can be trained discriminatively to perform accurate
inference within a fixed network size. We show how this framework allows us to
interpret conventional networks as mean-field inference in Markov random
fields, and to obtain new architectures by instead using belief propagation as
the inference algorithm. We then show its application to a non-negative matrix
factorization model that incorporates the problem-domain knowledge that sound
sources are additive. Deep unfolding of this model yields a new kind of
non-negative deep neural network, that can be trained using a multiplicative
backpropagation-style update algorithm. We present speech enhancement
experiments showing that our approach is competitive with conventional neural
networks despite using far fewer parameters.
Recommending the Most Encompassing Opposing and Endorsing Arguments in Debates
Authors: Marius C. Silaghi, Roussi Roussev
Date: 2014-11-20 00:00:00

Arguments are essential objects in DirectDemocracyP2P, where they can occur
both in association with signatures for petitions, or in association with other
debated decisions, such as bug sorting by importance. The arguments of a signer
on a given issue are grouped into one single justification, are classified by
the type of signature (e.g., supporting or opposing), and can be subject to
various types of threading.
  Given the available inputs, the two addressed problems are: (i) how to
recommend the best justification, of a given type, to a new voter, (ii) how to
recommend a compact list of justifications subsuming the majority of known
arguments for (or against) an issue.
  We investigate solutions based on weighted bipartite graphs.
Stable Model Counting and Its Application in Probabilistic Logic Programming
Authors: Rehan Abdul Aziz, Geoffrey Chu, Christian Muise, Peter Stuckey
Date: 2014-11-20 00:00:00

Model counting is the problem of computing the number of models that satisfy
a given propositional theory. It has recently been applied to solving inference
tasks in probabilistic logic programming, where the goal is to compute the
probability of given queries being true provided a set of mutually independent
random variables, a model (a logic program) and some evidence. The core of
solving this inference task involves translating the logic program to a
propositional theory and using a model counter. In this paper, we show that for
some problems that involve inductive definitions like reachability in a graph,
the translation of logic programs to SAT can be expensive for the purpose of
solving inference tasks. For such problems, direct implementation of stable
model semantics allows for more efficient solving. We present two
implementation techniques, based on unfounded set detection, that extend a
propositional model counter to a stable model counter. Our experiments show
that for particular problems, our approach can outperform a state-of-the-art
probabilistic logic programming solver by several orders of magnitude in terms
of running time and space requirements, and can solve instances of
significantly larger sizes on which the current solver runs out of time or
memory.
Approximate evaluation of marginal association probabilities with belief propagation
Authors: Jason L. Williams, Roslyn A. Lau
Date: 2012-09-12 00:00:00

Data association, the problem of reasoning over correspondence between
targets and measurements, is a fundamental problem in tracking. This paper
presents a graphical model formulation of data association and applies an
approximate inference method, belief propagation (BP), to obtain estimates of
marginal association probabilities. We prove that BP is guaranteed to converge,
and bound the number of iterations necessary. Experiments reveal a favourable
comparison to prior methods in terms of accuracy and computational complexity.