pwc-archive/papers-with-abstracts huggingface dataset:
Tensor Decompositions: A New Concept in Brain Data Analysis?
Authors: Andrzej Cichocki
Date: 2013-05-02 00:00:00
Tasks: blind source separation, Classification, Clustering, Dimensionality Reduction, General Classification, regression

Matrix factorizations and their extensions to tensor factorizations and
decompositions have become prominent techniques for linear and multilinear
blind source separation (BSS), especially multiway Independent Component
Analysis (ICA), NonnegativeMatrix and Tensor Factorization (NMF/NTF), Smooth
Component Analysis (SmoCA) and Sparse Component Analysis (SCA). Moreover,
tensor decompositions have many other potential applications beyond multilinear
BSS, especially feature extraction, classification, dimensionality reduction
and multiway clustering. In this paper, we briefly overview new and emerging
models and approaches for tensor decompositions in applications to group and
linked multiway BSS/ICA, feature extraction, classification andMultiway Partial
Least Squares (MPLS) regression problems. Keywords: Multilinear BSS, linked
multiway BSS/ICA, tensor factorizations and decompositions, constrained Tucker
and CP models, Penalized Tensor Decompositions (PTD), feature extraction,
classification, multiway PLS and CCA.
Sparse/Robust Estimation and Kalman Smoothing with Nonsmooth Log-Concave Densities: Modeling, Computation, and Theory
Authors: Aleksandr Y. Aravkin, James V. Burke, Gianluigi Pillonetto
Date: 2013-01-19 00:00:00
Tasks: Computational Efficiency, Time Series Analysis

We introduce a class of quadratic support (QS) functions, many of which play
a crucial role in a variety of applications, including machine learning, robust
statistical inference, sparsity promotion, and Kalman smoothing. Well known
examples include the l2, Huber, l1 and Vapnik losses. We build on a dual
representation for QS functions using convex analysis, revealing the structure
necessary for a QS function to be interpreted as the negative log of a
probability density, and providing the foundation for statistical
interpretation and analysis of QS loss functions. For a subclass of QS
functions called piecewise linear quadratic (PLQ) penalties, we also develop
efficient numerical estimation schemes. These components form a flexible
statistical modeling framework for a variety of learning applications, together
with a toolbox of efficient numerical methods for inference. In particular, for
PLQ densities, interior point (IP) methods can be used. IP methods solve
nonsmooth optimization problems by working directly with smooth systems of
equations characterizing their optimality. The efficiency of the IP approach
depends on the structure of particular applications. We consider the class of
dynamic inverse problems using Kalman smoothing, where the aim is to
reconstruct the state of a dynamical system with known process and measurement
models starting from noisy output samples. In the classical case, Gaussian
errors are assumed in the process and measurement models. The extended
framework allows arbitrary PLQ densities to be used, and the proposed IP
approach solves the generalized Kalman smoothing problem while maintaining the
linear complexity in the size of the time series, just as in the Gaussian case.
This extends the computational efficiency of classic algorithms to a much
broader nonsmooth setting, and includes many recently proposed robust and
sparse smoothers as special cases.
Model Selection for High-Dimensional Regression under the Generalized Irrepresentability Condition
Authors: Adel Javanmard, Andrea Montanari
Date: 2013-05-02 00:00:00
Proceeding: NeurIPS 2013 12
Tasks: Model Selection, regression, Vocal Bursts Intensity Prediction

In the high-dimensional regression model a response variable is linearly
related to $p$ covariates, but the sample size $n$ is smaller than $p$. We
assume that only a small subset of covariates is `active' (i.e., the
corresponding coefficients are non-zero), and consider the model-selection
problem of identifying the active covariates. A popular approach is to estimate
the regression coefficients through the Lasso ($\ell_1$-regularized least
squares). This is known to correctly identify the active set only if the
irrelevant covariates are roughly orthogonal to the relevant ones, as
quantified through the so called `irrepresentability' condition. In this paper
we study the `Gauss-Lasso' selector, a simple two-stage method that first
solves the Lasso, and then performs ordinary least squares restricted to the
Lasso active set. We formulate `generalized irrepresentability condition'
(GIC), an assumption that is substantially weaker than irrepresentability. We
prove that, under GIC, the Gauss-Lasso correctly recovers the active set.
Modeling Stable Matching Problems with Answer Set Programming
Authors: Sofie De Clercq, Steven Schockaert, Martine De Cock, Ann Nowé
Date: 2013-02-28 00:00:00

The Stable Marriage Problem (SMP) is a well-known matching problem first
introduced and solved by Gale and Shapley (1962). Several variants and
extensions to this problem have since been investigated to cover a wider set of
applications. Each time a new variant is considered, however, a new algorithm
needs to be developed and implemented. As an alternative, in this paper we
propose an encoding of the SMP using Answer Set Programming (ASP). Our encoding
can easily be extended and adapted to the needs of specific applications. As an
illustration we show how stable matchings can be found when individuals may
designate unacceptable partners and ties between preferences are allowed.
Subsequently, we show how our ASP based encoding naturally allows us to select
specific stable matchings which are optimal according to a given criterion.
Each time, we can rely on generic and efficient off-the-shelf answer set
solvers to find (optimal) stable matchings.
An Adaptive Descriptor Design for Object Recognition in the Wild
Authors: Zhenyu Guo, Z. Jane Wang
Date: 2013-05-01 00:00:00
Tasks: Domain Adaptation, Object, Object Recognition

Digital images nowadays have various styles of appearance, in the aspects of
color tones, contrast, vignetting, and etc. These 'picture styles' are directly
related to the scene radiance, image pipeline of the camera, and post
processing functions. Due to the complexity and nonlinearity of these causes,
popular gradient-based image descriptors won't be invariant to different
picture styles, which will decline the performance of object recognition. Given
that images shared online or created by individual users are taken with a wide
range of devices and may be processed by various post processing functions, to
find a robust object recognition system is useful and challenging. In this
paper, we present the first study on the influence of picture styles for object
recognition, and propose an adaptive approach based on the kernel view of
gradient descriptors and multiple kernel learning, without estimating or
specifying the styles of images used in training and testing. We conduct
experiments on Domain Adaptation data set and Oxford Flower data set. The
experiments also include several variants of the flower data set by processing
the images with popular photo effects. The results demonstrate that our
proposed method improve from standard descriptors in all cases.
The Sample Complexity of Search over Multiple Populations
Authors: Matthew L. Malloy, Gongguo Tang, Robert D. Nowak
Date: 2012-09-06 00:00:00

This paper studies the sample complexity of searching over multiple
populations. We consider a large number of populations, each corresponding to
either distribution P0 or P1. The goal of the search problem studied here is to
find one population corresponding to distribution P1 with as few samples as
possible. The main contribution is to quantify the number of samples needed to
correctly find one such population. We consider two general approaches:
non-adaptive sampling methods, which sample each population a predetermined
number of times until a population following P1 is found, and adaptive sampling
methods, which employ sequential sampling schemes for each population. We first
derive a lower bound on the number of samples required by any sampling scheme.
We then consider an adaptive procedure consisting of a series of sequential
probability ratio tests, and show it comes within a constant factor of the
lower bound. We give explicit expressions for this constant when samples of the
populations follow Gaussian and Bernoulli distributions. An alternative
adaptive scheme is discussed which does not require full knowledge of P1, and
comes within a constant factor of the optimal scheme. For comparison, a lower
bound on the sampling requirements of any non-adaptive scheme is presented.
Video Segmentation via Diffusion Bases
Authors: Dina Dushnik, Alon Schclar, Amir Averbuch
Date: 2013-05-01 00:00:00
Tasks: Segmentation, Video Segmentation, Video Semantic Segmentation

Identifying moving objects in a video sequence, which is produced by a static
camera, is a fundamental and critical task in many computer-vision
applications. A common approach performs background subtraction, which
identifies moving objects as the portion of a video frame that differs
significantly from a background model. A good background subtraction algorithm
has to be robust to changes in the illumination and it should avoid detecting
non-stationary background objects such as moving leaves, rain, snow, and
shadows. In addition, the internal background model should quickly respond to
changes in background such as objects that start to move or stop. We present a
new algorithm for video segmentation that processes the input video sequence as
a 3D matrix where the third axis is the time domain. Our approach identifies
the background by reducing the input dimension using the \emph{diffusion bases}
methodology. Furthermore, we describe an iterative method for extracting and
deleting the background. The algorithm has two versions and thus covers the
complete range of backgrounds: one for scenes with static backgrounds and the
other for scenes with dynamic (moving) backgrounds.
MATAWS: A Multimodal Approach for Automatic WS Semantic Annotation
Authors: Cihan Aksoy, Vincent Labatut, Chantal Cherifi, Jean-François Santucci
Date: 2013-05-01 00:00:00

Many recent works aim at developing methods and tools for the processing of
semantic Web services. In order to be properly tested, these tools must be
applied to an appropriate benchmark, taking the form of a collection of
semantic WS descriptions. However, all of the existing publicly available
collections are limited by their size or their realism (use of randomly
generated or resampled descriptions). Larger and realistic syntactic (WSDL)
collections exist, but their semantic annotation requires a certain level of
automation, due to the number of operations to be processed. In this article,
we propose a fully automatic method to semantically annotate such large WS
collections. Our approach is multimodal, in the sense it takes advantage of the
latent semantics present not only in the parameter names, but also in the type
names and structures. Concept-to-word association is performed by using Sigma,
a mapping of WordNet to the SUMO ontology. After having described in details
our annotation method, we apply it to the larger collection of real-world
syntactic WS descriptions we could find, and assess its efficiency.
Benefits of Semantics on Web Service Composition from a Complex Network Perspective
Authors: Chantal Cherifi, Vincent Labatut, Jean-François Santucci
Date: 2013-05-01 00:00:00
Tasks: Service Composition

The number of publicly available Web services (WS) is continuously growing,
and in parallel, we are witnessing a rapid development in semantic-related web
technologies. The intersection of the semantic web and WS allows the
development of semantic WS. In this work, we adopt a complex network
perspective to perform a comparative analysis of the syntactic and semantic
approaches used to describe WS. From a collection of publicly available WS
descriptions, we extract syntactic and semantic WS interaction networks. We
take advantage of tools from the complex network field to analyze them and
determine their properties. We show that WS interaction networks exhibit some
of the typical characteristics observed in real-world networks, such as short
average distance between nodes and community structure. By comparing syntactic
and semantic networks through their properties, we show the introduction of
semantics in WS descriptions should improve the composition process.
A Community Based Algorithm for Large Scale Web Service Composition
Authors: Chantal Cherifi, Yvan Rivierre, Jean-Francois Santucci
Date: 2013-05-01 00:00:00
Tasks: Efficient Exploration, Service Composition

Web service composition is the process of synthesizing a new composite
service using a set of available Web services in order to satisfy a client
request that cannot be treated by any available Web services. The Web services
space is a dynamic environment characterized by a huge number of elements.
Furthermore, many Web services are offering similar functionalities. In this
paper we propose a model for Web service composition designed to address the
scale effect and the redundancy issue. The Web services space is represented by
a two-layered network architecture. A concrete similarity network layer
organizes the Web services operations into communities of functionally similar
operations. An abstract interaction network layer represents the composition
relationships between the sets of communities. Composition synthesis is
performed by a two-phased graph search algorithm. First, the interaction
network is mined in order to discover abstract solutions to the request goal.
Then, the abstract compositions are instantiated with concrete operations
selected from the similarity network. This strategy allows an efficient
exploration of the Web services space. Furthermore, operations grouped in a
community can be easily substituted if necessary during the composition's
synthesis's process.
Semi-Supervised Information-Maximization Clustering
Authors: Daniele Calandriello, Gang Niu, Masashi Sugiyama
Date: 2013-04-30 00:00:00
Tasks: Clustering

Semi-supervised clustering aims to introduce prior knowledge in the decision
process of a clustering algorithm. In this paper, we propose a novel
semi-supervised clustering algorithm based on the information-maximization
principle. The proposed method is an extension of a previous unsupervised
information-maximization clustering algorithm based on squared-loss mutual
information to effectively incorporate must-links and cannot-links. The
proposed method is computationally efficient because the clustering solution
can be obtained analytically via eigendecomposition. Furthermore, the proposed
method allows systematic optimization of tuning parameters such as the kernel
width, given the degree of belief in the must-links and cannot-links. The
usefulness of the proposed method is demonstrated through experiments.
The Annealing Sparse Bayesian Learning Algorithm
Authors: Benyuan Liu, Hongqi Fan, Zaiqi Lu, Qiang Fu
Date: 2012-09-05 00:00:00

In this paper we propose a two-level hierarchical Bayesian model and an
annealing schedule to re-enable the noise variance learning capability of the
fast marginalized Sparse Bayesian Learning Algorithms. The performance such as
NMSE and F-measure can be greatly improved due to the annealing technique. This
algorithm tends to produce the most sparse solution under moderate SNR
scenarios and can outperform most concurrent SBL algorithms while pertains
small computational load.
Training Neural Networks with Stochastic Hessian-Free Optimization
Authors: Ryan Kiros
Date: 2013-01-16 00:00:00
Tasks: General Classification

Hessian-free (HF) optimization has been successfully used for training deep
autoencoders and recurrent networks. HF uses the conjugate gradient algorithm
to construct update directions through curvature-vector products that can be
computed on the same order of time as gradients. In this paper we exploit this
property and study stochastic HF with gradient and curvature mini-batches
independent of the dataset size. We modify Martens' HF for these settings and
integrate dropout, a method for preventing co-adaptation of feature detectors,
to guard against overfitting. Stochastic Hessian-free optimization gives an
intermediary between SGD and HF that achieves competitive performance on both
classification and deep autoencoder experiments.
Clustering Unclustered Data: Unsupervised Binary Labeling of Two Datasets Having Different Class Balances
Authors: Marthinus Christoffel du Plessis, Masashi Sugiyama
Date: 2013-05-01 00:00:00
Tasks: Clustering, Density Estimation

We consider the unsupervised learning problem of assigning labels to
unlabeled data. A naive approach is to use clustering methods, but this works
well only when data is properly clustered and each cluster corresponds to an
underlying class. In this paper, we first show that this unsupervised labeling
problem in balanced binary cases can be solved if two unlabeled datasets having
different class balances are available. More specifically, estimation of the
sign of the difference between probability densities of two unlabeled datasets
gives the solution. We then introduce a new method to directly estimate the
sign of the density difference without density estimation. Finally, we
demonstrate the usefulness of the proposed method against several clustering
methods on various toy problems and real-world datasets.
Joint Training Deep Boltzmann Machines for Classification
Authors: Ian J. Goodfellow, Aaron Courville, Yoshua Bengio
Date: 2013-01-16 00:00:00
Tasks: Classification, General Classification

We introduce a new method for training deep Boltzmann machines jointly. Prior
methods of training DBMs require an initial learning pass that trains the model
greedily, one layer at a time, or do not perform well on classification tasks.
In our approach, we train all layers of the DBM simultaneously, using a novel
training procedure called multi-prediction training. The resulting model can
either be interpreted as a single generative model trained to maximize a
variational approximation to the generalized pseudolikelihood, or as a family
of recurrent networks that share parameters and may be approximately averaged
together using a novel technique we call the multi-inference trick. We show
that our approach performs competitively for classification and outperforms
previous methods in terms of accuracy of approximate inference and
classification with missing inputs.
Spectral Compressed Sensing via Structured Matrix Completion
Authors: Yuxin Chen, Yuejie Chi
Date: 2013-04-16 00:00:00
Tasks: compressed sensing, Matrix Completion, Super-Resolution

The paper studies the problem of recovering a spectrally sparse object from a
small number of time domain samples. Specifically, the object of interest with
ambient dimension $n$ is assumed to be a mixture of $r$ complex
multi-dimensional sinusoids, while the underlying frequencies can assume any
value in the unit disk. Conventional compressed sensing paradigms suffer from
the {\em basis mismatch} issue when imposing a discrete dictionary on the
Fourier representation. To address this problem, we develop a novel
nonparametric algorithm, called enhanced matrix completion (EMaC), based on
structured matrix completion. The algorithm starts by arranging the data into a
low-rank enhanced form with multi-fold Hankel structure, then attempts recovery
via nuclear norm minimization. Under mild incoherence conditions, EMaC allows
perfect recovery as soon as the number of samples exceeds the order of
$\mathcal{O}(r\log^{2} n)$. We also show that, in many instances, accurate
completion of a low-rank multi-fold Hankel matrix is possible when the number
of observed entries is proportional to the information theoretical limits
(except for a logarithmic gap). The robustness of EMaC against bounded noise
and its applicability to super resolution are further demonstrated by numerical
experiments.
Revealing social networks of spammers through spectral clustering
Authors: Kevin S. Xu, Mark Kliger, Yilun Chen, Peter J. Woolf, Alfred O. Hero III
Date: 2013-04-30 00:00:00
Tasks: Clustering

To date, most studies on spam have focused only on the spamming phase of the
spam cycle and have ignored the harvesting phase, which consists of the mass
acquisition of email addresses. It has been observed that spammers conceal
their identity to a lesser degree in the harvesting phase, so it may be
possible to gain new insights into spammers' behavior by studying the behavior
of harvesters, which are individuals or bots that collect email addresses. In
this paper, we reveal social networks of spammers by identifying communities of
harvesters with high behavioral similarity using spectral clustering. The data
analyzed was collected through Project Honey Pot, a distributed system for
monitoring harvesting and spamming. Our main findings are (1) that most
spammers either send only phishing emails or no phishing emails at all, (2)
that most communities of spammers also send only phishing emails or no phishing
emails at all, and (3) that several groups of spammers within communities
exhibit coherent temporal behavior and have similar IP addresses. Our findings
reveal some previously unknown behavior of spammers and suggest that there is
indeed social structure between spammers to be discovered.
Image Compression By Embedding Five Modulus Method Into JPEG
Authors: Firas A. Jassim
Date: 2013-04-30 00:00:00
Tasks: Image Compression

The standard JPEG format is almost the optimum format in image compression.
The compression ratio in JPEG sometimes reaches 30:1. The compression ratio of
JPEG could be increased by embedding the Five Modulus Method (FMM) into the
JPEG algorithm. The novel algorithm gives twice the time as the standard JPEG
algorithm or more. The novel algorithm was called FJPEG (Five-JPEG). The
quality of the reconstructed image after compression is approximately
approaches the JPEG. Standard test images have been used to support and
implement the suggested idea in this paper and the error metrics have been
computed and compared with JPEG.
Inferring ground truth from multi-annotator ordinal data: a probabilistic approach
Authors: Balaji Lakshminarayanan, Yee Whye Teh
Date: 2013-04-30 00:00:00
Tasks: Bayesian Inference, parameter estimation

A popular approach for large scale data annotation tasks is crowdsourcing,
wherein each data point is labeled by multiple noisy annotators. We consider
the problem of inferring ground truth from noisy ordinal labels obtained from
multiple annotators of varying and unknown expertise levels. Annotation models
for ordinal data have been proposed mostly as extensions of their
binary/categorical counterparts and have received little attention in the
crowdsourcing literature. We propose a new model for crowdsourced ordinal data
that accounts for instance difficulty as well as annotator expertise, and
derive a variational Bayesian inference algorithm for parameter estimation. We
analyze the ordinal extensions of several state-of-the-art annotator models for
binary/categorical labels and evaluate the performance of all the models on two
real world datasets containing ordinal query-URL relevance scores, collected
through Amazon's Mechanical Turk. Our results indicate that the proposed model
performs better or as well as existing state-of-the-art methods and is more
resistant to `spammy' annotators (i.e., annotators who assign labels randomly
without actually looking at the instance) than popular baselines such as mean,
median, and majority vote which do not account for annotator expertise.