pwc-archive/papers-with-abstracts huggingface dataset:
GAP Safe screening rules for sparse multi-task and multi-class models
Authors: Eugene Ndiaye, Olivier Fercoq, Alexandre Gramfort, Joseph Salmon
Date: 2015-06-11 00:00:00
Proceeding: NeurIPS 2015 12
Tasks: regression

High dimensional regression benefits from sparsity promoting regularizations.
Screening rules leverage the known sparsity of the solution by ignoring some
variables in the optimization, hence speeding up solvers. When the procedure is
proven not to discard features wrongly the rules are said to be \emph{safe}. In
this paper we derive new safe rules for generalized linear models regularized
with $\ell_1$ and $\ell_1/\ell_2$ norms. The rules are based on duality gap
computations and spherical safe regions whose diameters converge to zero. This
allows to discard safely more variables, in particular for low regularization
parameters. The GAP Safe rule can cope with any iterative solver and we
illustrate its performance on coordinate descent for multi-task Lasso, binary
and multinomial logistic regression, demonstrating significant speed ups on all
tested datasets with respect to previous safe rules.
Efficient Output Kernel Learning for Multiple Tasks
Authors: Pratik Jawanpuria, Maksim Lapin, Matthias Hein, Bernt Schiele
Date: 2015-11-18 00:00:00
Proceeding: NeurIPS 2015 12
Tasks: Computational Efficiency, Multi-Task Learning

The paradigm of multi-task learning is that one can achieve better
generalization by learning tasks jointly and thus exploiting the similarity
between the tasks rather than learning them independently of each other. While
previously the relationship between tasks had to be user-defined in the form of
an output kernel, recent approaches jointly learn the tasks and the output
kernel. As the output kernel is a positive semidefinite matrix, the resulting
optimization problems are not scalable in the number of tasks as an
eigendecomposition is required in each step. \mbox{Using} the theory of
positive semidefinite kernels we show in this paper that for a certain class of
regularizers on the output kernel, the constraint of being positive
semidefinite can be dropped as it is automatically satisfied for the relaxed
problem. This leads to an unconstrained dual problem which can be solved
efficiently. Experiments on several multi-task and multi-class data sets
illustrate the efficacy of our approach in terms of computational efficiency as
well as generalization performance.
Budget Online Multiple Kernel Learning
Authors: Jing Lu, Steven C. H. Hoi, Doyen Sahoo, Peilin Zhao
Date: 2015-11-16 00:00:00
Tasks: General Classification

Online learning with multiple kernels has gained increasing interests in
recent years and found many applications. For classification tasks, Online
Multiple Kernel Classification (OMKC), which learns a kernel based classifier
by seeking the optimal linear combination of a pool of single kernel
classifiers in an online fashion, achieves superior accuracy and enjoys great
flexibility compared with traditional single-kernel classifiers. Despite being
studied extensively, existing OMKC algorithms suffer from high computational
cost due to their unbounded numbers of support vectors. To overcome this
drawback, we present a novel framework of Budget Online Multiple Kernel
Learning (BOMKL) and propose a new Sparse Passive Aggressive learning to
perform effective budget online learning. Specifically, we adopt a simple yet
effective Bernoulli sampling to decide if an incoming instance should be added
to the current set of support vectors. By limiting the number of support
vectors, our method can significantly accelerate OMKC while maintaining
satisfactory accuracy that is comparable to that of the existing OMKC
algorithms. We theoretically prove that our new method achieves an optimal
regret bound in expectation, and empirically found that the proposed algorithm
outperforms various OMKC algorithms and can easily scale up to large-scale
datasets.
Compositional Memory for Visual Question Answering
Authors: Aiwen Jiang, Fang Wang, Fatih Porikli, Yi Li
Date: 2015-11-18 00:00:00
Tasks: Question Answering, Visual Question Answering, Visual Question Answering (VQA)

Visual Question Answering (VQA) emerges as one of the most fascinating topics
in computer vision recently. Many state of the art methods naively use holistic
visual features with language features into a Long Short-Term Memory (LSTM)
module, neglecting the sophisticated interaction between them. This coarse
modeling also blocks the possibilities of exploring finer-grained local
features that contribute to the question answering dynamically over time.
  This paper addresses this fundamental problem by directly modeling the
temporal dynamics between language and all possible local image patches. When
traversing the question words sequentially, our end-to-end approach explicitly
fuses the features associated to the words and the ones available at multiple
local patches in an attention mechanism, and further combines the fused
information to generate dynamic messages, which we call episode. We then feed
the episodes to a standard question answering module together with the
contextual visual information and linguistic information. Motivated by recent
practices in deep learning, we use auxiliary loss functions during training to
improve the performance. Our experiments on two latest public datasets suggest
that our method has a superior performance. Notably, on the DARQUAR dataset we
advanced the state of the art by 6$\%$, and we also evaluated our approach on
the most recent MSCOCO-VQA dataset.
Learning with $\ell^{0}$-Graph: $\ell^{0}$-Induced Sparse Subspace Clustering
Authors: Yingzhen Yang, Jiashi Feng, Jianchao Yang, Thomas S. Huang
Date: 2015-10-28 00:00:00
Tasks: Clustering

Sparse subspace clustering methods, such as Sparse Subspace Clustering (SSC)
\cite{ElhamifarV13} and $\ell^{1}$-graph \cite{YanW09,ChengYYFH10}, are
effective in partitioning the data that lie in a union of subspaces. Most of
those methods use $\ell^{1}$-norm or $\ell^{2}$-norm with thresholding to
impose the sparsity of the constructed sparse similarity graph, and certain
assumptions, e.g. independence or disjointness, on the subspaces are required
to obtain the subspace-sparse representation, which is the key to their
success. Such assumptions are not guaranteed to hold in practice and they limit
the application of sparse subspace clustering on subspaces with general
location. In this paper, we propose a new sparse subspace clustering method
named $\ell^{0}$-graph. In contrast to the required assumptions on subspaces
for most existing sparse subspace clustering methods, it is proved that
subspace-sparse representation can be obtained by $\ell^{0}$-graph for
arbitrary distinct underlying subspaces almost surely under the mild i.i.d.
assumption on the data generation. We develop a proximal method to obtain the
sub-optimal solution to the optimization problem of $\ell^{0}$-graph with
proved guarantee of convergence. Moreover, we propose a regularized
$\ell^{0}$-graph that encourages nearby data to have similar neighbors so that
the similarity graph is more aligned within each cluster and the graph
connectivity issue is alleviated. Extensive experimental results on various
data sets demonstrate the superiority of $\ell^{0}$-graph compared to other
competing clustering methods, as well as the effectiveness of regularized
$\ell^{0}$-graph.
Reversible Recursive Instance-level Object Segmentation
Authors: Xiaodan Liang, Yunchao Wei, Xiaohui Shen, Zequn Jie, Jiashi Feng, Liang Lin, Shuicheng Yan
Date: 2015-11-14 00:00:00
Proceeding: CVPR 2016 6
Tasks: Denoising, Object, Segmentation, Semantic Segmentation

In this work, we propose a novel Reversible Recursive Instance-level Object
Segmentation (R2-IOS) framework to address the challenging instance-level
object segmentation task. R2-IOS consists of a reversible proposal refinement
sub-network that predicts bounding box offsets for refining the object proposal
locations, and an instance-level segmentation sub-network that generates the
foreground mask of the dominant object instance in each proposal. By being
recursive, R2-IOS iteratively optimizes the two sub-networks during joint
training, in which the refined object proposals and improved segmentation
predictions are alternately fed into each other to progressively increase the
network capabilities. By being reversible, the proposal refinement sub-network
adaptively determines an optimal number of refinement iterations required for
each proposal during both training and testing. Furthermore, to handle multiple
overlapped instances within a proposal, an instance-aware denoising autoencoder
is introduced into the segmentation sub-network to distinguish the dominant
object from other distracting instances. Extensive experiments on the
challenging PASCAL VOC 2012 benchmark well demonstrate the superiority of
R2-IOS over other state-of-the-art methods. In particular, the $\text{AP}^r$
over $20$ classes at $0.5$ IoU achieves $66.7\%$, which significantly
outperforms the results of $58.7\%$ by PFN~\cite{PFN} and $46.3\%$
by~\cite{liu2015multi}.
Accelerating Very Deep Convolutional Networks for Classification and Detection
Authors: Xiangyu Zhang, Jianhua Zou, Kaiming He, Jian Sun
Date: 2015-05-26 00:00:00
Tasks: Classification, General Classification, Model Compression, object-detection, Object Detection

This paper aims to accelerate the test-time computation of convolutional
neural networks (CNNs), especially very deep CNNs that have substantially
impacted the computer vision community. Unlike previous methods that are
designed for approximating linear filters or linear responses, our method takes
the nonlinear units into account. We develop an effective solution to the
resulting nonlinear optimization problem without the need of stochastic
gradient descent (SGD). More importantly, while previous methods mainly focus
on optimizing one or two layers, our nonlinear method enables an asymmetric
reconstruction that reduces the rapidly accumulated error when multiple (e.g.,
>=10) layers are approximated. For the widely used very deep VGG-16 model, our
method achieves a whole-model speedup of 4x with merely a 0.3% increase of
top-5 error in ImageNet classification. Our 4x accelerated VGG-16 model also
shows a graceful accuracy degradation for object detection when plugged into
the Fast R-CNN detector.
Managing Multi-Granular Linguistic Distribution Assessments in Large-Scale Multi-Attribute Group Decision Making
Authors: Zhen Zhang, Chonghui Guo, Luis Martínez
Date: 2015-04-04 00:00:00
Tasks: Attribute, Decision Making

Linguistic large-scale group decision making (LGDM) problems are more and
more common nowadays. In such problems a large group of decision makers are
involved in the decision process and elicit linguistic information that are
usually assessed in different linguistic scales with diverse granularity
because of decision makers' distinct knowledge and background. To keep maximum
information in initial stages of the linguistic LGDM problems, the use of
multi-granular linguistic distribution assessments seems a suitable choice,
however to manage such multigranular linguistic distribution assessments, it is
necessary the development of a new linguistic computational approach. In this
paper it is proposed a novel computational model based on the use of extended
linguistic hierarchies, which not only can be used to operate with
multi-granular linguistic distribution assessments, but also can provide
interpretable linguistic results to decision makers. Based on this new
linguistic computational model, an approach to linguistic large-scale
multi-attribute group decision making is proposed and applied to a talent
selection process in universities.
Discovering Underlying Plans Based on Distributed Representations of Actions
Authors: Xin Tian, Hankz Hankui Zhuo, Subbarao Kambhampati
Date: 2015-11-18 00:00:00

Plan recognition aims to discover target plans (i.e., sequences of actions)
behind observed actions, with history plan libraries or domain models in hand.
Previous approaches either discover plans by maximally "matching" observed
actions to plan libraries, assuming target plans are from plan libraries, or
infer plans by executing domain models to best explain the observed actions,
assuming complete domain models are available. In real world applications,
however, target plans are often not from plan libraries and complete domain
models are often not available, since building complete sets of plans and
complete domain models are often difficult or expensive. In this paper we view
plan libraries as corpora and learn vector representations of actions using the
corpora; we then discover target plans based on the vector representations. Our
approach is capable of discovering underlying plans that are not from plan
libraries, without requiring domain models provided. We empirically demonstrate
the effectiveness of our approach by comparing its performance to traditional
plan recognition approaches in three planning domains.
Bayesian hypothesis testing for one bit compressed sensing with sensing matrix perturbation
Authors: H. Zayyani, M. Korki, F. Marvasti
Date: 2015-11-18 00:00:00
Tasks: compressed sensing, Two-sample testing

This letter proposes a low-computational Bayesian algorithm for noisy sparse
recovery in the context of one bit compressed sensing with sensing matrix
perturbation. The proposed algorithm which is called BHT-MLE comprises a sparse
support detector and an amplitude estimator. The support detector utilizes
Bayesian hypothesis test, while the amplitude estimator uses an ML estimator
which is obtained by solving a convex optimization problem. Simulation results
show that BHT-MLE algorithm offers more reconstruction accuracy than that of an
ML estimator (MLE) at a low computational cost.
Watch and Learn: Optimizing from Revealed Preferences Feedback
Authors: Aaron Roth, Jonathan Ullman, Zhiwei Steven Wu
Date: 2015-04-04 00:00:00

A Stackelberg game is played between a leader and a follower. The leader
first chooses an action, then the follower plays his best response. The goal of
the leader is to pick the action that will maximize his payoff given the
follower's best response. In this paper we present an approach to solving for
the leader's optimal strategy in certain Stackelberg games where the follower's
utility function (and thus the subsequent best response of the follower) is
unknown.
  Stackelberg games capture, for example, the following interaction between a
producer and a consumer. The producer chooses the prices of the goods he
produces, and then a consumer chooses to buy a utility maximizing bundle of
goods. The goal of the seller here is to set prices to maximize his
profit---his revenue, minus the production cost of the purchased bundle. It is
quite natural that the seller in this example should not know the buyer's
utility function. However, he does have access to revealed preference
feedback---he can set prices, and then observe the purchased bundle and his own
profit. We give algorithms for efficiently solving, in terms of both
computational and query complexity, a broad class of Stackelberg games in which
the follower's utility function is unknown, using only "revealed preference"
access to it. This class includes in particular the profit maximization
problem, as well as the optimal tolling problem in nonatomic congestion games,
when the latency functions are unknown. Surprisingly, we are able to solve
these problems even though the optimization problems are non-convex in the
leader's actions.
Coercive Region-level Registration for Multi-modal Images
Authors: Yu-Hui Chen, Dennis Wei, Gregory Newstadt, Jeffrey Simmons, Alfred Hero
Date: 2015-02-26 00:00:00

We propose a coercive approach to simultaneously register and segment
multi-modal images which share similar spatial structure. Registration is done
at the region level to facilitate data fusion while avoiding the need for
interpolation. The algorithm performs alternating minimization of an objective
function informed by statistical models for pixel values in different
modalities. Hypothesis tests are developed to determine whether to refine
segmentations by splitting regions. We demonstrate that our approach has
significantly better performance than the state-of-the-art registration and
segmentation methods on microscopy images.
Tree-Guided MCMC Inference for Normalized Random Measure Mixture Models
Authors: Juho Lee, Seungjin Choi
Date: 2015-11-18 00:00:00
Proceeding: NeurIPS 2015 12
Tasks: Clustering

Normalized random measures (NRMs) provide a broad class of discrete random
measures that are often used as priors for Bayesian nonparametric models.
Dirichlet process is a well-known example of NRMs. Most of posterior inference
methods for NRM mixture models rely on MCMC methods since they are easy to
implement and their convergence is well studied. However, MCMC often suffers
from slow convergence when the acceptance rate is low. Tree-based inference is
an alternative deterministic posterior inference method, where Bayesian
hierarchical clustering (BHC) or incremental Bayesian hierarchical clustering
(IBHC) have been developed for DP or NRM mixture (NRMM) models, respectively.
Although IBHC is a promising method for posterior inference for NRMM models due
to its efficiency and applicability to online inference, its convergence is not
guaranteed since it uses heuristics that simply selects the best solution after
multiple trials are made. In this paper, we present a hybrid inference
algorithm for NRMM models, which combines the merits of both MCMC and IBHC.
Trees built by IBHC outlines partitions of data, which guides
Metropolis-Hastings procedure to employ appropriate proposals. Inheriting the
nature of MCMC, our tree-guided MCMC (tgMCMC) is guaranteed to converge, and
enjoys the fast convergence thanks to the effective proposals guided by trees.
Experiments on both synthetic and real-world datasets demonstrate the benefit
of our method.
A New Smooth Approximation to the Zero One Loss with a Probabilistic Interpretation
Authors: Md. Kamrul Hasan, Christopher J. Pal
Date: 2015-11-18 00:00:00
Tasks: Structured Prediction

We examine a new form of smooth approximation to the zero one loss in which
learning is performed using a reformulation of the widely used logistic
function. Our approach is based on using the posterior mean of a novel
generalized Beta-Bernoulli formulation. This leads to a generalized logistic
function that approximates the zero one loss, but retains a probabilistic
formulation conferring a number of useful properties. The approach is easily
generalized to kernel logistic regression and easily integrated into methods
for structured prediction. We present experiments in which we learn such models
using an optimization method consisting of a combination of gradient descent
and coordinate descent using localized grid search so as to escape from local
minima. Our experiments indicate that optimization quality is improved when
learning meta-parameters are themselves optimized using a validation set. Our
experiments show improved performance relative to widely used logistic and
hinge loss methods on a wide variety of problems ranging from standard UC
Irvine and libSVM evaluation datasets to product review predictions and a
visual information extraction task. We observe that the approach: 1) is more
robust to outliers compared to the logistic and hinge losses; 2) outperforms
comparable logistic and max margin models on larger scale benchmark problems;
3) when combined with Gaussian- Laplacian mixture prior on parameters the
kernelized version of our formulation yields sparser solutions than Support
Vector Machine classifiers; and 4) when integrated into a probabilistic
structured prediction technique our approach provides more accurate
probabilities yielding improved inference and increasing information extraction
performance.
A state vector algebra for algorithmic implementation of second-order logic
Authors: Dmitry Lesnik, Tobias Schaefer
Date: 2013-12-09 00:00:00
Tasks: Automated Theorem Proving

We present a mathematical framework for mapping second-order logic relations
onto a simple state vector algebra. Using this algebra, basic theorems of set
theory can be proven in an algorithmic way, hence by an expert system. We
illustrate the use of the algebra with simple examples and show that, in
principle, all theorems of basic set theory can be recovered in an elementary
way. The developed technique can be used for an automated theorem proving in
the 1st and 2nd order logic.
Competitive Multi-scale Convolution
Authors: Zhibin Liao, Gustavo Carneiro
Date: 2015-11-18 00:00:00
Tasks: Image Classification

In this paper, we introduce a new deep convolutional neural network (ConvNet)
module that promotes competition among a set of multi-scale convolutional
filters. This new module is inspired by the inception module, where we replace
the original collaborative pooling stage (consisting of a concatenation of the
multi-scale filter outputs) by a competitive pooling represented by a maxout
activation unit. This extension has the following two objectives: 1) the
selection of the maximum response among the multi-scale filters prevents filter
co-adaptation and allows the formation of multiple sub-networks within the same
model, which has been shown to facilitate the training of complex learning
problems; and 2) the maxout unit reduces the dimensionality of the outputs from
the multi-scale filters. We show that the use of our proposed module in typical
deep ConvNets produces classification results that are either better than or
comparable to the state of the art on the following benchmark datasets: MNIST,
CIFAR-10, CIFAR-100 and SVHN.
MOEA/D-GM: Using probabilistic graphical models in MOEA/D for solving combinatorial optimization problems
Authors: Murilo Zangari de Souza, Roberto Santana, Aurora Trinidad Ramirez Pozo, Alexander Mendiburu
Date: 2015-11-18 00:00:00
Tasks: Combinatorial Optimization, Evolutionary Algorithms

Evolutionary algorithms based on modeling the statistical dependencies
(interactions) between the variables have been proposed to solve a wide range
of complex problems. These algorithms learn and sample probabilistic graphical
models able to encode and exploit the regularities of the problem. This paper
investigates the effect of using probabilistic modeling techniques as a way to
enhance the behavior of MOEA/D framework. MOEA/D is a decomposition based
evolutionary algorithm that decomposes a multi-objective optimization problem
(MOP) in a number of scalar single-objective subproblems and optimizes them in
a collaborative manner. MOEA/D framework has been widely used to solve several
MOPs. The proposed algorithm, MOEA/D using probabilistic Graphical Models
(MOEA/D-GM) is able to instantiate both univariate and multi-variate
probabilistic models for each subproblem. To validate the introduced framework
algorithm, an experimental study is conducted on a multi-objective version of
the deceptive function Trap5. The results show that the variant of the
framework (MOEA/D-Tree), where tree models are learned from the matrices of the
mutual information between the variables, is able to capture the structure of
the problem. MOEA/D-Tree is able to achieve significantly better results than
both MOEA/D using genetic operators and MOEA/D using univariate probability
models, in terms of the approximation to the true Pareto front.