pwc-archive/papers-with-abstracts huggingface dataset:
Bregman Alternating Direction Method of Multipliers
Authors: Huahua Wang, Arindam Banerjee
Date: 2013-06-13 00:00:00
Proceeding: NeurIPS 2014 12
Tasks: GPU

The mirror descent algorithm (MDA) generalizes gradient descent by using a
Bregman divergence to replace squared Euclidean distance. In this paper, we
similarly generalize the alternating direction method of multipliers (ADMM) to
Bregman ADMM (BADMM), which allows the choice of different Bregman divergences
to exploit the structure of problems. BADMM provides a unified framework for
ADMM and its variants, including generalized ADMM, inexact ADMM and Bethe ADMM.
  We establish the global convergence and the $O(1/T)$ iteration complexity for
BADMM. In some cases, BADMM can be faster than ADMM by a factor of
$O(n/\log(n))$. In solving the linear program of mass transportation problem,
BADMM leads to massive parallelism and can easily run on GPU. BADMM is several
times faster than highly optimized commercial software Gurobi.
Lexpresso: a Controlled Natural Language
Authors: Adam Saulwick
Date: 2014-07-08 00:00:00

This paper presents an overview of `Lexpresso', a Controlled Natural Language
developed at the Defence Science & Technology Organisation as a bidirectional
natural language interface to a high-level information fusion system. The paper
describes Lexpresso's main features including lexical coverage, expressiveness
and range of linguistic syntactic and semantic structures. It also touches on
its tight integration with a formal semantic formalism and tentatively
classifies it against the PENS system.
A higher-order MRF based variational model for multiplicative noise reduction
Authors: Yunjin Chen, Wensen Feng, René Ranftl, Hong Qiao, Thomas Pock
Date: 2014-04-21 00:00:00
Tasks: GPU, Image Restoration

The Fields of Experts (FoE) image prior model, a filter-based higher-order
Markov Random Fields (MRF) model, has been shown to be effective for many image
restoration problems. Motivated by the successes of FoE-based approaches, in
this letter, we propose a novel variational model for multiplicative noise
reduction based on the FoE image prior model. The resulted model corresponds to
a non-convex minimization problem, which can be solved by a recently published
non-convex optimization algorithm. Experimental results based on synthetic
speckle noise and real synthetic aperture radar (SAR) images suggest that the
performance of our proposed method is on par with the best published
despeckling algorithm. Besides, our proposed model comes along with an
additional advantage, that the inference is extremely efficient. {Our GPU based
implementation takes less than 1s to produce state-of-the-art despeckling
performance.}
Recommending Learning Algorithms and Their Associated Hyperparameters
Authors: Michael R. Smith, Logan Mitchell, Christophe Giraud-Carrier, Tony Martinez
Date: 2014-07-07 00:00:00
Tasks: BIG-bench Machine Learning, Collaborative Filtering, Meta-Learning

The success of machine learning on a given task dependson, among other
things, which learning algorithm is selected and its associated
hyperparameters. Selecting an appropriate learning algorithm and setting its
hyperparameters for a given data set can be a challenging task, especially for
users who are not experts in machine learning. Previous work has examined using
meta-features to predict which learning algorithm and hyperparameters should be
used. However, choosing a set of meta-features that are predictive of algorithm
performance is difficult. Here, we propose to apply collaborative filtering
techniques to learning algorithm and hyperparameter selection, and find that
doing so avoids determining which meta-features to use and outperforms
traditional meta-learning approaches in many cases.
Spectral norm of random tensors
Authors: Ryota Tomioka, Taiji Suzuki
Date: 2014-07-07 00:00:00

We show that the spectral norm of a random $n_1\times n_2\times \cdots \times
n_K$ tensor (or higher-order array) scales as
$O\left(\sqrt{(\sum_{k=1}^{K}n_k)\log(K)}\right)$ under some sub-Gaussian
assumption on the entries. The proof is based on a covering number argument.
Since the spectral norm is dual to the tensor nuclear norm (the tightest convex
relaxation of the set of rank one tensors), the bound implies that the convex
relaxation yields sample complexity that is linear in (the sum of) the number
of dimensions, which is much smaller than other recently proposed convex
relaxations of tensor rank that use unfolding.
Simultaneous Detection and Segmentation
Authors: Bharath Hariharan, Pablo Arbeláez, Ross Girshick, Jitendra Malik
Date: 2014-07-07 00:00:00
Tasks: Diagnostic, object-detection, Object Detection, Segmentation, Semantic Segmentation

We aim to detect all instances of a category in an image and, for each
instance, mark the pixels that belong to it. We call this task Simultaneous
Detection and Segmentation (SDS). Unlike classical bounding box detection, SDS
requires a segmentation and not just a box. Unlike classical semantic
segmentation, we require individual object instances. We build on recent work
that uses convolutional neural networks to classify category-independent region
proposals (R-CNN [16]), introducing a novel architecture tailored for SDS. We
then use category-specific, top- down figure-ground predictions to refine our
bottom-up proposals. We show a 7 point boost (16% relative) over our baselines
on SDS, a 5 point boost (10% relative) over state-of-the-art on semantic
segmentation, and state-of-the-art performance in object detection. Finally, we
provide diagnostic tools that unpack performance and provide directions for
future work.
DimmWitted: A Study of Main-Memory Statistical Analytics
Authors: Ce Zhang, Christopher Ré
Date: 2014-03-28 00:00:00

We perform the first study of the tradeoff space of access methods and
replication to support statistical analytics using first-order methods executed
in the main memory of a Non-Uniform Memory Access (NUMA) machine. Statistical
analytics systems differ from conventional SQL-analytics in the amount and
types of memory incoherence they can tolerate. Our goal is to understand
tradeoffs in accessing the data in row- or column-order and at what granularity
one should share the model and data for a statistical task. We study this new
tradeoff space, and discover there are tradeoffs between hardware and
statistical efficiency. We argue that our tradeoff study may provide valuable
information for designers of analytics engines: for each system we consider,
our prototype engine can run at least one popular task at least 100x faster. We
conduct our study across five architectures using popular models including
SVMs, logistic regression, Gibbs sampling, and neural networks.
Downscaling near-surface atmospheric fields with multi-objective Genetic Programming
Authors: Tanja Zerenner, Victor Venema, Petra Friederichs, Clemens Simmer
Date: 2014-07-07 00:00:00

The coupling of models for the different components of the
Soil-Vegetation-Atmosphere-System is required to investigate component
interactions and feedback processes. However, the component models for
atmosphere, land-surface and subsurface are usually operated at different
resolutions in space and time owing to the dominant processes. The
computationally often more expensive atmospheric models, for instance, are
typically employed at a coarser resolution than land-surface and subsurface
models. Thus up- and downscaling procedures are required at the interface
between the atmospheric model and the land-surface/subsurface models. We apply
multi-objective Genetic Programming (GP) to a training data set of
high-resolution atmospheric model runs to learn equations or short programs
that reconstruct the fine-scale fields (e.g., 400 m resolution) of the
near-surface atmospheric state variables from the coarse atmospheric model
output (e.g., 2.8 km resolution). Like artificial neural networks, GP can
flexibly incorporate multivariate and nonlinear relations, but offers the
advantage that the solutions are human readable and thus can be checked for
physical consistency. Using the Strength Pareto Approach for multi-objective
fitness assignment allows us to consider multiple characteristics of the
fine-scale fields during the learning procedure.
The Primal-Dual Hybrid Gradient Method for Semiconvex Splittings
Authors: Thomas Möllenhoff, Evgeny Strekalovskiy, Michael Moeller, Daniel Cremers
Date: 2014-07-07 00:00:00

This paper deals with the analysis of a recent reformulation of the
primal-dual hybrid gradient method [Zhu and Chan 2008, Pock, Cremers, Bischof
and Chambolle 2009, Esser, Zhang and Chan 2010, Chambolle and Pock 2011], which
allows to apply it to nonconvex regularizers as first proposed for truncated
quadratic penalization in [Strekalovskiy and Cremers 2014]. Particularly, it
investigates variational problems for which the energy to be minimized can be
written as $G(u) + F(Ku)$, where $G$ is convex, $F$ semiconvex, and $K$ is a
linear operator. We study the method and prove convergence in the case where
the nonconvexity of $F$ is compensated by the strong convexity of the $G$. The
convergence proof yields an interesting requirement for the choice of algorithm
parameters, which we show to not only be sufficient, but necessary.
Additionally, we show boundedness of the iterates under much weaker conditions.
Finally, we demonstrate effectiveness and convergence of the algorithm beyond
the theoretical guarantees in several numerical experiments.
WordRep: A Benchmark for Research on Learning Word Representations
Authors: Bin Gao, Jiang Bian, Tie-Yan Liu
Date: 2014-07-07 00:00:00
Tasks: Word Embeddings

WordRep is a benchmark collection for the research on learning distributed
word representations (or word embeddings), released by Microsoft Research. In
this paper, we describe the details of the WordRep collection and show how to
use it in different types of machine learning research related to word
embedding. Specifically, we describe how the evaluation tasks in WordRep are
selected, how the data are sampled, and how the evaluation tool is built. We
then compare several state-of-the-art word representations on WordRep, report
their evaluation performance, and make discussions on the results. After that,
we discuss new potential research topics that can be supported by WordRep, in
addition to algorithm comparison. We hope that this paper can help people gain
deeper understanding of WordRep, and enable more interesting research on
learning distributed word representations and related topics.
Neural Codes for Image Retrieval
Authors: Artem Babenko, Anton Slesarev, Alexandr Chigorin, Victor Lempitsky
Date: 2014-04-07 00:00:00
Tasks: Dimensionality Reduction, Image Retrieval, Retrieval

It has been shown that the activations invoked by an image within the top
layers of a large convolutional neural network provide a high-level descriptor
of the visual content of the image. In this paper, we investigate the use of
such descriptors (neural codes) within the image retrieval application. In the
experiments with several standard retrieval benchmarks, we establish that
neural codes perform competitively even when the convolutional neural network
has been trained for an unrelated classification task (e.g.\ Image-Net). We
also evaluate the improvement in the retrieval performance of neural codes,
when the network is retrained on a dataset of images that are similar to images
encountered at test time.
  We further evaluate the performance of the compressed neural codes and show
that a simple PCA compression provides very good short codes that give
state-of-the-art accuracy on a number of datasets. In general, neural codes
turn out to be much more resilient to such compression in comparison other
state-of-the-art descriptors. Finally, we show that discriminative
dimensionality reduction trained on a dataset of pairs of matched photographs
improves the performance of PCA-compressed neural codes even further. Overall,
our quantitative experiments demonstrate the promise of neural codes as visual
descriptors for image retrieval.
Les noms propres se traduisent-ils ? Étude d'un corpus multilingue
Authors: Émeline Lecuit, Denis Maurel, Dusko Vitas
Date: 2014-07-07 00:00:00
Tasks: Translation

In this paper, we tackle the problem of the translation of proper names. We
introduce our hypothesis according to which proper names can be translated more
often than most people seem to think. Then, we describe the construction of a
parallel multilingual corpus used to illustrate our point. We eventually
evaluate both the advantages and limits of this corpus in our study.
A Clustering Approach to Learn Sparsely-Used Overcomplete Dictionaries
Authors: Alekh Agarwal, Animashree Anandkumar, Praneeth Netrapalli
Date: 2013-09-08 00:00:00
Tasks: Clustering, regression

We consider the problem of learning overcomplete dictionaries in the context
of sparse coding, where each sample selects a sparse subset of dictionary
elements. Our main result is a strategy to approximately recover the unknown
dictionary using an efficient algorithm. Our algorithm is a clustering-style
procedure, where each cluster is used to estimate a dictionary element. The
resulting solution can often be further cleaned up to obtain a high accuracy
estimate, and we provide one simple scenario where $\ell_1$-regularized
regression can be used for such a second stage.
A Coordinated MDP Approach to Multi-Agent Planning for Resource Allocation, with Applications to Healthcare
Authors: Hadi Hosseini, Jesse Hoey, Robin Cohen
Date: 2014-07-07 00:00:00

This paper considers a novel approach to scalable multiagent resource
allocation in dynamic settings. We propose an approximate solution in which
each resource consumer is represented by an independent MDP-based agent that
models expected utility using an average model of its expected access to
resources given only limited information about all other agents. A global
auction-based mechanism is proposed for allocations based on expected regret.
We assume truthful bidding and a cooperative coordination mechanism, as we are
considering healthcare scenarios. We illustrate the performance of our
coordinated MDP approach against a Monte-Carlo based planning algorithm
intended for large-scale applications, as well as other approaches suitable for
allocating medical resources. The evaluations show that the global utility
value across all consumer agents is closer to optimal when using our algorithms
under certain time constraints, with low computational cost. As such, we offer
a promising approach for addressing complex resource allocation problems that
arise in healthcare settings.
Large-Scale Multi-Label Learning with Incomplete Label Assignments
Authors: Xiangnan Kong, Zhaoming Wu, Li-Jia Li, Ruofei Zhang, Philip S. Yu, Hang Wu, Wei Fan
Date: 2014-07-06 00:00:00
Tasks: Missing Labels, Multi-Label Learning

Multi-label learning deals with the classification problems where each
instance can be assigned with multiple labels simultaneously. Conventional
multi-label learning approaches mainly focus on exploiting label correlations.
It is usually assumed, explicitly or implicitly, that the label sets for
training instances are fully labeled without any missing labels. However, in
many real-world multi-label datasets, the label assignments for training
instances can be incomplete. Some ground-truth labels can be missed by the
labeler from the label set. This problem is especially typical when the number
instances is very large, and the labeling cost is very high, which makes it
almost impossible to get a fully labeled training set. In this paper, we study
the problem of large-scale multi-label learning with incomplete label
assignments. We propose an approach, called MPU, based upon positive and
unlabeled stochastic gradient descent and stacked models. Unlike prior works,
our method can effectively and efficiently consider missing labels and label
correlations simultaneously, and is very scalable, that has linear time
complexities over the size of the data. Extensive experiments on two real-world
multi-label datasets show that our MPU model consistently outperform other
commonly-used baselines.
Bandit Online Optimization Over the Permutahedron
Authors: Nir Ailon, Kohei Hatano, Eiji Takimoto
Date: 2013-12-05 00:00:00

The permutahedron is the convex polytope with vertex set consisting of the
vectors $(\pi(1),\dots, \pi(n))$ for all permutations (bijections) $\pi$ over
$\{1,\dots, n\}$. We study a bandit game in which, at each step $t$, an
adversary chooses a hidden weight weight vector $s_t$, a player chooses a
vertex $\pi_t$ of the permutahedron and suffers an observed loss of
$\sum_{i=1}^n \pi(i) s_t(i)$.
  A previous algorithm CombBand of Cesa-Bianchi et al (2009) guarantees a
regret of $O(n\sqrt{T \log n})$ for a time horizon of $T$. Unfortunately,
CombBand requires at each step an $n$-by-$n$ matrix permanent approximation to
within improved accuracy as $T$ grows, resulting in a total running time that
is super linear in $T$, making it impractical for large time horizons.
  We provide an algorithm of regret $O(n^{3/2}\sqrt{T})$ with total time
complexity $O(n^3T)$. The ideas are a combination of CombBand and a recent
algorithm by Ailon (2013) for online optimization over the permutahedron in the
full information setting. The technical core is a bound on the variance of the
Plackett-Luce noisy sorting process's "pseudo loss". The bound is obtained by
establishing positive semi-definiteness of a family of 3-by-3 matrices
generated from rational functions of exponentials of 3 parameters.
Large-scale Supervised Hierarchical Feature Learning for Face Recognition
Authors: Jianguo Li, Yurong Chen
Date: 2014-07-06 00:00:00
Tasks: Face Recognition

This paper proposes a novel face recognition algorithm based on large-scale supervised hierarchical feature learning. The approach consists of two parts: hierarchical feature learning and large-scale model learning. The hierarchical feature learning searches feature in three levels of granularity in a supervised way. First, face images are modeled by receptive field theory, and the representation is an image with many channels of Gaussian receptive maps. We activate a few most distinguish channels by supervised learning. Second, the face image is further represented by patches of picked channels, and we search from the over-complete patch pool to activate only those most discriminant patches. Third, the feature descriptor of each patch is further projected to lower dimension subspace with discriminant subspace analysis. Learned feature of activated patches are concatenated to get a full face representation.A linear classifier is learned to separate face pairs from same subjects and different subjects. As the number of face pairs are extremely large, we introduce ADMM (alternative direction method of multipliers) to train the linear classifier on a computing cluster. Experiments show that more training samples will bring notable accuracy improvement. We conduct experiments on FRGC and LFW. Results show that the proposed approach outperforms existing algorithms under the same protocol notably. Besides, the proposed approach is small in memory footprint, and low in computing cost, which makes it suitable for embedded applications.
Fuzzy Model on Human Emotions Recognition
Authors: Kaveh Bakhtiyari, Hafizah Husain
Date: 2014-07-06 00:00:00
Tasks: model

This paper discusses a fuzzy model for multi-level human emotions recognition
by computer systems through keyboard keystrokes, mouse and touchscreen
interactions. This model can also be used to detect the other possible emotions
at the time of recognition. Accuracy measurements of human emotions by the
fuzzy model are discussed through two methods; the first is accuracy analysis
and the second is false positive rate analysis. This fuzzy model detects more
emotions, but on the other hand, for some of emotions, a lower accuracy was
obtained with the comparison with the non-fuzzy human emotions detection
methods. This system was trained and tested by Support Vector Machine (SVM) to
recognize the users' emotions. Overall, this model represents a closer
similarity between human brain detection of emotions and computer systems.
Kinetic Energy Plus Penalty Functions for Sparse Estimation
Authors: Zhihua Zhang, Shibo Zhao, Zebang Shen, Shuchang Zhou
Date: 2013-07-22 00:00:00

In this paper we propose and study a family of sparsity-inducing penalty
functions. Since the penalty functions are related to the kinetic energy in
special relativity, we call them \emph{kinetic energy plus} (KEP) functions. We
construct the KEP function by using the concave conjugate of a
$\chi^2$-distance function and present several novel insights into the KEP
function with $q=1$. In particular, we derive a thresholding operator based on
the KEP function, and prove its mathematical properties and asymptotic
properties in sparsity modeling. Moreover, we show that a coordinate descent
algorithm is especially appropriate for the KEP function. Additionally, we
discuss the relationship of KEP with the penalty functions $\ell_{1/2}$ and
MCP. The theoretical and empirical analysis validates that the KEP function is
effective and efficient in high-dimensional data modeling.
Image Fusion Using LEP Filtering and Bilinear Interpolation
Authors: Haritha Raveendran, Deepa Thomas
Date: 2014-07-05 00:00:00
Tasks: SSIM

Image Fusion is the process in which core information from a set of component
images is merged to form a single image, which is more informative and complete
than the component input images in quality and appearance. This paper presents
a fast and effective image fusion method for creating high quality fused images
by merging component images. In the proposed method, the input image is broken
down to a two-scale image representation with a base layer having large scale
variations in intensity, and a detail layer containing small scale details.
Here fusion of the base and detail layers is implemented by means of a Local
Edge preserving filtering based technique. The proposed method is an efficient
image fusion technique in which the noise component is very low and quality of
the resultant image is high so that it can be used for applications like
medical image processing, requiring very accurate edge preserved images.
Performance is tested by calculating PSNR and SSIM of images. The benefit of
the proposed method is that it removes noise without altering the underlying
structures of the image. This paper also presents an image zooming technique
using bilinear interpolation in which a portion of the input image is cropped
and bilinear interpolation is applied. Experimental results showed that the
when PSNR value is calculated, the noise is found to be very low for the
resultant image portion.
A New Approach for Super resolution by Using Web Images and FFT Based Image Registration
Authors: Archana Vijayan, Vincy Salam
Date: 2014-07-05 00:00:00
Tasks: Image Registration, SSIM, Super-Resolution

Preserving accuracy is a challenging issue in super resolution images. In
this paper, we propose a new FFT based image registration algorithm and a
sparse based super resolution algorithm to improve the accuracy of super
resolution image. Given a low resolution image, our approach initially extracts
the local descriptors from the input and then the local descriptors from the
whole correlated images using the SIFT algorithm. Once this is completed, it
will compare the local descriptors on the basis of a threshold value. The
retrieved images could be having different focal length, illumination,
inclination and size. To overcome the above differences of the retrieved
images, we propose a new FFT based image registration algorithm. After the
registration stage, we apply a sparse based super resolution on the images for
recreating images with better resolution compared to the input. Based on the
PSSNR calculation and SSIM comparison, we can see that the new methodology
creates a better image than the traditional methods.