pwc-archive/papers-with-abstracts huggingface dataset:
Student-t Processes as Alternatives to Gaussian Processes
Authors: Amar Shah, Andrew Gordon Wilson, Zoubin Ghahramani
Date: 2014-02-18 00:00:00
Tasks: Bayesian Optimization, Gaussian Processes, Model Selection

We investigate the Student-t process as an alternative to the Gaussian
process as a nonparametric prior over functions. We derive closed form
expressions for the marginal likelihood and predictive distribution of a
Student-t process, by integrating away an inverse Wishart process prior over
the covariance kernel of a Gaussian process model. We show surprising
equivalences between different hierarchical Gaussian process models leading to
Student-t processes, and derive a new sampling scheme for the inverse Wishart
process, which helps elucidate these equivalences. Overall, we show that a
Student-t process can retain the attractive properties of a Gaussian process --
a nonparametric representation, analytic marginal and predictive distributions,
and easy model selection through covariance kernels -- but has enhanced
flexibility, and predictive covariances that, unlike a Gaussian process,
explicitly depend on the values of training observations. We verify empirically
that a Student-t process is especially useful in situations where there are
changes in covariance structure, or in applications like Bayesian optimization,
where accurate predictive covariances are critical for good performance. These
advantages come at no additional computational cost over Gaussian processes.
Analysis of Multibeam SONAR Data using Dissimilarity Representations
Authors: Iain Rice, Roger Benton, Les Hart, David Lowe
Date: 2014-02-19 00:00:00

This paper considers the problem of low-dimensional visualisation of very
high dimensional information sources for the purpose of situation awareness in
the maritime environment. In response to the requirement for human decision
support aids to reduce information overload (and specifically, data amenable to
inter-point relative similarity measures) appropriate to the below-water
maritime domain, we are investigating a preliminary prototype topographic
visualisation model. The focus of the current paper is on the mathematical
problem of exploiting a relative dissimilarity representation of signals in a
visual informatics mapping model, driven by real-world sonar systems. An
independent source model is used to analyse the sonar beams from which a simple
probabilistic input model to represent uncertainty is mapped to a latent
visualisation space where data uncertainty can be accommodated. The use of
euclidean and non-euclidean measures are used and the motivation for future use
of non-euclidean measures is made. Concepts are illustrated using a simulated
64 beam weak SNR dataset with realistic sonar targets.
Unsupervised Ranking of Multi-Attribute Objects Based on Principal Curves
Authors: Chun-Guo Li, Xing Mei, Bao-Gang Hu
Date: 2014-02-19 00:00:00
Tasks: Attribute

Unsupervised ranking faces one critical challenge in evaluation applications,
that is, no ground truth is available. When PageRank and its variants show a
good solution in related subjects, they are applicable only for ranking from
link-structure data. In this work, we focus on unsupervised ranking from
multi-attribute data which is also common in evaluation tasks. To overcome the
challenge, we propose five essential meta-rules for the design and assessment
of unsupervised ranking approaches: scale and translation invariance, strict
monotonicity, linear/nonlinear capacities, smoothness, and explicitness of
parameter size. These meta-rules are regarded as high level knowledge for
unsupervised ranking tasks. Inspired by the works in [8] and [14], we propose a
ranking principal curve (RPC) model, which learns a one-dimensional manifold
function to perform unsupervised ranking tasks on multi-attribute observations.
Furthermore, the RPC is modeled to be a cubic B\'ezier curve with control
points restricted in the interior of a hypercube, thereby complying with all
the five meta-rules to infer a reasonable ranking list. With control points as
the model parameters, one is able to understand the learned manifold and to
interpret the ranking list semantically. Numerical experiments of the presented
RPC model are conducted on two open datasets of different ranking applications.
In comparison with the state-of-the-art approaches, the new model is able to
show more reasonable ranking lists.
A Statistical Approach to Set Classification by Feature Selection with Applications to Classification of Histopathology Images
Authors: Sungkyu Jung, Xingye Qiao
Date: 2014-02-19 00:00:00
Tasks: Classification, feature selection, General Classification

Set classification problems arise when classification tasks are based on sets
of observations as opposed to individual observations. In set classification, a
classification rule is trained with $N$ sets of observations, where each set is
labeled with class information, and the prediction of a class label is
performed also with a set of observations. Data sets for set classification
appear, for example, in diagnostics of disease based on multiple cell nucleus
images from a single tissue. Relevant statistical models for set classification
are introduced, which motivate a set classification framework based on
context-free feature extraction. By understanding a set of observations as an
empirical distribution, we employ a data-driven method to choose those features
which contain information on location and major variation. In particular, the
method of principal component analysis is used to extract the features of major
variation. Multidimensional scaling is used to represent features as
vector-valued points on which conventional classifiers can be applied. The
proposed set classification approaches achieve better classification results
than competing methods in a number of simulated data examples. The benefits of
our method are demonstrated in an analysis of histopathology images of cell
nuclei related to liver cancer.
Off-Policy General Value Functions to Represent Dynamic Role Assignments in RoboCup 3D Soccer Simulation
Authors: Saminda Abeyruwan, Andreas Seekircher, Ubbo Visser
Date: 2014-02-18 00:00:00
Tasks: Reinforcement Learning, Reinforcement Learning (RL), World Knowledge

Collecting and maintaining accurate world knowledge in a dynamic, complex,
adversarial, and stochastic environment such as the RoboCup 3D Soccer
Simulation is a challenging task. Knowledge should be learned in real-time with
time constraints. We use recently introduced Off-Policy Gradient Descent
algorithms within Reinforcement Learning that illustrate learnable knowledge
representations for dynamic role assignments. The results show that the agents
have learned competitive policies against the top teams from the RoboCup 2012
competitions for three vs three, five vs five, and seven vs seven agents. We
have explicitly used subsets of agents to identify the dynamics and the
semantics for which the agents learn to maximize their performance measures,
and to gather knowledge about different objectives, so that all agents
participate effectively and efficiently within the group.
High Dimensional Semiparametric Scale-Invariant Principal Component Analysis
Authors: Fang Han, Han Liu
Date: 2014-02-18 00:00:00
Tasks: feature selection, Vocal Bursts Intensity Prediction

We propose a new high dimensional semiparametric principal component analysis
(PCA) method, named Copula Component Analysis (COCA). The semiparametric model
assumes that, after unspecified marginally monotone transformations, the
distributions are multivariate Gaussian. COCA improves upon PCA and sparse PCA
in three aspects: (i) It is robust to modeling assumptions; (ii) It is robust
to outliers and data contamination; (iii) It is scale-invariant and yields more
interpretable results. We prove that the COCA estimators obtain fast estimation
rates and are feature selection consistent when the dimension is nearly
exponentially large relative to the sample size. Careful experiments confirm
that COCA outperforms sparse PCA on both synthetic and real-world datasets.
Multi-GPU Training of ConvNets
Authors: Omry Yadan, Keith Adams, Yaniv Taigman, Marc'Aurelio Ranzato
Date: 2013-12-20 00:00:00
Tasks: GPU

In this work we evaluate different approaches to parallelize computation of
convolutional neural networks across several GPUs.
Concurrent Cube-and-Conquer
Authors: Peter van der Tak, Marijn J. H. Heule, Armin Biere
Date: 2014-02-18 00:00:00

Recent work introduced the cube-and-conquer technique to solve hard SAT
instances. It partitions the search space into cubes using a lookahead solver.
Each cube is tackled by a conflict-driven clause learning (CDCL) solver.
Crucial for strong performance is the cutoff heuristic that decides when to
switch from lookahead to CDCL. Yet, this offline heuristic is far from ideal.
In this paper, we present a novel hybrid solver that applies the cube and
conquer steps simultaneously. A lookahead and a CDCL solver work together on
each cube, while communication is restricted to synchronization. Our concurrent
cube-and-conquer solver can solve many instances faster than pure lookahead,
pure CDCL and offline cube-and-conquer, and can abort early in favor of a pure
CDCL search if an instance is not suitable for cube-and-conquer techniques.
Symbiosis of Search and Heuristics for Random 3-SAT
Authors: Sid Mijnders, Boris de Wilde, Marijn Heule
Date: 2014-02-18 00:00:00

When combined properly, search techniques can reveal the full potential of
sophisticated branching heuristics. We demonstrate this observation on the
well-known class of random 3-SAT formulae. First, a new branching heuristic is
presented, which generalizes existing work on this class. Much smaller search
trees can be constructed by using this heuristic. Second, we introduce a
variant of discrepancy search, called ALDS. Theoretical and practical evidence
support that ALDS traverses the search tree in a near-optimal order when
combined with the new heuristic. Both techniques, search and heuristic, have
been implemented in the look-ahead solver march. The SAT 2009 competition
results show that march is by far the strongest complete solver on random k-SAT
formulae.
Artificial Mutation inspired Hyper-heuristic for Runtime Usage of Multi-objective Algorithms
Authors: Donia El Kateb, François Fouquet, Johann Bourcier, Yves Le Traon
Date: 2014-02-18 00:00:00
Tasks: Evolutionary Algorithms

In the last years, multi-objective evolutionary algorithms (MOEA) have been
applied to different software engineering problems where many conflicting
objectives have to be optimized simultaneously. In theory, evolutionary
algorithms feature a nice property for runtime optimization as they can provide
a solution in any execution time. In practice, based on a Darwinian inspired
natural selection, these evolutionary algorithms produce many deadborn
solutions whose computation results in a computational resources wastage:
natural selection is naturally slow. In this paper, we reconsider this founding
analogy to accelerate convergence of MOEA, by looking at modern biology
studies: artificial selection has been used to achieve an anticipated specific
purpose instead of only relying on crossover and natural selection (i.e.,
Muller et al [18] research on artificial mutation of fruits with X-Ray).
Putting aside the analogy with natural selection , the present paper proposes
an hyper-heuristic for MOEA algorithms named Sputnik 1 that uses artificial
selective mutation to improve the convergence speed of MOEA. Sputnik leverages
the past history of mutation efficiency to select the most relevant mutations
to perform. We evaluate Sputnik on a cloud-reasoning engine, which drives
on-demand provisioning while considering conflicting performance and cost
objectives. We have conducted experiments to highlight the significant
performance improvement of Sputnik in terms of resolution time.
Towards Ultra Rapid Restarts
Authors: Shai Haim, Marijn Heule
Date: 2014-02-18 00:00:00

We observe a trend regarding restart strategies used in SAT solvers. A few
years ago, most state-of-the-art solvers restarted on average after a few
thousands of backtracks. Currently, restarting after a dozen backtracks results
in much better performance. The main reason for this trend is that heuristics
and data structures have become more restart-friendly. We expect further
continuation of this trend, so future SAT solvers will restart even more
rapidly. Additionally, we present experimental results to support our
observations.
Principled Non-Linear Feature Selection
Authors: Dimitrios Athanasakis, John Shawe-Taylor, Delmiro Fernandez-Reyes
Date: 2013-12-20 00:00:00
Tasks: feature selection, Position

Recent non-linear feature selection approaches employing greedy optimisation
of Centred Kernel Target Alignment(KTA) exhibit strong results in terms of
generalisation accuracy and sparsity. However, they are computationally
prohibitive for large datasets. We propose randSel, a randomised feature
selection algorithm, with attractive scaling properties. Our theoretical
analysis of randSel provides strong probabilistic guarantees for correct
identification of relevant features. RandSel's characteristics make it an ideal
candidate for identifying informative learned representations. We've conducted
experimentation to establish the performance of this approach, and present
encouraging results, including a 3rd position result in the recent ICML black
box learning challenge as well as competitive results for signal peptide
prediction, an important problem in bioinformatics.
Direct Processing of Run Length Compressed Document Image for Segmentation and Characterization of a Specified Block
Authors: Mohammed Javed, P. Nagabhushan, B. B. Chaudhuri
Date: 2014-02-09 00:00:00

Extracting a block of interest referred to as segmenting a specified block in
an image and studying its characteristics is of general research interest, and
could be a challenging if such a segmentation task has to be carried out
directly in a compressed image. This is the objective of the present research
work. The proposal is to evolve a method which would segment and extract a
specified block, and carry out its characterization without decompressing a
compressed image, for two major reasons that most of the image archives contain
images in compressed format and decompressing an image indents additional
computing time and space. Specifically in this research work, the proposal is
to work on run-length compressed document images.
Automatic Detection of Font Size Straight from Run Length Compressed Text Documents
Authors: Mohammed Javed, P. Nagabhushan, B. B. Chaudhuri
Date: 2014-02-18 00:00:00

Automatic detection of font size finds many applications in the area of
intelligent OCRing and document image analysis, which has been traditionally
practiced over uncompressed documents, although in real life the documents
exist in compressed form for efficient storage and transmission. It would be
novel and intelligent if the task of font size detection could be carried out
directly from the compressed data of these documents without decompressing,
which would result in saving of considerable amount of processing time and
space. Therefore, in this paper we present a novel idea of learning and
detecting font size directly from run-length compressed text documents at line
level using simple line height features, which paves the way for intelligent
OCRing and document analysis directly from compressed documents. In the
proposed model, the given mixed-case text documents of different font size are
segmented into compressed text lines and the features extracted such as line
height and ascender height are used to capture the pattern of font size in the
form of a regression line, using which the automatic detection of font size is
done during the recognition stage. The method is experimented with a dataset of
50 compressed documents consisting of 780 text lines of single font size and
375 text lines of mixed font size resulting in an overall accuracy of 99.67%.
Fast X-ray CT image reconstruction using the linearized augmented Lagrangian method with ordered subsets
Authors: Hung Nien, Jeffrey A. Fessler
Date: 2014-02-18 00:00:00
Tasks: Computed Tomography (CT), Image Reconstruction

The augmented Lagrangian (AL) method that solves convex optimization problems
with linear constraints has drawn more attention recently in imaging
applications due to its decomposable structure for composite cost functions and
empirical fast convergence rate under weak conditions. However, for problems
such as X-ray computed tomography (CT) image reconstruction and large-scale
sparse regression with "big data", where there is no efficient way to solve the
inner least-squares problem, the AL method can be slow due to the inevitable
iterative inner updates. In this paper, we focus on solving regularized
(weighted) least-squares problems using a linearized variant of the AL method
that replaces the quadratic AL penalty term in the scaled augmented Lagrangian
with its separable quadratic surrogate (SQS) function, thus leading to a much
simpler ordered-subsets (OS) accelerable splitting-based algorithm, OS-LALM,
for X-ray CT image reconstruction. To further accelerate the proposed
algorithm, we use a second-order recursive system analysis to design a
deterministic downward continuation approach that avoids tedious parameter
tuning and provides fast convergence. Experimental results show that the
proposed algorithm significantly accelerates the "convergence" of X-ray CT
image reconstruction with negligible overhead and greatly reduces the OS
artifacts in the reconstructed image when using many subsets for OS
acceleration.
A Comparative Study of Machine Learning Methods for Verbal Autopsy Text Classification
Authors: Samuel Danso, Eric Atwell, Owen Johnson
Date: 2014-02-18 00:00:00
Tasks: BIG-bench Machine Learning, Classification, General Classification, text-classification, Text Classification

A Verbal Autopsy is the record of an interview about the circumstances of an
uncertified death. In developing countries, if a death occurs away from health
facilities, a field-worker interviews a relative of the deceased about the
circumstances of the death; this Verbal Autopsy can be reviewed off-site. We
report on a comparative study of the processes involved in Text Classification
applied to classifying Cause of Death: feature value representation; machine
learning classification algorithms; and feature reduction strategies in order
to identify the suitable approaches applicable to the classification of Verbal
Autopsy text. We demonstrate that normalised term frequency and the standard
TFiDF achieve comparable performance across a number of classifiers. The
results also show Support Vector Machine is superior to other classification
algorithms employed in this research. Finally, we demonstrate the effectiveness
of employing a "locally-semi-supervised" feature reduction strategy in order to
increase performance accuracy.
Learning Type-Driven Tensor-Based Meaning Representations
Authors: Tamara Polajnar, Luana Fagarasan, Stephen Clark
Date: 2013-12-20 00:00:00
Tasks: Sentence, Vocal Bursts Type Prediction

This paper investigates the learning of 3rd-order tensors representing the
semantics of transitive verbs. The meaning representations are part of a
type-driven tensor-based semantic framework, from the newly emerging field of
compositional distributional semantics. Standard techniques from the neural
networks literature are used to learn the tensors, which are tested on a
selectional preference-style task with a simple 2-dimensional sentence space.
Promising results are obtained against a competitive corpus-based baseline. We
argue that extending this work beyond transitive verbs, and to
higher-dimensional sentence spaces, is an interesting and challenging problem
for the machine learning community to consider.
A convergence proof of the split Bregman method for regularized least-squares problems
Authors: Hung Nien, Jeffrey A. Fessler
Date: 2014-02-18 00:00:00
Tasks: compressed sensing, Computed Tomography (CT), Image Reconstruction, Image Restoration

The split Bregman (SB) method [T. Goldstein and S. Osher, SIAM J. Imaging
Sci., 2 (2009), pp. 323-43] is a fast splitting-based algorithm that solves
image reconstruction problems with general l1, e.g., total-variation (TV) and
compressed sensing (CS), regularizations by introducing a single variable split
to decouple the data-fitting term and the regularization term, yielding simple
subproblems that are separable (or partially separable) and easy to minimize.
Several convergence proofs have been proposed, and these proofs either impose a
"full column rank" assumption to the split or assume exact updates in all
subproblems. However, these assumptions are impractical in many applications
such as the X-ray computed tomography (CT) image reconstructions, where the
inner least-squares problem usually cannot be solved efficiently due to the
highly shift-variant Hessian. In this paper, we show that when the data-fitting
term is quadratic, the SB method is a convergent alternating direction method
of multipliers (ADMM), and a straightforward convergence proof with inexact
updates is given using [J. Eckstein and D. P. Bertsekas, Mathematical
Programming, 55 (1992), pp. 293-318, Theorem 8]. Furthermore, since the SB
method is just a special case of an ADMM algorithm, it seems likely that the
ADMM algorithm will be faster than the SB method if the augmented Largangian
(AL) penalty parameters are selected appropriately. To have a concrete example,
we conduct a convergence rate analysis of the ADMM algorithm using two splits
for image restoration problems with quadratic data-fitting term and
regularization term. According to our analysis, we can show that the two-split
ADMM algorithm can be faster than the SB method if the AL penalty parameter of
the SB method is suboptimal. Numerical experiments were conducted to verify our
analysis.
Hybrid SRL with Optimization Modulo Theories
Authors: Stefano Teso, Roberto Sebastiani, Andrea Passerini
Date: 2014-02-18 00:00:00
Tasks: Relational Reasoning

Generally speaking, the goal of constructive learning could be seen as, given
an example set of structured objects, to generate novel objects with similar
properties. From a statistical-relational learning (SRL) viewpoint, the task
can be interpreted as a constraint satisfaction problem, i.e. the generated
objects must obey a set of soft constraints, whose weights are estimated from
the data. Traditional SRL approaches rely on (finite) First-Order Logic (FOL)
as a description language, and on MAX-SAT solvers to perform inference. Alas,
FOL is unsuited for con- structive problems where the objects contain a mixture
of Boolean and numerical variables. It is in fact difficult to implement, e.g.
linear arithmetic constraints within the language of FOL. In this paper we
propose a novel class of hybrid SRL methods that rely on Satisfiability Modulo
Theories, an alternative class of for- mal languages that allow to describe,
and reason over, mixed Boolean-numerical objects and constraints. The resulting
methods, which we call Learning Mod- ulo Theories, are formulated within the
structured output SVM framework, and employ a weighted SMT solver as an
optimization oracle to perform efficient in- ference and discriminative max
margin weight learning. We also present a few examples of constructive learning
applications enabled by our method.
Distributional Models and Deep Learning Embeddings: Combining the Best of Both Worlds
Authors: Irina Sergienya, Hinrich Schütze
Date: 2013-12-19 00:00:00
Tasks: Deep Learning

There are two main approaches to the distributed representation of words:
low-dimensional deep learning embeddings and high-dimensional distributional
models, in which each dimension corresponds to a context word. In this paper,
we combine these two approaches by learning embeddings based on
distributional-model vectors - as opposed to one-hot vectors as is standardly
done in deep learning. We show that the combined approach has better
performance on a word relatedness judgment task.