pwc-archive/papers-with-abstracts huggingface dataset:
How Many Topics? Stability Analysis for Topic Models
Authors: Derek Greene, Derek O'Callaghan, Pádraig Cunningham
Date: 2014-04-16 00:00:00
Tasks: Clustering, Diversity, Model Selection, Topic Models

Topic modeling refers to the task of discovering the underlying thematic
structure in a text corpus, where the output is commonly presented as a report
of the top terms appearing in each topic. Despite the diversity of topic
modeling algorithms that have been proposed, a common challenge in successfully
applying these techniques is the selection of an appropriate number of topics
for a given corpus. Choosing too few topics will produce results that are
overly broad, while choosing too many will result in the "over-clustering" of a
corpus into many small, highly-similar topics. In this paper, we propose a
term-centric stability analysis strategy to address this issue, the idea being
that a model with an appropriate number of topics will be more robust to
perturbations in the data. Using a topic modeling approach based on matrix
factorization, evaluations performed on a range of corpora show that this
strategy can successfully guide the model selection process.
Racing Multi-Objective Selection Probabilities
Authors: Gaétan Marceau, Marc Schoenauer
Date: 2014-06-19 00:00:00

In the context of Noisy Multi-Objective Optimization, dealing with
uncertainties requires the decision maker to define some preferences about how
to handle them, through some statistics (e.g., mean, median) to be used to
evaluate the qualities of the solutions, and define the corresponding Pareto
set. Approximating these statistics requires repeated samplings of the
population, drastically increasing the overall computational cost. To tackle
this issue, this paper proposes to directly estimate the probability of each
individual to be selected, using some Hoeffding races to dynamically assign the
estimation budget during the selection step. The proposed racing approach is
validated against static budget approaches with NSGA-II on noisy versions of
the ZDT benchmark functions.
Majority Vote of Diverse Classifiers for Late Fusion
Authors: Emilie Morvant, Amaury Habrard, Stéphane Ayache
Date: 2014-04-30 00:00:00
Tasks: Diversity

In the past few years, a lot of attention has been devoted to multimedia
indexing by fusing multimodal informations. Two kinds of fusion schemes are
generally considered: The early fusion and the late fusion. We focus on late
classifier fusion, where one combines the scores of each modality at the
decision level. To tackle this problem, we investigate a recent and elegant
well-founded quadratic program named MinCq coming from the machine learning
PAC-Bayesian theory. MinCq looks for the weighted combination, over a set of
real-valued functions seen as voters, leading to the lowest misclassification
rate, while maximizing the voters' diversity. We propose an extension of MinCq
tailored to multimedia indexing. Our method is based on an order-preserving
pairwise loss adapted to ranking that allows us to improve Mean Averaged
Precision measure while taking into account the diversity of the voters that we
want to fuse. We provide evidence that this method is naturally adapted to late
fusion procedures and confirm the good behavior of our approach on the
challenging PASCAL VOC'07 benchmark.
On the Application of Generic Summarization Algorithms to Music
Authors: Francisco Raposo, Ricardo Ribeiro, David Martins de Matos
Date: 2014-06-18 00:00:00
Tasks: General Classification

Several generic summarization algorithms were developed in the past and
successfully applied in fields such as text and speech summarization. In this
paper, we review and apply these algorithms to music. To evaluate this
summarization's performance, we adopt an extrinsic approach: we compare a Fado
Genre Classifier's performance using truncated contiguous clips against the
summaries extracted with those algorithms on 2 different datasets. We show that
Maximal Marginal Relevance (MMR), LexRank and Latent Semantic Analysis (LSA)
all improve classification performance in both datasets used for testing.
Improved Densification of One Permutation Hashing
Authors: Anshumali Shrivastava, Ping Li
Date: 2014-06-18 00:00:00

The existing work on densification of one permutation hashing reduces the
query processing cost of the $(K,L)$-parameterized Locality Sensitive Hashing
(LSH) algorithm with minwise hashing, from $O(dKL)$ to merely $O(d + KL)$,
where $d$ is the number of nonzeros of the data vector, $K$ is the number of
hashes in each hash table, and $L$ is the number of hash tables. While that is
a substantial improvement, our analysis reveals that the existing densification
scheme is sub-optimal. In particular, there is no enough randomness in that
procedure, which affects its accuracy on very sparse datasets.
  In this paper, we provide a new densification procedure which is provably
better than the existing scheme. This improvement is more significant for very
sparse datasets which are common over the web. The improved technique has the
same cost of $O(d + KL)$ for query processing, thereby making it strictly
preferable over the existing procedure. Experimental evaluations on public
datasets, in the task of hashing based near neighbor search, support our
theoretical findings.
Predictive Modelling of Bone Age through Classification and Regression of Bone Shapes
Authors: Anthony Bagnall, Luke Davis
Date: 2014-06-18 00:00:00
Tasks: General Classification, regression

Bone age assessment is a task performed daily in hospitals worldwide. This
involves a clinician estimating the age of a patient from a radiograph of the
non-dominant hand.
  Our approach to automated bone age assessment is to modularise the algorithm
into the following three stages: segment and verify hand outline; segment and
verify bones; use the bone outlines to construct models of age. In this paper
we address the final question: given outlines of bones, can we learn how to
predict the bone age of the patient? We examine two alternative approaches.
Firstly, we attempt to train classifiers on individual bones to predict the
bone stage categories commonly used in bone ageing. Secondly, we construct
regression models to directly predict patient age.
  We demonstrate that models built on summary features of the bone outline
perform better than those built using the one dimensional representation of the
outline, and also do at least as well as other automated systems. We show that
models constructed on just three bones are as accurate at predicting age as
expert human assessors using the standard technique. We also demonstrate the
utility of the model by quantifying the importance of ethnicity and sex on age
development. Our conclusion is that the feature based system of separating the
image processing from the age modelling is the best approach for automated bone
ageing, since it offers flexibility and transparency and produces accurate
estimates.
Deep Learning Face Representation by Joint Identification-Verification
Authors: Yi Sun, Xiaogang Wang, Xiaoou Tang
Date: 2014-06-18 00:00:00
Proceeding: NeurIPS 2014 12
Tasks: Deep Learning, Face Identification, Face Recognition, Face Verification

The key challenge of face recognition is to develop effective feature representations for reducing intra-personal variations while enlarging inter-personal differences. In this paper, we show that it can be well solved with deep learning and using both face identification and verification signals as supervision. The Deep IDentification-verification features (DeepID2) are learned with carefully designed deep convolutional networks. The face identification task increases the inter-personal variations by drawing DeepID2 extracted from different identities apart, while the face verification task reduces the intra-personal variations by pulling DeepID2 extracted from the same identity together, both of which are essential to face recognition. The learned DeepID2 features can be well generalized to new identities unseen in the training data. On the challenging LFW dataset, 99.15% face verification accuracy is achieved. Compared with the best deep learning result on LFW, the error rate has been significantly reduced by 67%.
Mass Classification Method in Mammogram Using Fuzzy K-Nearest Neighbour Equality
Authors: I. Laurence Aroquiaraj, K. Thangavel
Date: 2014-06-18 00:00:00
Tasks: Classification, General Classification, Specificity

Mass classification of objects is an important area of research and
application in a variety of fields. In this paper, we present an efficient
computer aided mass classification method in digitized mammograms using Fuzzy
K-Nearest Neighbor Equality, which performs benign or malignant classification
on region of interest that contains mass. One of the major mammographic
characteristics for mass classification is texture. Fuzzy K-Nearest Neighbor
Equality exploits this important factor to classify the mass into benign or
malignant. The statistical textural features used in characterizing the masses
are Haralick and Run length features. The main aim of the method is to increase
the effectiveness and efficiency of the classification process in an objective
manner to reduce the numbers of false positive of malignancies. In this paper
proposes a novel Fuzzy K-Nearest Neighbor Equality algorithm for classifying
the marked regions into benign and malignant and 94.46 sensitivity,96.81
specificity and 96.52 accuracy is achieved that is very much promising compare
to the radiologists' accuracy.
An Experimental Evaluation of Nearest Neighbour Time Series Classification
Authors: Anthony Bagnall, Jason Lines
Date: 2014-06-18 00:00:00
Tasks: Classification, Dynamic Time Warping, General Classification, Time Series, Time Series Analysis, Time Series Classification, valid

Data mining research into time series classification (TSC) has focussed on
alternative distance measures for nearest neighbour classifiers. It is standard
practice to use 1-NN with Euclidean or dynamic time warping (DTW) distance as a
straw man for comparison. As part of a wider investigation into elastic
distance measures for TSC~\cite{lines14elastic}, we perform a series of
experiments to test whether this standard practice is valid.
  Specifically, we compare 1-NN classifiers with Euclidean and DTW distance to
standard classifiers, examine whether the performance of 1-NN Euclidean
approaches that of 1-NN DTW as the number of cases increases, assess whether
there is any benefit of setting $k$ for $k$-NN through cross validation whether
it is worth setting the warping path for DTW through cross validation and
finally is it better to use a window or weighting for DTW. Based on experiments
on 77 problems, we conclude that 1-NN with Euclidean distance is fairly easy to
beat but 1-NN with DTW is not, if window size is set through cross validation.
Typed Hilbert Epsilon Operators and the Semantics of Determiner Phrases (Invited Lecture)
Authors: Christian Retoré
Date: 2014-06-18 00:00:00
Tasks: Sentence

The semantics of determiner phrases, be they definite de- scriptions,
indefinite descriptions or quantified noun phrases, is often as- sumed to be a
fully solved question: common nouns are properties, and determiners are
generalised quantifiers that apply to two predicates: the property
corresponding to the common noun and the one corresponding to the verb phrase.
We first present a criticism of this standard view. Firstly, the semantics of
determiners does not follow the syntactical structure of the sentence. Secondly
the standard interpretation of the indefinite article cannot ac- count for
nominal sentences. Thirdly, the standard view misses the linguis- tic asymmetry
between the two properties of a generalised quantifier. In the sequel, we
propose a treatment of determiners and quantifiers as Hilbert terms in a richly
typed system that we initially developed for lexical semantics, using a many
sorted logic for semantical representations. We present this semantical
framework called the Montagovian generative lexicon and show how these terms
better match the syntactical structure and avoid the aforementioned problems of
the standard approach. Hilbert terms rather differ from choice functions in
that there is one polymorphic operator and not one operator per formula. They
also open an intriguing connection between the logic for meaning assembly, the
typed lambda calculus handling compositionality and the many-sorted logic for
semantical representations. Furthermore epsilon terms naturally introduce
type-judgements and confirm the claim that type judgment are a form of
presupposition.
Self-Learning Camera: Autonomous Adaptation of Object Detectors to Unlabeled Video Streams
Authors: Adrien Gaidon, Gloria Zen, Jose A. Rodriguez-Serrano
Date: 2014-06-17 00:00:00
Tasks: Multi-Task Learning, Object, Self-Learning

Learning object detectors requires massive amounts of labeled training
samples from the specific data source of interest. This is impractical when
dealing with many different sources (e.g., in camera networks), or constantly
changing ones such as mobile cameras (e.g., in robotics or driving assistant
systems). In this paper, we address the problem of self-learning detectors in
an autonomous manner, i.e. (i) detectors continuously updating themselves to
efficiently adapt to streaming data sources (contrary to transductive
algorithms), (ii) without any labeled data strongly related to the target data
stream (contrary to self-paced learning), and (iii) without manual intervention
to set and update hyper-parameters. To that end, we propose an unsupervised,
on-line, and self-tuning learning algorithm to optimize a multi-task learning
convex objective. Our method uses confident but laconic oracles (high-precision
but low-recall off-the-shelf generic detectors), and exploits the structure of
the problem to jointly learn on-line an ensemble of instance-level trackers,
from which we derive an adapted category-level object detector. Our approach is
validated on real-world publicly available video object datasets.
The Frobenius anatomy of word meanings II: possessive relative pronouns
Authors: Mehrnoosh Sadrzadeh, Stephen Clark, Bob Coecke
Date: 2014-06-18 00:00:00
Tasks: Anatomy, Object

Within the categorical compositional distributional model of meaning, we
provide semantic interpretations for the subject and object roles of the
possessive relative pronoun `whose'. This is done in terms of Frobenius
algebras over compact closed categories. These algebras and their diagrammatic
language expose how meanings of words in relative clauses interact with each
other. We show how our interpretation is related to Montague-style semantics
and provide a truth-theoretic interpretation. We also show how vector spaces
provide a concrete interpretation and provide preliminary corpus-based
experimental evidence. In a prequel to this paper, we used similar methods and
dealt with the case of subject and object relative pronouns.
Exact Decoding on Latent Variable Conditional Models is NP-Hard
Authors: Xu Sun
Date: 2014-06-18 00:00:00

Latent variable conditional models, including the latent conditional random
fields as a special case, are popular models for many natural language
processing and vision processing tasks. The computational complexity of the
exact decoding/inference in latent conditional random fields is unclear. In
this paper, we try to clarify the computational complexity of the exact
decoding. We analyze the complexity and demonstrate that it is an NP-hard
problem even on a sequential labeling setting. Furthermore, we propose the
latent-dynamic inference (LDI-Naive) method and its bounded version
(LDI-Bounded), which are able to perform exact-inference or
almost-exact-inference by using top-$n$ search and dynamic programming.
The Propagation Depth of Local Consistency
Authors: Christoph Berkholz
Date: 2014-06-18 00:00:00
Tasks: ARC

We establish optimal bounds on the number of nested propagation steps in
$k$-consistency tests. It is known that local consistency algorithms such as
arc-, path- and $k$-consistency are not efficiently parallelizable. Their
inherent sequential nature is caused by long chains of nested propagation
steps, which cannot be executed in parallel. This motivates the question "What
is the minimum number of nested propagation steps that have to be performed by
$k$-consistency algorithms on (binary) constraint networks with $n$ variables
and domain size $d$?"
  It was known before that 2-consistency requires $\Omega(nd)$ and
3-consistency requires $\Omega(n^2)$ sequential propagation steps. We answer
the question exhaustively for every $k\geq 2$: there are binary constraint
networks where any $k$-consistency procedure has to perform
$\Omega(n^{k-1}d^{k-1})$ nested propagation steps before local inconsistencies
were detected. This bound is tight, because the overall number of propagation
steps performed by $k$-consistency is at most $n^{k-1}d^{k-1}$.
Notes on hierarchical ensemble methods for DAG-structured taxonomies
Authors: Giorgio Valentini
Date: 2014-06-17 00:00:00
Tasks: Classification, General Classification, Hierarchical Multi-label Classification, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION, text-classification, Text Classification

Several real problems ranging from text classification to computational
biology are characterized by hierarchical multi-label classification tasks.
Most of the methods presented in literature focused on tree-structured
taxonomies, but only few on taxonomies structured according to a Directed
Acyclic Graph (DAG). In this contribution novel classification ensemble
algorithms for DAG-structured taxonomies are introduced. In particular
Hierarchical Top-Down (HTD-DAG) and True Path Rule (TPR-DAG) for DAGs are
presented and discussed.
RAPID: Rapidly Accelerated Proximal Gradient Algorithms for Convex Minimization
Authors: Ziming Zhang, Venkatesh Saligrama
Date: 2014-06-13 00:00:00

In this paper, we propose a new algorithm to speed-up the convergence of
accelerated proximal gradient (APG) methods. In order to minimize a convex
function $f(\mathbf{x})$, our algorithm introduces a simple line search step
after each proximal gradient step in APG so that a biconvex function
$f(\theta\mathbf{x})$ is minimized over scalar variable $\theta>0$ while fixing
variable $\mathbf{x}$. We propose two new ways of constructing the auxiliary
variables in APG based on the intermediate solutions of the proximal gradient
and the line search steps. We prove that at arbitrary iteration step $t
(t\geq1)$, our algorithm can achieve a smaller upper-bound for the gap between
the current and optimal objective values than those in the traditional APG
methods such as FISTA, making it converge faster in practice. In fact, our
algorithm can be potentially applied to many important convex optimization
problems, such as sparse linear regression and kernel SVMs. Our experimental
results clearly demonstrate that our algorithm converges faster than APG in all
of the applications above, even comparable to some sophisticated solvers.
A Sober Look at Spectral Learning
Authors: Han Zhao, Pascal Poupart
Date: 2014-06-18 00:00:00

Spectral learning recently generated lots of excitement in machine learning,
largely because it is the first known method to produce consistent estimates
(under suitable conditions) for several latent variable models. In contrast,
maximum likelihood estimates may get trapped in local optima due to the
non-convex nature of the likelihood function of latent variable models. In this
paper, we do an empirical evaluation of spectral learning (SL) and expectation
maximization (EM), which reveals an important gap between the theory and the
practice. First, SL often leads to negative probabilities. Second, EM often
yields better estimates than spectral learning and it does not seem to get
stuck in local optima. We discuss how the rank of the model parameters and the
amount of training data can yield negative probabilities. We also question the
common belief that maximum likelihood estimators are necessarily inconsistent.
A Generalized Markov-Chain Modelling Approach to $(1,λ)$-ES Linear Optimization: Technical Report
Authors: Alexandre Chotard, Martin Holena
Date: 2014-06-18 00:00:00

Several recent publications investigated Markov-chain modelling of linear
optimization by a $(1,\lambda)$-ES, considering both unconstrained and linearly
constrained optimization, and both constant and varying step size. All of them
assume normality of the involved random steps, and while this is consistent
with a black-box scenario, information on the function to be optimized (e.g.
separability) may be exploited by the use of another distribution. The
objective of our contribution is to complement previous studies realized with
normal steps, and to give sufficient conditions on the distribution of the
random steps for the success of a constant step-size $(1,\lambda)$-ES on the
simple problem of a linear function with a linear constraint. The decomposition
of a multidimensional distribution into its marginals and the copula combining
them is applied to the new distributional assumptions, particular attention
being paid to distributions with Archimedean copulas.
Primitives for Dynamic Big Model Parallelism
Authors: Seunghak Lee, Jin Kyu Kim, Xun Zheng, Qirong Ho, Garth A. Gibson, Eric P. Xing
Date: 2014-06-18 00:00:00
Tasks: model, Scheduling

When training large machine learning models with many variables or
parameters, a single machine is often inadequate since the model may be too
large to fit in memory, while training can take a long time even with
stochastic updates. A natural recourse is to turn to distributed cluster
computing, in order to harness additional memory and processors. However,
naive, unstructured parallelization of ML algorithms can make inefficient use
of distributed memory, while failing to obtain proportional convergence
speedups - or can even result in divergence. We develop a framework of
primitives for dynamic model-parallelism, STRADS, in order to explore
partitioning and update scheduling of model variables in distributed ML
algorithms - thus improving their memory efficiency while presenting new
opportunities to speed up convergence without compromising inference
correctness. We demonstrate the efficacy of model-parallel algorithms
implemented in STRADS versus popular implementations for Topic Modeling, Matrix
Factorization and Lasso.
DFacTo: Distributed Factorization of Tensors
Authors: Joon Hee Choi, S. V. N. Vishwanathan
Date: 2014-06-17 00:00:00
Proceeding: NeurIPS 2014 12

We present a technique for significantly speeding up Alternating Least
Squares (ALS) and Gradient Descent (GD), two widely used algorithms for tensor
factorization. By exploiting properties of the Khatri-Rao product, we show how
to efficiently address a computationally challenging sub-step of both
algorithms. Our algorithm, DFacTo, only requires two sparse matrix-vector
products and is easy to parallelize. DFacTo is not only scalable but also on
average 4 to 10 times faster than competing algorithms on a variety of
datasets. For instance, DFacTo only takes 480 seconds on 4 machines to perform
one iteration of the ALS algorithm and 1,143 seconds to perform one iteration
of the GD algorithm on a 6.5 million x 2.5 million x 1.5 million dimensional
tensor with 1.2 billion non-zero entries.