pwc-archive/papers-with-abstracts huggingface dataset:
Scaling-up Empirical Risk Minimization: Optimization of Incomplete U-statistics
Authors: Stéphan Clémençon, Aurélien Bellet, Igor Colin
Date: 2015-01-12 00:00:00
Tasks: Clustering, Metric Learning, Model Selection

In a wide range of statistical learning problems such as ranking, clustering
or metric learning among others, the risk is accurately estimated by
$U$-statistics of degree $d\geq 1$, i.e. functionals of the training data with
low variance that take the form of averages over $k$-tuples. From a
computational perspective, the calculation of such statistics is highly
expensive even for a moderate sample size $n$, as it requires averaging
$O(n^d)$ terms. This makes learning procedures relying on the optimization of
such data functionals hardly feasible in practice. It is the major goal of this
paper to show that, strikingly, such empirical risks can be replaced by
drastically computationally simpler Monte-Carlo estimates based on $O(n)$ terms
only, usually referred to as incomplete $U$-statistics, without damaging the
$O_{\mathbb{P}}(1/\sqrt{n})$ learning rate of Empirical Risk Minimization (ERM)
procedures. For this purpose, we establish uniform deviation results describing
the error made when approximating a $U$-process by its incomplete version under
appropriate complexity assumptions. Extensions to model selection, fast rate
situations and various sampling techniques are also considered, as well as an
application to stochastic gradient descent for ERM. Finally, numerical examples
are displayed in order to provide strong empirical evidence that the approach
we promote largely surpasses more naive subsampling techniques.
Improving Raw Image Storage Efficiency by Exploiting Similarity
Authors: Binqi Zhang, Chen Wang, Bing Bing Zhou, Albert Y. Zomaya
Date: 2016-04-19 00:00:00
Tasks: Retrieval

To improve the temporal and spatial storage efficiency, researchers have
intensively studied various techniques, including compression and
deduplication. Through our evaluation, we find that methods such as photo tags
or local features help to identify the content-based similar- ity between raw
images. The images can then be com- pressed more efficiently to get better
storage space sav- ings. Furthermore, storing similar raw images together
enables rapid data sorting, searching and retrieval if the images are stored in
a distributed and large-scale envi- ronment by reducing fragmentation. In this
paper, we evaluated the compressibility by designing experiments and observing
the results. We found that on a statistical basis the higher similarity photos
have, the better com- pression results are. This research helps provide a clue
for future large-scale storage system design.
Comparative Study of Instance Based Learning and Back Propagation for Classification Problems
Authors: Nadia Kanwal, Erkan Bostanci
Date: 2016-04-19 00:00:00
Tasks: General Classification, Imputation, Missing Values

The paper presents a comparative study of the performance of Back Propagation
and Instance Based Learning Algorithm for classification tasks. The study is
carried out by a series of experiments will all possible combinations of
parameter values for the algorithms under evaluation. The algorithm's
classification accuracy is compared over a range of datasets and measurements
like Cross Validation, Kappa Statistics, Root Mean Squared Value and True
Positive vs False Positive rate have been used to evaluate their performance.
Along with performance comparison, techniques of handling missing values have
also been compared that include Mean or Mode replacement and Multiple
Imputation. The results showed that parameter adjustment plays vital role in
improving an algorithm's accuracy and therefore, Back Propagation has shown
better results as compared to Instance Based Learning. Furthermore, the problem
of missing values was better handled by Multiple imputation method, however,
not suitable for less amount of data.
Extending the Harper Identity to Iterated Belief Change
Authors: Jake Chandler, Richard Booth
Date: 2016-04-19 00:00:00

The field of iterated belief change has focused mainly on revision, with the
other main operator of AGM belief change theory, i.e. contraction, receiving
relatively little attention. In this paper we extend the Harper Identity from
single-step change to define iterated contraction in terms of iterated
revision. Specifically, just as the Harper Identity provides a recipe for
defining the belief set resulting from contracting A in terms of (i) the
initial belief set and (ii) the belief set resulting from revision by not-A, we
look at ways to define the plausibility ordering over worlds resulting from
contracting A in terms of (iii) the initial plausibility ordering, and (iv) the
plausibility ordering resulting from revision by not-A. After noting that the
most straightforward such extension leads to a trivialisation of the space of
permissible orderings, we provide a family of operators for combining
plausibility orderings that avoid such a result. These operators are
characterised in our domain of interest by a pair of intuitively compelling
properties, which turn out to enable the derivation of a number of iterated
contraction postulates from postulates for iterated revision. We finish by
observing that a salient member of this family allows for the derivation of
counterparts for contraction of some well known iterated revision operators, as
well as for defining new iterated contraction operators.
Comparing Face Detection and Recognition Techniques
Authors: Jyothi Korra
Date: 2016-04-19 00:00:00
Tasks: Face Detection, Face Recognition

This paper implements and compares different techniques for face detection
and recognition. One is find where the face is located in the images that is
face detection and second is face recognition that is identifying the person.
We study three techniques in this paper: Face detection using self organizing
map (SOM), Face recognition by projection and nearest neighbor and Face
recognition using SVM.
Multi-Source Multi-View Clustering via Discrepancy Penalty
Authors: Weixiang Shao, Jiawei Zhang, Lifang He, Philip S. Yu
Date: 2016-04-14 00:00:00
Tasks: Clustering

With the advance of technology, entities can be observed in multiple views.
Multiple views containing different types of features can be used for
clustering. Although multi-view clustering has been successfully applied in
many applications, the previous methods usually assume the complete instance
mapping between different views. In many real-world applications, information
can be gathered from multiple sources, while each source can contain multiple
views, which are more cohesive for learning. The views under the same source
are usually fully mapped, but they can be very heterogeneous. Moreover, the
mappings between different sources are usually incomplete and partially
observed, which makes it more difficult to integrate all the views across
different sources. In this paper, we propose MMC (Multi-source Multi-view
Clustering), which is a framework based on collective spectral clustering with
a discrepancy penalty across sources, to tackle these challenges. MMC has
several advantages compared with other existing methods. First, MMC can deal
with incomplete mapping between sources. Second, it considers the disagreements
between sources while treating views in the same source as a cohesive set.
Third, MMC also tries to infer the instance similarities across sources to
enhance the clustering performance. Extensive experiments conducted on
real-world data demonstrate the effectiveness of the proposed approach.
Cognitive state classification using transformed fMRI data
Authors: Hariharan Ramasangu, Neelam Sinha
Date: 2016-04-19 00:00:00
Tasks: Classification, General Classification, Sentence, Time Series Analysis

One approach, for understanding human brain functioning, is to analyze the
changes in the brain while performing cognitive tasks. Towards this, Functional
Magnetic Resonance (fMR) images of subjects performing well-defined tasks are
widely utilized for task-specific analyses. In this work, we propose a
procedure to enable classification between two chosen cognitive tasks, using
their respective fMR image sequences. The time series of expert-marked
anatomically-mapped relevant voxels are processed and fed as input to the
classical Naive Bayesian and SVM classifiers. The processing involves use of
random sieve function, phase information in the data transformed using Fourier
and Hilbert transformations. This processing results in improved
classification, as against using the voxel intensities directly, as
illustrated. The novelty of the proposed method lies in utilizing the phase
information in the transformed domain, for classifying between the cognitive
tasks along with random sieve function chosen with a particular probability
distribution. The proposed classification procedure is applied on a publicly
available dataset, StarPlus data, with 6 subjects performing the two distinct
cognitive tasks of watching either a picture or a sentence. The classification
accuracy stands at an average of 65.6%(using Naive Bayes classifier) and
76.4%(using SVM classifier) for raw data. The corresponding classification
accuracy stands at 96.8% and 97.5% for Fourier transformed data. For Hilbert
transformed data, it is 93.7% and 99%, for 6 subjects, on 2 cognitive tasks.
A Critical Connectivity Radius for Segmenting Randomly-Generated, High Dimensional Data Points
Authors: Robert A. Murphy
Date: 2016-02-11 00:00:00
Tasks: Edge Detection, Image Segmentation, Semantic Segmentation, Unsupervised Image Segmentation

Motivated by a $2$-dimensional (unsupervised) image segmentation task whereby local regions of pixels are clustered via edge detection methods, a more general probabilistic mathematical framework is devised. Critical thresholds are calculated that indicate strong correlation between randomly-generated, high dimensional data points that have been projected into structures in a partition of a bounded, $2$-dimensional area, of which, an image is a special case. A neighbor concept for structures in the partition is defined and a critical radius is uncovered. Measured from a central structure in localized regions of the partition, the radius indicates strong, long and short range correlation in the count of occupied structures. The size of a short interval of radii is estimated upon which the transition from short-to-long range correlation is virtually assured, which defines a demarcation of when an image ceases to be "interesting".
End-to-End Training of Deep Visuomotor Policies
Authors: Sergey Levine, Chelsea Finn, Trevor Darrell, Pieter Abbeel
Date: 2015-04-02 00:00:00
Tasks: Reinforcement Learning, State Estimation

Policy search methods can allow robots to learn control policies for a wide
range of tasks, but practical applications of policy search often require
hand-engineered components for perception, state estimation, and low-level
control. In this paper, we aim to answer the following question: does training
the perception and control systems jointly end-to-end provide better
performance than training each component separately? To this end, we develop a
method that can be used to learn policies that map raw image observations
directly to torques at the robot's motors. The policies are represented by deep
convolutional neural networks (CNNs) with 92,000 parameters, and are trained
using a partially observed guided policy search method, which transforms policy
search into supervised learning, with supervision provided by a simple
trajectory-centric reinforcement learning method. We evaluate our method on a
range of real-world manipulation tasks that require close coordination between
vision and control, such as screwing a cap onto a bottle, and present simulated
comparisons to a range of prior policy search methods.
An Adaptive Learning Mechanism for Selection of Increasingly More Complex Systems
Authors: Fouad Khan
Date: 2016-04-19 00:00:00

Recently it has been demonstrated that causal entropic forces can lead to the
emergence of complex phenomena associated with human cognitive niche such as
tool use and social cooperation. Here I show that even more fundamental traits
associated with human cognition such as 'self-awareness' can easily be
demonstrated to be arising out of merely a selection for 'better regulators';
i.e. systems which respond comparatively better to threats to their existence
which are internal to themselves. A simple model demonstrates how indeed the
average self-awareness for a universe of systems continues to rise as less
self-aware systems are eliminated. The model also demonstrates however that the
maximum attainable self-awareness for any system is limited by the plasticity
and energy availability for that typology of systems. I argue that this rise in
self-awareness may be the reason why systems tend towards greater complexity.
Learning Dense Correspondence via 3D-guided Cycle Consistency
Authors: Tinghui Zhou, Philipp Krähenbühl, Mathieu Aubry, Qi-Xing Huang, Alexei A. Efros
Date: 2016-04-18 00:00:00
Proceeding: CVPR 2016 6

Discriminative deep learning approaches have shown impressive results for
problems where human-labeled ground truth is plentiful, but what about tasks
where labels are difficult or impossible to obtain? This paper tackles one such
problem: establishing dense visual correspondence across different object
instances. For this task, although we do not know what the ground-truth is, we
know it should be consistent across instances of that category. We exploit this
consistency as a supervisory signal to train a convolutional neural network to
predict cross-instance correspondences between pairs of images depicting
objects of the same category. For each pair of training images we find an
appropriate 3D CAD model and render two synthetic views to link in with the
pair, establishing a correspondence flow 4-cycle. We use ground-truth
synthetic-to-synthetic correspondences, provided by the rendering engine, to
train a ConvNet to predict synthetic-to-real, real-to-real and
real-to-synthetic correspondences that are cycle-consistent with the
ground-truth. At test time, no CAD models are required. We demonstrate that our
end-to-end trained ConvNet supervised by cycle-consistency outperforms
state-of-the-art pairwise matching methods in correspondence-related tasks.
Churn analysis using deep convolutional neural networks and autoencoders
Authors: Artit Wangperawong, Cyrille Brun, Olav Laudy, Rujikorn Pavasuthipaisit
Date: 2016-04-18 00:00:00
Tasks: General Classification, image-classification, Image Classification

Customer temporal behavioral data was represented as images in order to
perform churn prediction by leveraging deep learning architectures prominent in
image classification. Supervised learning was performed on labeled data of over
6 million customers using deep convolutional neural networks, which achieved an
AUC of 0.743 on the test dataset using no more than 12 temporal features for
each customer. Unsupervised learning was conducted using autoencoders to better
understand the reasons for customer churn. Images that maximally activate the
hidden units of an autoencoder trained with churned customers reveal ample
opportunities for action to be taken to prevent churn among strong data, no
voice users.
Clustering Comparable Corpora of Russian and Ukrainian Academic Texts: Word Embeddings and Semantic Fingerprints
Authors: Andrey Kutuzov, Mikhail Kopotev, Tatyana Sviridenko, Lyubov Ivanova
Date: 2016-04-18 00:00:00
Tasks: Clustering, Translation, Word Embeddings

We present our experience in applying distributional semantics (neural word
embeddings) to the problem of representing and clustering documents in a
bilingual comparable corpus. Our data is a collection of Russian and Ukrainian
academic texts, for which topics are their academic fields. In order to build
language-independent semantic representations of these documents, we train
neural distributional models on monolingual corpora and learn the optimal
linear transformation of vectors from one language to another. The resulting
vectors are then used to produce `semantic fingerprints' of documents, serving
as input to a clustering algorithm.
  The presented method is compared to several baselines including `orthographic
translation' with Levenshtein edit distance and outperforms them by a large
margin. We also show that language-independent `semantic fingerprints' are
superior to multi-lingual clustering algorithms proposed in the previous work,
at the same time requiring less linguistic resources.
Text-based LSTM networks for Automatic Music Composition
Authors: Keunwoo Choi, George Fazekas, Mark Sandler
Date: 2016-04-18 00:00:00
Tasks: Diversity

In this paper, we introduce new methods and discuss results of text-based
LSTM (Long Short-Term Memory) networks for automatic music composition. The
proposed network is designed to learn relationships within text documents that
represent chord progressions and drum tracks in two case studies. In the
experiments, word-RNNs (Recurrent Neural Networks) show good results for both
cases, while character-based RNNs (char-RNNs) only succeed to learn chord
progressions. The proposed system can be used for fully automatic composition
or as semi-automatic systems that help humans to compose music by controlling a
diversity parameter of the model.
Time-Sensitive Bayesian Information Aggregation for Crowdsourcing Systems
Authors: Matteo Venanzi, John Guiver, Pushmeet Kohli, Nick Jennings
Date: 2015-10-21 00:00:00
Tasks: Bayesian Inference, Entity Linking, valid

Crowdsourcing systems commonly face the problem of aggregating multiple
judgments provided by potentially unreliable workers. In addition, several
aspects of the design of efficient crowdsourcing processes, such as defining
worker's bonuses, fair prices and time limits of the tasks, involve knowledge
of the likely duration of the task at hand. Bringing this together, in this
work we introduce a new time--sensitive Bayesian aggregation method that
simultaneously estimates a task's duration and obtains reliable aggregations of
crowdsourced judgments. Our method, called BCCTime, builds on the key insight
that the time taken by a worker to perform a task is an important indicator of
the likely quality of the produced judgment. To capture this, BCCTime uses
latent variables to represent the uncertainty about the workers' completion
time, the tasks' duration and the workers' accuracy. To relate the quality of a
judgment to the time a worker spends on a task, our model assumes that each
task is completed within a latent time window within which all workers with a
propensity to genuinely attempt the labelling task (i.e., no spammers) are
expected to submit their judgments. In contrast, workers with a lower
propensity to valid labeling, such as spammers, bots or lazy labelers, are
assumed to perform tasks considerably faster or slower than the time required
by normal workers. Specifically, we use efficient message-passing Bayesian
inference to learn approximate posterior probabilities of (i) the confusion
matrix of each worker, (ii) the propensity to valid labeling of each worker,
(iii) the unbiased duration of each task and (iv) the true label of each task.
Using two real-world public datasets for entity linking tasks, we show that
BCCTime produces up to 11% more accurate classifications and up to 100% more
informative estimates of a task's duration compared to state-of-the-art
methods.
Efficient Calculation of Bigram Frequencies in a Corpus of Short Texts
Authors: Melvyn Drag, Gauthaman Vasudevan
Date: 2016-04-18 00:00:00

We show that an efficient and popular method for calculating bigram
frequencies is unsuitable for bodies of short texts and offer a simple
alternative. Our method has the same computational complexity as the old method
and offers an exact count instead of an approximation.
Learning Possibilistic Logic Theories from Default Rules
Authors: Ondrej Kuzelka, Jesse Davis, Steven Schockaert
Date: 2016-04-18 00:00:00
Tasks: Learning Theory

We introduce a setting for learning possibilistic logic theories from
defaults of the form "if alpha then typically beta". We first analyse this
problem from the point of view of machine learning theory, determining the VC
dimension of possibilistic stratifications as well as the complexity of the
associated learning problems, after which we present a heuristic learning
algorithm that can easily scale to thousands of defaults. An important property
of our approach is that it is inherently able to handle noisy and conflicting
sets of defaults. Among others, this allows us to learn possibilistic logic
theories from crowdsourced data and to approximate propositional Markov logic
networks using heuristic MAP solvers. We present experimental results that
demonstrate the effectiveness of this approach.
Chained Gaussian Processes
Authors: Alan D. Saul, James Hensman, Aki Vehtari, Neil D. Lawrence
Date: 2016-04-18 00:00:00
Tasks: Additive models, Gaussian Processes

Gaussian process models are flexible, Bayesian non-parametric approaches to
regression. Properties of multivariate Gaussians mean that they can be combined
linearly in the manner of additive models and via a link function (like in
generalized linear models) to handle non-Gaussian data. However, the link
function formalism is restrictive, link functions are always invertible and
must convert a parameter of interest to a linear combination of the underlying
processes. There are many likelihoods and models where a non-linear combination
is more appropriate. We term these more general models Chained Gaussian
Processes: the transformation of the GPs to the likelihood parameters will not
generally be invertible, and that implies that linearisation would only be
possible with multiple (localized) links, i.e. a chain. We develop an
approximate inference procedure for Chained GPs that is scalable and applicable
to any factorized likelihood. We demonstrate the approximation on a range of
likelihood functions.
Kernel Distribution Embeddings: Universal Kernels, Characteristic Kernels and Kernel Metrics on Distributions
Authors: Carl-Johann Simon-Gabriel, Bernhard Schölkopf
Date: 2016-04-18 00:00:00
Tasks: BIG-bench Machine Learning

Kernel mean embeddings have recently attracted the attention of the machine learning community. They map measures $\mu$ from some set $M$ to functions in a reproducing kernel Hilbert space (RKHS) with kernel $k$. The RKHS distance of two mapped measures is a semi-metric $d_k$ over $M$. We study three questions. (I) For a given kernel, what sets $M$ can be embedded? (II) When is the embedding injective over $M$ (in which case $d_k$ is a metric)? (III) How does the $d_k$-induced topology compare to other topologies on $M$? The existing machine learning literature has addressed these questions in cases where $M$ is (a subset of) the finite regular Borel measures. We unify, improve and generalise those results. Our approach naturally leads to continuous and possibly even injective embeddings of (Schwartz-) distributions, i.e., generalised measures, but the reader is free to focus on measures only. In particular, we systemise and extend various (partly known) equivalences between different notions of universal, characteristic and strictly positive definite kernels, and show that on an underlying locally compact Hausdorff space, $d_k$ metrises the weak convergence of probability measures if and only if $k$ is continuous and characteristic.