pwc-archive/papers-with-abstracts huggingface dataset:
Acceleration of Deep Neural Network Training with Resistive Cross-Point Devices
Authors: Tayfun Gokmen, Yurii Vlasov
Date: 2016-03-23 00:00:00
Tasks: Data Integration, object-detection, Object Detection, speech-recognition, Speech Recognition

In recent years, deep neural networks (DNN) have demonstrated significant
business impact in large scale analysis and classification tasks such as speech
recognition, visual object detection, pattern extraction, etc. Training of
large DNNs, however, is universally considered as time consuming and
computationally intensive task that demands datacenter-scale computational
resources recruited for many days. Here we propose a concept of resistive
processing unit (RPU) devices that can potentially accelerate DNN training by
orders of magnitude while using much less power. The proposed RPU device can
store and update the weight values locally thus minimizing data movement during
training and allowing to fully exploit the locality and the parallelism of the
training algorithm. We identify the RPU device and system specifications for
implementation of an accelerator chip for DNN training in a realistic
CMOS-compatible technology. For large DNNs with about 1 billion weights this
massively parallel RPU architecture can achieve acceleration factors of 30,000X
compared to state-of-the-art microprocessors while providing power efficiency
of 84,000 GigaOps/s/W. Problems that currently require days of training on a
datacenter-size cluster with thousands of machines can be addressed within
hours on a single RPU accelerator. A system consisted of a cluster of RPU
accelerators will be able to tackle Big Data problems with trillions of
parameters that is impossible to address today like, for example, natural
speech recognition and translation between all world languages, real-time
analytics on large streams of business and scientific data, integration and
analysis of multimodal sensory data flows from massive number of IoT (Internet
of Things) sensors.
CONDITOR1: Topic Maps and DITA labelling tool for textual documents with historical information
Authors: Piedad Garrido, Jesus Tramullas, Manuel Coll
Date: 2016-03-23 00:00:00
Tasks: Information Retrieval, Recommendation Systems, Retrieval

Conditor is a software tool which works with textual documents containing
historical information. The purpose of this work two-fold: firstly to show the
validity of the developed engine to correctly identify and label the entities
of the universe of discourse with a labelled-combined XTM-DITA model. Secondly
to explain the improvements achieved in the information retrieval process
thanks to the use of a object-oriented database (JPOX) as well as its
integration into the Lucene-type database search process to not only accomplish
more accurate searches, but to also help the future development of a
recommender system. We finish with a brief demo in a 3D-graph of the results of
the aforementioned search.
Debugging Machine Learning Tasks
Authors: Aleksandar Chakarov, Aditya Nori, Sriram Rajamani, Shayak Sen, Deepak Vijaykeerthy
Date: 2016-03-23 00:00:00
Tasks: BIG-bench Machine Learning

Unlike traditional programs (such as operating systems or word processors)
which have large amounts of code, machine learning tasks use programs with
relatively small amounts of code (written in machine learning libraries), but
voluminous amounts of data. Just like developers of traditional programs debug
errors in their code, developers of machine learning tasks debug and fix errors
in their data. However, algorithms and tools for debugging and fixing errors in
data are less common, when compared to their counterparts for detecting and
fixing errors in code. In this paper, we consider classification tasks where
errors in training data lead to misclassifications in test points, and propose
an automated method to find the root causes of such misclassifications. Our
root cause analysis is based on Pearl's theory of causation, and uses Pearl's
PS (Probability of Sufficiency) as a scoring metric. Our implementation, Psi,
encodes the computation of PS as a probabilistic program, and uses recent work
on probabilistic programs and transformations on probabilistic programs (along
with gray-box models of machine learning algorithms) to efficiently compute PS.
Psi is able to identify root causes of data errors in interesting data sets.
Fast moment estimation for generalized latent Dirichlet models
Authors: Shiwen Zhao, Barbara E. Engelhardt, Sayan Mukherjee, David B. Dunson
Date: 2016-03-17 00:00:00
Tasks: parameter estimation, Variational Inference

We develop a generalized method of moments (GMM) approach for fast parameter
estimation in a new class of Dirichlet latent variable models with mixed data
types. Parameter estimation via GMM has been demonstrated to have computational
and statistical advantages over alternative methods, such as expectation
maximization, variational inference, and Markov chain Monte Carlo. The key
computational advan- tage of our method (MELD) is that parameter estimation
does not require instantiation of the latent variables. Moreover, a
representational advantage of the GMM approach is that the behavior of the
model is agnostic to distributional assumptions of the observations. We derive
population moment conditions after marginalizing out the sample-specific
Dirichlet latent variables. The moment conditions only depend on component mean
parameters. We illustrate the utility of our approach on simulated data,
comparing results from MELD to alternative methods, and we show the promise of
our approach through the application of MELD to several data sets.
Nuclear norm penalization and optimal rates for noisy low rank matrix completion
Authors: Vladimir Koltchinskii, Alexandre B. Tsybakov, Karim Lounici
Date: 2010-11-29 00:00:00
Tasks: Low-Rank Matrix Completion, Matrix Completion, regression, valid

This paper deals with the trace regression model where $n$ entries or linear
combinations of entries of an unknown $m_1\times m_2$ matrix $A_0$ corrupted by
noise are observed. We propose a new nuclear norm penalized estimator of $A_0$
and establish a general sharp oracle inequality for this estimator for
arbitrary values of $n,m_1,m_2$ under the condition of isometry in expectation.
Then this method is applied to the matrix completion problem. In this case, the
estimator admits a simple explicit form and we prove that it satisfies oracle
inequalities with faster rates of convergence than in the previous works. They
are valid, in particular, in the high-dimensional setting $m_1m_2\gg n$. We
show that the obtained rates are optimal up to logarithmic factors in a minimax
sense and also derive, for any fixed matrix $A_0$, a non-minimax lower bound on
the rate of convergence of our estimator, which coincides with the upper bound
up to a constant factor. Finally, we show that our procedure provides an exact
recovery of the rank of $A_0$ with probability close to 1. We also discuss the
statistical learning setting where there is no underlying model determined by
$A_0$ and the aim is to find the best trace regression model approximating the
data.
Gaussian Process Morphable Models
Authors: Marcel Lüthi, Christoph Jud, Thomas Gerig, Thomas Vetter
Date: 2016-03-23 00:00:00
Tasks: Gaussian Processes

Statistical shape models (SSMs) represent a class of shapes as a normal
distribution of point variations, whose parameters are estimated from example
shapes. Principal component analysis (PCA) is applied to obtain a
low-dimensional representation of the shape variation in terms of the leading
principal components. In this paper, we propose a generalization of SSMs,
called Gaussian Process Morphable Models (GPMMs). We model the shape variations
with a Gaussian process, which we represent using the leading components of its
Karhunen-Loeve expansion. To compute the expansion, we make use of an
approximation scheme based on the Nystrom method. The resulting model can be
seen as a continuous analogon of an SSM. However, while for SSMs the shape
variation is restricted to the span of the example data, with GPMMs we can
define the shape variation using any Gaussian process. For example, we can
build shape models that correspond to classical spline models, and thus do not
require any example data. Furthermore, Gaussian processes make it possible to
combine different models. For example, an SSM can be extended with a spline
model, to obtain a model that incorporates learned shape characteristics, but
is flexible enough to explain shapes that cannot be represented by the SSM. We
introduce a simple algorithm for fitting a GPMM to a surface or image. This
results in a non-rigid registration approach, whose regularization properties
are defined by a GPMM. We show how we can obtain different registration
schemes,including methods for multi-scale, spatially-varying or hybrid
registration, by constructing an appropriate GPMM. As our approach strictly
separates modelling from the fitting process, this is all achieved without
changes to the fitting algorithm. We show the applicability and versatility of
GPMMs on a clinical use case, where the goal is the model-based segmentation of
3D forearm images.
A Tutorial on Deep Neural Networks for Intelligent Systems
Authors: Juan C. Cuevas-Tello, Manuel Valenzuela-Rendon, Juan A. Nolazco-Flores
Date: 2016-03-23 00:00:00
Tasks: General Classification, speech-recognition, Speech Recognition

Developing Intelligent Systems involves artificial intelligence approaches
including artificial neural networks. Here, we present a tutorial of Deep
Neural Networks (DNNs), and some insights about the origin of the term "deep";
references to deep learning are also given. Restricted Boltzmann Machines,
which are the core of DNNs, are discussed in detail. An example of a simple
two-layer network, performing unsupervised learning for unlabeled data, is
shown. Deep Belief Networks (DBNs), which are used to build networks with more
than two layers, are also described. Moreover, examples for supervised learning
with DNNs performing simple prediction and classification tasks, are presented
and explained. This tutorial includes two intelligent pattern recognition
applications: hand- written digits (benchmark known as MNIST) and speech
recognition.
Lightweight Unsupervised Domain Adaptation by Convolutional Filter Reconstruction
Authors: Rahaf Aljundi, Tinne Tuytelaars
Date: 2016-03-23 00:00:00
Tasks: Domain Adaptation, Unsupervised Domain Adaptation

End-to-end learning methods have achieved impressive results in many areas of
computer vision. At the same time, these methods still suffer from a
degradation in performance when testing on new datasets that stem from a
different distribution. This is known as the domain shift effect. Recently
proposed adaptation methods focus on retraining the network parameters.
However, this requires access to all (labeled) source data, a large amount of
(unlabeled) target data, and plenty of computational resources. In this work,
we propose a lightweight alternative, that allows adapting to the target domain
based on a limited number of target samples in a matter of minutes rather than
hours, days or even weeks. To this end, we first analyze the output of each
convolutional layer from a domain adaptation perspective. Surprisingly, we find
that already at the very first layer, domain shift effects pop up. We then
propose a new domain adaptation method, where first layer convolutional filters
that are badly affected by the domain shift are reconstructed based on less
affected ones. This improves the performance of the deep network on various
benchmark datasets.
Scalable Metric Learning via Weighted Approximate Rank Component Analysis
Authors: Cijo Jose, Francois Fleuret
Date: 2016-03-01 00:00:00
Tasks: Metric Learning, Person Re-Identification, Stochastic Optimization

We are interested in the large-scale learning of Mahalanobis distances, with
a particular focus on person re-identification.
  We propose a metric learning formulation called Weighted Approximate Rank
Component Analysis (WARCA). WARCA optimizes the precision at top ranks by
combining the WARP loss with a regularizer that favors orthonormal linear
mappings, and avoids rank-deficient embeddings. Using this new regularizer
allows us to adapt the large-scale WSABIE procedure and to leverage the Adam
stochastic optimization algorithm, which results in an algorithm that scales
gracefully to very large data-sets. Also, we derive a kernelized version which
allows to take advantage of state-of-the-art features for re-identification
when data-set size permits kernel computation.
  Benchmarks on recent and standard re-identification data-sets show that our
method beats existing state-of-the-art techniques both in term of accuracy and
speed. We also provide experimental analysis to shade lights on the properties
of the regularizer we use, and how it improves performance.
A Decentralized Quasi-Newton Method for Dual Formulations of Consensus Optimization
Authors: Mark Eisen, Aryan Mokhtari, Alejandro Ribeiro
Date: 2016-03-23 00:00:00
Tasks: Second-order methods

This paper considers consensus optimization problems where each node of a
network has access to a different summand of an aggregate cost function. Nodes
try to minimize the aggregate cost function, while they exchange information
only with their neighbors. We modify the dual decomposition method to
incorporate a curvature correction inspired by the
Broyden-Fletcher-Goldfarb-Shanno (BFGS) quasi-Newton method. The resulting dual
D-BFGS method is a fully decentralized algorithm in which nodes approximate
curvature information of themselves and their neighbors through the
satisfaction of a secant condition. Dual D-BFGS is of interest in consensus
optimization problems that are not well conditioned, making first order
decentralized methods ineffective, and in which second order information is not
readily available, making decentralized second order methods infeasible.
Asynchronous implementation is discussed and convergence of D-BFGS is
established formally for both synchronous and asynchronous implementations.
Performance advantages relative to alternative decentralized algorithms are
shown numerically.
Enabling Cognitive Intelligence Queries in Relational Databases using Low-dimensional Word Embeddings
Authors: Rajesh Bordawekar, Oded Shmueli
Date: 2016-03-23 00:00:00
Tasks: Word Embeddings

We apply distributed language embedding methods from Natural Language
Processing to assign a vector to each database entity associated token (for
example, a token may be a word occurring in a table row, or the name of a
column). These vectors, of typical dimension 200, capture the meaning of tokens
based on the contexts in which the tokens appear together. To form vectors, we
apply a learning method to a token sequence derived from the database. We
describe various techniques for extracting token sequences from a database. The
techniques differ in complexity, in the token sequences they output and in the
database information used (e.g., foreign keys). The vectors can be used to
algebraically quantify semantic relationships between the tokens such as
similarities and analogies. Vectors enable a dual view of the data: relational
and (meaningful rather than purely syntactical) text. We introduce and explore
a new class of queries called cognitive intelligence (CI) queries that extract
information from the database based, in part, on the relationships encoded by
vectors. We have implemented a prototype system on top of Spark to exhibit the
power of CI queries. Here, CI queries are realized via SQL UDFs. This power
goes far beyond text extensions to relational systems due to the information
encoded in vectors. We also consider various extensions to the basic scheme,
including using a collection of views derived from the database to focus on a
domain of interest, utilizing vectors and/or text from external sources,
maintaining vectors as the database evolves and exploring a database without
utilizing its schema. For the latter, we consider minimal extensions to SQL to
vastly improve query expressiveness.
The Anatomy of a Search and Mining System for Digital Archives
Authors: Martyn Harris, Mark Levene, Dell Zhang, Dan Levene
Date: 2016-03-23 00:00:00
Tasks: Anatomy, Language Modeling, Language Modelling, Retrieval

Samtla (Search And Mining Tools with Linguistic Analysis) is a digital
humanities system designed in collaboration with historians and linguists to
assist them with their research work in quantifying the content of any textual
corpora through approximate phrase search and document comparison. The
retrieval engine uses a character-based n-gram language model rather than the
conventional word-based one so as to achieve great flexibility in language
agnostic query processing.
  The index is implemented as a space-optimised character-based suffix tree
with an accompanying database of document content and metadata. A number of
text mining tools are integrated into the system to allow researchers to
discover textual patterns, perform comparative analysis, and find out what is
currently popular in the research community.
  Herein we describe the system architecture, user interface, models and
algorithms, and data storage of the Samtla system. We also present several case
studies of its usage in practice together with an evaluation of the systems'
ranking performance through crowdsourcing.
BreakingNews: Article Annotation by Image and Text Processing
Authors: Arnau Ramisa, Fei Yan, Francesc Moreno-Noguer, Krystian Mikolajczyk
Date: 2016-03-23 00:00:00
Tasks: Articles, Image Retrieval, Retrieval, Transfer Learning

Building upon recent Deep Neural Network architectures, current approaches
lying in the intersection of computer vision and natural language processing
have achieved unprecedented breakthroughs in tasks like automatic captioning or
image retrieval. Most of these learning methods, though, rely on large training
sets of images associated with human annotations that specifically describe the
visual content. In this paper we propose to go a step further and explore the
more complex cases where textual descriptions are loosely related to the
images. We focus on the particular domain of News articles in which the textual
content often expresses connotative and ambiguous relations that are only
suggested but not directly inferred from images. We introduce new deep learning
methods that address source detection, popularity prediction, article
illustration and geolocation of articles. An adaptive CNN architecture is
proposed, that shares most of the structure for all the tasks, and is suitable
for multitask and transfer learning. Deep Canonical Correlation Analysis is
deployed for article illustration, and a new loss function based on Great
Circle Distance is proposed for geolocation. Furthermore, we present
BreakingNews, a novel dataset with approximately 100K news articles including
images, text and captions, and enriched with heterogeneous meta-data (such as
GPS coordinates and popularity metrics). We show this dataset to be appropriate
to explore all aforementioned problems, for which we provide a baseline
performance using various Deep Learning architectures, and different
representations of the textual and visual features. We report very promising
results and bring to light several limitations of current state-of-the-art in
this kind of domain, which we hope will help spur progress in the field.
Comparison of Bayesian predictive methods for model selection
Authors: Juho Piironen, Aki Vehtari
Date: 2015-03-30 00:00:00
Tasks: model, Model Selection

The goal of this paper is to compare several widely used Bayesian model
selection methods in practical model selection problems, highlight their
differences and give recommendations about the preferred approaches. We focus
on the variable subset selection for regression and classification and perform
several numerical experiments using both simulated and real world data. The
results show that the optimization of a utility estimate such as the
cross-validation (CV) score is liable to finding overfitted models due to
relatively high variance in the utility estimates when the data is scarce. This
can also lead to substantial selection induced bias and optimism in the
performance evaluation for the selected model. From a predictive viewpoint,
best results are obtained by accounting for model uncertainty by forming the
full encompassing model, such as the Bayesian model averaging solution over the
candidate models. If the encompassing model is too complex, it can be robustly
simplified by the projection method, in which the information of the full model
is projected onto the submodels. This approach is substantially less prone to
overfitting than selection based on CV-score. Overall, the projection method
appears to outperform also the maximum a posteriori model and the selection of
the most probable variables. The study also demonstrates that the model
selection can greatly benefit from using cross-validation outside the searching
process both for guiding the model size selection and assessing the predictive
performance of the finally selected model.
Robust cDNA microarray image segmentation and analysis technique based on Hough circle transform
Authors: R. M. Farouk, M. A. SayedElahl
Date: 2016-03-23 00:00:00
Tasks: Image Segmentation, Segmentation, Semantic Segmentation

One of the most challenging tasks in microarray image analysis is spot
segmentation. A solution to this problem is to provide an algorithm than can be
used to find any spot within the microarray image. Circular Hough
Transformation (CHT) is a powerful feature extraction technique used in image
analysis, computer vision, and digital image processing. CHT algorithm is
applied on the cDNA microarray images to develop the accuracy and the
efficiency of the spots localization, addressing and segmentation process. The
purpose of the applied technique is to find imperfect instances of spots within
a certain class of circles by applying a voting procedure on the cDNA
microarray images for spots localization, addressing and characterizing the
pixels of each spot into foreground pixels and background simultaneously.
Intensive experiments on the University of North Carolina (UNC) microarray
database indicate that the proposed method is superior to the K-means method
and the Support vector machine (SVM). Keywords: Hough circle transformation,
cDNA microarray image analysis, cDNA microarray image segmentation, spots
localization and addressing, spots segmentation
Predicting Glaucoma Visual Field Loss by Hierarchically Aggregating Clustering-based Predictors
Authors: Motohide Higaki, Kai Morino, Hiroshi Murata, Ryo Asaoka, Kenji Yamanishi
Date: 2016-03-23 00:00:00
Tasks: Clustering, Prediction

This study addresses the issue of predicting the glaucomatous visual field
loss from patient disease datasets. Our goal is to accurately predict the
progress of the disease in individual patients. As very few measurements are
available for each patient, it is difficult to produce good predictors for
individuals. A recently proposed clustering-based method enhances the power of
prediction using patient data with similar spatiotemporal patterns. Each
patient is categorized into a cluster of patients, and a predictive model is
constructed using all of the data in the class. Predictions are highly
dependent on the quality of clustering, but it is difficult to identify the
best clustering method. Thus, we propose a method for aggregating cluster-based
predictors to obtain better prediction accuracy than from a single
cluster-based prediction. Further, the method shows very high performances by
hierarchically aggregating experts generated from several cluster-based
methods. We use real datasets to demonstrate that our method performs
significantly better than conventional clustering-based and patient-wise
regression methods, because the hierarchical aggregating strategy has a
mechanism whereby good predictors in a small community can thrive.