pwc-archive/papers-with-abstracts huggingface dataset:
Global and Local Uncertainty Principles for Signals on Graphs
Authors: Nathanael Perraudin, Benjamin Ricaud, David Shuman, Pierre Vandergheynst
Date: 2016-03-10 00:00:00

Uncertainty principles such as Heisenberg's provide limits on the
time-frequency concentration of a signal, and constitute an important
theoretical tool for designing and evaluating linear signal transforms.
Generalizations of such principles to the graph setting can inform dictionary
design for graph signals, lead to algorithms for reconstructing missing
information from graph signals via sparse representations, and yield new graph
analysis tools. While previous work has focused on generalizing notions of
spreads of a graph signal in the vertex and graph spectral domains, our
approach is to generalize the methods of Lieb in order to develop uncertainty
principles that provide limits on the concentration of the analysis
coefficients of any graph signal under a dictionary transform whose atoms are
jointly localized in the vertex and graph spectral domains. One challenge we
highlight is that due to the inhomogeneity of the underlying graph data domain,
the local structure in a single small region of the graph can drastically
affect the uncertainty bounds for signals concentrated in different regions of
the graph, limiting the information provided by global uncertainty principles.
Accordingly, we suggest a new way to incorporate a notion of locality, and
develop local uncertainty principles that bound the concentration of the
analysis coefficients of each atom of a localized graph spectral filter frame
in terms of quantities that depend on the local structure of the graph around
the center vertex of the given atom. Finally, we demonstrate how our proposed
local uncertainty measures can improve the random sampling of graph signals.
Building a Fine-Grained Entity Typing System Overnight for a New X (X = Language, Domain, Genre)
Authors: Lifu Huang, Jonathan May, Xiaoman Pan, Heng Ji
Date: 2016-03-10 00:00:00
Tasks: Abstract Meaning Representation, Clustering, Entity Typing

Recent research has shown great progress on fine-grained entity typing. Most
existing methods require pre-defining a set of types and training a multi-class
classifier from a large labeled data set based on multi-level linguistic
features. They are thus limited to certain domains, genres and languages. In
this paper, we propose a novel unsupervised entity typing framework by
combining symbolic and distributional semantics. We start from learning general
embeddings for each entity mention, compose the embeddings of specific contexts
using linguistic structures, link the mention to knowledge bases and learn its
related knowledge representations. Then we develop a novel joint hierarchical
clustering and linking algorithm to type all mentions using these
representations. This framework doesn't rely on any annotated data, predefined
typing schema, or hand-crafted features, therefore it can be quickly adapted to
a new domain, genre and language. Furthermore, it has great flexibility at
incorporating linguistic structures (e.g., Abstract Meaning Representation
(AMR), dependency relations) to improve specific context representation.
Experiments on genres (news and discussion forum) show comparable performance
with state-of-the-art supervised typing systems trained from a large amount of
labeled data. Results on various languages (English, Chinese, Japanese, Hausa,
and Yoruba) and domains (general and biomedical) demonstrate the portability of
our framework.
Recursive Recurrent Nets with Attention Modeling for OCR in the Wild
Authors: Chen-Yu Lee, Simon Osindero
Date: 2016-03-09 00:00:00
Proceeding: CVPR 2016 6
Tasks: Language Modeling, Language Modelling, Optical Character Recognition, Optical Character Recognition (OCR)

We present recursive recurrent neural networks with attention modeling
(R$^2$AM) for lexicon-free optical character recognition in natural scene
images. The primary advantages of the proposed method are: (1) use of recursive
convolutional neural networks (CNNs), which allow for parametrically efficient
and effective image feature extraction; (2) an implicitly learned
character-level language model, embodied in a recurrent neural network which
avoids the need to use N-grams; and (3) the use of a soft-attention mechanism,
allowing the model to selectively exploit image features in a coordinated way,
and allowing for end-to-end training within a standard backpropagation
framework. We validate our method with state-of-the-art performance on
challenging benchmark datasets: Street View Text, IIIT5k, ICDAR and Synth90k.
Blind Source Separation: Fundamentals and Recent Advances (A Tutorial Overview Presented at SBrT-2001)
Authors: Eleftherios Kofidis
Date: 2016-03-09 00:00:00
Tasks: blind source separation

Blind source separation (BSS), i.e., the decoupling of unknown signals that
have been mixed in an unknown way, has been a topic of great interest in the
signal processing community for the last decade, covering a wide range of
applications in such diverse fields as digital communications, pattern
recognition, biomedical engineering, and financial data analysis, among others.
This course aims at an introduction to the BSS problem via an exposition of
well-known and established as well as some more recent approaches to its
solution. A unified way is followed in presenting the various results so as to
more easily bring out their similarities/differences and emphasize their
relative advantages/disadvantages. Only a representative sample of the existing
knowledge on BSS will be included in this course. The interested readers are
encouraged to consult the list of bibliographical references for more details
on this exciting and always active research topic.
Dynamic Privacy For Distributed Machine Learning Over Network
Authors: Tao Zhang, Quanyan Zhu
Date: 2016-01-14 00:00:00
Tasks: BIG-bench Machine Learning, Privacy Preserving

Privacy-preserving distributed machine learning becomes increasingly
important due to the recent rapid growth of data. This paper focuses on a class
of regularized empirical risk minimization (ERM) machine learning problems, and
develops two methods to provide differential privacy to distributed learning
algorithms over a network. We first decentralize the learning algorithm using
the alternating direction method of multipliers (ADMM), and propose the methods
of dual variable perturbation and primal variable perturbation to provide
dynamic differential privacy. The two mechanisms lead to algorithms that can
provide privacy guarantees under mild conditions of the convexity and
differentiability of the loss function and the regularizer. We study the
performance of the algorithms, and show that the dual variable perturbation
outperforms its primal counterpart. To design an optimal privacy mechanisms, we
analyze the fundamental tradeoff between privacy and accuracy, and provide
guidelines to choose privacy parameters. Numerical experiments using customer
information database are performed to corroborate the results on privacy and
utility tradeoffs and design.
Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D Images
Authors: Shuran Song, Jianxiong Xiao
Date: 2015-11-07 00:00:00
Proceeding: CVPR 2016 6
Tasks: 3D Object Detection, Object, object-detection, Object Detection, Object Recognition, Region Proposal

We focus on the task of amodal 3D object detection in RGB-D images, which
aims to produce a 3D bounding box of an object in metric form at its full
extent. We introduce Deep Sliding Shapes, a 3D ConvNet formulation that takes a
3D volumetric scene from a RGB-D image as input and outputs 3D object bounding
boxes. In our approach, we propose the first 3D Region Proposal Network (RPN)
to learn objectness from geometric shapes and the first joint Object
Recognition Network (ORN) to extract geometric features in 3D and color
features in 2D. In particular, we handle objects of various sizes by training
an amodal RPN at two different scales and an ORN to regress 3D bounding boxes.
Experiments show that our algorithm outperforms the state-of-the-art by 13.8 in
mAP and is 200x faster than the original Sliding Shapes. All source code and
pre-trained models will be available at GitHub.
Robot Dream
Authors: Alexander Tchitchigin, Max Talanov, Larisa Safina, Manuel Mazzara
Date: 2016-03-09 00:00:00
Tasks: Position

In this position paper we present a novel approach to neurobiologically
plausible implementation of emotional reactions and behaviors for real-time
autonomous robotic systems. The working metaphor we use is the "day" and
"night" phases of mammalian life. During the "day" phase a robotic system
stores the inbound information and is controlled by a light-weight rule-based
system in real time. In contrast to that, during the "night" phase the stored
information is been transferred to the supercomputing system to update the
realistic neural network: emotional and behavioral strategies.
Summarization of Films and Documentaries Based on Subtitles and Scripts
Authors: Marta Aparício, Paulo Figueiredo, Francisco Raposo, David Martins de Matos, Ricardo Ribeiro, Luís Marujo
Date: 2015-06-03 00:00:00
Tasks: Articles, Text Summarization

We assess the performance of generic text summarization algorithms applied to
films and documentaries, using the well-known behavior of summarization of news
articles as reference. We use three datasets: (i) news articles, (ii) film
scripts and subtitles, and (iii) documentary subtitles. Standard ROUGE metrics
are used for comparing generated summaries against news abstracts, plot
summaries, and synopses. We show that the best performing algorithms are LSA,
for news articles and documentaries, and LexRank and Support Sets, for films.
Despite the different nature of films and documentaries, their relative
behavior is in accordance with that obtained for news articles.
Using Generic Summarization to Improve Music Information Retrieval Tasks
Authors: Francisco Raposo, Ricardo Ribeiro, David Martins de Matos
Date: 2015-03-23 00:00:00
Tasks: General Classification, Genre classification, Information Retrieval, Music Genre Classification, Music Information Retrieval, Retrieval

In order to satisfy processing time constraints, many MIR tasks process only
a segment of the whole music signal. This practice may lead to decreasing
performance, since the most important information for the tasks may not be in
those processed segments. In this paper, we leverage generic summarization
algorithms, previously applied to text and speech summarization, to summarize
items in music datasets. These algorithms build summaries, that are both
concise and diverse, by selecting appropriate segments from the input signal
which makes them good candidates to summarize music as well. We evaluate the
summarization process on binary and multiclass music genre classification
tasks, by comparing the performance obtained using summarized datasets against
the performances obtained using continuous segments (which is the traditional
method used for addressing the previously mentioned time constraints) and full
songs of the same original dataset. We show that GRASSHOPPER, LexRank, LSA,
MMR, and a Support Sets-based Centrality model improve classification
performance when compared to selected 30-second baselines. We also show that
summarized datasets lead to a classification performance whose difference is
not statistically significant from using full songs. Furthermore, we make an
argument stating the advantages of sharing summarized datasets for future MIR
research.
Nonparametric Bayesian Double Articulation Analyzer for Direct Language Acquisition from Continuous Speech Signals
Authors: Tadahiro Taniguchi, Ryo Nakashima, Shogo Nagasaka
Date: 2015-06-22 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Language Acquisition, Language Modeling, Language Modelling, speech-recognition, Speech Recognition, Time Series Analysis

Human infants can discover words directly from unsegmented speech signals
without any explicitly labeled data. In this paper, we develop a novel machine
learning method called nonparametric Bayesian double articulation analyzer
(NPB-DAA) that can directly acquire language and acoustic models from observed
continuous speech signals. For this purpose, we propose an integrative
generative model that combines a language model and an acoustic model into a
single generative model called the "hierarchical Dirichlet process hidden
language model" (HDP-HLM). The HDP-HLM is obtained by extending the
hierarchical Dirichlet process hidden semi-Markov model (HDP-HSMM) proposed by
Johnson et al. An inference procedure for the HDP-HLM is derived using the
blocked Gibbs sampler originally proposed for the HDP-HSMM. This procedure
enables the simultaneous and direct inference of language and acoustic models
from continuous speech signals. Based on the HDP-HLM and its inference
procedure, we developed a novel double articulation analyzer. By assuming
HDP-HLM as a generative model of observed time series data, and by inferring
latent variables of the model, the method can analyze latent double
articulation structure, i.e., hierarchically organized latent words and
phonemes, of the data in an unsupervised manner. The novel unsupervised double
articulation analyzer is called NPB-DAA.
  The NPB-DAA can automatically estimate double articulation structure embedded
in speech signals. We also carried out two evaluation experiments using
synthetic data and actual human continuous speech signals representing Japanese
vowel sequences. In the word acquisition and phoneme categorization tasks, the
NPB-DAA outperformed a conventional double articulation analyzer (DAA) and
baseline automatic speech recognition system whose acoustic model was trained
in a supervised manner.
Patch-based Convolutional Neural Network for Whole Slide Tissue Image Classification
Authors: Le Hou, Dimitris Samaras, Tahsin M. Kurc, Yi Gao, James E. Davis, Joel H. Saltz
Date: 2015-04-29 00:00:00
Proceeding: CVPR 2016 6
Tasks: Classification, General Classification, image-classification, Image Classification

Convolutional Neural Networks (CNN) are state-of-the-art models for many
image classification tasks. However, to recognize cancer subtypes
automatically, training a CNN on gigapixel resolution Whole Slide Tissue Images
(WSI) is currently computationally impossible. The differentiation of cancer
subtypes is based on cellular-level visual features observed on image patch
scale. Therefore, we argue that in this situation, training a patch-level
classifier on image patches will perform better than or similar to an
image-level classifier. The challenge becomes how to intelligently combine
patch-level classification results and model the fact that not all patches will
be discriminative. We propose to train a decision fusion model to aggregate
patch-level predictions given by patch-level CNNs, which to the best of our
knowledge has not been shown before. Furthermore, we formulate a novel
Expectation-Maximization (EM) based method that automatically locates
discriminative patches robustly by utilizing the spatial relationships of
patches. We apply our method to the classification of glioma and non-small-cell
lung carcinoma cases into subtypes. The classification accuracy of our method
is similar to the inter-observer agreement between pathologists. Although it is
impossible to train CNNs on WSIs, we experimentally demonstrate using a
comparable non-cancer dataset of smaller images that a patch-based CNN can
outperform an image-based CNN.
Unsupervised word segmentation and lexicon discovery using acoustic word embeddings
Authors: Herman Kamper, Aren Jansen, Sharon Goldwater
Date: 2016-03-09 00:00:00
Tasks: Language Acquisition, Language Modelling, Word Embeddings

In settings where only unlabelled speech data is available, speech technology
needs to be developed without transcriptions, pronunciation dictionaries, or
language modelling text. A similar problem is faced when modelling infant
language acquisition. In these cases, categorical linguistic structure needs to
be discovered directly from speech audio. We present a novel unsupervised
Bayesian model that segments unlabelled speech and clusters the segments into
hypothesized word groupings. The result is a complete unsupervised tokenization
of the input speech in terms of discovered word types. In our approach, a
potential word segment (of arbitrary length) is embedded in a fixed-dimensional
acoustic vector space. The model, implemented as a Gibbs sampler, then builds a
whole-word acoustic model in this space while jointly performing segmentation.
We report word error rates in a small-vocabulary connected digit recognition
task by mapping the unsupervised decoded output to ground truth transcriptions.
The model achieves around 20% error rate, outperforming a previous HMM-based
system by about 10% absolute. Moreover, in contrast to the baseline, our model
does not require a pre-specified vocabulary size.
Note on the equivalence of hierarchical variational models and auxiliary deep generative models
Authors: Niko Brümmer
Date: 2016-03-08 00:00:00
Tasks: BIG-bench Machine Learning

This note compares two recently published machine learning methods for
constructing flexible, but tractable families of variational hidden-variable
posteriors. The first method, called "hierarchical variational models" enriches
the inference model with an extra variable, while the other, called "auxiliary
deep generative models", enriches the generative model instead. We conclude
that the two methods are mathematically equivalent.
Faster learning of deep stacked autoencoders on multi-core systems using synchronized layer-wise pre-training
Authors: Anirban Santara, Debapriya Maji, DP Tejas, Pabitra Mitra, Arobinda Gupta
Date: 2016-03-09 00:00:00
Tasks: Dimensionality Reduction, Unsupervised Pre-training

Deep neural networks are capable of modelling highly non-linear functions by
capturing different levels of abstraction of data hierarchically. While
training deep networks, first the system is initialized near a good optimum by
greedy layer-wise unsupervised pre-training. However, with burgeoning data and
increasing dimensions of the architecture, the time complexity of this approach
becomes enormous. Also, greedy pre-training of the layers often turns
detrimental by over-training a layer causing it to lose harmony with the rest
of the network. In this paper a synchronized parallel algorithm for
pre-training deep networks on multi-core machines has been proposed. Different
layers are trained by parallel threads running on different cores with regular
synchronization. Thus the pre-training process becomes faster and chances of
over-training are reduced. This is experimentally validated using a stacked
autoencoder for dimensionality reduction of MNIST handwritten digit database.
The proposed algorithm achieved 26\% speed-up compared to greedy layer-wise
pre-training for achieving the same reconstruction accuracy substantiating its
potential as an alternative.
Optimized Kernel Entropy Components
Authors: Emma Izquierdo-Verdiguier, Valero Laparra, Robert Jenssen, Luis Gómez-Chova, Gustau Camps-Valls
Date: 2016-03-09 00:00:00
Tasks: Density Estimation

This work addresses two main issues of the standard Kernel Entropy Component
Analysis (KECA) algorithm: the optimization of the kernel decomposition and the
optimization of the Gaussian kernel parameter. KECA roughly reduces to a
sorting of the importance of kernel eigenvectors by entropy instead of by
variance as in Kernel Principal Components Analysis. In this work, we propose
an extension of the KECA method, named Optimized KECA (OKECA), that directly
extracts the optimal features retaining most of the data entropy by means of
compacting the information in very few features (often in just one or two). The
proposed method produces features which have higher expressive power. In
particular, it is based on the Independent Component Analysis (ICA) framework,
and introduces an extra rotation to the eigen-decomposition, which is optimized
via gradient ascent search. This maximum entropy preservation suggests that
OKECA features are more efficient than KECA features for density estimation. In
addition, a critical issue in both methods is the selection of the kernel
parameter since it critically affects the resulting performance. Here we
analyze the most common kernel length-scale selection criteria. Results of both
methods are illustrated in different synthetic and real problems. Results show
that 1) OKECA returns projections with more expressive power than KECA, 2) the
most successful rule for estimating the kernel parameter is based on maximum
likelihood, and 3) OKECA is more robust to the selection of the length-scale
parameter in kernel density estimation.
Selective Inference Approach for Statistically Sound Predictive Pattern Mining
Authors: Shinya Suzumura, Kazuya Nakagawa, Mahito Sugiyama, Koji Tsuda, Ichiro Takeuchi
Date: 2016-02-15 00:00:00
Tasks: Selection bias, Two-sample testing

Discovering statistically significant patterns from databases is an important
challenging problem. The main obstacle of this problem is in the difficulty of
taking into account the selection bias, i.e., the bias arising from the fact
that patterns are selected from extremely large number of candidates in
databases. In this paper, we introduce a new approach for predictive pattern
mining problems that can address the selection bias issue. Our approach is
built on a recently popularized statistical inference framework called
selective inference. In selective inference, statistical inferences (such as
statistical hypothesis testing) are conducted based on sampling distributions
conditional on a selection event. If the selection event is characterized in a
tractable way, statistical inferences can be made without minding selection
bias issue. However, in pattern mining problems, it is difficult to
characterize the entire selection process of mining algorithms. Our main
contribution in this paper is to solve this challenging problem for a class of
predictive pattern mining problems by introducing a novel algorithmic
framework. We demonstrate that our approach is useful for finding statistically
significant patterns from databases.
Bipartite Correlation Clustering -- Maximizing Agreements
Authors: Megasthenis Asteris, Anastasios Kyrillidis, Dimitris Papailiopoulos, Alexandros G. Dimakis
Date: 2016-03-09 00:00:00
Tasks: Clustering

In Bipartite Correlation Clustering (BCC) we are given a complete bipartite
graph $G$ with `+' and `-' edges, and we seek a vertex clustering that
maximizes the number of agreements: the number of all `+' edges within clusters
plus all `-' edges cut across clusters. BCC is known to be NP-hard.
  We present a novel approximation algorithm for $k$-BCC, a variant of BCC with
an upper bound $k$ on the number of clusters. Our algorithm outputs a
$k$-clustering that provably achieves a number of agreements within a
multiplicative ${(1-\delta)}$-factor from the optimal, for any desired accuracy
$\delta$. It relies on solving a combinatorially constrained bilinear
maximization on the bi-adjacency matrix of $G$. It runs in time exponential in
$k$ and $\delta^{-1}$, but linear in the size of the input.
  Further, we show that, in the (unconstrained) BCC setting, an
${(1-\delta)}$-approximation can be achieved by $O(\delta^{-1})$ clusters
regardless of the size of the graph. In turn, our $k$-BCC algorithm implies an
Efficient PTAS for the BCC objective of maximizing agreements.