pwc-archive/papers-with-abstracts huggingface dataset:
Unsupervised Representation Learning of Structured Radio Communication Signals
Authors: Timothy J. O'Shea, Johnathan Corgan, T. Charles Clancy
Date: 2016-04-24 00:00:00
Tasks: Representation Learning, Time Series, Time Series Analysis

We explore unsupervised representation learning of radio communication
signals in raw sampled time series representation. We demonstrate that we can
learn modulation basis functions using convolutional autoencoders and visually
recognize their relationship to the analytic bases used in digital
communications. We also propose and evaluate quantitative met- rics for quality
of encoding using domain relevant performance metrics.
A note on the evaluation of generative models
Authors: Lucas Theis, Aäron van den Oord, Matthias Bethge
Date: 2015-11-05 00:00:00
Tasks: Denoising, Texture Synthesis

Probabilistic generative models can be used for compression, denoising,
inpainting, texture synthesis, semi-supervised learning, unsupervised feature
learning, and other tasks. Given this wide range of applications, it is not
surprising that a lot of heterogeneity exists in the way these models are
formulated, trained, and evaluated. As a consequence, direct comparison between
models is often difficult. This article reviews mostly known but often
underappreciated properties relating to the evaluation and interpretation of
generative models with a focus on image models. In particular, we show that
three of the currently most commonly used criteria---average log-likelihood,
Parzen window estimates, and visual fidelity of samples---are largely
independent of each other when the data is high-dimensional. Good performance
with respect to one criterion therefore need not imply good performance with
respect to the other criteria. Our results show that extrapolation from one
criterion to another is not warranted and generative models need to be
evaluated directly with respect to the application(s) they were intended for.
In addition, we provide examples demonstrating that Parzen window estimates
should generally be avoided.
Binary Codes for Tagging X-Ray Images via Deep De-Noising Autoencoders
Authors: Antonio Sze-To, Hamid. R. Tizhoosh, Andrew K. C. Wong
Date: 2016-04-24 00:00:00
Tasks: Content-Based Image Retrieval, Image Retrieval, Retrieval

A Content-Based Image Retrieval (CBIR) system which identifies similar
medical images based on a query image can assist clinicians for more accurate
diagnosis. The recent CBIR research trend favors the construction and use of
binary codes to represent images. Deep architectures could learn the non-linear
relationship among image pixels adaptively, allowing the automatic learning of
high-level features from raw pixels. However, most of them require class
labels, which are expensive to obtain, particularly for medical images. The
methods which do not need class labels utilize a deep autoencoder for binary
hashing, but the code construction involves a specific training algorithm and
an ad-hoc regularization technique. In this study, we explored using a deep
de-noising autoencoder (DDA), with a new unsupervised training scheme using
only backpropagation and dropout, to hash images into binary codes. We
conducted experiments on more than 14,000 x-ray images. By using class labels
only for evaluating the retrieval results, we constructed a 16-bit DDA and a
512-bit DDA independently. Comparing to other unsupervised methods, we
succeeded to obtain the lowest total error by using the 512-bit codes for
retrieval via exhaustive search, and speed up 9.27 times with the use of the
16-bit codes while keeping a comparable total error. We found that our new
training scheme could reduce the total retrieval error significantly by 21.9%.
To further boost the image retrieval performance, we developed Radon
Autoencoder Barcode (RABC) which are learned from the Radon projections of
images using a de-noising autoencoder. Experimental results demonstrated its
superior performance in retrieval when it was combined with DDA binary codes.
Optimization as Estimation with Gaussian Processes in Bandit Settings
Authors: Zi Wang, Bolei Zhou, Stefanie Jegelka
Date: 2015-10-21 00:00:00
Tasks: Bayesian Optimization, Gaussian Processes

Recently, there has been rising interest in Bayesian optimization -- the
optimization of an unknown function with assumptions usually expressed by a
Gaussian Process (GP) prior. We study an optimization strategy that directly
uses an estimate of the argmax of the function. This strategy offers both
practical and theoretical advantages: no tradeoff parameter needs to be
selected, and, moreover, we establish close connections to the popular GP-UCB
and GP-PI strategies. Our approach can be understood as automatically and
adaptively trading off exploration and exploitation in GP-UCB and GP-PI. We
illustrate the effects of this adaptive tuning via bounds on the regret as well
as an extensive empirical evaluation on robotics and vision tasks,
demonstrating the robustness of this strategy for a range of performance
criteria.
Learning and Tuning Meta-heuristics in Plan Space Planning
Authors: Shashank Shekhar, Deepak Khemani
Date: 2016-01-27 00:00:00
Tasks: Informativeness

In recent years, the planning community has observed that techniques for
learning heuristic functions have yielded improvements in performance. One
approach is to use offline learning to learn predictive models from existing
heuristics in a domain dependent manner. These learned models are deployed as
new heuristic functions. The learned models can in turn be tuned online using a
domain independent error correction approach to further enhance their
informativeness. The online tuning approach is domain independent but instance
specific, and contributes to improved performance for individual instances as
planning proceeds. Consequently it is more effective in larger problems.
  In this paper, we mention two approaches applicable in Partial Order Causal
Link (POCL) Planning that is also known as Plan Space Planning. First, we
endeavor to enhance the performance of a POCL planner by giving an algorithm
for supervised learning. Second, we then discuss an online error minimization
approach in POCL framework to minimize the step-error associated with the
offline learned models thus enhancing their informativeness. Our evaluation
shows that the learning approaches scale up the performance of the planner over
standard benchmarks, specially for larger problems.
Investigating echo state networks dynamics by means of recurrence analysis
Authors: Filippo Maria Bianchi, Lorenzo Livi, Cesare Alippi
Date: 2016-01-26 00:00:00
Tasks: Time Series, Time Series Analysis

In this paper, we elaborate over the well-known interpretability issue in
echo state networks. The idea is to investigate the dynamics of reservoir
neurons with time-series analysis techniques taken from research on complex
systems. Notably, we analyze time-series of neuron activations with Recurrence
Plots (RPs) and Recurrence Quantification Analysis (RQA), which permit to
visualize and characterize high-dimensional dynamical systems. We show that
this approach is useful in a number of ways. First, the two-dimensional
representation offered by RPs provides a way for visualizing the
high-dimensional dynamics of a reservoir. Our results suggest that, if the
network is stable, reservoir and input denote similar line patterns in the
respective RPs. Conversely, the more unstable the ESN, the more the RP of the
reservoir presents instability patterns. As a second result, we show that the
$\mathrm{L_{max}}$ measure is highly correlated with the well-established
maximal local Lyapunov exponent. This suggests that complexity measures based
on RP diagonal lines distribution provide a valuable tool to quantify the
degree of network stability. Finally, our analysis shows that all RQA measures
fluctuate on the proximity of the so-called edge of stability, where an ESN
typically achieves maximum computational capability. We verify that the
determination of the edge of stability provided by such RQA measures is more
accurate than two well-known criteria based on the Jacobian matrix of the
reservoir. Therefore, we claim that RPs and RQA-based analyses can be used as
valuable tools to design an effective network given a specific problem.
Matrix completion with column manipulation: Near-optimal sample-robustness-rank tradeoffs
Authors: Yudong Chen, Huan Xu, Constantine Caramanis, Sujay Sanghavi
Date: 2011-02-10 00:00:00
Tasks: Collaborative Filtering, Matrix Completion

This paper considers the problem of matrix completion when some number of the
columns are completely and arbitrarily corrupted, potentially by a malicious
adversary. It is well-known that standard algorithms for matrix completion can
return arbitrarily poor results, if even a single column is corrupted. One
direct application comes from robust collaborative filtering. Here, some number
of users are so-called manipulators who try to skew the predictions of the
algorithm by calibrating their inputs to the system. In this paper, we develop
an efficient algorithm for this problem based on a combination of a trimming
procedure and a convex program that minimizes the nuclear norm and the
$\ell_{1,2}$ norm. Our theoretical results show that given a vanishing fraction
of observed entries, it is nevertheless possible to complete the underlying
matrix even when the number of corrupted columns grows. Significantly, our
results hold without any assumptions on the locations or values of the observed
entries of the manipulated columns. Moreover, we show by an
information-theoretic argument that our guarantees are nearly optimal in terms
of the fraction of sampled entries on the authentic columns, the fraction of
corrupted columns, and the rank of the underlying matrix. Our results therefore
sharply characterize the tradeoffs between sample, robustness and rank in
matrix completion.
Cardiac Motion Analysis by Temporal Flow Graphs
Authors: V. S. R. Veeravasarapu, Jayanthi Sivaswamy, Vishanji Karani
Date: 2016-04-24 00:00:00

Cardiac motion analysis from B-mode ultrasound sequence is a key task in
assessing the health of the heart. The paper proposes a new methodology for
cardiac motion analysis based on the temporal behaviour of points of interest
on the myocardium. We define a new signal called the Temporal Flow Graph (TFG)
which depicts the movement of a point of interest over time. It is a graphical
representation derived from a flow field and describes the temporal evolution
of a point. We prove that TFG for an object undergoing periodic motion is also
periodic. This principle can be utilized to derive both global and local
information from a given sequence. We demonstrate this for detecting motion
irregularities at the sequence, as well as regional levels on real and
synthetic data. A coarse localisation of anatomical landmarks such as centres
of left/right cavities and valve points is also demonstrated using TFGs.
Extracted Social Network Mining
Authors: Mahyuddin K. M. Nasution
Date: 2016-04-24 00:00:00
Tasks: Relation

In this paper we study the relationship between the resources of social
networks by exploring the Web as big data based on a simple search engine. We
have used set theory by utilizing the occurrence and co-occurrence for defining
the singleton or doubleton spaces of event in a search engine model, and then
provided them as representation of social actors and their relationship in
clusters. Thus, there are behaviors of social actors and their relation based
on Web.
Bayesian Inference of Recursive Sequences of Group Activities from Tracks
Authors: Ernesto Brau, Colin Dawson, Alfredo Carrillo, David Sidi, Clayton T. Morrison
Date: 2016-04-24 00:00:00
Tasks: Bayesian Inference

We present a probabilistic generative model for inferring a description of
coordinated, recursively structured group activities at multiple levels of
temporal granularity based on observations of individuals' trajectories. The
model accommodates: (1) hierarchically structured groups, (2) activities that
are temporally and compositionally recursive, (3) component roles assigning
different subactivity dynamics to subgroups of participants, and (4) a
nonparametric Gaussian Process model of trajectories. We present an MCMC
sampling framework for performing joint inference over recursive activity
descriptions and assignment of trajectories to groups, integrating out
continuous parameters. We demonstrate the model's expressive power in several
simulated and complex real-world scenarios from the VIRAT and UCLA Aerial Event
video data sets.
Efficient AUC Optimization for Information Ranking Applications
Authors: Sean J. Welleck
Date: 2015-11-16 00:00:00
Tasks: Information Retrieval, regression, Retrieval

Adequate evaluation of an information retrieval system to estimate future
performance is a crucial task. Area under the ROC curve (AUC) is widely used to
evaluate the generalization of a retrieval system. However, the objective
function optimized in many retrieval systems is the error rate and not the AUC
value. This paper provides an efficient and effective non-linear approach to
optimize AUC using additive regression trees, with a special emphasis on the
use of multi-class AUC (MAUC) because multiple relevance levels are widely used
in many ranking applications. Compared to a conventional linear approach, the
performance of the non-linear approach is comparable on binary-relevance
benchmark datasets and is better on multi-relevance benchmark datasets.
Net2Net: Accelerating Learning via Knowledge Transfer
Authors: Tianqi Chen, Ian Goodfellow, Jonathon Shlens
Date: 2015-11-18 00:00:00
Tasks: Transfer Learning

We introduce techniques for rapidly transferring the information stored in
one neural net into another neural net. The main purpose is to accelerate the
training of a significantly larger neural net. During real-world workflows, one
often trains very many different neural networks during the experimentation and
design process. This is a wasteful process in which each new model is trained
from scratch. Our Net2Net technique accelerates the experimentation process by
instantaneously transferring the knowledge from a previous network to each new
deeper or wider network. Our techniques are based on the concept of
function-preserving transformations between neural network specifications. This
differs from previous approaches to pre-training that altered the function
represented by a neural net when adding layers to it. Using our knowledge
transfer mechanism to add depth to Inception modules, we demonstrate a new
state of the art accuracy rating on the ImageNet dataset.
RHOG: A Refinement-Operator Library for Directed Labeled Graphs
Authors: Santiago Ontañón
Date: 2016-04-23 00:00:00

This document provides the foundations behind the functionality provided by the $\rho$G library (https://github.com/santiontanon/RHOG), focusing on the basic operations the library provides: subsumption, refinement of directed labeled graphs, and distance/similarity assessment between directed labeled graphs. $\rho$G development was initially supported by the National Science Foundation, by the EAGER grant IIS-1551338.
Probably certifiably correct k-means clustering
Authors: Takayuki Iguchi, Dustin G. Mixon, Jesse Peterson, Soledad Villar
Date: 2015-09-26 00:00:00
Tasks: Clustering

Recently, Bandeira [arXiv:1509.00824] introduced a new type of algorithm (the
so-called probably certifiably correct algorithm) that combines fast solvers
with the optimality certificates provided by convex relaxations. In this paper,
we devise such an algorithm for the problem of k-means clustering. First, we
prove that Peng and Wei's semidefinite relaxation of k-means is tight with high
probability under a distribution of planted clusters called the stochastic ball
model. Our proof follows from a new dual certificate for integral solutions of
this semidefinite program. Next, we show how to test the optimality of a
proposed k-means solution using this dual certificate in quasilinear time.
Finally, we analyze a version of spectral clustering from Peng and Wei that is
designed to solve k-means in the case of two clusters. In particular, we show
that this quasilinear-time method typically recovers planted clusters under the
stochastic ball model.
Memory and Information Processing in Recurrent Neural Networks
Authors: Alireza Goudarzi, Sarah Marzen, Peter Banda, Guy Feldman, Christof Teuscher, Darko Stefanovic
Date: 2016-04-23 00:00:00

Recurrent neural networks (RNN) are simple dynamical systems whose
computational power has been attributed to their short-term memory. Short-term
memory of RNNs has been previously studied analytically only for the case of
orthogonal networks, and only under annealed approximation, and uncorrelated
input. Here for the first time, we present an exact solution to the memory
capacity and the task-solving performance as a function of the structure of a
given network instance, enabling direct determination of the
function--structure relation in RNNs. We calculate the memory capacity for
arbitrary networks with exponentially correlated input and further related it
to the performance of the system on signal processing tasks in a supervised
learning setup. We compute the expected error and the worst-case error bound as
a function of the spectra of the network and the correlation structure of its
inputs and outputs. Our results give an explanation for learning and
generalization of task solving using short-term memory, which is crucial for
building alternative computer architectures using physical phenomena based on
the short-term memory principle.
Learning Document Embeddings by Predicting N-grams for Sentiment Classification of Long Movie Reviews
Authors: Bofang Li, Tao Liu, Xiaoyong Du, Deyuan Zhang, Zhe Zhao
Date: 2015-12-27 00:00:00
Tasks: General Classification, Sentiment Analysis, Sentiment Classification

Despite the loss of semantic information, bag-of-ngram based methods still
achieve state-of-the-art results for tasks such as sentiment classification of
long movie reviews. Many document embeddings methods have been proposed to
capture semantics, but they still can't outperform bag-of-ngram based methods
on this task. In this paper, we modify the architecture of the recently
proposed Paragraph Vector, allowing it to learn document vectors by predicting
not only words, but n-gram features as well. Our model is able to capture both
semantics and word order in documents while keeping the expressive power of
learned vectors. Experimental results on IMDB movie review dataset shows that
our model outperforms previous deep learning models and bag-of-ngram based
models due to the above advantages. More robust results are also obtained when
our model is combined with other models. The source code of our model will be
also published together with this paper.
On the Sample Complexity of End-to-end Training vs. Semantic Abstraction Training
Authors: Shai Shalev-Shwartz, Amnon Shashua
Date: 2016-04-23 00:00:00
Tasks: Autonomous Driving

We compare the end-to-end training approach to a modular approach in which a
system is decomposed into semantically meaningful components. We focus on the
sample complexity aspect, in the regime where an extremely high accuracy is
necessary, as is the case in autonomous driving applications. We demonstrate
cases in which the number of training examples required by the end-to-end
approach is exponentially larger than the number of examples required by the
semantic abstraction approach.
TagBook: A Semantic Video Representation without Supervision for Event Detection
Authors: Masoud Mazloom, Xirong Li, Cees G. M. Snoek
Date: 2015-10-10 00:00:00
Tasks: Event Detection, Image Retrieval, Retrieval, TAG

We consider the problem of event detection in video for scenarios where only
few, or even zero examples are available for training. For this challenging
setting, the prevailing solutions in the literature rely on a semantic video
representation obtained from thousands of pre-trained concept detectors.
Different from existing work, we propose a new semantic video representation
that is based on freely available social tagged videos only, without the need
for training any intermediate concept detectors. We introduce a simple
algorithm that propagates tags from a video's nearest neighbors, similar in
spirit to the ones used for image retrieval, but redesign it for video event
detection by including video source set refinement and varying the video tag
assignment. We call our approach TagBook and study its construction,
descriptiveness and detection performance on the TRECVID 2013 and 2014
multimedia event detection datasets and the Columbia Consumer Video dataset.
Despite its simple nature, the proposed TagBook video representation is
remarkably effective for few-example and zero-example event detection, even
outperforming very recent state-of-the-art alternatives building on supervised
representations.
Dimensionality-Dependent Generalization Bounds for $k$-Dimensional Coding Schemes
Authors: Tongliang Liu, DaCheng Tao, Dong Xu
Date: 2016-01-03 00:00:00
Tasks: Clustering, Dictionary Learning, Generalization Bounds, Quantization

The $k$-dimensional coding schemes refer to a collection of methods that
attempt to represent data using a set of representative $k$-dimensional
vectors, and include non-negative matrix factorization, dictionary learning,
sparse coding, $k$-means clustering and vector quantization as special cases.
Previous generalization bounds for the reconstruction error of the
$k$-dimensional coding schemes are mainly dimensionality independent. A major
advantage of these bounds is that they can be used to analyze the
generalization error when data is mapped into an infinite- or high-dimensional
feature space. However, many applications use finite-dimensional data features.
Can we obtain dimensionality-dependent generalization bounds for
$k$-dimensional coding schemes that are tighter than dimensionality-independent
bounds when data is in a finite-dimensional feature space? The answer is
positive. In this paper, we address this problem and derive a
dimensionality-dependent generalization bound for $k$-dimensional coding
schemes by bounding the covering number of the loss function class induced by
the reconstruction error. The bound is of order
$\mathcal{O}\left(\left(mk\ln(mkn)/n\right)^{\lambda_n}\right)$, where $m$ is
the dimension of features, $k$ is the number of the columns in the linear
implementation of coding schemes, $n$ is the size of sample, $\lambda_n>0.5$
when $n$ is finite and $\lambda_n=0.5$ when $n$ is infinite. We show that our
bound can be tighter than previous results, because it avoids inducing the
worst-case upper bound on $k$ of the loss function and converges faster. The
proposed generalization bound is also applied to some specific coding schemes
to demonstrate that the dimensionality-dependent bound is an indispensable
complement to these dimensionality-independent generalization bounds.