pwc-archive/papers-with-abstracts huggingface dataset:
Guaranteed inference in topic models
Authors: Khoat Than, Tung Doan
Date: 2015-12-10 00:00:00
Tasks: Topic Models

One of the core problems in statistical models is the estimation of a
posterior distribution. For topic models, the problem of posterior inference
for individual texts is particularly important, especially when dealing with
data streams, but is often intractable in the worst case. As a consequence,
existing methods for posterior inference are approximate and do not have any
guarantee on neither quality nor convergence rate. In this paper, we introduce
a provably fast algorithm, namely Online Maximum a Posteriori Estimation (OPE),
for posterior inference in topic models. OPE has more attractive properties
than existing inference approaches, including theoretical guarantees on quality
and fast rate of convergence to a local maximal/stationary point of the
inference problem. The discussions about OPE are very general and hence can be
easily employed in a wide range of contexts. Finally, we employ OPE to design
three methods for learning Latent Dirichlet Allocation from text streams or
large corpora. Extensive experiments demonstrate some superior behaviors of OPE
and of our new learning methods.
Hard Clusters Maximize Mutual Information
Authors: Bernhard C. Geiger, Rana Ali Amjad
Date: 2016-08-17 00:00:00
Tasks: Clustering

In this paper, we investigate mutual information as a cost function for
clustering, and show in which cases hard, i.e., deterministic, clusters are
optimal. Using convexity properties of mutual information, we show that certain
formulations of the information bottleneck problem are solved by hard clusters.
Similarly, hard clusters are optimal for the information-theoretic
co-clustering problem that deals with simultaneous clustering of two dependent
data sets. If both data sets have to be clustered using the same cluster
assignment, hard clusters are not optimal in general. We point at interesting
and practically relevant special cases of this so-called pairwise clustering
problem, for which we can either prove or have evidence that hard clusters are
optimal. Our results thus show that one can relax the otherwise combinatorial
hard clustering problem to a real-valued optimization problem with the same
global optimum.
A Convolutional Autoencoder for Multi-Subject fMRI Data Aggregation
Authors: Po-Hsuan Chen, Xia Zhu, Hejia Zhang, Javier S. Turek, Janice Chen, Theodore L. Willke, Uri Hasson, Peter J. Ramadge
Date: 2016-08-17 00:00:00
Tasks: Anatomy

Finding the most effective way to aggregate multi-subject fMRI data is a
long-standing and challenging problem. It is of increasing interest in
contemporary fMRI studies of human cognition due to the scarcity of data per
subject and the variability of brain anatomy and functional response across
subjects. Recent work on latent factor models shows promising results in this
task but this approach does not preserve spatial locality in the brain. We
examine two ways to combine the ideas of a factor model and a searchlight based
analysis to aggregate multi-subject fMRI data while preserving spatial
locality. We first do this directly by combining a recent factor method known
as a shared response model with searchlight analysis. Then we design a
multi-view convolutional autoencoder for the same task. Both approaches
preserve spatial locality and have competitive or better performance compared
with standard searchlight analysis and the shared response model applied across
the whole brain. We also report a system design to handle the computational
challenge of training the convolutional autoencoder.
Lecture Notes on Spectral Graph Methods
Authors: Michael W. Mahoney
Date: 2016-08-17 00:00:00

These are lecture notes that are based on the lectures from a class I taught
on the topic of Spectral Graph Methods at UC Berkeley during the Spring 2015
semester.
Outlier Detection on Mixed-Type Data: An Energy-based Approach
Authors: Kien Do, Truyen Tran, Dinh Phung, Svetha Venkatesh
Date: 2016-08-17 00:00:00
Tasks: Outlier Detection, Vocal Bursts Type Prediction

Outlier detection amounts to finding data points that differ significantly
from the norm. Classic outlier detection methods are largely designed for
single data type such as continuous or discrete. However, real world data is
increasingly heterogeneous, where a data point can have both discrete and
continuous attributes. Handling mixed-type data in a disciplined way remains a
great challenge. In this paper, we propose a new unsupervised outlier detection
method for mixed-type data based on Mixed-variate Restricted Boltzmann Machine
(Mv.RBM). The Mv.RBM is a principled probabilistic method that models data
density. We propose to use \emph{free-energy} derived from Mv.RBM as outlier
score to detect outliers as those data points lying in low density regions. The
method is fast to learn and compute, is scalable to massive datasets. At the
same time, the outlier score is identical to data negative log-density up-to an
additive constant. We evaluate the proposed method on synthetic and real-world
datasets and demonstrate that (a) a proper handling mixed-types is necessary in
outlier detection, and (b) free-energy of Mv.RBM is a powerful and efficient
outlier scoring method, which is highly competitive against state-of-the-arts.
Modelling Temporal Information Using Discrete Fourier Transform for Video Classification
Authors: Haimin Zhang
Date: 2016-03-20 00:00:00
Tasks: Action Recognition, Classification, Emotion Classification, General Classification, Temporal Action Localization, Video Classification

Recently, video classification attracts intensive research efforts. However,
most existing works are based on framelevel visual features, which might fail
to model the temporal information, e.g. characteristics accumulated along time.
In order to capture video temporal information, we propose to analyse features
in frequency domain transformed by discrete Fourier transform (DFT features).
Frame-level features are firstly extract by a pre-trained deep convolutional
neural network (CNN). Then, time domain features are transformed and
interpolated into DFT features. CNN and DFT features are further encoded by
using different pooling methods and fused for video classification. In this
way, static image features extracted from a pre-trained deep CNN and temporal
information represented by DFT features are jointly considered for video
classification. We test our method for video emotion classification and action
recognition. Experimental results demonstrate that combining DFT features can
effectively capture temporal information and therefore improve the performance
of both video emotion classification and action recognition. Our approach has
achieved a state-of-the-art performance on the largest video emotion dataset
(VideoEmotion-8 dataset) and competitive results on UCF-101.
Guaranteed bounds on the Kullback-Leibler divergence of univariate mixtures using piecewise log-sum-exp inequalities
Authors: Frank Nielsen, Ke Sun
Date: 2016-06-19 00:00:00
Tasks: Form

Information-theoretic measures such as the entropy, cross-entropy and the
Kullback-Leibler divergence between two mixture models is a core primitive in
many signal processing tasks. Since the Kullback-Leibler divergence of mixtures
provably does not admit a closed-form formula, it is in practice either
estimated using costly Monte-Carlo stochastic integration, approximated, or
bounded using various techniques. We present a fast and generic method that
builds algorithmically closed-form lower and upper bounds on the entropy, the
cross-entropy and the Kullback-Leibler divergence of mixtures. We illustrate
the versatile method by reporting on our experiments for approximating the
Kullback-Leibler divergence between univariate exponential mixtures, Gaussian
mixtures, Rayleigh mixtures, and Gamma mixtures.
Recovering Spatiotemporal Correspondence between Deformable Objects by Exploiting Consistent Foreground Motion in Video
Authors: Luca Del Pero, Susanna Ricco, Rahul Sukthankar, Vittorio Ferrari
Date: 2014-12-01 00:00:00
Tasks: Object

Given unstructured videos of deformable objects, we automatically recover
spatiotemporal correspondences to map one object to another (such as animals in
the wild). While traditional methods based on appearance fail in such
challenging conditions, we exploit consistency in object motion between
instances. Our approach discovers pairs of short video intervals where the
object moves in a consistent manner and uses these candidates as seeds for
spatial alignment. We model the spatial correspondence between the point
trajectories on the object in one interval to those in the other using a
time-varying Thin Plate Spline deformation model. On a large dataset of tiger
and horse videos, our method automatically aligns thousands of pairs of frames
to a high accuracy, and outperforms the popular SIFT Flow algorithm.
Modelling Student Behavior using Granular Large Scale Action Data from a MOOC
Authors: Steven Tang, Joshua C. Peterson, Zachary A. Pardos
Date: 2016-08-16 00:00:00
Tasks: Language Modelling, Sentence, Time Series Analysis

Digital learning environments generate a precise record of the actions
learners take as they interact with learning materials and complete exercises
towards comprehension. With this high quantity of sequential data comes the
potential to apply time series models to learn about underlying behavioral
patterns and trends that characterize successful learning based on the granular
record of student actions. There exist several methods for looking at
longitudinal, sequential data like those recorded from learning environments.
In the field of language modelling, traditional n-gram techniques and modern
recurrent neural network (RNN) approaches have been applied to algorithmically
find structure in language and predict the next word given the previous words
in the sentence or paragraph as input. In this paper, we draw an analogy to
this work by treating student sequences of resource views and interactions in a
MOOC as the inputs and predicting students' next interaction as outputs. In
this study, we train only on students who received a certificate of completion.
In doing so, the model could potentially be used for recommendation of
sequences eventually leading to success, as opposed to perpetuating
unproductive behavior. Given that the MOOC used in our study had over 3,500
unique resources, predicting the exact resource that a student will interact
with next might appear to be a difficult classification problem. We find that
simply following the syllabus (built-in structure of the course) gives on
average 23% accuracy in making this prediction, followed by the n-gram method
with 70.4%, and RNN based methods with 72.2%. This research lays the ground
work for recommendation in a MOOC and other digital learning environments where
high volumes of sequential data exist.
Application of multiview techniques to NHANES dataset
Authors: Aileme Omogbai
Date: 2016-08-16 00:00:00
Tasks: Classification, Disease Prediction, General Classification, Multiview Learning

Disease prediction or classification using health datasets involve using
well-known predictors associated with the disease as features for the models.
This study considers multiple data components of an individual's health, using
the relationship between variables to generate features that may improve the
performance of disease classification models. In order to capture information
from different aspects of the data, this project uses a multiview learning
approach, using Canonical Correlation Analysis (CCA), a technique that finds
projections with maximum correlations between two data views. Data categories
collected from the NHANES survey (1999-2014) are used as views to learn the
multiview representations. The usefulness of the representations is
demonstrated by applying them as features in a Diabetes classification task.
Faster Sublinear Algorithms using Conditional Sampling
Authors: Themistoklis Gouleakis, Christos Tzamos, Manolis Zampetakis
Date: 2016-08-16 00:00:00
Tasks: Clustering

A conditional sampling oracle for a probability distribution D returns
samples from the conditional distribution of D restricted to a specified subset
of the domain. A recent line of work (Chakraborty et al. 2013 and Cannone et
al. 2014) has shown that having access to such a conditional sampling oracle
requires only polylogarithmic or even constant number of samples to solve
distribution testing problems like identity and uniformity. This significantly
improves over the standard sampling model where polynomially many samples are
necessary.
  Inspired by these results, we introduce a computational model based on
conditional sampling to develop sublinear algorithms with exponentially faster
runtimes compared to standard sublinear algorithms. We focus on geometric
optimization problems over points in high dimensional Euclidean space. Access
to these points is provided via a conditional sampling oracle that takes as
input a succinct representation of a subset of the domain and outputs a
uniformly random point in that subset. We study two well studied problems:
k-means clustering and estimating the weight of the minimum spanning tree. In
contrast to prior algorithms for the classic model, our algorithms have time,
space and sample complexity that is polynomial in the dimension and
polylogarithmic in the number of points.
  Finally, we comment on the applicability of the model and compare with
existing ones like streaming, parallel and distributed computational models.
Horizon Lines in the Wild
Authors: Scott Workman, Menghua Zhai, Nathan Jacobs
Date: 2016-04-07 00:00:00
Tasks: Attribute, Horizon Line Estimation

The horizon line is an important contextual attribute for a wide variety of
image understanding tasks. As such, many methods have been proposed to estimate
its location from a single image. These methods typically require the image to
contain specific cues, such as vanishing points, coplanar circles, and regular
textures, thus limiting their real-world applicability. We introduce a large,
realistic evaluation dataset, Horizon Lines in the Wild (HLW), containing
natural images with labeled horizon lines. Using this dataset, we investigate
the application of convolutional neural networks for directly estimating the
horizon line, without requiring any explicit geometric constraints or other
special cues. An extensive evaluation shows that using our CNNs, either in
isolation or in conjunction with a previous geometric approach, we achieve
state-of-the-art results on the challenging HLW dataset and two existing
benchmark datasets.
Applying Deep Learning to Basketball Trajectories
Authors: Rajiv Shah, Rob Romijnders
Date: 2016-08-12 00:00:00
Tasks: BIG-bench Machine Learning, Deep Learning, Feature Engineering, Sports Analytics

One of the emerging trends for sports analytics is the growing use of player
and ball tracking data. A parallel development is deep learning predictive
approaches that use vast quantities of data with less reliance on feature
engineering. This paper applies recurrent neural networks in the form of
sequence modeling to predict whether a three-point shot is successful. The
models are capable of learning the trajectory of a basketball without any
knowledge of physics. For comparison, a baseline static machine learning model
with a full set of features, such as angle and velocity, in addition to the
positional data is also tested. Using a dataset of over 20,000 three pointers
from NBA SportVu data, the models based simply on sequential positional data
outperform a static feature rich machine learning model in predicting whether a
three-point shot is successful. This suggests deep learning models may offer an
improvement to traditional feature based machine learning methods for tracking
data.
A Data-Driven Approach to Estimating the Number of Clusters in Hierarchical Clustering
Authors: Antoine Zambelli
Date: 2016-08-16 00:00:00
Tasks: Clustering

We propose two new methods for estimating the number of clusters in a
hierarchical clustering framework in the hopes of creating a fully automated
process with no human intervention. The methods are completely data-driven and
require no input from the researcher, and as such are fully automated. They are
quite easy to implement and not computationally intensive in the least. We
analyze performance on several simulated data sets and the Biobase Gene
Expression Set, comparing our methods to the established Gap statistic and
Elbow methods and outperforming both in multi-cluster scenarios.
Evaluating Causal Models by Comparing Interventional Distributions
Authors: Dan Garant, David Jensen
Date: 2016-08-16 00:00:00

The predominant method for evaluating the quality of causal models is to
measure the graphical accuracy of the learned model structure. We present an
alternative method for evaluating causal models that directly measures the
accuracy of estimated interventional distributions. We contrast such
distributional measures with structural measures, such as structural Hamming
distance and structural intervention distance, showing that structural measures
often correspond poorly to the accuracy of estimated interventional
distributions. We use a number of real and synthetic datasets to illustrate
various scenarios in which structural measures provide misleading results with
respect to algorithm selection and parameter tuning, and we recommend that
distributional measures become the new standard for evaluating causal models.
A Shallow High-Order Parametric Approach to Data Visualization and Compression
Authors: Martin Renqiang Min, Hongyu Guo, Dongjin Song
Date: 2016-08-16 00:00:00
Tasks: Computational Efficiency, Data Visualization, General Classification, Information Retrieval, Retrieval, Vocal Bursts Intensity Prediction

Explicit high-order feature interactions efficiently capture essential
structural knowledge about the data of interest and have been used for
constructing generative models. We present a supervised discriminative
High-Order Parametric Embedding (HOPE) approach to data visualization and
compression. Compared to deep embedding models with complicated deep
architectures, HOPE generates more effective high-order feature mapping through
an embarrassingly simple shallow model. Furthermore, two approaches to
generating a small number of exemplars conveying high-order interactions to
represent large-scale data sets are proposed. These exemplars in combination
with the feature mapping learned by HOPE effectively capture essential data
variations. Moreover, through HOPE, these exemplars are employed to increase
the computational efficiency of kNN classification for fast information
retrieval by thousands of times. For classification in two-dimensional
embedding space on MNIST and USPS datasets, our shallow method HOPE with simple
Sigmoid transformations significantly outperforms state-of-the-art supervised
deep embedding models based on deep neural networks, and even achieved
historically low test error rate of 0.65% in two-dimensional space on MNIST,
which demonstrates the representational efficiency and power of supervised
shallow models with high-order feature interactions.
Shape Constrained Tensor Decompositions using Sparse Representations in Over-Complete Libraries
Authors: Bethany Lusch, Eric C. Chi, J. Nathan Kutz
Date: 2016-08-16 00:00:00
Tasks: Tensor Decomposition

We consider $N$-way data arrays and low-rank tensor factorizations where the
time mode is coded as a sparse linear combination of temporal elements from an
over-complete library. Our method, Shape Constrained Tensor Decomposition
(SCTD) is based upon the CANDECOMP/PARAFAC (CP) decomposition which produces
$r$-rank approximations of data tensors via outer products of vectors in each
dimension of the data. By constraining the vector in the temporal dimension to
known analytic forms which are selected from a large set of candidate
functions, more readily interpretable decompositions are achieved and analytic
time dependencies discovered. The SCTD method circumvents traditional {\em
flattening} techniques where an $N$-way array is reshaped into a matrix in
order to perform a singular value decomposition. A clear advantage of the SCTD
algorithm is its ability to extract transient and intermittent phenomena which
is often difficult for SVD-based methods. We motivate the SCTD method using
several intuitively appealing results before applying it on a number of
high-dimensional, real-world data sets in order to illustrate the efficiency of
the algorithm in extracting interpretable spatio-temporal modes. With the rise
of data-driven discovery methods, the decomposition proposed provides a viable
technique for analyzing multitudes of data in a more comprehensible fashion.
Temporally Consistent Motion Segmentation from RGB-D Video
Authors: Peter Bertholet, Alexandru-Eugen Ichim, Matthias Zwicker
Date: 2016-08-16 00:00:00
Tasks: Motion Segmentation, Object, Segmentation

We present a method for temporally consistent motion segmentation from RGB-D
videos assuming a piecewise rigid motion model. We formulate global energies
over entire RGB-D sequences in terms of the segmentation of each frame into a
number of objects, and the rigid motion of each object through the sequence. We
develop a novel initialization procedure that clusters feature tracks obtained
from the RGB data by leveraging the depth information. We minimize the energy
using a coordinate descent approach that includes novel techniques to assemble
object motion hypotheses. A main benefit of our approach is that it enables us
to fuse consistently labeled object segments from all RGB-D frames of an input
sequence into individual 3D object reconstructions.
Training Echo State Networks with Regularization through Dimensionality Reduction
Authors: Sigurd Løkse, Filippo Maria Bianchi, Robert Jenssen
Date: 2016-08-16 00:00:00
Tasks: Dimensionality Reduction, Time Series, Time Series Analysis

In this paper we introduce a new framework to train an Echo State Network to
predict real valued time-series. The method consists in projecting the output
of the internal layer of the network on a space with lower dimensionality,
before training the output layer to learn the target task. Notably, we enforce
a regularization constraint that leads to better generalization capabilities.
We evaluate the performances of our approach on several benchmark tests, using
different techniques to train the readout of the network, achieving superior
predictive performance when using the proposed framework. Finally, we provide
an insight on the effectiveness of the implemented mechanics through a
visualization of the trajectory in the phase space and relying on the
methodologies of nonlinear time-series analysis. By applying our method on well
known chaotic systems, we provide evidence that the lower dimensional embedding
retains the dynamical properties of the underlying system better than the
full-dimensional internal states of the network.