pwc-archive/papers-with-abstracts huggingface dataset:
Training Deep Gaussian Processes using Stochastic Expectation Propagation and Probabilistic Backpropagation
Authors: Thang D. Bui, José Miguel Hernández-Lobato, Yingzhen Li, Daniel Hernández-Lobato, Richard E. Turner
Date: 2015-11-11 00:00:00
Tasks: Gaussian Processes

Deep Gaussian processes (DGPs) are multi-layer hierarchical generalisations
of Gaussian processes (GPs) and are formally equivalent to neural networks with
multiple, infinitely wide hidden layers. DGPs are probabilistic and
non-parametric and as such are arguably more flexible, have a greater capacity
to generalise, and provide better calibrated uncertainty estimates than
alternative deep models. The focus of this paper is scalable approximate
Bayesian learning of these networks. The paper develops a novel and efficient
extension of probabilistic backpropagation, a state-of-the-art method for
training Bayesian neural networks, that can be used to train DGPs. The new
method leverages a recently proposed method for scaling Expectation
Propagation, called stochastic Expectation Propagation. The method is able to
automatically discover useful input warping, expansion or compression, and it
is therefore is a flexible form of Bayesian kernel design. We demonstrate the
success of the new method for supervised learning on several real-world
datasets, showing that it typically outperforms GP regression and is never much
worse.
A GMM-Based Stair Quality Model for Human Perceived JPEG Images
Authors: Sudeng Hu, Haiqiang Wang, C. -C. Jay Kuo
Date: 2015-11-11 00:00:00
Tasks: Clustering

Based on the notion of just noticeable differences (JND), a stair quality
function (SQF) was recently proposed to model human perception on JPEG images.
Furthermore, a k-means clustering algorithm was adopted to aggregate JND data
collected from multiple subjects to generate a single SQF. In this work, we
propose a new method to derive the SQF using the Gaussian Mixture Model (GMM).
The newly derived SQF can be interpreted as a way to characterize the mean
viewer experience. Furthermore, it has a lower information criterion (BIC)
value than the previous one, indicating that it offers a better model. A
specific example is given to demonstrate the advantages of the new approach.
High Performance Latent Variable Models
Authors: Aaron Q. Li, Amr Ahmed, Mu Li, Vanja Josifovski
Date: 2015-10-21 00:00:00
Tasks: Vocal Bursts Intensity Prediction

Latent variable models have accumulated a considerable amount of interest
from the industry and academia for their versatility in a wide range of
applications. A large amount of effort has been made to develop systems that is
able to extend the systems to a large scale, in the hope to make use of them on
industry scale data. In this paper, we describe a system that operates at a
scale orders of magnitude higher than previous works, and an order of magnitude
faster than state-of-the-art system at the same scale, at the same time showing
more robustness and more accurate results.
  Our system uses a number of advances in distributed inference: high
performance in synchronization of sufficient statistics with relaxed
consistency model; fast sampling, using the Metropolis-Hastings-Walker method
to overcome dense generative models; statistical modeling, moving beyond Latent
Dirichlet Allocation (LDA) to Pitman-Yor distributions (PDP) and Hierarchical
Dirichlet Process (HDP) models; sophisticated parameter projection schemes, to
resolve the conflicts within the constraint between parameters arising from the
relaxed consistency model.
  This work significantly extends the domain of applicability of what is
commonly known as the Parameter Server. We obtain results with up to hundreds
billion oftokens, thousands of topics, and a vocabulary of a few million
token-types, using up to 60,000 processor cores operating on a production
cluster of a large Internet company. This demonstrates the feasibility to scale
to problems orders of magnitude larger than any previously published work.
Instance Optimal Learning
Authors: Gregory Valiant, Paul Valiant
Date: 2015-04-21 00:00:00

We consider the following basic learning task: given independent draws from
an unknown distribution over a discrete support, output an approximation of the
distribution that is as accurate as possible in $\ell_1$ distance (i.e. total
variation or statistical distance). Perhaps surprisingly, it is often possible
to "de-noise" the empirical distribution of the samples to return an
approximation of the true distribution that is significantly more accurate than
the empirical distribution, without relying on any prior assumptions on the
distribution. We present an instance optimal learning algorithm which optimally
performs this de-noising for every distribution for which such a de-noising is
possible. More formally, given $n$ independent draws from a distribution $p$,
our algorithm returns a labelled vector whose expected distance from $p$ is
equal to the minimum possible expected error that could be obtained by any
algorithm that knows the true unlabeled vector of probabilities of distribution
$p$ and simply needs to assign labels, up to an additive subconstant term that
is independent of $p$ and goes to zero as $n$ gets large. One conceptual
implication of this result is that for large samples, Bayesian assumptions on
the "shape" or bounds on the tail probabilities of a distribution over discrete
support are not helpful for the task of learning the distribution.
  As a consequence of our techniques, we also show that given a set of $n$
samples from an arbitrary distribution, one can accurately estimate the
expected number of distinct elements that will be observed in a sample of any
size up to $n \log n$. This sort of extrapolation is practically relevant,
particularly to domains such as genomics where it is important to understand
how much more might be discovered given larger sample sizes, and we are
optimistic that our approach is practically viable.
Multimodal MRI Neuroimaging with Motion Compensation Based on Particle Filtering
Authors: Yu-Hui Chen, Roni Mittelman, Boklye Kim, Charles Meyer, Alfred Hero
Date: 2015-11-11 00:00:00
Tasks: Image Registration, Motion Compensation, Motion Estimation

Head movement during scanning impedes activation detection in fMRI studies.
Head motion in fMRI acquired using slice-based Echo Planar Imaging (EPI) can be
estimated and compensated by aligning the images onto a reference volume
through image registration. However, registering EPI images volume to volume
fails to consider head motion between slices, which may lead to severely biased
head motion estimates. Slice-to-volume registration can be used to estimate
motion parameters for each slice by more accurately representing the image
acquisition sequence. However, accurate slice to volume mapping is dependent on
the information content of the slices: middle slices are information rich,
while edge slides are information poor and more prone to distortion. In this
work, we propose a Gaussian particle filter based head motion tracking
algorithm to reduce the image misregistration errors. The algorithm uses a
dynamic state space model of head motion with an observation equation that
models continuous slice acquisition of the scanner. Under this model the
particle filter provides more accurate motion estimates and voxel position
estimates. We demonstrate significant performance improvement of the proposed
approach as compared to registration-only methods of head motion estimation and
brain activation detection.
Facial Expression Detection using Patch-based Eigen-face Isomap Networks
Authors: Sohini Roychowdhury
Date: 2015-11-11 00:00:00
Tasks: Classification, Clustering, General Classification

Automated facial expression detection problem pose two primary challenges
that include variations in expression and facial occlusions (glasses, beard,
mustache or face covers). In this paper we introduce a novel automated patch
creation technique that masks a particular region of interest in the face,
followed by Eigen-value decomposition of the patched faces and generation of
Isomaps to detect underlying clustering patterns among faces. The proposed
masked Eigen-face based Isomap clustering technique achieves 75% sensitivity
and 66-73% accuracy in classification of faces with occlusions and smiling
faces in around 1 second per image. Also, betweenness centrality, Eigen
centrality and maximum information flow can be used as network-based measures
to identify the most significant training faces for expression classification
tasks. The proposed method can be used in combination with feature-based
expression classification methods in large data sets for improving expression
classification accuracies.
Discovery Radiomics via StochasticNet Sequencers for Cancer Detection
Authors: Mohammad Javad Shafiee, Audrey G. Chung, Devinder Kumar, Farzad Khalvati, Masoom Haider, Alexander Wong
Date: 2015-11-11 00:00:00
Tasks: Binary Classification

Radiomics has proven to be a powerful prognostic tool for cancer detection,
and has previously been applied in lung, breast, prostate, and head-and-neck
cancer studies with great success. However, these radiomics-driven methods rely
on pre-defined, hand-crafted radiomic feature sets that can limit their ability
to characterize unique cancer traits. In this study, we introduce a novel
discovery radiomics framework where we directly discover custom radiomic
features from the wealth of available medical imaging data. In particular, we
leverage novel StochasticNet radiomic sequencers for extracting custom radiomic
features tailored for characterizing unique cancer tissue phenotype. Using
StochasticNet radiomic sequencers discovered using a wealth of lung CT data, we
perform binary classification on 42,340 lung lesions obtained from the CT scans
of 93 patients in the LIDC-IDRI dataset. Preliminary results show significant
improvement over previous state-of-the-art methods, indicating the potential of
the proposed discovery radiomics framework for improving cancer screening and
diagnosis.
Principal Autoparallel Analysis: Data Analysis in Weitzenböck Space
Authors: Stephen Marsland, Carole J. Twining
Date: 2015-11-11 00:00:00

The statistical analysis of data lying on a differentiable, locally
Euclidean, manifold introduces a variety of challenges because the analogous
measures to standard Euclidean statistics are local, that is only defined
within a neighbourhood of each datapoint. This is because the curvature of the
space means that the connection of Riemannian geometry is path dependent. In
this paper we transfer the problem to Weitzenb\"{o}ck space, which has torsion,
but not curvature, meaning that parallel transport is path independent, and
rather than considering geodesics, it is natural to consider autoparallels,
which are `straight' in the sense that they follow the local basis vectors. We
demonstrate how to generate these autoparallels in a data-driven fashion, and
show that the resulting representation of the data is a useful space in which
to perform further analysis.
Fusing Multi-Stream Deep Networks for Video Classification
Authors: Zuxuan Wu, Yu-Gang Jiang, Xi Wang, Hao Ye, xiangyang xue, Jun Wang
Date: 2015-09-21 00:00:00
Tasks: Classification, General Classification, Video Classification

This paper studies deep network architectures to address the problem of video
classification. A multi-stream framework is proposed to fully utilize the rich
multimodal information in videos. Specifically, we first train three
Convolutional Neural Networks to model spatial, short-term motion and audio
clues respectively. Long Short Term Memory networks are then adopted to explore
long-term temporal dynamics. With the outputs of the individual streams, we
propose a simple and effective fusion method to generate the final predictions,
where the optimal fusion weights are learned adaptively for each class, and the
learning process is regularized by automatically estimated class relationships.
Our contributions are two-fold. First, the proposed multi-stream framework is
able to exploit multimodal features that are more comprehensive than those
previously attempted. Second, we demonstrate that the adaptive fusion method
using the class relationship as a regularizer outperforms traditional
alternatives that estimate the weights in a "free" fashion. Our framework
produces significantly better results than the state of the arts on two popular
benchmarks, 92.2\% on UCF-101 (without using audio) and 84.9\% on Columbia
Consumer Videos.
Scalable Semi-Supervised Aggregation of Classifiers
Authors: Akshay Balsubramani, Yoav Freund
Date: 2015-06-18 00:00:00
Proceeding: NeurIPS 2015 12

We present and empirically evaluate an efficient algorithm that learns to
aggregate the predictions of an ensemble of binary classifiers. The algorithm
uses the structure of the ensemble predictions on unlabeled data to yield
significant performance improvements. It does this without making assumptions
on the structure or origin of the ensemble, without parameters, and as scalably
as linear learning. We empirically demonstrate these performance gains with
random forests.
Anchored Discrete Factor Analysis
Authors: Yoni Halpern, Steven Horng, David Sontag
Date: 2015-11-10 00:00:00
Tasks: Medical Diagnosis, TAG

We present a semi-supervised learning algorithm for learning discrete factor
analysis models with arbitrary structure on the latent variables. Our algorithm
assumes that every latent variable has an "anchor", an observed variable with
only that latent variable as its parent. Given such anchors, we show that it is
possible to consistently recover moments of the latent variables and use these
moments to learn complete models. We also introduce a new technique for
improving the robustness of method-of-moment algorithms by optimizing over the
marginal polytope or its relaxations. We evaluate our algorithm using two
real-world tasks, tag prediction on questions from the Stack Overflow website
and medical diagnosis in an emergency department.
From Images to Sentences through Scene Description Graphs using Commonsense Reasoning and Knowledge
Authors: Somak Aditya, Yezhou Yang, Chitta Baral, Cornelia Fermuller, Yiannis Aloimonos
Date: 2015-11-10 00:00:00
Tasks: image-sentence alignment, Sentence

In this paper we propose the construction of linguistic descriptions of
images. This is achieved through the extraction of scene description graphs
(SDGs) from visual scenes using an automatically constructed knowledge base.
SDGs are constructed using both vision and reasoning. Specifically, commonsense
reasoning is applied on (a) detections obtained from existing perception
methods on given images, (b) a "commonsense" knowledge base constructed using
natural language processing of image annotations and (c) lexical ontological
knowledge from resources such as WordNet. Amazon Mechanical Turk(AMT)-based
evaluations on Flickr8k, Flickr30k and MS-COCO datasets show that in most
cases, sentences auto-constructed from SDGs obtained by our method give a more
relevant and thorough description of an image than a recent state-of-the-art
image caption based approach. Our Image-Sentence Alignment Evaluation results
are also comparable to that of the recent state-of-the art approaches.
Online Supervised Hashing for Ever-Growing Datasets
Authors: Fatih Cakir, Sarah Adel Bargal, Stan Sclaroff
Date: 2015-11-10 00:00:00

Supervised hashing methods are widely-used for nearest neighbor search in
computer vision applications. Most state-of-the-art supervised hashing
approaches employ batch-learners. Unfortunately, batch-learning strategies can
be inefficient when confronted with large training datasets. Moreover, with
batch-learners, it is unclear how to adapt the hash functions as a dataset
continues to grow and diversify over time. Yet, in many practical scenarios the
dataset grows and diversifies; thus, both the hash functions and the indexing
must swiftly accommodate these changes. To address these issues, we propose an
online hashing method that is amenable to changes and expansions of the
datasets. Since it is an online algorithm, our approach offers linear
complexity with the dataset size. Our solution is supervised, in that we
incorporate available label information to preserve the semantic neighborhood.
Such an adaptive hashing method is attractive; but it requires recomputing the
hash table as the hash functions are updated. If the frequency of update is
high, then recomputing the hash table entries may cause inefficiencies in the
system, especially for large indexes. Thus, we also propose a framework to
reduce hash table updates. We compare our method to state-of-the-art solutions
on two benchmarks and demonstrate significant improvements over previous work.
StochasticNet: Forming Deep Neural Networks via Stochastic Connectivity
Authors: Mohammad Javad Shafiee, Parthipan Siva, Alexander Wong
Date: 2015-08-22 00:00:00
Tasks: Functional Connectivity

Deep neural networks is a branch in machine learning that has seen a meteoric
rise in popularity due to its powerful abilities to represent and model
high-level abstractions in highly complex data. One area in deep neural
networks that is ripe for exploration is neural connectivity formation. A
pivotal study on the brain tissue of rats found that synaptic formation for
specific functional connectivity in neocortical neural microcircuits can be
surprisingly well modeled and predicted as a random formation. Motivated by
this intriguing finding, we introduce the concept of StochasticNet, where deep
neural networks are formed via stochastic connectivity between neurons. As a
result, any type of deep neural networks can be formed as a StochasticNet by
allowing the neuron connectivity to be stochastic. Stochastic synaptic
formations, in a deep neural network architecture, can allow for efficient
utilization of neurons for performing specific tasks. To evaluate the
feasibility of such a deep neural network architecture, we train a
StochasticNet using four different image datasets (CIFAR-10, MNIST, SVHN, and
STL-10). Experimental results show that a StochasticNet, using less than half
the number of neural connections as a conventional deep neural network,
achieves comparable accuracy and reduces overfitting on the CIFAR-10, MNIST and
SVHN dataset. Interestingly, StochasticNet with less than half the number of
neural connections, achieved a higher accuracy (relative improvement in test
error rate of ~6% compared to ConvNet) on the STL-10 dataset than a
conventional deep neural network. Finally, StochasticNets have faster
operational speeds while achieving better or similar accuracy performances.
Are Slepian-Wolf Rates Necessary for Distributed Parameter Estimation?
Authors: Mostafa El Gamal, Lifeng Lai
Date: 2015-08-11 00:00:00
Tasks: parameter estimation

We consider a distributed parameter estimation problem, in which multiple
terminals send messages related to their local observations using limited rates
to a fusion center who will obtain an estimate of a parameter related to
observations of all terminals. It is well known that if the transmission rates
are in the Slepian-Wolf region, the fusion center can fully recover all
observations and hence can construct an estimator having the same performance
as that of the centralized case. One natural question is whether Slepian-Wolf
rates are necessary to achieve the same estimation performance as that of the
centralized case. In this paper, we show that the answer to this question is
negative. We establish our result by explicitly constructing an asymptotically
minimum variance unbiased estimator (MVUE) that has the same performance as
that of the optimal estimator in the centralized case while requiring
information rates less than the conditions required in the Slepian-Wolf rate
region.
Fast Algorithms for Convolutional Neural Networks
Authors: Andrew Lavin, Scott Gray
Date: 2015-09-30 00:00:00
Proceeding: CVPR 2016 6
Tasks: GPU, Pedestrian Detection, Self-Driving Cars

Deep convolutional neural networks take GPU days of compute time to train on
large data sets. Pedestrian detection for self driving cars requires very low
latency. Image recognition for mobile phones is constrained by limited
processing resources. The success of convolutional neural networks in these
situations is limited by how fast we can compute them. Conventional FFT based
convolution is fast for large filters, but state of the art convolutional
neural networks use small, 3x3 filters. We introduce a new class of fast
algorithms for convolutional neural networks using Winograd's minimal filtering
algorithms. The algorithms compute minimal complexity convolution over small
tiles, which makes them fast with small filters and small batch sizes. We
benchmark a GPU implementation of our algorithm with the VGG network and show
state of the art throughput at batch sizes from 1 to 64.
A Distributed One-Step Estimator
Authors: Cheng Huang, Xiaoming Huo
Date: 2015-11-04 00:00:00

Distributed statistical inference has recently attracted enormous attention.
Many existing work focuses on the averaging estimator. We propose a one-step
approach to enhance a simple-averaging based distributed estimator. We derive
the corresponding asymptotic properties of the newly proposed estimator. We
find that the proposed one-step estimator enjoys the same asymptotic properties
as the centralized estimator. The proposed one-step approach merely requires
one additional round of communication in relative to the averaging estimator;
so the extra communication burden is insignificant. In finite sample cases,
numerical examples show that the proposed estimator outperforms the simple
averaging estimator with a large margin in terms of the mean squared errors. A
potential application of the one-step approach is that one can use multiple
machines to speed up large scale statistical inference with little compromise
in the quality of estimators. The proposed method becomes more valuable when
data can only be available at distributed machines with limited communication
bandwidth.
TemplateNet for Depth-Based Object Instance Recognition
Authors: Ujwal Bonde, Vijay Badrinarayanan, Roberto Cipolla, Minh-Tri Pham
Date: 2015-11-10 00:00:00
Tasks: Object

We present a novel deep architecture termed templateNet for depth based
object instance recognition. Using an intermediate template layer we exploit
prior knowledge of an object's shape to sparsify the feature maps. This has
three advantages: (i) the network is better regularised resulting in structured
filters; (ii) the sparse feature maps results in intuitive features been learnt
which can be visualized as the output of the template layer and (iii) the
resulting network achieves state-of-the-art performance. The network benefits
from this without any additional parametrization from the template layer. We
derive the weight updates needed to efficiently train this network in an
end-to-end manner. We benchmark the templateNet for depth based object instance
recognition using two publicly available datasets. The datasets present
multiple challenges of clutter, large pose variations and similar looking
distractors. Through our experiments we show that with the addition of a
template layer, a depth based CNN is able to outperform existing
state-of-the-art methods in the field.