pwc-archive/papers-with-abstracts huggingface dataset:
8-Bit Approximations for Parallelism in Deep Learning
Authors: Tim Dettmers
Date: 2015-11-14 00:00:00
Tasks: Deep Learning

The creation of practical deep learning data-products often requires
parallelization across processors and computers to make deep learning feasible
on large data sets, but bottlenecks in communication bandwidth make it
difficult to attain good speedups through parallelism. Here we develop and test
8-bit approximation algorithms which make better use of the available bandwidth
by compressing 32-bit gradients and nonlinear activations to 8-bit
approximations. We show that these approximations do not decrease predictive
performance on MNIST, CIFAR10, and ImageNet for both model and data parallelism
and provide a data transfer speedup of 2x relative to 32-bit parallelism. We
build a predictive model for speedups based on our experimental data, verify
its validity on known speedup data, and show that we can obtain a speedup of
50x and more on a system of 96 GPUs compared to a speedup of 23x for 32-bit. We
compare our data types with other methods and show that 8-bit approximations
achieve state-of-the-art speedups for model parallelism. Thus 8-bit
approximation is an efficient method to parallelize convolutional networks on
very large systems of GPUs.
Node-By-Node Greedy Deep Learning for Interpretable Features
Authors: Ke Wu, Malik Magdon-Ismail
Date: 2016-02-19 00:00:00
Tasks: Deep Learning

Multilayer networks have seen a resurgence under the umbrella of deep
learning. Current deep learning algorithms train the layers of the network
sequentially, improving algorithmic performance as well as providing some
regularization. We present a new training algorithm for deep networks which
trains \emph{each node in the network} sequentially. Our algorithm is orders of
magnitude faster, creates more interpretable internal representations at the
node level, while not sacrificing on the ultimate out-of-sample performance.
TribeFlow: Mining & Predicting User Trajectories
Authors: Flavio Figueiredo, Bruno Ribeiro, Jussara Almeida, Christos Faloutsos
Date: 2015-11-03 00:00:00
Tasks: Prediction, Product Recommendation, Trajectory Prediction

Which song will Smith listen to next? Which restaurant will Alice go to
tomorrow? Which product will John click next? These applications have in common
the prediction of user trajectories that are in a constant state of flux over a
hidden network (e.g. website links, geographic location). What users are doing
now may be unrelated to what they will be doing in an hour from now. Mindful of
these challenges we propose TribeFlow, a method designed to cope with the
complex challenges of learning personalized predictive models of
non-stationary, transient, and time-heterogeneous user trajectories. TribeFlow
is a general method that can perform next product recommendation, next song
recommendation, next location prediction, and general arbitrary-length user
trajectory prediction without domain-specific knowledge. TribeFlow is more
accurate and up to 413x faster than top competitors.
Real Time Video Quality Representation Classification of Encrypted HTTP Adaptive Video Streaming - the Case of Safari
Authors: Ran Dubin, Amit Dvir, Ofir Pele, Ofer Hadar, Itay Richman, Ofir Trabelsi
Date: 2016-02-01 00:00:00
Tasks: Classification, General Classification, Traffic Classification

The increasing popularity of HTTP adaptive video streaming services has
dramatically increased bandwidth requirements on operator networks, which
attempt to shape their traffic through Deep Packet Inspection (DPI). However,
Google and certain content providers have started to encrypt their video
services. As a result, operators often encounter difficulties in shaping their
encrypted video traffic via DPI. This highlights the need for new traffic
classification methods for encrypted HTTP adaptive video streaming to enable
smart traffic shaping. These new methods will have to effectively estimate the
quality representation layer and playout buffer. We present a new method and
show for the first time that video quality representation classification for
(YouTube) encrypted HTTP adaptive streaming is possible. We analyze the
performance of this classification method with Safari over HTTPS. Based on a
large number of offline and online traffic classification experiments, we
demonstrate that it can independently classify, in real time, every video
segment into one of the quality representation layers with 97.18% average
accuracy.
All you need is a good init
Authors: Dmytro Mishkin, Jiri Matas
Date: 2015-11-19 00:00:00
Proceeding: ICLR 2015 11
Tasks: All, Image Classification

Layer-sequential unit-variance (LSUV) initialization - a simple method for
weight initialization for deep net learning - is proposed. The method consists
of the two steps. First, pre-initialize weights of each convolution or
inner-product layer with orthonormal matrices. Second, proceed from the first
to the final layer, normalizing the variance of the output of each layer to be
equal to one.
  Experiment with different activation functions (maxout, ReLU-family, tanh)
show that the proposed initialization leads to learning of very deep nets that
(i) produces networks with test accuracy better or equal to standard methods
and (ii) is at least as fast as the complex schemes proposed specifically for
very deep nets such as FitNets (Romero et al. (2015)) and Highway (Srivastava
et al. (2015)).
  Performance is evaluated on GoogLeNet, CaffeNet, FitNets and Residual nets
and the state-of-the-art, or very close to it, is achieved on the MNIST,
CIFAR-10/100 and ImageNet datasets.
Depth-Based Object Tracking Using a Robust Gaussian Filter
Authors: Jan Issac, Manuel Wüthrich, Cristina Garcia Cifuentes, Jeannette Bohg, Sebastian Trimpe, Stefan Schaal
Date: 2016-02-19 00:00:00
Tasks: Computational Efficiency, Object, Object Tracking, Outlier Detection

We consider the problem of model-based 3D-tracking of objects given dense
depth images as input. Two difficulties preclude the application of a standard
Gaussian filter to this problem. First of all, depth sensors are characterized
by fat-tailed measurement noise. To address this issue, we show how a recently
published robustification method for Gaussian filters can be applied to the
problem at hand. Thereby, we avoid using heuristic outlier detection methods
that simply reject measurements if they do not match the model. Secondly, the
computational cost of the standard Gaussian filter is prohibitive due to the
high-dimensional measurement, i.e. the depth image. To address this problem, we
propose an approximation to reduce the computational complexity of the filter.
In quantitative experiments on real data we show how our method clearly
outperforms the standard Gaussian filter. Furthermore, we compare its
performance to a particle-filter-based tracking method, and observe comparable
computational efficiency and improved accuracy and smoothness of the estimates.
Large age-gap face verification by feature injection in deep networks
Authors: Simone Bianco
Date: 2016-02-19 00:00:00
Tasks: Age-Invariant Face Recognition, Face Recognition, Face Verification

This paper introduces a new method for face verification across large age
gaps and also a dataset containing variations of age in the wild, the Large
Age-Gap (LAG) dataset, with images ranging from child/young to adult/old. The
proposed method exploits a deep convolutional neural network (DCNN) pre-trained
for the face recognition task on a large dataset and then fine-tuned for the
large age-gap face verification task. Finetuning is performed in a Siamese
architecture using a contrastive loss function. A feature injection layer is
introduced to boost verification accuracy, showing the ability of the DCNN to
learn a similarity metric leveraging external features. Experimental results on
the LAG dataset show that our method is able to outperform the face
verification solutions in the state of the art considered.
Ordonnancement d'entités pour la rencontre du web des documents et du web des données
Authors: Mazen Alsarem, Pierre-Edouard Portier, Sylvie Calabretto, Harald Kosch
Date: 2016-02-19 00:00:00
Tasks: Dimensionality Reduction

The advances of the Linked Open Data (LOD) initiative are giving rise to a
more structured web of data. Indeed, a few datasets act as hubs (e.g., DBpedia)
connecting many other datasets. They also made possible new web services for
entity detection inside plain text (e.g., DBpedia Spotlight), thus allowing for
new applications that will benefit from a combination of the web of documents
and the web of data. To ease the emergence of these new use-cases, we propose a
query-biased algorithm for the ranking of entities detected inside a web page.
Our algorithm combine link analysis with dimensionality reduction. We use
crowdsourcing for building a publicly available and reusable dataset on which
we compare our algorithm to the state of the art. Finally, we use this
algorithm for the construction of semantic snippets for which we evaluate the
usability and the usefulness with a crowdsourcing-based approach.
Descriptors and regions of interest fusion for gender classification in the wild. Comparison and combination with Convolutional Neural Networks
Authors: M. Castrillón-Santana, J. Lorenzo-Navarro, E. Ramón-Balmaseda
Date: 2015-07-24 00:00:00
Tasks: Gender Classification, General Classification, MORPH

Gender classification (GC) has achieved high accuracy in different
experimental evaluations based mostly on inner facial details. However, these
results do not generalize well in unrestricted datasets and particularly in
cross-database experiments, where the performance drops drastically. In this
paper, we analyze the state-of-the-art GC accuracy on three large datasets:
MORPH, LFW and GROUPS. We discuss their respective difficulties and bias,
concluding that the most challenging and wildest complexity is present in
GROUPS. This dataset covers hard conditions such as low resolution imagery and
cluttered background. Firstly, we analyze in depth the performance of different
descriptors extracted from the face and its local context on this dataset.
Selecting the bests and studying their most suitable combination allows us to
design a solution that beats any previously published results for GROUPS with
the Dago's protocol, reaching an accuracy over 94.2%, reducing the gap with
other simpler datasets. The chosen solution based on local descriptors is later
evaluated in a cross-database scenario with the three mentioned datasets, and
full dataset 5-fold cross validation. The achieved results are compared with a
Convolutional Neural Network approach, achieving rather similar marks. Finally,
a solution is proposed combining both focuses, exhibiting great
complementarity, boosting GC performance to beat previously published results
in GC both cross-database, and full in-database evaluations.
Alternative Markov and Causal Properties for Acyclic Directed Mixed Graphs
Authors: Jose M. Peña
Date: 2015-11-18 00:00:00

We extend Andersson-Madigan-Perlman chain graphs by (i) relaxing the
semidirected acyclity constraint so that only directed cycles are forbidden,
and (ii) allowing up to two edges between any pair of nodes. We introduce
global, and ordered local and pairwise Markov properties for the new models. We
show the equivalence of these properties for strictly positive probability
distributions. We also show that when the random variables are continuous, the
new models can be interpreted as systems of structural equations with
correlated errors. This enables us to adapt Pearl's do-calculus to them.
Finally, we describe an exact algorithm for learning the new models from
observational and interventional data via answer set programming.
Uniresolution representations of white-matter data from CoCoMac
Authors: Raghavendra Singh
Date: 2016-02-19 00:00:00

Tracing data as collated by CoCoMac, a seminal neuroinformatics database, is
at multiple resolutions -- white matter tracts were studied for areas and their
subdivisions by different reports. Network theoretic analysis of this
multi-resolution data often assumes that the data at various resolutions is
equivalent, which may not be correct. In this paper we propose three methods to
resolve the multi-resolution issue such that the resultant networks have
connectivity data at only one resolution. The different resultant networks are
compared in terms of their network analysis metrics and degree distributions.
Modularity Component Analysis versus Principal Component Analysis
Authors: Hansi Jiang, Carl Meyer
Date: 2015-10-19 00:00:00
Tasks: Relation

In this paper the exact linear relation between the leading eigenvectors of
the modularity matrix and the singular vectors of an uncentered data matrix is
developed. Based on this analysis the concept of a modularity component is
defined, and its properties are developed. It is shown that modularity
component analysis can be used to cluster data similar to how traditional
principal component analysis is used except that modularity component analysis
does not require data centering.
First-order Methods for Geodesically Convex Optimization
Authors: Hongyi Zhang, Suvrit Sra
Date: 2016-02-19 00:00:00

Geodesic convexity generalizes the notion of (vector space) convexity to
nonlinear metric spaces. But unlike convex optimization, geodesically convex
(g-convex) optimization is much less developed. In this paper we contribute to
the understanding of g-convex optimization by developing iteration complexity
analysis for several first-order algorithms on Hadamard manifolds.
Specifically, we prove upper bounds for the global complexity of deterministic
and stochastic (sub)gradient methods for optimizing smooth and nonsmooth
g-convex functions, both with and without strong g-convexity. Our analysis also
reveals how the manifold geometry, especially \emph{sectional curvature},
impacts convergence rates. To the best of our knowledge, our work is the first
to provide global complexity analysis for first-order algorithms for general
g-convex optimization.
Strong Backdoors for Default Logic
Authors: Johannes K. Fichte, Arne Meier, Irina Schindler
Date: 2016-02-19 00:00:00

In this paper, we introduce a notion of backdoors to Reiter's propositional
default logic and study structural properties of it. Also we consider the
problems of backdoor detection (parameterised by the solution size) as well as
backdoor evaluation (parameterised by the size of the given backdoor), for
various kinds of target classes (cnf, horn, krom, monotone, identity). We show
that backdoor detection is fixed-parameter tractable for the considered target
classes, and backdoor evaluation is either fixed-parameter tractable, in
para-DP2 , or in para-NP, depending on the target class.
Asynchronous Distributed ADMM for Large-Scale Optimization- Part I: Algorithm and Convergence Analysis
Authors: Tsung-Hui Chang, Mingyi Hong, Wei-Cheng Liao, Xiangfeng Wang
Date: 2015-09-09 00:00:00
Tasks: Distributed Optimization

Aiming at solving large-scale learning problems, this paper studies
distributed optimization methods based on the alternating direction method of
multipliers (ADMM). By formulating the learning problem as a consensus problem,
the ADMM can be used to solve the consensus problem in a fully parallel fashion
over a computer network with a star topology. However, traditional synchronized
computation does not scale well with the problem size, as the speed of the
algorithm is limited by the slowest workers. This is particularly true in a
heterogeneous network where the computing nodes experience different
computation and communication delays. In this paper, we propose an asynchronous
distributed ADMM (AD-AMM) which can effectively improve the time efficiency of
distributed optimization. Our main interest lies in analyzing the convergence
conditions of the AD-ADMM, under the popular partially asynchronous model,
which is defined based on a maximum tolerable delay of the network.
Specifically, by considering general and possibly non-convex cost functions, we
show that the AD-ADMM is guaranteed to converge to the set of
Karush-Kuhn-Tucker (KKT) points as long as the algorithm parameters are chosen
appropriately according to the network delay. We further illustrate that the
asynchrony of the ADMM has to be handled with care, as slightly modifying the
implementation of the AD-ADMM can jeopardize the algorithm convergence, even
under a standard convex setting.
Scaling up Dynamic Topic Models
Authors: Arnab Bhadury, Jianfei Chen, Jun Zhu, Shixia Liu
Date: 2016-02-19 00:00:00
Tasks: Time Series, Time Series Analysis, Topic Models

Dynamic topic models (DTMs) are very effective in discovering topics and
capturing their evolution trends in time series data. To do posterior inference
of DTMs, existing methods are all batch algorithms that scan the full dataset
before each update of the model and make inexact variational approximations
with mean-field assumptions. Due to a lack of a more scalable inference
algorithm, despite the usefulness, DTMs have not captured large topic dynamics.
  This paper fills this research void, and presents a fast and parallelizable
inference algorithm using Gibbs Sampling with Stochastic Gradient Langevin
Dynamics that does not make any unwarranted assumptions. We also present a
Metropolis-Hastings based $O(1)$ sampler for topic assignments for each word
token. In a distributed environment, our algorithm requires very little
communication between workers during sampling (almost embarrassingly parallel)
and scales up to large-scale applications. We are able to learn the largest
Dynamic Topic Model to our knowledge, and learned the dynamics of 1,000 topics
from 2.6 million documents in less than half an hour, and our empirical results
show that our algorithm is not only orders of magnitude faster than the
baselines but also achieves lower perplexity.
Spectral Learning for Supervised Topic Models
Authors: Yong Ren, Yining Wang, Jun Zhu
Date: 2016-02-19 00:00:00
Tasks: regression, Topic Models

Supervised topic models simultaneously model the latent topic structure of
large collections of documents and a response variable associated with each
document. Existing inference methods are based on variational approximation or
Monte Carlo sampling, which often suffers from the local minimum defect.
Spectral methods have been applied to learn unsupervised topic models, such as
latent Dirichlet allocation (LDA), with provable guarantees. This paper
investigates the possibility of applying spectral methods to recover the
parameters of supervised LDA (sLDA). We first present a two-stage spectral
method, which recovers the parameters of LDA followed by a power update method
to recover the regression model parameters. Then, we further present a
single-phase spectral algorithm to jointly recover the topic distribution
matrix as well as the regression weights. Our spectral algorithms are provably
correct and computationally efficient. We prove a sample complexity bound for
each algorithm and subsequently derive a sufficient condition for the
identifiability of sLDA. Thorough experiments on synthetic and real-world
datasets verify the theory and demonstrate the practical effectiveness of the
spectral algorithms. In fact, our results on a large-scale review rating
dataset demonstrate that our single-phase spectral algorithm alone gets
comparable or even better performance than state-of-the-art methods, while
previous work on spectral methods has rarely reported such promising
performance.
Structured illumination microscopy image reconstruction algorithm
Authors: Amit Lal, Chunyan Shan, Peng Xi
Date: 2016-02-19 00:00:00
Tasks: Image Reconstruction, Super-Resolution

Structured illumination microscopy (SIM) is a very important super-resolution
microscopy technique, which provides high speed super-resolution with about
two-fold spatial resolution enhancement. Several attempts aimed at improving
the performance of SIM reconstruction algorithm have been reported. However,
most of these highlight only one specific aspect of the SIM reconstruction --
such as the determination of the illumination pattern phase shift accurately --
whereas other key elements -- such as determination of modulation factor,
estimation of object power spectrum, Wiener filtering frequency components with
inclusion of object power spectrum information, translocating and the merging
of the overlapping frequency components -- are usually glossed over
superficially. In addition, most of the work reported lie scattered throughout
the literature and a comprehensive review of the theoretical background is
found lacking. The purpose of the present work is two-fold: 1) to collect the
essential theoretical details of SIM algorithm at one place, thereby making
them readily accessible to readers for the first time; and 2) to provide an
open source SIM reconstruction code (named OpenSIM), which enables users to
interactively vary the code parameters and study it's effect on reconstructed
SIM image.
Ground-truth dataset and baseline evaluations for image base-detail separation algorithms
Authors: Xuan Dong, Boyan Bonev, Weixin Li, Weichao Qiu, Xianjie Chen, Alan Yuille
Date: 2015-11-21 00:00:00

Base-detail separation is a fundamental computer vision problem consisting of
modeling a smooth base layer with the coarse structures, and a detail layer
containing the texture-like structures. One of the challenges of estimating the
base is to preserve sharp boundaries between objects or parts to avoid halo
artifacts. Many methods have been proposed to address this problem, but there
is no ground-truth dataset of real images for quantitative evaluation. We
proposed a procedure to construct such a dataset, and provide two datasets:
Pascal Base-Detail and Fashionista Base-Detail, containing 1000 and 250 images,
respectively. Our assumption is that the base is piecewise smooth and we label
the appearance of each piece by a polynomial model. The pieces are objects and
parts of objects, obtained from human annotations. Finally, we proposed a way
to evaluate methods with our base-detail ground-truth and we compared the
performances of seven state-of-the-art algorithms.
A Nonparametric Framework for Quantifying Generative Inference on Neuromorphic Systems
Authors: Ojash Neopane, Srinjoy Das, Ery Arias-Castro, Kenneth Kreutz-Delgado
Date: 2016-02-18 00:00:00
Tasks: Motion Synthesis

Restricted Boltzmann Machines and Deep Belief Networks have been successfully
used in probabilistic generative model applications such as image occlusion
removal, pattern completion and motion synthesis. Generative inference in such
algorithms can be performed very efficiently on hardware using a Markov Chain
Monte Carlo procedure called Gibbs sampling, where stochastic samples are drawn
from noisy integrate and fire neurons implemented on neuromorphic substrates.
Currently, no satisfactory metrics exist for evaluating the generative
performance of such algorithms implemented on high-dimensional data for
neuromorphic platforms. This paper demonstrates the application of
nonparametric goodness-of-fit testing to both quantify the generative
performance as well as provide decision-directed criteria for choosing the
parameters of the neuromorphic Gibbs sampler and optimizing usage of hardware
resources used during sampling.