pwc-archive/papers-with-abstracts huggingface dataset:
Poseidon: An Efficient Communication Architecture for Distributed Deep Learning on GPU Clusters
Authors: Hao Zhang, Zeyu Zheng, Shizhen Xu, Wei Dai, Qirong Ho, Xiaodan Liang, Zhiting Hu, Jinliang Wei, Pengtao Xie, Eric P. Xing
Date: 2017-06-11 00:00:00
Tasks: GPU, image-classification, Image Classification

Deep learning models can take weeks to train on a single GPU-equipped
machine, necessitating scaling out DL training to a GPU-cluster. However,
current distributed DL implementations can scale poorly due to substantial
parameter synchronization over the network, because the high throughput of GPUs
allows more data batches to be processed per unit time than CPUs, leading to
more frequent network synchronization. We present Poseidon, an efficient
communication architecture for distributed DL on GPUs. Poseidon exploits the
layered model structures in DL programs to overlap communication and
computation, reducing bursty network communication. Moreover, Poseidon uses a
hybrid communication scheme that optimizes the number of bytes required to
synchronize each layer, according to layer properties and the number of
machines. We show that Poseidon is applicable to different DL frameworks by
plugging Poseidon into Caffe and TensorFlow. We show that Poseidon enables
Caffe and TensorFlow to achieve 15.5x speed-up on 16 single-GPU machines, even
with limited bandwidth (10GbE) and the challenging VGG19-22K network for image
classification. Moreover, Poseidon-enabled TensorFlow achieves 31.5x speed-up
with 32 single-GPU machines on Inception-V3, a 50% improvement over the
open-source TensorFlow (20x speed-up).
Recovering 6D Object Pose: A Review and Multi-modal Analysis
Authors: Caner Sahin, Tae-Kyun Kim
Date: 2017-06-10 00:00:00
Tasks: 6D Pose Estimation, 6D Pose Estimation using RGB, Object, object-detection, Object Detection, Pose Estimation

A large number of studies analyse object detection and pose estimation at
visual level in 2D, discussing the effects of challenges such as occlusion,
clutter, texture, etc., on the performances of the methods, which work in the
context of RGB modality. Interpreting the depth data, the study in this paper
presents thorough multi-modal analyses. It discusses the above-mentioned
challenges for full 6D object pose estimation in RGB-D images comparing the
performances of several 6D detectors in order to answer the following
questions: What is the current position of the computer vision community for
maintaining "automation" in robotic manipulation? What next steps should the
community take for improving "autonomy" in robotics while handling objects? Our
findings include: (i) reasonably accurate results are obtained on
textured-objects at varying viewpoints with cluttered backgrounds. (ii) Heavy
existence of occlusion and clutter severely affects the detectors, and
similar-looking distractors is the biggest challenge in recovering instances'
6D. (iii) Template-based methods and random forest-based learning algorithms
underlie object detection and 6D pose estimation. Recent paradigm is to learn
deep discriminative feature representations and to adopt CNNs taking RGB images
as input. (iv) Depending on the availability of large-scale 6D annotated depth
datasets, feature representations can be learnt on these datasets, and then the
learnt representations can be customized for the 6D problem.
Deep Recurrent Neural Networks for seizure detection and early seizure detection systems
Authors: Sachin S. Talathi
Date: 2017-06-10 00:00:00
Tasks: EEG, Electroencephalogram (EEG), Seizure Detection

Epilepsy is common neurological diseases, affecting about 0.6-0.8 % of world
population. Epileptic patients suffer from chronic unprovoked seizures, which
can result in broad spectrum of debilitating medical and social consequences.
Since seizures, in general, occur infrequently and are unpredictable, automated
seizure detection systems are recommended to screen for seizures during
long-term electroencephalogram (EEG) recordings. In addition, systems for early
seizure detection can lead to the development of new types of intervention
systems that are designed to control or shorten the duration of seizure events.
In this article, we investigate the utility of recurrent neural networks (RNNs)
in designing seizure detection and early seizure detection systems. We propose
a deep learning framework via the use of Gated Recurrent Unit (GRU) RNNs for
seizure detection. We use publicly available data in order to evaluate our
method and demonstrate very promising evaluation results with overall accuracy
close to 100 %. We also systematically investigate the application of our
method for early seizure warning systems. Our method can detect about 98% of
seizure events within the first 5 seconds of the overall epileptic seizure
duration.
Multidimensional Dynamic Pricing for Welfare Maximization
Authors: Aaron Roth, Aleksandrs Slivkins, Jonathan Ullman, Zhiwei Steven Wu
Date: 2016-07-19 00:00:00

We study the problem of a seller dynamically pricing $d$ distinct types of
indivisible goods, when faced with the online arrival of unit-demand buyers
drawn independently from an unknown distribution. The goods are not in limited
supply, but can only be produced at a limited rate and are costly to produce.
The seller observes only the bundle of goods purchased at each day, but nothing
else about the buyer's valuation function. Our main result is a dynamic pricing
algorithm for optimizing welfare (including the seller's cost of production)
that runs in time and a number of rounds that are polynomial in $d$ and the
approximation parameter. We are able to do this despite the fact that (i) the
price-response function is not continuous, and even its fractional relaxation
is a non-concave function of the prices, and (ii) the welfare is not observable
to the seller.
  We derive this result as an application of a general technique for optimizing
welfare over \emph{divisible} goods, which is of independent interest. When
buyers have strongly concave, H\"older continuous valuation functions over $d$
divisible goods, we give a general polynomial time dynamic pricing technique.
We are able to apply this technique to the setting of unit demand buyers
despite the fact that in that setting the goods are not divisible, and the
natural fractional relaxation of a unit demand valuation is not strongly
concave. In order to apply our general technique, we introduce a novel price
randomization procedure which has the effect of implicitly inducing buyers to
"regularize" their valuations with a strongly concave function. Finally, we
also extend our results to a limited-supply setting in which the number of
copies of each good cannot be replenished.
Learning Continuous Semantic Representations of Symbolic Expressions
Authors: Miltiadis Allamanis, Pankajan Chanthirasegaran, Pushmeet Kohli, Charles Sutton
Date: 2016-11-04 00:00:00
Proceeding: ICML 2017 8
Tasks: Representation Learning

Combining abstract, symbolic reasoning with continuous neural reasoning is a
grand challenge of representation learning. As a step in this direction, we
propose a new architecture, called neural equivalence networks, for the problem
of learning continuous semantic representations of algebraic and logical
expressions. These networks are trained to represent semantic equivalence, even
of expressions that are syntactically very different. The challenge is that
semantic representations must be computed in a syntax-directed manner, because
semantics is compositional, but at the same time, small changes in syntax can
lead to very large changes in semantics, which can be difficult for continuous
neural architectures. We perform an exhaustive evaluation on the task of
checking equivalence on a highly diverse class of symbolic algebraic and
boolean expression types, showing that our model significantly outperforms
existing architectures.
An Online Learning Approach to Generative Adversarial Networks
Authors: Paulina Grnarova, Kfir. Y. Levy, Aurelien Lucchi, Thomas Hofmann, Andreas Krause
Date: 2017-06-10 00:00:00
Proceeding: ICLR 2018 1
Tasks: Generative Adversarial Network

We consider the problem of training generative models with a Generative
Adversarial Network (GAN). Although GANs can accurately model complex
distributions, they are known to be difficult to train due to instabilities
caused by a difficult minimax optimization problem. In this paper, we view the
problem of training GANs as finding a mixed strategy in a zero-sum game.
Building on ideas from online learning we propose a novel training method named
Chekhov GAN 1 . On the theory side, we show that our method provably converges
to an equilibrium for semi-shallow GAN architectures, i.e. architectures where
the discriminator is a one layer network and the generator is arbitrary. On the
practical side, we develop an efficient heuristic guided by our theoretical
results, which we apply to commonly used deep GAN architectures. On several
real world tasks our approach exhibits improved stability and performance
compared to standard GAN training.
An Alternative to EM for Gaussian Mixture Models: Batch and Stochastic Riemannian Optimization
Authors: Reshad Hosseini, Suvrit Sra
Date: 2017-06-10 00:00:00
Tasks: Density Estimation, parameter estimation, Riemannian optimization

We consider maximum likelihood estimation for Gaussian Mixture Models (Gmms).
This task is almost invariably solved (in theory and practice) via the
Expectation Maximization (EM) algorithm. EM owes its success to various
factors, of which is its ability to fulfill positive definiteness constraints
in closed form is of key importance. We propose an alternative to EM by
appealing to the rich Riemannian geometry of positive definite matrices, using
which we cast Gmm parameter estimation as a Riemannian optimization problem.
Surprisingly, such an out-of-the-box Riemannian formulation completely fails
and proves much inferior to EM. This motivates us to take a closer look at the
problem geometry, and derive a better formulation that is much more amenable to
Riemannian optimization. We then develop (Riemannian) batch and stochastic
gradient algorithms that outperform EM, often substantially. We provide a
non-asymptotic convergence analysis for our stochastic method, which is also
the first (to our knowledge) such global analysis for Riemannian stochastic
gradient. Numerous empirical results are included to demonstrate the
effectiveness of our methods.
Stepwise regression for unsupervised learning
Authors: Jonathan Landy
Date: 2017-06-10 00:00:00
Tasks: feature selection, regression

I consider unsupervised extensions of the fast stepwise linear regression
algorithm \cite{efroymson1960multiple}. These extensions allow one to
efficiently identify highly-representative feature variable subsets within a
given set of jointly distributed variables. This in turn allows for the
efficient dimensional reduction of large data sets via the removal of redundant
features. Fast search is effected here through the avoidance of repeat
computations across trial fits, allowing for a full representative-importance
ranking of a set of feature variables to be carried out in $O(n^2 m)$ time,
where $n$ is the number of variables and $m$ is the number of data samples
available. This runtime complexity matches that needed to carry out a single
regression and is $O(n^2)$ faster than that of naive implementations. I present
pseudocode suitable for efficient forward, reverse, and forward-reverse
unsupervised feature selection. To illustrate the algorithm's application, I
apply it to the problem of identifying representative stocks within a given
financial market index -- a challenge relevant to the design of Exchange Traded
Funds (ETFs). I also characterize the growth of numerical error with iteration
step in these algorithms, and finally demonstrate and rationalize the
observation that the forward and reverse algorithms return exactly inverted
feature orderings in the weakly-correlated feature set regime.
First Person Action-Object Detection with EgoNet
Authors: Gedas Bertasius, Hyun Soo Park, Stella X. Yu, Jianbo Shi
Date: 2016-03-15 00:00:00
Tasks: Human-Object Interaction Detection, Object, object-detection, Object Detection

Unlike traditional third-person cameras mounted on robots, a first-person
camera, captures a person's visual sensorimotor object interactions from up
close. In this paper, we study the tight interplay between our momentary visual
attention and motor action with objects from a first-person camera. We propose
a concept of action-objects---the objects that capture person's conscious
visual (watching a TV) or tactile (taking a cup) interactions. Action-objects
may be task-dependent but since many tasks share common person-object spatial
configurations, action-objects exhibit a characteristic 3D spatial distance and
orientation with respect to the person.
  We design a predictive model that detects action-objects using EgoNet, a
joint two-stream network that holistically integrates visual appearance (RGB)
and 3D spatial layout (depth and height) cues to predict per-pixel likelihood
of action-objects. Our network also incorporates a first-person coordinate
embedding, which is designed to learn a spatial distribution of the
action-objects in the first-person data. We demonstrate EgoNet's predictive
power, by showing that it consistently outperforms previous baseline
approaches. Furthermore, EgoNet also exhibits a strong generalization ability,
i.e., it predicts semantically meaningful objects in novel first-person
datasets. Our method's ability to effectively detect action-objects could be
used to improve robots' understanding of human-object interactions.
A Bayesian Hyperprior Approach for Joint Image Denoising and Interpolation, with an Application to HDR Imaging
Authors: Cecilia Aguerrebere, Andrés Almansa, Julie Delon, Yann Gousseau, Pablo Musé
Date: 2017-06-10 00:00:00
Tasks: Denoising, Image Denoising

Recently, impressive denoising results have been achieved by Bayesian
approaches which assume Gaussian models for the image patches. This improvement
in performance can be attributed to the use of per-patch models. Unfortunately
such an approach is particularly unstable for most inverse problems beyond
denoising. In this work, we propose the use of a hyperprior to model image
patches, in order to stabilize the estimation procedure. There are two main
advantages to the proposed restoration scheme: Firstly it is adapted to
diagonal degradation matrices, and in particular to missing data problems (e.g.
inpainting of missing pixels or zooming). Secondly it can deal with signal
dependent noise models, particularly suited to digital cameras. As such, the
scheme is especially adapted to computational photography. In order to
illustrate this point, we provide an application to high dynamic range imaging
from a single image taken with a modified sensor, which shows the effectiveness
of the proposed scheme.
Progressive Neural Networks for Transfer Learning in Emotion Recognition
Authors: John Gideon, Soheil Khorram, Zakaria Aldeneh, Dimitrios Dimitriadis, Emily Mower Provost
Date: 2017-06-10 00:00:00
Tasks: Emotion Recognition, Transfer Learning

Many paralinguistic tasks are closely related and thus representations
learned in one domain can be leveraged for another. In this paper, we
investigate how knowledge can be transferred between three paralinguistic
tasks: speaker, emotion, and gender recognition. Further, we extend this
problem to cross-dataset tasks, asking how knowledge captured in one emotion
dataset can be transferred to another. We focus on progressive neural networks
and compare these networks to the conventional deep learning method of
pre-training and fine-tuning. Progressive neural networks provide a way to
transfer knowledge and avoid the forgetting effect present when pre-training
neural networks on different tasks. Our experiments demonstrate that: (1)
emotion recognition can benefit from using representations originally learned
for different paralinguistic tasks and (2) transfer learning can effectively
leverage additional datasets to improve the performance of emotion recognition
systems.
Causal Discovery in the Presence of Measurement Error: Identifiability Conditions
Authors: Kun Zhang, Mingming Gong, Joseph Ramsey, Kayhan Batmanghelich, Peter Spirtes, Clark Glymour
Date: 2017-06-10 00:00:00
Tasks: Causal Discovery

Measurement error in the observed values of the variables can greatly change
the output of various causal discovery methods. This problem has received much
attention in multiple fields, but it is not clear to what extent the causal
model for the measurement-error-free variables can be identified in the
presence of measurement error with unknown variance. In this paper, we study
precise sufficient identifiability conditions for the measurement-error-free
causal model and show what information of the causal model can be recovered
from observed data. In particular, we present two different sets of
identifiability conditions, based on the second-order statistics and
higher-order statistics of the data, respectively. The former was inspired by
the relationship between the generating model of the
measurement-error-contaminated data and the factor analysis model, and the
latter makes use of the identifiability result of the over-complete independent
component analysis problem.
Learning Gradient Descent: Better Generalization and Longer Horizons
Authors: Kaifeng Lv, Shunhua Jiang, Jian Li
Date: 2017-03-10 00:00:00
Proceeding: ICML 2017 8
Tasks: Scheduling

Training deep neural networks is a highly nontrivial task, involving
carefully selecting appropriate training algorithms, scheduling step sizes and
tuning other hyperparameters. Trying different combinations can be quite
labor-intensive and time consuming. Recently, researchers have tried to use
deep learning algorithms to exploit the landscape of the loss function of the
training problem of interest, and learn how to optimize over it in an automatic
way. In this paper, we propose a new learning-to-learn model and some useful
and practical tricks. Our optimizer outperforms generic, hand-crafted
optimization algorithms and state-of-the-art learning-to-learn optimizers by
DeepMind in many tasks. We demonstrate the effectiveness of our algorithms on a
number of tasks, including deep MLPs, CNNs, and simple LSTMs.
Evidence Updating for Stream-Processing in Big-Data: Robust Conditioning in Soft and Hard Fusion Environments
Authors: Thanuka Wickramarathne
Date: 2017-03-20 00:00:00

Robust belief revision methods are crucial in streaming data situations for
updating existing knowledge or beliefs with new incoming evidence. Bayes
conditioning is the primary mechanism in use for belief revision in data fusion
systems that use probabilistic inference. However, traditional conditioning
methods face several challenges due to inherent data/source imperfections in
big-data environments that harness soft (i.e., human or human-based) sources in
addition to hard (i.e., physics-based) sensors. The objective of this paper is
to investigate the most natural extension of Bayes conditioning that is
suitable for evidence updating in the presence of such uncertainties. By
viewing the evidence updating process as a thought experiment, an elegant
strategy is derived for robust evidence updating in the presence of extreme
uncertainties that are characteristic of big-data environments. In particular,
utilizing the Fagin-Halpern conditional notions, a natural extension to Bayes
conditioning is derived for evidence that takes the form of a general belief
function. The presented work differs fundamentally from the Conditional Update
Equation (CUE) and authors own extensions of it. An overview of this
development is provided via illustrative examples. Furthermore, insights into
parameter selection under various fusion contexts are also provided.
Exploring Convolutional Networks for End-to-End Visual Servoing
Authors: Aseem Saxena, Harit Pandya, Gourav Kumar, Ayush Gaud, K. Madhava Krishna
Date: 2017-06-10 00:00:00

Present image based visual servoing approaches rely on extracting hand
crafted visual features from an image. Choosing the right set of features is
important as it directly affects the performance of any approach. Motivated by
recent breakthroughs in performance of data driven methods on recognition and
localization tasks, we aim to learn visual feature representations suitable for
servoing tasks in unstructured and unknown environments. In this paper, we
present an end-to-end learning based approach for visual servoing in diverse
scenes where the knowledge of camera parameters and scene geometry is not
available a priori. This is achieved by training a convolutional neural network
over color images with synchronised camera poses. Through experiments performed
in simulation and on a quadrotor, we demonstrate the efficacy and robustness of
our approach for a wide range of camera poses in both indoor as well as outdoor
environments.
Improving Legal Information Retrieval by Distributional Composition with Term Order Probabilities
Authors: Danilo S. Carvalho, Duc-Vu Tran, Van-Khanh Tran, Le-Nguyen Minh
Date: 2017-06-04 00:00:00
Tasks: Information Retrieval, Question Answering, Retrieval, Sentence

Legal professionals worldwide are currently trying to get up-to-pace with the
explosive growth in legal document availability through digital means. This
drives a need for high efficiency Legal Information Retrieval (IR) and Question
Answering (QA) methods. The IR task in particular has a set of unique
challenges that invite the use of semantic motivated NLP techniques. In this
work, a two-stage method for Legal Information Retrieval is proposed, combining
lexical statistics and distributional sentence representations in the context
of Competition on Legal Information Extraction/Entailment (COLIEE). The
combination is done with the use of disambiguation rules, applied over the
rankings obtained through n-gram statistics. After the ranking is done, its
results are evaluated for ambiguity, and disambiguation is done if a result is
decided to be unreliable for a given query. Competition and experimental
results indicate small gains in overall retrieval performance using the
proposed approach. Additionally, an analysis of error and improvement cases is
presented for a better understanding of the contributions.
Towards Statistical Reasoning in Description Logics over Finite Domains (Full Version)
Authors: Rafael Peñaloza, Nico Potyka
Date: 2017-06-10 00:00:00

We present a probabilistic extension of the description logic $\mathcal{ALC}$
for reasoning about statistical knowledge. We consider conditional statements
over proportions of the domain and are interested in the probabilistic-logical
consequences of these proportions. After introducing some general reasoning
problems and analyzing their properties, we present first algorithms and
complexity results for reasoning in some fragments of Statistical
$\mathcal{ALC}$.