pwc-archive/papers-with-abstracts huggingface dataset:
Learning to Synthesize Motion Blur
Authors: Tim Brooks, Jonathan T. Barron
Date: 2018-11-27 00:00:00
Proceeding: CVPR 2019 6

We present a technique for synthesizing a motion blurred image from a pair of unblurred images captured in succession. To build this system we motivate and design a differentiable "line prediction" layer to be used as part of a neural network architecture, with which we can learn a system to regress from image pairs to motion blurred images that span the capture time of the input image pair. Training this model requires an abundance of data, and so we design and execute a strategy for using frame interpolation techniques to generate a large-scale synthetic dataset of motion blurred images and their respective inputs. We additionally capture a high quality test set of real motion blurred images, synthesized from slow motion videos, with which we evaluate our model against several baseline techniques that can be used to synthesize motion blur. Our model produces higher accuracy output than our baselines, and is significantly faster than baselines with competitive accuracy.
DeepPos: Deep Supervised Autoencoder Network for CSI Based Indoor Localization
Authors: Peyman Yazdanian, Vahid Pourahmadi
Date: 2018-11-27 00:00:00
Tasks: Indoor Localization

The widespread mobile devices facilitated the emergence of many new
applications and services. Among them are location-based services (LBS) that
provide services based on user's location. Several techniques have been
presented to enable LBS even in indoor environments where Global Positioning
System (GPS) has low localization accuracy. These methods use some environment
measurements (like Channel State Information (CSI) or Received Signal Strength
(RSS)) for user localization. In this paper, we will use CSI and a novel deep
learning algorithm to design a robust and efficient system for indoor
localization. More precisely, we use supervised autoencoder (SAE) to model the
environment using the data collected during the training phase. Then, during
the testing phase, we use the trained model and estimate the coordinates of the
unknown point by checking different possible labels. Unlike the previous
fingerprinting approaches, in this work, we do not store the {CSI/RSS} of
fingerprints and instead we model the environment only with a single SAE. The
performance of the proposed scheme is then evaluated in two indoor environments
and compared with that of similar approaches.
Bayesian Neural Network Ensembles
Authors: Tim Pearce, Mohamed Zaki, Andy Neely
Date: 2018-11-27 00:00:00

Ensembles of neural networks (NNs) have long been used to estimate predictive
uncertainty; a small number of NNs are trained from different initialisations
and sometimes on differing versions of the dataset. The variance of the
ensemble's predictions is interpreted as its epistemic uncertainty. The appeal
of ensembling stems from being a collection of regular NNs - this makes them
both scalable and easily implementable. They have achieved strong empirical
results in recent years, often presented as a practical alternative to more
costly Bayesian NNs (BNNs). The departure from Bayesian methodology is of
concern since the Bayesian framework provides a principled, widely-accepted
approach to handling uncertainty. In this extended abstract we derive and
implement a modified NN ensembling scheme, which provides a consistent
estimator of the Bayesian posterior in wide NNs - regularising parameters about
values drawn from a prior distribution.
SLAYER: Spike Layer Error Reassignment in Time
Authors: Sumit Bam Shrestha, Garrick Orchard
Date: 2018-09-05 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: GPU

Configuring deep Spiking Neural Networks (SNNs) is an exciting research
avenue for low power spike event based computation. However, the spike
generation function is non-differentiable and therefore not directly compatible
with the standard error backpropagation algorithm. In this paper, we introduce
a new general backpropagation mechanism for learning synaptic weights and
axonal delays which overcomes the problem of non-differentiability of the spike
function and uses a temporal credit assignment policy for backpropagating error
to preceding layers. We describe and release a GPU accelerated software
implementation of our method which allows training both fully connected and
convolutional neural network (CNN) architectures. Using our software, we
compare our method against existing SNN based learning approaches and standard
ANN to SNN conversion techniques and show that our method achieves state of the
art performance for an SNN on the MNIST, NMNIST, DVS Gesture, and TIDIGITS
datasets.
Improved Sample Complexity for Stochastic Compositional Variance Reduced Gradient
Authors: Tianyi Lin, Chenyou Fan, Mengdi Wang, Michael. I. Jordan
Date: 2018-06-01 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Convex composition optimization is an emerging topic that covers a wide range of applications arising from stochastic optimal control, reinforcement learning and multi-stage stochastic programming. Existing algorithms suffer from unsatisfactory sample complexity and practical issues since they ignore the convexity structure in the algorithmic design. In this paper, we develop a new stochastic compositional variance-reduced gradient algorithm with the sample complexity of $O((m+n)\log(1/\epsilon)+1/\epsilon^3)$ where $m+n$ is the total number of samples. Our algorithm is near-optimal as the dependence on $m+n$ is optimal up to a logarithmic factor. Experimental results on real-world datasets demonstrate the effectiveness and efficiency of the new algorithm.
An Introductory Study on Time Series Modeling and Forecasting
Authors: Ratnadip Adhikari, R. K. Agrawal
Date: 2013-02-26 00:00:00
Tasks: Time Series, Time Series Analysis, Time Series Forecasting

Time series modeling and forecasting has fundamental importance to various
practical domains. Thus a lot of active research works is going on in this
subject during several years. Many important models have been proposed in
literature for improving the accuracy and effectiveness of time series
forecasting. The aim of this dissertation work is to present a concise
description of some popular time series forecasting models used in practice,
with their salient features. In this thesis, we have described three important
classes of time series models, viz. the stochastic, neural networks and SVM
based models, together with their inherent forecasting strengths and
weaknesses. We have also discussed about the basic issues related to time
series modeling, such as stationarity, parsimony, overfitting, etc. Our
discussion about different time series models is supported by giving the
experimental forecast results, performed on six real time series datasets.
While fitting a model to a dataset, special care is taken to select the most
parsimonious one. To evaluate forecast accuracy as well as to compare among
different models fitted to a time series, we have used the five performance
measures, viz. MSE, MAD, RMSE, MAPE and Theil's U-statistics. For each of the
six datasets, we have shown the obtained forecast diagram which graphically
depicts the closeness between the original and forecasted observations. To have
authenticity as well as clarity in our discussion about time series modeling
and forecasting, we have taken the help of various published research works
from reputed journals and some standard books.
Multi-Adversarial Domain Adaptation
Authors: Zhongyi Pei, Zhangjie Cao, Mingsheng Long, Jian-Min Wang
Date: 2018-09-04 00:00:00
Tasks: Domain Adaptation

Recent advances in deep domain adaptation reveal that adversarial learning
can be embedded into deep networks to learn transferable features that reduce
distribution discrepancy between the source and target domains. Existing domain
adversarial adaptation methods based on single domain discriminator only align
the source and target data distributions without exploiting the complex
multimode structures. In this paper, we present a multi-adversarial domain
adaptation (MADA) approach, which captures multimode structures to enable
fine-grained alignment of different data distributions based on multiple domain
discriminators. The adaptation can be achieved by stochastic gradient descent
with the gradients computed by back-propagation in linear-time. Empirical
evidence demonstrates that the proposed model outperforms state of the art
methods on standard domain adaptation datasets.
19 dubious ways to compute the marginal likelihood of a phylogenetic tree topology
Authors: Mathieu Fourment, Andrew F. Magee, Chris Whidden, Arman Bilge, Frederick A. Matsen IV, Vladimir N. Minin
Date: 2018-11-28 00:00:00

The marginal likelihood of a model is a key quantity for assessing the
evidence provided by the data in support of a model. The marginal likelihood is
the normalizing constant for the posterior density, obtained by integrating the
product of the likelihood and the prior with respect to model parameters. Thus,
the computational burden of computing the marginal likelihood scales with the
dimension of the parameter space. In phylogenetics, where we work with tree
topologies that are high-dimensional models, standard approaches to computing
marginal likelihoods are very slow. Here we study methods to quickly compute
the marginal likelihood of a single fixed tree topology. We benchmark the speed
and accuracy of 19 different methods to compute the marginal likelihood of
phylogenetic topologies on a suite of real datasets. These methods include
several new ones that we develop explicitly to solve this problem, as well as
existing algorithms that we apply to phylogenetic models for the first time.
Altogether, our results show that the accuracy of these methods varies widely,
and that accuracy does not necessarily correlate with computational burden. Our
newly developed methods are orders of magnitude faster than standard
approaches, and in some cases, their accuracy rivals the best established
estimators.
Lightweight and Efficient Image Super-Resolution with Block State-based Recursive Network
Authors: Jun-Ho Choi, Jun-Hyuk Kim, Manri Cheon, Jong-Seok Lee
Date: 2018-11-30 00:00:00
Tasks: Image Super-Resolution, Super-Resolution

Recently, several deep learning-based image super-resolution methods have
been developed by stacking massive numbers of layers. However, this leads too
large model sizes and high computational complexities, thus some recursive
parameter-sharing methods have been also proposed. Nevertheless, their designs
do not properly utilize the potential of the recursive operation. In this
paper, we propose a novel, lightweight, and efficient super-resolution method
to maximize the usefulness of the recursive architecture, by introducing block
state-based recursive network. By taking advantage of utilizing the block
state, the recursive part of our model can easily track the status of the
current image features. We show the benefits of the proposed method in terms of
model size, speed, and efficiency. In addition, we show that our method
outperforms the other state-of-the-art methods.
On the Computational Inefficiency of Large Batch Sizes for Stochastic Gradient Descent
Authors: Noah Golmant, Nikita Vemuri, Zhewei Yao, Vladimir Feinberg, Amir Gholami, Kai Rothauge, Michael W. Mahoney, Joseph Gonzalez
Date: 2018-11-30 00:00:00
Tasks: image-classification, Image Classification, Image Segmentation, Language Modeling, Language Modelling, Semantic Segmentation

Increasing the mini-batch size for stochastic gradient descent offers
significant opportunities to reduce wall-clock training time, but there are a
variety of theoretical and systems challenges that impede the widespread
success of this technique. We investigate these issues, with an emphasis on
time to convergence and total computational cost, through an extensive
empirical analysis of network training across several architectures and problem
domains, including image classification, image segmentation, and language
modeling. Although it is common practice to increase the batch size in order to
fully exploit available computational resources, we find a substantially more
nuanced picture. Our main finding is that across a wide range of network
architectures and problem domains, increasing the batch size beyond a certain
point yields no decrease in wall-clock time to convergence for \emph{either}
train or test loss. This batch size is usually substantially below the capacity
of current systems. We show that popular training strategies for large batch
size optimization begin to fail before we can populate all available compute
resources, and we show that the point at which these methods break down depends
more on attributes like model architecture and data complexity than it does
directly on the size of the dataset.
Advance Prediction of Ventricular Tachyarrhythmias using Patient Metadata and Multi-Task Networks
Authors: Marek Rei, Joshua Oppenheimer, Marek Sirendi
Date: 2018-11-30 00:00:00
Tasks: Heart Rate Variability, Prediction, Specificity

We describe a novel neural network architecture for the prediction of
ventricular tachyarrhythmias. The model receives input features that capture
the change in RR intervals and ectopic beats, along with features based on
heart rate variability and frequency analysis. Patient age is also included as
a trainable embedding, while the whole network is optimized with multi-task
objectives. Each of these modifications provides a consistent improvement to
the model performance, achieving 74.02% prediction accuracy and 77.22%
specificity 60 seconds in advance of the episode.
Recurrent machines for likelihood-free inference
Authors: Arthur Pesah, Antoine Wehenkel, Gilles Louppe
Date: 2018-11-30 00:00:00
Tasks: Meta-Learning

Likelihood-free inference is concerned with the estimation of the parameters
of a non-differentiable stochastic simulator that best reproduce real
observations. In the absence of a likelihood function, most of the existing
inference methods optimize the simulator parameters through a handcrafted
iterative procedure that tries to make the simulated data more similar to the
observations. In this work, we explore whether meta-learning can be used in the
likelihood-free context, for learning automatically from data an iterative
optimization procedure that would solve likelihood-free inference problems. We
design a recurrent inference machine that learns a sequence of parameter
updates leading to good parameter estimates, without ever specifying some
explicit notion of divergence between the simulated data and the real data
distributions. We demonstrate our approach on toy simulators, showing promising
results both in terms of performance and robustness.
Online Abstraction with MDP Homomorphisms for Deep Learning
Authors: Ondrej Biza, Robert Platt
Date: 2018-11-30 00:00:00
Tasks: Deep Learning

Abstraction of Markov Decision Processes is a useful tool for solving complex
problems, as it can ignore unimportant aspects of an environment, simplifying
the process of learning an optimal policy. In this paper, we propose a new
algorithm for finding abstract MDPs in environments with continuous state
spaces. It is based on MDP homomorphisms, a structure-preserving mapping
between MDPs. We demonstrate our algorithm's ability to learn abstractions from
collected experience and show how to reuse the abstractions to guide
exploration in new tasks the agent encounters. Our novel task transfer method
outperforms baselines based on a deep Q-network in the majority of our
experiments. The source code is at https://github.com/ondrejba/aamas_19.
Automated Tactical Decision Planning Model with Strategic Values Guidance for Local Action-Value-Ambiguity
Authors: Daniel Muller, Erez Karpas
Date: 2018-11-30 00:00:00
Tasks: Informativeness

In many real-world planning problems, action's impact differs with a place,
time and the context in which the action is applied. The same action with the
same effects in a different context or states can cause a different change. In
actions with incomplete precondition list, that applicable in several states
and circumstances, ambiguity regarding the impact of the action is challenging
even in small domains. To estimate the real impact of actions, an evaluation of
the effect list will not be enough; a relative estimation is more informative
and suitable for estimation of action's real impact. Recent work on
Over-subscription Planning (OSP) defined the net utility of action as the net
change in the state's value caused by the action. The notion of net utility of
action allows for a broader perspective on value action impact and use for a
more accurate evaluation of achievements of the action, considering inter-state
and intra-state dependencies. To achieve value-rational decisions in complex
reality often requires strategic, high level, planning with a global
perspective and values, while many local tactical decisions require real-time
information to estimate the impact of actions. This paper proposes an offline
action-value structure analysis to exploit the compactly represented
informativeness of net utility of actions to extend the scope of planning to
value uncertainty scenarios and to provide a real-time value-rational decision
planning tool. The result of the offline pre-processing phase is a compact
decision planning model representation for flexible, local reasoning of net
utility of actions with (offline) value ambiguity. The obtained flexibility is
beneficial for the online planning phase and real-time execution of actions
with value ambiguity. Our empirical evaluation shows the effectiveness of this
approach in domains with value ambiguity in their action-value-structure.
Optimal Data Driven Resource Allocation under Multi-Armed Bandit Observations
Authors: Apostolos N. Burnetas, Odysseas Kanavetas, Michael N. Katehakis
Date: 2018-11-30 00:00:00

This paper introduces the first asymptotically optimal strategy for a multi armed bandit (MAB) model under side constraints. The side constraints model situations in which bandit activations are limited by the availability of certain resources that are replenished at a constant rate. The main result involves the derivation of an asymptotic lower bound for the regret of feasible uniformly fast policies and the construction of policies that achieve this lower bound, under pertinent conditions. Further, we provide the explicit form of such policies for the case in which the unknown distributions are Normal with unknown means and known variances, for the case of Normal distributions with unknown means and unknown variances and for the case of arbitrary discrete distributions with finite support.
Detecting Offensive Content in Open-domain Conversations using Two Stage Semi-supervision
Authors: Chandra Khatri, Behnam Hedayatnia, Rahul Goel, Anushree Venkatesh, Raefer Gabriel, Arindam Mandal
Date: 2018-11-30 00:00:00
Tasks: Chatbot

As open-ended human-chatbot interaction becomes commonplace, sensitive
content detection gains importance. In this work, we propose a two stage
semi-supervised approach to bootstrap large-scale data for automatic sensitive
language detection from publicly available web resources. We explore various
data selection methods including 1) using a blacklist to rank online discussion
forums by the level of their sensitiveness followed by randomly sampling
utterances and 2) training a weakly supervised model in conjunction with the
blacklist for scoring sentences from online discussion forums to curate a
dataset. Our data collection strategy is flexible and allows the models to
detect implicit sensitive content for which manual annotations may be
difficult. We train models using publicly available annotated datasets as well
as using the proposed large-scale semi-supervised datasets. We evaluate the
performance of all the models on Twitter and Toxic Wikipedia comments testsets
as well as on a manually annotated spoken language dataset collected during a
large scale chatbot competition. Results show that a model trained on this
collected data outperforms the baseline models by a large margin on both
in-domain and out-of-domain testsets, achieving an F1 score of 95.5% on an
out-of-domain testset compared to a score of 75% for models trained on public
datasets. We also showcase that large scale two stage semi-supervision
generalizes well across multiple classes of sensitivities such as hate speech,
racism, sexual and pornographic content, etc. without even providing explicit
labels for these classes, leading to an average recall of 95.5% versus the
models trained using annotated public datasets which achieve an average recall
of 73.2% across seven sensitive classes on out-of-domain testsets.
Flexible and Scalable State Tracking Framework for Goal-Oriented Dialogue Systems
Authors: Rahul Goel, Shachi Paul, Tagyoung Chung, Jeremie Lecomte, Arindam Mandal, Dilek Hakkani-Tur
Date: 2018-11-30 00:00:00
Tasks: Dialogue State Tracking, Goal-Oriented Dialogue Systems, Word Embeddings

Goal-oriented dialogue systems typically rely on components specifically
developed for a single task or domain. This limits such systems in two
different ways: If there is an update in the task domain, the dialogue system
usually needs to be updated or completely re-trained. It is also harder to
extend such dialogue systems to different and multiple domains. The dialogue
state tracker in conventional dialogue systems is one such component - it is
usually designed to fit a well-defined application domain. For example, it is
common for a state variable to be a categorical distribution over a
manually-predefined set of entities (Henderson et al., 2013), resulting in an
inflexible and hard-to-extend dialogue system. In this paper, we propose a new
approach for dialogue state tracking that can generalize well over multiple
domains without incorporating any domain-specific knowledge. Under this
framework, discrete dialogue state variables are learned independently and the
information of a predefined set of possible values for dialogue state variables
is not required. Furthermore, it enables adding arbitrary dialogue context as
features and allows for multiple values to be associated with a single state
variable. These characteristics make it much easier to expand the dialogue
state space. We evaluate our framework using the widely used dialogue state
tracking challenge data set (DSTC2) and show that our framework yields
competitive results with other state-of-the-art results despite incorporating
little domain knowledge. We also show that this framework can benefit from
widely available external resources such as pre-trained word embeddings.
Systematic Generalization: What Is Required and Can It Be Learned?
Authors: Dzmitry Bahdanau, Shikhar Murty, Michael Noukhovitch, Thien Huu Nguyen, Harm de Vries, Aaron Courville
Date: 2018-11-30 00:00:00
Proceeding: ICLR 2019 5
Tasks: Systematic Generalization, Visual Question Answering (VQA)

Numerous models for grounded language understanding have been recently
proposed, including (i) generic models that can be easily adapted to any given
task and (ii) intuitively appealing modular models that require background
knowledge to be instantiated. We compare both types of models in how much they
lend themselves to a particular form of systematic generalization. Using a
synthetic VQA test, we evaluate which models are capable of reasoning about all
possible object pairs after training on only a small subset of them. Our
findings show that the generalization of modular models is much more systematic
and that it is highly sensitive to the module layout, i.e. to how exactly the
modules are connected. We furthermore investigate if modular models that
generalize well could be made more end-to-end by learning their layout and
parametrization. We find that end-to-end methods from prior work often learn
inappropriate layouts or parametrizations that do not facilitate systematic
generalization. Our results suggest that, in addition to modularity, systematic
generalization in language understanding may require explicit regularizers or
priors.