pwc-archive/papers-with-abstracts huggingface dataset:
Dynamic Routing Between Capsules
Authors: Sara Sabour, Nicholas Frosst, Geoffrey E. Hinton
Date: 2017-10-26 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Image Classification

A capsule is a group of neurons whose activity vector represents the
instantiation parameters of a specific type of entity such as an object or an
object part. We use the length of the activity vector to represent the
probability that the entity exists and its orientation to represent the
instantiation parameters. Active capsules at one level make predictions, via
transformation matrices, for the instantiation parameters of higher-level
capsules. When multiple predictions agree, a higher level capsule becomes
active. We show that a discrimininatively trained, multi-layer capsule system
achieves state-of-the-art performance on MNIST and is considerably better than
a convolutional net at recognizing highly overlapping digits. To achieve these
results we use an iterative routing-by-agreement mechanism: A lower-level
capsule prefers to send its output to higher level capsules whose activity
vectors have a big scalar product with the prediction coming from the
lower-level capsule.
What are the visual features underlying human versus machine vision?
Authors: Drew Linsley, Sven Eberhardt, Tarun Sharma, Pankaj Gupta, Thomas Serre
Date: 2017-01-10 00:00:00
Tasks: Diagnostic, Object, Object Recognition

Although Deep Convolutional Networks (DCNs) are approaching the accuracy of
human observers at object recognition, it is unknown whether they leverage
similar visual representations to achieve this performance. To address this, we
introduce Clicktionary, a web-based game for identifying visual features used
by human observers during object recognition. Importance maps derived from the
game are consistent across participants and uncorrelated with image saliency
measures. These results suggest that Clicktionary identifies image regions that
are meaningful and diagnostic for object recognition but different than those
driving eye movements. Surprisingly, Clicktionary importance maps are only
weakly correlated with relevance maps derived from DCNs trained for object
recognition. Our study demonstrates that the narrowing gap between the object
recognition accuracy of human observers and DCNs obscures distinct visual
strategies used by each to achieve this performance.
Simple and Effective Multi-Paragraph Reading Comprehension
Authors: Christopher Clark, Matt Gardner
Date: 2017-10-29 00:00:00
Proceeding: ACL 2018 7
Tasks: Question Answering, Reading Comprehension, TriviaQA

We consider the problem of adapting neural paragraph-level question answering
models to the case where entire documents are given as input. Our proposed
solution trains models to produce well calibrated confidence scores for their
results on individual paragraphs. We sample multiple paragraphs from the
documents during training, and use a shared-normalization training objective
that encourages the model to produce globally correct output. We combine this
method with a state-of-the-art pipeline for training models on document QA
data. Experiments demonstrate strong performance on several document QA
datasets. Overall, we are able to achieve a score of 71.3 F1 on the web portion
of TriviaQA, a large improvement from the 56.7 F1 of the previous best system.
Latent hypernet: Exploring all Layers from Convolutional Neural Networks
Authors: Artur Jordao, Ricardo Kloss, William Robson Schwartz
Date: 2017-11-07 00:00:00
Tasks: Activity Recognition, All, Human Activity Recognition

Since Convolutional Neural Networks (ConvNets) are able to simultaneously
learn features and classifiers to discriminate different categories of
activities, recent works have employed ConvNets approaches to perform human
activity recognition (HAR) based on wearable sensors, allowing the removal of
expensive human work and expert knowledge. However, these approaches have their
power of discrimination limited mainly by the large number of parameters that
compose the network and the reduced number of samples available for training.
Inspired by this, we propose an accurate and robust approach, referred to as
Latent HyperNet (LHN). The LHN uses feature maps from early layers (hyper) and
projects them, individually, onto a low dimensionality space (latent). Then,
these latent features are concatenated and presented to a classifier. To
demonstrate the robustness and accuracy of the LHN, we evaluate it using four
different networks architectures in five publicly available HAR datasets based
on wearable sensors, which vary in the sampling rate and number of activities.
Our experiments demonstrate that the proposed LHN is able to produce rich
information, improving the results regarding the original ConvNets.
Furthermore, the method outperforms existing state-of-the-art methods.
Theoretical limitations of Encoder-Decoder GAN architectures
Authors: Sanjeev Arora, Andrej Risteski, Yi Zhang
Date: 2017-11-07 00:00:00
Tasks: Decoder

Encoder-decoder GANs architectures (e.g., BiGAN and ALI) seek to add an
inference mechanism to the GANs setup, consisting of a small encoder deep net
that maps data-points to their succinct encodings. The intuition is that being
forced to train an encoder alongside the usual generator forces the system to
learn meaningful mappings from the code to the data-point and vice-versa, which
should improve the learning of the target distribution and ameliorate
mode-collapse. It should also yield meaningful codes that are useful as
features for downstream tasks. The current paper shows rigorously that even on
real-life distributions of images, the encode-decoder GAN training objectives
(a) cannot prevent mode collapse; i.e. the objective can be near-optimal even
when the generated distribution has low and finite support (b) cannot prevent
learning meaningless codes for data -- essentially white noise. Thus if
encoder-decoder GANs do indeed work then it must be due to reasons as yet not
understood, since the training objective can be low even for meaningless
solutions.
Preorder-Based Triangle: A Modified Version of Bilattice-Based Triangle for Belief Revision in Nonmonotonic Reasoning
Authors: Kumar Sankar Ray, Sandip Paul, Diganta Saha
Date: 2016-09-19 00:00:00

Bilattice-based triangle provides an elegant algebraic structure for
reasoning with vague and uncertain information. But the truth and knowledge
ordering of intervals in bilattice-based triangle can not handle repetitive
belief revisions which is an essential characteristic of nonmonotonic
reasoning. Moreover the ordering induced over the intervals by the
bilattice-based triangle is not sometimes intuitive. In this work, we construct
an alternative algebraic structure, namely preorder-based triangle and we
formulate proper logical connectives for this. It is also demonstrated that
Preorder-based triangle serves to be a better alternative to the
bilattice-based triangle for reasoning in application areas, that involve
nonmonotonic fuzzy reasoning with uncertain information.
Identification of Gaussian Process State Space Models
Authors: Stefanos Eleftheriadis, Thomas F. W. Nicholson, Marc Peter Deisenroth, James Hensman
Date: 2017-05-30 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: State Estimation, State Space Models

The Gaussian process state space model (GPSSM) is a non-linear dynamical
system, where unknown transition and/or measurement mappings are described by
GPs. Most research in GPSSMs has focussed on the state estimation problem,
i.e., computing a posterior of the latent state given the model. However, the
key challenge in GPSSMs has not been satisfactorily addressed yet: system
identification, i.e., learning the model. To address this challenge, we impose
a structured Gaussian variational posterior distribution over the latent
states, which is parameterised by a recognition model in the form of a
bi-directional recurrent neural network. Inference with this structure allows
us to recover a posterior smoothed over sequences of data. We provide a
practical algorithm for efficiently computing a lower bound on the marginal
likelihood using the reparameterisation trick. This further allows for the use
of arbitrary kernels within the GPSSM. We demonstrate that the learnt GPSSM can
efficiently generate plausible future trajectories of the identified system
after only observing a small number of episodes from the true system.
Boltzmann Exploration Done Right
Authors: Nicolò Cesa-Bianchi, Claudio Gentile, Gábor Lugosi, Gergely Neu
Date: 2017-05-29 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Decision Making, Decision Making Under Uncertainty, Multi-Armed Bandits, Reinforcement Learning, Reinforcement Learning (RL), Sequential Decision Making

Boltzmann exploration is a classic strategy for sequential decision-making
under uncertainty, and is one of the most standard tools in Reinforcement
Learning (RL). Despite its widespread use, there is virtually no theoretical
understanding about the limitations or the actual benefits of this exploration
scheme. Does it drive exploration in a meaningful way? Is it prone to
misidentifying the optimal actions or spending too much time exploring the
suboptimal ones? What is the right tuning for the learning rate? In this paper,
we address several of these questions in the classic setup of stochastic
multi-armed bandits. One of our main results is showing that the Boltzmann
exploration strategy with any monotone learning-rate sequence will induce
suboptimal behavior. As a remedy, we offer a simple non-monotone schedule that
guarantees near-optimal performance, albeit only when given prior access to key
problem parameters that are typically not available in practical situations
(like the time horizon $T$ and the suboptimality gap $\Delta$). More
importantly, we propose a novel variant that uses different learning rates for
different arms, and achieves a distribution-dependent regret bound of order
$\frac{K\log^2 T}{\Delta}$ and a distribution-independent bound of order
$\sqrt{KT}\log K$ without requiring such prior knowledge. To demonstrate the
flexibility of our technique, we also propose a variant that guarantees the
same performance bounds even if the rewards are heavy-tailed.
Safe Adaptive Importance Sampling
Authors: Sebastian U. Stich, Anant Raj, Martin Jaggi
Date: 2017-11-07 00:00:00
Proceeding: NeurIPS 2017 12

Importance sampling has become an indispensable strategy to speed up
optimization algorithms for large-scale applications. Improved adaptive
variants - using importance values defined by the complete gradient information
which changes during optimization - enjoy favorable theoretical properties, but
are typically computationally infeasible. In this paper we propose an efficient
approximation of gradient-based sampling, which is based on safe bounds on the
gradient. The proposed sampling distribution is (i) provably the best sampling
with respect to the given bounds, (ii) always better than uniform sampling and
fixed importance sampling and (iii) can efficiently be computed - in many
applications at negligible extra cost. The proposed sampling scheme is generic
and can easily be integrated into existing algorithms. In particular, we show
that coordinate-descent (CD) and stochastic gradient descent (SGD) can enjoy
significant a speed-up under the novel scheme. The proven efficiency of the
proposed sampling is verified by extensive numerical testing.
Information-Theoretic Bounds and Approximations in Neural Population Coding
Authors: Wentao Huang, Kechen Zhang
Date: 2016-11-04 00:00:00
Tasks: Dimensionality Reduction, valid

While Shannon's mutual information has widespread applications in many
disciplines, for practical applications it is often difficult to calculate its
value accurately for high-dimensional variables because of the curse of
dimensionality. This paper is focused on effective approximation methods for
evaluating mutual information in the context of neural population coding. For
large but finite neural populations, we derive several information-theoretic
asymptotic bounds and approximation formulas that remain valid in
high-dimensional spaces. We prove that optimizing the population density
distribution based on these approximation formulas is a convex optimization
problem which allows efficient numerical solutions. Numerical simulation
results confirmed that our asymptotic formulas were highly accurate for
approximating mutual information for large neural populations. In special
cases, the approximation formulas are exactly equal to the true mutual
information. We also discuss techniques of variable transformation and
dimensionality reduction to facilitate computation of the approximations.
Tensor-Generative Adversarial Network with Two-dimensional Sparse Coding: Application to Real-time Indoor Localization
Authors: Chenxiao Zhu, Lingqing Xu, Xiao-Yang Liu, Feng Qian
Date: 2017-11-07 00:00:00
Tasks: Generative Adversarial Network, Indoor Localization, Super-Resolution

Localization technology is important for the development of indoor
location-based services (LBS). Global Positioning System (GPS) becomes invalid
in indoor environments due to the non-line-of-sight issue, so it is urgent to
develop a real-time high-accuracy localization approach for smartphones.
However, accurate localization is challenging due to issues such as real-time
response requirements, limited fingerprint samples and mobile device storage.
To address these problems, we propose a novel deep learning architecture:
Tensor-Generative Adversarial Network (TGAN).
  We first introduce a transform-based 3D tensor to model fingerprint samples.
Instead of those passive methods that construct a fingerprint database as a
prior, our model applies artificial neural network with deep learning to train
network classifiers and then gives out estimations. Then we propose a novel
tensor-based super-resolution scheme using the generative adversarial network
(GAN) that adopts sparse coding as the generator network and a residual
learning network as the discriminator. Further, we analyze the performance of
tensor-GAN and implement a trace-based localization experiment, which achieves
better performance. Compared to existing methods for smartphones indoor
positioning, that are energy-consuming and high demands on devices, TGAN can
give out an improved solution in localization accuracy, response time and
implementation complexity.
Extractive Multi-document Summarization Using Multilayer Networks
Authors: Jorge V. Tohalino, Diego R. Amancio
Date: 2017-11-07 00:00:00
Tasks: Document Summarization, Multi-Document Summarization

Huge volumes of textual information has been produced every single day. In
order to organize and understand such large datasets, in recent years,
summarization techniques have become popular. These techniques aims at finding
relevant, concise and non-redundant content from such a big data. While network
methods have been adopted to model texts in some scenarios, a systematic
evaluation of multilayer network models in the multi-document summarization
task has been limited to a few studies. Here, we evaluate the performance of a
multilayer-based method to select the most relevant sentences in the context of
an extractive multi document summarization (MDS) task. In the adopted model,
nodes represent sentences and edges are created based on the number of shared
words between sentences. Differently from previous studies in multi-document
summarization, we make a distinction between edges linking sentences from
different documents (inter-layer) and those connecting sentences from the same
document (intra-layer). As a proof of principle, our results reveal that such a
discrimination between intra- and inter-layer in a multilayered representation
is able to improve the quality of the generated summaries. This piece of
information could be used to improve current statistical methods and related
textual models.
Unbounded cache model for online language modeling with open vocabulary
Authors: Edouard Grave, Moustapha Cisse, Armand Joulin
Date: 2017-11-07 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Language Modeling, Language Modelling, Quantization

Recently, continuous cache models were proposed as extensions to recurrent
neural network language models, to adapt their predictions to local changes in
the data distribution. These models only capture the local context, of up to a
few thousands tokens. In this paper, we propose an extension of continuous
cache models, which can scale to larger contexts. In particular, we use a large
scale non-parametric memory component that stores all the hidden activations
seen in the past. We leverage recent advances in approximate nearest neighbor
search and quantization algorithms to store millions of representations while
searching them efficiently. We conduct extensive experiments showing that our
approach significantly improves the perplexity of pre-trained language models
on new distributions, and can scale efficiently to much larger contexts than
previously proposed local cache models.
Attend and Predict: Understanding Gene Regulation by Selective Attention on Chromatin
Authors: Ritambhara Singh, Jack Lanchantin, Arshdeep Sekhon, Yanjun Qi
Date: 2017-08-01 00:00:00
Proceeding: NeurIPS 2017 12

The past decade has seen a revolution in genomic technologies that enable a
flood of genome-wide profiling of chromatin marks. Recent literature tried to
understand gene regulation by predicting gene expression from large-scale
chromatin measurements. Two fundamental challenges exist for such learning
tasks: (1) genome-wide chromatin signals are spatially structured,
high-dimensional and highly modular; and (2) the core aim is to understand what
are the relevant factors and how they work together? Previous studies either
failed to model complex dependencies among input signals or relied on separate
feature analysis to explain the decisions. This paper presents an
attention-based deep learning approach; we call AttentiveChrome, that uses a
unified architecture to model and to interpret dependencies among chromatin
factors for controlling gene regulation. AttentiveChrome uses a hierarchy of
multiple Long short-term memory (LSTM) modules to encode the input signals and
to model how various chromatin marks cooperate automatically. AttentiveChrome
trains two levels of attention jointly with the target prediction, enabling it
to attend differentially to relevant marks and to locate important positions
per mark. We evaluate the model across 56 different cell types (tasks) in
human. Not only is the proposed architecture more accurate, but its attention
scores also provide a better interpretation than state-of-the-art feature
visualization methods such as saliency map.
  Code and data are shared at www.deepchrome.org
Streaming Robust Submodular Maximization: A Partitioned Thresholding Approach
Authors: Slobodan Mitrović, Ilija Bogunovic, Ashkan Norouzi-Fard, Jakub Tarnawski, Volkan Cevher
Date: 2017-11-07 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Data Summarization

We study the classical problem of maximizing a monotone submodular function
subject to a cardinality constraint k, with two additional twists: (i) elements
arrive in a streaming fashion, and (ii) m items from the algorithm's memory are
removed after the stream is finished. We develop a robust submodular algorithm
STAR-T. It is based on a novel partitioning structure and an exponentially
decreasing thresholding rule. STAR-T makes one pass over the data and retains a
short but robust summary. We show that after the removal of any m elements from
the obtained summary, a simple greedy algorithm STAR-T-GREEDY that runs on the
remaining elements achieves a constant-factor approximation guarantee. In two
different data summarization tasks, we demonstrate that it matches or
outperforms existing greedy and streaming methods, even if they are allowed the
benefit of knowing the removed subset in advance.
Image Captioning and Classification of Dangerous Situations
Authors: Octavio Arriaga, Paul Plöger, Matias Valdenegro-Toro
Date: 2017-11-07 00:00:00
Tasks: Anomaly Detection, Classification, General Classification, Image Captioning

Current robot platforms are being employed to collaborate with humans in a
wide range of domestic and industrial tasks. These environments require
autonomous systems that are able to classify and communicate anomalous
situations such as fires, injured persons, car accidents; or generally, any
potentially dangerous situation for humans. In this paper we introduce an
anomaly detection dataset for the purpose of robot applications as well as the
design and implementation of a deep learning architecture that classifies and
describes dangerous situations using only a single image as input. We report a
classification accuracy of 97 % and METEOR score of 16.2. We will make the
dataset publicly available after this paper is accepted.
A Convolutional Neural Network for Search Term Detection
Authors: Hojjat Salehinejad, Joseph Barfett, Parham Aarabi, Shahrokh Valaee, Errol Colak, Bruce Gray, Tim Dowdell
Date: 2017-08-07 00:00:00

Pathfinding in hospitals is challenging for patients, visitors, and even
employees. Many people have experienced getting lost due to lack of clear
guidance, large footprint of hospitals, and confusing array of hospital wings.
In this paper, we propose Halo; An indoor navigation application based on
voice-user interaction to help provide directions for users without assistance
of a localization system. The main challenge is accurate detection of origin
and destination search terms. A custom convolutional neural network (CNN) is
proposed to detect origin and destination search terms from transcription of a
submitted speech query. The CNN is trained based on a set of queries tailored
specifically for hospital and clinic environments. Performance of the proposed
model is studied and compared with Levenshtein distance-based word matching.
Learning Distributed Representations of Texts and Entities from Knowledge Base
Authors: Ikuya Yamada, Hiroyuki Shindo, Hideaki Takeda, Yoshiyasu Takefuji
Date: 2017-05-06 00:00:00
Proceeding: TACL 2017 1
Tasks: Entity Disambiguation, Entity Linking, Question Answering, Sentence

We describe a neural network model that jointly learns distributed
representations of texts and knowledge base (KB) entities. Given a text in the
KB, we train our proposed model to predict entities that are relevant to the
text. Our model is designed to be generic with the ability to address various
NLP tasks with ease. We train the model using a large corpus of texts and their
entity annotations extracted from Wikipedia. We evaluated the model on three
important NLP tasks (i.e., sentence textual similarity, entity linking, and
factoid question answering) involving both unsupervised and supervised
settings. As a result, we achieved state-of-the-art results on all three of
these tasks. Our code and trained models are publicly available for further
academic research.
Learning High Dynamic Range from Outdoor Panoramas
Authors: Jinsong Zhang, Jean-François Lalonde
Date: 2017-03-29 00:00:00
Proceeding: ICCV 2017 10
Tasks: Vocal Bursts Intensity Prediction

Outdoor lighting has extremely high dynamic range. This makes the process of
capturing outdoor environment maps notoriously challenging since special
equipment must be used. In this work, we propose an alternative approach. We
first capture lighting with a regular, LDR omnidirectional camera, and aim to
recover the HDR after the fact via a novel, learning-based inverse tonemapping
method. We propose a deep autoencoder framework which regresses linear, high
dynamic range data from non-linear, saturated, low dynamic range panoramas. We
validate our method through a wide set of experiments on synthetic data, as
well as on a novel dataset of real photographs with ground truth. Our approach
finds applications in a variety of settings, ranging from outdoor light capture
to image matching.