pwc-archive/papers-with-abstracts huggingface dataset:
Occlusion-aware Hand Pose Estimation Using Hierarchical Mixture Density Network
Authors: Qi Ye, Tae-Kyun Kim
Date: 2017-11-29 00:00:00
Proceeding: ECCV 2018 9
Tasks: Hand Pose Estimation, Pose Estimation

Learning and predicting the pose parameters of a 3D hand model given an
image, such as locations of hand joints, is challenging due to large viewpoint
changes and articulations, and severe self-occlusions exhibited particularly in
egocentric views. Both feature learning and prediction modeling have been
investigated to tackle the problem. Though effective, most existing
discriminative methods yield a single deterministic estimation of target poses.
Due to their single-value mapping intrinsic, they fail to adequately handle
self-occlusion problems, where occluded joints present multiple modes. In this
paper, we tackle the self-occlusion issue and provide a complete description of
observed poses given an input depth image by a novel method called hierarchical
mixture density networks (HMDN). The proposed method leverages the
state-of-the-art hand pose estimators based on Convolutional Neural Networks to
facilitate feature learning, while it models the multiple modes in a two-level
hierarchy to reconcile single-valued and multi-valued mapping in its output.
The whole framework with a mixture of two differentiable density functions is
naturally end-to-end trainable. In the experiments, HMDN produces interpretable
and diverse candidate samples, and significantly outperforms the
state-of-the-art methods on two benchmarks with occlusions, and performs
comparably on another benchmark free of occlusions.
Learning Selfie-Friendly Abstraction from Artistic Style Images
Authors: Yicun Liu, Jimmy Ren, Jianbo Liu, Jiawei Zhang, Xiaohao Chen
Date: 2018-05-05 00:00:00
Tasks: Style Transfer

Artistic style transfer can be thought as a process to generate different
versions of abstraction of the original image. However, most of the artistic
style transfer operators are not optimized for human faces thus mainly suffers
from two undesirable features when applying them to selfies. First, the edges
of human faces may unpleasantly deviate from the ones in the original image.
Second, the skin color is far from faithful to the original one which is
usually problematic in producing quality selfies. In this paper, we take a
different approach and formulate this abstraction process as a gradient domain
learning problem. We aim to learn a type of abstraction which not only achieves
the specified artistic style but also circumvents the two aforementioned
drawbacks thus highly applicable to selfie photography. We also show that our
method can be directly generalized to videos with high inter-frame consistency.
Our method is also robust to non-selfie images, and the generalization to
various kinds of real-life scenes is discussed. We will make our code publicly
available.
A Simple Cache Model for Image Recognition
Authors: A. Emin Orhan
Date: 2018-05-21 00:00:00
Proceeding: NeurIPS 2018
Tasks: General Classification, Language Modeling, Language Modelling, model

Training large-scale image recognition models is computationally expensive.
This raises the question of whether there might be simple ways to improve the
test performance of an already trained model without having to re-train or
fine-tune it with new data. Here, we show that, surprisingly, this is indeed
possible. The key observation we make is that the layers of a deep network
close to the output layer contain independent, easily extractable
class-relevant information that is not contained in the output layer itself. We
propose to extract this extra class-relevant information using a simple
key-value cache memory to improve the classification performance of the model
at test time. Our cache memory is directly inspired by a similar cache model
previously proposed for language modeling (Grave et al., 2017). This cache
component does not require any training or fine-tuning; it can be applied to
any pre-trained model and, by properly setting only two hyper-parameters, leads
to significant improvements in its classification performance. Improvements are
observed across several architectures and datasets. In the cache component,
using features extracted from layers close to the output (but not from the
output layer itself) as keys leads to the largest improvements. Concatenating
features from multiple layers to form keys can further improve performance over
using single-layer features as keys. The cache component also has a
regularizing effect, a simple consequence of which is that it substantially
increases the robustness of models against adversarial attacks.
Learning Maximum-A-Posteriori Perturbation Models for Structured Prediction in Polynomial Time
Authors: Asish Ghoshal, Jean Honorio
Date: 2018-05-21 00:00:00
Proceeding: ICML 2018 7
Tasks: Structured Prediction

MAP perturbation models have emerged as a powerful framework for inference in
structured prediction. Such models provide a way to efficiently sample from the
Gibbs distribution and facilitate predictions that are robust to random noise.
In this paper, we propose a provably polynomial time randomized algorithm for
learning the parameters of perturbed MAP predictors. Our approach is based on
minimizing a novel Rademacher-based generalization bound on the expected loss
of a perturbed MAP predictor, which can be computed in polynomial time. We
obtain conditions under which our randomized learning algorithm can guarantee
generalization to unseen examples.
Depth-Limited Solving for Imperfect-Information Games
Authors: Noam Brown, Tuomas Sandholm, Brandon Amos
Date: 2018-05-21 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: CPU

A fundamental challenge in imperfect-information games is that states do not
have well-defined values. As a result, depth-limited search algorithms used in
single-agent settings and perfect-information games do not apply. This paper
introduces a principled way to conduct depth-limited solving in
imperfect-information games by allowing the opponent to choose among a number
of strategies for the remainder of the game at the depth limit. Each one of
these strategies results in a different set of values for leaf nodes. This
forces an agent to be robust to the different strategies an opponent may
employ. We demonstrate the effectiveness of this approach by building a
master-level heads-up no-limit Texas hold'em poker AI that defeats two prior
top agents using only a 4-core CPU and 16 GB of memory. Developing such a
powerful agent would have previously required a supercomputer.
Non-Vacuous Generalization Bounds at the ImageNet Scale: A PAC-Bayesian Compression Approach
Authors: Wenda Zhou, Victor Veitch, Morgane Austern, Ryan P. Adams, Peter Orbanz
Date: 2018-04-16 00:00:00
Proceeding: ICLR 2019 5
Tasks: Generalization Bounds

Modern neural networks are highly overparameterized, with capacity to
substantially overfit to training data. Nevertheless, these networks often
generalize well in practice. It has also been observed that trained networks
can often be "compressed" to much smaller representations. The purpose of this
paper is to connect these two empirical observations. Our main technical result
is a generalization bound for compressed networks based on the compressed size.
Combined with off-the-shelf compression algorithms, the bound leads to state of
the art generalization guarantees; in particular, we provide the first
non-vacuous generalization guarantees for realistic architectures applied to
the ImageNet classification problem. As additional evidence connecting
compression and generalization, we show that compressibility of models that
tend to overfit is limited: We establish an absolute limit on expected
compressibility as a function of expected generalization error, where the
expectations are over the random choice of training examples. The bounds are
complemented by empirical results that show an increase in overfitting implies
an increase in the number of bits required to describe a trained network.
Masking: A New Perspective of Noisy Supervision
Authors: Bo Han, Jiangchao Yao, Gang Niu, Mingyuan Zhou, Ivor Tsang, Ya zhang, Masashi Sugiyama
Date: 2018-05-21 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Image Classification

It is important to learn various types of classifiers given training data
with noisy labels. Noisy labels, in the most popular noise model hitherto, are
corrupted from ground-truth labels by an unknown noise transition matrix. Thus,
by estimating this matrix, classifiers can escape from overfitting those noisy
labels. However, such estimation is practically difficult, due to either the
indirect nature of two-step approaches, or not big enough data to afford
end-to-end approaches. In this paper, we propose a human-assisted approach
called Masking that conveys human cognition of invalid class transitions and
naturally speculates the structure of the noise transition matrix. To this end,
we derive a structure-aware probabilistic model incorporating a structure
prior, and solve the challenges from structure extraction and structure
alignment. Thanks to Masking, we only estimate unmasked noise transition
probabilities and the burden of estimation is tremendously reduced. We conduct
extensive experiments on CIFAR-10 and CIFAR-100 with three noise structures as
well as the industrial-level Clothing1M with agnostic noise structure, and the
results show that Masking can improve the robustness of classifiers
significantly.
Hierarchically Structured Reinforcement Learning for Topically Coherent Visual Story Generation
Authors: Qiuyuan Huang, Zhe Gan, Asli Celikyilmaz, Dapeng Wu, Jian-Feng Wang, Xiaodong He
Date: 2018-05-21 00:00:00
Tasks: Decoder, Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Sentence, Story Generation, Visual Storytelling

We propose a hierarchically structured reinforcement learning approach to
address the challenges of planning for generating coherent multi-sentence
stories for the visual storytelling task. Within our framework, the task of
generating a story given a sequence of images is divided across a two-level
hierarchical decoder. The high-level decoder constructs a plan by generating a
semantic concept (i.e., topic) for each image in sequence. The low-level
decoder generates a sentence for each image using a semantic compositional
network, which effectively grounds the sentence generation conditioned on the
topic. The two decoders are jointly trained end-to-end using reinforcement
learning. We evaluate our model on the visual storytelling (VIST) dataset.
Empirical results from both automatic and human evaluations demonstrate that
the proposed hierarchically structured reinforced training achieves
significantly better performance compared to a strong flat deep reinforcement
learning baseline.
The Robust Reading Competition Annotation and Evaluation Platform
Authors: Dimosthenis Karatzas, Lluis Gómez, Anguelos Nicolaou, Marçal Rusiñol
Date: 2017-10-18 00:00:00
Tasks: Management

The ICDAR Robust Reading Competition (RRC), initiated in 2003 and
re-established in 2011, has become a de-facto evaluation standard for robust
reading systems and algorithms. Concurrent with its second incarnation in 2011,
a continuous effort started to develop an on-line framework to facilitate the
hosting and management of competitions. This paper outlines the Robust Reading
Competition Annotation and Evaluation Platform, the backbone of the
competitions. The RRC Annotation and Evaluation Platform is a modular
framework, fully accessible through on-line interfaces. It comprises a
collection of tools and services for managing all processes involved with
defining and evaluating a research task, from dataset definition to annotation
management, evaluation specification and results analysis. Although the
framework has been designed with robust reading research in mind, many of the
provided tools are generic by design. All aspects of the RRC Annotation and
Evaluation Framework are available for research use.
Party Matters: Enhancing Legislative Embeddings with Author Attributes for Vote Prediction
Authors: Anastassia Kornilova, Daniel Argyle, Vlad Eidelman
Date: 2018-05-21 00:00:00
Proceeding: ACL 2018 7

Predicting how Congressional legislators will vote is important for
understanding their past and future behavior. However, previous work on
roll-call prediction has been limited to single session settings, thus did not
consider generalization across sessions. In this paper, we show that metadata
is crucial for modeling voting outcomes in new contexts, as changes between
sessions lead to changes in the underlying data generation process. We show how
augmenting bill text with the sponsors' ideologies in a neural network model
can achieve an average of a 4% boost in accuracy over the previous
state-of-the-art.
Hierarchical Reinforcement Learning with Hindsight
Authors: Andrew Levy, Robert Platt, Kate Saenko
Date: 2018-05-21 00:00:00
Proceeding: ICLR 2019 5
Tasks: Hierarchical Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Reinforcement Learning (RL) algorithms can suffer from poor sample efficiency
when rewards are delayed and sparse. We introduce a solution that enables
agents to learn temporally extended actions at multiple levels of abstraction
in a sample efficient and automated fashion. Our approach combines universal
value functions and hindsight learning, allowing agents to learn policies
belonging to different time scales in parallel. We show that our method
significantly accelerates learning in a variety of discrete and continuous
tasks.
Understanding the Role of Adaptivity in Machine Teaching: The Case of Version Space Learners
Authors: Yuxin Chen, Adish Singla, Oisin Mac Aodha, Pietro Perona, Yisong Yue
Date: 2018-02-14 00:00:00
Proceeding: NeurIPS 2018 12

In real-world applications of education, an effective teacher adaptively
chooses the next example to teach based on the learner's current state.
However, most existing work in algorithmic machine teaching focuses on the
batch setting, where adaptivity plays no role. In this paper, we study the case
of teaching consistent, version space learners in an interactive setting. At
any time step, the teacher provides an example, the learner performs an update,
and the teacher observes the learner's new state. We highlight that adaptivity
does not speed up the teaching process when considering existing models of
version space learners, such as "worst-case" (the learner picks the next
hypothesis randomly from the version space) and "preference-based" (the learner
picks hypothesis according to some global preference). Inspired by human
teaching, we propose a new model where the learner picks hypotheses according
to some local preference defined by the current hypothesis. We show that our
model exhibits several desirable properties, e.g., adaptivity plays a key role,
and the learner's transitions over hypotheses are smooth/interpretable. We
develop efficient teaching algorithms and demonstrate our results via
simulation and user studies.
Reproducibility Report for "Learning To Count Objects In Natural Images For Visual Question Answering"
Authors: Shagun Sodhani, Vardaan Pahuja
Date: 2018-05-21 00:00:00
Tasks: Question Answering, Visual Question Answering, Visual Question Answering (VQA)

This is the reproducibility report for the paper "Learning To Count Objects
In Natural Images For Visual QuestionAnswering"
Turbo Learning for Captionbot and Drawingbot
Authors: Qiuyuan Huang, Pengchuan Zhang, Dapeng Wu, Lei Zhang
Date: 2018-05-21 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Image Captioning, Image Generation, Image to text, Text Generation, Text to Image Generation, Text-to-Image Generation

We study in this paper the problems of both image captioning and
text-to-image generation, and present a novel turbo learning approach to
jointly training an image-to-text generator (a.k.a. CaptionBot) and a
text-to-image generator (a.k.a. DrawingBot). The key idea behind the joint
training is that image-to-text generation and text-to-image generation as dual
problems can form a closed loop to provide informative feedback to each other.
Based on such feedback, we introduce a new loss metric by comparing the
original input with the output produced by the closed loop. In addition to the
old loss metrics used in CaptionBot and DrawingBot, this extra loss metric
makes the jointly trained CaptionBot and DrawingBot better than the separately
trained CaptionBot and DrawingBot. Furthermore, the turbo-learning approach
enables semi-supervised learning since the closed loop can provide
pseudo-labels for unlabeled samples. Experimental results on the COCO dataset
demonstrate that the proposed turbo learning can significantly improve the
performance of both CaptionBot and DrawingBot by a large margin.
Learning to Optimize Tensor Programs
Authors: Tianqi Chen, Lianmin Zheng, Eddie Yan, Ziheng Jiang, Thierry Moreau, Luis Ceze, Carlos Guestrin, Arvind Krishnamurthy
Date: 2018-05-21 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: CPU, Deep Learning, GPU

We introduce a learning-based framework to optimize tensor programs for deep
learning workloads. Efficient implementations of tensor operators, such as
matrix multiplication and high dimensional convolution, are key enablers of
effective deep learning systems. However, existing systems rely on manually
optimized libraries such as cuDNN where only a narrow range of server class
GPUs are well-supported. The reliance on hardware-specific operator libraries
limits the applicability of high-level graph optimizations and incurs
significant engineering costs when deploying to new hardware targets. We use
learning to remove this engineering burden. We learn domain-specific
statistical cost models to guide the search of tensor operator implementations
over billions of possible program variants. We further accelerate the search by
effective model transfer across workloads. Experimental results show that our
framework delivers performance competitive with state-of-the-art hand-tuned
libraries for low-power CPU, mobile GPU, and server-class GPU.
AGI Safety Literature Review
Authors: Tom Everitt, Gary Lea, Marcus Hutter
Date: 2018-05-03 00:00:00

The development of Artificial General Intelligence (AGI) promises to be a
major event. Along with its many potential benefits, it also raises serious
safety concerns (Bostrom, 2014). The intention of this paper is to provide an
easily accessible and up-to-date collection of references for the emerging
field of AGI safety. A significant number of safety problems for AGI have been
identified. We list these, and survey recent research on solving them. We also
cover works on how best to think of AGI from the limited knowledge we have
today, predictions for when AGI will first be created, and what will happen
after its creation. Finally, we review the current public policy on AGI.
VideoCapsuleNet: A Simplified Network for Action Detection
Authors: Kevin Duarte, Yogesh S Rawat, Mubarak Shah
Date: 2018-05-21 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Action Classification, Action Detection, Action Localization, Action Segmentation, Classification, General Classification, Multiple Action Detection, Optical Flow Estimation

The recent advances in Deep Convolutional Neural Networks (DCNNs) have shown
extremely good results for video human action classification, however, action
detection is still a challenging problem. The current action detection
approaches follow a complex pipeline which involves multiple tasks such as tube
proposals, optical flow, and tube classification. In this work, we present a
more elegant solution for action detection based on the recently developed
capsule network. We propose a 3D capsule network for videos, called
VideoCapsuleNet: a unified network for action detection which can jointly
perform pixel-wise action segmentation along with action classification. The
proposed network is a generalization of capsule network from 2D to 3D, which
takes a sequence of video frames as input. The 3D generalization drastically
increases the number of capsules in the network, making capsule routing
computationally expensive. We introduce capsule-pooling in the convolutional
capsule layer to address this issue which makes the voting algorithm tractable.
The routing-by-agreement in the network inherently models the action
representations and various action characteristics are captured by the
predicted capsules. This inspired us to utilize the capsules for action
localization and the class-specific capsules predicted by the network are used
to determine a pixel-wise localization of actions. The localization is further
improved by parameterized skip connections with the convolutional capsule
layers and the network is trained end-to-end with a classification as well as
localization loss. The proposed network achieves sate-of-the-art performance on
multiple action detection datasets including UCF-Sports, J-HMDB, and UCF-101
(24 classes) with an impressive ~20% improvement on UCF-101 and ~15%
improvement on J-HMDB in terms of v-mAP scores.
Multi-Perspective Relevance Matching with Hierarchical ConvNets for Social Media Search
Authors: Jinfeng Rao, Wei Yang, Yuhao Zhang, Ferhan Ture, Jimmy Lin
Date: 2018-05-21 00:00:00
Tasks: Information Retrieval, Retrieval

Despite substantial interest in applications of neural networks to information retrieval, neural ranking models have only been applied to standard ad hoc retrieval tasks over web pages and newswire documents. This paper proposes MP-HCNN (Multi-Perspective Hierarchical Convolutional Neural Network) a novel neural ranking model specifically designed for ranking short social media posts. We identify document length, informal language, and heterogeneous relevance signals as features that distinguish documents in our domain, and present a model specifically designed with these characteristics in mind. Our model uses hierarchical convolutional layers to learn latent semantic soft-match relevance signals at the character, word, and phrase levels. A pooling-based similarity measurement layer integrates evidence from multiple types of matches between the query, the social media post, as well as URLs contained in the post. Extensive experiments using Twitter data from the TREC Microblog Tracks 2011--2014 show that our model significantly outperforms prior feature-based as well and existing neural ranking models. To our best knowledge, this paper presents the first substantial work tackling search over social media posts using neural ranking models.
Numeracy for Language Models: Evaluating and Improving their Ability to Predict Numbers
Authors: Georgios P. Spithourakis, Sebastian Riedel
Date: 2018-05-21 00:00:00
Proceeding: ACL 2018 7
Tasks: Language Modelling

Numeracy is the ability to understand and work with numbers. It is a
necessary skill for composing and understanding documents in clinical,
scientific, and other technical domains. In this paper, we explore different
strategies for modelling numerals with language models, such as memorisation
and digit-by-digit composition, and propose a novel neural architecture that
uses a continuous probability density function to model numerals from an open
vocabulary. Our evaluation on clinical and scientific datasets shows that using
hierarchical models to distinguish numerals from words improves a perplexity
metric on the subset of numerals by 2 and 4 orders of magnitude, respectively,
over non-hierarchical models. A combination of strategies can further improve
perplexity. Our continuous probability density function model reduces mean
absolute percentage errors by 18% and 54% in comparison to the second best
strategy for each dataset, respectively.