pwc-archive/papers-with-abstracts huggingface dataset:
Multi-modal Feedback for Affordance-driven Interactive Reinforcement Learning
Authors: Francisco Cruz, German I. Parisi, Stefan Wermter
Date: 2018-07-26 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Interactive reinforcement learning (IRL) extends traditional reinforcement
learning (RL) by allowing an agent to interact with parent-like trainers during
a task. In this paper, we present an IRL approach using dynamic audio-visual
input in terms of vocal commands and hand gestures as feedback. Our
architecture integrates multi-modal information to provide robust commands from
multiple sensory cues along with a confidence value indicating the
trustworthiness of the feedback. The integration process also considers the
case in which the two modalities convey incongruent information. Additionally,
we modulate the influence of sensory-driven feedback in the IRL task using
goal-oriented knowledge in terms of contextual affordances. We implement a
neural network architecture to predict the effect of performed actions with
different objects to avoid failed-states, i.e., states from which it is not
possible to accomplish the task. In our experimental setup, we explore the
interplay of multimodal feedback and task-specific affordances in a robot
cleaning scenario. We compare the learning performance of the agent under four
different conditions: traditional RL, multi-modal IRL, and each of these two
setups with the use of contextual affordances. Our experiments show that the
best performance is obtained by using audio-visual feedback with
affordancemodulated IRL. The obtained results demonstrate the importance of
multi-modal sensory processing integrated with goal-oriented knowledge in IRL
tasks.
Recurrent Fusion Network for Image Captioning
Authors: Wenhao Jiang, Lin Ma, Yu-Gang Jiang, Wei Liu, Tong Zhang
Date: 2018-07-26 00:00:00
Proceeding: ECCV 2018 9
Tasks: Decoder, Image Captioning

Recently, much advance has been made in image captioning, and an
encoder-decoder framework has been adopted by all the state-of-the-art models.
Under this framework, an input image is encoded by a convolutional neural
network (CNN) and then translated into natural language with a recurrent neural
network (RNN). The existing models counting on this framework merely employ one
kind of CNNs, e.g., ResNet or Inception-X, which describe image contents from
only one specific view point. Thus, the semantic meaning of an input image
cannot be comprehensively understood, which restricts the performance of
captioning. In this paper, in order to exploit the complementary information
from multiple encoders, we propose a novel Recurrent Fusion Network (RFNet) for
tackling image captioning. The fusion process in our model can exploit the
interactions among the outputs of the image encoders and then generate new
compact yet informative representations for the decoder. Experiments on the
MSCOCO dataset demonstrate the effectiveness of our proposed RFNet, which sets
a new state-of-the-art for image captioning.
Bayesian Optimal Design of Experiments For Inferring The Statistical Expectation Of A Black-Box Function
Authors: Piyush Pandita, Ilias Bilionis, Jitesh Panchal
Date: 2018-07-26 00:00:00

Bayesian optimal design of experiments (BODE) has been successful in
acquiring information about a quantity of interest (QoI) which depends on a
black-box function. BODE is characterized by sequentially querying the function
at specific designs selected by an infill-sampling criterion. However, most
current BODE methods operate in specific contexts like optimization, or
learning a universal representation of the black-box function. The objective of
this paper is to design a BODE for estimating the statistical expectation of a
physical response surface. This QoI is omnipresent in uncertainty propagation
and design under uncertainty problems. Our hypothesis is that an optimal BODE
should be maximizing the expected information gain in the QoI. We represent the
information gain from a hypothetical experiment as the Kullback-Liebler (KL)
divergence between the prior and the posterior probability distributions of the
QoI. The prior distribution of the QoI is conditioned on the observed data and
the posterior distribution of the QoI is conditioned on the observed data and a
hypothetical experiment. The main contribution of this paper is the derivation
of a semi-analytic mathematical formula for the expected information gain about
the statistical expectation of a physical response. The developed BODE is
validated on synthetic functions with varying number of input-dimensions. We
demonstrate the performance of the methodology on a steel wire manufacturing
problem.
Person Re-identification with Deep Similarity-Guided Graph Neural Network
Authors: Yantao Shen, Hongsheng Li, Shuai Yi, Dapeng Chen, Xiaogang Wang
Date: 2018-07-26 00:00:00
Proceeding: ECCV 2018 9
Tasks: Graph Neural Network, Person Re-Identification, Relation

The person re-identification task requires to robustly estimate visual
similarities between person images. However, existing person re-identification
models mostly estimate the similarities of different image pairs of probe and
gallery images independently while ignores the relationship information between
different probe-gallery pairs. As a result, the similarity estimation of some
hard samples might not be accurate. In this paper, we propose a novel deep
learning framework, named Similarity-Guided Graph Neural Network (SGGNN) to
overcome such limitations. Given a probe image and several gallery images,
SGGNN creates a graph to represent the pairwise relationships between
probe-gallery pairs (nodes) and utilizes such relationships to update the
probe-gallery relation features in an end-to-end manner. Accurate similarity
estimation can be achieved by using such updated probe-gallery relation
features for prediction. The input features for nodes on the graph are the
relation features of different probe-gallery image pairs. The probe-gallery
relation feature updating is then performed by the messages passing in SGGNN,
which takes other nodes' information into account for similarity estimation.
Different from conventional GNN approaches, SGGNN learns the edge weights with
rich labels of gallery instance pairs directly, which provides relation fusion
more precise information. The effectiveness of our proposed method is validated
on three public person re-identification datasets.
Bottom-up Pose Estimation of Multiple Person with Bounding Box Constraint
Authors: Miaopeng Li, Zimeng Zhou, Jie Li, Xinguo Liu
Date: 2018-07-26 00:00:00
Tasks: Multi-Person Pose Estimation, Pose Estimation

In this work, we propose a new method for multi-person pose estimation which
combines the traditional bottom-up and the top-down methods. Specifically, we
perform the network feed-forwarding in a bottom-up manner, and then parse the
poses with bounding box constraints in a top-down manner. In contrast to the
previous top-down methods, our method is robust to bounding box shift and
tightness. We extract features from an original image by a residual network and
train the network to learn both the confidence maps of joints and the
connection relationships between joints. During testing, the predicted
confidence maps, the connection relationships and the bounding boxes are used
to parse the poses of all persons. The experimental results showed that our
method learns more accurate human poses especially in challenging situations
and gains better time performance, compared with the bottom-up and the top-down
methods.
A Minimal Closed-Form Solution for Multi-Perspective Pose Estimation using Points and Lines
Authors: Pedro Miraldo, Tiago Dias, Srikumar Ramalingam
Date: 2018-07-26 00:00:00
Proceeding: ECCV 2018 9
Tasks: Computational Efficiency, Form, Pose Estimation, Self-Driving Cars

We propose a minimal solution for pose estimation using both points and lines
for a multi-perspective camera. In this paper, we treat the multi-perspective
camera as a collection of rigidly attached perspective cameras. These type of
imaging devices are useful for several computer vision applications that
require a large coverage such as surveillance, self-driving cars, and
motion-capture studios. While prior methods have considered the cases using
solely points or lines, the hybrid case involving both points and lines has not
been solved for multi-perspective cameras. We present the solutions for two
cases. In the first case, we are given 2D to 3D correspondences for two points
and one line. In the later case, we are given 2D to 3D correspondences for one
point and two lines. We show that the solution for the case of two points and
one line can be formulated as a fourth degree equation. This is interesting
because we can get a closed-form solution and thereby achieve high
computational efficiency. The later case involving two lines and one point can
be mapped to an eighth degree equation. We show simulations and real
experiments to demonstrate the advantages and benefits over existing methods.
Face De-Spoofing: Anti-Spoofing via Noise Modeling
Authors: Amin Jourabloo, Yaojie Liu, Xiaoming Liu
Date: 2018-07-26 00:00:00
Proceeding: ECCV 2018 9
Tasks: Denoising, Face Anti-Spoofing

Many prior face anti-spoofing works develop discriminative models for
recognizing the subtle differences between live and spoof faces. Those
approaches often regard the image as an indivisible unit, and process it
holistically, without explicit modeling of the spoofing process. In this work,
motivated by the noise modeling and denoising algorithms, we identify a new
problem of face de-spoofing, for the purpose of anti-spoofing: inversely
decomposing a spoof face into a spoof noise and a live face, and then utilizing
the spoof noise for classification. A CNN architecture with proper constraints
and supervisions is proposed to overcome the problem of having no ground truth
for the decomposition. We evaluate the proposed method on multiple face
anti-spoofing databases. The results show promising improvements due to our
spoof noise modeling. Moreover, the estimated spoof noise provides a
visualization which helps to understand the added spoof noise by each spoof
medium.
A Collaborative Approach to Angel and Venture Capital Investment Recommendations
Authors: Xinyi Liu, Artit Wangperawong
Date: 2018-07-26 00:00:00
Tasks: Prediction

Matrix factorization was used to generate investment recommendations for
investors. An iterative conjugate gradient method was used to optimize the
regularized squared-error loss function. The number of latent factors, number
of iterations, and regularization values were explored. Overfitting can be
addressed by either early stopping or regularization parameter tuning. The
model achieved the highest average prediction accuracy of 13.3%. With a similar
model, the same dataset was used to generate investor recommendations for
companies undergoing fundraising, which achieved highest prediction accuracy of
11.1%.
Learning to guide task and motion planning using score-space representation
Authors: Beomjoon Kim, Zi Wang, Leslie Pack Kaelbling, Tomas Lozano-Perez
Date: 2018-07-26 00:00:00
Tasks: Motion Planning, Task and Motion Planning

In this paper, we propose a learning algorithm that speeds up the search in
task and motion planning problems. Our algorithm proposes solutions to three
different challenges that arise in learning to improve planning efficiency:
what to predict, how to represent a planning problem instance, and how to
transfer knowledge from one problem instance to another. We propose a method
that predicts constraints on the search space based on a generic representation
of a planning problem instance, called score-space, where we represent a
problem instance in terms of the performance of a set of solutions attempted so
far. Using this representation, we transfer knowledge, in the form of
constraints, from previous problems based on the similarity in score space. We
design a sequential algorithm that efficiently predicts these constraints, and
evaluate it in three different challenging task and motion planning problems.
Results indicate that our approach performs orders of magnitudes faster than an
unguided planner
Iterative Crowd Counting
Authors: Viresh Ranjan, Hieu Le, Minh Hoai
Date: 2018-07-26 00:00:00
Proceeding: ECCV 2018 9
Tasks: Crowd Counting, Density Estimation

In this work, we tackle the problem of crowd counting in images. We present a
Convolutional Neural Network (CNN) based density estimation approach to solve
this problem. Predicting a high resolution density map in one go is a
challenging task. Hence, we present a two branch CNN architecture for
generating high resolution density maps, where the first branch generates a low
resolution density map, and the second branch incorporates the low resolution
prediction and feature maps from the first branch to generate a high resolution
density map. We also propose a multi-stage extension of our approach where each
stage in the pipeline utilizes the predictions from all the previous stages.
Empirical comparison with the previous state-of-the-art crowd counting methods
shows that our method achieves the lowest mean absolute error on three
challenging crowd counting benchmarks: Shanghaitech, WorldExpo'10, and UCF
datasets.
Rethinking the Form of Latent States in Image Captioning
Authors: Bo Dai, Deming Ye, Dahua Lin
Date: 2018-07-26 00:00:00
Proceeding: ECCV 2018 9
Tasks: Caption Generation, Form, Image Captioning

RNNs and their variants have been widely adopted for image captioning. In
RNNs, the production of a caption is driven by a sequence of latent states.
Existing captioning models usually represent latent states as vectors, taking
this practice for granted. We rethink this choice and study an alternative
formulation, namely using two-dimensional maps to encode latent states. This is
motivated by the curiosity about a question: how the spatial structures in the
latent states affect the resultant captions? Our study on MSCOCO and Flickr30k
leads to two significant observations. First, the formulation with 2D states is
generally more effective in captioning, consistently achieving higher
performance with comparable parameter sizes. Second, 2D states preserve spatial
locality. Taking advantage of this, we visually reveal the internal dynamics in
the process of caption generation, as well as the connections between input
visual domain and output linguistic domain.
Pythia v0.1: the Winning Entry to the VQA Challenge 2018
Authors: Yu Jiang, Vivek Natarajan, Xinlei Chen, Marcus Rohrbach, Dhruv Batra, Devi Parikh
Date: 2018-07-26 00:00:00
Tasks: Data Augmentation, Visual Question Answering (VQA)

This document describes Pythia v0.1, the winning entry from Facebook AI
Research (FAIR)'s A-STAR team to the VQA Challenge 2018.
  Our starting point is a modular re-implementation of the bottom-up top-down
(up-down) model. We demonstrate that by making subtle but important changes to
the model architecture and the learning rate schedule, fine-tuning image
features, and adding data augmentation, we can significantly improve the
performance of the up-down model on VQA v2.0 dataset -- from 65.67% to 70.22%.
  Furthermore, by using a diverse ensemble of models trained with different
features and on different datasets, we are able to significantly improve over
the 'standard' way of ensembling (i.e. same model with different random seeds)
by 1.31%. Overall, we achieve 72.27% on the test-std split of the VQA v2.0
dataset. Our code in its entirety (training, evaluation, data-augmentation,
ensembling) and pre-trained models are publicly available at:
https://github.com/facebookresearch/pythia
Multi-temporal Sentinel-1 and -2 Data Fusion for Optical Image Simulation
Authors: Wei He, Naoto Yokoya
Date: 2018-07-26 00:00:00
Tasks: Cloud Removal, Generative Adversarial Network

In this paper, we present the optical image simulation from a synthetic
aperture radar (SAR) data using deep learning based methods. Two models, i.e.,
optical image simulation directly from the SAR data and from multi-temporal
SARoptical data, are proposed to testify the possibilities. The deep learning
based methods that we chose to achieve the models are a convolutional neural
network (CNN) with a residual architecture and a conditional generative
adversarial network (cGAN). We validate our models using the Sentinel-1 and -2
datasets. The experiments demonstrate that the model with multi-temporal
SAR-optical data can successfully simulate the optical image, meanwhile, the
model with simple SAR data as input failed. The optical image simulation
results indicate the possibility of SARoptical information blending for the
subsequent applications such as large-scale cloud removal, and optical data
temporal superresolution. We also investigate the sensitivity of the proposed
models against the training samples, and reveal possible future directions.
Learning to Forecast and Refine Residual Motion for Image-to-Video Generation
Authors: Long Zhao, Xi Peng, Yu Tian, Mubbasir Kapadia, Dimitris Metaxas
Date: 2018-07-26 00:00:00
Proceeding: ECCV 2018 9
Tasks: Human Pose Forecasting, Image to Video Generation, Translation, Video Generation

We consider the problem of image-to-video translation, where an input image
is translated into an output video containing motions of a single object.
Recent methods for such problems typically train transformation networks to
generate future frames conditioned on the structure sequence. Parallel work has
shown that short high-quality motions can be generated by spatiotemporal
generative networks that leverage temporal knowledge from the training data. We
combine the benefits of both approaches and propose a two-stage generation
framework where videos are generated from structures and then refined by
temporal signals. To model motions more efficiently, we train networks to learn
residual motion between the current and future frames, which avoids learning
motion-irrelevant details. We conduct extensive experiments on two
image-to-video translation tasks: facial expression retargeting and human pose
forecasting. Superior results over the state-of-the-art methods on both tasks
demonstrate the effectiveness of our approach.
Variational Memory Encoder-Decoder
Authors: Hung Le, Truyen Tran, Thin Nguyen, Svetha Venkatesh
Date: 2018-07-26 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Decoder

Introducing variability while maintaining coherence is a core task in
learning to generate utterances in conversation. Standard neural
encoder-decoder models and their extensions using conditional variational
autoencoder often result in either trivial or digressive responses. To overcome
this, we explore a novel approach that injects variability into neural
encoder-decoder via the use of external memory as a mixture model, namely
Variational Memory Encoder-Decoder (VMED). By associating each memory read with
a mode in the latent mixture distribution at each timestep, our model can
capture the variability observed in sequential data such as natural
conversations. We empirically compare the proposed model against other recent
approaches on various conversational datasets. The results show that VMED
consistently achieves significant improvement over others in both metric-based
and qualitative evaluations.
Computationally Efficient Measures of Internal Neuron Importance
Authors: Avanti Shrikumar, Jocelin Su, Anshul Kundaje
Date: 2018-07-26 00:00:00

The challenge of assigning importance to individual neurons in a network is
of interest when interpreting deep learning models. In recent work, Dhamdhere
et al. proposed Total Conductance, a "natural refinement of Integrated
Gradients" for attributing importance to internal neurons. Unfortunately, the
authors found that calculating conductance in tensorflow required the addition
of several custom gradient operators and did not scale well. In this work, we
show that the formula for Total Conductance is mathematically equivalent to
Path Integrated Gradients computed on a hidden layer in the network. We provide
a scalable implementation of Total Conductance using standard tensorflow
gradient operators that we call Neuron Integrated Gradients. We compare Neuron
Integrated Gradients to DeepLIFT, a pre-existing computationally efficient
approach that is applicable to calculating internal neuron importance. We find
that DeepLIFT produces strong empirical results and is faster to compute, but
because it lacks the theoretical properties of Neuron Integrated Gradients, it
may not always be preferred in practice. Colab notebook reproducing results:
http://bit.ly/neuronintegratedgradients
On Strengthening the Logic of Iterated Belief Revision: Proper Ordinal Interval Operators
Authors: Richard Booth, Jake Chandler
Date: 2018-07-26 00:00:00
Tasks: Relation

Darwiche and Pearl's seminal 1997 article outlined a number of baseline
principles for a logic of iterated belief revision. These principles, the DP
postulates, have been supplemented in a number of alternative ways. Most of the
suggestions made have resulted in a form of `reductionism' that identifies
belief states with orderings of worlds. However, this position has recently
been criticised as being unacceptably strong. Other proposals, such as the
popular principle (P), aka `Independence', characteristic of `admissible'
revision operators, remain commendably more modest. In this paper, we
supplement both the DP postulates and (P) with a number of novel conditions.
While the DP postulates constrain the relation between a prior and a posterior
conditional belief set, our new principles notably govern the relation between
two posterior conditional belief sets obtained from a common prior by different
revisions. We show that operators from the resulting family, which subsumes
both lexicographic and restrained revision, can be represented as relating
belief states that are associated with a `proper ordinal interval' (POI)
assignment, a structure more fine-grained than a simple ordering of worlds. We
close the paper by noting that these operators satisfy iterated versions of a
large number of AGM era postulates, including Superexpansion, that are not
sound for admissible operators in general.
Maximum Inner-Product Search using Tree Data-structures
Authors: Parikshit Ram, Alexander G. Gray
Date: 2012-02-28 00:00:00

The problem of {\em efficiently} finding the best match for a query in a
given set with respect to the Euclidean distance or the cosine similarity has
been extensively studied in literature. However, a closely related problem of
efficiently finding the best match with respect to the inner product has never
been explored in the general setting to the best of our knowledge. In this
paper we consider this general problem and contrast it with the existing
best-match algorithms. First, we propose a general branch-and-bound algorithm
using a tree data structure. Subsequently, we present a dual-tree algorithm for
the case where there are multiple queries. Finally we present a new data
structure for increasing the efficiency of the dual-tree algorithm. These
branch-and-bound algorithms involve novel bounds suited for the purpose of
best-matching with inner products. We evaluate our proposed algorithms on a
variety of data sets from various applications, and exhibit up to five orders
of magnitude improvement in query time over the naive search technique.