pwc-archive/papers-with-abstracts huggingface dataset:
One-Shot Imitation Learning
Authors: Yan Duan, Marcin Andrychowicz, Bradly C. Stadie, Jonathan Ho, Jonas Schneider, Ilya Sutskever, Pieter Abbeel, Wojciech Zaremba
Date: 2017-03-21 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Feature Engineering, Imitation Learning, Meta-Learning

Imitation learning has been commonly applied to solve different tasks in
isolation. This usually requires either careful feature engineering, or a
significant number of samples. This is far from what we desire: ideally, robots
should be able to learn from very few demonstrations of any given task, and
instantly generalize to new situations of the same task, without requiring
task-specific engineering. In this paper, we propose a meta-learning framework
for achieving such capability, which we call one-shot imitation learning.
  Specifically, we consider the setting where there is a very large set of
tasks, and each task has many instantiations. For example, a task could be to
stack all blocks on a table into a single tower, another task could be to place
all blocks on a table into two-block towers, etc. In each case, different
instances of the task would consist of different sets of blocks with different
initial states. At training time, our algorithm is presented with pairs of
demonstrations for a subset of all tasks. A neural net is trained that takes as
input one demonstration and the current state (which initially is the initial
state of the other demonstration of the pair), and outputs an action with the
goal that the resulting sequence of states and actions matches as closely as
possible with the second demonstration. At test time, a demonstration of a
single instance of a new task is presented, and the neural net is expected to
perform well on new instances of this new task. The use of soft attention
allows the model to generalize to conditions and tasks unseen in the training
data. We anticipate that by training this model on a much greater variety of
tasks and settings, we will obtain a general system that can turn any
demonstrations into robust policies that can accomplish an overwhelming variety
of tasks.
  Videos available at https://bit.ly/nips2017-oneshot .
Linearly-Recurrent Autoencoder Networks for Learning Dynamics
Authors: Samuel E. Otto, Clarence W. Rowley
Date: 2017-12-04 00:00:00

This paper describes a method for learning low-dimensional approximations of
nonlinear dynamical systems, based on neural-network approximations of the
underlying Koopman operator. Extended Dynamic Mode Decomposition (EDMD)
provides a useful data-driven approximation of the Koopman operator for
analyzing dynamical systems. This paper addresses a fundamental problem
associated with EDMD: a trade-off between representational capacity of the
dictionary and over-fitting due to insufficient data. A new neural network
architecture combining an autoencoder with linear recurrent dynamics in the
encoded state is used to learn a low-dimensional and highly informative
Koopman-invariant subspace of observables. A method is also presented for
balanced model reduction of over-specified EDMD systems in feature space.
Nonlinear reconstruction using partially linear multi-kernel regression aims to
improve reconstruction accuracy from the low-dimensional state when the data
has complex but intrinsically low-dimensional structure. The techniques
demonstrate the ability to identify Koopman eigenfunctions of the unforced
Duffing equation, create accurate low-dimensional models of an unstable
cylinder wake flow, and make short-time predictions of the chaotic
Kuramoto-Sivashinsky equation.
A+D Net: Training a Shadow Detector with Adversarial Shadow Attenuation
Authors: Hieu Le, Tomas F. Yago Vicente, Vu Nguyen, Minh Hoai, Dimitris Samaras
Date: 2017-12-04 00:00:00
Proceeding: ECCV 2018 9
Tasks: Detecting Shadows, Shadow Detection

We propose a novel GAN-based framework for detecting shadows in images, in
which a shadow detection network (D-Net) is trained together with a shadow
attenuation network (A-Net) that generates adversarial training examples. The
A-Net modifies the original training images constrained by a simplified
physical shadow model and is focused on fooling the D-Net's shadow predictions.
Hence, it is effectively augmenting the training data for D-Net with
hard-to-predict cases. The D-Net is trained to predict shadows in both original
images and generated images from the A-Net. Our experimental results show that
the additional training data from A-Net significantly improves the shadow
detection accuracy of D-Net. Our method outperforms the state-of-the-art
methods on the most challenging shadow detection benchmark (SBU) and also
obtains state-of-the-art results on a cross-dataset task, testing on UCF.
Furthermore, the proposed method achieves accurate real-time shadow detection
at 45 frames per second.
ClusterNet: Detecting Small Objects in Large Scenes by Exploiting Spatio-Temporal Information
Authors: Rodney LaLonde, Dong Zhang, Mubarak Shah
Date: 2017-04-10 00:00:00
Proceeding: CVPR 2018 6
Tasks: Object, object-detection, Object Detection

Object detection in wide area motion imagery (WAMI) has drawn the attention
of the computer vision research community for a number of years. WAMI proposes
a number of unique challenges including extremely small object sizes, both
sparse and densely-packed objects, and extremely large search spaces (large
video frames). Nearly all state-of-the-art methods in WAMI object detection
report that appearance-based classifiers fail in this challenging data and
instead rely almost entirely on motion information in the form of background
subtraction or frame-differencing. In this work, we experimentally verify the
failure of appearance-based classifiers in WAMI, such as Faster R-CNN and a
heatmap-based fully convolutional neural network (CNN), and propose a novel
two-stage spatio-temporal CNN which effectively and efficiently combines both
appearance and motion information to significantly surpass the state-of-the-art
in WAMI object detection. To reduce the large search space, the first stage
(ClusterNet) takes in a set of extremely large video frames, combines the
motion and appearance information within the convolutional architecture, and
proposes regions of objects of interest (ROOBI). These ROOBI can contain from
one to clusters of several hundred objects due to the large video frame size
and varying object density in WAMI. The second stage (FoveaNet) then estimates
the centroid location of all objects in that given ROOBI simultaneously via
heatmap estimation. The proposed method exceeds state-of-the-art results on the
WPAFB 2009 dataset by 5-16% for moving objects and nearly 50% for stopped
objects, as well as being the first proposed method in wide area motion imagery
to detect completely stationary objects.
3D Semantic Trajectory Reconstruction from 3D Pixel Continuum
Authors: Jae Shin Yoon, Ziwei Li, Hyun Soo Park
Date: 2017-12-04 00:00:00
Proceeding: CVPR 2018 6

This paper presents a method to reconstruct dense semantic trajectory stream
of human interactions in 3D from synchronized multiple videos. The interactions
inherently introduce self-occlusion and illumination/appearance/shape changes,
resulting in highly fragmented trajectory reconstruction with noisy and coarse
semantic labels. Our conjecture is that among many views, there exists a set of
views that can confidently recognize the visual semantic label of a 3D
trajectory. We introduce a new representation called 3D semantic map---a
probability distribution over the semantic labels per trajectory. We construct
the 3D semantic map by reasoning about visibility and 2D recognition confidence
based on view-pooling, i.e., finding the view that best represents the
semantics of the trajectory. Using the 3D semantic map, we precisely infer all
trajectory labels jointly by considering the affinity between long range
trajectories via estimating their local rigid transformations. This inference
quantitatively outperforms the baseline approaches in terms of predictive
validity, representation robustness, and affinity effectiveness. We demonstrate
that our algorithm can robustly compute the semantic labels of a large scale
trajectory set involving real-world human interactions with object, scenes, and
people.
Self-supervised Learning of Motion Capture
Authors: Hsiao-Yu Fish Tung, Hsiao-Wei Tung, Ersin Yumer, Katerina Fragkiadaki
Date: 2017-12-04 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: 3D Human Pose Estimation, 3D Human Reconstruction, Optical Flow Estimation, Self-Supervised Learning, Weakly-supervised 3D Human Pose Estimation

Current state-of-the-art solutions for motion capture from a single camera
are optimization driven: they optimize the parameters of a 3D human model so
that its re-projection matches measurements in the video (e.g. person
segmentation, optical flow, keypoint detections etc.). Optimization models are
susceptible to local minima. This has been the bottleneck that forced using
clean green-screen like backgrounds at capture time, manual initialization, or
switching to multiple cameras as input resource. In this work, we propose a
learning based motion capture model for single camera input. Instead of
optimizing mesh and skeleton parameters directly, our model optimizes neural
network weights that predict 3D shape and skeleton configurations given a
monocular RGB video. Our model is trained using a combination of strong
supervision from synthetic data, and self-supervision from differentiable
rendering of (a) skeletal keypoints, (b) dense 3D mesh motion, and (c)
human-background segmentation, in an end-to-end framework. Empirically we show
our model combines the best of both worlds of supervised learning and test-time
optimization: supervised learning initializes the model parameters in the right
regime, ensuring good pose and surface initialization at test time, without
manual effort. Self-supervision by back-propagating through differentiable
rendering allows (unsupervised) adaptation of the model to the test data, and
offers much tighter fit than a pretrained fixed model. We show that the
proposed model improves with experience and converges to low-error solutions
where previous optimization methods fail.
Modelling collective motion based on the principle of agency
Authors: Katja Ried, Thomas Müller, Hans J. Briegel
Date: 2017-12-04 00:00:00

Collective motion is an intriguing phenomenon, especially considering that it
arises from a set of simple rules governing local interactions between
individuals. In theoretical models, these rules are normally \emph{assumed} to
take a particular form, possibly constrained by heuristic arguments. We propose
a new class of models, which describe the individuals as \emph{agents}, capable
of deciding for themselves how to act and learning from their experiences. The
local interaction rules do not need to be postulated in this model, since they
\emph{emerge} from the learning process. We apply this ansatz to a concrete
scenario involving marching locusts, in order to model the phenomenon of
density-dependent alignment. We show that our learning agent-based model can
account for a Fokker-Planck equation that describes the collective motion and,
most notably, that the agents can learn the appropriate local interactions,
requiring no strong previous assumptions on their form. These results suggest
that learning agent-based models are a powerful tool for studying a broader
class of problems involving collective motion and animal agency in general.
Examining Cooperation in Visual Dialog Models
Authors: Mircea Mironenco, Dana Kianfar, Ke Tran, Evangelos Kanoulas, Efstratios Gavves
Date: 2017-12-04 00:00:00
Tasks: Visual Dialog

In this work we propose a blackbox intervention method for visual dialog
models, with the aim of assessing the contribution of individual linguistic or
visual components. Concretely, we conduct structured or randomized
interventions that aim to impair an individual component of the model, and
observe changes in task performance. We reproduce a state-of-the-art visual
dialog model and demonstrate that our methodology yields surprising insights,
namely that both dialog and image information have minimal contributions to
task performance. The intervention method presented here can be applied as a
sanity check for the strength and robustness of each component in visual dialog
systems.
Learning User Intent from Action Sequences on Interactive Systems
Authors: Rakshit Agrawal, Anwar Habeeb, Chih-Hsin Hsueh
Date: 2017-12-04 00:00:00
Tasks: Intent Recognition, Marketing, Time Series, Time Series Analysis

Interactive systems have taken over the web and mobile space with increasing
participation from users. Applications across every marketing domain can now be
accessed through mobile or web where users can directly perform certain actions
and reach a desired outcome. Actions of user on a system, though, can be
representative of a certain intent. Ability to learn this intent through user's
actions can help draw certain insight into the behavior of users on a system.
  In this paper, we present models to optimize interactive systems by learning
and analyzing user intent through their actions on the system. We present a
four phased model that uses time-series of interaction actions sequentially
using a Long Short-Term Memory (LSTM) based sequence learning system that helps
build a model for intent recognition. Our system then provides an objective
specific maximization followed by analysis and contrasting methods in order to
identify spaces of improvement in the interaction system. We discuss deployment
scenarios for such a system and present results from evaluation on an online
marketplace using user clickstream data.
Learning Sparse Neural Networks through $L_0$ Regularization
Authors: Christos Louizos, Max Welling, Diederik P. Kingma
Date: 2017-12-04 00:00:00
Tasks: Model Selection

We propose a practical method for $L_0$ norm regularization for neural
networks: pruning the network during training by encouraging weights to become
exactly zero. Such regularization is interesting since (1) it can greatly speed
up training and inference, and (2) it can improve generalization. AIC and BIC,
well-known model selection criteria, are special cases of $L_0$ regularization.
However, since the $L_0$ norm of weights is non-differentiable, we cannot
incorporate it directly as a regularization term in the objective function. We
propose a solution through the inclusion of a collection of non-negative
stochastic gates, which collectively determine which weights to set to zero. We
show that, somewhat surprisingly, for certain distributions over the gates, the
expected $L_0$ norm of the resulting gated weights is differentiable with
respect to the distribution parameters. We further propose the \emph{hard
concrete} distribution for the gates, which is obtained by "stretching" a
binary concrete distribution and then transforming its samples with a
hard-sigmoid. The parameters of the distribution over the gates can then be
jointly optimized with the original network parameters. As a result our method
allows for straightforward and efficient learning of model structures with
stochastic gradient descent and allows for conditional computation in a
principled way. We perform various experiments to demonstrate the effectiveness
of the resulting approach and regularizer.
Multi-View Deep Learning for Consistent Semantic Mapping with RGB-D Cameras
Authors: Lingni Ma, Jörg Stückler, Christian Kerl, Daniel Cremers
Date: 2017-03-26 00:00:00
Tasks: Scene Understanding, Segmentation, Semantic Segmentation

Visual scene understanding is an important capability that enables robots to
purposefully act in their environment. In this paper, we propose a novel
approach to object-class segmentation from multiple RGB-D views using deep
learning. We train a deep neural network to predict object-class semantics that
is consistent from several view points in a semi-supervised way. At test time,
the semantics predictions of our network can be fused more consistently in
semantic keyframe maps than predictions of a network trained on individual
views. We base our network architecture on a recent single-view deep learning
approach to RGB and depth fusion for semantic object-class segmentation and
enhance it with multi-scale loss minimization. We obtain the camera trajectory
using RGB-D SLAM and warp the predictions of RGB-D images into ground-truth
annotated frames in order to enforce multi-view consistency during training. At
test time, predictions from multiple views are fused into keyframes. We propose
and analyze several methods for enforcing multi-view consistency during
training and testing. We evaluate the benefit of multi-view consistency
training and demonstrate that pooling of deep features and fusion over multiple
views outperforms single-view baselines on the NYUDv2 benchmark for semantic
segmentation. Our end-to-end trained network achieves state-of-the-art
performance on the NYUDv2 dataset in single-view segmentation as well as
multi-view semantic fusion.
Saturating Splines and Feature Selection
Authors: Nicholas Boyd, Trevor Hastie, Stephen Boyd, Benjamin Recht, Michael Jordan
Date: 2016-09-21 00:00:00
Tasks: Additive models, feature selection

We extend the adaptive regression spline model by incorporating saturation,
the natural requirement that a function extend as a constant outside a certain
range. We fit saturating splines to data using a convex optimization problem
over a space of measures, which we solve using an efficient algorithm based on
the conditional gradient method. Unlike many existing approaches, our algorithm
solves the original infinite-dimensional (for splines of degree at least two)
optimization problem without pre-specified knot locations. We then adapt our
algorithm to fit generalized additive models with saturating splines as
coordinate functions and show that the saturation requirement allows our model
to simultaneously perform feature selection and nonlinear function fitting.
Finally, we briefly sketch how the method can be extended to higher order
splines and to different requirements on the extension outside the data range.
Linear-Complexity Exponentially-Consistent Tests for Universal Outlying Sequence Detection
Authors: Yuheng Bu, Shaofeng Zou, Venugopal V. Veeravalli
Date: 2017-01-21 00:00:00
Tasks: Clustering

The problem of universal outlying sequence detection is studied, where the
goal is to detect outlying sequences among $M$ sequences of samples. A sequence
is considered as outlying if the observations therein are generated by a
distribution different from those generating the observations in the majority
of the sequences. In the universal setting, we are interested in identifying
all the outlying sequences without knowing the underlying generating
distributions. In this paper, a class of tests based on distribution clustering
is proposed. These tests are shown to be exponentially consistent with linear
time complexity in $M$. Numerical results demonstrate that our clustering-based
tests achieve similar performance to existing tests, while being considerably
more computationally efficient.
An Equivalence of Fully Connected Layer and Convolutional Layer
Authors: Wei Ma, Jun Lu
Date: 2017-12-04 00:00:00

This article demonstrates that convolutional operation can be converted to
matrix multiplication, which has the same calculation way with fully connected
layer. The article is helpful for the beginners of the neural network to
understand how fully connected layer and the convolutional layer work in the
backend. To be concise and to make the article more readable, we only consider
the linear case. It can be extended to the non-linear case easily through
plugging in a non-linear encapsulation to the values like this $\sigma(x)$
denoted as $x^{\prime}$.
Clustering Stable Instances of Euclidean k-means
Authors: Abhratanu Dutta, Aravindan Vijayaraghavan, Alex Wang
Date: 2017-12-04 00:00:00
Tasks: Clustering

The Euclidean k-means problem is arguably the most widely-studied clustering
problem in machine learning. While the k-means objective is NP-hard in the
worst-case, practitioners have enjoyed remarkable success in applying
heuristics like Lloyd's algorithm for this problem. To address this disconnect,
we study the following question: what properties of real-world instances will
enable us to design efficient algorithms and prove guarantees for finding the
optimal clustering? We consider a natural notion called additive perturbation
stability that we believe captures many practical instances. Stable instances
have unique optimal k-means solutions that do not change even when each point
is perturbed a little (in Euclidean distance). This captures the property that
the k-means optimal solution should be tolerant to measurement errors and
uncertainty in the points. We design efficient algorithms that provably recover
the optimal clustering for instances that are additive perturbation stable.
When the instance has some additional separation, we show an efficient
algorithm with provable guarantees that is also robust to outliers. We
complement these results by studying the amount of stability in real datasets
and demonstrating that our algorithm performs well on these benchmark datasets.
Learning by Asking Questions
Authors: Ishan Misra, Ross Girshick, Rob Fergus, Martial Hebert, Abhinav Gupta, Laurens van der Maaten
Date: 2017-12-04 00:00:00
Proceeding: CVPR 2018 6
Tasks: Question Answering, Visual Question Answering, Visual Question Answering (VQA)

We introduce an interactive learning framework for the development and
testing of intelligent visual systems, called learning-by-asking (LBA). We
explore LBA in context of the Visual Question Answering (VQA) task. LBA differs
from standard VQA training in that most questions are not observed during
training time, and the learner must ask questions it wants answers to. Thus,
LBA more closely mimics natural learning and has the potential to be more
data-efficient than the traditional VQA setting. We present a model that
performs LBA on the CLEVR dataset, and show that it automatically discovers an
easy-to-hard curriculum when learning interactively from an oracle. Our LBA
generated data consistently matches or outperforms the CLEVR train data and is
more sample efficient. We also show that our model asks questions that
generalize to state-of-the-art VQA models and to novel test time distributions.
On the Real-time Vehicle Placement Problem
Authors: Abhinav Jauhri, Carlee Joe-Wong, John Paul Shen
Date: 2017-12-04 00:00:00
Tasks: Scheduling

Motivated by ride-sharing platforms' efforts to reduce their riders' wait
times for a vehicle, this paper introduces a novel problem of placing vehicles
to fulfill real-time pickup requests in a spatially and temporally changing
environment. The real-time nature of this problem makes it fundamentally
different from other placement and scheduling problems, as it requires not only
real-time placement decisions but also handling real-time request dynamics,
which are influenced by human mobility patterns. We use a dataset of ten
million ride requests from four major U.S. cities to show that the requests
exhibit significant self-similarity. We then propose distributed online
learning algorithms for the real-time vehicle placement problem and bound their
expected performance under this observed self-similarity.