pwc-archive/papers-with-abstracts huggingface dataset:
Stochastic Prediction of Multi-Agent Interactions from Partial Observations
Authors: Chen Sun, Per Karlsson, Jiajun Wu, Joshua B. Tenenbaum, Kevin Murphy
Date: 2019-02-25 00:00:00

We present a method that learns to integrate temporal information, from a
learned dynamics model, with ambiguous visual information, from a learned
vision model, in the context of interacting agents. Our method is based on a
graph-structured variational recurrent neural network (Graph-VRNN), which is
trained end-to-end to infer the current state of the (partially observed)
world, as well as to forecast future states. We show that our method
outperforms various baselines on two sports datasets, one based on real
basketball trajectories, and one generated by a soccer game engine.
NAS-Bench-101: Towards Reproducible Neural Architecture Search
Authors: Chris Ying, Aaron Klein, Esteban Real, Eric Christiansen, Kevin Murphy, Frank Hutter
Date: 2019-02-25 00:00:00
Tasks: Benchmarking, Neural Architecture Search

Recent advances in neural architecture search (NAS) demand tremendous computational resources, which makes it difficult to reproduce experiments and imposes a barrier-to-entry to researchers without access to large-scale computation. We aim to ameliorate these problems by introducing NAS-Bench-101, the first public architecture dataset for NAS research. To build NAS-Bench-101, we carefully constructed a compact, yet expressive, search space, exploiting graph isomorphisms to identify 423k unique convolutional architectures. We trained and evaluated all of these architectures multiple times on CIFAR-10 and compiled the results into a large dataset of over 5 million trained models. This allows researchers to evaluate the quality of a diverse range of models in milliseconds by querying the pre-computed dataset. We demonstrate its utility by analyzing the dataset as a whole and by benchmarking a range of architecture optimization algorithms.
TraVeLGAN: Image-to-image Translation by Transformation Vector Learning
Authors: Matthew Amodio, Smita Krishnaswamy
Date: 2019-02-25 00:00:00
Proceeding: CVPR 2019 6
Tasks: Image-to-Image Translation, Translation

Interest in image-to-image translation has grown substantially in recent
years with the success of unsupervised models based on the cycle-consistency
assumption. The achievements of these models have been limited to a particular
subset of domains where this assumption yields good results, namely homogeneous
domains that are characterized by style or texture differences. We tackle the
challenging problem of image-to-image translation where the domains are defined
by high-level shapes and contexts, as well as including significant clutter and
heterogeneity. For this purpose, we introduce a novel GAN based on preserving
intra-domain vector transformations in a latent space learned by a siamese
network. The traditional GAN system introduced a discriminator network to guide
the generator into generating images in the target domain. To this two-network
system we add a third: a siamese network that guides the generator so that each
original image shares semantics with its generated version. With this new
three-network system, we no longer need to constrain the generators with the
ubiquitous cycle-consistency restraint. As a result, the generators can learn
mappings between more complex domains that differ from each other by large
differences - not just style or texture.
Flappy Hummingbird: An Open Source Dynamic Simulation of Flapping Wing Robots and Animals
Authors: Fan Fei, Zhan Tu, Yilun Yang, Jian Zhang, Xinyan Deng
Date: 2019-02-25 00:00:00
Tasks: Deep Reinforcement Learning, OpenAI Gym, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Insects and hummingbirds exhibit extraordinary flight capabilities and can
simultaneously master seemingly conflicting goals: stable hovering and
aggressive maneuvering, unmatched by small scale man-made vehicles. Flapping
Wing Micro Air Vehicles (FWMAVs) hold great promise for closing this
performance gap. However, design and control of such systems remain challenging
due to various constraints. Here, we present an open source high fidelity
dynamic simulation for FWMAVs to serve as a testbed for the design,
optimization and flight control of FWMAVs. For simulation validation, we
recreated the hummingbird-scale robot developed in our lab in the simulation.
System identification was performed to obtain the model parameters. The force
generation, open-loop and closed-loop dynamic response between simulated and
experimental flights were compared and validated. The unsteady aerodynamics and
the highly nonlinear flight dynamics present challenging control problems for
conventional and learning control algorithms such as Reinforcement Learning.
The interface of the simulation is fully compatible with OpenAI Gym
environment. As a benchmark study, we present a linear controller for hovering
stabilization and a Deep Reinforcement Learning control policy for
goal-directed maneuvering. Finally, we demonstrate direct simulation-to-real
transfer of both control policies onto the physical robot, further
demonstrating the fidelity of the simulation.
Learning Extreme Hummingbird Maneuvers on Flapping Wing Robots
Authors: Fan Fei, Zhan Tu, Jian Zhang, Xinyan Deng
Date: 2019-02-25 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Biological studies show that hummingbirds can perform extreme aerobatic
maneuvers during fast escape. Given a sudden looming visual stimulus at hover,
a hummingbird initiates a fast backward translation coupled with a 180-degree
yaw turn, which is followed by instant posture stabilization in just under 10
wingbeats. Consider the wingbeat frequency of 40Hz, this aggressive maneuver is
carried out in just 0.2 seconds. Inspired by the hummingbirds' near-maximal
performance during such extreme maneuvers, we developed a flight control
strategy and experimentally demonstrated that such maneuverability can be
achieved by an at-scale 12-gram hummingbird robot equipped with just two
actuators. The proposed hybrid control policy combines model-based nonlinear
control with model-free reinforcement learning. We use model-based nonlinear
control for nominal flight control, as the dynamic model is relatively accurate
for these conditions. However, during extreme maneuver, the modeling error
becomes unmanageable. A model-free reinforcement learning policy trained in
simulation was optimized to 'destabilize' the system and maximize the
performance during maneuvering. The hybrid policy manifests a maneuver that is
close to that observed in hummingbirds. Direct simulation-to-real transfer is
achieved, demonstrating the hummingbird-like fast evasive maneuvers on the
at-scale hummingbird robot.
On Binscatter
Authors: Matias D. Cattaneo, Richard K. Crump, Max H. Farrell, Yingjie Feng
Date: 2019-02-25 00:00:00

Binscatter is a popular method for visualizing bivariate relationships and conducting informal specification testing. We study the properties of this method formally and develop enhanced visualization and econometric binscatter tools. These include estimating conditional means with optimal binning and quantifying uncertainty. We also highlight a methodological problem related to covariate adjustment that can yield incorrect conclusions. We revisit two applications using our methodology and find substantially different results relative to those obtained using prior informal binscatter methods. General purpose software in Python, R, and Stata is provided. Our technical work is of independent interest for the nonparametric partition-based estimation literature.
Analyzing Data Selection Techniques with Tools from the Theory of Information Losses
Authors: Brandon Foggo, Nanpeng Yu
Date: 2019-02-25 00:00:00
Tasks: Active Learning, Experimental Design, General Classification, Learning Theory

In this paper, we present and illustrate some new tools for rigorously analyzing training data selection methods. These tools focus on the information theoretic losses that occur when sampling data. We use this framework to prove that two methods, Facility Location Selection and Transductive Experimental Design, reduce these losses. These are meant to act as generalizable theoretical examples of applying the field of Information Theoretic Deep Learning Theory to the fields of data selection and active learning. Both analyses yield insight into their respective methods and increase their interpretability. In the case of Transductive Experimental Design, the provided analysis greatly increases the method's scope as well.
Short-term Road Traffic Prediction based on Deep Cluster at Large-scale Networks
Authors: Lingyi Han, Kan Zheng, Long Zhao, Xianbin Wang, Xuemin Shen
Date: 2019-02-25 00:00:00
Tasks: Clustering, Deep Clustering, Representation Learning, Time Series Analysis, Traffic Prediction, Triplet

Short-term road traffic prediction (STTP) is one of the most important
modules in Intelligent Transportation Systems (ITS). However, network-level
STTP still remains challenging due to the difficulties both in modeling the
diverse traffic patterns and tacking high-dimensional time series with low
latency. Therefore, a framework combining with a deep clustering (DeepCluster)
module is developed for STTP at largescale networks in this paper. The
DeepCluster module is proposed to supervise the representation learning in a
visualized way from the large unlabeled dataset. More specifically, to fully
exploit the traffic periodicity, the raw series is first split into a number of
sub-series for triplets generation. The convolutional neural networks (CNNs)
with triplet loss are utilized to extract the features of shape by transferring
the series into visual images. The shape-based representations are then used
for road segments clustering. Thereafter, motivated by the fact that the road
segments in a group have similar patterns, a model sharing strategy is further
proposed to build recurrent NNs (RNNs)-based predictions through a group-based
model (GM), instead of individual-based model (IM) in which one model are built
for one road exclusively. Our framework can not only significantly reduce the
number of models and cost, but also increase the number of training data and
the diversity of samples. In the end, we evaluate the proposed framework over
the network of Liuli Bridge in Beijing. Experimental results show that the
DeepCluster can effectively cluster the road segments and GM can achieve
comparable performance against the IM with less number of models.
Convolutional Neural Networks for Automatic Meter Reading
Authors: Rayson Laroca, Victor Barroso, Matheus A. Diniz, Gabriel R. Gonçalves, William Robson Schwartz, David Menotti
Date: 2019-02-25 00:00:00
Tasks: Counter Recognition, Data Augmentation, Image-based Automatic Meter Reading, Meter Reading, Optical Character Recognition (OCR)

In this paper, we tackle Automatic Meter Reading (AMR) by leveraging the high
capability of Convolutional Neural Networks (CNNs). We design a two-stage
approach that employs the Fast-YOLO object detector for counter detection and
evaluates three different CNN-based approaches for counter recognition. In the
AMR literature, most datasets are not available to the research community since
the images belong to a service company. In this sense, we introduce a new
public dataset, called UFPR-AMR dataset, with 2,000 fully and manually
annotated images. This dataset is, to the best of our knowledge, three times
larger than the largest public dataset found in the literature and contains a
well-defined evaluation protocol to assist the development and evaluation of
AMR methods. Furthermore, we propose the use of a data augmentation technique
to generate a balanced training set with many more examples to train the CNN
models for counter recognition. In the proposed dataset, impressive results
were obtained and a detailed speed/accuracy trade-off evaluation of each model
was performed. In a public dataset, state-of-the-art results were achieved
using less than 200 images for training.
MisGAN: Learning from Incomplete Data with Generative Adversarial Networks
Authors: Steven Cheng-Xian Li, Bo Jiang, Benjamin Marlin
Date: 2019-02-25 00:00:00
Proceeding: ICLR 2019 5
Tasks: Abstract Argumentation

Generative adversarial networks (GANs) have been shown to provide an
effective way to model complex distributions and have obtained impressive
results on various challenging tasks. However, typical GANs require
fully-observed data during training. In this paper, we present a GAN-based
framework for learning from complex, high-dimensional incomplete data. The
proposed framework learns a complete data generator along with a mask generator
that models the missing data distribution. We further demonstrate how to impute
missing data by equipping our framework with an adversarially trained imputer.
We evaluate the proposed framework using a series of experiments with several
types of missing data processes under the missing completely at random
assumption.
Unsupervised learning-based long-term superpixel tracking
Authors: Pierre-Henri Conze, Florian Tilquin, Mathieu Lamard, Fabrice Heitz, Gwenolé Quellec
Date: 2019-02-25 00:00:00
Tasks: Object Tracking, Superpixels, Video Object Tracking

Finding correspondences between structural entities decomposing images is of
high interest for computer vision applications. In particular, we analyze how
to accurately track superpixels - visual primitives generated by aggregating
adjacent pixels sharing similar characteristics - over extended time periods
relying on unsupervised learning and temporal integration. A two-step video
processing pipeline dedicated to long-term superpixel tracking is proposed.
First, unsupervised learning-based superpixel matching provides correspondences
between consecutive and distant frames using new context-rich features extended
from greyscale to multi-channel and forward-backward consistency contraints.
Resulting elementary matches are then combined along multi-step paths running
through the whole sequence with various inter-frame distances. This produces a
large set of candidate long-term superpixel pairings upon which majority voting
is performed. Video object tracking experiments demonstrate the accuracy of our
elementary estimator against state-of-the-art methods and proves the ability of
multi-step integration to provide accurate long-term superpixel matches
compared to usual direct and sequential integration.
Verification of Non-Linear Specifications for Neural Networks
Authors: Chongli Qin, Krishnamurthy, Dvijotham, Brendan O'Donoghue, Rudy Bunel, Robert Stanforth, Sven Gowal, Jonathan Uesato, Grzegorz Swirszcz, Pushmeet Kohli
Date: 2019-02-25 00:00:00
Proceeding: ICLR 2019 5

Prior work on neural network verification has focused on specifications that
are linear functions of the output of the network, e.g., invariance of the
classifier output under adversarial perturbations of the input. In this paper,
we extend verification algorithms to be able to certify richer properties of
neural networks. To do this we introduce the class of convex-relaxable
specifications, which constitute nonlinear specifications that can be verified
using a convex relaxation. We show that a number of important properties of
interest can be modeled within this class, including conservation of energy in
a learned dynamics model of a physical system; semantic consistency of a
classifier's output labels under adversarial perturbations and bounding errors
in a system that predicts the summation of handwritten digits. Our experimental
evaluation shows that our method is able to effectively verify these
specifications. Moreover, our evaluation exposes the failure modes in models
which cannot be verified to satisfy these specifications. Thus, emphasizing the
importance of training models not just to fit training data but also to be
consistent with specifications.
The State of Sparsity in Deep Neural Networks
Authors: Trevor Gale, Erich Elsen, Sara Hooker
Date: 2019-02-25 00:00:00
Tasks: Model Compression, Sparse Learning

We rigorously evaluate three state-of-the-art techniques for inducing
sparsity in deep neural networks on two large-scale learning tasks: Transformer
trained on WMT 2014 English-to-German, and ResNet-50 trained on ImageNet.
Across thousands of experiments, we demonstrate that complex techniques
(Molchanov et al., 2017; Louizos et al., 2017b) shown to yield high compression
rates on smaller datasets perform inconsistently, and that simple magnitude
pruning approaches achieve comparable or better results. Additionally, we
replicate the experiments performed by (Frankle & Carbin, 2018) and (Liu et
al., 2018) at scale and show that unstructured sparse architectures learned
through pruning cannot be trained from scratch to the same test set performance
as a model trained with joint sparsification and optimization. Together, these
results highlight the need for large-scale benchmarks in the field of model
compression. We open-source our code, top performing model checkpoints, and
results of all hyperparameter configurations to establish rigorous baselines
for future work on compression and sparsification.
Anomaly Detection for an E-commerce Pricing System
Authors: Jagdish Ramakrishnan, Elham Shaabani, Chao Li, Mátyás A. Sustik
Date: 2019-02-25 00:00:00
Tasks: Anomaly Detection, Model Selection, Outlier Detection, Supervised Anomaly Detection

Online retailers execute a very large number of price updates when compared to brick-and-mortar stores. Even a few mis-priced items can have a significant business impact and result in a loss of customer trust. Early detection of anomalies in an automated real-time fashion is an important part of such a pricing system. In this paper, we describe unsupervised and supervised anomaly detection approaches we developed and deployed for a large-scale online pricing system at Walmart. Our system detects anomalies both in batch and real-time streaming settings, and the items flagged are reviewed and actioned based on priority and business impact. We found that having the right architecture design was critical to facilitate model performance at scale, and business impact and speed were important factors influencing model selection, parameter choice, and prioritization in a production environment for a large-scale system. We conducted analyses on the performance of various approaches on a test set using real-world retail data and fully deployed our approach into production. We found that our approach was able to detect the most important anomalies with high precision.
Using Deep Object Features for Image Descriptions
Authors: Ashutosh Mishra, Marcus Liwicki
Date: 2019-02-25 00:00:00
Tasks: Decoder, Language Modeling, Language Modelling, Machine Translation, Object, Translation

Inspired by recent advances in leveraging multiple modalities in machine
translation, we introduce an encoder-decoder pipeline that uses (1) specific
objects within an image and their object labels, (2) a language model for
decoding joint embedding of object features and the object labels. Our pipeline
merges prior detected objects from the image and their object labels and then
learns the sequences of captions describing the particular image. The decoder
model learns to extract descriptions for the image from scratch by decoding the
joint representation of the object visual features and their object classes
conditioned by the encoder component. The idea of the model is to concentrate
only on the specific objects of the image and their labels for generating
descriptions of the image rather than visual feature of the entire image. The
model needs to be calibrated more by adjusting the parameters and settings to
result in better accuracy and performance.
Epileptic seizure classification using statistical sampling and a novel feature selection algorithm
Authors: Md Mursalin, Syed Shamsul Islam, Md Kislu Noman, Adel Ali Al-Jumaily
Date: 2019-02-25 00:00:00
Tasks: EEG, Electroencephalogram (EEG), feature selection, General Classification, Seizure Detection

Epilepsy is a well-known neuronal disorder that can be identified by
interpretation of the electroencephalogram (EEG) signal. Usually, the length of
an EEG signal is quite long which is challenging to interpret manually. In this
work, we propose an automated epileptic seizure detection method by applying a
two-step minimization technique: first, we reduce the data points using a
statistical sampling technique and then, we minimize the number of features
using our novel feature selection algorithm. We then apply different machine
learning algorithms for performance measurement of the proposed feature
selection algorithm. The experimental results outperform some of the
state-of-the-art methods for seizure detection using the reduced data points
and the least number of features.
Residual-CNDS for Grand Challenge Scene Dataset
Authors: Hussein A. Al-Barazanchi, Hussam Qassim, David Feinzimer, Abhishek Verma
Date: 2019-01-13 00:00:00

Increasing depth of convolutional neural networks (CNNs) is a highly
promising method of increasing the accuracy of the (CNNs). Increased CNN depth
will also result in increased layer count (parameters), leading to a slow
backpropagation convergence prone to overfitting. We trained our model
(Residual-CNDS) to classify very large-scale scene datasets MIT Places 205, and
MIT Places 365-Standard. The outcome result from the two datasets proved our
proposed model (Residual-CNDS) effectively handled the slow convergence,
overfitting, and degradation. CNNs that include deep supervision (CNDS) add
supplementary branches to the deep convolutional neural network in specified
layers by calculating vanishing, effectively addressing delayed convergence and
overfitting. Nevertheless, (CNDS) does not resolve degradation; hence, we add
residual learning to the (CNDS) in certain layers after studying the best place
in which to add it. With this approach we overcome degradation in the very deep
network. We have built two models (Residual-CNDS 8), and (Residual-CNDS 10).
Moreover, we tested our models on two large-scale datasets, and we compared our
results with other recently introduced cutting-edge networks in the domain of
top-1 and top-5 classification accuracy. As a result, both of models have shown
good improvement, which supports the assertion that the addition of residual
connections enhances network CNDS accuracy without adding any computation
complexity.
Attention-driven Tree-structured Convolutional LSTM for High Dimensional Data Understanding
Authors: Bin Kong, Xin Wang, Junjie Bai, Yi Lu, Feng Gao, Kunlin Cao, Qi Song, Shaoting Zhang, Siwei Lyu, Youbing Yin
Date: 2019-01-29 00:00:00
Tasks: Vocal Bursts Intensity Prediction

Modeling the sequential information of image sequences has been a vital step
of various vision tasks and convolutional long short-term memory (ConvLSTM) has
demonstrated its superb performance in such spatiotemporal problems.
Nevertheless, the hierarchical data structures in a significant amount of tasks
(e.g., human body parts and vessel/airway tree in biomedical images) cannot be
properly modeled by sequential models. Thus, ConvLSTM is not suitable for
tree-structured image data analysis. In order to address these limitations, we
present tree-structured ConvLSTM models for tree-structured image analysis
tasks which can be trained end-to-end. To demonstrate the effectiveness of the
proposed tree-structured ConvLSTM model, we present a tree-structured
segmentation framework which consists of a tree-structured ConvLSTM and an
attention fully convolutional network (FCN) model. The proposed framework is
extensively validated on four large-scale coronary artery datasets. The results
demonstrate the effectiveness and efficiency of the proposed method.