pwc-archive/papers-with-abstracts huggingface dataset:
Causality-Aided Falsification
Authors: Takumi Akazaki, Yoshihiro Kumazawa, Ichiro Hasuo
Date: 2017-09-08 00:00:00
Tasks: Stochastic Optimization

Falsification is drawing attention in quality assurance of heterogeneous
systems whose complexities are beyond most verification techniques'
scalability. In this paper we introduce the idea of causality aid in
falsification: by providing a falsification solver -- that relies on stochastic
optimization of a certain cost function -- with suitable causal information
expressed by a Bayesian network, search for a falsifying input value can be
efficient. Our experiment results show the idea's viability.
Low-memory GEMM-based convolution algorithms for deep neural networks
Authors: Andrew Anderson, Aravind Vasudevan, Cormac Keane, David Gregg
Date: 2017-09-08 00:00:00

Deep neural networks (DNNs) require very large amounts of computation both
for training and for inference when deployed in the field. A common approach to
implementing DNNs is to recast the most computationally expensive operations as
general matrix multiplication (GEMM). However, as we demonstrate in this paper,
there are a great many different ways to express DNN convolution operations
using GEMM. Although different approaches all perform the same number of
operations, the size of temporary data structures differs significantly.
Convolution of an input matrix with dimensions $C \times H \times W$, requires
$O(K^2CHW)$ additional space using the classical im2col approach. More recently
memory-efficient approaches requiring just $O(KCHW)$ auxiliary space have been
proposed.
  We present two novel GEMM-based algorithms that require just $O(MHW)$ and
$O(KW)$ additional space respectively, where $M$ is the number of channels in
the result of the convolution. These algorithms dramatically reduce the space
overhead of DNN convolution, making it much more suitable for memory-limited
embedded systems. Experimental evaluation shows that our low-memory algorithms
are just as fast as the best patch-building approaches despite requiring just a
fraction of the amount of additional memory. Our low-memory algorithms have
excellent data locality which gives them a further edge over patch-building
algorithms when multiple cores are used. As a result, our low memory algorithms
often outperform the best patch-building algorithms using multiple threads.
Uncertainty-Aware Learning from Demonstration using Mixture Density Networks with Sampling-Free Variance Modeling
Authors: Sungjoon Choi, Kyungjae Lee, Sungbin Lim, Songhwai Oh
Date: 2017-09-03 00:00:00
Tasks: Autonomous Driving

In this paper, we propose an uncertainty-aware learning from demonstration
method by presenting a novel uncertainty estimation method utilizing a mixture
density network appropriate for modeling complex and noisy human behaviors. The
proposed uncertainty acquisition can be done with a single forward path without
Monte Carlo sampling and is suitable for real-time robotics applications. The
properties of the proposed uncertainty measure are analyzed through three
different synthetic examples, absence of data, heavy measurement noise, and
composition of functions scenarios. We show that each case can be distinguished
using the proposed uncertainty measure and presented an uncertainty-aware
learn- ing from demonstration method of an autonomous driving using this
property. The proposed uncertainty-aware learning from demonstration method
outperforms other compared methods in terms of safety using a complex
real-world driving dataset.
A Novel Low-Complexity Framework in Ultra-Wideband Imaging for Breast Cancer Detection
Authors: Yasaman Ettefagh, Mohammad Hossein Moghaddam, Saeed Vahidian
Date: 2017-09-08 00:00:00
Tasks: Breast Cancer Detection

In this research work, a novel framework is pro- posed as an efficient
successor to traditional imaging methods for breast cancer detection in order
to decrease the computational complexity. In this framework, the breast is
devided into seg- ments in an iterative process and in each iteration, the one
having the most probability of containing tumor with lowest possible resolution
is selected by using suitable decision metrics. After finding the smallest
tumor-containing segment, the resolution is increased in the detected
tumor-containing segment, leaving the other parts of the breast image with low
resolution. Our framework is applied on the most common used beamforming
techniques, such as delay and sum (DAS) and delay multiply and sum (DMAS) and
according to simulation results, our framework can decrease the computational
complexity significantly for both DAS and DMAS without imposing any degradation
on accuracy of basic algorithms. The amount of complexity reduction can be
determined manually or automatically based on two proposed methods that are
described in this framework.
Optimizing scoring function of dynamic programming of pairwise profile alignment using derivative free neural network
Authors: Kazunori D Yamada
Date: 2017-08-30 00:00:00
Tasks: Multiple Sequence Alignment

A profile comparison method with position-specific scoring matrix (PSSM) is
one of the most accurate alignment methods. Currently, cosine similarity and
correlation coefficient are used as scoring functions of dynamic programming to
calculate similarity between PSSMs. However, it is unclear that these functions
are optimal for profile alignment methods. At least, by definition, these
functions cannot capture non-linear relationships between profiles. Therefore,
in this study, we attempted to discover a novel scoring function, which was
more suitable for the profile comparison method than the existing ones. Firstly
we implemented a new derivative free neural network by combining the
conventional neural network with evolutionary strategy optimization method.
Next, using the framework, the scoring function was optimized for aligning
remote sequence pairs. Nepal, the pairwise profile aligner with the novel
scoring function significantly improved both alignment sensitivity and
precision, compared to aligners with the existing functions. Nepal improved
alignment quality because of adaptation to remote sequence alignment and
increasing the expressive power of similarity score. The novel scoring function
can be realized using a simple matrix operation and easily incorporated into
other aligners. With our scoring function, the performance of homology
detection and/or multiple sequence alignment for remote homologous sequences
would be further improved.
Implicit Regularization in Deep Learning
Authors: Behnam Neyshabur
Date: 2017-09-06 00:00:00
Tasks: Deep Learning

In an attempt to better understand generalization in deep learning, we study
several possible explanations. We show that implicit regularization induced by
the optimization method is playing a key role in generalization and success of
deep learning models. Motivated by this view, we study how different complexity
measures can ensure generalization and explain how optimization algorithms can
implicitly regularize complexity measures. We empirically investigate the
ability of these measures to explain different observed phenomena in deep
learning. We further study the invariances in neural networks, suggest
complexity measures and optimization algorithms that have similar invariances
to those in neural networks and evaluate them on a number of learning tasks.
Intelligent Subset Selection of Power Generators for Economic Dispatch
Authors: Biswarup Bhattacharya, Abhishek Sinha
Date: 2017-09-08 00:00:00
Tasks: Deep Learning

Sustainable and economical generation of electrical power is an essential and
mandatory component of infrastructure in today's world. Optimal generation
(generator subset selection) of power requires a careful evaluation of various
factors like type of source, generation, transmission & storage capacities,
congestion among others which makes this a difficult task. We created a grid to
simulate various conditions including stimuli like generator supply, weather
and load demand using Siemens PSS/E software and this data is trained using
deep learning methods and subsequently tested. The results are highly
encouraging. As per our knowledge, this is the first paper to propose a working
and scalable deep learning model for this problem.
Deep Subspace Clustering Networks
Authors: Pan Ji, Tong Zhang, Hongdong Li, Mathieu Salzmann, Ian Reid
Date: 2017-09-08 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Clustering, Decoder, Image Clustering

We present a novel deep neural network architecture for unsupervised subspace
clustering. This architecture is built upon deep auto-encoders, which
non-linearly map the input data into a latent space. Our key idea is to
introduce a novel self-expressive layer between the encoder and the decoder to
mimic the "self-expressiveness" property that has proven effective in
traditional subspace clustering. Being differentiable, our new self-expressive
layer provides a simple but effective way to learn pairwise affinities between
all data points through a standard back-propagation procedure. Being nonlinear,
our neural-network based method is able to cluster data points having complex
(often nonlinear) structures. We further propose pre-training and fine-tuning
strategies that let us effectively learn the parameters of our subspace
clustering networks. Our experiments show that the proposed method
significantly outperforms the state-of-the-art unsupervised subspace clustering
methods.
DeepFeat: A Bottom Up and Top Down Saliency Model Based on Deep Features of Convolutional Neural Nets
Authors: Ali Mahdi, Jun Qin
Date: 2017-09-08 00:00:00

A deep feature based saliency model (DeepFeat) is developed to leverage the
understanding of the prediction of human fixations. Traditional saliency models
often predict the human visual attention relying on few level image cues.
Although such models predict fixations on a variety of image complexities,
their approaches are limited to the incorporated features. In this study, we
aim to provide an intuitive interpretation of convolu- tional neural network
deep features by combining low and high level visual factors. We exploit four
evaluation metrics to evaluate the correspondence between the proposed
framework and the ground-truth fixations. The key findings of the results
demon- strate that the DeepFeat algorithm, incorporation of bottom up and top
down saliency maps, outperforms the individual bottom up and top down approach.
Moreover, in comparison to nine 9 state-of-the-art saliency models, our
proposed DeepFeat model achieves satisfactory performance based on all four
evaluation metrics.
GOOWE: Geometrically Optimum and Online-Weighted Ensemble Classifier for Evolving Data Streams
Authors: Hamed R. Bonab, Fazli Can
Date: 2017-09-08 00:00:00

Designing adaptive classifiers for an evolving data stream is a challenging
task due to the data size and its dynamically changing nature. Combining
individual classifiers in an online setting, the ensemble approach, is a
well-known solution. It is possible that a subset of classifiers in the
ensemble outperforms others in a time-varying fashion. However, optimum weight
assignment for component classifiers is a problem which is not yet fully
addressed in online evolving environments. We propose a novel data stream
ensemble classifier, called Geometrically Optimum and Online-Weighted Ensemble
(GOOWE), which assigns optimum weights to the component classifiers using a
sliding window containing the most recent data instances. We map vote scores of
individual classifiers and true class labels into a spatial environment. Based
on the Euclidean distance between vote scores and ideal-points, and using the
linear least squares (LSQ) solution, we present a novel, dynamic, and online
weighting approach. While LSQ is used for batch mode ensemble classifiers, it
is the first time that we adapt and use it for online environments by providing
a spatial modeling of online ensembles. In order to show the robustness of the
proposed algorithm, we use real-world datasets and synthetic data generators
using the MOA libraries. First, we analyze the impact of our weighting system
on prediction accuracy through two scenarios. Second, we compare GOOWE with 8
state-of-the-art ensemble classifiers in a comprehensive experimental
environment. Our experiments show that GOOWE provides improved reactions to
different types of concept drift compared to our baselines. The statistical
tests indicate a significant improvement in accuracy, with conservative time
and memory requirements.
Synthesizing Training Data for Object Detection in Indoor Scenes
Authors: Georgios Georgakis, Arsalan Mousavian, Alexander C. Berg, Jana Kosecka
Date: 2017-02-25 00:00:00
Tasks: Object, object-detection, Object Detection, Object Detection In Indoor Scenes

Detection of objects in cluttered indoor environments is one of the key
enabling functionalities for service robots. The best performing object
detection approaches in computer vision exploit deep Convolutional Neural
Networks (CNN) to simultaneously detect and categorize the objects of interest
in cluttered scenes. Training of such models typically requires large amounts
of annotated training data which is time consuming and costly to obtain. In
this work we explore the ability of using synthetically generated composite
images for training state-of-the-art object detectors, especially for object
instance detection. We superimpose 2D images of textured object models into
images of real environments at variety of locations and scales. Our experiments
evaluate different superimposition strategies ranging from purely image-based
blending all the way to depth and semantics informed positioning of the object
models into real scenes. We demonstrate the effectiveness of these object
detector training strategies on two publicly available datasets, the
GMU-Kitchens and the Washington RGB-D Scenes v2. As one observation, augmenting
some hand-labeled training data with synthetic examples carefully composed onto
scenes yields object detectors with comparable performance to using much more
hand-labeled data. Broadly, this work charts new opportunities for training
detectors for new objects by exploiting existing object model repositories in
either a purely automatic fashion or with only a very small number of
human-annotated examples.
Learned Optimizers that Scale and Generalize
Authors: Olga Wichrowska, Niru Maheswaranathan, Matthew W. Hoffman, Sergio Gomez Colmenarejo, Misha Denil, Nando de Freitas, Jascha Sohl-Dickstein
Date: 2017-03-14 00:00:00
Proceeding: ICML 2017 8

Learning to learn has emerged as an important direction for achieving
artificial intelligence. Two of the primary barriers to its adoption are an
inability to scale to larger problems and a limited ability to generalize to
new tasks. We introduce a learned gradient descent optimizer that generalizes
well to new tasks, and which has significantly reduced memory and computation
overhead. We achieve this by introducing a novel hierarchical RNN architecture,
with minimal per-parameter overhead, augmented with additional architectural
features that mirror the known structure of optimization tasks. We also develop
a meta-training ensemble of small, diverse optimization tasks capturing common
properties of loss landscapes. The optimizer learns to outperform RMSProp/ADAM
on problems in this corpus. More importantly, it performs comparably or better
when applied to small convolutional neural networks, despite seeing no neural
networks in its meta-training set. Finally, it generalizes to train Inception
V3 and ResNet V2 architectures on the ImageNet dataset for thousands of steps,
optimization problems that are of a vastly different scale than those it was
trained on. We release an open source implementation of the meta-training
algorithm.
Evolution Strategies as a Scalable Alternative to Reinforcement Learning
Authors: Tim Salimans, Jonathan Ho, Xi Chen, Szymon Sidor, Ilya Sutskever
Date: 2017-03-10 00:00:00
Tasks: Atari Games, MuJoCo, Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We explore the use of Evolution Strategies (ES), a class of black box
optimization algorithms, as an alternative to popular MDP-based RL techniques
such as Q-learning and Policy Gradients. Experiments on MuJoCo and Atari show
that ES is a viable solution strategy that scales extremely well with the
number of CPUs available: By using a novel communication strategy based on
common random numbers, our ES implementation only needs to communicate scalars,
making it possible to scale to over a thousand parallel workers. This allows us
to solve 3D humanoid walking in 10 minutes and obtain competitive results on
most Atari games after one hour of training. In addition, we highlight several
advantages of ES as a black box optimization technique: it is invariant to
action frequency and delayed rewards, tolerant of extremely long horizons, and
does not need temporal discounting or value function approximation.
Scalable Annotation of Fine-Grained Categories Without Experts
Authors: Timnit Gebru, Jonathan Krause, Jia Deng, Li Fei-Fei
Date: 2017-09-07 00:00:00

We present a crowdsourcing workflow to collect image annotations for visually
similar synthetic categories without requiring experts. In animals, there is a
direct link between taxonomy and visual similarity: e.g. a collie (type of dog)
looks more similar to other collies (e.g. smooth collie) than a greyhound
(another type of dog). However, in synthetic categories such as cars, objects
with similar taxonomy can have very different appearance: e.g. a 2011 Ford
F-150 Supercrew-HD looks the same as a 2011 Ford F-150 Supercrew-LL but very
different from a 2011 Ford F-150 Supercrew-SVT. We introduce a graph based
crowdsourcing algorithm to automatically group visually indistinguishable
objects together. Using our workflow, we label 712,430 images by ~1,000 Amazon
Mechanical Turk workers; resulting in the largest fine-grained visual dataset
reported to date with 2,657 categories of cars annotated at 1/20th the cost of
hiring experts.
Fine-Grained Car Detection for Visual Census Estimation
Authors: Timnit Gebru, Jonathan Krause, Yi-Lun Wang, Duyun Chen, Jia Deng, Li Fei-Fei
Date: 2017-09-07 00:00:00

Targeted socioeconomic policies require an accurate understanding of a
country's demographic makeup. To that end, the United States spends more than 1
billion dollars a year gathering census data such as race, gender, education,
occupation and unemployment rates. Compared to the traditional method of
collecting surveys across many years which is costly and labor intensive,
data-driven, machine learning driven approaches are cheaper and faster--with
the potential ability to detect trends in close to real time. In this work, we
leverage the ubiquity of Google Street View images and develop a computer
vision pipeline to predict income, per capita carbon emission, crime rates and
other city attributes from a single source of publicly available visual data.
We first detect cars in 50 million images across 200 of the largest US cities
and train a model to predict demographic attributes using the detected cars. To
facilitate our work, we have collected the largest and most challenging
fine-grained dataset reported to date consisting of over 2600 classes of cars
comprised of images from Google Street View and other web sources, classified
by car experts to account for even the most subtle of visual differences. We
use this data to construct the largest scale fine-grained detection system
reported to date. Our prediction results correlate well with ground truth
income data (r=0.82), Massachusetts department of vehicle registration, and
sources investigating crime rates, income segregation, per capita carbon
emission, and other market research. Finally, we learn interesting
relationships between cars and neighborhoods allowing us to perform the first
large scale sociological analysis of cities using computer vision techniques.
Network learning via multi-agent inverse transportation problems
Authors: Susan Jia Xu, Mehdi Nourinejad, Xuebo Lai, Joseph Y. J. Chow
Date: 2016-09-14 00:00:00
Tasks: Descriptive

Despite the ubiquity of transportation data, methods to infer the state
parameters of a network either ignore sensitivity of route decisions, require
route enumeration for parameterizing descriptive models of route selection, or
require complex bilevel models of route assignment behavior. These limitations
prevent modelers from fully exploiting ubiquitous data in monitoring
transportation networks. Inverse optimization methods that capture network
route choice behavior can address this gap, but they are designed to take
observations of the same model to learn the parameters of that model, which is
statistically inefficient (e.g. requires estimating population route and link
flows). New inverse optimization models and supporting algorithms are proposed
to learn the parameters of heterogeneous travelers' route behavior to infer
shared network state parameters (e.g. link capacity dual prices). The inferred
values are consistent with observations of each agent's optimization behavior.
We prove that the method can obtain unique dual prices for a network shared by
these agents in polynomial time. Four experiments are conducted. The first one,
conducted on a 4-node network, verifies the methodology to obtain heterogeneous
link cost parameters even when multinomial or mixed logit models would not be
meaningfully estimated. The second is a parameter recovery test on the
Nguyen-Dupuis network that shows that unique latent link capacity dual prices
can be inferred using the proposed method. The third test on the same network
demonstrates how a monitoring system in an online learning environment can be
designed using this method. The last test demonstrates this learning on real
data obtained from a freeway network in Queens, New York, using only real-time
Google Maps queries.
Inferring Generative Model Structure with Static Analysis
Authors: Paroma Varma, Bryan He, Payal Bajaj, Imon Banerjee, Nishith Khandwala, Daniel L. Rubin, Christopher Ré
Date: 2017-09-07 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: model

Obtaining enough labeled data to robustly train complex discriminative models
is a major bottleneck in the machine learning pipeline. A popular solution is
combining multiple sources of weak supervision using generative models. The
structure of these models affects training label quality, but is difficult to
learn without any ground truth labels. We instead rely on these weak
supervision sources having some structure by virtue of being encoded
programmatically. We present Coral, a paradigm that infers generative model
structure by statically analyzing the code for these heuristics, thus reducing
the data required to learn structure significantly. We prove that Coral's
sample complexity scales quasilinearly with the number of heuristics and number
of relations found, improving over the standard sample complexity, which is
exponential in $n$ for identifying $n^{\textrm{th}}$ degree relations.
Experimentally, Coral matches or outperforms traditional structure learning
approaches by up to 3.81 F1 points. Using Coral to model dependencies instead
of assuming independence results in better performance than a fully supervised
model by 3.07 accuracy points when heuristics are used to label radiology data
without ground truth labels.