pwc-archive/papers-with-abstracts huggingface dataset:
Feature-based Attention in Convolutional Neural Networks
Authors: Grace W. Lindsay
Date: 2015-11-19 00:00:00
Tasks: Object, object-detection, Object Detection, Object Recognition

Convolutional neural networks (CNNs) have proven effective for image
processing tasks, such as object recognition and classification. Recently, CNNs
have been enhanced with concepts of attention, similar to those found in
biology. Much of this work on attention has focused on effective serial spatial
processing. In this paper, I introduce a simple procedure for applying
feature-based attention (FBA) to CNNs and compare multiple implementation
options. FBA is a top-down signal applied globally to an input image which
aides in detecting chosen objects in cluttered or noisy settings. The concept
of FBA and the implementation details tested here were derived from what is
known (and debated) about biological object- and feature-based attention. The
implementations of FBA described here increase performance on challenging
object detection tasks using a procedure that is simple, fast, and does not
require additional iterative training. Furthermore, the comparisons performed
here suggest that a proposed model of biological FBA (the "feature similarity
gain model") is effective in increasing performance.
Get More With Less: Near Real-Time Image Clustering on Mobile Phones
Authors: Jorge Ortiz, Chien-chin Huang, Supriyo Chakraborty
Date: 2015-12-09 00:00:00
Tasks: Clustering, Image Clustering

Machine learning algorithms, in conjunction with user data, hold the promise
of revolutionizing the way we interact with our phones, and indeed their
widespread adoption in the design of apps bear testimony to this promise.
However, currently, the computationally expensive segments of the learning
pipeline, such as feature extraction and model training, are offloaded to the
cloud, resulting in an over-reliance on the network and under-utilization of
computing resources available on mobile platforms. In this paper, we show that
by combining the computing power distributed over a number of phones, judicious
optimization choices, and contextual information it is possible to execute the
end-to-end pipeline entirely on the phones at the edge of the network,
efficiently. We also show that by harnessing the power of this combination, it
is possible to execute a computationally expensive pipeline at near real-time.
  To demonstrate our approach, we implement an end-to-end image-processing
pipeline -- that includes feature extraction, vocabulary learning,
vectorization, and image clustering -- on a set of mobile phones. Our results
show a 75% improvement over the standard, full pipeline implementation running
on the phones without modification -- reducing the time to one minute under
certain conditions. We believe that this result is a promising indication that
fully distributed, infrastructure-less computing is possible on networks of
mobile phones; enabling a new class of mobile applications that are less
reliant on the cloud.
Video captioning with recurrent networks based on frame- and video-level features and visual content classification
Authors: Rakshith Shetty, Jorma Laaksonen
Date: 2015-12-09 00:00:00
Tasks: Caption Generation, General Classification, Image Captioning, Video Captioning

In this paper, we describe the system for generating textual descriptions of
short video clips using recurrent neural networks (RNN), which we used while
participating in the Large Scale Movie Description Challenge 2015 in ICCV 2015.
Our work builds on static image captioning systems with RNN based language
models and extends this framework to videos utilizing both static image
features and video-specific features. In addition, we study the usefulness of
visual content classifiers as a source of additional information for caption
generation. With experimental results we show that utilizing keyframe based
features, dense trajectory video features and content classifier outputs
together gives better performance than any one of them individually.
Stochastic Interpretation of Quasi-periodic Event-based Systems
Authors: Hesham Mostafa, Giacomo Indiveri
Date: 2015-12-09 00:00:00

Many networks used in machine learning and as models of biological neural
networks make use of stochastic neurons or neuron-like units. We show that
stochastic artificial neurons can be realized on silicon chips by exploiting
the quasi-periodic behavior of mismatched analog oscillators to approximate the
neuron's stochastic activation function. We represent neurons by finite state
machines (FSMs) that communicate using digital events and whose transitions are
event-triggered. The event generation times of each neuron are controlled by an
analog oscillator internal to that neuron/FSM and the frequencies of the
oscillators in different FSMs are incommensurable. We show that within this
quasi-periodic system, the transition graph of a FSM can be interpreted as the
transition graph of a Markov chain and we show that by using different FSMs, we
can obtain approximations of different stochastic activation functions. We
investigate the quality of the stochastic interpretation of such a
deterministic system and we use the system to realize and sample from a
restricted Boltzmann machine. We implemented the quasi-periodic event-based
system on a custom silicon chip and we show that the chip behavior can be used
to closely approximate a stochastic sampling task.
Yet Another Statistical Analysis of Bob Ross Paintings
Authors: Christopher Steven Marcum
Date: 2015-12-09 00:00:00

In this paper, we analyze a sample of clippings from paintings by the late
artist Bob Ross. Previous work focused on the qualitative themes of his
paintings (Hickey, 2014); here, we expand on that line of research by
considering the colorspace and luminosity values as our data. Our results
demonstrate the subtle aesthetics of the average Ross painting, the common
variation shared by his paintings, and the structure of the relationships
between each painting in our sample. We reveal, for the first time, renderings
of the average paintings and introduce "eigenross" components to identify and
evaluate shared variance. Additionally, all data and code are embedded in this
document to encourage future research, and, in the spirit of Bob Ross, to teach
others how to do so.
Where You Are Is Who You Are: User Identification by Matching Statistics
Authors: Farid M. Naini, Jayakrishnan Unnikrishnan, Patrick Thiran, Martin Vetterli
Date: 2015-12-09 00:00:00
Tasks: User Identification

Most users of online services have unique behavioral or usage patterns. These
behavioral patterns can be exploited to identify and track users by using only
the observed patterns in the behavior. We study the task of identifying users
from statistics of their behavioral patterns. Specifically, we focus on the
setting in which we are given histograms of users' data collected during two
different experiments. We assume that, in the first dataset, the users'
identities are anonymized or hidden and that, in the second dataset, their
identities are known. We study the task of identifying the users by matching
the histograms of their data in the first dataset with the histograms from the
second dataset. In recent works, the optimal algorithm for this user
identification task is introduced. In this paper, we evaluate the effectiveness
of this method on three different types of datasets and in multiple scenarios.
Using datasets such as call data records, web browsing histories, and GPS
trajectories, we show that a large fraction of users can be easily identified
given only histograms of their data; hence these histograms can act as users'
fingerprints. We also verify that simultaneous identification of users achieves
better performance compared to one-by-one user identification. We show that
using the optimal method for identification gives higher identification
accuracy than heuristics-based approaches in practical scenarios. The accuracy
obtained under this optimal method can thus be used to quantify the maximum
level of user identification that is possible in such settings. We show that
the key factors affecting the accuracy of the optimal identification algorithm
are the duration of the data collection, the number of users in the anonymized
dataset, and the resolution of the dataset. We analyze the effectiveness of
k-anonymization in resisting user identification attacks on these datasets.
Multi-Player Bandits -- a Musical Chairs Approach
Authors: Jonathan Rosenski, Ohad Shamir, Liran Szlak
Date: 2015-12-09 00:00:00

We consider a variant of the stochastic multi-armed bandit problem, where
multiple players simultaneously choose from the same set of arms and may
collide, receiving no reward. This setting has been motivated by problems
arising in cognitive radio networks, and is especially challenging under the
realistic assumption that communication between players is limited. We provide
a communication-free algorithm (Musical Chairs) which attains constant regret
with high probability, as well as a sublinear-regret, communication-free
algorithm (Dynamic Musical Chairs) for the more difficult setting of players
dynamically entering and leaving throughout the game. Moreover, both algorithms
do not require prior knowledge of the number of players. To the best of our
knowledge, these are the first communication-free algorithms with these types
of formal guarantees. We also rigorously compare our algorithms to previous
works, and complement our theoretical findings with experiments.
Bigger Buffer k-d Trees on Multi-Many-Core Systems
Authors: Fabian Gieseke, Cosmin Eugen Oancea, Ashish Mahabal, Christian Igel, Tom Heskes
Date: 2015-12-09 00:00:00
Tasks: Astronomy

A buffer k-d tree is a k-d tree variant for massively-parallel nearest
neighbor search. While providing valuable speed-ups on modern many-core devices
in case both a large number of reference and query points are given, buffer k-d
trees are limited by the amount of points that can fit on a single device. In
this work, we show how to modify the original data structure and the associated
workflow to make the overall approach capable of dealing with massive data
sets. We further provide a simple yet efficient way of using multiple devices
given in a single workstation. The applicability of the modified framework is
demonstrated in the context of astronomy, a field that is faced with huge
amounts of data.
Risk Minimization in Structured Prediction using Orbit Loss
Authors: Danny Karmon, Joseph Keshet
Date: 2015-12-07 00:00:00
Tasks: Decoder, Prediction, Structured Prediction

We introduce a new surrogate loss function called orbit loss in the
structured prediction framework, which has good theoretical and practical
advantages. While the orbit loss is not convex, it has a simple analytical
gradient and a simple perceptron-like learning rule. We analyze the new loss
theoretically and state a PAC-Bayesian generalization bound. We also prove that
the new loss is consistent in the strong sense; namely, the risk achieved by
the set of the trained parameters approaches the infimum risk achievable by any
linear decoder over the given features. Methods that are aimed at risk
minimization, such as the structured ramp loss, the structured probit loss and
the direct loss minimization require at least two inference operations per
training iteration. In this sense, the orbit loss is more efficient as it
requires only one inference operation per training iteration, while yields
similar performance. We conclude the paper with an empirical comparison of the
proposed loss function to the structured hinge loss, the structured ramp loss,
the structured probit loss and the direct loss minimization method on several
benchmark datasets and tasks.
Sensor Fusion of Camera, GPS and IMU using Fuzzy Adaptive Multiple Motion Models
Authors: Erkan Bostanci, Betul Bostanci, Nadia Kanwal, Adrian F. Clark
Date: 2015-12-09 00:00:00
Tasks: Motion Estimation, Pose Estimation, Sensor Fusion

A tracking system that will be used for Augmented Reality (AR) applications
has two main requirements: accuracy and frame rate. The first requirement is
related to the performance of the pose estimation algorithm and how accurately
the tracking system can find the position and orientation of the user in the
environment. Accuracy problems of current tracking devices, considering that
they are low-cost devices, cause static errors during this motion estimation
process. The second requirement is related to dynamic errors (the end-to-end
system delay; occurring because of the delay in estimating the motion of the
user and displaying images based on this estimate. This paper investigates
combining the vision-based estimates with measurements from other sensors, GPS
and IMU, in order to improve the tracking accuracy in outdoor environments. The
idea of using Fuzzy Adaptive Multiple Models (FAMM) was investigated using a
novel fuzzy rule-based approach to decide on the model that results in improved
accuracy and faster convergence for the fusion filter. Results show that the
developed tracking system is more accurate than a conventional GPS-IMU fusion
approach due to additional estimates from a camera and fuzzy motion models. The
paper also presents an application in cultural heritage context.
Return of Frustratingly Easy Domain Adaptation
Authors: Baochen Sun, Jiashi Feng, Kate Saenko
Date: 2015-11-17 00:00:00
Tasks: BIG-bench Machine Learning, Domain Adaptation, Unsupervised Domain Adaptation

Unlike human learning, machine learning often fails to handle changes between
training (source) and test (target) input distributions. Such domain shifts,
common in practical scenarios, severely damage the performance of conventional
machine learning methods. Supervised domain adaptation methods have been
proposed for the case when the target data have labels, including some that
perform very well despite being "frustratingly easy" to implement. However, in
practice, the target domain is often unlabeled, requiring unsupervised
adaptation. We propose a simple, effective, and efficient method for
unsupervised domain adaptation called CORrelation ALignment (CORAL). CORAL
minimizes domain shift by aligning the second-order statistics of source and
target distributions, without requiring any target labels. Even though it is
extraordinarily simple--it can be implemented in four lines of Matlab
code--CORAL performs remarkably well in extensive evaluations on standard
benchmark datasets.
Scalable, High-Quality Object Detection
Authors: Christian Szegedy, Scott Reed, Dumitru Erhan, Dragomir Anguelov, Sergey Ioffe
Date: 2014-12-03 00:00:00
Tasks: Object, object-detection, Object Detection, Vocal Bursts Intensity Prediction

Current high-quality object detection approaches use the scheme of
salience-based object proposal methods followed by post-classification using
deep convolutional features. This spurred recent research in improving object
proposal methods. However, domain agnostic proposal generation has the
principal drawback that the proposals come unranked or with very weak ranking,
making it hard to trade-off quality for running time. This raises the more
fundamental question of whether high-quality proposal generation requires
careful engineering or can be derived just from data alone. We demonstrate that
learning-based proposal methods can effectively match the performance of
hand-engineered methods while allowing for very efficient runtime-quality
trade-offs. Using the multi-scale convolutional MultiBox (MSC-MultiBox)
approach, we substantially advance the state-of-the-art on the ILSVRC 2014
detection challenge data set, with $0.5$ mAP for a single model and $0.52$ mAP
for an ensemble of two models. MSC-Multibox significantly improves the proposal
quality over its predecessor MultiBox~method: AP increases from $0.42$ to
$0.53$ for the ILSVRC detection challenge. Finally, we demonstrate improved
bounding-box recall compared to Multiscale Combinatorial Grouping with less
proposals on the Microsoft-COCO data set.
Window-Object Relationship Guided Representation Learning for Generic Object Detections
Authors: Xingyu Zeng, Wanli Ouyang, Xiaogang Wang
Date: 2015-12-09 00:00:00
Tasks: Object, object-detection, Object Detection, Representation Learning

In existing works that learn representation for object detection, the
relationship between a candidate window and the ground truth bounding box of an
object is simplified by thresholding their overlap. This paper shows
information loss in this simplification and picks up the relative location/size
information discarded by thresholding. We propose a representation learning
pipeline to use the relationship as supervision for improving the learned
representation in object detection. Such relationship is not limited to object
of the target category, but also includes surrounding objects of other
categories. We show that image regions with multiple contexts and multiple
rotations are effective in capturing such relationship during the
representation learning process and in handling the semantic and visual
variation caused by different window-object configurations. Experimental
results show that the representation learned by our approach can improve the
object detection accuracy by 6.4% in mean average precision (mAP) on
ILSVRC2014. On the challenging ILSVRC2014 test dataset, 48.6% mAP is achieved
by our single model and it is the best among published results. On PASCAL VOC,
it outperforms the state-of-the-art result of Fast RCNN by 3.3% in absolute
mAP.
Level-Based Analysis of Genetic Algorithms for Combinatorial Optimization
Authors: Duc-Cuong Dang, Anton V. Eremeev, Per Kristian Lehre
Date: 2015-12-07 00:00:00
Tasks: Combinatorial Optimization

The paper is devoted to upper bounds on run-time of Non-Elitist Genetic
Algorithms until some target subset of solutions is visited for the first time.
In particular, we consider the sets of optimal solutions and the sets of local
optima as the target subsets. Previously known upper bounds are improved by
means of drift analysis. Finally, we propose conditions ensuring that a
Non-Elitist Genetic Algorithm efficiently finds approximate solutions with
constant approximation ratio on the class of combinatorial optimization
problems with guaranteed local optima (GLO).
Reinforcement Control with Hierarchical Backpropagated Adaptive Critics
Authors: John W. Jameson
Date: 2015-12-08 00:00:00
Tasks: Incremental Learning

Present incremental learning methods are limited in the ability to achieve
reliable credit assignment over a large number time steps (or events). However,
this situation is typical for cases where the dynamical system to be controlled
requires relatively frequent control updates in order to maintain stability or
robustness yet has some action-consequences which must be established over
relatively long periods of time. To address this problem, the learning
capabilities of a control architecture comprised of two Backpropagated Adaptive
Critics (BACs) in a two-level hierarchy with continuous actions are explored.
The high-level BAC updates less frequently than the low-level BAC and controls
the latter to some degree. The response of the low-level to high-level signals
can either be determined a priori or it can emerge during learning. A general
approach called Response Induction Learning is introduced to address the latter
case.
Deep Reinforcement Learning with Double Q-learning
Authors: Hado van Hasselt, Arthur Guez, David Silver
Date: 2015-09-22 00:00:00
Tasks: Atari Games, Deep Reinforcement Learning, Q-Learning, Reinforcement Learning, Reinforcement Learning (RL)

The popular Q-learning algorithm is known to overestimate action values under
certain conditions. It was not previously known whether, in practice, such
overestimations are common, whether they harm performance, and whether they can
generally be prevented. In this paper, we answer all these questions
affirmatively. In particular, we first show that the recent DQN algorithm,
which combines Q-learning with a deep neural network, suffers from substantial
overestimations in some games in the Atari 2600 domain. We then show that the
idea behind the Double Q-learning algorithm, which was introduced in a tabular
setting, can be generalized to work with large-scale function approximation. We
propose a specific adaptation to the DQN algorithm and show that the resulting
algorithm not only reduces the observed overestimations, as hypothesized, but
that this also leads to much better performance on several games.
Optimal Testing for Properties of Distributions
Authors: Jayadev Acharya, Constantinos Daskalakis, Gautam Kamath
Date: 2015-07-21 00:00:00
Proceeding: NeurIPS 2015 12

Given samples from an unknown distribution $p$, is it possible to distinguish
whether $p$ belongs to some class of distributions $\mathcal{C}$ versus $p$
being far from every distribution in $\mathcal{C}$? This fundamental question
has received tremendous attention in statistics, focusing primarily on
asymptotic analysis, and more recently in information theory and theoretical
computer science, where the emphasis has been on small sample size and
computational complexity. Nevertheless, even for basic properties of
distributions such as monotonicity, log-concavity, unimodality, independence,
and monotone-hazard rate, the optimal sample complexity is unknown.
  We provide a general approach via which we obtain sample-optimal and
computationally efficient testers for all these distribution families. At the
core of our approach is an algorithm which solves the following problem: Given
samples from an unknown distribution $p$, and a known distribution $q$, are $p$
and $q$ close in $\chi^2$-distance, or far in total variation distance?
  The optimality of our testers is established by providing matching lower
bounds with respect to both $n$ and $\varepsilon$. Finally, a necessary
building block for our testers and an important byproduct of our work are the
first known computationally efficient proper learners for discrete log-concave
and monotone hazard rate distributions.
Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
Authors: Dario Amodei, Rishita Anubhai, Eric Battenberg, Carl Case, Jared Casper, Bryan Catanzaro, Jingdong Chen, Mike Chrzanowski, Adam Coates, Greg Diamos, Erich Elsen, Jesse Engel, Linxi Fan, Christopher Fougner, Tony Han, Awni Hannun, Billy Jun, Patrick LeGresley, Libby Lin, Sharan Narang, Andrew Ng, Sherjil Ozair, Ryan Prenger, Jonathan Raiman, Sanjeev Satheesh, David Seetapun, Shubho Sengupta, Yi Wang, Zhiqian Wang, Chong Wang, Bo Xiao, Dani Yogatama, Jun Zhan, Zhenyao Zhu
Date: 2015-12-08 00:00:00
Tasks: Accented Speech Recognition, Noisy Speech Recognition, Speech Recognition

We show that an end-to-end deep learning approach can be used to recognize
either English or Mandarin Chinese speech--two vastly different languages.
Because it replaces entire pipelines of hand-engineered components with neural
networks, end-to-end learning allows us to handle a diverse variety of speech
including noisy environments, accents and different languages. Key to our
approach is our application of HPC techniques, resulting in a 7x speedup over
our previous system. Because of this efficiency, experiments that previously
took weeks now run in days. This enables us to iterate more quickly to identify
superior architectures and algorithms. As a result, in several cases, our
system is competitive with the transcription of human workers when benchmarked
on standard datasets. Finally, using a technique called Batch Dispatch with
GPUs in the data center, we show that our system can be inexpensively deployed
in an online setting, delivering low latency when serving users at scale.