pwc-archive/papers-with-abstracts huggingface dataset:
The Barbados 2018 List of Open Issues in Continual Learning
Authors: Tom Schaul, Hado van Hasselt, Joseph Modayil, Martha White, Adam White, Pierre-Luc Bacon, Jean Harb, Shibl Mourad, Marc Bellemare, Doina Precup
Date: 2018-11-16 00:00:00
Tasks: Continual Learning

We want to make progress toward artificial general intelligence, namely
general-purpose agents that autonomously learn how to competently act in
complex environments. The purpose of this report is to sketch a research
outline, share some of the most important open issues we are facing, and
stimulate further discussion in the community. The content is based on some of
our discussions during a week-long workshop held in Barbados in February 2018.
Data-Efficient Graph Embedding Learning for PCB Component Detection
Authors: Chia-Wen Kuo, Jacob Ashmore, David Huggins, Zsolt Kira
Date: 2018-11-16 00:00:00
Tasks: Graph Embedding, object-detection, Object Detection, Region Proposal

This paper presents a challenging computer vision task, namely the detection
of generic components on a PCB, and a novel set of deep-learning methods that
are able to jointly leverage the appearance of individual components and the
propagation of information across the structure of the board to accurately
detect and identify various types of components on a PCB. Due to the expense of
manual data labeling, a highly unbalanced distribution of component types, and
significant domain shift across boards, most earlier attempts based on
traditional image processing techniques fail to generalize well to PCB images
with various quality, lighting conditions, etc. Newer object detection
pipelines such as Faster R-CNN, on the other hand, require a large amount of
labeled data, do not deal with domain shift, and do not leverage structure. To
address these issues, we propose a three stage pipeline in which a
class-agnostic region proposal network is followed by a low-shot similarity
prediction classifier. In order to exploit the data dependency within a PCB, we
design a novel Graph Network block to refine the component features conditioned
on each PCB. To the best of our knowledge, this is one of the earliest attempts
to train a deep learning based model for such tasks, and we demonstrate
improvements over recent graph networks for this task. We also provide in-depth
analysis and discussion for this challenging task, pointing to future research.
Image Classification at Supercomputer Scale
Authors: Chris Ying, Sameer Kumar, Dehao Chen, Tao Wang, Youlong Cheng
Date: 2018-11-16 00:00:00
Tasks: Classification, Deep Learning, General Classification, image-classification, Image Classification

Deep learning is extremely computationally intensive, and hardware vendors
have responded by building faster accelerators in large clusters. Training deep
learning models at petaFLOPS scale requires overcoming both algorithmic and
systems software challenges. In this paper, we discuss three systems-related
optimizations: (1) distributed batch normalization to control per-replica batch
sizes, (2) input pipeline optimizations to sustain model throughput, and (3)
2-D torus all-reduce to speed up gradient summation. We combine these
optimizations to train ResNet-50 on ImageNet to 76.3% accuracy in 2.2 minutes
on a 1024-chip TPU v3 Pod with a training throughput of over 1.05 million
images/second and no accuracy drop.
Learned Video Compression
Authors: Oren Rippel, Sanjay Nair, Carissa Lew, Steve Branson, Alexander G. Anderson, Lubomir Bourdev
Date: 2018-11-16 00:00:00
Proceeding: ICCV 2019 10
Tasks: Blocking, Motion Estimation, Optical Flow Estimation, Video Compression

We present a new algorithm for video coding, learned end-to-end for the
low-latency mode. In this setting, our approach outperforms all existing video
codecs across nearly the entire bitrate range. To our knowledge, this is the
first ML-based method to do so.
  We evaluate our approach on standard video compression test sets of varying
resolutions, and benchmark against all mainstream commercial codecs, in the
low-latency mode. On standard-definition videos, relative to our algorithm,
HEVC/H.265, AVC/H.264 and VP9 typically produce codes up to 60% larger. On
high-definition 1080p videos, H.265 and VP9 typically produce codes up to 20%
larger, and H.264 up to 35% larger. Furthermore, our approach does not suffer
from blocking artifacts and pixelation, and thus produces videos that are more
visually pleasing.
  We propose two main contributions. The first is a novel architecture for
video compression, which (1) generalizes motion estimation to perform any
learned compensation beyond simple translations, (2) rather than strictly
relying on previously transmitted reference frames, maintains a state of
arbitrary information learned by the model, and (3) enables jointly compressing
all transmitted signals (such as optical flow and residual).
  Secondly, we present a framework for ML-based spatial rate control: namely, a
mechanism for assigning variable bitrates across space for each frame. This is
a critical component for video coding, which to our knowledge had not been
developed within a machine learning setting.
Beam Search Decoding using Manner of Articulation Detection Knowledge Derived from Connectionist Temporal Classification
Authors: Pradeep Rangan, Sreenivasa Rao K
Date: 2018-11-16 00:00:00
Tasks: Attribute, General Classification, Manner Of Articulation Detection

Manner of articulation detection using deep neural networks require a priori
knowledge of the attribute discriminative features or the decent phoneme
alignments. However generating an appropriate phoneme alignment is complex and
its performance depends on the choice of optimal number of senones, Gaussians,
etc. In the first part of our work, we exploit the manner of articulation
detection using connectionist temporal classification (CTC) which doesn't need
any phoneme alignment. Later we modify the state-of-the-art character based
posteriors generated by CTC using the manner of articulation CTC detector. Beam
search decoding is performed on the modified posteriors and it's impact on open
source datasets such as AN4 and LibriSpeech is observed.
Optimizing Photonic Nanostructures via Multi-fidelity Gaussian Processes
Authors: Jialin Song, Yury S. Tokpanov, Yuxin Chen, Dagny Fleischman, Kate T. Fountaine, Harry A. Atwater, Yisong Yue
Date: 2018-11-15 00:00:00
Tasks: Gaussian Processes

We apply numerical methods in combination with finite-difference-time-domain
(FDTD) simulations to optimize transmission properties of plasmonic mirror
color filters using a multi-objective figure of merit over a five-dimensional
parameter space by utilizing novel multi-fidelity Gaussian processes approach.
We compare these results with conventional derivative-free global search
algorithms, such as (single-fidelity) Gaussian Processes optimization scheme,
and Particle Swarm Optimization---a commonly used method in nanophotonics
community, which is implemented in Lumerical commercial photonics software. We
demonstrate the performance of various numerical optimization approaches on
several pre-collected real-world datasets and show that by properly trading off
expensive information sources with cheap simulations, one can more effectively
optimize the transmission properties with a fixed budget.
Towards Large-Scale Exploratory Search over Heterogeneous Sources
Authors: Mariia Seleznova, Anton Belyy, Aleksei Sholokhov
Date: 2018-11-15 00:00:00
Tasks: Diversity

Since time immemorial, people have been looking for ways to organize
scientific knowledge into some systems to facilitate search and discovery of
new ideas. The problem was partially solved in the pre-Internet era using
library classifications, but nowadays it is nearly impossible to classify all
scientific and popular scientific knowledge manually. There is a clear gap
between the diversity and the amount of data available on the Internet and the
algorithms for automatic structuring of such data. In our preliminary study, we
approach the problem of knowledge discovery on web-scale data with diverse text
sources and propose an algorithm to aggregate multiple collections into a
single hierarchical topic model. We implement a web service named Rysearch to
demonstrate the concept of topical exploratory search and make it available
online.
Tetris: Re-architecting Convolutional Neural Network Computation for Machine Learning Accelerators
Authors: Hang Lu, Xin Wei, Ning Lin, Guihai Yan, and Xiaowei Li
Date: 2018-11-14 00:00:00
Tasks: BIG-bench Machine Learning

Inference efficiency is the predominant consideration in designing deep
learning accelerators. Previous work mainly focuses on skipping zero values to
deal with remarkable ineffectual computation, while zero bits in non-zero
values, as another major source of ineffectual computation, is often ignored.
The reason lies on the difficulty of extracting essential bits during operating
multiply-and-accumulate (MAC) in the processing element. Based on the fact that
zero bits occupy as high as 68.9% fraction in the overall weights of modern
deep convolutional neural network models, this paper firstly proposes a weight
kneading technique that could eliminate ineffectual computation caused by
either zero value weights or zero bits in non-zero weights, simultaneously.
Besides, a split-and-accumulate (SAC) computing pattern in replacement of
conventional MAC, as well as the corresponding hardware accelerator design
called Tetris are proposed to support weight kneading at the hardware level.
Experimental results prove that Tetris could speed up inference up to 1.50x,
and improve power efficiency up to 5.33x compared with the state-of-the-art
baselines.
Pareto Optimization for Subset Selection with Dynamic Cost Constraints
Authors: Vahid Roostapour, Aneta Neumann, Frank Neumann, Tobias Friedrich
Date: 2018-11-14 00:00:00

We consider the subset selection problem for function $f$ with constraint bound $B$ that changes over time. Within the area of submodular optimization, various greedy approaches are commonly used. For dynamic environments we observe that the adaptive variants of these greedy approaches are not able to maintain their approximation quality. Investigating the recently introduced POMC Pareto optimization approach, we show that this algorithm efficiently computes a $\phi= (\alpha_f/2)(1-\frac{1}{e^{\alpha_f}})$-approximation, where $\alpha_f$ is the submodularity ratio of $f$, for each possible constraint bound $b \leq B$. Furthermore, we show that POMC is able to adapt its set of solutions quickly in the case that $B$ increases. Our experimental investigations for the influence maximization in social networks show the advantage of POMC over generalized greedy algorithms. We also consider EAMC, a new evolutionary algorithm with polynomial expected time guarantee to maintain $\phi$ approximation ratio, and NSGA-II with two different population sizes as advanced multi-objective optimization algorithm, to demonstrate their challenges in optimizing the maximum coverage problem. Our empirical analysis shows that, within the same number of evaluations, POMC is able to perform as good as NSGA-II under linear constraint, while EAMC performs significantly worse than all considered algorithms in most cases.
A New SVDD-Based Multivariate Non-parametric Process Capability Index
Authors: Deovrat Kakde, Arin Chaudhuri, Diana Shaw
Date: 2018-11-13 00:00:00

Process capability index (PCI) is a commonly used statistic to measure
ability of a process to operate within the given specifications or to produce
products which meet the required quality specifications. PCI can be univariate
or multivariate depending upon the number of process specifications or quality
characteristics of interest. Most PCIs make distributional assumptions which
are often unrealistic in practice.
  This paper proposes a new multivariate non-parametric process capability
index. This index can be used when distribution of the process or quality
parameters is either unknown or does not follow commonly used distributions
such as multivariate normal.
Modelling the Dynamic Joint Policy of Teammates with Attention Multi-agent DDPG
Authors: Hangyu Mao, Zhengchao Zhang, Zhen Xiao, Zhibo Gong
Date: 2018-11-13 00:00:00
Tasks: Reinforcement Learning, Reinforcement Learning (RL)

Modelling and exploiting teammates' policies in cooperative multi-agent
systems have long been an interest and also a big challenge for the
reinforcement learning (RL) community. The interest lies in the fact that if
the agent knows the teammates' policies, it can adjust its own policy
accordingly to arrive at proper cooperations; while the challenge is that the
agents' policies are changing continuously due to they are learning
concurrently, which imposes difficulty to model the dynamic policies of
teammates accurately. In this paper, we present \emph{ATTention Multi-Agent
Deep Deterministic Policy Gradient} (ATT-MADDPG) to address this challenge.
ATT-MADDPG extends DDPG, a single-agent actor-critic RL method, with two
special designs. First, in order to model the teammates' policies, the agent
should get access to the observations and actions of teammates. ATT-MADDPG
adopts a centralized critic to collect such information. Second, to model the
teammates' policies using the collected information in an effective way,
ATT-MADDPG enhances the centralized critic with an attention mechanism. This
attention mechanism introduces a special structure to explicitly model the
dynamic joint policy of teammates, making sure that the collected information
can be processed efficiently. We evaluate ATT-MADDPG on both benchmark tasks
and the real-world packet routing tasks. Experimental results show that it not
only outperforms the state-of-the-art RL-based methods and rule-based methods
by a large margin, but also achieves better performance in terms of scalability
and robustness.
An Infinite Parade of Giraffes: Expressive Augmentation and Complexity Layers for Cartoon Drawing
Authors: K. G. Greene
Date: 2018-11-08 00:00:00
Tasks: Image Generation, Small Data Image Classification

In this paper, we explore creative image generation constrained by small
data. To partially automate the creation of cartoon sketches consistent with a
specific designer's style, where acquiring a very large original image set is
impossible or cost prohibitive, we exploit domain specific knowledge for a huge
reduction in original image requirements, creating an effectively infinite
number of cartoon giraffes from just nine original drawings. We introduce
"expressive augmentations" for cartoon sketches, mathematical transformations
that create broad domain appropriate variation, far beyond the usual affine
transformations, and we show that chained GANs models trained on the temporal
stages of drawing or "complexity layers" can effectively add character
appropriate details and finish new drawings in the designer's style.
  We discuss the application of these tools in design processes for textiles,
graphics, architectural elements and interior design.
Scalable and interpretable product recommendations via overlapping co-clustering
Authors: Reinhard Heckel, Michail Vlachos, Thomas Parnell, Celestine Dünner
Date: 2016-04-07 00:00:00
Tasks: Clustering

We consider the problem of generating interpretable recommendations by
identifying overlapping co-clusters of clients and products, based only on
positive or implicit feedback. Our approach is applicable on very large
datasets because it exhibits almost linear complexity in the input examples and
the number of co-clusters. We show, both on real industrial data and on
publicly available datasets, that the recommendation accuracy of our algorithm
is competitive to that of state-of-art matrix factorization techniques. In
addition, our technique has the advantage of offering recommendations that are
textually and visually interpretable. Finally, we examine how to implement our
technique efficiently on Graphical Processing Units (GPUs).
Latent Factor Interpretations for Collaborative Filtering
Authors: Anupam Datta, Sophia Kovaleva, Piotr Mardziel, Shayak Sen
Date: 2017-11-29 00:00:00
Tasks: Collaborative Filtering, Recommendation Systems

Many machine learning systems utilize latent factors as internal
representations for making predictions. Since these latent factors are largely
uninterpreted, however, predictions made using them are opaque. Collaborative
filtering via matrix factorization is a prime example of such an algorithm that
uses uninterpreted latent features, and yet has seen widespread adoption for
many recommendation tasks. We present Latent Factor Interpretation (LFI), a
method for interpreting models by leveraging interpretations of latent factors
in terms of human-understandable features. The interpretation of latent factors
can then replace the uninterpreted latent factors, resulting in a new model
that expresses predictions in terms of interpretable features. This new model
can then be interpreted using recently developed model explanation techniques.
In this paper we develop LFI for collaborative filtering based recommender
systems. We illustrate the use of LFI interpretations on the MovieLens dataset,
integrating auxiliary features from IMDB and DB tropes, and show that latent
factors can be predicted with sufficient accuracy for replicating the
predictions of the true model.
MultiNest: an efficient and robust Bayesian inference tool for cosmology and particle physics
Authors: F. Feroz, M. P. Hobson, M. Bridges
Date: 2008-09-19 00:00:00
Tasks: Bayesian Inference

We present further development and the first public release of our multimodal
nested sampling algorithm, called MultiNest. This Bayesian inference tool
calculates the evidence, with an associated error estimate, and produces
posterior samples from distributions that may contain multiple modes and
pronounced (curving) degeneracies in high dimensions. The developments
presented here lead to further substantial improvements in sampling efficiency
and robustness, as compared to the original algorithm presented in Feroz &
Hobson (2008), which itself significantly outperformed existing MCMC techniques
in a wide range of astrophysical inference problems. The accuracy and economy
of the MultiNest algorithm is demonstrated by application to two toy problems
and to a cosmological inference problem focussing on the extension of the
vanilla $\Lambda$CDM model to include spatial curvature and a varying equation
of state for dark energy. The MultiNest software, which is fully parallelized
using MPI and includes an interface to CosmoMC, is available at
http://www.mrao.cam.ac.uk/software/multinest/. It will also be released as part
of the SuperBayeS package, for the analysis of supersymmetric theories of
particle physics, at http://www.superbayes.org
WEST: Word Encoded Sequence Transducers
Authors: Ehsan Variani, Ananda Theertha Suresh, Mitchel Weintraub
Date: 2018-11-20 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Federated Learning, speech-recognition, Speech Recognition

Most of the parameters in large vocabulary models are used in embedding layer
to map categorical features to vectors and in softmax layer for classification
weights. This is a bottle-neck in memory constraint on-device training
applications like federated learning and on-device inference applications like
automatic speech recognition (ASR). One way of compressing the embedding and
softmax layers is to substitute larger units such as words with smaller
sub-units such as characters. However, often the sub-unit models perform poorly
compared to the larger unit models. We propose WEST, an algorithm for encoding
categorical features and output classes with a sequence of random or domain
dependent sub-units and demonstrate that this transduction can lead to
significant compression without compromising performance. WEST bridges the gap
between larger unit and sub-unit models and can be interpreted as a MaxEnt
model over sub-unit features, which can be of independent interest.
Visual SLAM-based Localization and Navigation for Service Robots: The Pepper Case
Authors: Cristopher Gómez, Matías Mattamala, Tim Resink, Javier Ruiz-del-Solar
Date: 2018-11-20 00:00:00

We propose a Visual-SLAM based localization and navigation system for service
robots. Our system is built on top of the ORB-SLAM monocular system but
extended by the inclusion of wheel odometry in the estimation procedures. As a
case study, the proposed system is validated using the Pepper robot, whose
short-range LIDARs and RGB-D camera do not allow the robot to self-localize in
large environments. The localization system is tested in navigation tasks using
Pepper in two different environments: a medium-size laboratory, and a
large-size hall.
Sampling Can Be Faster Than Optimization
Authors: Yi-An Ma, Yuansi Chen, Chi Jin, Nicolas Flammarion, Michael. I. Jordan
Date: 2018-11-20 00:00:00

Optimization algorithms and Monte Carlo sampling algorithms have provided the computational foundations for the rapid growth in applications of statistical machine learning in recent years. There is, however, limited theoretical understanding of the relationships between these two kinds of methodology, and limited understanding of relative strengths and weaknesses. Moreover, existing results have been obtained primarily in the setting of convex functions (for optimization) and log-concave functions (for sampling). In this setting, where local properties determine global properties, optimization algorithms are unsurprisingly more efficient computationally than sampling algorithms. We instead examine a class of nonconvex objective functions that arise in mixture modeling and multi-stable systems. In this nonconvex setting, we find that the computational complexity of sampling algorithms scales linearly with the model dimension while that of optimization algorithms scales exponentially.
A Baseline for Multi-Label Image Classification Using An Ensemble of Deep Convolutional Neural Networks
Authors: Qian Wang, Ning Jia, Toby P. Breckon
Date: 2018-11-20 00:00:00
Tasks: Classification, Data Augmentation, General Classification, image-classification, Image Classification, Multi-Label Image Classification, Region Proposal

Recent studies on multi-label image classification have focused on designing more complex architectures of deep neural networks such as the use of attention mechanisms and region proposal networks. Although performance gains have been reported, the backbone deep models of the proposed approaches and the evaluation metrics employed in different works vary, making it difficult to compare each fairly. Moreover, due to the lack of properly investigated baselines, the advantage introduced by the proposed techniques are often ambiguous. To address these issues, we make a thorough investigation of the mainstream deep convolutional neural network architectures for multi-label image classification and present a strong baseline. With the use of proper data augmentation techniques and model ensembles, the basic deep architectures can achieve better performance than many existing more complex ones on three benchmark datasets, providing great insight for the future studies on multi-label image classification.