pwc-archive/papers-with-abstracts huggingface dataset:
Modeling Uncertainty with Hedged Instance Embedding
Authors: Seong Joon Oh, Kevin Murphy, Jiyan Pan, Joseph Roth, Florian Schroff, Andrew Gallagher
Date: 2018-09-30 00:00:00
Tasks: Clustering, Metric Learning, Retrieval

Instance embeddings are an efficient and versatile image representation that facilitates applications like recognition, verification, retrieval, and clustering. Many metric learning methods represent the input as a single point in the embedding space. Often the distance between points is used as a proxy for match confidence. However, this can fail to represent uncertainty arising when the input is ambiguous, e.g., due to occlusion or blurriness. This work addresses this issue and explicitly models the uncertainty by hedging the location of each input in the embedding space. We introduce the hedged instance embedding (HIB) in which embeddings are modeled as random variables and the model is trained under the variational information bottleneck principle. Empirical results on our new N-digit MNIST dataset show that our method leads to the desired behavior of hedging its bets across the embedding space upon encountering ambiguous inputs. This results in improved performance for image matching and classification tasks, more structure in the learned embedding space, and an ability to compute a per-exemplar uncertainty measure that is correlated with downstream performance.
Convex Relaxation Methods for Community Detection
Authors: Xiao-Dong Li, Yudong Chen, Jiaming Xu
Date: 2018-09-30 00:00:00
Tasks: Community Detection, Survey

This paper surveys recent theoretical advances in convex optimization
approaches for community detection. We introduce some important theoretical
techniques and results for establishing the consistency of convex community
detection under various statistical models. In particular, we discuss the basic
techniques based on the primal and dual analysis. We also present results that
demonstrate several distinctive advantages of convex community detection,
including robustness against outlier nodes, consistency under weak
assortativity, and adaptivity to heterogeneous degrees.
  This survey is not intended to be a complete overview of the vast literature
on this fast-growing topic. Instead, we aim to provide a big picture of the
remarkable recent development in this area and to make the survey accessible to
a broad audience. We hope that this expository article can serve as an
introductory guide for readers who are interested in using, designing, and
analyzing convex relaxation methods in network analysis.
Posture recognition using an RGB-D camera : exploring 3D body modeling and deep learning approaches
Authors: Mohamed El Amine Elforaici, Ismail Chaaraoui, Wassim Bouachir, Youssef Ouakrim, Neila Mezghani
Date: 2018-09-30 00:00:00
Tasks: Transfer Learning

The emergence of RGB-D sensors offered new possibilities for addressing
complex artificial vision problems efficiently. Human posture recognition is
among these computer vision problems, with a wide range of applications such as
ambient assisted living and intelligent health care systems. In this context,
our paper presents novel methods and ideas to design automatic posture
recognition systems using an RGB-D camera. More specifically, we introduce two
supervised methods to learn and recognize human postures using the main types
of visual data provided by an RGB-D camera. The first method is based on
convolutional features extracted from 2D images. Convolutional Neural Networks
(CNNs) are trained to recognize human postures using transfer learning on RGB
and depth images. Secondly, we propose to model the posture using the body
joint configuration in the 3D space. Posture recognition is then performed
through SVM classification of 3D skeleton-based features. To evaluate the
proposed methods, we created a challenging posture recognition dataset with a
considerable variability regarding the acquisition conditions. The experimental
results demonstrated comparable performances and high precision for both
methods in recognizing human postures, with a slight superiority for the
CNN-based method when applied on depth images. Moreover, the two approaches
demonstrated a high robustness to several perturbation factors, such as scale
and orientation change.
Mini-batch Serialization: CNN Training with Inter-layer Data Reuse
Authors: Sangkug Lym, Armand Behroozi, Wei Wen, Ge Li, Yongkee Kwon, Mattan Erez
Date: 2018-09-30 00:00:00

Training convolutional neural networks (CNNs) requires intense computations and high memory bandwidth. We find that bandwidth today is over-provisioned because most memory accesses in CNN training can be eliminated by rearranging computation to better utilize on-chip buffers and avoid traffic resulting from large per-layer memory footprints. We introduce the MBS CNN training approach that significantly reduces memory traffic by partially serializing mini-batch processing across groups of layers. This optimizes reuse within on-chip buffers and balances both intra-layer and inter-layer reuse. We also introduce the WaveCore CNN training accelerator that effectively trains CNNs in the MBS approach with high functional-unit utilization. Combined, WaveCore and MBS reduce DRAM traffic by 75%, improve performance by 53%, and save 26% system energy for modern deep CNN training compared to conventional training mechanisms and accelerators.
Correlation Propagation Networks for Scene Text Detection
Authors: Zichuan Liu, Guosheng Lin, Wang Ling Goh, Fayao Liu, Chunhua Shen, Xiaokang Yang
Date: 2018-09-30 00:00:00
Tasks: Scene Text Detection, Text Detection

In this work, we propose a novel hybrid method for scene text detection
namely Correlation Propagation Network (CPN). It is an end-to-end trainable
framework engined by advanced Convolutional Neural Networks. Our CPN predicts
text objects according to both top-down observations and the bottom-up cues.
Multiple candidate boxes are assembled by a spatial communication mechanism
call Correlation Propagation (CP). The extracted spatial features by CNN are
regarded as node features in a latticed graph and Correlation Propagation
algorithm runs distributively on each node to update the hypothesis of
corresponding object centers. The CP process can flexibly handle scale-varying
and rotated text objects without using predefined bounding box templates.
Benefit from its distributive nature, CPN is computationally efficient and
enjoys a high level of parallelism. Moreover, we introduce deformable
convolution to the backbone network to enhance the adaptability to long texts.
The evaluation on public benchmarks shows that the proposed method achieves
state-of-art performance, and it significantly outperforms the existing methods
for handling multi-scale and multi-oriented text objects with much lower
computation cost.
Newton-MR: Inexact Newton Method With Minimum Residual Sub-problem Solver
Authors: Fred Roosta, Yang Liu, Peng Xu, Michael W. Mahoney
Date: 2018-09-30 00:00:00

We consider a variant of inexact Newton Method, called Newton-MR, in which the least-squares sub-problems are solved approximately using Minimum Residual method. By construction, Newton-MR can be readily applied for unconstrained optimization of a class of non-convex problems known as invex, which subsumes convexity as a sub-class. For invex optimization, instead of the classical Lipschitz continuity assumptions on gradient and Hessian, Newton-MR's global convergence can be guaranteed under a weaker notion of joint regularity of Hessian and gradient. We also obtain Newton-MR's problem-independent local convergence to the set of minima. We show that fast local/global convergence can be guaranteed under a novel inexactness condition, which, to our knowledge, is much weaker than the prior related works. Numerical results demonstrate the performance of Newton-MR as compared with several other Newton-type alternatives on a few machine learning problems.
DIMENSION: Dynamic MR Imaging with Both K-space and Spatial Prior Knowledge Obtained via Multi-Supervised Network Training
Authors: Shan-Shan Wang, Ziwen Ke, Huitao Cheng, Sen Jia, Ying Leslie, Hairong Zheng, Dong Liang
Date: 2018-09-30 00:00:00
Tasks: Image Reconstruction

Dynamic MR image reconstruction from incomplete k-space data has generated
great research interest due to its capability in reducing scan time.
Nevertheless, the reconstruction problem is still challenging due to its
ill-posed nature. Most existing methods either suffer from long iterative
reconstruction time or explore limited prior knowledge. This paper proposes a
dynamic MR imaging method with both k-space and spatial prior knowledge
integrated via multi-supervised network training, dubbed as DIMENSION.
Specifically, the DIMENSION architecture consists of a frequential prior
network for updating the k-space with its network prediction and a spatial
prior network for capturing image structures and details. Furthermore, a
multisupervised network training technique is developed to constrain the
frequency domain information and reconstruction results at different levels.
The comparisons with classical k-t FOCUSS, k-t SLR, L+S and the
state-of-the-art CNN-based method on in vivo datasets show our method can
achieve improved reconstruction results in shorter time.
Training Behavior of Sparse Neural Network Topologies
Authors: Simon Alford, Ryan Robinett, Lauren Milechin, Jeremy Kepner
Date: 2018-09-30 00:00:00

Improvements in the performance of deep neural networks have often come through the design of larger and more complex networks. As a result, fast memory is a significant limiting factor in our ability to improve network performance. One approach to overcoming this limit is the design of sparse neural networks, which can be both very large and efficiently trained. In this paper we experiment training on sparse neural network topologies. We test pruning-based topologies, which are derived from an initially dense network whose connections are pruned, as well as RadiX-Nets, a class of network topologies with proven connectivity and sparsity properties. Results show that sparse networks obtain accuracies comparable to dense networks, but extreme levels of sparsity cause instability in training, which merits further study.
Automatic Skin Lesion Segmentation Using GrabCut in HSV Colour Space
Authors: Fakrul Islam Tushar
Date: 2018-09-30 00:00:00
Tasks: Lesion Segmentation, Segmentation, Skin Lesion Segmentation

Skin lesion segmentation is one of the first steps towards automatic
Computer-Aided Diagnosis of skin cancer. Vast variety in the appearance of the
skin lesion makes this task very challenging. The contribution of this paper is
to apply a power foreground extraction technique called GrabCut for automatic
skin lesion segmentation with minimal human interaction in HSV color space.
Preprocessing was performed for removing the outer black border. Jaccard Index
was measured to evaluate the performance of the segmentation method. On
average, 0.71 Jaccard Index was achieved on 1000 images from ISIC challenge
2017 Training Dataset.
META-DES: A Dynamic Ensemble Selection Framework using Meta-Learning
Authors: Rafael M. O. Cruz, Robert Sabourin, George D. C. Cavalcanti, Tsang Ing Ren
Date: 2018-09-30 00:00:00
Tasks: General Classification, Meta-Learning

Dynamic ensemble selection systems work by estimating the level of competence
of each classifier from a pool of classifiers. Only the most competent ones are
selected to classify a given test sample. This is achieved by defining a
criterion to measure the level of competence of a base classifier, such as, its
accuracy in local regions of the feature space around the query instance.
However, using only one criterion about the behavior of a base classifier is
not sufficient to accurately estimate its level of competence. In this paper,
we present a novel dynamic ensemble selection framework using meta-learning. We
propose five distinct sets of meta-features, each one corresponding to a
different criterion to measure the level of competence of a classifier for the
classification of input samples. The meta-features are extracted from the
training data and used to train a meta-classifier to predict whether or not a
base classifier is competent enough to classify an input instance. During the
generalization phase, the meta-features are extracted from the query instance
and passed down as input to the meta-classifier. The meta-classifier estimates,
whether a base classifier is competent enough to be added to the ensemble.
Experiments are conducted over several small sample size classification
problems, i.e., problems with a high degree of uncertainty due to the lack of
training data. Experimental results show the proposed meta-learning framework
greatly improves classification accuracy when compared against current
state-of-the-art dynamic ensemble selection techniques.
MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling
Authors: Paweł Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, Iñigo Casanueva, Stefan Ultes, Osman Ramadan, Milica Gašić
Date: 2018-09-29 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Response Generation

Even though machine learning has become the major scene in dialogue research community, the real breakthrough has been blocked by the scale of data available. To address this fundamental obstacle, we introduce the Multi-Domain Wizard-of-Oz dataset (MultiWOZ), a fully-labeled collection of human-human written conversations spanning over multiple domains and topics. At a size of $10$k dialogues, it is at least one order of magnitude larger than all previous annotated task-oriented corpora. The contribution of this work apart from the open-sourced dataset labelled with dialogue belief states and dialogue actions is two-fold: firstly, a detailed description of the data collection procedure along with a summary of data structure and analysis is provided. The proposed data-collection pipeline is entirely based on crowd-sourcing without the need of hiring professional annotators; secondly, a set of benchmark results of belief tracking, dialogue act and response generation is reported, which shows the usability of the data and sets a baseline for future studies.
A fast quasi-Newton-type method for large-scale stochastic optimisation
Authors: Adrian Wills, Carl Jidling, Thomas Schon
Date: 2018-09-29 00:00:00
Proceeding: ICLR 2019 5
Tasks: Vocal Bursts Type Prediction

During recent years there has been an increased interest in stochastic
adaptations of limited memory quasi-Newton methods, which compared to pure
gradient-based routines can improve the convergence by incorporating second
order information. In this work we propose a direct least-squares approach
conceptually similar to the limited memory quasi-Newton methods, but that
computes the search direction in a slightly different way. This is achieved in
a fast and numerically robust manner by maintaining a Cholesky factor of low
dimension. This is combined with a stochastic line search relying upon
fulfilment of the Wolfe condition in a backtracking manner, where the step
length is adaptively modified with respect to the optimisation progress. We
support our new algorithm by providing several theoretical results guaranteeing
its performance. The performance is demonstrated on real-world benchmark
problems which shows improved results in comparison with already established
methods.
Pulsed Schlieren Imaging of Ultrasonic Haptics and Levitation using Phased Arrays
Authors: Michele Iodice, William Frier, James Wilcox, Ben Long, Orestis Georgiou
Date: 2018-09-29 00:00:00

Ultrasonic acoustic fields have recently been used to generate haptic effects
on the human skin as well as to levitate small sub-wavelength size particles.
Schlieren imaging and background-oriented schlieren techniques can be used for
acoustic wave pattern and beam shape visualization. These techniques exploit
variations in the refractive index of a propagation medium by applying
refractive optics or cross-correlation algorithms of photographs of illuminated
background patterns. Here both background-oriented and traditional schlieren
systems are used to visualize the regions of the acoustic power involved in
creating dynamic haptic sensations and dynamic levitation traps. We demonstrate
for the first time the application of back-ground-oriented schlieren for
imaging ultrasonic fields in air. We detail our imaging apparatus and present
improved algorithms used to visualize these phenomena that we have produced
using multiple phased arrays. Moreover, to improve imaging, we leverage an
electronically controlled, high-output LED which is pulsed in synchrony with
the ultrasonic carrier frequency.
Directed-Info GAIL: Learning Hierarchical Policies from Unsegmented Demonstrations using Directed Information
Authors: Arjun Sharma, Mohit Sharma, Nicholas Rhinehart, Kris M. Kitani
Date: 2018-09-29 00:00:00
Proceeding: ICLR 2019 5
Tasks: Imitation Learning

The use of imitation learning to learn a single policy for a complex task
that has multiple modes or hierarchical structure can be challenging. In fact,
previous work has shown that when the modes are known, learning separate
policies for each mode or sub-task can greatly improve the performance of
imitation learning. In this work, we discover the interaction between sub-tasks
from their resulting state-action trajectory sequences using a directed
graphical model. We propose a new algorithm based on the generative adversarial
imitation learning framework which automatically learns sub-task policies from
unsegmented demonstrations. Our approach maximizes the directed information
flow in the graphical model between sub-task latent variables and their
generated trajectories. We also show how our approach connects with the
existing Options framework, which is commonly used to learn hierarchical
policies.
CAAD 2018: Generating Transferable Adversarial Examples
Authors: Yash Sharma, Tien-Dung Le, Moustafa Alzantot
Date: 2018-09-29 00:00:00
Tasks: Adversarial Attack, Adversarial Defense, Adversarial Robustness

Deep neural networks (DNNs) are vulnerable to adversarial examples,
perturbations carefully crafted to fool the targeted DNN, in both the
non-targeted and targeted case. In the non-targeted case, the attacker simply
aims to induce misclassification. In the targeted case, the attacker aims to
induce classification to a specified target class. In addition, it has been
observed that strong adversarial examples can transfer to unknown models,
yielding a serious security concern. The NIPS 2017 competition was organized to
accelerate research in adversarial attacks and defenses, taking place in the
realistic setting where submitted adversarial attacks attempt to transfer to
submitted defenses. The CAAD 2018 competition took place with nearly identical
rules to the NIPS 2017 one. Given the requirement that the NIPS 2017
submissions were to be open-sourced, participants in the CAAD 2018 competition
were able to directly build upon previous solutions, and thus improve the
state-of-the-art in this setting. Our team participated in the CAAD 2018
competition, and won 1st place in both attack subtracks, non-targeted and
targeted adversarial attacks, and 3rd place in defense. We outline our
solutions and development results in this article. We hope our results can
inform researchers in both generating and defending against adversarial
examples.
Training Machine Learning Models by Regularizing their Explanations
Authors: Andrew Slavin Ross
Date: 2018-09-29 00:00:00
Tasks: BIG-bench Machine Learning

Neural networks are among the most accurate supervised learning methods in
use today. However, their opacity makes them difficult to trust in critical
applications, especially when conditions in training may differ from those in
practice. Recent efforts to develop explanations for neural networks and
machine learning models more generally have produced tools to shed light on the
implicit rules behind predictions. These tools can help us identify when models
are right for the wrong reasons. However, they do not always scale to
explaining predictions for entire datasets, are not always at the right level
of abstraction, and most importantly cannot correct the problems they reveal.
In this thesis, we explore the possibility of training machine learning models
(with a particular focus on neural networks) using explanations themselves. We
consider approaches where models are penalized not only for making incorrect
predictions but also for providing explanations that are either inconsistent
with domain knowledge or overly complex. These methods let us train models
which can not only provide more interpretable rationales for their predictions
but also generalize better when training data is confounded or meaningfully
different from test data (even adversarially so).
Reinforcement Learning in R
Authors: Nicolas Pröllochs, Stefan Feuerriegel
Date: 2018-09-29 00:00:00
Tasks: Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Reinforcement learning refers to a group of methods from artificial
intelligence where an agent performs learning through trial and error. It
differs from supervised learning, since reinforcement learning requires no
explicit labels; instead, the agent interacts continuously with its
environment. That is, the agent starts in a specific state and then performs an
action, based on which it transitions to a new state and, depending on the
outcome, receives a reward. Different strategies (e.g. Q-learning) have been
proposed to maximize the overall reward, resulting in a so-called policy, which
defines the best possible action in each state. Mathematically, this process
can be formalized by a Markov decision process and it has been implemented by
packages in R; however, there is currently no package available for
reinforcement learning. As a remedy, this paper demonstrates how to perform
reinforcement learning in R and, for this purpose, introduces the
ReinforcementLearning package. The package provides a remarkably flexible
framework and is easily applied to a wide range of different problems. We
demonstrate its use by drawing upon common examples from the literature (e.g.
finding optimal game strategies).
Deep Adversarial Training for Multi-Organ Nuclei Segmentation in Histopathology Images
Authors: Faisal Mahmood, Daniel Borders, Richard Chen, Gregory N. McKay, Kevan J. Salimian, Alexander Baras, Nicholas J. Durr
Date: 2018-09-29 00:00:00
Tasks: Generative Adversarial Network, Segmentation, Structured Prediction

Nuclei segmentation is a fundamental task that is critical for various
computational pathology applications including nuclei morphology analysis, cell
type classification, and cancer grading. Conventional vision-based methods for
nuclei segmentation struggle in challenging cases and deep learning approaches
have proven to be more robust and generalizable. However, CNNs require large
amounts of labeled histopathology data. Moreover, conventional CNN-based
approaches lack structured prediction capabilities which are required to
distinguish overlapping and clumped nuclei. Here, we present an approach to
nuclei segmentation that overcomes these challenges by utilizing a conditional
generative adversarial network (cGAN) trained with synthetic and real data. We
generate a large dataset of H&E training images with perfect nuclei
segmentation labels using an unpaired GAN framework. This synthetic data along
with real histopathology data from six different organs are used to train a
conditional GAN with spectral normalization and gradient penalty for nuclei
segmentation. This adversarial regression framework enforces higher order
consistency when compared to conventional CNN models. We demonstrate that this
nuclei segmentation approach generalizes across different organs, sites,
patients and disease states, and outperforms conventional approaches,
especially in isolating individual and overlapping nuclei.