pwc-archive/papers-with-abstracts huggingface dataset:
On Latent Distributions Without Finite Mean in Generative Models
Authors: Damian Leśniak, Igor Sieradzki, Igor Podolak
Date: 2018-06-05 00:00:00

We investigate the properties of multidimensional probability distributions
in the context of latent space prior distributions of implicit generative
models. Our work revolves around the phenomena arising while decoding linear
interpolations between two random latent vectors -- regions of latent space in
close proximity to the origin of the space are sampled causing distribution
mismatch. We show that due to the Central Limit Theorem, this region is almost
never sampled during the training process. As a result, linear interpolations
may generate unrealistic data and their usage as a tool to check quality of the
trained model is questionable. We propose to use multidimensional Cauchy
distribution as the latent prior. Cauchy distribution does not satisfy the
assumptions of the CLT and has a number of properties that allow it to work
well in conjunction with linear interpolations. We also provide two general
methods of creating non-linear interpolations that are easily applicable to a
large family of common latent distributions. Finally we empirically analyze the
quality of data generated from low-probability-mass regions for the DCGAN model
on the CelebA dataset.
Phonocardiographic Sensing using Deep Learning for Abnormal Heartbeat Detection
Authors: Siddique Latif, Muhammad Usman, Rajib Rana, Junaid Qadir
Date: 2018-01-25 00:00:00
Tasks: Deep Learning, Heartbeat Classification

Cardiac auscultation involves expert interpretation of abnormalities in heart sounds using stethoscope. Deep learning based cardiac auscultation is of significant interest to the healthcare community as it can help reducing the burden of manual auscultation with automated detection of abnormal heartbeats. However, the problem of automatic cardiac auscultation is complicated due to the requirement of reliability and high accuracy, and due to the presence of background noise in the heartbeat sound. In this work, we propose a Recurrent Neural Networks (RNNs) based automated cardiac auscultation solution. Our choice of RNNs is motivated by the great success of deep learning in medical applications and by the observation that RNNs represent the deep learning configuration most suitable for dealing with sequential or temporal data even in the presence of noise. We explore the use of various RNN models, and demonstrate that these models deliver the abnormal heartbeat classification score with significant improvement. Our proposed approach using RNNs can be potentially be used for real-time abnormal heartbeat detection in the Internet of Medical Things for remote monitoring applications.
Hierarchical Graph Clustering using Node Pair Sampling
Authors: Thomas Bonald, Bertrand Charpentier, Alexis Galland, Alexandre Hollocou
Date: 2018-06-05 00:00:00
Tasks: Clustering, Graph Clustering

We present a novel hierarchical graph clustering algorithm inspired by
modularity-based clustering techniques. The algorithm is agglomerative and
based on a simple distance between clusters induced by the probability of
sampling node pairs. We prove that this distance is reducible, which enables
the use of the nearest-neighbor chain to speed up the agglomeration. The output
of the algorithm is a regular dendrogram, which reveals the multi-scale
structure of the graph. The results are illustrated on both synthetic and real
datasets.
Deep Gaussian Processes with Convolutional Kernels
Authors: Vinayak Kumar, Vaibhav Singh, P. K. Srijith, Andreas Damianou
Date: 2018-06-05 00:00:00
Tasks: Gaussian Processes, image-classification, Image Classification

Deep Gaussian processes (DGPs) provide a Bayesian non-parametric alternative
to standard parametric deep learning models. A DGP is formed by stacking
multiple GPs resulting in a well-regularized composition of functions. The
Bayesian framework that equips the model with attractive properties, such as
implicit capacity control and predictive uncertainty, makes it at the same time
challenging to combine with a convolutional structure. This has hindered the
application of DGPs in computer vision tasks, an area where deep parametric
models (i.e. CNNs) have made breakthroughs. Standard kernels used in DGPs such
as radial basis functions (RBFs) are insufficient for handling pixel
variability in raw images. In this paper, we build on the recent convolutional
GP to develop Convolutional DGP (CDGP) models which effectively capture image
level features through the use of convolution kernels, therefore opening up the
way for applying DGPs to computer vision tasks. Our model learns local spatial
influence and outperforms strong GP based baselines on multi-class image
classification. We also consider various constructions of convolution kernel
over the image patches, analyze the computational trade-offs and provide an
efficient framework for convolutional DGP models. The experimental results on
image data such as MNIST, rectangles-image, CIFAR10 and Caltech101 demonstrate
the effectiveness of the proposed approaches.
EffNet: An Efficient Structure for Convolutional Neural Networks
Authors: Ido Freeman, Lutz Roese-Koerner, Anton Kummert
Date: 2018-01-19 00:00:00

With the ever increasing application of Convolutional Neural Networks to
customer products the need emerges for models to efficiently run on embedded,
mobile hardware. Slimmer models have therefore become a hot research topic with
various approaches which vary from binary networks to revised convolution
layers. We offer our contribution to the latter and propose a novel convolution
block which significantly reduces the computational burden while surpassing the
current state-of-the-art. Our model, dubbed EffNet, is optimised for models
which are slim to begin with and is created to tackle issues in existing models
such as MobileNet and ShuffleNet.
Real-time Lane Marker Detection Using Template Matching with RGB-D Camera
Authors: Cong Hoang Quach, Van Lien Tran, Duy Hung Nguyen, Viet Thang Nguyen, Minh Trien Pham, Manh Duong Phung
Date: 2018-06-05 00:00:00
Tasks: Autonomous Driving, Lane Detection, Template Matching

This paper addresses the problem of lane detection which is fundamental for
self-driving vehicles. Our approach exploits both colour and depth information
recorded by a single RGB-D camera to better deal with negative factors such as
lighting conditions and lane-like objects. In the approach, colour and depth
images are first converted to a half-binary format and a 2D matrix of 3D
points. They are then used as the inputs of template matching and geometric
feature extraction processes to form a response map so that its values
represent the probability of pixels being lane markers. To further improve the
results, the template and lane surfaces are finally refined by principal
component analysis and lane model fitting techniques. A number of experiments
have been conducted on both synthetic and real datasets. The result shows that
the proposed approach can effectively eliminate unwanted noise to accurately
detect lane markers in various scenarios. Moreover, the processing speed of 20
frames per second under hardware configuration of a popular laptop computer
allows the proposed algorithm to be implemented for real-time autonomous
driving applications.
Explaining Away Syntactic Structure in Semantic Document Representations
Authors: Erik Holmer, Andreas Marfurt
Date: 2018-06-05 00:00:00

Most generative document models act on bag-of-words input in an attempt to
focus on the semantic content and thereby partially forego syntactic
information. We argue that it is preferable to keep the original word order
intact and explicitly account for the syntactic structure instead. We propose
an extension to the Neural Variational Document Model (Miao et al., 2016) that
does exactly that to separate local (syntactic) context from the global
(semantic) representation of the document. Our model builds on the variational
autoencoder framework to define a generative document model based on next-word
prediction. We name our approach Sequence-Aware Variational Autoencoder since
in contrast to its predecessor, it operates on the true input sequence. In a
series of experiments we observe stronger topicality of the learned
representations as well as increased robustness to syntactic noise in our
training data.
BOCK : Bayesian Optimization with Cylindrical Kernels
Authors: Changyong Oh, Efstratios Gavves, Max Welling
Date: 2018-06-05 00:00:00
Proceeding: ICML 2018 7
Tasks: Bayesian Optimization

A major challenge in Bayesian Optimization is the boundary issue (Swersky, 2017) where an algorithm spends too many evaluations near the boundary of its search space. In this paper, we propose BOCK, Bayesian Optimization with Cylindrical Kernels, whose basic idea is to transform the ball geometry of the search space using a cylindrical transformation. Because of the transformed geometry, the Gaussian Process-based surrogate model spends less budget searching near the boundary, while concentrating its efforts relatively more near the center of the search region, where we expect the solution to be located. We evaluate BOCK extensively, showing that it is not only more accurate and efficient, but it also scales successfully to problems with a dimensionality as high as 500. We show that the better accuracy and scalability of BOCK even allows optimizing modestly sized neural network layers, as well as neural network hyperparameters.
Good and safe uses of AI Oracles
Authors: Stuart Armstrong, Xavier O'Rorke
Date: 2017-11-15 00:00:00
Tasks: counterfactual

It is possible that powerful and potentially dangerous artificial
intelligence (AI) might be developed in the future. An Oracle is a design which
aims to restrain the impact of a potentially dangerous AI by restricting the
agent to no actions besides answering questions. Unfortunately, most Oracles
will be motivated to gain more control over the world by manipulating users
through the content of their answers, and Oracles of potentially high
intelligence might be very successful at this
\citep{DBLP:journals/corr/AlfonsecaCACAR16}. In this paper we present two
designs for Oracles which, even under pessimistic assumptions, will not
manipulate their users into releasing them and yet will still be incentivised
to provide their users with helpful answers. The first design is the
counterfactual Oracle -- which choses its answer as if it expected nobody to
ever read it. The second design is the low-bandwidth Oracle -- which is limited
by the quantity of information it can transmit.
'Indifference' methods for managing agent rewards
Authors: Stuart Armstrong, Xavier O'Rourke
Date: 2017-12-18 00:00:00

`Indifference' refers to a class of methods used to control reward based
agents. Indifference techniques aim to achieve one or more of three distinct
goals: rewards dependent on certain events (without the agent being motivated
to manipulate the probability of those events), effective disbelief (where
agents behave as if particular events could never happen), and seamless
transition from one reward function to another (with the agent acting as if
this change is unanticipated). This paper presents several methods for
achieving these goals in the POMDP setting, establishing their uses, strengths,
and requirements. These methods of control work even when the implications of
the agent's reward are otherwise not fully understood.
ALMN: Deep Embedding Learning with Geometrical Virtual Point Generating
Authors: Binghui Chen, Weihong Deng
Date: 2018-06-04 00:00:00
Tasks: Clustering, Image Retrieval, Open-Ended Question Answering, Retrieval

Deep embedding learning becomes more attractive for discriminative feature
learning, but many methods still require hard-class mining, which is
computationally complex and performance-sensitive. To this end, we propose
Adaptive Large Margin N-Pair loss (ALMN) to address the aforementioned issues.
Instead of exploring hard example-mining strategy, we introduce the concept of
large margin constraint. This constraint aims at encouraging local-adaptive
large angular decision margin among dissimilar samples in multimodal feature
space so as to significantly encourage intraclass compactness and interclass
separability. And it is mainly achieved by a simple yet novel geometrical
Virtual Point Generating (VPG) method, which converts artificially setting a
fixed margin into automatically generating a boundary training sample in
feature space and is an open question. We demonstrate the effectiveness of our
method on several popular datasets for image retrieval and clustering tasks.
Low Cost Edge Sensing for High Quality Demosaicking
Authors: Yan Niu, Jihong Ouyang, Wanli Zuo, Fuxin Wang
Date: 2018-06-03 00:00:00
Tasks: Demosaicking, Vocal Bursts Intensity Prediction

Digital cameras that use Color Filter Arrays (CFA) entail a demosaicking
procedure to form full RGB images. As today's camera users generally require
images to be viewed instantly, demosaicking algorithms for real applications
must be fast. Moreover, the associated cost should be lower than the cost saved
by using CFA. For this purpose, we revisit the classical Hamilton-Adams (HA)
algorithm, which outperforms many sophisticated techniques in both speed and
accuracy. Inspired by HA's strength and weakness, we design a very low cost
edge sensing scheme. Briefly, it guides demosaicking by a logistic functional
of the difference between directional variations. We extensively compare our
algorithm with 28 demosaicking algorithms by running their open source codes on
benchmark datasets. Compared to methods of similar computational cost, our
method achieves substantially higher accuracy, Whereas compared to methods of
similar accuracy, our method has significantly lower cost. Moreover, on test
images of currently popular resolution, the quality of our algorithm is
comparable to top performers, whereas its speed is tens of times faster.
Layer rotation: a surprisingly powerful indicator of generalization in deep networks?
Authors: Simon Carbonnelle, Christophe De Vleeschouwer
Date: 2018-06-05 00:00:00

Our work presents extensive empirical evidence that layer rotation, i.e. the evolution across training of the cosine distance between each layer's weight vector and its initialization, constitutes an impressively consistent indicator of generalization performance. In particular, larger cosine distances between final and initial weights of each layer consistently translate into better generalization performance of the final model. Interestingly, this relation admits a network independent optimum: training procedures during which all layers' weights reach a cosine distance of 1 from their initialization consistently outperform other configurations -by up to 30% test accuracy. Moreover, we show that layer rotations are easily monitored and controlled (helpful for hyperparameter tuning) and potentially provide a unified framework to explain the impact of learning rate tuning, weight decay, learning rate warmups and adaptive gradient methods on generalization and training speed. In an attempt to explain the surprising properties of layer rotation, we show on a 1-layer MLP trained on MNIST that layer rotation correlates with the degree to which features of intermediate layers have been trained.
Dense Pooling layers in Fully Convolutional Network for Skin Lesion Segmentation
Authors: Ebrahim Nasr-Esfahani, Shima Rafiei, Mohammad H. Jafari, Nader Karimi, James S. Wrobel, S. M. Reza Soroushmehr, Shadrokh Samavi, Kayvan Najarian
Date: 2017-12-29 00:00:00
Tasks: Lesion Segmentation, Medical Image Analysis, Segmentation, Skin Lesion Segmentation

One of the essential tasks in medical image analysis is segmentation and accurate detection of borders. Lesion segmentation in skin images is an essential step in the computerized detection of skin cancer. However, many of the state-of-the-art segmentation methods have deficiencies in their border detection phase. In this paper, a new class of fully convolutional network is proposed, with new dense pooling layers for segmentation of lesion regions in skin images. This network leads to highly accurate segmentation of lesions on skin lesion datasets which outperforms state-of-the-art algorithms in the skin lesion segmentation.
Stochastic Optimization from Distributed, Streaming Data in Rate-limited Networks
Authors: Matthew Nokleby, Waheed U. Bajwa
Date: 2017-04-25 00:00:00
Tasks: Stochastic Optimization

Motivated by machine learning applications in networks of sensors,
internet-of-things (IoT) devices, and autonomous agents, we propose techniques
for distributed stochastic convex learning from high-rate data streams. The
setup involves a network of nodes---each one of which has a stream of data
arriving at a constant rate---that solve a stochastic convex optimization
problem by collaborating with each other over rate-limited communication links.
To this end, we present and analyze two algorithms---termed distributed
stochastic approximation mirror descent (D-SAMD) and accelerated distributed
stochastic approximation mirror descent (AD-SAMD)---that are based on two
stochastic variants of mirror descent and in which nodes collaborate via
approximate averaging of the local, noisy subgradients using distributed
consensus. Our main contributions are (i) bounds on the convergence rates of
D-SAMD and AD-SAMD in terms of the number of nodes, network topology, and ratio
of the data streaming and communication rates, and (ii) sufficient conditions
for order-optimum convergence of these algorithms. In particular, we show that
for sufficiently well-connected networks, distributed learning schemes can
obtain order-optimum convergence even if the communications rate is small.
Further we find that the use of accelerated methods significantly enlarges the
regime in which order-optimum convergence is achieved; this is in contrast to
the centralized setting, where accelerated methods usually offer only a modest
improvement. Finally, we demonstrate the effectiveness of the proposed
algorithms using numerical experiments.
Accelerated Randomized Coordinate Descent Algorithms for Stochastic Optimization and Online Learning
Authors: Akshita Bhandari, Chandramani Singh
Date: 2018-06-05 00:00:00
Tasks: Stochastic Optimization

We propose accelerated randomized coordinate descent algorithms for
stochastic optimization and online learning. Our algorithms have significantly
less per-iteration complexity than the known accelerated gradient algorithms.
The proposed algorithms for online learning have better regret performance than
the known randomized online coordinate descent algorithms. Furthermore, the
proposed algorithms for stochastic optimization exhibit as good convergence
rates as the best known randomized coordinate descent algorithms. We also show
simulation results to demonstrate performance of the proposed algorithms.
Stochastic Gradient Descent with Hyperbolic-Tangent Decay on Classification
Authors: Bo Yang Hsueh, Wei Li, I-Chen Wu
Date: 2018-06-05 00:00:00
Tasks: Classification, General Classification, Scheduling

Learning rate scheduler has been a critical issue in the deep neural network
training. Several schedulers and methods have been proposed, including step
decay scheduler, adaptive method, cosine scheduler and cyclical scheduler. This
paper proposes a new scheduling method, named hyperbolic-tangent decay (HTD).
We run experiments on several benchmarks such as: ResNet, Wide ResNet and
DenseNet for CIFAR-10 and CIFAR-100 datasets, LSTM for PAMAP2 dataset, ResNet
on ImageNet and Fashion-MNIST datasets. In our experiments, HTD outperforms
step decay and cosine scheduler in nearly all cases, while requiring less
hyperparameters than step decay, and more flexible than cosine scheduler. Code
is available at https://github.com/BIGBALLON/HTD.
L4: Practical loss-based stepsize adaptation for deep learning
Authors: Michal Rolinek, Georg Martius
Date: 2018-02-14 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Deep Learning

We propose a stepsize adaptation scheme for stochastic gradient descent. It
operates directly with the loss function and rescales the gradient in order to
make fixed predicted progress on the loss. We demonstrate its capabilities by
conclusively improving the performance of Adam and Momentum optimizers. The
enhanced optimizers with default hyperparameters consistently outperform their
constant stepsize counterparts, even the best ones, without a measurable
increase in computational cost. The performance is validated on multiple
architectures including dense nets, CNNs, ResNets, and the recurrent
Differential Neural Computer on classical datasets MNIST, fashion MNIST,
CIFAR10 and others.
Discovering and Removing Exogenous State Variables and Rewards for Reinforcement Learning
Authors: Thomas G. Dietterich, George Trimponias, Zhitang Chen
Date: 2018-06-05 00:00:00
Proceeding: ICML 2018 7
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Exogenous state variables and rewards can slow down reinforcement learning by
injecting uncontrolled variation into the reward signal. We formalize exogenous
state variables and rewards and identify conditions under which an MDP with
exogenous state can be decomposed into an exogenous Markov Reward Process
involving only the exogenous state+reward and an endogenous Markov Decision
Process defined with respect to only the endogenous rewards. We also derive a
variance-covariance condition under which Monte Carlo policy evaluation on the
endogenous MDP is accelerated compared to using the full MDP. Similar speedups
are likely to carry over to all RL algorithms. We develop two algorithms for
discovering the exogenous variables and test them on several MDPs. Results show
that the algorithms are practical and can significantly speed up reinforcement
learning.
GuideR: a guided separate-and-conquer rule learning in classification, regression, and survival settings
Authors: Marek Sikora, Łukasz Wróbel, Adam Gudyś
Date: 2018-06-05 00:00:00
Tasks: Attribute, General Classification, regression, Survival Analysis

This article presents GuideR, a user-guided rule induction algorithm, which
overcomes the largest limitation of the existing methods-the lack of the
possibility to introduce user's preferences or domain knowledge to the rule
learning process. Automatic selection of attributes and attribute ranges often
leads to the situation in which resulting rules do not contain interesting
information. We propose an induction algorithm which takes into account user's
requirements. Our method uses the sequential covering approach and is suitable
for classification, regression, and survival analysis problems. The
effectiveness of the algorithm in all these tasks has been verified
experimentally, confirming guided rule induction to be a powerful data analysis
tool.