pwc-archive/papers-with-abstracts huggingface dataset:
Regret Analysis for Continuous Dueling Bandit
Authors: Wataru Kumagai
Date: 2017-11-21 00:00:00
Proceeding: NeurIPS 2017 12

The dueling bandit is a learning framework wherein the feedback information
in the learning process is restricted to a noisy comparison between a pair of
actions. In this research, we address a dueling bandit problem based on a cost
function over a continuous space. We propose a stochastic mirror descent
algorithm and show that the algorithm achieves an $O(\sqrt{T\log T})$-regret
bound under strong convexity and smoothness assumptions for the cost function.
Subsequently, we clarify the equivalence between regret minimization in dueling
bandit and convex optimization for the cost function. Moreover, when
considering a lower bound in convex optimization, our algorithm is shown to
achieve the optimal convergence rate in convex optimization and the optimal
regret in dueling bandit except for a logarithmic factor.
Transportation analysis of denoising autoencoders: a novel method for analyzing deep neural networks
Authors: Sho Sonoda, Noboru Murata
Date: 2017-12-12 00:00:00
Tasks: Denoising

The feature map obtained from the denoising autoencoder (DAE) is investigated
by determining transportation dynamics of the DAE, which is a cornerstone for
deep learning. Despite the rapid development in its application, deep neural
networks remain analytically unexplained, because the feature maps are nested
and parameters are not faithful. In this paper, we address the problem of the
formulation of nested complex of parameters by regarding the feature map as a
transport map. Even when a feature map has different dimensions between input
and output, we can regard it as a transportation map by considering that both
the input and output spaces are embedded in a common high-dimensional space. In
addition, the trajectory is a geometric object and thus, is independent of
parameterization. In this manner, transportation can be regarded as a universal
character of deep neural networks. By determining and analyzing the
transportation dynamics, we can understand the behavior of a deep neural
network. In this paper, we investigate a fundamental case of deep neural
networks: the DAE. We derive the transport map of the DAE, and reveal that the
infinitely deep DAE transports mass to decrease a certain quantity, such as
entropy, of the data distribution. These results though analytically simple,
shed light on the correspondence between deep neural networks and the
Wasserstein gradient flows.
Information Perspective to Probabilistic Modeling: Boltzmann Machines versus Born Machines
Authors: Song Cheng, Jing Chen, Lei Wang
Date: 2017-12-12 00:00:00

We compare and contrast the statistical physics and quantum physics inspired
approaches for unsupervised generative modeling of classical data. The two
approaches represent probabilities of observed data using energy-based models
and quantum states respectively.Classical and quantum information patterns of
the target datasets therefore provide principled guidelines for structural
design and learning in these two approaches. Taking the restricted Boltzmann
machines (RBM) as an example, we analyze the information theoretical bounds of
the two approaches. We verify our reasonings by comparing the performance of
RBMs of various architectures on the standard MNIST datasets.
Deep learning enhanced mobile-phone microscopy
Authors: Yair Rivenson, Hatice Ceylan Koydemir, Hongda Wang, Zhensong Wei, Zhengshuang Ren, Harun Gunaydin, Yibo Zhang, Zoltan Gorocs, Kyle Liang, Derek Tseng, Aydogan Ozcan
Date: 2017-12-12 00:00:00
Tasks: Deep Learning

Mobile-phones have facilitated the creation of field-portable, cost-effective
imaging and sensing technologies that approach laboratory-grade instrument
performance. However, the optical imaging interfaces of mobile-phones are not
designed for microscopy and produce spatial and spectral distortions in imaging
microscopic specimens. Here, we report on the use of deep learning to correct
such distortions introduced by mobile-phone-based microscopes, facilitating the
production of high-resolution, denoised and colour-corrected images, matching
the performance of benchtop microscopes with high-end objective lenses, also
extending their limited depth-of-field. After training a convolutional neural
network, we successfully imaged various samples, including blood smears,
histopathology tissue sections, and parasites, where the recorded images were
highly compressed to ease storage and transmission for telemedicine
applications. This method is applicable to other low-cost, aberrated imaging
systems, and could offer alternatives for costly and bulky microscopes, while
also providing a framework for standardization of optical images for clinical
and biomedical applications.
A vision based system for underwater docking
Authors: Shuang Liu, Mete Ozay, Takayuki Okatani, Hongli Xu, Kai Sun, Yang Lin
Date: 2017-12-12 00:00:00
Tasks: Pose Estimation, Position

Autonomous underwater vehicles (AUVs) have been deployed for underwater
exploration. However, its potential is confined by its limited on-board battery
energy and data storage capacity. This problem has been addressed using docking
systems by underwater recharging and data transfer for AUVs. In this work, we
propose a vision based framework for underwater docking following these
systems. The proposed framework comprises two modules; (i) a detection module
which provides location information on underwater docking stations in 2D images
captured by an on-board camera, and (ii) a pose estimation module which
recovers the relative 3D position and orientation between docking stations and
AUVs from the 2D images. For robust and credible detection of docking stations,
we propose a convolutional neural network called Docking Neural Network (DoNN).
For accurate pose estimation, a perspective-n-point algorithm is integrated
into our framework. In order to examine our framework in underwater docking
tasks, we collected a dataset of 2D images, named Underwater Docking Images
Dataset (UDID), in an experimental water pool. To the best of our knowledge,
UDID is the first publicly available underwater docking dataset. In the
experiments, we first evaluate performance of the proposed detection module on
UDID and its deformed variations. Next, we assess the accuracy of the pose
estimation module by ground experiments, since it is not feasible to obtain
true relative position and orientation between docking stations and AUVs under
water. Then, we examine the pose estimation module by underwater experiments in
our experimental water pool. Experimental results show that the proposed
framework can be used to detect docking stations and estimate their relative
pose efficiently and successfully, compared to the state-of-the-art baseline
systems.
Time Matters: Multi-scale Temporalization of Social Media Popularity
Authors: Bo Wu, Wen-Huang Cheng, Yongdong Zhang, Tao Mei
Date: 2017-12-12 00:00:00
Tasks: Social Media Popularity Prediction

The evolution of social media popularity exhibits rich temporality, i.e.,
popularities change over time at various levels of temporal granularity. This
is influenced by temporal variations of public attentions or user activities.
For example, popularity patterns of street snap on Flickr are observed to
depict distinctive fashion styles at specific time scales, such as season-based
periodic fluctuations for Trench Coat or one-off peak in days for Evening
Dress. However, this fact is often overlooked by existing research of
popularity modeling. We present the first study to incorporate multiple
time-scale dynamics into predicting online popularity. We propose a novel
computational framework in the paper, named Multi-scale Temporalization, for
estimating popularity based on multi-scale decomposition and structural
reconstruction in a tensor space of user, post, and time by joint low-rank
constraints. By considering the noise caused by context inconsistency, we
design a data rearrangement step based on context aggregation as preprocessing
to enhance contextual relevance of neighboring data in the tensor space. As a
result, our approach can leverage multiple levels of temporal characteristics
and reduce the noise of data decomposition to improve modeling effectiveness.
We evaluate our approach on two large-scale Flickr image datasets with over 1.8
million photos in total, for the task of popularity prediction. The results
show that our approach significantly outperforms state-of-the-art popularity
prediction techniques, with a relative improvement of 10.9%-47.5% in terms of
prediction accuracy.
Sequential Prediction of Social Media Popularity with Deep Temporal Context Networks
Authors: Bo Wu, Wen-Huang Cheng, Yongdong Zhang, Qiushi Huang, Jintao Li, Tao Mei
Date: 2017-12-12 00:00:00
Tasks: Prediction, Social Media Popularity Prediction

Prediction of popularity has profound impact for social media, since it
offers opportunities to reveal individual preference and public attention from
evolutionary social systems. Previous research, although achieves promising
results, neglects one distinctive characteristic of social data, i.e.,
sequentiality. For example, the popularity of online content is generated over
time with sequential post streams of social media. To investigate the
sequential prediction of popularity, we propose a novel prediction framework
called Deep Temporal Context Networks (DTCN) by incorporating both temporal
context and temporal attention into account. Our DTCN contains three main
components, from embedding, learning to predicting. With a joint embedding
network, we obtain a unified deep representation of multi-modal user-post data
in a common embedding space. Then, based on the embedded data sequence over
time, temporal context learning attempts to recurrently learn two adaptive
temporal contexts for sequential popularity. Finally, a novel temporal
attention is designed to predict new popularity (the popularity of a new
user-post pair) with temporal coherence across multiple time-scales.
Experiments on our released image dataset with about 600K Flickr photos
demonstrate that DTCN outperforms state-of-the-art deep prediction algorithms,
with an average of 21.51% relative performance improvement in the popularity
prediction (Spearman Ranking Correlation).
Predictive-Corrective Networks for Action Detection
Authors: Achal Dave, Olga Russakovsky, Deva Ramanan
Date: 2017-04-12 00:00:00
Proceeding: CVPR 2017 7
Tasks: Action Detection, Optical Flow Estimation, Time Series, Time Series Analysis

While deep feature learning has revolutionized techniques for static-image
understanding, the same does not quite hold for video processing. Architectures
and optimization techniques used for video are largely based off those for
static images, potentially underutilizing rich video information. In this work,
we rethink both the underlying network architecture and the stochastic learning
paradigm for temporal data. To do so, we draw inspiration from classic theory
on linear dynamic systems for modeling time series. By extending such models to
include nonlinear mappings, we derive a series of novel recurrent neural
networks that sequentially make top-down predictions about the future and then
correct those predictions with bottom-up observations. Predictive-corrective
networks have a number of desirable properties: (1) they can adaptively focus
computation on "surprising" frames where predictions require large corrections,
(2) they simplify learning in that only "residual-like" corrective terms need
to be learned over time and (3) they naturally decorrelate an input data stream
in a hierarchical fashion, producing a more reliable signal for learning at
each layer of a network. We provide an extensive analysis of our lightweight
and interpretable framework, and demonstrate that our model is competitive with
the two-stream network on three challenging datasets without the need for
computationally expensive optical flow.
Deep Learning for Reliable Mobile Edge Analytics in Intelligent Transportation Systems
Authors: Aidin Ferdowsi, Ursula Challita, Walid Saad
Date: 2017-12-12 00:00:00
Tasks: Deep Learning, Edge-computing

Intelligent transportation systems (ITSs) will be a major component of
tomorrow's smart cities. However, realizing the true potential of ITSs requires
ultra-low latency and reliable data analytics solutions that can combine, in
real-time, a heterogeneous mix of data stemming from the ITS network and its
environment. Such data analytics capabilities cannot be provided by
conventional cloud-centric data processing techniques whose communication and
computing latency can be high. Instead, edge-centric solutions that are
tailored to the unique ITS environment must be developed. In this paper, an
edge analytics architecture for ITSs is introduced in which data is processed
at the vehicle or roadside smart sensor level in order to overcome the ITS
latency and reliability challenges. With a higher capability of passengers'
mobile devices and intra-vehicle processors, such a distributed edge computing
architecture can leverage deep learning techniques for reliable mobile sensing
in ITSs. In this context, the ITS mobile edge analytics challenges pertaining
to heterogeneous data, autonomous control, vehicular platoon control, and
cyber-physical security are investigated. Then, different deep learning
solutions for such challenges are proposed. The proposed deep learning
solutions will enable ITS edge analytics by endowing the ITS devices with
powerful computer vision and signal processing functions. Preliminary results
show that the proposed edge analytics architecture, coupled with the power of
deep learning algorithms, can provide a reliable, secure, and truly smart
transportation environment.
Outlier Detection by Consistent Data Selection Method
Authors: Utkarsh Porwal, Smruthi Mukund
Date: 2017-12-12 00:00:00
Tasks: Anomaly Detection, Clustering, One-class classifier, Outlier Detection, Spam detection

Often the challenge associated with tasks like fraud and spam detection[1] is
the lack of all likely patterns needed to train suitable supervised learning
models. In order to overcome this limitation, such tasks are attempted as
outlier or anomaly detection tasks. We also hypothesize that out- liers have
behavioral patterns that change over time. Limited data and continuously
changing patterns makes learning significantly difficult. In this work we are
proposing an approach that detects outliers in large data sets by relying on
data points that are consistent. The primary contribution of this work is that
it will quickly help retrieve samples for both consistent and non-outlier data
sets and is also mindful of new outlier patterns. No prior knowledge of each
set is required to extract the samples. The method consists of two phases, in
the first phase, consistent data points (non- outliers) are retrieved by an
ensemble method of unsupervised clustering techniques and in the second phase a
one class classifier trained on the consistent data point set is ap- plied on
the remaining sample set to identify the outliers. The approach is tested on
three publicly available data sets and the performance scores are competitive.
Deep Potential Molecular Dynamics: a scalable model with the accuracy of quantum mechanics
Authors: Linfeng Zhang, Jiequn Han, Han Wang, Roberto Car, Weinan E
Date: 2017-07-30 00:00:00

We introduce a scheme for molecular simulations, the Deep Potential Molecular
Dynamics (DeePMD) method, based on a many-body potential and interatomic forces
generated by a carefully crafted deep neural network trained with ab initio
data. The neural network model preserves all the natural symmetries in the
problem. It is "first principle-based" in the sense that there are no ad hoc
components aside from the network model. We show that the proposed scheme
provides an efficient and accurate protocol in a variety of systems, including
bulk materials and molecules. In all these cases, DeePMD gives results that are
essentially indistinguishable from the original data, at a cost that scales
linearly with system size.
Modular Continual Learning in a Unified Visual Environment
Authors: Kevin T. Feigelis, Blue Sheffer, Daniel L. K. Yamins
Date: 2017-11-20 00:00:00
Proceeding: ICLR 2018 1
Tasks: Continual Learning, Reinforcement Learning

A core aspect of human intelligence is the ability to learn new tasks quickly
and switch between them flexibly. Here, we describe a modular continual
reinforcement learning paradigm inspired by these abilities. We first introduce
a visual interaction environment that allows many types of tasks to be unified
in a single framework. We then describe a reward map prediction scheme that
learns new tasks robustly in the very large state and action spaces required by
such an environment. We investigate how properties of module architecture
influence efficiency of task learning, showing that a module motif
incorporating specific design principles (e.g. early bottlenecks, low-order
polynomial nonlinearities, and symmetry) significantly outperforms more
standard neural network motifs, needing fewer training examples and fewer
neurons to achieve high levels of performance. Finally, we present a
meta-controller architecture for task switching based on a dynamic neural
voting scheme, which allows new modules to use information learned from
previously-seen tasks to substantially improve their own learning efficiency.
GibbsNet: Iterative Adversarial Inference for Deep Graphical Models
Authors: Alex Lamb, Devon Hjelm, Yaroslav Ganin, Joseph Paul Cohen, Aaron Courville, Yoshua Bengio
Date: 2017-12-12 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Attribute

Directed latent variable models that formulate the joint distribution as
$p(x,z) = p(z) p(x \mid z)$ have the advantage of fast and exact sampling.
However, these models have the weakness of needing to specify $p(z)$, often
with a simple fixed prior that limits the expressiveness of the model.
Undirected latent variable models discard the requirement that $p(z)$ be
specified with a prior, yet sampling from them generally requires an iterative
procedure such as blocked Gibbs-sampling that may require many steps to draw
samples from the joint distribution $p(x, z)$. We propose a novel approach to
learning the joint distribution between the data and a latent code which uses
an adversarially learned iterative procedure to gradually refine the joint
distribution, $p(x, z)$, to better match with the data distribution on each
step. GibbsNet is the best of both worlds both in theory and in practice.
Achieving the speed and simplicity of a directed latent variable model, it is
guaranteed (assuming the adversarial game reaches the virtual training criteria
global minimum) to produce samples from $p(x, z)$ with only a few sampling
iterations. Achieving the expressiveness and flexibility of an undirected
latent variable model, GibbsNet does away with the need for an explicit $p(z)$
and has the ability to do attribute prediction, class-conditional generation,
and joint image-attribute modeling in a single model which is not trained for
any of these specific tasks. We show empirically that GibbsNet is able to learn
a more complex $p(z)$ and show that this leads to improved inpainting and
iterative refinement of $p(x, z)$ for dozens of steps and stable generation
without collapse for thousands of steps, despite being trained on only a few
steps.
200x Low-dose PET Reconstruction using Deep Learning
Authors: Junshen Xu, Enhao Gong, John Pauly, Greg Zaharchuk
Date: 2017-12-12 00:00:00
Tasks: Decoder, Deep Learning, Image Reconstruction

Positron emission tomography (PET) is widely used in various clinical
applications, including cancer diagnosis, heart disease and neuro disorders.
The use of radioactive tracer in PET imaging raises concerns due to the risk of
radiation exposure. To minimize this potential risk in PET imaging, efforts
have been made to reduce the amount of radio-tracer usage. However, lowing dose
results in low Signal-to-Noise-Ratio (SNR) and loss of information, both of
which will heavily affect clinical diagnosis. Besides, the ill-conditioning of
low-dose PET image reconstruction makes it a difficult problem for iterative
reconstruction algorithms. Previous methods proposed are typically complicated
and slow, yet still cannot yield satisfactory results at significantly low
dose. Here, we propose a deep learning method to resolve this issue with an
encoder-decoder residual deep network with concatenate skip connections.
Experiments shows the proposed method can reconstruct low-dose PET image to a
standard-dose quality with only two-hundredth dose. Different cost functions
for training model are explored. Multi-slice input strategy is introduced to
provide the network with more structural information and make it more robust to
noise. Evaluation on ultra-low-dose clinical data shows that the proposed
method can achieve better result than the state-of-the-art methods and
reconstruct images with comparable quality using only 0.5% of the original
regular dose.
Neural Component Analysis for Fault Detection
Authors: Haitao Zhao
Date: 2017-12-12 00:00:00
Tasks: Chemical Process, Fault Detection

Principal component analysis (PCA) is largely adopted for chemical process
monitoring and numerous PCA-based systems have been developed to solve various
fault detection and diagnosis problems. Since PCA-based methods assume that the
monitored process is linear, nonlinear PCA models, such as autoencoder models
and kernel principal component analysis (KPCA), has been proposed and applied
to nonlinear process monitoring. However, KPCA-based methods need to perform
eigen-decomposition (ED) on the kernel Gram matrix whose dimensions depend on
the number of training data. Moreover, prefixed kernel parameters cannot be
most effective for different faults which may need different parameters to
maximize their respective detection performances. Autoencoder models lack the
consideration of orthogonal constraints which is crucial for PCA-based
algorithms. To address these problems, this paper proposes a novel nonlinear
method, called neural component analysis (NCA), which intends to train a
feedforward neural work with orthogonal constraints such as those used in PCA.
NCA can adaptively learn its parameters through backpropagation and the
dimensionality of the nonlinear features has no relationship with the number of
training samples. Extensive experimental results on the Tennessee Eastman (TE)
benchmark process show the superiority of NCA in terms of missed detection rate
(MDR) and false alarm rate (FAR). The source code of NCA can be found in
https://github.com/haitaozhao/Neural-Component-Analysis.git.