pwc-archive/papers-with-abstracts huggingface dataset:
Fast High-Dimensional Kernel Filtering
Authors: Pravin Nair, Kunal. N. Chaudhury
Date: 2019-01-18 00:00:00
Tasks: Vocal Bursts Intensity Prediction

The bilateral and nonlocal means filters are instances of kernel-based
filters that are popularly used in image processing. It was recently shown that
fast and accurate bilateral filtering of grayscale images can be performed
using a low-rank approximation of the kernel matrix. More specifically, based
on the eigendecomposition of the kernel matrix, the overall filtering was
approximated using spatial convolutions, for which efficient algorithms are
available. Unfortunately, this technique cannot be scaled to high-dimensional
data such as color and hyperspectral images. This is simply because one needs
to compute/store a large matrix and perform its eigendecomposition in this
case. We show how this problem can be solved using the Nystr\"om method, which
is generally used for approximating the eigendecomposition of large matrices.
The resulting algorithm can also be used for nonlocal means filtering. We
demonstrate the effectiveness of our proposal for bilateral and nonlocal means
filtering of color and hyperspectral images. In particular, our method is shown
to be competitive with state-of-the-art fast algorithms, and moreover it comes
with a theoretical guarantee on the approximation error.
CRDN: Cascaded Residual Dense Networks for Dynamic MR Imaging with Edge-enhanced Loss Constraint
Authors: Ziwen Ke, Shan-Shan Wang, Huitao Cheng, Leslie Ying, Qiegen Liu, Hairong Zheng, Dong Liang
Date: 2019-01-18 00:00:00
Tasks: compressed sensing

Dynamic magnetic resonance (MR) imaging has generated great research
interest, as it can provide both spatial and temporal information for clinical
diagnosis. However, slow imaging speed or long scanning time is still one of
the challenges for dynamic MR imaging. Most existing methods reconstruct
Dynamic MR images from incomplete k-space data under the guidance of compressed
sensing (CS) or low rank theory, which suffer from long iterative
reconstruction time. Recently, deep learning has shown great potential in
accelerating dynamic MR. Our previous work proposed a dynamic MR imaging method
with both k-space and spatial prior knowledge integrated via multi-supervised
network training. Nevertheless, there was still a certain degree of smooth in
the reconstructed images at high acceleration factors. In this work, we propose
cascaded residual dense networks for dynamic MR imaging with edge-enhance loss
constraint, dubbed as CRDN. Specifically, the cascaded residual dense networks
fully exploit the hierarchical features from all the convolutional layers with
both local and global feature fusion. We further utilize the total variation
(TV) loss function, which has the edge enhancement properties, for training the
networks.
Linearized ADMM and Fast Nonlocal Denoising for Efficient Plug-and-Play Restoration
Authors: Unni V. S., Sanjay Ghosh, Kunal. N. Chaudhury
Date: 2019-01-18 00:00:00
Tasks: Denoising, Image Restoration, Super-Resolution

In plug-and-play image restoration, the regularization is performed using
powerful denoisers such as nonlocal means (NLM) or BM3D. This is done within
the framework of alternating direction method of multipliers (ADMM), where the
regularization step is formally replaced by an off-the-shelf denoiser. Each
plug-and-play iteration involves the inversion of the forward model followed by
a denoising step. In this paper, we present a couple of ideas for improving the
efficiency of the inversion and denoising steps. First, we propose to use
linearized ADMM, which generally allows us to perform the inversion at a lower
cost than standard ADMM. Moreover, we can easily incorporate hard constraints
into the optimization framework as a result. Second, we develop a fast
algorithm for doubly stochastic NLM, originally proposed by Sreehari et al.
(IEEE TCI, 2016), which is about 80x faster than brute-force computation. This
particular denoiser can be expressed as the proximal map of a convex
regularizer and, as a consequence, we can guarantee convergence for linearized
plug-and-play ADMM. We demonstrate the effectiveness of our proposals for
super-resolution and single-photon imaging.
Exploring Semi-supervised Variational Autoencoders for Biomedical Relation Extraction
Authors: Yijia Zhang, Zhiyong Lu
Date: 2019-01-18 00:00:00
Tasks: Decoder, Relation, Relation Extraction

The biomedical literature provides a rich source of knowledge such as
protein-protein interactions (PPIs), drug-drug interactions (DDIs) and
chemical-protein interactions (CPIs). Biomedical relation extraction aims to
automatically extract biomedical relations from biomedical text for various
biomedical research. State-of-the-art methods for biomedical relation
extraction are primarily based on supervised machine learning and therefore
depend on (sufficient) labeled data. However, creating large sets of training
data is prohibitively expensive and labor-intensive, especially so in
biomedicine as domain knowledge is required. In contrast, there is a large
amount of unlabeled biomedical text available in PubMed. Hence, computational
methods capable of employing unlabeled data to reduce the burden of manual
annotation are of particular interest in biomedical relation extraction. We
present a novel semi-supervised approach based on variational autoencoder (VAE)
for biomedical relation extraction. Our model consists of the following three
parts, a classifier, an encoder and a decoder. The classifier is implemented
using multi-layer convolutional neural networks (CNNs), and the encoder and
decoder are implemented using both bidirectional long short-term memory
networks (Bi-LSTMs) and CNNs, respectively. The semi-supervised mechanism
allows our model to learn features from both the labeled and unlabeled data. We
evaluate our method on multiple public PPI, DDI and CPI corpora. Experimental
results show that our method effectively exploits the unlabeled data to improve
the performance and reduce the dependence on labeled data. To our best
knowledge, this is the first semi-supervised VAE-based method for (biomedical)
relation extraction. Our results suggest that exploiting such unlabeled data
can be greatly beneficial to improved performance in various biomedical
relation extraction.
Transfer Learning and Meta Classification Based Deep Churn Prediction System for Telecom Industry
Authors: Uzair Ahmed, Asifullah Khan, Saddam Hussain Khan, Abdul Basit, Irfan Ul Haq, Yeon Soo Lee
Date: 2019-01-18 00:00:00
Tasks: General Classification, Prediction, Transfer Learning

A churn prediction system guides telecom service providers to reduce revenue
loss. However, the development of a churn prediction system for a telecom
industry is a challenging task, mainly due to the large size of the data, high
dimensional features, and imbalanced distribution of the data. In this paper,
we present a solution to the inherent problems of churn prediction, using the
concept of Transfer Learning (TL) and Ensemble-based Meta-Classification. The
proposed method TL-DeepE is applied in two stages. The first stage employs TL
by fine-tuning multiple pre-trained Deep Convolution Neural Networks (CNNs).
Telecom datasets are normally in vector form, which is converted into 2D images
because Deep CNNs have high learning capacity on images. In the second stage,
predictions from these Deep CNNs are appended to the original feature vector
and thus are used to build a final feature vector for the high-level Genetic
Programming (GP) and AdaBoost based ensemble classifier. Thus, the experiments
are conducted using various CNNs as base classifiers and the GP-AdaBoost as a
meta-classifier. By using 10-fold cross-validation, the performance of the
proposed TL-DeepE system is compared with existing techniques, for two standard
telecommunication datasets; Orange and Cell2cell. Performing experiments on
Orange and Cell2cell datasets, the prediction accuracy obtained was 75.4% and
68.2%, while the area under the curve was 0.83 and 0.74, respectively.
WALL-E: An Efficient Reinforcement Learning Research Framework
Authors: Tianbing Xu, Andrew Zhang, Liang Zhao
Date: 2019-01-18 00:00:00
Tasks: MuJoCo, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

There are two halves to RL systems: experience collection time and policy
learning time. For a large number of samples in rollouts, experience collection
time is the major bottleneck. Thus, it is necessary to speed up the rollout
generation time with multi-process architecture support. Our work, dubbed
WALL-E, utilizes multiple rollout samplers running in parallel to rapidly
generate experience. Due to our parallel samplers, we experience not only
faster convergence times, but also higher average reward thresholds. For
example, on the MuJoCo HalfCheetah-v2 task, with $N = 10$ parallel sampler
processes, we are able to achieve much higher average return than those from
using only a single process architecture.
Theory of Minds: Understanding Behavior in Groups Through Inverse Planning
Authors: Michael Shum, Max Kleiman-Weiner, Michael L. Littman, Joshua B. Tenenbaum
Date: 2019-01-18 00:00:00
Tasks: Action Understanding, Bayesian Inference, Multi-agent Reinforcement Learning, Reinforcement Learning

Human social behavior is structured by relationships. We form teams, groups,
tribes, and alliances at all scales of human life. These structures guide
multi-agent cooperation and competition, but when we observe others these
underlying relationships are typically unobservable and hence must be inferred.
Humans make these inferences intuitively and flexibly, often making rapid
generalizations about the latent relationships that underlie behavior from just
sparse and noisy observations. Rapid and accurate inferences are important for
determining who to cooperate with, who to compete with, and how to cooperate in
order to compete. Towards the goal of building machine-learning algorithms with
human-like social intelligence, we develop a generative model of multi-agent
action understanding based on a novel representation for these latent
relationships called Composable Team Hierarchies (CTH). This representation is
grounded in the formalism of stochastic games and multi-agent reinforcement
learning. We use CTH as a target for Bayesian inference yielding a new
algorithm for understanding behavior in groups that can both infer hidden
relationships as well as predict future actions for multiple agents interacting
together. Our algorithm rapidly recovers an underlying causal model of how
agents relate in spatial stochastic games from just a few observations. The
patterns of inference made by this algorithm closely correspond with human
judgments and the algorithm makes the same rapid generalizations that people
do.
Probabilistic symmetries and invariant neural networks
Authors: Benjamin Bloem-Reddy, Yee Whye Teh
Date: 2019-01-18 00:00:00

Treating neural network inputs and outputs as random variables, we characterize the structure of neural networks that can be used to model data that are invariant or equivariant under the action of a compact group. Much recent research has been devoted to encoding invariance under symmetry transformations into neural network architectures, in an effort to improve the performance of deep neural networks in data-scarce, non-i.i.d., or unsupervised settings. By considering group invariance from the perspective of probabilistic symmetry, we establish a link between functional and probabilistic symmetry, and obtain generative functional representations of probability distributions that are invariant or equivariant under the action of a compact group. Our representations completely characterize the structure of neural networks that can be used to model such distributions and yield a general program for constructing invariant stochastic or deterministic neural networks. We demonstrate that examples from the recent literature are special cases, and develop the details of the general program for exchangeable sequences and arrays.
DeepOtsu: Document Enhancement and Binarization using Iterative Deep Learning
Authors: Sheng He, Lambert Schomaker
Date: 2019-01-18 00:00:00
Tasks: Binarization, Deep Learning, Document Enhancement

This paper presents a novel iterative deep learning framework and apply it
for document enhancement and binarization. Unlike the traditional methods which
predict the binary label of each pixel on the input image, we train the neural
network to learn the degradations in document images and produce the uniform
images of the degraded input images, which allows the network to refine the
output iteratively. Two different iterative methods have been studied in this
paper: recurrent refinement (RR) which uses the same trained neural network in
each iteration for document enhancement and stacked refinement (SR) which uses
a stack of different neural networks for iterative output refinement. Given the
learned uniform and enhanced image, the binarization map can be easy to obtain
by a global or local threshold. The experimental results on several public
benchmark data sets show that our proposed methods provide a new clean version
of the degraded image which is suitable for visualization and promising results
of binarization using the global Otsu's threshold based on the enhanced images
learned iteratively by the neural network.
Accelerated Experimental Design for Pairwise Comparisons
Authors: Yuan Guo, Jennifer Dy, Deniz Erdogmus, Jayashree Kalpathy-Cramer, Susan Ostmo, J. Peter Campbell, Michael F. Chiang, Stratis Ioannidis
Date: 2019-01-18 00:00:00
Tasks: 2k, Experimental Design

Pairwise comparison labels are more informative and less variable than class
labels, but generating them poses a challenge: their number grows quadratically
in the dataset size. We study a natural experimental design objective, namely,
D-optimality, that can be used to identify which $K$ pairwise comparisons to
generate. This objective is known to perform well in practice, and is
submodular, making the selection approximable via the greedy algorithm. A
na\"ive greedy implementation has $O(N^2d^2K)$ complexity, where $N$ is the
dataset size, $d$ is the feature space dimension, and $K$ is the number of
generated comparisons. We show that, by exploiting the inherent geometry of the
dataset--namely, that it consists of pairwise comparisons--the greedy
algorithm's complexity can be reduced to $O(N^2(K+d)+N(dK+d^2) +d^2K).$ We
apply the same acceleration also to the so-called lazy greedy algorithm. When
combined, the above improvements lead to an execution time of less than 1 hour
for a dataset with $10^8$ comparisons; the na\"ive greedy algorithm on the same
dataset would require more than 10 days to terminate.
Chinese Word Segmentation: Another Decade Review (2007-2017)
Authors: Hai Zhao, Deng Cai, Changning Huang, Chunyu Kit
Date: 2019-01-18 00:00:00
Tasks: Chinese Word Segmentation

This paper reviews the development of Chinese word segmentation (CWS) in the
most recent decade, 2007-2017. Special attention was paid to the deep learning
technologies that has already permeated into most areas of natural language
processing (NLP). The basic view we have arrived at is that compared to
traditional supervised learning methods, neural network based methods have not
shown any superior performance. The most critical challenge still lies on
balancing of recognition of in-vocabulary (IV) and out-of-vocabulary (OOV)
words. However, as neural models have potentials to capture the essential
linguistic structure of natural language, we are optimistic about significant
progresses may arrive in the near future.
Kernel Change-point Detection with Auxiliary Deep Generative Models
Authors: Wei-Cheng Chang, Chun-Liang Li, Yiming Yang, Barnabás Póczos
Date: 2019-01-18 00:00:00
Proceeding: ICLR 2019 5
Tasks: Change Point Detection, Time Series, Time Series Analysis

Detecting the emergence of abrupt property changes in time series is a
challenging problem. Kernel two-sample test has been studied for this task
which makes fewer assumptions on the distributions than traditional parametric
approaches. However, selecting kernels is non-trivial in practice. Although
kernel selection for two-sample test has been studied, the insufficient samples
in change point detection problem hinder the success of those developed kernel
selection algorithms. In this paper, we propose KL-CPD, a novel kernel learning
framework for time series CPD that optimizes a lower bound of test power via an
auxiliary generative model. With deep kernel parameterization, KL-CPD endows
kernel two-sample test with the data-driven kernel to detect different types of
change-points in real-world applications. The proposed approach significantly
outperformed other state-of-the-art methods in our comparative evaluation of
benchmark datasets and simulation studies.
Splitting Methods for Convex Bi-Clustering and Co-Clustering
Authors: Michael Weylandt
Date: 2019-01-18 00:00:00
Tasks: Clustering

Co-Clustering, the problem of simultaneously identifying clusters across multiple aspects of a data set, is a natural generalization of clustering to higher-order structured data. Recent convex formulations of bi-clustering and tensor co-clustering, which shrink estimated centroids together using a convex fusion penalty, allow for global optimality guarantees and precise theoretical analysis, but their computational properties have been less well studied. In this note, we present three efficient operator-splitting methods for the convex co-clustering problem: a standard two-block ADMM, a Generalized ADMM which avoids an expensive tensor Sylvester equation in the primal update, and a three-block ADMM based on the operator splitting scheme of Davis and Yin. Theoretical complexity analysis suggests, and experimental evidence confirms, that the Generalized ADMM is far more efficient for large problems.
Quasi-potential as an implicit regularizer for the loss function in the stochastic gradient descent
Authors: Wenqing Hu, Zhanxing Zhu, Haoyi Xiong, Jun Huan
Date: 2019-01-18 00:00:00
Tasks: Relation, Variational Inference

We interpret the variational inference of the Stochastic Gradient Descent
(SGD) as minimizing a new potential function named the
\textit{quasi-potential}. We analytically construct the quasi-potential
function in the case when the loss function is convex and admits only one
global minimum point. We show in this case that the quasi-potential function is
related to the noise covariance structure of SGD via a partial differential
equation of Hamilton-Jacobi type. This relation helps us to show that
anisotropic noise leads to faster escape than isotropic noise. We then consider
the dynamics of SGD in the case when the loss function is non-convex and admits
several different local minima. In this case, we demonstrate an example that
shows how the noise covariance structure plays a role in "implicit
regularization", a phenomenon in which SGD favors some particular local minimum
points. This is done through the relation between the noise covariance
structure and the quasi-potential function. Our analysis is based on Large
Deviations Theory (LDT), and they are validated by numerical experiments.
A Tail-Index Analysis of Stochastic Gradient Noise in Deep Neural Networks
Authors: Umut Simsekli, Levent Sagun, Mert Gurbuzbalaban
Date: 2019-01-18 00:00:00

The gradient noise (GN) in the stochastic gradient descent (SGD) algorithm is
often considered to be Gaussian in the large data regime by assuming that the
classical central limit theorem (CLT) kicks in. This assumption is often made
for mathematical convenience, since it enables SGD to be analyzed as a
stochastic differential equation (SDE) driven by a Brownian motion. We argue
that the Gaussianity assumption might fail to hold in deep learning settings
and hence render the Brownian motion-based analyses inappropriate. Inspired by
non-Gaussian natural phenomena, we consider the GN in a more general context
and invoke the generalized CLT (GCLT), which suggests that the GN converges to
a heavy-tailed $\alpha$-stable random variable. Accordingly, we propose to
analyze SGD as an SDE driven by a L\'{e}vy motion. Such SDEs can incur `jumps',
which force the SDE transition from narrow minima to wider minima, as proven by
existing metastability theory. To validate the $\alpha$-stable assumption, we
conduct extensive experiments on common deep learning architectures and show
that in all settings, the GN is highly non-Gaussian and admits heavy-tails. We
further investigate the tail behavior in varying network architectures and
sizes, loss functions, and datasets. Our results open up a different
perspective and shed more light on the belief that SGD prefers wide minima.
Learning Mutually Local-global U-nets For High-resolution Retinal Lesion Segmentation in Fundus Images
Authors: Zizheng Yan, Xiaoguang Han, Changmiao Wang, Yuda Qiu, Zixiang Xiong, Shuguang Cui
Date: 2019-01-18 00:00:00
Tasks: Decoder, Lesion Segmentation, Segmentation

Diabetic retinopathy is the most important complication of diabetes. Early
diagnosis of retinal lesions helps to avoid visual loss or blindness. Due to
high-resolution and small-size lesion regions, applying existing methods, such
as U-Nets, to perform segmentation on fundus photography is very challenging.
Although downsampling the input images could simplify the problem, it loses
detailed information. Conducting patch-level analysis helps reaching fine-scale
segmentation yet usually leads to misunderstanding as the lack of context
information. In this paper, we propose an efficient network that combines them
together, not only being aware of local details but also taking fully use of
the context perceptions. This is implemented by integrating the decoder parts
of a global-level U-net and a patch-level one. The two streams are jointly
optimized, ensuring that they are enhanced mutually. Experimental results
demonstrate our new framework significantly outperforms existing patch-based
and global-based methods, especially when the lesion regions are scattered and
small-scaled.
Good Similar Patches for Image Denoising
Authors: Si Lu
Date: 2019-01-18 00:00:00
Tasks: Clustering, Denoising, Image Denoising

Patch-based denoising algorithms like BM3D have achieved outstanding
performance. An important idea for the success of these methods is to exploit
the recurrence of similar patches in an input image to estimate the underlying
image structures. However, in these algorithms, the similar patches used for
denoising are obtained via Nearest Neighbour Search (NNS) and are sometimes not
optimal. First, due to the existence of noise, NNS can select similar patches
with similar noise patterns to the reference patch. Second, the unreliable
noisy pixels in digital images can bring a bias to the patch searching process
and result in a loss of color fidelity in the final denoising result. We
observe that given a set of good similar patches, their distribution is not
necessarily centered at the noisy reference patch and can be approximated by a
Gaussian component. Based on this observation, we present a patch searching
method that clusters similar patch candidates into patch groups using Gaussian
Mixture Model-based clustering, and selects the patch group that contains the
reference patch as the final patches for denoising. We also use an unreliable
pixel estimation algorithm to pre-process the input noisy images to further
improve the patch searching. Our experiments show that our approach can better
capture the underlying patch structures and can consistently enable the
state-of-the-art patch-based denoising algorithms, such as BM3D, LPCA and PLOW,
to better denoise images by providing them with patches found by our approach
while without modifying these algorithms.