pwc-archive/papers-with-abstracts huggingface dataset:
Dynamic Feature Acquisition Using Denoising Autoencoders
Authors: Mohammad Kachuee, Sajad Darabi, Babak Moatamed, Majid Sarrafzadeh
Date: 2018-11-03 00:00:00
Tasks: Denoising, Density Estimation, Informativeness, Sensitivity

In real-world scenarios, different features have different acquisition costs
at test-time which necessitates cost-aware methods to optimize the cost and
performance trade-off. This paper introduces a novel and scalable approach for
cost-aware feature acquisition at test-time. The method incrementally asks for
features based on the available context that are known feature values. The
proposed method is based on sensitivity analysis in neural networks and density
estimation using denoising autoencoders with binary representation layers. In
the proposed architecture, a denoising autoencoder is used to handle unknown
features (i.e., features that are yet to be acquired), and the sensitivity of
predictions with respect to each unknown feature is used as a context-dependent
measure of informativeness. We evaluated the proposed method on eight different
real-world datasets as well as one synthesized dataset and compared its
performance with several other approaches in the literature. According to the
results, the suggested method is capable of efficiently acquiring features at
test-time in a cost- and context-aware fashion.
Stochastic Neighbor Embedding under f-divergences
Authors: Daniel Jiwoong Im, Nakul Verma, Kristin Branson
Date: 2018-11-03 00:00:00

The t-distributed Stochastic Neighbor Embedding (t-SNE) is a powerful and
popular method for visualizing high-dimensional data. It minimizes the
Kullback-Leibler (KL) divergence between the original and embedded data
distributions. In this work, we propose extending this method to other
f-divergences. We analytically and empirically evaluate the types of latent
structure-manifold, cluster, and hierarchical-that are well-captured using both
the original KL-divergence as well as the proposed f-divergence generalization,
and find that different divergences perform better for different types of
structure.
  A common concern with $t$-SNE criterion is that it is optimized using
gradient descent, and can become stuck in poor local minima. We propose
optimizing the f-divergence based loss criteria by minimizing a variational
bound. This typically performs better than optimizing the primal form, and our
experiments show that it can improve upon the embedding results obtained from
the original $t$-SNE criterion as well.
Wizard of Wikipedia: Knowledge-Powered Conversational agents
Authors: Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, Jason Weston
Date: 2018-11-03 00:00:00
Proceeding: ICLR 2019 5
Tasks: Dialogue Generation

In open-domain dialogue intelligent agents should exhibit the use of
knowledge, however there are few convincing demonstrations of this to date. The
most popular sequence to sequence models typically "generate and hope" generic
utterances that can be memorized in the weights of the model when mapping from
input utterance(s) to output, rather than employing recalled knowledge as
context. Use of knowledge has so far proved difficult, in part because of the
lack of a supervised learning benchmark task which exhibits knowledgeable open
dialogue with clear grounding. To that end we collect and release a large
dataset with conversations directly grounded with knowledge retrieved from
Wikipedia. We then design architectures capable of retrieving knowledge,
reading and conditioning on it, and finally generating natural responses. Our
best performing dialogue models are able to conduct knowledgeable discussions
on open-domain topics as evaluated by automatic metrics and human evaluations,
while our new benchmark allows for measuring further improvements in this
important research direction.
Deep Learning based Computer-Aided Diagnosis Systems for Diabetic Retinopathy: A Survey
Authors: Norah Asiri, Muhammad Hussain, Fadwa Al Adel, Nazih Alzaidi
Date: 2018-11-03 00:00:00
Tasks: Deep Learning, Medical Image Analysis

Diabetic retinopathy (DR) results in vision loss if not treated early. A computer-aided diagnosis (CAD) system based on retinal fundus images is an efficient and effective method for early DR diagnosis and assisting experts. A computer-aided diagnosis (CAD) system involves various stages like detection, segmentation and classification of lesions in fundus images. Many traditional machine-learning (ML) techniques based on hand-engineered features have been introduced. The recent emergence of deep learning (DL) and its decisive victory over traditional ML methods for various applications motivated the researchers to employ it for DR diagnosis, and many deep-learning-based methods have been introduced. In this paper, we review these methods, highlighting their pros and cons. In addition, we point out the challenges to be addressed in designing and learning about efficient, effective and robust deep-learning algorithms for various problems in DR diagnosis and draw attention to directions for future research.
Relation Mention Extraction from Noisy Data with Hierarchical Reinforcement Learning
Authors: Jun Feng, Minlie Huang, Yijie Zhang, Yang Yang, Xiaoyan Zhu
Date: 2018-11-03 00:00:00
Tasks: Denoising, Hierarchical Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Relation, Relation Mention Extraction, Sentence

In this paper we address a task of relation mention extraction from noisy
data: extracting representative phrases for a particular relation from noisy
sentences that are collected via distant supervision. Despite its significance
and value in many downstream applications, this task is less studied on noisy
data. The major challenges exists in 1) the lack of annotation on mention
phrases, and more severely, 2) handling noisy sentences which do not express a
relation at all. To address the two challenges, we formulate the task as a
semi-Markov decision process and propose a novel hierarchical reinforcement
learning model. Our model consists of a top-level sentence selector to remove
noisy sentences, a low-level mention extractor to extract relation mentions,
and a reward estimator to provide signals to guide data denoising and mention
extraction without explicit annotations. Experimental results show that our
model is effective to extract relation mentions from noisy data.
CAAD 2018: Powerful None-Access Black-Box Attack Based on Adversarial Transformation Network
Authors: Xiaoyi Dong, Weiming Zhang, Nenghai Yu
Date: 2018-11-03 00:00:00

In this paper, we propose an improvement of Adversarial Transformation
Networks(ATN) to generate adversarial examples, which can fool white-box models
and black-box models with a state of the art performance and won the 2rd place
in the non-target task in CAAD 2018.
Sharp worst-case evaluation complexity bounds for arbitrary-order nonconvex optimization with inexpensive constraints
Authors: Coralia Cartis, Nick I. M. Gould, Philippe L. Toint
Date: 2018-11-03 00:00:00

We provide sharp worst-case evaluation complexity bounds for nonconvex
minimization problems with general inexpensive constraints, i.e.\ problems
where the cost of evaluating/enforcing of the (possibly nonconvex or even
disconnected) constraints, if any, is negligible compared to that of evaluating
the objective function. These bounds unify, extend or improve all known upper
and lower complexity bounds for unconstrained and convexly-constrained
problems. It is shown that, given an accuracy level $\epsilon$, a degree of
highest available Lipschitz continuous derivatives $p$ and a desired optimality
order $q$ between one and $p$, a conceptual regularization algorithm requires
no more than $O(\epsilon^{-\frac{p+1}{p-q+1}})$ evaluations of the objective
function and its derivatives to compute a suitably approximate $q$-th order
minimizer. With an appropriate choice of the regularization, a similar result
also holds if the $p$-th derivative is merely H\"older rather than Lipschitz
continuous. We provide an example that shows that the above complexity bound is
sharp for unconstrained and a wide class of constrained problems, we also give
reasons for the optimality of regularization methods from a worst-case
complexity point of view, within a large class of algorithms that use the same
derivative information.
Learning sparse mixtures of rankings from noisy information
Authors: Anindya De, Ryan O'Donnell, Rocco Servedio
Date: 2018-11-03 00:00:00

We study the problem of learning an unknown mixture of $k$ rankings over $n$
elements, given access to noisy samples drawn from the unknown mixture. We
consider a range of different noise models, including natural variants of the
"heat kernel" noise framework and the Mallows model. For each of these noise
models we give an algorithm which, under mild assumptions, learns the unknown
mixture to high accuracy and runs in $n^{O(\log k)}$ time. The best previous
algorithms for closely related problems have running times which are
exponential in $k$.
Learning to Defend by Learning to Attack
Authors: Haoming Jiang, Zhehui Chen, Yuyang Shi, Bo Dai, Tuo Zhao
Date: 2018-11-03 00:00:00
Tasks: Adversarial Attack, Adversarial Defense, Bilevel Optimization, Computational Efficiency, Imitation Learning

Adversarial training provides a principled approach for training robust neural networks. From an optimization perspective, adversarial training is essentially solving a bilevel optimization problem. The leader problem is trying to learn a robust classifier, while the follower problem is trying to generate adversarial samples. Unfortunately, such a bilevel problem is difficult to solve due to its highly complicated structure. This work proposes a new adversarial training method based on a generic learning-to-learn (L2L) framework. Specifically, instead of applying existing hand-designed algorithms for the inner problem, we learn an optimizer, which is parametrized as a convolutional neural network. At the same time, a robust classifier is learned to defense the adversarial attack generated by the learned optimizer. Experiments over CIFAR-10 and CIFAR-100 datasets demonstrate that L2L outperforms existing adversarial training methods in both classification accuracy and computational efficiency. Moreover, our L2L framework can be extended to generative adversarial imitation learning and stabilize the training.
DUNet: A deformable network for retinal vessel segmentation
Authors: Qiangguo Jin, Zhaopeng Meng, Tuan D. Pham, Qi Chen, Leyi Wei, Ran Su
Date: 2018-11-03 00:00:00
Tasks: Retinal Vessel Segmentation, Segmentation

Automatic segmentation of retinal vessels in fundus images plays an important
role in the diagnosis of some diseases such as diabetes and hypertension. In
this paper, we propose Deformable U-Net (DUNet), which exploits the retinal
vessels' local features with a U-shape architecture, in an end to end manner
for retinal vessel segmentation. Inspired by the recently introduced deformable
convolutional networks, we integrate the deformable convolution into the
proposed network. The DUNet, with upsampling operators to increase the output
resolution, is designed to extract context information and enable precise
localization by combining low-level feature maps with high-level ones.
Furthermore, DUNet captures the retinal vessels at various shapes and scales by
adaptively adjusting the receptive fields according to vessels' scales and
shapes. Three public datasets DRIVE, STARE and CHASE_DB1 are used to train and
test our model. Detailed comparisons between the proposed network and the
deformable neural network, U-Net are provided in our study. Results show that
more detailed vessels are extracted by DUNet and it exhibits state-of-the-art
performance for retinal vessel segmentation with a global accuracy of
0.9697/0.9722/0.9724 and AUC of 0.9856/0.9868/0.9863 on DRIVE, STARE and
CHASE_DB1 respectively. Moreover, to show the generalization ability of the
DUNet, we used another two retinal vessel data sets, one is named WIDE and the
other is a synthetic data set with diverse styles, named SYNTHE, to
qualitatively and quantitatively analyzed and compared with other methods.
Results indicates that DUNet outperforms other state-of-the-arts.
A biconvex optimization for solving semidefinite programs via bilinear factorization
Authors: En-Liang Hu
Date: 2018-11-03 00:00:00

Many problems in machine learning can be reduced to learning a low-rank positive semidefinite matrix (denoted as $Z$), which encounters semidefinite program (SDP). Existing SDP solvers by classical convex optimization are expensive to solve large-scale problems. Employing the low rank of solution, Burer-Monteiro's method reformulated SDP as a nonconvex problem via the $quadratic$ factorization ($Z$ as $XX^\top$). However, this would lose the structure of problem in optimization. In this paper, we propose to convert SDP into a biconvex problem via the $bilinear$ factorization ($Z$ as $XY^\top$), and while adding the term $\frac{\gamma}{2}||X-Y||_F^2$ to penalize the difference of $X$ and $Y$. Thus, the biconvex structure (w.r.t. $X$ and $Y$) can be exploited naturally in optimization. As a theoretical result, we provide a bound to the penalty parameter $\gamma$ under the assumption of $L$-Lipschitz smoothness and $\sigma $-strongly biconvexity, such that, at stationary points, the proposed bilinear factorization is equivalent to Burer-Monteiro's factorization when the bound is arrived, that is $\gamma>\frac{1}{4}(L-\sigma)_+$. Our proposal opens up a new way to surrogate SDP by biconvex program. Experiments on two SDP-related applications demonstrate that the proposed method is effective as the state-of-the-art.
Pushing the boundaries of audiovisual word recognition using Residual Networks and LSTMs
Authors: Themos Stafylakis, Muhammad Haris Khan, Georgios Tzimiropoulos
Date: 2018-11-03 00:00:00
Tasks: Lipreading, speech-recognition, Speech Recognition

Visual and audiovisual speech recognition are witnessing a renaissance which
is largely due to the advent of deep learning methods. In this paper, we
present a deep learning architecture for lipreading and audiovisual word
recognition, which combines Residual Networks equipped with spatiotemporal
input layers and Bidirectional LSTMs. The lipreading architecture attains
11.92% misclassification rate on the challenging Lipreading-In-The-Wild
database, which is composed of excerpts from BBC-TV, each containing one of the
500 target words. Audiovisual experiments are performed using both intermediate
and late integration, as well as several types and levels of environmental
noise, and notable improvements over the audio-only network are reported, even
in the case of clean speech. A further analysis on the utility of target word
boundaries is provided, as well as on the capacity of the network in modeling
the linguistic context of the target word. Finally, we examine difficult word
pairs and discuss how visual information helps towards attaining higher
recognition accuracy.
Unsupervised Identification of Study Descriptors in Toxicology Research: An Experimental Study
Authors: Drahomira Herrmannova, Steven R. Young, Robert M. Patton, Christopher G. Stahl, Nicole C. Kleinstreuer, Mary S. Wolfe
Date: 2018-11-03 00:00:00
Proceeding: WS 2018 10

Identifying and extracting data elements such as study descriptors in
publication full texts is a critical yet manual and labor-intensive step
required in a number of tasks. In this paper we address the question of
identifying data elements in an unsupervised manner. Specifically, provided a
set of criteria describing specific study parameters, such as species, route of
administration, and dosing regimen, we develop an unsupervised approach to
identify text segments (sentences) relevant to the criteria. A binary
classifier trained to identify publications that met the criteria performs
better when trained on the candidate sentences than when trained on sentences
randomly picked from the text, supporting the intuition that our method is able
to accurately identify study descriptors.
Stochastic Primal-Dual Method for Empirical Risk Minimization with $\mathcal{O}(1)$ Per-Iteration Complexity
Authors: Conghui Tan, Tong Zhang, Shiqian Ma, Ji Liu
Date: 2018-11-03 00:00:00
Proceeding: NeurIPS 2018
Tasks: BIG-bench Machine Learning

Regularized empirical risk minimization problem with linear predictor appears
frequently in machine learning. In this paper, we propose a new stochastic
primal-dual method to solve this class of problems. Different from existing
methods, our proposed methods only require O(1) operations in each iteration.
We also develop a variance-reduction variant of the algorithm that converges
linearly. Numerical experiments suggest that our methods are faster than
existing ones such as proximal SGD, SVRG and SAGA on high-dimensional problems.
Large-scale Heteroscedastic Regression via Gaussian Process
Authors: Haitao Liu, Yew-Soon Ong, Jianfei Cai
Date: 2018-11-03 00:00:00
Tasks: regression, Variational Inference

Heteroscedastic regression considering the varying noises among observations has many applications in the fields like machine learning and statistics. Here we focus on the heteroscedastic Gaussian process (HGP) regression which integrates the latent function and the noise function together in a unified non-parametric Bayesian framework. Though showing remarkable performance, HGP suffers from the cubic time complexity, which strictly limits its application to big data. To improve the scalability, we first develop a variational sparse inference algorithm, named VSHGP, to handle large-scale datasets. Furthermore, two variants are developed to improve the scalability and capability of VSHGP. The first is stochastic VSHGP (SVSHGP) which derives a factorized evidence lower bound, thus enhancing efficient stochastic variational inference. The second is distributed VSHGP (DVSHGP) which (i) follows the Bayesian committee machine formalism to distribute computations over multiple local VSHGP experts with many inducing points; and (ii) adopts hybrid parameters for experts to guard against over-fitting and capture local variety. The superiority of DVSHGP and SVSHGP as compared to existing scalable heteroscedastic/homoscedastic GPs is then extensively verified on various datasets.
Nonparallel Emotional Speech Conversion
Authors: Jian Gao, Deep Chakraborty, Hamidou Tembine, Olaitan Olaleye
Date: 2018-11-03 00:00:00
Tasks: Decoder, Style Transfer, Translation

We propose a nonparallel data-driven emotional speech conversion method. It enables the transfer of emotion-related characteristics of a speech signal while preserving the speaker's identity and linguistic content. Most existing approaches require parallel data and time alignment, which is not available in most real applications. We achieve nonparallel training based on an unsupervised style transfer technique, which learns a translation model between two distributions instead of a deterministic one-to-one mapping between paired examples. The conversion model consists of an encoder and a decoder for each emotion domain. We assume that the speech signal can be decomposed into an emotion-invariant content code and an emotion-related style code in latent space. Emotion conversion is performed by extracting and recombining the content code of the source speech and the style code of the target emotion. We tested our method on a nonparallel corpora with four emotions. Both subjective and objective evaluations show the effectiveness of our approach.
Radius-margin bounds for deep neural networks
Authors: Mayank Sharma, Jayadeva, Sumit Soman
Date: 2018-11-03 00:00:00

Explaining the unreasonable effectiveness of deep learning has eluded
researchers around the globe. Various authors have described multiple metrics
to evaluate the capacity of deep architectures. In this paper, we allude to the
radius margin bounds described for a support vector machine (SVM) with hinge
loss, apply the same to the deep feed-forward architectures and derive the
Vapnik-Chervonenkis (VC) bounds which are different from the earlier bounds
proposed in terms of number of weights of the network. In doing so, we also
relate the effectiveness of techniques like Dropout and Dropconnect in bringing
down the capacity of the network. Finally, we describe the effect of maximizing
the input as well as the output margin to achieve an input noise-robust deep
architecture.
Convergence of the Deep BSDE Method for Coupled FBSDEs
Authors: Jiequn Han, Jihao Long
Date: 2018-11-03 00:00:00

The recently proposed numerical algorithm, deep BSDE method, has shown remarkable performance in solving high-dimensional forward-backward stochastic differential equations (FBSDEs) and parabolic partial differential equations (PDEs). This article lays a theoretical foundation for the deep BSDE method in the general case of coupled FBSDEs. In particular, a posteriori error estimation of the solution is provided and it is proved that the error converges to zero given the universal approximation capability of neural networks. Numerical results are presented to demonstrate the accuracy of the analyzed algorithm in solving high-dimensional coupled FBSDEs.
Understanding and Comparing Scalable Gaussian Process Regression for Big Data
Authors: Haitao Liu, Jianfei Cai, Yew-Soon Ong, Yi Wang
Date: 2018-11-03 00:00:00
Tasks: regression

As a non-parametric Bayesian model which produces informative predictive
distribution, Gaussian process (GP) has been widely used in various fields,
like regression, classification and optimization. The cubic complexity of
standard GP however leads to poor scalability, which poses challenges in the
era of big data. Hence, various scalable GPs have been developed in the
literature in order to improve the scalability while retaining desirable
prediction accuracy. This paper devotes to investigating the methodological
characteristics and performance of representative global and local scalable GPs
including sparse approximations and local aggregations from four main
perspectives: scalability, capability, controllability and robustness. The
numerical experiments on two toy examples and five real-world datasets with up
to 250K points offer the following findings. In terms of scalability, most of
the scalable GPs own a time complexity that is linear to the training size. In
terms of capability, the sparse approximations capture the long-term spatial
correlations, the local aggregations capture the local patterns but suffer from
over-fitting in some scenarios. In terms of controllability, we could improve
the performance of sparse approximations by simply increasing the inducing
size. But this is not the case for local aggregations. In terms of robustness,
local aggregations are robust to various initializations of hyperparameters due
to the local attention mechanism. Finally, we highlight that the proper hybrid
of global and local scalable GPs may be a promising way to improve both the
model capability and scalability for big data.
Boosted Sparse and Low-Rank Tensor Regression
Authors: Lifang He, Kun Chen, Wanwan Xu, Jiayu Zhou, Fei Wang
Date: 2018-11-03 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: regression

We propose a sparse and low-rank tensor regression model to relate a
univariate outcome to a feature tensor, in which each unit-rank tensor from the
CP decomposition of the coefficient tensor is assumed to be sparse. This
structure is both parsimonious and highly interpretable, as it implies that the
outcome is related to the features through a few distinct pathways, each of
which may only involve subsets of feature dimensions. We take a
divide-and-conquer strategy to simplify the task into a set of sparse unit-rank
tensor regression problems. To make the computation efficient and scalable, for
the unit-rank tensor regression, we propose a stagewise estimation procedure to
efficiently trace out its entire solution path. We show that as the step size
goes to zero, the stagewise solution paths converge exactly to those of the
corresponding regularized regression. The superior performance of our approach
is demonstrated on various real-world and synthetic examples.