pwc-archive/papers-with-abstracts huggingface dataset:
Zonotope hit-and-run for efficient sampling from projection DPPs
Authors: Guillaume Gautier, Rémi Bardenet, Michal Valko
Date: 2017-05-30 00:00:00
Proceeding: ICML 2017 8
Tasks: Diversity, Point Processes, Recommendation Systems

Determinantal point processes (DPPs) are distributions over sets of items
that model diversity using kernels. Their applications in machine learning
include summary extraction and recommendation systems. Yet, the cost of
sampling from a DPP is prohibitive in large-scale applications, which has
triggered an effort towards efficient approximate samplers. We build a novel
MCMC sampler that combines ideas from combinatorial geometry, linear
programming, and Monte Carlo methods to sample from DPPs with a fixed sample
cardinality, also called projection DPPs. Our sampler leverages the ability of
the hit-and-run MCMC kernel to efficiently move across convex bodies. Previous
theoretical results yield a fast mixing time of our chain when targeting a
distribution that is close to a projection DPP, but not a DPP in general. Our
empirical results demonstrate that this extends to sampling projection DPPs,
i.e., our sampler is more sample-efficient than previous approaches which in
turn translates to faster convergence when dealing with costly-to-evaluate
functions, such as summary extraction in our experiments.
By-passing the Kohn-Sham equations with machine learning
Authors: Felix Brockherde, Leslie Vogt, Li Li, Mark E. Tuckerman, Kieron Burke, Klaus-Robert Müller
Date: 2016-09-09 00:00:00
Tasks: BIG-bench Machine Learning

Last year, at least 30,000 scientific papers used the Kohn-Sham scheme of
density functional theory to solve electronic structure problems in a wide
variety of scientific fields, ranging from materials science to biochemistry to
astrophysics. Machine learning holds the promise of learning the kinetic energy
functional via examples, by-passing the need to solve the Kohn-Sham equations.
This should yield substantial savings in computer time, allowing either larger
systems or longer time-scales to be tackled, but attempts to machine-learn this
functional have been limited by the need to find its derivative. The present
work overcomes this difficulty by directly learning the density-potential and
energy-density maps for test systems and various molecules. Both improved
accuracy and lower computational cost with this method are demonstrated by
reproducing DFT energies for a range of molecular geometries generated during
molecular dynamics simulations. Moreover, the methodology could be applied
directly to quantum chemical calculations, allowing construction of density
functionals of quantum-chemical accuracy.
Stochastic Training of Neural Networks via Successive Convex Approximations
Authors: Simone Scardapane, Paolo Di Lorenzo
Date: 2017-06-15 00:00:00

This paper proposes a new family of algorithms for training neural networks
(NNs). These are based on recent developments in the field of non-convex
optimization, going under the general name of successive convex approximation
(SCA) techniques. The basic idea is to iteratively replace the original
(non-convex, highly dimensional) learning problem with a sequence of (strongly
convex) approximations, which are both accurate and simple to optimize.
Differently from similar ideas (e.g., quasi-Newton algorithms), the
approximations can be constructed using only first-order information of the
neural network function, in a stochastic fashion, while exploiting the overall
structure of the learning problem for a faster convergence. We discuss several
use cases, based on different choices for the loss function (e.g., squared loss
and cross-entropy loss), and for the regularization of the NN's weights. We
experiment on several medium-sized benchmark problems, and on a large-scale
dataset involving simulated physical data. The results show how the algorithm
outperforms state-of-the-art techniques, providing faster convergence to a
better minimum. Additionally, we show how the algorithm can be easily
parallelized over multiple computational units without hindering its
performance. In particular, each computational unit can optimize a tailored
surrogate function defined on a randomly assigned subset of the input
variables, whose dimension can be selected depending entirely on the available
computational power.
Efficient Representative Subset Selection over Sliding Windows
Authors: Yanhao Wang, Yuchen Li, Kian-Lee Tan
Date: 2017-06-15 00:00:00

Representative subset selection (RSS) is an important tool for users to draw
insights from massive datasets. Existing literature models RSS as the
submodular maximization problem to capture the "diminishing returns" property
of the representativeness of selected subsets, but often only has a single
constraint (e.g., cardinality), which limits its applications in many
real-world problems. To capture the data recency issue and support different
types of constraints, we formulate dynamic RSS in data streams as maximizing
submodular functions subject to general $d$-knapsack constraints (SMDK) over
sliding windows. We propose a \textsc{KnapWindow} framework (KW) for SMDK. KW
utilizes the \textsc{KnapStream} algorithm (KS) for SMDK in append-only streams
as a subroutine. It maintains a sequence of checkpoints and KS instances over
the sliding window. Theoretically, KW is
$\frac{1-\varepsilon}{1+d}$-approximate for SMDK. Furthermore, we propose a
\textsc{KnapWindowPlus} framework (KW$^{+}$) to improve upon KW. KW$^{+}$
builds an index \textsc{SubKnapChk} to manage the checkpoints and KS instances.
\textsc{SubKnapChk} deletes a checkpoint whenever it can be approximated by its
successors. By keeping much fewer checkpoints, KW$^{+}$ achieves higher
efficiency than KW while still guaranteeing a
$\frac{1-\varepsilon'}{2+2d}$-approximate solution for SMDK. Finally, we
evaluate the efficiency and solution quality of KW and KW$^{+}$ in real-world
datasets. The experimental results demonstrate that KW achieves more than two
orders of magnitude speedups over the batch baseline and preserves high-quality
solutions for SMDK over sliding windows. KW$^{+}$ further runs 5-10 times
faster than KW while providing solutions with equivalent or even better
utilities.
Video Imagination from a Single Image with Transformation Generation
Authors: Baoyang Chen, Wenmin Wang, Jinzhuo Wang, Xiongtao Chen
Date: 2017-06-13 00:00:00
Tasks: Image Quality Assessment

In this work, we focus on a challenging task: synthesizing multiple imaginary
videos given a single image. Major problems come from high dimensionality of
pixel space and the ambiguity of potential motions. To overcome those problems,
we propose a new framework that produce imaginary videos by transformation
generation. The generated transformations are applied to the original image in
a novel volumetric merge network to reconstruct frames in imaginary video.
Through sampling different latent variables, our method can output different
imaginary video samples. The framework is trained in an adversarial way with
unsupervised learning. For evaluation, we propose a new assessment metric
$RIQA$. In experiments, we test on 3 datasets varying from synthetic data to
natural scene. Our framework achieves promising performance in image quality
assessment. The visual inspection indicates that it can successfully generate
diverse five-frame videos in acceptable perceptual quality.
Extracting Formal Models from Normative Texts
Authors: John J. Camilleri, Normunds Grūzītis, Gerardo Schneider
Date: 2017-06-15 00:00:00
Tasks: Sentence

We are concerned with the analysis of normative texts - documents based on
the deontic notions of obligation, permission, and prohibition. Our goal is to
make queries about these notions and verify that a text satisfies certain
properties concerning causality of actions and timing constraints. This
requires taking the original text and building a representation (model) of it
in a formal language, in our case the C-O Diagram formalism. We present an
experimental, semi-automatic aid that helps to bridge the gap between a
normative text in natural language and its C-O Diagram representation. Our
approach consists of using dependency structures obtained from the
state-of-the-art Stanford Parser, and applying our own rules and heuristics in
order to extract the relevant components. The result is a tabular data
structure where each sentence is split into suitable fields, which can then be
converted into a C-O Diagram. The process is not fully automatic however, and
some post-editing is generally required of the user. We apply our tool and
perform experiments on documents from different domains, and report an initial
evaluation of the accuracy and feasibility of our approach.
Suggestive Annotation: A Deep Active Learning Framework for Biomedical Image Segmentation
Authors: Lin Yang, Yizhe Zhang, Jianxu Chen, Si-Yuan Zhang, Danny Z. Chen
Date: 2017-06-15 00:00:00
Tasks: Active Learning, Image Segmentation, Segmentation, Semantic Segmentation

Image segmentation is a fundamental problem in biomedical image analysis.
Recent advances in deep learning have achieved promising results on many
biomedical image segmentation benchmarks. However, due to large variations in
biomedical images (different modalities, image settings, objects, noise, etc),
to utilize deep learning on a new application, it usually needs a new set of
training data. This can incur a great deal of annotation effort and cost,
because only biomedical experts can annotate effectively, and often there are
too many instances in images (e.g., cells) to annotate. In this paper, we aim
to address the following question: With limited effort (e.g., time) for
annotation, what instances should be annotated in order to attain the best
performance? We present a deep active learning framework that combines fully
convolutional network (FCN) and active learning to significantly reduce
annotation effort by making judicious suggestions on the most effective
annotation areas. We utilize uncertainty and similarity information provided by
FCN and formulate a generalized version of the maximum set cover problem to
determine the most representative and uncertain areas for annotation. Extensive
experiments using the 2015 MICCAI Gland Challenge dataset and a lymph node
ultrasound image segmentation dataset show that, using annotation suggestions
by our method, state-of-the-art segmentation performance can be achieved by
using only 50% of training data.
Sequential detection of low-rank changes using extreme eigenvalues
Authors: Liyan Xie, Yao Xie
Date: 2017-06-15 00:00:00
Tasks: Change Point Detection

We study the problem of detecting an abrupt change to the signal covariance
matrix. In particular, the covariance changes from a "white" identity matrix to
an unknown spiked or low-rank matrix. Two sequential change-point detection
procedures are presented, based on the largest and the smallest eigenvalues of
the sample covariance matrix. To control false-alarm-rate, we present an
accurate theoretical approximation to the average-run-length (ARL) and expected
detection delay (EDD) of the detection, leveraging the extreme eigenvalue
distributions from random matrix theory and by capturing a non-negligible
temporal correlation in the sequence of scan statistics due to the sliding
window approach. Real data examples demonstrate the good performance of our
method for detecting behavior change of a swarm.
Effective Representations of Clinical Notes
Authors: Sebastien Dubois, Nathanael Romano, David C. Kale, Nigam Shah, Kenneth Jung
Date: 2017-05-19 00:00:00
Tasks: Feature Engineering, Transfer Learning

Clinical notes are a rich source of information about patient state. However,
using them to predict clinical events with machine learning models is
challenging. They are very high dimensional, sparse and have complex structure.
Furthermore, training data is often scarce because it is expensive to obtain
reliable labels for many clinical events. These difficulties have traditionally
been addressed by manual feature engineering encoding task specific domain
knowledge. We explored the use of neural networks and transfer learning to
learn representations of clinical notes that are useful for predicting future
clinical events of interest, such as all causes mortality, inpatient
admissions, and emergency room visits. Our data comprised 2.7 million notes and
115 thousand patients at Stanford Hospital. We used the learned
representations, along with commonly used bag of words and topic model
representations, as features for predictive models of clinical events. We
evaluated the effectiveness of these representations with respect to the
performance of the models trained on small datasets. Models using the neural
network derived representations performed significantly better than models
using the baseline representations with small ($N < 1000$) training datasets.
The learned representations offer significant performance gains over commonly
used baseline representations for a range of predictive modeling tasks and
cohort sizes, offering an effective alternative to task specific feature
engineering when plentiful labeled training data is not available.
Text Summarization using Deep Learning and Ridge Regression
Authors: Karthik Bangalore Mani
Date: 2016-12-26 00:00:00
Tasks: Deep Learning, regression, Sentence, Text Summarization

We develop models and extract relevant features for automatic text
summarization and investigate the performance of different models on the DUC
2001 dataset. Two different models were developed, one being a ridge regressor
and the other one was a multi-layer perceptron. The hyperparameters were varied
and their performance were noted. We segregated the summarization task into 2
main steps, the first being sentence ranking and the second step being sentence
selection. In the first step, given a document, we sort the sentences based on
their Importance, and in the second step, in order to obtain non-redundant
sentences, we weed out the sentences that are have high similarity with the
previously selected sentences.
The Compressed Model of Residual CNDS
Authors: Hussam Qassim, David Feinzimer, Abhishek Verma
Date: 2017-06-15 00:00:00
Tasks: model

Convolutional neural networks have achieved a great success in the recent
years. Although, the way to maximize the performance of the convolutional
neural networks still in the beginning. Furthermore, the optimization of the
size and the time that need to train the convolutional neural networks is very
far away from reaching the researcher's ambition. In this paper, we proposed a
new convolutional neural network that combined several techniques to boost the
optimization of the convolutional neural network in the aspects of speed and
size. As we used our previous model Residual-CNDS (ResCNDS), which solved the
problems of slower convergence, overfitting, and degradation, and compressed
it. The outcome model called Residual-Squeeze-CNDS (ResSquCNDS), which we
demonstrated on our sold technique to add residual learning and our model of
compressing the convolutional neural networks. Our model of compressing adapted
from the SQUEEZENET model, but our model is more generalizable, which can be
applied almost to any neural network model, and fully integrated into the
residual learning, which addresses the problem of the degradation very
successfully. Our proposed model trained on very large-scale MIT
Places365-Standard scene datasets, which backing our hypothesis that the new
compressed model inherited the best of the previous ResCNDS8 model, and almost
get the same accuracy in the validation Top-1 and Top-5 with 87.64% smaller in
size and 13.33% faster in the training time.
Recent Progress of Face Image Synthesis
Authors: Zhihe Lu, Zhihang Li, Jie Cao, Ran He, Zhenan Sun
Date: 2017-06-15 00:00:00
Tasks: Face Generation, Face Recognition, Image Generation

Face synthesis has been a fascinating yet challenging problem in computer
vision and machine learning. Its main research effort is to design algorithms
to generate photo-realistic face images via given semantic domain. It has been
a crucial prepossessing step of main-stream face recognition approaches and an
excellent test of AI ability to use complicated probability distributions. In
this paper, we provide a comprehensive review of typical face synthesis works
that involve traditional methods as well as advanced deep learning approaches.
Particularly, Generative Adversarial Net (GAN) is highlighted to generate
photo-realistic and identity preserving results. Furthermore, the public
available databases and evaluation metrics are introduced in details. We end
the review with discussing unsolved difficulties and promising directions for
future research.
Visual Question Answering: Datasets, Algorithms, and Future Challenges
Authors: Kushal Kafle, Christopher Kanan
Date: 2016-10-05 00:00:00
Tasks: Question Answering, Visual Question Answering, Visual Question Answering (VQA)

Visual Question Answering (VQA) is a recent problem in computer vision and
natural language processing that has garnered a large amount of interest from
the deep learning, computer vision, and natural language processing
communities. In VQA, an algorithm needs to answer text-based questions about
images. Since the release of the first VQA dataset in 2014, additional datasets
have been released and many algorithms have been proposed. In this review, we
critically examine the current state of VQA in terms of problem formulation,
existing datasets, evaluation metrics, and algorithms. In particular, we
discuss the limitations of current datasets with regard to their ability to
properly train and assess VQA algorithms. We then exhaustively review existing
algorithms for VQA. Finally, we discuss possible future directions for VQA and
image understanding research.
Dueling Bandits with Dependent Arms
Authors: Bangrui Chen, Peter I. Frazier
Date: 2016-05-28 00:00:00

We study dueling bandits with weak utility-based regret when preferences over
arms have a total order and carry observable feature vectors. The order is
assumed to be determined by these feature vectors, an unknown preference
vector, and a known utility function. This structure introduces dependence
between preferences for pairs of arms, and allows learning about the preference
over one pair of arms from the preference over another pair of arms. We propose
an algorithm for this setting called Comparing The Best (CTB), which we show
has constant expected cumulative weak utility-based regret. We provide a
Bayesian interpretation for CTB, an implementation appropriate for a small
number of arms, and an alternate implementation for many arms that can be used
when the input parameters satisfy a decomposability condition. We demonstrate
through numerical experiments that CTB with appropriate input parameters
outperforms all benchmarks considered.
Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
Authors: Weinan E, Jiequn Han, Arnulf Jentzen
Date: 2017-06-15 00:00:00
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We propose a new algorithm for solving parabolic partial differential
equations (PDEs) and backward stochastic differential equations (BSDEs) in high
dimension, by making an analogy between the BSDE and reinforcement learning
with the gradient of the solution playing the role of the policy function, and
the loss function given by the error between the prescribed terminal condition
and the solution of the BSDE. The policy function is then approximated by a
neural network, as is done in deep reinforcement learning. Numerical results
using TensorFlow illustrate the efficiency and accuracy of the proposed
algorithms for several 100-dimensional nonlinear PDEs from physics and finance
such as the Allen-Cahn equation, the Hamilton-Jacobi-Bellman equation, and a
nonlinear pricing model for financial derivatives.
Adversarial Example Defenses: Ensembles of Weak Defenses are not Strong
Authors: Warren He, James Wei, Xinyun Chen, Nicholas Carlini, Dawn Song
Date: 2017-06-15 00:00:00

Ongoing research has proposed several methods to defend neural networks
against adversarial examples, many of which researchers have shown to be
ineffective. We ask whether a strong defense can be created by combining
multiple (possibly weak) defenses. To answer this question, we study three
defenses that follow this approach. Two of these are recently proposed defenses
that intentionally combine components designed to work well together. A third
defense combines three independent defenses. For all the components of these
defenses and the combined defenses themselves, we show that an adaptive
adversary can create adversarial examples successfully with low distortion.
Thus, our work implies that ensemble of weak defenses is not sufficient to
provide strong defense against adversarial examples.
Deep Generalized Canonical Correlation Analysis
Authors: Adrian Benton, Huda Khayrallah, Biman Gujral, Dee Ann Reisinger, Sheng Zhang, Raman Arora
Date: 2017-02-08 00:00:00
Proceeding: WS 2019 8
Tasks: Representation Learning, Stochastic Optimization

We present Deep Generalized Canonical Correlation Analysis (DGCCA) -- a
method for learning nonlinear transformations of arbitrarily many views of
data, such that the resulting transformations are maximally informative of each
other. While methods for nonlinear two-view representation learning (Deep CCA,
(Andrew et al., 2013)) and linear many-view representation learning
(Generalized CCA (Horst, 1961)) exist, DGCCA is the first CCA-style multiview
representation learning technique that combines the flexibility of nonlinear
(deep) representation learning with the statistical power of incorporating
information from many independent sources, or views. We present the DGCCA
formulation as well as an efficient stochastic optimization algorithm for
solving it. We learn DGCCA representations on two distinct datasets for three
downstream tasks: phonetic transcription from acoustic and articulatory
measurements, and recommending hashtags and friends on a dataset of Twitter
users. We find that DGCCA representations soundly beat existing methods at
phonetic transcription and hashtag recommendation, and in general perform no
worse than standard linear many-view techniques.
SEVEN: Deep Semi-supervised Verification Networks
Authors: Vahid Noroozi, Lei Zheng, Sara Bahaadini, Sihong Xie, Philip S. Yu
Date: 2017-06-12 00:00:00
Tasks: fingerprint verification

Verification determines whether two samples belong to the same class or not,
and has important applications such as face and fingerprint verification, where
thousands or millions of categories are present but each category has scarce
labeled examples, presenting two major challenges for existing deep learning
models. We propose a deep semi-supervised model named SEmi-supervised
VErification Network (SEVEN) to address these challenges. The model consists of
two complementary components. The generative component addresses the lack of
supervision within each category by learning general salient structures from a
large amount of data across categories. The discriminative component exploits
the learned general features to mitigate the lack of supervision within
categories, and also directs the generative component to find more informative
structures of the whole data manifold. The two components are tied together in
SEVEN to allow an end-to-end training of the two components. Extensive
experiments on four verification tasks demonstrate that SEVEN significantly
outperforms other state-of-the-art deep semi-supervised techniques when labeled
data are in short supply. Furthermore, SEVEN is competitive with fully
supervised baselines trained with a larger amount of labeled data. It indicates
the importance of the generative component in SEVEN.