pwc-archive/papers-with-abstracts huggingface dataset:
Dynamic Network Model from Partial Observations
Authors: Elahe Ghalebi, Baharan Mirzasoleiman, Radu Grosu, Jure Leskovec
Date: 2018-05-27 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: model, Open-Ended Question Answering

Can evolving networks be inferred and modeled without directly observing
their nodes and edges? In many applications, the edges of a dynamic network
might not be observed, but one can observe the dynamics of stochastic cascading
processes (e.g., information diffusion, virus propagation) occurring over the
unobserved network. While there have been efforts to infer networks based on
such data, providing a generative probabilistic model that is able to identify
the underlying time-varying network remains an open question. Here we consider
the problem of inferring generative dynamic network models based on network
cascade diffusion data. We propose a novel framework for providing a
non-parametric dynamic network model--based on a mixture of coupled
hierarchical Dirichlet processes-- based on data capturing cascade node
infection times. Our approach allows us to infer the evolving community
structure in networks and to obtain an explicit predictive distribution over
the edges of the underlying network--including those that were not involved in
transmission of any cascade, or are likely to appear in the future. We show the
effectiveness of our approach using extensive experiments on synthetic as well
as real-world networks.
PAC-Bayes bounds for stable algorithms with instance-dependent priors
Authors: Omar Rivasplata, Emilio Parrado-Hernandez, John Shawe-Taylor, Shiliang Sun, Csaba Szepesvari
Date: 2018-06-18 00:00:00
Proceeding: NeurIPS 2018 12

PAC-Bayes bounds have been proposed to get risk estimates based on a training
sample. In this paper the PAC-Bayes approach is combined with stability of the
hypothesis learned by a Hilbert space valued algorithm. The PAC-Bayes setting
is used with a Gaussian prior centered at the expected output. Thus a novelty
of our paper is using priors defined in terms of the data-generating
distribution. Our main result estimates the risk of the randomized algorithm in
terms of the hypothesis stability coefficients. We also provide a new bound for
the SVM classifier, which is compared to other known bounds experimentally.
Ours appears to be the first stability-based bound that evaluates to
non-trivial values.
Automated Bridge Component Recognition using Video Data
Authors: Yasutaka Narazaki, Vedhus Hoskere, Tu A. Hoang, Billie F. Spencer Jr
Date: 2018-06-18 00:00:00
Tasks: BIG-bench Machine Learning

This paper investigates the automated recognition of structural bridge
components using video data. Although understanding video data for structural
inspections is straightforward for human inspectors, the implementation of the
same task using machine learning methods has not been fully realized. In
particular, single-frame image processing techniques, such as convolutional
neural networks (CNNs), are not expected to identify structural components
accurately when the image is a close-up view, lacking contextual information
regarding where on the structure the image originates. Inspired by the
significant progress in video processing techniques, this study investigates
automated bridge component recognition using video data, where the information
from the past frames is used to augment the understanding of the current frame.
A new simulated video dataset is created to train the machine learning
algorithms. Then, convolutional Neural Networks (CNNs) with recurrent
architectures are designed and applied to implement the automated bridge
component recognition task. Results are presented for simulated video data, as
well as video collected in the field.
Gradient descent with identity initialization efficiently learns positive definite linear transformations by deep residual networks
Authors: Peter L. Bartlett, David P. Helmbold, Philip M. Long
Date: 2018-02-16 00:00:00
Proceeding: ICML 2018

We analyze algorithms for approximating a function $f(x) = \Phi x$ mapping
$\Re^d$ to $\Re^d$ using deep linear neural networks, i.e. that learn a
function $h$ parameterized by matrices $\Theta_1,...,\Theta_L$ and defined by
$h(x) = \Theta_L \Theta_{L-1} ... \Theta_1 x$. We focus on algorithms that
learn through gradient descent on the population quadratic loss in the case
that the distribution over the inputs is isotropic.
  We provide polynomial bounds on the number of iterations for gradient descent
to approximate the least squares matrix $\Phi$, in the case where the initial
hypothesis $\Theta_1 = ... = \Theta_L = I$ has excess loss bounded by a small
enough constant. On the other hand, we show that gradient descent fails to
converge for $\Phi$ whose distance from the identity is a larger constant, and
we show that some forms of regularization toward the identity in each layer do
not help.
  If $\Phi$ is symmetric positive definite, we show that an algorithm that
initializes $\Theta_i = I$ learns an $\epsilon$-approximation of $f$ using a
number of updates polynomial in $L$, the condition number of $\Phi$, and
$\log(d/\epsilon)$. In contrast, we show that if the least squares matrix
$\Phi$ is symmetric and has a negative eigenvalue, then all members of a class
of algorithms that perform gradient descent with identity initialization, and
optionally regularize toward the identity in each layer, fail to converge.
  We analyze an algorithm for the case that $\Phi$ satisfies $u^{\top} \Phi u >
0$ for all $u$, but may not be symmetric. This algorithm uses two regularizers:
one that maintains the invariant $u^{\top} \Theta_L \Theta_{L-1} ... \Theta_1 u
> 0$ for all $u$, and another that "balances" $\Theta_1, ..., \Theta_L$ so that
they have the same singular values.
Temporal coherence-based self-supervised learning for laparoscopic workflow analysis
Authors: Isabel Funke, Alexander Jenke, Sören Torge Mees, Jürgen Weitz, Stefanie Speidel, Sebastian Bodenstedt
Date: 2018-06-18 00:00:00
Tasks: Self-Supervised Learning, Surgical phase recognition

In order to provide the right type of assistance at the right time,
computer-assisted surgery systems need context awareness. To achieve this,
methods for surgical workflow analysis are crucial. Currently, convolutional
neural networks provide the best performance for video-based workflow analysis
tasks. For training such networks, large amounts of annotated data are
necessary. However, collecting a sufficient amount of data is often costly,
time-consuming, and not always feasible. In this paper, we address this problem
by presenting and comparing different approaches for self-supervised
pretraining of neural networks on unlabeled laparoscopic videos using temporal
coherence. We evaluate our pretrained networks on Cholec80, a publicly
available dataset for surgical phase segmentation, on which a maximum F1 score
of 84.6 was reached. Furthermore, we were able to achieve an increase of the F1
score of up to 10 points when compared to a non-pretrained neural network.
Better Runtime Guarantees Via Stochastic Domination
Authors: Benjamin Doerr
Date: 2018-01-13 00:00:00
Tasks: Evolutionary Algorithms

Apart from few exceptions, the mathematical runtime analysis of evolutionary
algorithms is mostly concerned with expected runtimes. In this work, we argue
that stochastic domination is a notion that should be used more frequently in
this area. Stochastic domination allows to formulate much more informative
performance guarantees, it allows to decouple the algorithm analysis into the
true algorithmic part of detecting a domination statement and the
probability-theoretical part of deriving the desired probabilistic guarantees
from this statement, and it helps finding simpler and more natural proofs.
  As particular results, we prove a fitness level theorem which shows that the
runtime is dominated by a sum of independent geometric random variables, we
prove the first tail bounds for several classic runtime problems, and we give a
short and natural proof for Witt's result that the runtime of any $(\mu,p)$
mutation-based algorithm on any function with unique optimum is subdominated by
the runtime of a variant of the \oea on the \onemax function.
  As side-products, we determine the fastest unbiased (1+1) algorithm for the
\leadingones benchmark problem, both in the general case and when restricted to
static mutation operators, and we prove a Chernoff-type tail bound for sums of
independent coupon collector distributions.
Scaling Neural Machine Translation
Authors: Myle Ott, Sergey Edunov, David Grangier, Michael Auli
Date: 2018-06-01 00:00:00
Proceeding: WS 2018 10
Tasks: GPU, Machine Translation, Question Answering, Translation

Sequence to sequence learning models still require several days to reach
state of the art performance on large benchmark datasets using a single
machine. This paper shows that reduced precision and large batch training can
speedup training by nearly 5x on a single 8-GPU machine with careful tuning and
implementation. On WMT'14 English-German translation, we match the accuracy of
Vaswani et al. (2017) in under 5 hours when training on 8 GPUs and we obtain a
new state of the art of 29.3 BLEU after training for 85 minutes on 128 GPUs. We
further improve these results to 29.8 BLEU by training on the much larger
Paracrawl dataset. On the WMT'14 English-French task, we obtain a
state-of-the-art BLEU of 43.2 in 8.5 hours on 128 GPUs.
Interpretable Almost Matching Exactly for Causal Inference
Authors: Yameng Liu, Aw Dieng, Sudeepa Roy, Cynthia Rudin, Alexander Volfovsky
Date: 2018-06-18 00:00:00
Tasks: Causal Inference

We aim to create the highest possible quality of treatment-control matches for categorical data in the potential outcomes framework. Matching methods are heavily used in the social sciences due to their interpretability, but most matching methods do not pass basic sanity checks: they fail when irrelevant variables are introduced, and tend to be either computationally slow or produce low-quality matches. The method proposed in this work aims to match units on a weighted Hamming distance, taking into account the relative importance of the covariates; the algorithm aims to match units on as many relevant variables as possible. To do this, the algorithm creates a hierarchy of covariate combinations on which to match (similar to downward closure), in the process solving an optimization problem for each unit in order to construct the optimal matches. The algorithm uses a single dynamic program to solve all of the optimization problems simultaneously. Notable advantages of our method over existing matching procedures are its high-quality matches, versatility in handling different data distributions that may have irrelevant variables, and ability to handle missing data by matching on as many available covariates as possible.
Deep Spatiotemporal Representation of the Face for Automatic Pain Intensity Estimation
Authors: Mohammad Tavakolian, Abdenour Hadid
Date: 2018-06-18 00:00:00

Automatic pain intensity assessment has a high value in disease diagnosis
applications. Inspired by the fact that many diseases and brain disorders can
interrupt normal facial expression formation, we aim to develop a computational
model for automatic pain intensity assessment from spontaneous and micro facial
variations. For this purpose, we propose a 3D deep architecture for dynamic
facial video representation. The proposed model is built by stacking several
convolutional modules where each module encompasses a 3D convolution kernel
with a fixed temporal depth, several parallel 3D convolutional kernels with
different temporal depths, and an average pooling layer. Deploying variable
temporal depths in the proposed architecture allows the model to effectively
capture a wide range of spatiotemporal variations on the faces. Extensive
experiments on the UNBC-McMaster Shoulder Pain Expression Archive database show
that our proposed model yields in a promising performance compared to the
state-of-the-art in automatic pain intensity estimation.
Robust inference on the average treatment effect using the outcome highly adaptive lasso
Authors: Cheng Ju, David Benkeser, Mark J. Van Der Laan
Date: 2018-06-18 00:00:00
Tasks: regression

Many estimators of the average effect of a treatment on an outcome require estimation of the propensity score, the outcome regression, or both. It is often beneficial to utilize flexible techniques such as semiparametric regression or machine learning to estimate these quantities. However, optimal estimation of these regressions does not necessarily lead to optimal estimation of the average treatment effect, particularly in settings with strong instrumental variables. A recent proposal addressed these issues via the outcome-adaptive lasso, a penalized regression technique for estimating the propensity score that seeks to minimize the impact of instrumental variables on treatment effect estimators. However, a notable limitation of this approach is that its application is restricted to parametric models. We propose a more flexible alternative that we call the outcome highly adaptive lasso. We discuss large sample theory for this estimator and propose closed form confidence intervals based on the proposed estimator. We show via simulation that our method offers benefits over several popular approaches.
Consistent Individualized Feature Attribution for Tree Ensembles
Authors: Scott M. Lundberg, Gabriel G. Erion, Su-In Lee
Date: 2018-02-12 00:00:00
Tasks: BIG-bench Machine Learning

A unified approach to explain the output of any machine learning model.
BinGAN: Learning Compact Binary Descriptors with a Regularized GAN
Authors: Maciej Zieba, Piotr Semberecki, Tarek El-Gaaly, Tomasz Trzcinski
Date: 2018-06-18 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Dimensionality Reduction, Retrieval

In this paper, we propose a novel regularization method for Generative
Adversarial Networks, which allows the model to learn discriminative yet
compact binary representations of image patches (image descriptors). We employ
the dimensionality reduction that takes place in the intermediate layers of the
discriminator network and train binarized low-dimensional representation of the
penultimate layer to mimic the distribution of the higher-dimensional preceding
layers. To achieve this, we introduce two loss terms that aim at: (i) reducing
the correlation between the dimensions of the binarized low-dimensional
representation of the penultimate layer i. e. maximizing joint entropy) and
(ii) propagating the relations between the dimensions in the high-dimensional
space to the low-dimensional space. We evaluate the resulting binary image
descriptors on two challenging applications, image matching and retrieval, and
achieve state-of-the-art results.
Multiscale Fisher's Independence Test for Multivariate Dependence
Authors: Shai Gorsky, Li Ma
Date: 2018-06-18 00:00:00

Identifying dependency in multivariate data is a common inference task that arises in numerous applications. However, existing nonparametric independence tests typically require computation that scales at least quadratically with the sample size, making it difficult to apply them to massive data. Moreover, resampling is usually necessary to evaluate the statistical significance of the resulting test statistics at finite sample sizes, further worsening the computational burden. We introduce a scalable, resampling-free approach to testing the independence between two random vectors by breaking down the task into simple univariate tests of independence on a collection of 2x2 contingency tables constructed through sequential coarse-to-fine discretization of the sample space, transforming the inference task into a multiple testing problem that can be completed with almost linear complexity with respect to the sample size. To address increasing dimensionality, we introduce a coarse-to-fine sequential adaptive procedure that exploits the spatial features of dependency structures to more effectively examine the sample space. We derive a finite-sample theory that guarantees the inferential validity of our adaptive procedure at any given sample size. In particular, we show that our approach can achieve strong control of the family-wise error rate without resampling or large-sample approximation. We demonstrate the substantial computational advantage of the procedure in comparison to existing approaches as well as its decent statistical power under various dependency scenarios through an extensive simulation study, and illustrate how the divide-and-conquer nature of the procedure can be exploited to not just test independence but to learn the nature of the underlying dependency. Finally, we demonstrate the use of our method through analyzing a large data set from a flow cytometry experiment.
Kernel-based Outlier Detection using the Inverse Christoffel Function
Authors: Armin Askari, Forest Yang, Laurent El Ghaoui
Date: 2018-06-18 00:00:00
Tasks: Outlier Detection

Outlier detection methods have become increasingly relevant in recent years
due to increased security concerns and because of its vast application to
different fields. Recently, Pauwels and Lasserre (2016) noticed that the
sublevel sets of the inverse Christoffel function accurately depict the shape
of a cloud of data using a sum-of-squares polynomial and can be used to perform
outlier detection. In this work, we propose a kernelized variant of the inverse
Christoffel function that makes it computationally tractable for data sets with
a large number of features. We compare our approach to current methods on 15
different data sets and achieve the best average area under the precision
recall curve (AUPRC) score, the best average rank and the lowest root mean
square deviation.
Kid-Net: Convolution Networks for Kidney Vessels Segmentation from CT-Volumes
Authors: Ahmed Taha, Pechin Lo, Junning Li, Tao Zhao
Date: 2018-06-18 00:00:00
Tasks: Anatomy, Image Segmentation, Segmentation, Semantic Segmentation

Semantic image segmentation plays an important role in modeling
patient-specific anatomy. We propose a convolution neural network, called
Kid-Net, along with a training schema to segment kidney vessels: artery, vein
and collecting system. Such segmentation is vital during the surgical planning
phase in which medical decisions are made before surgical incision. Our main
contribution is developing a training schema that handles unbalanced data,
reduces false positives and enables high-resolution segmentation with a limited
memory budget. These objectives are attained using dynamic weighting, random
sampling and 3D patch segmentation. Manual medical image annotation is both
time-consuming and expensive. Kid-Net reduces kidney vessels segmentation time
from matter of hours to minutes. It is trained end-to-end using 3D patches from
volumetric CT-images. A complete segmentation for a 512x512x512 CT-volume is
obtained within a few minutes (1-2 mins) by stitching the output 3D patches
together. Feature down-sampling and up-sampling are utilized to achieve higher
classification and localization accuracies. Quantitative and qualitative
evaluation results on a challenging testing dataset show Kid-Net competence.
Modularity Matters: Learning Invariant Relational Reasoning Tasks
Authors: Jason Jo, Vikas Verma, Yoshua Bengio
Date: 2018-06-18 00:00:00
Tasks: Mixture-of-Experts, Relational Reasoning, Visual Reasoning

We focus on two supervised visual reasoning tasks whose labels encode a
semantic relational rule between two or more objects in an image: the MNIST
Parity task and the colorized Pentomino task. The objects in the images undergo
random translation, scaling, rotation and coloring transformations. Thus these
tasks involve invariant relational reasoning. We report uneven performance of
various deep CNN models on these two tasks. For the MNIST Parity task, we
report that the VGG19 model soundly outperforms a family of ResNet models.
Moreover, the family of ResNet models exhibits a general sensitivity to random
initialization for the MNIST Parity task. For the colorized Pentomino task, now
both the VGG19 and ResNet models exhibit sluggish optimization and very poor
test generalization, hovering around 30% test error. The CNN we tested all
learn hierarchies of fully distributed features and thus encode the distributed
representation prior. We are motivated by a hypothesis from cognitive
neuroscience which posits that the human visual cortex is modularized, and this
allows the visual cortex to learn higher order invariances. To this end, we
consider a modularized variant of the ResNet model, referred to as a Residual
Mixture Network (ResMixNet) which employs a mixture-of-experts architecture to
interleave distributed representations with more specialized, modular
representations. We show that very shallow ResMixNets are capable of learning
each of the two tasks well, attaining less than 2% and 1% test error on the
MNIST Parity and the colorized Pentomino tasks respectively. Most importantly,
the ResMixNet models are extremely parameter efficient: generalizing better
than various non-modular CNNs that have over 10x the number of parameters.
These experimental results support the hypothesis that modularity is a robust
prior for learning invariant relational reasoning.
Closing the Generalization Gap of Adaptive Gradient Methods in Training Deep Neural Networks
Authors: Jinghui Chen, Dongruo Zhou, Yiqi Tang, Ziyan Yang, Yuan Cao, Quanquan Gu
Date: 2018-06-18 00:00:00

Adaptive gradient methods, which adopt historical gradient information to automatically adjust the learning rate, despite the nice property of fast convergence, have been observed to generalize worse than stochastic gradient descent (SGD) with momentum in training deep neural networks. This leaves how to close the generalization gap of adaptive gradient methods an open problem. In this work, we show that adaptive gradient methods such as Adam, Amsgrad, are sometimes "over adapted". We design a new algorithm, called Partially adaptive momentum estimation method, which unifies the Adam/Amsgrad with SGD by introducing a partial adaptive parameter $p$, to achieve the best from both worlds. We also prove the convergence rate of our proposed algorithm to a stationary point in the stochastic nonconvex optimization setting. Experiments on standard benchmarks show that our proposed algorithm can maintain a fast convergence rate as Adam/Amsgrad while generalizing as well as SGD in training deep neural networks. These results would suggest practitioners pick up adaptive gradient methods once again for faster training of deep neural networks.
A Memory Network Approach for Story-based Temporal Summarization of 360° Videos
Authors: Sang-ho Lee, Jinyoung Sung, Youngjae Yu, Gunhee Kim
Date: 2018-05-08 00:00:00
Proceeding: CVPR 2018
Tasks: Video Summarization

We address the problem of story-based temporal summarization of long
360{\deg} videos. We propose a novel memory network model named Past-Future
Memory Network (PFMN), in which we first compute the scores of 81 normal field
of view (NFOV) region proposals cropped from the input 360{\deg} video, and
then recover a latent, collective summary using the network with two external
memories that store the embeddings of previously selected subshots and future
candidate subshots. Our major contributions are two-fold. First, our work is
the first to address story-based temporal summarization of 360{\deg} videos.
Second, our model is the first attempt to leverage memory networks for video
summarization tasks. For evaluation, we perform three sets of experiments.
First, we investigate the view selection capability of our model on the
Pano2Vid dataset. Second, we evaluate the temporal summarization with a newly
collected 360{\deg} video dataset. Finally, we experiment our model's
performance in another domain, with image-based storytelling VIST dataset. We
verify that our model achieves state-of-the-art performance on all the tasks.