pwc-archive/papers-with-abstracts huggingface dataset:
Extracting News Events from Microblogs
Authors: Øystein Repp, Heri Ramampiaro
Date: 2018-06-20 00:00:00
Tasks: Clustering

Twitter stream has become a large source of information for many people, but
the magnitude of tweets and the noisy nature of its content have made
harvesting the knowledge from Twitter a challenging task for researchers for a
long time. Aiming at overcoming some of the main challenges of extracting the
hidden information from tweet streams, this work proposes a new approach for
real-time detection of news events from the Twitter stream. We divide our
approach into three steps. The first step is to use a neural network or deep
learning to detect news-relevant tweets from the stream. The second step is to
apply a novel streaming data clustering algorithm to the detected news tweets
to form news events. The third and final step is to rank the detected events
based on the size of the event clusters and growth speed of the tweet
frequencies. We evaluate the proposed system on a large, publicly available
corpus of annotated news events from Twitter. As part of the evaluation, we
compare our approach with a related state-of-the-art solution. Overall, our
experiments and user-based evaluation show that our approach on detecting
current (real) news events delivers a state-of-the-art performance.
Wall Stress Estimation of Cerebral Aneurysm based on Zernike Convolutional Neural Networks
Authors: Zhiyu Sun, Jia Lu, Stephen Baek
Date: 2018-06-20 00:00:00

Convolutional neural networks (ConvNets) have demonstrated an exceptional
capacity to discern visual patterns from digital images and signals.
Unfortunately, such powerful ConvNets do not generalize well to
arbitrary-shaped manifolds, where data representation does not fit into a
tensor-like grid. Hence, many fields of science and engineering, where data
points possess some manifold structure, cannot enjoy the full benefits of the
recent advances in ConvNets. The aneurysm wall stress estimation problem
introduced in this paper is one of many such problems. The problem is
well-known to be of a paramount clinical importance, but yet, traditional
ConvNets cannot be applied due to the manifold structure of the data, neither
does the state-of-the-art geometric ConvNets perform well. Motivated by this,
we propose a new geometric ConvNet method named ZerNet, which builds upon our
novel mathematical generalization of convolution and pooling operations on
manifolds. Our study shows that the ZerNet outperforms the other
state-of-the-art geometric ConvNets in terms of accuracy.
Neural Tangent Kernel: Convergence and Generalization in Neural Networks
Authors: Arthur Jacot, Franck Gabriel, Clément Hongler
Date: 2018-06-20 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Gaussian Processes

At initialization, artificial neural networks (ANNs) are equivalent to Gaussian processes in the infinite-width limit, thus connecting them to kernel methods. We prove that the evolution of an ANN during training can also be described by a kernel: during gradient descent on the parameters of an ANN, the network function $f_\theta$ (which maps input vectors to output vectors) follows the kernel gradient of the functional cost (which is convex, in contrast to the parameter cost) w.r.t. a new kernel: the Neural Tangent Kernel (NTK). This kernel is central to describe the generalization features of ANNs. While the NTK is random at initialization and varies during training, in the infinite-width limit it converges to an explicit limiting kernel and it stays constant during training. This makes it possible to study the training of ANNs in function space instead of parameter space. Convergence of the training can then be related to the positive-definiteness of the limiting NTK. We prove the positive-definiteness of the limiting NTK when the data is supported on the sphere and the non-linearity is non-polynomial. We then focus on the setting of least-squares regression and show that in the infinite-width limit, the network function $f_\theta$ follows a linear differential equation during training. The convergence is fastest along the largest kernel principal components of the input data with respect to the NTK, hence suggesting a theoretical motivation for early stopping. Finally we study the NTK numerically, observe its behavior for wide networks, and compare it to the infinite-width limit.
A Distributed Second-Order Algorithm You Can Trust
Authors: Celestine Dünner, Aurelien Lucchi, Matilde Gargiani, An Bian, Thomas Hofmann, Martin Jaggi
Date: 2018-06-20 00:00:00
Proceeding: ICML 2018 7
Tasks: Distributed Optimization, Second-order methods

Due to the rapid growth of data and computational resources, distributed
optimization has become an active research area in recent years. While
first-order methods seem to dominate the field, second-order methods are
nevertheless attractive as they potentially require fewer communication rounds
to converge. However, there are significant drawbacks that impede their wide
adoption, such as the computation and the communication of a large Hessian
matrix. In this paper we present a new algorithm for distributed training of
generalized linear models that only requires the computation of diagonal blocks
of the Hessian matrix on the individual workers. To deal with this approximate
information we propose an adaptive approach that - akin to trust-region methods
- dynamically adapts the auxiliary model to compensate for modeling errors. We
provide theoretical rates of convergence for a wide class of problems including
L1-regularized objectives. We also demonstrate that our approach achieves
state-of-the-art results on multiple large benchmark datasets.
Doubly Nested Network for Resource-Efficient Inference
Authors: Jaehong Kim, Sungeun Hong, Yongseok Choi, Jiwon Kim
Date: 2018-06-20 00:00:00

We propose doubly nested network(DNNet) where all neurons represent their own
sub-models that solve the same task. Every sub-model is nested both layer-wise
and channel-wise. While nesting sub-models layer-wise is straight-forward with
deep-supervision as proposed in \cite{xie2015holistically}, channel-wise
nesting has not been explored in the literature to our best knowledge.
Channel-wise nesting is non-trivial as neurons between consecutive layers are
all connected to each other. In this work, we introduce a technique to solve
this problem by sorting channels topologically and connecting neurons
accordingly. For the purpose, channel-causal convolutions are used. Slicing
doubly nested network gives a working sub-network. The most notable application
of our proposed network structure with slicing operation is resource-efficient
inference. At test time, computing resources such as time and memory available
for running the prediction algorithm can significantly vary across devices and
applications. Given a budget constraint, we can slice the network accordingly
and use a sub-model for inference within budget, requiring no additional
computation such as training or fine-tuning after deployment. We demonstrate
the effectiveness of our approach in several practical scenarios of utilizing
available resource efficiently.
Locating Objects Without Bounding Boxes
Authors: Javier Ribera, David Güera, Yuhao Chen, Edward J. Delp
Date: 2018-06-20 00:00:00
Proceeding: CVPR 2019 6
Tasks: Object, Object Localization, Pupil Tracking

Recent advances in convolutional neural networks (CNN) have achieved
remarkable results in locating objects in images. In these networks, the
training procedure usually requires providing bounding boxes or the maximum
number of expected objects. In this paper, we address the task of estimating
object locations without annotated bounding boxes which are typically
hand-drawn and time consuming to label. We propose a loss function that can be
used in any fully convolutional network (FCN) to estimate object locations.
This loss function is a modification of the average Hausdorff distance between
two unordered sets of points. The proposed method has no notion of bounding
boxes, region proposals, or sliding windows. We evaluate our method with three
datasets designed to locate people's heads, pupil centers and plant centers. We
outperform state-of-the-art generic object detectors and methods fine-tuned for
pupil tracking.
Efficient inference in stochastic block models with vertex labels
Authors: Clara Stegehuis, Laurent Massoulié
Date: 2018-06-20 00:00:00
Tasks: Stochastic Block Model

We study the stochastic block model with two communities where vertices
contain side information in the form of a vertex label. These vertex labels may
have arbitrary label distributions, depending on the community memberships. We
analyze a linearized version of the popular belief propagation algorithm. We
show that this algorithm achieves the highest accuracy possible whenever a
certain function of the network parameters has a unique fixed point. Whenever
this function has multiple fixed points, the belief propagation algorithm may
not perform optimally. We show that increasing the information in the vertex
labels may reduce the number of fixed points and hence lead to optimality of
belief propagation.
Stagewise Safe Bayesian Optimization with Gaussian Processes
Authors: Yanan Sui, Vincent Zhuang, Joel W. Burdick, Yisong Yue
Date: 2018-06-20 00:00:00
Proceeding: ICML 2018 7
Tasks: Bayesian Optimization, Decision Making, Decision Making Under Uncertainty, Gaussian Processes, Sequential Decision Making

Enforcing safety is a key aspect of many problems pertaining to sequential decision making under uncertainty, which require the decisions made at every step to be both informative of the optimal decision and also safe. For example, we value both efficacy and comfort in medical therapy, and efficiency and safety in robotic control. We consider this problem of optimizing an unknown utility function with absolute feedback or preference feedback subject to unknown safety constraints. We develop an efficient safe Bayesian optimization algorithm, StageOpt, that separates safe region expansion and utility function maximization into two distinct stages. Compared to existing approaches which interleave between expansion and optimization, we show that StageOpt is more efficient and naturally applicable to a broader class of problems. We provide theoretical guarantees for both the satisfaction of safety constraints as well as convergence to the optimal utility value. We evaluate StageOpt on both a variety of synthetic experiments, as well as in clinical practice. We demonstrate that StageOpt is more effective than existing safe optimization approaches, and is able to safely and effectively optimize spinal cord stimulation therapy in our clinical experiments.
Automatic detection of lumen and media in the IVUS images using U-Net with VGG16 Encoder
Authors: Chirag Balakrishna, Sarshar Dadashzadeh, Sara Soltaninejad
Date: 2018-06-20 00:00:00

Coronary heart disease is one of the top rank leading cause of mortality in
the world which can be because of plaque burden inside the arteries.
Intravascular Ultrasound (IVUS) has been recognized as power- ful imaging
technology which captures the real time and high resolution images of the
coronary arteries and can be used for the analysis of these plaques. The IVUS
segmentation involves the extraction of two arterial walls components namely,
lumen and media. In this paper, we investi- gate the effectiveness of
Convolutional Neural Networks including U-Net to segment ultrasound scans of
arteries. In particular, the proposed seg- mentation network was built based on
the the U-Net with the VGG16 encoder. Experiments were done for evaluating the
proposed segmen- tation architecture which show promising quantitative and
qualitative results.
Interpretable to Whom? A Role-based Model for Analyzing Interpretable Machine Learning Systems
Authors: Richard Tomsett, Dave Braines, Dan Harborne, Alun Preece, Supriyo Chakraborty
Date: 2018-06-20 00:00:00
Tasks: BIG-bench Machine Learning, Interpretable Machine Learning, Relation

Several researchers have argued that a machine learning system's
interpretability should be defined in relation to a specific agent or task: we
should not ask if the system is interpretable, but to whom is it interpretable.
We describe a model intended to help answer this question, by identifying
different roles that agents can fulfill in relation to the machine learning
system. We illustrate the use of our model in a variety of scenarios, exploring
how an agent's role influences its goals, and the implications for defining
interpretability. Finally, we make suggestions for how our model could be
useful to interpretability researchers, system developers, and regulatory
bodies auditing machine learning systems.
Binary Ensemble Neural Network: More Bits per Network or More Networks per Bit?
Authors: Shilin Zhu, Xin Dong, Hao Su
Date: 2018-06-20 00:00:00
Proceeding: CVPR 2019 6

Binary neural networks (BNN) have been studied extensively since they run
dramatically faster at lower memory and power consumption than floating-point
networks, thanks to the efficiency of bit operations. However, contemporary
BNNs whose weights and activations are both single bits suffer from severe
accuracy degradation. To understand why, we investigate the representation
ability, speed and bias/variance of BNNs through extensive experiments. We
conclude that the error of BNNs is predominantly caused by the intrinsic
instability (training time) and non-robustness (train & test time). Inspired by
this investigation, we propose the Binary Ensemble Neural Network (BENN) which
leverages ensemble methods to improve the performance of BNNs with limited
efficiency cost. While ensemble techniques have been broadly believed to be
only marginally helpful for strong classifiers such as deep neural networks,
our analyses and experiments show that they are naturally a perfect fit to
boost BNNs. We find that our BENN, which is faster and much more robust than
state-of-the-art binary networks, can even surpass the accuracy of the
full-precision floating number network with the same architecture.
Towards Robust Interpretability with Self-Explaining Neural Networks
Authors: David Alvarez-Melis, Tommi S. Jaakkola
Date: 2018-06-20 00:00:00
Proceeding: NeurIPS 2018

Most recent work on interpretability of complex machine learning models has
focused on estimating $\textit{a posteriori}$ explanations for previously
trained models around specific predictions. $\textit{Self-explaining}$ models
where interpretability plays a key role already during learning have received
much less attention. We propose three desiderata for explanations in general --
explicitness, faithfulness, and stability -- and show that existing methods do
not satisfy them. In response, we design self-explaining models in stages,
progressively generalizing linear classifiers to complex yet architecturally
explicit models. Faithfulness and stability are enforced via regularization
specifically tailored to such models. Experimental results across various
benchmark datasets show that our framework offers a promising direction for
reconciling model complexity and interpretability.
An Asynchronous Distributed Expectation Maximization Algorithm For Massive Data: The DEM Algorithm
Authors: Sanvesh Srivastava, Glen DePalma, Chuanhai Liu
Date: 2018-06-20 00:00:00

The family of Expectation-Maximization (EM) algorithms provides a general
approach to fitting flexible models for large and complex data. The expectation
(E) step of EM-type algorithms is time-consuming in massive data applications
because it requires multiple passes through the full data. We address this
problem by proposing an asynchronous and distributed generalization of the EM
called the Distributed EM (DEM). Using DEM, existing EM-type algorithms are
easily extended to massive data settings by exploiting the divide-and-conquer
technique and widely available computing power, such as grid computing. The DEM
algorithm reserves two groups of computing processes called \emph{workers} and
\emph{managers} for performing the E step and the maximization step (M step),
respectively. The samples are randomly partitioned into a large number of
disjoint subsets and are stored on the worker processes. The E step of DEM
algorithm is performed in parallel on all the workers, and every worker
communicates its results to the managers at the end of local E step. The
managers perform the M step after they have received results from a
$\gamma$-fraction of the workers, where $\gamma$ is a fixed constant in $(0,
1]$. The sequence of parameter estimates generated by the DEM algorithm retains
the attractive properties of EM: convergence of the sequence of parameter
estimates to a local mode and linear global rate of convergence. Across diverse
simulations focused on linear mixed-effects models, the DEM algorithm is
significantly faster than competing EM-type algorithms while having a similar
accuracy. The DEM algorithm maintains its superior empirical performance on a
movie ratings database consisting of 10 million ratings.
Uncertainty in Multitask Transfer Learning
Authors: Alexandre Lacoste, Boris Oreshkin, Wonchang Chung, Thomas Boquet, Negar Rostamzadeh, David Krueger
Date: 2018-06-20 00:00:00
Proceeding: ICLR 2019 5
Tasks: Few-Shot Learning, Transfer Learning

Using variational Bayes neural networks, we develop an algorithm capable of
accumulating knowledge into a prior from multiple different tasks. The result
is a rich and meaningful prior capable of few-shot learning on new tasks. The
posterior can go beyond the mean field approximation and yields good
uncertainty on the performed experiments. Analysis on toy tasks shows that it
can learn from significantly different tasks while finding similarities among
them. Experiments of Mini-Imagenet yields the new state of the art with 74.5%
accuracy on 5 shot learning. Finally, we provide experiments showing that other
existing methods can fail to perform well in different benchmarks.
Fluid Annotation: A Human-Machine Collaboration Interface for Full Image Annotation
Authors: Mykhaylo Andriluka, Jasper R. R. Uijlings, Vittorio Ferrari
Date: 2018-06-20 00:00:00

We introduce Fluid Annotation, an intuitive human-machine collaboration
interface for annotating the class label and outline of every object and
background region in an image. Fluid annotation is based on three principles:
(I) Strong Machine-Learning aid. We start from the output of a strong neural
network model, which the annotator can edit by correcting the labels of
existing regions, adding new regions to cover missing objects, and removing
incorrect regions. The edit operations are also assisted by the model. (II)
Full image annotation in a single pass. As opposed to performing a series of
small annotation tasks in isolation, we propose a unified interface for full
image annotation in a single pass. (III) Empower the annotator. We empower the
annotator to choose what to annotate and in which order. This enables
concentrating on what the machine does not already know, i.e. putting human
effort only on the errors it made. This helps using the annotation budget
effectively. Through extensive experiments on the COCO+Stuff dataset, we
demonstrate that Fluid Annotation leads to accurate annotations very
efficiently, taking three times less annotation time than the popular LabelMe
interface.
A Simple Fusion of Deep and Shallow Learning for Acoustic Scene Classification
Authors: Eduardo Fonseca, Rong Gong, Xavier Serra
Date: 2018-06-19 00:00:00
Tasks: Acoustic Scene Classification, Classification, Feature Engineering, General Classification, Scene Classification

In the past, Acoustic Scene Classification systems have been based on hand
crafting audio features that are input to a classifier. Nowadays, the common
trend is to adopt data driven techniques, e.g., deep learning, where audio
representations are learned from data. In this paper, we propose a system that
consists of a simple fusion of two methods of the aforementioned types: a deep
learning approach where log-scaled mel-spectrograms are input to a
convolutional neural network, and a feature engineering approach, where a
collection of hand-crafted features is input to a gradient boosting machine. We
first show that both methods provide complementary information to some extent.
Then, we use a simple late fusion strategy to combine both methods. We report
classification accuracy of each method individually and the combined system on
the TUT Acoustic Scenes 2017 dataset. The proposed fused system outperforms
each of the individual methods and attains a classification accuracy of 72.8%
on the evaluation set, improving the baseline system by 11.8%.
A Latent Variable Approach to Gaussian Process Modeling with Qualitative and Quantitative Factors
Authors: Yichi Zhang, Siyu Tao, Wei Chen, Daniel W. Apley
Date: 2018-06-19 00:00:00
Tasks: Dimensionality Reduction

Computer simulations often involve both qualitative and numerical inputs.
Existing Gaussian process (GP) methods for handling this mainly assume a
different response surface for each combination of levels of the qualitative
factors and relate them via a multiresponse cross-covariance matrix. We
introduce a substantially different approach that maps each qualitative factor
to an underlying numerical latent variable (LV), with the mapped value for each
level estimated similarly to the correlation parameters. This provides a
parsimonious GP parameterization that treats qualitative factors the same as
numerical variables and views them as effecting the response via similar
physical mechanisms. This has strong physical justification, as the effects of
a qualitative factor in any physics-based simulation model must always be due
to some underlying numerical variables. Even when the underlying variables are
many, sufficient dimension reduction arguments imply that their effects can be
represented by a low-dimensional LV. This conjecture is supported by the
superior predictive performance observed across a variety of examples.
Moreover, the mapped LVs provide substantial insight into the nature and
effects of the qualitative factors.
Defining Locality for Surrogates in Post-hoc Interpretablity
Authors: Thibault Laugel, Xavier Renard, Marie-Jeanne Lesot, Christophe Marsala, Marcin Detyniecki
Date: 2018-06-19 00:00:00
Tasks: Prediction

Local surrogate models, to approximate the local decision boundary of a
black-box classifier, constitute one approach to generate explanations for the
rationale behind an individual prediction made by the back-box. This paper
highlights the importance of defining the right locality, the neighborhood on
which a local surrogate is trained, in order to approximate accurately the
local black-box decision boundary. Unfortunately, as shown in this paper, this
issue is not only a parameter or sampling distribution challenge and has a
major impact on the relevance and quality of the approximation of the local
black-box decision boundary and thus on the meaning and accuracy of the
generated explanation. To overcome the identified problems, quantified with an
adapted measure and procedure, we propose to generate surrogate-based
explanations for individual predictions based on a sampling centered on
particular place of the decision boundary, relevant for the prediction to be
explained, rather than on the prediction itself as it is classically done. We
evaluate the novel approach compared to state-of-the-art methods and a
straightforward improvement thereof on four UCI datasets.