pwc-archive/papers-with-abstracts huggingface dataset:
Faster gaze prediction with dense networks and Fisher pruning
Authors: Lucas Theis, Iryna Korshunova, Alykhan Tejani, Ferenc Huszár
Date: 2018-01-17 00:00:00
Proceeding: Twitter 2018 1
Tasks: Gaze Estimation, Gaze Prediction, Knowledge Distillation, Object Recognition, Prediction, Saliency Prediction

Predicting human fixations from images has recently seen large improvements
by leveraging deep representations which were pretrained for object
recognition. However, as we show in this paper, these networks are highly
overparameterized for the task of fixation prediction. We first present a
simple yet principled greedy pruning method which we call Fisher pruning.
Through a combination of knowledge distillation and Fisher pruning, we obtain
much more runtime-efficient architectures for saliency prediction, achieving a
10x speedup for the same AUC performance as a state of the art network on the
CAT2000 dataset. Speeding up single-image gaze prediction is important for many
real-world applications, but it is also a crucial step in the development of
video saliency models, where the amount of data to be processed is
substantially larger.
Estimating the error variance in a high-dimensional linear model
Authors: Guo Yu, Jacob Bien
Date: 2017-12-06 00:00:00
Tasks: Vocal Bursts Intensity Prediction

The lasso has been studied extensively as a tool for estimating the coefficient vector in the high-dimensional linear model; however, considerably less is known about estimating the error variance in this context. In this paper, we propose the natural lasso estimator for the error variance, which maximizes a penalized likelihood objective. A key aspect of the natural lasso is that the likelihood is expressed in terms of the natural parameterization of the multiparameter exponential family of a Gaussian with unknown mean and variance. The result is a remarkably simple estimator of the error variance with provably good performance in terms of mean squared error. These theoretical results do not require placing any assumptions on the design matrix or the true regression coefficients. We also propose a companion estimator, called the organic lasso, which theoretically does not require tuning of the regularization parameter. Both estimators do well empirically compared to preexisting methods, especially in settings where successful recovery of the true support of the coefficient vector is hard. Finally, we show that existing methods can do well under fewer assumptions than previously known, thus providing a fuller story about the problem of estimating the error variance in high-dimensional linear models.
Ranking Data with Continuous Labels through Oriented Recursive Partitions
Authors: Stephan Clémençon, Mastane Achab
Date: 2018-01-17 00:00:00
Proceeding: NeurIPS 2017 12

We formulate a supervised learning problem, referred to as continuous
ranking, where a continuous real-valued label Y is assigned to an observable
r.v. X taking its values in a feature space $\mathcal{X}$ and the goal is to
order all possible observations x in $\mathcal{X}$ by means of a scoring
function $s:\mathcal{X}\rightarrow \mathbb{R}$ so that s(X) and Y tend to
increase or decrease together with highest probability. This problem
generalizes bi/multi-partite ranking to a certain extent and the task of
finding optimal scoring functions s(x) can be naturally cast as optimization of
a dedicated functional criterion, called the IROC curve here, or as
maximization of the Kendall ${\tau}$ related to the pair (s(X), Y ). From the
theoretical side, we describe the optimal elements of this problem and provide
statistical guarantees for empirical Kendall ${\tau}$ maximization under
appropriate conditions for the class of scoring function candidates. We also
propose a recursive statistical learning algorithm tailored to empirical IROC
curve optimization and producing a piecewise constant scoring function that is
fully described by an oriented binary tree. Preliminary numerical experiments
highlight the difference in nature between regression and continuous ranking
and provide strong empirical evidence of the performance of empirical
optimizers of the criteria proposed.
Poisson multi-Bernoulli mixture filter: direct derivation and implementation
Authors: Ángel F. García-Fernández, Jason L. Williams, Karl Granström, Lennart Svensson
Date: 2017-03-13 00:00:00

We provide a derivation of the Poisson multi-Bernoulli mixture (PMBM) filter
for multi-target tracking with the standard point target measurements without
using probability generating functionals or functional derivatives. We also
establish the connection with the \delta-generalised labelled multi-Bernoulli
(\delta-GLMB) filter, showing that a \delta-GLMB density represents a
multi-Bernoulli mixture with labelled targets so it can be seen as a special
case of PMBM. In addition, we propose an implementation for linear/Gaussian
dynamic and measurement models and how to efficiently obtain typical estimators
in the literature from the PMBM. The PMBM filter is shown to outperform other
filters in the literature in a challenging scenario.
Experience-driven Networking: A Deep Reinforcement Learning based Approach
Authors: Zhiyuan Xu, Jian Tang, Jingsong Meng, Weiyi Zhang, Yanzhi Wang, Chi Harold Liu, Dejun Yang
Date: 2018-01-17 00:00:00
Tasks: continuous-control, Continuous Control, Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Modern communication networks have become very complicated and highly
dynamic, which makes them hard to model, predict and control. In this paper, we
develop a novel experience-driven approach that can learn to well control a
communication network from its own experience rather than an accurate
mathematical model, just as a human learns a new skill (such as driving,
swimming, etc). Specifically, we, for the first time, propose to leverage
emerging Deep Reinforcement Learning (DRL) for enabling model-free control in
communication networks; and present a novel and highly effective DRL-based
control framework, DRL-TE, for a fundamental networking problem: Traffic
Engineering (TE). The proposed framework maximizes a widely-used utility
function by jointly learning network environment and its dynamics, and making
decisions under the guidance of powerful Deep Neural Networks (DNNs). We
propose two new techniques, TE-aware exploration and actor-critic-based
prioritized experience replay, to optimize the general DRL framework
particularly for TE. To validate and evaluate the proposed framework, we
implemented it in ns-3, and tested it comprehensively with both representative
and randomly generated network topologies. Extensive packet-level simulation
results show that 1) compared to several widely-used baseline methods, DRL-TE
significantly reduces end-to-end delay and consistently improves the network
utility, while offering better or comparable throughput; 2) DRL-TE is robust to
network changes; and 3) DRL-TE consistently outperforms a state-ofthe-art DRL
method (for continuous control), Deep Deterministic Policy Gradient (DDPG),
which, however, does not offer satisfying performance.
Seismic-Net: A Deep Densely Connected Neural Network to Detect Seismic Events
Authors: Yue Wu, Youzuo Lin, Zheng Zhou, Andrew Delorey
Date: 2018-01-17 00:00:00
Tasks: Event Detection, Time Series, Time Series Analysis

One of the risks of large-scale geologic carbon sequestration is the
potential migration of fluids out of the storage formations. Accurate and fast
detection of this fluids migration is not only important but also challenging,
due to the large subsurface uncertainty and complex governing physics.
Traditional leakage detection and monitoring techniques rely on geophysical
observations including seismic. However, the resulting accuracy of these
methods is limited because of indirect information they provide requiring
expert interpretation, therefore yielding in-accurate estimates of leakage
rates and locations. In this work, we develop a novel machine-learning
detection package, named "Seismic-Net", which is based on the deep densely
connected neural network. To validate the performance of our proposed leakage
detection method, we employ our method to a natural analog site at Chimay\'o,
New Mexico. The seismic events in the data sets are generated because of the
eruptions of geysers, which is due to the leakage of $\mathrm{CO}_\mathrm{2}$.
In particular, we demonstrate the efficacy of our Seismic-Net by formulating
our detection problem as an event detection problem with time series data. A
fixed-length window is slid throughout the time series data and we build a deep
densely connected network to classify each window to determine if a geyser
event is included. Through our numerical tests, we show that our model achieves
precision/recall as high as 0.889/0.923. Therefore, our Seismic-Net has a great
potential for detection of $\mathrm{CO}_\mathrm{2}$ leakage.
TernausNet: U-Net with VGG11 Encoder Pre-Trained on ImageNet for Image Segmentation
Authors: Vladimir Iglovikov, Alexey Shvets
Date: 2018-01-17 00:00:00
Tasks: Image Segmentation, Segmentation, Semantic Segmentation

Pixel-wise image segmentation is demanding task in computer vision. Classical
U-Net architectures composed of encoders and decoders are very popular for
segmentation of medical images, satellite images etc. Typically, neural network
initialized with weights from a network pre-trained on a large data set like
ImageNet shows better performance than those trained from scratch on a small
dataset. In some practical applications, particularly in medicine and traffic
safety, the accuracy of the models is of utmost importance. In this paper, we
demonstrate how the U-Net type architecture can be improved by the use of the
pre-trained encoder. Our code and corresponding pre-trained weights are
publicly available at https://github.com/ternaus/TernausNet. We compare three
weight initialization schemes: LeCun uniform, the encoder with weights from
VGG11 and full network trained on the Carvana dataset. This network
architecture was a part of the winning solution (1st out of 735) in the Kaggle:
Carvana Image Masking Challenge.
In-network Neural Networks
Authors: Giuseppe Siracusano, Roberto Bifulco
Date: 2018-01-17 00:00:00

We present N2Net, a system that implements binary neural networks using
commodity switching chips deployed in network switches and routers. Our system
shows that these devices can run simple neural network models, whose input is
encoded in the network packets' header, at packet processing speeds (billions
of packets per second). Furthermore, our experience highlights that switching
chips could support even more complex models, provided that some minor and
cheap modifications to the chip's design are applied. We believe N2Net provides
an interesting building block for future end-to-end networked systems.
Deep Learning: An Introduction for Applied Mathematicians
Authors: Catherine F. Higham, Desmond J. Higham
Date: 2018-01-17 00:00:00
Tasks: Deep Learning, image-classification, Image Classification

Multilayered artificial neural networks are becoming a pervasive tool in a
host of application fields. At the heart of this deep learning revolution are
familiar concepts from applied and computational mathematics; notably, in
calculus, approximation theory, optimization and linear algebra. This article
provides a very brief introduction to the basic ideas that underlie deep
learning from an applied mathematics perspective. Our target audience includes
postgraduate and final year undergraduate students in mathematics who are keen
to learn about the area. The article may also be useful for instructors in
mathematics who wish to enliven their classes with references to the
application of deep learning techniques. We focus on three fundamental
questions: what is a deep neural network? how is a network trained? what is the
stochastic gradient method? We illustrate the ideas with a short MATLAB code
that sets up and trains a network. We also show the use of state-of-the art
software on a large scale image classification problem. We finish with
references to the current literature.
Learning Combinations of Sigmoids Through Gradient Estimation
Authors: Stratis Ioannidis, Andrea Montanari
Date: 2017-08-22 00:00:00
Tasks: regression

We develop a new approach to learn the parameters of regression models with
hidden variables. In a nutshell, we estimate the gradient of the regression
function at a set of random points, and cluster the estimated gradients. The
centers of the clusters are used as estimates for the parameters of hidden
units. We justify this approach by studying a toy model, whereby the regression
function is a linear combination of sigmoids. We prove that indeed the
estimated gradients concentrate around the parameter vectors of the hidden
units, and provide non-asymptotic bounds on the number of required samples. To
the best of our knowledge, no comparable guarantees have been proven for linear
combinations of sigmoids.
The QLBS Q-Learner Goes NuQLear: Fitted Q Iteration, Inverse RL, and Option Portfolios
Authors: Igor Halperin
Date: 2018-01-17 00:00:00
Tasks: Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

The QLBS model is a discrete-time option hedging and pricing model that is
based on Dynamic Programming (DP) and Reinforcement Learning (RL). It combines
the famous Q-Learning method for RL with the Black-Scholes (-Merton) model's
idea of reducing the problem of option pricing and hedging to the problem of
optimal rebalancing of a dynamic replicating portfolio for the option, which is
made of a stock and cash. Here we expand on several NuQLear (Numerical
Q-Learning) topics with the QLBS model. First, we investigate the performance
of Fitted Q Iteration for a RL (data-driven) solution to the model, and
benchmark it versus a DP (model-based) solution, as well as versus the BSM
model. Second, we develop an Inverse Reinforcement Learning (IRL) setting for
the model, where we only observe prices and actions (re-hedges) taken by a
trader, but not rewards. Third, we outline how the QLBS model can be used for
pricing portfolios of options, rather than a single option in isolation, thus
providing its own, data-driven and model independent solution to the (in)famous
volatility smile problem of the Black-Scholes model.
A modified fuzzy C means algorithm for shading correction in craniofacial CBCT images
Authors: Awais Ashfaq, Jonas Adler
Date: 2018-01-17 00:00:00

CBCT images suffer from acute shading artifacts primarily due to scatter.
Numerous image-domain correction algorithms have been proposed in the
literature that use patient-specific planning CT images to estimate shading
contributions in CBCT images. However, in the context of radiosurgery
applications such as gamma knife, planning images are often acquired through
MRI which impedes the use of polynomial fitting approaches for shading
correction. We present a new shading correction approach that is independent of
planning CT images. Our algorithm is based on the assumption that true CBCT
images follow a uniform volumetric intensity distribution per material, and
scatter perturbs this uniform texture by contributing cupping and shading
artifacts in the image domain. The framework is a combination of fuzzy C-means
coupled with a neighborhood regularization term and Otsu's method. Experimental
results on artificially simulated craniofacial CBCT images are provided to
demonstrate the effectiveness of our algorithm. Spatial non-uniformity is
reduced from 16% to 7% in soft tissue and from 44% to 8% in bone regions. With
shading-correction, thresholding based segmentation accuracy for bone pixels is
improved from 85% to 91% when compared to thresholding without
shading-correction. The proposed algorithm is thus practical and qualifies as a
plug and play extension into any CBCT reconstruction software for shading
correction.
Face Recognition via Centralized Coordinate Learning
Authors: Xianbiao Qi, Lei Zhang
Date: 2018-01-17 00:00:00
Tasks: Classification, Face Recognition, General Classification

Owe to the rapid development of deep neural network (DNN) techniques and the
emergence of large scale face databases, face recognition has achieved a great
success in recent years. During the training process of DNN, the face features
and classification vectors to be learned will interact with each other, while
the distribution of face features will largely affect the convergence status of
network and the face similarity computing in test stage. In this work, we
formulate jointly the learning of face features and classification vectors, and
propose a simple yet effective centralized coordinate learning (CCL) method,
which enforces the features to be dispersedly spanned in the coordinate space
while ensuring the classification vectors to lie on a hypersphere. An adaptive
angular margin is further proposed to enhance the discrimination capability of
face features. Extensive experiments are conducted on six face benchmarks,
including those have large age gap and hard negative samples. Trained only on
the small-scale CASIA Webface dataset with 460K face images from about 10K
subjects, our CCL model demonstrates high effectiveness and generality, showing
consistently competitive performance across all the six benchmark databases.
Innateness, AlphaZero, and Artificial Intelligence
Authors: Gary Marcus
Date: 2018-01-17 00:00:00

The concept of innateness is rarely discussed in the context of artificial
intelligence. When it is discussed, or hinted at, it is often the context of
trying to reduce the amount of innate machinery in a given system. In this
paper, I consider as a test case a recent series of papers by Silver et al
(Silver et al., 2017a) on AlphaGo and its successors that have been presented
as an argument that a "even in the most challenging of domains: it is possible
to train to superhuman level, without human examples or guidance", "starting
tabula rasa."
  I argue that these claims are overstated, for multiple reasons. I close by
arguing that artificial intelligence needs greater attention to innateness, and
I point to some proposals about what that innateness might look like.
MR image reconstruction using deep density priors
Authors: Kerem C. Tezcan, Christian F. Baumgartner, Roger Luechinger, Klaas P. Pruessmann, Ender Konukoglu
Date: 2017-11-30 00:00:00
Tasks: Density Estimation, Image Reconstruction

Algorithms for Magnetic Resonance (MR) image reconstruction from undersampled
measurements exploit prior information to compensate for missing k-space data.
Deep learning (DL) provides a powerful framework for extracting such
information from existing image datasets, through learning, and then using it
for reconstruction. Leveraging this, recent methods employed DL to learn
mappings from undersampled to fully sampled images using paired datasets,
including undersampled and corresponding fully sampled images, integrating
prior knowledge implicitly. In this article, we propose an alternative approach
that learns the probability distribution of fully sampled MR images using
unsupervised DL, specifically Variational Autoencoders (VAE), and use this as
an explicit prior term in reconstruction, completely decoupling the encoding
operation from the prior. The resulting reconstruction algorithm enjoys a
powerful image prior to compensate for missing k-space data without requiring
paired datasets for training nor being prone to associated sensitivities, such
as deviations in undersampling patterns used in training and test time or coil
settings. We evaluated the proposed method with T1 weighted images from a
publicly available dataset, multi-coil complex images acquired from healthy
volunteers (N=8) and images with white matter lesions. The proposed algorithm,
using the VAE prior, produced visually high quality reconstructions and
achieved low RMSE values, outperforming most of the alternative methods on the
same dataset. On multi-coil complex data, the algorithm yielded accurate
magnitude and phase reconstruction results. In the experiments on images with
white matter lesions, the method faithfully reconstructed the lesions.
  Keywords: Reconstruction, MRI, prior probability, machine learning, deep
learning, unsupervised learning, density estimation
MORF: A Framework for Predictive Modeling and Replication At Scale With Privacy-Restricted MOOC Data
Authors: Josh Gardner, Christopher Brooks, Juan Miguel L. Andres, Ryan Baker
Date: 2018-01-16 00:00:00

Big data repositories from online learning platforms such as Massive Open
Online Courses (MOOCs) represent an unprecedented opportunity to advance
research on education at scale and impact a global population of learners. To
date, such research has been hindered by poor reproducibility and a lack of
replication, largely due to three types of barriers: experimental, inferential,
and data. We present a novel system for large-scale computational research, the
MOOC Replication Framework (MORF), to jointly address these barriers. We
discuss MORF's architecture, an open-source platform-as-a-service (PaaS) which
includes a simple, flexible software API providing for multiple modes of
research (predictive modeling or production rule analysis) integrated with a
high-performance computing environment. All experiments conducted on MORF use
executable Docker containers which ensure complete reproducibility while
allowing for the use of any software or language which can be installed in the
linux-based Docker container. Each experimental artifact is assigned a DOI and
made publicly available. MORF has the potential to accelerate and democratize
research on its massive data repository, which currently includes over 200
MOOCs, as demonstrated by initial research conducted on the platform. We also
highlight ways in which MORF represents a solution template to a more general
class of problems faced by computational researchers in other domains.
Individual Recognition in Schizophrenia using Deep Learning Methods with Random Forest and Voting Classifiers: Insights from Resting State EEG Streams
Authors: Lei Chu, Robert Qiu, Haichun Liu, Zenan Ling, Tianhong Zhang, Jijun Wang
Date: 2017-06-20 00:00:00
Tasks: Classification, Deep Learning, EEG, Electroencephalogram (EEG), General Classification

Recently, there has been a growing interest in monitoring brain activity for
individual recognition system. So far these works are mainly focussing on
single channel data or fragment data collected by some advanced brain
monitoring modalities. In this study we propose new individual recognition
schemes based on spatio-temporal resting state Electroencephalography (EEG)
data. Besides, instead of using features derived from artificially-designed
procedures, modified deep learning architectures which aim to automatically
extract an individual's unique features are developed to conduct
classification. Our designed deep learning frameworks are proved of a small but
consistent advantage of replacing the $softmax$ layer with Random Forest.
Additionally, a voting layer is added at the top of designed neural networks in
order to tackle the classification problem arisen from EEG streams. Lastly,
various experiments are implemented to evaluate the performance of the designed
deep learning architectures; Results indicate that the proposed EEG-based
individual recognition scheme yields a high degree of classification accuracy:
$81.6\%$ for characteristics in high risk (CHR) individuals, $96.7\%$ for
clinically stable first episode patients with schizophrenia (FES) and $99.2\%$
for healthy controls (HC).