pwc-archive/papers-with-abstracts huggingface dataset:
Adversarial Time-to-Event Modeling
Authors: Paidamoyo Chapfuwa, Chenyang Tao, Chunyuan Li, Courtney Page, Benjamin Goldstein, Lawrence Carin, Ricardo Henao
Date: 2018-04-09 00:00:00
Proceeding: ICML 2018 7
Tasks: Survival Analysis

Modern health data science applications leverage abundant molecular and
electronic health data, providing opportunities for machine learning to build
statistical models to support clinical practice. Time-to-event analysis, also
called survival analysis, stands as one of the most representative examples of
such statistical models. We present a deep-network-based approach that
leverages adversarial learning to address a key challenge in modern
time-to-event modeling: nonparametric estimation of event-time distributions.
We also introduce a principled cost function to exploit information from
censored events (events that occur subsequent to the observation window).
Unlike most time-to-event models, we focus on the estimation of time-to-event
distributions, rather than time ordering. We validate our model on both
benchmark and real datasets, demonstrating that the proposed formulation yields
significant performance gains relative to a parametric alternative, which we
also propose.
Convolutional Imputation of Matrix Networks
Authors: Qingyun Sun, Mengyuan Yan David Donoho, Stephen Boyd
Date: 2016-06-02 00:00:00
Proceeding: ICML 2018 7
Tasks: Imputation

A matrix network is a family of matrices, with relatedness modeled by a
weighted graph. We consider the task of completing a partially observed matrix
network. We assume a novel sampling scheme where a fraction of matrices might
be completely unobserved. How can we recover the entire matrix network from
incomplete observations? This mathematical problem arises in many applications
including medical imaging and social networks.
  To recover the matrix network, we propose a structural assumption that the
matrices have a graph Fourier transform which is low-rank. We formulate a
convex optimization problem and prove an exact recovery guarantee for the
optimization problem. Furthermore, we numerically characterize the exact
recovery regime for varying rank and sampling rate and discover a new phase
transition phenomenon. Then we give an iterative imputation algorithm to
efficiently solve the optimization problem and complete large scale matrix
networks. We demonstrate the algorithm with a variety of applications such as
MRI and Facebook user network.
Modeling 4D fMRI Data via Spatio-Temporal Convolutional Neural Networks (ST-CNN)
Authors: Yu Zhao, Xiang Li, Wei zhang, Shijie Zhao, Milad Makkie, Mo Zhang, Quanzheng Li, Tianming Liu
Date: 2018-05-31 00:00:00
Tasks: Brain Decoding, Medical Image Analysis, Network Identification

Simultaneous modeling of the spatio-temporal variation patterns of brain
functional network from 4D fMRI data has been an important yet challenging
problem for the field of cognitive neuroscience and medical image analysis.
Inspired by the recent success in applying deep learning for functional brain
decoding and encoding, in this work we propose a spatio-temporal convolutional
neural network (ST-CNN)to jointly learn the spatial and temporal patterns of
targeted network from the training data and perform automatic, pin-pointing
functional network identification. The proposed ST-CNN is evaluated by the task
of identifying the Default Mode Network (DMN) from fMRI data. Results show that
while the framework is only trained on one fMRI dataset,it has the sufficient
generalizability to identify the DMN from different populations of data as well
as different cognitive tasks. Further investigation into the results show that
the superior performance of ST-CNN is driven by the jointly-learning scheme,
which capture the intrinsic relationship between the spatial and temporal
characteristic of DMN and ensures the accurate identification.
Probabilistic FastText for Multi-Sense Word Embeddings
Authors: Ben Athiwaratkun, Andrew Gordon Wilson, Anima Anandkumar
Date: 2018-06-07 00:00:00
Proceeding: ACL 2018 7
Tasks: Word Embeddings, Word Similarity

We introduce Probabilistic FastText, a new model for word embeddings that can
capture multiple word senses, sub-word structure, and uncertainty information.
In particular, we represent each word with a Gaussian mixture density, where
the mean of a mixture component is given by the sum of n-grams. This
representation allows the model to share statistical strength across sub-word
structures (e.g. Latin roots), producing accurate representations of rare,
misspelt, or even unseen words. Moreover, each component of the mixture can
capture a different word sense. Probabilistic FastText outperforms both
FastText, which has no probabilistic model, and dictionary-level probabilistic
embeddings, which do not incorporate subword structures, on several
word-similarity benchmarks, including English RareWord and foreign language
datasets. We also achieve state-of-art performance on benchmarks that measure
ability to discern different meanings. Thus, the proposed model is the first to
achieve multi-sense representations while having enriched semantics on rare
words.
Training Faster by Separating Modes of Variation in Batch-normalized Models
Authors: Mahdi M. Kalayeh, Mubarak Shah
Date: 2018-06-07 00:00:00
Tasks: image-classification, Image Classification

Batch Normalization (BN) is essential to effectively train state-of-the-art
deep Convolutional Neural Networks (CNN). It normalizes inputs to the layers
during training using the statistics of each mini-batch. In this work, we study
BN from the viewpoint of Fisher kernels. We show that assuming samples within a
mini-batch are from the same probability density function, then BN is identical
to the Fisher vector of a Gaussian distribution. That means BN can be explained
in terms of kernels that naturally emerge from the probability density function
of the underlying data distribution. However, given the rectifying
non-linearities employed in CNN architectures, distribution of inputs to the
layers show heavy tail and asymmetric characteristics. Therefore, we propose
approximating underlying data distribution not with one, but a mixture of
Gaussian densities. Deriving Fisher vector for a Gaussian Mixture Model (GMM),
reveals that BN can be improved by independently normalizing with respect to
the statistics of disentangled sub-populations. We refer to our proposed soft
piecewise version of BN as Mixture Normalization (MN). Through extensive set of
experiments on CIFAR-10 and CIFAR-100, we show that MN not only effectively
accelerates training image classification and Generative Adversarial networks,
but also reaches higher quality models.
Revisiting the Importance of Individual Units in CNNs via Ablation
Authors: Bolei Zhou, Yiyou Sun, David Bau, Antonio Torralba
Date: 2018-06-07 00:00:00
Tasks: General Classification

We revisit the importance of the individual units in Convolutional Neural
Networks (CNNs) for visual recognition. By conducting unit ablation experiments
on CNNs trained on large scale image datasets, we demonstrate that, though
ablating any individual unit does not hurt overall classification accuracy, it
does lead to significant damage on the accuracy of specific classes. This
result shows that an individual unit is specialized to encode information
relevant to a subset of classes. We compute the correlation between the
accuracy drop under unit ablation and various attributes of an individual unit
such as class selectivity and weight L1 norm. We confirm that unit attributes
such as class selectivity are a poor predictor for impact on overall accuracy
as found previously in recent work \cite{morcos2018importance}. However, our
results show that class selectivity along with other attributes are good
predictors of the importance of one unit to individual classes. We evaluate the
impact of random rotation, batch normalization, and dropout to the importance
of units to specific classes. Our results show that units with high selectivity
play an important role in network classification power at the individual class
level. Understanding and interpreting the behavior of these units is necessary
and meaningful.
Inertial Odometry on Handheld Smartphones
Authors: Arno Solin, Santiago Cortes, Esa Rahtu, Juho Kannala
Date: 2017-03-01 00:00:00

Building a complete inertial navigation system using the limited quality data
provided by current smartphones has been regarded challenging, if not
impossible. This paper shows that by careful crafting and accounting for the
weak information in the sensor samples, smartphones are capable of pure
inertial navigation. We present a probabilistic approach for orientation and
use-case free inertial odometry, which is based on double-integrating rotated
accelerations. The strength of the model is in learning additive and
multiplicative IMU biases online. We are able to track the phone position,
velocity, and pose in real-time and in a computationally lightweight fashion by
solving the inference with an extended Kalman filter. The information fusion is
completed with zero-velocity updates (if the phone remains stationary),
altitude correction from barometric pressure readings (if available), and
pseudo-updates constraining the momentary speed. We demonstrate our approach
using an iPad and iPhone in several indoor dead-reckoning applications and in a
measurement tool setup.
Real-time coherent diffraction inversion using deep generative networks
Authors: Mathew J. Cherukara, Youssef S. G. Nashed, Ross J. Harder
Date: 2018-06-07 00:00:00
Tasks: Retrieval

Phase retrieval, or the process of recovering phase information in reciprocal
space to reconstruct images from measured intensity alone, is the underlying
basis to a variety of imaging applications including coherent diffraction
imaging (CDI). Typical phase retrieval algorithms are iterative in nature, and
hence, are time-consuming and computationally expensive, precluding real-time
imaging. Furthermore, iterative phase retrieval algorithms struggle to converge
to the correct solution especially in the presence of strong phase structures.
In this work, we demonstrate the training and testing of CDI NN, a pair of deep
deconvolutional networks trained to predict structure and phase in real space
of a 2D object from its corresponding far-field diffraction intensities alone.
Once trained, CDI NN can invert a diffraction pattern to an image within a few
milliseconds of compute time on a standard desktop machine, opening the door to
real-time imaging.
Generating Artificial Data for Private Deep Learning
Authors: Aleksei Triastcyn, Boi Faltings
Date: 2018-03-08 00:00:00
Tasks: BIG-bench Machine Learning, Deep Learning, Generative Adversarial Network, Privacy Preserving

In this paper, we propose generating artificial data that retain statistical
properties of real data as the means of providing privacy with respect to the
original dataset. We use generative adversarial network to draw
privacy-preserving artificial data samples and derive an empirical method to
assess the risk of information disclosure in a differential-privacy-like way.
Our experiments show that we are able to generate artificial data of high
quality and successfully train and validate machine learning models on this
data while limiting potential privacy loss.
Constant-Time Predictive Distributions for Gaussian Processes
Authors: Geoff Pleiss, Jacob R. Gardner, Kilian Q. Weinberger, Andrew Gordon Wilson
Date: 2018-03-16 00:00:00
Proceeding: ICML 2018 7
Tasks: Gaussian Processes, regression

One of the most compelling features of Gaussian process (GP) regression is
its ability to provide well-calibrated posterior distributions. Recent advances
in inducing point methods have sped up GP marginal likelihood and posterior
mean computations, leaving posterior covariance estimation and sampling as the
remaining computational bottlenecks. In this paper we address these
shortcomings by using the Lanczos algorithm to rapidly approximate the
predictive covariance matrix. Our approach, which we refer to as LOVE (LanczOs
Variance Estimates), substantially improves time and space complexity. In our
experiments, LOVE computes covariances up to 2,000 times faster and draws
samples 18,000 times faster than existing methods, all without sacrificing
accuracy.
Correspondence of Deep Neural Networks and the Brain for Visual Textures
Authors: Md Nasir Uddin Laskar, Luis G. Sanchez Giraldo, Odelia Schwartz
Date: 2018-06-07 00:00:00
Tasks: Sensitivity

Deep convolutional neural networks (CNNs) trained on objects and scenes have
shown intriguing ability to predict some response properties of visual cortical
neurons. However, the factors and computations that give rise to such ability,
and the role of intermediate processing stages in explaining changes that
develop across areas of the cortical hierarchy, are poorly understood. We
focused on the sensitivity to textures as a paradigmatic example, since recent
neurophysiology experiments provide rich data pointing to texture sensitivity
in secondary but not primary visual cortex. We developed a quantitative
approach for selecting a subset of the neural unit population from the CNN that
best describes the brain neural recordings. We found that the first two layers
of the CNN showed qualitative and quantitative correspondence to the cortical
data across a number of metrics. This compatibility was reduced for the
architecture alone rather than the learned weights, for some other related
hierarchical models, and only mildly in the absence of a nonlinear computation
akin to local divisive normalization. Our results show that the CNN class of
model is effective for capturing changes that develop across early areas of
cortex, and has the potential to facilitate understanding of the computations
that give rise to hierarchical processing in the brain.
Residual Unfairness in Fair Machine Learning from Prejudiced Data
Authors: Nathan Kallus, Angela Zhou
Date: 2018-06-07 00:00:00
Proceeding: ICML 2018 7
Tasks: BIG-bench Machine Learning, Fairness

Recent work in fairness in machine learning has proposed adjusting for
fairness by equalizing accuracy metrics across groups and has also studied how
datasets affected by historical prejudices may lead to unfair decision
policies. We connect these lines of work and study the residual unfairness that
arises when a fairness-adjusted predictor is not actually fair on the target
population due to systematic censoring of training data by existing biased
policies. This scenario is particularly common in the same applications where
fairness is a concern. We characterize theoretically the impact of such
censoring on standard fairness metrics for binary classifiers and provide
criteria for when residual unfairness may or may not appear. We prove that,
under certain conditions, fairness-adjusted classifiers will in fact induce
residual unfairness that perpetuates the same injustices, against the same
groups, that biased the data to begin with, thus showing that even
state-of-the-art fair machine learning can have a "bias in, bias out" property.
When certain benchmark data is available, we show how sample reweighting can
estimate and adjust fairness metrics while accounting for censoring. We use
this to study the case of Stop, Question, and Frisk (SQF) and demonstrate that
attempting to adjust for fairness perpetuates the same injustices that the
policy is infamous for.
Hilbert Space Methods for Reduced-Rank Gaussian Process Regression
Authors: Arno Solin, Simo Särkkä
Date: 2014-01-21 00:00:00
Tasks: regression

This paper proposes a novel scheme for reduced-rank Gaussian process regression. The method is based on an approximate series expansion of the covariance function in terms of an eigenfunction expansion of the Laplace operator in a compact subset of $\mathbb{R}^d$. On this approximate eigenbasis the eigenvalues of the covariance function can be expressed as simple functions of the spectral density of the Gaussian process, which allows the GP inference to be solved under a computational cost scaling as $\mathcal{O}(nm^2)$ (initial) and $\mathcal{O}(m^3)$ (hyperparameter learning) with $m$ basis functions and $n$ data points. Furthermore, the basis functions are independent of the parameters of the covariance function, which allows for very fast hyperparameter learning. The approach also allows for rigorous error analysis with Hilbert space theory, and we show that the approximation becomes exact when the size of the compact subset and the number of eigenfunctions go to infinity. We also show that the convergence rate of the truncation error is independent of the input dimensionality provided that the differentiability order of the covariance function is increases appropriately, and for the squared exponential covariance function it is always bounded by ${\sim}1/m$ regardless of the input dimensionality. The expansion generalizes to Hilbert spaces with an inner product which is defined as an integral over a specified input density. The method is compared to previously proposed methods theoretically and through empirical tests with simulated and real data.
MINE: Mutual Information Neural Estimation
Authors: Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeswar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, R. Devon Hjelm
Date: 2018-01-12 00:00:00
Tasks: General Classification

We argue that the estimation of mutual information between high dimensional continuous random variables can be achieved by gradient descent over neural networks. We present a Mutual Information Neural Estimator (MINE) that is linearly scalable in dimensionality as well as in sample size, trainable through back-prop, and strongly consistent. We present a handful of applications on which MINE can be used to minimize or maximize mutual information. We apply MINE to improve adversarially trained generative models. We also use MINE to implement Information Bottleneck, applying it to supervised classification; our results demonstrate substantial improvement in flexibility and performance in these settings.
Pros and Cons of GAN Evaluation Measures
Authors: Ali Borji
Date: 2018-02-09 00:00:00

Generative models, in particular generative adversarial networks (GANs), have
received significant attention recently. A number of GAN variants have been
proposed and have been utilized in many applications. Despite large strides in
terms of theoretical progress, evaluating and comparing GANs remains a daunting
task. While several measures have been introduced, as of yet, there is no
consensus as to which measure best captures strengths and limitations of models
and should be used for fair model comparison. As in other areas of computer
vision and machine learning, it is critical to settle on one or few good
measures to steer the progress in this field. In this paper, I review and
critically discuss more than 24 quantitative and 5 qualitative measures for
evaluating generative models with a particular emphasis on GAN-derived models.
I also provide a set of 7 desiderata followed by an evaluation of whether a
given measure or a family of measures is compatible with them.
Learning Tasks for Multitask Learning: Heterogenous Patient Populations in the ICU
Authors: Harini Suresh, Jen J. Gong, John Guttag
Date: 2018-06-07 00:00:00

Machine learning approaches have been effective in predicting adverse
outcomes in different clinical settings. These models are often developed and
evaluated on datasets with heterogeneous patient populations. However, good
predictive performance on the aggregate population does not imply good
performance for specific groups.
  In this work, we present a two-step framework to 1) learn relevant patient
subgroups, and 2) predict an outcome for separate patient populations in a
multi-task framework, where each population is a separate task. We demonstrate
how to discover relevant groups in an unsupervised way with a
sequence-to-sequence autoencoder. We show that using these groups in a
multi-task framework leads to better predictive performance of in-hospital
mortality both across groups and overall. We also highlight the need for more
granular evaluation of performance when dealing with heterogeneous populations.
An Exploration of Unreliable News Classification in Brazil and The U.S
Authors: Mauricio Gruppi, Benjamin D. Horne, Sibel Adali
Date: 2018-06-07 00:00:00
Tasks: Articles, General Classification, News Classification

The propagation of unreliable information is on the rise in many places
around the world. This expansion is facilitated by the rapid spread of
information and anonymity granted by the Internet. The spread of unreliable
information is a wellstudied issue and it is associated with negative social
impacts. In a previous work, we have identified significant differences in the
structure of news articles from reliable and unreliable sources in the US
media. Our goal in this work was to explore such differences in the Brazilian
media. We found significant features in two data sets: one with Brazilian news
in Portuguese and another one with US news in English. Our results show that
features related to the writing style were prominent in both data sets and,
despite the language difference, some features have a universal behavior, being
significant to both US and Brazilian news articles. Finally, we combined both
data sets and used the universal features to build a machine learning
classifier to predict the source type of a news article as reliable or
unreliable.
On the Topic of Jets: Disentangling Quarks and Gluons at Colliders
Authors: Eric M. Metodiev, Jesse Thaler
Date: 2018-01-31 00:00:00

We introduce jet topics: a framework to identify underlying classes of jets
from collider data. Because of a close mathematical relationship between
distributions of observables in jets and emergent themes in sets of documents,
we can apply recent techniques in "topic modeling" to extract jet topics from
data with minimal or no input from simulation or theory. As a proof of concept
with parton shower samples, we apply jet topics to determine separate quark and
gluon jet distributions for constituent multiplicity. We also determine
separate quark and gluon rapidity spectra from a mixed Z-plus-jet sample. While
jet topics are defined directly from hadron-level multi-differential cross
sections, one can also predict jet topics from first-principles theoretical
calculations, with potential implications for how to define quark and gluon
jets beyond leading-logarithmic accuracy. These investigations suggest that jet
topics will be useful for extracting underlying jet distributions and fractions
in a wide range of contexts at the Large Hadron Collider.
Direct Optimization through $\arg \max$ for Discrete Variational Auto-Encoder
Authors: Guy Lorberbom, Andreea Gane, Tommi Jaakkola, Tamir Hazan
Date: 2018-06-07 00:00:00
Proceeding: ICLR 2019 5

Reparameterization of variational auto-encoders with continuous random variables is an effective method for reducing the variance of their gradient estimates. In the discrete case, one can perform reparametrization using the Gumbel-Max trick, but the resulting objective relies on an $\arg \max$ operation and is non-differentiable. In contrast to previous works which resort to softmax-based relaxations, we propose to optimize it directly by applying the direct loss minimization approach. Our proposal extends naturally to structured discrete latent variable models when evaluating the $\arg \max$ operation is tractable. We demonstrate empirically the effectiveness of the direct loss minimization technique in variational autoencoders with both unstructured and structured discrete latent variables.