pwc-archive/papers-with-abstracts huggingface dataset:
Using Social Dynamics to Make Individual Predictions: Variational Inference with a Stochastic Kinetic Model
Authors: Zhen Xu, Wen Dong, Sargur Srihari
Date: 2016-11-07 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Variational Inference

Social dynamics is concerned primarily with interactions among individuals
and the resulting group behaviors, modeling the temporal evolution of social
systems via the interactions of individuals within these systems. In
particular, the availability of large-scale data from social networks and
sensor networks offers an unprecedented opportunity to predict state-changing
events at the individual level. Examples of such events include disease
transmission, opinion transition in elections, and rumor propagation. Unlike
previous research focusing on the collective effects of social systems, this
study makes efficient inferences at the individual level. In order to cope with
dynamic interactions among a large number of individuals, we introduce the
stochastic kinetic model to capture adaptive transition probabilities and
propose an efficient variational inference algorithm the complexity of which
grows linearly --- rather than exponentially --- with the number of
individuals. To validate this method, we have performed epidemic-dynamics
experiments on wireless sensor network data collected from more than ten
thousand people over three years. The proposed algorithm was used to track
disease transmission and predict the probability of infection for each
individual. Our results demonstrate that this method is more efficient than
sampling while nonetheless achieving high accuracy.
Importance Weighted Autoencoders
Authors: Yuri Burda, Roger Grosse, Ruslan Salakhutdinov
Date: 2015-09-01 00:00:00
Tasks: Density Estimation

The variational autoencoder (VAE; Kingma, Welling (2014)) is a recently
proposed generative model pairing a top-down generative network with a
bottom-up recognition network which approximates posterior inference. It
typically makes strong assumptions about posterior inference, for instance that
the posterior distribution is approximately factorial, and that its parameters
can be approximated with nonlinear regression from the observations. As we show
empirically, the VAE objective can lead to overly simplified representations
which fail to use the network's entire modeling capacity. We present the
importance weighted autoencoder (IWAE), a generative model with the same
architecture as the VAE, but which uses a strictly tighter log-likelihood lower
bound derived from importance weighting. In the IWAE, the recognition network
uses multiple samples to approximate the posterior, giving it increased
flexibility to model complex posteriors which do not fit the VAE modeling
assumptions. We show empirically that IWAEs learn richer latent space
representations than VAEs, leading to improved test log-likelihood on density
estimation benchmarks.
Spatiotemporal Residual Networks for Video Action Recognition
Authors: Christoph Feichtenhofer, Axel Pinz, Richard P. Wildes
Date: 2016-11-07 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Action Recognition, Action Recognition In Videos, Temporal Action Localization

Two-stream Convolutional Networks (ConvNets) have shown strong performance
for human action recognition in videos. Recently, Residual Networks (ResNets)
have arisen as a new technique to train extremely deep architectures. In this
paper, we introduce spatiotemporal ResNets as a combination of these two
approaches. Our novel architecture generalizes ResNets for the spatiotemporal
domain by introducing residual connections in two ways. First, we inject
residual connections between the appearance and motion pathways of a two-stream
architecture to allow spatiotemporal interaction between the two streams.
Second, we transform pretrained image ConvNets into spatiotemporal networks by
equipping these with learnable convolutional filters that are initialized as
temporal residual connections and operate on adjacent feature maps in time.
This approach slowly increases the spatiotemporal receptive field as the depth
of the model increases and naturally integrates image ConvNet design
principles. The whole model is trained end-to-end to allow hierarchical
learning of complex spatiotemporal features. We evaluate our novel
spatiotemporal ResNet using two widely used action recognition benchmarks where
it exceeds the previous state-of-the-art.
Crowdsourcing in Computer Vision
Authors: Adriana Kovashka, Olga Russakovsky, Li Fei-Fei, Kristen Grauman
Date: 2016-11-07 00:00:00
Tasks: Object Recognition

Computer vision systems require large amounts of manually annotated data to
properly learn challenging visual concepts. Crowdsourcing platforms offer an
inexpensive method to capture human knowledge and understanding, for a vast
number of visual perception tasks. In this survey, we describe the types of
annotations computer vision researchers have collected using crowdsourcing, and
how they have ensured that this data is of high quality while annotation effort
is minimized. We begin by discussing data collection on both classic (e.g.,
object recognition) and recent (e.g., visual story-telling) vision tasks. We
then summarize key design decisions for creating effective data collection
interfaces and workflows, and present strategies for intelligently selecting
the most important data instances to annotate. Finally, we conclude with some
thoughts on the future of crowdsourcing in computer vision.
Computational Imaging for VLBI Image Reconstruction
Authors: Katherine L. Bouman, Michael D. Johnson, Daniel Zoran, Vincent L. Fish, Sheperd S. Doeleman, William T. Freeman
Date: 2015-12-04 00:00:00
Proceeding: CVPR 2016 6
Tasks: Image Reconstruction

Very long baseline interferometry (VLBI) is a technique for imaging celestial
radio emissions by simultaneously observing a source from telescopes
distributed across Earth. The challenges in reconstructing images from fine
angular resolution VLBI data are immense. The data is extremely sparse and
noisy, thus requiring statistical image models such as those designed in the
computer vision community. In this paper we present a novel Bayesian approach
for VLBI image reconstruction. While other methods often require careful tuning
and parameter selection for different types of data, our method (CHIRP)
produces good results under different settings such as low SNR or extended
emission. The success of our method is demonstrated on realistic synthetic
experiments as well as publicly available real data. We present this problem in
a way that is accessible to members of the community, and provide a dataset
website (vlbiimaging.csail.mit.edu) that facilitates controlled comparisons
across algorithms.
Fairness in Learning: Classic and Contextual Bandits
Authors: Matthew Joseph, Michael Kearns, Jamie Morgenstern, Aaron Roth
Date: 2016-05-23 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Fairness, Multi-Armed Bandits

We introduce the study of fairness in multi-armed bandit problems. Our
fairness definition can be interpreted as demanding that given a pool of
applicants (say, for college admission or mortgages), a worse applicant is
never favored over a better one, despite a learning algorithm's uncertainty
over the true payoffs. We prove results of two types.
  First, in the important special case of the classic stochastic bandits
problem (i.e., in which there are no contexts), we provide a provably fair
algorithm based on "chained" confidence intervals, and provide a cumulative
regret bound with a cubic dependence on the number of arms. We further show
that any fair algorithm must have such a dependence. When combined with regret
bounds for standard non-fair algorithms such as UCB, this proves a strong
separation between fair and unfair learning, which extends to the general
contextual case.
  In the general contextual case, we prove a tight connection between fairness
and the KWIK (Knows What It Knows) learning model: a KWIK algorithm for a class
of functions can be transformed into a provably fair contextual bandit
algorithm, and conversely any fair contextual bandit algorithm can be
transformed into a KWIK learning algorithm. This tight connection allows us to
provide a provably fair algorithm for the linear contextual bandit problem with
a polynomial dependence on the dimension, and to show (for a different class of
functions) a worst-case exponential gap in regret between fair and non-fair
learning algorithms
Assessing and tuning brain decoders: cross-validation, caveats, and guidelines
Authors: Gaël Varoquaux, Pradeep Reddy Raamana, Denis Engemann, Andrés Hoyos-Idrobo, Yannick Schwartz, Bertrand Thirion
Date: 2016-06-16 00:00:00

Decoding, ie prediction from brain images or signals, calls for empirical
evaluation of its predictive power. Such evaluation is achieved via
cross-validation, a method also used to tune decoders' hyper-parameters. This
paper is a review on cross-validation procedures for decoding in neuroimaging.
It includes a didactic overview of the relevant theoretical considerations.
Practical aspects are highlighted with an extensive empirical study of the
common decoders in within-and across-subject predictions, on multiple datasets
--anatomical and functional MRI and MEG-- and simulations. Theory and
experiments outline that the popular " leave-one-out " strategy leads to
unstable and biased estimates, and a repeated random splits method should be
preferred. Experiments outline the large error bars of cross-validation in
neuroimaging settings: typical confidence intervals of 10%. Nested
cross-validation can tune decoders' parameters while avoiding circularity bias.
However we find that it can be more favorable to use sane defaults, in
particular for non-sparse decoders.
Neural Networks Designing Neural Networks: Multi-Objective Hyper-Parameter Optimization
Authors: Sean C. Smithson, Guang Yang, Warren J. Gross, Brett H. Meyer
Date: 2016-11-07 00:00:00
Tasks: BIG-bench Machine Learning, image-classification, Image Classification, speech-recognition, Speech Recognition

Artificial neural networks have gone through a recent rise in popularity,
achieving state-of-the-art results in various fields, including image
classification, speech recognition, and automated control. Both the performance
and computational complexity of such models are heavily dependant on the design
of characteristic hyper-parameters (e.g., number of hidden layers, nodes per
layer, or choice of activation functions), which have traditionally been
optimized manually. With machine learning penetrating low-power mobile and
embedded areas, the need to optimize not only for performance (accuracy), but
also for implementation complexity, becomes paramount. In this work, we present
a multi-objective design space exploration method that reduces the number of
solution networks trained and evaluated through response surface modelling.
Given spaces which can easily exceed 1020 solutions, manually designing a
near-optimal architecture is unlikely as opportunities to reduce network
complexity, while maintaining performance, may be overlooked. This problem is
exacerbated by the fact that hyper-parameters which perform well on specific
datasets may yield sub-par results on others, and must therefore be designed on
a per-application basis. In our work, machine learning is leveraged by training
an artificial neural network to predict the performance of future candidate
networks. The method is evaluated on the MNIST and CIFAR-10 image datasets,
optimizing for both recognition accuracy and computational complexity.
Experimental results demonstrate that the proposed method can closely
approximate the Pareto-optimal front, while only exploring a small fraction of
the design space.
Reinforcement-based Simultaneous Algorithm and its Hyperparameters Selection
Authors: Valeria Efimova, Andrey Filchenkov, Anatoly Shalyto
Date: 2016-11-07 00:00:00
Tasks: General Classification

Many algorithms for data analysis exist, especially for classification
problems. To solve a data analysis problem, a proper algorithm should be
chosen, and also its hyperparameters should be selected. In this paper, we
present a new method for the simultaneous selection of an algorithm and its
hyperparameters. In order to do so, we reduced this problem to the multi-armed
bandit problem. We consider an algorithm as an arm and algorithm
hyperparameters search during a fixed time as the corresponding arm play. We
also suggest a problem-specific reward function. We performed the experiments
on 10 real datasets and compare the suggested method with the existing one
implemented in Auto-WEKA. The results show that our method is significantly
better in most of the cases and never worse than the Auto-WEKA.
Reinforcement Learning Approach for Parallelization in Filters Aggregation Based Feature Selection Algorithms
Authors: Ivan Smetannikov, Ilya Isaev, Andrey Filchenkov
Date: 2016-11-07 00:00:00
Tasks: BIG-bench Machine Learning, feature selection, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

One of the classical problems in machine learning and data mining is feature
selection. A feature selection algorithm is expected to be quick, and at the
same time it should show high performance. MeLiF algorithm effectively solves
this problem using ensembles of ranking filters. This article describes two
different ways to improve MeLiF algorithm performance with parallelization.
Experiments show that proposed schemes significantly improves algorithm
performance and increase feature selection quality.
:telephone::person::sailboat::whale::okhand:; or "Call me Ishmael" - How do you translate emoji?
Authors: Will Radford, Andrew Chisholm, Ben Hachey, Bo Han
Date: 2016-11-07 00:00:00
Tasks: Part-Of-Speech Tagging, Translation, Word Alignment

We report on an exploratory analysis of Emoji Dick, a project that leverages
crowdsourcing to translate Melville's Moby Dick into emoji. This distinctive
use of emoji removes textual context, and leads to a varying translation
quality. In this paper, we use statistical word alignment and part-of-speech
tagging to explore how people use emoji. Despite these simple methods, we
observed differences in token and part-of-speech distributions. Experiments
also suggest that semantics are preserved in the translation, and repetition is
more common in emoji.
Presenting a New Dataset for the Timeline Generation Problem
Authors: Xavier Holt, Will Radford, Ben Hachey
Date: 2016-11-07 00:00:00
Proceeding: ALTA 2016 12
Tasks: Articles

The timeline generation task summarises an entity's biography by selecting
stories representing key events from a large pool of relevant documents. This
paper addresses the lack of a standard dataset and evaluative methodology for
the problem. We present and make publicly available a new dataset of 18,793
news articles covering 39 entities. For each entity, we provide a gold standard
timeline and a set of entity-related articles. We propose ROUGE as an
evaluation metric and validate our dataset by showing that top Google results
outperform straw-man baselines.
Deep Spiking Networks
Authors: Peter O'Connor, Max Welling
Date: 2016-02-26 00:00:00

We introduce an algorithm to do backpropagation on a spiking network. Our
network is "spiking" in the sense that our neurons accumulate their activation
into a potential over time, and only send out a signal (a "spike") when this
potential crosses a threshold and the neuron is reset. Neurons only update
their states when receiving signals from other neurons. Total computation of
the network thus scales with the number of spikes caused by an input rather
than network size. We show that the spiking Multi-Layer Perceptron behaves
identically, during both prediction and training, to a conventional deep
network of rectified-linear units, in the limiting case where we run the
spiking network for a long time. We apply this architecture to a conventional
classification problem (MNIST) and achieve performance very close to that of a
conventional Multi-Layer Perceptron with the same architecture. Our network is
a natural architecture for learning based on streaming event-based data, and is
a stepping stone towards using spiking neural networks to learn efficiently on
streaming data.
Multi-view Generative Adversarial Networks
Authors: Mickaël Chen, Ludovic Denoyer
Date: 2016-11-07 00:00:00
Tasks: Density Estimation, General Classification

Learning over multi-view data is a challenging problem with strong practical
applications. Most related studies focus on the classification point of view
and assume that all the views are available at any time. We consider an
extension of this framework in two directions. First, based on the BiGAN model,
the Multi-view BiGAN (MV-BiGAN) is able to perform density estimation from
multi-view inputs. Second, it can deal with missing views and is able to update
its prediction when additional views are provided. We illustrate these
properties on a set of experiments over different datasets.
Keyphrase Annotation with Graph Co-Ranking
Authors: Adrien Bougouin, Florian Boudin, Béatrice Daille
Date: 2016-11-07 00:00:00
Proceeding: COLING 2016 12
Tasks: Keyphrase Extraction

Keyphrase annotation is the task of identifying textual units that represent
the main content of a document. Keyphrase annotation is either carried out by
extracting the most important phrases from a document, keyphrase extraction, or
by assigning entries from a controlled domain-specific vocabulary, keyphrase
assignment. Assignment methods are generally more reliable. They provide
better-formed keyphrases, as well as keyphrases that do not occur in the
document. But they are often silent on the contrary of extraction methods that
do not depend on manually built resources. This paper proposes a new method to
perform both keyphrase extraction and keyphrase assignment in an integrated and
mutual reinforcing manner. Experiments have been carried out on datasets
covering different domains of humanities and social sciences. They show
statistically significant improvements compared to both keyphrase extraction
and keyphrase assignment state-of-the art methods.
Optimal Binary Classifier Aggregation for General Losses
Authors: Akshay Balsubramani, Yoav Freund
Date: 2015-10-01 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Binary Classification, General Classification

We address the problem of aggregating an ensemble of predictors with known
loss bounds in a semi-supervised binary classification setting, to minimize
prediction loss incurred on the unlabeled data. We find the minimax optimal
predictions for a very general class of loss functions including all convex and
many non-convex losses, extending a recent analysis of the problem for
misclassification error. The result is a family of semi-supervised ensemble
aggregation algorithms which are as efficient as linear learning by convex
optimization, but are minimax optimal without any relaxations. Their decision
rules take a form familiar in decision theory -- applying sigmoid functions to
a notion of ensemble margin -- without the assumptions typically made in
margin-based learning.
Log-time and Log-space Extreme Classification
Authors: Kalina Jasinska, Nikos Karampatziakis
Date: 2016-11-07 00:00:00
Tasks: Classification, General Classification, Prediction, Structured Prediction

We present LTLS, a technique for multiclass and multilabel prediction that
can perform training and inference in logarithmic time and space. LTLS embeds
large classification problems into simple structured prediction problems and
relies on efficient dynamic programming algorithms for inference. We train LTLS
with stochastic gradient descent on a number of multiclass and multilabel
datasets and show that despite its small memory footprint it is often
competitive with existing approaches.
High-Resolution Semantic Labeling with Convolutional Neural Networks
Authors: Emmanuel Maggiori, Yuliya Tarabalka, Guillaume Charpiat, Pierre Alliez
Date: 2016-11-07 00:00:00
Tasks: Image Categorization, Vocal Bursts Intensity Prediction

Convolutional neural networks (CNNs) have received increasing attention over
the last few years. They were initially conceived for image categorization,
i.e., the problem of assigning a semantic label to an entire input image.
  In this paper we address the problem of dense semantic labeling, which
consists in assigning a semantic label to every pixel in an image. Since this
requires a high spatial accuracy to determine where labels are assigned,
categorization CNNs, intended to be highly robust to local deformations, are
not directly applicable.
  By adapting categorization networks, many semantic labeling CNNs have been
recently proposed. Our first contribution is an in-depth analysis of these
architectures. We establish the desired properties of an ideal semantic
labeling CNN, and assess how those methods stand with regard to these
properties. We observe that even though they provide competitive results, these
CNNs often underexploit properties of semantic labeling that could lead to more
effective and efficient architectures.
  Out of these observations, we then derive a CNN framework specifically
adapted to the semantic labeling problem. In addition to learning features at
different resolutions, it learns how to combine these features. By integrating
local and global information in an efficient and flexible manner, it
outperforms previous techniques. We evaluate the proposed framework and compare
it with state-of-the-art architectures on public benchmarks of high-resolution
aerial image labeling.
Decision Tree Classification with Differential Privacy: A Survey
Authors: Sam Fletcher, Md Zahidul Islam
Date: 2016-11-07 00:00:00
Tasks: Classification, General Classification, Survey

Data mining information about people is becoming increasingly important in the data-driven society of the 21st century. Unfortunately, sometimes there are real-world considerations that conflict with the goals of data mining; sometimes the privacy of the people being data mined needs to be considered. This necessitates that the output of data mining algorithms be modified to preserve privacy while simultaneously not ruining the predictive power of the outputted model. Differential privacy is a strong, enforceable definition of privacy that can be used in data mining algorithms, guaranteeing that nothing will be learned about the people in the data that could not already be discovered without their participation. In this survey, we focus on one particular data mining algorithm -- decision trees -- and how differential privacy interacts with each of the components that constitute decision tree algorithms. We analyze both greedy and random decision trees, and the conflicts that arise when trying to balance privacy requirements with the accuracy of the model.