pwc-archive/papers-with-abstracts huggingface dataset:
Bias and high-dimensional adjustment in observational studies of peer effects
Authors: Dean Eckles, Eytan Bakshy
Date: 2017-06-14 00:00:00
Tasks: Causal Inference, Vocal Bursts Intensity Prediction

Peer effects, in which the behavior of an individual is affected by the
behavior of their peers, are posited by multiple theories in the social
sciences. Other processes can also produce behaviors that are correlated in
networks and groups, thereby generating debate about the credibility of
observational (i.e. nonexperimental) studies of peer effects. Randomized field
experiments that identify peer effects, however, are often expensive or
infeasible. Thus, many studies of peer effects use observational data, and
prior evaluations of causal inference methods for adjusting observational data
to estimate peer effects have lacked an experimental "gold standard" for
comparison. Here we show, in the context of information and media diffusion on
Facebook, that high-dimensional adjustment of a nonexperimental control group
(677 million observations) using propensity score models produces estimates of
peer effects statistically indistinguishable from those from using a large
randomized experiment (220 million observations). Naive observational
estimators overstate peer effects by 320% and commonly used variables (e.g.,
demographics) offer little bias reduction, but adjusting for a measure of prior
behaviors closely related to the focal behavior reduces bias by 91%.
High-dimensional models adjusting for over 3,700 past behaviors provide
additional bias reduction, such that the full model reduces bias by over 97%.
This experimental evaluation demonstrates that detailed records of individuals'
past behavior can improve studies of social influence, information diffusion,
and imitation; these results are encouraging for the credibility of some
studies but also cautionary for studies of rare or new behaviors. More
generally, these results show how large, high-dimensional data sets and
statistical learning techniques can be used to improve causal inference in the
behavioral sciences.
Adaptive Feature Selection: Computationally Efficient Online Sparse Linear Regression under RIP
Authors: Satyen Kale, Zohar Karnin, Tengyuan Liang, Dávid Pál
Date: 2017-06-14 00:00:00
Proceeding: ICML 2017 8
Tasks: feature selection, regression

Online sparse linear regression is an online problem where an algorithm
repeatedly chooses a subset of coordinates to observe in an adversarially
chosen feature vector, makes a real-valued prediction, receives the true label,
and incurs the squared loss. The goal is to design an online learning algorithm
with sublinear regret to the best sparse linear predictor in hindsight. Without
any assumptions, this problem is known to be computationally intractable. In
this paper, we make the assumption that data matrix satisfies restricted
isometry property, and show that this assumption leads to computationally
efficient algorithms with sublinear regret for two variants of the problem. In
the first variant, the true label is generated according to a sparse linear
model with additive Gaussian noise. In the second, the true label is chosen
adversarially.
A Practical Method for Solving Contextual Bandit Problems Using Decision Trees
Authors: Adam N. Elmachtoub, Ryan McNellis, Sechan Oh, Marek Petrik
Date: 2017-06-14 00:00:00
Tasks: Thompson Sampling

Many efficient algorithms with strong theoretical guarantees have been
proposed for the contextual multi-armed bandit problem. However, applying these
algorithms in practice can be difficult because they require domain expertise
to build appropriate features and to tune their parameters. We propose a new
method for the contextual bandit problem that is simple, practical, and can be
applied with little or no domain expertise. Our algorithm relies on decision
trees to model the context-reward relationship. Decision trees are
non-parametric, interpretable, and work well without hand-crafted features. To
guide the exploration-exploitation trade-off, we use a bootstrapping approach
which abstracts Thompson sampling to non-Bayesian settings. We also discuss
several computational heuristics and demonstrate the performance of our method
on several datasets.
Feature Enhancement in Visually Impaired Images
Authors: Madhuri Suthar, Mohammad Asghari, Bahram Jalali
Date: 2017-06-14 00:00:00
Tasks: Edge Detection

One of the major open problems in computer vision is detection of features in
visually impaired images. In this paper, we describe a potential solution using
Phase Stretch Transform, a new computational approach for image analysis, edge
detection and resolution enhancement that is inspired by the physics of the
photonic time stretch technique. We mathematically derive the intrinsic
nonlinear transfer function and demonstrate how it leads to (1) superior
performance at low contrast levels and (2) a reconfigurable operator for
hyper-dimensional classification. We prove that the Phase Stretch Transform
equalizes the input image brightness across the range of intensities resulting
in a high dynamic range in visually impaired images. We also show further
improvement in the dynamic range by combining our method with the conventional
techniques. Finally, our results show a method for computation of mathematical
derivatives via group delay dispersion operations.
LinkNet: Exploiting Encoder Representations for Efficient Semantic Segmentation
Authors: Abhishek Chaurasia, Eugenio Culurciello
Date: 2017-06-14 00:00:00
Tasks: GPU, Scene Understanding, Semantic Segmentation, Thermal Image Segmentation

Pixel-wise semantic segmentation for visual scene understanding not only
needs to be accurate, but also efficient in order to find any use in real-time
application. Existing algorithms even though are accurate but they do not focus
on utilizing the parameters of neural network efficiently. As a result they are
huge in terms of parameters and number of operations; hence slow too. In this
paper, we propose a novel deep neural network architecture which allows it to
learn without any significant increase in number of parameters. Our network
uses only 11.5 million parameters and 21.2 GFLOPs for processing an image of
resolution 3x640x360. It gives state-of-the-art performance on CamVid and
comparable results on Cityscapes dataset. We also compare our networks
processing time on NVIDIA GPU and embedded system device with existing
state-of-the-art architectures for different image resolutions.
Differentially Private Learning of Undirected Graphical Models using Collective Graphical Models
Authors: Garrett Bernstein, Ryan McKenna, Tao Sun, Daniel Sheldon, Michael Hay, Gerome Miklau
Date: 2017-06-14 00:00:00

We investigate the problem of learning discrete, undirected graphical models
in a differentially private way. We show that the approach of releasing noisy
sufficient statistics using the Laplace mechanism achieves a good trade-off
between privacy, utility, and practicality. A naive learning algorithm that
uses the noisy sufficient statistics "as is" outperforms general-purpose
differentially private learning algorithms. However, it has three limitations:
it ignores knowledge about the data generating process, rests on uncertain
theoretical foundations, and exhibits certain pathologies. We develop a more
principled approach that applies the formalism of collective graphical models
to perform inference over the true sufficient statistics within an
expectation-maximization framework. We show that this learns better models than
competing approaches on both synthetic data and on real human mobility data
used as a case study.
Stochastic Gradient MCMC Methods for Hidden Markov Models
Authors: Yi-An Ma, Nicholas J. Foti, Emily B. Fox
Date: 2017-06-14 00:00:00
Proceeding: ICML 2017 8
Tasks: Bayesian Inference

Stochastic gradient MCMC (SG-MCMC) algorithms have proven useful in scaling
Bayesian inference to large datasets under an assumption of i.i.d data. We
instead develop an SG-MCMC algorithm to learn the parameters of hidden Markov
models (HMMs) for time-dependent data. There are two challenges to applying
SG-MCMC in this setting: The latent discrete states, and needing to break
dependencies when considering minibatches. We consider a marginal likelihood
representation of the HMM and propose an algorithm that harnesses the inherent
memory decay of the process. We demonstrate the effectiveness of our algorithm
on synthetic experiments and an ion channel recording data, with runtimes
significantly outperforming batch MCMC.
Input Convex Neural Networks
Authors: Brandon Amos, Lei Xu, J. Zico Kolter
Date: 2016-09-22 00:00:00
Proceeding: ICML 2017 8
Tasks: Imputation, Inference Optimization, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Structured Prediction

This paper presents the input convex neural network architecture. These are
scalar-valued (potentially deep) neural networks with constraints on the
network parameters such that the output of the network is a convex function of
(some of) the inputs. The networks allow for efficient inference via
optimization over some inputs to the network given others, and can be applied
to settings including structured prediction, data imputation, reinforcement
learning, and others. In this paper we lay the basic groundwork for these
models, proposing methods for inference, optimization and learning, and analyze
their representational power. We show that many existing neural network
architectures can be made input-convex with a minor modification, and develop
specialized optimization algorithms tailored to this setting. Finally, we
highlight the performance of the methods on multi-label prediction, image
completion, and reinforcement learning problems, where we show improvement over
the existing state of the art in many cases.
CortexNet: a Generic Network Family for Robust Visual Temporal Representations
Authors: Alfredo Canziani, Eugenio Culurciello
Date: 2017-06-08 00:00:00
Tasks: Object Recognition

In the past five years we have observed the rise of incredibly well
performing feed-forward neural networks trained supervisedly for vision related
tasks. These models have achieved super-human performance on object
recognition, localisation, and detection in still images. However, there is a
need to identify the best strategy to employ these networks with temporal
visual inputs and obtain a robust and stable representation of video data.
Inspired by the human visual system, we propose a deep neural network family,
CortexNet, which features not only bottom-up feed-forward connections, but also
it models the abundant top-down feedback and lateral connections, which are
present in our visual cortex. We introduce two training schemes - the
unsupervised MatchNet and weakly supervised TempoNet modes - where a network
learns how to correctly anticipate a subsequent frame in a video clip or the
identity of its predominant subject, by learning egomotion clues and how to
automatically track several objects in the current scene. Find the project
website at https://engineering.purdue.edu/elab/CortexNet/.
Provable benefits of representation learning
Authors: Sanjeev Arora, Andrej Risteski
Date: 2017-06-14 00:00:00
Tasks: Clustering, Representation Learning, Transfer Learning

There is general consensus that learning representations is useful for a
variety of reasons, e.g. efficient use of labeled data (semi-supervised
learning), transfer learning and understanding hidden structure of data.
Popular techniques for representation learning include clustering, manifold
learning, kernel-learning, autoencoders, Boltzmann machines, etc.
  To study the relative merits of these techniques, it's essential to formalize
the definition and goals of representation learning, so that they are all
become instances of the same definition. This paper introduces such a formal
framework that also formalizes the utility of learning the representation. It
is related to previous Bayesian notions, but with some new twists. We show the
usefulness of our framework by exhibiting simple and natural settings -- linear
mixture models and loglinear models, where the power of representation learning
can be formally shown. In these examples, representation learning can be
performed provably and efficiently under plausible assumptions (despite being
NP-hard), and furthermore: (i) it greatly reduces the need for labeled data
(semi-supervised learning) and (ii) it allows solving classification tasks when
simpler approaches like nearest neighbors require too much data (iii) it is
more powerful than manifold learning methods.
Learning without Prejudice: Avoiding Bias in Webly-Supervised Action Recognition
Authors: Christian Rupprecht, Ansh Kapil, Nan Liu, Lamberto Ballan, Federico Tombari
Date: 2017-06-14 00:00:00
Tasks: Action Recognition, Optical Flow Estimation, Temporal Action Localization, Video Understanding

Webly-supervised learning has recently emerged as an alternative paradigm to
traditional supervised learning based on large-scale datasets with manual
annotations. The key idea is that models such as CNNs can be learned from the
noisy visual data available on the web. In this work we aim to exploit web data
for video understanding tasks such as action recognition and detection. One of
the main problems in webly-supervised learning is cleaning the noisy labeled
data from the web. The state-of-the-art paradigm relies on training a first
classifier on noisy data that is then used to clean the remaining dataset. Our
key insight is that this procedure biases the second classifier towards samples
that the first one understands. Here we train two independent CNNs, a RGB
network on web images and video frames and a second network using temporal
information from optical flow. We show that training the networks independently
is vastly superior to selecting the frames for the flow classifier by using our
RGB network. Moreover, we show benefits in enriching the training set with
different data sources from heterogeneous public web databases. We demonstrate
that our framework outperforms all other webly-supervised methods on two public
benchmarks, UCF-101 and Thumos'14.
On The Projection Operator to A Three-view Cardinality Constrained Set
Authors: Haichuan Yang, Shupeng Gui, Chuyang Ke, Daniel Stefankovic, Ryohei Fujimaki, Ji Liu
Date: 2017-03-21 00:00:00
Proceeding: ICML 2017 8
Tasks: compressed sensing, feature selection, Sparse Learning

The cardinality constraint is an intrinsic way to restrict the solution
structure in many domains, for example, sparse learning, feature selection, and
compressed sensing. To solve a cardinality constrained problem, the key
challenge is to solve the projection onto the cardinality constraint set, which
is NP-hard in general when there exist multiple overlapped cardinality
constraints. In this paper, we consider the scenario where the overlapped
cardinality constraints satisfy a Three-view Cardinality Structure (TVCS),
which reflects the natural restriction in many applications, such as
identification of gene regulatory networks and task-worker assignment problem.
We cast the projection into a linear programming, and show that for TVCS, the
vertex solution of this linear programming is the solution for the original
projection problem. We further prove that such solution can be found with the
complexity proportional to the number of variables and constraints. We finally
use synthetic experiments and two interesting applications in bioinformatics
and crowdsourcing to validate the proposed TVCS model and method.
Deep Learning Methods for Efficient Large Scale Video Labeling
Authors: Miha Skalic, Marcin Pekalski, Xingguo E. Pan
Date: 2017-06-14 00:00:00
Tasks: Deep Learning, Video Understanding

We present a solution to "Google Cloud and YouTube-8M Video Understanding
Challenge" that ranked 5th place. The proposed model is an ensemble of three
model families, two frame level and one video level. The training was performed
on augmented dataset, with cross validation.
The Inflation Technique for Causal Inference with Latent Variables
Authors: Elie Wolfe, Robert W. Spekkens, Tobias Fritz
Date: 2016-09-02 00:00:00
Tasks: Causal Inference

The problem of causal inference is to determine if a given probability distribution on observed variables is compatible with some causal structure. The difficult case is when the causal structure includes latent variables. We here introduce the $\textit{inflation technique}$ for tackling this problem. An inflation of a causal structure is a new causal structure that can contain multiple copies of each of the original variables, but where the ancestry of each copy mirrors that of the original. To every distribution of the observed variables that is compatible with the original causal structure, we assign a family of marginal distributions on certain subsets of the copies that are compatible with the inflated causal structure. It follows that compatibility constraints for the inflation can be translated into compatibility constraints for the original causal structure. Even if the constraints at the level of inflation are weak, such as observable statistical independences implied by disjoint causal ancestry, the translated constraints can be strong. We apply this method to derive new inequalities whose violation by a distribution witnesses that distribution's incompatibility with the causal structure (of which Bell inequalities and Pearl's instrumental inequality are prominent examples). We describe an algorithm for deriving all such inequalities for the original causal structure that follow from ancestral independences in the inflation. For three observed binary variables with pairwise common causes, it yields inequalities that are stronger in at least some aspects than those obtainable by existing methods. We also describe an algorithm that derives a weaker set of inequalities but is more efficient. Finally, we discuss which inflations are such that the inequalities one obtains from them remain valid even for quantum (and post-quantum) generalizations of the notion of a causal model.
Full Quantification of Left Ventricle via Deep Multitask Learning Network Respecting Intra- and Inter-Task Relatedness
Authors: Wufeng Xue, Andrea Lum, Ashley Mercado, Mark Landis, James Warringto, Shuo Li
Date: 2017-06-06 00:00:00
Tasks: feature selection

Cardiac left ventricle (LV) quantification is among the most clinically
important tasks for identification and diagnosis of cardiac diseases, yet still
a challenge due to the high variability of cardiac structure and the complexity
of temporal dynamics. Full quantification, i.e., to simultaneously quantify all
LV indices including two areas (cavity and myocardium), six regional wall
thicknesses (RWT), three LV dimensions, and one cardiac phase, is even more
challenging since the uncertain relatedness intra and inter each type of
indices may hinder the learning procedure from better convergence and
generalization. In this paper, we propose a newly-designed multitask learning
network (FullLVNet), which is constituted by a deep convolution neural network
(CNN) for expressive feature embedding of cardiac structure; two followed
parallel recurrent neural network (RNN) modules for temporal dynamic modeling;
and four linear models for the final estimation. During the final estimation,
both intra- and inter-task relatedness are modeled to enforce improvement of
generalization: 1) respecting intra-task relatedness, group lasso is applied to
each of the regression tasks for sparse and common feature selection and
consistent prediction; 2) respecting inter-task relatedness, three phase-guided
constraints are proposed to penalize violation of the temporal behavior of the
obtained LV indices. Experiments on MR sequences of 145 subjects show that
FullLVNet achieves high accurate prediction with our intra- and inter-task
relatedness, leading to MAE of 190mm$^2$, 1.41mm, 2.68mm for average areas,
RWT, dimensions and error rate of 10.4\% for the phase classification. This
endows our method a great potential in comprehensive clinical assessment of
global, regional and dynamic cardiac function.
Variational Inference for Sparse and Undirected Models
Authors: John Ingraham, Debora Marks
Date: 2016-02-11 00:00:00
Proceeding: ICML 2017 8
Tasks: Bayesian Inference, Protein Structure Prediction, Variational Inference

Undirected graphical models are applied in genomics, protein structure
prediction, and neuroscience to identify sparse interactions that underlie
discrete data. Although Bayesian methods for inference would be favorable in
these contexts, they are rarely used because they require doubly intractable
Monte Carlo sampling. Here, we develop a framework for scalable Bayesian
inference of discrete undirected models based on two new methods. The first is
Persistent VI, an algorithm for variational inference of discrete undirected
models that avoids doubly intractable MCMC and approximations of the partition
function. The second is Fadeout, a reparameterization approach for variational
inference under sparsity-inducing priors that captures a posteriori
correlations between parameters and hyperparameters with noncentered
parameterizations. We find that, together, these methods for variational
inference substantially improve learning of sparse undirected graphical models
in simulated and real problems from physics and biology.
Reinforcement Learning with Budget-Constrained Nonparametric Function Approximation for Opportunistic Spectrum Access
Authors: Theodoros Tsiligkaridis, David Romero
Date: 2017-06-14 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Opportunistic spectrum access is one of the emerging techniques for
maximizing throughput in congested bands and is enabled by predicting idle
slots in spectrum. We propose a kernel-based reinforcement learning approach
coupled with a novel budget-constrained sparsification technique that
efficiently captures the environment to find the best channel access actions.
This approach allows learning and planning over the intrinsic state-action
space and extends well to large state spaces. We apply our methods to evaluate
coexistence of a reinforcement learning-based radio with a multi-channel
adversarial radio and a single-channel CSMA-CA radio. Numerical experiments
show the performance gains over carrier-sense systems.
Small-footprint Deep Neural Networks with Highway Connections for Speech Recognition
Authors: Liang Lu, Steve Renals
Date: 2015-12-14 00:00:00
Tasks: speech-recognition, Speech Recognition

For speech recognition, deep neural networks (DNNs) have significantly
improved the recognition accuracy in most of benchmark datasets and application
domains. However, compared to the conventional Gaussian mixture models,
DNN-based acoustic models usually have much larger number of model parameters,
making it challenging for their applications in resource constrained platforms,
e.g., mobile devices. In this paper, we study the application of the recently
proposed highway network to train small-footprint DNNs, which are {\it thinner}
and {\it deeper}, and have significantly smaller number of model parameters
compared to conventional DNNs. We investigated this approach on the AMI meeting
speech transcription corpus which has around 70 hours of audio data. The
highway neural networks constantly outperformed their plain DNN counterparts,
and the number of model parameters can be reduced significantly without
sacrificing the recognition accuracy.