pwc-archive/papers-with-abstracts huggingface dataset:
MLBench: How Good Are Machine Learning Clouds for Binary Classification Tasks on Structured Data?
Authors: Yu Liu, Hantian Zhang, Luyuan Zeng, Wentao Wu, Ce Zhang
Date: 2017-07-29 00:00:00
Tasks: BIG-bench Machine Learning, Binary Classification, General Classification

We conduct an empirical study of machine learning functionalities provided by
major cloud service providers, which we call machine learning clouds. Machine
learning clouds hold the promise of hiding all the sophistication of running
large-scale machine learning: Instead of specifying how to run a machine
learning task, users only specify what machine learning task to run and the
cloud figures out the rest. Raising the level of abstraction, however, rarely
comes free - a performance penalty is possible. How good, then, are current
machine learning clouds on real-world machine learning workloads?
  We study this question with a focus on binary classication problems. We
present mlbench, a novel benchmark constructed by harvesting datasets from
Kaggle competitions. We then compare the performance of the top winning code
available from Kaggle with that of running machine learning clouds from both
Azure and Amazon on mlbench. Our comparative study reveals the strength and
weakness of existing machine learning clouds and points out potential future
directions for improvement.
Deep Feature Learning for Graphs
Authors: Ryan A. Rossi, Rong Zhou, Nesreen K. Ahmed
Date: 2017-04-28 00:00:00
Tasks: Graph Representation Learning, Representation Learning, Transfer Learning

This paper presents a general graph representation learning framework called
DeepGL for learning deep node and edge representations from large (attributed)
graphs. In particular, DeepGL begins by deriving a set of base features (e.g.,
graphlet features) and automatically learns a multi-layered hierarchical graph
representation where each successive layer leverages the output from the
previous layer to learn features of a higher-order. Contrary to previous work,
DeepGL learns relational functions (each representing a feature) that
generalize across-networks and therefore useful for graph-based transfer
learning tasks. Moreover, DeepGL naturally supports attributed graphs, learns
interpretable features, and is space-efficient (by learning sparse feature
vectors). In addition, DeepGL is expressive, flexible with many interchangeable
components, efficient with a time complexity of $\mathcal{O}(|E|)$, and
scalable for large networks via an efficient parallel implementation. Compared
with the state-of-the-art method, DeepGL is (1) effective for across-network
transfer learning tasks and attributed graph representation learning, (2)
space-efficient requiring up to 6x less memory, (3) fast with up to 182x
speedup in runtime performance, and (4) accurate with an average improvement of
20% or more on many learning tasks.
Fair Kernel Learning
Authors: Adrián Pérez-Suay, Valero Laparra, Gonzalo Mateo-García, Jordi Muñoz-Marí, Luis Gómez-Chova, Gustau Camps-Valls
Date: 2017-10-16 00:00:00
Tasks: BIG-bench Machine Learning, Dimensionality Reduction, Ethics, Fairness

New social and economic activities massively exploit big data and machine
learning algorithms to do inference on people's lives. Applications include
automatic curricula evaluation, wage determination, and risk assessment for
credits and loans. Recently, many governments and institutions have raised
concerns about the lack of fairness, equity and ethics in machine learning to
treat these problems. It has been shown that not including sensitive features
that bias fairness, such as gender or race, is not enough to mitigate the
discrimination when other related features are included. Instead, including
fairness in the objective function has been shown to be more efficient.
  We present novel fair regression and dimensionality reduction methods built
on a previously proposed fair classification framework. Both methods rely on
using the Hilbert Schmidt independence criterion as the fairness term. Unlike
previous approaches, this allows us to simplify the problem and to use multiple
sensitive variables simultaneously. Replacing the linear formulation by kernel
functions allows the methods to deal with nonlinear problems. For both linear
and nonlinear formulations the solution reduces to solving simple matrix
inversions or generalized eigenvalue problems. This simplifies the evaluation
of the solutions for different trade-off values between the predictive error
and fairness terms. We illustrate the usefulness of the proposed methods in toy
examples, and evaluate their performance on real world datasets to predict
income using gender and/or race discrimination as sensitive variables, and
contraceptive method prediction under demographic and socio-economic sensitive
descriptors.
Convergence Analysis of l0-RLS Adaptive Filter
Authors: B. K. Das, S. Mukhopadhyay, M. Chakraborty
Date: 2017-10-16 00:00:00

This paper presents first and second order convergence analysis of the
sparsity aware l0-RLS adaptive filter. The theorems 1 and 2 state the steady
state value of mean and mean square deviation of the adaptive filter weight
vector.
Coresets for Dependency Networks
Authors: Alejandro Molina, Alexander Munteanu, Kristian Kersting
Date: 2017-10-09 00:00:00

Many applications infer the structure of a probabilistic graphical model from
data to elucidate the relationships between variables. But how can we train
graphical models on a massive data set? In this paper, we show how to construct
coresets -compressed data sets which can be used as proxy for the original data
and have provably bounded worst case error- for Gaussian dependency networks
(DNs), i.e., cyclic directed graphical models over Gaussians, where the parents
of each variable are its Markov blanket. Specifically, we prove that Gaussian
DNs admit coresets of size independent of the size of the data set.
Unfortunately, this does not extend to DNs over members of the exponential
family in general. As we will prove, Poisson DNs do not admit small coresets.
Despite this worst-case result, we will provide an argument why our coreset
construction for DNs can still work well in practice on count data. To
corroborate our theoretical results, we empirically evaluated the resulting
Core DNs on real data sets. The results
Fully adaptive algorithm for pure exploration in linear bandits
Authors: Liyuan Xu, Junya Honda, Masashi Sugiyama
Date: 2017-10-16 00:00:00

We propose the first fully-adaptive algorithm for pure exploration in linear
bandits---the task to find the arm with the largest expected reward, which
depends on an unknown parameter linearly. While existing methods partially or
entirely fix sequences of arm selections before observing rewards, our method
adaptively changes the arm selection strategy based on past observations at
each round. We show our sample complexity matches the achievable lower bound up
to a constant factor in an extreme case. Furthermore, we evaluate the
performance of the methods by simulations based on both synthetic setting and
real-world data, in which our method shows vast improvement over existing
methods.
A Maximal Heterogeneity Based Clustering Approach for Obtaining Samples
Authors: Megha Mishra, Chandrasekaran Anirudh Bhardwaj, Kalyani Desikan
Date: 2017-09-02 00:00:00
Tasks: Clustering

Medical and social sciences demand sampling techniques which are robust,
reliable, replicable and have the least dissimilarity between the samples
obtained. Majority of the applications of sampling use randomized sampling,
albeit with stratification where applicable. The randomized technique is not
consistent, and may provide different samples each time, and the different
samples themselves may not be similar to each other. In this paper, we
introduce a novel non-statistical no-replacement sampling technique called
Wobbly Center Algorithm, which relies on building clusters iteratively based on
maximizing the heterogeneity inside each cluster. The algorithm works on the
principle of stepwise building of clusters by finding the points with the
maximal distance from the cluster center. The obtained results are validated
statistically using Analysis of Variance tests by comparing the samples
obtained to check if they are representative of each other. The obtained
results generated from running the Wobbly Center algorithm on benchmark
datasets when compared against other sampling algorithms indicate the
superiority of the Wobbly Center Algorithm.
Entanglement Entropy of Target Functions for Image Classification and Convolutional Neural Network
Authors: Ya-Hui Zhang
Date: 2017-10-16 00:00:00
Tasks: Classification, General Classification, image-classification, Image Classification

The success of deep convolutional neural network (CNN) in computer vision
especially image classification problems requests a new information theory for
function of image, instead of image itself. In this article, after establishing
a deep mathematical connection between image classification problem and quantum
spin model, we propose to use entanglement entropy, a generalization of
classical Boltzmann-Shannon entropy, as a powerful tool to characterize the
information needed for representation of general function of image. We prove
that there is a sub-volume-law bound for entanglement entropy of target
functions of reasonable image classification problems. Therefore target
functions of image classification only occupy a small subspace of the whole
Hilbert space. As a result, a neural network with polynomial number of
parameters is efficient for representation of such target functions of image.
The concept of entanglement entropy can also be useful to characterize the
expressive power of different neural networks. For example, we show that to
maintain the same expressive power, number of channels $D$ in a convolutional
neural network should scale with the number of convolution layers $n_c$ as
$D\sim D_0^{\frac{1}{n_c}}$. Therefore, deeper CNN with large $n_c$ is more
efficient than shallow ones.
Robust Maximum Likelihood Estimation of Sparse Vector Error Correction Model
Authors: Ziping Zhao, Daniel P. Palomar
Date: 2017-10-16 00:00:00
Tasks: Dimensionality Reduction, Econometrics, feature selection, Time Series, Time Series Analysis

In econometrics and finance, the vector error correction model (VECM) is an
important time series model for cointegration analysis, which is used to
estimate the long-run equilibrium variable relationships. The traditional
analysis and estimation methodologies assume the underlying Gaussian
distribution but, in practice, heavy-tailed data and outliers can lead to the
inapplicability of these methods. In this paper, we propose a robust model
estimation method based on the Cauchy distribution to tackle this issue. In
addition, sparse cointegration relations are considered to realize feature
selection and dimension reduction. An efficient algorithm based on the
majorization-minimization (MM) method is applied to solve the proposed
nonconvex problem. The performance of this algorithm is shown through numerical
simulations.
Aligned Image-Word Representations Improve Inductive Transfer Across Vision-Language Tasks
Authors: Tanmay Gupta, Kevin Shih, Saurabh Singh, Derek Hoiem
Date: 2017-04-02 00:00:00
Proceeding: ICCV 2017 10
Tasks: Multi-Task Learning, Question Answering, Visual Question Answering, Visual Question Answering (VQA)

An important goal of computer vision is to build systems that learn visual
representations over time that can be applied to many tasks. In this paper, we
investigate a vision-language embedding as a core representation and show that
it leads to better cross-task transfer than standard multi-task learning. In
particular, the task of visual recognition is aligned to the task of visual
question answering by forcing each to use the same word-region embeddings. We
show this leads to greater inductive transfer from recognition to VQA than
standard multitask learning. Visual recognition also improves, especially for
categories that have relatively few recognition training labels but appear
often in the VQA setting. Thus, our paper takes a small step towards creating
more general vision systems by showing the benefit of interpretable, flexible,
and trainable core representations.
The Feeling of Success: Does Touch Sensing Help Predict Grasp Outcomes?
Authors: Roberto Calandra, Andrew Owens, Manu Upadhyaya, Wenzhen Yuan, Justin Lin, Edward H. Adelson, Sergey Levine
Date: 2017-10-16 00:00:00
Tasks: Industrial Robots, Robotic Grasping

A successful grasp requires careful balancing of the contact forces. Deducing whether a particular grasp will be successful from indirect measurements, such as vision, is therefore quite challenging, and direct sensing of contacts through touch sensing provides an appealing avenue toward more successful and consistent robotic grasping. However, in order to fully evaluate the value of touch sensing for grasp outcome prediction, we must understand how touch sensing can influence outcome prediction accuracy when combined with other modalities. Doing so using conventional model-based techniques is exceptionally difficult. In this work, we investigate the question of whether touch sensing aids in predicting grasp outcomes within a multimodal sensing framework that combines vision and touch. To that end, we collected more than 9,000 grasping trials using a two-finger gripper equipped with GelSight high-resolution tactile sensors on each finger, and evaluated visuo-tactile deep neural network models to directly predict grasp outcomes from either modality individually, and from both modalities together. Our experimental results indicate that incorporating tactile readings substantially improve grasping performance.
Toward Crowd-Sensitive Path Planning
Authors: Anoop Aroor, Susan L. Epstein
Date: 2017-10-16 00:00:00

If a robot can predict crowds in parts of its environment that are
inaccessible to its sensors, then it can plan to avoid them. This paper
proposes a fast, online algorithm that learns average crowd densities in
different areas. It also describes how these densities can be incorporated into
existing navigation architectures. In simulation across multiple challenging
crowd scenarios, the robot reaches its target faster, travels less, and risks
fewer collisions than if it were to plan with the traditional A* algorithm.
A multi-branch convolutional neural network for detecting double JPEG compression
Authors: Bin Li, Hu Luo, Haoxin Zhang, Shunquan Tan, Zhongzhou Ji
Date: 2017-10-16 00:00:00

Detection of double JPEG compression is important to forensics analysis. A
few methods were proposed based on convolutional neural networks (CNNs). These
methods only accept inputs from pre-processed data, such as histogram features
and/or decompressed images. In this paper, we present a CNN solution by using
raw DCT (discrete cosine transformation) coefficients from JPEG images as
input. Considering the DCT sub-band nature in JPEG, a multiple-branch CNN
structure has been designed to reveal whether a JPEG format image has been
doubly compressed. Comparing with previous methods, the proposed method
provides end-to-end detection capability. Extensive experiments have been
carried out to demonstrate the effectiveness of the proposed network.
FRIST - Flipping and Rotation Invariant Sparsifying Transform Learning and Applications
Authors: Bihan Wen, Saiprasad Ravishankar, Yoram Bresler
Date: 2015-11-19 00:00:00
Tasks: compressed sensing, Denoising, Dictionary Learning, Image Reconstruction

Features based on sparse representation, especially using the synthesis
dictionary model, have been heavily exploited in signal processing and computer
vision. However, synthesis dictionary learning typically involves NP-hard
sparse coding and expensive learning steps. Recently, sparsifying transform
learning received interest for its cheap computation and its optimal updates in
the alternating algorithms. In this work, we develop a methodology for learning
Flipping and Rotation Invariant Sparsifying Transforms, dubbed FRIST, to better
represent natural images that contain textures with various geometrical
directions. The proposed alternating FRIST learning algorithm involves
efficient optimal updates. We provide a convergence guarantee, and demonstrate
the empirical convergence behavior of the proposed FRIST learning approach.
Preliminary experiments show the promising performance of FRIST learning for
sparse image representation, segmentation, denoising, robust inpainting, and
compressed sensing-based magnetic resonance image reconstruction.
Safe Learning of Quadrotor Dynamics Using Barrier Certificates
Authors: Li Wang, Evangelos A. Theodorou, Magnus Egerstedt
Date: 2017-10-16 00:00:00
Tasks: Gaussian Processes

To effectively control complex dynamical systems, accurate nonlinear models
are typically needed. However, these models are not always known. In this
paper, we present a data-driven approach based on Gaussian processes that
learns models of quadrotors operating in partially unknown environments. What
makes this challenging is that if the learning process is not carefully
controlled, the system will go unstable, i.e., the quadcopter will crash. To
this end, barrier certificates are employed for safe learning. The barrier
certificates establish a non-conservative forward invariant safe region, in
which high probability safety guarantees are provided based on the statistics
of the Gaussian Process. A learning controller is designed to efficiently
explore those uncertain states and expand the barrier certified safe region
based on an adaptive sampling scheme. In addition, a recursive Gaussian Process
prediction method is developed to learn the complex quadrotor dynamics in
real-time. Simulation results are provided to demonstrate the effectiveness of
the proposed approach.
Flow: A Modular Learning Framework for Mixed Autonomy Traffic
Authors: Cathy Wu, Aboudy Kreidieh, Kanaad Parvate, Eugene Vinitsky, Alexandre M. Bayen
Date: 2017-10-16 00:00:00
Tasks: Autonomous Vehicles, Deep Reinforcement Learning, Reinforcement Learning, Reinforcement Learning (RL)

The rapid development of autonomous vehicles (AVs) holds vast potential for transportation systems through improved safety, efficiency, and access to mobility. However, the progression of these impacts, as AVs are adopted, is not well understood. Numerous technical challenges arise from the goal of analyzing the partial adoption of autonomy: partial control and observation, multi-vehicle interactions, and the sheer variety of scenarios represented by real-world networks. To shed light into near-term AV impacts, this article studies the suitability of deep reinforcement learning (RL) for overcoming these challenges in a low AV-adoption regime. A modular learning framework is presented, which leverages deep RL to address complex traffic dynamics. Modules are composed to capture common traffic phenomena (stop-and-go traffic jams, lane changing, intersections). Learned control laws are found to improve upon human driving performance, in terms of system-level velocity, by up to 57% with only 4-7% adoption of AVs. Furthermore, in single-lane traffic, a small neural network control law with only local observation is found to eliminate stop-and-go traffic - surpassing all known model-based controllers to achieve near-optimal performance - and generalize to out-of-distribution traffic densities.
Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networks
Authors: Peter L. Bartlett, Nick Harvey, Chris Liaw, Abbas Mehrabian
Date: 2017-03-08 00:00:00

We prove new upper and lower bounds on the VC-dimension of deep neural
networks with the ReLU activation function. These bounds are tight for almost
the entire range of parameters. Letting $W$ be the number of weights and $L$ be
the number of layers, we prove that the VC-dimension is $O(W L \log(W))$, and
provide examples with VC-dimension $\Omega( W L \log(W/L) )$. This improves
both the previously known upper bounds and lower bounds. In terms of the number
$U$ of non-linear units, we prove a tight bound $\Theta(W U)$ on the
VC-dimension. All of these bounds generalize to arbitrary piecewise linear
activation functions, and also hold for the pseudodimensions of these function
classes.
  Combined with previous results, this gives an intriguing range of
dependencies of the VC-dimension on depth for networks with different
non-linearities: there is no dependence for piecewise-constant, linear
dependence for piecewise-linear, and no more than quadratic dependence for
general piecewise-polynomial.
Emergence of Invariance and Disentanglement in Deep Representations
Authors: Alessandro Achille, Stefano Soatto
Date: 2017-06-05 00:00:00
Tasks: Disentanglement

Using established principles from Statistics and Information Theory, we show
that invariance to nuisance factors in a deep neural network is equivalent to
information minimality of the learned representation, and that stacking layers
and injecting noise during training naturally bias the network towards learning
invariant representations. We then decompose the cross-entropy loss used during
training and highlight the presence of an inherent overfitting term. We propose
regularizing the loss by bounding such a term in two equivalent ways: One with
a Kullbach-Leibler term, which relates to a PAC-Bayes perspective; the other
using the information in the weights as a measure of complexity of a learned
model, yielding a novel Information Bottleneck for the weights. Finally, we
show that invariance and independence of the components of the representation
learned by the network are bounded above and below by the information in the
weights, and therefore are implicitly optimized during training. The theory
enables us to quantify and predict sharp phase transitions between underfitting
and overfitting of random labels when using our regularized loss, which we
verify in experiments, and sheds light on the relation between the geometry of
the loss function, invariance properties of the learned representation, and
generalization error.
Semi-Supervised Approach to Monitoring Clinical Depressive Symptoms in Social Media
Authors: Amir Hossein Yazdavar, Hussein S. Al-Olimat, Monireh Ebrahimi, Goonmeet Bajaj, Tanvi Banerjee, Krishnaprasad Thirunarayan, Jyotishman Pathak, Amit Sheth
Date: 2017-10-16 00:00:00

With the rise of social media, millions of people are routinely expressing
their moods, feelings, and daily struggles with mental health issues on social
media platforms like Twitter. Unlike traditional observational cohort studies
conducted through questionnaires and self-reported surveys, we explore the
reliable detection of clinical depression from tweets obtained unobtrusively.
Based on the analysis of tweets crawled from users with self-reported
depressive symptoms in their Twitter profiles, we demonstrate the potential for
detecting clinical depression symptoms which emulate the PHQ-9 questionnaire
clinicians use today. Our study uses a semi-supervised statistical model to
evaluate how the duration of these symptoms and their expression on Twitter (in
terms of word usage patterns and topical preferences) align with the medical
findings reported via the PHQ-9. Our proactive and automatic screening tool is
able to identify clinical depressive symptoms with an accuracy of 68% and
precision of 72%.