pwc-archive/papers-with-abstracts huggingface dataset:
Deep convolutional Gaussian processes
Authors: Kenneth Blomqvist, Samuel Kaski, Markus Heinonen
Date: 2018-10-06 00:00:00
Tasks: Classification, Gaussian Processes, General Classification, image-classification, Image Classification

We propose deep convolutional Gaussian processes, a deep Gaussian process
architecture with convolutional structure. The model is a principled Bayesian
framework for detecting hierarchical combinations of local features for image
classification. We demonstrate greatly improved image classification
performance compared to current Gaussian process approaches on the MNIST and
CIFAR-10 datasets. In particular, we improve CIFAR-10 accuracy by over 10
percentage points.
Provable Subspace Tracking from Missing Data and Matrix Completion
Authors: Praneeth Narayanamurthy, Vahid Daneshpajooh, Namrata Vaswani
Date: 2018-10-06 00:00:00
Tasks: Low-Rank Matrix Completion, Matrix Completion

We study the problem of subspace tracking in the presence of missing data (ST-miss). In recent work, we studied a related problem called robust ST. In this work, we show that a simple modification of our robust ST solution also provably solves ST-miss and robust ST-miss. To our knowledge, our result is the first `complete' guarantee for ST-miss. This means that we can prove that under assumptions on only the algorithm inputs, the output subspace estimates are close to the true data subspaces at all times. Our guarantees hold under mild and easily interpretable assumptions, and allow the underlying subspace to change with time in a piecewise constant fashion. In contrast, all existing guarantees for ST are partial results and assume a fixed unknown subspace. Extensive numerical experiments are shown to back up our theoretical claims. Finally, our solution can be interpreted as a provably correct mini-batch and memory-efficient solution to low-rank Matrix Completion (MC).
Bayes-CPACE: PAC Optimal Exploration in Continuous Space Bayes-Adaptive Markov Decision Processes
Authors: Gilwoo Lee, Sanjiban Choudhury, Brian Hou, Siddhartha S. Srinivasa
Date: 2018-10-06 00:00:00

We present the first PAC optimal algorithm for Bayes-Adaptive Markov Decision
Processes (BAMDPs) in continuous state and action spaces, to the best of our
knowledge. The BAMDP framework elegantly addresses model uncertainty by
incorporating Bayesian belief updates into long-term expected return. However,
computing an exact optimal Bayesian policy is intractable. Our key insight is
to compute a near-optimal value function by covering the continuous
state-belief-action space with a finite set of representative samples and
exploiting the Lipschitz continuity of the value function. We prove the
near-optimality of our algorithm and analyze a number of schemes that boost the
algorithm's efficiency. Finally, we empirically validate our approach on a
number of discrete and continuous BAMDPs and show that the learned policy has
consistently competitive performance against baseline approaches.
Artificial Intelligence for Diabetes Case Management: The Intersection of Physical and Mental Health
Authors: Casey C. Bennett
Date: 2018-10-06 00:00:00
Tasks: Clustering, Feature Engineering, Management

Diabetes is a major public health problem in the United States, affecting roughly 30 million people. Diabetes complications, along with the mental health comorbidities that often co-occur with them, are major drivers of high healthcare costs, poor outcomes, and reduced treatment adherence in diabetes. Here, we evaluate in a large state-wide population whether we can use artificial intelligence (AI) techniques to identify clusters of patient trajectories within the broader diabetes population in order to create cost-effective, narrowly-focused case management intervention strategies to reduce development of complications. This approach combined data from: 1) claims, 2) case management notes, and 3) social determinants of health from ~300,000 real patients between 2014 and 2016. We categorized complications as five types: Cardiovascular, Neuropathy, Opthalmic, Renal, and Other. Modeling was performed combining a variety of machine learning algorithms, including supervised classification, unsupervised clustering, natural language processing of unstructured care notes, and feature engineering. The results showed that we can predict development of diabetes complications roughly 83.5% of the time using claims data or social determinants of health data. They also showed we can reveal meaningful clusters in the patient population related to complications and mental health that can be used to cost-effective screening program, reducing the number of patients to be screened down by 85%. This study outlines creation of an AI framework to develop protocols to better address mental health comorbidities that lead to complications development in the diabetes population. Future work is described that outlines potential lines of research and the need for better addressing the 'people side' of the equation.
Robustness via Retrying: Closed-Loop Robotic Manipulation with Self-Supervised Learning
Authors: Frederik Ebert, Sudeep Dasari, Alex X. Lee, Sergey Levine, Chelsea Finn
Date: 2018-10-06 00:00:00
Tasks: Image Registration, Self-Supervised Learning, Video Prediction

Prediction is an appealing objective for self-supervised learning of
behavioral skills, particularly for autonomous robots. However, effectively
utilizing predictive models for control, especially with raw image inputs,
poses a number of major challenges. How should the predictions be used? What
happens when they are inaccurate? In this paper, we tackle these questions by
proposing a method for learning robotic skills from raw image observations,
using only autonomously collected experience. We show that even an imperfect
model can complete complex tasks if it can continuously retry, but this
requires the model to not lose track of the objective (e.g., the object of
interest). To enable a robot to continuously retry a task, we devise a
self-supervised algorithm for learning image registration, which can keep track
of objects of interest for the duration of the trial. We demonstrate that this
idea can be combined with a video-prediction based controller to enable complex
behaviors to be learned from scratch using only raw visual inputs, including
grasping, repositioning objects, and non-prehensile manipulation. Our
real-world experiments demonstrate that a model trained with 160 robot hours of
autonomously collected, unlabeled data is able to successfully perform complex
manipulation tasks with a wide range of objects not seen during training.
Why do Larger Models Generalize Better? A Theoretical Perspective via the XOR Problem
Authors: Alon Brutzkus, Amir Globerson
Date: 2018-10-06 00:00:00
Proceeding: ICLR 2019 5
Tasks: Clustering

Empirical evidence suggests that neural networks with ReLU activations
generalize better with over-parameterization. However, there is currently no
theoretical analysis that explains this observation. In this work, we provide
theoretical and empirical evidence that, in certain cases, overparameterized
convolutional networks generalize better than small networks because of an
interplay between weight clustering and feature exploration at initialization.
We demonstrate this theoretically for a 3-layer convolutional neural network
with max-pooling, in a novel setting which extends the XOR problem. We show
that this interplay implies that with overparamterization, gradient descent
converges to global minima with better generalization performance compared to
global minima of small networks. Empirically, we demonstrate these phenomena
for a 3-layer convolutional neural network in the MNIST task.
Constructing Graph Node Embeddings via Discrimination of Similarity Distributions
Authors: Stanislav Tsepa, Maxim Panov
Date: 2018-10-06 00:00:00
Tasks: Link Prediction

The problem of unsupervised learning node embeddings in graphs is one of the
important directions in modern network science. In this work we propose a novel
framework, which is aimed to find embeddings by \textit{discriminating
distributions of similarities (DDoS)} between nodes in the graph. The general
idea is implemented by maximizing the \textit{earth mover distance} between
distributions of decoded similarities of similar and dissimilar nodes. The
resulting algorithm generates embeddings which give a state-of-the-art
performance in the problem of link prediction in real-world graphs.
Text-based Sentiment Analysis and Music Emotion Recognition
Authors: Erion Çano
Date: 2018-10-06 00:00:00
Tasks: Emotion Recognition, Music Emotion Recognition, Recommendation Systems, Sentiment Analysis, Word Embeddings

Sentiment polarity of tweets, blog posts or product reviews has become highly
attractive and is utilized in recommender systems, market predictions, business
intelligence and more. Deep learning techniques are becoming top performers on
analyzing such texts. There are however several problems that need to be solved
for efficient use of deep neural networks on text mining and text polarity
analysis. First, deep neural networks need to be fed with data sets that are
big in size as well as properly labeled. Second, there are various
uncertainties regarding the use of word embedding vectors: should they be
generated from the same data set that is used to train the model or it is
better to source them from big and popular collections? Third, to simplify
model creation it is convenient to have generic neural network architectures
that are effective and can adapt to various texts, encapsulating much of design
complexity. This thesis addresses the above problems to provide methodological
and practical insights for utilizing neural networks on sentiment analysis of
texts and achieving state of the art results. Regarding the first problem, the
effectiveness of various crowdsourcing alternatives is explored and two
medium-sized and emotion-labeled song data sets are created utilizing social
tags. To address the second problem, a series of experiments with large text
collections of various contents and domains were conducted, trying word
embeddings of various parameters. Regarding the third problem, a series of
experiments involving convolution and max-pooling neural layers were conducted.
Combining convolutions of words, bigrams, and trigrams with regional
max-pooling layers in a couple of stacks produced the best results. The derived
architecture achieves competitive performance on sentiment polarity analysis of
movie, business and product reviews.
Discretizing Logged Interaction Data Biases Learning for Decision-Making
Authors: Peter Schulam, Suchi Saria
Date: 2018-10-06 00:00:00
Tasks: Decision Making, Time Series, Time Series Analysis

Time series data that are not measured at regular intervals are commonly
discretized as a preprocessing step. For example, data about customer arrival
times might be simplified by summing the number of arrivals within hourly
intervals, which produces a discrete-time time series that is easier to model.
In this abstract, we show that discretization introduces a bias that affects
models trained for decision-making. We refer to this phenomenon as
discretization bias, and show that we can avoid it by using continuous-time
models instead.
Learning to Optimize under Non-Stationarity
Authors: Wang Chi Cheung, David Simchi-Levi, Ruihao Zhu
Date: 2018-10-06 00:00:00

We introduce algorithms that achieve state-of-the-art \emph{dynamic regret} bounds for non-stationary linear stochastic bandit setting. It captures natural applications such as dynamic pricing and ads allocation in a changing environment. We show how the difficulty posed by the non-stationarity can be overcome by a novel marriage between stochastic and adversarial bandits learning algorithms. Defining $d,B_T,$ and $T$ as the problem dimension, the \emph{variation budget}, and the total time horizon, respectively, our main contributions are the tuned Sliding Window UCB (\texttt{SW-UCB}) algorithm with optimal $\widetilde{O}(d^{2/3}(B_T+1)^{1/3}T^{2/3})$ dynamic regret, and the tuning free bandit-over-bandit (\texttt{BOB}) framework built on top of the \texttt{SW-UCB} algorithm with best $\widetilde{O}(d^{2/3}(B_T+1)^{1/4}T^{3/4})$ dynamic regret.
h-detach: Modifying the LSTM Gradient Towards Better Optimization
Authors: Devansh Arpit, Bhargav Kanuparthi, Giancarlo Kerg, Nan Rosemary Ke, Ioannis Mitliagkas, Yoshua Bengio
Date: 2018-10-06 00:00:00
Proceeding: ICLR 2019 5

Recurrent neural networks are known for their notorious exploding and
vanishing gradient problem (EVGP). This problem becomes more evident in tasks
where the information needed to correctly solve them exist over long time
scales, because EVGP prevents important gradient components from being
back-propagated adequately over a large number of steps. We introduce a simple
stochastic algorithm (\textit{h}-detach) that is specific to LSTM optimization
and targeted towards addressing this problem. Specifically, we show that when
the LSTM weights are large, the gradient components through the linear path
(cell state) in the LSTM computational graph get suppressed. Based on the
hypothesis that these components carry information about long term dependencies
(which we show empirically), their suppression can prevent LSTMs from capturing
them. Our algorithm\footnote{Our code is available at
https://github.com/bhargav104/h-detach.} prevents gradients flowing through
this path from getting suppressed, thus allowing the LSTM to capture such
dependencies better. We show significant improvements over vanilla LSTM
gradient based training in terms of convergence speed, robustness to seed and
learning rate, and generalization using our modification of LSTM gradient on
various benchmark datasets.
When logic lays down the law
Authors: Bjørn Jespersen, Ana de Almeida Borges, Jorge del Castillo Tierz, Juan José Conejero Rodríguez, Eric Sancho Adamson, Aleix Solé Sánchez, Nika Pona, Joost J. Joosten
Date: 2018-10-06 00:00:00

We analyse so-called computable laws, i.e., laws that can be enforced by
automatic procedures. These laws should be logically perfect and unambiguous,
but sometimes they are not. We use a regulation on road transport to illustrate
this issue, and show what some fragments of this regulation would look like if
rewritten in the image of logic. We further propose desiderata to be fulfilled
by computable laws, and provide a critical platform from which to assess
existing laws and a guideline for composing future ones.
Context-Aware Deep Spatio-Temporal Network for Hand Pose Estimation from Depth Images
Authors: Yiming Wu, Wei Ji, Xi Li, Gang Wang, Jianwei Yin, Fei Wu
Date: 2018-10-06 00:00:00
Tasks: Hand Pose Estimation, Pose Estimation

As a fundamental and challenging problem in computer vision, hand pose
estimation aims to estimate the hand joint locations from depth images.
Typically, the problem is modeled as learning a mapping function from images to
hand joint coordinates in a data-driven manner. In this paper, we propose
Context-Aware Deep Spatio-Temporal Network (CADSTN), a novel method to jointly
model the spatio-temporal properties for hand pose estimation. Our proposed
network is able to learn the representations of the spatial information and the
temporal structure from the image sequences. Moreover, by adopting adaptive
fusion method, the model is capable of dynamically weighting different
predictions to lay emphasis on sufficient context. Our method is examined on
two common benchmarks, the experimental results demonstrate that our proposed
approach achieves the best or the second-best performance with state-of-the-art
methods and runs in 60fps.
Camera Model Identification Using Convolutional Neural Networks
Authors: Artur Kuzin, Artur Fattakhov, Ilya Kibardin, Vladimir Iglovikov, Ruslan Dautov
Date: 2018-10-06 00:00:00
Tasks: model

Source camera identification is the process of determining which camera or
model has been used to capture an image. In the recent years, there has been a
rapid growth of research interest in the domain of forensics. In the current
work, we describe our Deep Learning approach to the camera detection task of 10
cameras as a part of the Camera Model Identification Challenge hosted by
Kaggle.com where our team finished 2nd out of 582 teams with the accuracy on
the unseen data of 98%. We used aggressive data augmentations that allowed a
model to stay robust against transformations. A number of experiments are
carried out on datasets collected by organizers and scraped from the web.
Personality facets recognition from text
Authors: Wesley Ramos dos Santos, Ivandre Paraboni
Date: 2018-10-06 00:00:00

Fundamental Big Five personality traits (e.g., Extraversion) and their facets
(e.g., Activity) are known to correlate with a broad range of linguistic
features and, accordingly, the recognition of personality traits from text is a
well-known Natural Language Processing task. Labelling text data with facets
information, however, may require the use of lengthy personality inventories,
and perhaps for that reason existing computational models of this kind are
usually limited to the recognition of the fundamental traits. Based on these
observations, this paper investigates the issue of personality facets
recognition from text labelled only with information available from a shorter
personality inventory. In doing so, we provide a low-cost model for the
recognition of certain personality facets, and present reference results for
further studies in this field.
Anytime Stochastic Gradient Descent: A Time to Hear from all the Workers
Authors: Nuwan Ferdinand, Stark Draper
Date: 2018-10-06 00:00:00
Tasks: All

In this paper, we focus on approaches to parallelizing stochastic gradient
descent (SGD) wherein data is farmed out to a set of workers, the results of
which, after a number of updates, are then combined at a central master node.
Although such synchronized SGD approaches parallelize well in idealized
computing environments, they often fail to realize their promised computational
acceleration in practical settings. One cause is slow workers, termed
stragglers, who can cause the fusion step at the master node to stall, which
greatly slowing convergence. In many straggler mitigation approaches work
completed by these nodes, while only partial, is discarded completely. In this
paper, we propose an approach to parallelizing synchronous SGD that exploits
the work completed by all workers. The central idea is to fix the computation
time of each worker and then to combine distinct contributions of all workers.
We provide a convergence analysis and optimize the combination function. Our
numerical results demonstrate an improvement of several factors of magnitude in
comparison to existing methods.
Constraint-Based Visual Generation
Authors: Giuseppe Marra, Francesco Giannini, Michelangelo Diligenti, Marco Gori
Date: 2018-07-16 00:00:00

In the last few years the systematic adoption of deep learning to visual generation has produced impressive results that, amongst others, definitely benefit from the massive exploration of convolutional architectures. In this paper, we propose a general approach to visual generation that combines learning capabilities with logic descriptions of the target to be generated. The process of generation is regarded as a constrained satisfaction problem, where the constraints describe a set of properties that characterize the target. Interestingly, the constraints can also involve logic variables, while all of them are converted into real-valued functions by means of the t-norm theory. We use deep architectures to model the involved variables, and propose a computational scheme where the learning process carries out a satisfaction of the constraints. We propose some examples in which the theory can naturally be used, including the modeling of GAN and auto-encoders, and report promising results in problems with the generation of handwritten characters and face transformations.
Understanding Recurrent Neural Architectures by Analyzing and Synthesizing Long Distance Dependencies in Benchmark Sequential Datasets
Authors: Abhijit Mahalunkar, John D. Kelleher
Date: 2018-10-06 00:00:00
Tasks: Benchmarking, Language Modeling, Language Modelling

In order to build efficient deep recurrent neural architectures, it is essential to analyze the complexityof long distance dependencies (LDDs) of the dataset being modeled. In this paper, we presentdetailed analysis of the dependency decay curve exhibited by various datasets. The datasets sampledfrom a similar process (e.g. natural language, sequential MNIST, Strictlyk-Piecewise languages,etc) display variations in the properties of the dependency decay curve. Our analysis reveal thefactors resulting in these variations; such as (i) number of unique symbols in a dataset, (ii) size ofthe dataset, (iii) number of interacting symbols within a given LDD, and (iv) the distance betweenthe interacting symbols. We test these factors by generating synthesized datasets of the Strictlyk-Piecewise languages. Another advantage of these synthesized datasets is that they enable targetedtesting of deep recurrent neural architectures in terms of their ability to model LDDs with differentcharacteristics. We also demonstrate that analysing dependency decay curves can inform the selectionof optimal hyper-parameters for SOTA deep recurrent neural architectures. This analysis can directlycontribute to the development of more accurate and efficient sequential models.
Higher-order Spectral Clustering for Heterogeneous Graphs
Authors: Aldo G. Carranza, Ryan A. Rossi, Anup Rao, Eunyee Koh
Date: 2018-10-06 00:00:00
Tasks: Clustering, Link Prediction

Higher-order connectivity patterns such as small induced sub-graphs called
graphlets (network motifs) are vital to understand the important components
(modules/functional units) governing the configuration and behavior of complex
networks. Existing work in higher-order clustering has focused on simple
homogeneous graphs with a single node/edge type. However, heterogeneous graphs
consisting of nodes and edges of different types are seemingly ubiquitous in
the real-world. In this work, we introduce the notion of typed-graphlet that
explicitly captures the rich (typed) connectivity patterns in heterogeneous
networks. Using typed-graphlets as a basis, we develop a general principled
framework for higher-order clustering in heterogeneous networks. The framework
provides mathematical guarantees on the optimality of the higher-order
clustering obtained. The experiments demonstrate the effectiveness of the
framework quantitatively for three important applications including (i)
clustering, (ii) link prediction, and (iii) graph compression. In particular,
the approach achieves a mean improvement of 43x over all methods and graphs for
clustering while achieving a 18.7% and 20.8% improvement for link prediction
and graph compression, respectively.