pwc-archive/papers-with-abstracts huggingface dataset:
Asynchronous Stochastic Gradient MCMC with Elastic Coupling
Authors: Jost Tobias Springenberg, Aaron Klein, Stefan Falkner, Frank Hutter
Date: 2016-12-02 00:00:00

We consider parallel asynchronous Markov Chain Monte Carlo (MCMC) sampling
for problems where we can leverage (stochastic) gradients to define continuous
dynamics which explore the target distribution. We outline a solution strategy
for this setting based on stochastic gradient Hamiltonian Monte Carlo sampling
(SGHMC) which we alter to include an elastic coupling term that ties together
multiple MCMC instances. The proposed strategy turns inherently sequential HMC
algorithms into asynchronous parallel versions. First experiments empirically
show that the resulting parallel sampler significantly speeds up exploration of
the target distribution, when compared to standard SGHMC, and is less prone to
the harmful effects of stale gradients than a naive parallelization approach.
Distributed Optimization of Multi-Class SVMs
Authors: Maximilian Alber, Julian Zimmert, Urun Dogan, Marius Kloft
Date: 2016-11-25 00:00:00
Tasks: Image Classification

Training of one-vs.-rest SVMs can be parallelized over the number of classes
in a straight forward way. Given enough computational resources, one-vs.-rest
SVMs can thus be trained on data involving a large number of classes. The same
cannot be stated, however, for the so-called all-in-one SVMs, which require
solving a quadratic program of size quadratically in the number of classes. We
develop distributed algorithms for two all-in-one SVM formulations (Lee et al.
and Weston and Watkins) that parallelize the computation evenly over the number
of classes. This allows us to compare these models to one-vs.-rest SVMs on
unprecedented scale. The results indicate superior accuracy on text
classification data.
An unexpected unity among methods for interpreting model predictions
Authors: Scott Lundberg, Su-In Lee
Date: 2016-11-22 00:00:00
Tasks: Unity

Understanding why a model made a certain prediction is crucial in many data
science fields. Interpretable predictions engender appropriate trust and
provide insight into how the model may be improved. However, with large modern
datasets the best accuracy is often achieved by complex models even experts
struggle to interpret, which creates a tension between accuracy and
interpretability. Recently, several methods have been proposed for interpreting
predictions from complex models by estimating the importance of input features.
Here, we present how a model-agnostic additive representation of the importance
of input features unifies current methods. This representation is optimal, in
the sense that it is the only set of additive values that satisfies important
properties. We show how we can leverage these properties to create novel visual
explanations of model predictions. The thread of unity that this representation
weaves through the literature indicates that there are common principles to be
learned about the interpretation of model predictions that apply in many
scenarios.
Multi-class classification: mirror descent approach
Authors: Daria Reshetova
Date: 2016-06-30 00:00:00
Tasks: Classification, General Classification, Multi-class Classification

We consider the problem of multi-class classification and a stochastic opti-
mization approach to it. We derive risk bounds for stochastic mirror descent
algorithm and provide examples of set geometries that make the use of the
algorithm efficient in terms of error in k.
Contextual Visual Similarity
Authors: Xiaofang Wang, Kris M. Kitani, Martial Hebert
Date: 2016-12-08 00:00:00
Tasks: Attribute, Image Retrieval, Image Similarity Search

Measuring visual similarity is critical for image understanding. But what
makes two images similar? Most existing work on visual similarity assumes that
images are similar because they contain the same object instance or category.
However, the reason why images are similar is much more complex. For example,
from the perspective of category, a black dog image is similar to a white dog
image. However, in terms of color, a black dog image is more similar to a black
horse image than the white dog image. This example serves to illustrate that
visual similarity is ambiguous but can be made precise when given an explicit
contextual perspective. Based on this observation, we propose the concept of
contextual visual similarity. To be concrete, we examine the concept of
contextual visual similarity in the application domain of image search. Instead
of providing only a single image for image similarity search (\eg, Google image
search), we require three images. Given a query image, a second positive image
and a third negative image, dissimilar to the first two images, we define a
contextualized similarity search criteria. In particular, we learn feature
weights over all the feature dimensions of each image such that the distance
between the query image and the positive image is small and their distances to
the negative image are large after reweighting their features. The learned
feature weights encode the contextualized visual similarity specified by the
user and can be used for attribute specific image search. We also show the
usefulness of our contextualized similarity weighting scheme for different
tasks, such as answering visual analogy questions and unsupervised attribute
discovery.
Smoothing Effects of Bagging: Von Mises Expansions of Bagged Statistical Functionals
Authors: Andreas Buja, Werner Stuetzle
Date: 2016-12-08 00:00:00

Bagging is a device intended for reducing the prediction error of learning
algorithms. In its simplest form, bagging draws bootstrap samples from the
training sample, applies the learning algorithm to each bootstrap sample, and
then averages the resulting prediction rules.
  We extend the definition of bagging from statistics to statistical
functionals and study the von Mises expansion of bagged statistical
functionals. We show that the expansion is related to the Efron-Stein ANOVA
expansion of the raw (unbagged) functional. The basic observation is that a
bagged functional is always smooth in the sense that the von Mises expansion
exists and is finite of length 1 + resample size $M$. This holds even if the
raw functional is rough or unstable. The resample size $M$ acts as a smoothing
parameter, where a smaller $M$ means more smoothing.
Geometric Decomposition of Feed Forward Neural Networks
Authors: Sven Cattell
Date: 2016-12-08 00:00:00

There have been several attempts to mathematically understand neural networks
and many more from biological and computational perspectives. The field has
exploded in the last decade, yet neural networks are still treated much like a
black box. In this work we describe a structure that is inherent to a feed
forward neural network. This will provide a framework for future work on neural
networks to improve training algorithms, compute the homology of the network,
and other applications. Our approach takes a more geometric point of view and
is unlike other attempts to mathematically understand neural networks that rely
on a functional perspective.
An Efficient Algorithm for the Piecewise-Smooth Model with Approximately Explicit Solutions
Authors: Huihui Song, yuhui Zheng, Kaihua Zhang
Date: 2016-12-08 00:00:00
Tasks: Image Segmentation, Segmentation, Semantic Segmentation

This paper presents an efficient approach to image segmentation that
approximates the piecewise-smooth (PS) functional in [12] with explicit
solutions. By rendering some rational constraints on the initial conditions and
the final solutions of the PS functional, we propose two novel formulations
which can be approximated to be the explicit solutions of the evolution partial
differential equations (PDEs) of the PS model, in which only one PDE needs to
be solved efficiently. Furthermore, an energy term that regularizes the level
set function to be a signed distance function is incorporated into our
evolution formulation, and the time-consuming re-initialization is avoided.
Experiments on synthetic and real images show that our method is more efficient
than both the PS model and the local binary fitting (LBF) model [4], while
having similar segmentation accuracy as the LBF model.
Stochastic Primal-Dual Methods and Sample Complexity of Reinforcement Learning
Authors: Yi-Chen Chen, Mengdi Wang
Date: 2016-12-08 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We study the online estimation of the optimal policy of a Markov decision
process (MDP). We propose a class of Stochastic Primal-Dual (SPD) methods which
exploit the inherent minimax duality of Bellman equations. The SPD methods
update a few coordinates of the value and policy estimates as a new state
transition is observed. These methods use small storage and has low
computational complexity per iteration. The SPD methods find an
absolute-$\epsilon$-optimal policy, with high probability, using
$\mathcal{O}\left(\frac{|\mathcal{S}|^4 |\mathcal{A}|^2\sigma^2
}{(1-\gamma)^6\epsilon^2} \right)$ iterations/samples for the infinite-horizon
discounted-reward MDP and $\mathcal{O}\left(\frac{|\mathcal{S}|^4
|\mathcal{A}|^2H^6\sigma^2 }{\epsilon^2} \right)$ for the finite-horizon MDP.
Complex Matrix Factorization for Face Recognition
Authors: Viet-Hang Duong, Yuan-Shan Lee, Bach-Tung Pham, Seksan Mathulaprangsan, Pham The Bao, Jia-Ching Wang
Date: 2016-12-08 00:00:00
Tasks: Face Recognition

This work developed novel complex matrix factorization methods for face
recognition; the methods were complex matrix factorization (CMF), sparse
complex matrix factorization (SpaCMF), and graph complex matrix factorization
(GraCMF). After real-valued data are transformed into a complex field, the
complex-valued matrix will be decomposed into two matrices of bases and
coefficients, which are derived from solutions to an optimization problem in a
complex domain. The generated objective function is the real-valued function of
the reconstruction error, which produces a parametric description. Factorizing
the matrix of complex entries directly transformed the constrained optimization
problem into an unconstrained optimization problem. Additionally, a complex
vector space with N dimensions can be regarded as a 2N-dimensional real vector
space. Accordingly, all real analytic properties can be exploited in the
complex field. The ability to exploit these important characteristics motivated
the development herein of a simpler framework that can provide better
recognition results. The effectiveness of this framework will be clearly
elucidated in later sections in this paper.
Discrete Schroedinger Transform For Texture Recognition
Authors: João B. Florindo, Odemir M. Bruno
Date: 2016-12-08 00:00:00

This work presents a new procedure to extract features of grey-level texture
images based on the discrete Schroedinger transform. This is a non-linear
transform where the image is mapped as the initial probability distribution of
a wave function and such distribution evolves in time following the
Schroedinger equation from Quantum Mechanics. The features are provided by
statistical moments of the distribution measured at different times. The
proposed method is applied to the classification of three databases of textures
used for benchmark and compared to other well-known texture descriptors in the
literature, such as textons, local binary patterns, multifractals, among
others. All of them are outperformed by the proposed method in terms of
percentage of images correctly classified. The proposal is also applied to the
identification of plant species using scanned images of leaves and again it
outperforms other texture methods. A test with images affected by Gaussian and
"salt \& pepper" noise is also carried out, also with the best performance
achieved by the Schroedinger descriptors.
Parallel Chromatic MCMC with Spatial Partitioning
Authors: Jun Song, David A. Moore
Date: 2016-12-02 00:00:00
Tasks: Event Detection

We introduce a novel approach for parallelizing MCMC inference in models with
spatially determined conditional independence relationships, for which existing
techniques exploiting graphical model structure are not applicable. Our
approach is motivated by a model of seismic events and signals, where events
detected in distant regions are approximately independent given those in
intermediate regions. We perform parallel inference by coloring a factor graph
defined over regions of latent space, rather than individual model variables.
Evaluating on a model of seismic event detection, we achieve significant
speedups over serial MCMC with no degradation in inference quality.
Research on the Multiple Feature Fusion Image Retrieval Algorithm based on Texture Feature and Rough Set Theory
Authors: Xiaojie Shi, Yijun Shao
Date: 2016-12-08 00:00:00
Tasks: Image Retrieval, Retrieval

Recently, we have witnessed the explosive growth of images with complex
information and content. In order to effectively and precisely retrieve desired
images from a large-scale image database with low time-consuming, we propose
the multiple feature fusion image retrieval algorithm based on the texture
feature and rough set theory in this paper. In contrast to the conventional
approaches that only use the single feature or standard, we fuse the different
features with operation of normalization. The rough set theory will assist us
to enhance the robustness of retrieval system when facing with incomplete data
warehouse. To enhance the texture extraction paradigm, we use the wavelet Gabor
function that holds better robustness. In addition, from the perspectives of
the internal and external normalization, we re-organize extracted feature with
the better combination. The numerical experiment has verified general
feasibility of our methodology. We enhance the overall accuracy compared with
the other state-of-the-art algorithms.
Bridging Medical Data Inference to Achilles Tendon Rupture Rehabilitation
Authors: An Qu, Cheng Zhang, Paul Ackermann, Hedvig Kjellström
Date: 2016-12-07 00:00:00
Tasks: Collaborative Filtering, Decision Making, Imputation, Predicting Patient Outcomes, Prediction, Recommendation Systems

Imputing incomplete medical tests and predicting patient outcomes are crucial
for guiding the decision making for therapy, such as after an Achilles Tendon
Rupture (ATR). We formulate the problem of data imputation and prediction for
ATR relevant medical measurements into a recommender system framework. By
applying MatchBox, which is a collaborative filtering approach, on a real
dataset collected from 374 ATR patients, we aim at offering personalized
medical data imputation and prediction. In this work, we show the feasibility
of this approach and discuss potential research directions by conducting
initial qualitative evaluations.
A Review of Intelligent Practices for Irrigation Prediction
Authors: Hans Krupakar, Akshay Jayakumar, Dhivya G
Date: 2016-12-07 00:00:00
Tasks: Management, Prediction, Time Series, Time Series Analysis

Population growth and increasing droughts are creating unprecedented strain
on the continued availability of water resources. Since irrigation is a major
consumer of fresh water, wastage of resources in this sector could have strong
consequences. To address this issue, irrigation water management and prediction
techniques need to be employed effectively and should be able to account for
the variabilities present in the environment. The different techniques surveyed
in this paper can be classified into two categories: computational and
statistical. Computational methods deal with scientific correlations between
physical parameters whereas statistical methods involve specific prediction
algorithms that can be used to automate the process of irrigation water
prediction. These algorithms interpret semantic relationships between the
various parameters of temperature, pressure, evapotranspiration etc. and store
them as numerical precomputed entities specific to the conditions and the area
used as the data for the training corpus used to train it. We focus on
reviewing the computational methods used to determine Evapotranspiration and
its implications. We compare the efficiencies of different data mining and
machine learning methods implemented in this area, such as Logistic Regression,
Decision Tress Classifier, SysFor, Support Vector Machine(SVM), Fuzzy Logic
techniques, Artifical Neural Networks(ANNs) and various hybrids of Genetic
Algorithms (GA) applied to irrigation prediction. We also recommend a possible
technique for the same based on its superior results in other such time series
analysis tasks.
Learning from Maps: Visual Common Sense for Autonomous Driving
Authors: Ari Seff, Jianxiong Xiao
Date: 2016-11-25 00:00:00
Tasks: Autonomous Driving, Autonomous Vehicles, Common Sense Reasoning, Navigate, Scene Understanding

Today's autonomous vehicles rely extensively on high-definition 3D maps to
navigate the environment. While this approach works well when these maps are
completely up-to-date, safe autonomous vehicles must be able to corroborate the
map's information via a real time sensor-based system. Our goal in this work is
to develop a model for road layout inference given imagery from on-board
cameras, without any reliance on high-definition maps. However, no sufficient
dataset for training such a model exists. Here, we leverage the availability of
standard navigation maps and corresponding street view images to construct an
automatically labeled, large-scale dataset for this complex scene understanding
problem. By matching road vectors and metadata from navigation maps with Google
Street View images, we can assign ground truth road layout attributes (e.g.,
distance to an intersection, one-way vs. two-way street) to the images. We then
train deep convolutional networks to predict these road layout attributes given
a single monocular RGB image. Experimental evaluation demonstrates that our
model learns to correctly infer the road attributes using only panoramas
captured by car-mounted cameras as input. Additionally, our results indicate
that this method may be suitable to the novel application of recommending
safety improvements to infrastructure (e.g., suggesting an alternative speed
limit for a street).
Towards the Modeling of Behavioral Trajectories of Users in Online Social Media
Authors: Alessandro Bessi
Date: 2016-11-17 00:00:00
Tasks: Clustering, Temporal Sequences

In this paper, we introduce a methodology that allows to model behavioral
trajectories of users in online social media. First, we illustrate how to
leverage the probabilistic framework provided by Hidden Markov Models (HMMs) to
represent users by embedding the temporal sequences of actions they performed
online. We then derive a model-based distance between trained HMMs, and we use
spectral clustering to find homogeneous clusters of users showing similar
behavioral trajectories. To provide platform-agnostic results, we apply the
proposed approach to two different online social media --- i.e. Facebook and
YouTube. We conclude discussing merits and limitations of our approach as well
as future and promising research directions.
Composing Music with Grammar Argumented Neural Networks and Note-Level Encoding
Authors: Zheng Sun, Jiaqi Liu, Zewang Zhang, Jingwen Chen, Zhao Huo, Ching Hua Lee, Xiao Zhang
Date: 2016-11-16 00:00:00
Tasks: Music Generation

Creating aesthetically pleasing pieces of art, including music, has been a
long-term goal for artificial intelligence research. Despite recent successes
of long-short term memory (LSTM) recurrent neural networks (RNNs) in sequential
learning, LSTM neural networks have not, by themselves, been able to generate
natural-sounding music conforming to music theory. To transcend this
inadequacy, we put forward a novel method for music composition that combines
the LSTM with Grammars motivated by music theory. The main tenets of music
theory are encoded as grammar argumented (GA) filters on the training data,
such that the machine can be trained to generate music inheriting the
naturalness of human-composed pieces from the original dataset while adhering
to the rules of music theory. Unlike previous approaches, pitches and durations
are encoded as one semantic entity, which we refer to as note-level encoding.
This allows easy implementation of music theory grammars, as well as closer
emulation of the thinking pattern of a musician. Although the GA rules are
applied to the training data and never directly to the LSTM music generation,
our machine still composes music that possess high incidences of diatonic scale
notes, small pitch intervals and chords, in deference to music theory.
Annotation Order Matters: Recurrent Image Annotator for Arbitrary Length Image Tagging
Authors: Jiren Jin, Hideki Nakayama
Date: 2016-04-18 00:00:00
Tasks: Image Captioning, Machine Translation, Management, Retrieval, TAG, Translation

Automatic image annotation has been an important research topic in
facilitating large scale image management and retrieval. Existing methods focus
on learning image-tag correlation or correlation between tags to improve
annotation accuracy. However, most of these methods evaluate their performance
using top-k retrieval performance, where k is fixed. Although such setting
gives convenience for comparing different methods, it is not the natural way
that humans annotate images. The number of annotated tags should depend on
image contents. Inspired by the recent progress in machine translation and
image captioning, we propose a novel Recurrent Image Annotator (RIA) model that
forms image annotation task as a sequence generation problem so that RIA can
natively predict the proper length of tags according to image contents. We
evaluate the proposed model on various image annotation datasets. In addition
to comparing our model with existing methods using the conventional top-k
evaluation measures, we also provide our model as a high quality baseline for
the arbitrary length image tagging task. Moreover, the results of our
experiments show that the order of tags in training phase has a great impact on
the final annotation performance.
Automatic Detection of ADHD and ASD from Expressive Behaviour in RGBD Data
Authors: Shashank Jaiswal, Michel Valstar, Alinda Gillott, David Daley
Date: 2016-12-07 00:00:00
Tasks: Diagnostic

Attention Deficit Hyperactivity Disorder (ADHD) and Autism Spectrum Disorder
(ASD) are neurodevelopmental conditions which impact on a significant number of
children and adults. Currently, the diagnosis of such disorders is done by
experts who employ standard questionnaires and look for certain behavioural
markers through manual observation. Such methods for their diagnosis are not
only subjective, difficult to repeat, and costly but also extremely time
consuming. In this work, we present a novel methodology to aid diagnostic
predictions about the presence/absence of ADHD and ASD by automatic visual
analysis of a person's behaviour. To do so, we conduct the questionnaires in a
computer-mediated way while recording participants with modern RGBD
(Colour+Depth) sensors. In contrast to previous automatic approaches which have
focussed only detecting certain behavioural markers, our approach provides a
fully automatic end-to-end system for directly predicting ADHD and ASD in
adults. Using state of the art facial expression analysis based on Dynamic Deep
Learning and 3D analysis of behaviour, we attain classification rates of 96%
for Controls vs Condition (ADHD/ASD) group and 94% for Comorbid (ADHD+ASD) vs
ASD only group. We show that our system is a potentially useful time saving
contribution to the diagnostic field of ADHD and ASD.