pwc-archive/papers-with-abstracts huggingface dataset:
Convergence of Value Aggregation for Imitation Learning
Authors: Ching-An Cheng, Byron Boots
Date: 2018-01-22 00:00:00
Tasks: Imitation Learning

Value aggregation is a general framework for solving imitation learning
problems. Based on the idea of data aggregation, it generates a policy sequence
by iteratively interleaving policy optimization and evaluation in an online
learning setting. While the existence of a good policy in the policy sequence
can be guaranteed non-asymptotically, little is known about the convergence of
the sequence or the performance of the last policy. In this paper, we debunk
the common belief that value aggregation always produces a convergent policy
sequence with improving performance. Moreover, we identify a critical stability
condition for convergence and provide a tight non-asymptotic bound on the
performance of the last policy. These new theoretical insights let us stabilize
problems with regularization, which removes the inconvenient process of
identifying the best policy in the policy sequence in stochastic problems.
Statistical physics of inference: Thresholds and algorithms
Authors: Lenka Zdeborová, Florent Krzakala
Date: 2015-11-08 00:00:00
Tasks: Community Detection, compressed sensing

Many questions of fundamental interest in todays science can be formulated as
inference problems: Some partial, or noisy, observations are performed over a
set of variables and the goal is to recover, or infer, the values of the
variables based on the indirect information contained in the measurements. For
such problems, the central scientific questions are: Under what conditions is
the information contained in the measurements sufficient for a satisfactory
inference to be possible? What are the most efficient algorithms for this task?
A growing body of work has shown that often we can understand and locate these
fundamental barriers by thinking of them as phase transitions in the sense of
statistical physics. Moreover, it turned out that we can use the gained
physical insight to develop new promising algorithms. Connection between
inference and statistical physics is currently witnessing an impressive
renaissance and we review here the current state-of-the-art, with a pedagogical
focus on the Ising model which formulated as an inference problem we call the
planted spin glass. In terms of applications we review two classes of problems:
(i) inference of clusters on graphs and networks, with community detection as a
special case and (ii) estimating a signal from its noisy linear measurements,
with compressed sensing as a case of sparse estimation. Our goal is to provide
a pedagogical review for researchers in physics and other fields interested in
this fascinating topic.
Automated dataset generation for image recognition using the example of taxonomy
Authors: Jaro Milan Zink
Date: 2018-01-22 00:00:00
Tasks: Dataset Generation, General Classification

This master thesis addresses the subject of automatically generating a
dataset for image recognition, which takes a lot of time when being done
manually. As the thesis was written with motivation from the context of the
biodiversity workgroup at the City University of Applied Sciences Bremen, the
classification of taxonomic entries was chosen as an exemplary use case. In
order to automate the dataset creation, a prototype was conceptualized and
implemented after working out knowledge basics and analyzing requirements for
it. It makes use of an pre-trained abstract artificial intelligence which is
able to sort out images that do not contain the desired content. Subsequent to
the implementation and the automated dataset creation resulting from it, an
evaluation was performed. Other, manually collected datasets were compared to
the one the prototype produced in means of specifications and accuracy. The
results were more than satisfactory and showed that automatically generating a
dataset for image recognition is not only possible, but also might be a decent
alternative to spending time and money in doing this task manually. At the very
end of this work, an idea of how to use the principle of employing abstract
artificial intelligences for step-by-step classification of deeper taxonomic
layers in a productive system is presented and discussed.
Food recognition and recipe analysis: integrating visual content, context and external knowledge
Authors: Luis Herranz, Weiqing Min, Shuqiang Jiang
Date: 2018-01-22 00:00:00
Tasks: Food Recognition, Food recommendation, Retrieval

The central role of food in our individual and social life, combined with
recent technological advances, has motivated a growing interest in applications
that help to better monitor dietary habits as well as the exploration and
retrieval of food-related information. We review how visual content, context
and external knowledge can be integrated effectively into food-oriented
applications, with special focus on recipe analysis and retrieval, food
recommendation, and the restaurant context as emerging directions.
Variational Inference for Gaussian Process Models with Linear Complexity
Authors: Ching-An Cheng, Byron Boots
Date: 2017-11-28 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Variational Inference

Large-scale Gaussian process inference has long faced practical challenges
due to time and space complexity that is superlinear in dataset size. While
sparse variational Gaussian process models are capable of learning from
large-scale data, standard strategies for sparsifying the model can prevent the
approximation of complex functions. In this work, we propose a novel
variational Gaussian process model that decouples the representation of mean
and covariance functions in reproducing kernel Hilbert space. We show that this
new parametrization generalizes previous models. Furthermore, it yields a
variational inference problem that can be solved by stochastic gradient ascent
with time and space complexity that is only linear in the number of mean
function parameters, regardless of the choice of kernels, likelihoods, and
inducing points. This strategy makes the adoption of large-scale expressive
Gaussian process models possible. We run several experiments on regression
tasks and show that this decoupled approach greatly outperforms previous sparse
variational Gaussian process inference procedures.
Improving TSP Solutions Using GA with a New Hybrid Mutation Based on Knowledge and Randomness
Authors: Esra'a Alkafaween, Ahmad B. A. Hassanat
Date: 2018-01-22 00:00:00
Tasks: Traveling Salesman Problem

Genetic algorithm (GA) is an efficient tool for solving optimization problems
by evolving solutions, as it mimics the Darwinian theory of natural evolution.
The mutation operator is one of the key success factors in GA, as it is
considered the exploration operator of GA. Various mutation operators exist to
solve hard combinatorial problems such as the TSP. In this paper, we propose a
hybrid mutation operator called "IRGIBNNM", this mutation is a combination of
two existing mutations, a knowledge-based mutation, and a random-based
mutation. We also improve the existing "select best mutation" strategy using
the proposed mutation. We conducted several experiments on twelve benchmark
Symmetric traveling salesman problem (STSP) instances. The results of our
experiments show the efficiency of the proposed mutation, particularly when we
use it with some other mutations. Keyword: Knowledge-based mutation, Inversion
mutation, Slide mutation, RGIBNNM, SBM.
DiscrimNet: Semi-Supervised Action Recognition from Videos using Generative Adversarial Networks
Authors: Unaiza Ahsan, Chen Sun, Irfan Essa
Date: 2018-01-22 00:00:00
Tasks: Action Recognition, Generative Adversarial Network, Temporal Action Localization, Unsupervised Pre-training

We propose an action recognition framework using Gen- erative Adversarial
Networks. Our model involves train- ing a deep convolutional generative
adversarial network (DCGAN) using a large video activity dataset without la-
bel information. Then we use the trained discriminator from the GAN model as an
unsupervised pre-training step and fine-tune the trained discriminator model on
a labeled dataset to recognize human activities. We determine good network
architectural and hyperparameter settings for us- ing the discriminator from
DCGAN as a trained model to learn useful representations for action
recognition. Our semi-supervised framework using only appearance infor- mation
achieves superior or comparable performance to the current state-of-the-art
semi-supervised action recog- nition methods on two challenging video activity
datasets: UCF101 and HMDB51.
Combinatorial framework for planning in geological exploration
Authors: Mark Sh. Levin
Date: 2018-01-22 00:00:00
Tasks: Attribute, Multiple-choice

The paper describes combinatorial framework for planning of geological
exploration for oil-gas fields. The suggested scheme of the geological
exploration involves the following stages: (1) building of special 4-layer
tree-like model (layer of geological exploration): productive layer, group of
productive layers, oil-gas field, oil-gas region (or group of the fields); (2)
generations of local design (exploration) alternatives for each low-layer
geological objects: conservation, additional search, independent utilization,
joint utilization; (3) multicriteria (i.e., multi-attribute) assessment of the
design (exploration) alternatives and their interrelation (compatibility) and
mapping if the obtained vector estimates into integrated ordinal scale; (4)
hierarchical design ('bottom-up') of composite exploration plans for each
oil-gas field; (5) integration of the plans into region plans and (6)
aggregation of the region plans into a general exploration plan. Stages 2, 3,
4, and 5 are based on hierarchical multicriteria morphological design (HMMD)
method (assessment of ranking of alternatives, selection and composition of
alternatives into composite alternatives). The composition problem is based on
morphological clique model. Aggregation of the obtained modular alternatives
(stage 6) is based on detection of a alternatives 'kernel' and its extension by
addition of elements (multiple choice model). In addition, the usage of
multiset estimates for alternatives is described as well. The alternative
estimates are based on expert judgment. The suggested combinatorial planning
methodology is illustrated by numerical examples for geological exploration of
Yamal peninsula.
Optimal Convergence for Distributed Learning with Stochastic Gradient Methods and Spectral Algorithms
Authors: Junhong Lin, Volkan Cevher
Date: 2018-01-22 00:00:00
Tasks: regression

We study generalization properties of distributed algorithms in the setting
of nonparametric regression over a reproducing kernel Hilbert space (RKHS). We
first investigate distributed stochastic gradient methods (SGM), with
mini-batches and multi-passes over the data. We show that optimal
generalization error bounds can be retained for distributed SGM provided that
the partition level is not too large. We then extend our results to
spectral-regularization algorithms (SRA), including kernel ridge regression
(KRR), kernel principal component analysis, and gradient methods. Our results
are superior to the state-of-the-art theory. Particularly, our results show
that distributed SGM has a smaller theoretical computational complexity,
compared with distributed KRR and classic SGM. Moreover, even for
non-distributed SRA, they provide the first optimal, capacity-dependent
convergence rates, considering the case that the regression function may not be
in the RKHS.
Get Your Workload in Order: Game Theoretic Prioritization of Database Auditing
Authors: Chao Yan, Bo Li, Yevgeniy Vorobeychik, Aron Laszka, Daniel Fabbri, Bradley Malin
Date: 2018-01-22 00:00:00
Tasks: Heuristic Search

For enhancing the privacy protections of databases, where the increasing
amount of detailed personal data is stored and processed, multiple mechanisms
have been developed, such as audit logging and alert triggers, which notify
administrators about suspicious activities; however, the two main limitations
in common are: 1) the volume of such alerts is often substantially greater than
the capabilities of resource-constrained organizations, and 2) strategic
attackers may disguise their actions or carefully choosing which records they
touch, making incompetent the statistical detection models. For solving them,
we introduce a novel approach to database auditing that explicitly accounts for
adversarial behavior by 1) prioritizing the order in which types of alerts are
investigated and 2) providing an upper bound on how much resource to allocate
for each type. We model the interaction between a database auditor and
potential attackers as a Stackelberg game in which the auditor chooses an
auditing policy and attackers choose which records to target. A corresponding
approach combining linear programming, column generation, and heuristic search
is proposed to derive an auditing policy. For testing the policy-searching
performance, a publicly available credit card application dataset are adopted,
on which it shows that our methods produce high-quality mixed strategies as
database audit policies, and our general approach significantly outperforms
non-game-theoretic baselines.
Optimizing Prediction Intervals by Tuning Random Forest via Meta-Validation
Authors: Sean Bayley, Davide Falessi
Date: 2018-01-22 00:00:00
Tasks: Prediction, Prediction Intervals

Recent studies have shown that tuning prediction models increases prediction
accuracy and that Random Forest can be used to construct prediction intervals.
However, to our best knowledge, no study has investigated the need to, and the
manner in which one can, tune Random Forest for optimizing prediction intervals
{ this paper aims to fill this gap. We explore a tuning approach that combines
an effectively exhaustive search with a validation technique on a single Random
Forest parameter. This paper investigates which, out of eight validation
techniques, are beneficial for tuning, i.e., which automatically choose a
Random Forest configuration constructing prediction intervals that are reliable
and with a smaller width than the default configuration. Additionally, we
present and validate three meta-validation techniques to determine which are
beneficial, i.e., those which automatically chose a beneficial validation
technique. This study uses data from our industrial partner (Keymind Inc.) and
the Tukutuku Research Project, related to post-release defect prediction and
Web application effort estimation, respectively. Results from our study
indicate that: i) the default configuration is frequently unreliable, ii) most
of the validation techniques, including previously successfully adopted ones
such as 50/50 holdout and bootstrap, are counterproductive in most of the
cases, and iii) the 75/25 holdout meta-validation technique is always
beneficial; i.e., it avoids the likely counterproductive effects of validation
techniques.
SHADHO: Massively Scalable Hardware-Aware Distributed Hyperparameter Optimization
Authors: Jeff Kinnison, Nathaniel Kremer-Herman, Douglas Thain, Walter Scheirer
Date: 2017-07-05 00:00:00
Tasks: Cell Segmentation, Hyperparameter Optimization

Computer vision is experiencing an AI renaissance, in which machine learning
models are expediting important breakthroughs in academic research and
commercial applications. Effectively training these models, however, is not
trivial due in part to hyperparameters: user-configured values that control a
model's ability to learn from data. Existing hyperparameter optimization
methods are highly parallel but make no effort to balance the search across
heterogeneous hardware or to prioritize searching high-impact spaces. In this
paper, we introduce a framework for massively Scalable Hardware-Aware
Distributed Hyperparameter Optimization (SHADHO). Our framework calculates the
relative complexity of each search space and monitors performance on the
learning task over all trials. These metrics are then used as heuristics to
assign hyperparameters to distributed workers based on their hardware. We first
demonstrate that our framework achieves double the throughput of a standard
distributed hyperparameter optimization framework by optimizing SVM for MNIST
using 150 distributed workers. We then conduct model search with SHADHO over
the course of one week using 74 GPUs across two compute clusters to optimize
U-Net for a cell segmentation task, discovering 515 models that achieve a lower
validation loss than standard U-Net.
Unsupervised Open Relation Extraction
Authors: Hady Elsahar, Elena Demidova, Simon Gottschalk, Christophe Gravier, Frederique Laforest
Date: 2018-01-22 00:00:00
Tasks: Clustering, Relation, Relation Extraction, Word Embeddings

We explore methods to extract relations between named entities from free text
in an unsupervised setting. In addition to standard feature extraction, we
develop a novel method to re-weight word embeddings. We alleviate the problem
of features sparsity using an individual feature reduction. Our approach
exhibits a significant improvement by 5.8% over the state-of-the-art relation
clustering scoring a F1-score of 0.416 on the NYT-FB dataset.
Scale-invariant Feature Extraction of Neural Network and Renormalization Group Flow
Authors: Satoshi Iso, Shotaro Shiba, Sumito Yokoo
Date: 2018-01-22 00:00:00

Theoretical understanding of how deep neural network (DNN) extracts features
from input images is still unclear, but it is widely believed that the
extraction is performed hierarchically through a process of coarse-graining. It
reminds us of the basic concept of renormalization group (RG) in statistical
physics. In order to explore possible relations between DNN and RG, we use the
Restricted Boltzmann machine (RBM) applied to Ising model and construct a flow
of model parameters (in particular, temperature) generated by the RBM. We show
that the unsupervised RBM trained by spin configurations at various
temperatures from $T=0$ to $T=6$ generates a flow along which the temperature
approaches the critical value $T_c=2.27$. This behavior is opposite to the
typical RG flow of the Ising model. By analyzing various properties of the
weight matrices of the trained RBM, we discuss why it flows towards $T_c$ and
how the RBM learns to extract features of spin configurations.
A Novel Weighted Distance Measure for Multi-Attributed Graph
Authors: Muhammad Abulaish, Jahiruddin
Date: 2018-01-22 00:00:00
Tasks: Clustering, General Classification, Graph Generation

Due to exponential growth of complex data, graph structure has become
increasingly important to model various entities and their interactions, with
many interesting applications including, bioinformatics, social network
analysis, etc. Depending on the complexity of the data, the underlying graph
model can be a simple directed/undirected and/or weighted/un-weighted graph to
a complex graph (aka multi-attributed graph) where vertices and edges are
labelled with multi-dimensional vectors. In this paper, we present a novel
weighted distance measure based on weighted Euclidean norm which is defined as
a function of both vertex and edge attributes, and it can be used for various
graph analysis tasks including classification and cluster analysis. The
proposed distance measure has flexibility to increase/decrease the weightage of
edge labels while calculating the distance between vertex-pairs. We have also
proposed a MAGDist algorithm, which reads multi-attributed graph stored in CSV
files containing the list of vertex vectors and edge vectors, and calculates
the distance between each vertex-pair using the proposed weighted distance
measure. Finally, we have proposed a multi-attributed similarity graph
generation algorithm, MAGSim, which reads the output of MAGDist algorithm and
generates a similarity graph that can be analysed using classification and
clustering algorithms. The significance and accuracy of the proposed distance
measure and algorithms is evaluated on Iris and Twitter data sets, and it is
found that the similarity graph generated by our proposed method yields better
clustering results than the existing similarity graph generation methods.
E-swish: Adjusting Activations to Different Network Depths
Authors: Eric Alcaide
Date: 2018-01-22 00:00:00

Activation functions have a notorious impact on neural networks on both
training and testing the models against the desired problem. Currently, the
most used activation function is the Rectified Linear Unit (ReLU). This paper
introduces a new and novel activation function, closely related with the new
activation $Swish = x * sigmoid(x)$ (Ramachandran et al., 2017) which
generalizes it. We call the new activation $E-swish = \beta x * sigmoid(x)$. We
show that E-swish outperforms many other well-known activations including both
ReLU and Swish. For example, using E-swish provided 1.5% and 4.6% accuracy
improvements on Cifar10 and Cifar100 respectively for the WRN 10-2 when
compared to ReLU and 0.35% and 0.6% respectively when compared to Swish. The
code to reproduce all our experiments can be found at
https://github.com/EricAlcaide/E-swish
Ultimate SLAM? Combining Events, Images, and IMU for Robust Visual SLAM in HDR and High Speed Scenarios
Authors: Antoni Rosinol Vidal, Henri Rebecq, Timo Horstschaefer, Davide Scaramuzza
Date: 2017-09-19 00:00:00
Tasks: State Estimation

Event cameras are bio-inspired vision sensors that output pixel-level
brightness changes instead of standard intensity frames. These cameras do not
suffer from motion blur and have a very high dynamic range, which enables them
to provide reliable visual information during high speed motions or in scenes
characterized by high dynamic range. However, event cameras output only little
information when the amount of motion is limited, such as in the case of almost
still motion. Conversely, standard cameras provide instant and rich information
about the environment most of the time (in low-speed and good lighting
scenarios), but they fail severely in case of fast motions, or difficult
lighting such as high dynamic range or low light scenes. In this paper, we
present the first state estimation pipeline that leverages the complementary
advantages of these two sensors by fusing in a tightly-coupled manner events,
standard frames, and inertial measurements. We show on the publicly available
Event Camera Dataset that our hybrid pipeline leads to an accuracy improvement
of 130% over event-only pipelines, and 85% over standard-frames-only
visual-inertial systems, while still being computationally tractable.
Furthermore, we use our pipeline to demonstrate - to the best of our knowledge
- the first autonomous quadrotor flight using an event camera for state
estimation, unlocking flight scenarios that were not reachable with traditional
visual-inertial odometry, such as low-light environments and high-dynamic range
scenes.
Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning
Authors: Qimai Li, Zhichao Han, Xiao-Ming Wu
Date: 2018-01-22 00:00:00
Tasks: Model Selection, Node Classification

Many interesting problems in machine learning are being revisited with new
deep learning tools. For graph-based semisupervised learning, a recent
important development is graph convolutional networks (GCNs), which nicely
integrate local vertex features and graph topology in the convolutional layers.
Although the GCN model compares favorably with other state-of-the-art methods,
its mechanisms are not clear and it still requires a considerable amount of
labeled data for validation and model selection. In this paper, we develop
deeper insights into the GCN model and address its fundamental limits. First,
we show that the graph convolution of the GCN model is actually a special form
of Laplacian smoothing, which is the key reason why GCNs work, but it also
brings potential concerns of over-smoothing with many convolutional layers.
Second, to overcome the limits of the GCN model with shallow architectures, we
propose both co-training and self-training approaches to train GCNs. Our
approaches significantly improve GCNs in learning with very few labels, and
exempt them from requiring additional labels for validation. Extensive
experiments on benchmarks have verified our theory and proposals.