pwc-archive/papers-with-abstracts huggingface dataset:
Metric Learning on Manifolds
Authors: Max Aalto, Nakul Verma
Date: 2019-02-05 00:00:00
Tasks: Clustering, Metric Learning

Recent literature has shown that symbolic data, such as text and graphs, is
often better represented by points on a curved manifold, rather than in
Euclidean space. However, geometrical operations on manifolds are generally
more complicated than in Euclidean space, and thus many techniques for
processing and analysis taken for granted in Euclidean space are difficult on
manifolds. A priori, it is not obvious how we may generalize such methods to
manifolds. We consider specifically the problem of distance metric learning,
and present a framework that solves it on a large class of manifolds, such that
similar data are located in closer proximity with respect to the manifold
distance function. In particular, we extend the existing metric learning
algorithms, and derive the corresponding sample complexity rates for the case
of manifolds. Additionally, we demonstrate an improvement of performance in
$k$-means clustering and $k$-nearest neighbor classification on real-world
complex networks using our methods.
Deep Tree Transductions - A Short Survey
Authors: Davide Bacciu, Antonio Bruno
Date: 2019-02-05 00:00:00
Tasks: Survey

The paper surveys recent extensions of the Long-Short Term Memory networks to
handle tree structures from the perspective of learning non-trivial forms of
isomorph structured transductions. It provides a discussion of modern TreeLSTM
models, showing the effect of the bias induced by the direction of tree
processing. An empirical analysis is performed on real-world benchmarks,
highlighting how there is no single model adequate to effectively approach all
transduction problems.
Robust Regression via Online Feature Selection under Adversarial Data Corruption
Authors: Xuchao Zhang, Shuo Lei, Liang Zhao, Arnold P. Boedihardjo, Chang-Tien Lu
Date: 2019-02-05 00:00:00
Tasks: feature selection, regression

The presence of data corruption in user-generated streaming data, such as
social media, motivates a new fundamental problem that learns reliable
regression coefficient when features are not accessible entirely at one time.
Until now, several important challenges still cannot be handled concurrently:
1) corrupted data estimation when only partial features are accessible; 2)
online feature selection when data contains adversarial corruption; and 3)
scaling to a massive dataset. This paper proposes a novel RObust regression
algorithm via Online Feature Selection (\textit{RoOFS}) that concurrently
addresses all the above challenges. Specifically, the algorithm iteratively
updates the regression coefficients and the uncorrupted set via a robust online
feature substitution method. We also prove that our algorithm has a restricted
error bound compared to the optimal solution. Extensive empirical experiments
in both synthetic and real-world datasets demonstrated that the effectiveness
of our new method is superior to that of existing methods in the recovery of
both feature selection and regression coefficients, with very competitive
efficiency.
6D Object Pose Estimation without PnP
Authors: Jin Liu, Sheng He
Date: 2019-02-05 00:00:00
Tasks: 6D Pose Estimation using RGB, GPU, Object, Pose Estimation, Translation

In this paper, we propose an efficient end-to-end algorithm to tackle the
problem of estimating the 6D pose of objects from a single RGB image. Our
system trains a fully convolutional network to regress the 3D rotation and the
3D translation in region layer. On this basis, a special layer, Collinear
Equation Layer, is added next to region layer to output the 2D projections of
the 3D bounding boxs corners. In the back propagation stage, the 6D pose
network are adjusted according to the error of the 2D projections. In the
detection phase, we directly output the position and pose through the region
layer. Besides, we introduce a novel and concise representation of 3D rotation
to make the regression more precise and easier. Experiments show that our
method outperforms base-line and state of the art methods both at accuracy and
efficiency. In the LineMod dataset, our algorithm achieves less than 18
ms/object on a GeForce GTX 1080Ti GPU, while the translational error and
rotational error are less than 1.67 cm and 2.5 degree.
AlphaStar: An Evolutionary Computation Perspective
Authors: Kai Arulkumaran, Antoine Cully, Julian Togelius
Date: 2019-02-05 00:00:00
Tasks: Diversity, Reinforcement Learning, Reinforcement Learning (RL), Starcraft II

In January 2019, DeepMind revealed AlphaStar to the world-the first artificial intelligence (AI) system to beat a professional player at the game of StarCraft II-representing a milestone in the progress of AI. AlphaStar draws on many areas of AI research, including deep learning, reinforcement learning, game theory, and evolutionary computation (EC). In this paper we analyze AlphaStar primarily through the lens of EC, presenting a new look at the system and relating it to many concepts in the field. We highlight some of its most interesting aspects-the use of Lamarckian evolution, competitive co-evolution, and quality diversity. In doing so, we hope to provide a bridge between the wider EC community and one of the most significant AI systems developed in recent times.
Total stochastic gradient algorithms and applications in reinforcement learning
Authors: Paavo Parmas
Date: 2019-02-05 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Density Estimation, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Backpropagation and the chain rule of derivatives have been prominent;
however, the total derivative rule has not enjoyed the same amount of
attention. In this work we show how the total derivative rule leads to an
intuitive visual framework for creating gradient estimators on graphical
models. In particular, previous "policy gradient theorems" are easily derived.
We derive new gradient estimators based on density estimation, as well as a
likelihood ratio gradient, which "jumps" to an intermediate node, not directly
to the objective function. We evaluate our methods on model-based policy
gradient algorithms, achieve good performance, and present evidence towards
demystifying the success of the popular PILCO algorithm.
End-to-End Open-Domain Question Answering with BERTserini
Authors: Wei Yang, Yuqing Xie, Aileen Lin, Xingyu Li, Luchen Tan, Kun Xiong, Ming Li, Jimmy Lin
Date: 2019-02-05 00:00:00
Proceeding: NAACL 2019 6
Tasks: Articles, Information Retrieval, Open-Domain Question Answering, Question Answering, Reading Comprehension, Retrieval

We demonstrate an end-to-end question answering system that integrates BERT with the open-source Anserini information retrieval toolkit. In contrast to most question answering and reading comprehension models today, which operate over small amounts of input text, our system integrates best practices from IR with a BERT-based reader to identify answers from a large corpus of Wikipedia articles in an end-to-end fashion. We report large improvements over previous results on a standard benchmark test collection, showing that fine-tuning pretrained BERT with SQuAD is sufficient to achieve high accuracy in identifying answer spans.
Optimal Nonparametric Inference via Deep Neural Network
Authors: Ruiqi Liu, Ben Boukai, Zuofeng Shang
Date: 2019-02-05 00:00:00
Tasks: Two-sample testing

Deep neural network is a state-of-art method in modern science and technology. Much statistical literature have been devoted to understanding its performance in nonparametric estimation, whereas the results are suboptimal due to a redundant logarithmic sacrifice. In this paper, we show that such log-factors are not necessary. We derive upper bounds for the $L^2$ minimax risk in nonparametric estimation. Sufficient conditions on network architectures are provided such that the upper bounds become optimal (without log-sacrifice). Our proof relies on an explicitly constructed network estimator based on tensor product B-splines. We also derive asymptotic distributions for the constructed network and a relating hypothesis testing procedure. The testing procedure is further proven as minimax optimal under suitable network architectures.
The Probabilistic Fault Tolerance of Neural Networks in the Continuous Limit
Authors: El-Mahdi El-Mhamdi, Rachid Guerraoui, Andrei Kucharavy, Sergei Volodin
Date: 2019-02-05 00:00:00
Proceeding: ICLR 2020 1

The loss of a few neurons in a brain rarely results in any visible loss of function. However, the insight into what "few" means in this context is unclear. How many random neuron failures will it take to lead to a visible loss of function? In this paper, we address the fundamental question of the impact of the crash of a random subset of neurons on the overall computation of a neural network and the error in the output it produces. We study fault tolerance of neural networks subject to small random neuron/weight crash failures in a probabilistic setting. We give provable guarantees on the robustness of the network to these crashes. Our main contribution is a bound on the error in the output of a network under small random Bernoulli crashes proved by using a Taylor expansion in the continuous limit, where close-by neurons at a layer are similar. The failure mode we adopt in our model is characteristic of neuromorphic hardware, a promising technology to speed up artificial neural networks, as well as of biological networks. We show that our theoretical bounds can be used to compare the fault tolerance of different architectures and to design a regularizer improving the fault tolerance of a given architecture. We design an algorithm achieving fault tolerance using a reasonable number of neurons. In addition to the theoretical proof, we also provide experimental validation of our results and suggest a connection to the generalization capacity problem.
Interactively shaping robot behaviour with unlabeled human instructions
Authors: Anis Najar, Olivier Sigaud, Mohamed Chetouani
Date: 2019-02-05 00:00:00
Tasks: Reinforcement Learning, Reinforcement Learning (RL)

In this paper, we propose a framework that enables a human teacher to shape a robot behaviour by interactively providing it with unlabeled instructions. We ground the meaning of instruction signals in the task-learning process, and use them simultaneously for guiding the latter. We implement our framework as a modular architecture, named TICS (Task-Instruction-Contingency-Shaping) that combines different information sources: a predefined reward function, human evaluative feedback and unlabeled instructions. This approach provides a novel perspective for robotic task learning that lies between Reinforcement Learning and Supervised Learning paradigms. We evaluate our framework both in simulation and with a real robot. The experimental results demonstrate the effectiveness of our framework in accelerating the task-learning process and in reducing the number of required teaching signals.
Early Recognition of Sepsis with Gaussian Process Temporal Convolutional Networks and Dynamic Time Warping
Authors: Michael Moor, Max Horn, Bastian Rieck, Damian Roqueiro, Karsten Borgwardt
Date: 2019-02-05 00:00:00
Tasks: Deep Learning, Dynamic Time Warping, Management, Time Series, Time Series Analysis, Time Series Classification

Sepsis is a life-threatening host response to infection associated with high mortality, morbidity, and health costs. Its management is highly time-sensitive since each hour of delayed treatment increases mortality due to irreversible organ damage. Meanwhile, despite decades of clinical research, robust biomarkers for sepsis are missing. Therefore, detecting sepsis early by utilizing the affluence of high-resolution intensive care records has become a challenging machine learning problem. Recent advances in deep learning and data mining promise to deliver a powerful set of tools to efficiently address this task. This empirical study proposes two novel approaches for the early detection of sepsis: a deep learning model and a lazy learner based on time series distances. Our deep learning model employs a temporal convolutional network that is embedded in a Multi-task Gaussian Process Adapter framework, making it directly applicable to irregularly-spaced time series data. Our lazy learner, by contrast, is an ensemble approach that employs dynamic time warping. We frame the timely detection of sepsis as a supervised time series classification task. For this, we derive the most recent sepsis definition in an hourly resolution to provide the first fully accessible early sepsis detection environment. Seven hours before sepsis onset, our methods improve area under the precision--recall curve from 0.25 to 0.35/0.40 over the state of the art. This demonstrates that they are well-suited for detecting sepsis in the crucial earlier stages when management is most effective.
DVOLVER: Efficient Pareto-Optimal Neural Network Architecture Search
Authors: Guillaume Michel, Mohammed Amine Alaoui, Alice Lebois, Amal Feriani, Mehdi Felhi
Date: 2019-02-05 00:00:00
Proceeding: ICLR 2019 5
Tasks: Neural Architecture Search

Automatic search of neural network architectures is a standing research
topic. In addition to the fact that it presents a faster alternative to
hand-designed architectures, it can improve their efficiency and for instance
generate Convolutional Neural Networks (CNN) adapted for mobile devices. In
this paper, we present a multi-objective neural architecture search method to
find a family of CNN models with the best accuracy and computational resources
tradeoffs, in a search space inspired by the state-of-the-art findings in
neural search. Our work, called Dvolver, evolves a population of architectures
and iteratively improves an approximation of the optimal Pareto front. Applying
Dvolver on the model accuracy and on the number of floating points operations
as objective functions, we are able to find, in only 2.5 days, a set of
competitive mobile models on ImageNet. Amongst these models one architecture
has the same Top-1 accuracy on ImageNet as NASNet-A mobile with 8% less
floating point operations and another one has a Top-1 accuracy of 75.28% on
ImageNet exceeding by 0.28% the best MobileNetV2 model for the same
computational resources.
On the Convergence of Projected-Gradient Methods with Low-Rank Projections for Smooth Convex Minimization over Trace-Norm Balls and Related Problems
Authors: Dan Garber
Date: 2019-02-05 00:00:00

Smooth convex minimization over the unit trace-norm ball is an important optimization problem in machine learning, signal processing, statistics and other fields, that underlies many tasks in which one wishes to recover a low-rank matrix given certain measurements. While first-order methods for convex optimization enjoy optimal convergence rates, they require in worst-case to compute a full-rank SVD on each iteration, in order to compute the projection onto the trace-norm ball. These full-rank SVD computations however prohibit the application of such methods to large problems. A simple and natural heuristic to reduce the computational cost is to approximate the projection using only a low-rank SVD. This raises the question if, and under what conditions, this simple heuristic can indeed result in provable convergence to the optimal solution. In this paper we show that any optimal solution is a center of a Euclid. ball inside-which the projected-gradient mapping admits rank that is at most the multiplicity of the largest singular value of the gradient vector. Moreover, the radius of the ball scales with the spectral gap of this gradient vector. We show how this readily implies the local convergence (i.e., from a "warm-start" initialization) of standard first-order methods, using only low-rank SVD computations. We also quantify the effect of "over-parameterization", i.e., using SVD computations with higher rank, on the radius of this ball, showing it can increase dramatically with moderately larger rank. We extend our results also to the setting of optimization with trace-norm regularization and optimization over bounded-trace positive semidefinite matrices. Our theoretical investigation is supported by concrete empirical evidence that demonstrates the \textit{correct} convergence of first-order methods with low-rank projections on real-world datasets.
Exploiting locality in high-dimensional factorial hidden Markov models
Authors: Lorenzo Rimella, Nick Whiteley
Date: 2019-02-05 00:00:00
Tasks: Vocal Bursts Intensity Prediction

We propose algorithms for approximate filtering and smoothing in high-dimensional Factorial hidden Markov models. The approximation involves discarding, in a principled way, likelihood factors according to a notion of locality in a factor graph associated with the emission distribution. This allows the exponential-in-dimension cost of exact filtering and smoothing to be avoided. We prove that the approximation accuracy, measured in a local total variation norm, is "dimension-free" in the sense that as the overall dimension of the model increases the error bounds we derive do not necessarily degrade. A key step in the analysis is to quantify the error introduced by localizing the likelihood function in a Bayes' rule update. The factorial structure of the likelihood function which we exploit arises naturally when data have known spatial or network structure. We demonstrate the new algorithms on synthetic examples and a London Underground passenger flow problem, where the factor graph is effectively given by the train network.
A Universal Algorithm for Variational Inequalities Adaptive to Smoothness and Noise
Authors: Francis Bach, Kfir. Y. Levy
Date: 2019-02-05 00:00:00

We consider variational inequalities coming from monotone operators, a
setting that includes convex minimization and convex-concave saddle-point
problems. We assume an access to potentially noisy unbiased values of the
monotone operators and assess convergence through a compatible gap function
which corresponds to the standard optimality criteria in the aforementioned
subcases. We present a universal algorithm for these inequalities based on the
Mirror-Prox algorithm. Concretely, our algorithm simultaneously achieves the
optimal rates for the smooth/non-smooth, and noisy/noiseless settings. This is
done without any prior knowledge of these properties, and in the general set-up
of arbitrary norms and compatible Bregman divergences. For convex minimization
and convex-concave saddle-point problems, this leads to new adaptive
algorithms. Our method relies on a novel yet simple adaptive choice of the
step-size, which can be seen as the appropriate extension of AdaGrad to handle
constrained problems.
Riemannian optimization with a preconditioning scheme on the generalized Stiefel manifold
Authors: Boris Shustin, Haim Avron
Date: 2019-02-05 00:00:00
Tasks: Dimensionality Reduction, Riemannian optimization

Optimization problems on the generalized Stiefel manifold (and products of it) are prevalent across science and engineering. For example, in computational science they arise in symmetric (generalized) eigenvalue problems, in nonlinear eigenvalue problems, and in electronic structures computations, to name a few problems. In statistics and machine learning, they arise, for example, in various dimensionality reduction techniques such as canonical correlation analysis. In deep learning, regularization and improved stability can be obtained by constraining some layers to have parameter matrices that belong to the Stiefel manifold. Solving problems on the generalized Stiefel manifold can be approached via the tools of Riemannian optimization. However, using the standard geometric components for the generalized Stiefel manifold has two possible shortcomings: computing some of the geometric components can be too expensive and convergence can be rather slow in certain cases. Both shortcomings can be addressed using a technique called Riemannian preconditioning, which amounts to using geometric components derived by a precoditioner that defines a Riemannian metric on the constraint manifold. In this paper we develop the geometric components required to perform Riemannian optimization on the generalized Stiefel manifold equipped with a non-standard metric, and illustrate theoretically and numerically the use of those components and the effect of Riemannian preconditioning for solving optimization problems on the generalized Stiefel manifold.
Minimum description length as an objective function for non-negative matrix factorization
Authors: Steven Squires, Adam Prugel Bennett, Mahesan Niranjan
Date: 2019-02-05 00:00:00
Tasks: Dimensionality Reduction

Non-negative matrix factorization (NMF) is a dimensionality reduction
technique which tends to produce a sparse representation of data. Commonly, the
error between the actual and recreated matrices is used as an objective
function, but this method may not produce the type of representation we desire
as it allows for the complexity of the model to grow, constrained only by the
size of the subspace and the non-negativity requirement. If additional
constraints, such as sparsity, are imposed the question of parameter selection
becomes critical. Instead of adding sparsity constraints in an ad-hoc manner we
propose a novel objective function created by using the principle of minimum
description length (MDL). Our formulation, MDL-NMF, automatically trades off
between the complexity and accuracy of the model using a principled approach
with little parameter selection or the need for domain expertise. We
demonstrate our model works effectively on three heterogeneous data-sets and on
a range of semi-synthetic data showing the broad applicability of our method.
EasyLabel: A Semi-Automatic Pixel-wise Object Annotation Tool for Creating Robotic RGB-D Datasets
Authors: Markus Suchi, Timothy Patten, David Fischinger, Markus Vincze
Date: 2019-02-05 00:00:00
Tasks: Object, Semantic Segmentation

Developing robot perception systems for recognizing objects in the real-world
requires computer vision algorithms to be carefully scrutinized with respect to
the expected operating domain. This demands large quantities of ground truth
data to rigorously evaluate the performance of algorithms. This paper presents
the EasyLabel tool for easily acquiring high quality ground truth annotation of
objects at the pixel-level in densely cluttered scenes. In a semi-automatic
process, complex scenes are incrementally built and EasyLabel exploits depth
change to extract precise object masks at each step. We use this tool to
generate the Object Cluttered Indoor Dataset (OCID) that captures diverse
settings of objects, background, context, sensor to scene distance, viewpoint
angle and lighting conditions. OCID is used to perform a systematic comparison
of existing object segmentation methods. The baseline comparison supports the
need for pixel- and object-wise annotation to progress robot vision towards
realistic applications. This insight reveals the usefulness of EasyLabel and
OCID to better understand the challenges that robots face in the real-world.
  Copyright 20XX IEEE. Personal use of this material is permitted. Permission
from IEEE must be obtained for all other uses, in any current or future media,
including reprinting/republishing this material for advertising or promotional
purposes, creating new collective works, for resale or redistribution to
servers or lists, or reuse of any copyrighted component of this work in other
works.