pwc-archive/papers-with-abstracts huggingface dataset:
QuaSI: Quantile Sparse Image Prior for Spatio-Temporal Denoising of Retinal OCT Data
Authors: Franziska Schirrmacher, Thomas Köhler, Lennart Husvogt, James G. Fujimoto, Joachim Hornegger, Andreas K. Maier
Date: 2017-03-08 00:00:00
Tasks: Denoising, Diagnostic

Optical coherence tomography (OCT) enables high-resolution and non-invasive
3D imaging of the human retina but is inherently impaired by speckle noise.
This paper introduces a spatio-temporal denoising algorithm for OCT data on a
B-scan level using a novel quantile sparse image (QuaSI) prior. To remove
speckle noise while preserving image structures of diagnostic relevance, we
implement our QuaSI prior via median filter regularization coupled with a Huber
data fidelity model in a variational approach. For efficient energy
minimization, we develop an alternating direction method of multipliers (ADMM)
scheme using a linearization of median filtering. Our spatio-temporal method
can handle both, denoising of single B-scans and temporally consecutive
B-scans, to gain volumetric OCT data with enhanced signal-to-noise ratio. Our
algorithm based on 4 B-scans only achieved comparable performance to averaging
13 B-scans and outperformed other current denoising methods.
Fast Gesture Recognition with Multiple Stream Discrete HMMs on 3D Skeletons
Authors: Guido Borghi, Roberto Vezzani, Rita Cucchiara
Date: 2017-03-08 00:00:00
Tasks: All, Classification, General Classification, Gesture Recognition

HMMs are widely used in action and gesture recognition due to their
implementation simplicity, low computational requirement, scalability and high
parallelism. They have worth performance even with a limited training set. All
these characteristics are hard to find together in other even more accurate
methods. In this paper, we propose a novel double-stage classification
approach, based on Multiple Stream Discrete Hidden Markov Models (MSD-HMM) and
3D skeleton joint data, able to reach high performances maintaining all
advantages listed above. The approach allows both to quickly classify
pre-segmented gestures (offline classification), and to perform temporal
segmentation on streams of gestures (online classification) faster than real
time. We test our system on three public datasets, MSRAction3D, UTKinect-Action
and MSRDailyAction, and on a new dataset, Kinteract Dataset, explicitly created
for Human Computer Interaction (HCI). We obtain state of the art performances
on all of them.
Transformation-Grounded Image Generation Network for Novel 3D View Synthesis
Authors: Eunbyung Park, Jimei Yang, Ersin Yumer, Duygu Ceylan, Alexander C. Berg
Date: 2017-03-08 00:00:00
Proceeding: CVPR 2017 7
Tasks: Image Generation, Novel View Synthesis

We present a transformation-grounded image generation network for novel 3D
view synthesis from a single image. Instead of taking a 'blank slate' approach,
we first explicitly infer the parts of the geometry visible both in the input
and novel views and then re-cast the remaining synthesis problem as image
completion. Specifically, we both predict a flow to move the pixels from the
input to the novel view along with a novel visibility map that helps deal with
occulsion/disocculsion. Next, conditioned on those intermediate results, we
hallucinate (infer) parts of the object invisible in the input image. In
addition to the new network structure, training with a combination of
adversarial and perceptual loss results in a reduction in common artifacts of
novel view synthesis such as distortions and holes, while successfully
generating high frequency details and preserving visual aspects of the input
image. We evaluate our approach on a wide range of synthetic and real examples.
Both qualitative and quantitative results show our method achieves
significantly better results compared to existing methods.
Dropout Inference in Bayesian Neural Networks with Alpha-divergences
Authors: Yingzhen Li, Yarin Gal
Date: 2017-03-08 00:00:00
Proceeding: ICML 2017 8
Tasks: Variational Inference

To obtain uncertainty estimates with real-world Bayesian deep learning
models, practical inference approximations are needed. Dropout variational
inference (VI) for example has been used for machine vision and medical
applications, but VI can severely underestimates model uncertainty.
Alpha-divergences are alternative divergences to VI's KL objective, which are
able to avoid VI's uncertainty underestimation. But these are hard to use in
practice: existing techniques can only use Gaussian approximating
distributions, and require existing models to be changed radically, thus are of
limited use for practitioners. We propose a re-parametrisation of the
alpha-divergence objectives, deriving a simple inference technique which,
together with dropout, can be easily implemented with existing models by simply
changing the loss of the model. We demonstrate improved uncertainty estimates
and accuracy compared to VI in dropout networks. We study our model's epistemic
uncertainty far away from the data using adversarial images, showing that these
can be distinguished from non-adversarial images by examining our model's
uncertainty.
STransE: a novel embedding model of entities and relationships in knowledge bases
Authors: Dat Quoc Nguyen, Kairit Sirts, Lizhen Qu, Mark Johnson
Date: 2016-06-27 00:00:00
Proceeding: NAACL 2016 6
Tasks: Knowledge Base Completion, Link Prediction, Prediction, Translation

Knowledge bases of real-world facts about entities and their relationships
are useful resources for a variety of natural language processing tasks.
However, because knowledge bases are typically incomplete, it is useful to be
able to perform link prediction or knowledge base completion, i.e., predict
whether a relationship not in the knowledge base is likely to be true. This
paper combines insights from several previous link prediction models into a new
embedding model STransE that represents each entity as a low-dimensional
vector, and each relation by two matrices and a translation vector. STransE is
a simple combination of the SE and TransE models, but it obtains better link
prediction performance on two benchmark datasets than previous embedding
models. Thus, STransE can serve as a new baseline for the more complex models
in the link prediction task.
Deep Bayesian Active Learning with Image Data
Authors: Yarin Gal, Riashat Islam, Zoubin Ghahramani
Date: 2017-03-08 00:00:00
Proceeding: ICML 2017 8
Tasks: Active Learning, Deep Learning

Even though active learning forms an important pillar of machine learning,
deep learning tools are not prevalent within it. Deep learning poses several
difficulties when used in an active learning setting. First, active learning
(AL) methods generally rely on being able to learn and update models from small
amounts of data. Recent advances in deep learning, on the other hand, are
notorious for their dependence on large amounts of data. Second, many AL
acquisition functions rely on model uncertainty, yet deep learning methods
rarely represent such model uncertainty. In this paper we combine recent
advances in Bayesian deep learning into the active learning framework in a
practical way. We develop an active learning framework for high dimensional
data, a task which has been extremely challenging so far, with very sparse
existing literature. Taking advantage of specialised models such as Bayesian
convolutional neural networks, we demonstrate our active learning techniques
with image data, obtaining a significant improvement on existing active
learning approaches. We demonstrate this on both the MNIST dataset, as well as
for skin cancer diagnosis from lesion images (ISIC2016 task).
Model-Based Policy Search for Automatic Tuning of Multivariate PID Controllers
Authors: Andreas Doerr, Duy Nguyen-Tuong, Alonso Marco, Stefan Schaal, Sebastian Trimpe
Date: 2017-03-08 00:00:00

PID control architectures are widely used in industrial applications. Despite
their low number of open parameters, tuning multiple, coupled PID controllers
can become tedious in practice. In this paper, we extend PILCO, a model-based
policy search framework, to automatically tune multivariate PID controllers
purely based on data observed on an otherwise unknown system. The system's
state is extended appropriately to frame the PID policy as a static state
feedback policy. This renders PID tuning possible as the solution of a finite
horizon optimal control problem without further a priori knowledge. The
framework is applied to the task of balancing an inverted pendulum on a seven
degree-of-freedom robotic arm, thereby demonstrating its capabilities of fast
and data-efficient policy learning, even on complex real world problems.
Memory Enriched Big Bang Big Crunch Optimization Algorithm for Data Clustering
Authors: Kayvan Bijari, Hadi Zare, Hadi Veisi, Hossein Bobarshad
Date: 2017-03-08 00:00:00
Tasks: Clustering, Decision Making

Cluster analysis plays an important role in decision making process for many
knowledge-based systems. There exist a wide variety of different approaches for
clustering applications including the heuristic techniques, probabilistic
models, and traditional hierarchical algorithms. In this paper, a novel
heuristic approach based on big bang-big crunch algorithm is proposed for
clustering problems. The proposed method not only takes advantage of heuristic
nature to alleviate typical clustering algorithms such as k-means, but it also
benefits from the memory based scheme as compared to its similar heuristic
techniques. Furthermore, the performance of the proposed algorithm is
investigated based on several benchmark test functions as well as on the
well-known datasets. The experimental results show the significant superiority
of the proposed method over the similar algorithms.
Design of the Artificial: lessons from the biological roots of general intelligence
Authors: Nima Dehghani
Date: 2017-03-07 00:00:00

Our fascination with intelligent machines goes back to ancient times with the mythical automaton Talos, Aristotle's mode of mechanical thought (syllogism) and Heron of Alexandria's mechanical machines. However, the quest for Artificial General Intelligence (AGI) has been troubled with repeated failures. Recently, there has been a shift towards bio-inspired software and hardware, but their singular design focus makes them inefficient in achieving AGI. Which set of requirements have to be met in the design of AGI? What are the limits in the design of the artificial? A careful examination of computation in biological systems suggests that evolutionary tinkering of contextual processing of information enabled by a hierarchical architecture is key to building AGI.
FALCON: Feature Driven Selective Classification for Energy-Efficient Image Recognition
Authors: Priyadarshini Panda, Aayush Ankit, Parami Wijesinghe, Kaushik Roy
Date: 2016-09-12 00:00:00
Tasks: BIG-bench Machine Learning, Classification, General Classification

Machine-learning algorithms have shown outstanding image recognition or
classification performance for computer vision applications. However, the
compute and energy requirement for implementing such classifier models for
large-scale problems is quite high. In this paper, we propose Feature Driven
Selective Classification (FALCON) inspired by the biological visual attention
mechanism in the brain to optimize the energy-efficiency of machine-learning
classifiers. We use the consensus in the characteristic features
(color/texture) across images in a dataset to decompose the original
classification problem and construct a tree of classifiers (nodes) with a
generic-to-specific transition in the classification hierarchy. The initial
nodes of the tree separate the instances based on feature information and
selectively enable the latter nodes to perform object specific classification.
The proposed methodology allows selective activation of only those branches and
nodes of the classification tree that are relevant to the input while keeping
the remaining nodes idle. Additionally, we propose a programmable and scalable
Neuromorphic Engine (NeuE) that utilizes arrays of specialized neural
computational elements to execute the FALCON based classifier models for
diverse datasets. The structure of FALCON facilitates the reuse of nodes while
scaling up from small classification problems to larger ones thus allowing us
to construct classifier implementations that are significantly more efficient.
We evaluate our approach for a 12-object classification task on the Caltech101
dataset and 10-object task on CIFAR-10 dataset by constructing FALCON models on
the NeuE platform in 45nm technology. Our results demonstrate significant
improvement in energy-efficiency and training time for minimal loss in output
quality.
Spice up Your Chat: The Intentions and Sentiment Effects of Using Emoji
Authors: Tianran Hu, Han Guo, Hao Sun, Thuy-vy Thi Nguyen, Jiebo Luo
Date: 2017-03-08 00:00:00

Emojis, as a new way of conveying nonverbal cues, are widely adopted in
computer-mediated communications. In this paper, first from a message sender
perspective, we focus on people's motives in using four types of emojis --
positive, neutral, negative, and non-facial. We compare the willingness levels
of using these emoji types for seven typical intentions that people usually
apply nonverbal cues for in communication. The results of extensive statistical
hypothesis tests not only report the popularities of the intentions, but also
uncover the subtle differences between emoji types in terms of intended uses.
Second, from a perspective of message recipients, we further study the
sentiment effects of emojis, as well as their duplications, on verbal messages.
Different from previous studies in emoji sentiment, we study the sentiments of
emojis and their contexts as a whole. The experiment results indicate that the
powers of conveying sentiment are different between four emoji types, and the
sentiment effects of emojis vary in the contexts of different valences.
Exact Dimensionality Selection for Bayesian PCA
Authors: Charles Bouveyron, Pierre Latouche, Pierre-Alexandre Mattei
Date: 2017-03-08 00:00:00
Tasks: Model Selection

We present a Bayesian model selection approach to estimate the intrinsic dimensionality of a high-dimensional dataset. To this end, we introduce a novel formulation of the probabilisitic principal component analysis model based on a normal-gamma prior distribution. In this context, we exhibit a closed-form expression of the marginal likelihood which allows to infer an optimal number of components. We also propose a heuristic based on the expected shape of the marginal likelihood curve in order to choose the hyperparameters. In non-asymptotic frameworks, we show on simulated data that this exact dimensionality selection approach is competitive with both Bayesian and frequentist state-of-the-art methods.
Introduction to Formal Concept Analysis and Its Applications in Information Retrieval and Related Fields
Authors: Dmitry I. Ignatov
Date: 2017-03-08 00:00:00
Tasks: Attribute, BIG-bench Machine Learning, Information Retrieval, Retrieval

This paper is a tutorial on Formal Concept Analysis (FCA) and its
applications. FCA is an applied branch of Lattice Theory, a mathematical
discipline which enables formalisation of concepts as basic units of human
thinking and analysing data in the object-attribute form. Originated in early
80s, during the last three decades, it became a popular human-centred tool for
knowledge representation and data analysis with numerous applications. Since
the tutorial was specially prepared for RuSSIR 2014, the covered FCA topics
include Information Retrieval with a focus on visualisation aspects, Machine
Learning, Data Mining and Knowledge Discovery, Text Mining and several others.
An Integrated and Scalable Platform for Proactive Event-Driven Traffic Management
Authors: Alain Kibangou, Alexander Artikis, Evangelos Michelioudakis, Georgios Paliouras, Marius Schmitt, John Lygeros, Chris Baber, Natan Morar, Fabiana Fournier, Inna Skarbovsky
Date: 2017-03-08 00:00:00
Tasks: Decision Making, Management

Traffic on freeways can be managed by means of ramp meters from Road Traffic
Control rooms. Human operators cannot efficiently manage a network of ramp
meters. To support them, we present an intelligent platform for traffic
management which includes a new ramp metering coordination scheme in the
decision making module, an efficient dashboard for interacting with human
operators, machine learning tools for learning event definitions and Complex
Event Processing tools able to deal with uncertainties inherent to the traffic
use case. Unlike the usual approach, the devised event-driven platform is able
to predict a congestion up to 4 minutes before it really happens. Proactive
decision making can then be established leading to significant improvement of
traffic conditions.
Deep Reservoir Computing Using Cellular Automata
Authors: Stefano Nichele, Andreas Molund
Date: 2017-03-08 00:00:00
Tasks: speech-recognition, Speech Recognition

Recurrent Neural Networks (RNNs) have been a prominent concept within
artificial intelligence. They are inspired by Biological Neural Networks (BNNs)
and provide an intuitive and abstract representation of how BNNs work. Derived
from the more generic Artificial Neural Networks (ANNs), the recurrent ones are
meant to be used for temporal tasks, such as speech recognition, because they
are capable of memorizing historic input. However, such networks are very time
consuming to train as a result of their inherent nature. Recently, Echo State
Networks and Liquid State Machines have been proposed as possible RNN
alternatives, under the name of Reservoir Computing (RC). RCs are far more easy
to train. In this paper, Cellular Automata are used as reservoir, and are
tested on the 5-bit memory task (a well known benchmark within the RC
community). The work herein provides a method of mapping binary inputs from the
task onto the automata, and a recurrent architecture for handling the
sequential aspects of it. Furthermore, a layered (deep) reservoir architecture
is proposed. Performances are compared towards earlier work, in addition to its
single-layer version. Results show that the single CA reservoir system yields
similar results to state-of-the-art work. The system comprised of two layered
reservoirs do show a noticeable improvement compared to a single CA reservoir.
This indicates potential for further research and provides valuable insight on
how to design CA reservoir systems.
DeepCoder: Learning to Write Programs
Authors: Matej Balog, Alexander L. Gaunt, Marc Brockschmidt, Sebastian Nowozin, Daniel Tarlow
Date: 2016-11-07 00:00:00
Tasks: Enumerative Search, Program Synthesis

We develop a first line of attack for solving programming competition-style
problems from input-output examples using deep learning. The approach is to
train a neural network to predict properties of the program that generated the
outputs from the inputs. We use the neural network's predictions to augment
search techniques from the programming languages community, including
enumerative search and an SMT-based solver. Empirically, we show that our
approach leads to an order of magnitude speedup over the strong non-augmented
baselines and a Recurrent Neural Network approach, and that we are able to
solve problems of difficulty comparable to the simplest problems on programming
competition websites.
Oracle Complexity of Second-Order Methods for Finite-Sum Problems
Authors: Yossi Arjevani, Ohad Shamir
Date: 2016-11-15 00:00:00
Proceeding: ICML 2017 8
Tasks: Second-order methods

Finite-sum optimization problems are ubiquitous in machine learning, and are
commonly solved using first-order methods which rely on gradient computations.
Recently, there has been growing interest in \emph{second-order} methods, which
rely on both gradients and Hessians. In principle, second-order methods can
require much fewer iterations than first-order methods, and hold the promise
for more efficient algorithms. Although computing and manipulating Hessians is
prohibitive for high-dimensional problems in general, the Hessians of
individual functions in finite-sum problems can often be efficiently computed,
e.g. because they possess a low-rank structure. Can second-order information
indeed be used to solve such problems more efficiently? In this paper, we
provide evidence that the answer -- perhaps surprisingly -- is negative, at
least in terms of worst-case guarantees. However, we also discuss what
additional assumptions and algorithmic approaches might potentially circumvent
this negative result.
Byzantine-Tolerant Machine Learning
Authors: Peva Blanchard, El Mahdi El Mhamdi, Rachid Guerraoui, Julien Stainer
Date: 2017-03-08 00:00:00
Tasks: BIG-bench Machine Learning

The growth of data, the need for scalability and the complexity of models
used in modern machine learning calls for distributed implementations. Yet, as
of today, distributed machine learning frameworks have largely ignored the
possibility of arbitrary (i.e., Byzantine) failures. In this paper, we study
the robustness to Byzantine failures at the fundamental level of stochastic
gradient descent (SGD), the heart of most machine learning algorithms. Assuming
a set of $n$ workers, up to $f$ of them being Byzantine, we ask how robust can
SGD be, without limiting the dimension, nor the size of the parameter space.
  We first show that no gradient descent update rule based on a linear
combination of the vectors proposed by the workers (i.e, current approaches)
tolerates a single Byzantine failure. We then formulate a resilience property
of the update rule capturing the basic requirements to guarantee convergence
despite $f$ Byzantine workers. We finally propose Krum, an update rule that
satisfies the resilience property aforementioned. For a $d$-dimensional
learning problem, the time complexity of Krum is $O(n^2 \cdot (d + \log n))$.
Clustering-Based Relational Unsupervised Representation Learning with an Explicit Distributed Representation
Authors: Sebastijan Dumancic, Hendrik Blockeel
Date: 2016-06-28 00:00:00
Tasks: Clustering, General Classification, Relational Reasoning, Representation Learning

The goal of unsupervised representation learning is to extract a new
representation of data, such that solving many different tasks becomes easier.
Existing methods typically focus on vectorized data and offer little support
for relational data, which additionally describe relationships among instances.
In this work we introduce an approach for relational unsupervised
representation learning. Viewing a relational dataset as a hypergraph, new
features are obtained by clustering vertices and hyperedges. To find a
representation suited for many relational learning tasks, a wide range of
similarities between relational objects is considered, e.g. feature and
structural similarities. We experimentally evaluate the proposed approach and
show that models learned on such latent representations perform better, have
lower complexity, and outperform the existing approaches on classification
tasks.
Kiefer Wolfowitz Algorithm is Asymptotically Optimal for a Class of Non-Stationary Bandit Problems
Authors: Rahul Singh, Taposh Banerjee
Date: 2017-02-26 00:00:00

We consider the problem of designing an allocation rule or an "online
learning algorithm" for a class of bandit problems in which the set of control
actions available at each time $s$ is a convex, compact subset of
$\mathbb{R}^d$. Upon choosing an action $x$ at time $s$, the algorithm obtains
a noisy value of the unknown and time-varying function $f_s$ evaluated at $x$.
The "regret" of an algorithm is the gap between its expected reward, and the
reward earned by a strategy which has the knowledge of the function $f_s$ at
each time $s$ and hence chooses the action $x_s$ that maximizes $f_s$.
  For this non-stationary bandit problem set-up, we consider two variants of
the Kiefer Wolfowitz (KW) algorithm i) KW with fixed step-size $\beta$, and ii)
KW with sliding window of length $L$. We show that if the number of times that
the function $f_s$ varies during time $T$ is $o(T)$, and if the learning rates
of the proposed algorithms are chosen "optimally", then the regret of the
proposed algorithms is $o(T)$, and hence the algorithms are asymptotically
efficient.