pwc-archive/papers-with-abstracts huggingface dataset:
High Dimensional Bayesian Optimisation and Bandits via Additive Models
Authors: Kirthevasan Kandasamy, Jeff Schneider, Barnabas Poczos
Date: 2015-03-05 00:00:00
Tasks: Additive models, Bayesian Optimisation, Face Detection, Vocal Bursts Intensity Prediction

Bayesian Optimisation (BO) is a technique used in optimising a
$D$-dimensional function which is typically expensive to evaluate. While there
have been many successes for BO in low dimensions, scaling it to high
dimensions has been notoriously difficult. Existing literature on the topic are
under very restrictive settings. In this paper, we identify two key challenges
in this endeavour. We tackle these challenges by assuming an additive structure
for the function. This setting is substantially more expressive and contains a
richer class of functions than previous work. We prove that, for additive
functions the regret has only linear dependence on $D$ even though the function
depends on all $D$ dimensions. We also demonstrate several other statistical
and computational benefits in our framework. Via synthetic examples, a
scientific simulation and a face detection problem we demonstrate that our
method outperforms naive BO on additive functions and on several examples where
the function is not additive.
Learning scale-variant and scale-invariant features for deep image classification
Authors: Nanne van Noord, Eric Postma
Date: 2016-02-03 00:00:00
Tasks: General Classification, image-classification, Image Classification

Convolutional Neural Networks (CNNs) require large image corpora to be
trained on classification tasks. The variation in image resolutions, sizes of
objects and patterns depicted, and image scales, hampers CNN training and
performance, because the task-relevant information varies over spatial scales.
Previous work attempting to deal with such scale variations focused on
encouraging scale-invariant CNN representations. However, scale-invariant
representations are incomplete representations of images, because images
contain scale-variant information as well. This paper addresses the combined
development of scale-invariant and scale-variant representations. We propose a
multi- scale CNN method to encourage the recognition of both types of features
and evaluate it on a challenging image classification task involving
task-relevant characteristics at multiple scales. The results show that our
multi-scale CNN outperforms single-scale CNN. This leads to the conclusion that
encouraging the combined development of a scale-invariant and scale-variant
representation in CNNs is beneficial to image recognition performance.
Deep Gate Recurrent Neural Network
Authors: Yuan Gao, Dorota Glowacka
Date: 2016-04-11 00:00:00

This paper introduces two recurrent neural network structures called Simple
Gated Unit (SGU) and Deep Simple Gated Unit (DSGU), which are general
structures for learning long term dependencies. Compared to traditional Long
Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU), both structures
require fewer parameters and less computation time in sequence classification
tasks. Unlike GRU and LSTM, which require more than one gates to control
information flow in the network, SGU and DSGU only use one multiplicative gate
to control the flow of information. We show that this difference can accelerate
the learning speed in tasks that require long dependency information. We also
show that DSGU is more numerically stable than SGU. In addition, we also
propose a standard way of representing inner structure of RNN called RNN
Conventional Graph (RCG), which helps analyzing the relationship between input
units and hidden units of RNN.
Structured Receptive Fields in CNNs
Authors: Jörn-Henrik Jacobsen, Jan van Gemert, Zhongyu Lou, Arnold W. M. Smeulders
Date: 2016-05-10 00:00:00
Proceeding: CVPR 2016 6

Learning powerful feature representations with CNNs is hard when training
data are limited. Pre-training is one way to overcome this, but it requires
large datasets sufficiently similar to the target domain. Another option is to
design priors into the model, which can range from tuned hyperparameters to
fully engineered representations like Scattering Networks. We combine these
ideas into structured receptive field networks, a model which has a fixed
filter basis and yet retains the flexibility of CNNs. This flexibility is
achieved by expressing receptive fields in CNNs as a weighted sum over a fixed
basis which is similar in spirit to Scattering Networks. The key difference is
that we learn arbitrary effective filter sets from the basis rather than
modeling the filters. This approach explicitly connects classical multiscale
image analysis with general CNNs. With structured receptive field networks, we
improve considerably over unstructured CNNs for small and medium dataset
scenarios as well as over Scattering for large datasets. We validate our
findings on ILSVRC2012, Cifar-10, Cifar-100 and MNIST. As a realistic small
dataset example, we show state-of-the-art classification results on popular 3D
MRI brain-disease datasets where pre-training is difficult due to a lack of
large public datasets in a similar domain.
Natural Language Semantics and Computability
Authors: Richard Moot, Christian Retoré
Date: 2016-05-13 00:00:00

This paper is a reflexion on the computability of natural language semantics.
It does not contain a new model or new results in the formal semantics of
natural language: it is rather a computational analysis of the logical models
and algorithms currently used in natural language semantics, defined as the
mapping of a statement to logical formulas - formulas, because a statement can
be ambiguous. We argue that as long as possible world semantics is left out,
one can compute the semantic representation(s) of a given statement, including
aspects of lexical meaning. We also discuss the algorithmic complexity of this
process.
Fast methods for training Gaussian processes on large data sets
Authors: Christopher J. Moore, Alvin J. K. Chua, Christopher P. L. Berry, Jonathan R. Gair
Date: 2016-04-05 00:00:00
Tasks: Gaussian Processes, GPR, regression

Gaussian process regression (GPR) is a non-parametric Bayesian technique for
interpolating or fitting data. The main barrier to further uptake of this
powerful tool rests in the computational costs associated with the matrices
which arise when dealing with large data sets. Here, we derive some simple
results which we have found useful for speeding up the learning stage in the
GPR algorithm, and especially for performing Bayesian model comparison between
different covariance functions. We apply our techniques to both synthetic and
real data and quantify the speed-up relative to using nested sampling to
numerically evaluate model evidences.
Wisdom of Crowds cluster ensemble
Authors: Hosein Alizadeh, Muhammad Yousefnezhad, Behrouz Minaei Bidgoli
Date: 2016-05-13 00:00:00
Tasks: Decision Making, Diversity

The Wisdom of Crowds is a phenomenon described in social science that
suggests four criteria applicable to groups of people. It is claimed that, if
these criteria are satisfied, then the aggregate decisions made by a group will
often be better than those of its individual members. Inspired by this concept,
we present a novel feedback framework for the cluster ensemble problem, which
we call Wisdom of Crowds Cluster Ensemble (WOCCE). Although many conventional
cluster ensemble methods focusing on diversity have recently been proposed,
WOCCE analyzes the conditions necessary for a crowd to exhibit this collective
wisdom. These include decentralization criteria for generating primary results,
independence criteria for the base algorithms, and diversity criteria for the
ensemble members. We suggest appropriate procedures for evaluating these
measures, and propose a new measure to assess the diversity. We evaluate the
performance of WOCCE against some other traditional base algorithms as well as
state-of-the-art ensemble methods. The results demonstrate the efficiency of
WOCCE's aggregate decision-making compared to other algorithms.
Towards Empathetic Human-Robot Interactions
Authors: Pascale Fung, Dario Bertero, Yan Wan, Anik Dey, Ricky Ho Yin Chan, Farhad Bin Siddique, Yang Yang, Chien-Sheng Wu, Ruixi Lin
Date: 2016-05-13 00:00:00
Tasks: Sentiment Analysis

Since the late 1990s when speech companies began providing their
customer-service software in the market, people have gotten used to speaking to
machines. As people interact more often with voice and gesture controlled
machines, they expect the machines to recognize different emotions, and
understand other high level communication features such as humor, sarcasm and
intention. In order to make such communication possible, the machines need an
empathy module in them which can extract emotions from human speech and
behavior and can decide the correct response of the robot. Although research on
empathetic robots is still in the early stage, we described our approach using
signal processing techniques, sentiment analysis and machine learning
algorithms to make robots that can "understand" human emotion. We propose Zara
the Supergirl as a prototype system of empathetic robots. It is a software
based virtual android, with an animated cartoon character to present itself on
the screen. She will get "smarter" and more empathetic through its deep
learning algorithms, and by gathering more data and learning from it. In this
paper, we present our work so far in the areas of deep learning of emotion and
sentiment recognition, as well as humor recognition. We hope to explore the
future direction of android development and how it can help improve people's
lives.
A Reinforcement Learning System to Encourage Physical Activity in Diabetes Patients
Authors: Irit Hochberg, Guy Feraru, Mark Kozdoba, Shie Mannor, Moshe Tennenholtz, Elad Yom-Tov
Date: 2016-05-13 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Regular physical activity is known to be beneficial to people suffering from
diabetes type 2. Nevertheless, most such people are sedentary. Smartphones
create new possibilities for helping people to adhere to their physical
activity goals, through continuous monitoring and communication, coupled with
personalized feedback.
  We provided 27 sedentary diabetes type 2 patients with a smartphone-based
pedometer and a personal plan for physical activity. Patients were sent SMS
messages to encourage physical activity between once a day and once per week.
Messages were personalized through a Reinforcement Learning (RL) algorithm
which optimized messages to improve each participant's compliance with the
activity regimen. The RL algorithm was compared to a static policy for sending
messages and to weekly reminders.
  Our results show that participants who received messages generated by the RL
algorithm increased the amount of activity and pace of walking, while the
control group patients did not. Patients assigned to the RL algorithm group
experienced a superior reduction in blood glucose levels (HbA1c) compared to
control policies, and longer participation caused greater reductions in blood
glucose levels. The learning algorithm improved gradually in predicting which
messages would lead participants to exercise.
  Our results suggest that a mobile phone application coupled with a learning
algorithm can improve adherence to exercise in diabetic patients. As a learning
algorithm is automated, and delivers personalized messages, it could be used in
large populations of diabetic patients to improve health and glycemic control.
Our results can be expanded to other areas where computer-led health coaching
of humans may have a positive impact.
Fast Semantic Image Segmentation with High Order Context and Guided Filtering
Authors: Falong Shen, Gang Zeng
Date: 2016-05-13 00:00:00
Tasks: Image Segmentation, Semantic Segmentation, Vocal Bursts Intensity Prediction

This paper describes a fast and accurate semantic image segmentation approach
that encodes not only the discriminative features from deep neural networks,
but also the high-order context compatibility among adjacent objects as well as
low level image features. We formulate the underlying problem as the
conditional random field that embeds local feature extraction, clique potential
construction, and guided filtering within the same framework, and provide an
efficient coarse-to-fine solver. At the coarse level, we combine local feature
representation and context interaction using a deep convolutional network, and
directly learn the interaction from high order cliques with a message passing
routine, avoiding time-consuming explicit graph inference for joint probability
distribution. At the fine level, we introduce a guided filtering interpretation
for the mean field algorithm, and achieve accurate object boundaries with 100+
faster than classic learning methods. The two parts are connected and jointly
trained in an end-to-end fashion. Experimental results on Pascal VOC 2012
dataset have shown that the proposed algorithm outperforms the
state-of-the-art, and that it achieves the rank 1 performance at the time of
submission, both of which prove the effectiveness of this unified framework for
semantic image segmentation.
Modeling the Mind: A brief review
Authors: Gabriel Makdah
Date: 2015-07-04 00:00:00

The brain is a powerful tool used to achieve amazing feats. There have been
several significant advances in neuroscience and artificial brain research in
the past two decades. This article is a review of such advances, ranging from
the concepts of connectionism, to neural network architectures and
high-dimensional representations. There have also been advances in biologically
inspired cognitive architectures of which we will cite a few. We will be
positioning relatively specific models in a much broader perspective, while
comparing and contrasting their advantages and weaknesses. The projects
presented are targeted to model the brain at different levels, utilizing
different methodologies.
Track Extraction with Hidden Reciprocal Chain Models
Authors: George Stamatescu, Langford B. White, Riley Bruce-Doust
Date: 2016-05-13 00:00:00

This paper develops Bayesian track extraction algorithms for targets modelled
as hidden reciprocal chains (HRC). HRC are a class of finite-state random
process models that generalise the familiar hidden Markov chains (HMC). HRC are
able to model the "intention" of a target to proceed from a given origin to a
destination, behaviour which cannot be properly captured by a HMC. While
Bayesian estimation problems for HRC have previously been studied, this paper
focusses principally on the problem of track extraction, of which the primary
task is confirming target existence in a set of detections obtained from
thresholding sensor measurements. Simulation examples are presented which show
that the additional model information contained in a HRC improves detection
performance when compared to HMC models.
Cross-Domain Visual Matching via Generalized Similarity Measure and Feature Learning
Authors: Liang Lin, Guangrun Wang, WangMeng Zuo, Xiangchu Feng, Lei Zhang
Date: 2016-05-13 00:00:00
Tasks: Face Verification, Model Optimization, Person Re-Identification, Representation Learning

Cross-domain visual data matching is one of the fundamental problems in many
real-world vision tasks, e.g., matching persons across ID photos and
surveillance videos. Conventional approaches to this problem usually involves
two steps: i) projecting samples from different domains into a common space,
and ii) computing (dis-)similarity in this space based on a certain distance.
In this paper, we present a novel pairwise similarity measure that advances
existing models by i) expanding traditional linear projections into affine
transformations and ii) fusing affine Mahalanobis distance and Cosine
similarity by a data-driven combination. Moreover, we unify our similarity
measure with feature representation learning via deep convolutional neural
networks. Specifically, we incorporate the similarity measure matrix into the
deep architecture, enabling an end-to-end way of model optimization. We
extensively evaluate our generalized similarity model in several challenging
cross-domain matching tasks: person re-identification under different views and
face verification over different modalities (i.e., faces from still images and
videos, older and younger faces, and sketch and photo portraits). The
experimental results demonstrate superior performance of our model over other
state-of-the-art methods.
Transfer Hashing with Privileged Information
Authors: Joey Tianyi Zhou, Xinxing Xu, Sinno Jialin Pan, Ivor W. Tsang, Zheng Qin, Rick Siow Mong Goh
Date: 2016-05-13 00:00:00
Tasks: Quantization, Transfer Learning

Most existing learning to hash methods assume that there are sufficient data,
either labeled or unlabeled, on the domain of interest (i.e., the target
domain) for training. However, this assumption cannot be satisfied in some
real-world applications. To address this data sparsity issue in hashing,
inspired by transfer learning, we propose a new framework named Transfer
Hashing with Privileged Information (THPI). Specifically, we extend the
standard learning to hash method, Iterative Quantization (ITQ), in a transfer
learning manner, namely ITQ+. In ITQ+, a new slack function is learned from
auxiliary data to approximate the quantization error in ITQ. We developed an
alternating optimization approach to solve the resultant optimization problem
for ITQ+. We further extend ITQ+ to LapITQ+ by utilizing the geometry structure
among the auxiliary data for learning more precise binary codes in the target
domain. Extensive experiments on several benchmark datasets verify the
effectiveness of our proposed approaches through comparisons with several
state-of-the-art baselines.
Tensor decomposition with generalized lasso penalties
Authors: Oscar Hernan Madrid Padilla, James G. Scott
Date: 2015-02-24 00:00:00
Tasks: regression, Tensor Decomposition

We present an approach for penalized tensor decomposition (PTD) that
estimates smoothly varying latent factors in multi-way data. This generalizes
existing work on sparse tensor decomposition and penalized matrix
decompositions, in a manner parallel to the generalized lasso for regression
and smoothing problems. Our approach presents many nontrivial challenges at the
intersection of modeling and computation, which are studied in detail. An
efficient coordinate-wise optimization algorithm for (PTD) is presented, and
its convergence properties are characterized. The method is applied both to
simulated data and real data on flu hospitalizations in Texas. These results
show that our penalized tensor decomposition can offer major improvements on
existing methods for analyzing multi-way data that exhibit smooth spatial or
temporal features.
Optimizing human-interpretable dialog management policy using Genetic Algorithm
Authors: Hang Ren, Weiqun Xu, Yonghong Yan
Date: 2016-05-12 00:00:00
Tasks: Management, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), User Simulation

Automatic optimization of spoken dialog management policies that are robust
to environmental noise has long been the goal for both academia and industry.
Approaches based on reinforcement learning have been proved to be effective.
However, the numerical representation of dialog policy is
human-incomprehensible and difficult for dialog system designers to verify or
modify, which limits its practical application. In this paper we propose a
novel framework for optimizing dialog policies specified in domain language
using genetic algorithm. The human-interpretable representation of policy makes
the method suitable for practical employment. We present learning algorithms
using user simulation and real human-machine dialogs respectively.Empirical
experimental results are given to show the effectiveness of the proposed
approach.
How do neurons operate on sparse distributed representations? A mathematical theory of sparsity, neurons and active dendrites
Authors: Subutai Ahmad, Jeff Hawkins
Date: 2016-01-05 00:00:00

We propose a formal mathematical model for sparse representations and active
dendrites in neocortex. Our model is inspired by recent experimental findings
on active dendritic processing and NMDA spikes in pyramidal neurons. These
experimental and modeling studies suggest that the basic unit of pattern memory
in the neocortex is instantiated by small clusters of synapses operated on by
localized non-linear dendritic processes. We derive a number of scaling laws
that characterize the accuracy of such dendrites in detecting activation
patterns in a neuronal population under adverse conditions. We introduce the
union property which shows that synapses for multiple patterns can be randomly
mixed together within a segment and still lead to highly accurate recognition.
We describe simulation results that provide further insight into sparse
representations as well as two primary results. First we show that pattern
recognition by a neuron with active dendrites can be extremely accurate and
robust with high dimensional sparse inputs even when using a tiny number of
synapses to recognize large patterns. Second, equations representing
recognition accuracy of a dendrite predict optimal NMDA spiking thresholds
under a generous set of assumptions. The prediction tightly matches NMDA
spiking thresholds measured in the literature. Our model matches many of the
known properties of pyramidal neurons. As such the theory provides a
mathematical framework for understanding the benefits and limits of sparse
representations in cortical networks.
Machine Comprehension Based on Learning to Rank
Authors: Tian Tian, Yuezhang Li
Date: 2016-05-11 00:00:00
Tasks: BIG-bench Machine Learning, Learning-To-Rank, Reading Comprehension

Machine comprehension plays an essential role in NLP and has been widely
explored with dataset like MCTest. However, this dataset is too simple and too
small for learning true reasoning abilities. \cite{hermann2015teaching}
therefore release a large scale news article dataset and propose a deep LSTM
reader system for machine comprehension. However, the training process is
expensive. We therefore try feature-engineered approach with semantics on the
new dataset to see how traditional machine learning technique and semantics can
help with machine comprehension. Meanwhile, our proposed L2R reader system
achieves good performance with efficiency and less training data.
Joint Embeddings of Hierarchical Categories and Entities
Authors: Yuezhang Li, Ronghuo Zheng, Tian Tian, Zhiting Hu, Rahul Iyer, Katia Sycara
Date: 2016-05-12 00:00:00

Due to the lack of structured knowledge applied in learning distributed
representation of categories, existing work cannot incorporate category
hierarchies into entity information.~We propose a framework that embeds
entities and categories into a semantic space by integrating structured
knowledge and taxonomy hierarchy from large knowledge bases. The framework
allows to compute meaningful semantic relatedness between entities and
categories.~Compared with the previous state of the art, our framework can
handle both single-word concepts and multiple-word concepts with superior
performance in concept categorization and semantic relatedness.