pwc-archive/papers-with-abstracts huggingface dataset:
Which phoneme-to-viseme maps best improve visual-only computer lip-reading?
Authors: Helen L. Bear, Richard W. Harvey, Barry-John Theobald, Yuxuan Lan
Date: 2017-10-03 00:00:00
Tasks: Lip Reading, speech-recognition, Speech Recognition, Visual Speech Recognition

A critical assumption of all current visual speech recognition systems is
that there are visual speech units called visemes which can be mapped to units
of acoustic speech, the phonemes. Despite there being a number of published
maps it is infrequent to see the effectiveness of these tested, particularly on
visual-only lip-reading (many works use audio-visual speech). Here we examine
120 mappings and consider if any are stable across talkers. We show a method
for devising maps based on phoneme confusions from an automated lip-reading
system, and we present new mappings that show improvements for individual
talkers.
Some observations on computer lip-reading: moving from the dream to the reality
Authors: Helen L. Bear, Gari Owen, Richard Harvey, Barry-John Theobald
Date: 2017-10-03 00:00:00
Tasks: Lip Reading

In the quest for greater computer lip-reading performance there are a number
of tacit assumptions which are either present in the datasets (high resolution
for example) or in the methods (recognition of spoken visual units called
visemes for example). Here we review these and other assumptions and show the
surprising result that computer lip-reading is not heavily constrained by video
resolution, pose, lighting and other practical factors. However, the working
assumption that visemes, which are the visual equivalent of phonemes, are the
best unit for recognition does need further examination. We conclude that
visemes, which were defined over a century ago, are unlikely to be optimal for
a modern computer lip-reading system.
Optimal DNN Primitive Selection with Partitioned Boolean Quadratic Programming
Authors: Andrew Anderson, David Gregg
Date: 2017-10-03 00:00:00

Deep Neural Networks (DNNs) require very large amounts of computation both
for training and for inference when deployed in the field. Many different
algorithms have been proposed to implement the most computationally expensive
layers of DNNs. Further, each of these algorithms has a large number of
variants, which offer different trade-offs of parallelism, data locality,
memory footprint, and execution time. In addition, specific algorithms operate
much more efficiently on specialized data layouts and formats.
  We state the problem of optimal primitive selection in the presence of data
format transformations, and show that it is NP-hard by demonstrating an
embedding in the Partitioned Boolean Quadratic Assignment problem (PBQP).
  We propose an analytic solution via a PBQP solver, and evaluate our approach
experimentally by optimizing several popular DNNs using a library of more than
70 DNN primitives, on an embedded platform and a general purpose platform. We
show experimentally that significant gains are possible versus the state of the
art vendor libraries by using a principled analytic solution to the problem of
layout selection in the presence of data format transformations.
Resolution limits on visual speech recognition
Authors: Helen L. Bear, Richard Harvey, Barry-John Theobald, Yuxuan Lan
Date: 2017-10-03 00:00:00
Tasks: Lip Reading, speech-recognition, Speech Recognition, Visual Speech Recognition

Visual-only speech recognition is dependent upon a number of factors that can
be difficult to control, such as: lighting; identity; motion; emotion and
expression. But some factors, such as video resolution are controllable, so it
is surprising that there is not yet a systematic study of the effect of
resolution on lip-reading. Here we use a new data set, the Rosetta Raven data,
to train and test recognizers so we can measure the affect of video resolution
on recognition accuracy. We conclude that, contrary to common practice,
resolution need not be that great for automatic lip-reading. However it is
highly unlikely that automatic lip-reading can work reliably when the distance
between the bottom of the lower lip and the top of the upper lip is less than
four pixels at rest.
Remote Sensing Image Classification with Large Scale Gaussian Processes
Authors: Pablo Morales-Alvarez, Adrian Perez-Suay, Rafael Molina, Gustau Camps-Valls
Date: 2017-10-02 00:00:00
Tasks: Classification, Cloud Detection, Gaussian Processes, General Classification, image-classification, Image Classification, Remote Sensing Image Classification

Current remote sensing image classification problems have to deal with an
unprecedented amount of heterogeneous and complex data sources. Upcoming
missions will soon provide large data streams that will make land cover/use
classification difficult. Machine learning classifiers can help at this, and
many methods are currently available. A popular kernel classifier is the
Gaussian process classifier (GPC), since it approaches the classification
problem with a solid probabilistic treatment, thus yielding confidence
intervals for the predictions as well as very competitive results to
state-of-the-art neural networks and support vector machines. However, its
computational cost is prohibitive for large scale applications, and constitutes
the main obstacle precluding wide adoption. This paper tackles this problem by
introducing two novel efficient methodologies for Gaussian Process (GP)
classification. We first include the standard random Fourier features
approximation into GPC, which largely decreases its computational cost and
permits large scale remote sensing image classification. In addition, we
propose a model which avoids randomly sampling a number of Fourier frequencies,
and alternatively learns the optimal ones within a variational Bayes approach.
The performance of the proposed methods is illustrated in complex problems of
cloud detection from multispectral imagery and infrared sounding data.
Excellent empirical results support the proposal in both computational cost and
accuracy.
A Mutually-Dependent Hadamard Kernel for Modelling Latent Variable Couplings
Authors: Sami Remes, Markus Heinonen, Samuel Kaski
Date: 2017-02-27 00:00:00
Tasks: Bayesian Inference, Classification, EEG, Electroencephalogram (EEG), General Classification

We introduce a novel kernel that models input-dependent couplings across
multiple latent processes. The pairwise joint kernel measures covariance along
inputs and across different latent signals in a mutually-dependent fashion. A
latent correlation Gaussian process (LCGP) model combines these non-stationary
latent components into multiple outputs by an input-dependent mixing matrix.
Probit classification and support for multiple observation sets are derived by
Variational Bayesian inference. Results on several datasets indicate that the
LCGP model can recover the correlations between latent signals while
simultaneously achieving state-of-the-art performance. We highlight the latent
covariances with an EEG classification dataset where latent brain processes and
their couplings simultaneously emerge from the model.
Simulating Structure-from-Motion
Authors: Martin Hahner, Orestis Varesis, Panagiotis Bountouris
Date: 2017-10-03 00:00:00

The implementation of a Structure-from-Motion (SfM) pipeline from a
synthetically generated scene as well as the investigation of the faithfulness
of diverse reconstructions is the subject of this project. A series of
different SfM reconstructions are implemented and their camera pose estimations
are being contrasted with their respective ground truth locations. Finally,
injection of ground truth location data into the rendered images in order to
reduce the estimation error of the camera poses is studied as well.
Generative Models for Learning from Crowds
Authors: Chi Hong
Date: 2017-06-13 00:00:00
Tasks: Variational Inference

In this paper, we propose generative probabilistic models for label
aggregation. We use Gibbs sampling and a novel variational inference algorithm
to perform the posterior inference. Empirical results show that our methods
consistently outperform state-of-the-art methods.
Learning Affinity via Spatial Propagation Networks
Authors: Sifei Liu, Shalini De Mello, Jinwei Gu, Guangyu Zhong, Ming-Hsuan Yang, Jan Kautz
Date: 2017-10-03 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Colorization, Face Parsing, Image Matting, Image Segmentation, Segmentation, Semantic Segmentation

In this paper, we propose spatial propagation networks for learning the
affinity matrix for vision tasks. We show that by constructing a row/column
linear propagation model, the spatially varying transformation matrix exactly
constitutes an affinity matrix that models dense, global pairwise relationships
of an image. Specifically, we develop a three-way connection for the linear
propagation model, which (a) formulates a sparse transformation matrix, where
all elements can be the output from a deep CNN, but (b) results in a dense
affinity matrix that effectively models any task-specific pairwise similarity
matrix. Instead of designing the similarity kernels according to image features
of two points, we can directly output all the similarities in a purely
data-driven manner. The spatial propagation network is a generic framework that
can be applied to many affinity-related tasks, including but not limited to
image matting, segmentation and colorization, to name a few. Essentially, the
model can learn semantically-aware affinity values for high-level vision tasks
due to the powerful learning capability of the deep neural network classifier.
We validate the framework on the task of refinement for image segmentation
boundaries. Experiments on the HELEN face parsing and PASCAL VOC-2012 semantic
segmentation tasks show that the spatial propagation network provides a
general, effective and efficient solution for generating high-quality
segmentation results.
NonSTOP: A NonSTationary Online Prediction Method for Time Series
Authors: Christopher Xie, Avleen Bijral, Juan Lavista Ferres
Date: 2016-11-08 00:00:00
Tasks: Prediction, Time Series, Time Series Analysis

We present online prediction methods for time series that let us explicitly
handle nonstationary artifacts (e.g. trend and seasonality) present in most
real time series. Specifically, we show that applying appropriate
transformations to such time series before prediction can lead to improved
theoretical and empirical prediction performance. Moreover, since these
transformations are usually unknown, we employ the learning with experts
setting to develop a fully online method (NonSTOP-NonSTationary Online
Prediction) for predicting nonstationary time series. This framework allows for
seasonality and/or other trends in univariate time series and cointegration in
multivariate time series. Our algorithms and regret analysis subsume recent
related work while significantly expanding the applicability of such methods.
For all the methods, we provide sub-linear regret bounds using relaxed
assumptions. The theoretical guarantees do not fully capture the benefits of
the transformations, thus we provide a data-dependent analysis of the
follow-the-leader algorithm that provides insight into the success of using
such transformations. We support all of our results with experiments on
simulated and real data.
Training Feedforward Neural Networks with Standard Logistic Activations is Feasible
Authors: Emanuele Sansone, Francesco G. B. De Natale
Date: 2017-10-03 00:00:00

Training feedforward neural networks with standard logistic activations is
considered difficult because of the intrinsic properties of these sigmoidal
functions. This work aims at showing that these networks can be trained to
achieve generalization performance comparable to those based on hyperbolic
tangent activations. The solution consists on applying a set of conditions in
parameter initialization, which have been derived from the study of the
properties of a single neuron from an information-theoretic perspective. The
proposed initialization is validated through an extensive experimental
analysis.
Is Structure Necessary for Modeling Argument Expectations in Distributional Semantics?
Authors: Emmanuele Chersoni, Enrico Santus, Philippe Blache, Alessandro Lenci
Date: 2017-10-03 00:00:00
Proceeding: WS 2017 1

Despite the number of NLP studies dedicated to thematic fit estimation,
little attention has been paid to the related task of composing and updating
verb argument expectations. The few exceptions have mostly modeled this
phenomenon with structured distributional models, implicitly assuming a
similarly structured representation of events. Recent experimental evidence,
however, suggests that human processing system could also exploit an
unstructured "bag-of-arguments" type of event representation to predict
upcoming input. In this paper, we re-implement a traditional structured model
and adapt it to compare the different hypotheses concerning the degree of
structure in our event knowledge, evaluating their relative performance in the
task of the argument expectations update.
Annotation and Detection of Emotion in Text-based Dialogue Systems with CNN
Authors: Jialiang Zhao, Qi Gao
Date: 2017-10-03 00:00:00

Knowledge of users' emotion states helps improve human-computer interaction.
In this work, we presented EmoNet, an emotion detector of Chinese daily
dialogues based on deep convolutional neural networks. In order to maintain the
original linguistic features, such as the order, commonly used methods like
segmentation and keywords extraction were not adopted, instead we increased the
depth of CNN and tried to let CNN learn inner linguistic relationships. Our
main contribution is that we presented a new model and a new pipeline which can
be used in multi-language environment to solve sentimental problems.
Experimental results shows EmoNet has a great capacity in learning the emotion
of dialogues and achieves a better result than other state of art detectors do.
Joint Person Re-identification and Camera Network Topology Inference in Multiple Cameras
Authors: Yeong-Jun Cho, Su-A Kim, Jae-Han Park, Kyuewang Lee, Kuk-Jin Yoon
Date: 2017-10-03 00:00:00
Tasks: Person Re-Identification

Person re-identification is the task of recognizing or identifying a person
across multiple views in multi-camera networks. Although there has been much
progress in person re-identification, person re-identification in large-scale
multi-camera networks still remains a challenging task because of the large
spatio-temporal uncertainty and high complexity due to a large number of
cameras and people. To handle these difficulties, additional information such
as camera network topology should be provided, which is also difficult to
automatically estimate, unfortunately. In this study, we propose a unified
framework which jointly solves both person re-identification and camera network
topology inference problems with minimal prior knowledge about the
environments. The proposed framework takes general multi-camera network
environments into account and can be applied to online person re-identification
in large-scale multi-camera networks. In addition, to effectively show the
superiority of the proposed framework, we provide a new person
re-identification dataset with full annotations, named SLP, captured in the
multi-camera network consisting of nine non-overlapping cameras. Experimental
results using our person re-identification and public datasets show that the
proposed methods are promising for both person re-identification and camera
topology inference tasks.
SwGridNet: A Deep Convolutional Neural Network based on Grid Topology for Image Classification
Authors: Atsushi Takeda
Date: 2017-09-22 00:00:00
Tasks: Classification, Ensemble Learning, General Classification, image-classification, Image Classification

Deep convolutional neural networks (CNNs) achieve remarkable performance on
image classification tasks. Recent studies, however, have demonstrated that
generalization abilities are more important than the depth of neural networks
for improving performance on image classification tasks. Herein, a new neural
network called SwGridNet is proposed. A SwGridNet includes many convolutional
processing units which connect mutually as a grid network where many processing
paths exist between input and output. A SwGridNet has high generalization
capability because the multipath architecture has the same effect of ensemble
learning. As described in this paper, details of the SwGridNet network
architecture are presented. Experimentally obtained results presented in this
paper show that SwGridNets respectively achieve test error rates of 2.95% and
15.67% in a CIFAR-10 and CIFAR-100 classification tasks. The results indicate
that the SwGridNet performance approximates that of state-of-the-art deep CNNs.
Supervised Q-walk for Learning Vector Representation of Nodes in Networks
Authors: Naimish Agarwal, G. C. Nandi
Date: 2017-10-03 00:00:00
Tasks: Classification, General Classification, Node Classification, Q-Learning

Automatic feature learning algorithms are at the forefront of modern day
machine learning research. We present a novel algorithm, supervised Q-walk,
which applies Q-learning to generate random walks on graphs such that the walks
prove to be useful for learning node features suitable for tackling with the
node classification problem. We present another novel algorithm, k-hops
neighborhood based confidence values learner, which learns confidence values of
labels for unlabelled nodes in the network without first learning the node
embedding. These confidence values aid in learning an apt reward function for
Q-learning.
  We demonstrate the efficacy of supervised Q-walk approach over existing
state-of-the-art random walk based node embedding learners in solving the
single / multi-label multi-class node classification problem using several real
world datasets.
  Summarising, our approach represents a novel state-of-the-art technique to
learn features, for nodes in networks, tailor-made for dealing with the node
classification problem.
Facial Key Points Detection using Deep Convolutional Neural Network - NaimishNet
Authors: Naimish Agarwal, Artus Krohn-Grimberghe, Ranjana Vyas
Date: 2017-10-03 00:00:00
Tasks: 3D Face Reconstruction, BIG-bench Machine Learning

Facial Key Points (FKPs) Detection is an important and challenging problem in
the fields of computer vision and machine learning. It involves predicting the
co-ordinates of the FKPs, e.g. nose tip, center of eyes, etc, for a given face.
In this paper, we propose a LeNet adapted Deep CNN model - NaimishNet, to
operate on facial key points data and compare our model's performance against
existing state of the art approaches.
A concatenating framework of shortcut convolutional neural networks
Authors: Yujian Li, Ting Zhang, Zhaoying Liu, Haihe Hu
Date: 2017-10-03 00:00:00
Tasks: Classification, Gender Classification, General Classification, image-classification, Image Classification, Object Recognition, Texture Classification

It is well accepted that convolutional neural networks play an important role
in learning excellent features for image classification and recognition.
However, in tradition they only allow adjacent layers connected, limiting
integration of multi-scale information. To further improve their performance,
we present a concatenating framework of shortcut convolutional neural networks.
This framework can concatenate multi-scale features by shortcut connections to
the fully-connected layer that is directly fed to the output layer. We do a
large number of experiments to investigate performance of the shortcut
convolutional neural networks on many benchmark visual datasets for different
tasks. The datasets include AR, FERET, FaceScrub, CelebA for gender
classification, CUReT for texture classification, MNIST for digit recognition,
and CIFAR-10 for object recognition. Experimental results show that the
shortcut convolutional neural networks can achieve better results than the
traditional ones on these tasks, with more stability in different settings of
pooling schemes, activation functions, optimizations, initializations, kernel
numbers and kernel sizes.
Event Identification as a Decision Process with Non-linear Representation of Text
Authors: Yukun Yan, Daqi Zheng, Zhengdong Lu, Sen Song
Date: 2017-10-03 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We propose scale-free Identifier Network(sfIN), a novel model for event
identification in documents. In general, sfIN first encodes a document into
multi-scale memory stacks, then extracts special events via conducting
multi-scale actions, which can be considered as a special type of sequence
labelling. The design of large scale actions makes it more efficient processing
a long document. The whole model is trained with both supervised learning and
reinforcement learning.
Monte Carlo approximation certificates for k-means clustering
Authors: Dustin G. Mixon, Soledad Villar
Date: 2017-10-03 00:00:00
Tasks: Clustering

Efficient algorithms for $k$-means clustering frequently converge to
suboptimal partitions, and given a partition, it is difficult to detect
$k$-means optimality. In this paper, we develop an a posteriori certifier of
approximate optimality for $k$-means clustering. The certifier is a sub-linear
Monte Carlo algorithm based on Peng and Wei's semidefinite relaxation of
$k$-means. In particular, solving the relaxation for small random samples of
the dataset produces a high-confidence lower bound on the $k$-means objective,
and being sub-linear, our algorithm is faster than $k$-means++ when the number
of data points is large. We illustrate the performance of our algorithm with
both numerical experiments and a performance guarantee: If the data points are
drawn independently from any mixture of two Gaussians over $\mathbb{R}^m$ with
identity covariance, then with probability $1-O(1/m)$, our
$\operatorname{poly}(m)$-time algorithm produces a 3-approximation certificate
with 99% confidence.
Parameter Sharing Deep Deterministic Policy Gradient for Cooperative Multi-agent Reinforcement Learning
Authors: Xiangxiang Chu, Hangjun Ye
Date: 2017-10-01 00:00:00
Tasks: Deep Reinforcement Learning, Multi-agent Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Deep reinforcement learning for multi-agent cooperation and competition has
been a hot topic recently. This paper focuses on cooperative multi-agent
problem based on actor-critic methods under local observations settings. Multi
agent deep deterministic policy gradient obtained state of art results for some
multi-agent games, whereas, it cannot scale well with growing amount of agents.
In order to boost scalability, we propose a parameter sharing deterministic
policy gradient method with three variants based on neural networks, including
actor-critic sharing, actor sharing and actor sharing with partially shared
critic. Benchmarks from rllab show that the proposed method has advantages in
learning speed and memory efficiency, well scales with growing amount of
agents, and moreover, it can make full use of reward sharing and
exchangeability if possible.