pwc-archive/papers-with-abstracts huggingface dataset:
Improving Speaker-Independent Lipreading with Domain-Adversarial Training
Authors: Michael Wand, Juergen Schmidhuber
Date: 2017-08-04 00:00:00
Tasks: Lipreading, speech-recognition, Speech Recognition

We present a Lipreading system, i.e. a speech recognition system using only
visual features, which uses domain-adversarial training for speaker
independence. Domain-adversarial training is integrated into the optimization
of a lipreader based on a stack of feedforward and LSTM (Long Short-Term
Memory) recurrent neural networks, yielding an end-to-end trainable system
which only requires a very small number of frames of untranscribed target data
to substantially improve the recognition accuracy on the target speaker. On
pairs of different source and target speakers, we achieve a relative accuracy
improvement of around 40% with only 15 to 20 seconds of untranscribed target
speech data. On multi-speaker training setups, the accuracy improvements are
smaller but still substantial.
Associations among Image Assessments as Cost Functions in Linear Decomposition: MSE, SSIM, and Correlation Coefficient
Authors: Jianji Wang, Nanning Zheng, Badong Chen, Jose C. Principe
Date: 2017-08-04 00:00:00
Tasks: SSIM

The traditional methods of image assessment, such as mean squared error
(MSE), signal-to-noise ratio (SNR), and Peak signal-to-noise ratio (PSNR), are
all based on the absolute error of images. Pearson's inner-product correlation
coefficient (PCC) is also usually used to measure the similarity between
images. Structural similarity (SSIM) index is another important measurement
which has been shown to be more effective in the human vision system (HVS).
Although there are many essential differences among these image assessments,
some important associations among them as cost functions in linear
decomposition are discussed in this paper. Firstly, the selected bases from a
basis set for a target vector are the same in the linear decomposition schemes
with different cost functions MSE, SSIM, and PCC. Moreover, for a target
vector, the ratio of the corresponding affine parameters in the MSE-based
linear decomposition scheme and the SSIM-based scheme is a constant, which is
just the value of PCC between the target vector and its estimated vector.
Can string kernels pass the test of time in Native Language Identification?
Authors: Radu Tudor Ionescu, Marius Popescu
Date: 2017-07-26 00:00:00
Proceeding: WS 2017 9
Tasks: Language Identification, Native Language Identification

We describe a machine learning approach for the 2017 shared task on Native
Language Identification (NLI). The proposed approach combines several kernels
using multiple kernel learning. While most of our kernels are based on
character p-grams (also known as n-grams) extracted from essays or speech
transcripts, we also use a kernel based on i-vectors, a low-dimensional
representation of audio recordings, provided by the shared task organizers. For
the learning stage, we choose Kernel Discriminant Analysis (KDA) over Kernel
Ridge Regression (KRR), because the former classifier obtains better results
than the latter one on the development set. In our previous work, we have used
a similar machine learning approach to achieve state-of-the-art NLI results.
The goal of this paper is to demonstrate that our shallow and simple approach
based on string kernels (with minor improvements) can pass the test of time and
reach state-of-the-art performance in the 2017 NLI shared task, despite the
recent advances in natural language processing. We participated in all three
tracks, in which the competitors were allowed to use only the essays (essay
track), only the speech transcripts (speech track), or both (fusion track).
Using only the data provided by the organizers for training our models, we have
reached a macro F1 score of 86.95% in the closed essay track, a macro F1 score
of 87.55% in the closed speech track, and a macro F1 score of 93.19% in the
closed fusion track. With these scores, our team (UnibucKernel) ranked in the
first group of teams in all three tracks, while attaining the best scores in
the speech and the fusion tracks.
Learning Model Reparametrizations: Implicit Variational Inference by Fitting MCMC distributions
Authors: Michalis K. Titsias
Date: 2017-08-04 00:00:00
Tasks: Variational Inference

We introduce a new algorithm for approximate inference that combines
reparametrization, Markov chain Monte Carlo and variational methods. We
construct a very flexible implicit variational distribution synthesized by an
arbitrary Markov chain Monte Carlo operation and a deterministic transformation
that can be optimized using the reparametrization trick. Unlike current methods
for implicit variational inference, our method avoids the computation of log
density ratios and therefore it is easily applicable to arbitrary continuous
and differentiable models. We demonstrate the proposed algorithm for fitting
banana-shaped distributions and for training variational autoencoders.
A Latent Variable Model for Two-Dimensional Canonical Correlation Analysis and its Variational Inference
Authors: Mehran Safayani, Saeid Momenzadeh
Date: 2017-08-04 00:00:00
Tasks: Dimensionality Reduction, Variational Inference

Describing the dimension reduction (DR) techniques by means of probabilistic
models has recently been given special attention. Probabilistic models, in
addition to a better interpretability of the DR methods, provide a framework
for further extensions of such algorithms. One of the new approaches to the
probabilistic DR methods is to preserving the internal structure of data. It is
meant that it is not necessary that the data first be converted from the matrix
or tensor format to the vector format in the process of dimensionality
reduction. In this paper, a latent variable model for matrix-variate data for
canonical correlation analysis (CCA) is proposed. Since in general there is not
any analytical maximum likelihood solution for this model, we present two
approaches for learning the parameters. The proposed methods are evaluated
using the synthetic data in terms of convergence and quality of mappings. Also,
real data set is employed for assessing the proposed methods with several
probabilistic and none-probabilistic CCA based approaches. The results confirm
the superiority of the proposed methods with respect to the competing
algorithms. Moreover, this model can be considered as a framework for further
extensions.
Semi-supervised Learning based on Distributionally Robust Optimization
Authors: Jose Blanchet, Yang Kang
Date: 2017-02-28 00:00:00
Tasks: Dimensionality Reduction

We propose a novel method for semi-supervised learning (SSL) based on
data-driven distributionally robust optimization (DRO) using optimal transport
metrics. Our proposed method enhances generalization error by using the
unlabeled data to restrict the support of the worst case distribution in our
DRO formulation. We enable the implementation of our DRO formulation by
proposing a stochastic gradient descent algorithm which allows to easily
implement the training procedure. We demonstrate that our Semi-supervised DRO
method is able to improve the generalization error over natural supervised
procedures and state-of-the-art SSL estimators. Finally, we include a
discussion on the large sample behavior of the optimal uncertainty region in
the DRO formulation. Our discussion exposes important aspects such as the role
of dimension reduction in SSL.
A Bayesian Approach to Policy Recognition and State Representation Learning
Authors: Adrian Šošić, Abdelhak M. Zoubir, Heinz Koeppl
Date: 2016-05-04 00:00:00
Tasks: Representation Learning

Learning from demonstration (LfD) is the process of building behavioral
models of a task from demonstrations provided by an expert. These models can be
used e.g. for system control by generalizing the expert demonstrations to
previously unencountered situations. Most LfD methods, however, make strong
assumptions about the expert behavior, e.g. they assume the existence of a
deterministic optimal ground truth policy or require direct monitoring of the
expert's controls, which limits their practical use as part of a general system
identification framework. In this work, we consider the LfD problem in a more
general setting where we allow for arbitrary stochastic expert policies,
without reasoning about the optimality of the demonstrations. Following a
Bayesian methodology, we model the full posterior distribution of possible
expert controllers that explain the provided demonstration data. Moreover, we
show that our methodology can be applied in a nonparametric context to infer
the complexity of the state representation used by the expert, and to learn
task-appropriate partitionings of the system state space.
Multi-modal Factorized Bilinear Pooling with Co-Attention Learning for Visual Question Answering
Authors: Zhou Yu, Jun Yu, Jianping Fan, DaCheng Tao
Date: 2017-08-04 00:00:00
Proceeding: ICCV 2017 10
Tasks: Question Answering, Visual Question Answering, Visual Question Answering (VQA)

Visual question answering (VQA) is challenging because it requires a
simultaneous understanding of both the visual content of images and the textual
content of questions. The approaches used to represent the images and questions
in a fine-grained manner and questions and to fuse these multi-modal features
play key roles in performance. Bilinear pooling based models have been shown to
outperform traditional linear models for VQA, but their high-dimensional
representations and high computational complexity may seriously limit their
applicability in practice. For multi-modal feature fusion, here we develop a
Multi-modal Factorized Bilinear (MFB) pooling approach to efficiently and
effectively combine multi-modal features, which results in superior performance
for VQA compared with other bilinear pooling approaches. For fine-grained image
and question representation, we develop a co-attention mechanism using an
end-to-end deep network architecture to jointly learn both the image and
question attentions. Combining the proposed MFB approach with co-attention
learning in a new network architecture provides a unified model for VQA. Our
experimental results demonstrate that the single MFB with co-attention model
achieves new state-of-the-art performance on the real-world VQA dataset. Code
available at https://github.com/yuzcccc/mfb.
Towards Instance Segmentation with Object Priority: Prominent Object Detection and Recognition
Authors: Hamed R. -Tavakoli, Jorma Laaksonen
Date: 2017-04-24 00:00:00
Tasks: Instance Segmentation, Object, object-detection, Object Detection, Object Recognition, Saliency Detection, Semantic Segmentation

This manuscript introduces the problem of prominent object detection and
recognition inspired by the fact that human seems to priorities perception of
scene elements. The problem deals with finding the most important region of
interest, segmenting the relevant item/object in that area, and assigning it an
object class label. In other words, we are solving the three problems of
saliency modeling, saliency detection, and object recognition under one
umbrella. The motivation behind such a problem formulation is (1) the benefits
to the knowledge representation-based vision pipelines, and (2) the potential
improvements in emulating bio-inspired vision systems by solving these three
problems together. We are foreseeing extending this problem formulation to
fully semantically segmented scenes with instance object priority for
high-level inferences in various applications including assistive vision. Along
with a new problem definition, we also propose a method to achieve such a task.
The proposed model predicts the most important area in the image, segments the
associated objects, and labels them. The proposed problem and method are
evaluated against human fixations, annotated segmentation masks, and object
class categories. We define a chance level for each of the evaluation criterion
to compare the proposed algorithm with. Despite the good performance of the
proposed baseline, the overall evaluations indicate that the problem of
prominent object detection and recognition is a challenging task that is still
worth investigating further.
Massively Multilingual Neural Grapheme-to-Phoneme Conversion
Authors: Ben Peters, Jon Dehdari, Josef van Genabith
Date: 2017-08-04 00:00:00
Proceeding: WS 2017 9
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Decoder, Grapheme-to-Phoneme Conversion, speech-recognition, Speech Recognition, text-to-speech, Text to Speech

Grapheme-to-phoneme conversion (g2p) is necessary for text-to-speech and
automatic speech recognition systems. Most g2p systems are monolingual: they
require language-specific data or handcrafting of rules. Such systems are
difficult to extend to low resource languages, for which data and handcrafted
rules are not available. As an alternative, we present a neural
sequence-to-sequence approach to g2p which is trained on
spelling--pronunciation pairs in hundreds of languages. The system shares a
single encoder and decoder across all languages, allowing it to utilize the
intrinsic similarities between different writing systems. We show an 11%
improvement in phoneme error rate over an approach based on adapting
high-resource monolingual g2p models to low-resource languages. Our model is
also much more compact relative to previous approaches.
Paying Attention to Descriptions Generated by Image Captioning Models
Authors: Hamed R. -Tavakoli, Rakshith Shetty, Ali Borji, Jorma Laaksonen
Date: 2017-04-24 00:00:00
Proceeding: ICCV 2017 10
Tasks: Image Captioning

To bridge the gap between humans and machines in image understanding and
describing, we need further insight into how people describe a perceived scene.
In this paper, we study the agreement between bottom-up saliency-based visual
attention and object referrals in scene description constructs. We investigate
the properties of human-written descriptions and machine-generated ones. We
then propose a saliency-boosted image captioning model in order to investigate
benefits from low-level cues in language models. We learn that (1) humans
mention more salient objects earlier than less salient ones in their
descriptions, (2) the better a captioning model performs, the better attention
agreement it has with human descriptions, (3) the proposed saliency-boosted
model, compared to its baseline form, does not improve significantly on the MS
COCO database, indicating explicit bottom-up boosting does not help when the
task is well learnt and tuned on a data, (4) a better generalization is,
however, observed for the saliency-boosted model on unseen data.
View Adaptive Recurrent Neural Networks for High Performance Human Action Recognition from Skeleton Data
Authors: Pengfei Zhang, Cuiling Lan, Junliang Xing, Wen-Jun Zeng, Jianru Xue, Nanning Zheng
Date: 2017-03-24 00:00:00
Proceeding: ICCV 2017 10
Tasks: Action Recognition, Skeleton Based Action Recognition, Temporal Action Localization

Skeleton-based human action recognition has recently attracted increasing
attention due to the popularity of 3D skeleton data. One main challenge lies in
the large view variations in captured human actions. We propose a novel view
adaptation scheme to automatically regulate observation viewpoints during the
occurrence of an action. Rather than re-positioning the skeletons based on a
human defined prior criterion, we design a view adaptive recurrent neural
network (RNN) with LSTM architecture, which enables the network itself to adapt
to the most suitable observation viewpoints from end to end. Extensive
experiment analyses show that the proposed view adaptive RNN model strives to
(1) transform the skeletons of various views to much more consistent viewpoints
and (2) maintain the continuity of the action rather than transforming every
frame to the same position with the same body orientation. Our model achieves
significant improvement over the state-of-the-art approaches on three benchmark
datasets.
Comparison of Distances for Supervised Segmentation of White Matter Tractography
Authors: Emanuele Olivetti, Giulia Bertò, Pietro Gori, Nusrat Sharmin, Paolo Avesani
Date: 2017-08-04 00:00:00
Tasks: Diffusion  MRI, Segmentation

Tractograms are mathematical representations of the main paths of axons
within the white matter of the brain, from diffusion MRI data. Such
representations are in the form of polylines, called streamlines, and one
streamline approximates the common path of tens of thousands of axons. The
analysis of tractograms is a task of interest in multiple fields, like
neurosurgery and neurology. A basic building block of many pipelines of
analysis is the definition of a distance function between streamlines. Multiple
distance functions have been proposed in the literature, and different authors
use different distances, usually without a specific reason other than invoking
the "common practice". To this end, in this work we want to test such common
practices, in order to obtain factual reasons for choosing one distance over
another. For these reasons, in this work we compare many streamline distance
functions available in the literature. We focus on the common task of automatic
bundle segmentation and we adopt the recent approach of supervised segmentation
from expert-based examples. Using the HCP dataset, we compare several distances
obtaining guidelines on the choice of which distance function one should use
for supervised bundle segmentation.
Online Visual Multi-Object Tracking via Labeled Random Finite Set Filtering
Authors: Du Yong Kim, Ba-Ngu Vo, Ba-Tuong Vo
Date: 2016-11-18 00:00:00
Tasks: Management, Multi-Object Tracking, Object, Object Tracking, Occlusion Handling, State Estimation

This paper proposes an online visual multi-object tracking algorithm using a
top-down Bayesian formulation that seamlessly integrates state estimation,
track management, clutter rejection, occlusion and mis-detection handling into
a single recursion. This is achieved by modeling the multi-object state as
labeled random finite set and using the Bayes recursion to propagate the
multi-object filtering density forward in time. The proposed filter updates
tracks with detections but switches to image data when mis-detection occurs,
thereby exploiting the efficiency of detection data and the accuracy of image
data. Furthermore the labeled random finite set framework enables the
incorporation of prior knowledge that mis-detections of long tracks which occur
in the middle of the scene are likely to be due to occlusions. Such prior
knowledge can be exploited to improve occlusion handling, especially long
occlusions that can lead to premature track termination in on-line multi-object
tracking. Tracking performance are compared to state-of-the-art algorithms on
well-known benchmark video datasets.
On the Selective and Invariant Representation of DCNN for High-Resolution Remote Sensing Image Recognition
Authors: Jie Chen, Chao Yuan, Min Deng, Chao Tao, Jian Peng, Haifeng Li
Date: 2017-08-04 00:00:00
Tasks: General Classification, Scene Recognition

Human vision possesses strong invariance in image recognition. The cognitive
capability of deep convolutional neural network (DCNN) is close to the human
visual level because of hierarchical coding directly from raw image. Owing to
its superiority in feature representation, DCNN has exhibited remarkable
performance in scene recognition of high-resolution remote sensing (HRRS)
images and classification of hyper-spectral remote sensing images. In-depth
investigation is still essential for understanding why DCNN can accurately
identify diverse ground objects via its effective feature representation. Thus,
we train the deep neural network called AlexNet on our large scale remote
sensing image recognition benchmark. At the neuron level in each convolution
layer, we analyze the general properties of DCNN in HRRS image recognition by
use of a framework of visual stimulation-characteristic response combined with
feature coding-classification decoding. Specifically, we use histogram
statistics, representational dissimilarity matrix, and class activation mapping
to observe the selective and invariance representations of DCNN in HRRS image
recognition. We argue that selective and invariance representations play
important roles in remote sensing images tasks, such as classification,
detection, and segment. Also selective and invariance representations are
significant to design new DCNN liked models for analyzing and understanding
remote sensing images.
Adaptive Plant Propagation Algorithm for Solving Economic Load Dispatch Problem
Authors: Sayan Nag
Date: 2017-08-04 00:00:00
Tasks: Metaheuristic Optimization

Optimization problems in design engineering are complex by nature, often
because of the involvement of critical objective functions accompanied by a
number of rigid constraints associated with the products involved. One such
problem is Economic Load Dispatch (ED) problem which focuses on the
optimization of the fuel cost while satisfying some system constraints.
Classical optimization algorithms are not sufficient and also inefficient for
the ED problem involving highly nonlinear, and non-convex functions both in the
objective and in the constraints. This led to the development of metaheuristic
optimization approaches which can solve the ED problem almost efficiently. This
paper presents a novel robust plant intelligence based Adaptive Plant
Propagation Algorithm (APPA) which is used to solve the classical ED problem.
The application of the proposed method to the 3-generator and 6-generator
systems shows the efficiency and robustness of the proposed algorithm. A
comparative study with another state-of-the-art algorithm (APSO) demonstrates
the quality of the solution achieved by the proposed method along with the
convergence characteristics of the proposed approach.
The All-Paths and Cycles Graph Kernel
Authors: P. -L. Giscard, R. C. Wilson
Date: 2017-08-04 00:00:00
Tasks: All

With the recent rise in the amount of structured data available, there has
been considerable interest in methods for machine learning with graphs. Many of
these approaches have been kernel methods, which focus on measuring the
similarity between graphs. These generally involving measuring the similarity
of structural elements such as walks or paths. Borgwardt and Kriegel proposed
the all-paths kernel but emphasized that it is NP-hard to compute and
infeasible in practice, favouring instead the shortest-path kernel. In this
paper, we introduce a new algorithm for computing the all-paths kernel which is
very efficient and enrich it further by including the simple cycles as well. We
demonstrate how it is feasible even on large datasets to compute all the paths
and simple cycles up to a moderate length. We show how to count labelled
paths/simple cycles between vertices of a graph and evaluate a labelled path
and simple cycles kernel. Extensive evaluations on a variety of graph datasets
demonstrate that the all-paths and cycles kernel has superior performance to
the shortest-path kernel and state-of-the-art performance overall.