pwc-archive/papers-with-abstracts huggingface dataset:
Understanding VAEs in Fisher-Shannon Plane
Authors: Huangjie Zheng, Jiangchao Yao, Ya zhang, Ivor W. Tsang, Jia Wang
Date: 2018-07-10 00:00:00
Tasks: Decoder, Representation Learning

In information theory, Fisher information and Shannon information (entropy)
are respectively used to quantify the uncertainty associated with the
distribution modeling and the uncertainty in specifying the outcome of given
variables. These two quantities are complementary and are jointly applied to
information behavior analysis in most cases. The uncertainty property in
information asserts a fundamental trade-off between Fisher information and
Shannon information, which enlightens us the relationship between the encoder
and the decoder in variational auto-encoders (VAEs). In this paper, we
investigate VAEs in the Fisher-Shannon plane and demonstrate that the
representation learning and the log-likelihood estimation are intrinsically
related to these two information quantities. Through extensive qualitative and
quantitative experiments, we provide with a better comprehension of VAEs in
tasks such as high-resolution reconstruction, and representation learning in
the perspective of Fisher information and Shannon information. We further
propose a variant of VAEs, termed as Fisher auto-encoder (FAE), for practical
needs to balance Fisher information and Shannon information. Our experimental
results have demonstrated its promise in improving the reconstruction accuracy
and avoiding the non-informative latent code as occurred in previous works.
Geometric Generalization Based Zero-Shot Learning Dataset Infinite World: Simple Yet Powerful
Authors: Rajesh Chidambaram, Michael Kampffmeyer, Willie Neiswanger, Xiaodan Liang, Thomas Lachmann, Eric Xing
Date: 2018-07-10 00:00:00
Tasks: Zero-Shot Learning

Raven's Progressive Matrices are one of the widely used tests in evaluating
the human test taker's fluid intelligence. Analogously, this paper introduces
geometric generalization based zero-shot learning tests to measure the rapid
learning ability and the internal consistency of deep generative models. Our
empirical research analysis on state-of-the-art generative models discern their
ability to generalize concepts across classes. In the process, we introduce
Infinite World, an evaluable, scalable, multi-modal, light-weight dataset and
Zero-Shot Intelligence Metric ZSI. The proposed tests condenses human-level
spatial and numerical reasoning tasks to its simplistic geometric forms. The
dataset is scalable to a theoretical limit of infinity, in numerical features
of the generated geometric figures, image size and in quantity. We
systematically analyze state-of-the-art model's internal consistency, identify
their bottlenecks and propose a pro-active optimization method for few-shot and
zero-shot learning.
Recurrent Auto-Encoder Model for Large-Scale Industrial Sensor Signal Analysis
Authors: Timothy Wong, Zhiyuan Luo
Date: 2018-07-10 00:00:00
Tasks: Clustering, Decoder, Time Series, Time Series Analysis

Recurrent auto-encoder model summarises sequential data through an encoder
structure into a fixed-length vector and then reconstructs the original
sequence through the decoder structure. The summarised vector can be used to
represent time series features. In this paper, we propose relaxing the
dimensionality of the decoder output so that it performs partial
reconstruction. The fixed-length vector therefore represents features in the
selected dimensions only. In addition, we propose using rolling fixed window
approach to generate training samples from unbounded time series data. The
change of time series features over time can be summarised as a smooth
trajectory path. The fixed-length vectors are further analysed using additional
visualisation and unsupervised clustering techniques. The proposed method can
be applied in large-scale industrial processes for sensors signal analysis
purpose, where clusters of the vector representations can reflect the operating
states of the industrial system.
Deterministic Policy Gradients With General State Transitions
Authors: Qingpeng Cai, Ling Pan, Pingzhong Tang
Date: 2018-07-10 00:00:00
Tasks: continuous-control, Continuous Control, Reinforcement Learning

We study a reinforcement learning setting, where the state transition
function is a convex combination of a stochastic continuous function and a
deterministic function. Such a setting generalizes the widely-studied
stochastic state transition setting, namely the setting of deterministic policy
gradient (DPG).
  We firstly give a simple example to illustrate that the deterministic policy
gradient may be infinite under deterministic state transitions, and introduce a
theoretical technique to prove the existence of the policy gradient in this
generalized setting. Using this technique, we prove that the deterministic
policy gradient indeed exists for a certain set of discount factors, and
further prove two conditions that guarantee the existence for all discount
factors. We then derive a closed form of the policy gradient whenever exists.
Furthermore, to overcome the challenge of high sample complexity of DPG in this
setting, we propose the Generalized Deterministic Policy Gradient (GDPG)
algorithm. The main innovation of the algorithm is a new method of applying
model-based techniques to the model-free algorithm, the deep deterministic
policy gradient algorithm (DDPG). GDPG optimize the long-term rewards of the
model-based augmented MDP subject to a constraint that the long-rewards of the
MDP is less than the original one.
  We finally conduct extensive experiments comparing GDPG with state-of-the-art
methods and the direct model-based extension method of DDPG on several standard
continuous control benchmarks. Results demonstrate that GDPG substantially
outperforms DDPG, the model-based extension of DDPG and other baselines in
terms of both convergence and long-term rewards in most environments.
Deep Learning for Audio Transcription on Low-Resource Datasets
Authors: Veronica Morfi, Dan Stowell
Date: 2018-07-10 00:00:00
Tasks: Deep Learning

In training a deep learning system to perform audio transcription, two
practical problems may arise. Firstly, most datasets are weakly labelled,
having only a list of events present in each recording without any temporal
information for training. Secondly, deep neural networks need a very large
amount of labelled training data to achieve good quality performance, yet in
practice it is difficult to collect enough samples for most classes of
interest. In this paper, we propose factorising the final task of audio
transcription into multiple intermediate tasks in order to improve the training
performance when dealing with this kind of low-resource datasets. We evaluate
three data-efficient approaches of training a stacked convolutional and
recurrent neural network for the intermediate tasks. Our results show that
different methods of training have different advantages and disadvantages.
IAM at CLEF eHealth 2018: Concept Annotation and Coding in French Death Certificates
Authors: Sébastien Cossin, Vianney Jouhet, Fleur Mougin, Gayo Diallo, Frantz Thiessard
Date: 2018-07-10 00:00:00

In this paper, we describe the approach and results for our participation in
the task 1 (multilingual information extraction) of the CLEF eHealth 2018
challenge. We addressed the task of automatically assigning ICD-10 codes to
French death certificates. We used a dictionary-based approach using materials
provided by the task organizers. The terms of the ICD-10 terminology were
normalized, tokenized and stored in a tree data structure. The Levenshtein
distance was used to detect typos. Frequent abbreviations were detected by
manually creating a small set of them. Our system achieved an F-score of 0.786
(precision: 0.794, recall: 0.779). These scores were substantially higher than
the average score of the systems that participated in the challenge.
Enriching Knowledge Bases with Counting Quantifiers
Authors: Paramita Mirza, Simon Razniewski, Fariz Darari, Gerhard Weikum
Date: 2018-07-10 00:00:00

Information extraction traditionally focuses on extracting relations between
identifiable entities, such as <Monterey, locatedIn, California>. Yet, texts
often also contain Counting information, stating that a subject is in a
specific relation with a number of objects, without mentioning the objects
themselves, for example, "California is divided into 58 counties". Such
counting quantifiers can help in a variety of tasks such as query answering or
knowledge base curation, but are neglected by prior work. This paper develops
the first full-fledged system for extracting counting information from text,
called CINEX. We employ distant supervision using fact counts from a knowledge
base as training seeds, and develop novel techniques for dealing with several
challenges: (i) non-maximal training seeds due to the incompleteness of
knowledge bases, (ii) sparse and skewed observations in text sources, and (iii)
high diversity of linguistic patterns. Experiments with five human-evaluated
relations show that CINEX can achieve 60% average precision for extracting
counting information. In a large-scale experiment, we demonstrate the potential
for knowledge base enrichment by applying CINEX to 2,474 frequent relations in
Wikidata. CINEX can assert the existence of 2.5M facts for 110 distinct
relations, which is 28% more than the existing Wikidata facts for these
relations.
Linguistic Characteristics of Censorable Language on SinaWeibo
Authors: Kei Yin Ng, Anna Feldman, Jing Peng, Chris Leberknight
Date: 2018-07-10 00:00:00
Proceeding: COLING 2018 8

This paper investigates censorship from a linguistic perspective. We collect
a corpus of censored and uncensored posts on a number of topics, build a
classifier that predicts censorship decisions independent of discussion topics.
Our investigation reveals that the strongest linguistic indicator of censored
content of our corpus is its readability.
Handling Incomplete Heterogeneous Data using VAEs
Authors: Alfredo Nazabal, Pablo M. Olmos, Zoubin Ghahramani, Isabel Valera
Date: 2018-07-10 00:00:00
Tasks: Imputation

Variational autoencoders (VAEs), as well as other generative models, have been shown to be efficient and accurate for capturing the latent structure of vast amounts of complex high-dimensional data. However, existing VAEs can still not directly handle data that are heterogenous (mixed continuous and discrete) or incomplete (with missing data at random), which is indeed common in real-world applications. In this paper, we propose a general framework to design VAEs suitable for fitting incomplete heterogenous data. The proposed HI-VAE includes likelihood models for real-valued, positive real valued, interval, categorical, ordinal and count data, and allows accurate estimation (and potentially imputation) of missing data. Furthermore, HI-VAE presents competitive predictive performance in supervised tasks, outperforming supervised models when trained on incomplete data.
Towards Head Motion Compensation Using Multi-Scale Convolutional Neural Networks
Authors: Omer Rajput, Nils Gessert, Martin Gromniak, Lars Matthäus, Alexander Schlaefer
Date: 2018-07-10 00:00:00
Tasks: Head Pose Estimation, Motion Compensation, Pose Estimation

Head pose estimation and tracking is useful in variety of medical
applications. With the advent of RGBD cameras like Kinect, it has become
feasible to do markerless tracking by estimating the head pose directly from
the point clouds. One specific medical application is robot assisted
transcranial magnetic stimulation (TMS) where any patient motion is compensated
with the help of a robot. For increased patient comfort, it is important to
track the head without markers. In this regard, we address the head pose
estimation problem using two different approaches. In the first approach, we
build upon the more traditional approach of model based head tracking, where a
head model is morphed according to the particular head to be tracked and the
morphed model is used to track the head in the point cloud streams. In the
second approach, we propose a new multi-scale convolutional neural network
architecture for more accurate pose regression. Additionally, we outline a
systematic data set acquisition strategy using a head phantom mounted on the
robot and ground-truth labels generated using a highly accurate tracking
system.
Convolutional neural network based automatic plaque characterization from intracoronary optical coherence tomography images
Authors: Shenghua He, Jie Zheng, Akiko Maehara, Gary Mintz, Dalin Tang, Mark Anastasio, Hua Li
Date: 2018-07-10 00:00:00
Tasks: Classification, feature selection, General Classification, Translation

Optical coherence tomography (OCT) can provide high-resolution
cross-sectional images for analyzing superficial plaques in coronary arteries.
Commonly, plaque characterization using intra-coronary OCT images is performed
manually by expert observers. This manual analysis is time consuming and its
accuracy heavily relies on the experience of human observers. Traditional
machine learning based methods, such as the least squares support vector
machine and random forest methods, have been recently employed to automatically
characterize plaque regions in OCT images. Several processing steps, including
feature extraction, informative feature selection, and final pixel
classification, are commonly used in these traditional methods. Therefore, the
final classification accuracy can be jeopardized by error or inaccuracy within
each of these steps. In this study, we proposed a convolutional neural network
(CNN) based method to automatically characterize plaques in OCT images. Unlike
traditional methods, our method uses the image as a direct input and performs
classification as a single-step process. The experiments on 269 OCT images
showed that the average prediction accuracy of CNN-based method was 0.866,
which indicated a great promise for clinical translation.
Window Opening Model using Deep Learning Methods
Authors: Romana Markovic, Eva Grintal, Daniel Wölki, Jérôme Frisch, Christoph van Treeck
Date: 2018-07-10 00:00:00
Tasks: Deep Learning, model

Occupant behavior (OB) and in particular window openings need to be
considered in building performance simulation (BPS), in order to realistically
model the indoor climate and energy consumption for heating ventilation and air
conditioning (HVAC). However, the proposed OB window opening models are often
biased towards the over-represented class where windows remained closed. In
addition, they require tuning for each occupant which can not be efficiently
scaled to the increased number of occupants. This paper presents a window
opening model for commercial buildings using deep learning methods. The model
is trained using data from occupants from an office building in Germany. In
total the model is evaluated using almost 20 mio. data points from 3
independent buildings, located in Aachen, Frankfurt and Philadelphia.
Eventually, the results of 3100 core hours of model development are summarized,
which makes this study the largest of its kind in window states modeling.
Additionally, the practical potential of the proposed model was tested by
incorporating it in the Modelica-based thermal building simulation. The
resulting evaluation accuracy and F1 scores on the office buildings ranged
between 86-89 % and 0.53-0.65 respectively. The performance dropped around 15 %
points in case of sparse input data, while the F1 score remained high.
Essential Tensor Learning for Multi-view Spectral Clustering
Authors: Jianlong Wu, Zhouchen Lin, Hongbin Zha
Date: 2018-07-10 00:00:00
Tasks: Clustering

Multi-view clustering attracts much attention recently, which aims to take advantage of multi-view information to improve the performance of clustering. However, most recent work mainly focus on self-representation based subspace clustering, which is of high computation complexity. In this paper, we focus on the Markov chain based spectral clustering method and propose a novel essential tensor learning method to explore the high order correlations for multi-view representation. We first construct a tensor based on multi-view transition probability matrices of the Markov chain. By incorporating the idea from robust principle component analysis, tensor singular value decomposition (t-SVD) based tensor nuclear norm is imposed to preserve the low-rank property of the essential tensor, which can well capture the principle information from multiple views. We also employ the tensor rotation operator for this task to better investigate the relationship among views as well as reduce the computation complexity. The proposed method can be efficiently optimized by the alternating direction method of multipliers~(ADMM). Extensive experiments on six real world datasets corresponding to five different applications show that our method achieves superior performance over other state-of-the-art methods.
Revisiting the Hierarchical Multiscale LSTM
Authors: Ákos Kádár, Marc-Alexandre Côté, Grzegorz Chrupała, Afra Alishahi
Date: 2018-07-10 00:00:00
Proceeding: COLING 2018 8
Tasks: Language Modeling, Language Modelling

Hierarchical Multiscale LSTM (Chung et al., 2016a) is a state-of-the-art
language model that learns interpretable structure from character-level input.
Such models can provide fertile ground for (cognitive) computational
linguistics studies. However, the high complexity of the architecture, training
procedure and implementations might hinder its applicability. We provide a
detailed reproduction and ablation study of the architecture, shedding light on
some of the potential caveats of re-purposing complex deep-learning
architectures. We further show that simplifying certain aspects of the
architecture can in fact improve its performance. We also investigate the
linguistic units (segments) learned by various levels of the model, and argue
that their quality does not correlate with the overall performance of the model
on language modeling.
Efficient Evaluation of the Number of False Alarm Criterion
Authors: Sylvie Le Hégarat-Mascle, Emanuel Aldea, Jennifer Vandoni
Date: 2018-07-10 00:00:00

This paper proposes a method for computing efficiently the significance of a
parametric pattern inside a binary image. On the one hand, a-contrario
strategies avoid the user involvement for tuning detection thresholds, and
allow one to account fairly for different pattern sizes. On the other hand,
a-contrario criteria become intractable when the pattern complexity in terms of
parametrization increases. In this work, we introduce a strategy which relies
on the use of a cumulative space of reduced dimensionality, derived from the
coupling of a classic (Hough) cumulative space with an integral histogram
trick. This space allows us to store partial computations which are required by
the a-contrario criterion, and to evaluate the significance with a lower
computational cost than by following a straightforward approach. The method is
illustrated on synthetic examples on patterns with various parametrizations up
to five dimensions. In order to demonstrate how to apply this generic concept
in a real scenario, we consider a difficult crack detection task in still
images, which has been addressed in the literature with various local and
global detection strategies. We model cracks as bounded segments, detected by
the proposed a-contrario criterion, which allow us to introduce additional
spatial constraints based on their relative alignment. On this application, the
proposed strategy yields state-of the-art results, and underlines its potential
for handling complex pattern detection tasks.
Paired Comparison Sentiment Scores
Authors: Christoph Dalitz, Jens Wilberg, Katrin E. Bednarek
Date: 2018-07-10 00:00:00

The method of paired comparisons is an established method in psychology. In
this article, it is applied to obtain continuous sentiment scores for words
from comparisons made by test persons. We created an initial lexicon with
$n=199$ German words from a two-fold all-pair comparison experiment with ten
different test persons. From the probabilistic models taken into account, the
logistic model showed the best agreement with the results of the comparison
experiment. The initial lexicon can then be used in different ways. One is to
create special purpose sentiment lexica through the addition of arbitrary words
that are compared with some of the initial words by test persons. A
cross-validation experiment suggests that only about 18 two-fold comparisons
are necessary to estimate the score of a new, yet unknown word, provided these
words are selected by a modification of a method by Silverstein & Farrell.
Another application of the initial lexicon is the evaluation of automatically
created corpus-based lexica. By such an evaluation, we compared the
corpus-based lexica SentiWS, SenticNet, and SentiWordNet, of which SenticNet 4
performed best. This technical report is a corrected and extended version of a
presentation made at the ICDM Sentire workshop in 2016.
Two-stage iterative Procrustes match algorithm and its application for VQ-based speaker verification
Authors: Richeng Tan, Jing Li
Date: 2018-07-10 00:00:00
Tasks: Quantization, Speaker Verification

In the past decades, Vector Quantization (VQ) model has been very popular
across different pattern recognition areas, especially for feature-based tasks.
However, the classification or regression performance of VQ-based systems
always confronts the feature mismatch problem, which will heavily affect the
performance of them. In this paper, we propose a two-stage iterative Procrustes
algorithm (TIPM) to address the feature mismatch problem for VQ-based
applications. At the first stage, the algorithm will remove mismatched feature
vector pairs for a pair of input feature sets. Then, the second stage will
collect those correct matched feature pairs that were discarded during the
first stage. To evaluate the effectiveness of the proposed TIPM algorithm,
speaker verification is used as the case study in this paper. The experiments
were conducted on the TIMIT database and the results show that TIPM can improve
VQ-based speaker verification performance clean condition and all noisy
conditions.
Difficulty Controllable Generation of Reading Comprehension Questions
Authors: Yifan Gao, Lidong Bing, Wang Chen, Michael R. Lyu, Irwin King
Date: 2018-07-10 00:00:00
Tasks: Question Generation, Question-Generation, Reading Comprehension, Sentence

We investigate the difficulty levels of questions in reading comprehension datasets such as SQuAD, and propose a new question generation setting, named Difficulty-controllable Question Generation (DQG). Taking as input a sentence in the reading comprehension paragraph and some of its text fragments (i.e., answers) that we want to ask questions about, a DQG method needs to generate questions each of which has a given text fragment as its answer, and meanwhile the generation is under the control of specified difficulty labels---the output questions should satisfy the specified difficulty as much as possible. To solve this task, we propose an end-to-end framework to generate questions of designated difficulty levels by exploring a few important intuitions. For evaluation, we prepared the first dataset of reading comprehension questions with difficulty labels. The results show that the question generated by our framework not only have better quality under the metrics like BLEU, but also comply with the specified difficulty labels.