pwc-archive/papers-with-abstracts huggingface dataset:
Bias and Generalization in Deep Generative Models: An Empirical Study
Authors: Shengjia Zhao, Hongyu Ren, Arianna Yuan, Jiaming Song, Noah Goodman, Stefano Ermon
Date: 2018-11-08 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Density Estimation, Inductive Bias

In high dimensional settings, density estimation algorithms rely crucially on
their inductive bias. Despite recent empirical success, the inductive bias of
deep generative models is not well understood. In this paper we propose a
framework to systematically investigate bias and generalization in deep
generative models of images. Inspired by experimental methods from cognitive
psychology, we probe each learning algorithm with carefully designed training
datasets to characterize when and how existing models generate novel attributes
and their combinations. We identify similarities to human psychology and verify
that these patterns are consistent across commonly used models and
architectures.
Gaussian-Constrained training for speaker verification
Authors: Lantian Li, Zhiyuan Tang, Ying Shi, Dong Wang
Date: 2018-11-08 00:00:00
Tasks: Speaker Verification

Neural models, in particular the d-vector and x-vector architectures, have
produced state-of-the-art performance on many speaker verification tasks.
However, two potential problems of these neural models deserve more
investigation. Firstly, both models suffer from `information leak', which means
that some parameters participating in model training will be discarded during
inference, i.e, the layers that are used as the classifier. Secondly, these
models do not regulate the distribution of the derived speaker vectors. This
`unconstrained distribution' may degrade the performance of the subsequent
scoring component, e.g., PLDA. This paper proposes a Gaussian-constrained
training approach that (1) discards the parametric classifier, and (2) enforces
the distribution of the derived speaker vectors to be Gaussian. Our experiments
on the VoxCeleb and SITW databases demonstrated that this new training approach
produced more representative and regular speaker embeddings, leading to
consistent performance improvement.
Phonetic-attention scoring for deep speaker features in speaker verification
Authors: Lantian Li, Zhiyuan Tang, Ying Shi, Dong Wang
Date: 2018-11-08 00:00:00
Tasks: Machine Translation, Speaker Verification, Translation

Recent studies have shown that frame-level deep speaker features can be
derived from a deep neural network with the training target set to discriminate
speakers by a short speech segment. By pooling the frame-level features,
utterance-level representations, called d-vectors, can be derived and used in
the automatic speaker verification (ASV) task. This simple average pooling,
however, is inherently sensitive to the phonetic content of the utterance. An
interesting idea borrowed from machine translation is the attention-based
mechanism, where the contribution of an input word to the translation at a
particular time is weighted by an attention score. This score reflects the
relevance of the input word and the present translation. We can use the same
idea to align utterances with different phonetic contents. This paper proposes
a phonetic-attention scoring approach for d-vector systems. By this approach,
an attention score is computed for each frame pair. This score reflects the
similarity of the two frames in phonetic content, and is used to weigh the
contribution of this frame pair in the utterance-based scoring. This new
scoring approach emphasizes the frame pairs with similar phonetic contents,
which essentially provides a soft alignment for utterances with any phonetic
contents. Experimental results show that compared with the naive average
pooling, this phonetic-attention scoring approach can deliver consistent
performance improvement in ASV tasks of both text-dependent and
text-independent.
Similarity Learning with Higher-Order Graph Convolutions for Brain Network Analysis
Authors: Guixiang Ma, Nesreen K. Ahmed, Ted Willke, Dipanjan Sengupta, Michael W. Cole, Nicholas B. Turk-Browne, Philip S. Yu
Date: 2018-11-02 00:00:00
Tasks: Graph Similarity

Learning a similarity metric has gained much attention recently, where the
goal is to learn a function that maps input patterns to a target space while
preserving the semantic distance in the input space. While most related work
focused on images, we focus instead on learning a similarity metric for
neuroimages, such as fMRI and DTI images. We propose an end-to-end similarity
learning framework called Higher-order Siamese GCN for multi-subject fMRI data
analysis. The proposed framework learns the brain network representations via a
supervised metric-based approach with siamese neural networks using two graph
convolutional networks as the twin networks. Our proposed framework performs
higher-order convolutions by incorporating higher-order proximity in graph
convolutional networks to characterize and learn the community structure in
brain connectivity networks. To the best of our knowledge, this is the first
community-preserving similarity learning framework for multi-subject brain
network analysis. Experimental results on four real fMRI datasets demonstrate
the potential use cases of the proposed framework for multi-subject brain
analysis in health and neuropsychiatric disorders. Our proposed approach
achieves an average AUC gain of 75% compared to PCA, an average AUC gain of
65.5% compared to Spectral Embedding, and an average AUC gain of 24.3% compared
to S-GCN across the four datasets, indicating promising application in clinical
investigation and brain disease diagnosis.
Model Selection for Generalized Zero-shot Learning
Authors: Hongguang Zhang, Piotr Koniusz
Date: 2018-11-08 00:00:00
Tasks: Generalized Zero-Shot Learning, Generative Adversarial Network, model, Model Selection, Zero-Shot Learning

In the problem of generalized zero-shot learning, the datapoints from unknown
classes are not available during training. The main challenge for generalized
zero-shot learning is the unbalanced data distribution which makes it hard for
the classifier to distinguish if a given testing sample comes from a seen or
unseen class. However, using Generative Adversarial Network (GAN) to generate
auxiliary datapoints by the semantic embeddings of unseen classes alleviates
the above problem. Current approaches combine the auxiliary datapoints and
original training data to train the generalized zero-shot learning model and
obtain state-of-the-art results. Inspired by such models, we propose to feed
the generated data via a model selection mechanism. Specifically, we leverage
two sources of datapoints (observed and auxiliary) to train some classifier to
recognize which test datapoints come from seen and which from unseen classes.
This way, generalized zero-shot learning can be divided into two disjoint
classification tasks, thus reducing the negative influence of the unbalanced
data distribution. Our evaluations on four publicly available datasets for
generalized zero-shot learning show that our model obtains state-of-the-art
results.
ABC: Efficient Selection of Machine Learning Configuration on Large Dataset
Authors: Silu Huang, Chi Wang, Bolin Ding, Surajit Chaudhuri
Date: 2018-11-08 00:00:00
Tasks: BIG-bench Machine Learning

A machine learning configuration refers to a combination of preprocessor,
learner, and hyperparameters. Given a set of configurations and a large dataset
randomly split into training and testing set, we study how to efficiently
select the best configuration with approximately the highest testing accuracy
when trained from the training set. To guarantee small accuracy loss, we
develop a solution using confidence interval (CI)-based progressive sampling
and pruning strategy. Compared to using full data to find the exact best
configuration, our solution achieves more than two orders of magnitude speedup,
while the returned top configuration has identical or close test accuracy.
Evaluating the Complementarity of Taxonomic Relation Extraction Methods Across Different Languages
Authors: Roger Granada, Renata Vieira, Cassia Trojahn, Nathalie Aussenac-Gilles
Date: 2018-11-08 00:00:00
Tasks: Relation, Relation Extraction

Modern information systems are changing the idea of "data processing" to the
idea of "concept processing", meaning that instead of processing words, such
systems process semantic concepts which carry meaning and share contexts with
other concepts. Ontology is commonly used as a structure that captures the
knowledge about a certain area via providing concepts and relations between
them. Traditionally, concept hierarchies have been built manually by knowledge
engineers or domain experts. However, the manual construction of a concept
hierarchy suffers from several limitations such as its coverage and the
enormous costs of its extension and maintenance. Ontology learning, usually
referred to the (semi-)automatic support in ontology development, is usually
divided into steps, going from concepts identification, passing through
hierarchy and non-hierarchy relations detection and, seldom, axiom extraction.
It is reasonable to say that among these steps the current frontier is in the
establishment of concept hierarchies, since this is the backbone of ontologies
and, therefore, a good concept hierarchy is already a valuable resource for
many ontology applications. The automatic construction of concept hierarchies
from texts is a complex task and much work have been proposing approaches to
better extract relations between concepts. These different proposals have never
been contrasted against each other on the same set of data and across different
languages. Such comparison is important to see whether they are complementary
or incremental. Also, we can see whether they present different tendencies
towards recall and precision. This paper evaluates these different methods on
the basis of hierarchy metrics such as density and depth, and evaluation
metrics such as Recall and Precision. Results shed light over the comprehensive
set of methods according to the literature in the area.
Short Term Load Forecasting Using Deep Neural Networks
Authors: Faisal Mohammad, Ki Boem Lee, Young-Chon Kim
Date: 2018-11-08 00:00:00
Tasks: Load Forecasting

Electricity load forecasting plays an important role in the energy planning
such as generation and distribution. However, the nonlinearity and dynamic
uncertainties in the smart grid environment are the main obstacles in
forecasting accuracy. Deep Neural Network (DNN) is a set of intelligent
computational algorithms that provide a comprehensive solution for modelling a
complicated nonlinear relationship between the input and output through
multiple hidden layers. In this paper, we propose DNN based electricity load
forecasting system to manage the energy consumption in an efficient manner. We
investigate the applicability of two deep neural network architectures
Feed-forward Deep Neural Network (Deep-FNN) and Recurrent Deep Neural Network
(Deep-RNN) to the New York Independent System Operator (NYISO) electricity load
forecasting task. We test our algorithm with various activation functions such
as Sigmoid, Hyperbolic Tangent (tanh) and Rectifier Linear Unit (ReLU). The
performance measurement of two network architectures is compared in terms of
Mean Absolute Percentage Error (MAPE) metric.
High Speed Tracking With A Fourier Domain Kernelized Correlation Filter
Authors: Mingyang Guan, Zhengguo Li, Renjie He, Changyun Wen
Date: 2018-11-08 00:00:00
Tasks: Attribute, Vocal Bursts Intensity Prediction

It is challenging to design a high speed tracking approach using l1-norm due
to its non-differentiability. In this paper, a new kernelized correlation
filter is introduced by leveraging the sparsity attribute of l1-norm based
regularization to design a high speed tracker. We combine the l1-norm and
l2-norm based regularizations in one Huber-type loss function, and then
formulate an optimization problem in the Fourier Domain for fast computation,
which enables the tracker to adaptively ignore the noisy features produced from
occlusion and illumination variation, while keep the advantages of l2-norm
based regression. This is achieved due to the attribute of Convolution Theorem
that the correlation in spatial domain corresponds to an element-wise product
in the Fourier domain, resulting in that the l1-norm optimization problem could
be decomposed into multiple sub-optimization spaces in the Fourier domain. But
the optimized variables in the Fourier domain are complex, which makes using
the l1-norm impossible if the real and imaginary parts of the variables cannot
be separated. However, our proposed optimization problem is formulated in such
a way that their real part and imaginary parts are indeed well separated. As
such, the proposed optimization problem can be solved efficiently to obtain
their optimal values independently with closed-form solutions. Extensive
experiments on two large benchmark datasets demonstrate that the proposed
tracking algorithm significantly improves the tracking accuracy of the original
kernelized correlation filter (KCF) while with little sacrifice on tracking
speed. Moreover, it outperforms the state-of-the-art approaches in terms of
accuracy, efficiency, and robustness.
Knowledge Transfer via Distillation of Activation Boundaries Formed by Hidden Neurons
Authors: Byeongho Heo, Minsik Lee, Sangdoo Yun, Jin Young Choi
Date: 2018-11-08 00:00:00
Tasks: Transfer Learning

An activation boundary for a neuron refers to a separating hyperplane that
determines whether the neuron is activated or deactivated. It has been long
considered in neural networks that the activations of neurons, rather than
their exact output values, play the most important role in forming
classification friendly partitions of the hidden feature space. However, as far
as we know, this aspect of neural networks has not been considered in the
literature of knowledge transfer. In this paper, we propose a knowledge
transfer method via distillation of activation boundaries formed by hidden
neurons. For the distillation, we propose an activation transfer loss that has
the minimum value when the boundaries generated by the student coincide with
those by the teacher. Since the activation transfer loss is not differentiable,
we design a piecewise differentiable loss approximating the activation transfer
loss. By the proposed method, the student learns a separating boundary between
activation region and deactivation region formed by each neuron in the teacher.
Through the experiments in various aspects of knowledge transfer, it is
verified that the proposed method outperforms the current state-of-the-art.
Advanced machine learning informatics modeling using clinical and radiological imaging metrics for characterizing breast tumor characteristics with the OncotypeDX gene array
Authors: Michael A. Jacobs, Christopher Umbricht, Vishwa Parekh, Riham El Khouli, Leslie Cope, Katarzyna J. Macura, Susan Harvey, Antonio C. Wolff
Date: 2018-11-08 00:00:00
Tasks: BIG-bench Machine Learning, General Classification, Sensitivity, Specificity

Purpose-Optimal use of established and imaging methods, such as
multiparametric magnetic resonance imaging(mpMRI) can simultaneously identify
key functional parameters and provide unique imaging phenotypes of breast
cancer. Therefore, we have developed and implemented a new machine-learning
informatic system that integrates clinical variables, derived from imaging and
clinical health records, to compare with the 21-gene array assay, OncotypeDX.
Materials and methods-We tested our informatics modeling in a subset of
patients (n=81) who had ER+ disease and underwent OncotypeDX gene expression
and breast mpMRI testing. The machine-learning informatic method is termed
Integrated Radiomic Informatic System-IRIS was applied to the mpMRI, clinical
and pathologic descriptors, as well as a gene array analysis. The IRIS method
using an advanced graph theoretic model and quantitative metrics. Summary
statistics (mean and standard deviations) for the quantitative imaging
parameters were obtained. Sensitivity and specificity and Area Under the Curve
were calculated for the classification of the patients. Results-The OncotypeDX
classification by IRIS model had sensitivity of 95% and specificity of 89% with
AUC of 0.92. The breast lesion size was larger for the high-risk groups and
lower for both low risk and intermediate risk groups. There were significant
differences in PK-DCE and ADC map values in each group. The ADC map values for
high- and intermediate-risk groups were significantly lower than the low-risk
group. Conclusion-These initial studies provide deeper understandings of
imaging features and molecular gene array OncotypeDX score. This insight
provides the foundation to relate these imaging features to the assessment of
treatment response for improved personalized medicine.
RGB-D SLAM in Dynamic Environments Using Point Correlations
Authors: Weichen Dai, Yu Zhang, Ping Li, Zheng Fang, Sebastian Scherer
Date: 2018-11-08 00:00:00
Tasks: Motion Estimation, Simultaneous Localization and Mapping

In this paper, a simultaneous localization and mapping (SLAM) method that eliminates the influence of moving objects in dynamic environments is proposed. This method utilizes the correlation between map points to separate points that are part of the static scene and points that are part of different moving objects into different groups. A sparse graph is first created using Delaunay triangulation from all map points. In this graph, the vertices represent map points, and each edge represents the correlation between adjacent points. If the relative position between two points remains consistent over time, there is correlation between them, and they are considered to be moving together rigidly. If not, they are considered to have no correlation and to be in separate groups. After the edges between the uncorrelated points are removed during point-correlation optimization, the remaining graph separates the map points of the moving objects from the map points of the static scene. The largest group is assumed to be the group of reliable static map points. Finally, motion estimation is performed using only these points. The proposed method was implemented for RGB-D sensors, evaluated with a public RGB-D benchmark, and tested in several additional challenging environments. The experimental results demonstrate that robust and accurate performance can be achieved by the proposed SLAM method in both slightly and highly dynamic environments. Compared with other state-of-the-art methods, the proposed method can provide competitive accuracy with good real-time performance.
Facial Landmark Detection for Manga Images
Authors: Marco Stricker, Olivier Augereau, Koichi Kise, Motoi Iwata
Date: 2018-11-08 00:00:00
Tasks: Facial Landmark Detection

The topic of facial landmark detection has been widely covered for pictures
of human faces, but it is still a challenge for drawings. Indeed, the
proportions and symmetry of standard human faces are not always used for comics
or mangas. The personal style of the author, the limitation of colors, etc.
makes the landmark detection on faces in drawings a difficult task. Detecting
the landmarks on manga images will be useful to provide new services for easily
editing the character faces, estimating the character emotions, or generating
automatically some animations such as lip or eye movements.
  This paper contains two main contributions: 1) a new landmark annotation
model for manga faces, and 2) a deep learning approach to detect these
landmarks. We use the "Deep Alignment Network", a multi stage architecture
where the first stage makes an initial estimation which gets refined in further
stages. The first results show that the proposed method succeed to accurately
find the landmarks in more than 80% of the cases.
Deep Semantic Instance Segmentation of Tree-like Structures Using Synthetic Data
Authors: Kerry Halupka, Rahil Garnavi, Stephen Moore
Date: 2018-11-08 00:00:00
Tasks: Instance Segmentation, Segmentation, Semantic Segmentation

Tree-like structures, such as blood vessels, often express complexity at very
fine scales, requiring high-resolution grids to adequately describe their
shape. Such sparse morphology can alternately be represented by locations of
centreline points, but learning from this type of data with deep learning is
challenging due to it being unordered, and permutation invariant. In this work,
we propose a deep neural network that directly consumes unordered points along
the centreline of a branching structure, to identify the topology of the
represented structure in a single-shot. Key to our approach is the use of a
novel multi-task loss function, enabling instance segmentation of arbitrarily
complex branching structures. We train the network solely using synthetically
generated data, utilizing domain randomization to facilitate the transfer to
real 2D and 3D data. Results show that our network can reliably extract
meaningful information about branch locations, bifurcations and endpoints, and
sets a new benchmark for semantic instance segmentation in branching
structures.
Robustness of Conditional GANs to Noisy Labels
Authors: Kiran Koshy Thekumparampil, Ashish Khetan, Zinan Lin, Sewoong Oh
Date: 2018-11-08 00:00:00
Proceeding: NeurIPS 2018 12

We study the problem of learning conditional generators from noisy labeled
samples, where the labels are corrupted by random noise. A standard training of
conditional GANs will not only produce samples with wrong labels, but also
generate poor quality samples. We consider two scenarios, depending on whether
the noise model is known or not. When the distribution of the noise is known,
we introduce a novel architecture which we call Robust Conditional GAN (RCGAN).
The main idea is to corrupt the label of the generated sample before feeding to
the adversarial discriminator, forcing the generator to produce samples with
clean labels. This approach of passing through a matching noisy channel is
justified by corresponding multiplicative approximation bounds between the loss
of the RCGAN and the distance between the clean real distribution and the
generator distribution. This shows that the proposed approach is robust, when
used with a carefully chosen discriminator architecture, known as projection
discriminator. When the distribution of the noise is not known, we provide an
extension of our architecture, which we call RCGAN-U, that learns the noise
model simultaneously while training the generator. We show experimentally on
MNIST and CIFAR-10 datasets that both the approaches consistently improve upon
baseline approaches, and RCGAN-U closely matches the performance of RCGAN.