pwc-archive/papers-with-abstracts huggingface dataset:
Hi-Fi: Hierarchical Feature Integration for Skeleton Detection
Authors: Kai Zhao, Wei Shen, Shang-Hua Gao, Dandan Li, Ming-Ming Cheng
Date: 2018-01-05 00:00:00
Tasks: Object, Object Skeleton Detection

In natural images, the scales (thickness) of object skeletons may
dramatically vary among objects and object parts, making object skeleton
detection a challenging problem. We present a new convolutional neural network
(CNN) architecture by introducing a novel hierarchical feature integration
mechanism, named Hi-Fi, to address the skeleton detection problem. The proposed
CNN-based approach has a powerful multi-scale feature integration ability that
intrinsically captures high-level semantics from deeper layers as well as
low-level details from shallower layers. % By hierarchically integrating
different CNN feature levels with bidirectional guidance, our approach (1)
enables mutual refinement across features of different levels, and (2)
possesses the strong ability to capture both rich object context and
high-resolution details. Experimental results show that our method
significantly outperforms the state-of-the-art methods in terms of effectively
fusing features from very different scales, as evidenced by a considerable
performance improvement on several benchmarks.
Dense Adaptive Cascade Forest: A Self Adaptive Deep Ensemble for Classification Problems
Authors: Haiyang Wang, Yong Tang, Ziyang Jia, Fei Ye
Date: 2018-04-29 00:00:00
Tasks: Ensemble Learning, General Classification

Recent researches have shown that deep forest ensemble achieves a considerable increase in classification accuracy compared with the general ensemble learning methods, especially when the training set is small. In this paper, we take advantage of deep forest ensemble and introduce the Dense Adaptive Cascade Forest (daForest). Our model has a better performance than the original Cascade Forest with three major features: first, we apply SAMME.R boosting algorithm to improve the performance of the model. It guarantees the improvement as the number of layers increases. Second, our model connects each layer to the subsequent ones in a feed-forward fashion, which enhances the capability of the model to resist performance degeneration. Third, we add a hyper-parameters optimization layer before the first classification layer, making our model spend less time to set up and find the optimal hyper-parameters. Experimental results show that daForest performs significantly well, and in some cases, even outperforms neural networks and achieves state-of-the-art results.
Virtual-to-Real: Learning to Control in Visual Semantic Segmentation
Authors: Zhang-Wei Hong, Chen Yu-Ming, Shih-Yang Su, Tzu-Yun Shann, Yi-Hsiang Chang, Hsuan-Kung Yang, Brian Hsi-Lin Ho, Chih-Chieh Tu, Yueh-Chuan Chang, Tsu-Ching Hsiao, Hsin-Wei Hsiao, Sih-Pin Lai, Chun-Yi Lee
Date: 2018-02-01 00:00:00
Tasks: Deep Reinforcement Learning, Image Segmentation, Reinforcement Learning, Segmentation, Semantic Segmentation

Collecting training data from the physical world is usually time-consuming
and even dangerous for fragile robots, and thus, recent advances in robot
learning advocate the use of simulators as the training platform.
Unfortunately, the reality gap between synthetic and real visual data prohibits
direct migration of the models trained in virtual worlds to the real world.
This paper proposes a modular architecture for tackling the virtual-to-real
problem. The proposed architecture separates the learning model into a
perception module and a control policy module, and uses semantic image
segmentation as the meta representation for relating these two modules. The
perception module translates the perceived RGB image to semantic image
segmentation. The control policy module is implemented as a deep reinforcement
learning agent, which performs actions based on the translated image
segmentation. Our architecture is evaluated in an obstacle avoidance task and a
target following task. Experimental results show that our architecture
significantly outperforms all of the baseline methods in both virtual and real
environments, and demonstrates a faster learning curve than them. We also
present a detailed analysis for a variety of variant configurations, and
validate the transferability of our modular architecture.
TreeSegNet: Adaptive Tree CNNs for Subdecimeter Aerial Image Segmentation
Authors: Kai Yue, Lei Yang, Ruirui Li, Wei Hu, Fan Zhang, Wei Li
Date: 2018-04-29 00:00:00
Tasks: Image Segmentation, Segmentation, Semantic Segmentation

For the task of subdecimeter aerial imagery segmentation, fine-grained
semantic segmentation results are usually difficult to obtain because of
complex remote sensing content and optical conditions. Recently, convolutional
neural networks (CNNs) have shown outstanding performance on this task.
Although many deep neural network structures and techniques have been applied
to improve the accuracy, few have paid attention to better differentiating the
easily confused classes. In this paper, we propose TreeSegNet which adopts an
adaptive network to increase the classification rate at the pixelwise level.
Specifically, based on the infrastructure of DeepUNet, a Tree-CNN block in
which each node represents a ResNeXt unit is constructed adaptively according
to the confusion matrix and the proposed TreeCutting algorithm. By transporting
feature maps through concatenating connections, the Tree-CNN block fuses
multiscale features and learns best weights for the model. In experiments on
the ISPRS 2D semantic labeling Potsdam dataset, the results obtained by
TreeSegNet are better than those of other published state-of-the-art methods.
Detailed comparison and analysis show that the improvement brought by the
adaptive Tree-CNN block is significant.
Online Second Order Methods for Non-Convex Stochastic Optimizations
Authors: Xi-Lin Li
Date: 2018-03-26 00:00:00
Tasks: Second-order methods

This paper proposes a family of online second order methods for possibly
non-convex stochastic optimizations based on the theory of preconditioned
stochastic gradient descent (PSGD), which can be regarded as an enhance
stochastic Newton method with the ability to handle gradient noise and
non-convexity simultaneously. We have improved the implementations of the
original PSGD in several ways, e.g., new forms of preconditioners, more
accurate Hessian vector product calculations, and better numerical stability
with vanishing or ill-conditioned Hessian, etc.. We also have unrevealed the
relationship between feature normalization and PSGD with Kronecker product
preconditioners, which explains the excellent performance of Kronecker product
preconditioners in deep neural network learning. A software package
(https://github.com/lixilinx/psgd_tf) implemented in Tensorflow is provided to
compare variations of stochastic gradient descent (SGD) and PSGD with five
different preconditioners on a wide range of benchmark problems with commonly
used neural network architectures, e.g., convolutional and recurrent neural
networks. Experimental results clearly demonstrate the advantages of PSGD in
terms of generalization performance and convergence speed.
Evolved Policy Gradients
Authors: Rein Houthooft, Richard Y. Chen, Phillip Isola, Bradly C. Stadie, Filip Wolski, Jonathan Ho, Pieter Abbeel
Date: 2018-02-13 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Reinforcement Learning, Reinforcement Learning (RL)

We propose a metalearning approach for learning gradient-based reinforcement
learning (RL) algorithms. The idea is to evolve a differentiable loss function,
such that an agent, which optimizes its policy to minimize this loss, will
achieve high rewards. The loss is parametrized via temporal convolutions over
the agent's experience. Because this loss is highly flexible in its ability to
take into account the agent's history, it enables fast task learning. Empirical
results show that our evolved policy gradient algorithm (EPG) achieves faster
learning on several randomized environments compared to an off-the-shelf policy
gradient method. We also demonstrate that EPG's learned loss can generalize to
out-of-distribution test time tasks, and exhibits qualitatively different
behavior from other popular metalearning algorithms.
Quantum dynamical mode (QDM): A possible extension of belief function
Authors: Fuyuan Xiao
Date: 2018-01-16 00:00:00
Tasks: Decision Making

Dempster-Shafer evidence theory has been widely used in various fields of
applications, because of the flexibility and effectiveness in modeling
uncertainties without prior information. Besides, it has been proven that the
quantum theory has powerful capabilities of solving the decision making
problems, especially for modelling human decision and cognition. However, the
classical Dempster-Shafer evidence theory modelled by real numbers cannot be
integrated directly with the quantum theory modelled by complex numbers. So,
how can we establish a bridge of communications between the classical
Dempster-Shafer evidence theory and the quantum theory? To answer this
question, a generalized Dempster-Shafer evidence theory is proposed in this
paper. The main contribution in this study is that, unlike the existing
evidence theory, a mass function in the generalized Dempster-Shafer evidence
theory is modelled by a complex number, called as a complex mass function. In
addition, compared with the classical Dempster's combination rule, the
condition in terms of the conflict coefficient between two evidences K < 1 is
released in the generalized Dempster's combination rule so that it is more
general and applicable than the classical Dempster's combination rule. When the
complex mass function is degenerated from complex numbers to real numbers, the
generalized Dempster's combination rule degenerates to the classical evidence
theory under the condition that the conflict coefficient between the evidences
K is less than 1. Numerical examples are illustrated to show the efficiency of
the generalized Dempster-Shafer evidence theory. Finally, an application of an
evidential quantum dynamical model is implemented by integrating the
generalized Dempster-Shafer evidence theory with the quantum dynamical model.
From the experimental results, it validates the feasibility and effectiveness
of the proposed method.
On the Reconstruction of Face Images from Deep Face Templates
Authors: Guangcan Mai, Kai Cao, Pong C. Yuen, Anil K. Jain
Date: 2017-03-02 00:00:00
Tasks: Face Recognition, Reconstruction Attack, TAR

State-of-the-art face recognition systems are based on deep (convolutional)
neural networks. Therefore, it is imperative to determine to what extent face
templates derived from deep networks can be inverted to obtain the original
face image. In this paper, we study the vulnerabilities of a state-of-the-art
face recognition system based on template reconstruction attack. We propose a
neighborly de-convolutional neural network (\textit{NbNet}) to reconstruct face
images from their deep templates. In our experiments, we assumed that no
knowledge about the target subject and the deep network are available. To train
the \textit{NbNet} reconstruction models, we augmented two benchmark face
datasets (VGG-Face and Multi-PIE) with a large collection of images synthesized
using a face generator. The proposed reconstruction was evaluated using type-I
(comparing the reconstructed images against the original face images used to
generate the deep template) and type-II (comparing the reconstructed images
against a different face image of the same subject) attacks. Given the images
reconstructed from \textit{NbNets}, we show that for verification, we achieve
TAR of 95.20\% (58.05\%) on LFW under type-I (type-II) attacks @ FAR of 0.1\%.
Besides, 96.58\% (92.84\%) of the images reconstruction from templates of
partition \textit{fa} (\textit{fb}) can be identified from partition
\textit{fa} in color FERET. Our study demonstrates the need to secure deep
templates in face recognition systems.
Imbalanced Deep Learning by Minority Class Incremental Rectification
Authors: Qi Dong, Shaogang Gong, Xiatian Zhu
Date: 2018-04-28 00:00:00
Tasks: Attribute, Deep Learning, Facial Attribute Classification

Model learning from class imbalanced training data is a long-standing and
significant challenge for machine learning. In particular, existing deep
learning methods consider mostly either class balanced data or moderately
imbalanced data in model training, and ignore the challenge of learning from
significantly imbalanced training data. To address this problem, we formulate a
class imbalanced deep learning model based on batch-wise incremental minority
(sparsely sampled) class rectification by hard sample mining in majority
(frequently sampled) classes during model training. This model is designed to
minimise the dominant effect of majority classes by discovering sparsely
sampled boundaries of minority classes in an iterative batch-wise learning
process. To that end, we introduce a Class Rectification Loss (CRL) function
that can be deployed readily in deep network architectures. Extensive
experimental evaluations are conducted on three imbalanced person attribute
benchmark datasets (CelebA, X-Domain, DeepFashion) and one balanced object
category benchmark dataset (CIFAR-100). These experimental results demonstrate
the performance advantages and model scalability of the proposed batch-wise
incremental minority class rectification model over the existing
state-of-the-art models for addressing the problem of imbalanced data learning.
Deep BCD-Net Using Identical Encoding-Decoding CNN Structures for Iterative Image Recovery
Authors: Il Yong Chun, Jeffrey A. Fessler
Date: 2018-02-20 00:00:00
Tasks: Denoising

In "extreme" computational imaging that collects extremely undersampled or
noisy measurements, obtaining an accurate image within a reasonable computing
time is challenging. Incorporating image mapping convolutional neural networks
(CNN) into iterative image recovery has great potential to resolve this issue.
This paper 1) incorporates image mapping CNN using identical convolutional
kernels in both encoders and decoders into a block coordinate descent (BCD)
signal recovery method and 2) applies alternating direction method of
multipliers to train the aforementioned image mapping CNN. We refer to the
proposed recurrent network as BCD-Net using identical encoding-decoding CNN
structures. Numerical experiments show that, for a) denoising low
signal-to-noise-ratio images and b) extremely undersampled magnetic resonance
imaging, the proposed BCD-Net achieves significantly more accurate image
recovery, compared to BCD-Net using distinct encoding-decoding structures
and/or the conventional image recovery model using both wavelets and total
variation.
Drug Similarity Integration Through Attentive Multi-view Graph Auto-Encoders
Authors: Tengfei Ma, Cao Xiao, Jiayu Zhou, Fei Wang
Date: 2018-04-28 00:00:00

Drug similarity has been studied to support downstream clinical tasks such as
inferring novel properties of drugs (e.g. side effects, indications,
interactions) from known properties. The growing availability of new types of
drug features brings the opportunity of learning a more comprehensive and
accurate drug similarity that represents the full spectrum of underlying drug
relations. However, it is challenging to integrate these heterogeneous, noisy,
nonlinear-related information to learn accurate similarity measures especially
when labels are scarce. Moreover, there is a trade-off between accuracy and
interpretability. In this paper, we propose to learn accurate and interpretable
similarity measures from multiple types of drug features. In particular, we
model the integration using multi-view graph auto-encoders, and add attentive
mechanism to determine the weights for each view with respect to corresponding
tasks and features for better interpretability. Our model has flexible design
for both semi-supervised and unsupervised settings. Experimental results
demonstrated significant predictive accuracy improvement. Case studies also
showed better model capacity (e.g. embed node features) and interpretability.
Probabilistic supervised learning
Authors: Frithjof Gressmann, Franz J. Király, Bilal Mateen, Harald Oberhauser
Date: 2018-01-02 00:00:00
Tasks: BIG-bench Machine Learning, Decision Making, Prediction

Predictive modelling and supervised learning are central to modern data science. With predictions from an ever-expanding number of supervised black-box strategies - e.g., kernel methods, random forests, deep learning aka neural networks - being employed as a basis for decision making processes, it is crucial to understand the statistical uncertainty associated with these predictions. As a general means to approach the issue, we present an overarching framework for black-box prediction strategies that not only predict the target but also their own predictions' uncertainty. Moreover, the framework allows for fair assessment and comparison of disparate prediction strategies. For this, we formally consider strategies capable of predicting full distributions from feature variables, so-called probabilistic supervised learning strategies. Our work draws from prior work including Bayesian statistics, information theory, and modern supervised machine learning, and in a novel synthesis leads to (a) new theoretical insights such as a probabilistic bias-variance decomposition and an entropic formulation of prediction, as well as to (b) new algorithms and meta-algorithms, such as composite prediction strategies, probabilistic boosting and bagging, and a probabilistic predictive independence test. Our black-box formulation also leads (c) to a new modular interface view on probabilistic supervised learning and a modelling workflow API design, which we have implemented in the newly released skpro machine learning toolbox, extending the familiar modelling interface and meta-modelling functionality of sklearn. The skpro package provides interfaces for construction, composition, and tuning of probabilistic supervised learning strategies, together with orchestration features for validation and comparison of any such strategy - be it frequentist, Bayesian, or other.
Predictive Independence Testing, Predictive Conditional Independence Testing, and Predictive Graphical Modelling
Authors: Samuel Burkart, Franz J. Király
Date: 2017-11-16 00:00:00
Tasks: Philosophy

Testing (conditional) independence of multivariate random variables is a task
central to statistical inference and modelling in general - though
unfortunately one for which to date there does not exist a practicable
workflow. State-of-art workflows suffer from the need for heuristic or
subjective manual choices, high computational complexity, or strong parametric
assumptions.
  We address these problems by establishing a theoretical link between
multivariate/conditional independence testing, and model comparison in the
multivariate predictive modelling aka supervised learning task. This link
allows advances in the extensively studied supervised learning workflow to be
directly transferred to independence testing workflows - including automated
tuning of machine learning type which addresses the need for a heuristic
choice, the ability to quantitatively trade-off computational demand with
accuracy, and the modern black-box philosophy for checking and interfacing.
  As a practical implementation of this link between the two workflows, we
present a python package 'pcit', which implements our novel multivariate and
conditional independence tests, interfacing the supervised learning API of the
scikit-learn package. Theory and package also allow for straightforward
independence test based learning of graphical model structure.
  We empirically show that our proposed predictive independence test outperform
or are on par to current practice, and the derived graphical model structure
learning algorithms asymptotically recover the 'true' graph. This paper, and
the 'pcit' package accompanying it, thus provide powerful, scalable,
generalizable, and easy-to-use methods for multivariate and conditional
independence testing, as well as for graphical model structure learning.
CRAM: Clued Recurrent Attention Model
Authors: Minki Chung, Sungzoon Cho
Date: 2018-04-28 00:00:00
Tasks: Decoder, image-classification, Image Classification, model

To overcome the poor scalability of convolutional neural network, recurrent
attention model(RAM) selectively choose what and where to look on the image. By
directing recurrent attention model how to look the image, RAM can be even more
successful in that the given clue narrow down the scope of the possible focus
zone. In this perspective, this work proposes clued recurrent attention model
(CRAM) which add clue or constraint on the RAM better problem solving. CRAM
follows encoder-decoder framework, encoder utilizes recurrent attention model
with spatial transformer network and decoder which varies depending on the
task. To ensure the performance, CRAM tackles two computer vision task. One is
the image classification task, with clue given as the binary image saliency
which indicates the approximate location of object. The other is the inpainting
task, with clue given as binary mask which indicates the occluded part. In both
tasks, CRAM shows better performance than existing methods showing the
successful extension of RAM.
Closed-form detector for solid sub-pixel targets in multivariate t-distributed background clutter
Authors: James Theiler, Beate Zimmer, Amanda Ziemann
Date: 2018-04-05 00:00:00
Tasks: Form

The generalized likelihood ratio test (GLRT) is used to derive a detector for
solid sub-pixel targets in hyperspectral imagery. A closed-form solution is
obtained that optimizes the replacement target model when the background is a
fat-tailed elliptically-contoured multivariate t-distribution. This generalizes
GLRT-based detectors that have previously been derived for the replacement
target model with Gaussian background, and for the additive target model with
an elliptically-contoured background. Experiments with simulated hyperspectral
data illustrate the performance of this detector in various parameter regimes.
Learning from multivariate discrete sequential data using a restricted Boltzmann machine model
Authors: Jefferson Hernandez, Andres G. Abad
Date: 2018-04-28 00:00:00
Tasks: Collaborative Filtering, Time Series, Time Series Analysis

A restricted Boltzmann machine (RBM) is a generative neural-network model
with many novel applications such as collaborative filtering and acoustic
modeling. An RBM lacks the capacity to retain memory, making it inappropriate
for dynamic data modeling as in time-series analysis. In this paper we address
this issue by proposing the p-RBM model, a generalization of the regular RBM
model, capable of retaining memory of p past states. We further show how to
train the p-RBM model using contrastive divergence and test our model on the
problem of predicting the stock market direction considering 100 stocks of the
NASDAQ-100 index. Obtained results show that the p-RBM offer promising
prediction potential.