pwc-archive/papers-with-abstracts huggingface dataset:
Comparing Dynamics: Deep Neural Networks versus Glassy Systems
Authors: M. Baity-Jesi, L. Sagun, M. Geiger, S. Spigler, G. Ben Arous, C. Cammarota, Y. LeCun, M. Wyart, G. Biroli
Date: 2018-03-19 00:00:00
Proceeding: ICML 2018

We analyze numerically the training dynamics of deep neural networks (DNN) by
using methods developed in statistical physics of glassy systems. The two main
issues we address are (1) the complexity of the loss landscape and of the
dynamics within it, and (2) to what extent DNNs share similarities with glassy
systems. Our findings, obtained for different architectures and datasets,
suggest that during the training process the dynamics slows down because of an
increasingly large number of flat directions. At large times, when the loss is
approaching zero, the system diffuses at the bottom of the landscape. Despite
some similarities with the dynamics of mean-field glassy systems, in
particular, the absence of barrier crossing, we find distinctive dynamical
behaviors in the two cases, showing that the statistical properties of the
corresponding loss and energy landscapes are different. In contrast, when the
network is under-parametrized we observe a typical glassy behavior, thus
suggesting the existence of different phases depending on whether the network
is under-parametrized or over-parametrized.
Generative Adversarial Networks for Realistic Synthesis of Hyperspectral Samples
Authors: Nicolas Audebert, Bertrand Le Saux, Sébastien Lefèvre
Date: 2018-06-07 00:00:00
Tasks: Data Augmentation

This work addresses the scarcity of annotated hyperspectral data required to
train deep neural networks. Especially, we investigate generative adversarial
networks and their application to the synthesis of consistent labeled spectra.
By training such networks on public datasets, we show that these models are not
only able to capture the underlying distribution, but also to generate
genuine-looking and physically plausible spectra. Moreover, we experimentally
validate that the synthetic samples can be used as an effective data
augmentation strategy. We validate our approach on several public
hyper-spectral datasets using a variety of deep classifiers.
Critères de qualité d'un classifieur généraliste
Authors: Gilles R. Ducharme
Date: 2018-02-10 00:00:00

This paper considers the problem of choosing a good classifier. For each
problem there exist an optimal classifier, but none are optimal, regarding the
error rate, in all cases. Because there exists a large number of classifiers, a
user would rather prefer an all-purpose classifier that is easy to adjust, in
the hope that it will do almost as good as the optimal. In this paper we
establish a list of criteria that a good generalist classifier should satisfy .
We first discuss data analytic, these criteria are presented. Six among the
most popular classifiers are selected and scored according to these criteria.
Tables allow to easily appreciate the relative values of each. In the end,
random forests turn out to be the best classifiers.
AI-based Two-Stage Intrusion Detection for Software Defined IoT Networks
Authors: Jiaqi Li, Zhifeng Zhao, Rongpeng Li, Honggang Zhang
Date: 2018-06-07 00:00:00
Tasks: Intrusion Detection, Management, Vocal Bursts Valence Prediction

Software Defined Internet of Things (SD-IoT) Networks profits from
centralized management and interactive resource sharing which enhances the
efficiency and scalability of IoT applications. But with the rapid growth in
services and applications, it is vulnerable to possible attacks and faces
severe security challenges. Intrusion detection has been widely used to ensure
network security, but classical detection means are usually signature-based or
explicit-behavior-based and fail to detect unknown attacks intelligently, which
are hard to satisfy the requirements of SD-IoT Networks. In this paper, we
propose an AI-based two-stage intrusion detection empowered by software defined
technology. It flexibly captures network flows with a globle view and detects
attacks intelligently through applying AI algorithms. We firstly leverage Bat
algorithm with swarm division and Differential Mutation to select typical
features. Then, we exploit Random forest through adaptively altering the
weights of samples using weighted voting mechanism to classify flows.
Evaluation results prove that the modified intelligent algorithms select more
important features and achieve superior performance in flow classification. It
is also verified that intelligent intrusion detection shows better accuracy
with lower overhead comparied with existing solutions.
On the Effect of Inter-observer Variability for a Reliable Estimation of Uncertainty of Medical Image Segmentation
Authors: Alain Jungo, Raphael Meier, Ekin Ermis, Marcela Blatti-Moreno, Evelyn Herrmann, Roland Wiest, Mauricio Reyes
Date: 2018-06-07 00:00:00
Tasks: Image Segmentation, Medical Image Segmentation, Segmentation, Semantic Segmentation

Uncertainty estimation methods are expected to improve the understanding and
quality of computer-assisted methods used in medical applications (e.g.,
neurosurgical interventions, radiotherapy planning), where automated medical
image segmentation is crucial. In supervised machine learning, a common
practice to generate ground truth label data is to merge observer annotations.
However, as many medical image tasks show a high inter-observer variability
resulting from factors such as image quality, different levels of user
expertise and domain knowledge, little is known as to how inter-observer
variability and commonly used fusion methods affect the estimation of
uncertainty of automated image segmentation. In this paper we analyze the
effect of common image label fusion techniques on uncertainty estimation, and
propose to learn the uncertainty among observers. The results highlight the
negative effect of fusion methods applied in deep learning, to obtain reliable
estimates of segmentation uncertainty. Additionally, we show that the learned
observers' uncertainty can be combined with current standard Monte Carlo
dropout Bayesian neural networks to characterize uncertainty of model's
parameters.
Shape Robust Text Detection with Progressive Scale Expansion Network
Authors: Xiang Li, Wenhai Wang, Wenbo Hou, Ruo-Ze Liu, Tong Lu, Jian Yang
Date: 2018-06-07 00:00:00
Tasks: Curved Text Detection, Scene Text Detection, Text Detection

The challenges of shape robust text detection lie in two aspects: 1) most
existing quadrangular bounding box based detectors are difficult to locate
texts with arbitrary shapes, which are hard to be enclosed perfectly in a
rectangle; 2) most pixel-wise segmentation-based detectors may not separate the
text instances that are very close to each other. To address these problems, we
propose a novel Progressive Scale Expansion Network (PSENet), designed as a
segmentation-based detector with multiple predictions for each text instance.
These predictions correspond to different `kernels' produced by shrinking the
original text instance into various scales. Consequently, the final detection
can be conducted through our progressive scale expansion algorithm which
gradually expands the kernels with minimal scales to the text instances with
maximal and complete shapes. Due to the fact that there are large geometrical
margins among these minimal kernels, our method is effective to distinguish the
adjacent text instances and is robust to arbitrary shapes. The state-of-the-art
results on ICDAR 2015 and ICDAR 2017 MLT benchmarks further confirm the great
effectiveness of PSENet. Notably, PSENet outperforms the previous best record
by absolute 6.37\% on the curve text dataset SCUT-CTW1500. Code will be
available in https://github.com/whai362/PSENet.
Online Convolutional Sparse Coding with Sample-Dependent Dictionary
Authors: Yaqing Wang, Quanming Yao, James T. Kwok, Lionel M. Ni
Date: 2018-04-27 00:00:00
Proceeding: ICML 2018 7

Convolutional sparse coding (CSC) has been popularly used for the learning of
shift-invariant dictionaries in image and signal processing. However, existing
methods have limited scalability. In this paper, instead of convolving with a
dictionary shared by all samples, we propose the use of a sample-dependent
dictionary in which filters are obtained as linear combinations of a small set
of base filters learned from the data. This added flexibility allows a large
number of sample-dependent patterns to be captured, while the resultant model
can still be efficiently learned by online learning. Extensive experimental
results show that the proposed method outperforms existing CSC algorithms with
significantly reduced time and space requirements.
Grouped Gaussian Processes for Solar Power Prediction
Authors: Astrid Dahl, Edwin V. Bonilla
Date: 2018-06-07 00:00:00
Tasks: Gaussian Processes, Prediction

We consider multi-task regression models where the observations are assumed
to be a linear combination of several latent node functions and weight
functions, which are both drawn from Gaussian process priors. Driven by the
problem of developing scalable methods for forecasting distributed solar and
other renewable power generation, we propose coupled priors over groups of
(node or weight) processes to exploit spatial dependence between functions. We
estimate forecast models for solar power at multiple distributed sites and
ground wind speed at multiple proximate weather stations. Our results show that
our approach maintains or improves point-prediction accuracy relative to
competing solar benchmarks and improves over wind forecast benchmark models on
all measures. Our approach consistently dominates the equivalent model without
coupled priors, achieving faster gains in forecast accuracy. At the same time
our approach provides better quantification of predictive uncertainties.
Segment-Based Credit Scoring Using Latent Clusters in the Variational Autoencoder
Authors: Rogelio Andrade Mancisidor, Michael Kampffmeyer, Kjersti Aas, Robert Jenssen
Date: 2018-06-07 00:00:00
Tasks: Clustering, Marketing

Identifying customer segments in retail banking portfolios with different
risk profiles can improve the accuracy of credit scoring. The Variational
Autoencoder (VAE) has shown promising results in different research domains,
and it has been documented the powerful information embedded in the latent
space of the VAE. We use the VAE and show that transforming the input data into
a meaningful representation, it is possible to steer configurations in the
latent space of the VAE. Specifically, the Weight of Evidence (WoE)
transformation encapsulates the propensity to fall into financial distress and
the latent space in the VAE preserves this characteristic in a well-defined
clustering structure. These clusters have considerably different risk profiles
and therefore are suitable not only for credit scoring but also for marketing
and customer purposes. This new clustering methodology offers solutions to some
of the challenges in the existing clustering algorithms, e.g., suggests the
number of clusters, assigns cluster labels to new customers, enables cluster
visualization, scales to large datasets, captures non-linear relationships
among others. Finally, for portfolios with a large number of customers in each
cluster, developing one classifier model per cluster can improve the credit
scoring assessment.
Deep supervision with additional labels for retinal vessel segmentation task
Authors: Yishuo Zhang, Albert C. S. Chung
Date: 2018-06-06 00:00:00
Tasks: Retinal Vessel Segmentation

Automatic analysis of retinal blood images is of vital importance in
diagnosis tasks of retinopathy. Segmenting vessels accurately is a fundamental
step in analysing retinal images. However, it is usually difficult due to
various imaging conditions, low image contrast and the appearance of
pathologies such as micro-aneurysms. In this paper, we propose a novel method
with deep neural networks to solve this problem. We utilize U-net with residual
connection to detect vessels. To achieve better accuracy, we introduce an
edge-aware mechanism, in which we convert the original task into a multi-class
task by adding additional labels on boundary areas. In this way, the network
will pay more attention to the boundary areas of vessels and achieve a better
performance, especially in tiny vessels detecting. Besides, side output layers
are applied in order to give deep supervision and therefore help convergence.
We train and evaluate our model on three databases: DRIVE, STARE, and CHASEDB1.
Experimental results show that our method has a comparable performance with AUC
of 97.99% on DRIVE and an efficient running time compared to the
state-of-the-art methods.
Instance Segmentation and Tracking with Cosine Embeddings and Recurrent Hourglass Networks
Authors: Christian Payer, Darko Štern, Thomas Neff, Horst Bischof, Martin Urschler
Date: 2018-06-06 00:00:00
Tasks: Instance Segmentation, Segmentation, Semantic Segmentation

Different to semantic segmentation, instance segmentation assigns unique
labels to each individual instance of the same class. In this work, we propose
a novel recurrent fully convolutional network architecture for tracking such
instance segmentations over time. The network architecture incorporates
convolutional gated recurrent units (ConvGRU) into a stacked hourglass network
to utilize temporal video information. Furthermore, we train the network with a
novel embedding loss based on cosine similarities, such that the network
predicts unique embeddings for every instance throughout videos. Afterwards,
these embeddings are clustered among subsequent video frames to create the
final tracked instance segmentations. We evaluate the recurrent hourglass
network by segmenting left ventricles in MR videos of the heart, where it
outperforms a network that does not incorporate video information. Furthermore,
we show applicability of the cosine embedding loss for segmenting leaf
instances on still images of plants. Finally, we evaluate the framework for
instance segmentation and tracking on six datasets of the ISBI celltracking
challenge, where it shows state-of-the-art performance.
Supervised Saliency Map Driven Segmentation of the Lesions in Dermoscopic Images
Authors: Mostafa Jahanifar, Neda Zamani Tajeddin, Babak Mohammadzadeh Asl, Ali Gooya
Date: 2017-02-28 00:00:00
Tasks: Lesion Segmentation, Saliency Detection, Segmentation

Lesion segmentation is the first step in most automatic melanoma recognition
systems. Deficiencies and difficulties in dermoscopic images such as color
inconstancy, hair occlusion, dark corners and color charts make lesion
segmentation an intricate task. In order to detect the lesion in the presence
of these problems, we propose a supervised saliency detection method tailored
for dermoscopic images based on the discriminative regional feature integration
(DRFI). DRFI method incorporates multi-level segmentation, regional contrast,
property, background descriptors, and a random forest regressor to create
saliency scores for each region in the image. In our improved saliency
detection method, mDRFI, we have added some new features to regional property
descriptors. Also, in order to achieve more robust regional background
descriptors, a thresholding algorithm is proposed to obtain a new
pseudo-background region. Findings reveal that mDRFI is superior to DRFI in
detecting the lesion as the salient object in dermoscopic images. The proposed
overall lesion segmentation framework uses detected saliency map to construct
an initial mask of the lesion through thresholding and post-processing
operations. The initial mask is then evolving in a level set framework to fit
better on the lesion's boundaries. The results of evaluation tests on three
public datasets show that our proposed segmentation method outperforms the
other conventional state-of-the-art segmentation algorithms and its performance
is comparable with most recent approaches that are based on deep convolutional
neural networks.
Conditional End-to-End Audio Transforms
Authors: Albert Haque, Michelle Guo, Prateek Verma
Date: 2018-03-30 00:00:00

We present an end-to-end method for transforming audio from one style to
another. For the case of speech, by conditioning on speaker identities, we can
train a single model to transform words spoken by multiple people into multiple
target voices. For the case of music, we can specify musical instruments and
achieve the same result. Architecturally, our method is a fully-differentiable
sequence-to-sequence model based on convolutional and hierarchical recurrent
neural networks. It is designed to capture long-term acoustic dependencies,
requires minimal post-processing, and produces realistic audio transforms.
Ablation studies confirm that our model can separate speaker and instrument
properties from acoustic content at different receptive fields. Empirically,
our method achieves competitive performance on community-standard datasets.
Respond-CAM: Analyzing Deep Models for 3D Imaging Data by Visualizations
Authors: Guannan Zhao, Bo Zhou, Kaiwen Wang, Rui Jiang, Min Xu
Date: 2018-05-31 00:00:00

The convolutional neural network (CNN) has become a powerful tool for various
biomedical image analysis tasks, but there is a lack of visual explanation for
the machinery of CNNs. In this paper, we present a novel algorithm,
Respond-weighted Class Activation Mapping (Respond-CAM), for making CNN-based
models interpretable by visualizing input regions that are important for
predictions, especially for biomedical 3D imaging data inputs. Our method uses
the gradients of any target concept (e.g. the score of target class) that flows
into a convolutional layer. The weighted feature maps are combined to produce a
heatmap that highlights the important regions in the image for predicting the
target concept. We prove a preferable sum-to-score property of the Respond-CAM
and verify its significant improvement on 3D images from the current
state-of-the-art approach. Our tests on Cellular Electron Cryo-Tomography 3D
images show that Respond-CAM achieves superior performance on visualizing the
CNNs with 3D biomedical images inputs, and is able to get reasonably good
results on visualizing the CNNs with natural image inputs. The Respond-CAM is
an efficient and reliable approach for visualizing the CNN machinery, and is
applicable to a wide variety of CNN model families and image analysis tasks.
Toward Diverse Text Generation with Inverse Reinforcement Learning
Authors: Zhan Shi, Xinchi Chen, Xipeng Qiu, Xuanjing Huang
Date: 2018-04-30 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Text Generation

Text generation is a crucial task in NLP. Recently, several adversarial
generative models have been proposed to improve the exposure bias problem in
text generation. Though these models gain great success, they still suffer from
the problems of reward sparsity and mode collapse. In order to address these
two problems, in this paper, we employ inverse reinforcement learning (IRL) for
text generation. Specifically, the IRL framework learns a reward function on
training data, and then an optimal policy to maximum the expected total reward.
Similar to the adversarial models, the reward and policy function in IRL are
optimized alternately. Our method has two advantages: (1) the reward function
can produce more dense reward signals. (2) the generation policy, trained by
"entropy regularized" policy gradient, encourages to generate more diversified
texts. Experiment results demonstrate that our proposed method can generate
higher quality texts than the previous methods.
Information-Maximizing Sampling to Promote Tracking-by-Detection
Authors: Kourosh Meshgi, Maryam Sadat Mirzaei, Shigeyuki Oba
Date: 2018-06-07 00:00:00
Tasks: General Classification

The performance of an adaptive tracking-by-detection algorithm not only
depends on the classification and updating processes but also on the sampling.
Typically, such trackers select their samples from the vicinity of the last
predicted object location, or from its expected location using a pre-defined
motion model, which does not exploit the contents of the samples nor the
information provided by the classifier. We introduced the idea of most
informative sampling, in which the sampler attempts to select samples that
trouble the classifier of a discriminative tracker. We then proposed an active
discriminative co-tracker that embed an adversarial sampler to increase its
robustness against various tracking challenges. Experiments show that our
proposed tracker outperforms state-of-the-art trackers on various benchmark
videos.
Efficient Diverse Ensemble for Discriminative Co-Tracking
Authors: Kourosh Meshgi, Shigeyuki Oba, Shin Ishii
Date: 2017-11-16 00:00:00
Proceeding: CVPR 2018 6
Tasks: Ensemble Learning

Ensemble discriminative tracking utilizes a committee of classifiers, to
label data samples, which are in turn, used for retraining the tracker to
localize the target using the collective knowledge of the committee. Committee
members could vary in their features, memory update schemes, or training data,
however, it is inevitable to have committee members that excessively agree
because of large overlaps in their version space. To remove this redundancy and
have an effective ensemble learning, it is critical for the committee to
include consistent hypotheses that differ from one-another, covering the
version space with minimum overlaps. In this study, we propose an online
ensemble tracker that directly generates a diverse committee by generating an
efficient set of artificial training. The artificial data is sampled from the
empirical distribution of the samples taken from both target and background,
whereas the process is governed by query-by-committee to shrink the overlap
between classifiers. The experimental results demonstrate that the proposed
scheme outperforms conventional ensemble trackers on public benchmarks.
Global-Local Airborne Mapping (GLAM): Reconstructing a City from Aerial Videos
Authors: Hasnain Vohra, Maxim Bazik, Matthew Antone, Joseph Mundy, William Stephenson
Date: 2017-06-06 00:00:00

Monocular visual SLAM has become an attractive practical approach for robot
localization and 3D environment mapping, since cameras are small, lightweight,
inexpensive, and produce high-rate, high-resolution data streams. Although
numerous robust tools have been developed, most existing systems are designed
to operate in terrestrial environments and at relatively small scale (a few
thousand frames) due to constraints on computation and storage.
  In this paper, we present a feature-based visual SLAM system for aerial video
whose simple design permits near real-time operation, and whose scalability
permits large-area mapping using tens of thousands of frames, all on a single
conventional computer. Our approach consists of two parallel threads: the first
incrementally creates small locally consistent submaps and estimates camera
poses at video rate; the second aligns these submaps with one another to
produce a single globally consistent map via factor graph optimization over
both poses and landmarks. Scale drift is minimized through the use of
7-degree-of-freedom similarity transformations during submap alignment.
  We quantify our system's performance on both simulated and real data sets,
and demonstrate city-scale map reconstruction accurate to within 2 meters using
nearly 90,000 aerial video frames - to our knowledge, the largest and fastest
such reconstruction to date.