pwc-archive/papers-with-abstracts huggingface dataset:
Improving Neural Architecture Search Image Classifiers via Ensemble Learning
Authors: Vladimir Macko, Charles Weill, Hanna Mazzawi, Javier Gonzalvo
Date: 2019-03-14 00:00:00
Tasks: Ensemble Learning, Image Classification, Knowledge Distillation, Neural Architecture Search

Finding the best neural network architecture requires significant time,
resources, and human expertise. These challenges are partially addressed by
neural architecture search (NAS) which is able to find the best convolutional
layer or cell that is then used as a building block for the network. However,
once a good building block is found, manual design is still required to
assemble the final architecture as a combination of multiple blocks under a
predefined parameter budget constraint. A common solution is to stack these
blocks into a single tower and adjust the width and depth to fill the parameter
budget. However, these single tower architectures may not be optimal. Instead,
in this paper we present the AdaNAS algorithm, that uses ensemble techniques to
compose a neural network as an ensemble of smaller networks automatically.
Additionally, we introduce a novel technique based on knowledge distillation to
iteratively train the smaller networks using the previous ensemble as a
teacher. Our experiments demonstrate that ensembles of networks improve
accuracy upon a single neural network while keeping the same number of
parameters. Our models achieve comparable results with the state-of-the-art on
CIFAR-10 and sets a new state-of-the-art on CIFAR-100.
Teaching with IMPACT
Authors: Carl Trimbach, Michael Littman
Date: 2019-03-14 00:00:00

Like many problems in AI in their general form, supervised learning is
computationally intractable. We hypothesize that an important reason humans can
learn highly complex and varied concepts, in spite of the computational
difficulty, is that they benefit tremendously from experienced and insightful
teachers. This paper proposes a new learning framework that provides a role for
a knowledgeable, benevolent teacher to guide the process of learning a target
concept in a series of "curricular" phases or rounds. In each round, the
teacher's role is to act as a moderator, exposing the learner to a subset of
the available training data to move it closer to mastering the target concept.
Via both theoretical and empirical evidence, we argue that this framework
enables simple, efficient learners to acquire very complex concepts from
examples. In particular, we provide multiple examples of concept classes that
are known to be unlearnable in the standard PAC setting along with provably
efficient algorithms for learning them in our extended setting. A key focus of
our work is the ability to learn complex concepts on top of simpler, previously
learned, concepts---a direction with the potential of creating more competent
artificial agents.
No-regret Exploration in Contextual Reinforcement Learning
Authors: Aditya Modi, Ambuj Tewari
Date: 2019-03-14 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We consider the recently proposed reinforcement learning (RL) framework of Contextual Markov Decision Processes (CMDP), where the agent interacts with a (potentially adversarial) sequence of episodic tabular MDPs. In addition, a context vector determining the MDP parameters is available to the agent at the start of each episode, thereby allowing it to learn a context-dependent near-optimal policy. In this paper, we propose a no-regret online RL algorithm in the setting where the MDP parameters are obtained from the context using generalized linear mappings (GLMs). We propose and analyze optimistic and randomized exploration methods which make (time and space) efficient online updates. The GLM based model subsumes previous work in this area and also improves previous known bounds in the special case where the contextual mapping is linear. In addition, we demonstrate a generic template to derive confidence sets using an online learning oracle and give a lower bound for the setting.
Stripe: Tensor Compilation via the Nested Polyhedral Model
Authors: Tim Zerrell, Jeremy Bruestle
Date: 2019-03-14 00:00:00
Tasks: Code Generation, model

Hardware architectures and machine learning (ML) libraries evolve rapidly.
Traditional compilers often fail to generate high-performance code across the
spectrum of new hardware offerings. To mitigate, engineers develop hand-tuned
kernels for each ML library update and hardware upgrade. Unfortunately, this
approach requires excessive engineering effort to scale or maintain with any
degree of state-of-the-art performance. Here we present a Nested Polyhedral
Model for representing highly parallelizable computations with limited
dependencies between iterations. This model provides an underlying framework
for an intermediate representation (IR) called Stripe, amenable to standard
compiler techniques while naturally modeling key aspects of modern ML
computing. Stripe represents parallelism, efficient memory layout, and multiple
compute units at a level of abstraction amenable to automatic optimization. We
describe how Stripe enables a compiler for ML in the style of LLVM that allows
independent development of algorithms, optimizations, and hardware
accelerators. We also discuss the design exploration advantages of Stripe over
kernel libraries and schedule-based or schedule-space-based code generation.
Learning Latent Representations of Bank Customers With The Variational Autoencoder
Authors: Rogelio A. Mancisidor, Michael Kampffmeyer, Kjersti Aas, Robert Jenssen
Date: 2019-03-14 00:00:00
Tasks: Clustering, Management, Marketing

Learning data representations that reflect the customers' creditworthiness
can improve marketing campaigns, customer relationship management, data and
process management or the credit risk assessment in retail banks. In this
research, we adopt the Variational Autoencoder (VAE), which has the ability to
learn latent representations that contain useful information. We show that it
is possible to steer the latent representations in the latent space of the VAE
using the Weight of Evidence and forming a specific grouping of the data that
reflects the customers' creditworthiness. Our proposed method learns a latent
representation of the data, which shows a well-defied clustering structure
capturing the customers' creditworthiness. These clusters are well suited for
the aforementioned banks' activities. Further, our methodology generalizes to
new customers, captures high-dimensional and complex financial data, and scales
to large data sets.
Unsupervised and interpretable scene discovery with Discrete-Attend-Infer-Repeat
Authors: Duo Wang, Mateja Jamnik, Pietro Lio
Date: 2019-03-14 00:00:00
Tasks: Attribute

In this work we present Discrete Attend Infer Repeat (Discrete-AIR), a
Recurrent Auto-Encoder with structured latent distributions containing discrete
categorical distributions, continuous attribute distributions, and factorised
spatial attention. While inspired by the original AIR model andretaining AIR
model's capability in identifying objects in an image, Discrete-AIR provides
direct interpretability of the latent codes. We show that for Multi-MNIST and a
multiple-objects version of dSprites dataset, the Discrete-AIR model needs just
one categorical latent variable, one attribute variable (for Multi-MNIST only),
together with spatial attention variables, for efficient inference. We perform
analysis to show that the learnt categorical distributions effectively capture
the categories of objects in the scene for Multi-MNIST and for Multi-Sprites.
Episodic Memory Reader: Learning What to Remember for Question Answering from Streaming Data
Authors: Moonsu Han, Minki Kang, Hyunwoo Jung, Sung Ju Hwang
Date: 2019-03-14 00:00:00
Proceeding: ACL 2019 7
Tasks: Question Answering, Reading Comprehension, Scheduling, TriviaQA

We consider a novel question answering (QA) task where the machine needs to read from large streaming data (long documents or videos) without knowing when the questions will be given, which is difficult to solve with existing QA methods due to their lack of scalability. To tackle this problem, we propose a novel end-to-end deep network model for reading comprehension, which we refer to as Episodic Memory Reader (EMR) that sequentially reads the input contexts into an external memory, while replacing memories that are less important for answering \emph{unseen} questions. Specifically, we train an RL agent to replace a memory entry when the memory is full, in order to maximize its QA accuracy at a future timepoint, while encoding the external memory using either the GRU or the Transformer architecture to learn representations that considers relative importance between the memory entries. We validate our model on a synthetic dataset (bAbI) as well as real-world large-scale textual QA (TriviaQA) and video QA (TVQA) datasets, on which it achieves significant improvements over rule-based memory scheduling policies or an RL-based baseline that independently learns the query-specific importance of each memory.
Superpixel Contracted Graph-Based Learning for Hyperspectral Image Classification
Authors: Philip Sellars, Angelica Aviles-Rivero, Carola-Bibiane Schönlieb
Date: 2019-03-14 00:00:00
Tasks: Classification, General Classification, Hyperspectral Image Classification, image-classification, Image Classification, Superpixels

A central problem in hyperspectral image classification is obtaining high
classification accuracy when using a limited amount of labelled data. In this
paper we present a novel graph-based framework, which aims to tackle this
problem in the presence of large scale data input. Our approach utilises a
novel superpixel method, specifically designed for hyperspectral data, to
define meaningful local regions in an image, which with high probability share
the same classification label. We then extract spectral and spatial features
from these regions and use these to produce a contracted weighted
graph-representation, where each node represents a region rather than a pixel.
Our graph is then fed into a graph-based semi-supervised classifier which gives
the final classification. We show that using superpixels in a graph
representation is an effective tool for speeding up graphical classifiers
applied to hyperspectral images. We demonstrate through exhaustive quantitative
and qualitative results that our proposed method produces accurate
classifications when an incredibly small amount of labelled data is used. We
show that our approach mitigates the major drawbacks of existing approaches,
resulting in our approach outperforming several comparative state-of-the-art
techniques.
Constrained Mutual Convex Cone Method for Image Set Based Recognition
Authors: Naoya Sogi, Rui Zhu, Jing-Hao Xue, Kazuhiro Fukui
Date: 2019-03-14 00:00:00
Tasks: Classification, General Classification

In this paper, we propose a method for image-set classification based on
convex cone models. Image set classification aims to classify a set of images,
which were usually obtained from video frames or multi-view cameras, into a
target object. To accurately and stably classify a set, it is essential to
represent structural information of the set accurately. There are various
representative image features, such as histogram based features, HLAC, and
Convolutional Neural Network (CNN) features. We should note that most of them
have non-negativity and thus can be effectively represented by a convex cone.
This leads us to introduce the convex cone representation to image-set
classification. To establish a convex cone based framework, we mathematically
define multiple angles between two convex cones, and then define the geometric
similarity between the cones using the angles. Moreover, to enhance the
framework, we introduce a discriminant space that maximizes the between-class
variance (gaps) and minimizes the within-class variance of the projected convex
cones onto the discriminant space, similar to the Fisher discriminant analysis.
Finally, the classification is performed based on the similarity between
projected convex cones. The effectiveness of the proposed method is
demonstrated experimentally by using five databases: CMU PIE dataset, ETH-80,
CMU Motion of Body dataset, Youtube Celebrity dataset, and a private database
of multi-view hand shapes.
Spectroscopic Approach to Correction and Visualisation of Bright-Field Light Transmission Microscopy Biological Data
Authors: Ganna Platonova, Dalibor Stys, Pavel Soucek, Kirill Lonhus, Jan Valenta, Renata Rychtarikova
Date: 2019-03-14 00:00:00

The most realistic information about the transparent sample such as a live cell can be obtained only using bright-field light microscopy. At high-intensity pulsing LED illumination, we captured a primary 12-bit-per-channel (bpc) response from an observed sample using a bright-field microscope equipped with a high-resolution (4872x3248) image sensor. In order to suppress data distortions originating from the light interactions with elements in the optical path, poor sensor reproduction (geometrical defects of the camera sensor and some peculiarities of sensor sensitivity), we propose a spectroscopic approach for the correction of this uncompressed 12-bpc data by simultaneous calibration of all parts of the experimental arrangement. Moreover, the final intensities of the corrected images are proportional to the photon fluxes detected by a camera sensor. It can be visualized in 8-bpc intensity depth after the Least Information Loss compression.
Learning Representations from Persian Handwriting for Offline Signature Verification, a Deep Transfer Learning Approach
Authors: Omid Mersa, Farhood Etaati, Saeed Masoudnia, Babak N. Araabi
Date: 2019-02-28 00:00:00
Tasks: Transfer Learning

Offline Signature Verification (OSV) is a challenging pattern recognition
task, especially when it is expected to generalize well on the skilled
forgeries that are not available during the training. Its challenges also
include small training sample and large intra-class variations. Considering the
limitations, we suggest a novel transfer learning approach from Persian
handwriting domain to multi-language OSV domain. We train two Residual CNNs on
the source domain separately based on two different tasks of word
classification and writer identification. Since identifying a person signature
resembles identifying ones handwriting, it seems perfectly convenient to use
handwriting for the feature learning phase. The learned representation on the
more varied and plentiful handwriting dataset can compensate for the lack of
training data in the original task, i.e. OSV, without sacrificing the
generalizability. Our proposed OSV system includes two steps: learning
representation and verification of the input signature. For the first step, the
signature images are fed into the trained Residual CNNs. The output
representations are then used to train SVMs for the verification. We test our
OSV system on three different signature datasets, including MCYT (a Spanish
signature dataset), UTSig (a Persian one) and GPDS-Synthetic (an artificial
dataset). On UT-SIG, we achieved 9.80% Equal Error Rate (EER) which showed
substantial improvement over the best EER in the literature, 17.45%. Our
proposed method surpassed state-of-the-arts by 6% on GPDS-Synthetic, achieving
6.81%. On MCYT, EER of 3.98% was obtained which is comparable to the best
previously reported results.
Active Transfer Learning for Persian Offline Signature Verification
Authors: Taraneh Younesian, Saeed Masoudnia, Reshad Hosseini, Babak N. Araabi
Date: 2019-02-28 00:00:00
Tasks: Active Learning, Transfer Learning

Offline Signature Verification (OSV) remains a challenging pattern
recognition task, especially in the presence of skilled forgeries that are not
available during the training. This challenge is aggravated when there are
small labeled training data available but with large intra-personal variations.
In this study, we address this issue by employing an active learning approach,
which selects the most informative instances to label and therefore reduces the
human labeling effort significantly. Our proposed OSV includes three steps:
feature learning, active learning, and final verification. We benefit from
transfer learning using a pre-trained CNN for feature learning. We also propose
SVM-based active learning for each user to separate his genuine signatures from
the random forgeries. We finally used the SVMs to verify the authenticity of
the questioned signature. We examined our proposed active transfer learning
method on UTSig: A Persian offline signature dataset. We achieved near 13%
improvement compared to the random selection of instances. Our results also
showed 1% improvement over the state-of-the-art method in which a fully
supervised setting with five more labeled instances per user was used.
Hyperspectral Image Classification with Deep Metric Learning and Conditional Random Field
Authors: Yi Liang, Xin Zhao, Alan J. X. Guo, Fei Zhu
Date: 2019-03-04 00:00:00
Tasks: General Classification, Hyperspectral Image Classification, image-classification, Image Classification, Image Segmentation, Metric Learning, Semantic Segmentation

To improve the classification performance in the context of hyperspectral image processing, many works have been developed based on two common strategies, namely the spatial-spectral information integration and the utilization of neural networks. However, both strategies typically require more training data than the classical algorithms, aggregating the shortage of labeled samples. In this letter, we propose a novel framework that organically combines the spectrum-based deep metric learning model and the conditional random field algorithm. The deep metric learning model is supervised by the center loss to produce spectrum-based features that gather more tightly in Euclidean space within classes. The conditional random field with Gaussian edge potentials, which is firstly proposed for image segmentation tasks, is introduced to give the pixel-wise classification over the hyperspectral image by utilizing both the geographical distances between pixels and the Euclidean distances between the features produced by the deep metric learning model. The proposed framework is trained by spectral pixels at the deep metric learning stage and utilizes the half handcrafted spatial features at the conditional random field stage. This settlement alleviates the shortage of training data to some extent. Experiments on two real hyperspectral images demonstrate the advantages of the proposed method in terms of both classification accuracy and computation cost.
Conditional GANs For Painting Generation
Authors: Adeel Mufti, Biagio Antonelli, Julius Monello
Date: 2019-03-06 00:00:00

We examined the use of modern Generative Adversarial Nets to generate novel
images of oil paintings using the Painter By Numbers dataset. We implemented
Spectral Normalization GAN (SN-GAN) and Spectral Normalization GAN with
Gradient Penalty, and compared their outputs to a Deep Convolutional GAN.
Visually, and quantitatively according to the Sliced Wasserstein Distance
metric, we determined that the SN-GAN produced paintings that were most
comparable to our training dataset. We then performed a series of experiments
to add supervised conditioning to SN-GAN, the culmination of which is what we
believe to be a novel architecture that can generate face paintings with
user-specified characteristics.
Mixture Modeling of Global Shape Priors and Autoencoding Local Intensity Priors for Left Atrium Segmentation
Authors: Tim Sodergren, Riddhish Bhalodia, Ross Whitaker, Joshua Cates, Nassir Marrouche, Shireen Elhabian
Date: 2019-03-06 00:00:00
Tasks: Density Estimation, Image Segmentation, Left Atrium Segmentation, Segmentation, Semantic Segmentation

Difficult image segmentation problems, for instance left atrium MRI, can be
addressed by incorporating shape priors to find solutions that are consistent
with known objects. Nonetheless, a single multivariate Gaussian is not an
adequate model in cases with significant nonlinear shape variation or where the
prior distribution is multimodal. Nonparametric density estimation is more
general, but has a ravenous appetite for training samples and poses serious
challenges in optimization, especially in high dimensional spaces. Here, we
propose a maximum-a-posteriori formulation that relies on a generative image
model by incorporating both local intensity and global shape priors. We use
deep autoencoders to capture the complex intensity distribution while avoiding
the careful selection of hand-crafted features. We formulate the shape prior as
a mixture of Gaussians and learn the corresponding parameters in a
high-dimensional shape space rather than pre-projecting onto a low-dimensional
subspace. In segmentation, we treat the identity of the mixture component as a
latent variable and marginalize it within a generalized
expectation-maximization framework. We present a conditional maximization-based
scheme that alternates between a closed-form solution for component-specific
shape parameters that provides a global update-based optimization strategy, and
an intensity-based energy minimization that translates the global notion of a
nonlinear shape prior into a set of local penalties. We demonstrate our
approach on the left atrial segmentation from gadolinium-enhanced MRI, which is
useful in quantifying the atrial geometry in patients with atrial fibrillation.
Noisy Supervision for Correcting Misaligned Cadaster Maps Without Perfect Ground Truth Data
Authors: Nicolas Girard, Guillaume Charpiat, Yuliya Tarabalka
Date: 2019-03-12 00:00:00

In machine learning the best performance on a certain task is achieved by
fully supervised methods when perfect ground truth labels are available.
However, labels are often noisy, especially in remote sensing where manually
curated public datasets are rare. We study the multi-modal cadaster map
alignment problem for which available annotations are mis-aligned polygons,
resulting in noisy supervision. We subsequently set up a multiple-rounds
training scheme which corrects the ground truth annotations at each round to
better train the model at the next round. We show that it is possible to reduce
the noise of the dataset by iteratively training a better alignment model to
correct the annotation alignment.
Spiking-YOLO: Spiking Neural Network for Energy-Efficient Object Detection
Authors: Seijoon Kim, Seongsik Park, Byunggook Na, Sungroh Yoon
Date: 2019-03-12 00:00:00
Tasks: image-classification, Image Classification, object-detection, Object Detection, Real-Time Object Detection

Over the past decade, deep neural networks (DNNs) have demonstrated remarkable performance in a variety of applications. As we try to solve more advanced problems, increasing demands for computing and power resources has become inevitable. Spiking neural networks (SNNs) have attracted widespread interest as the third-generation of neural networks due to their event-driven and low-powered nature. SNNs, however, are difficult to train, mainly owing to their complex dynamics of neurons and non-differentiable spike operations. Furthermore, their applications have been limited to relatively simple tasks such as image classification. In this study, we investigate the performance degradation of SNNs in a more challenging regression problem (i.e., object detection). Through our in-depth analysis, we introduce two novel methods: channel-wise normalization and signed neuron with imbalanced threshold, both of which provide fast and accurate information transmission for deep SNNs. Consequently, we present a first spiked-based object detection model, called Spiking-YOLO. Our experiments show that Spiking-YOLO achieves remarkable results that are comparable (up to 98%) to those of Tiny YOLO on non-trivial datasets, PASCAL VOC and MS COCO. Furthermore, Spiking-YOLO on a neuromorphic chip consumes approximately 280 times less energy than Tiny YOLO and converges 2.3 to 4 times faster than previous SNN conversion methods.