pwc-archive/papers-with-abstracts huggingface dataset:
Compressed Domain Image Classification Using a Dynamic-Rate Neural Network
Authors: Yibo Xu, Weidi Liu, Kevin F. Kelly
Date: 2019-01-28 00:00:00
Tasks: Classification, Compressive Sensing, General Classification, image-classification, Image Classification, Image Reconstruction, object-detection, Object Detection, valid

Compressed domain image classification performs classification directly on compressive measurements acquired from the single-pixel camera, bypassing the image reconstruction step. It is of great importance for extending high-speed object detection and classification beyond the visible spectrum in a cost-effective manner especially for resource-limited platforms. Previous neural network methods require training a dedicated neural network for each different measurement rate (MR), which is costly in computation and storage. In this work, we develop an efficient training scheme that provides a neural network with dynamic-rate property, where a single neural network is capable of classifying over any MR within the range of interest with a given sensing matrix. This training scheme uses only a few selected MRs for training and the trained neural network is valid over the full range of MRs of interest. We demonstrate the performance of the dynamic-rate neural network on datasets of MNIST, CIFAR-10, Fashion-MNIST, COIL-100, and show that it generates approximately equal performance at each MR as that of a single-rate neural network valid only for one MR. Robustness to noise of the dynamic-rate model is also demonstrated. The dynamic-rate training scheme can be regarded as a general approach compatible with different types of sensing matrices, various neural network architectures, and is a valuable step towards wider adoption of compressive inference techniques and other compressive sensing related tasks via neural networks.
Improving Adversarial Robustness of Ensembles with Diversity Training
Authors: Sanjay Kariyappa, Moinuddin K. Qureshi
Date: 2019-01-28 00:00:00
Tasks: Adversarial Robustness, Diversity

Deep Neural Networks are vulnerable to adversarial attacks even in settings
where the attacker has no direct access to the model being attacked. Such
attacks usually rely on the principle of transferability, whereby an attack
crafted on a surrogate model tends to transfer to the target model. We show
that an ensemble of models with misaligned loss gradients can provide an
effective defense against transfer-based attacks. Our key insight is that an
adversarial example is less likely to fool multiple models in the ensemble if
their loss functions do not increase in a correlated fashion. To this end, we
propose Diversity Training, a novel method to train an ensemble of models with
uncorrelated loss functions. We show that our method significantly improves the
adversarial robustness of ensembles and can also be combined with existing
methods to create a stronger defense.
Heartbeat Anomaly Detection using Adversarial Oversampling
Authors: Jefferson L. P. Lima, David Macêdo, Cleber Zanchettin
Date: 2019-01-28 00:00:00
Tasks: Anomaly Detection, Diagnostic, General Classification

Cardiovascular diseases are one of the most common causes of death in the
world. Prevention, knowledge of previous cases in the family, and early
detection is the best strategy to reduce this fact. Different machine learning
approaches to automatic diagnostic are being proposed to this task. As in most
health problems, the imbalance between examples and classes is predominant in
this problem and affects the performance of the automated solution. In this
paper, we address the classification of heartbeats images in different
cardiovascular diseases. We propose a two-dimensional Convolutional Neural
Network for classification after using a InfoGAN architecture for generating
synthetic images to unbalanced classes. We call this proposal Adversarial
Oversampling and compare it with the classical oversampling methods as SMOTE,
ADASYN, and RandomOversampling. The results show that the proposed approach
improves the classifier performance for the minority classes without harming
the performance in the balanced classes.
DeGraF-Flow: Extending DeGraF Features for accurate and efficient sparse-to-dense optical flow estimation
Authors: Felix Stephenson, Toby Breckon, Ioannis Katramados
Date: 2019-01-28 00:00:00
Tasks: Computational Efficiency, GPU, Optical Flow Estimation

Modern optical flow methods make use of salient scene feature points detected and matched within the scene as a basis for sparse-to-dense optical flow estimation. Current feature detectors however either give sparse, non uniform point clouds (resulting in flow inaccuracies) or lack the efficiency for frame-rate real-time applications. In this work we use the novel Dense Gradient Based Features (DeGraF) as the input to a sparse-to-dense optical flow scheme. This consists of three stages: 1) efficient detection of uniformly distributed Dense Gradient Based Features (DeGraF); 2) feature tracking via robust local optical flow; and 3) edge preserving flow interpolation to recover overall dense optical flow. The tunable density and uniformity of DeGraF features yield superior dense optical flow estimation compared to other popular feature detectors within this three stage pipeline. Furthermore, the comparable speed of feature detection also lends itself well to the aim of real-time optical flow recovery. Evaluation on established real-world benchmark datasets show test performance in an autonomous vehicle setting where DeGraF-Flow shows promising results in terms of accuracy with competitive computational efficiency among non-GPU based methods, including a marked increase in speed over the conceptually similar EpicFlow approach.
Lie Group Auto-Encoder
Authors: Liyu Gong, Qiang Cheng
Date: 2019-01-28 00:00:00
Tasks: Decoder

In this paper, we propose an auto-encoder based generative neural network
model whose encoder compresses the inputs into vectors in the tangent space of
a special Lie group manifold: upper triangular positive definite affine
transform matrices (UTDATs). UTDATs are representations of Gaussian
distributions and can straightforwardly generate Gaussian distributed samples.
Therefore, the encoder is trained together with a decoder (generator) which
takes Gaussian distributed latent vectors as input. Compared with related
generative models such as variational auto-encoder, the proposed model
incorporates the information on geometric properties of Gaussian distributions.
As a special case, we derive an exponential mapping layer for diagonal Gaussian
UTDATs which eliminates matrix exponential operator compared with general
exponential mapping in Lie group theory. Moreover, we derive an intrinsic loss
for UTDAT Lie group which can be calculated as l-2 loss in the tangent space.
Furthermore, inspired by the Lie group theory, we propose to use the Lie
algebra vectors rather than the raw parameters (e.g. mean) of Gaussian
distributions as compressed representations of original inputs. Experimental
results verity the effectiveness of the proposed new generative model and the
benefits gained from the Lie group structural information of UTDATs.
OpenHowNet: An Open Sememe-based Lexical Knowledge Base
Authors: Fanchao Qi, Chenghao Yang, Zhiyuan Liu, Qiang Dong, Maosong Sun, Zhendong Dong
Date: 2019-01-28 00:00:00

In this paper, we present an open sememe-based lexical knowledge base
OpenHowNet. Based on well-known HowNet, OpenHowNet comprises three components:
core data which is composed of more than 100 thousand senses annotated with
sememes, OpenHowNet Web which gives a brief introduction to OpenHowNet as well
as provides online exhibition of OpenHowNet information, and OpenHowNet API
which includes several useful APIs such as accessing OpenHowNet core data and
drawing sememe tree structures of senses. In the main text, we first give some
backgrounds including definition of sememe and details of HowNet. And then we
introduce some previous HowNet and sememe-based research works. Last but not
least, we detail the constituents of OpenHowNet and their basic features and
functionalities. Additionally, we briefly make a summary and list some future
works.
TGAN: Deep Tensor Generative Adversarial Nets for Large Image Generation
Authors: Zihan Ding, Xiao-Yang Liu, Miao Yin, Linghe Kong
Date: 2019-01-28 00:00:00
Tasks: Dictionary Learning, Image Generation, Super-Resolution

Deep generative models have been successfully applied to many applications.
However, existing works experience limitations when generating large images
(the literature usually generates small images, e.g. 32 * 32 or 128 * 128). In
this paper, we propose a novel scheme, called deep tensor adversarial
generative nets (TGAN), that generates large high-quality images by exploring
tensor structures. Essentially, the adversarial process of TGAN takes place in
a tensor space. First, we impose tensor structures for concise image
representation, which is superior in capturing the pixel proximity information
and the spatial patterns of elementary objects in images, over the
vectorization preprocess in existing works. Secondly, we propose TGAN that
integrates deep convolutional generative adversarial networks and tensor
super-resolution in a cascading manner, to generate high-quality images from
random distributions. More specifically, we design a tensor super-resolution
process that consists of tensor dictionary learning and tensor coefficients
learning. Finally, on three datasets, the proposed TGAN generates images with
more realistic textures, compared with state-of-the-art adversarial
autoencoders. The size of the generated images is increased by over 8.5 times,
namely 374 * 374 in PASCAL2.
Surrogate Gradient Learning in Spiking Neural Networks
Authors: Emre O. Neftci, Hesham Mostafa, Friedemann Zenke
Date: 2019-01-28 00:00:00

Spiking neural networks are nature's versatile solution to fault-tolerant and energy efficient signal processing. To translate these benefits into hardware, a growing number of neuromorphic spiking neural network processors attempt to emulate biological neural networks. These developments have created an imminent need for methods and tools to enable such systems to solve real-world signal processing problems. Like conventional neural networks, spiking neural networks can be trained on real, domain specific data. However, their training requires overcoming a number of challenges linked to their binary and dynamical nature. This article elucidates step-by-step the problems typically encountered when training spiking neural networks, and guides the reader through the key concepts of synaptic plasticity and data-driven learning in the spiking setting. To that end, it gives an overview of existing approaches and provides an introduction to surrogate gradient methods, specifically, as a particularly flexible and efficient method to overcome the aforementioned challenges.
Designing a Multi-Objective Reward Function for Creating Teams of Robotic Bodyguards Using Deep Reinforcement Learning
Authors: Hassam Ullah Sheikh, Ladislau Bölöni
Date: 2019-01-28 00:00:00
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We are considering a scenario where a team of bodyguard robots provides
physical protection to a VIP in a crowded public space. We use deep
reinforcement learning to learn the policy to be followed by the robots. As the
robot bodyguards need to follow several difficult-to-reconcile goals, we study
several primitive and composite reward functions and their impact on the
overall behavior of the robotic bodyguards.
Inferring Heterogeneous Causal Effects in Presence of Spatial Confounding
Authors: Muhammad Osama, Dave Zachariah, Thomas B. Schön
Date: 2019-01-28 00:00:00

We address the problem of inferring the causal effect of an exposure on an outcome across space, using observational data. The data is possibly subject to unmeasured confounding variables which, in a standard approach, must be adjusted for by estimating a nuisance function. Here we develop a method that eliminates the nuisance function, while mitigating the resulting errors-in-variables. The result is a robust and accurate inference method for spatially varying heterogeneous causal effects. The properties of the method are demonstrated on synthetic as well as real data from Germany and the US.
Testing Conditional Independence in Supervised Learning Algorithms
Authors: David S. Watson, Marvin N. Wright
Date: 2019-01-28 00:00:00
Tasks: Causal Discovery, Model Selection, valid

We propose the conditional predictive impact (CPI), a consistent and unbiased estimator of the association between one or several features and a given outcome, conditional on a reduced feature set. Building on the knockoff framework of Cand\`es et al. (2018), we develop a novel testing procedure that works in conjunction with any valid knockoff sampler, supervised learning algorithm, and loss function. The CPI can be efficiently computed for high-dimensional data without any sparsity constraints. We demonstrate convergence criteria for the CPI and develop statistical inference procedures for evaluating its magnitude, significance, and precision. These tests aid in feature and model selection, extending traditional frequentist and Bayesian techniques to general supervised learning tasks. The CPI may also be applied in causal discovery to identify underlying multivariate graph structures. We test our method using various algorithms, including linear regression, neural networks, random forests, and support vector machines. Empirical results show that the CPI compares favorably to alternative variable importance measures and other nonparametric tests of conditional independence on a diverse array of real and simulated datasets. Simulations confirm that our inference procedures successfully control Type I error and achieve nominal coverage probability. Our method has been implemented in an R package, cpi, which can be downloaded from https://github.com/dswatson/cpi.
Hierarchically Clustered Representation Learning
Authors: Su-Jin Shin, Kyungwoo Song, Il-Chul Moon
Date: 2019-01-28 00:00:00
Proceeding: ICLR 2019 5
Tasks: Clustering, Representation Learning

The joint optimization of representation learning and clustering in the
embedding space has experienced a breakthrough in recent years. In spite of the
advance, clustering with representation learning has been limited to flat-level
categories, which often involves cohesive clustering with a focus on instance
relations. To overcome the limitations of flat clustering, we introduce
hierarchically-clustered representation learning (HCRL), which simultaneously
optimizes representation learning and hierarchical clustering in the embedding
space. Compared with a few prior works, HCRL firstly attempts to consider a
generation of deep embeddings from every component of the hierarchy, not just
leaf components. In addition to obtaining hierarchically clustered embeddings,
we can reconstruct data by the various abstraction levels, infer the intrinsic
hierarchical structure, and learn the level-proportion features. We conducted
evaluations with image and text domains, and our quantitative analyses showed
competent likelihoods and the best accuracies compared with the baselines.
FPSA: A Full System Stack Solution for Reconfigurable ReRAM-based NN Accelerator Architecture
Authors: Yu Ji, Youyang Zhang, Xinfeng Xie, Shuangchen Li, Peiqi Wang, Xing Hu, YouHui Zhang, Yuan Xie
Date: 2019-01-28 00:00:00

Neural Network (NN) accelerators with emerging ReRAM (resistive random access
memory) technologies have been investigated as one of the promising solutions
to address the \textit{memory wall} challenge, due to the unique capability of
\textit{processing-in-memory} within ReRAM-crossbar-based processing elements
(PEs). However, the high efficiency and high density advantages of ReRAM have
not been fully utilized due to the huge communication demands among PEs and the
overhead of peripheral circuits.
  In this paper, we propose a full system stack solution, composed of a
reconfigurable architecture design, Field Programmable Synapse Array (FPSA) and
its software system including neural synthesizer, temporal-to-spatial mapper,
and placement & routing. We highly leverage the software system to make the
hardware design compact and efficient. To satisfy the high-performance
communication demand, we optimize it with a reconfigurable routing architecture
and the placement & routing tool. To improve the computational density, we
greatly simplify the PE circuit with the spiking schema and then adopt neural
synthesizer to enable the high density computation-resources to support
different kinds of NN operations. In addition, we provide spiking memory blocks
(SMBs) and configurable logic blocks (CLBs) in hardware and leverage the
temporal-to-spatial mapper to utilize them to balance the storage and
computation requirements of NN. Owing to the end-to-end software system, we can
efficiently deploy existing deep neural networks to FPSA. Evaluations show
that, compared to one of state-of-the-art ReRAM-based NN accelerators, PRIME,
the computational density of FPSA improves by 31x; for representative NNs, its
inference performance can achieve up to 1000x speedup.
The CM Algorithm for the Maximum Mutual Information Classifications of Unseen Instances
Authors: Chenguang Lu
Date: 2019-01-28 00:00:00

The Maximum Mutual Information (MMI) criterion is different from the Least
Error Rate (LER) criterion. It can reduce failing to report small probability
events. This paper introduces the Channels Matching (CM) algorithm for the MMI
classifications of unseen instances. It also introduces some semantic
information methods, which base the CM algorithm. In the CM algorithm, label
learning is to let the semantic channel match the Shannon channel (Matching I)
whereas classifying is to let the Shannon channel match the semantic channel
(Matching II). We can achieve the MMI classifications by repeating Matching I
and II. For low-dimensional feature spaces, we only use parameters to construct
n likelihood functions for n different classes (rather than to construct
partitioning boundaries as gradient descent) and expresses the boundaries by
numerical values. Without searching in parameter spaces, the computation of the
CM algorithm for low-dimensional feature spaces is very simple and fast. Using
a two-dimensional example, we test the speed and reliability of the CM
algorithm by different initial partitions. For most initial partitions, two
iterations can make the mutual information surpass 99% of the convergent MMI.
The analysis indicates that for high-dimensional feature spaces, we may combine
the CM algorithm with neural networks to improve the MMI classifications for
faster and more reliable convergence.
Modularization of End-to-End Learning: Case Study in Arcade Games
Authors: Andrew Melnik, Sascha Fleer, Malte Schilling, Helge Ritter
Date: 2019-01-27 00:00:00
Tasks: Atari Games, Reinforcement Learning

Complex environments and tasks pose a difficult problem for holistic
end-to-end learning approaches. Decomposition of an environment into
interacting controllable and non-controllable objects allows supervised
learning for non-controllable objects and universal value function approximator
learning for controllable objects. Such decomposition should lead to a shorter
learning time and better generalisation capability. Here, we consider
arcade-game environments as sets of interacting objects (controllable,
non-controllable) and propose a set of functional modules that are specialized
on mastering different types of interactions in a broad range of environments.
The modules utilize regression, supervised learning, and reinforcement learning
algorithms. Results of this case study in different Atari games suggest that
human-level performance can be achieved by a learning agent within a human
amount of game experience (10-15 minutes game time) when a proper decomposition
of an environment or a task is provided. However, automatization of such
decomposition remains a challenging problem. This case study shows how a model
of a causal structure underlying an environment or a task can benefit learning
time and generalization capability of the agent, and argues in favor of
exploiting modular structure in contrast to using pure end-to-end learning
approaches.
Composing Distributed Data-intensive Web Services Using a Flexible Memetic Algorithm
Authors: Soheila Sadeghiram, Hui Ma, Gang Chen
Date: 2019-01-26 00:00:00
Tasks: Service Composition

Web Service Composition (WSC) is a particularly promising application of Web
services, where multiple individual services with specific functionalities are
composed to accomplish a more complex task, which must fulfil functional
requirements and optimise Quality of Service (QoS) attributes, simultaneously.
Additionally, large quantities of data, produced by technological advances,
need to be exchanged between services. Data-intensive Web services, which
manipulate and deal with those data, are of great interest to implement
data-intensive processes, such as distributed Data-intensive Web Service
Composition (DWSC). Researchers have proposed Evolutionary Computing (EC)
fully-automated WSC techniques that meet all the above factors. Some of these
works employed Memetic Algorithms (MAs) to enhance the performance of EC
through increasing its exploitation ability of in searching neighbourhood area
of a solution. However, those works are not efficient or effective. This paper
proposes an MA-based approach to solving the problem of distributed DWSC in an
effective and efficient manner. In particular, we develop an MA that hybridises
EC with a flexible local search technique incorporating distance of services.
An evaluation using benchmark datasets is carried out, comparing existing
state-of-the-art methods. Results show that our proposed method has the highest
quality and an acceptable execution time overall.
A Black-box Attack on Neural Networks Based on Swarm Evolutionary Algorithm
Authors: Xiaolei Liu, Yuheng Luo, Xiaosong Zhang, Qingxin Zhu
Date: 2019-01-26 00:00:00
Tasks: Evolutionary Algorithms

Neural networks play an increasingly important role in the field of machine
learning and are included in many applications in society. Unfortunately,
neural networks suffer from adversarial samples generated to attack them.
However, most of the generation approaches either assume that the attacker has
full knowledge of the neural network model or are limited by the type of
attacked model. In this paper, we propose a new approach that generates a
black-box attack to neural networks based on the swarm evolutionary algorithm.
Benefiting from the improvements in the technology and theoretical
characteristics of evolutionary algorithms, our approach has the advantages of
effectiveness, black-box attack, generality, and randomness. Our experimental
results show that both the MNIST images and the CIFAR-10 images can be
perturbed to successful generate a black-box attack with 100\% probability on
average. In addition, the proposed attack, which is successful on distilled
neural networks with almost 100\% probability, is resistant to defensive
distillation. The experimental results also indicate that the robustness of the
artificial intelligence algorithm is related to the complexity of the model and
the data set. In addition, we find that the adversarial samples to some extent
reproduce the characteristics of the sample data learned by the neural network
model.