pwc-archive/papers-with-abstracts huggingface dataset:
Deep One-Class Classification Using Intra-Class Splitting
Authors: Patrick Schlachter, Yiwen Liao, Bin Yang
Date: 2019-02-04 00:00:00
Tasks: Classification, General Classification, One-Class Classification

This paper introduces a generic method which enables to use conventional deep neural networks as end-to-end one-class classifiers. The method is based on splitting given data from one class into two subsets. In one-class classification, only samples of one normal class are available for training. During inference, a closed and tight decision boundary around the training samples is sought which conventional binary or multi-class neural networks are not able to provide. By splitting data into typical and atypical normal subsets, the proposed method can use a binary loss and defines an auxiliary subnetwork for distance constraints in the latent space. Various experiments on three well-known image datasets showed the effectiveness of the proposed method which outperformed seven baselines and had a better or comparable performance to the state-of-the-art.
Solving Nurse Scheduling Problem Using Constraint Programming Technique
Authors: O. M. Alade, A. O. Amusat
Date: 2019-02-04 00:00:00
Tasks: Management, Scheduling

Staff scheduling is a universal problem that can be encountered in many
organizations, such as call centers, educational institution, industry,
hospital, and any other public services. It is one of the most important
aspects of workforce management strategy and the one that is most prone to
errors or issues as there are many entities should be considered, such as the
staff turnover, employee availability, time between rotations, unusual periods
of activity, and even the last-minute shift changes. The nurse scheduling
problem is a variant of staff scheduling problems which appoints nurses to
shifts as well as rooms per day taking both hard constraints, i.e., hospital
requirements, and soft constraints, i.e., nurse preferences, into account. Most
algorithms used for scheduling problems fall short when it comes to the number
of inputs they can handle. In this paper, constraint programming was developed
to solve the nurse scheduling problem. The developed constraint programming
model was then implemented using python programming language.
Constructing the Matrix Multilayer Perceptron and its Application to the VAE
Authors: Jalil Taghia, Maria Bånkestad, Fredrik Lindsten, Thomas B. Schön
Date: 2019-02-04 00:00:00

Like most learning algorithms, the multilayer perceptrons (MLP) is designed
to learn a vector of parameters from data. However, in certain scenarios we are
interested in learning structured parameters (predictions) in the form of
symmetric positive definite matrices. Here, we introduce a variant of the MLP,
referred to as the matrix MLP, that is specialized at learning symmetric
positive definite matrices. We also present an application of the model within
the context of the variational autoencoder (VAE). Our formulation of the VAE
extends the vanilla formulation to the cases where the recognition and the
generative networks can be from the parametric family of distributions with
dense covariance matrices. Two specific examples are discussed in more detail:
the dense covariance Gaussian and its generalization, the power exponential
distribution. Our new developments are illustrated using both synthetic and
real data.
Unsupervised Clinical Language Translation
Authors: Wei-Hung Weng, Yu-An Chung, Peter Szolovits
Date: 2019-02-04 00:00:00
Tasks: Clinical Language Translation, Machine Translation, Representation Learning, Sentence, Translation, Word Translation

As patients' access to their doctors' clinical notes becomes common, translating professional, clinical jargon to layperson-understandable language is essential to improve patient-clinician communication. Such translation yields better clinical outcomes by enhancing patients' understanding of their own health conditions, and thus improving patients' involvement in their own care. Existing research has used dictionary-based word replacement or definition insertion to approach the need. However, these methods are limited by expert curation, which is hard to scale and has trouble generalizing to unseen datasets that do not share an overlapping vocabulary. In contrast, we approach the clinical word and sentence translation problem in a completely unsupervised manner. We show that a framework using representation learning, bilingual dictionary induction and statistical machine translation yields the best precision at 10 of 0.827 on professional-to-consumer word translation, and mean opinion scores of 4.10 and 4.28 out of 5 for clinical correctness and layperson readability, respectively, on sentence translation. Our fully-unsupervised strategy overcomes the curation problem, and the clinically meaningful evaluation reduces biases from inappropriate evaluators, which are critical in clinical machine learning.
CapStore: Energy-Efficient Design and Management of the On-Chip Memory for CapsuleNet Inference Accelerators
Authors: Alberto Marchisio, Muhammad Abdullah Hanif, Mohammad Taghi Teimoori, Muhammad Shafique
Date: 2019-02-04 00:00:00
Tasks: Management

Deep Neural Networks (DNNs) have been established as the state-of-the-art
algorithm for advanced machine learning applications. Recently, CapsuleNets
have improved the generalization ability, as compared to DNNs, due to their
multi-dimensional capsules. However, they pose high computational and memory
requirements, which makes energy-efficient inference a challenging task. In
this paper, we perform an extensive analysis to demonstrate their key
limitations due to intense memory accesses and large on-chip memory
requirements. To enable efficient CaspuleNet inference accelerators, we propose
a specialized on-chip memory hierarchy which minimizes the off-chip memory
accesses, while efficiently feeding the data to the accelerator. We analyze the
on-chip memory requirements for each memory component of the architecture. By
leveraging this analysis, we propose a methodology to explore different on-chip
memory designs and a power-gating technique to further reduce the energy
consumption, depending upon the utilization across different operations of a
CapsuleNet. Our memory designs can significantly reduce the energy consumption
of the on-chip memory by up to 86%, when compared to a state-of-the-art memory
design. Since the power consumption of the memory elements is the major
contributor in the power breakdown of the CapsuleNet accelerator, as we will
also show in our analyses, the proposed memory design can effectively reduce
the overall energy consumption of the complete CapsuleNet accelerator
architecture.
Theoretical evidence for adversarial robustness through randomization
Authors: Rafael Pinot, Laurent Meunier, Alexandre Araujo, Hisashi Kashima, Florian Yger, Cédric Gouy-Pailler, Jamal Atif
Date: 2019-02-04 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Adversarial Robustness

This paper investigates the theory of robustness against adversarial attacks. It focuses on the family of randomization techniques that consist in injecting noise in the network at inference time. These techniques have proven effective in many contexts, but lack theoretical arguments. We close this gap by presenting a theoretical analysis of these approaches, hence explaining why they perform well in practice. More precisely, we make two new contributions. The first one relates the randomization rate to robustness to adversarial attacks. This result applies for the general family of exponential distributions, and thus extends and unifies the previous approaches. The second contribution consists in devising a new upper bound on the adversarial generalization gap of randomized neural networks. We support our theoretical claims with a set of experiments.
Is Spiking Secure? A Comparative Study on the Security Vulnerabilities of Spiking and Deep Neural Networks
Authors: Alberto Marchisio, Giorgio Nanfa, Faiq Khalid, Muhammad Abdullah Hanif, Maurizio Martina, Muhammad Shafique
Date: 2019-02-04 00:00:00
Tasks: Data Poisoning

Spiking Neural Networks (SNNs) claim to present many advantages in terms of biological plausibility and energy efficiency compared to standard Deep Neural Networks (DNNs). Recent works have shown that DNNs are vulnerable to adversarial attacks, i.e., small perturbations added to the input data can lead to targeted or random misclassifications. In this paper, we aim at investigating the key research question: ``Are SNNs secure?'' Towards this, we perform a comparative study of the security vulnerabilities in SNNs and DNNs w.r.t. the adversarial noise. Afterwards, we propose a novel black-box attack methodology, i.e., without the knowledge of the internal structure of the SNN, which employs a greedy heuristic to automatically generate imperceptible and robust adversarial examples (i.e., attack images) for the given SNN. We perform an in-depth evaluation for a Spiking Deep Belief Network (SDBN) and a DNN having the same number of layers and neurons (to obtain a fair comparison), in order to study the efficiency of our methodology and to understand the differences between SNNs and DNNs w.r.t. the adversarial examples. Our work opens new avenues of research towards the robustness of the SNNs, considering their similarities to the human brain's functionality.
Riemannian adaptive stochastic gradient algorithms on matrix manifolds
Authors: Hiroyuki Kasai, Pratik Jawanpuria, Bamdev Mishra
Date: 2019-02-04 00:00:00

Adaptive stochastic gradient algorithms in the Euclidean space have attracted much attention lately. Such explorations on Riemannian manifolds, on the other hand, are relatively new, limited, and challenging. This is because of the intrinsic non-linear structure of the underlying manifold and the absence of a canonical coordinate system. In machine learning applications, however, most manifolds of interest are represented as matrices with notions of row and column subspaces. In addition, the implicit manifold-related constraints may also lie on such subspaces. For example, the Grassmann manifold is the set of column subspaces. To this end, such a rich structure should not be lost by transforming matrices to just a stack of vectors while developing optimization algorithms on manifolds. We propose novel stochastic gradient algorithms for problems on Riemannian matrix manifolds by adapting the row and column subspaces of gradients. Our algorithms are provably convergent and they achieve the convergence rate of order $\mathcal{O}(\log (T)/\sqrt{T})$, where $T$ is the number of iterations. Our experiments illustrate the efficacy of the proposed algorithms on several applications.
Active Automata Learning with Adaptive Distinguishing Sequences
Authors: Markus Theo Frohme
Date: 2019-02-04 00:00:00

This document investigates the integration of adaptive distinguishing
sequences into the process of active automata learning (AAL). A novel AAL
algorithm "ADT" (adaptive discrimination tree) is developed and presented.
Since the submission of the original thesis, the presented algorithm has been
integrated into LearnLib - an open-source library for active automata learning
- and has been successfully used in related fields of research.
A Unified Framework for Marketing Budget Allocation
Authors: Kui Zhao, Junhao Hua, Ling Yan, Qi Zhang, Huan Xu, Cheng Yang
Date: 2019-02-04 00:00:00
Tasks: Decision Making, Marketing

While marketing budget allocation has been studied for decades in traditional business, nowadays online business brings much more challenges due to the dynamic environment and complex decision-making process. In this paper, we present a novel unified framework for marketing budget allocation. By leveraging abundant data, the proposed data-driven approach can help us to overcome the challenges and make more informed decisions. In our approach, a semi-black-box model is built to forecast the dynamic market response and an efficient optimization method is proposed to solve the complex allocation task. First, the response in each market-segment is forecasted by exploring historical data through a semi-black-box model, where the capability of logit demand curve is enhanced by neural networks. The response model reveals relationship between sales and marketing cost. Based on the learned model, budget allocation is then formulated as an optimization problem, and we design efficient algorithms to solve it in both continuous and discrete settings. Several kinds of business constraints are supported in one unified optimization paradigm, including cost upper bound, profit lower bound, or ROI lower bound. The proposed framework is easy to implement and readily to handle large-scale problems. It has been successfully applied to many scenarios in Alibaba Group. The results of both offline experiments and online A/B testing demonstrate its effectiveness.
The Natural Language of Actions
Authors: Guy Tennenholtz, Shie Mannor
Date: 2019-02-04 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Starcraft, Starcraft II

We introduce Act2Vec, a general framework for learning context-based action representation for Reinforcement Learning. Representing actions in a vector space help reinforcement learning algorithms achieve better performance by grouping similar actions and utilizing relations between different actions. We show how prior knowledge of an environment can be extracted from demonstrations and injected into action vector representations that encode natural compatible behavior. We then use these for augmenting state representations as well as improving function approximation of Q-values. We visualize and test action embeddings in three domains including a drawing task, a high dimensional navigation task, and the large action space domain of StarCraft II.
Dual Path Multi-Scale Fusion Networks with Attention for Crowd Counting
Authors: Liang Zhu, Zhijian Zhao, Chao Lu, Yining Lin, Yao Peng, Tangren Yao
Date: 2019-02-04 00:00:00
Tasks: Crowd Counting

The task of crowd counting in varying density scenes is an extremely
difficult challenge due to large scale variations. In this paper, we propose a
novel dual path multi-scale fusion network architecture with attention
mechanism named SFANet that can perform accurate count estimation as well as
present high-resolution density maps for highly congested crowd scenes. The
proposed SFANet contains two main components: a VGG backbone convolutional
neural network (CNN) as the front-end feature map extractor and a dual path
multi-scale fusion networks as the back-end to generate density map. These dual
path multi-scale fusion networks have the same structure, one path is
responsible for generating attention map by highlighting crowd regions in
images, the other path is responsible for fusing multi-scale features as well
as attention map to generate the final high-quality high-resolution density
maps. SFANet can be easily trained in an end-to-end way by dual path joint
training. We have evaluated our method on four crowd counting datasets
(ShanghaiTech, UCF CC 50, UCSD and UCF-QRNF). The results demonstrate that with
attention mechanism and multi-scale feature fusion, the proposed SFANet
achieves the best performance on all these datasets and generates better
quality density maps compared with other state-of-the-art approaches.
Strategies for Structuring Story Generation
Authors: Angela Fan, Mike Lewis, Yann Dauphin
Date: 2019-02-04 00:00:00
Proceeding: ACL 2019 7
Tasks: Diversity, Story Generation, Text Generation

Writers generally rely on plans or sketches to write long stories, but most current language models generate word by word from left to right. We explore coarse-to-fine models for creating narrative texts of several hundred words, and introduce new models which decompose stories by abstracting over actions and entities. The model first generates the predicate-argument structure of the text, where different mentions of the same entity are marked with placeholder tokens. It then generates a surface realization of the predicate-argument structure, and finally replaces the entity placeholders with context-sensitive names and references. Human judges prefer the stories from our models to a wide range of previous approaches to hierarchical text generation. Extensive analysis shows that our methods can help improve the diversity and coherence of events and entities in generated stories.
2-D Embedding of Large and High-dimensional Data with Minimal Memory and Computational Time Requirements
Authors: Witold Dzwinel, Rafal Wcislo, Stan Matwin
Date: 2019-02-04 00:00:00

In the advent of big data era, interactive visualization of large data sets
consisting of M*10^5+ high-dimensional feature vectors of length N (N ~ 10^3+),
is an indispensable tool for data exploratory analysis. The state-of-the-art
data embedding (DE) methods of N-D data into 2-D (3-D) visually perceptible
space (e.g., based on t-SNE concept) are too demanding computationally to be
efficiently employed for interactive data analytics of large and
high-dimensional datasets. Herein we present a simple method, ivhd (interactive
visualization of high-dimensional data tool), which radically outperforms the
modern data-embedding algorithms in both computational and memory loads, while
retaining high quality of N-D data embedding in 2-D (3-D). We show that DE
problem is equivalent to the nearest neighbor nn-graph visualization, where
only indices of a few nearest neighbors of each data sample has to be known,
and binary distance between data samples -- 0 to the nearest and 1 to the other
samples -- is defined. These improvements reduce the time-complexity and memory
load from O(M log M) to O(M), and ensure minimal O(M) proportionality
coefficient as well. We demonstrate high efficiency, quality and robustness of
ivhd on popular benchmark datasets such as MNIST, 20NG, NORB and RCV1.
Compatible and Diverse Fashion Image Inpainting
Authors: Xintong Han, Zuxuan Wu, Weilin Huang, Matthew R. Scott, Larry S. Davis
Date: 2019-02-04 00:00:00
Tasks: Fashion Synthesis, Image Generation, Image Inpainting

Visual compatibility is critical for fashion analysis, yet is missing in
existing fashion image synthesis systems. In this paper, we propose to
explicitly model visual compatibility through fashion image inpainting. To this
end, we present Fashion Inpainting Networks (FiNet), a two-stage image-to-image
generation framework that is able to perform compatible and diverse inpainting.
Disentangling the generation of shape and appearance to ensure photorealistic
results, our framework consists of a shape generation network and an appearance
generation network. More importantly, for each generation network, we introduce
two encoders interacting with one another to learn latent code in a shared
compatibility space. The latent representations are jointly optimized with the
corresponding generation network to condition the synthesis process,
encouraging a diverse set of generated results that are visually compatible
with existing fashion garments. In addition, our framework is readily extended
to clothing reconstruction and fashion transfer, with impressive results.
Extensive experiments with comparisons with state-of-the-art approaches on
fashion synthesis task quantitatively and qualitatively demonstrate the
effectiveness of our method.
Robust Environmental Sound Recognition with Sparse Key-point Encoding and Efficient Multi-spike Learning
Authors: Qiang Yu, Yanli Yao, Longbiao Wang, Huajin Tang, Jianwu Dang, Kay Chen Tan
Date: 2019-02-04 00:00:00
Tasks: Decision Making

The capability for environmental sound recognition (ESR) can determine the
fitness of individuals in a way to avoid dangers or pursue opportunities when
critical sound events occur. It still remains mysterious about the fundamental
principles of biological systems that result in such a remarkable ability.
Additionally, the practical importance of ESR has attracted an increasing
amount of research attention, but the chaotic and non-stationary difficulties
continue to make it a challenging task. In this study, we propose a spike-based
framework from a more brain-like perspective for the ESR task. Our framework is
a unifying system with a consistent integration of three major functional parts
which are sparse encoding, efficient learning and robust readout. We first
introduce a simple sparse encoding where key-points are used for feature
representation, and demonstrate its generalization to both spike and non-spike
based systems. Then, we evaluate the learning properties of different learning
rules in details with our contributions being added for improvements. Our
results highlight the advantages of the multi-spike learning, providing a
selection reference for various spike-based developments. Finally, we combine
the multi-spike readout with the other parts to form a system for ESR.
Experimental results show that our framework performs the best as compared to
other baseline approaches. In addition, we show that our spike-based framework
has several advantageous characteristics including early decision making, small
dataset acquiring and ongoing dynamic processing. Our framework is the first
attempt to apply the multi-spike characteristic of nervous neurons to ESR. The
outstanding performance of our approach would potentially contribute to draw
more research efforts to push the boundaries of spike-based paradigm to a new
horizon.
Computational Limitations in Robust Classification and Win-Win Results
Authors: Akshay Degwekar, Preetum Nakkiran, Vinod Vaikuntanathan
Date: 2019-02-04 00:00:00
Tasks: Classification, General Classification, Robust classification

We continue the study of statistical/computational tradeoffs in learning robust classifiers, following the recent work of Bubeck, Lee, Price and Razenshteyn who showed examples of classification tasks where (a) an efficient robust classifier exists, in the small-perturbation regime; (b) a non-robust classifier can be learned efficiently; but (c) it is computationally hard to learn a robust classifier, assuming the hardness of factoring large numbers. The question of whether a robust classifier for their task exists in the large perturbation regime seems related to important open questions in computational number theory. In this work, we extend their work in three directions. First, we demonstrate classification tasks where computationally efficient robust classification is impossible, even when computationally unbounded robust classifiers exist. For this, we rely on the existence of average-case hard functions. Second, we show hard-to-robustly-learn classification tasks in the large-perturbation regime. Namely, we show that even though an efficient classifier that is robust to large perturbations exists, it is computationally hard to learn any non-trivial robust classifier. Our first construction relies on the existence of one-way functions, and the second on the hardness of the learning parity with noise problem. In the latter setting, not only does a non-robust classifier exist, but also an efficient algorithm that generates fresh new labeled samples given access to polynomially many training examples (termed as generation by Kearns et. al. (1994)). Third, we show that any such counterexample implies the existence of cryptographic primitives such as one-way functions. This leads us to a win-win scenario: either we can learn an efficient robust classifier, or we can construct new instances of cryptographic primitives.