pwc-archive/papers-with-abstracts huggingface dataset:
A representer theorem for deep kernel learning
Authors: Bastian Bohn, Michael Griebel, Christian Rieger
Date: 2017-09-29 00:00:00
Tasks: BIG-bench Machine Learning

In this paper we provide a finite-sample and an infinite-sample representer
theorem for the concatenation of (linear combinations of) kernel functions of
reproducing kernel Hilbert spaces. These results serve as mathematical
foundation for the analysis of machine learning algorithms based on
compositions of functions. As a direct consequence in the finite-sample case,
the corresponding infinite-dimensional minimization problems can be recast into
(nonlinear) finite-dimensional minimization problems, which can be tackled with
nonlinear optimization algorithms. Moreover, we show how concatenated machine
learning problems can be reformulated as neural networks and how our
representer theorem applies to a broad class of state-of-the-art deep learning
methods.
CondenseNet: An Efficient DenseNet using Learned Group Convolutions
Authors: Gao Huang, Shichen Liu, Laurens van der Maaten, Kilian Q. Weinberger
Date: 2017-11-25 00:00:00
Proceeding: CVPR 2018 6

Deep neural networks are increasingly used on mobile devices, where
computational resources are limited. In this paper we develop CondenseNet, a
novel network architecture with unprecedented efficiency. It combines dense
connectivity with a novel module called learned group convolution. The dense
connectivity facilitates feature re-use in the network, whereas learned group
convolutions remove connections between layers for which this feature re-use is
superfluous. At test time, our model can be implemented using standard group
convolutions, allowing for efficient computation in practice. Our experiments
show that CondenseNets are far more efficient than state-of-the-art compact
convolutional networks such as MobileNets and ShuffleNets.
A Study of EV BMS Cyber Security Based on Neural Network SOC Prediction
Authors: Syed Rahman, Haneen Aburub, Yemeserach Mekonnen, Arif I. Sarwat
Date: 2018-06-07 00:00:00

Recent changes to greenhouse gas emission policies are catalyzing the
electric vehicle (EV) market making it readily accessible to consumers. While
there are challenges that arise with dense deployment of EVs, one of the major
future concerns is cyber security threat. In this paper, cyber security threats
in the form of tampering with EV battery's State of Charge (SOC) was explored.
A Back Propagation (BP) Neural Network (NN) was trained and tested based on
experimental data to estimate SOC of battery under normal operation and
cyber-attack scenarios. NeuralWare software was used to run scenarios.
Different statistic metrics of the predicted values were compared against the
actual values of the specific battery tested to measure the stability and
accuracy of the proposed BP network under different operating conditions. The
results showed that BP NN was able to capture and detect the false entries due
to a cyber-attack on its network.
SBNet: Sparse Blocks Network for Fast Inference
Authors: Mengye Ren, Andrei Pokrovsky, Bin Yang, Raquel Urtasun
Date: 2018-01-07 00:00:00
Proceeding: CVPR 2018 6
Tasks: 3D Object Detection, Object, object-detection, Object Detection, Semantic Segmentation

Conventional deep convolutional neural networks (CNNs) apply convolution
operators uniformly in space across all feature maps for hundreds of layers -
this incurs a high computational cost for real-time applications. For many
problems such as object detection and semantic segmentation, we are able to
obtain a low-cost computation mask, either from a priori problem knowledge, or
from a low-resolution segmentation network. We show that such computation masks
can be used to reduce computation in the high-resolution main network. Variants
of sparse activation CNNs have previously been explored on small-scale tasks
and showed no degradation in terms of object classification accuracy, but often
measured gains in terms of theoretical FLOPs without realizing a practical
speed-up when compared to highly optimized dense convolution implementations.
In this work, we leverage the sparsity structure of computation masks and
propose a novel tiling-based sparse convolution algorithm. We verified the
effectiveness of our sparse CNN on LiDAR-based 3D object detection, and we
report significant wall-clock speed-ups compared to dense convolution without
noticeable loss of accuracy.
Randomized Optimal Transport on a Graph: framework and new distance measures
Authors: Guillaume Guex, Ilkka Kivimäki, Marco Saerens
Date: 2018-06-07 00:00:00
Tasks: Clustering

The recently developed bag-of-paths (BoP) framework consists in setting a
Gibbs-Boltzmann distribution on all feasible paths of a graph. This probability
distribution favors short paths over long ones, with a free parameter (the
temperature $T$) controlling the entropic level of the distribution. This
formalism enables the computation of new distances or dissimilarities,
interpolating between the shortest-path and the resistance distance, which have
been shown to perform well in clustering and classification tasks. In this
work, the bag-of-paths formalism is extended by adding two independent equality
constraints fixing starting and ending nodes distributions of paths (margins).
When the temperature is low, this formalism is shown to be equivalent to a
relaxation of the optimal transport problem on a network where paths carry a
flow between two discrete distributions on nodes. The randomization is achieved
by considering free energy minimization instead of traditional cost
minimization. Algorithms computing the optimal free energy solution are
developed for two types of paths: hitting (or absorbing) paths and non-hitting,
regular, paths, and require the inversion of an $n \times n$ matrix with $n$
being the number of nodes. Interestingly, for regular paths on an undirected
graph, the resulting optimal policy interpolates between the deterministic
optimal transport policy ($T \rightarrow 0^{+}$) and the solution to the
corresponding electrical circuit ($T \rightarrow \infty$). Two distance
measures between nodes and a dissimilarity between groups of nodes, both
integrating weights on nodes, are derived from this framework.
Multi-Scale Dense Networks for Resource Efficient Image Classification
Authors: Gao Huang, Danlu Chen, Tianhong Li, Felix Wu, Laurens van der Maaten, Kilian Q. Weinberger
Date: 2017-03-29 00:00:00
Proceeding: ICLR 2018 1
Tasks: General Classification, Handwritten Mathmatical Expression Recognition, image-classification, Image Classification

In this paper we investigate image classification with computational resource
limits at test time. Two such settings are: 1. anytime classification, where
the network's prediction for a test example is progressively updated,
facilitating the output of a prediction at any time; and 2. budgeted batch
classification, where a fixed amount of computation is available to classify a
set of examples that can be spent unevenly across "easier" and "harder" inputs.
In contrast to most prior work, such as the popular Viola and Jones algorithm,
our approach is based on convolutional neural networks. We train multiple
classifiers with varying resource demands, which we adaptively apply during
test time. To maximally re-use computation between the classifiers, we
incorporate them as early-exits into a single deep convolutional neural network
and inter-connect them with dense connectivity. To facilitate high quality
classification early on, we use a two-dimensional multi-scale network
architecture that maintains coarse and fine level features all-throughout the
network. Experiments on three image-classification tasks demonstrate that our
framework substantially improves the existing state-of-the-art in both
settings.
Multiobjective Test Problems with Degenerate Pareto Fronts
Authors: Liangli Zhen, Miqing Li, Ran Cheng, Dezhong Peng, Xin Yao
Date: 2018-06-07 00:00:00
Tasks: Multiobjective Optimization

In multiobjective optimisation, a set of scalable test problems with a variety of features allow researchers to investigate and evaluate the abilities of different optimisation algorithms, and thus can help them to design and develop more effective and efficient approaches. Existing test problem suites mainly focus on situations where all the objectives are fully conflicting with each other. In such cases, an m-objective optimisation problem has an (m-1)-dimensional Pareto front in the objective space. However, in some optimisation problems, there may be unexpected characteristics among objectives, e.g., redundancy. The redundancy of some objectives can lead to the multiobjective problem having a degenerate Pareto front, i.e., the dimension of the Pareto front of the $m$-objective problem be less than (m-1). In this paper, we systematically study degenerate multiobjective problems. We abstract three general characteristics of degenerate problems, which are not formulated and systematically investigated in the literature. Based on these characteristics, we present a set of test problems to support the investigation of multiobjective optimisation algorithms under situations with redundant objectives. To the best of our knowledge, this work is the first one that explicitly formulates these three characteristics of degenerate problems, thus allowing the resulting test problems to be featured by their generality, in contrast to existing test problems designed for specific purposes (e.g., visualisation).
Efficient semantic image segmentation with superpixel pooling
Authors: Mathijs Schuurmans, Maxim Berman, Matthew B. Blaschko
Date: 2018-06-07 00:00:00
Tasks: GPU, Image Segmentation, Semantic Segmentation

In this work, we evaluate the use of superpixel pooling layers in deep
network architectures for semantic segmentation. Superpixel pooling is a
flexible and efficient replacement for other pooling strategies that
incorporates spatial prior information. We propose a simple and efficient
GPU-implementation of the layer and explore several designs for the integration
of the layer into existing network architectures. We provide experimental
results on the IBSR and Cityscapes dataset, demonstrating that superpixel
pooling can be leveraged to consistently increase network accuracy with minimal
computational overhead. Source code is available at
https://github.com/bermanmaxim/superpixPool
First Order Generative Adversarial Networks
Authors: Calvin Seward, Thomas Unterthiner, Urs Bergmann, Nikolay Jetchev, Sepp Hochreiter
Date: 2018-02-13 00:00:00
Proceeding: ICML 2018 7
Tasks: Image Generation, Text Generation

GANs excel at learning high dimensional distributions, but they can update
generator parameters in directions that do not correspond to the steepest
descent direction of the objective. Prominent examples of problematic update
directions include those used in both Goodfellow's original GAN and the
WGAN-GP. To formally describe an optimal update direction, we introduce a
theoretical framework which allows the derivation of requirements on both the
divergence and corresponding method for determining an update direction, with
these requirements guaranteeing unbiased mini-batch updates in the direction of
steepest descent. We propose a novel divergence which approximates the
Wasserstein distance while regularizing the critic's first order information.
Together with an accompanying update direction, this divergence fulfills the
requirements for unbiased steepest descent updates. We verify our method, the
First Order GAN, with image generation on CelebA, LSUN and CIFAR-10 and set a
new state of the art on the One Billion Word language generation task. Code to
reproduce experiments is available.
Does Distributionally Robust Supervised Learning Give Robust Classifiers?
Authors: Weihua Hu, Gang Niu, Issei Sato, Masashi Sugiyama
Date: 2016-11-07 00:00:00
Proceeding: ICML 2018 7
Tasks: BIG-bench Machine Learning, General Classification, Image Classification

Distributionally Robust Supervised Learning (DRSL) is necessary for building
reliable machine learning systems. When machine learning is deployed in the
real world, its performance can be significantly degraded because test data may
follow a different distribution from training data. DRSL with f-divergences
explicitly considers the worst-case distribution shift by minimizing the
adversarially reweighted training loss. In this paper, we analyze this DRSL,
focusing on the classification scenario. Since the DRSL is explicitly
formulated for a distribution shift scenario, we naturally expect it to give a
robust classifier that can aggressively handle shifted distributions. However,
surprisingly, we prove that the DRSL just ends up giving a classifier that
exactly fits the given training distribution, which is too pessimistic. This
pessimism comes from two sources: the particular losses used in classification
and the fact that the variety of distributions to which the DRSL tries to be
robust is too wide. Motivated by our analysis, we propose simple DRSL that
overcomes this pessimism and empirically demonstrate its effectiveness.
Differential Diagnosis for Pancreatic Cysts in CT Scans Using Densely-Connected Convolutional Networks
Authors: Hongwei Li, Kanru Lin, Maximilian Reichert, Lina Xu, Rickmer Braren, Deliang Fu, Roland Schmid, Ji Li, Bjoern Menze, Kuangyu Shi
Date: 2018-06-04 00:00:00

The lethal nature of pancreatic ductal adenocarcinoma (PDAC) calls for early
differential diagnosis of pancreatic cysts, which are identified in up to 16%
of normal subjects, and some of which may develop into PDAC. Previous
computer-aided developments have achieved certain accuracy for classification
on segmented cystic lesions in CT. However, pancreatic cysts have a large
variation in size and shape, and the precise segmentation of them remains
rather challenging, which restricts the computer-aided interpretation of CT
images acquired for differential diagnosis. We propose a computer-aided
framework for early differential diagnosis of pancreatic cysts without
pre-segmenting the lesions using densely-connected convolutional networks
(Dense-Net). The Dense-Net learns high-level features from whole abnormal
pancreas and builds mappings between medical imaging appearance to different
pathological types of pancreatic cysts. To enhance the clinical applicability,
we integrate saliency maps in the framework to assist the physicians to
understand the decision of the deep learning method. The test on a cohort of
206 patients with 4 pathologically confirmed subtypes of pancreatic cysts has
achieved an overall accuracy of 72.8%, which is significantly higher than the
baseline accuracy of 48.1%, which strongly supports the clinical potential of
our developed method.
Semi-Blind Spatially-Variant Deconvolution in Optical Microscopy with Local Point Spread Function Estimation By Use Of Convolutional Neural Networks
Authors: Adrian Shajkofci, Michael Liebling
Date: 2018-03-20 00:00:00
Tasks: regression

We present a semi-blind, spatially-variant deconvolution technique aimed at optical microscopy that combines a local estimation step of the point spread function (PSF) and deconvolution using a spatially variant, regularized Richardson-Lucy algorithm. To find the local PSF map in a computationally tractable way, we train a convolutional neural network to perform regression of an optical parametric model on synthetically blurred image patches. We deconvolved both synthetic and experimentally-acquired data, and achieved an improvement of image SNR of 1.00 dB on average, compared to other deconvolution algorithms.
Semi-Supervised Learning via Compact Latent Space Clustering
Authors: Konstantinos Kamnitsas, Daniel C. Castro, Loic Le Folgoc, Ian Walker, Ryutaro Tanno, Daniel Rueckert, Ben Glocker, Antonio Criminisi, Aditya Nori
Date: 2018-06-07 00:00:00
Proceeding: ICML 2018 7
Tasks: Clustering

We present a novel cost function for semi-supervised learning of neural
networks that encourages compact clustering of the latent space to facilitate
separation. The key idea is to dynamically create a graph over embeddings of
labeled and unlabeled samples of a training batch to capture underlying
structure in feature space, and use label propagation to estimate its high and
low density regions. We then devise a cost function based on Markov chains on
the graph that regularizes the latent space to form a single compact cluster
per class, while avoiding to disturb existing clusters during optimization. We
evaluate our approach on three benchmarks and compare to state-of-the art with
promising results. Our approach combines the benefits of graph-based
regularization with efficient, inductive inference, does not require
modifications to a network architecture, and can thus be easily applied to
existing networks to enable an effective use of unlabeled data.
Attention-based Deep Multiple Instance Learning
Authors: Maximilian Ilse, Jakub M. Tomczak, Max Welling
Date: 2018-02-13 00:00:00
Proceeding: ICML 2018 7
Tasks: Aerial Scene Classification, Multiple Instance Learning

Multiple instance learning (MIL) is a variation of supervised learning where
a single class label is assigned to a bag of instances. In this paper, we state
the MIL problem as learning the Bernoulli distribution of the bag label where
the bag label probability is fully parameterized by neural networks.
Furthermore, we propose a neural network-based permutation-invariant
aggregation operator that corresponds to the attention mechanism. Notably, an
application of the proposed attention-based operator provides insight into the
contribution of each instance to the bag label. We show empirically that our
approach achieves comparable performance to the best MIL methods on benchmark
MIL datasets and it outperforms other methods on a MNIST-based MIL dataset and
two real-life histopathology datasets without sacrificing interpretability.
FaceShop: Deep Sketch-based Face Image Editing
Authors: Tiziano Portenier, Qiyang Hu, Attila Szabó, Siavash Arjomand Bigdeli, Paolo Favaro, Matthias Zwicker
Date: 2018-04-24 00:00:00
Tasks: Image Manipulation

We present a novel system for sketch-based face image editing, enabling users
to edit images intuitively by sketching a few strokes on a region of interest.
Our interface features tools to express a desired image manipulation by
providing both geometry and color constraints as user-drawn strokes. As an
alternative to the direct user input, our proposed system naturally supports a
copy-paste mode, which allows users to edit a given image region by using parts
of another exemplar image without the need of hand-drawn sketching at all. The
proposed interface runs in real-time and facilitates an interactive and
iterative workflow to quickly express the intended edits. Our system is based
on a novel sketch domain and a convolutional neural network trained end-to-end
to automatically learn to render image regions corresponding to the input
strokes. To achieve high quality and semantically consistent results we train
our neural network on two simultaneous tasks, namely image completion and image
translation. To the best of our knowledge, we are the first to combine these
two tasks in a unified framework for interactive image editing. Our results
show that the proposed sketch domain, network architecture, and training
procedure generalize well to real user input and enable high quality synthesis
results without additional post-processing.
Probabilistic AND-OR Attribute Grouping for Zero-Shot Learning
Authors: Yuval Atzmon, Gal Chechik
Date: 2018-06-07 00:00:00
Tasks: Attribute, Zero-Shot Learning

In zero-shot learning (ZSL), a classifier is trained to recognize visual
classes without any image samples. Instead, it is given semantic information
about the class, like a textual description or a set of attributes. Learning
from attributes could benefit from explicitly modeling structure of the
attribute space. Unfortunately, learning of general structure from empirical
samples is hard with typical dataset sizes.
  Here we describe LAGO, a probabilistic model designed to capture natural soft
and-or relations across groups of attributes. We show how this model can be
learned end-to-end with a deep attribute-detection model. The soft group
structure can be learned from data jointly as part of the model, and can also
readily incorporate prior knowledge about groups if available. The soft and-or
structure succeeds to capture meaningful and predictive structures, improving
the accuracy of zero-shot learning on two of three benchmarks.
  Finally, LAGO reveals a unified formulation over two ZSL approaches: DAP
(Lampert et al., 2009) and ESZSL (Romera-Paredes & Torr, 2015). Interestingly,
taking only one singleton group for each attribute, introduces a new
soft-relaxation of DAP, that outperforms DAP by ~40.
Quantization Mimic: Towards Very Tiny CNN for Object Detection
Authors: Yi Wei, Xinyu Pan, Hongwei Qin, Wanli Ouyang, Junjie Yan
Date: 2018-05-06 00:00:00
Proceeding: ECCV 2018 9
Tasks: Object, object-detection, Object Detection, Quantization

In this paper, we propose a simple and general framework for training very
tiny CNNs for object detection. Due to limited representation ability, it is
challenging to train very tiny networks for complicated tasks like detection.
To the best of our knowledge, our method, called Quantization Mimic, is the
first one focusing on very tiny networks. We utilize two types of acceleration
methods: mimic and quantization. Mimic improves the performance of a student
network by transfering knowledge from a teacher network. Quantization converts
a full-precision network to a quantized one without large degradation of
performance. If the teacher network is quantized, the search scope of the
student network will be smaller. Using this feature of the quantization, we
propose Quantization Mimic. It first quantizes the large network, then mimic a
quantized small network. The quantization operation can help student network to
better match the feature maps from teacher network. To evaluate our approach,
we carry out experiments on various popular CNNs including VGG and Resnet, as
well as different detection frameworks including Faster R-CNN and R-FCN.
Experiments on Pascal VOC and WIDER FACE verify that our Quantization Mimic
algorithm can be applied on various settings and outperforms state-of-the-art
model acceleration methods given limited computing resouces.
Scalable Multi-Class Bayesian Support Vector Machines for Structured and Unstructured Data
Authors: Martin Wistuba, Ambrish Rawat
Date: 2018-06-07 00:00:00
Tasks: Active Learning, General Classification, Variational Inference

We introduce a new Bayesian multi-class support vector machine by formulating
a pseudo-likelihood for a multi-class hinge loss in the form of a
location-scale mixture of Gaussians. We derive a variational-inference-based
training objective for gradient-based learning. Additionally, we employ an
inducing point approximation which scales inference to large data sets.
Furthermore, we develop hybrid Bayesian neural networks that combine standard
deep learning components with the proposed model to enable learning for
unstructured data. We provide empirical evidence that our model outperforms the
competitor methods with respect to both training time and accuracy in
classification experiments on 68 structured and two unstructured data sets.
Finally, we highlight the key capability of our model in yielding prediction
uncertainty for classification by demonstrating its effectiveness in the tasks
of large-scale active learning and detection of adversarial images.