pwc-archive/papers-with-abstracts huggingface dataset:
CBAM: Convolutional Block Attention Module
Authors: Sanghyun Woo, Jongchan Park, Joon-Young Lee, In So Kweon
Date: 2018-07-17 00:00:00
Proceeding: ECCV 2018 9
Tasks: General Classification, Image Classification, Object Detection

We propose Convolutional Block Attention Module (CBAM), a simple yet
effective attention module for feed-forward convolutional neural networks.
Given an intermediate feature map, our module sequentially infers attention
maps along two separate dimensions, channel and spatial, then the attention
maps are multiplied to the input feature map for adaptive feature refinement.
Because CBAM is a lightweight and general module, it can be integrated into any
CNN architectures seamlessly with negligible overheads and is end-to-end
trainable along with base CNNs. We validate our CBAM through extensive
experiments on ImageNet-1K, MS~COCO detection, and VOC~2007 detection datasets.
Our experiments show consistent improvements in classification and detection
performances with various models, demonstrating the wide applicability of CBAM.
The code and models will be publicly available.
Large-Scale Multi-Domain Belief Tracking with Knowledge Sharing
Authors: Osman Ramadan, Paweł Budzianowski, Milica Gašić
Date: 2018-07-17 00:00:00
Proceeding: ACL 2018 7
Tasks: Multi-domain Dialogue State Tracking, Semantic Similarity, Semantic Textual Similarity

Robust dialogue belief tracking is a key component in maintaining good
quality dialogue systems. The tasks that dialogue systems are trying to solve
are becoming increasingly complex, requiring scalability to multi domain,
semantically rich dialogues. However, most current approaches have difficulty
scaling up with domains because of the dependency of the model parameters on
the dialogue ontology. In this paper, a novel approach is introduced that fully
utilizes semantic similarity between dialogue utterances and the ontology
terms, allowing the information to be shared across domains. The evaluation is
performed on a recently collected multi-domain dialogues dataset, one order of
magnitude larger than currently available corpora. Our model demonstrates great
capability in handling multi-domain dialogues, simultaneously outperforming
existing state-of-the-art models in single-domain dialogue tracking tasks.
BAM: Bottleneck Attention Module
Authors: Jongchan Park, Sanghyun Woo, Joon-Young Lee, In So Kweon
Date: 2018-07-17 00:00:00
Tasks: Neural Architecture Search

Recent advances in deep neural networks have been developed via architecture
search for stronger representational power. In this work, we focus on the
effect of attention in general deep neural networks. We propose a simple and
effective attention module, named Bottleneck Attention Module (BAM), that can
be integrated with any feed-forward convolutional neural networks. Our module
infers an attention map along two separate pathways, channel and spatial. We
place our module at each bottleneck of models where the downsampling of feature
maps occurs. Our module constructs a hierarchical attention at bottlenecks with
a number of parameters and it is trainable in an end-to-end manner jointly with
any feed-forward models. We validate our BAM through extensive experiments on
CIFAR-100, ImageNet-1K, VOC 2007 and MS COCO benchmarks. Our experiments show
consistent improvement in classification and detection performances with
various models, demonstrating the wide applicability of BAM. The code and
models will be publicly available.
Chinese Poetry Generation with Flexible Styles
Authors: Jiyuan Zhang, Dong Wang
Date: 2018-07-17 00:00:00

Research has shown that sequence-to-sequence neural models, particularly
those with the attention mechanism, can successfully generate classical Chinese
poems. However, neural models are not capable of generating poems that match
specific styles, such as the impulsive style of Li Bai, a famous poet in the
Tang Dynasty. This work proposes a memory-augmented neural model to enable the
generation of style-specific poetry. The key idea is a memory structure that
stores how poems with a desired style were generated by humans, and uses
similar fragments to adjust the generation. We demonstrate that the proposed
algorithm generates poems with flexible styles, including styles of a
particular era and an individual poet.
Continuous Assortment Optimization with Logit Choice Probabilities under Incomplete Information
Authors: Yannik Peeters, Arnoud V. den Boer, Michel Mandjes
Date: 2018-07-17 00:00:00
Tasks: Assortment Optimization, Density Estimation

We consider assortment optimization over a continuous spectrum of products represented by the unit interval, where the seller's problem consists of determining the optimal subset of products to offer to potential customers. To describe the relation between assortment and customer choice, we propose a probabilistic choice model that forms the continuous counterpart of the widely studied discrete multinomial logit model. We consider the seller's problem under incomplete information, propose a stochastic-approximation type of policy, and show that its regret -- its performance loss compared to the optimal policy -- is only logarithmic in the time horizon. We complement this result by showing a matching lower bound on the regret of any policy, implying that our policy is asymptotically optimal. We then show that adding a capacity constraint significantly changes the structure of the problem: we construct a policy and show that its regret after $T$ time periods is bounded above by a constant times $T^{2/3}$ (up to a logarithmic term); in addition, we show that the regret of any policy is bounded from below by a positive constant times $T^{2/3}$, so that also in the capacitated case we obtain asymptotic optimality. Numerical illustrations show that our policies outperform or are on par with alternatives.
Automated Treatment Planning in Radiation Therapy using Generative Adversarial Networks
Authors: Rafid Mahmood, Aaron Babier, Andrea McNiven, Adam Diamant, Timothy C. Y. Chan
Date: 2018-07-17 00:00:00
Tasks: Feature Engineering, Generative Adversarial Network

Knowledge-based planning (KBP) is an automated approach to radiation therapy
treatment planning that involves predicting desirable treatment plans before
they are then corrected to deliverable ones. We propose a generative
adversarial network (GAN) approach for predicting desirable 3D dose
distributions that eschews the previous paradigms of site-specific feature
engineering and predicting low-dimensional representations of the plan.
Experiments on a dataset of oropharyngeal cancer patients show that our
approach significantly outperforms previous methods on several clinical
satisfaction criteria and similarity metrics.
Contextual Memory Trees
Authors: Wen Sun, Alina Beygelzimer, Hal Daumé III, John Langford, Paul Mineiro
Date: 2018-07-17 00:00:00
Tasks: General Classification, Image Captioning, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION, Retrieval

We design and study a Contextual Memory Tree (CMT), a learning memory controller that inserts new memories into an experience store of unbounded size. It is designed to efficiently query for memories from that store, supporting logarithmic time insertion and retrieval operations. Hence CMT can be integrated into existing statistical learning algorithms as an augmented memory unit without substantially increasing training and inference computation. Furthermore CMT operates as a reduction to classification, allowing it to benefit from advances in representation or architecture. We demonstrate the efficacy of CMT by augmenting existing multi-class and multi-label classification algorithms with CMT and observe statistical improvement. We also test CMT learning on several image-captioning tasks to demonstrate that it performs computationally better than a simple nearest neighbors memory system while benefitting from reward learning.
Automatic Skin Lesion Segmentation Using Deep Fully Convolutional Networks
Authors: Hongming Xu, Tae Hyun Hwang
Date: 2018-07-17 00:00:00
Tasks: Lesion Segmentation, Segmentation, Skin Lesion Segmentation

This paper summarizes our method and validation results for the ISIC
Challenge 2018 - Skin Lesion Analysis Towards Melanoma Detection - Task 1:
Lesion Segmentation
A Dense CNN approach for skin lesion classification
Authors: Pierluigi Carcagnì, Andrea Cuna, Cosimo Distante
Date: 2018-07-17 00:00:00
Tasks: Classification, General Classification, Lesion Classification, Skin Lesion Classification

This article presents a Deep CNN, based on the DenseNet architecture jointly
with a highly discriminating learning methodology, in order to classify seven
kinds of skin lesions: Melanoma, Melanocytic nevus, Basal cell carcinoma,
Actinic keratosis / Bowen's disease, Benign keratosis, Dermatofibroma, Vascular
lesion. In particular a 61 layers DenseNet, pre-trained on IMAGENET dataset,
has been fine-tuned on ISIC 2018 Task 3 Challenge Dataset exploiting a Center
Loss function.
Are Efficient Deep Representations Learnable?
Authors: Maxwell Nye, Andrew Saxe
Date: 2018-07-17 00:00:00
Tasks: Deep Learning

Many theories of deep learning have shown that a deep network can require
dramatically fewer resources to represent a given function compared to a
shallow network. But a question remains: can these efficient representations be
learned using current deep learning techniques? In this work, we test whether
standard deep learning methods can in fact find the efficient representations
posited by several theories of deep representation. Specifically, we train deep
neural networks to learn two simple functions with known efficient solutions:
the parity function and the fast Fourier transform. We find that using
gradient-based optimization, a deep network does not learn the parity function,
unless initialized very close to a hand-coded exact solution. We also find that
a deep linear neural network does not learn the fast Fourier transform, even in
the best-case scenario of infinite training data, unless the weights are
initialized very close to the exact hand-coded solution. Our results suggest
that not every element of the class of compositional functions can be learned
efficiently by a deep network, and further restrictions are necessary to
understand what functions are both efficiently representable and learnable.
Expressing Linear Orders Requires Exponential-Size DNNFs
Authors: Ronald de Haan
Date: 2018-07-17 00:00:00

We show that any DNNF circuit that expresses the set of linear orders over a set of $n$ candidates must be of size $2^{\Omega(n)}$. Moreover, we show that there exist DNNF circuits of size $2^{O(n)}$ expressing linear orders over $n$ candidates.
Learning to Listen, Read, and Follow: Score Following as a Reinforcement Learning Game
Authors: Matthias Dorfer, Florian Henkel, Gerhard Widmer
Date: 2018-07-17 00:00:00
Tasks: Decision Making, Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Sequential Decision Making

Score following is the process of tracking a musical performance (audio) with
respect to a known symbolic representation (a score). We start this paper by
formulating score following as a multimodal Markov Decision Process, the
mathematical foundation for sequential decision making. Given this formal
definition, we address the score following task with state-of-the-art deep
reinforcement learning (RL) algorithms such as synchronous advantage actor
critic (A2C). In particular, we design multimodal RL agents that simultaneously
learn to listen to music, read the scores from images of sheet music, and
follow the audio along in the sheet, in an end-to-end fashion. All this
behavior is learned entirely from scratch, based on a weak and potentially
delayed reward signal that indicates to the agent how close it is to the
correct position in the score. Besides discussing the theoretical advantages of
this learning paradigm, we show in experiments that it is in fact superior
compared to previously proposed methods for score following in raw sheet music
images.
Confidence Intervals for Testing Disparate Impact in Fair Learning
Authors: Philippe Besse, Eustasio del Barrio, Paula Gordaliza, Jean-Michel Loubes
Date: 2018-07-17 00:00:00
Tasks: BIG-bench Machine Learning

We provide the asymptotic distribution of the major indexes used in the
statistical literature to quantify disparate treatment in machine learning. We
aim at promoting the use of confidence intervals when testing the so-called
group disparate impact. We illustrate on some examples the importance of using
confidence intervals and not a single value.
Lesion Analysis and Diagnosis with Mask-RCNN
Authors: Andrey Sorokin
Date: 2018-07-10 00:00:00
Tasks: Segmentation, Task 2

This project applies Mask R-CNN method to ISIC 2018 challenge tasks: lesion
boundary segmentation (task1), lesion attributes detection (task 2), lesion
diagnosis (task 3), a solution to the latter is using a trained model for task
1 and a simple voting procedure.
IntroVAE: Introspective Variational Autoencoders for Photographic Image Synthesis
Authors: Huaibo Huang, Zhihang Li, Ran He, Zhenan Sun, Tieniu Tan
Date: 2018-07-17 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Image Generation

We present a novel introspective variational autoencoder (IntroVAE) model for
synthesizing high-resolution photographic images. IntroVAE is capable of
self-evaluating the quality of its generated samples and improving itself
accordingly. Its inference and generator models are jointly trained in an
introspective way. On one hand, the generator is required to reconstruct the
input images from the noisy outputs of the inference model as normal VAEs. On
the other hand, the inference model is encouraged to classify between the
generated and real samples while the generator tries to fool it as GANs. These
two famous generative frameworks are integrated in a simple yet efficient
single-stream architecture that can be trained in a single stage. IntroVAE
preserves the advantages of VAEs, such as stable training and nice latent
manifold. Unlike most other hybrid models of VAEs and GANs, IntroVAE requires
no extra discriminators, because the inference model itself serves as a
discriminator to distinguish between the generated and real samples.
Experiments demonstrate that our method produces high-resolution
photo-realistic images (e.g., CELEBA images at \(1024^{2}\)), which are
comparable to or better than the state-of-the-art GANs.
Magnetic Resonance Fingerprinting Reconstruction via Spatiotemporal Convolutional Neural Networks
Authors: Fabian Balsiger, Amaresha Shridhar Konar, Shivaprasad Chikop, Vimal Chandran, Olivier Scheidegger, Sairam Geethanath, Mauricio Reyes
Date: 2018-07-17 00:00:00
Tasks: Magnetic Resonance Fingerprinting

Magnetic resonance fingerprinting (MRF) quantifies multiple nuclear magnetic
resonance parameters in a single and fast acquisition. Standard MRF
reconstructs parametric maps using dictionary matching, which lacks scalability
due to computational inefficiency. We propose to perform MRF map reconstruction
using a spatiotemporal convolutional neural network, which exploits the
relationship between neighboring MRF signal evolutions to replace the
dictionary matching. We evaluate our method on multiparametric brain scans and
compare it to three recent MRF reconstruction approaches. Our method achieves
state-of-the-art reconstruction accuracy and yields qualitatively more
appealing maps compared to other reconstruction methods. In addition, the
reconstruction time is significantly reduced compared to a dictionary-based
approach.
Battery health prediction under generalized conditions using a Gaussian process transition model
Authors: Robert R. Richardson, Michael A. Osborne, David A. Howey
Date: 2018-07-17 00:00:00
Tasks: feature selection

Accurately predicting the future health of batteries is necessary to ensure
reliable operation, minimise maintenance costs, and calculate the value of
energy storage investments. The complex nature of degradation renders
data-driven approaches a promising alternative to mechanistic modelling. This
study predicts the changes in battery capacity over time using a Bayesian
non-parametric approach based on Gaussian process regression. These changes can
be integrated against an arbitrary input sequence to predict capacity fade in a
variety of usage scenarios, forming a generalised health model. The approach
naturally incorporates varying current, voltage and temperature inputs, crucial
for enabling real world application. A key innovation is the feature selection
step, where arbitrary length current, voltage and temperature measurement
vectors are mapped to fixed size feature vectors, enabling them to be
efficiently used as exogenous variables. The approach is demonstrated on the
open-source NASA Randomised Battery Usage Dataset, with data of 26 cells aged
under randomized operational conditions. Using half of the cells for training,
and half for validation, the method is shown to accurately predict non-linear
capacity fade, with a best case normalised root mean square error of 4.3%,
including accurate estimation of prediction uncertainty.
Learning with SGD and Random Features
Authors: Luigi Carratino, Alessandro Rudi, Lorenzo Rosasco
Date: 2018-07-17 00:00:00
Proceeding: NeurIPS 2018 12

Sketching and stochastic gradient methods are arguably the most common
techniques to derive efficient large scale learning algorithms. In this paper,
we investigate their application in the context of nonparametric statistical
learning. More precisely, we study the estimator defined by stochastic gradient
with mini batches and random features. The latter can be seen as form of
nonlinear sketching and used to define approximate kernel methods. The
considered estimator is not explicitly penalized/constrained and regularization
is implicit. Indeed, our study highlights how different parameters, such as
number of features, iterations, step-size and mini-batch size control the
learning properties of the solutions. We do this by deriving optimal finite
sample bounds, under standard assumptions. The obtained results are
corroborated and illustrated by numerical experiments.
Foundations for Restraining Bolts: Reinforcement Learning with LTLf/LDLf restraining specifications
Authors: Giuseppe De Giacomo, Luca Iocchi, Marco Favorito, Fabio Patrizi
Date: 2018-07-17 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In this work we investigate on the concept of "restraining bolt", envisioned in Science Fiction. Specifically we introduce a novel problem in AI. We have two distinct sets of features extracted from the world, one by the agent and one by the authority imposing restraining specifications (the "restraining bolt"). The two sets are apparently unrelated since of interest to independent parties, however they both account for (aspects of) the same world. We consider the case in which the agent is a reinforcement learning agent on the first set of features, while the restraining bolt is specified logically using linear time logic on finite traces LTLf/LDLf over the second set of features. We show formally, and illustrate with examples, that, under general circumstances, the agent can learn while shaping its goals to suitably conform (as much as possible) to the restraining bolt specifications.
Saliency Map Estimation for Omni-Directional Image Considering Prior Distributions
Authors: Tatsuya Suzuki, Takao Yamanaka
Date: 2018-07-17 00:00:00

In recent years, the deep learning techniques have been applied to the
estimation of saliency maps, which represent probability density functions of
fixations when people look at the images. Although the methods of saliency-map
estimation have been actively studied for 2-dimensional planer images, the
methods for omni-directional images to be utilized in virtual environments had
not been studied, until a competition of saliency-map estimation for the
omni-directional images was held in ICME2017. In this paper, novel methods for
estimating saliency maps for the omni-directional images are proposed
considering the properties of prior distributions for fixations in the planar
images and the omni-directional images.
Learning Neuron Non-Linearities with Kernel-Based Deep Neural Networks
Authors: Giuseppe Marra, Dario Zanca, Alessandro Betti, Marco Gori
Date: 2018-07-17 00:00:00
Proceeding: ICLR 2019 5

The effectiveness of deep neural architectures has been widely supported in
terms of both experimental and foundational principles. There is also clear
evidence that the activation function (e.g. the rectifier and the LSTM units)
plays a crucial role in the complexity of learning. Based on this remark, this
paper discusses an optimal selection of the neuron non-linearity in a
functional framework that is inspired from classic regularization arguments. It
is shown that the best activation function is represented by a kernel expansion
in the training set, that can be effectively approximated over an opportune set
of points modeling 1-D clusters. The idea can be naturally extended to
recurrent networks, where the expressiveness of kernel-based activation
functions turns out to be a crucial ingredient to capture long-term
dependencies. We give experimental evidence of this property by a set of
challenging experiments, where we compare the results with neural architectures
based on state of the art LSTM cells.
GeoDesc: Learning Local Descriptors by Integrating Geometry Constraints
Authors: Zixin Luo, Tianwei Shen, Lei Zhou, Siyu Zhu, Runze Zhang, Yao Yao, Tian Fang, Long Quan
Date: 2018-07-17 00:00:00
Proceeding: ECCV 2018 9
Tasks: 3D Reconstruction

Learned local descriptors based on Convolutional Neural Networks (CNNs) have
achieved significant improvements on patch-based benchmarks, whereas not having
demonstrated strong generalization ability on recent benchmarks of image-based
3D reconstruction. In this paper, we mitigate this limitation by proposing a
novel local descriptor learning approach that integrates geometry constraints
from multi-view reconstructions, which benefits the learning process in terms
of data generation, data sampling and loss computation. We refer to the
proposed descriptor as GeoDesc, and demonstrate its superior performance on
various large-scale benchmarks, and in particular show its great success on
challenging reconstruction tasks. Moreover, we provide guidelines towards
practical integration of learned descriptors in Structure-from-Motion (SfM)
pipelines, showing the good trade-off that GeoDesc delivers to 3D
reconstruction tasks between accuracy and efficiency.
PointSeg: Real-Time Semantic Segmentation Based on 3D LiDAR Point Cloud
Authors: Yu-An Wang, Tianyue Shi, Peng Yun, Lei Tai, Ming Liu
Date: 2018-07-17 00:00:00
Tasks: 3D Object Detection, Autonomous Driving, GPU, object-detection, Object Detection, Real-Time Semantic Segmentation, Semantic Segmentation

In this paper, we propose PointSeg, a real-time end-to-end semantic
segmentation method for road-objects based on spherical images. We take the
spherical image, which is transformed from the 3D LiDAR point clouds, as input
of the convolutional neural networks (CNNs) to predict the point-wise semantic
map. To make PointSeg applicable on a mobile system, we build the model based
on the light-weight network, SqueezeNet, with several improvements. It
maintains a good balance between memory cost and prediction performance. Our
model is trained on spherical images and label masks projected from the KITTI
3D object detection dataset. Experiments show that PointSeg can achieve
competitive accuracy with 90fps on a single GPU 1080ti. which makes it quite
compatible for autonomous driving applications.