pwc-archive/papers-with-abstracts huggingface dataset:
Implicit Reparameterization Gradients
Authors: Michael Figurnov, Shakir Mohamed, andriy mnih
Date: 2018-05-22 00:00:00
Proceeding: NeurIPS 2018 12

By providing a simple and efficient way of computing low-variance gradients
of continuous random variables, the reparameterization trick has become the
technique of choice for training a variety of latent variable models. However,
it is not applicable to a number of important continuous distributions. We
introduce an alternative approach to computing reparameterization gradients
based on implicit differentiation and demonstrate its broader applicability by
applying it to Gamma, Beta, Dirichlet, and von Mises distributions, which
cannot be used with the classic reparameterization trick. Our experiments show
that the proposed approach is faster and more accurate than the existing
gradient estimators for these distributions.
Natural Language Generation for Electronic Health Records
Authors: Scott Lee
Date: 2018-06-01 00:00:00
Tasks: Decoder, De-identification, Machine Translation, Text Generation, Translation

A variety of methods existing for generating synthetic electronic health
records (EHRs), but they are not capable of generating unstructured text, like
emergency department (ED) chief complaints, history of present illness or
progress notes. Here, we use the encoder-decoder model, a deep learning
algorithm that features in many contemporary machine translation systems, to
generate synthetic chief complaints from discrete variables in EHRs, like age
group, gender, and discharge diagnosis. After being trained end-to-end on
authentic records, the model can generate realistic chief complaint text that
preserves much of the epidemiological information in the original data. As a
side effect of the model's optimization goal, these synthetic chief complaints
are also free of relatively uncommon abbreviation and misspellings, and they
include none of the personally-identifiable information (PII) that was in the
training data, suggesting it may be used to support the de-identification of
text in EHRs. When combined with algorithms like generative adversarial
networks (GANs), our model could be used to generate fully-synthetic EHRs,
facilitating data sharing between healthcare providers and researchers and
improving our ability to develop machine learning methods tailored to the
information in healthcare data.
Artificial Immune Systems Can Find Arbitrarily Good Approximations for the NP-Hard Number Partitioning Problem
Authors: Dogan Corus, Pietro S. Oliveto, Donya Yazdani
Date: 2018-06-01 00:00:00
Tasks: Evolutionary Algorithms

Typical artificial immune system (AIS) operators such as hypermutations with
mutation potential and ageing allow to efficiently overcome local optima from
which evolutionary algorithms (EAs) struggle to escape. Such behaviour has been
shown for artificial example functions constructed especially to show
difficulties that EAs may encounter during the optimisation process.
{\color{black}However, no evidence is available indicating that these two
operators have similar behaviour also in more realistic problems.} In this
paper we perform an analysis for the standard NP-hard \partition problem from
combinatorial optimisation and rigorously show that hypermutations and ageing
allow AISs to efficiently escape from local optima where standard EAs require
exponential time. As a result we prove that while EAs and random local search
(RLS) may get trapped on 4/3 approximations, AISs find arbitrarily good
approximate solutions of ratio (1+$\epsilon$) {\color{black}within $n(\epsilon
^{-(2/\epsilon)-1})(1-\epsilon)^{-2} e^{3} 2^{2/\epsilon} + 2n^3 2^{2/\epsilon}
+ 2n^3$ function evaluations in expectation. This expectation is polynomial in
the problem size and exponential only in $1/\epsilon$}.
Fast Artificial Immune Systems
Authors: Dogan Corus, Pietro S. Oliveto, Donya Yazdani
Date: 2018-06-01 00:00:00
Tasks: Evolutionary Algorithms

Various studies have shown that characteristic Artificial Immune System (AIS)
operators such as hypermutations and ageing can be very efficient at escaping
local optima of multimodal optimisation problems. However, this efficiency
comes at the expense of considerably slower runtimes during the exploitation
phase compared to standard evolutionary algorithms. We propose modifications to
the traditional `hypermutations with mutation potential' (HMP) that allow them
to be efficient at exploitation as well as maintaining their effective
explorative characteristics. Rather than deterministically evaluating fitness
after each bitflip of a hypermutation, we sample the fitness function
stochastically with a `parabolic' distribution which allows the `stop at first
constructive mutation' (FCM) variant of HMP to reduce the linear amount of
wasted function evaluations when no improvement is found to a constant. By
returning the best sampled solution during the hypermutation, rather than the
first constructive mutation, we then turn the extremely inefficient HMP
operator without FCM, into a very effective operator for the standard Opt-IA
AIS using hypermutation, cloning and ageing. We rigorously prove the
effectiveness of the two proposed operators by analysing them on all problems
where the performance of HPM is rigorously understood in the literature. %
Automatic Detection of Neurons in NeuN-stained Histological Images of Human Brain
Authors: Andrija Štajduhar, Domagoj Džaja, Miloš Judaš, Sven Lončarić
Date: 2018-06-01 00:00:00

In this paper, we present a novel use of an anisotropic diffusion model for
automatic detection of neurons in histological sections of the adult human
brain cortex. We use a partial differential equation model to process high
resolution images to acquire locations of neuronal bodies. We also present a
novel approach in model training and evaluation that considers variability
among the human experts, addressing the issue of existence and correctness of
the golden standard for neuron and cell counting, used in most of relevant
papers. Our method, trained on dataset manually labeled by three experts, has
correctly distinguished over 95% of neuron bodies in test data, doing so in
time much shorter than other comparable methods.
Sparse Pursuit and Dictionary Learning for Blind Source Separation in Polyphonic Music Recordings
Authors: Sören Schulze, Emily J. King
Date: 2018-06-01 00:00:00
Tasks: blind source separation, Dictionary Learning

We propose an algorithm for the blind separation of single-channel audio signals. It is based on a parametric model that describes the spectral properties of the sounds of musical instruments independently of pitch. We develop a novel sparse pursuit algorithm that can match the discrete frequency spectra from the recorded signal with the continuous spectra delivered by the model. We first use this algorithm to convert an STFT spectrogram from the recording into a novel form of log-frequency spectrogram whose resolution exceeds that of the mel spectrogram. We then make use of the pitch-invariant properties of that representation in order to identify the sounds of the instruments via the same sparse pursuit method. As the model parameters which characterize the musical instruments are not known beforehand, we train a dictionary that contains them, using a modified version of Adam. Applying the algorithm on various audio samples, we find that it is capable of producing high-quality separation results when the model assumptions are satisfied and the instruments are clearly distinguishable, but combinations of instruments with similar spectral characteristics pose a conceptual difficulty. While a key feature of the model is that it explicitly models inharmonicity, its presence can also still impede performance of the sparse pursuit algorithm. In general, due to its pitch-invariance, our method is especially suitable for dealing with spectra from acoustic instruments, requiring only a minimal number of hyperparameters to be preset. Additionally, we demonstrate that the dictionary that is constructed for one recording can be applied to a different recording with similar instruments without additional training.
Generative Modeling by Inclusive Neural Random Fields with Applications in Image Generation and Anomaly Detection
Authors: Yunfu Song, Zhijian Ou
Date: 2018-06-01 00:00:00
Tasks: Anomaly Detection, Image Generation

Neural random fields (NRFs), referring to a class of generative models that use neural networks to implement potential functions in random fields (a.k.a. energy-based models), are not new but receive less attention with slow progress. Different from various directed graphical models such as generative adversarial networks (GANs), NRFs provide an interesting family of undirected graphical models for generative modeling. In this paper we propose a new approach, the inclusive-NRF approach, to learning NRFs for continuous data (e.g. images), by introducing inclusive-divergence minimized auxiliary generators and developing stochastic gradient sampling in an augmented space. Based on the new approach, specific inclusive-NRF models are developed and thoroughly evaluated in two important generative modeling applications - image generation and anomaly detection. The proposed models consistently improve over state-of-the-art results in both applications. Remarkably, in addition to superior sample generation, one additional benefit of our inclusive-NRF approach is that, unlike GANs, it can directly provide (unnormalized) density estimate for sample evaluation. With these contributions and results, this paper significantly advances the learning and applications of NRFs to a new level, both theoretically and empirically, which have never been obtained before.
Active Learning for Convolutional Neural Networks: A Core-Set Approach
Authors: Ozan Sener, Silvio Savarese
Date: 2017-08-01 00:00:00
Proceeding: ICLR 2018 1
Tasks: Active Learning, image-classification, Image Classification

Convolutional neural networks (CNNs) have been successfully applied to many
recognition and learning tasks using a universal recipe; training a deep model
on a very large dataset of supervised examples. However, this approach is
rather restrictive in practice since collecting a large set of labeled images
is very expensive. One way to ease this problem is coming up with smart ways
for choosing images to be labelled from a very large collection (ie. active
learning).
  Our empirical study suggests that many of the active learning heuristics in
the literature are not effective when applied to CNNs in batch setting.
Inspired by these limitations, we define the problem of active learning as
core-set selection, ie. choosing set of points such that a model learned over
the selected subset is competitive for the remaining data points. We further
present a theoretical result characterizing the performance of any selected
subset using the geometry of the datapoints. As an active learning algorithm,
we choose the subset which is expected to yield best result according to our
characterization. Our experiments show that the proposed method significantly
outperforms existing approaches in image classification experiments by a large
margin.
Learn the new, keep the old: Extending pretrained models with new anatomy and images
Authors: Firat Ozdemir, Philipp Fuernstahl, Orcun Goksel
Date: 2018-06-01 00:00:00
Tasks: Anatomy, Image Segmentation, Incremental Learning, Medical Image Segmentation, Segmentation, Semantic Segmentation

Deep learning has been widely accepted as a promising solution for medical
image segmentation, given a sufficiently large representative dataset of images
with corresponding annotations. With ever increasing amounts of annotated
medical datasets, it is infeasible to train a learning method always with all
data from scratch. This is also doomed to hit computational limits, e.g.,
memory or runtime feasible for training. Incremental learning can be a
potential solution, where new information (images or anatomy) is introduced
iteratively. Nevertheless, for the preservation of the collective information,
it is essential to keep some "important" (i.e. representative) images and
annotations from the past, while adding new information. In this paper, we
introduce a framework for applying incremental learning for segmentation and
propose novel methods for selecting representative data therein. We
comparatively evaluate our methods in different scenarios using MR images and
validate the increased learning capacity with using our methods.
Combining Pyramid Pooling and Attention Mechanism for Pelvic MR Image Semantic Segmentaion
Authors: Ting-Ting Liang, Satoshi Tsutsui, Liangcai Gao, Jing-Jing Lu, Mengyan Sun
Date: 2018-06-01 00:00:00
Tasks: Data Augmentation, Image Segmentation, Segmentation, Semantic Segmentation

One of the time-consuming routine work for a radiologist is to discern
anatomical structures from tomographic images. For assisting radiologists, this
paper develops an automatic segmentation method for pelvic magnetic resonance
(MR) images. The task has three major challenges 1) A pelvic organ can have
various sizes and shapes depending on the axial image, which requires local
contexts to segment correctly. 2) Different organs often have quite similar
appearance in MR images, which requires global context to segment. 3) The
number of available annotated images are very small to use the latest
segmentation algorithms. To address the challenges, we propose a novel
convolutional neural network called Attention-Pyramid network (APNet) that
effectively exploits both local and global contexts, in addition to a
data-augmentation technique that is particularly effective for MR images. In
order to evaluate our method, we construct fine-grained (50 pelvic organs) MR
image segmentation dataset, and experimentally confirm the superior performance
of our techniques over the state-of-the-art image segmentation methods.
Adaptation to criticality through organizational invariance in embodied agents
Authors: Miguel Aguilera, Manuel G. Bedia
Date: 2017-12-13 00:00:00
Tasks: Acrobot, Reinforcement Learning

Many biological and cognitive systems do not operate deep within one or other
regime of activity. Instead, they are poised at critical points located at
phase transitions in their parameter space. The pervasiveness of criticality
suggests that there may be general principles inducing this behaviour, yet
there is no well-founded theory for understanding how criticality is generated
at a wide span of levels and contexts. In order to explore how criticality
might emerge from general adaptive mechanisms, we propose a simple learning
rule that maintains an internal organizational structure from a specific family
of systems at criticality. We implement the mechanism in artificial embodied
agents controlled by a neural network maintaining a correlation structure
randomly sampled from an Ising model at critical temperature. Agents are
evaluated in two classical reinforcement learning scenarios: the Mountain Car
and the Acrobot double pendulum. In both cases the neural controller appears to
reach a point of criticality, which coincides with a transition point between
two regimes of the agent's behaviour. These results suggest that adaptation to
criticality could be used as a general adaptive mechanism in some
circumstances, providing an alternative explanation for the pervasive presence
of criticality in biological and cognitive systems.
A Survey of Domain Adaptation for Neural Machine Translation
Authors: Chenhui Chu, Rui Wang
Date: 2018-06-01 00:00:00
Proceeding: COLING 2018 8
Tasks: Domain Adaptation, Machine Translation, NMT, Survey, Translation

Neural machine translation (NMT) is a deep learning based approach for
machine translation, which yields the state-of-the-art translation performance
in scenarios where large-scale parallel corpora are available. Although the
high-quality and domain-specific translation is crucial in the real world,
domain-specific corpora are usually scarce or nonexistent, and thus vanilla NMT
performs poorly in such scenarios. Domain adaptation that leverages both
out-of-domain parallel corpora as well as monolingual corpora for in-domain
translation, is very important for domain-specific translation. In this paper,
we give a comprehensive survey of the state-of-the-art domain adaptation
techniques for NMT.
DENSER: Deep Evolutionary Network Structured Representation
Authors: Filipe Assunção, Nuno Lourenço, Penousal Machado, Bernardete Ribeiro
Date: 2018-01-04 00:00:00
Tasks: Data Augmentation, Image Classification

Deep Evolutionary Network Structured Representation (DENSER) is a novel
approach to automatically design Artificial Neural Networks (ANNs) using
Evolutionary Computation. The algorithm not only searches for the best network
topology (e.g., number of layers, type of layers), but also tunes
hyper-parameters, such as, learning parameters or data augmentation parameters.
The automatic design is achieved using a representation with two distinct
levels, where the outer level encodes the general structure of the network,
i.e., the sequence of layers, and the inner level encodes the parameters
associated with each layer. The allowed layers and range of the
hyper-parameters values are defined by means of a human-readable Context-Free
Grammar. DENSER was used to evolve ANNs for CIFAR-10, obtaining an average test
accuracy of 94.13%. The networks evolved for the CIFA--10 are tested on the
MNIST, Fashion-MNIST, and CIFAR-100; the results are highly competitive, and on
the CIFAR-100 we report a test accuracy of 78.75%. To the best of our
knowledge, our CIFAR-100 results are the highest performing models generated by
methods that aim at the automatic design of Convolutional Neural Networks
(CNNs), and are amongst the best for manually designed and fine-tuned CNNs.
Fighting Fake News: Image Splice Detection via Learned Self-Consistency
Authors: Minyoung Huh, Andrew Liu, Andrew Owens, Alexei A. Efros
Date: 2018-05-10 00:00:00
Proceeding: ECCV 2018 9
Tasks: Image Forensics

Advances in photo editing and manipulation tools have made it significantly
easier to create fake imagery. Learning to detect such manipulations, however,
remains a challenging problem due to the lack of sufficient amounts of
manipulated training data. In this paper, we propose a learning algorithm for
detecting visual image manipulations that is trained only using a large dataset
of real photographs. The algorithm uses the automatically recorded photo EXIF
metadata as supervisory signal for training a model to determine whether an
image is self-consistent -- that is, whether its content could have been
produced by a single imaging pipeline. We apply this self-consistency model to
the task of detecting and localizing image splices. The proposed method obtains
state-of-the-art performance on several image forensics benchmarks, despite
never seeing any manipulated images at training. That said, it is merely a step
in the long quest for a truly general purpose visual forensics tool.
TAPAS: Train-less Accuracy Predictor for Architecture Search
Authors: R. Istrate, F. Scheidegger, G. Mariani, D. Nikolopoulos, C. Bekas, A. C. I. Malossi
Date: 2018-06-01 00:00:00
Tasks: GPU, Neural Architecture Search, Reinforcement Learning

In recent years an increasing number of researchers and practitioners have
been suggesting algorithms for large-scale neural network architecture search:
genetic algorithms, reinforcement learning, learning curve extrapolation, and
accuracy predictors. None of them, however, demonstrated high-performance
without training new experiments in the presence of unseen datasets. We propose
a new deep neural network accuracy predictor, that estimates in fractions of a
second classification performance for unseen input datasets, without training.
In contrast to previously proposed approaches, our prediction is not only
calibrated on the topological network information, but also on the
characterization of the dataset-difficulty which allows us to re-tune the
prediction without any training. Our predictor achieves a performance which
exceeds 100 networks per second on a single GPU, thus creating the opportunity
to perform large-scale architecture search within a few minutes. We present
results of two searches performed in 400 seconds on a single GPU. Our best
discovered networks reach 93.67% accuracy for CIFAR-10 and 81.01% for
CIFAR-100, verified by training. These networks are performance competitive
with other automatically discovered state-of-the-art networks however we only
needed a small fraction of the time to solution and computational resources.
Deep Predictive Models in Interactive Music
Authors: Charles P. Martin, Kai Olav Ellefsen, Jim Torresen
Date: 2018-01-31 00:00:00
Tasks: Prediction

Musical performance requires prediction to operate instruments, to perform in
groups and to improvise. In this paper, we investigate how a number of digital
musical instruments (DMIs), including two of our own, have applied predictive
machine learning models that assist users by predicting unknown states of
musical processes. We characterise these predictions as focussed within a
musical instrument, at the level of individual performers, and between members
of an ensemble. These models can connect to existing frameworks for DMI design
and have parallels in the cognitive predictions of human musicians.
  We discuss how recent advances in deep learning highlight the role of
prediction in DMIs, by allowing data-driven predictive models with a long
memory of past states. The systems we review are used to motivate musical
use-cases where prediction is a necessary component, and to highlight a number
of challenges for DMI designers seeking to apply deep predictive models in
interactive music systems of the future.
Generative Adversarial Networks for Unsupervised Object Co-localization
Authors: Junsuk Choe, Joo Hyun Park, Hyunjung Shim
Date: 2018-06-01 00:00:00
Tasks: Diversity, Object, Object Localization

This paper introduces a novel approach for unsupervised object
co-localization using Generative Adversarial Networks (GANs). GAN is a powerful
tool that can implicitly learn unknown data distributions in an unsupervised
manner. From the observation that GAN discriminator is highly influenced by
pixels where objects appear, we analyze the internal layers of discriminator
and visualize the activated pixels. Our important finding is that high image
diversity of GAN, which is a main goal in GAN research, is ironically
disadvantageous for object localization, because such discriminators focus not
only on the target object, but also on the various objects, such as background
objects. Based on extensive evaluations and experimental studies, we show the
image diversity and localization performance have a negative correlation. In
addition, our approach achieves meaningful accuracy for unsupervised object
co-localization using publicly available benchmark datasets, even comparable to
state-of-the-art weakly-supervised approach.