pwc-archive/papers-with-abstracts huggingface dataset:
Adversarial Sampling for Active Learning
Authors: Christoph Mayer, Radu Timofte
Date: 2018-08-20 00:00:00
Proceeding: ICLR 2019 5
Tasks: Active Learning, General Classification, Multi-Label Classification

This paper proposes asal, a new GAN based active learning method that generates high entropy samples. Instead of directly annotating the synthetic samples, ASAL searches similar samples from the pool and includes them for training. Hence, the quality of new samples is high and annotations are reliable. To the best of our knowledge, ASAL is the first GAN based AL method applicable to multi-class problems that outperforms random sample selection. Another benefit of ASAL is its small run-time complexity (sub-linear) compared to traditional uncertainty sampling (linear). We present a comprehensive set of experiments on multiple traditional data sets and show that ASAL outperforms similar methods and clearly exceeds the established baseline (random sampling). In the discussion section we analyze in which situations ASAL performs best and why it is sometimes hard to outperform random sample selection.
Learning deep representations by mutual information estimation and maximization
Authors: R. Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Phil Bachman, Adam Trischler, Yoshua Bengio
Date: 2018-08-20 00:00:00
Proceeding: ICLR 2019 5
Tasks: General Classification, Mutual Information Estimation, Representation Learning

In this work, we perform unsupervised learning of representations by
maximizing mutual information between an input and the output of a deep neural
network encoder. Importantly, we show that structure matters: incorporating
knowledge about locality of the input to the objective can greatly influence a
representation's suitability for downstream tasks. We further control
characteristics of the representation by matching to a prior distribution
adversarially. Our method, which we call Deep InfoMax (DIM), outperforms a
number of popular unsupervised learning methods and competes with
fully-supervised learning on several classification tasks. DIM opens new
avenues for unsupervised learning of representations and is an important step
towards flexible formulations of representation-learning objectives for
specific end-goals.
Out-of-Distribution Detection using Multiple Semantic Label Representations
Authors: Gabi Shalev, Yossi Adi, Joseph Keshet
Date: 2018-08-20 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Out-of-Distribution Detection

Deep Neural Networks are powerful models that attained remarkable results on
a variety of tasks. These models are shown to be extremely efficient when
training and test data are drawn from the same distribution. However, it is not
clear how a network will act when it is fed with an out-of-distribution
example. In this work, we consider the problem of out-of-distribution detection
in neural networks. We propose to use multiple semantic dense representations
instead of sparse representation as the target label. Specifically, we propose
to use several word representations obtained from different corpora or
architectures as target labels. We evaluated the proposed model on computer
vision, and speech commands detection tasks and compared it to previous
methods. Results suggest that our method compares favorably with previous work.
Besides, we present the efficiency of our approach for detecting wrongly
classified and adversarial examples.
A Hybrid Differential Evolution Approach to Designing Deep Convolutional Neural Networks for Image Classification
Authors: Bin Wang, Yanan sun, Bing Xue, Mengjie Zhang
Date: 2018-08-20 00:00:00
Tasks: General Classification, image-classification, Image Classification

Convolutional Neural Networks (CNNs) have demonstrated their superiority in
image classification, and evolutionary computation (EC) methods have recently
been surging to automatically design the architectures of CNNs to save the
tedious work of manually designing CNNs. In this paper, a new hybrid
differential evolution (DE) algorithm with a newly added crossover operator is
proposed to evolve the architectures of CNNs of any lengths, which is named
DECNN. There are three new ideas in the proposed DECNN method. Firstly, an
existing effective encoding scheme is refined to cater for variable-length CNN
architectures; Secondly, the new mutation and crossover operators are developed
for variable-length DE to optimise the hyperparameters of CNNs; Finally, the
new second crossover is introduced to evolve the depth of the CNN
architectures. The proposed algorithm is tested on six widely-used benchmark
datasets and the results are compared to 12 state-of-the-art methods, which
shows the proposed method is vigorously competitive to the state-of-the-art
algorithms. Furthermore, the proposed method is also compared with a method
using particle swarm optimisation with a similar encoding strategy named IPPSO,
and the proposed DECNN outperforms IPPSO in terms of the accuracy.
Privacy Amplification by Iteration
Authors: Vitaly Feldman, Ilya Mironov, Kunal Talwar, Abhradeep Thakurta
Date: 2018-08-20 00:00:00

Many commonly used learning algorithms work by iteratively updating an
intermediate solution using one or a few data points in each iteration.
Analysis of differential privacy for such algorithms often involves ensuring
privacy of each step and then reasoning about the cumulative privacy cost of
the algorithm. This is enabled by composition theorems for differential privacy
that allow releasing of all the intermediate results. In this work, we
demonstrate that for contractive iterations, not releasing the intermediate
results strongly amplifies the privacy guarantees.
  We describe several applications of this new analysis technique to solving
convex optimization problems via noisy stochastic gradient descent. For
example, we demonstrate that a relatively small number of non-private data
points from the same distribution can be used to close the gap between private
and non-private convex optimization. In addition, we demonstrate that we can
achieve guarantees similar to those obtainable using the
privacy-amplification-by-sampling technique in several natural settings where
that technique cannot be applied.
Inverse Problems in Asteroseismology
Authors: Earl Patrick Bellinger
Date: 2018-08-20 00:00:00

Asteroseismology allows us to probe the internal structure of stars through
their global modes of oscillation. Thanks to missions such as the NASA Kepler
space observatory, we now have high-quality asteroseismic data for nearly 100
solar-type stars. In this thesis, new techniques to measure the ages, masses,
and radii of stars are presented, as well as a way to infer their internal
structure.
Stochastic Combinatorial Ensembles for Defending Against Adversarial Examples
Authors: George A. Adam, Petr Smirnov, David Duvenaud, Benjamin Haibe-Kains, Anna Goldenberg
Date: 2018-08-20 00:00:00
Tasks: Adversarial Attack, Metric Learning

Many deep learning algorithms can be easily fooled with simple adversarial
examples. To address the limitations of existing defenses, we devised a
probabilistic framework that can generate an exponentially large ensemble of
models from a single model with just a linear cost. This framework takes
advantage of neural network depth and stochastically decides whether or not to
insert noise removal operators such as VAEs between layers. We show empirically
the important role that model gradients have when it comes to determining
transferability of adversarial examples, and take advantage of this result to
demonstrate that it is possible to train models with limited adversarial attack
transferability. Additionally, we propose a detection method based on metric
learning in order to detect adversarial examples that have no hope of being
cleaned of maliciously engineered noise.
Adversarial Removal of Demographic Attributes from Text Data
Authors: Yanai Elazar, Yoav Goldberg
Date: 2018-08-20 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Representation Learning

Recent advances in Representation Learning and Adversarial Training seem to
succeed in removing unwanted features from the learned representation. We show
that demographic information of authors is encoded in -- and can be recovered
from -- the intermediate representations learned by text-based neural
classifiers. The implication is that decisions of classifiers trained on
textual data are not agnostic to -- and likely condition on -- demographic
attributes. When attempting to remove such demographic information using
adversarial training, we find that while the adversarial component achieves
chance-level development-set accuracy during training, a post-hoc classifier,
trained on the encoded sentences from the first part, still manages to reach
substantially higher classification accuracies on the same data. This behavior
is consistent across several tasks, demographic properties and datasets. We
explore several techniques to improve the effectiveness of the adversarial
component. Our main conclusion is a cautionary one: do not rely on the
adversarial training to achieve invariant representation to sensitive features.
Supervised Kernel PCA For Longitudinal Data
Authors: Patrick Staples, Min Ouyang, Robert F. Dougherty, Gregory A. Ryslik, Paul Dagum
Date: 2018-08-20 00:00:00
Tasks: Dimensionality Reduction

In statistical learning, high covariate dimensionality poses challenges for
robust prediction and inference. To address this challenge, supervised
dimension reduction is often performed, where dependence on the outcome is
maximized for a selected covariate subspace with smaller dimensionality.
Prevalent dimension reduction techniques assume data are $i.i.d.$, which is not
appropriate for longitudinal data comprising multiple subjects with repeated
measurements over time. In this paper, we derive a decomposition of the
Hilbert-Schmidt Independence Criterion as a supervised loss function for
longitudinal data, enabling dimension reduction between and within clusters
separately, and propose a dimensionality-reduction technique, $sklPCA$, that
performs this decomposed dimension reduction. We also show that this technique
yields superior model accuracy compared to the model it extends.
PACO: Global Signal Restoration via PAtch COnsensus
Authors: Ignacio Francisco Ramírez Paulino
Date: 2018-08-20 00:00:00
Tasks: Denoising, Image Denoising, Image Inpainting

Many signal processing algorithms break the target signal into overlapping segments (also called windows, or patches), process them separately, and then stitch them back into place to produce a unified output. At the overlaps, the final value of those samples that are estimated more than once needs to be decided in some way. Averaging, the simplest approach, often leads to unsatisfactory results. Significant work has been devoted to this issue in recent years. Several works explore the idea of a weighted average of the overlapped patches and/or pixels; others promote agreement (consensus) between the patches at their intersections. Agreement can be either encouraged or imposed as a hard constraint. This work develops on the latter case. The result is a variational signal processing framework, named PACO, which features a number of appealing theoretical and practical properties. The PACO framework consists of a variational formulation that fits a wide variety of problems, and a general ADMMbased algorithm for minimizing the resulting energies. As a byproduct, we show that the consensus step of the algorithm, which is the main bottleneck of similar methods, can be solved efficiently and easily for any arbitrary patch decomposition scheme. We demonstrate the flexibility and power of PACO on three different problems: image inpainting (which we have already covered in previous works), image denoising, and contrast enhancement, using different cost functions including Laplacian and Gaussian Mixture Models.
End to End Vehicle Lateral Control Using a Single Fisheye Camera
Authors: Marin Toromanoff, Emilie Wirbel, Frédéric Wilhelm, Camilo Vejarano, Xavier Perrotton, Fabien Moutarde
Date: 2018-08-20 00:00:00

Convolutional neural networks are commonly used to control the steering angle
for autonomous cars. Most of the time, multiple long range cameras are used to
generate lateral failure cases. In this paper we present a novel model to
generate this data and label augmentation using only one short range fisheye
camera. We present our simulator and how it can be used as a consistent metric
for lateral end-to-end control evaluation. Experiments are conducted on a
custom dataset corresponding to more than 10000 km and 200 hours of open road
driving. Finally we evaluate this model on real world driving scenarios, open
road and a custom test track with challenging obstacle avoidance and sharp
turns. In our simulator based on real-world videos, the final model was capable
of more than 99% autonomy on urban road
Artificial Neural Networks in Fluid Dynamics: A Novel Approach to the Navier-Stokes Equations
Authors: Megan McCracken
Date: 2018-08-19 00:00:00

Neural networks have been used to solve different types of large data related
problems in many different fields.This project takes a novel approach to
solving the Navier-Stokes Equations for turbulence by training a neural network
using Bayesian Cluster and SOM neighbor weighting to map ionospheric velocity
fields based on 3-dimensional inputs. Parameters used in this problem included
the velocity, Reynolds number, Prandtl number, and temperature. In this project
data was obtained from Johns-Hopkins University to train the neural network
using MATLAB. The neural network was able to map the velocity fields within a
sixty-seven percent accuracy of the validation data used. Further studies will
focus on higher accuracy and solving further non-linear differential equations
using convolutional neural networks.
Segmentation of Microscopy Data for finding Nuclei in Divergent Images
Authors: Shivam Singh, Stuti Pathak
Date: 2018-08-19 00:00:00
Tasks: Cell Segmentation, Image Segmentation, Medical Image Segmentation, Segmentation, Semantic Segmentation

Every year millions of people die due to disease of Cancer. Due to its
invasive nature it is very complex to cure even in primary stages. Hence, only
method to survive this disease completely is via forecasting by analyzing the
early mutation in cells of the patient biopsy. Cell Segmentation can be used to
find cell which have left their nuclei. This enables faster cure and high rate
of survival. Cell counting is a hard, yet tedious task that would greatly
benefit from automation. To accomplish this task, segmentation of cells need to
be accurate. In this paper, we have improved the learning of training data by
our network. It can annotate precise masks on test data. we examine the
strength of activation functions in medical image segmentation task by
improving learning rates by our proposed Carving Technique. Identifying the
cells nuclei is the starting point for most analyses, identifying nuclei allows
researchers to identify each individual cell in a sample, and by measuring how
cells react to various treatments, the researcher can understand the underlying
biological processes at work. Experimental results shows the efficiency of the
proposed work.
Linked Recurrent Neural Networks
Authors: Zhiwei Wang, Yao Ma, Dawei Yin, Jiliang Tang
Date: 2018-08-19 00:00:00
Tasks: Document Classification, Machine Translation, speech-recognition, Speech Recognition, Translation

Recurrent Neural Networks (RNNs) have been proven to be effective in modeling
sequential data and they have been applied to boost a variety of tasks such as
document classification, speech recognition and machine translation. Most of
existing RNN models have been designed for sequences assumed to be identically
and independently distributed (i.i.d). However, in many real-world
applications, sequences are naturally linked. For example, web documents are
connected by hyperlinks; and genes interact with each other. On the one hand,
linked sequences are inherently not i.i.d., which poses tremendous challenges
to existing RNN models. On the other hand, linked sequences offer link
information in addition to the sequential information, which enables
unprecedented opportunities to build advanced RNN models. In this paper, we
study the problem of RNN for linked sequences. In particular, we introduce a
principled approach to capture link information and propose a linked Recurrent
Neural Network (LinkedRNN), which models sequential and link information
coherently. We conduct experiments on real-world datasets from multiple domains
and the experimental results validate the effectiveness of the proposed
framework.
Generalized Bregman and Jensen divergences which include some f-divergences
Authors: Tomohiro Nishiyama
Date: 2018-08-19 00:00:00

In this paper, we introduce new classes of divergences by extending the
definitions of the Bregman divergence and the skew Jensen divergence. These new
divergence classes (g-Bregman divergence and skew g-Jensen divergence) satisfy
some properties similar to the Bregman or skew Jensen divergence. We show these
g-divergences include divergences which belong to a class of f-divergence (the
Hellinger distance, the chi-square divergence and the alpha-divergence in
addition to the Kullback-Leibler divergence). Moreover, we derive an inequality
between the g-Bregman divergence and the skew g-Jensen divergence and show this
inequality is a generalization of Lin's inequality.
Network-based Biased Tree Ensembles (NetBiTE) for Drug Sensitivity Prediction and Drug Sensitivity Biomarker Identification in Cancer
Authors: Ali Oskooei, Matteo Manica, Roland Mathis, Maria Rodriguez Martinez
Date: 2018-08-18 00:00:00
Tasks: Sensitivity

We present the Network-based Biased Tree Ensembles (NetBiTE) method for drug
sensitivity prediction and drug sensitivity biomarker identification in cancer
using a combination of prior knowledge and gene expression data. Our devised
method consists of a biased tree ensemble that is built according to a
probabilistic bias weight distribution. The bias weight distribution is
obtained from the assignment of high weights to the drug targets and
propagating the assigned weights over a protein-protein interaction network
such as STRING. The propagation of weights, defines neighborhoods of influence
around the drug targets and as such simulates the spread of perturbations
within the cell, following drug administration. Using a synthetic dataset, we
showcase how application of biased tree ensembles (BiTE) results in significant
accuracy gains at a much lower computational cost compared to the unbiased
random forests (RF) algorithm. We then apply NetBiTE to the Genomics of Drug
Sensitivity in Cancer (GDSC) dataset and demonstrate that NetBiTE outperforms
RF in predicting IC50 drug sensitivity, only for drugs that target membrane
receptor pathways (MRPs): RTK, EGFR and IGFR signaling pathways. We propose
based on the NetBiTE results, that for drugs that inhibit MRPs, the expression
of target genes prior to drug administration is a biomarker for IC50 drug
sensitivity following drug administration. We further verify and reinforce this
proposition through control studies on, PI3K/MTOR signaling pathway inhibitors,
a drug category that does not target MRPs, and through assignment of dummy
targets to MRP inhibiting drugs and investigating the variation in NetBiTE
accuracy.
The Distribution of Reversible Functions is Normal
Authors: W. B. Langdon
Date: 2018-08-18 00:00:00

The distribution of reversible programs tends to a limit as their size
increases. For problems with a Hamming distance fitness function the limiting
distribution is binomial with an exponentially small chance (but non~zero)
chance of perfect solution. Sufficiently good reversible circuits are more
common. Expected RMS error is also calculated. Random unitary matrices may
suggest possible extension to quantum computing. Using the genetic programming
(GP) benchmark, the six multiplexor, circuits of Toffoli gates are shown to
give a fitness landscape amenable to evolutionary search. Minimal CCNOT
solutions to the six multiplexer are found but larger circuits are more
evolvable.
The variable quality of metadata about biological samples used in biomedical experiments
Authors: Rafael S. Gonçalves, Mark A. Musen
Date: 2018-08-17 00:00:00
Tasks: Clustering

We present an analytical study of the quality of metadata about samples used
in biomedical experiments. The metadata under analysis are stored in two
well-known databases: BioSample---a repository managed by the National Center
for Biotechnology Information (NCBI), and BioSamples---a repository managed by
the European Bioinformatics Institute (EBI). We tested whether 11.4M sample
metadata records in the two repositories are populated with values that fulfill
the stated requirements for such values. Our study revealed multiple anomalies
in the metadata. Most metadata field names and their values are not
standardized or controlled. Even simple binary or numeric fields are often
populated with inadequate values of different data types. By clustering
metadata field names, we discovered there are often many distinct ways to
represent the same aspect of a sample. Overall, the metadata we analyzed reveal
that there is a lack of principled mechanisms to enforce and validate metadata
requirements. The significant aberrancies that we found in the metadata are
likely to impede search and secondary use of the associated datasets.
Immunological recognition by artificial neural networks
Authors: Jin Xu, Junghyo Jo
Date: 2018-08-10 00:00:00

The binding affinity between the T-cell receptors (TCRs) and antigenic peptides mainly determines immunological recognition. It is not a trivial task that T cells identify the digital sequences of peptide amino acids by simply relying on the integrated binding affinity between TCRs and antigenic peptides. To address this problem, we examine whether the affinity-based discrimination of peptide sequences is learnable and generalizable by artificial neural networks (ANNs) that process the digital experimental amino acid sequence information of receptors and peptides. A pair of TCR and peptide sequences correspond to the input for ANNs, while the success or failure of the immunological recognition correspond to the output. The output is obtained by both theoretical model and experimental data. In either case, we confirmed that ANNs could learn the immunological recognition. We also found that a homogenized encoding of amino acid sequence was more effective for the supervised learning task.
Stacked Pooling: Improving Crowd Counting by Boosting Scale Invariance
Authors: Siyu Huang, Xi Li, Zhi-Qi Cheng, Zhongfei Zhang, Alexander Hauptmann
Date: 2018-08-22 00:00:00
Tasks: Crowd Counting, Density Estimation

In this work, we explore the cross-scale similarity in crowd counting
scenario, in which the regions of different scales often exhibit high visual
similarity. This feature is universal both within an image and across different
images, indicating the importance of scale invariance of a crowd counting
model. Motivated by this, in this paper we propose simple but effective
variants of pooling module, i.e., multi-kernel pooling and stacked pooling, to
boost the scale invariance of convolutional neural networks (CNNs), benefiting
much the crowd density estimation and counting. Specifically, the multi-kernel
pooling comprises of pooling kernels with multiple receptive fields to capture
the responses at multi-scale local ranges. The stacked pooling is an equivalent
form of multi-kernel pooling, while, it reduces considerable computing cost.
Our proposed pooling modules do not introduce extra parameters into model and
can easily take place of the vanilla pooling layer in implementation. In
empirical study on two benchmark crowd counting datasets, the stacked pooling
beats the vanilla pooling layer in most cases.