pwc-archive/papers-with-abstracts huggingface dataset:
User Loss -- A Forced-Choice-Inspired Approach to Train Neural Networks directly by User Interaction
Authors: Shahab Zarei, Bernhard Stimpel, Christopher Syben, Andreas Maier
Date: 2018-07-24 00:00:00
Tasks: Denoising, Image Denoising, Medical Image Denoising

In this paper, we investigate whether is it possible to train a neural
network directly from user inputs. We consider this approach to be highly
relevant for applications in which the point of optimality is not well-defined
and user-dependent. Our application is medical image denoising which is
essential in fluoroscopy imaging. In this field every user, i.e. physician, has
a different flavor and image quality needs to be tailored towards each
individual.
  To address this important problem, we propose to construct a loss function
derived from a forced-choice experiment. In order to make the learning problem
feasible, we operate in the domain of precision learning, i.e., we inspire the
network architecture by traditional signal processing methods in order to
reduce the number of trainable parameters. The algorithm that was used for this
is a Laplacian pyramid with only six trainable parameters.
  In the experimental results, we demonstrate that two image experts who prefer
different filter characteristics between sharpness and de-noising can be
created using our approach. Also models trained for a specific user perform
best on this users test data. This approach opens the way towards
implementation of direct user feedback in deep learning and is applicable for a
wide range of application.
Ensemble of Multi-sized FCNs to Improve White Matter Lesion Segmentation
Authors: Zhewei Wang, Charles D. Smith, Jundong Liu
Date: 2018-07-24 00:00:00
Tasks: Lesion Segmentation, Segmentation

In this paper, we develop a two-stage neural network solution for the
challenging task of white-matter lesion segmentation. To cope with the vast
vari- ability in lesion sizes, we sample brain MR scans with patches at three
differ- ent dimensions and feed them into separate fully convolutional neural
networks (FCNs). In the second stage, we process large and small lesion
separately, and use ensemble-nets to combine the segmentation results generated
from the FCNs. A novel activation function is adopted in the ensemble-nets to
improve the segmen- tation accuracy measured by Dice Similarity Coefficient.
Experiments on MICCAI 2017 White Matter Hyperintensities (WMH) Segmentation
Challenge data demonstrate that our two-stage-multi-sized FCN approach, as well
as the new activation function, are effective in capturing white-matter lesions
in MR images.
Improved Training with Curriculum GANs
Authors: Rishi Sharma, Shane Barratt, Stefano Ermon, Vijay Pande
Date: 2018-07-24 00:00:00
Tasks: Image Generation

In this paper we introduce Curriculum GANs, a curriculum learning strategy
for training Generative Adversarial Networks that increases the strength of the
discriminator over the course of training, thereby making the learning task
progressively more difficult for the generator. We demonstrate that this
strategy is key to obtaining state-of-the-art results in image generation. We
also show evidence that this strategy may be broadly applicable to improving
GAN training in other data modalities.
Noise Contrastive Priors for Functional Uncertainty
Authors: Danijar Hafner, Dustin Tran, Timothy Lillicrap, Alex Irpan, James Davidson
Date: 2018-07-24 00:00:00
Proceeding: ICLR 2019 5
Tasks: Active Learning

Obtaining reliable uncertainty estimates of neural network predictions is a long standing challenge. Bayesian neural networks have been proposed as a solution, but it remains open how to specify their prior. In particular, the common practice of an independent normal prior in weight space imposes relatively weak constraints on the function posterior, allowing it to generalize in unforeseen ways on inputs outside of the training distribution. We propose noise contrastive priors (NCPs) to obtain reliable uncertainty estimates. The key idea is to train the model to output high uncertainty for data points outside of the training distribution. NCPs do so using an input prior, which adds noise to the inputs of the current mini batch, and an output prior, which is a wide distribution given these inputs. NCPs are compatible with any model that can output uncertainty estimates, are easy to scale, and yield reliable uncertainty estimates throughout training. Empirically, we show that NCPs prevent overfitting outside of the training distribution and result in uncertainty estimates that are useful for active learning. We demonstrate the scalability of our method on the flight delays data set, where we significantly improve upon previously published results.
Convolutional Neural Networks In Classifying Cancer Through DNA Methylation
Authors: Soham Chatterjee, Archana Iyer, Satya Avva, Abhai Kollara, Malaikannan Sankarasubbu
Date: 2018-07-24 00:00:00

DNA Methylation has been the most extensively studied epigenetic mark.
Usually a change in the genotype, DNA sequence, leads to a change in the
phenotype, observable characteristics of the individual. But DNA methylation,
which happens in the context of CpG (cytosine and guanine bases linked by
phosphate backbone) dinucleotides, does not lead to a change in the original
DNA sequence but has the potential to change the phenotype. DNA methylation is
implicated in various biological processes and diseases including cancer. Hence
there is a strong interest in understanding the DNA methylation patterns across
various epigenetic related ailments in order to distinguish and diagnose the
type of disease in its early stages. In this work, the relationship between
methylated versus unmethylated CpG regions and cancer types is explored using
Convolutional Neural Networks (CNNs). A CNN based Deep Learning model that can
classify the cancer of a new DNA methylation profile based on the learning from
publicly available DNA methylation datasets is then proposed.
Improved Adaptive Brovey as a New Method for Image Fusion
Authors: Hamid Reza Shahdoosti
Date: 2018-07-24 00:00:00

An ideal fusion method preserves the Spectral information in fused image and
adds spatial information to it with no spectral distortion. Among the existing
fusion algorithms, the contourlet-based fusion method is the most frequently
discussed one in recent publications, because the contourlet has the ability to
capture and link the point of discontinuities to form a linear structure. The
Brovey is a popular pan-sharpening method owing to its efficiency and high
spatial resolution. This method can be explained by mathematical model of
optical remote sensing sensors. This study presents a new fusion approach that
integrates the advantages of both the Brovey and the cotourlet techniques to
reduce the color distortion of fusion results. Visual and statistical analyzes
show that the proposed algorithm clearly improves the merging quality in terms
of: correlation coefficient, ERGAS, UIQI, and Q4; compared to fusion methods
including IHS, PCA, Adaptive IHS, and Improved Adaptive PCA.
Text Classification based on Multiple Block Convolutional Highways
Authors: Seyed Mahdi Rezaeinia, Ali Ghodsi, Rouhollah Rahmani
Date: 2018-07-23 00:00:00
Tasks: Classification, General Classification, Sentiment Analysis, text-classification, Text Classification

In the Text Classification areas of Sentiment Analysis,
Subjectivity/Objectivity Analysis, and Opinion Polarity, Convolutional Neural
Networks have gained special attention because of their performance and
accuracy. In this work, we applied recent advances in CNNs and propose a novel
architecture, Multiple Block Convolutional Highways (MBCH), which achieves
improved accuracy on multiple popular benchmark datasets, compared to previous
architectures. The MBCH is based on new techniques and architectures including
highway networks, DenseNet, batch normalization and bottleneck layers. In
addition, to cope with the limitations of existing pre-trained word vectors
which are used as inputs for the CNN, we propose a novel method, Improved Word
Vectors (IWV). The IWV improves the accuracy of CNNs which are used for text
classification tasks.
Acoustic-to-Word Recognition with Sequence-to-Sequence Models
Authors: Shruti Palaskar, Florian Metze
Date: 2018-07-23 00:00:00
Tasks: Language Modeling, Language Modelling, speech-recognition, Speech Recognition

Acoustic-to-Word recognition provides a straightforward solution to
end-to-end speech recognition without needing external decoding, language model
re-scoring or lexicon. While character-based models offer a natural solution to
the out-of-vocabulary problem, word models can be simpler to decode and may
also be able to directly recognize semantically meaningful units. We present
effective methods to train Sequence-to-Sequence models for direct word-level
recognition (and character-level recognition) and show an absolute improvement
of 4.4-5.0\% in Word Error Rate on the Switchboard corpus compared to prior
work. In addition to these promising results, word-based models are more
interpretable than character models, which have to be composed into words using
a separate decoding step. We analyze the encoder hidden states and the
attention behavior, and show that location-aware attention naturally represents
words as a single speech-word-vector, despite spanning multiple frames in the
input. We finally show that the Acoustic-to-Word model also learns to segment
speech into words with a mean standard deviation of 3 frames as compared with
human annotated forced-alignments for the Switchboard corpus.
Contextual Stochastic Block Models
Authors: Yash Deshpande, Andrea Montanari, Elchanan Mossel, Subhabrata Sen
Date: 2018-07-23 00:00:00
Proceeding: NeurIPS 2018 12

We provide the first information theoretic tight analysis for inference of
latent community structure given a sparse graph along with high dimensional
node covariates, correlated with the same latent communities. Our work bridges
recent theoretical breakthroughs in the detection of latent community structure
without nodes covariates and a large body of empirical work using diverse
heuristics for combining node covariates with graphs for inference. The
tightness of our analysis implies in particular, the information theoretical
necessity of combining the different sources of information. Our analysis holds
for networks of large degrees as well as for a Gaussian version of the model.
Redundancy Coefficient Gradual Up-weighting-based Mutual Information Feature Selection Technique for Crypto-ransomware Early Detection
Authors: Bander Ali Saleh Al-rimy, Mohd Aizaini Maarof, Syed Zainudeen Mohd Shaid
Date: 2018-07-22 00:00:00
Tasks: feature selection

Crypto-ransomware is characterized by its irreversible effect even after the
detection and removal. As such, the early detection is crucial to protect user
data and files of being held to ransom. Several solutions have proposed
utilizing the data extracted during the initial phases of the attacks before
the encryption takes place. However, the lack of enough data at the early
phases of the attack along with high dimensional features space renders the
model prone to overfitting which decreases its detection accuracy. To this end,
this paper proposed a novel redundancy coefficient gradual up-weighting
approach that was incorporated to the calculation of redundancy term of mutual
information to improve the feature selection process and enhance the accuracy
of the detection model. Several machine learning classifiers were used to
evaluate the detection performance of the proposed techniques. The experimental
results show that the accuracy of proposed techniques achieved higher detection
accuracy. Those results demonstrate the efficacy of the proposed techniques for
the early detection tasks.
Applying Domain Randomization to Synthetic Data for Object Category Detection
Authors: João Borrego, Atabak Dehban, Rui Figueiredo, Plinio Moreno, Alexandre Bernardino, José Santos-Victor
Date: 2018-07-16 00:00:00
Tasks: object-detection, Object Detection

Recent advances in deep learning-based object detection techniques have
revolutionized their applicability in several fields. However, since these
methods rely on unwieldy and large amounts of data, a common practice is to
download models pre-trained on standard datasets and fine-tune them for
specific application domains with a small set of domain relevant images. In
this work, we show that using synthetic datasets that are not necessarily
photo-realistic can be a better alternative to simply fine-tune pre-trained
networks. Specifically, our results show an impressive 25% improvement in the
mAP metric over a fine-tuning baseline when only about 200 labelled images are
available to train. Finally, an ablation study of our results is presented to
delineate the individual contribution of different components in the
randomization pipeline.
Evaluating and Understanding the Robustness of Adversarial Logit Pairing
Authors: Logan Engstrom, Andrew Ilyas, Anish Athalye
Date: 2018-07-26 00:00:00
Tasks: Adversarial Attack

We evaluate the robustness of Adversarial Logit Pairing, a recently proposed
defense against adversarial examples. We find that a network trained with
Adversarial Logit Pairing achieves 0.6% accuracy in the threat model in which
the defense is considered. We provide a brief overview of the defense and the
threat models/claims considered, as well as a discussion of the methodology and
results of our attack, which may offer insights into the reasons underlying the
vulnerability of ALP to adversarial attack.
Premise selection with neural networks and distributed representation of features
Authors: Andrzej Stanisław Kucik, Konstantin Korovin
Date: 2018-07-26 00:00:00
Tasks: Binary Classification, Dimensionality Reduction

We present the problem of selecting relevant premises for a proof of a given
statement. When stated as a binary classification task for pairs (conjecture,
axiom), it can be efficiently solved using artificial neural networks. The key
difference between our advance to solve this problem and previous approaches is
the use of just functional signatures of premises. To further improve the
performance of the model, we use dimensionality reduction technique, to replace
long and sparse signature vectors with their compact and dense embedded
versions. These are obtained by firstly defining the concept of a context for
each functor symbol, and then training a simple neural network to predict the
distribution of other functor symbols in the context of this functor. After
training the network, the output of its hidden layer is used to construct a
lower dimensional embedding of a functional signature (for each premise) with a
distributed representation of features. This allows us to use 512-dimensional
embeddings for conjecture-axiom pairs, containing enough information about the
original statements to reach the accuracy of 76.45% in premise selection task,
only with simple two-layer densely connected neural networks.
Generating 3D faces using Convolutional Mesh Autoencoders
Authors: Anurag Ranjan, Timo Bolkart, Soubhik Sanyal, Michael J. Black
Date: 2018-07-26 00:00:00
Proceeding: ECCV 2018 9
Tasks: 3D Face Modelling, Face Alignment, Face Model

Learned 3D representations of human faces are useful for computer vision
problems such as 3D face tracking and reconstruction from images, as well as
graphics applications such as character generation and animation. Traditional
models learn a latent representation of a face using linear subspaces or
higher-order tensor generalizations. Due to this linearity, they can not
capture extreme deformations and non-linear expressions. To address this, we
introduce a versatile model that learns a non-linear representation of a face
using spectral convolutions on a mesh surface. We introduce mesh sampling
operations that enable a hierarchical mesh representation that captures
non-linear variations in shape and expression at multiple scales within the
model. In a variational setting, our model samples diverse realistic 3D faces
from a multivariate Gaussian distribution. Our training data consists of 20,466
meshes of extreme expressions captured over 12 different subjects. Despite
limited training data, our trained model outperforms state-of-the-art face
models with 50% lower reconstruction error, while using 75% fewer parameters.
We also show that, replacing the expression space of an existing
state-of-the-art face model with our autoencoder, achieves a lower
reconstruction error. Our data, model and code are available at
http://github.com/anuragranj/coma
Layer-structured 3D Scene Inference via View Synthesis
Authors: Shubham Tulsiani, Richard Tucker, Noah Snavely
Date: 2018-07-26 00:00:00
Proceeding: ECCV 2018 9

We present an approach to infer a layer-structured 3D representation of a
scene from a single input image. This allows us to infer not only the depth of
the visible pixels, but also to capture the texture and depth for content in
the scene that is not directly visible. We overcome the challenge posed by the
lack of direct supervision by instead leveraging a more naturally available
multi-view supervisory signal. Our insight is to use view synthesis as a proxy
task: we enforce that our representation (inferred from a single image), when
rendered from a novel perspective, matches the true observed image. We present
a learning framework that operationalizes this insight using a new,
differentiable novel view renderer. We provide qualitative and quantitative
validation of our approach in two different settings, and demonstrate that we
can learn to capture the hidden aspects of a scene.
Seeded Graph Matching via Large Neighborhood Statistics
Authors: Elchanan Mossel, Jiaming Xu
Date: 2018-07-26 00:00:00
Tasks: Graph Matching

We study a well known noisy model of the graph isomorphism problem. In this
model, the goal is to perfectly recover the vertex correspondence between two
edge-correlated Erd\H{o}s-R\'{e}nyi random graphs, with an initial seed set of
correctly matched vertex pairs revealed as side information. For seeded
problems, our result provides a significant improvement over previously known
results. We show that it is possible to achieve the information-theoretic limit
of graph sparsity in time polynomial in the number of vertices $n$. Moreover,
we show the number of seeds needed for exact recovery in polynomial-time can be
as low as $n^{3\epsilon}$ in the sparse graph regime (with the average degree
smaller than $n^{\epsilon}$) and $\Omega(\log n)$ in the dense graph regime.
  Our results also shed light on the unseeded problem. In particular, we give
sub-exponential time algorithms for sparse models and an $n^{O(\log n)}$
algorithm for dense models for some parameters, including some that are not
covered by recent results of Barak et al.
Novelty Detection Meets Collider Physics
Authors: Jan Hajer, Ying-Ying Li, Tao Liu, He Wang
Date: 2018-07-26 00:00:00
Tasks: Clustering, Novelty Detection

Novelty detection is the machine learning task to recognize data, which
belong to an unknown pattern. Complementary to supervised learning, it allows
to analyze data model-independently. We demonstrate the potential role of
novelty detection in collider physics, using autoencoder-based deep neural
network. Explicitly, we develop a set of density-based novelty evaluators,
which are sensitive to the clustering of unknown-pattern testing data or
new-physics signal events, for the design of detection algorithms. We also
explore the influence of the known-pattern data fluctuations, arising from
non-signal regions, on detection sensitivity. Strategies to address it are
proposed. The algorithms are applied to detecting fermionic di-top partner and
resonant di-top productions at LHC, and exotic Higgs decays of two specific
modes at a $e^+e^-$ future collider. With parton-level analysis, we conclude
that potentially the new-physics benchmarks can be recognized with high
efficiency.
From handcrafted to deep local features
Authors: Gabriela Csurka, Christopher R. Dance, Martin Humenberger
Date: 2018-07-26 00:00:00
Tasks: 3D Reconstruction, BIG-bench Machine Learning

This paper presents an overview of the evolution of local features from handcrafted to deep-learning-based methods, followed by a discussion of several benchmarks and papers evaluating such local features. Our investigations are motivated by 3D reconstruction problems, where the precise location of the features is important. As we describe these methods, we highlight and explain the challenges of feature extraction and potential ways to overcome them. We first present handcrafted methods, followed by methods based on classical machine learning and finally we discuss methods based on deep-learning. This largely chronologically-ordered presentation will help the reader to fully understand the topic of image and region description in order to make best use of it in modern computer vision applications. In particular, understanding handcrafted methods and their motivation can help to understand modern approaches and how machine learning is used to improve the results. We also provide references to most of the relevant literature and code.
Aggregated Learning: A Deep Learning Framework Based on Information-Bottleneck Vector Quantization
Authors: Hongyu Guo, Yongyi Mao, Ali Al-Bashabsheh, Richong Zhang
Date: 2018-07-26 00:00:00
Tasks: Image Classification, Quantization, text-classification, Text Classification

Based on the notion of information bottleneck (IB), we formulate a
quantization problem called "IB quantization". We show that IB quantization is
equivalent to learning based on the IB principle. Under this equivalence, the
standard neural network models can be viewed as scalar (single sample) IB
quantizers. It is known, from conventional rate-distortion theory, that scalar
quantizers are inferior to vector (multi-sample) quantizers. Such a deficiency
then inspires us to develop a novel learning framework, AgrLearn, that
corresponds to vector IB quantizers for learning with neural networks. Unlike
standard networks, AgrLearn simultaneously optimizes against multiple data
samples. We experimentally verify that AgrLearn can result in significant
improvements when applied to several current deep learning architectures for
image recognition and text classification. We also empirically show that
AgrLearn can reduce up to 80% of the training samples needed for ResNet
training.
Medical Image Synthesis for Data Augmentation and Anonymization using Generative Adversarial Networks
Authors: Hoo-chang Shin, Neil A. Tenenholtz, Jameson K Rogers, Christopher G Schwarz, Matthew L Senjem, Jeffrey L Gunter, Katherine Andriole, Mark Michalski
Date: 2018-07-26 00:00:00
Tasks: Data Augmentation, Diversity, Generative Adversarial Network, Image Generation, Medical Image Generation, Segmentation, Tumor Segmentation

Data diversity is critical to success when training deep learning models.
Medical imaging data sets are often imbalanced as pathologic findings are
generally rare, which introduces significant challenges when training deep
learning models. In this work, we propose a method to generate synthetic
abnormal MRI images with brain tumors by training a generative adversarial
network using two publicly available data sets of brain MRI. We demonstrate two
unique benefits that the synthetic images provide. First, we illustrate
improved performance on tumor segmentation by leveraging the synthetic images
as a form of data augmentation. Second, we demonstrate the value of generative
models as an anonymization tool, achieving comparable tumor segmentation
results when trained on the synthetic data versus when trained on real subject
data. Together, these results offer a potential solution to two of the largest
challenges facing machine learning in medical imaging, namely the small
incidence of pathological findings, and the restrictions around sharing of
patient data.