pwc-archive/papers-with-abstracts huggingface dataset:
On the cross-validation bias due to unsupervised pre-processing
Authors: Amit Moscovich, Saharon Rosset
Date: 2019-01-25 00:00:00
Tasks: Dimensionality Reduction, feature selection, Model Selection

Cross-validation is the de facto standard for predictive model evaluation and selection. In proper use, it provides an unbiased estimate of a model's predictive performance. However, data sets often undergo various forms of data-dependent preprocessing, such as mean-centering, rescaling, dimensionality reduction, and outlier removal. It is often believed that such preprocessing stages, if done in an unsupervised manner (that does not incorporate the class labels or response values) are generally safe to do prior to cross-validation. In this paper, we study three commonly-practiced preprocessing procedures prior to a regression analysis: (i) variance-based feature selection; (ii) grouping of rare categorical features; and (iii) feature rescaling. We demonstrate that unsupervised preprocessing can, in fact, introduce a substantial bias into cross-validation estimates and potentially hurt model selection. This bias may be either positive or negative and its exact magnitude depends on all the parameters of the problem in an intricate manner. Further research is needed to understand the real-world impact of this bias across different application domains, particularly when dealing with small sample sizes and high-dimensional data.
FaceForensics++: Learning to Detect Manipulated Facial Images
Authors: Andreas Rössler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, Matthias Nießner
Date: 2019-01-25 00:00:00
Tasks: DeepFake Detection, Face Swapping, Fake Image Detection, Image Generation

The rapid progress in synthetic image generation and manipulation has now come to a point where it raises significant concerns for the implications towards society. At best, this leads to a loss of trust in digital content, but could potentially cause further harm by spreading false information or fake news. This paper examines the realism of state-of-the-art image manipulations, and how difficult it is to detect them, either automatically or by humans. To standardize the evaluation of detection methods, we propose an automated benchmark for facial manipulation detection. In particular, the benchmark is based on DeepFakes, Face2Face, FaceSwap and NeuralTextures as prominent representatives for facial manipulations at random compression level and size. The benchmark is publicly available and contains a hidden test set as well as a database of over 1.8 million manipulated images. This dataset is over an order of magnitude larger than comparable, publicly available, forgery datasets. Based on this data, we performed a thorough analysis of data-driven forgery detectors. We show that the use of additional domainspecific knowledge improves forgery detection to unprecedented accuracy, even in the presence of strong compression, and clearly outperforms human observers.
Skip-GANomaly: Skip Connected and Adversarially Trained Encoder-Decoder Anomaly Detection
Authors: Samet Akçay, Amir Atapour-Abarghouei, Toby P. Breckon
Date: 2019-01-25 00:00:00
Tasks: Anomaly Detection, Decoder, Scene Understanding, Unsupervised Anomaly Detection

Despite inherent ill-definition, anomaly detection is a research endeavor of
great interest within machine learning and visual scene understanding alike.
Most commonly, anomaly detection is considered as the detection of outliers
within a given data distribution based on some measure of normality. The most
significant challenge in real-world anomaly detection problems is that
available data is highly imbalanced towards normality (i.e. non-anomalous) and
contains a most a subset of all possible anomalous samples - hence limiting the
use of well-established supervised learning methods. By contrast, we introduce
an unsupervised anomaly detection model, trained only on the normal
(non-anomalous, plentiful) samples in order to learn the normality distribution
of the domain and hence detect abnormality based on deviation from this model.
Our proposed approach employs an encoder-decoder convolutional neural network
with skip connections to thoroughly capture the multi-scale distribution of the
normal data distribution in high-dimensional image space. Furthermore,
utilizing an adversarial training scheme for this chosen architecture provides
superior reconstruction both within high-dimensional image space and a
lower-dimensional latent vector space encoding. Minimizing the reconstruction
error metric within both the image and hidden vector spaces during training
aids the model to learn the distribution of normality as required. Higher
reconstruction metrics during subsequent test and deployment are thus
indicative of a deviation from this normal distribution, hence indicative of an
anomaly. Experimentation over established anomaly detection benchmarks and
challenging real-world datasets, within the context of X-ray security
screening, shows the unique promise of such a proposed approach.
Subspace Robust Wasserstein Distances
Authors: François-Pierre Paty, Marco Cuturi
Date: 2019-01-25 00:00:00
Tasks: Quantization

Making sense of Wasserstein distances between discrete measures in high-dimensional settings remains a challenge. Recent work has advocated a two-step approach to improve robustness and facilitate the computation of optimal transport, using for instance projections on random real lines, or a preliminary quantization of the measures to reduce the size of their support. We propose in this work a "max-min" robust variant of the Wasserstein distance by considering the maximal possible distance that can be realized between two measures, assuming they can be projected orthogonally on a lower $k$-dimensional subspace. Alternatively, we show that the corresponding "min-max" OT problem has a tight convex relaxation which can be cast as that of finding an optimal transport plan with a low transportation cost, where the cost is alternatively defined as the sum of the $k$ largest eigenvalues of the second order moment matrix of the displacements (or matchings) corresponding to that plan (the usual OT definition only considers the trace of that matrix). We show that both quantities inherit several favorable properties from the OT geometry. We propose two algorithms to compute the latter formulation using entropic regularization, and illustrate the interest of this approach empirically.
Pricing options and computing implied volatilities using neural networks
Authors: Shuaiqiang Liu, Cornelis W. Oosterlee, Sander M. Bohte
Date: 2019-01-25 00:00:00

This paper proposes a data-driven approach, by means of an Artificial Neural
Network (ANN), to value financial options and to calculate implied volatilities
with the aim of accelerating the corresponding numerical methods. With ANNs
being universal function approximators, this method trains an optimized ANN on
a data set generated by a sophisticated financial model, and runs the trained
ANN as an agent of the original solver in a fast and efficient way. We test
this approach on three different types of solvers, including the analytic
solution for the Black-Scholes equation, the COS method for the Heston
stochastic volatility model and Brent's iterative root-finding method for the
calculation of implied volatilities. The numerical results show that the ANN
solver can reduce the computing time significantly.
Improving Image Captioning by Leveraging Knowledge Graphs
Authors: Yimin Zhou, Yiwei Sun, Vasant Honavar
Date: 2019-01-25 00:00:00
Tasks: Image Captioning, Knowledge Graphs

We explore the use of a knowledge graphs, that capture general or commonsense
knowledge, to augment the information extracted from images by the
state-of-the-art methods for image captioning. The results of our experiments,
on several benchmark data sets such as MS COCO, as measured by CIDEr-D, a
performance metric for image captioning, show that the variants of the
state-of-the-art methods for image captioning that make use of the information
extracted from knowledge graphs can substantially outperform those that rely
solely on the information extracted from images.
Self-Supervised Generalisation with Meta Auxiliary Learning
Authors: Shikun Liu, Andrew J. Davison, Edward Johns
Date: 2019-01-25 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Auxiliary Learning, Meta-Learning, Multi-Task Learning

Learning with auxiliary tasks can improve the ability of a primary task to generalise. However, this comes at the cost of manually labelling auxiliary data. We propose a new method which automatically learns appropriate labels for an auxiliary task, such that any supervised learning task can be improved without requiring access to any further data. The approach is to train two neural networks: a label-generation network to predict the auxiliary labels, and a multi-task network to train the primary task alongside the auxiliary task. The loss for the label-generation network incorporates the loss of the multi-task network, and so this interaction between the two networks can be seen as a form of meta learning with a double gradient. We show that our proposed method, Meta AuXiliary Learning (MAXL), outperforms single-task learning on 7 image datasets, without requiring any additional data. We also show that MAXL outperforms several other baselines for generating auxiliary labels, and is even competitive when compared with human-defined auxiliary labels. The self-supervised nature of our method leads to a promising new direction towards automated generalisation. Source code can be found at https://github.com/lorenmt/maxl.
Dense 3D Point Cloud Reconstruction Using a Deep Pyramid Network
Authors: Priyanka Mandikal, R. Venkatesh Babu
Date: 2019-01-25 00:00:00
Tasks: 3D Point Cloud Reconstruction, Point cloud reconstruction

Reconstructing a high-resolution 3D model of an object is a challenging task
in computer vision. Designing scalable and light-weight architectures is
crucial while addressing this problem. Existing point-cloud based
reconstruction approaches directly predict the entire point cloud in a single
stage. Although this technique can handle low-resolution point clouds, it is
not a viable solution for generating dense, high-resolution outputs. In this
work, we introduce DensePCR, a deep pyramidal network for point cloud
reconstruction that hierarchically predicts point clouds of increasing
resolution. Towards this end, we propose an architecture that first predicts a
low-resolution point cloud, and then hierarchically increases the resolution by
aggregating local and global point features to deform a grid. Our method
generates point clouds that are accurate, uniform and dense. Through extensive
quantitative and qualitative evaluation on synthetic and real datasets, we
demonstrate that DensePCR outperforms the existing state-of-the-art point cloud
reconstruction works, while also providing a light-weight and scalable
architecture for predicting high-resolution outputs.
Comparing of Term Clustering Frameworks for Modular Ontology Learning
Authors: Ziwei Xu, Mounira Harzallah, Fabrice Guillet
Date: 2019-01-25 00:00:00
Tasks: Clustering, Dimensionality Reduction, Feature Compression, feature selection, Sentence

This paper aims to use term clustering to build a modular ontology according
to core ontology from domain-specific text. The acquisition of semantic
knowledge focuses on noun phrase appearing with the same syntactic roles in
relation to a verb or its preposition combination in a sentence. The
construction of this co-occurrence matrix from context helps to build feature
space of noun phrases, which is then transformed to several encoding
representations including feature selection and dimensionality reduction. In
addition, the content has also been presented with the construction of word
vectors. These representations are clustered respectively with K-Means and
Affinity Propagation (AP) methods, which differentiate into the term clustering
frameworks. Due to the randomness of K-Means, iteration efforts are adopted to
find the optimal parameter. The frameworks are evaluated extensively where AP
shows dominant effectiveness for co-occurred terms and NMF encoding technique
is salient by its promising facilities in feature compression.
Bayesian surrogate learning in dynamic simulator-based regression problems
Authors: Xi Chen, Mike Hobson
Date: 2019-01-25 00:00:00
Tasks: parameter estimation, regression

The estimation of unknown values of parameters (or hidden variables, control
variables) that characterise a physical system often relies on the comparison
of measured data with synthetic data produced by some numerical simulator of
the system as the parameter values are varied. This process often encounters
two major difficulties: the generation of synthetic data for each considered
set of parameter values can be computationally expensive if the system model is
complicated; and the exploration of the parameter space can be inefficient
and/or incomplete, a typical example being when the exploration becomes trapped
in a local optimum of the objection function that characterises the mismatch
between the measured and synthetic data. A method to address both these issues
is presented, whereby: a surrogate model (or proxy), which emulates the
computationally expensive system simulator, is constructed using deep recurrent
networks (DRN); and a nested sampling (NS) algorithm is employed to perform
efficient and robust exploration of the parameter space. The analysis is
performed in a Bayesian context, in which the samples characterise the full
joint posterior distribution of the parameters, from which parameter estimates
and uncertainties are easily derived. The proposed approach is compared with
conventional methods in some numerical examples, for which the results
demonstrate that one can accelerate the parameter estimation process by at
least an order of magnitude.
Vision-based inspection system employing computer vision & neural networks for detection of fractures in manufactured components
Authors: Sarthak J. Shetty
Date: 2019-01-25 00:00:00
Tasks: BIG-bench Machine Learning

We are proceeding towards the age of automation and robotic integration of
our production lines [5]. Effective quality-control systems have to be put in
place to maintain the quality of manufactured components. Among different
quality-control systems, vision-based inspection systems have gained
considerable amount of popularity [8] due to developments in computing power
and image processing techniques. In this paper, we present a vision-based
inspection system (VBI) as a quality-control system, which not only detects the
presence of defects, such as in conventional VBIs, but also leverage
developments in machine learning to predict the presence of surface fractures
and wearing. We use OpenCV, an open source computer-vision framework, and
Tensorflow, an open source machine-learning framework developed by Google Inc.,
to accomplish the tasks of detection and prediction of presence of surface
defects such as fractures of manufactured gears.
Improving Adversarial Robustness via Promoting Ensemble Diversity
Authors: Tianyu Pang, Kun Xu, Chao Du, Ning Chen, Jun Zhu
Date: 2019-01-25 00:00:00
Tasks: Adversarial Robustness, Diversity

Though deep neural networks have achieved significant progress on various tasks, often enhanced by model ensemble, existing high-performance models can be vulnerable to adversarial attacks. Many efforts have been devoted to enhancing the robustness of individual networks and then constructing a straightforward ensemble, e.g., by directly averaging the outputs, which ignores the interaction among networks. This paper presents a new method that explores the interaction among individual networks to improve robustness for ensemble models. Technically, we define a new notion of ensemble diversity in the adversarial setting as the diversity among non-maximal predictions of individual members, and present an adaptive diversity promoting (ADP) regularizer to encourage the diversity, which leads to globally better robustness for the ensemble by making adversarial examples difficult to transfer among individual members. Our method is computationally efficient and compatible with the defense methods acting on individual networks. Empirical results on various datasets verify that our method can improve adversarial robustness while maintaining state-of-the-art accuracy on normal examples.
Bayes metaclassifier and Soft-confusion-matrix classifier in the task of multi-label classification
Authors: Pawel Trajdos, Marcin Majak
Date: 2019-01-25 00:00:00
Tasks: Classification, General Classification, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION

The aim of this paper was to compare soft confusion matrix approach and Bayes
metaclassifier under the multi-label classification framework. Although the
methods were successfully applied under the multi-label classification
framework, they have not been compared directly thus far. Such comparison is of
vital importance because both methods are quite similar as they are both based
on the concept of randomized reference classifier. Since both algorithms were
designed to deal with single-label problems, they are combined with the
problem-transformation approach to multi-label classification. Present study
included 29 benchmark datasets and four different base classifiers. The
algorithms were compared in terms of 11 quality criteria and the results were
subjected to statistical analysis.
Joint shape learning and segmentation for medical images using a minimalistic deep network
Authors: Balamurali Murugesan, Kaushik Sarveswaran, Sharath M. Shankaranarayana, Keerthi Ram, Mohanasankar Sivaprakasam
Date: 2019-01-25 00:00:00
Tasks: Decoder, Image Segmentation, Medical Image Segmentation, Multi-Task Learning, Segmentation, Semantic Segmentation

Recently, state-of-the-art results have been achieved in semantic
segmentation using fully convolutional networks (FCNs). Most of these networks
employ encoder-decoder style architecture similar to U-Net and are trained with
images and the corresponding segmentation maps as a pixel-wise classification
task. Such frameworks only exploit class information by using the ground truth
segmentation maps. In this paper, we propose a multi-task learning framework
with the main aim of exploiting structural and spatial information along with
the class information. We modify the decoder part of the FCN to exploit class
information and the structural information as well. We intend to do this while
also keeping the parameters of the network as low as possible. We obtain the
structural information using either of the two ways: i) using the contour map
and ii) using the distance map, both of which can be obtained from ground truth
segmentation maps with no additional annotation costs. We also explore
different ways in which distance maps can be computed and study the effects of
different distance maps on the segmentation performance. We also experiment
extensively on two different medical image segmentation applications: i.e i)
using color fundus images for optic disc and cup segmentation and ii) using
endoscopic images for polyp segmentation. Through our experiments, we report
results comparable to, and in some cases performing better than the current
state-of-the-art architectures and with an order of 2x reduction in the number
of parameters.
State-Regularized Recurrent Neural Networks
Authors: Cheng Wang, Mathias Niepert
Date: 2019-01-25 00:00:00
Tasks: Language Modeling, Language Modelling, Memorization, Object Recognition, Sentiment Analysis

Recurrent neural networks are a widely used class of neural architectures. They have, however, two shortcomings. First, it is difficult to understand what exactly they learn. Second, they tend to work poorly on sequences requiring long-term memorization, despite having this capacity in principle. We aim to address both shortcomings with a class of recurrent networks that use a stochastic state transition mechanism between cell applications. This mechanism, which we term state-regularization, makes RNNs transition between a finite set of learnable states. We evaluate state-regularized RNNs on (1) regular languages for the purpose of automata extraction; (2) nonregular languages such as balanced parentheses, palindromes, and the copy task where external memory is required; and (3) real-word sequence learning tasks for sentiment analysis, visual object recognition, and language modeling. We show that state-regularization (a) simplifies the extraction of finite state automata modeling an RNN's state transition dynamics; (b) forces RNNs to operate more like automata with external memory and less like finite state machines; (c) makes RNNs have better interpretability and explainability.
Empowering individual trait prediction using interactions
Authors: Damian Gola, Inke R. König
Date: 2019-01-25 00:00:00
Tasks: Dimensionality Reduction, Epidemiology, Prediction

One component of precision medicine is to construct prediction models with
their predictive ability as high as possible, e.g. to enable individual risk
prediction. In genetic epidemiology, complex diseases have a polygenic basis
and a common assumption is that biological and genetic features affect the
outcome under consideration via interactions. In the case of omics data, the
use of standard approaches such as generalized linear models may be suboptimal
and machine learning methods are appealing to make individual predictions.
However, most of these algorithms focus mostly on main or marginal effects of
the single features in a dataset. On the other hand, the detection of
interacting features is an active area of research in the realm of genetic
epidemiology. One big class of algorithms to detect interacting features is
based on the multifactor dimensionality reduction (MDR). Here, we extend the
model-based MDR (MB-MDR), a powerful extension of the original MDR algorithm,
to enable interaction empowered individual prediction. Using a comprehensive
simulation study we show that our new algorithm can use information hidden in
interactions more efficiently than two other state-of-the-art algorithms,
namely the Random Forest and Elastic Net, and clearly outperforms these if
interactions are present. The performance of these algorithms is comparable if
no interactions are present. Further, we show that our new algorithm is
applicable to real data by comparing the performance of the three algorithms on
a dataset of rheumatoid arthritis cases and healthy controls. As our new
algorithm is not only applicable to biological/genetic data but to all datasets
with discrete features, it may have practical implications in other
applications as well, and we made our method available as an R package.
Proceedings of AAAI 2019 Workshop on Network Interpretability for Deep Learning
Authors: Quanshi Zhang, Lixin Fan, Bolei Zhou
Date: 2019-01-25 00:00:00
Tasks: Deep Learning

This is the Proceedings of AAAI 2019 Workshop on Network Interpretability for Deep Learning
Face morphing detection in the presence of printing/scanning and heterogeneous image sources
Authors: Matteo Ferrara, Annalisa Franco, Davide Maltoni
Date: 2019-01-25 00:00:00
Tasks: Data Augmentation, Face Recognition

Face morphing represents nowadays a big security threat in the context of electronic identity documents as well as an interesting challenge for researchers in the field of face recognition. Despite of the good performance obtained by state-of-the-art approaches on digital images, no satisfactory solutions have been identified so far to deal with cross-database testing and printed-scanned images (typically used in many countries for document issuing). In this work, novel approaches are proposed to train Deep Neural Networks for morphing detection: in particular generation of simulated printed-scanned images together with other data augmentation strategies and pre-training on large face recognition datasets, allowed to reach state-of-the-art accuracy on challenging datasets from heterogeneous image sources.