pwc-archive/papers-with-abstracts huggingface dataset:
Disentangling Latent Factors of Variational Auto-Encoder with Whitening
Authors: Sangchul Hahn, Heeyoul Choi
Date: 2018-11-08 00:00:00
Tasks: Image Generation

After deep generative models were successfully applied to image generation tasks, learning disentangled latent variables of data has become a crucial part of deep generative model research. Many models have been proposed to learn an interpretable and factorized representation of latent variable by modifying their objective function or model architecture. To disentangle the latent variable, some models show lower quality of reconstructed images and others increase the model complexity which is hard to train. In this paper, we propose a simple disentangling method based on a traditional whitening process. The proposed method is applied to the latent variables of variational auto-encoder (VAE), although it can be applied to any generative models with latent variables. In experiment, we apply the proposed method to simple VAE models and experiment results confirm that our method finds more interpretable factors from the latent space while keeping the reconstruction error the same as the conventional VAE's error.
Alpha-Integration Pooling for Convolutional Neural Networks
Authors: Hayoung Eom, Heeyoul Choi
Date: 2018-11-08 00:00:00
Tasks: Philosophy

Convolutional neural networks (CNNs) have achieved remarkable performance in many applications, especially in image recognition tasks. As a crucial component of CNNs, sub-sampling plays an important role for efficient training or invariance property, and max-pooling and arithmetic average-pooling are commonly used sub-sampling methods. In addition to the two pooling methods, however, there could be many other pooling types, such as geometric average, harmonic average, and so on. Since it is not easy for algorithms to find the best pooling method, usually the pooling types are assumed a priority, which might not be optimal for different tasks. In line with the deep learning philosophy, the type of pooling can be driven by data for a given task. In this paper, we propose {\it $\alpha$-integration pooling} ($\alpha$I-pooling), which has a trainable parameter $\alpha$ to find the type of pooling. $\alpha$I-pooling is a general pooling method including max-pooling and arithmetic average-pooling as a special case, depending on the parameter $\alpha$. Experiments show that $\alpha$I-pooling outperforms other pooling methods including max-pooling, in image recognition tasks. Also, it turns out that each layer has different optimal pooling type.
Explainable cardiac pathology classification on cine MRI with motion characterization by semi-supervised learning of apparent flow
Authors: Qiao Zheng, Hervé Delingette, Nicholas Ayache
Date: 2018-11-08 00:00:00
Tasks: Cardiac Segmentation, General Classification, Time Series, Time Series Analysis

We propose a method to classify cardiac pathology based on a novel approach
to extract image derived features to characterize the shape and motion of the
heart. An original semi-supervised learning procedure, which makes efficient
use of a large amount of non-segmented images and a small amount of images
segmented manually by experts, is developed to generate pixel-wise apparent
flow between two time points of a 2D+t cine MRI image sequence. Combining the
apparent flow maps and cardiac segmentation masks, we obtain a local apparent
flow corresponding to the 2D motion of myocardium and ventricular cavities.
This leads to the generation of time series of the radius and thickness of
myocardial segments to represent cardiac motion. These time series of motion
features are reliable and explainable characteristics of pathological cardiac
motion. Furthermore, they are combined with shape-related features to classify
cardiac pathologies. Using only nine feature values as input, we propose an
explainable, simple and flexible model for pathology classification. On ACDC
training set and testing set, the model achieves 95% and 94% respectively as
classification accuracy. Its performance is hence comparable to that of the
state-of-the-art. Comparison with various other models is performed to outline
some advantages of our model.
A Factor Graph Approach to Automated Design of Bayesian Signal Processing Algorithms
Authors: Marco Cox, Thijs van de Laar, Bert de Vries
Date: 2018-11-08 00:00:00
Tasks: Bayesian Inference, parameter estimation, Probabilistic Programming, State Space Models, Variational Inference

The benefits of automating design cycles for Bayesian inference-based
algorithms are becoming increasingly recognized by the machine learning
community. As a result, interest in probabilistic programming frameworks has
much increased over the past few years. This paper explores a specific
probabilistic programming paradigm, namely message passing in Forney-style
factor graphs (FFGs), in the context of automated design of efficient Bayesian
signal processing algorithms. To this end, we developed "ForneyLab"
(https://github.com/biaslab/ForneyLab.jl) as a Julia toolbox for message
passing-based inference in FFGs. We show by example how ForneyLab enables
automatic derivation of Bayesian signal processing algorithms, including
algorithms for parameter estimation and model comparison. Crucially, due to the
modular makeup of the FFG framework, both the model specification and inference
methods are readily extensible in ForneyLab. In order to test this framework,
we compared variational message passing as implemented by ForneyLab with
automatic differentiation variational inference (ADVI) and Monte Carlo methods
as implemented by state-of-the-art tools "Edward" and "Stan". In terms of
performance, extensibility and stability issues, ForneyLab appears to enjoy an
edge relative to its competitors for automated inference in state-space models.
ExGate: Externally Controlled Gating for Feature-based Attention in Artificial Neural Networks
Authors: Jarryd Son, Amit Mishra
Date: 2018-11-08 00:00:00
Tasks: General Classification

Perceptual capabilities of artificial systems have come a long way since the
advent of deep learning. These methods have proven to be effective, however
they are not as efficient as their biological counterparts. Visual attention is
a set of mechanisms that are employed in biological visual systems to ease
computational load by only processing pertinent parts of the stimuli. This
paper addresses the implementation of top-down, feature-based attention in an
artificial neural network by use of externally controlled neuron gating. Our
results showed a 5% increase in classification accuracy on the CIFAR-10 dataset
versus a non-gated version, while adding very few parameters. Our gated model
also produces more reasonable errors in predictions by drastically reducing
prediction of classes that belong to a different category to the true class.
A Survey on Data Collection for Machine Learning: a Big Data -- AI Integration Perspective
Authors: Yuji Roh, Geon Heo, Steven Euijong Whang
Date: 2018-11-08 00:00:00
Tasks: BIG-bench Machine Learning, Feature Engineering, Management

Data collection is a major bottleneck in machine learning and an active research topic in multiple communities. There are largely two reasons data collection has recently become a critical issue. First, as machine learning is becoming more widely-used, we are seeing new applications that do not necessarily have enough labeled data. Second, unlike traditional machine learning, deep learning techniques automatically generate features, which saves feature engineering costs, but in return may require larger amounts of labeled data. Interestingly, recent research in data collection comes not only from the machine learning, natural language, and computer vision communities, but also from the data management community due to the importance of handling large amounts of data. In this survey, we perform a comprehensive study of data collection from a data management point of view. Data collection largely consists of data acquisition, data labeling, and improvement of existing data or models. We provide a research landscape of these operations, provide guidelines on which technique to use when, and identify interesting research challenges. The integration of machine learning and data management for data collection is part of a larger trend of Big data and Artificial Intelligence (AI) integration and opens many opportunities for new research.
Untangling the GDPR Using ConRelMiner
Authors: Karolin Winter, Stefanie Rinderle-Ma
Date: 2018-11-08 00:00:00

The General Data Protection Regulation (GDPR) poses enormous challenges on
companies and organizations with respect to understanding, implementing, and
maintaining the contained constraints. We report on how the ConRelMiner method
can be used for untangling the GDPR. For this, the GDPR is filtered and grouped
along the roles mentioned by the GDPR and the reduction of sentences to be read
by analysts is shown. Moreover, the output of the ConRelMiner - a cluster graph
with relations between the sentences - is displayed and interpreted. Overall
the goal is to illustrate how the effort for implementing the GDPR can be
reduced and a structured and meaningful representation of the relevant GDPR
sentences can be found.
Transformative Machine Learning
Authors: Ivan Olier, Oghenejokpeme I. Orhobor, Joaquin Vanschoren, Ross D. King
Date: 2018-11-08 00:00:00
Tasks: BIG-bench Machine Learning, Drug Design, Explainable Models, Meta-Learning, Multi-Task Learning, Representation Learning

The key to success in machine learning (ML) is the use of effective data
representations. Traditionally, data representations were hand-crafted.
Recently it has been demonstrated that, given sufficient data, deep neural
networks can learn effective implicit representations from simple input
representations. However, for most scientific problems, the use of deep
learning is not appropriate as the amount of available data is limited, and/or
the output models must be explainable. Nevertheless, many scientific problems
do have significant amounts of data available on related tasks, which makes
them amenable to multi-task learning, i.e. learning many related problems
simultaneously. Here we propose a novel and general representation learning
approach for multi-task learning that works successfully with small amounts of
data. The fundamental new idea is to transform an input intrinsic data
representation (i.e., handcrafted features), to an extrinsic representation
based on what a pre-trained set of models predict about the examples. This
transformation has the dual advantages of producing significantly more accurate
predictions, and providing explainable models. To demonstrate the utility of
this transformative learning approach, we have applied it to three real-world
scientific problems: drug-design (quantitative structure activity relationship
learning), predicting human gene expression (across different tissue types and
drug treatments), and meta-learning for machine learning (predicting which
machine learning methods work best for a given problem). In all three problems,
transformative machine learning significantly outperforms the best intrinsic
representation.
Bayesian Deep Learning for Exoplanet Atmospheric Retrieval
Authors: Frank Soboczenski, Michael D. Himes, Molly D. O'Beirne, Simone Zorzan, Atilim Gunes Baydin, Adam D. Cobb, Yarin Gal, Daniel Angerhausen, Massimo Mascaro, Giada N. Arney, Shawn D. Domagal-Goldman
Date: 2018-11-08 00:00:00
Tasks: Deep Learning, Retrieval

Over the past decade, the study of extrasolar planets has evolved rapidly
from plain detection and identification to comprehensive categorization and
characterization of exoplanet systems and their atmospheres. Atmospheric
retrieval, the inverse modeling technique used to determine an exoplanetary
atmosphere's temperature structure and composition from an observed spectrum,
is both time-consuming and compute-intensive, requiring complex algorithms that
compare thousands to millions of atmospheric models to the observational data
to find the most probable values and associated uncertainties for each model
parameter. For rocky, terrestrial planets, the retrieved atmospheric
composition can give insight into the surface fluxes of gaseous species
necessary to maintain the stability of that atmosphere, which may in turn
provide insight into the geological and/or biological processes active on the
planet. These atmospheres contain many molecules, some of them biosignatures,
spectral fingerprints indicative of biological activity, which will become
observable with the next generation of telescopes. Runtimes of traditional
retrieval models scale with the number of model parameters, so as more
molecular species are considered, runtimes can become prohibitively long.
Recent advances in machine learning (ML) and computer vision offer new ways to
reduce the time to perform a retrieval by orders of magnitude, given a
sufficient data set to train with. Here we present an ML-based retrieval
framework called Intelligent exoplaNet Atmospheric RetrievAl (INARA) that
consists of a Bayesian deep learning model for retrieval and a data set of
3,000,000 synthetic rocky exoplanetary spectra generated using the NASA
Planetary Spectrum Generator. Our work represents the first ML retrieval model
for rocky, terrestrial exoplanets and the first synthetic data set of
terrestrial spectra generated at this scale.
Knowledge Tracing Machines: Factorization Machines for Knowledge Tracing
Authors: Jill-Jênn Vie, Hisashi Kashima
Date: 2018-11-08 00:00:00
Tasks: Knowledge Tracing

Knowledge tracing is a sequence prediction problem where the goal is to
predict the outcomes of students over questions as they are interacting with a
learning platform. By tracking the evolution of the knowledge of some student,
one can optimize instruction. Existing methods are either based on temporal
latent variable models, or factor analysis with temporal features. We here show
that factorization machines (FMs), a model for regression or classification,
encompasses several existing models in the educational literature as special
cases, notably additive factor model, performance factor model, and
multidimensional item response theory. We show, using several real datasets of
tens of thousands of users and items, that FMs can estimate student knowledge
accurately and fast even when student data is sparsely observed, and handle
side information such as multiple knowledge components and number of attempts
at item or skill level. Our approach allows to fit student models of higher
dimension than existing models, and provides a testbed to try new combinations
of features in order to improve existing models.
Prediction of laparoscopic procedure duration using unlabeled, multimodal sensor data
Authors: Sebastian Bodenstedt, Martin Wagner, Lars Mündermann, Hannes Kenngott, Beat Müller-Stich, Michael Breucha, Sören Torge Mees, Jürgen Weitz, Stefanie Speidel
Date: 2018-11-08 00:00:00

Purpose The course of surgical procedures is often unpredictable, making it
difficult to estimate the duration of procedures beforehand. A context-aware
method that analyses the workflow of an intervention online and automatically
predicts the remaining duration would alleviate these problems. As basis for
such an estimate, information regarding the current state of the intervention
is required. Methods Today, the operating room contains a diverse range of
sensors. During laparoscopic interventions, the endoscopic video stream is an
ideal source of such information. Extracting quantitative information from the
video is challenging though, due to its high dimensionality. Other surgical
devices (e.g. insufflator, lights, etc.) provide data streams which are, in
contrast to the video stream, more compact and easier to quantify. Though
whether such streams offer sufficient information for estimating the duration
of surgery is uncertain. Here, we propose and compare methods, based on
convolutional neural networks, for continuously predicting the duration of
laparoscopic interventions based on unlabeled data, such as from endoscopic
images and surgical device streams. Results The methods are evaluated on 80
laparoscopic interventions of various types, for which surgical device data and
the endoscopic video are available. Here the combined method performs best with
an overall average error of 37% and an average halftime error of 28%.
Conclusion In this paper, we present, to our knowledge, the first approach for
online procedure duration prediction using unlabeled endoscopic video data and
surgical device data in a laparoscopic setting. We also show that a method
incorporating both vision and device data performs better than methods based
only on vision, while methods only based on tool usage and surgical device data
perform poorly, showing the importance of the visual channel.
Active Learning using Deep Bayesian Networks for Surgical Workflow Analysis
Authors: Sebastian Bodenstedt, Dominik Rivoir, Alexander Jenke, Martin Wagner, Michael Breucha, Beat Müller-Stich, Sören Torge Mees, Jürgen Weitz, Stefanie Speidel
Date: 2018-11-08 00:00:00
Tasks: Active Learning, BIG-bench Machine Learning

For many applications in the field of computer assisted surgery, such as
providing the position of a tumor, specifying the most probable tool required
next by the surgeon or determining the remaining duration of surgery, methods
for surgical workflow analysis are a prerequisite. Often machine learning based
approaches serve as basis for surgical workflow analysis. In general machine
learning algorithms, such as convolutional neural networks (CNN), require large
amounts of labeled data. While data is often available in abundance, many tasks
in surgical workflow analysis need data annotated by domain experts, making it
difficult to obtain a sufficient amount of annotations.
  The aim of using active learning to train a machine learning model is to
reduce the annotation effort. Active learning methods determine which unlabeled
data points would provide the most information according to some metric, such
as prediction uncertainty. Experts will then be asked to only annotate these
data points. The model is then retrained with the new data and used to select
further data for annotation. Recently, active learning has been applied to CNN
by means of Deep Bayesian Networks (DBN). These networks make it possible to
assign uncertainties to predictions.
  In this paper, we present a DBN-based active learning approach adapted for
image-based surgical workflow analysis task. Furthermore, by using a recurrent
architecture, we extend this network to video-based surgical workflow analysis.
We evaluate these approaches on the Cholec80 dataset by performing instrument
presence detection and surgical phase segmentation. Here we are able to show
that using a DBN-based active learning approach for selecting what data points
to annotate next outperforms a baseline based on randomly selecting data
points.
Activation Functions: Comparison of trends in Practice and Research for Deep Learning
Authors: Chigozie Nwankpa, Winifred Ijomah, Anthony Gachagan, Stephen Marshall
Date: 2018-11-08 00:00:00
Tasks: Deep Learning

Deep neural networks have been successfully used in diverse emerging domains
to solve real world complex problems with may more deep learning(DL)
architectures, being developed to date. To achieve these state-of-the-art
performances, the DL architectures use activation functions (AFs), to perform
diverse computations between the hidden layers and the output layers of any
given DL architecture. This paper presents a survey on the existing AFs used in
deep learning applications and highlights the recent trends in the use of the
activation functions for deep learning applications. The novelty of this paper
is that it compiles majority of the AFs used in DL and outlines the current
trends in the applications and usage of these functions in practical deep
learning deployments against the state-of-the-art research results. This
compilation will aid in making effective decisions in the choice of the most
suitable and appropriate activation function for any given application, ready
for deployment. This paper is timely because most research papers on AF
highlights similar works and results while this paper will be the first, to
compile the trends in AF applications in practice against the research results
from literature, found in deep learning research to date.
Spectral Simplicial Theory for Feature Selection and Applications to Genomics
Authors: Kiya W. Govek, Venkata S. Yamajala, Pablo G. Camara
Date: 2018-11-08 00:00:00
Tasks: feature selection, Topological Data Analysis

The scale and complexity of modern data sets and the limitations associated
with testing large numbers of hypotheses underline the need for feature
selection methods. Spectral techniques rank features according to their degree
of consistency with an underlying metric structure, but their current
graph-based formulation restricts their applicability to point features. We
extend spectral methods for feature selection to abstract simplicial complexes
and present a general framework which can be applied to 2-point and
higher-order features. Combinatorial Laplacian scores take into account the
topology spanned by the data and reduce to the ordinary Laplacian score in the
case of point features. We demonstrate the utility of spectral simplicial
methods for feature selection with several examples of application to the
analysis of gene expression and multi-modal genomic data. Our results provide a
unifying perspective on topological data analysis and manifold learning
approaches.
Meta-Learning for Multi-objective Reinforcement Learning
Authors: Xi Chen, Ali Ghadirzadeh, Mårten Björkman, Patric Jensfelt
Date: 2018-11-08 00:00:00
Tasks: Computational Efficiency, continuous-control, Continuous Control, Decision Making, Meta-Learning, Multi-Objective Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Sequential Decision Making

Multi-objective reinforcement learning (MORL) is the generalization of standard reinforcement learning (RL) approaches to solve sequential decision making problems that consist of several, possibly conflicting, objectives. Generally, in such formulations, there is no single optimal policy which optimizes all the objectives simultaneously, and instead, a number of policies has to be found each optimizing a preference of the objectives. In other words, the MORL is framed as a meta-learning problem, with the task distribution given by a distribution over the preferences. We demonstrate that such a formulation results in a better approximation of the Pareto optimal solutions in terms of both the optimality and the computational efficiency. We evaluated our method on obtaining Pareto optimal policies using a number of continuous control problems with high degrees of freedom.
Linear Memory Networks
Authors: Davide Bacciu, Antonio Carta, Alessandro Sperduti
Date: 2018-11-08 00:00:00

Recurrent neural networks can learn complex transduction problems that
require maintaining and actively exploiting a memory of their inputs. Such
models traditionally consider memory and input-output functionalities
indissolubly entangled. We introduce a novel recurrent architecture based on
the conceptual separation between the functional input-output transformation
and the memory mechanism, showing how they can be implemented through different
neural components. By building on such conceptualization, we introduce the
Linear Memory Network, a recurrent model comprising a feedforward neural
network, realizing the non-linear functional transformation, and a linear
autoencoder for sequences, implementing the memory component. The resulting
architecture can be efficiently trained by building on closed-form solutions to
linear optimization problems. Further, by exploiting equivalence results
between feedforward and recurrent neural networks we devise a pretraining
schema for the proposed architecture. Experiments on polyphonic music datasets
show competitive results against gated recurrent networks and other state of
the art models.