pwc-archive/papers-with-abstracts huggingface dataset:
MILD-Net: Minimal Information Loss Dilated Network for Gland Instance Segmentation in Colon Histology Images
Authors: Simon Graham, Hao Chen, Jevgenij Gamper, Qi Dou, Pheng-Ann Heng, David Snead, Yee Wah Tsang, Nasir Rajpoot
Date: 2018-06-05 00:00:00
Tasks: Colorectal Gland Segmentation:, Decision Making, Diagnostic, Instance Segmentation, Segmentation, Semantic Segmentation, whole slide images

The analysis of glandular morphology within colon histopathology images is an
important step in determining the grade of colon cancer. Despite the importance
of this task, manual segmentation is laborious, time-consuming and can suffer
from subjectivity among pathologists. The rise of computational pathology has
led to the development of automated methods for gland segmentation that aim to
overcome the challenges of manual segmentation. However, this task is
non-trivial due to the large variability in glandular appearance and the
difficulty in differentiating between certain glandular and non-glandular
histological structures. Furthermore, a measure of uncertainty is essential for
diagnostic decision making. To address these challenges, we propose a fully
convolutional neural network that counters the loss of information caused by
max-pooling by re-introducing the original image at multiple points within the
network. We also use atrous spatial pyramid pooling with varying dilation rates
for preserving the resolution and multi-level aggregation. To incorporate
uncertainty, we introduce random transformations during test time for an
enhanced segmentation result that simultaneously generates an uncertainty map,
highlighting areas of ambiguity. We show that this map can be used to define a
metric for disregarding predictions with high uncertainty. The proposed network
achieves state-of-the-art performance on the GlaS challenge dataset and on a
second independent colorectal adenocarcinoma dataset. In addition, we perform
gland instance segmentation on whole-slide images from two further datasets to
highlight the generalisability of our method. As an extension, we introduce
MILD-Net+ for simultaneous gland and lumen segmentation, to increase the
diagnostic power of the network.
Domain Adversarial Training for Accented Speech Recognition
Authors: Sining Sun, Ching-Feng Yeh, Mei-Yuh Hwang, Mari Ostendorf, Lei Xie
Date: 2018-06-07 00:00:00
Tasks: Accented Speech Recognition, Multi-Task Learning, speech-recognition, Speech Recognition

In this paper, we propose a domain adversarial training (DAT) algorithm to
alleviate the accented speech recognition problem. In order to reduce the
mismatch between labeled source domain data ("standard" accent) and unlabeled
target domain data (with heavy accents), we augment the learning objective for
a Kaldi TDNN network with a domain adversarial training (DAT) objective to
encourage the model to learn accent-invariant features. In experiments with
three Mandarin accents, we show that DAT yields up to 7.45% relative character
error rate reduction when we do not have transcriptions of the accented speech,
compared with the baseline trained on standard accent data only. We also find a
benefit from DAT when used in combination with training from automatic
transcriptions on the accented data. Furthermore, we find that DAT is superior
to multi-task learning for accented speech recognition.
Zeroth-Order Stochastic Variance Reduction for Nonconvex Optimization
Authors: Sijia Liu, Bhavya Kailkhura, Pin-Yu Chen, Pai-Shun Ting, Shiyu Chang, Lisa Amini
Date: 2018-05-25 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Material Classification, Stochastic Optimization

As application demands for zeroth-order (gradient-free) optimization
accelerate, the need for variance reduced and faster converging approaches is
also intensifying. This paper addresses these challenges by presenting: a) a
comprehensive theoretical analysis of variance reduced zeroth-order (ZO)
optimization, b) a novel variance reduced ZO algorithm, called ZO-SVRG, and c)
an experimental evaluation of our approach in the context of two compelling
applications, black-box chemical material classification and generation of
adversarial examples from black-box deep neural network models. Our theoretical
analysis uncovers an essential difficulty in the analysis of ZO-SVRG: the
unbiased assumption on gradient estimates no longer holds. We prove that
compared to its first-order counterpart, ZO-SVRG with a two-point random
gradient estimator could suffer an additional error of order $O(1/b)$, where
$b$ is the mini-batch size. To mitigate this error, we propose two accelerated
versions of ZO-SVRG utilizing variance reduced gradient estimators, which
achieve the best rate known for ZO stochastic optimization (in terms of
iterations). Our extensive experimental results show that our approaches
outperform other state-of-the-art ZO algorithms, and strike a balance between
the convergence rate and the function query complexity.
More Adaptive Algorithms for Adversarial Bandits
Authors: Chen-Yu Wei, Haipeng Luo
Date: 2018-01-10 00:00:00

We develop a novel and generic algorithm for the adversarial multi-armed
bandit problem (or more generally the combinatorial semi-bandit problem). When
instantiated differently, our algorithm achieves various new data-dependent
regret bounds improving previous work. Examples include: 1) a regret bound
depending on the variance of only the best arm; 2) a regret bound depending on
the first-order path-length of only the best arm; 3) a regret bound depending
on the sum of first-order path-lengths of all arms as well as an important
negative term, which together lead to faster convergence rates for some normal
form games with partial feedback; 4) a regret bound that simultaneously implies
small regret when the best arm has small loss and logarithmic regret when there
exists an arm whose expected loss is always smaller than those of others by a
fixed gap (e.g. the classic i.i.d. setting). In some cases, such as the last
two results, our algorithm is completely parameter-free.
  The main idea of our algorithm is to apply the optimism and adaptivity
techniques to the well-known Online Mirror Descent framework with a special
log-barrier regularizer. The challenges are to come up with appropriate
optimistic predictions and correction terms in this framework. Some of our
results also crucially rely on using a sophisticated increasing learning rate
schedule.
Stein Variational Gradient Descent Without Gradient
Authors: Jun Han, Qiang Liu
Date: 2018-06-07 00:00:00
Proceeding: ICML 2018 7

Stein variational gradient decent (SVGD) has been shown to be a powerful
approximate inference algorithm for complex distributions. However, the
standard SVGD requires calculating the gradient of the target density and
cannot be applied when the gradient is unavailable. In this work, we develop a
gradient-free variant of SVGD (GF-SVGD), which replaces the true gradient with
a surrogate gradient, and corrects the induced bias by re-weighting the
gradients in a proper form. We show that our GF-SVGD can be viewed as the
standard SVGD with a special choice of kernel, and hence directly inherits the
theoretical properties of SVGD. We shed insights on the empirical choice of the
surrogate gradient and propose an annealed GF-SVGD that leverages the idea of
simulated annealing to improve the performance on high dimensional complex
distributions. Empirical studies show that our method consistently outperforms
a number of recent advanced gradient-free MCMC methods.
Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
Authors: Anish Athalye, Nicholas Carlini, David Wagner
Date: 2018-02-01 00:00:00
Proceeding: ICML 2018 7
Tasks: Adversarial Attack, Adversarial Defense

We identify obfuscated gradients, a kind of gradient masking, as a phenomenon
that leads to a false sense of security in defenses against adversarial
examples. While defenses that cause obfuscated gradients appear to defeat
iterative optimization-based attacks, we find defenses relying on this effect
can be circumvented. We describe characteristic behaviors of defenses
exhibiting the effect, and for each of the three types of obfuscated gradients
we discover, we develop attack techniques to overcome it. In a case study,
examining non-certified white-box-secure defenses at ICLR 2018, we find
obfuscated gradients are a common occurrence, with 7 of 9 defenses relying on
obfuscated gradients. Our new attacks successfully circumvent 6 completely, and
1 partially, in the original threat model each paper considers.
Synthesizing Robust Adversarial Examples
Authors: Anish Athalye, Logan Engstrom, Andrew Ilyas, Kevin Kwok
Date: 2017-07-24 00:00:00

Standard methods for generating adversarial examples for neural networks do
not consistently fool neural network classifiers in the physical world due to a
combination of viewpoint shifts, camera noise, and other natural
transformations, limiting their relevance to real-world systems. We demonstrate
the existence of robust 3D adversarial objects, and we present the first
algorithm for synthesizing examples that are adversarial over a chosen
distribution of transformations. We synthesize two-dimensional adversarial
images that are robust to noise, distortion, and affine transformation. We
apply our algorithm to complex three-dimensional objects, using 3D-printing to
manufacture the first physical adversarial objects. Our results demonstrate the
existence of 3D adversarial objects in the physical world.
DEEPEYE: A Compact and Accurate Video Comprehension at Terminal Devices Compressed with Quantization and Tensorization
Authors: Yuan Cheng, Guangya Li, Hai-Bao Chen, Sheldon X. -D. Tan, Hao Yu
Date: 2018-05-21 00:00:00
Tasks: Action Recognition, General Classification, Model Compression, object-detection, Object Detection, Quantization, Temporal Action Localization

As it requires a huge number of parameters when exposed to high dimensional
inputs in video detection and classification, there is a grand challenge to
develop a compact yet accurate video comprehension at terminal devices. Current
works focus on optimizations of video detection and classification in a
separated fashion. In this paper, we introduce a video comprehension (object
detection and action recognition) system for terminal devices, namely DEEPEYE.
Based on You Only Look Once (YOLO), we have developed an 8-bit quantization
method when training YOLO; and also developed a tensorized-compression method
of Recurrent Neural Network (RNN) composed of features extracted from YOLO. The
developed quantization and tensorization can significantly compress the
original network model yet with maintained accuracy. Using the challenging
video datasets: MOMENTS and UCF11 as benchmarks, the results show that the
proposed DEEPEYE achieves 3.994x model compression rate with only 0.47% mAP
decreased; and 15,047x parameter reduction and 2.87x speed-up with 16.58%
accuracy improvement.
Evaluating surgical skills from kinematic data using convolutional neural networks
Authors: Hassan Ismail Fawaz, Germain Forestier, Jonathan Weber, Lhassane Idoumghar, Pierre-Alain Muller
Date: 2018-06-07 00:00:00
Tasks: General Classification, Skills Assessment, Skills Evaluation, Surgical Skills Evaluation

The need for automatic surgical skills assessment is increasing, especially
because manual feedback from senior surgeons observing junior surgeons is prone
to subjectivity and time consuming. Thus, automating surgical skills evaluation
is a very important step towards improving surgical practice. In this paper, we
designed a Convolutional Neural Network (CNN) to evaluate surgeon skills by
extracting patterns in the surgeon motions performed in robotic surgery. The
proposed method is validated on the JIGSAWS dataset and achieved very
competitive results with 100% accuracy on the suturing and needle passing
tasks. While we leveraged from the CNNs efficiency, we also managed to mitigate
its black-box effect using class activation map. This feature allows our method
to automatically highlight which parts of the surgical task influenced the
skill prediction and can be used to explain the classification and to provide
personalized feedback to the trainee.
Discovering space - Grounding spatial topology and metric regularity in a naive agent's sensorimotor experience
Authors: Alban Laflaquière, J. Kevin O'Regan, Bruno Gas, Alexander Terekhov
Date: 2018-06-07 00:00:00

In line with the sensorimotor contingency theory, we investigate the problem
of the perception of space from a fundamental sensorimotor perspective. Despite
its pervasive nature in our perception of the world, the origin of the concept
of space remains largely mysterious. For example in the context of artificial
perception, this issue is usually circumvented by having engineers pre-define
the spatial structure of the problem the agent has to face. We here show that
the structure of space can be autonomously discovered by a naive agent in the
form of sensorimotor regularities, that correspond to so called compensable
sensory experiences: these are experiences that can be generated either by the
agent or its environment. By detecting such compensable experiences the agent
can infer the topological and metric structure of the external space in which
its body is moving. We propose a theoretical description of the nature of these
regularities and illustrate the approach on a simulated robotic arm equipped
with an eye-like sensor, and which interacts with an object. Finally we show
how these regularities can be used to build an internal representation of the
sensor's external spatial configuration.
Joint mean and covariance estimation with unreplicated matrix-variate data
Authors: Michael Hornstein, Roger Fan, Kerby Shedden, Shuheng Zhou
Date: 2016-11-13 00:00:00

It has been proposed that complex populations, such as those that arise in
genomics studies, may exhibit dependencies among observations as well as among
variables. This gives rise to the challenging problem of analyzing unreplicated
high-dimensional data with unknown mean and dependence structures.
Matrix-variate approaches that impose various forms of (inverse) covariance
sparsity allow flexible dependence structures to be estimated, but cannot
directly be applied when the mean and covariance matrices are estimated
jointly. We present a practical method utilizing generalized least squares and
penalized (inverse) covariance estimation to address this challenge. We
establish consistency and obtain rates of convergence for estimating the mean
parameters and covariance matrices. The advantages of our approaches are: (i)
dependence graphs and covariance structures can be estimated in the presence of
unknown mean structure, (ii) the mean structure becomes more efficiently
estimated when accounting for the dependence structure among observations; and
(iii) inferences about the mean parameters become correctly calibrated. We use
simulation studies and analysis of genomic data from a twin study of ulcerative
colitis to illustrate the statistical convergence and the performance of our
methods in practical settings. Several lines of evidence show that the test
statistics for differential gene expression produced by our methods are
correctly calibrated and improve power over conventional methods.
Where are we now? A large benchmark study of recent symbolic regression methods
Authors: Patryk Orzechowski, William La Cava, Jason H. Moore
Date: 2018-04-25 00:00:00
Tasks: Benchmarking, BIG-bench Machine Learning, regression, Symbolic Regression

In this paper we provide a broad benchmarking of recent genetic programming
approaches to symbolic regression in the context of state of the art machine
learning approaches. We use a set of nearly 100 regression benchmark problems
culled from open source repositories across the web. We conduct a rigorous
benchmarking of four recent symbolic regression approaches as well as nine
machine learning approaches from scikit-learn. The results suggest that
symbolic regression performs strongly compared to state-of-the-art gradient
boosting algorithms, although in terms of running times is among the slowest of
the available methodologies. We discuss the results in detail and point to
future research directions that may allow symbolic regression to gain wider
adoption in the machine learning community.
Online Adaptive Machine Learning Based Algorithm for Implied Volatility Surface Modeling
Authors: Yaxiong Zeng, Diego Klabjan
Date: 2017-06-06 00:00:00
Tasks: BIG-bench Machine Learning, CPU, regression

In this work, we design a machine learning based method, online adaptive
primal support vector regression (SVR), to model the implied volatility surface
(IVS). The algorithm proposed is the first derivation and implementation of an
online primal kernel SVR. It features enhancements that allow efficient online
adaptive learning by embedding the idea of local fitness and budget maintenance
to dynamically update support vectors upon pattern drifts. For algorithm
acceleration, we implement its most computationally intensive parts in a Field
Programmable Gate Arrays hardware, where a 132x speedup over CPU is achieved
during online prediction. Using intraday tick data from the E-mini S&P 500
options market, we show that the Gaussian kernel outperforms the linear kernel
in regulating the size of support vectors, and that our empirical IVS algorithm
beats two competing online methods with regards to model complexity and
regression errors (the mean absolute percentage error of our algorithm is up to
13%). Best results are obtained at the center of the IVS grid due to its larger
number of adjacent support vectors than the edges of the grid. Sensitivity
analysis is also presented to demonstrate how hyper parameters affect the error
rates and model complexity.
Can Machines Design? An Artificial General Intelligence Approach
Authors: Andreas Makoto Hein, Hélène Condat
Date: 2018-06-06 00:00:00
Tasks: Novel Concepts

Can machines design? Can they come up with creative solutions to problems and
build tools and artifacts across a wide range of domains? Recent advances in
the field of computational creativity and formal Artificial General
Intelligence (AGI) provide frameworks for machines with the general ability to
design. In this paper we propose to integrate a formal computational creativity
framework into the G\"odel machine framework. We call the resulting framework
design G\"odel machine. Such a machine could solve a variety of design problems
by generating novel concepts. In addition, it could change the way these
concepts are generated by modifying itself. The design G\"odel machine is able
to improve its initial design program, once it has proven that a modification
would increase its return on the utility function. Finally, we sketch out a
specific version of the design G\"odel machine which specifically addresses the
design of complex software and hardware systems. Future work aims at the
development of a more formal version of the design G\"odel machine and a proof
of concept implementation.
Assessing the impact of machine intelligence on human behaviour: an interdisciplinary endeavour
Authors: Emilia Gómez, Carlos Castillo, Vicky Charisi, Verónica Dahl, Gustavo Deco, Blagoj Delipetrev, Nicole Dewandre, Miguel Ángel González-Ballester, Fabien Gouyon, José Hernández-Orallo, Perfecto Herrera, Anders Jonsson, Ansgar Koene, Martha Larson, Ramón López de Mántaras, Bertin Martens, Marius Miron, Rubén Moreno-Bote, Nuria Oliver, Antonio Puertas Gallardo, Heike Schweitzer, Nuria Sebastian, Xavier Serra, Joan Serrà, Songül Tolan, Karina Vold
Date: 2018-06-07 00:00:00
Tasks: Decision Making

This document contains the outcome of the first Human behaviour and machine
intelligence (HUMAINT) workshop that took place 5-6 March 2018 in Barcelona,
Spain. The workshop was organized in the context of a new research programme at
the Centre for Advanced Studies, Joint Research Centre of the European
Commission, which focuses on studying the potential impact of artificial
intelligence on human behaviour. The workshop gathered an interdisciplinary
group of experts to establish the state of the art research in the field and a
list of future research challenges to be addressed on the topic of human and
machine intelligence, algorithm's potential impact on human cognitive
capabilities and decision making, and evaluation and regulation needs. The
document is made of short position statements and identification of challenges
provided by each expert, and incorporates the result of the discussions carried
out during the workshop. In the conclusion section, we provide a list of
emerging research topics and strategies to be addressed in the near future.
Taylor's law for Human Linguistic Sequences
Authors: Tatsuru Kobayashi, Kumiko Tanaka-Ishii
Date: 2018-04-21 00:00:00
Proceeding: ACL 2018 7
Tasks: Time Series, Time Series Analysis

Taylor's law describes the fluctuation characteristics underlying a system in
which the variance of an event within a time span grows by a power law with
respect to the mean. Although Taylor's law has been applied in many natural and
social systems, its application for language has been scarce. This article
describes a new quantification of Taylor's law in natural language and reports
an analysis of over 1100 texts across 14 languages. The Taylor exponents of
written natural language texts were found to exhibit almost the same value. The
exponent was also compared for other language-related data, such as the
child-directed speech, music, and programming language code. The results show
how the Taylor exponent serves to quantify the fundamental structural
complexity underlying linguistic time series. The article also shows the
applicability of these findings in evaluating language models.
Speaker-Follower Models for Vision-and-Language Navigation
Authors: Daniel Fried, Ronghang Hu, Volkan Cirik, Anna Rohrbach, Jacob Andreas, Louis-Philippe Morency, Taylor Berg-Kirkpatrick, Kate Saenko, Dan Klein, Trevor Darrell
Date: 2018-06-07 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Data Augmentation, Vision and Language Navigation

Navigation guided by natural language instructions presents a challenging
reasoning problem for instruction followers. Natural language instructions
typically identify only a few high-level decisions and landmarks rather than
complete low-level motor behaviors; much of the missing information must be
inferred based on perceptual context. In machine learning settings, this is
doubly challenging: it is difficult to collect enough annotated data to enable
learning of this reasoning process from scratch, and also difficult to
implement the reasoning process using generic sequence models. Here we describe
an approach to vision-and-language navigation that addresses both these issues
with an embedded speaker model. We use this speaker model to (1) synthesize new
instructions for data augmentation and to (2) implement pragmatic reasoning,
which evaluates how well candidate action sequences explain an instruction.
Both steps are supported by a panoramic action space that reflects the
granularity of human-generated instructions. Experiments show that all three
components of this approach---speaker-driven data augmentation, pragmatic
reasoning and panoramic action space---dramatically improve the performance of
a baseline instruction follower, more than doubling the success rate over the
best existing approach on a standard benchmark.