pwc-archive/papers-with-abstracts huggingface dataset:
Curriculum Learning by Transfer Learning: Theory and Experiments with Deep Networks
Authors: Daphna Weinshall, Gad Cohen, Dan Amir
Date: 2018-02-11 00:00:00
Proceeding: ICML 2018 7
Tasks: Learning Theory, Transfer Learning

We provide theoretical investigation of curriculum learning in the context of
stochastic gradient descent when optimizing the convex linear regression loss.
We prove that the rate of convergence of an ideal curriculum learning method is
monotonically increasing with the difficulty of the examples. Moreover, among
all equally difficult points, convergence is faster when using points which
incur higher loss with respect to the current hypothesis. We then analyze
curriculum learning in the context of training a CNN. We describe a method
which infers the curriculum by way of transfer learning from another network,
pre-trained on a different task. While this approach can only approximate the
ideal curriculum, we observe empirically similar behavior to the one predicted
by the theory, namely, a significant boost in convergence speed at the
beginning of training. When the task is made more difficult, improvement in
generalization performance is also observed. Finally, curriculum learning
exhibits robustness against unfavorable conditions such as excessive
regularization.
Policy Gradient as a Proxy for Dynamic Oracles in Constituency Parsing
Authors: Daniel Fried, Dan Klein
Date: 2018-06-08 00:00:00
Proceeding: ACL 2018 7
Tasks: Constituency Parsing

Dynamic oracles provide strong supervision for training constituency parsers
with exploration, but must be custom defined for a given parser's transition
system. We explore using a policy gradient method as a parser-agnostic
alternative. In addition to directly optimizing for a tree-level metric such as
F1, policy gradient has the potential to reduce exposure bias by allowing
exploration during training; moreover, it does not require a dynamic oracle for
supervision. On four constituency parsers in three languages, the method
substantially outperforms static oracle likelihood training in almost all
settings. For parsers where a dynamic oracle is available (including a novel
oracle which we define for the transition system of Dyer et al. 2016), policy
gradient typically recaptures a substantial fraction of the performance gain
afforded by the dynamic oracle.
Stabiliser states are efficiently PAC-learnable
Authors: Andrea Rocchetto
Date: 2017-04-30 00:00:00
Tasks: Learning Theory, Quantum State Tomography

The exponential scaling of the wave function is a fundamental property of
quantum systems with far reaching implications in our ability to process
quantum information. A problem where these are particularly relevant is quantum
state tomography. State tomography, whose objective is to obtain a full
description of a quantum system, can be analysed in the framework of
computational learning theory. In this model, quantum states have been shown to
be Probably Approximately Correct (PAC)-learnable with sample complexity linear
in the number of qubits. However, it is conjectured that in general quantum
states require an exponential amount of computation to be learned. Here, using
results from the literature on the efficient classical simulation of quantum
systems, we show that stabiliser states are efficiently PAC-learnable. Our
results solve an open problem formulated by Aaronson [Proc. R. Soc. A, 2088,
(2007)] and propose learning theory as a tool for exploring the power of
quantum computation.
Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware
Authors: Florian Tramèr, Dan Boneh
Date: 2018-06-08 00:00:00
Proceeding: ICLR 2019 5
Tasks: GPU

As Machine Learning (ML) gets applied to security-critical or sensitive
domains, there is a growing need for integrity and privacy for outsourced ML
computations. A pragmatic solution comes from Trusted Execution Environments
(TEEs), which use hardware and software protections to isolate sensitive
computations from the untrusted software stack. However, these isolation
guarantees come at a price in performance, compared to untrusted alternatives.
This paper initiates the study of high performance execution of Deep Neural
Networks (DNNs) in TEEs by efficiently partitioning DNN computations between
trusted and untrusted devices. Building upon an efficient outsourcing scheme
for matrix multiplication, we propose Slalom, a framework that securely
delegates execution of all linear layers in a DNN from a TEE (e.g., Intel SGX
or Sanctum) to a faster, yet untrusted, co-located processor. We evaluate
Slalom by running DNNs in an Intel SGX enclave, which selectively delegates
work to an untrusted GPU. For canonical DNNs (VGG16, MobileNet and ResNet
variants) we obtain 6x to 20x increases in throughput for verifiable inference,
and 4x to 11x for verifiable and private inference.
Regression with Comparisons: Escaping the Curse of Dimensionality with Ordinal Information
Authors: Yichong Xu, Sivaraman Balakrishnan, Aarti Singh, Artur Dubrawski
Date: 2018-06-08 00:00:00
Proceeding: ICML 2018 7
Tasks: regression

In supervised learning, we typically leverage a fully labeled dataset to design methods for function estimation or prediction. In many practical situations, we are able to obtain alternative feedback, possibly at a low cost. A broad goal is to understand the usefulness of, and to design algorithms to exploit, this alternative feedback. In this paper, we consider a semi-supervised regression setting, where we obtain additional ordinal (or comparison) information for the unlabeled samples. We consider ordinal feedback of varying qualities where we have either a perfect ordering of the samples, a noisy ordering of the samples or noisy pairwise comparisons between the samples. We provide a precise quantification of the usefulness of these types of ordinal feedback in both nonparametric and linear regression, showing that in many cases it is possible to accurately estimate an underlying function with a very small labeled set, effectively \emph{escaping the curse of dimensionality}. We also present lower bounds, that establish fundamental limits for the task and show that our algorithms are optimal in a variety of settings. Finally, we present extensive experiments on new datasets that demonstrate the efficacy and practicality of our algorithms and investigate their robustness to various sources of noise and model misspecification.
Blind Justice: Fairness with Encrypted Sensitive Attributes
Authors: Niki Kilbertus, Adrià Gascón, Matt J. Kusner, Michael Veale, Krishna P. Gummadi, Adrian Weller
Date: 2018-06-08 00:00:00
Proceeding: ICML 2018 7
Tasks: Fairness

Recent work has explored how to train machine learning models which do not
discriminate against any subgroup of the population as determined by sensitive
attributes such as gender or race. To avoid disparate treatment, sensitive
attributes should not be considered. On the other hand, in order to avoid
disparate impact, sensitive attributes must be examined, e.g., in order to
learn a fair model, or to check if a given model is fair. We introduce methods
from secure multi-party computation which allow us to avoid both. By encrypting
sensitive attributes, we show how an outcome-based fair model may be learned,
checked, or have its outputs verified and held to account, without users
revealing their sensitive attributes.
Multilingual Neural Machine Translation with Task-Specific Attention
Authors: Graeme Blackwood, Miguel Ballesteros, Todd Ward
Date: 2018-06-08 00:00:00
Proceeding: COLING 2018 8
Tasks: Machine Translation, NMT, Translation

Multilingual machine translation addresses the task of translating between
multiple source and target languages. We propose task-specific attention
models, a simple but effective technique for improving the quality of
sequence-to-sequence neural multilingual translation. Our approach seeks to
retain as much of the parameter sharing generalization of NMT models as
possible, while still allowing for language-specific specialization of the
attention model to a particular language-pair or task. Our experiments on four
languages of the Europarl corpus show that using a target-specific model of
attention provides consistent gains in translation quality for all possible
translation directions, compared to a model in which all parameters are shared.
We observe improved translation quality even in the (extreme) low-resource
zero-shot translation directions for which the model never saw explicitly
paired parallel data.
Towards Dependability Metrics for Neural Networks
Authors: Chih-Hong Cheng, Georg Nührenberg, Chung-Hao Huang, Harald Ruess, Hirotoshi Yasuoka
Date: 2018-06-06 00:00:00

Artificial neural networks (NN) are instrumental in realizing
highly-automated driving functionality. An overarching challenge is to identify
best safety engineering practices for NN and other learning-enabled components.
In particular, there is an urgent need for an adequate set of metrics for
measuring all-important NN dependability attributes. We address this challenge
by proposing a number of NN-specific and efficiently computable metrics for
measuring NN dependability attributes including robustness, interpretability,
completeness, and correctness.
CuisineNet: Food Attributes Classification using Multi-scale Convolution Network
Authors: Md. Mostafa Kamal Sarker, Mohammed Jabreel, Hatem A. Rashwan, Syeda Furruka Banu, Antonio Moreno, Petia Radeva, Domenec Puig
Date: 2018-05-30 00:00:00
Tasks: Classification, Cultural Vocal Bursts Intensity Prediction, Diversity, General Classification, Multi-modal Classification

Diversity of food and its attributes represents the culinary habits of
peoples from different countries. Thus, this paper addresses the problem of
identifying food culture of people around the world and its flavor by
classifying two main food attributes, cuisine and flavor. A deep learning model
based on multi-scale convotuional networks is proposed for extracting more
accurate features from input images. The aggregation of multi-scale convolution
layers with different kernel size is also used for weighting the features
results from different scales. In addition, a joint loss function based on
Negative Log Likelihood (NLL) is used to fit the model probability to multi
labeled classes for multi-modal classification task. Furthermore, this work
provides a new dataset for food attributes, so-called Yummly48K, extracted from
the popular food website, Yummly. Our model is assessed on the constructed
Yummly48K dataset. The experimental results show that our proposed method
yields 65% and 62% average F1 score on validation and test set which
outperforming the state-of-the-art models.
Orbital Petri Nets: A Novel Petri Net Approach
Authors: Mohamed Yorky, Aboul Ella Hassanien
Date: 2018-06-08 00:00:00

Petri Nets is very interesting tool for studying and simulating different
behaviors of information systems. It can be used in different applications
based on the appropriate class of Petri Nets whereas it is classical, colored
or timed Petri Nets. In this paper we introduce a new approach of Petri Nets
called orbital Petri Nets (OPN) for studying the orbital rotating systems
within a specific domain. The study investigated and analyzed OPN with
highlighting the problem of space debris collision problem as a case study. The
mathematical investigation results of two OPN models proved that space debris
collision problem can be prevented based on the new method of firing sequence
in OPN. By this study, new smart algorithms can be implemented and simulated by
orbital Petri Nets for mitigating the space debris collision problem as a next
work.
Sheep identity recognition, age and weight estimation datasets
Authors: Aya Salama Abdelhady, Aboul Ella Hassanenin, Aly Fahmy
Date: 2018-06-08 00:00:00
Tasks: Management

Increased interest of scientists, producers and consumers in sheep
identification has been stimulated by the dramatic increase in population and
the urge to increase productivity. The world population is expected to exceed
9.6 million in 2050. For this reason, awareness is raised towards the necessity
of effective livestock production. Sheep is considered as one of the main of
food resources. Most of the research now is directed towards developing real
time applications that facilitate sheep identification for breed management and
gathering related information like weight and age. Weight and age are key
matrices in assessing the effectiveness of production. For this reason, visual
analysis proved recently its significant success over other approaches. Visual
analysis techniques need enough images for testing and study completion. For
this reason, collecting sheep images database is a vital step to fulfill such
objective. We provide here datasets for testing and comparing such algorithms
which are under development. Our collected dataset consists of 416 color images
for different features of sheep in different postures. Images were collected
fifty two sheep at a range of year from three months to six years. For each
sheep, two images were captured for both sides of the body, two images for both
sides of the face, one image from the top view, one image for the hip and one
image for the teeth. The collected images cover different illumination, quality
levels and angle of rotation. The allocated data set can be used to test sheep
identification, weigh estimation, and age detection algorithms. Such algorithms
are crucial for disease management, animal assessment and ownership.
PatchFCN for Intracranial Hemorrhage Detection
Authors: Wei-cheng Kuo, Christian Häne, Esther Yuh, Pratik Mukherjee, Jitendra Malik
Date: 2018-06-08 00:00:00
Tasks: Computed Tomography (CT), Diversity, object-detection, Object Detection, Segmentation, Semantic Segmentation

This paper studies the problem of detecting and segmenting acute intracranial
hemorrhage on head computed tomography (CT) scans. We propose to solve both
tasks as a semantic segmentation problem using a patch-based fully
convolutional network (PatchFCN). This formulation allows us to accurately
localize hemorrhages while bypassing the complexity of object detection. Our
system demonstrates competitive performance with a human expert and the
state-of-the-art on classification tasks (0.976, 0.966 AUC of ROC on
retrospective and prospective test sets) and on segmentation tasks (0.785 pixel
AP, 0.766 Dice score), while using much less data and a simpler system. In
addition, we conduct a series of controlled experiments to understand "why"
PatchFCN outperforms standard FCN. Our studies show that PatchFCN finds a good
trade-off between batch diversity and the amount of context during training.
These findings may also apply to other medical segmentation tasks.
Information-based inference for singular models and finite sample sizes: A frequentist information criterion
Authors: Colin H. LaMont, Paul A. Wiggins
Date: 2015-06-19 00:00:00
Tasks: Model Selection

In the information-based paradigm of inference, model selection is performed
by selecting the candidate model with the best estimated predictive
performance. The success of this approach depends on the accuracy of the
estimate of the predictive complexity. In the large-sample-size limit of a
regular model, the predictive performance is well estimated by the Akaike
Information Criterion (AIC). However, this approximation can either
significantly under or over-estimating the complexity in a wide range of
important applications where models are either non-regular or
finite-sample-size corrections are significant. We introduce an improved
approximation for the complexity that is used to define a new information
criterion: the Frequentist Information Criterion (QIC). QIC extends the
applicability of information-based inference to the finite-sample-size regime
of regular models and to singular models. We demonstrate the power and the
comparative advantage of QIC in a number of example analyses.
Deep learning with convolutional neural networks for EEG decoding and visualization
Authors: Robin Tibor Schirrmeister, Jost Tobias Springenberg, Lukas Dominique Josef Fiederer, Martin Glasstetter, Katharina Eggensperger, Michael Tangermann, Frank Hutter, Wolfram Burgard, Tonio Ball
Date: 2017-03-15 00:00:00
Tasks: EEG, Eeg Decoding, Electroencephalogram (EEG)

PLEASE READ AND CITE THE REVISED VERSION at Human Brain Mapping:
http://onlinelibrary.wiley.com/doi/10.1002/hbm.23730/full
  Code available here: https://github.com/robintibor/braindecode
Automatic View Planning with Multi-scale Deep Reinforcement Learning Agents
Authors: Amir Alansary, Loic Le Folgoc, Ghislain Vaillant, Ozan Oktay, Yuanwei Li, Wenjia Bai, Jonathan Passerat-Palmbach, Ricardo Guerrero, Konstantinos Kamnitsas, Benjamin Hou, Steven McDonagh, Ben Glocker, Bernhard Kainz, Daniel Rueckert
Date: 2018-06-08 00:00:00
Tasks: Anatomy, Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We propose a fully automatic method to find standardized view planes in 3D
image acquisitions. Standard view images are important in clinical practice as
they provide a means to perform biometric measurements from similar anatomical
regions. These views are often constrained to the native orientation of a 3D
image acquisition. Navigating through target anatomy to find the required view
plane is tedious and operator-dependent. For this task, we employ a multi-scale
reinforcement learning (RL) agent framework and extensively evaluate several
Deep Q-Network (DQN) based strategies. RL enables a natural learning paradigm
by interaction with the environment, which can be used to mimic experienced
operators. We evaluate our results using the distance between the anatomical
landmarks and detected planes, and the angles between their normal vector and
target. The proposed algorithm is assessed on the mid-sagittal and
anterior-posterior commissure planes of brain MRI, and the 4-chamber long-axis
plane commonly used in cardiac MRI, achieving accuracy of 1.53mm, 1.98mm and
4.84mm, respectively.
Concurrent Spatial and Channel Squeeze & Excitation in Fully Convolutional Networks
Authors: Abhijit Guha Roy, Nassir Navab, Christian Wachinger
Date: 2018-03-07 00:00:00
Tasks: Brain Segmentation, image-classification, Image Classification, Image Segmentation, Organ Segmentation, Segmentation, Semantic Segmentation

Fully convolutional neural networks (F-CNNs) have set the state-of-the-art in
image segmentation for a plethora of applications. Architectural innovations
within F-CNNs have mainly focused on improving spatial encoding or network
connectivity to aid gradient flow. In this paper, we explore an alternate
direction of recalibrating the feature maps adaptively, to boost meaningful
features, while suppressing weak ones. We draw inspiration from the recently
proposed squeeze & excitation (SE) module for channel recalibration of feature
maps for image classification. Towards this end, we introduce three variants of
SE modules for image segmentation, (i) squeezing spatially and exciting
channel-wise (cSE), (ii) squeezing channel-wise and exciting spatially (sSE)
and (iii) concurrent spatial and channel squeeze & excitation (scSE). We
effectively incorporate these SE modules within three different
state-of-the-art F-CNNs (DenseNet, SD-Net, U-Net) and observe consistent
improvement of performance across all architectures, while minimally effecting
model complexity. Evaluations are performed on two challenging applications:
whole brain segmentation on MRI scans (Multi-Atlas Labelling Challenge Dataset)
and organ segmentation on whole body contrast enhanced CT scans (Visceral
Dataset).
Inherent Brain Segmentation Quality Control from Fully ConvNet Monte Carlo Sampling
Authors: Abhijit Guha Roy, Sailesh Conjeti, Nassir Navab, Christian Wachinger
Date: 2018-04-19 00:00:00
Tasks: Brain Segmentation, Segmentation, Translation

We introduce inherent measures for effective quality control of brain
segmentation based on a Bayesian fully convolutional neural network, using
model uncertainty. Monte Carlo samples from the posterior distribution are
efficiently generated using dropout at test time. Based on these samples, we
introduce next to a voxel-wise uncertainty map also three metrics for
structure-wise uncertainty. We then incorporate these structure-wise
uncertainty in group analyses as a measure of confidence in the observation.
Our results show that the metrics are highly correlated to segmentation
accuracy and therefore present an inherent measure of segmentation quality.
Furthermore, group analysis with uncertainty results in effect sizes closer to
that of manual annotations. The introduced uncertainty metrics can not only be
very useful in translation to clinical practice but also provide automated
quality control and group analyses in processing large data repositories.
ChangeMyView Through Concessions: Do Concessions Increase Persuasion?
Authors: Elena Musi, Debanjan Ghosh, Smaranda Muresan
Date: 2018-06-08 00:00:00

In discourse studies concessions are considered among those argumentative
strategies that increase persuasion. We aim to empirically test this hypothesis
by calculating the distribution of argumentative concessions in persuasive vs.
non-persuasive comments from the ChangeMyView subreddit. This constitutes a
challenging task since concessions are not always part of an argument. Drawing
from a theoretically-informed typology of concessions, we conduct an annotation
task to label a set of polysemous lexical markers as introducing an
argumentative concession or not and we observe their distribution in threads
that achieved and did not achieve persuasion. For the annotation, we used both
expert and novice annotators. With the ultimate goal of conducting the study on
large datasets, we present a self-training method to automatically identify
argumentative concessions using linguistically motivated features. We achieve a
moderate F1 of 57.4% on the development set and 46.0% on the test set via the
self-training method. These results are comparable to state of the art results
on similar tasks of identifying explicit discourse connective types from the
Penn Discourse Treebank. Our findings from the manual labeling and the
classification experiments indicate that the type of argumentative concessions
we investigated is almost equally likely to be used in winning and losing
arguments from the ChangeMyView dataset. While this result seems to contradict
theoretical assumptions, we provide some reasons for this discrepancy related
to the ChangeMyView subreddit.
Learning to Reweight Examples for Robust Deep Learning
Authors: Mengye Ren, Wenyuan Zeng, Bin Yang, Raquel Urtasun
Date: 2018-03-24 00:00:00
Proceeding: ICML 2018 7
Tasks: Deep Learning, Meta-Learning

Deep neural networks have been shown to be very powerful modeling tools for many supervised learning tasks involving complex input patterns. However, they can also easily overfit to training set biases and label noises. In addition to various regularizers, example reweighting algorithms are popular solutions to these problems, but they require careful tuning of additional hyperparameters, such as example mining schedules and regularization hyperparameters. In contrast to past reweighting methods, which typically consist of functions of the cost value of each example, in this work we propose a novel meta-learning algorithm that learns to assign weights to training examples based on their gradient directions. To determine the example weights, our method performs a meta gradient descent step on the current mini-batch example weights (which are initialized from zero) to minimize the loss on a clean unbiased validation set. Our proposed method can be easily implemented on any type of deep network, does not require any additional hyperparameter tuning, and achieves impressive performance on class imbalance and corrupted label problems where only a small amount of clean validation data is available.