pwc-archive/papers-with-abstracts huggingface dataset:
Convolutional Neural Network Quantization using Generalized Gamma Distribution
Authors: Doyun Kim, Han Young Yim, Sanghyuck Ha, Changgwun Lee, Inyup Kang
Date: 2018-10-31 00:00:00
Tasks: Quantization

As edge applications using convolutional neural networks (CNN) models grow,
it is becoming necessary to introduce dedicated hardware accelerators in which
network parameters and feature-map data are represented with limited precision.
In this paper we propose a novel quantization algorithm for energy-efficient
deployment of the hardware accelerators. For weights and biases, the optimal
bit length of the fractional part is determined so that the quantization error
is minimized over their distribution. For feature-map data, meanwhile, their
sample distribution is well approximated with the generalized gamma
distribution (GGD), and accordingly the optimal quantization step size can be
obtained through the asymptotical closed form solution of GGD. The proposed
quantization algorithm has a higher signal-to-quantization-noise ratio (SQNR)
than other quantization schemes previously proposed for CNNs, and even can be
more improved by tuning the quantization parameters, resulting in efficient
implementation of the hardware accelerators for CNNs in terms of power
consumption and memory bandwidth.
Cross-Lingual Transfer Learning for Multilingual Task Oriented Dialog
Authors: Sebastian Schuster, Sonal Gupta, Rushin Shah, Mike Lewis
Date: 2018-10-31 00:00:00
Proceeding: NAACL 2019 6
Tasks: Cross-Lingual Transfer, Machine Translation, Transfer Learning

One of the first steps in the utterance interpretation pipeline of many
task-oriented conversational AI systems is to identify user intents and the
corresponding slots. Since data collection for machine learning models for this
task is time-consuming, it is desirable to make use of existing data in a
high-resource language to train models in low-resource languages. However,
development of such models has largely been hindered by the lack of
multilingual training data. In this paper, we present a new data set of 57k
annotated utterances in English (43k), Spanish (8.6k) and Thai (5k) across the
domains weather, alarm, and reminder. We use this data set to evaluate three
different cross-lingual transfer methods: (1) translating the training data,
(2) using cross-lingual pre-trained embeddings, and (3) a novel method of using
a multilingual machine translation encoder as contextual word representations.
We find that given several hundred training examples in the the target
language, the latter two methods outperform translating the training data.
Further, in very low-resource settings, multilingual contextual word
representations give better results than using cross-lingual static embeddings.
We also compare the cross-lingual methods to using monolingual resources in the
form of contextual ELMo representations and find that given just small amounts
of target language data, this method outperforms all cross-lingual methods,
which highlights the need for more sophisticated cross-lingual methods.
Convolutional Self-Attention Network
Authors: Baosong Yang, Long-Yue Wang, Derek F. Wong, Lidia S. Chao, Zhaopeng Tu
Date: 2018-10-31 00:00:00
Tasks: Translation

Self-attention network (SAN) has recently attracted increasing interest due
to its fully parallelized computation and flexibility in modeling dependencies.
It can be further enhanced with multi-headed attention mechanism by allowing
the model to jointly attend to information from different representation
subspaces at different positions (Vaswani et al., 2017). In this work, we
propose a novel convolutional self-attention network (CSAN), which offers SAN
the abilities to 1) capture neighboring dependencies, and 2) model the
interaction between multiple attention heads. Experimental results on WMT14
English-to-German translation task demonstrate that the proposed approach
outperforms both the strong Transformer baseline and other existing works on
enhancing the locality of SAN. Comparing with previous work, our model does not
introduce any new parameters.
Contrastive Multivariate Singular Spectrum Analysis
Authors: Abdi-Hakin Dirie, Abubakar Abid, James Zou
Date: 2018-10-31 00:00:00
Tasks: Clustering, Dimensionality Reduction, Time Series, Time Series Analysis

We introduce Contrastive Multivariate Singular Spectrum Analysis, a novel
unsupervised method for dimensionality reduction and signal decomposition of
time series data. By utilizing an appropriate background dataset, the method
transforms a target time series dataset in a way that evinces the sub-signals
that are enhanced in the target dataset, as opposed to only those that account
for the greatest variance. This shifts the goal from finding signals that
explain the most variance to signals that matter the most to the analyst. We
demonstrate our method on an illustrative synthetic example, as well as show
the utility of our method in the downstream clustering of electrocardiogram
signals from the public MHEALTH dataset.
Crowdsourcing with Fairness, Diversity and Budget Constraints
Authors: Naman Goel, Boi Faltings
Date: 2018-10-31 00:00:00
Tasks: BIG-bench Machine Learning, Diversity, Fairness

Recent studies have shown that the labels collected from crowdworkers can be
discriminatory with respect to sensitive attributes such as gender and race.
This raises questions about the suitability of using crowdsourced data for
further use, such as for training machine learning algorithms. In this work, we
address the problem of fair and diverse data collection from a crowd under
budget constraints. We propose a novel algorithm which maximizes the expected
accuracy of the collected data, while ensuring that the errors satisfy desired
notions of fairness. We provide guarantees on the performance of our algorithm
and show that the algorithm performs well in practice through experiments on a
real dataset.
Automated Machine Learning: From Principles to Practices
Authors: Zhenqian Shen, Yongqi Zhang, Lanning Wei, Huan Zhao, Quanming Yao
Date: 2018-10-31 00:00:00
Tasks: AutoML, BIG-bench Machine Learning, Neural Architecture Search, Survey

Machine learning (ML) methods have been developing rapidly, but configuring and selecting proper methods to achieve a desired performance is increasingly difficult and tedious. To address this challenge, automated machine learning (AutoML) has emerged, which aims to generate satisfactory ML configurations for given tasks in a data-driven way. In this paper, we provide a comprehensive survey on this topic. We begin with the formal definition of AutoML and then introduce its principles, including the bi-level learning objective, the learning strategy, and the theoretical interpretation. Then, we summarize the AutoML practices by setting up the taxonomy of existing works based on three main factors: the search space, the search algorithm, and the evaluation strategy. Each category is also explained with the representative methods. Then, we illustrate the principles and practices with exemplary applications from configuring ML pipeline, one-shot neural architecture search, and integration with foundation models. Finally, we highlight the emerging directions of AutoML and conclude the survey.
Acute and sub-acute stroke lesion segmentation from multimodal MRI
Authors: Albert Clèrigues, Sergi Valverde, Jose Bernal, Jordi Freixenet, Arnau Oliver, Xavier Lladó
Date: 2018-10-31 00:00:00
Tasks: Acute Stroke Lesion Segmentation, Anatomy, Ischemic Stroke Lesion Segmentation, Lesion Segmentation, MRI segmentation, Outcome Prediction In Multimodal Mri, Segmentation

Acute stroke lesion segmentation tasks are of great clinical interest as they
can help doctors make better informed treatment decisions. Magnetic resonance
imaging (MRI) is time demanding but can provide images that are considered gold
standard for diagnosis. Automated stroke lesion segmentation can provide with
an estimate of the location and volume of the lesioned tissue, which can help
in the clinical practice to better assess and evaluate the risks of each
treatment. We propose a deep learning methodology for acute and sub-acute
stroke lesion segmentation using multimodal MR imaging. The proposed method is
evaluated using two public datasets from the 2015 Ischemic Stroke Lesion
Segmentation challenge (ISLES 2015). These involve the tasks of sub-acute
stroke lesion segmentation (SISS) and acute stroke penumbra estimation (SPES)
from diffusion, perfusion and anatomical MRI modalities. The performance is
compared against state-of-the-art methods with a blind online testing set
evaluation on each of the challenges. At the time of submitting this
manuscript, our approach is the first method in the online rankings for the
SISS (DSC=0.59$\pm$0.31) and SPES sub-tasks (DSC=0.84$\pm$0.10). When compared
with the rest of submitted strategies, we achieve top rank performance with a
lower Hausdorff distance. Better segmentation results are obtained by
leveraging the anatomy and pathophysiology of acute stroke lesions and using a
combined approach to minimize the effects of class imbalance. The same training
procedure is used for both tasks, showing the proposed methodology can
generalize well enough to deal with different unrelated tasks and imaging
modalities without training hyper-parameter tuning. A public version of the
proposed method has been released to the scientific community at
https://github.com/NIC-VICOROB/stroke-mri-segmentation.
On Exploration, Exploitation and Learning in Adaptive Importance Sampling
Authors: Xiaoyu Lu, Tom Rainforth, Yuan Zhou, Jan-Willem van de Meent, Yee Whye Teh
Date: 2018-10-31 00:00:00

We study adaptive importance sampling (AIS) as an online learning problem and
argue for the importance of the trade-off between exploration and exploitation
in this adaptation. Borrowing ideas from the bandits literature, we propose
Daisee, a partition-based AIS algorithm. We further introduce a notion of
regret for AIS and show that Daisee has $\mathcal{O}(\sqrt{T}(\log
T)^{\frac{3}{4}})$ cumulative pseudo-regret, where $T$ is the number of
iterations. We then extend Daisee to adaptively learn a hierarchical
partitioning of the sample space for more efficient sampling and confirm the
performance of both algorithms empirically.
Anomaly Detection With Multiple-Hypotheses Predictions
Authors: Duc Tam Nguyen, Zhongyu Lou, Michael Klar, Thomas Brox
Date: 2018-10-31 00:00:00
Proceeding: ICLR 2019
Tasks: Anomaly Detection, Diversity

In one-class-learning tasks, only the normal case (foreground) can be modeled with data, whereas the variation of all possible anomalies is too erratic to be described by samples. Thus, due to the lack of representative data, the wide-spread discriminative approaches cannot cover such learning tasks, and rather generative models, which attempt to learn the input density of the foreground, are used. However, generative models suffer from a large input dimensionality (as in images) and are typically inefficient learners. We propose to learn the data distribution of the foreground more efficiently with a multi-hypotheses autoencoder. Moreover, the model is criticized by a discriminator, which prevents artificial data modes not supported by data, and enforces diversity across hypotheses. Our multiple-hypothesesbased anomaly detection framework allows the reliable identification of out-of-distribution samples. For anomaly detection on CIFAR-10, it yields up to 3.9% points improvement over previously reported results. On a real anomaly detection task, the approach reduces the error of the baseline models from 6.8% to 1.5%.
The Medico-Task 2018: Disease Detection in the Gastrointestinal Tract using Global Features and Deep Learning
Authors: Vajira Thambawita, Debesh Jha, Michael Riegler, Pål Halvorsen, Hugo Lewi Hammer, Håvard D. Johansen, Dag Johansen
Date: 2018-10-31 00:00:00

In this paper, we present our approach for the 2018 Medico Task classifying
diseases in the gastrointestinal tract. We have proposed a system based on
global features and deep neural networks. The best approach combines two neural
networks, and the reproducible experimental results signify the efficiency of
the proposed model with an accuracy rate of 95.80%, a precision of 95.87%, and
an F1-score of 95.80%.
Ionospheric activity prediction using convolutional recurrent neural networks
Authors: Alexandre Boulch, Noëlie Cherrier, Thibaut Castaings
Date: 2018-10-31 00:00:00
Tasks: Activity Prediction, Prediction

The ionosphere electromagnetic activity is a major factor of the quality of
satellite telecommunications, Global Navigation Satellite Systems (GNSS) and
other vital space applications. Being able to forecast globally the Total
Electron Content (TEC) would enable a better anticipation of potential
performance degradations. A few studies have proposed models able to predict
the TEC locally, but not worldwide for most of them. Thanks to a large record
of past TEC maps publicly available, we propose a method based on Deep Neural
Networks (DNN) to forecast a sequence of global TEC maps consecutive to an
input sequence of TEC maps, without introducing any prior knowledge other than
Earth rotation periodicity. By combining several state-of-the-art
architectures, the proposed approach is competitive with previous works on TEC
forecasting while predicting the TEC globally.
Non-linear Canonical Correlation Analysis: A Compressed Representation Approach
Authors: Amichai Painsky, Meir Feder, Naftali Tishby
Date: 2018-10-31 00:00:00
Tasks: Dimensionality Reduction, Quantization, Representation Learning

Canonical Correlation Analysis (CCA) is a linear representation learning method that seeks maximally correlated variables in multi-view data. Non-linear CCA extends this notion to a broader family of transformations, which are more powerful in many real-world applications. Given the joint probability, the Alternating Conditional Expectation (ACE) algorithm provides an optimal solution to the non-linear CCA problem. However, it suffers from limited performance and an increasing computational burden when only a finite number of samples is available. In this work we introduce an information-theoretic compressed representation framework for the non-linear CCA problem (CRCCA), which extends the classical ACE approach. Our suggested framework seeks compact representations of the data that allow a maximal level of correlation. This way we control the trade-off between the flexibility and the complexity of the model. CRCCA provides theoretical bounds and optimality conditions, as we establish fundamental connections to rate-distortion theory, the information bottleneck and remote source coding. In addition, it allows a soft dimensionality reduction, as the compression level is determined by the mutual information between the original noisy data and the extracted signals. Finally, we introduce a simple implementation of the CRCCA framework, based on lattice quantization.
On Fast Leverage Score Sampling and Optimal Learning
Authors: Alessandro Rudi, Daniele Calandriello, Luigi Carratino, Lorenzo Rosasco
Date: 2018-10-31 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: regression

Leverage score sampling provides an appealing way to perform approximate
computations for large matrices. Indeed, it allows to derive faithful
approximations with a complexity adapted to the problem at hand. Yet,
performing leverage scores sampling is a challenge in its own right requiring
further approximations. In this paper, we study the problem of leverage score
sampling for positive definite matrices defined by a kernel. Our contribution
is twofold. First we provide a novel algorithm for leverage score sampling and
second, we exploit the proposed method in statistical learning by deriving a
novel solver for kernel ridge regression. Our main technical contribution is
showing that the proposed algorithms are currently the most efficient and
accurate for these problems.
Methods for Segmentation and Classification of Digital Microscopy Tissue Images
Authors: Quoc Dang Vu, Simon Graham, Minh Nguyen Nhat To, Muhammad Shaban, Talha Qaiser, Navid Alemi Koohbanani, Syed Ali Khurram, Tahsin Kurc, Keyvan Farahani, Tianhao Zhao, Rajarsi Gupta, Jin Tae Kwak, Nasir Rajpoot, Joel Saltz
Date: 2018-10-31 00:00:00
Tasks: Classification, General Classification, image-classification, Image Classification, Segmentation

High-resolution microscopy images of tissue specimens provide detailed
information about the morphology of normal and diseased tissue. Image analysis
of tissue morphology can help cancer researchers develop a better understanding
of cancer biology. Segmentation of nuclei and classification of tissue images
are two common tasks in tissue image analysis. Development of accurate and
efficient algorithms for these tasks is a challenging problem because of the
complexity of tissue morphology and tumor heterogeneity. In this paper we
present two computer algorithms; one designed for segmentation of nuclei and
the other for classification of whole slide tissue images. The segmentation
algorithm implements a multiscale deep residual aggregation network to
accurately segment nuclear material and then separate clumped nuclei into
individual nuclei. The classification algorithm initially carries out
patch-level classification via a deep learning method, then patch-level
statistical and morphological features are used as input to a random forest
regression model for whole slide image classification. The segmentation and
classification algorithms were evaluated in the MICCAI 2017 Digital Pathology
challenge. The segmentation algorithm achieved an accuracy score of 0.78. The
classification algorithm achieved an accuracy score of 0.81.
SURFACE: Semantically Rich Fact Validation with Explanations
Authors: Ankur Padia, Francis Ferraro, Tim Finin
Date: 2018-10-31 00:00:00
Tasks: General Classification, Sentence

Judging the veracity of a sentence making one or more claims is an important
and challenging problem with many dimensions. The recent FEVER task asked
participants to classify input sentences as either SUPPORTED, REFUTED or
NotEnoughInfo using Wikipedia as a source of true facts. SURFACE does this task
and explains its decision through a selection of sentences from the trusted
source. Our multi-task neural approach uses semantic lexical frames from
FrameNet to jointly (i) find relevant evidential sentences in the trusted
source and (ii) use them to classify the input sentence's veracity. An
evaluation of our efficient three-parameter model on the FEVER dataset showed
an improvement of 90% over the state-of-the-art baseline on retrieving relevant
sentences and a 70% relative improvement in classification.
Multi-Task Learning for Left Atrial Segmentation on GE-MRI
Authors: Chen Chen, Wenjia Bai, Daniel Rueckert
Date: 2018-10-31 00:00:00
Tasks: Anatomy, General Classification, Multi-Task Learning, Segmentation

Segmentation of the left atrium (LA) is crucial for assessing its anatomy in
both pre-operative atrial fibrillation (AF) ablation planning and
post-operative follow-up studies. In this paper, we present a fully automated
framework for left atrial segmentation in gadolinium-enhanced magnetic
resonance images (GE-MRI) based on deep learning. We propose a fully
convolutional neural network and explore the benefits of multi-task learning
for performing both atrial segmentation and pre/post ablation classification.
Our results show that, by sharing features between related tasks, the network
can gain additional anatomical information and achieve more accurate atrial
segmentation, leading to a mean Dice score of 0.901 on a test set of 20 3D MRI
images. Code of our proposed algorithm is available at
https://github.com/cherise215/atria_segmentation_2018/.
Compressive Single-pixel Fourier Transform Imaging using Structured Illumination
Authors: Amirafshar Moshtaghpour, José M. Bioucas-Dias, Laurent Jacques
Date: 2018-10-31 00:00:00
Tasks: Compressive Sensing

Single Pixel (SP) imaging is now a reality in many applications, e.g.,
biomedical ultrathin endoscope and fluorescent spectroscopy. In this context,
many schemes exist to improve the light throughput of these device, e.g., using
structured illumination driven by compressive sensing theory. In this work, we
consider the combination of SP imaging with Fourier Transform Interferometry
(SP-FTI) to reach high-resolution HyperSpectral (HS) imaging, as desirable,
e.g., in fluorescent spectroscopy. While this association is not new, we here
focus on optimizing the spatial illumination, structured as Hadamard patterns,
during the optical path progression. We follow a variable density sampling
strategy for space-time coding of the light illumination, and show
theoretically and numerically that this scheme allows us to reduce the number
of measurements and light-exposure of the observed object compared to
conventional compressive SP-FTI.
The Many Moods of Emotion
Authors: Valentin Vielzeuf, Corentin Kervadec, Stéphane Pateux, Frédéric Jurie
Date: 2018-10-31 00:00:00
Tasks: Emotion Classification, Facial expression generation, Generative Adversarial Network

This paper presents a novel approach to the facial expression generation
problem. Building upon the assumption of the psychological community that
emotion is intrinsically continuous, we first design our own continuous emotion
representation with a 3-dimensional latent space issued from a neural network
trained on discrete emotion classification. The so-obtained representation can
be used to annotate large in the wild datasets and later used to trained a
Generative Adversarial Network. We first show that our model is able to map
back to discrete emotion classes with a objectively and subjectively better
quality of the images than usual discrete approaches. But also that we are able
to pave the larger space of possible facial expressions, generating the many
moods of emotion. Moreover, two axis in this space may be found to generate
similar expression changes as in traditional continuous representations such as
arousal-valence. Finally we show from visual interpretation, that the third
remaining dimension is highly related to the well-known dominance dimension
from psychology.
Nearly-tight bounds on linear regions of piecewise linear neural networks
Authors: Qiang Hu, Hao Zhang
Date: 2018-10-31 00:00:00

The developments of deep neural networks (DNN) in recent years have ushered a
brand new era of artificial intelligence. DNNs are proved to be excellent in
solving very complex problems, e.g., visual recognition and text understanding,
to the extent of competing with or even surpassing people. Despite inspiring
and encouraging success of DNNs, thorough theoretical analyses still lack to
unravel the mystery of their magics. The design of DNN structure is dominated
by empirical results in terms of network depth, number of neurons and
activations. A few of remarkable works published recently in an attempt to
interpret DNNs have established the first glimpses of their internal
mechanisms. Nevertheless, research on exploring how DNNs operate is still at
the initial stage with plenty of room for refinement. In this paper, we extend
precedent research on neural networks with piecewise linear activations (PLNN)
concerning linear regions bounds. We present (i) the exact maximal number of
linear regions for single layer PLNNs; (ii) a upper bound for multi-layer
PLNNs; and (iii) a tighter upper bound for the maximal number of liner regions
on rectifier networks. The derived bounds also indirectly explain why deep
models are more powerful than shallow counterparts, and how non-linearity of
activation functions impacts on expressiveness of networks.