pwc-archive/papers-with-abstracts huggingface dataset:
FALKON: An Optimal Large Scale Kernel Method
Authors: Alessandro Rudi, Luigi Carratino, Lorenzo Rosasco
Date: 2017-05-31 00:00:00
Proceeding: NeurIPS 2017 12

Kernel methods provide a principled way to perform non linear, nonparametric
learning. They rely on solid functional analytic foundations and enjoy optimal
statistical properties. However, at least in their basic form, they have
limited applicability in large scale scenarios because of stringent
computational requirements in terms of time and especially memory. In this
paper, we take a substantial step in scaling up kernel methods, proposing
FALKON, a novel algorithm that allows to efficiently process millions of
points. FALKON is derived combining several algorithmic principles, namely
stochastic subsampling, iterative solvers and preconditioning. Our theoretical
analysis shows that optimal statistical accuracy is achieved requiring
essentially $O(n)$ memory and $O(n\sqrt{n})$ time. An extensive experimental
analysis on large scale datasets shows that, even with a single machine, FALKON
outperforms previous state of the art solutions, which exploit
parallel/distributed architectures.
Distinguishing Cause from Effect Using Quantiles: Bivariate Quantile Causal Discovery
Authors: Natasa Tagasovska, Valérie Chavez-Demoulin, Thibault Vatter
Date: 2018-01-31 00:00:00
Tasks: Causal Discovery, Causal Inference, quantile regression, regression, Selection bias

Causal inference using observational data is challenging, especially in the bivariate case. Through the minimum description length principle, we link the postulate of independence between the generating mechanisms of the cause and of the effect given the cause to quantile regression. Based on this theory, we develop Bivariate Quantile Causal Discovery (bQCD), a new method to distinguish cause from effect assuming no confounding, selection bias or feedback. Because it uses multiple quantile levels instead of the conditional mean only, bQCD is adaptive not only to additive, but also to multiplicative or even location-scale generating mechanisms. To illustrate the effectiveness of our approach, we perform an extensive empirical comparison on both synthetic and real datasets. This study shows that bQCD is robust across different implementations of the method (i.e., the quantile regression), computationally efficient, and compares favorably to state-of-the-art methods.
Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach
Authors: Tsui-Wei Weng, huan zhang, Pin-Yu Chen, Jin-Feng Yi, Dong Su, Yupeng Gao, Cho-Jui Hsieh, Luca Daniel
Date: 2018-01-31 00:00:00
Proceeding: ICLR 2018 1

The robustness of neural networks to adversarial examples has received great
attention due to security implications. Despite various attack approaches to
crafting visually imperceptible adversarial examples, little has been developed
towards a comprehensive measure of robustness. In this paper, we provide a
theoretical justification for converting robustness analysis into a local
Lipschitz constant estimation problem, and propose to use the Extreme Value
Theory for efficient evaluation. Our analysis yields a novel robustness metric
called CLEVER, which is short for Cross Lipschitz Extreme Value for nEtwork
Robustness. The proposed CLEVER score is attack-agnostic and computationally
feasible for large neural networks. Experimental results on various networks,
including ResNet, Inception-v3 and MobileNet, show that (i) CLEVER is aligned
with the robustness indication measured by the $\ell_2$ and $\ell_\infty$ norms
of adversarial examples from powerful attacks, and (ii) defended networks using
defensive distillation or bounded ReLU indeed achieve better CLEVER scores. To
the best of our knowledge, CLEVER is the first attack-independent robustness
metric that can be applied to any neural network classifier.
Generalization Properties of Learning with Random Features
Authors: Alessandro Rudi, Lorenzo Rosasco
Date: 2016-02-14 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: regression

We study the generalization properties of ridge regression with random features in the statistical learning framework. We show for the first time that $O(1/\sqrt{n})$ learning bounds can be achieved with only $O(\sqrt{n}\log n)$ random features rather than $O({n})$ as suggested by previous results. Further, we prove faster learning rates and show that they might require more random features, unless they are sampled according to a possibly problem dependent distribution. Our results shed light on the statistical computational trade-offs in large scale kernelized learning, showing the potential effectiveness of random features in reducing the computational complexity while keeping optimal generalization properties.
Naive Bayes Entrapment Detection for Planetary Rovers
Authors: Dicong Qiu
Date: 2018-01-31 00:00:00

Entrapment detection is a prerequisite for planetary rovers to perform
autonomous rescue procedure. In this study, rover entrapment and approximated
entrapment criteria are formally defined. Entrapment detection using Naive
Bayes classifiers is proposed and discussed along with results from experiments
where the Naive Bayes entrapment detector is applied to AutoKralwer rovers. And
final conclusions and further discussions are presented in the final section.
Feature Decomposition Based Saliency Detection in Electron Cryo-Tomograms
Authors: Bo Zhou, Qiang Guo, Xiangrui Zeng, Min Xu
Date: 2018-01-31 00:00:00
Tasks: Saliency Detection, Segmentation

Electron Cryo-Tomography (ECT) allows 3D visualization of subcellular
structures at the submolecular resolution in close to the native state.
However, due to the high degree of structural complexity and imaging limits,
the automatic segmentation of cellular components from ECT images is very
difficult. To complement and speed up existing segmentation methods, it is
desirable to develop a generic cell component segmentation method that is 1)
not specific to particular types of cellular components, 2) able to segment
unknown cellular components, 3) fully unsupervised and does not rely on the
availability of training data. As an important step towards this goal, in this
paper, we propose a saliency detection method that computes the likelihood that
a subregion in a tomogram stands out from the background. Our method consists
of four steps: supervoxel over-segmentation, feature extraction, feature matrix
decomposition, and computation of saliency. The method produces a distribution
map that represents the regions' saliency in tomograms. Our experiments show
that our method can successfully label most salient regions detected by a human
observer, and able to filter out regions not containing cellular components.
Therefore, our method can remove the majority of the background region, and
significantly speed up the subsequent processing of segmentation and
recognition of cellular components captured by ECT.
Speaker Diarization with LSTM
Authors: Quan Wang, Carlton Downey, Li Wan, Philip Andrew Mansfield, Ignacio Lopez Moreno
Date: 2017-10-28 00:00:00
Tasks: Clustering, speaker-diarization, Speaker Diarization, Speaker Verification

For many years, i-vector based audio embedding techniques were the dominant approach for speaker verification and speaker diarization applications. However, mirroring the rise of deep learning in various domains, neural network based audio embeddings, also known as d-vectors, have consistently demonstrated superior speaker verification performance. In this paper, we build on the success of d-vector based speaker verification systems to develop a new d-vector based approach to speaker diarization. Specifically, we combine LSTM-based d-vector audio embeddings with recent work in non-parametric clustering to obtain a state-of-the-art speaker diarization system. Our system is evaluated on three standard public datasets, suggesting that d-vector based diarization systems offer significant advantages over traditional i-vector based systems. We achieved a 12.0% diarization error rate on NIST SRE 2000 CALLHOME, while our model is trained with out-of-domain data from voice search logs.
Attacking Binarized Neural Networks
Authors: Angus Galloway, Graham W. Taylor, Medhat Moussa
Date: 2017-11-01 00:00:00
Proceeding: ICLR 2018 1
Tasks: Quantization

Neural networks with low-precision weights and activations offer compelling
efficiency advantages over their full-precision equivalents. The two most
frequently discussed benefits of quantization are reduced memory consumption,
and a faster forward pass when implemented with efficient bitwise operations.
We propose a third benefit of very low-precision neural networks: improved
robustness against some adversarial attacks, and in the worst case, performance
that is on par with full-precision models. We focus on the very low-precision
case where weights and activations are both quantized to $\pm$1, and note that
stochastically quantizing weights in just one layer can sharply reduce the
impact of iterative attacks. We observe that non-scaled binary neural networks
exhibit a similar effect to the original defensive distillation procedure that
led to gradient masking, and a false notion of security. We address this by
conducting both black-box and white-box experiments with binary models that do
not artificially mask gradients.
A family of OWA operators based on Faulhaber's formulas
Authors: Oscar Duarte, Sandra Téllez
Date: 2018-01-31 00:00:00

In this paper we develop a new family of Ordered Weighted Averaging (OWA)
operators. Weight vector is obtained from a desired orness of the operator.
Using Faulhaber's formulas we obtain direct and simple expressions for the
weight vector without any iteration loop. With the exception of one weight, the
remaining follow a straight line relation. As a result, a fast and robust
algorithm is developed. The resulting weight vector is suboptimal according
with the Maximum Entropy criterion, but it is very close to the optimal.
Comparisons are done with other procedures.
An Analysis of Human-centered Geolocation
Authors: Kaili Wang, Yu-Hui Huang, Jose Oramas, Luc van Gool, Tinne Tuytelaars
Date: 2017-07-10 00:00:00

Online social networks contain a constantly increasing amount of images -
most of them focusing on people. Due to cultural and climate factors, fashion
trends and physical appearance of individuals differ from city to city. In this
paper we investigate to what extent such cues can be exploited in order to
infer the geographic location, i.e. the city, where a picture was taken. We
conduct a user study, as well as an evaluation of automatic methods based on
convolutional neural networks. Experiments on the Fashion 144k and a
Pinterest-based dataset show that the automatic methods succeed at this task to
a reasonable extent. As a matter of fact, our empirical results suggest that
automatic methods can surpass human performance by a large margin. Further
inspection of the trained models shows that human-centered characteristics,
like clothing style, physical features, and accessories, are informative for
the task at hand. Moreover, it reveals that also contextual features, e.g. wall
type, natural environment, etc., are taken into account by the automatic
methods.
Latent Space Oddity: on the Curvature of Deep Generative Models
Authors: Georgios Arvanitidis, Lars Kai Hansen, Søren Hauberg
Date: 2017-10-31 00:00:00
Proceeding: ICLR 2018 1
Tasks: Clustering

Deep generative models provide a systematic way to learn nonlinear data distributions, through a set of latent variables and a nonlinear "generator" function that maps latent points into the input space. The nonlinearity of the generator imply that the latent space gives a distorted view of the input space. Under mild conditions, we show that this distortion can be characterized by a stochastic Riemannian metric, and demonstrate that distances and interpolants are significantly improved under this metric. This in turn improves probability distributions, sampling algorithms and clustering in the latent space. Our geometric analysis further reveals that current generators provide poor variance estimates and we propose a new generator architecture with vastly improved variance estimates. Results are demonstrated on convolutional and fully connected variational autoencoders, but the formalism easily generalize to other deep generative models.
Deep Reinforcement Learning for Programming Language Correction
Authors: Rahul Gupta, Aditya Kanade, Shirish Shevade
Date: 2018-01-31 00:00:00
Tasks: Deep Reinforcement Learning, Machine Translation, Program Repair, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Translation

Novice programmers often struggle with the formal syntax of programming
languages. To assist them, we design a novel programming language correction
framework amenable to reinforcement learning. The framework allows an agent to
mimic human actions for text navigation and editing. We demonstrate that the
agent can be trained through self-exploration directly from the raw input, that
is, program text itself, without any knowledge of the formal syntax of the
programming language. We leverage expert demonstrations for one tenth of the
training data to accelerate training. The proposed technique is evaluated on
6975 erroneous C programs with typographic errors, written by students during
an introductory programming course. Our technique fixes 14% more programs and
29% more compiler error messages relative to those fixed by a state-of-the-art
tool, DeepFix, which uses a fully supervised neural machine translation
approach.
Preparation of Improved Turkish DataSet for Sentiment Analysis in Social Media
Authors: Semiha Makinist, Ibrahim Riza Hallac, Betul Ay Karakus, Galip Aydin
Date: 2018-01-30 00:00:00
Tasks: Sentiment Analysis, Spelling Correction

A public dataset, with a variety of properties suitable for sentiment
analysis [1], event prediction, trend detection and other text mining
applications, is needed in order to be able to successfully perform analysis
studies. The vast majority of data on social media is text-based and it is not
possible to directly apply machine learning processes into these raw data,
since several different processes are required to prepare the data before the
implementation of the algorithms. For example, different misspellings of same
word enlarge the word vector space unnecessarily, thereby it leads to reduce
the success of the algorithm and increase the computational power requirement.
This paper presents an improved Turkish dataset with an effective spelling
correction algorithm based on Hadoop [2]. The collected data is recorded on the
Hadoop Distributed File System and the text based data is processed by
MapReduce programming model. This method is suitable for the storage and
processing of large sized text based social media data. In this study, movie
reviews have been automatically recorded with Apache ManifoldCF (MCF) [3] and
data clusters have been created. Various methods compared such as Levenshtein
and Fuzzy String Matching have been proposed to create a public dataset from
collected data. Experimental results show that the proposed algorithm, which
can be used as an open source dataset in sentiment analysis studies, have been
performed successfully to the detection and correction of spelling errors.
Denoising of 3D magnetic resonance images with multi-channel residual learning of convolutional neural network
Authors: Dongsheng Jiang, Weiqiang Dou, Luc Vosters, Xiayu Xu, Yue Sun, Tao Tan
Date: 2017-12-23 00:00:00
Tasks: Denoising

The denoising of magnetic resonance (MR) images is a task of great importance
for improving the acquired image quality. Many methods have been proposed in
the literature to retrieve noise free images with good performances. Howerever,
the state-of-the-art denoising methods, all needs a time-consuming optimization
processes and their performance strongly depend on the estimated noise level
parameter. Within this manuscript we propose the idea of denoising MRI Rician
noise using a convolutional neural network. The advantage of the proposed
methodology is that the learning based model can be directly used in the
denosing process without optimization and even without the noise level
parameter. Specifically, a ten convolutional layers neural network combined
with residual learning and multi-channel strategy was proposed. Two training
ways: training on a specific noise level and training on a general level were
conducted to demonstrate the capability of our methods. Experimental results
over synthetic and real 3D MR data demonstrate our proposed network can achieve
superior performance compared with other methods in term of both of the peak
signal to noise ratio and the global of structure similarity index. Without
noise level parameter, our general noise-applicable model is also better than
the other compared methods in two datasets. Furthermore, our training model
shows good general applicability.
Exploiting the potential of unlabeled endoscopic video data with self-supervised learning
Authors: Tobias Ross, David Zimmerer, Anant Vemuri, Fabian Isensee, Manuel Wiesenfarth, Sebastian Bodenstedt, Fabian Both, Philip Kessler, Martin Wagner, Beat Müller, Hannes Kenngott, Stefanie Speidel, Annette Kopp-Schneider, Klaus Maier-Hein, Lena Maier-Hein
Date: 2017-11-27 00:00:00
Tasks: Colorization, Generative Adversarial Network, Instance Segmentation, Self-Supervised Learning, Semantic Segmentation

Surgical data science is a new research field that aims to observe all
aspects of the patient treatment process in order to provide the right
assistance at the right time. Due to the breakthrough successes of deep
learning-based solutions for automatic image annotation, the availability of
reference annotations for algorithm training is becoming a major bottleneck in
the field. The purpose of this paper was to investigate the concept of
self-supervised learning to address this issue.
  Our approach is guided by the hypothesis that unlabeled video data can be
used to learn a representation of the target domain that boosts the performance
of state-of-the-art machine learning algorithms when used for pre-training.
Core of the method is an auxiliary task based on raw endoscopic video data of
the target domain that is used to initialize the convolutional neural network
(CNN) for the target task. In this paper, we propose the re-colorization of
medical images with a generative adversarial network (GAN)-based architecture
as auxiliary task. A variant of the method involves a second pre-training step
based on labeled data for the target task from a related domain. We validate
both variants using medical instrument segmentation as target task.
  The proposed approach can be used to radically reduce the manual annotation
effort involved in training CNNs. Compared to the baseline approach of
generating annotated data from scratch, our method decreases exploratively the
number of labeled images by up to 75% without sacrificing performance. Our
method also outperforms alternative methods for CNN pre-training, such as
pre-training on publicly available non-medical or medical data using the target
task (in this instance: segmentation).
  As it makes efficient use of available (non-)public and (un-)labeled data,
the approach has the potential to become a valuable tool for CNN
(pre-)training.
Recovering from Random Pruning: On the Plasticity of Deep Convolutional Neural Networks
Authors: Deepak Mittal, Shweta Bhardwaj, Mitesh M. Khapra, Balaraman Ravindran
Date: 2018-01-31 00:00:00
Tasks: image-classification, Image Classification, object-detection, Object Detection

Recently there has been a lot of work on pruning filters from deep
convolutional neural networks (CNNs) with the intention of reducing
computations. The key idea is to rank the filters based on a certain criterion
(say, $l_1$-norm, average percentage of zeros, etc) and retain only the top
ranked filters. Once the low scoring filters are pruned away the remainder of
the network is fine tuned and is shown to give performance comparable to the
original unpruned network. In this work, we report experiments which suggest
that the comparable performance of the pruned network is not due to the
specific criterion chosen but due to the inherent plasticity of deep neural
networks which allows them to recover from the loss of pruned filters once the
rest of the filters are fine-tuned. Specifically, we show counter-intuitive
results wherein by randomly pruning 25-50\% filters from deep CNNs we are able
to obtain the same performance as obtained by using state of the art pruning
methods. We empirically validate our claims by doing an exhaustive evaluation
with VGG-16 and ResNet-50. Further, we also evaluate a real world scenario
where a CNN trained on all 1000 ImageNet classes needs to be tested on only a
small set of classes at test time (say, only animals). We create a new
benchmark dataset from ImageNet to evaluate such class specific pruning and
show that even here a random pruning strategy gives close to state of the art
performance. Lastly, unlike existing approaches which mainly focus on the task
of image classification, in this work we also report results on object
detection. We show that using a simple random pruning strategy we can achieve
significant speed up in object detection (74$\%$ improvement in fps) while
retaining the same accuracy as that of the original Faster RCNN model.
Counting Cells in Time-Lapse Microscopy using Deep Neural Networks
Authors: Alexander Gomez Villa, Augusto Salazar, Igor Stefanini
Date: 2018-01-31 00:00:00

An automatic approach to counting any kind of cells could alleviate work of
the experts and boost the research in fields such as regenerative medicine. In
this paper, a method for microscopy cell counting using multiple frames (hence
temporal information) is proposed. Unlike previous approaches where the cell
counting is done independently in each frame (static cell counting), in this
work the cell counting prediction is done using multiple frames (dynamic cell
counting). A spatiotemporal model using ConvNets and long short term memory
(LSTM) recurrent neural networks is proposed to overcome temporal variations.
The model outperforms static cell counting in a publicly available dataset of
stem cells. The advantages, working conditions and limitations of the
ConvNet-LSTM method are discussed. Although our method is tested in cell
counting, it can be extrapolated to quantify in video (or correlated image
series) any kind of objects or volumes.
From Benedict Cumberbatch to Sherlock Holmes: Character Identification in TV series without a Script
Authors: Arsha Nagrani, Andrew Zisserman
Date: 2018-01-31 00:00:00
Tasks: Speaker Identification

The goal of this paper is the automatic identification of characters in TV
and feature film material. In contrast to standard approaches to this task,
which rely on the weak supervision afforded by transcripts and subtitles, we
propose a new method requiring only a cast list. This list is used to obtain
images of actors from freely available sources on the web, providing a form of
partial supervision for this task. In using images of actors to recognize
characters, we make the following three contributions: (i) We demonstrate that
an automated semi-supervised learning approach is able to adapt from the
actor's face to the character's face, including the face context of the hair;
(ii) By building voice models for every character, we provide a bridge between
frontal faces (for which there is plenty of actor-level supervision) and
profile (for which there is very little or none); and (iii) by combining face
context and speaker identification, we are able to identify characters with
partially occluded faces and extreme facial poses. Results are presented on the
TV series 'Sherlock' and the feature film 'Casablanca'. We achieve the
state-of-the-art on the Casablanca benchmark, surpassing previous methods that
have used the stronger supervision available from transcripts.