pwc-archive/papers-with-abstracts huggingface dataset:
Skin Lesion Classification Using Hybrid Deep Neural Networks
Authors: Amirreza Mahbod, Gerald Schaefer, Chunliang Wang, Rupert Ecker, Isabella Ellinger
Date: 2017-02-27 00:00:00
Tasks: Classification, General Classification, Lesion Classification, Object Detection, Skin Lesion Classification

Skin cancer is one of the major types of cancers with an increasing incidence
over the past decades. Accurately diagnosing skin lesions to discriminate
between benign and malignant skin lesions is crucial to ensure appropriate
patient treatment. While there are many computerised methods for skin lesion
classification, convolutional neural networks (CNNs) have been shown to be
superior over classical methods. In this work, we propose a fully automatic
computerised method for skin lesion classification which employs optimised deep
features from a number of well-established CNNs and from different abstraction
levels. We use three pre-trained deep models, namely AlexNet, VGG16 and
ResNet-18, as deep feature generators. The extracted features then are used to
train support vector machine classifiers. In the final stage, the classifier
outputs are fused to obtain a classification. Evaluated on the 150 validation
images from the ISIC 2017 classification challenge, the proposed method is
shown to achieve very good classification performance, yielding an area under
receiver operating characteristic curve of 83.83% for melanoma classification
and of 97.55% for seborrheic keratosis classification.
Improving Reliability of Word Similarity Evaluation by Redesigning Annotation Task and Performance Measure
Authors: Oded Avraham, Yoav Goldberg
Date: 2016-11-11 00:00:00
Proceeding: WS 2016 8
Tasks: Word Similarity

We suggest a new method for creating and using gold-standard datasets for
word similarity evaluation. Our goal is to improve the reliability of the
evaluation, and we do this by redesigning the annotation task to achieve higher
inter-rater agreement, and by defining a performance measure which takes the
reliability of each annotation decision in the dataset into account.
SAGA and Restricted Strong Convexity
Authors: Chao Qu, Yan Li, Huan Xu
Date: 2017-02-19 00:00:00
Tasks: regression

SAGA is a fast incremental gradient method on the finite sum problem and its
effectiveness has been tested on a vast of applications. In this paper, we
analyze SAGA on a class of non-strongly convex and non-convex statistical
problem such as Lasso, group Lasso, Logistic regression with $\ell_1$
regularization, linear regression with SCAD regularization and Correct Lasso.
We prove that SAGA enjoys the linear convergence rate up to the statistical
estimation accuracy, under the assumption of restricted strong convexity (RSC).
It significantly extends the applicability of SAGA in convex and non-convex
optimization.
An SDP-Based Algorithm for Linear-Sized Spectral Sparsification
Authors: Yin Tat Lee, He Sun
Date: 2017-02-27 00:00:00

For any undirected and weighted graph $G=(V,E,w)$ with $n$ vertices and $m$
edges, we call a sparse subgraph $H$ of $G$, with proper reweighting of the
edges, a $(1+\varepsilon)$-spectral sparsifier if \[
(1-\varepsilon)x^{\intercal}L_Gx\leq x^{\intercal} L_{H} x\leq (1+\varepsilon)
x^{\intercal} L_Gx \] holds for any $x\in\mathbb{R}^n$, where $L_G$ and $L_{H}$
are the respective Laplacian matrices of $G$ and $H$. Noticing that $\Omega(m)$
time is needed for any algorithm to construct a spectral sparsifier and a
spectral sparsifier of $G$ requires $\Omega(n)$ edges, a natural question is to
investigate, for any constant $\varepsilon$, if a $(1+\varepsilon)$-spectral
sparsifier of $G$ with $O(n)$ edges can be constructed in $\tilde{O}(m)$ time,
where the $\tilde{O}$ notation suppresses polylogarithmic factors. All previous
constructions on spectral sparsification require either super-linear number of
edges or $m^{1+\Omega(1)}$ time.
  In this work we answer this question affirmatively by presenting an algorithm
that, for any undirected graph $G$ and $\varepsilon>0$, outputs a
$(1+\varepsilon)$-spectral sparsifier of $G$ with $O(n/\varepsilon^2)$ edges in
$\tilde{O}(m/\varepsilon^{O(1)})$ time. Our algorithm is based on three novel
techniques: (1) a new potential function which is much easier to compute yet
has similar guarantees as the potential functions used in previous references;
(2) an efficient reduction from a two-sided spectral sparsifier to a one-sided
spectral sparsifier; (3) constructing a one-sided spectral sparsifier by a
semi-definite program.
Convolution Aware Initialization
Authors: Armen Aghajanyan
Date: 2017-02-21 00:00:00

Initialization of parameters in deep neural networks has been shown to have a
big impact on the performance of the networks (Mishkin & Matas, 2015). The
initialization scheme devised by He et al, allowed convolution activations to
carry a constrained mean which allowed deep networks to be trained effectively
(He et al., 2015a). Orthogonal initializations and more generally orthogonal
matrices in standard recurrent networks have been proved to eradicate the
vanishing and exploding gradient problem (Pascanu et al., 2012). Majority of
current initialization schemes do not take fully into account the intrinsic
structure of the convolution operator. Using the duality of the Fourier
transform and the convolution operator, Convolution Aware Initialization builds
orthogonal filters in the Fourier space, and using the inverse Fourier
transform represents them in the standard space. With Convolution Aware
Initialization we noticed not only higher accuracy and lower loss, but faster
convergence. We achieve new state of the art on the CIFAR10 dataset, and
achieve close to state of the art on various other tasks.
On Context-Dependent Clustering of Bandits
Authors: Claudio Gentile, Shuai Li, Purushottam Kar, Alexandros Karatzoglou, Evans Etrue, Giovanni Zappella
Date: 2016-08-06 00:00:00
Proceeding: ICML 2017 8
Tasks: Clustering

We investigate a novel cluster-of-bandit algorithm CAB for collaborative
recommendation tasks that implements the underlying feedback sharing mechanism
by estimating the neighborhood of users in a context-dependent manner. CAB
makes sharp departures from the state of the art by incorporating collaborative
effects into inference as well as learning processes in a manner that
seamlessly interleaving explore-exploit tradeoffs and collaborative steps. We
prove regret bounds under various assumptions on the data, which exhibit a
crisp dependence on the expected number of clusters over the users, a natural
measure of the statistical difficulty of the learning task. Experiments on
production and real-world datasets show that CAB offers significantly increased
prediction performance against a representative pool of state-of-the-art
methods.
Neural Map: Structured Memory for Deep Reinforcement Learning
Authors: Emilio Parisotto, Ruslan Salakhutdinov
Date: 2017-02-27 00:00:00
Proceeding: ICLR 2018 1
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

A critical component to enabling intelligent reasoning in partially
observable environments is memory. Despite this importance, Deep Reinforcement
Learning (DRL) agents have so far used relatively simple memory architectures,
with the main methods to overcome partial observability being either a temporal
convolution over the past k frames or an LSTM layer. More recent work (Oh et
al., 2016) has went beyond these architectures by using memory networks which
can allow more sophisticated addressing schemes over the past k frames. But
even these architectures are unsatisfactory due to the reason that they are
limited to only remembering information from the last k frames. In this paper,
we develop a memory system with an adaptable write operator that is customized
to the sorts of 3D environments that DRL agents typically interact with. This
architecture, called the Neural Map, uses a spatially structured 2D memory
image to learn to store arbitrary information about the environment over long
time lags. We demonstrate empirically that the Neural Map surpasses previous
DRL memories on a set of challenging 2D and 3D maze environments and show that
it is capable of generalizing to environments that were not seen during
training.
Spatio-Colour Asplünd 's Metric and Logarithmic Image Processing for Colour Images (LIPC)
Authors: Guillaume Noyel, Michel Jourlin
Date: 2016-08-31 00:00:00

Aspl\"und 's metric, which is useful for pattern matching, consists in a
double-sided probing, i.e. the over-graph and the sub-graph of a function are
probed jointly. This paper extends the Aspl\"und 's metric we previously
defined for colour and multivariate images using a marginal approach (i.e.
component by component) to the first spatio-colour Aspl\"und 's metric based on
the vectorial colour LIP model (LIPC). LIPC is a non-linear model with
operations between colour images which are consistent with the human visual
system. The defined colour metric is insensitive to lighting variations and a
variant which is robust to noise is used for colour pattern matching.
Multi-Label Segmentation via Residual-Driven Adaptive Regularization
Authors: Byung-Woo Hong, Ja-Keoung Koo, Stefano Soatto
Date: 2017-02-27 00:00:00

We present a variational multi-label segmentation algorithm based on a robust
Huber loss for both the data and the regularizer, minimized within a convex
optimization framework. We introduce a novel constraint on the common areas, to
bias the solution towards mutually exclusive regions. We also propose a
regularization scheme that is adapted to the spatial statistics of the residual
at each iteration, resulting in a varying degree of regularization being
applied as the algorithm proceeds: the effect of the regularizer is strongest
at initialization, and wanes as the solution increasingly fits the data. This
minimizes the bias induced by the regularizer at convergence. We design an
efficient convex optimization algorithm based on the alternating direction
method of multipliers using the equivalent relation between the Huber function
and the proximal operator of the one-norm. We empirically validate our proposed
algorithm on synthetic and real images and offer an information-theoretic
derivation of the cost-function that highlights the modeling choices made.
Visual Translation Embedding Network for Visual Relation Detection
Authors: Hanwang Zhang, Zawlin Kyaw, Shih-Fu Chang, Tat-Seng Chua
Date: 2017-02-27 00:00:00
Proceeding: CVPR 2017 7
Tasks: Object, object-detection, Object Detection, Relation, Representation Learning, Scene Understanding, Transfer Learning, Translation

Visual relations, such as "person ride bike" and "bike next to car", offer a
comprehensive scene understanding of an image, and have already shown their
great utility in connecting computer vision and natural language. However, due
to the challenging combinatorial complexity of modeling
subject-predicate-object relation triplets, very little work has been done to
localize and predict visual relations. Inspired by the recent advances in
relational representation learning of knowledge bases and convolutional object
detection networks, we propose a Visual Translation Embedding network (VTransE)
for visual relation detection. VTransE places objects in a low-dimensional
relation space where a relation can be modeled as a simple vector translation,
i.e., subject + predicate $\approx$ object. We propose a novel feature
extraction layer that enables object-relation knowledge transfer in a
fully-convolutional fashion that supports training and inference in a single
forward/backward pass. To the best of our knowledge, VTransE is the first
end-to-end relation detection network. We demonstrate the effectiveness of
VTransE over other state-of-the-art methods on two large-scale datasets: Visual
Relationship and Visual Genome. Note that even though VTransE is a purely
visual model, it is still competitive to the Lu's multi-modal model with
language priors.
Stochastic Configuration Networks: Fundamentals and Algorithms
Authors: Dianhui Wang, Ming Li
Date: 2017-02-10 00:00:00
Tasks: regression

This paper contributes to a development of randomized methods for neural
networks. The proposed learner model is generated incrementally by stochastic
configuration (SC) algorithms, termed as Stochastic Configuration Networks
(SCNs). In contrast to the existing randomised learning algorithms for single
layer feed-forward neural networks (SLFNNs), we randomly assign the input
weights and biases of the hidden nodes in the light of a supervisory mechanism,
and the output weights are analytically evaluated in either constructive or
selective manner. As fundamentals of SCN-based data modelling techniques, we
establish some theoretical results on the universal approximation property.
Three versions of SC algorithms are presented for regression problems
(applicable for classification problems as well) in this work. Simulation
results concerning both function approximation and real world data regression
indicate some remarkable merits of our proposed SCNs in terms of less human
intervention on the network size setting, the scope adaptation of random
parameters, fast learning and sound generalization.
Identifying beneficial task relations for multi-task learning in deep neural networks
Authors: Joachim Bingel, Anders Søgaard
Date: 2017-02-27 00:00:00
Proceeding: EACL 2017 4
Tasks: Multi-Task Learning

Multi-task learning (MTL) in deep neural networks for NLP has recently
received increasing interest due to some compelling benefits, including its
potential to efficiently regularize models and to reduce the need for labeled
data. While it has brought significant improvements in a number of NLP tasks,
mixed results have been reported, and little is known about the conditions
under which MTL leads to gains in NLP. This paper sheds light on the specific
task relations that can lead to gains from MTL models over single-task setups.
Bayesian selection for the l2-Potts model regularization parameter: 1D piecewise constant signal denoising
Authors: Jordan Frecon, Nelly Pustelnik, Nicolas Dobigeon, Herwig Wendt, Patrice Abry
Date: 2016-08-27 00:00:00
Tasks: Denoising

Piecewise constant denoising can be solved either by deterministic
optimization approaches, based on the Potts model, or by stochastic Bayesian
procedures. The former lead to low computational time but require the selection
of a regularization parameter, whose value significantly impacts the achieved
solution, and whose automated selection remains an involved and challenging
problem. Conversely, fully Bayesian formalisms encapsulate the regularization
parameter selection into hierarchical models, at the price of high
computational costs. This contribution proposes an operational strategy that
combines hierarchical Bayesian and Potts model formulations, with the double
aim of automatically tuning the regularization parameter and of maintaining
computational effciency. The proposed procedure relies on formally connecting a
Bayesian framework to a l2-Potts functional. Behaviors and performance for the
proposed piecewise constant denoising and regularization parameter tuning
techniques are studied qualitatively and assessed quantitatively, and shown to
compare favorably against those of a fully Bayesian hierarchical procedure,
both in accuracy and in computational load.
Adaptive Learning to Speed-Up Control of Prosthetic Hands: a Few Things Everybody Should Know
Authors: Valentina Gregori, Arjan Gijsberts, Barbara Caputo
Date: 2017-02-27 00:00:00
Tasks: Domain Adaptation, Transfer Learning

A number of studies have proposed to use domain adaptation to reduce the
training efforts needed to control an upper-limb prosthesis exploiting
pre-trained models from prior subjects. These studies generally reported
impressive reductions in the required number of training samples to achieve a
certain level of accuracy for intact subjects. We further investigate two
popular methods in this field to verify whether this result equally applies to
amputees. Our findings show instead that this improvement can largely be
attributed to a suboptimal hyperparameter configuration. When hyperparameters
are appropriately tuned, the standard approach that does not exploit prior
information performs on par with the more complicated transfer learning
algorithms. Additionally, earlier studies erroneously assumed that the number
of training samples relates proportionally to the efforts required from the
subject. However, a repetition of a movement is the atomic unit for subjects
and the total number of repetitions should therefore be used as reliable
measure for training efforts. Also when correcting for this mistake, we do not
find any performance increase due to the use of prior models.
Refined Lower Bounds for Adversarial Bandits
Authors: Sébastien Gerchinovitz, Tor Lattimore
Date: 2016-05-24 00:00:00
Proceeding: NeurIPS 2016 12

We provide new lower bounds on the regret that must be suffered by
adversarial bandit algorithms. The new results show that recent upper bounds
that either (a) hold with high-probability or (b) depend on the total lossof
the best arm or (c) depend on the quadratic variation of the losses, are close
to tight. Besides this we prove two impossibility results. First, the existence
of a single arm that is optimal in every round cannot improve the regret in the
worst case. Second, the regret cannot scale with the effective range of the
losses. In contrast, both results are possible in the full-information setting.
Approches d'analyse distributionnelle pour améliorer la désambiguïsation sémantique
Authors: Mokhtar Billami, Núria Gala
Date: 2017-02-27 00:00:00
Tasks: Information Retrieval, Lexical Simplification, Machine Translation, Retrieval, Translation, Word Sense Disambiguation

Word sense disambiguation (WSD) improves many Natural Language Processing
(NLP) applications such as Information Retrieval, Machine Translation or
Lexical Simplification. WSD is the ability of determining a word sense among
different ones within a polysemic lexical unit taking into account the context.
The most straightforward approach uses a semantic proximity measure between the
word sense candidates of the target word and those of its context. Such a
method very easily entails a combinatorial explosion. In this paper, we propose
two methods based on distributional analysis which enable to reduce the
exponential complexity without losing the coherence. We present a comparison
between the selection of distributional neighbors and the linearly nearest
neighbors. The figures obtained show that selecting distributional neighbors
leads to better results.
A Knowledge-Based Approach to Word Sense Disambiguation by distributional selection and semantic features
Authors: Mokhtar Billami
Date: 2017-02-27 00:00:00
Tasks: Combinatorial Optimization, Information Retrieval, Lexical Simplification, Machine Translation, Retrieval, Semantic Similarity, Semantic Textual Similarity, Translation, Word Sense Disambiguation

Word sense disambiguation improves many Natural Language Processing (NLP)
applications such as Information Retrieval, Information Extraction, Machine
Translation, or Lexical Simplification. Roughly speaking, the aim is to choose
for each word in a text its best sense. One of the most popular method
estimates local semantic similarity relatedness between two word senses and
then extends it to all words from text. The most direct method computes a rough
score for every pair of word senses and chooses the lexical chain that has the
best score (we can imagine the exponential complexity that returns this
comprehensive approach). In this paper, we propose to use a combinatorial
optimization metaheuristic for choosing the nearest neighbors obtained by
distributional selection around the word to disambiguate. The test and the
evaluation of our method concern a corpus written in French by means of the
semantic network BabelNet. The obtained accuracy rate is 78 % on all names and
verbs chosen for the evaluation.
Adaptive Ensemble Prediction for Deep Neural Networks based on Confidence Level
Authors: Hiroshi Inoue
Date: 2017-02-27 00:00:00

Ensembling multiple predictions is a widely used technique for improving the
accuracy of various machine learning tasks. One obvious drawback of ensembling
is its higher execution cost during inference. In this paper, we first describe
our insights on the relationship between the probability of prediction and the
effect of ensembling with current deep neural networks; ensembling does not
help mispredictions for inputs predicted with a high probability even when
there is a non-negligible number of mispredicted inputs. This finding motivated
us to develop a way to adaptively control the ensembling. If the prediction for
an input reaches a high enough probability, i.e., the output from the softmax
function, on the basis of the confidence level, we stop ensembling for this
input to avoid wasting computation power. We evaluated the adaptive ensembling
by using various datasets and showed that it reduces the computation cost
significantly while achieving accuracy similar to that of static ensembling
using a pre-defined number of local predictions. We also show that our
statistically rigorous confidence-level-based early-exit condition reduces the
burden of task-dependent threshold tuning better compared with naive early exit
based on a pre-defined threshold in addition to yielding a better accuracy with
the same cost.
Uniform Deviation Bounds for Unbounded Loss Functions like k-Means
Authors: Olivier Bachem, Mario Lucic, S. Hamed Hassani, Andreas Krause
Date: 2017-02-27 00:00:00
Tasks: Clustering

Uniform deviation bounds limit the difference between a model's expected loss
and its loss on an empirical sample uniformly for all models in a learning
problem. As such, they are a critical component to empirical risk minimization.
In this paper, we provide a novel framework to obtain uniform deviation bounds
for loss functions which are *unbounded*. In our main application, this allows
us to obtain bounds for $k$-Means clustering under weak assumptions on the
underlying distribution. If the fourth moment is bounded, we prove a rate of
$\mathcal{O}\left(m^{-\frac12}\right)$ compared to the previously known
$\mathcal{O}\left(m^{-\frac14}\right)$ rate. Furthermore, we show that the rate
also depends on the kurtosis - the normalized fourth moment which measures the
"tailedness" of a distribution. We further provide improved rates under
progressively stronger assumptions, namely, bounded higher moments,
subgaussianity and bounded support.
Variational Inference using Implicit Distributions
Authors: Ferenc Huszár
Date: 2017-02-27 00:00:00
Tasks: Denoising, Density Ratio Estimation, Image Reconstruction, Variational Inference

Generative adversarial networks (GANs) have given us a great tool to fit
implicit generative models to data. Implicit distributions are ones we can
sample from easily, and take derivatives of samples with respect to model
parameters. These models are highly expressive and we argue they can prove just
as useful for variational inference (VI) as they are for generative modelling.
Several papers have proposed GAN-like algorithms for inference, however,
connections to the theory of VI are not always well understood. This paper
provides a unifying review of existing algorithms establishing connections
between variational autoencoders, adversarially learned inference, operator VI,
GAN-based image reconstruction, and more. Secondly, the paper provides a
framework for building new algorithms: depending on the way the variational
bound is expressed we introduce prior-contrastive and joint-contrastive
methods, and show practical inference algorithms based on either density ratio
estimation or denoising.