pwc-archive/papers-with-abstracts huggingface dataset:
Capsule Networks for Brain Tumor Classification based on MRI Images and Course Tumor Boundaries
Authors: Parnian Afshar, Konstantinos N. Plataniotis, Arash Mohammadi
Date: 2018-11-01 00:00:00
Tasks: Brain Tumor Classification, General Classification, Miscellaneous

According to official statistics, cancer is considered as the second leading
cause of human fatalities. Among different types of cancer, brain tumor is seen
as one of the deadliest forms due to its aggressive nature, heterogeneous
characteristics, and low relative survival rate. Determining the type of brain
tumor has significant impact on the treatment choice and patient's survival.
Human-centered diagnosis is typically error-prone and unreliable resulting in a
recent surge of interest to automatize this process using convolutional neural
networks (CNNs). CNNs, however, fail to fully utilize spatial relations, which
is particularly harmful for tumor classification, as the relation between the
tumor and its surrounding tissue is a critical indicator of the tumor's type.
In our recent work, we have incorporated newly developed CapsNets to overcome
this shortcoming. CapsNets are, however, highly sensitive to the miscellaneous
image background. The paper addresses this gap. The main contribution is to
equip CapsNet with access to the tumor surrounding tissues, without distracting
it from the main target. A modified CapsNet architecture is, therefore,
proposed for brain tumor classification, which takes the tumor coarse
boundaries as extra inputs within its pipeline to increase the CapsNet's focus.
The proposed approach noticeably outperforms its counterparts.
Variational Dropout via Empirical Bayes
Authors: Valery Kharitonov, Dmitry Molchanov, Dmitry Vetrov
Date: 2018-11-01 00:00:00

We study the Automatic Relevance Determination procedure applied to deep
neural networks. We show that ARD applied to Bayesian DNNs with Gaussian
approximate posterior distributions leads to a variational bound similar to
that of variational dropout, and in the case of a fixed dropout rate,
objectives are exactly the same. Experimental results show that the two
approaches yield comparable results in practice even when the dropout rates are
trained. This leads to an alternative Bayesian interpretation of dropout and
mitigates some of the theoretical issues that arise with the use of improper
priors in the variational dropout model. Additionally, we explore the use of
the hierarchical priors in ARD and show that it helps achieve higher sparsity
for the same accuracy.
Multilingual Embeddings Jointly Induced from Contexts and Concepts: Simple, Strong and Scalable
Authors: Philipp Dufter, Mengjie Zhao, Hinrich Schütze
Date: 2018-11-01 00:00:00
Tasks: Multilingual Word Embeddings, Sentence, Word Embeddings

Word embeddings induced from local context are prevalent in NLP. A simple and effective context-based multilingual embedding learner is Levy et al. (2017)'s S-ID (sentence ID) method. Another line of work induces high-performing multilingual embeddings from concepts (Dufter et al., 2018). In this paper, we propose Co+Co, a simple and scalable method that combines context-based and concept-based learning. From a sentence aligned corpus, concepts are extracted via sampling; words are then associated with their concept ID and sentence ID in embedding learning. This is the first work that successfully combines context-based and concept-based embedding learning. We show that Co+Co performs well for two different application scenarios: the Parallel Bible Corpus (1000+ languages, low-resource) and EuroParl (12 languages, high-resource). Among methods applicable to both corpora, Co+Co performs best in our evaluation setup of six tasks.
Functional Nonlinear Sparse Models
Authors: Luiz. F. O. Chamon, Yonina C. Eldar, Alejandro Ribeiro
Date: 2018-11-01 00:00:00
Tasks: Robust classification, Spectrum Cartography, Super-Resolution

Signal processing is rich in inherently continuous and often nonlinear applications, such as spectral estimation, optical imaging, and super-resolution microscopy, in which sparsity plays a key role in obtaining state-of-the-art results. Coping with the infinite dimensionality and non-convexity of these problems typically involves discretization and convex relaxations, e.g., using atomic norms. Nevertheless, grid mismatch and other coherence issues often lead to discretized versions of sparse signals that are not sparse. Even if they are, recovering sparse solutions using convex relaxations requires assumptions that may be hard to meet in practice. What is more, problems involving nonlinear measurements remain non-convex even after relaxing the sparsity objective. We address these issues by directly tackling the continuous, nonlinear problem cast as a sparse functional optimization program. We prove that when these problems are non-atomic, they have no duality gap and can therefore be solved efficiently using duality and~(stochastic) convex optimization methods. We illustrate the wide range of applications of this approach by formulating and solving problems from nonlinear spectral estimation and robust classification.
Connections between physics, mathematics and deep learning
Authors: Jean Thierry-Mieg
Date: 2018-11-01 00:00:00
Tasks: Deep Learning

Starting from the Fermat's principle of least action, which governs classical and quantum mechanics and from the theory of exterior differential forms, which governs the geometry of curved manifolds, we show how to derive the equations governing neural networks in an intrinsic, coordinate invariant way, where the loss function plays the role of the Hamiltonian. To be covariant, these equations imply a layer metric which is instrumental in pretraining and explains the role of conjugation when using complex numbers. The differential formalism also clarifies the relation of the gradient descent optimizer with Aristotelian and Newtonian mechanics and why large learning steps break the logic of the linearization procedure. We hope that this formal presentation of the differential geometry of neural networks will encourage some physicists to dive into deep learning, and reciprocally, that the specialists of deep learning will better appreciate the close interconnection of their subject with the foundations of classical and quantum field theory.
Tropical Modeling of Weighted Transducer Algorithms on Graphs
Authors: Emmanouil Theodosis, Petros Maragos
Date: 2018-11-01 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), speech-recognition, Speech Recognition

Weighted Finite State Transducers (WFSTs) are versatile data structures that
can model a great number of problems, ranging from Automatic Speech Recognition
to DNA sequencing. Traditional computer science algorithms are employed when
working with these structures in order to optimise their size, but also the
runtime of decoding algorithms. However, these algorithms are not unified under
a common framework that would allow for their treatment as a whole. Moreover,
the inherent geometrical representation of WFSTs, coupled with the
topology-preserving algorithms that operate on them make the structures ideal
for tropical analysis. The benefits of such analysis have a twofold nature;
first, matrix operations offer a connection to nonlinear vector space and
spectral theory, and, second, tropical algebra offers a connection to tropical
geometry. In this work we model some of the most frequently used algorithms in
WFSTs by using tropical algebra; this provides a theoretical unification and
allows us to also analyse aspects of their tropical geometry. Further, we
provide insights via numerical examples.
On Difficulties of Cross-Lingual Transfer with Order Differences: A Case Study on Dependency Parsing
Authors: Wasi Uddin Ahmad, Zhisong Zhang, Xuezhe Ma, Eduard Hovy, Kai-Wei Chang, Nanyun Peng
Date: 2018-11-01 00:00:00
Proceeding: NAACL 2019 6
Tasks: Cross-Lingual Transfer, Dependency Parsing

Different languages might have different word orders. In this paper, we
investigate cross-lingual transfer and posit that an order-agnostic model will
perform better when transferring to distant foreign languages. To test our
hypothesis, we train dependency parsers on an English corpus and evaluate their
transfer performance on 30 other languages. Specifically, we compare encoders
and decoders based on Recurrent Neural Networks (RNNs) and modified
self-attentive architectures. The former relies on sequential information while
the latter is more flexible at modeling word order. Rigorous experiments and
detailed analysis shows that RNN-based architectures transfer well to languages
that are close to English, while self-attentive models have better overall
cross-lingual transferability and perform especially well on distant languages.
Multiple-Attribute Text Style Transfer
Authors: Sandeep Subramanian, Guillaume Lample, Eric Michael Smith, Ludovic Denoyer, Marc'Aurelio Ranzato, Y-Lan Boureau
Date: 2018-11-01 00:00:00
Tasks: Attribute, Disentanglement, Style Transfer, Text Style Transfer, Translation

The dominant approach to unsupervised "style transfer" in text is based on the idea of learning a latent representation, which is independent of the attributes specifying its "style". In this paper, we show that this condition is not necessary and is not always met in practice, even with domain adversarial training that explicitly aims at learning such disentangled representations. We thus propose a new model that controls several factors of variation in textual data where this condition on disentanglement is replaced with a simpler mechanism based on back-translation. Our method allows control over multiple attributes, like gender, sentiment, product type, etc., and a more fine-grained control on the trade-off between content preservation and change of style with a pooling operator in the latent space. Our experiments demonstrate that the fully entangled model produces better generations, even when tested on new and more challenging benchmarks comprising reviews with multiple sentences and multiple attributes.
Improving Information Retrieval Results for Persian Documents using FarsNet
Authors: Adel Rahimi, Mohammad bahrani
Date: 2018-11-01 00:00:00
Tasks: Information Retrieval, Retrieval

In this paper, we propose a new method for query expansion, which uses
FarsNet (Persian WordNet) to find similar tokens related to the query and
expand the semantic meaning of the query. For this purpose, we use synonymy
relations in FarsNet and extract the related synonyms to query words. This
algorithm is used to enhance information retrieval systems and improve search
results. The overall evaluation of this system in comparison to the baseline
method (without using query expansion) shows an improvement of about 9 percent
in Mean Average Precision (MAP).
Enhancing the Structural Performance of Additively Manufactured Objects
Authors: Erva Ulu
Date: 2018-11-01 00:00:00

The ability to accurately quantify the performance an additively manufactured
(AM) product is important for a widespread industry adoption of AM as the
design is required to: (1) satisfy geometrical constraints, (2) satisfy
structural constraints dictated by its intended function, and (3) be cost
effective compared to traditional manufacturing methods. Optimization
techniques offer design aids in creating cost-effective structures that meet
the prescribed structural objectives. The fundamental problem in existing
approaches lies in the difficulty to quantify the structural performance as
each unique design leads to a new set of analyses to determine the structural
robustness and such analyses can be very costly due to the complexity of in-use
forces experienced by the structure. This work develops computationally
tractable methods tailored to maximize the structural performance of AM
products. A geometry preserving build orientation optimization method as well
as data-driven shape optimization approaches to structural design are
presented. Proposed methods greatly enhance the value of AM technology by
taking advantage of the design space enabled by it for a broad class of
problems involving complex in-use loads.
Content preserving text generation with attribute controls
Authors: Lajanugen Logeswaran, Honglak Lee, Samy Bengio
Date: 2018-11-03 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Attribute, Sentence, Text Generation, Translation

In this work, we address the problem of modifying textual attributes of
sentences. Given an input sentence and a set of attribute labels, we attempt to
generate sentences that are compatible with the conditioning information. To
ensure that the model generates content compatible sentences, we introduce a
reconstruction loss which interpolates between auto-encoding and
back-translation loss components. We propose an adversarial loss to enforce
generated samples to be attribute compatible and realistic. Through
quantitative, qualitative and human evaluations we demonstrate that our model
is capable of generating fluent sentences that better reflect the conditioning
information compared to prior methods. We further demonstrate that the model is
capable of simultaneously controlling multiple attributes.
A Tutorial on Spectral Clustering
Authors: Ulrike von Luxburg
Date: 2007-11-01 00:00:00
Tasks: Clustering

In recent years, spectral clustering has become one of the most popular
modern clustering algorithms. It is simple to implement, can be solved
efficiently by standard linear algebra software, and very often outperforms
traditional clustering algorithms such as the k-means algorithm. On the first
glance spectral clustering appears slightly mysterious, and it is not obvious
to see why it works at all and what it really does. The goal of this tutorial
is to give some intuition on those questions. We describe different graph
Laplacians and their basic properties, present the most common spectral
clustering algorithms, and derive those algorithms from scratch by several
different approaches. Advantages and disadvantages of the different spectral
clustering algorithms are discussed.
Evolutionary Data Measures: Understanding the Difficulty of Text Classification Tasks
Authors: Edward Collins, Nikolai Rozanov, Bingbing Zhang
Date: 2018-11-05 00:00:00
Proceeding: CONLL 2018 10
Tasks: Classification, General Classification, text-classification, Text Classification

Classification tasks are usually analysed and improved through new model
architectures or hyperparameter optimisation but the underlying properties of
datasets are discovered on an ad-hoc basis as errors occur. However,
understanding the properties of the data is crucial in perfecting models. In
this paper we analyse exactly which characteristics of a dataset best determine
how difficult that dataset is for the task of text classification. We then
propose an intuitive measure of difficulty for text classification datasets
which is simple and fast to calculate. We show that this measure generalises to
unseen data by comparing it to state-of-the-art datasets and results. This
measure can be used to analyse the precise source of errors in a dataset and
allows fast estimation of how difficult a dataset is to learn. We searched for
this measure by training 12 classical and neural network based models on 78
real-world datasets, then use a genetic algorithm to discover the best measure
of difficulty. Our difficulty-calculating code ( https://github.com/Wluper/edm
) and datasets ( http://data.wluper.com ) are publicly available.
Fast Non-Bayesian Poisson Factorization for Implicit-Feedback Recommendations
Authors: David Cortes
Date: 2018-11-05 00:00:00
Tasks: Recommendation Systems, Variational Inference

This work explores non-negative low-rank matrix factorization based on regularized Poisson models (PF or "Poisson factorization" for short) for recommender systems with implicit-feedback data. The properties of Poisson likelihood allow a shortcut for very fast computations over zero-valued inputs, and oftentimes results in very sparse factors for both users and items. Compared to HPF (a popular Bayesian formulation of the problem with hierarchical priors), the frequentist optimization-based approach presented here tends to produce better top-N recommendations with significantly shorter fitting times, on top of having sparse solutions.
A Unified Framework of DNN Weight Pruning and Weight Clustering/Quantization Using ADMM
Authors: Shaokai Ye, Tianyun Zhang, Kaiqi Zhang, Jiayu Li, Jiaming Xie, Yun Liang, Sijia Liu, Xue Lin, Yanzhi Wang
Date: 2018-11-05 00:00:00
Tasks: Clustering, Model Compression, Quantization

Many model compression techniques of Deep Neural Networks (DNNs) have been
investigated, including weight pruning, weight clustering and quantization,
etc. Weight pruning leverages the redundancy in the number of weights in DNNs,
while weight clustering/quantization leverages the redundancy in the number of
bit representations of weights. They can be effectively combined in order to
exploit the maximum degree of redundancy. However, there lacks a systematic
investigation in literature towards this direction.
  In this paper, we fill this void and develop a unified, systematic framework
of DNN weight pruning and clustering/quantization using Alternating Direction
Method of Multipliers (ADMM), a powerful technique in optimization theory to
deal with non-convex optimization problems. Both DNN weight pruning and
clustering/quantization, as well as their combinations, can be solved in a
unified manner. For further performance improvement in this framework, we adopt
multiple techniques including iterative weight quantization and retraining,
joint weight clustering training and centroid updating, weight clustering
retraining, etc. The proposed framework achieves significant improvements both
in individual weight pruning and clustering/quantization problems, as well as
their combinations. For weight pruning alone, we achieve 167x weight reduction
in LeNet-5, 24.7x in AlexNet, and 23.4x in VGGNet, without any accuracy loss.
For the combination of DNN weight pruning and clustering/quantization, we
achieve 1,910x and 210x storage reduction of weight data on LeNet-5 and
AlexNet, respectively, without accuracy loss. Our codes and models are released
at the link http://bit.ly/2D3F0np
Lower Bounds for Parallel and Randomized Convex Optimization
Authors: Jelena Diakonikolas, Cristóbal Guzmán
Date: 2018-11-05 00:00:00

We study the question of whether parallelization in the exploration of the feasible set can be used to speed up convex optimization, in the local oracle model of computation. We show that the answer is negative for both deterministic and randomized algorithms applied to essentially any of the interesting geometries and nonsmooth, weakly-smooth, or smooth objective functions. In particular, we show that it is not possible to obtain a polylogarithmic (in the sequential complexity of the problem) number of parallel rounds with a polynomial (in the dimension) number of queries per round. In the majority of these settings and when the dimension of the space is polynomial in the inverse target accuracy, our lower bounds match the oracle complexity of sequential convex optimization, up to at most a logarithmic factor in the dimension, which makes them (nearly) tight. Prior to our work, lower bounds for parallel convex optimization algorithms were only known in a small fraction of the settings considered in this paper, mainly applying to Euclidean ($\ell_2$) and $\ell_\infty$ spaces. Our work provides a more general approach for proving lower bounds in the setting of parallel convex optimization.
Janossy Pooling: Learning Deep Permutation-Invariant Functions for Variable-Size Inputs
Authors: Ryan L. Murphy, Balasubramaniam Srinivasan, Vinayak Rao, Bruno Ribeiro
Date: 2018-11-05 00:00:00
Proceeding: ICLR 2019 5
Tasks: Stochastic Optimization

We consider a simple and overarching representation for permutation-invariant
functions of sequences (or multiset functions). Our approach, which we call
Janossy pooling, expresses a permutation-invariant function as the average of a
permutation-sensitive function applied to all reorderings of the input
sequence. This allows us to leverage the rich and mature literature on
permutation-sensitive functions to construct novel and flexible
permutation-invariant functions. If carried out naively, Janossy pooling can be
computationally prohibitive. To allow computational tractability, we consider
three kinds of approximations: canonical orderings of sequences, functions with
$k$-order interactions, and stochastic optimization algorithms with random
permutations. Our framework unifies a variety of existing work in the
literature, and suggests possible modeling and algorithmic extensions. We
explore a few in our experiments, which demonstrate improved performance over
current state-of-the-art methods.
Learning Two Layer Rectified Neural Networks in Polynomial Time
Authors: Ainesh Bakshi, Rajesh Jayaram, David P. Woodruff
Date: 2018-11-05 00:00:00
Tasks: Vocal Bursts Valence Prediction

Consider the following fundamental learning problem: given input examples $x
\in \mathbb{R}^d$ and their vector-valued labels, as defined by an underlying
generative neural network, recover the weight matrices of this network. We
consider two-layer networks, mapping $\mathbb{R}^d$ to $\mathbb{R}^m$, with $k$
non-linear activation units $f(\cdot)$, where $f(x) = \max \{x , 0\}$ is the
ReLU. Such a network is specified by two weight matrices, $\mathbf{U}^* \in
\mathbb{R}^{m \times k}, \mathbf{V}^* \in \mathbb{R}^{k \times d}$, such that
the label of an example $x \in \mathbb{R}^{d}$ is given by $\mathbf{U}^*
f(\mathbf{V}^* x)$, where $f(\cdot)$ is applied coordinate-wise. Given $n$
samples as a matrix $\mathbf{X} \in \mathbb{R}^{d \times n}$ and the (possibly
noisy) labels $\mathbf{U}^* f(\mathbf{V}^* \mathbf{X}) + \mathbf{E}$ of the
network on these samples, where $\mathbf{E}$ is a noise matrix, our goal is to
recover the weight matrices $\mathbf{U}^*$ and $\mathbf{V}^*$.
  In this work, we develop algorithms and hardness results under varying
assumptions on the input and noise. Although the problem is NP-hard even for
$k=2$, by assuming Gaussian marginals over the input $\mathbf{X}$ we are able
to develop polynomial time algorithms for the approximate recovery of
$\mathbf{U}^*$ and $\mathbf{V}^*$. Perhaps surprisingly, in the noiseless case
our algorithms recover $\mathbf{U}^*,\mathbf{V}^*$ exactly, i.e., with no
error. To the best of the our knowledge, this is the first algorithm to
accomplish exact recovery. For the noisy case, we give the first polynomial
time algorithm that approximately recovers the weights in the presence of
mean-zero noise $\mathbf{E}$. Our algorithms generalize to a larger class of
rectified activation functions, $f(x) = 0$ when $x\leq 0$, and $f(x) > 0$
otherwise.
Do RNNs learn human-like abstract word order preferences?
Authors: Richard Futrell, Roger P. Levy
Date: 2018-11-05 00:00:00
Proceeding: WS 2019 1
Tasks: Language Modeling, Language Modelling, Sentence

RNN language models have achieved state-of-the-art results on various tasks,
but what exactly they are representing about syntax is as yet unclear. Here we
investigate whether RNN language models learn humanlike word order preferences
in syntactic alternations. We collect language model surprisal scores for
controlled sentence stimuli exhibiting major syntactic alternations in English:
heavy NP shift, particle shift, the dative alternation, and the genitive
alternation. We show that RNN language models reproduce human preferences in
these alternations based on NP length, animacy, and definiteness. We collect
human acceptability ratings for our stimuli, in the first acceptability
judgment experiment directly manipulating the predictors of syntactic
alternations. We show that the RNNs' performance is similar to the human
acceptability ratings and is not matched by an n-gram baseline model. Our
results show that RNNs learn the abstract features of weight, animacy, and
definiteness which underlie soft constraints on syntactic alternations.