pwc-archive/papers-with-abstracts huggingface dataset:
Analysis of the Effect of Unexpected Outliers in the Classification of Spectroscopy Data
Authors: Frank G. Glavin, Michael G. Madden
Date: 2018-06-14 00:00:00
Tasks: Binary Classification, Classification, General Classification, Multi-class Classification

Multi-class classification algorithms are very widely used, but we argue that
they are not always ideal from a theoretical perspective, because they assume
all classes are characterized by the data, whereas in many applications,
training data for some classes may be entirely absent, rare, or statistically
unrepresentative. We evaluate one-sided classifiers as an alternative, since
they assume that only one class (the target) is well characterized. We consider
a task of identifying whether a substance contains a chlorinated solvent, based
on its chemical spectrum. For this application, it is not really feasible to
collect a statistically representative set of outliers, since that group may
contain \emph{anything} apart from the target chlorinated solvents. Using a new
one-sided classification toolkit, we compare a One-Sided k-NN algorithm with
two well-known binary classification algorithms, and conclude that the
one-sided classifier is more robust to unexpected outliers.
Low-rank geometric mean metric learning
Authors: Mukul Bhutani, Pratik Jawanpuria, Hiroyuki Kasai, Bamdev Mishra
Date: 2018-06-14 00:00:00
Tasks: Metric Learning

We propose a low-rank approach to learning a Mahalanobis metric from data.
Inspired by the recent geometric mean metric learning (GMML) algorithm, we
propose a low-rank variant of the algorithm. This allows to jointly learn a
low-dimensional subspace where the data reside and the Mahalanobis metric that
appropriately fits the data. Our results show that we compete effectively with
GMML at lower ranks.
4DFAB: A Large Scale 4D Facial Expression Database for Biometric Applications
Authors: Shiyang Cheng, Irene Kotsia, Maja Pantic, Stefanos Zafeiriou
Date: 2017-12-05 00:00:00
Tasks: Facial Expression Recognition, Facial Expression Recognition (FER)

The progress we are currently witnessing in many computer vision
applications, including automatic face analysis, would not be made possible
without tremendous efforts in collecting and annotating large scale visual
databases. To this end, we propose 4DFAB, a new large scale database of dynamic
high-resolution 3D faces (over 1,800,000 3D meshes). 4DFAB contains recordings
of 180 subjects captured in four different sessions spanning over a five-year
period. It contains 4D videos of subjects displaying both spontaneous and posed
facial behaviours. The database can be used for both face and facial expression
recognition, as well as behavioural biometrics. It can also be used to learn
very powerful blendshapes for parametrising facial behaviour. In this paper, we
conduct several experiments and demonstrate the usefulness of the database for
various applications. The database will be made publicly available for research
purposes.
Masked Autoregressive Flow for Density Estimation
Authors: George Papamakarios, Theo Pavlakou, Iain Murray
Date: 2017-05-19 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Density Estimation

Autoregressive models are among the best performing neural density
estimators. We describe an approach for increasing the flexibility of an
autoregressive model, based on modelling the random numbers that the model uses
internally when generating data. By constructing a stack of autoregressive
models, each modelling the random numbers of the next model in the stack, we
obtain a type of normalizing flow suitable for density estimation, which we
call Masked Autoregressive Flow. This type of flow is closely related to
Inverse Autoregressive Flow and is a generalization of Real NVP. Masked
Autoregressive Flow achieves state-of-the-art performance in a range of
general-purpose density estimation tasks.
Deep Generative Models in the Real-World: An Open Challenge from Medical Imaging
Authors: Xiaoran Chen, Nick Pawlowski, Martin Rajchl, Ben Glocker, Ender Konukoglu
Date: 2018-06-14 00:00:00
Tasks: Anomaly Detection

Recent advances in deep learning led to novel generative modeling techniques
that achieve unprecedented quality in generated samples and performance in
learning complex distributions in imaging data. These new models in medical
image computing have important applications that form clinically relevant and
very challenging unsupervised learning problems. In this paper, we explore the
feasibility of using state-of-the-art auto-encoder-based deep generative
models, such as variational and adversarial auto-encoders, for one such task:
abnormality detection in medical imaging. We utilize typical, publicly
available datasets with brain scans from healthy subjects and patients with
stroke lesions and brain tumors. We use the data from healthy subjects to train
different auto-encoder based models to learn the distribution of healthy images
and detect pathologies as outliers. Models that can better learn the data
distribution should be able to detect outliers more accurately. We evaluate the
detection performance of deep generative models and compare them with non-deep
learning based approaches to provide a benchmark of the current state of
research. We conclude that abnormality detection is a challenging task for deep
generative models and large room exists for improvement. In order to facilitate
further research, we aim to provide carefully pre-processed imaging data
available to the research community.
The committee machine: Computational to statistical gaps in learning a two-layers neural network
Authors: Benjamin Aubin, Antoine Maillard, Jean Barbier, Florent Krzakala, Nicolas Macris, Lenka Zdeborová
Date: 2018-06-14 00:00:00
Proceeding: NeurIPS 2018 12

Heuristic tools from statistical physics have been used in the past to locate the phase transitions and compute the optimal learning and generalization errors in the teacher-student scenario in multi-layer neural networks. In this contribution, we provide a rigorous justification of these approaches for a two-layers neural network model called the committee machine. We also introduce a version of the approximate message passing (AMP) algorithm for the committee machine that allows to perform optimal learning in polynomial time for a large set of parameters. We find that there are regimes in which a low generalization error is information-theoretically achievable while the AMP algorithm fails to deliver it, strongly suggesting that no efficient algorithm exists for those cases, and unveiling a large computational gap.
Stochastic Gradient Descent with Exponential Convergence Rates of Expected Classification Errors
Authors: Atsushi Nitanda, Taiji Suzuki
Date: 2018-06-14 00:00:00
Tasks: Binary Classification, Classification, General Classification

We consider stochastic gradient descent and its averaging variant for binary classification problems in a reproducing kernel Hilbert space. In the traditional analysis using a consistency property of loss functions, it is known that the expected classification error converges more slowly than the expected risk even when assuming a low-noise condition on the conditional label probabilities. Consequently, the resulting rate is sublinear. Therefore, it is important to consider whether much faster convergence of the expected classification error can be achieved. In recent research, an exponential convergence rate for stochastic gradient descent was shown under a strong low-noise condition but provided theoretical analysis was limited to the squared loss function, which is somewhat inadequate for binary classification tasks. In this paper, we show an exponential convergence of the expected classification error in the final phase of the stochastic gradient descent for a wide class of differentiable convex loss functions under similar assumptions. As for the averaged stochastic gradient descent, we show that the same convergence rate holds from the early phase of training. In experiments, we verify our analyses on the $L_2$-regularized logistic regression.
ServeNet: A Deep Neural Network for Web Services Classification
Authors: Yilong Yang, Nafees Qamar, Peng Liu, Katarina Grolinger, Weiru Wang, Zhi Li, Zhifang Liao
Date: 2018-06-14 00:00:00
Tasks: BIG-bench Machine Learning, Classification, Feature Engineering, General Classification

Automated service classification plays a crucial role in service discovery, selection, and composition. Machine learning has been widely used for service classification in recent years. However, the performance of conventional machine learning methods highly depends on the quality of manual feature engineering. In this paper, we present a novel deep neural network to automatically abstract low-level representation of both service name and service description to high-level merged features without feature engineering and the length limitation, and then predict service classification on 50 service categories. To demonstrate the effectiveness of our approach, we conduct a comprehensive experimental study by comparing 10 machine learning methods on 10,000 real-world web services. The result shows that the proposed deep neural network can achieve higher accuracy in classification and more robust than other machine learning methods.
Transfer Learning for Context-Aware Question Matching in Information-seeking Conversations in E-commerce
Authors: Minghui Qiu, Liu Yang, Feng Ji, Weipeng Zhao, Wei Zhou, Jun Huang, Haiqing Chen, W. Bruce Croft, Wei. Lin
Date: 2018-06-14 00:00:00
Proceeding: ACL 2018 7
Tasks: Chatbot, Text Matching, Transfer Learning

Building multi-turn information-seeking conversation systems is an important
and challenging research topic. Although several advanced neural text matching
models have been proposed for this task, they are generally not efficient for
industrial applications. Furthermore, they rely on a large amount of labeled
data, which may not be available in real-world applications. To alleviate these
problems, we study transfer learning for multi-turn information seeking
conversations in this paper. We first propose an efficient and effective
multi-turn conversation model based on convolutional neural networks. After
that, we extend our model to adapt the knowledge learned from a resource-rich
domain to enhance the performance. Finally, we deployed our model in an
industrial chatbot called AliMe Assist
(https://consumerservice.taobao.com/online-help) and observed a significant
improvement over the existing online model.
Urdu Word Segmentation using Conditional Random Fields (CRFs)
Authors: Haris Bin Zia, Agha Ali Raza, Awais Athar
Date: 2018-06-14 00:00:00
Proceeding: COLING 2018 8
Tasks: Segmentation

State-of-the-art Natural Language Processing algorithms rely heavily on
efficient word segmentation. Urdu is amongst languages for which word
segmentation is a complex task as it exhibits space omission as well as space
insertion issues. This is partly due to the Arabic script which although
cursive in nature, consists of characters that have inherent joining and
non-joining attributes regardless of word boundary. This paper presents a word
segmentation system for Urdu which uses a Conditional Random Field sequence
modeler with orthographic, linguistic and morphological features. Our proposed
model automatically learns to predict white space as word boundary as well as
Zero Width Non-Joiner (ZWNJ) as sub-word boundary. Using a manually annotated
corpus, our model achieves F1 score of 0.97 for word boundary identification
and 0.85 for sub-word boundary identification tasks. We have made our code and
corpus publicly available to make our results reproducible.
Only Bayes should learn a manifold (on the estimation of differential geometric structure from data)
Authors: Søren Hauberg
Date: 2018-06-13 00:00:00

We investigate learning of the differential geometric structure of a data manifold embedded in a high-dimensional Euclidean space. We first analyze kernel-based algorithms and show that under the usual regularizations, non-probabilistic methods cannot recover the differential geometric structure, but instead find mostly linear manifolds or spaces equipped with teleports. To properly learn the differential geometric structure, non-probabilistic methods must apply regularizations that enforce large gradients, which go against common wisdom. We repeat the analysis for probabilistic methods and find that under reasonable priors, the geometric structure can be recovered. Fully exploiting the recovered structure, however, requires the development of stochastic extensions to classic Riemannian geometry. We take early steps in that regard. Finally, we partly extend the analysis to modern models based on neural networks, thereby highlighting geometric and probabilistic shortcomings of current deep generative models.
Modeling Coherence for Neural Machine Translation with Dynamic and Topic Caches
Authors: Shaohui Kuang, Deyi Xiong, Weihua Luo, Guodong Zhou
Date: 2017-11-30 00:00:00
Proceeding: COLING 2018 8
Tasks: Machine Translation, NMT, Sentence, Translation

Sentences in a well-formed text are connected to each other via various links
to form the cohesive structure of the text. Current neural machine translation
(NMT) systems translate a text in a conventional sentence-by-sentence fashion,
ignoring such cross-sentence links and dependencies. This may lead to generate
an incoherent target text for a coherent source text. In order to handle this
issue, we propose a cache-based approach to modeling coherence for neural
machine translation by capturing contextual information either from recently
translated sentences or the entire document. Particularly, we explore two types
of caches: a dynamic cache, which stores words from the best translation
hypotheses of preceding sentences, and a topic cache, which maintains a set of
target-side topical words that are semantically related to the document to be
translated. On this basis, we build a new layer to score target words in these
two caches with a cache-based neural model. Here the estimated probabilities
from the cache-based neural model are combined with NMT probabilities into the
final word prediction probabilities via a gating mechanism. Finally, the
proposed cache-based neural model is trained jointly with NMT system in an
end-to-end manner. Experiments and analysis presented in this paper demonstrate
that the proposed cache-based model achieves substantial improvements over
several state-of-the-art SMT and NMT baselines.
Cross-modal Hallucination for Few-shot Fine-grained Recognition
Authors: Frederik Pahde, Patrick Jähnichen, Tassilo Klein, Moin Nabi
Date: 2018-06-13 00:00:00
Tasks: Hallucination

State-of-the-art deep learning algorithms generally require large amounts of
data for model training. Lack thereof can severely deteriorate the performance,
particularly in scenarios with fine-grained boundaries between categories. To
this end, we propose a multimodal approach that facilitates bridging the
information gap by means of meaningful joint embeddings. Specifically, we
present a benchmark that is multimodal during training (i.e. images and texts)
and single-modal in testing time (i.e. images), with the associated task to
utilize multimodal data in base classes (with many samples), to learn explicit
visual classifiers for novel classes (with few samples). Next, we propose a
framework built upon the idea of cross-modal data hallucination. In this
regard, we introduce a discriminative text-conditional GAN for sample
generation with a simple self-paced strategy for sample selection. We show the
results of our proposed discriminative hallucinated method for 1-, 2-, and 5-
shot learning on the CUB dataset, where the accuracy is improved by employing
multimodal data.
Ranking Recovery from Limited Comparisons using Low-Rank Matrix Completion
Authors: Tal Levy, Alireza Vahid, Raja Giryes
Date: 2018-06-14 00:00:00
Tasks: Form, Low-Rank Matrix Completion, Matrix Completion

This paper proposes a new method for solving the well-known rank aggregation
problem from pairwise comparisons using the method of low-rank matrix
completion. The partial and noisy data of pairwise comparisons is transformed
into a matrix form. We then use tools from matrix completion, which has served
as a major component in the low-rank completion solution of the Netflix
challenge, to construct the preference of the different objects. In our
approach, the data of multiple comparisons is used to create an estimate of the
probability of object i to win (or be chosen) over object j, where only a
partial set of comparisons between N objects is known. The data is then
transformed into a matrix form for which the noiseless solution has a known
rank of one. An alternating minimization algorithm, in which the target matrix
takes a bilinear form, is then used in combination with maximum likelihood
estimation for both factors. The reconstructed matrix is used to obtain the
true underlying preference intensity. This work demonstrates the improvement of
our proposed algorithm over the current state-of-the-art in both simulated
scenarios and real data.
Configurable Markov Decision Processes
Authors: Alberto Maria Metelli, Mirco Mutti, Marcello Restelli
Date: 2018-06-14 00:00:00
Proceeding: ICML 2018 7

In many real-world problems, there is the possibility to configure, to a
limited extent, some environmental parameters to improve the performance of a
learning agent. In this paper, we propose a novel framework, Configurable
Markov Decision Processes (Conf-MDPs), to model this new type of interaction
with the environment. Furthermore, we provide a new learning algorithm, Safe
Policy-Model Iteration (SPMI), to jointly and adaptively optimize the policy
and the environment configuration. After having introduced our approach and
derived some theoretical results, we present the experimental evaluation in two
explicative problems to show the benefits of the environment configurability on
the performance of the learned policy.
Learning Dynamics of Linear Denoising Autoencoders
Authors: Arnu Pretorius, Steve Kroon, Herman Kamper
Date: 2018-06-14 00:00:00
Proceeding: ICML 2018 7
Tasks: Denoising, Representation Learning

Denoising autoencoders (DAEs) have proven useful for unsupervised
representation learning, but a thorough theoretical understanding is still
lacking of how the input noise influences learning. Here we develop theory for
how noise influences learning in DAEs. By focusing on linear DAEs, we are able
to derive analytic expressions that exactly describe their learning dynamics.
We verify our theoretical predictions with simulations as well as experiments
on MNIST and CIFAR-10. The theory illustrates how, when tuned correctly, noise
allows DAEs to ignore low variance directions in the inputs while learning to
reconstruct them. Furthermore, in a comparison of the learning dynamics of DAEs
to standard regularised autoencoders, we show that noise has a similar
regularisation effect to weight decay, but with faster training dynamics. We
also show that our theoretical predictions approximate learning dynamics on
real-world data and qualitatively match observed dynamics in nonlinear DAEs.
An Approach to Vehicle Trajectory Prediction Using Automatically Generated Traffic Maps
Authors: Jannik Quehl, Haohao Hu, Sascha Wirges, Martin Lauer
Date: 2018-02-23 00:00:00
Tasks: Prediction, Trajectory Prediction

Trajectory and intention prediction of traffic participants is an important
task in automated driving and crucial for safe interaction with the
environment. In this paper, we present a new approach to vehicle trajectory
prediction based on automatically generated maps containing statistical
information about the behavior of traffic participants in a given area. These
maps are generated based on trajectory observations using image processing and
map matching techniques and contain all typical vehicle movements and
probabilities in the considered area. Our prediction approach matches an
observed trajectory to a behavior contained in the map and uses this
information to generate a prediction. We evaluated our approach on a dataset
containing over 14000 trajectories and found that it produces significantly
more precise mid-term predictions compared to motion model-based prediction
approaches.
Incremental Natural Language Processing: Challenges, Strategies, and Evaluation
Authors: Arne Köhn
Date: 2018-05-31 00:00:00

Incrementality is ubiquitous in human-human interaction and beneficial for
human-computer interaction. It has been a topic of research in different parts
of the NLP community, mostly with focus on the specific topic at hand even
though incremental systems have to deal with similar challenges regardless of
domain. In this survey, I consolidate and categorize the approaches,
identifying similarities and differences in the computation and data, and show
trade-offs that have to be considered. A focus lies on evaluating incremental
systems because the standard metrics often fail to capture the incremental
properties of a system and coming up with a suitable evaluation scheme is
non-trivial.
On the Perceptron's Compression
Authors: Shay Moran, Ido Nachum, Itai Panasoff, Amir Yehudayoff
Date: 2018-06-14 00:00:00

We study and provide exposition to several phenomena that are related to the
perceptron's compression. One theme concerns modifications of the perceptron
algorithm that yield better guarantees on the margin of the hyperplane it
outputs. These modifications can be useful in training neural networks as well,
and we demonstrate them with some experimental data. In a second theme, we
deduce conclusions from the perceptron's compression in various contexts.
Aggregating Predictions on Multiple Non-disclosed Datasets using Conformal Prediction
Authors: Ola Spjuth, Lars Carlsson, Niharika Gauraha
Date: 2018-06-11 00:00:00
Tasks: Conformal Prediction, Prediction, valid

Conformal Prediction is a machine learning methodology that produces valid
prediction regions under mild conditions. In this paper, we explore the
application of making predictions over multiple data sources of different sizes
without disclosing data between the sources. We propose that each data source
applies a transductive conformal predictor independently using the local data,
and that the individual predictions are then aggregated to form a combined
prediction region. We demonstrate the method on several data sets, and show
that the proposed method produces conservatively valid predictions and reduces
the variance in the aggregated predictions. We also study the effect that the
number of data sources and size of each source has on aggregated predictions,
as compared with equally sized sources and pooled data.
Dynamical Isometry and a Mean Field Theory of RNNs: Gating Enables Signal Propagation in Recurrent Neural Networks
Authors: Minmin Chen, Jeffrey Pennington, Samuel S. Schoenholz
Date: 2018-06-14 00:00:00
Proceeding: ICML 2018 7
Tasks: Language Modeling, Language Modelling

Recurrent neural networks have gained widespread use in modeling sequence
data across various domains. While many successful recurrent architectures
employ a notion of gating, the exact mechanism that enables such remarkable
performance is not well understood. We develop a theory for signal propagation
in recurrent networks after random initialization using a combination of mean
field theory and random matrix theory. To simplify our discussion, we introduce
a new RNN cell with a simple gating mechanism that we call the minimalRNN and
compare it with vanilla RNNs. Our theory allows us to define a maximum
timescale over which RNNs can remember an input. We show that this theory
predicts trainability for both recurrent architectures. We show that gated
recurrent networks feature a much broader, more robust, trainable region than
vanilla RNNs, which corroborates recent experimental findings. Finally, we
develop a closed-form critical initialization scheme that achieves dynamical
isometry in both vanilla RNNs and minimalRNNs. We show that this results in
significantly improvement in training dynamics. Finally, we demonstrate that
the minimalRNN achieves comparable performance to its more complex
counterparts, such as LSTMs or GRUs, on a language modeling task.