pwc-archive/papers-with-abstracts huggingface dataset:
Residual Squeeze VGG16
Authors: Hussam Qassim, David Feinzimer, Abhishek Verma
Date: 2017-05-05 00:00:00
Tasks: image-classification, Image Classification, object-detection, Object Detection

Deep learning has given way to a new era of machine learning, apart from
computer vision. Convolutional neural networks have been implemented in image
classification, segmentation and object detection. Despite recent advancements,
we are still in the very early stages and have yet to settle on best practices
for network architecture in terms of deep design, small in size and a short
training time. In this work, we propose a very deep neural network comprised of
16 Convolutional layers compressed with the Fire Module adapted from the
SQUEEZENET model. We also call for the addition of residual connections to help
suppress degradation. This model can be implemented on almost every neural
network model with fully incorporated residual learning. This proposed model
Residual-Squeeze-VGG16 (ResSquVGG16) trained on the large-scale MIT
Places365-Standard scene dataset. In our tests, the model performed with
accuracy similar to the pre-trained VGG16 model in Top-1 and Top-5 validation
accuracy while also enjoying a 23.86% reduction in training time and an 88.4%
reduction in size. In our tests, this model was trained from scratch.
A comprehensive study of batch construction strategies for recurrent neural networks in MXNet
Authors: Patrick Doetsch, Pavel Golik, Hermann Ney
Date: 2017-05-05 00:00:00
Tasks: Noisy Speech Recognition, speech-recognition, Speech Recognition

In this work we compare different batch construction methods for mini-batch
training of recurrent neural networks. While popular implementations like
TensorFlow and MXNet suggest a bucketing approach to improve the
parallelization capabilities of the recurrent training process, we propose a
simple ordering strategy that arranges the training sequences in a stochastic
alternatingly sorted way. We compare our method to sequence bucketing as well
as various other batch construction strategies on the CHiME-4 noisy speech
recognition corpus. The experiments show that our alternated sorting approach
is able to compete both in training time and recognition performance while
being conceptually simpler to implement.
Max-Pooling Loss Training of Long Short-Term Memory Networks for Small-Footprint Keyword Spotting
Authors: Ming Sun, Anirudh Raju, George Tucker, Sankaran Panchapagesan, Geng-Shen Fu, Arindam Mandal, Spyros Matsoukas, Nikko Strom, Shiv Vitaladevuni
Date: 2017-05-05 00:00:00
Tasks: CPU, Keyword Spotting, Small-Footprint Keyword Spotting

We propose a max-pooling based loss function for training Long Short-Term
Memory (LSTM) networks for small-footprint keyword spotting (KWS), with low
CPU, memory, and latency requirements. The max-pooling loss training can be
further guided by initializing with a cross-entropy loss trained network. A
posterior smoothing based evaluation approach is employed to measure keyword
spotting performance. Our experimental results show that LSTM models trained
using cross-entropy loss or max-pooling loss outperform a cross-entropy loss
trained baseline feed-forward Deep Neural Network (DNN). In addition,
max-pooling loss trained LSTM with randomly initialized network performs better
compared to cross-entropy loss trained LSTM. Finally, the max-pooling loss
trained LSTM initialized with a cross-entropy pre-trained network shows the
best performance, which yields $67.6\%$ relative reduction compared to baseline
feed-forward DNN in Area Under the Curve (AUC) measure.
Deep Recurrent Neural Networks for Supernovae Classification
Authors: Tom Charnock, Adam Moss
Date: 2016-06-23 00:00:00
Tasks: Classification, General Classification

We apply deep recurrent neural networks, which are capable of learning
complex sequential information, to classify supernovae\footnote{Code available
at
\href{https://github.com/adammoss/supernovae}{https://github.com/adammoss/supernovae}}.
The observational time and filter fluxes are used as inputs to the network, but
since the inputs are agnostic additional data such as host galaxy information
can also be included. Using the Supernovae Photometric Classification Challenge
(SPCC) data, we find that deep networks are capable of learning about light
curves, however the performance of the network is highly sensitive to the
amount of training data. For a training size of 50\% of the representational
SPCC dataset (around $10^4$ supernovae) we obtain a type-Ia vs. non-type-Ia
classification accuracy of 94.7\%, an area under the Receiver Operating
Characteristic curve AUC of 0.986 and a SPCC figure-of-merit $F_1=0.64$. When
using only the data for the early-epoch challenge defined by the SPCC we
achieve a classification accuracy of 93.1\%, AUC of 0.977 and $F_1=0.58$,
results almost as good as with the whole light-curve. By employing
bidirectional neural networks we can acquire impressive classification results
between supernovae types -I,~-II and~-III at an accuracy of 90.4\% and AUC of
0.974. We also apply a pre-trained model to obtain classification probabilities
as a function of time, and show it can give early indications of supernovae
type. Our method is competitive with existing algorithms and has applications
for future large-scale photometric surveys.
Building Morphological Chains for Agglutinative Languages
Authors: Serkan Ozen, Burcu Can
Date: 2017-05-05 00:00:00
Tasks: Segmentation

In this paper, we build morphological chains for agglutinative languages by
using a log-linear model for the morphological segmentation task. The model is
based on the unsupervised morphological segmentation system called
MorphoChains. We extend MorphoChains log linear model by expanding the
candidate space recursively to cover more split points for agglutinative
languages such as Turkish, whereas in the original model candidates are
generated by considering only binary segmentation of each word. The results
show that we improve the state-of-art Turkish scores by 12% having a F-measure
of 72% and we improve the English scores by 3% having a F-measure of 74%.
Eventually, the system outperforms both MorphoChains and other well-known
unsupervised morphological segmentation systems. The results indicate that
candidate generation plays an important role in such an unsupervised log-linear
model that is learned using contrastive estimation with negative samples.
A Time-Vertex Signal Processing Framework
Authors: Francesco Grassi, Andreas Loukas, Nathanaël Perraudin, Benjamin Ricaud
Date: 2017-05-05 00:00:00
Tasks: Denoising, Video Inpainting

An emerging way to deal with high-dimensional non-euclidean data is to assume
that the underlying structure can be captured by a graph. Recently, ideas have
begun to emerge related to the analysis of time-varying graph signals. This
work aims to elevate the notion of joint harmonic analysis to a full-fledged
framework denoted as Time-Vertex Signal Processing, that links together the
time-domain signal processing techniques with the new tools of graph signal
processing. This entails three main contributions: (a) We provide a formal
motivation for harmonic time-vertex analysis as an analysis tool for the state
evolution of simple Partial Differential Equations on graphs. (b) We improve
the accuracy of joint filtering operators by up-to two orders of magnitude. (c)
Using our joint filters, we construct time-vertex dictionaries analyzing the
different scales and the local time-frequency content of a signal. The utility
of our tools is illustrated in numerous applications and datasets, such as
dynamic mesh denoising and classification, still-video inpainting, and source
localization in seismic events. Our results suggest that joint analysis of
time-vertex signals can bring benefits to regression and learning.
Deep Speaker: an End-to-End Neural Speaker Embedding System
Authors: Chao Li, Xiaokong Ma, Bing Jiang, Xiangang Li, Xuewei Zhang, Xiao Liu, Ying Cao, Ajay Kannan, Zhenyao Zhu
Date: 2017-05-05 00:00:00
Tasks: Clustering, Speaker Identification, Speaker Recognition, Triplet

We present Deep Speaker, a neural speaker embedding system that maps
utterances to a hypersphere where speaker similarity is measured by cosine
similarity. The embeddings generated by Deep Speaker can be used for many
tasks, including speaker identification, verification, and clustering. We
experiment with ResCNN and GRU architectures to extract the acoustic features,
then mean pool to produce utterance-level speaker embeddings, and train using
triplet loss based on cosine similarity. Experiments on three distinct datasets
suggest that Deep Speaker outperforms a DNN-based i-vector baseline. For
example, Deep Speaker reduces the verification equal error rate by 50%
(relatively) and improves the identification accuracy by 60% (relatively) on a
text-independent dataset. We also present results that suggest adapting from a
model trained with Mandarin can improve accuracy for English speaker
recognition.
Locally Weighted Ensemble Clustering
Authors: Dong Huang, Chang-Dong Wang, Jian-Huang Lai
Date: 2016-05-17 00:00:00
Tasks: Clustering, Diversity

Due to its ability to combine multiple base clusterings into a probably better and more robust clustering, the ensemble clustering technique has been attracting increasing attention in recent years. Despite the significant success, one limitation to most of the existing ensemble clustering methods is that they generally treat all base clusterings equally regardless of their reliability, which makes them vulnerable to low-quality base clusterings. Although some efforts have been made to (globally) evaluate and weight the base clusterings, yet these methods tend to view each base clustering as an individual and neglect the local diversity of clusters inside the same base clustering. It remains an open problem how to evaluate the reliability of clusters and exploit the local diversity in the ensemble to enhance the consensus performance, especially in the case when there is no access to data features or specific assumptions on data distribution. To address this, in this paper, we propose a novel ensemble clustering approach based on ensemble-driven cluster uncertainty estimation and local weighting strategy. In particular, the uncertainty of each cluster is estimated by considering the cluster labels in the entire ensemble via an entropic criterion. A novel ensemble-driven cluster validity measure is introduced, and a locally weighted co-association matrix is presented to serve as a summary for the ensemble of diverse clusters. With the local diversity in ensembles exploited, two novel consensus functions are further proposed. Extensive experiments on a variety of real-world datasets demonstrate the superiority of the proposed approach over the state-of-the-art.
End-to-end Learning of Deep Visual Representations for Image Retrieval
Authors: Albert Gordo, Jon Almazan, Jerome Revaud, Diane Larlus
Date: 2016-10-25 00:00:00
Tasks: Image Retrieval, Quantization, Retrieval, Triplet

While deep learning has become a key ingredient in the top performing methods
for many computer vision tasks, it has failed so far to bring similar
improvements to instance-level image retrieval. In this article, we argue that
reasons for the underwhelming results of deep methods on image retrieval are
threefold: i) noisy training data, ii) inappropriate deep architecture, and
iii) suboptimal training procedure. We address all three issues.
  First, we leverage a large-scale but noisy landmark dataset and develop an
automatic cleaning method that produces a suitable training set for deep
retrieval. Second, we build on the recent R-MAC descriptor, show that it can be
interpreted as a deep and differentiable architecture, and present improvements
to enhance it. Last, we train this network with a siamese architecture that
combines three streams with a triplet loss. At the end of the training process,
the proposed architecture produces a global image representation in a single
forward pass that is well suited for image retrieval. Extensive experiments
show that our approach significantly outperforms previous retrieval approaches,
including state-of-the-art methods based on costly local descriptor indexing
and spatial verification. On Oxford 5k, Paris 6k and Holidays, we respectively
report 94.7, 96.6, and 94.8 mean average precision. Our representations can
also be heavily compressed using product quantization with little loss in
accuracy. For additional material, please see
www.xrce.xerox.com/Deep-Image-Retrieval.
Data Readiness Levels
Authors: Neil D. Lawrence
Date: 2017-05-05 00:00:00
Tasks: Management, Missing Values

Application of models to data is fraught. Data-generating collaborators often
only have a very basic understanding of the complications of collating,
processing and curating data. Challenges include: poor data collection
practices, missing values, inconvenient storage mechanisms, intellectual
property, security and privacy. All these aspects obstruct the sharing and
interconnection of data, and the eventual interpretation of data through
machine learning or other approaches. In project reporting, a major challenge
is in encapsulating these problems and enabling goals to be built around the
processing of data. Project overruns can occur due to failure to account for
the amount of time required to curate and collate. But to understand these
failures we need to have a common language for assessing the readiness of a
particular data set. This position paper proposes the use of data readiness
levels: it gives a rough outline of three stages of data preparedness and
speculates on how formalisation of these levels into a common language for data
readiness could facilitate project management.
Composition of Credal Sets via Polyhedral Geometry
Authors: Jiřina Vejnarová, Václav Kratochvíl
Date: 2017-05-05 00:00:00

Recently introduced composition operator for credal sets is an analogy of
such operators in probability, possibility, evidence and valuation-based
systems theories. It was designed to construct multidimensional models (in the
framework of credal sets) from a system of low- dimensional credal sets. In
this paper we study its potential from the computational point of view
utilizing methods of polyhedral geometry.
Graphical Models: An Extension to Random Graphs, Trees, and Other Objects
Authors: Neil Hallonquist
Date: 2017-04-14 00:00:00

In this work, we consider an extension of graphical models to random graphs,
trees, and other objects. To do this, many fundamental concepts for
multivariate random variables (e.g., marginal variables, Gibbs distribution,
Markov properties) must be extended to other mathematical objects; it turns out
that this extension is possible, as we will discuss, if we have a consistent,
complete system of projections on a given object. Each projection defines a
marginal random variable, allowing one to specify independence assumptions
between them. Furthermore, these independencies can be specified in terms of a
small subset of these marginal variables (which we call the atomic variables),
allowing the compact representation of independencies by a directed graph.
Projections also define factors, functions on the projected object space, and
hence a projection family defines a set of possible factorizations for a
distribution; these can be compactly represented by an undirected graph.
  The invariances used in graphical models are essential for learning
distributions, not just on multivariate random variables, but also on other
objects. When they are applied to random graphs and random trees, the result is
a general class of models that is applicable to a broad range of problems,
including those in which the graphs and trees have complicated edge structures.
These models need not be conditioned on a fixed number of vertices, as is often
the case in the literature for random graphs, and can be used for problems in
which attributes are associated with vertices and edges. For graphs,
applications include the modeling of molecules, neural networks, and relational
real-world scenes; for trees, applications include the modeling of infectious
diseases, cell fusion, the structure of language, and the structure of objects
in visual scenes. Many classic models are particular instances of this
framework.
Group invariance principles for causal generative models
Authors: Michel Besserve, Naji Shajarisales, Bernhard Schölkopf, Dominik Janzing
Date: 2017-05-05 00:00:00
Tasks: BIG-bench Machine Learning, Causal Discovery

The postulate of independence of cause and mechanism (ICM) has recently led
to several new causal discovery algorithms. The interpretation of independence
and the way it is utilized, however, varies across these methods. Our aim in
this paper is to propose a group theoretic framework for ICM to unify and
generalize these approaches. In our setting, the cause-mechanism relationship
is assessed by comparing it against a null hypothesis through the application
of random generic group transformations. We show that the group theoretic view
provides a very general tool to study the structure of data generating
mechanisms with direct applications to machine learning.
SLDR-DL: A Framework for SLD-Resolution with Deep Learning
Authors: Cheng-Hao Cai
Date: 2017-05-05 00:00:00
Tasks: Deep Learning

This paper introduces an SLD-resolution technique based on deep learning.
This technique enables neural networks to learn from old and successful
resolution processes and to use learnt experiences to guide new resolution
processes. An implementation of this technique is named SLDR-DL. It includes a
Prolog library of deep feedforward neural networks and some essential functions
of resolution. In the SLDR-DL framework, users can define logical rules in the
form of definite clauses and teach neural networks to use the rules in
reasoning processes.
SEAGLE: Sparsity-Driven Image Reconstruction under Multiple Scattering
Authors: Hsiou-Yuan Liu, Dehong Liu, Hassan Mansour, Petros T. Boufounos, Laura Waller, Ulugbek S. Kamilov
Date: 2017-05-05 00:00:00
Tasks: Image Reconstruction

Multiple scattering of an electromagnetic wave as it passes through an object
is a fundamental problem that limits the performance of current imaging
systems. In this paper, we describe a new technique-called Series Expansion
with Accelerated Gradient Descent on Lippmann-Schwinger Equation (SEAGLE)-for
robust imaging under multiple scattering based on a combination of a new
nonlinear forward model and a total variation (TV) regularizer. The proposed
forward model can account for multiple scattering, which makes it advantageous
in applications where linear models are inaccurate. Specifically, it
corresponds to a series expansion of the scattered wave with an
accelerated-gradient method. This expansion guarantees the convergence even for
strongly scattering objects. One of our key insights is that it is possible to
obtain an explicit formula for computing the gradient of our nonlinear forward
model with respect to the unknown object, thus enabling fast image
reconstruction with the state-of-the-art fast iterative shrinkage/thresholding
algorithm (FISTA). The proposed method is validated on both simulated and
experimentally measured data.
An analytic comparison of regularization methods for Gaussian Processes
Authors: Hossein Mohammadi, Rodolphe Le Riche, Nicolas Durrande, Eric Touboul, Xavier Bay
Date: 2016-02-02 00:00:00
Tasks: Gaussian Processes, global-optimization

Gaussian Processes (GPs) are a popular approach to predict the output of a
parameterized experiment. They have many applications in the field of Computer
Experiments, in particular to perform sensitivity analysis, adaptive design of
experiments and global optimization. Nearly all of the applications of GPs
require the inversion of a covariance matrix that, in practice, is often
ill-conditioned. Regularization methodologies are then employed with
consequences on the GPs that need to be better understood.The two principal
methods to deal with ill-conditioned covariance matrices are i) pseudoinverse
and ii) adding a positive constant to the diagonal (the so-called nugget
regularization).The first part of this paper provides an algebraic comparison
of PI and nugget regularizations. Redundant points, responsible for covariance
matrix singularity, are defined. It is proven that pseudoinverse
regularization, contrarily to nugget regularization, averages the output values
and makes the variance zero at redundant points. However, pseudoinverse and
nugget regularizations become equivalent as the nugget value vanishes. A
measure for data-model discrepancy is proposed which serves for choosing a
regularization technique.In the second part of the paper, a distribution-wise
GP is introduced that interpolates Gaussian distributions instead of data
points. Distribution-wise GP can be seen as an improved regularization method
for GPs.
Distributed Online Learning of Event Definitions
Authors: Nikos Katzouris, Alexander Artikis, Georgios Paliouras
Date: 2017-05-05 00:00:00
Tasks: Activity Recognition, Form, Inductive logic programming

Logic-based event recognition systems infer occurrences of events in time
using a set of event definitions in the form of first-order rules. The Event
Calculus is a temporal logic that has been used as a basis in event recognition
applications, providing among others, direct connections to machine learning,
via Inductive Logic Programming (ILP). OLED is a recently proposed ILP system
that learns event definitions in the form of Event Calculus theories, in a
single pass over a data stream. In this work we present a version of OLED that
allows for distributed, online learning. We evaluate our approach on a
benchmark activity recognition dataset and show that we can significantly
reduce training times, exchanging minimal information between processing nodes.
Sketching for Large-Scale Learning of Mixture Models
Authors: Nicolas Keriven, Anthony Bourrier, Rémi Gribonval, Patrick Pérez
Date: 2016-06-09 00:00:00
Tasks: Compressive Sensing, Speaker Verification

Learning parameters from voluminous data can be prohibitive in terms of
memory and computational requirements. We propose a "compressive learning"
framework where we estimate model parameters from a sketch of the training
data. This sketch is a collection of generalized moments of the underlying
probability distribution of the data. It can be computed in a single pass on
the training set, and is easily computable on streams or distributed datasets.
The proposed framework shares similarities with compressive sensing, which aims
at drastically reducing the dimension of high-dimensional signals while
preserving the ability to reconstruct them. To perform the estimation task, we
derive an iterative algorithm analogous to sparse reconstruction algorithms in
the context of linear inverse problems. We exemplify our framework with the
compressive estimation of a Gaussian Mixture Model (GMM), providing heuristics
on the choice of the sketching procedure and theoretical guarantees of
reconstruction. We experimentally show on synthetic data that the proposed
algorithm yields results comparable to the classical Expectation-Maximization
(EM) technique while requiring significantly less memory and fewer computations
when the number of database elements is large. We further demonstrate the
potential of the approach on real large-scale data (over 10 8 training samples)
for the task of model-based speaker verification. Finally, we draw some
connections between the proposed framework and approximate Hilbert space
embedding of probability distributions using random features. We show that the
proposed sketching operator can be seen as an innovative method to design
translation-invariant kernels adapted to the analysis of GMMs. We also use this
theoretical framework to derive information preservation guarantees, in the
spirit of infinite-dimensional compressive sensing.