pwc-archive/papers-with-abstracts huggingface dataset:
Scalable Training of Artificial Neural Networks with Adaptive Sparse Connectivity inspired by Network Science
Authors: Decebal Constantin Mocanu, Elena Mocanu, Peter Stone, Phuong H. Nguyen, Madeleine Gibescu, Antonio Liotta
Date: 2017-07-15 00:00:00
Tasks: Sparse Learning

Through the success of deep learning in various domains, artificial neural
networks are currently among the most used artificial intelligence methods.
Taking inspiration from the network properties of biological neural networks
(e.g. sparsity, scale-freeness), we argue that (contrary to general practice)
artificial neural networks, too, should not have fully-connected layers. Here
we propose sparse evolutionary training of artificial neural networks, an
algorithm which evolves an initial sparse topology (Erd\H{o}s-R\'enyi random
graph) of two consecutive layers of neurons into a scale-free topology, during
learning. Our method replaces artificial neural networks fully-connected layers
with sparse ones before training, reducing quadratically the number of
parameters, with no decrease in accuracy. We demonstrate our claims on
restricted Boltzmann machines, multi-layer perceptrons, and convolutional
neural networks for unsupervised and supervised learning on 15 datasets. Our
approach has the potential to enable artificial neural networks to scale up
beyond what is currently possible.
Topology Estimation in Bulk Power Grids: Guarantees on Exact Recovery
Authors: Deepjyoti Deka, Saurav Talukdar, Michael Chertkov, Murti Salapaka
Date: 2017-07-05 00:00:00

The topology of a power grid affects its dynamic operation and settlement in
the electricity market. Real-time topology identification can enable faster
control action following an emergency scenario like failure of a line. This
article discusses a graphical model framework for topology estimation in bulk
power grids (both loopy transmission and radial distribution) using
measurements of voltage collected from the grid nodes. The graphical model for
the probability distribution of nodal voltages in linear power flow models is
shown to include additional edges along with the operational edges in the true
grid. Our proposed estimation algorithms first learn the graphical model and
subsequently extract the operational edges using either thresholding or a
neighborhood counting scheme. For grid topologies containing no three-node
cycles (two buses do not share a common neighbor), we prove that an exact
extraction of the operational topology is theoretically guaranteed. This
includes a majority of distribution grids that have radial topologies. For
grids that include cycles of length three, we provide sufficient conditions
that ensure existence of algorithms for exact reconstruction. In particular,
for grids with constant impedance per unit length and uniform injection
covariances, this observation leads to conditions on geographical placement of
the buses. The performance of algorithms is demonstrated in test case
simulations.
Original Loop-closure Detection Algorithm for Monocular vSLAM
Authors: Andrey Bokovoy, Konstantin Yakovlev
Date: 2017-07-15 00:00:00
Tasks: Loop Closure Detection, Simultaneous Localization and Mapping

Vision-based simultaneous localization and mapping (vSLAM) is a
well-established problem in mobile robotics and monocular vSLAM is one of the
most challenging variations of that problem nowadays. In this work we study one
of the core post-processing optimization mechanisms in vSLAM, e.g. loop-closure
detection. We analyze the existing methods and propose original algorithm for
loop-closure detection, which is suitable for dense, semi-dense and
feature-based vSLAM methods. We evaluate the algorithm experimentally and show
that it contribute to more accurate mapping while speeding up the monocular
vSLAM pipeline to the extent the latter can be used in real-time for
controlling small multi-rotor vehicle (drone).
Not All Dialogues are Created Equal: Instance Weighting for Neural Conversational Models
Authors: Pierre Lison, Serge Bibauw
Date: 2017-04-28 00:00:00
Proceeding: WS 2017 8
Tasks: All, parameter estimation, Retrieval

Neural conversational models require substantial amounts of dialogue data for
their parameter estimation and are therefore usually learned on large corpora
such as chat forums or movie subtitles. These corpora are, however, often
challenging to work with, notably due to their frequent lack of turn
segmentation and the presence of multiple references external to the dialogue
itself. This paper shows that these challenges can be mitigated by adding a
weighting model into the architecture. The weighting model, which is itself
estimated from dialogue data, associates each training example to a numerical
weight that reflects its intrinsic quality for dialogue modelling. At training
time, these sample weights are included into the empirical loss to be
minimised. Evaluation results on retrieval-based models trained on movie and TV
subtitles demonstrate that the inclusion of such a weighting model improves the
model performance on unsupervised metrics.
The k-means-u* algorithm: non-local jumps and greedy retries improve k-means++ clustering
Authors: Bernd Fritzke
Date: 2017-06-27 00:00:00
Tasks: Clustering

We present a new clustering algorithm called k-means-u* which in many cases
is able to significantly improve the clusterings found by k-means++, the
current de-facto standard for clustering in Euclidean spaces. First we
introduce the k-means-u algorithm which starts from a result of k-means++ and
attempts to improve it with a sequence of non-local "jumps" alternated by runs
of standard k-means. Each jump transfers the "least useful" center towards the
center with the largest local error, offset by a small random vector. This is
continued as long as the error decreases and often leads to an improved
solution. Occasionally k-means-u terminates despite obvious remaining
optimization possibilities. By allowing a limited number of retries for the
last jump it is frequently possible to reach better local minima. The resulting
algorithm is called k-means-u* and dominates k-means++ wrt. solution quality
which is demonstrated empirically using various data sets. By construction the
logarithmic quality bound established for k-means++ holds for k-means-u* as
well.
Quantum Computation via Sparse Distributed Representation
Authors: Gerard J. Rinkus
Date: 2017-07-15 00:00:00

Quantum superposition says that any physical system simultaneously exists in
all of its possible states, the number of which is exponential in the number of
entities composing the system. The strength of presence of each possible state
in the superposition, i.e., its probability of being observed, is represented
by its probability amplitude coefficient. The assumption that these
coefficients must be represented physically disjointly from each other, i.e.,
localistically, is nearly universal in the quantum theory/computing literature.
Alternatively, these coefficients can be represented using sparse distributed
representations (SDR), wherein each coefficient is represented by small subset
of an overall population of units, and the subsets can overlap. Specifically, I
consider an SDR model in which the overall population consists of Q WTA
clusters, each with K binary units. Each coefficient is represented by a set of
Q units, one per cluster. Thus, K^Q coefficients can be represented with KQ
units. Thus, the particular world state, X, whose coefficient's representation,
R(X), is the set of Q units active at time t has the max probability and the
probability of every other state, Y_i, at time t, is measured by R(Y_i)'s
intersection with R(X). Thus, R(X) simultaneously represents both the
particular state, X, and the probability distribution over all states. Thus,
set intersection may be used to classically implement quantum superposition. If
algorithms exist for which the time it takes to store (learn) new
representations and to find the closest-matching stored representation
(probabilistic inference) remains constant as additional representations are
stored, this meets the criterion of quantum computing. Such an algorithm has
already been described: it achieves this "quantum speed-up" without esoteric
hardware, and in fact, on a single-processor, classical (Von Neumann) computer.
Time for a change: a tutorial for comparing multiple classifiers through Bayesian analysis
Authors: Alessio Benavoli, Giorgio Corani, Janez Demsar, Marco Zaffalon
Date: 2016-06-14 00:00:00
Tasks: BIG-bench Machine Learning

The machine learning community adopted the use of null hypothesis
significance testing (NHST) in order to ensure the statistical validity of
results. Many scientific fields however realized the shortcomings of
frequentist reasoning and in the most radical cases even banned its use in
publications. We should do the same: just as we have embraced the Bayesian
paradigm in the development of new machine learning methods, so we should also
use it in the analysis of our own results. We argue for abandonment of NHST by
exposing its fallacies and, more importantly, offer better - more sound and
useful - alternatives for it.
Ensembling Factored Neural Machine Translation Models for Automatic Post-Editing and Quality Estimation
Authors: Chris Hokamp
Date: 2017-06-15 00:00:00
Proceeding: WS 2017 9
Tasks: Automatic Post-Editing, Machine Translation, NMT, Translation

This work presents a novel approach to Automatic Post-Editing (APE) and
Word-Level Quality Estimation (QE) using ensembles of specialized Neural
Machine Translation (NMT) systems. Word-level features that have proven
effective for QE are included as input factors, expanding the representation of
the original source and the machine translation hypothesis, which are used to
generate an automatically post-edited hypothesis. We train a suite of NMT
models that use different input representations, but share the same output
space. These models are then ensembled together, and tuned for both the APE and
the QE task. We thus attempt to connect the state-of-the-art approaches to APE
and QE within a single framework. Our models achieve state-of-the-art results
in both tasks, with the only difference in the tuning step which learns weights
for each component of the ensemble.
Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the LUNA16 challenge
Authors: Arnaud Arindra Adiyoso Setio, Alberto Traverso, Thomas de Bel, Moira S. N. Berens, Cas van den Bogaard, Piergiorgio Cerello, Hao Chen, Qi Dou, Maria Evelina Fantacci, Bram Geurts, Robbert van der Gugten, Pheng Ann Heng, Bart Jansen, Michael M. J. de Kaste, Valentin Kotov, Jack Yu-Hung Lin, Jeroen T. M. C. Manders, Alexander Sónora-Mengana, Juan Carlos García-Naranjo, Evgenia Papavasileiou, Mathias Prokop, Marco Saletta, Cornelia M Schaefer-Prokop, Ernst T. Scholten, Luuk Scholten, Miranda M. Snoeren, Ernesto Lopez Torres, Jef Vandemeulebroucke, Nicole Walasek, Guido C. A. Zuidhof, Bram van Ginneken, Colin Jacobs
Date: 2016-12-23 00:00:00
Tasks: Computed Tomography (CT)

Automatic detection of pulmonary nodules in thoracic computed tomography (CT)
scans has been an active area of research for the last two decades. However,
there have only been few studies that provide a comparative performance
evaluation of different systems on a common database. We have therefore set up
the LUNA16 challenge, an objective evaluation framework for automatic nodule
detection algorithms using the largest publicly available reference database of
chest CT scans, the LIDC-IDRI data set. In LUNA16, participants develop their
algorithm and upload their predictions on 888 CT scans in one of the two
tracks: 1) the complete nodule detection track where a complete CAD system
should be developed, or 2) the false positive reduction track where a provided
set of nodule candidates should be classified. This paper describes the setup
of LUNA16 and presents the results of the challenge so far. Moreover, the
impact of combining individual systems on the detection performance was also
investigated. It was observed that the leading solutions employed convolutional
networks and used the provided set of nodule candidates. The combination of
these solutions achieved an excellent sensitivity of over 95% at fewer than 1.0
false positives per scan. This highlights the potential of combining algorithms
to improve the detection performance. Our observer study with four expert
readers has shown that the best system detects nodules that were missed by
expert readers who originally annotated the LIDC-IDRI data. We released this
set of additional nodules for further development of CAD systems.
Memoisation: Purely, Left-recursively, and with (Continuation Passing) Style
Authors: Samer Abdallah
Date: 2017-07-15 00:00:00

Memoisation, or tabling, is a well-known technique that yields large
improvements in the performance of some recursive computations. Tabled
resolution in Prologs such as XSB and B-Prolog can transform so called
left-recursive predicates from non-terminating computations into finite and
well-behaved ones. In the functional programming literature, memoisation has
usually been implemented in a way that does not handle left-recursion,
requiring supplementary mechanisms to prevent non-termination. A notable
exception is Johnson's (1995) continuation passing approach in Scheme. This,
however, relies on mutation of a memo table data structure and coding in
explicit continuation passing style. We show how Johnson's approach can be
implemented purely functionally in a modern, strongly typed functional language
(OCaml), presented via a monadic interface that hides the implementation
details, yet providing a way to return a compact represention of the memo
tables at the end of the computation.
Faster Kernel Ridge Regression Using Sketching and Preconditioning
Authors: Haim Avron, Kenneth L. Clarkson, David P. Woodruff
Date: 2016-11-10 00:00:00
Tasks: regression

Kernel Ridge Regression (KRR) is a simple yet powerful technique for
non-parametric regression whose computation amounts to solving a linear system.
This system is usually dense and highly ill-conditioned. In addition, the
dimensions of the matrix are the same as the number of data points, so direct
methods are unrealistic for large-scale datasets. In this paper, we propose a
preconditioning technique for accelerating the solution of the aforementioned
linear system. The preconditioner is based on random feature maps, such as
random Fourier features, which have recently emerged as a powerful technique
for speeding up and scaling the training of kernel-based methods, such as
kernel ridge regression, by resorting to approximations. However, random
feature maps only provide crude approximations to the kernel function, so
delivering state-of-the-art results by directly solving the approximated system
requires the number of random features to be very large. We show that random
feature maps can be much more effective in forming preconditioners, since under
certain conditions a not-too-large number of random features is sufficient to
yield an effective preconditioner. We empirically evaluate our method and show
it is highly effective for datasets of up to one million training examples.
Binarized Convolutional Neural Networks with Separable Filters for Efficient Hardware Acceleration
Authors: Jeng-Hau Lin, Tianwei Xing, Ritchie Zhao, Zhiru Zhang, Mani Srivastava, Zhuowen Tu, Rajesh K. Gupta
Date: 2017-07-15 00:00:00

State-of-the-art convolutional neural networks are enormously costly in both
compute and memory, demanding massively parallel GPUs for execution. Such
networks strain the computational capabilities and energy available to embedded
and mobile processing platforms, restricting their use in many important
applications. In this paper, we push the boundaries of hardware-effective CNN
design by proposing BCNN with Separable Filters (BCNNw/SF), which applies
Singular Value Decomposition (SVD) on BCNN kernels to further reduce
computational and storage complexity. To enable its implementation, we provide
a closed form of the gradient over SVD to calculate the exact gradient with
respect to every binarized weight in backward propagation. We verify BCNNw/SF
on the MNIST, CIFAR-10, and SVHN datasets, and implement an accelerator for
CIFAR-10 on FPGA hardware. Our BCNNw/SF accelerator realizes memory savings of
17% and execution time reduction of 31.3% compared to BCNN with only minor
accuracy sacrifices.
On the Performance of Forecasting Models in the Presence of Input Uncertainty
Authors: Hossein Sangrody, Morteza Sarailoo, Ning Zhou, Ahmad Shokrollahi, Elham Foruzan
Date: 2017-07-15 00:00:00

Nowadays, with the unprecedented penetration of renewable distributed energy
resources (DERs), the necessity of an efficient energy forecasting model is
more demanding than before. Generally, forecasting models are trained using
observed weather data while the trained models are applied for energy
forecasting using forecasted weather data. In this study, the performance of
several commonly used forecasting methods in the presence of weather predictors
with uncertainty is assessed and compared. Accordingly, both observed and
forecasted weather data are collected, then the influential predictors for
solar PV generation forecasting model are selected using several measures.
Using observed and forecasted weather data, an analysis on the uncertainty of
weather variables is represented by MAE and bootstrapping. The energy
forecasting model is trained using observed weather data, and finally, the
performance of several commonly used forecasting methods in solar energy
forecasting is simulated and compared for a real case study.
End-to-End Learning for Structured Prediction Energy Networks
Authors: David Belanger, Bishan Yang, Andrew McCallum
Date: 2017-03-16 00:00:00
Proceeding: ICML 2017 8
Tasks: Denoising, Image Denoising, Prediction, Semantic Role Labeling, Structured Prediction

Structured Prediction Energy Networks (SPENs) are a simple, yet expressive
family of structured prediction models (Belanger and McCallum, 2016). An energy
function over candidate structured outputs is given by a deep network, and
predictions are formed by gradient-based optimization. This paper presents
end-to-end learning for SPENs, where the energy function is discriminatively
trained by back-propagating through gradient-based prediction. In our
experience, the approach is substantially more accurate than the structured SVM
method of Belanger and McCallum (2016), as it allows us to use more
sophisticated non-convex energies. We provide a collection of techniques for
improving the speed, accuracy, and memory requirements of end-to-end SPENs, and
demonstrate the power of our method on 7-Scenes image denoising and CoNLL-2005
semantic role labeling tasks. In both, inexact minimization of non-convex SPEN
energies is superior to baseline methods that use simplistic energy functions
that can be minimized exactly.
Dominant Sets for "Constrained" Image Segmentation
Authors: Eyasu Zemene, Leulseged Tesfaye Alemu, Marcello Pelillo
Date: 2017-07-15 00:00:00
Tasks: Image Segmentation, Interactive Segmentation, Segmentation, Semantic Segmentation

Image segmentation has come a long way since the early days of computer
vision, and still remains a challenging task. Modern variations of the
classical (purely bottom-up) approach, involve, e.g., some form of user
assistance (interactive segmentation) or ask for the simultaneous segmentation
of two or more images (co-segmentation). At an abstract level, all these
variants can be thought of as "constrained" versions of the original
formulation, whereby the segmentation process is guided by some external source
of information. In this paper, we propose a new approach to tackle this kind of
problems in a unified way. Our work is based on some properties of a family of
quadratic optimization problems related to dominant sets, a well-known
graph-theoretic notion of a cluster which generalizes the concept of a maximal
clique to edge-weighted graphs. In particular, we show that by properly
controlling a regularization parameter which determines the structure and the
scale of the underlying problem, we are in a position to extract groups of
dominant-set clusters that are constrained to contain predefined elements. In
particular, we shall focus on interactive segmentation and co-segmentation (in
both the unsupervised and the interactive versions). The proposed algorithm can
deal naturally with several type of constraints and input modality, including
scribbles, sloppy contours, and bounding boxes, and is able to robustly handle
noisy annotations on the part of the user. Experiments on standard benchmark
datasets show the effectiveness of our approach as compared to state-of-the-art
algorithms on a variety of natural images under several input conditions and
constraints.
Lyrics-Based Music Genre Classification Using a Hierarchical Attention Network
Authors: Alexandros Tsaptsinos
Date: 2017-07-15 00:00:00
Tasks: Form, General Classification, Genre classification, Information Retrieval, Music Genre Classification, Music Information Retrieval, Retrieval

Music genre classification, especially using lyrics alone, remains a
challenging topic in Music Information Retrieval. In this study we apply
recurrent neural network models to classify a large dataset of intact song
lyrics. As lyrics exhibit a hierarchical layer structure - in which words
combine to form lines, lines form segments, and segments form a complete song -
we adapt a hierarchical attention network (HAN) to exploit these layers and in
addition learn the importance of the words, lines, and segments. We test the
model over a 117-genre dataset and a reduced 20-genre dataset. Experimental
results show that the HAN outperforms both non-neural models and simpler neural
models, whilst also classifying over a higher number of genres than previous
research. Through the learning process we can also visualise which words or
lines in a song the model believes are important to classifying the genre. As a
result the HAN provides insights, from a computational perspective, into
lyrical structure and language features that differentiate musical genres.
Knowledge-Guided Recurrent Neural Network Learning for Task-Oriented Action Prediction
Authors: Liang Lin, Lili Huang, Tianshui Chen, Yukang Gan, Hui Cheng
Date: 2017-07-15 00:00:00
Tasks: Common Sense Reasoning, valid

This paper aims at task-oriented action prediction, i.e., predicting a
sequence of actions towards accomplishing a specific task under a certain
scene, which is a new problem in computer vision research. The main challenges
lie in how to model task-specific knowledge and integrate it in the learning
procedure. In this work, we propose to train a recurrent long-short term memory
(LSTM) network for handling this problem, i.e., taking a scene image (including
pre-located objects) and the specified task as input and recurrently predicting
action sequences. However, training such a network usually requires large
amounts of annotated samples for covering the semantic space (e.g., diverse
action decomposition and ordering). To alleviate this issue, we introduce a
temporal And-Or graph (AOG) for task description, which hierarchically
represents a task into atomic actions. With this AOG representation, we can
produce many valid samples (i.e., action sequences according with common sense)
by training another auxiliary LSTM network with a small set of annotated
samples. And these generated samples (i.e., task-oriented action sequences)
effectively facilitate training the model for task-oriented action prediction.
In the experiments, we create a new dataset containing diverse daily tasks and
extensively evaluate the effectiveness of our approach.