pwc-archive/papers-with-abstracts huggingface dataset:
Machine Learning Approaches to Energy Consumption Forecasting in Households
Authors: Riccardo Bonetto, Michele Rossi
Date: 2017-06-29 00:00:00
Tasks: BIG-bench Machine Learning, Demand Forecasting

We consider the problem of power demand forecasting in residential
micro-grids. Several approaches using ARMA models, support vector machines, and
recurrent neural networks that perform one-step ahead predictions have been
proposed in the literature. Here, we extend them to perform multi-step ahead
forecasting and we compare their performance. Toward this end, we implement a
parallel and efficient training framework, using power demand traces from real
deployments to gauge the accuracy of the considered techniques. Our results
indicate that machine learning schemes achieve smaller prediction errors in the
mean and the variance with respect to ARMA, but there is no clear algorithm of
choice among them. Pros and cons of these approaches are discussed and the
solution of choice is found to depend on the specific use case requirements. A
hybrid approach, that is driven by the prediction interval, the target error,
and its uncertainty, is then recommended.
Weakly-supervised localization of diabetic retinopathy lesions in retinal fundus images
Authors: Waleed M. Gondal, Jan M. Köhler, René Grzeszick, Gernot A. Fink, Michael Hirsch
Date: 2017-06-29 00:00:00
Tasks: General Classification, image-classification, Image Classification

Convolutional neural networks (CNNs) show impressive performance for image
classification and detection, extending heavily to the medical image domain.
Nevertheless, medical experts are sceptical in these predictions as the
nonlinear multilayer structure resulting in a classification outcome is not
directly graspable. Recently, approaches have been shown which help the user to
understand the discriminative regions within an image which are decisive for
the CNN to conclude to a certain class. Although these approaches could help to
build trust in the CNNs predictions, they are only slightly shown to work with
medical image data which often poses a challenge as the decision for a class
relies on different lesion areas scattered around the entire image. Using the
DiaretDB1 dataset, we show that on retina images different lesion areas
fundamental for diabetic retinopathy are detected on an image level with high
accuracy, comparable or exceeding supervised methods. On lesion level, we
achieve few false positives with high sensitivity, though, the network is
solely trained on image-level labels which do not include information about
existing lesions. Classifying between diseased and healthy images, we achieve
an AUC of 0.954 on the DiaretDB1.
Model-Free Trajectory-based Policy Optimization with Monotonic Improvement
Authors: Riad Akrour, Abbas Abdolmaleki, Hany Abdulsamad, Jan Peters, Gerhard Neumann
Date: 2016-06-29 00:00:00

Many of the recent trajectory optimization algorithms alternate between
linear approximation of the system dynamics around the mean trajectory and
conservative policy update. One way of constraining the policy change is by
bounding the Kullback-Leibler (KL) divergence between successive policies.
These approaches already demonstrated great experimental success in challenging
problems such as end-to-end control of physical systems. However, the linear
approximation of the system dynamics can introduce a bias in the policy update
and prevent convergence to the optimal policy. In this article, we propose a
new model-free trajectory-based policy optimization algorithm with guaranteed
monotonic improvement. The algorithm backpropagates a local, quadratic and
time-dependent \qfunc~learned from trajectory data instead of a model of the
system dynamics. Our policy update ensures exact KL-constraint satisfaction
without simplifying assumptions on the system dynamics. We experimentally
demonstrate on highly non-linear control tasks the improvement in performance
of our algorithm in comparison to approaches linearizing the system dynamics.
In order to show the monotonic improvement of our algorithm, we additionally
conduct a theoretical analysis of our policy update scheme to derive a lower
bound of the change in policy return between successive iterations.
A Random Matrix Approach to Neural Networks
Authors: Cosme Louart, Zhenyu Liao, Romain Couillet
Date: 2017-02-17 00:00:00
Tasks: LEMMA

This article studies the Gram random matrix model $G=\frac1T\Sigma^{\rm
T}\Sigma$, $\Sigma=\sigma(WX)$, classically found in the analysis of random
feature maps and random neural networks, where $X=[x_1,\ldots,x_T]\in{\mathbb
R}^{p\times T}$ is a (data) matrix of bounded norm, $W\in{\mathbb R}^{n\times
p}$ is a matrix of independent zero-mean unit variance entries, and
$\sigma:{\mathbb R}\to{\mathbb R}$ is a Lipschitz continuous (activation)
function --- $\sigma(WX)$ being understood entry-wise. By means of a key
concentration of measure lemma arising from non-asymptotic random matrix
arguments, we prove that, as $n,p,T$ grow large at the same rate, the resolvent
$Q=(G+\gamma I_T)^{-1}$, for $\gamma>0$, has a similar behavior as that met in
sample covariance matrix models, involving notably the moment
$\Phi=\frac{T}n{\mathbb E}[G]$, which provides in passing a deterministic
equivalent for the empirical spectral measure of $G$. Application-wise, this
result enables the estimation of the asymptotic performance of single-layer
random neural networks. This in turn provides practical insights into the
underlying mechanisms into play in random neural networks, entailing several
unexpected consequences, as well as a fast practical means to tune the network
hyperparameters.
CS591 Report: Application of siamesa network in 2D transformation
Authors: Dorothy Chang
Date: 2017-06-29 00:00:00
Tasks: Deep Learning, General Classification, Image Registration, Image Retrieval, Retrieval

Deep learning has been extensively used various aspects of computer vision
area. Deep learning separate itself from traditional neural network by having a
much deeper and complicated network layers in its network structures.
Traditionally, deep neural network is abundantly used in computer vision tasks
including classification and detection and has achieve remarkable success and
set up a new state of the art results in these fields. Instead of using neural
network for vision recognition and detection. I will show the ability of neural
network to do image registration, synthesis of images and image retrieval in
this report.
Path Integral Networks: End-to-End Differentiable Optimal Control
Authors: Masashi Okada, Luca Rigazio, Takenobu Aoshima
Date: 2017-06-29 00:00:00
Tasks: continuous-control, Continuous Control, Imitation Learning, Reinforcement Learning, Reinforcement Learning (RL)

In this paper, we introduce Path Integral Networks (PI-Net), a recurrent
network representation of the Path Integral optimal control algorithm. The
network includes both system dynamics and cost models, used for optimal control
based planning. PI-Net is fully differentiable, learning both dynamics and cost
models end-to-end by back-propagation and stochastic gradient descent. Because
of this, PI-Net can learn to plan. PI-Net has several advantages: it can
generalize to unseen states thanks to planning, it can be applied to continuous
control tasks, and it allows for a wide variety learning schemes, including
imitation and reinforcement learning. Preliminary experiment results show that
PI-Net, trained by imitation learning, can mimic control demonstrations for two
simulated problems; a linear system and a pendulum swing-up problem. We also
show that PI-Net is able to learn dynamics and cost models latent in the
demonstrations.
Fast Separable Non-Local Means
Authors: S. Ghosh, K. N. Chaudhury
Date: 2014-07-09 00:00:00
Tasks: Denoising, SSIM

We propose a simple and fast algorithm called PatchLift for computing
distances between patches (contiguous block of samples) extracted from a given
one-dimensional signal. PatchLift is based on the observation that the patch
distances can be efficiently computed from a matrix that is derived from the
one-dimensional signal using lifting; importantly, the number of operations
required to compute the patch distances using this approach does not scale with
the patch length. We next demonstrate how PatchLift can be used for patch-based
denoising of images corrupted with Gaussian noise. In particular, we propose a
separable formulation of the classical Non-Local Means (NLM) algorithm that can
be implemented using PatchLift. We demonstrate that the PatchLift-based
implementation of separable NLM is few orders faster than standard NLM, and is
competitive with existing fast implementations of NLM. Moreover, its denoising
performance is shown to be consistently superior to that of NLM and some of its
variants, both in terms of PSNR/SSIM and visual quality.
Multi-scale Multi-band DenseNets for Audio Source Separation
Authors: Naoya Takahashi, Yuki Mitsufuji
Date: 2017-06-29 00:00:00
Tasks: Audio Source Separation, image-classification, Music Source Separation

This paper deals with the problem of audio source separation. To handle the
complex and ill-posed nature of the problems of audio source separation, the
current state-of-the-art approaches employ deep neural networks to obtain
instrumental spectra from a mixture. In this study, we propose a novel network
architecture that extends the recently developed densely connected
convolutional network (DenseNet), which has shown excellent results on image
classification tasks. To deal with the specific problem of audio source
separation, an up-sampling layer, block skip connection and band-dedicated
dense blocks are incorporated on top of DenseNet. The proposed approach takes
advantage of long contextual information and outperforms state-of-the-art
results on SiSEC 2016 competition by a large margin in terms of
signal-to-distortion ratio. Moreover, the proposed architecture requires
significantly fewer parameters and considerably less training time compared
with other methods.
Online Reweighted Least Squares Algorithm for Sparse Recovery and Application to Short-Wave Infrared Imaging
Authors: Subhadip Mukherjee, Deepak R., Huaijin Chen, Ashok Veeraraghavan, Chandra Sekhar Seelamantula
Date: 2017-06-29 00:00:00

We address the problem of sparse recovery in an online setting, where random
linear measurements of a sparse signal are revealed sequentially and the
objective is to recover the underlying signal. We propose a reweighted least
squares (RLS) algorithm to solve the problem of online sparse reconstruction,
wherein a system of linear equations is solved using conjugate gradient with
the arrival of every new measurement. The proposed online algorithm is useful
in a setting where one seeks to design a progressive decoding strategy to
reconstruct a sparse signal from linear measurements so that one does not have
to wait until all measurements are acquired. Moreover, the proposed algorithm
is also useful in applications where it is infeasible to process all the
measurements using a batch algorithm, owing to computational and storage
constraints. It is not needed a priori to collect a fixed number of
measurements; rather one can keep collecting measurements until the quality of
reconstruction is satisfactory and stop taking further measurements once the
reconstruction is sufficiently accurate. We provide a proof-of-concept by
comparing the performance of our algorithm with the RLS-based batch
reconstruction strategy, known as iteratively reweighted least squares (IRLS),
on natural images. Experiments on a recently proposed focal plane array-based
imaging setup show up to 1 dB improvement in output peak signal-to-noise ratio
as compared with the total variation-based reconstruction.
Nonbacktracking Bounds on the Influence in Independent Cascade Models
Authors: Emmanuel Abbe, Sanjeev Kulkarni, Eun Jee Lee
Date: 2017-05-24 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Community Detection

This paper develops upper and lower bounds on the influence measure in a
network, more precisely, the expected number of nodes that a seed set can
influence in the independent cascade model. In particular, our bounds exploit
nonbacktracking walks, Fortuin-Kasteleyn-Ginibre (FKG) type inequalities, and
are computed by message passing implementation. Nonbacktracking walks have
recently allowed for headways in community detection, and this paper shows that
their use can also impact the influence computation. Further, we provide a knob
to control the trade-off between the efficiency and the accuracy of the bounds.
Finally, the tightness of the bounds is illustrated with simulations on various
network models.
Recurrent neural networks with specialized word embeddings for health-domain named-entity recognition
Authors: Inigo Jauregi Unanue, Ehsan Zare Borzeshi, Massimo Piccardi
Date: 2017-06-29 00:00:00
Tasks: BIG-bench Machine Learning, Clinical Concept Extraction, Feature Engineering, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Word Embeddings

Background. Previous state-of-the-art systems on Drug Name Recognition (DNR)
and Clinical Concept Extraction (CCE) have focused on a combination of text
"feature engineering" and conventional machine learning algorithms such as
conditional random fields and support vector machines. However, developing good
features is inherently heavily time-consuming. Conversely, more modern machine
learning approaches such as recurrent neural networks (RNNs) have proved
capable of automatically learning effective features from either random
assignments or automated word "embeddings". Objectives. (i) To create a highly
accurate DNR and CCE system that avoids conventional, time-consuming feature
engineering. (ii) To create richer, more specialized word embeddings by using
health domain datasets such as MIMIC-III. (iii) To evaluate our systems over
three contemporary datasets. Methods. Two deep learning methods, namely the
Bidirectional LSTM and the Bidirectional LSTM-CRF, are evaluated. A CRF model
is set as the baseline to compare the deep learning systems to a traditional
machine learning approach. The same features are used for all the models.
Results. We have obtained the best results with the Bidirectional LSTM-CRF
model, which has outperformed all previously proposed systems. The specialized
embeddings have helped to cover unusual words in DDI-DrugBank and DDI-MedLine,
but not in the 2010 i2b2/VA IRB Revision dataset. Conclusion. We present a
state-of-the-art system for DNR and CCE. Automated word embeddings has allowed
us to avoid costly feature engineering and achieve higher accuracy.
Nevertheless, the embeddings need to be retrained over datasets that are
adequate for the domain, in order to adequately cover the domain-specific
vocabulary.
Frame-Based Continuous Lexical Semantics through Exponential Family Tensor Factorization and Semantic Proto-Roles
Authors: Francis Ferraro, Adam Poliak, Ryan Cotterell, Benjamin Van Durme
Date: 2017-06-29 00:00:00
Proceeding: SEMEVAL 2017 8

We study how different frame annotations complement one another when learning
continuous lexical semantics. We learn the representations from a tensorized
skip-gram model that consistently encodes syntactic-semantic content better,
with multiple 10% gains over baselines.
Audio Spectrogram Representations for Processing with Convolutional Neural Networks
Authors: L. Wyse
Date: 2017-06-29 00:00:00
Tasks: Style Transfer

One of the decisions that arise when designing a neural network for any
application is how the data should be represented in order to be presented to,
and possibly generated by, a neural network. For audio, the choice is less
obvious than it seems to be for visual images, and a variety of representations
have been used for different applications including the raw digitized sample
stream, hand-crafted features, machine discovered features, MFCCs and variants
that include deltas, and a variety of spectral representations. This paper
reviews some of these representations and issues that arise, focusing
particularly on spectrograms for generating audio using neural networks for
style transfer.
Talking Drums: Generating drum grooves with neural networks
Authors: P. Hutchings
Date: 2017-06-29 00:00:00
Tasks: Translation

Presented is a method of generating a full drum kit part for a provided
kick-drum sequence. A sequence to sequence neural network model used in natural
language translation was adopted to encode multiple musical styles and an
online survey was developed to test different techniques for sampling the
output of the softmax function. The strongest results were found using a
sampling technique that drew from the three most probable outputs at each
subdivision of the drum pattern but the consistency of output was found to be
heavily dependent on style.
Machine listening intelligence
Authors: C. E. Cella
Date: 2017-06-29 00:00:00
Tasks: Deep Learning

This manifesto paper will introduce machine listening intelligence, an
integrated research framework for acoustic and musical signals modelling, based
on signal processing, deep learning and computational musicology.
Vision-based Detection of Acoustic Timed Events: a Case Study on Clarinet Note Onsets
Authors: A. Bazzica, J. C. van Gemert, C. C. S. Liem, A. Hanjalic
Date: 2017-06-29 00:00:00
Tasks: Binary Classification, Onset Detection

Acoustic events often have a visual counterpart. Knowledge of visual
information can aid the understanding of complex auditory scenes, even when
only a stereo mixdown is available in the audio domain, \eg identifying which
musicians are playing in large musical ensembles. In this paper, we consider a
vision-based approach to note onset detection. As a case study we focus on
challenging, real-world clarinetist videos and carry out preliminary
experiments on a 3D convolutional neural network based on multiple streams and
purposely avoiding temporal pooling. We release an audiovisual dataset with 4.5
hours of clarinetist videos together with cleaned annotations which include
about 36,000 onsets and the coordinates for a number of salient points and
regions of interest. By performing several training trials on our dataset, we
learned that the problem is challenging. We found that the CNN model is highly
sensitive to the optimization algorithm and hyper-parameters, and that treating
the problem as binary classification may prevent the joint optimization of
precision and recall. To encourage further research, we publicly share our
dataset, annotations and all models and detail which issues we came across
during our preliminary experiments.
Music Signal Processing Using Vector Product Neural Networks
Authors: Z. C. Fan, T. S. Chan, Y. H. Yang, J. S. R. Jang
Date: 2017-06-29 00:00:00

We propose a novel neural network model for music signal processing using
vector product neurons and dimensionality transformations. Here, the inputs are
first mapped from real values into three-dimensional vectors then fed into a
three-dimensional vector product neural network where the inputs, outputs, and
weights are all three-dimensional values. Next, the final outputs are mapped
back to the reals. Two methods for dimensionality transformation are proposed,
one via context windows and the other via spectral coloring. Experimental
results on the iKala dataset for blind singing voice separation confirm the
efficacy of our model.
Transforming Musical Signals through a Genre Classifying Convolutional Neural Network
Authors: S. Geng, G. Ren, M. Ogihara
Date: 2017-06-29 00:00:00
Tasks: Decision Making, Genre classification

Convolutional neural networks (CNNs) have been successfully applied on both
discriminative and generative modeling for music-related tasks. For a
particular task, the trained CNN contains information representing the decision
making or the abstracting process. One can hope to manipulate existing music
based on this 'informed' network and create music with new features
corresponding to the knowledge obtained by the network. In this paper, we
propose a method to utilize the stored information from a CNN trained on
musical genre classification task. The network was composed of three
convolutional layers, and was trained to classify five-second song clips into
five different genres. After training, randomly selected clips were modified by
maximizing the sum of outputs from the network layers. In addition to the
potential of such CNNs to produce interesting audio transformation, more
information about the network and the original music could be obtained from the
analysis of the generated features since these features indicate how the
network 'understands' the music.
Chord Label Personalization through Deep Learning of Integrated Harmonic Interval-based Representations
Authors: H. V. Koops, W. B. de Haas, J. Bransen, A. Volk
Date: 2017-06-29 00:00:00

The increasing accuracy of automatic chord estimation systems, the
availability of vast amounts of heterogeneous reference annotations, and
insights from annotator subjectivity research make chord label personalization
increasingly important. Nevertheless, automatic chord estimation systems are
historically exclusively trained and evaluated on a single reference
annotation. We introduce a first approach to automatic chord label
personalization by modeling subjectivity through deep learning of a harmonic
interval-based chord label representation. After integrating these
representations from multiple annotators, we can accurately personalize chord
labels for individual annotators from a single model and the annotators' chord
label vocabulary. Furthermore, we show that chord personalization using
multiple reference annotations outperforms using a single reference annotation.
Toward Inverse Control of Physics-Based Sound Synthesis
Authors: A. Pfalz, E. Berdahl
Date: 2017-06-29 00:00:00

Long Short-Term Memory networks (LSTMs) can be trained to realize inverse
control of physics-based sound synthesizers. Physics-based sound synthesizers
simulate the laws of physics to produce output sound according to input gesture
signals. When a user's gestures are measured in real time, she or he can use
them to control physics-based sound synthesizers, thereby creating simulated
virtual instruments. An intriguing question is how to program a computer to
learn to play such physics-based models. This work demonstrates that LSTMs can
be trained to accomplish this inverse control task with four physics-based
sound synthesizers.
Modeling Musical Context with Word2vec
Authors: Dorien Herremans, Ching-Hua Chuan
Date: 2017-06-28 00:00:00

We present a semantic vector space model for capturing complex polyphonic
musical context. A word2vec model based on a skip-gram representation with
negative sampling was used to model slices of music from a dataset of
Beethoven's piano sonatas. A visualization of the reduced vector space using
t-distributed stochastic neighbor embedding shows that the resulting embedded
vector space captures tonal relationships, even without any explicit
information about the musical contents of the slices. Secondly, an excerpt of
the Moonlight Sonata from Beethoven was altered by replacing slices based on
context similarity. The resulting music shows that the selected slice based on
similar word2vec context also has a relatively short tonal distance from the
original slice.