pwc-archive/papers-with-abstracts huggingface dataset:
Using Big Data to Enhance the Bosch Production Line Performance: A Kaggle Challenge
Authors: Ankita Mangal, Nishant Kumar
Date: 2016-12-29 00:00:00

This paper describes our approach to the Bosch production line performance
challenge run by Kaggle.com. Maximizing the production yield is at the heart of
the manufacturing industry. At the Bosch assembly line, data is recorded for
products as they progress through each stage. Data science methods are applied
to this huge data repository consisting records of tests and measurements made
for each component along the assembly line to predict internal failures. We
found that it is possible to train a model that predicts which parts are most
likely to fail. Thus a smarter failure detection system can be built and the
parts tagged likely to fail can be salvaged to decrease operating costs and
increase the profit margins.
Motifs in Temporal Networks
Authors: Ashwin Paranjape, Austin R. Benson, Jure Leskovec
Date: 2016-12-29 00:00:00

Networks are a fundamental tool for modeling complex systems in a variety of
domains including social and communication networks as well as biology and
neuroscience. Small subgraph patterns in networks, called network motifs, are
crucial to understanding the structure and function of these systems. However,
the role of network motifs in temporal networks, which contain many timestamped
links between the nodes, is not yet well understood.
  Here we develop a notion of a temporal network motif as an elementary unit of
temporal networks and provide a general methodology for counting such motifs.
We define temporal network motifs as induced subgraphs on sequences of temporal
edges, design fast algorithms for counting temporal motifs, and prove their
runtime complexity. Our fast algorithms achieve up to 56.5x speedup compared to
a baseline method. Furthermore, we use our algorithms to count temporal motifs
in a variety of networks. Results show that networks from different domains
have significantly different motif counts, whereas networks from the same
domain tend to have similar motif counts. We also find that different motifs
occur at different time scales, which provides further insights into structure
and function of temporal networks.
Lifted Relational Algebra with Recursion and Connections to Modal Logic
Authors: Eugenia Ternovska
Date: 2016-12-29 00:00:00
Tasks: Negation

We propose a new formalism for specifying and reasoning about problems that
involve heterogeneous "pieces of information" -- large collections of data,
decision procedures of any kind and complexity and connections between them.
The essence of our proposal is to lift Codd's relational algebra from
operations on relational tables to operations on classes of structures (with
recursion), and to add a direction of information propagation. We observe the
presence of information propagation in several formalisms for efficient
reasoning and use it to express unary negation and operations used in graph
databases. We carefully analyze several reasoning tasks and establish a precise
connection between a generalized query evaluation and temporal logic model
checking. Our development allows us to reveal a general correspondence between
classical and modal logics and may shed a new light on the good computational
properties of modal logics and related formalisms.
SSD: Single Shot MultiBox Detector
Authors: Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, Alexander C. Berg
Date: 2015-12-08 00:00:00
Tasks: LIDAR Semantic Segmentation, Low-Light Image Enhancement, Node Property Prediction, Object, Object Detection, Surgical tool detection, Visual Object Tracking

We present a method for detecting objects in images using a single deep
neural network. Our approach, named SSD, discretizes the output space of
bounding boxes into a set of default boxes over different aspect ratios and
scales per feature map location. At prediction time, the network generates
scores for the presence of each object category in each default box and
produces adjustments to the box to better match the object shape. Additionally,
the network combines predictions from multiple feature maps with different
resolutions to naturally handle objects of various sizes. Our SSD model is
simple relative to methods that require object proposals because it completely
eliminates proposal generation and subsequent pixel or feature resampling stage
and encapsulates all computation in a single network. This makes SSD easy to
train and straightforward to integrate into systems that require a detection
component. Experimental results on the PASCAL VOC, MS COCO, and ILSVRC datasets
confirm that SSD has comparable accuracy to methods that utilize an additional
object proposal step and is much faster, while providing a unified framework
for both training and inference. Compared to other single stage methods, SSD
has much better accuracy, even with a smaller input image size. For $300\times
300$ input, SSD achieves 72.1% mAP on VOC2007 test at 58 FPS on a Nvidia Titan
X and for $500\times 500$ input, SSD achieves 75.1% mAP, outperforming a
comparable state of the art Faster R-CNN model. Code is available at
https://github.com/weiliu89/caffe/tree/ssd .
Verifying Heaps' law using Google Books Ngram data
Authors: Vladimir V. Bochkarev, Eduard Yu. Lerner, Anna V. Shevlyakova
Date: 2016-12-29 00:00:00
Tasks: Text Generation

This article is devoted to the verification of the empirical Heaps law in
European languages using Google Books Ngram corpus data. The connection between
word distribution frequency and expected dependence of individual word number
on text size is analysed in terms of a simple probability model of text
generation. It is shown that the Heaps exponent varies significantly within
characteristic time intervals of 60-100 years.
A hybrid approach to supervised machine learning for algorithmic melody composition
Authors: Rouven Bauer
Date: 2016-12-29 00:00:00
Tasks: BIG-bench Machine Learning

In this work we present an algorithm for composing monophonic melodies
similar in style to those of a given, phrase annotated, sample of melodies. For
implementation, a hybrid approach incorporating parametric Markov models of
higher order and a contour concept of phrases is used. This work is based on
the master thesis of Thayabaran Kathiresan (2015). An online listening test
conducted shows that enhancing a pure Markov model with musically relevant
context, like count and planed melody contour, improves the result
significantly.
Deep neural heart rate variability analysis
Authors: Tamas Madl
Date: 2016-12-29 00:00:00
Tasks: Electrocardiography (ECG), Heart Rate Variability

Despite of the pain and limited accuracy of blood tests for early recognition
of cardiovascular disease, they dominate risk screening and triage. On the
other hand, heart rate variability is non-invasive and cheap, but not
considered accurate enough for clinical practice. Here, we tackle heart beat
interval based classification with deep learning. We introduce an end to end
differentiable hybrid architecture, consisting of a layer of biological neuron
models of cardiac dynamics (modified FitzHugh Nagumo neurons) and several
layers of a standard feed-forward neural network. The proposed model is
evaluated on ECGs from 474 stable at-risk (coronary artery disease) patients,
and 1172 chest pain patients of an emergency department. We show that it can
significantly outperform models based on traditional heart rate variability
predictors, as well as approaching or in some cases outperforming clinical
blood tests, based only on 60 seconds of inter-beat intervals.
Quantum Clustering and Gaussian Mixtures
Authors: Mahajabin Rahman, Davi Geiger
Date: 2016-12-29 00:00:00
Tasks: Clustering

The mixture of Gaussian distributions, a soft version of k-means , is
considered a state-of-the-art clustering algorithm. It is widely used in
computer vision for selecting classes, e.g., color, texture, and shapes. In
this algorithm, each class is described by a Gaussian distribution, defined by
its mean and covariance. The data is described by a weighted sum of these
Gaussian distributions. We propose a new method, inspired by quantum
interference in physics. Instead of modeling each class distribution directly,
we model a class wave function such that its magnitude square is the class
Gaussian distribution. We then mix the class wave functions to create the
mixture wave function. The final mixture distribution is then the magnitude
square of the mixture wave function. As a result, we observe the quantum class
interference phenomena, not present in the Gaussian mixture model. We show that
the quantum method outperforms the Gaussian mixture method in every aspect of
the estimations. It provides more accurate estimations of all distribution
parameters, with much less fluctuations, and it is also more robust to data
deformations from the Gaussian assumptions. We illustrate our method for color
segmentation as an example application.
Unsupervised Learning for Computational Phenotyping
Authors: Chris Hodapp
Date: 2016-12-26 00:00:00
Tasks: Computational Phenotyping, Time Series, Time Series Analysis

With large volumes of health care data comes the research area of
computational phenotyping, making use of techniques such as machine learning to
describe illnesses and other clinical concepts from the data itself. The
"traditional" approach of using supervised learning relies on a domain expert,
and has two main limitations: requiring skilled humans to supply correct labels
limits its scalability and accuracy, and relying on existing clinical
descriptions limits the sorts of patterns that can be found. For instance, it
may fail to acknowledge that a disease treated as a single condition may really
have several subtypes with different phenotypes, as seems to be the case with
asthma and heart disease. Some recent papers cite successes instead using
unsupervised learning. This shows great potential for finding patterns in
Electronic Health Records that would otherwise be hidden and that can lead to
greater understanding of conditions and treatments. This work implements a
method derived strongly from Lasko et al., but implements it in Apache Spark
and Python and generalizes it to laboratory time-series data in MIMIC-III. It
is released as an open-source tool for exploration, analysis, and
visualization, available at https://github.com/Hodapp87/mimic3_phenotyping
High-dimensional Filtering using Nested Sequential Monte Carlo
Authors: Christian A. Naesseth, Fredrik Lindsten, Thomas B. Schön
Date: 2016-12-29 00:00:00
Tasks: State Space Models, Vocal Bursts Intensity Prediction

Sequential Monte Carlo (SMC) methods comprise one of the most successful
approaches to approximate Bayesian filtering. However, SMC without good
proposal distributions struggle in high dimensions. We propose nested
sequential Monte Carlo (NSMC), a methodology that generalises the SMC framework
by requiring only approximate, properly weighted, samples from the SMC proposal
distribution, while still resulting in a correct SMC algorithm. This way we can
exactly approximate the locally optimal proposal, and extend the class of
models for which we can perform efficient inference using SMC. We show improved
accuracy over other state-of-the-art methods on several spatio-temporal state
space models.
The interplay between system identification and machine learning
Authors: Gianluigi Pillonetto
Date: 2016-12-29 00:00:00
Tasks: BIG-bench Machine Learning

Learning from examples is one of the key problems in science and engineering.
It deals with function reconstruction from a finite set of direct and noisy
samples. Regularization in reproducing kernel Hilbert spaces (RKHSs) is widely
used to solve this task and includes powerful estimators such as regularization
networks. Recent achievements include the proof of the statistical consistency
of these kernel- based approaches. Parallel to this, many different system
identification techniques have been developed but the interaction with machine
learning does not appear so strong yet. One reason is that the RKHSs usually
employed in machine learning do not embed the information available on dynamic
systems, e.g. BIBO stability. In addition, in system identification the
independent data assumptions routinely adopted in machine learning are never
satisfied in practice. This paper provides new results which strengthen the
connection between system identification and machine learning. Our starting
point is the introduction of RKHSs of dynamic systems. They contain functionals
over spaces defined by system inputs and allow to interpret system
identification as learning from examples. In both linear and nonlinear
settings, it is shown that this perspective permits to derive in a relatively
simple way conditions on RKHS stability (i.e. the property of containing only
BIBO stable systems or predictors), also facilitating the design of new kernels
for system identification. Furthermore, we prove the convergence of the
regularized estimator to the optimal predictor under conditions typical of
dynamic systems.
From Virtual to Real World Visual Perception using Domain Adaptation -- The DPM as Example
Authors: Antonio M. Lopez, Jiaolong Xu, Jose L. Gomez, David Vazquez, German Ros
Date: 2016-12-29 00:00:00
Tasks: Domain Adaptation, vehicle detection

Supervised learning tends to produce more accurate classifiers than
unsupervised learning in general. This implies that training data is preferred
with annotations. When addressing visual perception challenges, such as
localizing certain object classes within an image, the learning of the involved
classifiers turns out to be a practical bottleneck. The reason is that, at
least, we have to frame object examples with bounding boxes in thousands of
images. A priori, the more complex the model is regarding its number of
parameters, the more annotated examples are required. This annotation task is
performed by human oracles, which ends up in inaccuracies and errors in the
annotations (aka ground truth) since the task is inherently very cumbersome and
sometimes ambiguous. As an alternative we have pioneered the use of virtual
worlds for collecting such annotations automatically and with high precision.
However, since the models learned with virtual data must operate in the real
world, we still need to perform domain adaptation (DA). In this chapter we
revisit the DA of a deformable part-based model (DPM) as an exemplifying case
of virtual- to-real-world DA. As a use case, we address the challenge of
vehicle detection for driver assistance, using different publicly available
virtual-world data. While doing so, we investigate questions such as: how does
the domain gap behave due to virtual-vs-real data with respect to dominant
object appearance per domain, as well as the role of photo-realism in the
virtual world.
Digital Makeup from Internet Images
Authors: Asad Khan, Muhammad Ahmad, Yudong Guo, Ligang Liu
Date: 2016-10-16 00:00:00
Tasks: Image Matting

We present a novel approach of color transfer between images by exploring
their high-level semantic information. First, we set up a database which
consists of the collection of downloaded images from the internet, which are
segmented automatically by using matting techniques. We then, extract image
foregrounds from both source and multiple target images. Then by using image
matting algorithms, the system extracts the semantic information such as faces,
lips, teeth, eyes, eyebrows, etc., from the extracted foregrounds of the source
image. And, then the color is transferred between corresponding parts with the
same semantic information. Next we get the color transferred result by
seamlessly compositing different parts together using alpha blending. In the
final step, we present an efficient method of color consistency to optimize the
color of a collection of images showing the common scene. The main advantage of
our method over existing techniques is that it does not need face matching, as
one could use more than one target images. It is not restricted to head shot
images as we can also change the color style in the wild. Moreover, our
algorithm does not require to choose the same color style, same pose and image
size between source and target images. Our algorithm is not restricted to
one-to-one image color transfer and can make use of more than one target images
to transfer the color in different parts in the source image. Comparing with
other approaches, our algorithm is much better in color blending in the input
data.
Modeling documents with Generative Adversarial Networks
Authors: John Glover
Date: 2016-12-29 00:00:00
Tasks: Denoising

This paper describes a method for using Generative Adversarial Networks to
learn distributed representations of natural language documents. We propose a
model that is based on the recently proposed Energy-Based GAN, but instead uses
a Denoising Autoencoder as the discriminator network. Document representations
are extracted from the hidden layer of the discriminator and evaluated both
quantitatively and qualitatively.
Deep Semi-Supervised Learning with Linguistically Motivated Sequence Labeling Task Hierarchies
Authors: Jonathan Godwin, Pontus Stenetorp, Sebastian Riedel
Date: 2016-12-29 00:00:00
Tasks: Chunking

In this paper we present a novel Neural Network algorithm for conducting
semi-supervised learning for sequence labeling tasks arranged in a
linguistically motivated hierarchy. This relationship is exploited to
regularise the representations of supervised tasks by backpropagating the error
of the unsupervised task through the supervised tasks. We introduce a neural
network where lower layers are supervised by junior downstream tasks and the
final layer task is an auxiliary unsupervised task. The architecture shows
improvements of up to two percentage points F1 for Chunking compared to a
plausible baseline.
Selecting Bases in Spectral learning of Predictive State Representations via Model Entropy
Authors: Yunlong Liu, Hexing Zhu
Date: 2016-12-29 00:00:00

Predictive State Representations (PSRs) are powerful techniques for modelling
dynamical systems, which represent a state as a vector of predictions about
future observable events (tests). In PSRs, one of the fundamental problems is
the learning of the PSR model of the underlying system. Recently, spectral
methods have been successfully used to address this issue by treating the
learning problem as the task of computing an singular value decomposition (SVD)
over a submatrix of a special type of matrix called the Hankel matrix. Under
the assumptions that the rows and columns of the submatrix of the Hankel Matrix
are sufficient~(which usually means a very large number of rows and columns,
and almost fails in practice) and the entries of the matrix can be estimated
accurately, it has been proven that the spectral approach for learning PSRs is
statistically consistent and the learned parameters can converge to the true
parameters. However, in practice, due to the limit of the computation ability,
only a finite set of rows or columns can be chosen to be used for the spectral
learning. While different sets of columns usually lead to variant accuracy of
the learned model, in this paper, we propose an approach for selecting the set
of columns, namely basis selection, by adopting a concept of model entropy to
measure the accuracy of the learned model. Experimental results are shown to
demonstrate the effectiveness of the proposed approach.
Linking the Neural Machine Translation and the Prediction of Organic Chemistry Reactions
Authors: Juno Nam, Jurae Kim
Date: 2016-12-29 00:00:00
Tasks: Machine Translation, Translation

Finding the main product of a chemical reaction is one of the important
problems of organic chemistry. This paper describes a method of applying a
neural machine translation model to the prediction of organic chemical
reactions. In order to translate 'reactants and reagents' to 'products', a
gated recurrent unit based sequence-to-sequence model and a parser to generate
input tokens for model from reaction SMILES strings were built. Training sets
are composed of reactions from the patent databases, and reactions manually
generated applying the elementary reactions in an organic chemistry textbook of
Wade. The trained models were tested by examples and problems in the textbook.
The prediction process does not need manual encoding of rules (e.g., SMARTS
transformations) to predict products, hence it only needs sufficient training
reaction sets to learn new types of reactions.
A Basic Recurrent Neural Network Model
Authors: Fathi M. Salem
Date: 2016-12-29 00:00:00
Tasks: model, Time Series, Time Series Analysis

We present a model of a basic recurrent neural network (or bRNN) that
includes a separate linear term with a slightly "stable" fixed matrix to
guarantee bounded solutions and fast dynamic response. We formulate a state
space viewpoint and adapt the constrained optimization Lagrange Multiplier
(CLM) technique and the vector Calculus of Variations (CoV) to derive the
(stochastic) gradient descent. In this process, one avoids the commonly used
re-application of the circular chain-rule and identifies the error
back-propagation with the co-state backward dynamic equations. We assert that
this bRNN can successfully perform regression tracking of time-series.
Moreover, the "vanishing and exploding" gradients are explicitly quantified and
explained through the co-state dynamics and the update laws. The adapted CoV
framework, in addition, can correctly and principally integrate new loss
functions in the network on any variable and for varied goals, e.g., for
supervised learning on the outputs and unsupervised learning on the internal
(hidden) states.
Learning Identity Mappings with Residual Gates
Authors: Pedro H. P. Savarese, Leonardo O. Mazza, Daniel R. Figueiredo
Date: 2016-11-04 00:00:00
Tasks: Image Classification

We propose a new layer design by adding a linear gating mechanism to shortcut
connections. By using a scalar parameter to control each gate, we provide a way
to learn identity mappings by optimizing only one parameter. We build upon the
motivation behind Residual Networks, where a layer is reformulated in order to
make learning identity mappings less problematic to the optimizer. The
augmentation introduces only one extra parameter per layer, and provides easier
optimization by making degeneration into identity mappings simpler. We propose
a new model, the Gated Residual Network, which is the result when augmenting
Residual Networks. Experimental results show that augmenting layers provides
better optimization, increased performance, and more layer independence. We
evaluate our method on MNIST using fully-connected networks, showing empirical
indications that our augmentation facilitates the optimization of deep models,
and that it provides high tolerance to full layer removal: the model retains
over 90% of its performance even after half of its layers have been randomly
removed. We also evaluate our model on CIFAR-10 and CIFAR-100 using Wide Gated
ResNets, achieving 3.65% and 18.27% error, respectively.
A Deep Learning Approach To Multiple Kernel Fusion
Authors: Huan Song, Jayaraman J. Thiagarajan, Prasanna Sattigeri, Karthikeyan Natesan Ramamurthy, Andreas Spanias
Date: 2016-12-28 00:00:00
Tasks: Activity Recognition, Deep Learning

Kernel fusion is a popular and effective approach for combining multiple
features that characterize different aspects of data. Traditional approaches
for Multiple Kernel Learning (MKL) attempt to learn the parameters for
combining the kernels through sophisticated optimization procedures. In this
paper, we propose an alternative approach that creates dense embeddings for
data using the kernel similarities and adopts a deep neural network
architecture for fusing the embeddings. In order to improve the effectiveness
of this network, we introduce the kernel dropout regularization strategy
coupled with the use of an expanded set of composition kernels. Experiment
results on a real-world activity recognition dataset show that the proposed
architecture is effective in fusing kernels and achieves state-of-the-art
performance.