pwc-archive/papers-with-abstracts huggingface dataset:
Training-image based geostatistical inversion using a spatial generative adversarial neural network
Authors: Eric Laloy, Romain Hérault, Diederik Jacques, Niklas Linde
Date: 2017-08-16 00:00:00
Tasks: Generative Adversarial Network

Probabilistic inversion within a multiple-point statistics framework is often
computationally prohibitive for high-dimensional problems. To partly address
this, we introduce and evaluate a new training-image based inversion approach
for complex geologic media. Our approach relies on a deep neural network of the
generative adversarial network (GAN) type. After training using a training
image (TI), our proposed spatial GAN (SGAN) can quickly generate 2D and 3D
unconditional realizations. A key characteristic of our SGAN is that it defines
a (very) low-dimensional parameterization, thereby allowing for efficient
probabilistic inversion using state-of-the-art Markov chain Monte Carlo (MCMC)
methods. In addition, available direct conditioning data can be incorporated
within the inversion. Several 2D and 3D categorical TIs are first used to
analyze the performance of our SGAN for unconditional geostatistical
simulation. Training our deep network can take several hours. After training,
realizations containing a few millions of pixels/voxels can be produced in a
matter of seconds. This makes it especially useful for simulating many
thousands of realizations (e.g., for MCMC inversion) as the relative cost of
the training per realization diminishes with the considered number of
realizations. Synthetic inversion case studies involving 2D steady-state flow
and 3D transient hydraulic tomography with and without direct conditioning data
are used to illustrate the effectiveness of our proposed SGAN-based inversion.
For the 2D case, the inversion rapidly explores the posterior model
distribution. For the 3D case, the inversion recovers model realizations that
fit the data close to the target level and visually resemble the true model
well.
Adaptive Threshold Sampling
Authors: Daniel Ting
Date: 2017-08-16 00:00:00

Sampling is a fundamental problem in computer science and statistics. However, for a given task and stream, it is often not possible to choose good sampling probabilities in advance. We derive a general framework for adaptively changing the sampling probabilities via a collection of thresholds.In general, adaptive sampling procedures introduce dependence amongst the sampled points, making it difficult to compute expectations and ensure estimators are unbiased or consistent. Our framework address this issue and further shows when adaptive thresholds can be treated as if they were fixed thresholds which samples items independently. This makes our adaptive sampling schemes simple to apply as there is no need to create custom estimators for the sampling method. Using our framework, we derive new samplers that can address a broad range of new and existing problems including sampling with memory rather than sample size budgets, stratified samples, multiple objectives, distinct counting, and sliding windows. In particular, we design a sampling procedure for the top-K problem where, unlike in the heavy-hitter problem, the sketch size and sampling probabilities are adaptively chosen.
Fault in your stars: An Analysis of Android App Reviews
Authors: Rahul Aralikatte, Giriprasad Sridhara, Neelamadhav Gantayat, Senthil Mani
Date: 2017-08-16 00:00:00

Mobile app distribution platforms such as Google Play Store allow users to
share their feedback about downloaded apps in the form of a review comment and
a corresponding star rating. Typically, the star rating ranges from one to five
stars, with one star denoting a high sense of dissatisfaction with the app and
five stars denoting a high sense of satisfaction.
  Unfortunately, due to a variety of reasons, often the star rating provided by
a user is inconsistent with the opinion expressed in the review. For example,
consider the following review for the Facebook App on Android; "Awesome App".
One would reasonably expect the rating for this review to be five stars, but
the actual rating is one star!
  Such inconsistent ratings can lead to a deflated (or inflated) overall
average rating of an app which can affect user downloads, as typically users
look at the average star ratings while making a decision on downloading an app.
Also, the app developers receive a biased feedback about the application that
does not represent ground reality. This is especially significant for small
apps with a few thousand downloads as even a small number of mismatched reviews
can bring down the average rating drastically.
  In this paper, we conducted a study on this review-rating mismatch problem.
We manually examined 8600 reviews from 10 popular Android apps and found that
20% of the ratings in our dataset were inconsistent with the review. Further,
we developed three systems; two of which were based on traditional machine
learning and one on deep learning to automatically identify reviews whose
rating did not match with the opinion expressed in the review. Our deep
learning system performed the best and had an accuracy of 92% in identifying
the correct star rating to be associated with a given review.
Hardware-efficient on-line learning through pipelined truncated-error backpropagation in binary-state networks
Authors: Hesham Mostafa, Bruno Pedroni, Sadique Sheik, Gert Cauwenberghs
Date: 2017-06-15 00:00:00

Artificial neural networks (ANNs) trained using backpropagation are powerful
learning architectures that have achieved state-of-the-art performance in
various benchmarks. Significant effort has been devoted to developing custom
silicon devices to accelerate inference in ANNs. Accelerating the training
phase, however, has attracted relatively little attention. In this paper, we
describe a hardware-efficient on-line learning technique for feedforward
multi-layer ANNs that is based on pipelined backpropagation. Learning is
performed in parallel with inference in the forward pass, removing the need for
an explicit backward pass and requiring no extra weight lookup. By using binary
state variables in the feedforward network and ternary errors in
truncated-error backpropagation, the need for any multiplications in the
forward and backward passes is removed, and memory requirements for the
pipelining are drastically reduced. Further reduction in addition operations
owing to the sparsity in the forward neural and backpropagating error signal
paths contributes to highly efficient hardware implementation. For
proof-of-concept validation, we demonstrate on-line learning of MNIST
handwritten digit classification on a Spartan 6 FPGA interfacing with an
external 1Gb DDR2 DRAM, that shows small degradation in test error performance
compared to an equivalently sized binary ANN trained off-line using standard
back-propagation and exact errors. Our results highlight an attractive synergy
between pipelined backpropagation and binary-state networks in substantially
reducing computation and memory requirements, making pipelined on-line learning
practical in deep networks.
Supervised learning based on temporal coding in spiking neural networks
Authors: Hesham Mostafa
Date: 2016-06-27 00:00:00
Tasks: Relation

Gradient descent training techniques are remarkably successful in training
analog-valued artificial neural networks (ANNs). Such training techniques,
however, do not transfer easily to spiking networks due to the spike generation
hard non-linearity and the discrete nature of spike communication. We show that
in a feedforward spiking network that uses a temporal coding scheme where
information is encoded in spike times instead of spike rates, the network
input-output relation is differentiable almost everywhere. Moreover, this
relation is piece-wise linear after a transformation of variables. Methods for
training ANNs thus carry directly to the training of such spiking networks as
we show when training on the permutation invariant MNIST task. In contrast to
rate-based spiking networks that are often used to approximate the behavior of
ANNs, the networks we present spike much more sparsely and their behavior can
not be directly approximated by conventional ANNs. Our results highlight a new
approach for controlling the behavior of spiking networks with realistic
temporal dynamics, opening up the potential for using these networks to process
spike patterns with complex temporal information.
Reconstruction-Based Disentanglement for Pose-invariant Face Recognition
Authors: Xi Peng, Xiang Yu, Kihyuk Sohn, Dimitris Metaxas, Manmohan Chandraker
Date: 2017-02-10 00:00:00
Proceeding: ICCV 2017 10
Tasks: Disentanglement, Face Recognition, Metric Learning, Robust Face Recognition

Deep neural networks (DNNs) trained on large-scale datasets have recently
achieved impressive improvements in face recognition. But a persistent
challenge remains to develop methods capable of handling large pose variations
that are relatively underrepresented in training data. This paper presents a
method for learning a feature representation that is invariant to pose, without
requiring extensive pose coverage in training data. We first propose to
generate non-frontal views from a single frontal face, in order to increase the
diversity of training data while preserving accurate facial details that are
critical for identity discrimination. Our next contribution is to seek a rich
embedding that encodes identity features, as well as non-identity ones such as
pose and landmark locations. Finally, we propose a new feature reconstruction
metric learning to explicitly disentangle identity and pose, by demanding
alignment between the feature reconstructions through various combinations of
identity and pose features, which is obtained from two images of the same
subject. Experiments on both controlled and in-the-wild face datasets, such as
MultiPIE, 300WLP and the profile view database CFP, show that our method
consistently outperforms the state-of-the-art, especially on images with large
head pose variations. Detail results and resource are referred to
https://sites.google.com/site/xipengcshomepage/iccv2017
Stacked Deconvolutional Network for Semantic Segmentation
Authors: Jun Fu, Jing Liu, Yuhang Wang, Hanqing Lu
Date: 2017-08-16 00:00:00
Tasks: Segmentation, Semantic Segmentation

Recent progress in semantic segmentation has been driven by improving the
spatial resolution under Fully Convolutional Networks (FCNs). To address this
problem, we propose a Stacked Deconvolutional Network (SDN) for semantic
segmentation. In SDN, multiple shallow deconvolutional networks, which are
called as SDN units, are stacked one by one to integrate contextual information
and guarantee the fine recovery of localization information. Meanwhile,
inter-unit and intra-unit connections are designed to assist network training
and enhance feature fusion since the connections improve the flow of
information and gradient propagation throughout the network. Besides,
hierarchical supervision is applied during the upsampling process of each SDN
unit, which guarantees the discrimination of feature representations and
benefits the network optimization. We carry out comprehensive experiments and
achieve the new state-of-the-art results on three datasets, including PASCAL
VOC 2012, CamVid, GATECH. In particular, our best model without CRF
post-processing achieves an intersection-over-union score of 86.6% in the test
set.
Time Series Compression Based on Adaptive Piecewise Recurrent Autoencoder
Authors: Daniel Hsu
Date: 2017-07-23 00:00:00
Tasks: Decoder, Time Series, Time Series Analysis

Time series account for a large proportion of the data stored in financial,
medical and scientific databases. The efficient storage of time series is
important in practical applications. In this paper, we propose a novel
compression scheme for time series. The encoder and decoder are both composed
by recurrent neural networks (RNN) such as long short-term memory (LSTM). There
is an autoencoder between encoder and decoder, which encodes the hidden state
and input together and decodes them at the decoder side. Moreover, we
pre-process the original time series by partitioning it into segments with
various lengths which have similar total variation. The experimental study
shows that the proposed algorithm can achieve competitive compression ratio on
real-world time series.
mAnI: Movie Amalgamation using Neural Imitation
Authors: Naveen Panwar, Shreya Khare, Neelamadhav Gantayat, Rahul Aralikatte, Senthil Mani, Anush Sankaran
Date: 2017-08-16 00:00:00
Tasks: Retrieval

Cross-modal data retrieval has been the basis of various creative tasks
performed by Artificial Intelligence (AI). One such highly challenging task for
AI is to convert a book into its corresponding movie, which most of the
creative film makers do as of today. In this research, we take the first step
towards it by visualizing the content of a book using its corresponding movie
visuals. Given a set of sentences from a book or even a fan-fiction written in
the same universe, we employ deep learning models to visualize the input by
stitching together relevant frames from the movie. We studied and compared
three different types of setting to match the book with the movie content: (i)
Dialog model: using only the dialog from the movie, (ii) Visual model: using
only the visual content from the movie, and (iii) Hybrid model: using the
dialog and the visual content from the movie. Experiments on the publicly
available MovieBook dataset shows the effectiveness of the proposed models.
Optimal Alarms for Vehicular Collision Detection
Authors: Michael Motro, Joydeep Ghosh, Chandra Bhat
Date: 2017-08-16 00:00:00
Tasks: BIG-bench Machine Learning

An important application of intelligent vehicles is advance detection of
dangerous events such as collisions. This problem is framed as a problem of
optimal alarm choice given predictive models for vehicle location and motion.
Techniques for real-time collision detection are surveyed and grouped into
three classes: random Monte Carlo sampling, faster deterministic
approximations, and machine learning models trained by simulation. Theoretical
guarantees on the performance of these collision detection techniques are
provided where possible, and empirical analysis is provided for two example
scenarios. Results validate Monte Carlo sampling as a robust solution despite
its simplicity.
Optimal Hyperparameters for Deep LSTM-Networks for Sequence Labeling Tasks
Authors: Nils Reimers, Iryna Gurevych
Date: 2017-07-21 00:00:00
Tasks: Chunking, Event Detection, Hyperparameter Optimization, Named Entity Recognition (NER), POS, Word Embeddings

Selecting optimal parameters for a neural network architecture can often make
the difference between mediocre and state-of-the-art performance. However,
little is published which parameters and design choices should be evaluated or
selected making the correct hyperparameter optimization often a "black art that
requires expert experiences" (Snoek et al., 2012). In this paper, we evaluate
the importance of different network design choices and hyperparameters for five
common linguistic sequence tagging tasks (POS, Chunking, NER, Entity
Recognition, and Event Detection). We evaluated over 50.000 different setups
and found, that some parameters, like the pre-trained word embeddings or the
last layer of the network, have a large impact on the performance, while other
parameters, for example the number of LSTM layers or the number of recurrent
units, are of minor importance. We give a recommendation on a configuration
that performs well among different tasks.
Towards Visual Explanations for Convolutional Neural Networks via Input Resampling
Authors: Benjamin J. Lengerich, Sandeep Konam, Eric P. Xing, Stephanie Rosenthal, Manuela Veloso
Date: 2017-07-30 00:00:00

The predictive power of neural networks often costs model interpretability.
Several techniques have been developed for explaining model outputs in terms of
input features; however, it is difficult to translate such interpretations into
actionable insight. Here, we propose a framework to analyze predictions in
terms of the model's internal features by inspecting information flow through
the network. Given a trained network and a test image, we select neurons by two
metrics, both measured over a set of images created by perturbations to the
input image: (1) magnitude of the correlation between the neuron activation and
the network output and (2) precision of the neuron activation. We show that the
former metric selects neurons that exert large influence over the network
output while the latter metric selects neurons that activate on generalizable
features. By comparing the sets of neurons selected by these two metrics, our
framework suggests a way to investigate the internal attention mechanisms of
convolutional neural networks.
Towards Semantic Fast-Forward and Stabilized Egocentric Videos
Authors: Michel Melo Silva, Washington Luis Souza Ramos, Joao Pedro Klock Ferreira, Mario Fernando Montenegro Campos, Erickson Rangel Nascimento
Date: 2017-08-14 00:00:00

The emergence of low-cost personal mobiles devices and wearable cameras and
the increasing storage capacity of video-sharing websites have pushed forward a
growing interest towards first-person videos. Since most of the recorded videos
compose long-running streams with unedited content, they are tedious and
unpleasant to watch. The fast-forward state-of-the-art methods are facing
challenges of balancing the smoothness of the video and the emphasis in the
relevant frames given a speed-up rate. In this work, we present a methodology
capable of summarizing and stabilizing egocentric videos by extracting the
semantic information from the frames. This paper also describes a dataset
collection with several semantically labeled videos and introduces a new
smoothness evaluation metric for egocentric videos that is used to test our
method.
Fast-Forward Video Based on Semantic Extraction
Authors: Washington Luis Souza Ramos, Michel Melo Silva, Mario Fernando Montenegro Campos, Erickson Rangel Nascimento
Date: 2017-08-14 00:00:00

Thanks to the low operational cost and large storage capacity of smartphones
and wearable devices, people are recording many hours of daily activities,
sport actions and home videos. These videos, also known as egocentric videos,
are generally long-running streams with unedited content, which make them
boring and visually unpalatable, bringing up the challenge to make egocentric
videos more appealing. In this work we propose a novel methodology to compose
the new fast-forward video by selecting frames based on semantic information
extracted from images. The experiments show that our approach outperforms the
state-of-the-art as far as semantic information is concerned and that it is
also able to produce videos that are more pleasant to be watched.
A deep architecture for unified aesthetic prediction
Authors: Naila Murray, Albert Gordo
Date: 2017-08-16 00:00:00
Tasks: Prediction

Image aesthetics has become an important criterion for visual content
curation on social media sites and media content repositories. Previous work on
aesthetic prediction models in the computer vision community has focused on
aesthetic score prediction or binary image labeling. However, raw aesthetic
annotations are in the form of score histograms and provide richer and more
precise information than binary labels or mean scores. Consequently, in this
work we focus on the rarely-studied problem of predicting aesthetic score
distributions and propose a novel architecture and training procedure for our
model. Our model achieves state-of-the-art results on the standard AVA
large-scale benchmark dataset for three tasks: (i) aesthetic quality
classification; (ii) aesthetic score regression; and (iii) aesthetic score
distribution prediction, all while using one model trained only for the
distribution prediction task. We also introduce a method to modify an image
such that its predicted aesthetics changes, and use this modification to gain
insight into our model.
Multi-task Neural Network for Non-discrete Attribute Prediction in Knowledge Graphs
Authors: Yi Tay, Luu Anh Tuan, Minh C. Phan, Siu Cheung Hui
Date: 2017-08-16 00:00:00
Tasks: Attribute, Knowledge Graphs, Multi-Task Learning, Prediction, Relational Reasoning, Triplet, Value prediction

Many popular knowledge graphs such as Freebase, YAGO or DBPedia maintain a
list of non-discrete attributes for each entity. Intuitively, these attributes
such as height, price or population count are able to richly characterize
entities in knowledge graphs. This additional source of information may help to
alleviate the inherent sparsity and incompleteness problem that are prevalent
in knowledge graphs. Unfortunately, many state-of-the-art relational learning
models ignore this information due to the challenging nature of dealing with
non-discrete data types in the inherently binary-natured knowledge graphs. In
this paper, we propose a novel multi-task neural network approach for both
encoding and prediction of non-discrete attribute information in a relational
setting. Specifically, we train a neural network for triplet prediction along
with a separate network for attribute value regression. Via multi-task
learning, we are able to learn representations of entities, relations and
attributes that encode information about both tasks. Moreover, such attributes
are not only central to many predictive tasks as an information source but also
as a prediction target. Therefore, models that are able to encode, incorporate
and predict such information in a relational learning context are highly
attractive as well. We show that our approach outperforms many state-of-the-art
methods for the tasks of relational triplet classification and attribute value
prediction.
Ultra-Fast Reactive Transport Simulations When Chemical Reactions Meet Machine Learning: Chemical Equilibrium
Authors: Allan M. M. Leal, Dmitrii A. Kulik, Martin O. Saar
Date: 2017-08-16 00:00:00
Tasks: BIG-bench Machine Learning

During reactive transport modeling, the computational cost associated with
chemical reaction calculations is often 10-100 times higher than that of
transport calculations. Most of these costs results from chemical equilibrium
calculations that are performed at least once in every mesh cell and at every
time step of the simulation. Calculating chemical equilibrium is an iterative
process, where each iteration is in general so computationally expensive that
even if every calculation converged in a single iteration, the resulting
speedup would not be significant. Thus, rather than proposing a fast-converging
numerical method for solving chemical equilibrium equations, we present a
machine learning method that enables new equilibrium states to be quickly and
accurately estimated, whenever a previous equilibrium calculation with similar
input conditions has been performed. We demonstrate the use of this smart
chemical equilibrium method in a reactive transport modeling example and show
that, even at early simulation times, the majority of all equilibrium
calculations are quickly predicted and, after some time steps, the
machine-learning-accelerated chemical solver has been fully trained to rapidly
perform all subsequent equilibrium calculations, resulting in speedups of
almost two orders of magnitude. We remark that our new on-demand machine
learning method can be applied to any case in which a massive number of
sequential/parallel evaluations of a computationally expensive function $f$
needs to be done, $y=f(x)$. We remark, that, in contrast to traditional machine
learning algorithms, our on-demand training approach does not require a
statistics-based training phase before the actual simulation of interest
commences. The introduced on-demand training scheme requires, however, the
first-order derivatives $\partial f/\partial x$ for later smart predictions.