pwc-archive/papers-with-abstracts huggingface dataset:
Language modeling with Neural trans-dimensional random fields
Authors: Bin Wang, Zhijian Ou
Date: 2017-07-23 00:00:00
Tasks: Language Modeling, Language Modelling, speech-recognition, Speech Recognition

Trans-dimensional random field language models (TRF LMs) have recently been
introduced, where sentences are modeled as a collection of random fields. The
TRF approach has been shown to have the advantages of being computationally
more efficient in inference than LSTM LMs with close performance and being able
to flexibly integrating rich features. In this paper we propose neural TRFs,
beyond of the previous discrete TRFs that only use linear potentials with
discrete features. The idea is to use nonlinear potentials with continuous
features, implemented by neural networks (NNs), in the TRF framework. Neural
TRFs combine the advantages of both NNs and TRFs. The benefits of word
embedding, nonlinear feature learning and larger context modeling are inherited
from the use of NNs. At the same time, the strength of efficient inference by
avoiding expensive softmax is preserved. A number of technical contributions,
including employing deep convolutional neural networks (CNNs) to define the
potentials and incorporating the joint stochastic approximation (JSA) strategy
in the training algorithm, are developed in this work, which enable us to
successfully train neural TRF LMs. Various LMs are evaluated in terms of speech
recognition WERs by rescoring the 1000-best lists of WSJ'92 test data. The
results show that neural TRF LMs not only improve over discrete TRF LMs, but
also perform slightly better than LSTM LMs with only one fifth of parameters
and 16x faster inference efficiency.
Unwritten Languages Demand Attention Too! Word Discovery with Encoder-Decoder Models
Authors: Marcely Zanon Boito, Alexandre Berard, Aline Villavicencio, Laurent Besacier
Date: 2017-09-17 00:00:00
Tasks: Decoder, Machine Translation, Translation

Word discovery is the task of extracting words from unsegmented text. In this
paper we examine to what extent neural networks can be applied to this task in
a realistic unwritten language scenario, where only small corpora and limited
annotations are available. We investigate two scenarios: one with no
supervision and another with limited supervision with access to the most
frequent words. Obtained results show that it is possible to retrieve at least
27% of the gold standard vocabulary by training an encoder-decoder neural
machine translation system with only 5,157 sentences. This result is close to
those obtained with a task-specific Bayesian nonparametric model. Moreover, our
approach has the advantage of generating translation alignments, which could be
used to create a bilingual lexicon. As a future perspective, this approach is
also well suited to work directly from speech.
Deep-Learnt Classification of Light Curves
Authors: Ashish Mahabal, Kshiteej Sheth, Fabian Gieseke, Akshay Pai, S. George Djorgovski, Andrew Drake, Matthew Graham, the CSS/CRTS/PTF Collaboration
Date: 2017-09-19 00:00:00
Tasks: Astronomy, Classification, General Classification, Time Series, Time Series Analysis

Astronomy light curves are sparse, gappy, and heteroscedastic. As a result
standard time series methods regularly used for financial and similar datasets
are of little help and astronomers are usually left to their own instruments
and techniques to classify light curves. A common approach is to derive
statistical features from the time series and to use machine learning methods,
generally supervised, to separate objects into a few of the standard classes.
In this work, we transform the time series to two-dimensional light curve
representations in order to classify them using modern deep learning
techniques. In particular, we show that convolutional neural networks based
classifiers work well for broad characterization and classification. We use
labeled datasets of periodic variables from CRTS survey and show how this opens
doors for a quick classification of diverse classes with several possible
exciting extensions.
Finite Sample Guarantees for PCA in Non-Isotropic and Data-Dependent Noise
Authors: Namrata Vaswani, Praneeth Narayanamurthy
Date: 2017-09-19 00:00:00

This work obtains novel finite sample guarantees for Principal Component
Analysis (PCA). These hold even when the corrupting noise is non-isotropic, and
a part (or all of it) is data-dependent. Because of the latter, in general, the
noise and the true data are correlated. The results in this work are a
significant improvement over those given in our earlier work where this
"correlated-PCA" problem was first studied. In fact, in certain regimes, our
results imply that the sample complexity required to achieve subspace recovery
error that is a constant fraction of the noise level is near-optimal. Useful
corollaries of our result include guarantees for PCA in sparse data-dependent
noise and for PCA with missing data. An important application of the former is
in proving correctness of the subspace update step of a popular online
algorithm for dynamic robust PCA.
Look Wider to Match Image Patches with Convolutional Neural Networks
Authors: Haesol Park, Kyoung Mu Lee
Date: 2017-09-19 00:00:00
Tasks: Stereo Matching, Stereo Matching Hand

When a human matches two images, the viewer has a natural tendency to view
the wide area around the target pixel to obtain clues of right correspondence.
However, designing a matching cost function that works on a large window in the
same way is difficult. The cost function is typically not intelligent enough to
discard the information irrelevant to the target pixel, resulting in
undesirable artifacts. In this paper, we propose a novel learn a stereo
matching cost with a large-sized window. Unlike conventional pooling layers
with strides, the proposed per-pixel pyramid-pooling layer can cover a large
area without a loss of resolution and detail. Therefore, the learned matching
cost function can successfully utilize the information from a large area
without introducing the fattening effect. The proposed method is robust despite
the presence of weak textures, depth discontinuity, illumination, and exposure
difference. The proposed method achieves near-peak performance on the
Middlebury benchmark.
Rethink ReLU to Training Better CNNs
Authors: Gangming Zhao, Zhao-Xiang Zhang, He Guan, Peng Tang, Jingdong Wang
Date: 2017-09-19 00:00:00

Most of convolutional neural networks share the same characteristic: each
convolutional layer is followed by a nonlinear activation layer where Rectified
Linear Unit (ReLU) is the most widely used. In this paper, we argue that the
designed structure with the equal ratio between these two layers may not be the
best choice since it could result in the poor generalization ability. Thus, we
try to investigate a more suitable method on using ReLU to explore the better
network architectures. Specifically, we propose a proportional module to keep
the ratio between convolution and ReLU amount to be N:M (N>M). The proportional
module can be applied in almost all networks with no extra computational cost
to improve the performance. Comprehensive experimental results indicate that
the proposed method achieves better performance on different benchmarks with
different network architectures, thus verify the superiority of our work.
Restoration of Atmospheric Turbulence-distorted Images via RPCA and Quasiconformal Maps
Authors: Chun Pong Lau, Yu Hin Lai, Lok Ming Lui
Date: 2017-04-11 00:00:00

We address the problem of restoring a high-quality image from an observed
image sequence strongly distorted by atmospheric turbulence. A novel algorithm
is proposed in this paper to reduce geometric distortion as well as
space-and-time-varying blur due to strong turbulence. By considering a suitable
energy functional, our algorithm first obtains a sharp reference image and a
subsampled image sequence containing sharp and mildly distorted image frames
with respect to the reference image. The subsampled image sequence is then
stabilized by applying the Robust Principal Component Analysis (RPCA) on the
deformation fields between image frames and warping the image frames by a
quasiconformal map associated with the low-rank part of the deformation matrix.
After image frames are registered to the reference image, the low-rank part of
them are deblurred via a blind deconvolution, and the deblurred frames are then
fused with the enhanced sparse part. Experiments have been carried out on both
synthetic and real turbulence-distorted video. Results demonstrate that our
method is effective in alleviating distortions and blur, restoring image
details and enhancing visual quality.
DESPOT: Online POMDP Planning with Regularization
Authors: Nan Ye, Adhiraj Somani, David Hsu, Wee Sun Lee
Date: 2016-09-12 00:00:00
Proceeding: NeurIPS 2013 12
Tasks: Autonomous Driving

The partially observable Markov decision process (POMDP) provides a
principled general framework for planning under uncertainty, but solving POMDPs
optimally is computationally intractable, due to the "curse of dimensionality"
and the "curse of history". To overcome these challenges, we introduce the
Determinized Sparse Partially Observable Tree (DESPOT), a sparse approximation
of the standard belief tree, for online planning under uncertainty. A DESPOT
focuses online planning on a set of randomly sampled scenarios and compactly
captures the "execution" of all policies under these scenarios. We show that
the best policy obtained from a DESPOT is near-optimal, with a regret bound
that depends on the representation size of the optimal policy. Leveraging this
result, we give an anytime online planning algorithm, which searches a DESPOT
for a policy that optimizes a regularized objective function. Regularization
balances the estimated value of a policy under the sampled scenarios and the
policy size, thus avoiding overfitting. The algorithm demonstrates strong
experimental results, compared with some of the best online POMDP algorithms
available. It has also been incorporated into an autonomous driving system for
real-time vehicle control. The source code for the algorithm is available
online.
CISRDCNN: Super-resolution of compressed images using deep convolutional neural networks
Authors: Honggang Chen, Xiaohai He, Chao Ren, Linbo Qing, Qizhi Teng
Date: 2017-09-19 00:00:00
Tasks: Image Super-Resolution, Super-Resolution

In recent years, much research has been conducted on image super-resolution
(SR). To the best of our knowledge, however, few SR methods were concerned with
compressed images. The SR of compressed images is a challenging task due to the
complicated compression artifacts, while many images suffer from them in
practice. The intuitive solution for this difficult task is to decouple it into
two sequential but independent subproblems, i.e., compression artifacts
reduction (CAR) and SR. Nevertheless, some useful details may be removed in CAR
stage, which is contrary to the goal of SR and makes the SR stage more
challenging. In this paper, an end-to-end trainable deep convolutional neural
network is designed to perform SR on compressed images (CISRDCNN), which
reduces compression artifacts and improves image resolution jointly.
Experiments on compressed images produced by JPEG (we take the JPEG as an
example in this paper) demonstrate that the proposed CISRDCNN yields
state-of-the-art SR performance on commonly used test images and imagesets. The
results of CISRDCNN on real low quality web images are also very impressive,
with obvious quality enhancement. Further, we explore the application of the
proposed SR method in low bit-rate image coding, leading to better
rate-distortion performance than JPEG.
Algorithm and Hardware Design of Discrete-Time Spiking Neural Networks Based on Back Propagation with Binary Activations
Authors: Shihui Yin, Shreyas K. Venkataramanaiah, Gregory K. Chen, Ram Krishnamurthy, Yu Cao, Chaitali Chakrabarti, Jae-sun Seo
Date: 2017-09-19 00:00:00
Tasks: General Classification

We present a new back propagation based training algorithm for discrete-time
spiking neural networks (SNN). Inspired by recent deep learning algorithms on
binarized neural networks, binary activation with a straight-through gradient
estimator is used to model the leaky integrate-fire spiking neuron, overcoming
the difficulty in training SNNs using back propagation. Two SNN training
algorithms are proposed: (1) SNN with discontinuous integration, which is
suitable for rate-coded input spikes, and (2) SNN with continuous integration,
which is more general and can handle input spikes with temporal information.
Neuromorphic hardware designed in 40nm CMOS exploits the spike sparsity and
demonstrates high classification accuracy (>98% on MNIST) and low energy
(48.4-773 nJ/image).
Protest Activity Detection and Perceived Violence Estimation from Social Media Images
Authors: Donghyeon Won, Zachary C. Steinert-Threlkeld, Jungseock Joo
Date: 2017-09-18 00:00:00
Tasks: Action Detection, Activity Detection

We develop a novel visual model which can recognize protesters, describe
their activities by visual attributes and estimate the level of perceived
violence in an image. Studies of social media and protests use natural language
processing to track how individuals use hashtags and links, often with a focus
on those items' diffusion. These approaches, however, may not be effective in
fully characterizing actual real-world protests (e.g., violent or peaceful) or
estimating the demographics of participants (e.g., age, gender, and race) and
their emotions. Our system characterizes protests along these dimensions. We
have collected geotagged tweets and their images from 2013-2017 and analyzed
multiple major protest events in that period. A multi-task convolutional neural
network is employed in order to automatically classify the presence of
protesters in an image and predict its visual attributes, perceived violence
and exhibited emotions. We also release the UCLA Protest Image Dataset, our
novel dataset of 40,764 images (11,659 protest images and hard negatives) with
various annotations of visual attributes and sentiments. Using this dataset, we
train our model and demonstrate its effectiveness. We also present experimental
results from various analysis on geotagged image data in several prevalent
protest events. Our dataset will be made accessible at
https://www.sscnet.ucla.edu/comm/jjoo/mm-protest/.
A Comparative Quantitative Analysis of Contemporary Big Data Clustering Algorithms for Market Segmentation in Hospitality Industry
Authors: Avishek Bose, Arslan Munir, Neda Shabani
Date: 2017-09-18 00:00:00
Tasks: Clustering, Marketing

The hospitality industry is one of the data-rich industries that receives
huge Volumes of data streaming at high Velocity with considerably Variety,
Veracity, and Variability. These properties make the data analysis in the
hospitality industry a big data problem. Meeting the customers' expectations is
a key factor in the hospitality industry to grasp the customers' loyalty. To
achieve this goal, marketing professionals in this industry actively look for
ways to utilize their data in the best possible manner and advance their data
analytic solutions, such as identifying a unique market segmentation clustering
and developing a recommendation system. In this paper, we present a
comprehensive literature review of existing big data clustering algorithms and
their advantages and disadvantages for various use cases. We implement the
existing big data clustering algorithms and provide a quantitative comparison
of the performance of different clustering algorithms for different scenarios.
We also present our insights and recommendations regarding the suitability of
different big data clustering algorithms for different use cases. These
recommendations will be helpful for hoteliers in selecting the appropriate
market segmentation clustering algorithm for different clustering datasets to
improve the customer experience and maximize the hotel revenue.
Human Understandable Explanation Extraction for Black-box Classification Models Based on Matrix Factorization
Authors: Jaedeok Kim, Jingoo Seo
Date: 2017-09-18 00:00:00
Tasks: Decision Making, Defect Detection, General Classification

In recent years, a number of artificial intelligent services have been
developed such as defect detection system or diagnosis system for customer
services. Unfortunately, the core in these services is a black-box in which
human cannot understand the underlying decision making logic, even though the
inspection of the logic is crucial before launching a commercial service. Our
goal in this paper is to propose an analytic method of a model explanation that
is applicable to general classification models. To this end, we introduce the
concept of a contribution matrix and an explanation embedding in a constraint
space by using a matrix factorization. We extract a rule-like model explanation
from the contribution matrix with the help of the nonnegative matrix
factorization. To validate our method, the experiment results provide with open
datasets as well as an industry dataset of a LTE network diagnosis and the
results show our method extracts reasonable explanations.
A Summary Of The Kernel Matrix, And How To Learn It Effectively Using Semidefinite Programming
Authors: Amir-Hossein Karimi
Date: 2017-09-18 00:00:00
Tasks: Partially Labeled Datasets

Kernel-based learning algorithms are widely used in machine learning for
problems that make use of the similarity between object pairs. Such algorithms
first embed all data points into an alternative space, where the inner product
between object pairs specifies their distance in the embedding space. Applying
kernel methods to partially labeled datasets is a classical challenge in this
regard, requiring that the distances between unlabeled pairs must somehow be
learnt using the labeled data. In this independent study, I will summarize the
work of G. Lanckriet et al.'s work on "Learning the Kernel Matrix with
Semidefinite Programming" used in support vector machines (SVM) algorithms for
the transduction problem. Throughout the report, I have provide alternative
explanations / derivations / analysis related to this work which is designed to
ease the understanding of the original article.
Measurement of amplitude of the moiré patterns in digital autostereoscopic 3D display
Authors: Vladimir Saveljev, Sung-Kyu Kim
Date: 2017-09-18 00:00:00

The article presents the experimental measurements of the amplitude of the
moir\'e patterns in a digital autostereoscopic barrier-type 3D display across a
wide angular range with a small increment. The period and orientation of the
moir\'e patterns were also measured as functions of the angle. Simultaneous
branches are observed and analyzed. The theoretical interpretation is also
given. The results can help preventing or minimizing the moir\'e effect in
displays.
DAC-h3: A Proactive Robot Cognitive Architecture to Acquire and Express Knowledge About the World and the Self
Authors: Clément Moulin-Frier, Tobias Fischer, Maxime Petit, Grégoire Pointeau, Jordi-Ysard Puigbo, Ugo Pattacini, Sock Ching Low, Daniel Camilleri, Phuong Nguyen, Matej Hoffmann, Hyung Jin Chang, Martina Zambelli, Anne-Laure Mealier, Andreas Damianou, Giorgio Metta, Tony J. Prescott, Yiannis Demiris, Peter Ford Dominey, Paul F. M. J. Verschure
Date: 2017-06-12 00:00:00

This paper introduces a cognitive architecture for a humanoid robot to engage
in a proactive, mixed-initiative exploration and manipulation of its
environment, where the initiative can originate from both the human and the
robot. The framework, based on a biologically-grounded theory of the brain and
mind, integrates a reactive interaction engine, a number of state-of-the-art
perceptual and motor learning algorithms, as well as planning abilities and an
autobiographical memory. The architecture as a whole drives the robot behavior
to solve the symbol grounding problem, acquire language capabilities, execute
goal-oriented behavior, and express a verbal narrative of its own experience in
the world. We validate our approach in human-robot interaction experiments with
the iCub humanoid robot, showing that the proposed cognitive architecture can
be applied in real time within a realistic scenario and that it can be used
with naive users.
Bias Correction with Jackknife, Bootstrap, and Taylor Series
Authors: Jiantao Jiao, Yanjun Han
Date: 2017-09-18 00:00:00

We analyze bias correction methods using jackknife, bootstrap, and Taylor series. We focus on the binomial model, and consider the problem of bias correction for estimating $f(p)$, where $f \in C[0,1]$ is arbitrary. We characterize the supremum norm of the bias of general jackknife and bootstrap estimators for any continuous functions, and demonstrate the in delete-$d$ jackknife, different values of $d$ may lead to drastically different behaviors in jackknife. We show that in the binomial model, iterating the bootstrap bias correction infinitely many times may lead to divergence of bias and variance, and demonstrate that the bias properties of the bootstrap bias corrected estimator after $r-1$ rounds are of the same order as that of the $r$-jackknife estimator if a bounded coefficients condition is satisfied.
Learning Methods for Dynamic Topic Modeling in Automated Behaviour Analysis
Authors: Olga Isupova, Danil Kuzin, Lyudmila Mihaylova
Date: 2016-11-02 00:00:00
Tasks: Dynamic Topic Modeling

Semi-supervised and unsupervised systems provide operators with invaluable
support and can tremendously reduce the operators load. In the light of the
necessity to process large volumes of video data and provide autonomous
decisions, this work proposes new learning algorithms for activity analysis in
video. The activities and behaviours are described by a dynamic topic model.
Two novel learning algorithms based on the expectation maximisation approach
and variational Bayes inference are proposed. Theoretical derivations of the
posterior of model parameters are given. The designed learning algorithms are
compared with the Gibbs sampling inference scheme introduced earlier in the
literature. A detailed comparison of the learning algorithms is presented on
real video data. We also propose an anomaly localisation procedure, elegantly
embedded in the topic modeling framework. The proposed framework can be applied
to a number of areas, including transportation systems, security and
surveillance.
DropoutDAgger: A Bayesian Approach to Safe Imitation Learning
Authors: Kunal Menda, Katherine Driggs-Campbell, Mykel J. Kochenderfer
Date: 2017-09-18 00:00:00
Tasks: Imitation Learning, MuJoCo

While imitation learning is becoming common practice in robotics, this
approach often suffers from data mismatch and compounding errors. DAgger is an
iterative algorithm that addresses these issues by continually aggregating
training data from both the expert and novice policies, but does not consider
the impact of safety. We present a probabilistic extension to DAgger, which
uses the distribution over actions provided by the novice policy, for a given
observation. Our method, which we call DropoutDAgger, uses dropout to train the
novice as a Bayesian neural network that provides insight to its confidence.
Using the distribution over the novice's actions, we estimate a probabilistic
measure of safety with respect to the expert action, tuned to balance
exploration and exploitation. The utility of this approach is evaluated on the
MuJoCo HalfCheetah and in a simple driving experiment, demonstrating improved
performance and safety compared to other DAgger variants and classic imitation
learning.