pwc-archive/papers-with-abstracts huggingface dataset:
Practical Learning of Predictive State Representations
Authors: Carlton Downey, Ahmed Hefny, Geoffrey Gordon
Date: 2017-02-14 00:00:00

Over the past decade there has been considerable interest in spectral
algorithms for learning Predictive State Representations (PSRs). Spectral
algorithms have appealing theoretical guarantees; however, the resulting models
do not always perform well on inference tasks in practice. One reason for this
behavior is the mismatch between the intended task (accurate filtering or
prediction) and the loss function being optimized by the algorithm (estimation
error in model parameters).
  A natural idea is to improve performance by refining PSRs using an algorithm
such as EM. Unfortunately it is not obvious how to apply apply an EM style
algorithm in the context of PSRs as the Log Likelihood is not well defined for
all PSRs. We show that it is possible to overcome this problem using ideas from
Predictive State Inference Machines.
  We combine spectral algorithms for PSRs as a consistent and efficient
initialization with PSIM-style updates to refine the resulting model
parameters. By combining these two ideas we develop Inference Gradients, a
simple, fast, and robust method for practical learning of PSRs. Inference
Gradients performs gradient descent in the PSR parameter space to optimize an
inference-based loss function like PSIM. Because Inference Gradients uses a
spectral initialization we get the same consistency benefits as PSRs. We show
that Inference Gradients outperforms both PSRs and PSIMs on real and synthetic
data sets.
Graph Based Over-Segmentation Methods for 3D Point Clouds
Authors: Yizhak Ben-Shabat, Tamar Avraham, Michael Lindenbaum, Anath Fischer
Date: 2017-02-14 00:00:00
Tasks: Segmentation

Over-segmentation, or super-pixel generation, is a common preliminary stage
for many computer vision applications. New acquisition technologies enable the
capturing of 3D point clouds that contain color and geometrical information.
This 3D information introduces a new conceptual change that can be utilized to
improve the results of over-segmentation, which uses mainly color information,
and to generate clusters of points we call super-points. We consider a variety
of possible 3D extensions of the Local Variation (LV) graph based
over-segmentation algorithms, and compare them thoroughly. We consider
different alternatives for constructing the connectivity graph, for assigning
the edge weights, and for defining the merge criterion, which must now account
for the geometric information and not only color. Following this evaluation, we
derive a new generic algorithm for over-segmentation of 3D point clouds. We
call this new algorithm Point Cloud Local Variation (PCLV). The advantages of
the new over-segmentation algorithm are demonstrated on both outdoor and
cluttered indoor scenes. Performance analysis of the proposed approach compared
to state-of-the-art 2D and 3D over-segmentation algorithms shows significant
improvement according to the common performance measures.
Efficient Linear Programming for Dense CRFs
Authors: Thalaiyasingam Ajanthan, Alban Desmaison, Rudy Bunel, Mathieu Salzmann, Philip H. S. Torr, M. Pawan Kumar
Date: 2016-11-29 00:00:00
Proceeding: CVPR 2017 7
Tasks: Semantic Segmentation

The fully connected conditional random field (CRF) with Gaussian pairwise
potentials has proven popular and effective for multi-class semantic
segmentation. While the energy of a dense CRF can be minimized accurately using
a linear programming (LP) relaxation, the state-of-the-art algorithm is too
slow to be useful in practice. To alleviate this deficiency, we introduce an
efficient LP minimization algorithm for dense CRFs. To this end, we develop a
proximal minimization framework, where the dual of each proximal problem is
optimized via block coordinate descent. We show that each block of variables
can be efficiently optimized. Specifically, for one block, the problem
decomposes into significantly smaller subproblems, each of which is defined
over a single pixel. For the other block, the problem is optimized via
conditional gradient descent. This has two advantages: 1) the conditional
gradient can be computed in a time linear in the number of pixels and labels;
and 2) the optimal step size can be computed analytically. Our experiments on
standard datasets provide compelling evidence that our approach outperforms all
existing baselines including the previous LP based approach for dense CRFs.
Supervised Learning of Semantics-Preserving Hash via Deep Convolutional Neural Networks
Authors: Huei-Fang Yang, Kevin Lin, Chu-Song Chen
Date: 2015-07-01 00:00:00
Tasks: Attribute, Classification, Deep Hashing, General Classification, Image Retrieval, Retrieval

This paper presents a simple yet effective supervised deep hash approach that
constructs binary hash codes from labeled data for large-scale image search. We
assume that the semantic labels are governed by several latent attributes with
each attribute on or off, and classification relies on these attributes. Based
on this assumption, our approach, dubbed supervised semantics-preserving deep
hashing (SSDH), constructs hash functions as a latent layer in a deep network
and the binary codes are learned by minimizing an objective function defined
over classification error and other desirable hash codes properties. With this
design, SSDH has a nice characteristic that classification and retrieval are
unified in a single learning model. Moreover, SSDH performs joint learning of
image representations, hash codes, and classification in a point-wised manner,
and thus is scalable to large-scale datasets. SSDH is simple and can be
realized by a slight enhancement of an existing deep architecture for
classification; yet it is effective and outperforms other hashing approaches on
several benchmarks and large datasets. Compared with state-of-the-art
approaches, SSDH achieves higher retrieval accuracy, while the classification
performance is not sacrificed.
JFLEG: A Fluency Corpus and Benchmark for Grammatical Error Correction
Authors: Courtney Napoles, Keisuke Sakaguchi, Joel Tetreault
Date: 2017-02-14 00:00:00
Proceeding: EACL 2017 4
Tasks: Grammatical Error Correction

We present a new parallel corpus, JHU FLuency-Extended GUG corpus (JFLEG) for
developing and evaluating grammatical error correction (GEC). Unlike other
corpora, it represents a broad range of language proficiency levels and uses
holistic fluency edits to not only correct grammatical errors but also make the
original text more native sounding. We describe the types of corrections made
and benchmark four leading GEC systems on this corpus, identifying specific
areas in which they do well and how they can improve. JFLEG fulfills the need
for a new gold standard to properly assess the current state of GEC.
Evolution-Preserving Dense Trajectory Descriptors
Authors: Yang Wang, Vinh Tran, Minh Hoai
Date: 2017-02-14 00:00:00
Tasks: Action Recognition, Deep Learning, Temporal Action Localization

Recently Trajectory-pooled Deep-learning Descriptors were shown to achieve
state-of-the-art human action recognition results on a number of datasets. This
paper improves their performance by applying rank pooling to each trajectory,
encoding the temporal evolution of deep learning features computed along the
trajectory. This leads to Evolution-Preserving Trajectory (EPT) descriptors, a
novel type of video descriptor that significantly outperforms Trajectory-pooled
Deep-learning Descriptors. EPT descriptors are defined based on dense
trajectories, and they provide complimentary benefits to video descriptors that
are not based on trajectories. In particular, we show that the combination of
EPT descriptors and VideoDarwin leads to state-of-the-art performance on
Hollywood2 and UCF101 datasets.
Cutting-off Redundant Repeating Generations for Neural Abstractive Summarization
Authors: Jun Suzuki, Masaaki Nagata
Date: 2016-12-31 00:00:00
Proceeding: EACL 2017 4
Tasks: Abstractive Text Summarization, Decoder

This paper tackles the reduction of redundant repeating generation that is
often observed in RNN-based encoder-decoder models. Our basic idea is to
jointly estimate the upper-bound frequency of each target vocabulary in the
encoder and control the output words based on the estimation in the decoder.
Our method shows significant improvement over a strong RNN-based
encoder-decoder baseline and achieved its best results on an abstractive
summarization benchmark.
Intercomparison of Machine Learning Methods for Statistical Downscaling: The Case of Daily and Extreme Precipitation
Authors: Thomas Vandal, Evan Kodra, Auroop R. Ganguly
Date: 2017-02-13 00:00:00
Tasks: BIG-bench Machine Learning

Statistical downscaling of global climate models (GCMs) allows researchers to
study local climate change effects decades into the future. A wide range of
statistical models have been applied to downscaling GCMs but recent advances in
machine learning have not been explored. In this paper, we compare four
fundamental statistical methods, Bias Correction Spatial Disaggregation (BCSD),
Ordinary Least Squares, Elastic-Net, and Support Vector Machine, with three
more advanced machine learning methods, Multi-task Sparse Structure Learning
(MSSL), BCSD coupled with MSSL, and Convolutional Neural Networks to downscale
daily precipitation in the Northeast United States. Metrics to evaluate of each
method's ability to capture daily anomalies, large scale climate shifts, and
extremes are analyzed. We find that linear methods, led by BCSD, consistently
outperform non-linear approaches. The direct application of state-of-the-art
machine learning methods to statistical downscaling does not provide
improvements over simpler, longstanding approaches.
Is a Data-Driven Approach still Better than Random Choice with Naive Bayes classifiers?
Authors: Piotr Szymański, Tomasz Kajdanowicz
Date: 2017-02-13 00:00:00
Tasks: General Classification, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION

We study the performance of data-driven, a priori and random approaches to
label space partitioning for multi-label classification with a Gaussian Naive
Bayes classifier. Experiments were performed on 12 benchmark data sets and
evaluated on 5 established measures of classification quality: micro and macro
averaged F1 score, Subset Accuracy and Hamming loss. Data-driven methods are
significantly better than an average run of the random baseline. In case of F1
scores and Subset Accuracy - data driven approaches were more likely to perform
better than random approaches than otherwise in the worst case. There always
exists a method that performs better than a priori methods in the worst case.
The advantage of data-driven methods against a priori methods with a weak
classifier is lesser than when tree classifiers are used.
Training Neural Networks Based on Imperialist Competitive Algorithm for Predicting Earthquake Intensity
Authors: Mohsen Moradi
Date: 2017-02-13 00:00:00
Tasks: Earthquake prediction

In this study we determined neural network weights and biases by Imperialist
Competitive Algorithm (ICA) in order to train network for predicting earthquake
intensity in Richter. For this reason, we used dependent parameters like
earthquake occurrence time, epicenter's latitude and longitude in degree, focal
depth in kilometer, and the seismological center distance from epicenter and
earthquake focal center in kilometer which has been provided by Berkeley data
base. The studied neural network has two hidden layer: its first layer has 16
neurons and the second layer has 24 neurons. By using ICA algorithm, average
error for testing data is 0.0007 with a variance equal to 0.318. The earthquake
prediction error in Richter by MSE criteria for ICA algorithm is 0.101, but by
using GA, the MSE value is 0.115.
Correlated signal inference by free energy exploration
Authors: Torsten A. Enßlin, Jakob Knollmüller
Date: 2016-12-26 00:00:00

The inference of correlated signal fields with unknown correlation structures
is of high scientific and technological relevance, but poses significant
conceptual and numerical challenges. To address these, we develop the
correlated signal inference (CSI) algorithm within information field theory
(IFT) and discuss its numerical implementation. To this end, we introduce the
free energy exploration (FrEE) strategy for numerical information field theory
(NIFTy) applications. The FrEE strategy is to let the mathematical structure of
the inference problem determine the dynamics of the numerical solver. FrEE uses
the Gibbs free energy formalism for all involved unknown fields and correlation
structures without marginalization of nuisance quantities. It thereby avoids
the complexity marginalization often impose to IFT equations. FrEE
simultaneously solves for the mean and the uncertainties of signal, nuisance,
and auxiliary fields, while exploiting any analytically calculable quantity.
Finally, FrEE uses a problem specific and self-tuning exploration strategy to
swiftly identify the optimal field estimates as well as their uncertainty maps.
For all estimated fields, properly weighted posterior samples drawn from their
exact, fully non-Gaussian distributions can be generated. Here, we develop the
FrEE strategies for the CSI of a normal, a log-normal, and a Poisson log-normal
IFT signal inference problem and demonstrate their performances via their NIFTy
implementations.
Bayesian Opponent Exploitation in Imperfect-Information Games
Authors: Sam Ganzfried, Qingyun Sun
Date: 2016-03-10 00:00:00

Two fundamental problems in computational game theory are computing a Nash
equilibrium and learning to exploit opponents given observations of their play
(opponent exploitation). The latter is perhaps even more important than the
former: Nash equilibrium does not have a compelling theoretical justification
in game classes other than two-player zero-sum, and for all games one can
potentially do better by exploiting perceived weaknesses of the opponent than
by following a static equilibrium strategy throughout the match. The natural
setting for opponent exploitation is the Bayesian setting where we have a prior
model that is integrated with observations to create a posterior opponent model
that we respond to. The most natural, and a well-studied prior distribution is
the Dirichlet distribution. An exact polynomial-time algorithm is known for
best-responding to the posterior distribution for an opponent assuming a
Dirichlet prior with multinomial sampling in normal-form games; however, for
imperfect-information games the best known algorithm is based on approximating
an infinite integral without theoretical guarantees. We present the first exact
algorithm for a natural class of imperfect-information games. We demonstrate
that our algorithm runs quickly in practice and outperforms the best prior
approaches. We also present an algorithm for the uniform prior setting.
metboost: Exploratory regression analysis with hierarchically clustered data
Authors: Patrick J. Miller, Daniel B. McArtor, Gitta H. Lubke
Date: 2017-02-13 00:00:00
Tasks: Math, Missing Values, Model Selection, regression, Variable Selection

As data collections become larger, exploratory regression analysis becomes
more important but more challenging. When observations are hierarchically
clustered the problem is even more challenging because model selection with
mixed effect models can produce misleading results when nonlinear effects are
not included into the model (Bauer and Cai, 2009). A machine learning method
called boosted decision trees (Friedman, 2001) is a good approach for
exploratory regression analysis in real data sets because it can detect
predictors with nonlinear and interaction effects while also accounting for
missing data. We propose an extension to boosted decision decision trees called
metboost for hierarchically clustered data. It works by constraining the
structure of each tree to be the same across groups, but allowing the terminal
node means to differ. This allows predictors and split points to lead to
different predictions within each group, and approximates nonlinear group
specific effects. Importantly, metboost remains computationally feasible for
thousands of observations and hundreds of predictors that may contain missing
values. We apply the method to predict math performance for 15,240 students
from 751 schools in data collected in the Educational Longitudinal Study 2002
(Ingels et al., 2007), allowing 76 predictors to have unique effects for each
school. When comparing results to boosted decision trees, metboost has 15%
improved prediction performance. Results of a large simulation study show that
metboost has up to 70% improved variable selection performance and up to 30%
improved prediction performance compared to boosted decision trees when group
sizes are small
Unsupervised Classification in Hyperspectral Imagery with Nonlocal Total Variation and Primal-Dual Hybrid Gradient Algorithm
Authors: Wei Zhu, Victoria Chayes, Alexandre Tiard, Stephanie Sanchez, Devin Dahlberg, Andrea L. Bertozzi, Stanley Osher, Dominique Zosso, Da Kuang
Date: 2016-04-27 00:00:00
Tasks: Classification Of Hyperspectral Images, Clustering, General Classification

In this paper, a graph-based nonlocal total variation method (NLTV) is
proposed for unsupervised classification of hyperspectral images (HSI). The
variational problem is solved by the primal-dual hybrid gradient (PDHG)
algorithm. By squaring the labeling function and using a stable simplex
clustering routine, an unsupervised clustering method with random
initialization can be implemented. The effectiveness of this proposed algorithm
is illustrated on both synthetic and real-world HSI, and numerical results show
that the proposed algorithm outperforms other standard unsupervised clustering
methods such as spherical K-means, nonnegative matrix factorization (NMF), and
the graph-based Merriman-Bence-Osher (MBO) scheme.
The Causal Role of Astrocytes in Slow-Wave Rhythmogenesis: A Computational Modelling Study
Authors: Leo Kozachkov, Konstantinos P. Michmizos
Date: 2017-02-13 00:00:00

Finding the origin of slow and infra-slow oscillations could reveal or
explain brain mechanisms in health and disease. Here, we present a
biophysically constrained computational model of a neural network where the
inclusion of astrocytes introduced slow and infra-slow-oscillations, through
two distinct mechanisms. Specifically, we show how astrocytes can modulate the
fast network activity through their slow inter-cellular calcium wave speed and
amplitude and possibly cause the oscillatory imbalances observed in diseases
commonly known for such abnormalities, namely Alzheimer's disease, Parkinson's
disease, epilepsy, depression and ischemic stroke. This work aims to increase
our knowledge on how astrocytes and neurons synergize to affect brain function
and dysfunction.
Sampling-based Gradient Regularization for Capturing Long-Term Dependencies in Recurrent Neural Networks
Authors: Artem Chernodub, Dimitri Nowicki
Date: 2016-06-24 00:00:00

Vanishing (and exploding) gradients effect is a common problem for recurrent
neural networks with nonlinear activation functions which use backpropagation
method for calculation of derivatives. Deep feedforward neural networks with
many hidden layers also suffer from this effect. In this paper we propose a
novel universal technique that makes the norm of the gradient stay in the
suitable range. We construct a way to estimate a contribution of each training
example to the norm of the long-term components of the target function s
gradient. Using this subroutine we can construct mini-batches for the
stochastic gradient descent (SGD) training that leads to high performance and
accuracy of the trained network even for very complex tasks. We provide a
straightforward mathematical estimation of minibatch s impact on for the
gradient norm and prove its correctness theoretically. To check our framework
experimentally we use some special synthetic benchmarks for testing RNNs on
ability to capture long-term dependencies. Our network can detect links between
events in the (temporal) sequence at the range approx. 100 and longer.
End-to-End Interpretation of the French Street Name Signs Dataset
Authors: Raymond Smith, Chunhui Gu, Dar-Shyang Lee, Huiyi Hu, Ranjith Unnikrishnan, Julian Ibarz, Sacha Arnoud, Sophia Lin
Date: 2017-02-13 00:00:00
Tasks: Optical Character Recognition (OCR)

We introduce the French Street Name Signs (FSNS) Dataset consisting of more
than a million images of street name signs cropped from Google Street View
images of France. Each image contains several views of the same street name
sign. Every image has normalized, title case folded ground-truth text as it
would appear on a map. We believe that the FSNS dataset is large and complex
enough to train a deep network of significant complexity to solve the street
name extraction problem "end-to-end" or to explore the design trade-offs
between a single complex engineered network and multiple sub-networks designed
and trained to solve sub-problems. We present such an "end-to-end"
network/graph for Tensor Flow and its results on the FSNS dataset.
The Parallel Meaning Bank: Towards a Multilingual Corpus of Translations Annotated with Compositional Meaning Representations
Authors: Lasha Abzianidze, Johannes Bjerva, Kilian Evang, Hessel Haagsma, Rik van Noord, Pierre Ludmann, Duc-Duy Nguyen, Johan Bos
Date: 2017-02-13 00:00:00
Proceeding: EACL 2017 4

The Parallel Meaning Bank is a corpus of translations annotated with shared,
formal meaning representations comprising over 11 million words divided over
four languages (English, German, Italian, and Dutch). Our approach is based on
cross-lingual projection: automatically produced (and manually corrected)
semantic annotations for English sentences are mapped onto their word-aligned
translations, assuming that the translations are meaning-preserving. The
semantic annotation consists of five main steps: (i) segmentation of the text
in sentences and lexical items; (ii) syntactic parsing with Combinatory
Categorial Grammar; (iii) universal semantic tagging; (iv) symbolization; and
(v) compositional semantic analysis based on Discourse Representation Theory.
These steps are performed using statistical models trained in a semi-supervised
manner. The employed annotation models are all language-neutral. Our first
results are promising.
Learning-based Quantum Robust Control: Algorithm, Applications and Experiments
Authors: Daoyi Dong, Xi Xing, Hailan Ma, Chunlin Chen, Zhixin Liu, Herschel Rabitz
Date: 2017-02-13 00:00:00

Robust control design for quantum systems has been recognized as a key task in quantum information technology, molecular chemistry and atomic physics. In this paper, an improved differential evolution algorithm, referred to as \emph{msMS}\_DE, is proposed to search robust fields for various quantum control problems. In \emph{msMS}\_DE, multiple samples are used for fitness evaluation and a mixed strategy is employed for the mutation operation. In particular, the \emph{msMS}\_DE algorithm is applied to the control problems of (i) open inhomogeneous quantum ensembles and (ii) the consensus goal of a quantum network with uncertainties. Numerical results are presented to demonstrate the excellent performance of the improved machine learning algorithm for these two classes of quantum robust control problems. Furthermore, \emph{msMS}\_DE is experimentally implemented on femtosecond laser control applications to optimize two-photon absorption and control fragmentation of the molecule $\text{CH}_2\text{BrI}$. Experimental results demonstrate excellent performance of \emph{msMS}\_DE in searching for effective femtosecond laser pulses for various tasks.