pwc-archive/papers-with-abstracts huggingface dataset:
Syntactic Structures and Code Parameters
Authors: Kevin Shu, Matilde Marcolli
Date: 2016-10-02 00:00:00

We assign binary and ternary error-correcting codes to the data of syntactic
structures of world languages and we study the distribution of code points in
the space of code parameters. We show that, while most codes populate the lower
region approximating a superposition of Thomae functions, there is a
substantial presence of codes above the Gilbert-Varshamov bound and even above
the asymptotic bound and the Plotkin bound. We investigate the dynamics induced
on the space of code parameters by spin glass models of language change, and
show that, in the presence of entailment relations between syntactic parameters
the dynamics can sometimes improve the code. For large sets of languages and
syntactic data, one can gain information on the spin glass dynamics from the
induced dynamics in the space of code parameters.
Deep Feature Consistent Variational Autoencoder
Authors: Xianxu Hou, Linlin Shen, Ke Sun, Guoping Qiu
Date: 2016-10-02 00:00:00
Tasks: Attribute, Style Transfer

We present a novel method for constructing Variational Autoencoder (VAE). Instead of using pixel-by-pixel loss, we enforce deep feature consistency between the input and the output of a VAE, which ensures the VAE's output to preserve the spatial correlation characteristics of the input, thus leading the output to have a more natural visual appearance and better perceptual quality. Based on recent deep learning works such as style transfer, we employ a pre-trained deep convolutional neural network (CNN) and use its hidden features to define a feature perceptual loss for VAE training. Evaluated on the CelebA face dataset, we show that our model produces better results than other methods in the literature. We also show that our method can produce latent vectors that can capture the semantic information of face expressions and can be used to achieve state-of-the-art performance in facial attribute prediction.
Deep Learning Algorithms for Signal Recognition in Long Perimeter Monitoring Distributed Fiber Optic Sensors
Authors: A. V. Makarenko
Date: 2016-10-02 00:00:00
Tasks: Event Detection

In this paper, we show an approach to build deep learning algorithms for
recognizing signals in distributed fiber optic monitoring and security systems
for long perimeters. Synthesizing such detection algorithms poses a non-trivial
research and development challenge, because these systems face stringent error
(type I and II) requirements and operate in difficult signal-jamming
environments, with intensive signal-like jamming and a variety of changing
possible signal portraits of possible recognized events. To address these
issues, we have developed a twolevel event detection architecture, where the
primary classifier is based on an ensemble of deep convolutional networks, can
recognize 7 classes of signals and receives time-space data frames as input.
Using real-life data, we have shown that the applied methods result in
efficient and robust multiclass detection algorithms that have a high degree of
adaptability.
Very Deep Convolutional Neural Networks for Robust Speech Recognition
Authors: Yanmin Qian, Philip C. Woodland
Date: 2016-10-02 00:00:00
Tasks: Robust Speech Recognition, speech-recognition, Speech Recognition

This paper describes the extension and optimization of our previous work on
very deep convolutional neural networks (CNNs) for effective recognition of
noisy speech in the Aurora 4 task. The appropriate number of convolutional
layers, the sizes of the filters, pooling operations and input feature maps are
all modified: the filter and pooling sizes are reduced and dimensions of input
feature maps are extended to allow adding more convolutional layers.
Furthermore appropriate input padding and input feature map selection
strategies are developed. In addition, an adaptation framework using joint
training of very deep CNN with auxiliary features i-vector and fMLLR features
is developed. These modifications give substantial word error rate reductions
over the standard CNN used as baseline. Finally the very deep CNN is combined
with an LSTM-RNN acoustic model and it is shown that state-level weighted log
likelihood score combination in a joint acoustic model decoding scheme is very
effective. On the Aurora 4 task, the very deep CNN achieves a WER of 8.81%,
further 7.99% with auxiliary feature joint training, and 7.09% with LSTM-RNN
joint decoding.
HNP3: A Hierarchical Nonparametric Point Process for Modeling Content Diffusion over Social Media
Authors: Seyed Abbas Hosseini, Ali Khodadadi, Soheil Arabzade, Hamid R. Rabiee
Date: 2016-10-02 00:00:00
Tasks: Point Processes

This paper introduces a novel framework for modeling temporal events with
complex longitudinal dependency that are generated by dependent sources. This
framework takes advantage of multidimensional point processes for modeling time
of events. The intensity function of the proposed process is a mixture of
intensities, and its complexity grows with the complexity of temporal patterns
of data. Moreover, it utilizes a hierarchical dependent nonparametric approach
to model marks of events. These capabilities allow the proposed model to adapt
its temporal and topical complexity according to the complexity of data, which
makes it a suitable candidate for real world scenarios. An online inference
algorithm is also proposed that makes the framework applicable to a vast range
of applications. The framework is applied to a real world application, modeling
the diffusion of contents over networks. Extensive experiments reveal the
effectiveness of the proposed framework in comparison with state-of-the-art
methods.
Deep unsupervised learning through spatial contrasting
Authors: Elad Hoffer, Itay Hubara, Nir Ailon
Date: 2016-10-02 00:00:00

Convolutional networks have marked their place over the last few years as the
best performing model for various visual tasks. They are, however, most suited
for supervised learning from large amounts of labeled data. Previous attempts
have been made to use unlabeled data to improve model performance by applying
unsupervised techniques. These attempts require different architectures and
training methods. In this work we present a novel approach for unsupervised
training of Convolutional networks that is based on contrasting between spatial
regions within images. This criterion can be employed within conventional
neural networks and trained using standard techniques such as SGD and
back-propagation, thus complementing supervised methods.
Text Network Exploration via Heterogeneous Web of Topics
Authors: Junxian He, Ying Huang, Changfeng Liu, Jiaming Shen, Yuting Jia, Xinbing Wang
Date: 2016-10-02 00:00:00

A text network refers to a data type that each vertex is associated with a
text document and the relationship between documents is represented by edges.
The proliferation of text networks such as hyperlinked webpages and academic
citation networks has led to an increasing demand for quickly developing a
general sense of a new text network, namely text network exploration. In this
paper, we address the problem of text network exploration through constructing
a heterogeneous web of topics, which allows people to investigate a text
network associating word level with document level. To achieve this, a
probabilistic generative model for text and links is proposed, where three
different relationships in the heterogeneous topic web are quantified. We also
develop a prototype demo system named TopicAtlas to exhibit such heterogeneous
topic web, and demonstrate how this system can facilitate the task of text
network exploration. Extensive qualitative analyses are included to verify the
effectiveness of this heterogeneous topic web. Besides, we validate our model
on real-life text networks, showing that it preserves good performance on
objective evaluation metrics.
Deep Double Sparsity Encoder: Learning to Sparsify Not Only Features But Also Parameters
Authors: Zhangyang Wang, Thomas S. Huang
Date: 2016-08-23 00:00:00
Tasks: Dictionary Learning

This paper emphasizes the significance to jointly exploit the problem
structure and the parameter structure, in the context of deep modeling. As a
specific and interesting example, we describe the deep double sparsity encoder
(DDSE), which is inspired by the double sparsity model for dictionary learning.
DDSE simultaneously sparsities the output features and the learned model
parameters, under one unified framework. In addition to its intuitive model
interpretation, DDSE also possesses compact model size and low complexity.
Extensive simulations compare DDSE with several carefully-designed baselines,
and verify the consistently superior performance of DDSE. We further apply DDSE
to the novel application domain of brain encoding, with promising preliminary
results achieved.
Two-stage Sampling, Prediction and Adaptive Regression via Correlation Screening (SPARCS)
Authors: Hamed Firouzi, Alfred Hero, Bala Rajaratnam
Date: 2015-02-22 00:00:00
Tasks: Prognosis, regression, Vocal Bursts Valence Prediction

This paper proposes a general adaptive procedure for budget-limited predictor
design in high dimensions called two-stage Sampling, Prediction and Adaptive
Regression via Correlation Screening (SPARCS). SPARCS can be applied to high
dimensional prediction problems in experimental science, medicine, finance, and
engineering, as illustrated by the following. Suppose one wishes to run a
sequence of experiments to learn a sparse multivariate predictor of a dependent
variable $Y$ (disease prognosis for instance) based on a $p$ dimensional set of
independent variables $\mathbf X=[X_1,\ldots, X_p]^T$ (assayed biomarkers).
Assume that the cost of acquiring the full set of variables $\mathbf X$
increases linearly in its dimension. SPARCS breaks the data collection into two
stages in order to achieve an optimal tradeoff between sampling cost and
predictor performance. In the first stage we collect a few ($n$) expensive
samples $\{y_i,\mathbf x_i\}_{i=1}^n$, at the full dimension $p\gg n$ of
$\mathbf X$, winnowing the number of variables down to a smaller dimension $l <
p$ using a type of cross-correlation or regression coefficient screening. In
the second stage we collect a larger number $(t-n)$ of cheaper samples of the
$l$ variables that passed the screening of the first stage. At the second
stage, a low dimensional predictor is constructed by solving the standard
regression problem using all $t$ samples of the selected variables. SPARCS is
an adaptive online algorithm that implements false positive control on the
selected variables, is well suited to small sample sizes, and is scalable to
high dimensions. We establish asymptotic bounds for the Familywise Error Rate
(FWER), specify high dimensional convergence rates for support recovery, and
establish optimal sample allocation rules to the first and second stages.
Tuning parameter calibration for $\ell_1$-regularized logistic regression
Authors: Wei Li, Johannes Lederer
Date: 2016-10-01 00:00:00
Tasks: feature selection, General Classification, regression

Feature selection is a standard approach to understanding and modeling
high-dimensional classification data, but the corresponding statistical methods
hinge on tuning parameters that are difficult to calibrate. In particular,
existing calibration schemes in the logistic regression framework lack any
finite sample guarantees. In this paper, we introduce a novel calibration
scheme for $\ell_1$-penalized logistic regression. It is based on simple tests
along the tuning parameter path and is equipped with optimal guarantees for
feature selection. It is also amenable to easy and efficient implementations,
and it rivals or outmatches existing methods in simulations and real data
applications.
Bacterial Foraging Optimized STATCOM for Stability Assessment in Power System
Authors: Shiba R. Paital, Prakash K. Ray, Asit Mohanty, Sandipan Patra, Harishchandra Dubey
Date: 2016-10-01 00:00:00

This paper presents a study of improvement in stability in a single machine
connected to infinite bus (SMIB) power system by using static compensator
(STATCOM). The gains of Proportional-Integral-Derivative (PID) controller in
STATCOM are being optimized by heuristic technique based on Particle swarm
optimization (PSO). Further, Bacterial Foraging Optimization (BFO) as an
alternative heuristic method is also applied to select optimal gains of PID
controller. The performance of STATCOM with the above soft-computing techniques
are studied and compared with the conventional PID controller under various
scenarios. The simulation results are accompanied with performance indices
based quantitative analysis. The analysis clearly signifies the robustness of
the new scheme in terms of stability and voltage regulation when compared with
conventional PID.
Provable Burer-Monteiro factorization for a class of norm-constrained matrix problems
Authors: Dohyung Park, Anastasios Kyrillidis, Srinadh Bhojanapalli, Constantine Caramanis, Sujay Sanghavi
Date: 2016-06-04 00:00:00
Tasks: LEMMA, Quantum State Tomography, Retrieval

We study the projected gradient descent method on low-rank matrix problems
with a strongly convex objective. We use the Burer-Monteiro factorization
approach to implicitly enforce low-rankness; such factorization introduces
non-convexity in the objective. We focus on constraint sets that include both
positive semi-definite (PSD) constraints and specific matrix norm-constraints.
Such criteria appear in quantum state tomography and phase retrieval
applications.
  We show that non-convex projected gradient descent favors local linear
convergence in the factored space. We build our theory on a novel descent
lemma, that non-trivially extends recent results on the unconstrained problem.
The resulting algorithm is Projected Factored Gradient Descent, abbreviated as
ProjFGD, and shows superior performance compared to state of the art on quantum
state tomography and sparse phase retrieval applications.
Convergence of a Grassmannian Gradient Descent Algorithm for Subspace Estimation From Undersampled Data
Authors: Dejiao Zhang, Laura Balzano
Date: 2016-10-01 00:00:00

Subspace learning and matrix factorization problems have great many applications in science and engineering, and efficient algorithms are critical as dataset sizes continue to grow. Many relevant problem formulations are non-convex, and in a variety of contexts it has been observed that solving the non-convex problem directly is not only efficient but reliably accurate. We discuss convergence theory for a particular method: first order incremental gradient descent constrained to the Grassmannian. The output of the algorithm is an orthonormal basis for a $d$-dimensional subspace spanned by an input streaming data matrix. We study two sampling cases: where each data vector of the streaming matrix is fully sampled, or where it is undersampled by a sampling matrix $A_t\in \mathbb{R}^{m\times n}$ with $m\ll n$. Our results cover two cases, where $A_t$ is Gaussian or a subset of rows of the identity matrix. We propose an adaptive stepsize scheme that depends only on the sampled data and algorithm outputs. We prove that with fully sampled data, the stepsize scheme maximizes the improvement of our convergence metric at each iteration, and this method converges from any random initialization to the true subspace, despite the non-convex formulation and orthogonality constraints. For the case of undersampled data, we establish monotonic expected improvement on the defined convergence metric for each iteration with high probability.
A Birth and Death Process for Bayesian Network Structure Inference
Authors: D. Jennings, J. N. Corcoran
Date: 2016-10-01 00:00:00

Bayesian networks (BNs) are graphical models that are useful for representing
high-dimensional probability distributions. There has been a great deal of
interest in recent years in the NP-hard problem of learning the structure of a
BN from observed data. Typically, one assigns a score to various structures and
the search becomes an optimization problem that can be approached with either
deterministic or stochastic methods. In this paper, we walk through the space
of graphs by modeling the appearance and disappearance of edges as a birth and
death process and compare our novel approach to the popular Metropolis-Hastings
search strategy. We give empirical evidence that the birth and death process
has superior mixing properties.
Near-Infrared Image Dehazing Via Color Regularization
Authors: Chang-Hwan Son, Xiao-Ping Zhang
Date: 2016-10-01 00:00:00
Tasks: Image Dehazing

Near-infrared imaging can capture haze-free near-infrared gray images and
visible color images, according to physical scattering models, e.g., Rayleigh
or Mie models. However, there exist serious discrepancies in brightness and
image structures between the near-infrared gray images and the visible color
images. The direct use of the near-infrared gray images brings about another
color distortion problem in the dehazed images. Therefore, the color distortion
should also be considered for near-infrared dehazing. To reflect this point,
this paper presents an approach of adding a new color regularization to
conventional dehazing framework. The proposed color regularization can model
the color prior for unknown haze-free images from two captured images. Thus,
natural-looking colors and fine details can be induced on the dehazed images.
The experimental results show that the proposed color regularization model can
help remove the color distortion and the haze at the same time. Also, the
effectiveness of the proposed color regularization is verified by comparing
with other conventional regularizations. It is also shown that the proposed
color regularization can remove the edge artifacts which arise from the use of
the conventional dark prior model.
TweeTime: A Minimally Supervised Method for Recognizing and Normalizing Time Expressions in Twitter
Authors: Jeniya Tabassum, Alan Ritter, Wei Xu
Date: 2016-08-09 00:00:00

We describe TweeTIME, a temporal tagger for recognizing and normalizing time expressions in Twitter. Most previous work in social media analysis has to rely on temporal resolvers that are designed for well-edited text, and therefore suffer from the reduced performance due to domain mismatch. We present a minimally supervised method that learns from large quantities of unlabeled data and requires no hand-engineered rules or hand-annotated training corpora. TweeTIME achieves 0.68 F1 score on the end-to-end task of resolving date expressions, outperforming a broad range of state-of-the-art systems.
Automated Visual Fin Identification of Individual Great White Sharks
Authors: Benjamin Hughes, Tilo Burghardt
Date: 2016-09-20 00:00:00

This paper discusses the automated visual identification of individual great
white sharks from dorsal fin imagery. We propose a computer vision photo ID
system and report recognition results over a database of thousands of
unconstrained fin images. To the best of our knowledge this line of work
establishes the first fully automated contour-based visual ID system in the
field of animal biometrics. The approach put forward appreciates shark fins as
textureless, flexible and partially occluded objects with an individually
characteristic shape. In order to recover animal identities from an image we
first introduce an open contour stroke model, which extends multi-scale region
segmentation to achieve robust fin detection. Secondly, we show that
combinatorial, scale-space selective fingerprinting can successfully encode fin
individuality. We then measure the species-specific distribution of visual
individuality along the fin contour via an embedding into a global `fin space'.
Exploiting this domain, we finally propose a non-linear model for individual
animal recognition and combine all approaches into a fine-grained
multi-instance framework. We provide a system evaluation, compare results to
prior work, and report performance and properties in detail.
Towards Deep Symbolic Reinforcement Learning
Authors: Marta Garnelo, Kai Arulkumaran, Murray Shanahan
Date: 2016-09-18 00:00:00
Tasks: Deep Reinforcement Learning, Game of Go, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Transfer Learning

Deep reinforcement learning (DRL) brings the power of deep neural networks to
bear on the generic task of trial-and-error learning, and its effectiveness has
been convincingly demonstrated on tasks such as Atari video games and the game
of Go. However, contemporary DRL systems inherit a number of shortcomings from
the current generation of deep learning techniques. For example, they require
very large datasets to work effectively, entailing that they are slow to learn
even when such datasets are available. Moreover, they lack the ability to
reason on an abstract level, which makes it difficult to implement high-level
cognitive functions such as transfer learning, analogical reasoning, and
hypothesis-based reasoning. Finally, their operation is largely opaque to
humans, rendering them unsuitable for domains in which verifiability is
important. In this paper, we propose an end-to-end reinforcement learning
architecture comprising a neural back end and a symbolic front end with the
potential to overcome each of these shortcomings. As proof-of-concept, we
present a preliminary implementation of the architecture and apply it to
several variants of a simple video game. We show that the resulting system --
though just a prototype -- learns effectively, and, by acquiring a set of
symbolic rules that are easily comprehensible to humans, dramatically
outperforms a conventional, fully neural DRL system on a stochastic variant of
the game.
How Transferable are CNN-based Features for Age and Gender Classification?
Authors: Gökhan Özbulak, Yusuf Aytar, Hazim Kemal Ekenel
Date: 2016-10-01 00:00:00
Tasks: Age And Gender Classification, Classification, Face Recognition, Gender Classification, General Classification

Age and gender are complementary soft biometric traits for face recognition.
Successful estimation of age and gender from facial images taken under
real-world conditions can contribute improving the identification results in
the wild. In this study, in order to achieve robust age and gender
classification in the wild, we have benefited from Deep Convolutional Neural
Networks based representation. We have explored transferability of existing
deep convolutional neural network (CNN) models for age and gender
classification. The generic AlexNet-like architecture and domain specific
VGG-Face CNN model are employed and fine-tuned with the Adience dataset
prepared for age and gender classification in uncontrolled environments. In
addition, task specific GilNet CNN model has also been utilized and used as a
baseline method in order to compare with transferred models. Experimental
results show that both transferred deep CNN models outperform the GilNet CNN
model, which is the state-of-the-art age and gender classification approach on
the Adience dataset, by an absolute increase of 7% and 4.5% in accuracy,
respectively. This outcome indicates that transferring a deep CNN model can
provide better classification performance than a task specific CNN model, which
has a limited number of layers and trained from scratch using a limited amount
of data as in the case of GilNet. Domain specific VGG-Face CNN model has been
found to be more useful and provided better performance for both age and gender
classification tasks, when compared with generic AlexNet-like model, which
shows that transfering from a closer domain is more useful.