pwc-archive/papers-with-abstracts huggingface dataset:
Iteratively reweighted adaptive lasso for conditional heteroscedastic time series with applications to AR-ARCH type processes
Authors: Florian Ziel
Date: 2015-02-23 00:00:00
Tasks: Time Series, Time Series Analysis

Shrinkage algorithms are of great importance in almost every area of
statistics due to the increasing impact of big data. Especially time series
analysis benefits from efficient and rapid estimation techniques such as the
lasso. However, currently lasso type estimators for autoregressive time series
models still focus on models with homoscedastic residuals. Therefore, an
iteratively reweighted adaptive lasso algorithm for the estimation of time
series models under conditional heteroscedasticity is presented in a
high-dimensional setting. The asymptotic behaviour of the resulting estimator
is analysed. It is found that the proposed estimation procedure performs
substantially better than its homoscedastic counterpart. A special case of the
algorithm is suitable to compute the estimated multivariate AR-ARCH type models
efficiently. Extensions to the model like periodic AR-ARCH, threshold AR-ARCH
or ARMA-GARCH are discussed. Finally, different simulation results and
applications to electricity market data and returns of metal prices are shown.
Deep Attention Recurrent Q-Network
Authors: Ivan Sorokin, Alexey Seleznev, Mikhail Pavlov, Aleksandr Fedorov, Anastasiia Ignateva
Date: 2015-12-05 00:00:00
Tasks: Atari Games, Deep Attention, Hard Attention, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

A deep learning approach to reinforcement learning led to a general learner
able to train on visual input to play a variety of arcade games at the human
and superhuman levels. Its creators at the Google DeepMind's team called the
approach: Deep Q-Network (DQN). We present an extension of DQN by "soft" and
"hard" attention mechanisms. Tests of the proposed Deep Attention Recurrent
Q-Network (DARQN) algorithm on multiple Atari 2600 games show level of
performance superior to that of DQN. Moreover, built-in attention mechanisms
allow a direct online monitoring of the training process by highlighting the
regions of the game screen the agent is focusing on when making decisions.
Feature Selection for Ridge Regression with Provable Guarantees
Authors: Saurabh Paul, Petros Drineas
Date: 2015-06-17 00:00:00
Tasks: Classification, feature selection, General Classification, regression

We introduce single-set spectral sparsification as a deterministic sampling
based feature selection technique for regularized least squares classification,
which is the classification analogue to ridge regression. The method is
unsupervised and gives worst-case guarantees of the generalization power of the
classification function after feature selection with respect to the
classification function obtained using all features. We also introduce
leverage-score sampling as an unsupervised randomized feature selection method
for ridge regression. We provide risk bounds for both single-set spectral
sparsification and leverage-score sampling on ridge regression in the fixed
design setting and show that the risk in the sampled space is comparable to the
risk in the full-feature space. We perform experiments on synthetic and
real-world datasets, namely a subset of TechTC-300 datasets, to support our
theory. Experimental results indicate that the proposed methods perform better
than the existing feature selection methods.
Maximum Entropy Binary Encoding for Face Template Protection
Authors: Rohit Kumar Pandey, Yingbo Zhou, Bhargava Urala Kota, Venu Govindaraju
Date: 2015-12-05 00:00:00

In this paper we present a framework for secure identification using deep
neural networks, and apply it to the task of template protection for face
authentication. We use deep convolutional neural networks (CNNs) to learn a
mapping from face images to maximum entropy binary (MEB) codes. The mapping is
robust enough to tackle the problem of exact matching, yielding the same code
for new samples of a user as the code assigned during training. These codes are
then hashed using any hash function that follows the random oracle model (like
SHA-512) to generate protected face templates (similar to text based password
protection). The algorithm makes no unrealistic assumptions and offers high
template security, cancelability, and state-of-the-art matching performance.
The efficacy of the approach is shown on CMU-PIE, Extended Yale B, and
Multi-PIE face databases. We achieve high (~95%) genuine accept rates (GAR) at
zero false accept rate (FAR) with up to 1024 bits of template security.
Neural Activation Constellations: Unsupervised Part Model Discovery with Convolutional Networks
Authors: Marcel Simon, Erik Rodner
Date: 2015-04-30 00:00:00
Proceeding: ICCV 2015 12
Tasks: Data Augmentation, Model Discovery

Part models of object categories are essential for challenging recognition
tasks, where differences in categories are subtle and only reflected in
appearances of small parts of the object. We present an approach that is able
to learn part models in a completely unsupervised manner, without part
annotations and even without given bounding boxes during learning. The key idea
is to find constellations of neural activation patterns computed using
convolutional neural networks. In our experiments, we outperform existing
approaches for fine-grained recognition on the CUB200-2011, NA birds, Oxford
PETS, and Oxford Flowers dataset in case no part or bounding box annotations
are available and achieve state-of-the-art performance for the Stanford Dog
dataset. We also show the benefits of neural constellation models as a data
augmentation technique for fine-tuning. Furthermore, our paper unites the areas
of generic and fine-grained classification, since our approach is suitable for
both scenarios. The source code of our method is available online at
http://www.inf-cv.uni-jena.de/part_discovery
A Shapley Value Solution to Game Theoretic-based Feature Reduction in False Alarm Detection
Authors: Fatemeh Afghah, Abolfazl Razi, Kayvan Najarian
Date: 2015-12-05 00:00:00
Tasks: General Classification

False alarm is one of the main concerns in intensive care units and can
result in care disruption, sleep deprivation, and insensitivity of care-givers
to alarms. Several methods have been proposed to suppress the false alarm rate
through improving the quality of physiological signals by filtering, and
developing more accurate sensors. However, significant intrinsic correlation
among the extracted features limits the performance of most currently available
data mining techniques, as they often discard the predictors with low
individual impact that may potentially have strong discriminatory power when
grouped with others. We propose a model based on coalition game theory that
considers the inter-features dependencies in determining the salient predictors
in respect to false alarm, which results in improved classification accuracy.
The superior performance of this method compared to current methods is shown in
simulation results using PhysionNet's MIMIC II database.
A Survey of the Trends in Facial and Expression Recognition Databases and Methods
Authors: Sohini Roychowdhury, Michelle Emmons
Date: 2015-11-07 00:00:00
Tasks: Facial Expression Recognition, Facial Expression Recognition (FER)

Automated facial identification and facial expression recognition have been
topics of active research over the past few decades. Facial and expression
recognition find applications in human-computer interfaces, subject tracking,
real-time security surveillance systems and social networking. Several holistic
and geometric methods have been developed to identify faces and expressions
using public and local facial image databases. In this work we present the
evolution in facial image data sets and the methodologies for facial
identification and recognition of expressions such as anger, sadness,
happiness, disgust, fear and surprise. We observe that most of the earlier
methods for facial and expression recognition aimed at improving the
recognition rates for facial feature-based methods using static images.
However, the recent methodologies have shifted focus towards robust
implementation of facial/expression recognition from large image databases that
vary with space (gathered from the internet) and time (video recordings). The
evolution trends in databases and methodologies for facial and expression
recognition can be useful for assessing the next-generation topics that may
have applications in security systems or personal identification systems that
involve "Quantitative face" assessments.
Stochastic Collapsed Variational Inference for Sequential Data
Authors: Pengyu Wang, Phil Blunsom
Date: 2015-12-05 00:00:00
Tasks: Variational Inference

Stochastic variational inference for collapsed models has recently been
successfully applied to large scale topic modelling. In this paper, we propose
a stochastic collapsed variational inference algorithm in the sequential data
setting. Our algorithm is applicable to both finite hidden Markov models and
hierarchical Dirichlet process hidden Markov models, and to any datasets
generated by emission distributions in the exponential family. Our experiment
results on two discrete datasets show that our inference is both more efficient
and more accurate than its uncollapsed version, stochastic variational
inference.
Stochastic Collapsed Variational Inference for Hidden Markov Models
Authors: Pengyu Wang, Phil Blunsom
Date: 2015-12-05 00:00:00
Tasks: Variational Inference

Stochastic variational inference for collapsed models has recently been
successfully applied to large scale topic modelling. In this paper, we propose
a stochastic collapsed variational inference algorithm for hidden Markov
models, in a sequential data setting. Given a collapsed hidden Markov Model, we
break its long Markov chain into a set of short subchains. We propose a novel
sum-product algorithm to update the posteriors of the subchains, taking into
account their boundary transitions due to the sequential dependencies. Our
experiments on two discrete datasets show that our collapsed algorithm is
scalable to very large datasets, memory efficient and significantly more
accurate than the existing uncollapsed algorithm.
A Picture is Worth a Billion Bits: Real-Time Image Reconstruction from Dense Binary Pixels
Authors: Tal Remez, Or Litany, Alex Bronstein
Date: 2015-10-15 00:00:00
Tasks: Image Reconstruction, Quantization

The pursuit of smaller pixel sizes at ever increasing resolution in digital
image sensors is mainly driven by the stringent price and form-factor
requirements of sensors and optics in the cellular phone market. Recently, Eric
Fossum proposed a novel concept of an image sensor with dense sub-diffraction
limit one-bit pixels jots, which can be considered a digital emulation of
silver halide photographic film. This idea has been recently embodied as the
EPFL Gigavision camera. A major bottleneck in the design of such sensors is the
image reconstruction process, producing a continuous high dynamic range image
from oversampled binary measurements. The extreme quantization of the Poisson
statistics is incompatible with the assumptions of most standard image
processing and enhancement frameworks. The recently proposed maximum-likelihood
(ML) approach addresses this difficulty, but suffers from image artifacts and
has impractically high computational complexity. In this work, we study a
variant of a sensor with binary threshold pixels and propose a reconstruction
algorithm combining an ML data fitting term with a sparse synthesis prior. We
also show an efficient hardware-friendly real-time approximation of this
inverse operator.Promising results are shown on synthetic data as well as on
HDR data emulated using multiple exposures of a regular CMOS sensor.
Spatially Coherent Random Forests
Authors: Tal Remez, Shai Avidan
Date: 2015-11-09 00:00:00
Tasks: Segmentation

Spatially Coherent Random Forest (SCRF) extends Random Forest to create
spatially coherent labeling. Each split function in SCRF is evaluated based on
a traditional information gain measure that is regularized by a spatial
coherency term. This way, SCRF is encouraged to choose split functions that
cluster pixels both in appearance space and in image space. In particular, we
use SCRF to detect contours in images, where contours are taken to be the
boundaries between different regions. Each tree in the forest produces a
segmentation of the image plane and the boundaries of the segmentations of all
trees are aggregated to produce a final hierarchical contour map. We show that
this modification improves the performance of regular Random Forest by about
10% on the standard Berkeley Segmentation Datasets. We believe that SCRF can be
used in other settings as well.
Regularized EM Algorithms: A Unified Framework and Statistical Guarantees
Authors: Xinyang Yi, Constantine Caramanis
Date: 2015-11-27 00:00:00
Proceeding: NeurIPS 2015 12
Tasks: regression

Latent variable models are a fundamental modeling tool in machine learning
applications, but they present significant computational and analytical
challenges. The popular EM algorithm and its variants, is a much used
algorithmic tool; yet our rigorous understanding of its performance is highly
incomplete. Recently, work in Balakrishnan et al. (2014) has demonstrated that
for an important class of problems, EM exhibits linear local convergence. In
the high-dimensional setting, however, the M-step may not be well defined. We
address precisely this setting through a unified treatment using
regularization. While regularization for high-dimensional problems is by now
well understood, the iterative EM algorithm requires a careful balancing of
making progress towards the solution while identifying the right structure
(e.g., sparsity or low-rank). In particular, regularizing the M-step using the
state-of-the-art high-dimensional prescriptions (e.g., Wainwright (2014)) is
not guaranteed to provide this balance. Our algorithm and analysis are linked
in a way that reveals the balance between optimization and statistical errors.
We specialize our general framework to sparse gaussian mixture models,
high-dimensional mixed regression, and regression with missing variables,
obtaining statistical guarantees for each of these examples.
A Deep Structured Model with Radius-Margin Bound for 3D Human Activity Recognition
Authors: Liang Lin, Keze Wang, WangMeng Zuo, Meng Wang, Jiebo Luo, Lei Zhang
Date: 2015-12-05 00:00:00
Tasks: Activity Recognition, Human Activity Recognition

Understanding human activity is very challenging even with the recently
developed 3D/depth sensors. To solve this problem, this work investigates a
novel deep structured model, which adaptively decomposes an activity instance
into temporal parts using the convolutional neural networks (CNNs). Our model
advances the traditional deep learning approaches in two aspects. First, { we
incorporate latent temporal structure into the deep model, accounting for large
temporal variations of diverse human activities. In particular, we utilize the
latent variables to decompose the input activity into a number of temporally
segmented sub-activities, and accordingly feed them into the parts (i.e.
sub-networks) of the deep architecture}. Second, we incorporate a radius-margin
bound as a regularization term into our deep model, which effectively improves
the generalization performance for classification. For model training, we
propose a principled learning algorithm that iteratively (i) discovers the
optimal latent variables (i.e. the ways of activity decomposition) for all
training instances, (ii) { updates the classifiers} based on the generated
features, and (iii) updates the parameters of multi-layer neural networks. In
the experiments, our approach is validated on several complex scenarios for
human activity recognition and demonstrates superior performances over other
state-of-the-art approaches.
Unsupervised comparable corpora preparation and exploration for bi-lingual translation equivalents
Authors: Krzysztof Wołk, Krzysztof Marasek
Date: 2015-12-05 00:00:00
Tasks: GPU, Translation

The multilingual nature of the world makes translation a crucial requirement
today. Parallel dictionaries constructed by humans are a widely-available
resource, but they are limited and do not provide enough coverage for good
quality translation purposes, due to out-of-vocabulary words and neologisms.
This motivates the use of statistical translation systems, which are
unfortunately dependent on the quantity and quality of training data. Such
systems have a very limited availability especially for some languages and very
narrow text domains. In this research we present our improvements to current
comparable corpora mining methodologies by re- implementation of the comparison
algorithms (using Needleman-Wunch algorithm), introduction of a tuning script
and computation time improvement by GPU acceleration. Experiments are carried
out on bilingual data extracted from the Wikipedia, on various domains. For the
Wikipedia itself, additional cross-lingual comparison heuristics were
introduced. The modifications made a positive impact on the quality and
quantity of mined data and on the translation quality.
PJAIT Systems for the IWSLT 2015 Evaluation Campaign Enhanced by Comparable Corpora
Authors: Krzysztof Wołk, Krzysztof Marasek
Date: 2015-12-05 00:00:00
Tasks: Domain Adaptation, Language Modeling, Language Modelling, Machine Translation, Translation, Transliteration, Word Alignment

In this paper, we attempt to improve Statistical Machine Translation (SMT)
systems on a very diverse set of language pairs (in both directions): Czech -
English, Vietnamese - English, French - English and German - English. To
accomplish this, we performed translation model training, created adaptations
of training settings for each language pair, and obtained comparable corpora
for our SMT systems. Innovative tools and data adaptation techniques were
employed. The TED parallel text corpora for the IWSLT 2015 evaluation campaign
were used to train language models, and to develop, tune, and test the system.
In addition, we prepared Wikipedia-based comparable corpora for use with our
SMT system. This data was specified as permissible for the IWSLT 2015
evaluation. We explored the use of domain adaptation techniques, symmetrized
word alignment models, the unsupervised transliteration models and the KenLM
language modeling tool. To evaluate the effects of different preparations on
translation results, we conducted experiments and used the BLEU, NIST and TER
metrics. Our results indicate that our approach produced a positive impact on
SMT quality.
Stochastic Proximal Gradient Descent for Nuclear Norm Regularization
Authors: Lijun Zhang, Tianbao Yang, Rong Jin, Zhi-Hua Zhou
Date: 2015-11-05 00:00:00
Tasks: Stochastic Optimization

In this paper, we utilize stochastic optimization to reduce the space
complexity of convex composite optimization with a nuclear norm regularizer,
where the variable is a matrix of size $m \times n$. By constructing a low-rank
estimate of the gradient, we propose an iterative algorithm based on stochastic
proximal gradient descent (SPGD), and take the last iterate of SPGD as the
final solution. The main advantage of the proposed algorithm is that its space
complexity is $O(m+n)$, in contrast, most of previous algorithms have a $O(mn)$
space complexity. Theoretical analysis shows that it achieves $O(\log
T/\sqrt{T})$ and $O(\log T/T)$ convergence rates for general convex functions
and strongly convex functions, respectively.
On Binary Embedding using Circulant Matrices
Authors: Felix X. Yu, Aditya Bhaskara, Sanjiv Kumar, Yunchao Gong, Shih-Fu Chang
Date: 2015-11-20 00:00:00

Binary embeddings provide efficient and powerful ways to perform operations
on large scale data. However binary embedding typically requires long codes in
order to preserve the discriminative power of the input space. Thus binary
coding methods traditionally suffer from high computation and storage costs in
such a scenario. To address this problem, we propose Circulant Binary Embedding
(CBE) which generates binary codes by projecting the data with a circulant
matrix. The circulant structure allows us to use Fast Fourier Transform
algorithms to speed up the computation. For obtaining $k$-bit binary codes from
$d$-dimensional data, this improves the time complexity from $O(dk)$ to
$O(d\log{d})$, and the space complexity from $O(dk)$ to $O(d)$.
  We study two settings, which differ in the way we choose the parameters of
the circulant matrix. In the first, the parameters are chosen randomly and in
the second, the parameters are learned using the data. For randomized CBE, we
give a theoretical analysis comparing it with binary embedding using an
unstructured random projection matrix. The challenge here is to show that the
dependencies in the entries of the circulant matrix do not lead to a loss in
performance. In the second setting, we design a novel time-frequency
alternating optimization to learn data-dependent circulant projections, which
alternatively minimizes the objective in original and Fourier domains. In both
the settings, we show by extensive experiments that the CBE approach gives much
better performance than the state-of-the-art approaches if we fix a running
time, and provides much faster computation with negligible performance
degradation if we fix the number of bits in the embedding.
Extracting Biomolecular Interactions Using Semantic Parsing of Biomedical Text
Authors: Sahil Garg, Aram Galstyan, Ulf Hermjakob, Daniel Marcu
Date: 2015-12-04 00:00:00
Tasks: Semantic Parsing, Sentence

We advance the state of the art in biomolecular interaction extraction with
three contributions: (i) We show that deep, Abstract Meaning Representations
(AMR) significantly improve the accuracy of a biomolecular interaction
extraction system when compared to a baseline that relies solely on surface-
and syntax-based features; (ii) In contrast with previous approaches that infer
relations on a sentence-by-sentence basis, we expand our framework to enable
consistent predictions over sets of sentences (documents); (iii) We further
modify and expand a graph kernel learning framework to enable concurrent
exploitation of automatically induced AMR (semantic) and dependency structure
(syntactic) representations. Our experiments show that our approach yields
interaction extraction systems that are more robust in environments where there
is a significant mismatch between training and test conditions.
Learning with Group Invariant Features: A Kernel Perspective
Authors: Youssef Mroueh, Stephen Voinea, Tomaso Poggio
Date: 2015-06-08 00:00:00
Proceeding: NeurIPS 2015

We analyze in this paper a random feature map based on a theory of invariance
I-theory introduced recently. More specifically, a group invariant signal
signature is obtained through cumulative distributions of group transformed
random projections. Our analysis bridges invariant feature learning with kernel
methods, as we show that this feature map defines an expected Haar integration
kernel that is invariant to the specified group action. We show how this
non-linear random feature map approximates this group invariant kernel
uniformly on a set of $N$ points. Moreover, we show that it defines a function
space that is dense in the equivalent Invariant Reproducing Kernel Hilbert
Space. Finally, we quantify error rates of the convergence of the empirical
risk minimization, as well as the reduction in the sample complexity of a
learning algorithm using such an invariant representation for signal
classification, in a classical supervised learning setting.