pwc-archive/papers-with-abstracts huggingface dataset:
High-Dimensional Regularized Discriminant Analysis
Authors: John A. Ramey, Caleb K. Stein, Phil D. Young, Dean M. Young
Date: 2016-02-03 00:00:00
Tasks: General Classification, Vocal Bursts Intensity Prediction

Regularized discriminant analysis (RDA), proposed by Friedman (1989), is a
widely popular classifier that lacks interpretability and is impractical for
high-dimensional data sets. Here, we present an interpretable and
computationally efficient classifier called high-dimensional RDA (HDRDA),
designed for the small-sample, high-dimensional setting. For HDRDA, we show
that each training observation, regardless of class, contributes to the class
covariance matrix, resulting in an interpretable estimator that borrows from
the pooled sample covariance matrix. Moreover, we show that HDRDA is equivalent
to a classifier in a reduced-feature space with dimension approximately equal
to the training sample size. As a result, the matrix operations employed by
HDRDA are computationally linear in the number of features, making the
classifier well-suited for high-dimensional classification in practice. We
demonstrate that HDRDA is often superior to several sparse and regularized
classifiers in terms of classification accuracy with three artificial and six
real high-dimensional data sets. Also, timing comparisons between our HDRDA
implementation in the sparsediscrim R package and the standard RDA formulation
in the klaR R package demonstrate that as the number of features increases, the
computational runtime of HDRDA is drastically smaller than that of RDA.
Printed Arabic Text Recognition using Linear and Nonlinear Regression
Authors: Ashraf A. Shahin
Date: 2017-02-05 00:00:00
Tasks: regression

Arabic language is one of the most popular languages in the world. Hundreds
of millions of people in many countries around the world speak Arabic as their
native speaking. However, due to complexity of Arabic language, recognition of
printed and handwritten Arabic text remained untouched for a very long time
compared with English and Chinese. Although, in the last few years, significant
number of researches has been done in recognizing printed and handwritten
Arabic text, it stills an open research field due to cursive nature of Arabic
script. This paper proposes automatic printed Arabic text recognition technique
based on linear and ellipse regression techniques. After collecting all
possible forms of each character, unique code is generated to represent each
character form. Each code contains a sequence of lines and ellipses. To
recognize fonts, a unique list of codes is identified to be used as a
fingerprint of font. The proposed technique has been evaluated using over 14000
different Arabic words with different fonts and experimental results show that
average recognition rate of the proposed technique is 86%.
A dataset and exploration of models for understanding video data through fill-in-the-blank question-answering
Authors: Tegan Maharaj, Nicolas Ballas, Anna Rohrbach, Aaron Courville, Christopher Pal
Date: 2016-11-23 00:00:00
Proceeding: CVPR 2017 7
Tasks: Descriptive, Language Modeling, Language Modelling, object-detection, Object Detection, Question Answering

While deep convolutional neural networks frequently approach or exceed
human-level performance at benchmark tasks involving static images, extending
this success to moving images is not straightforward. Having models which can
learn to understand video is of interest for many applications, including
content recommendation, prediction, summarization, event/object detection and
understanding human visual perception, but many domains lack sufficient data to
explore and perfect video models. In order to address the need for a simple,
quantitative benchmark for developing and understanding video, we present
MovieFIB, a fill-in-the-blank question-answering dataset with over 300,000
examples, based on descriptive video annotations for the visually impaired. In
addition to presenting statistics and a description of the dataset, we perform
a detailed analysis of 5 different models' predictions, and compare these with
human performance. We investigate the relative importance of language, static
(2D) visual features, and moving (3D) visual features; the effects of
increasing dataset size, the number of frames sampled; and of vocabulary size.
We illustrate that: this task is not solvable by a language model alone; our
model combining 2D and 3D visual information indeed provides the best result;
all models perform significantly worse than human-level. We provide human
evaluations for responses given by different models and find that accuracy on
the MovieFIB evaluation corresponds well with human judgement. We suggest
avenues for improving video models, and hope that the proposed dataset can be
useful for measuring and encouraging progress in this very interesting field.
Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering
Authors: Michaël Defferrard, Xavier Bresson, Pierre Vandergheynst
Date: 2016-06-30 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Node Classification, Skeleton Based Action Recognition

In this work, we are interested in generalizing convolutional neural networks
(CNNs) from low-dimensional regular grids, where image, video and speech are
represented, to high-dimensional irregular domains, such as social networks,
brain connectomes or words' embedding, represented by graphs. We present a
formulation of CNNs in the context of spectral graph theory, which provides the
necessary mathematical background and efficient numerical schemes to design
fast localized convolutional filters on graphs. Importantly, the proposed
technique offers the same linear computational complexity and constant learning
complexity as classical CNNs, while being universal to any graph structure.
Experiments on MNIST and 20NEWS demonstrate the ability of this novel deep
learning system to learn local, stationary, and compositional features on
graphs.
Operational Calculus for Differentiable Programming
Authors: Žiga Sajovic, Martin Vuk
Date: 2016-10-25 00:00:00

In this work we present a theoretical model for differentiable programming.
We construct an algebraic language that encapsulates formal semantics of
differentiable programs by way of Operational Calculus. The algebraic nature of
Operational Calculus can alter the properties of the programs that are
expressed within the language and transform them into their solutions.
  In our model programs are elements of programming spaces and viewed as maps
from the virtual memory space to itself. Virtual memory space is an algebra of
programs, an algebraic data structure one can calculate with. We define the
operator of differentiation ($\partial$) on programming spaces and, using its
powers, implement the general shift operator and the operator of program
composition. We provide the formula for the expansion of a differentiable
program into an infinite tensor series in terms of the powers of $\partial$. We
express the operator of program composition in terms of the generalized shift
operator and $\partial$, which implements a differentiable composition in the
language. Such operators serve as abstractions over the tensor series algebra,
as main actors in our language.
  We demonstrate our models usefulness in differentiable programming by using
it to analyse iterators, deriving fractional iterations and their iterating
velocities, and explicitly solve the special case of ReduceSum.
Shape-Based Approach to Household Load Curve Clustering and Prediction
Authors: Thanchanok Teeraratkul, Daniel O'Neill, Sanjay Lall
Date: 2017-02-05 00:00:00
Tasks: Clustering, Dynamic Time Warping

Consumer Demand Response (DR) is an important research and industry problem,
which seeks to categorize, predict and modify consumer's energy consumption.
Unfortunately, traditional clustering methods have resulted in many hundreds of
clusters, with a given consumer often associated with several clusters, making
it difficult to classify consumers into stable representative groups and to
predict individual energy consumption patterns. In this paper, we present a
shape-based approach that better classifies and predicts consumer energy
consumption behavior at the household level. The method is based on Dynamic
Time Warping. DTW seeks an optimal alignment between energy consumption
patterns reflecting the effect of hidden patterns of regular consumer behavior.
Using real consumer 24-hour load curves from Opower Corporation, our method
results in a 50% reduction in the number of representative groups and an
improvement in prediction accuracy measured under DTW distance. We extend the
approach to estimate which electrical devices will be used and in which hours.
A SMART Stochastic Algorithm for Nonconvex Optimization with Applications to Robust Machine Learning
Authors: Aleksandr Aravkin, Damek Davis
Date: 2016-10-04 00:00:00
Tasks: BIG-bench Machine Learning

In this paper, we show how to transform any optimization problem that arises
from fitting a machine learning model into one that (1) detects and removes
contaminated data from the training set while (2) simultaneously fitting the
trimmed model on the uncontaminated data that remains. To solve the resulting
nonconvex optimization problem, we introduce a fast stochastic
proximal-gradient algorithm that incorporates prior knowledge through nonsmooth
regularization. For datasets of size $n$, our approach requires
$O(n^{2/3}/\varepsilon)$ gradient evaluations to reach $\varepsilon$-accuracy
and, when a certain error bound holds, the complexity improves to $O(\kappa
n^{2/3}\log(1/\varepsilon))$. These rates are $n^{1/3}$ times better than those
achieved by typical, full gradient methods.
Language Models with Pre-Trained (GloVe) Word Embeddings
Authors: Victor Makarenkov, Bracha Shapira, Lior Rokach
Date: 2016-10-12 00:00:00
Tasks: Language Modeling, Language Modelling, Word Embeddings

In this work we implement a training of a Language Model (LM), using
Recurrent Neural Network (RNN) and GloVe word embeddings, introduced by
Pennigton et al. in [1]. The implementation is following the general idea of
training RNNs for LM tasks presented in [2], but is rather using Gated
Recurrent Unit (GRU) [3] for a memory cell, and not the more commonly used LSTM
[4].
Multi-feature combined cloud and cloud shadow detection in GaoFen-1 wide field of view imagery
Authors: Zhiwei Li, Huanfeng Shen, Huifang Li, Gui-Song Xia, Paolo Gamba, Liangpei Zhang
Date: 2016-06-17 00:00:00
Tasks: Cloud Detection, Earth Observation, Shadow Detection

The wide field of view (WFV) imaging system onboard the Chinese GaoFen-1
(GF-1) optical satellite has a 16-m resolution and four-day revisit cycle for
large-scale Earth observation. The advantages of the high temporal-spatial
resolution and the wide field of view make the GF-1 WFV imagery very popular.
However, cloud cover is an inevitable problem in GF-1 WFV imagery, which
influences its precise application. Accurate cloud and cloud shadow detection
in GF-1 WFV imagery is quite difficult due to the fact that there are only
three visible bands and one near-infrared band. In this paper, an automatic
multi-feature combined (MFC) method is proposed for cloud and cloud shadow
detection in GF-1 WFV imagery. The MFC algorithm first implements threshold
segmentation based on the spectral features and mask refinement based on guided
filtering to generate a preliminary cloud mask. The geometric features are then
used in combination with the texture features to improve the cloud detection
results and produce the final cloud mask. Finally, the cloud shadow mask can be
acquired by means of the cloud and shadow matching and follow-up correction
process. The method was validated using 108 globally distributed scenes. The
results indicate that MFC performs well under most conditions, and the average
overall accuracy of MFC cloud detection is as high as 96.8%. In the contrastive
analysis with the official provided cloud fractions, MFC shows a significant
improvement in cloud fraction estimation, and achieves a high accuracy for the
cloud and cloud shadow detection in the GF-1 WFV imagery with fewer spectral
bands. The proposed method could be used as a preprocessing step in the future
to monitor land-cover change, and it could also be easily extended to other
optical satellite imagery which has a similar spectral setting.
An Empirical Evaluation of Zero Resource Acoustic Unit Discovery
Authors: Chunxi Liu, Jinyi Yang, Ming Sun, Santosh Kesiraju, Alena Rott, Lucas Ondel, Pegah Ghahremani, Najim Dehak, Lukas Burget, Sanjeev Khudanpur
Date: 2017-02-05 00:00:00
Tasks: Acoustic Unit Discovery

Acoustic unit discovery (AUD) is a process of automatically identifying a
categorical acoustic unit inventory from speech and producing corresponding
acoustic unit tokenizations. AUD provides an important avenue for unsupervised
acoustic model training in a zero resource setting where expert-provided
linguistic knowledge and transcribed speech are unavailable. Therefore, to
further facilitate zero-resource AUD process, in this paper, we demonstrate
acoustic feature representations can be significantly improved by (i)
performing linear discriminant analysis (LDA) in an unsupervised self-trained
fashion, and (ii) leveraging resources of other languages through building a
multilingual bottleneck (BN) feature extractor to give effective cross-lingual
generalization. Moreover, we perform comprehensive evaluations of AUD efficacy
on multiple downstream speech applications, and their correlated performance
suggests that AUD evaluations are feasible using different alternative language
resources when only a subset of these evaluation resources can be available in
typical zero resource applications.
Entropy-guided Retinex anisotropic diffusion algorithm based on partial differential equations (PDE) for illumination correction
Authors: U. A. Nnolim
Date: 2017-02-04 00:00:00

This report describes the experimental results obtained using a proposed
variational Retinex algorithm for controlled illumination correction. Two
colour restoration and enhancement schemes of the algorithm are presented for
drastically improved results. The algorithm modifies the reflectance image
using global and local contrast enhancement approaches and gradually removes
the residual illumination to yield highly pleasing results. The proposed
algorithms are optimized by way of simultaneous perceptual quality metric (PQM)
stabilization and entropy maximization for fully automated processing solving
the problem of determination of stopping time. The usage of the HSI or HSV
colour space ensures a unique solution to the optimization problem unlike in
the RGB space where there is none (forcing manual selection of number of
iteration. The proposed approach preserves and enhances details in both bright
and dark regions of underexposed images in addition to eliminating the colour
distortion, over-exposure in bright image regions, halo effect and grey-world
violations observed in Retinex-based approaches. Extensive experiments indicate
consistent performance as the proposed approach exploits and augments the
advantages of PDE-based formulation, performing illumination correction, colour
enhancement correction and restoration, contrast enhancement and noise
suppression. Comparisons shows that the proposed approach surpasses most of the
other conventional algorithms found in the literature.
An Experimental Study of Deep Convolutional Features For Iris Recognition
Authors: Shervin Minaee, Amirali Abdolrashidi, Yao Wang
Date: 2017-02-04 00:00:00
Tasks: Face Recognition, Iris Recognition, object-detection, Object Detection

Iris is one of the popular biometrics that is widely used for identity
authentication. Different features have been used to perform iris recognition
in the past. Most of them are based on hand-crafted features designed by
biometrics experts. Due to tremendous success of deep learning in computer
vision problems, there has been a lot of interest in applying features learned
by convolutional neural networks on general image recognition to other tasks
such as segmentation, face recognition, and object detection. In this paper, we
have investigated the application of deep features extracted from VGG-Net for
iris recognition. The proposed scheme has been tested on two well-known iris
databases, and has shown promising results with the best accuracy rate of
99.4\%, which outperforms the previous best result.
Manyopt: An Extensible Tool for Mixed, Non-Linear Optimization Through SMT Solving
Authors: Andrea Callia D'Iddio, Michael Huth
Date: 2017-02-04 00:00:00
Tasks: Chemical Process, Management

Optimization of Mixed-Integer Non-Linear Programming (MINLP) supports
important decisions in applications such as Chemical Process Engineering. But
current solvers have limited ability for deductive reasoning or the use of
domain-specific theories, and the management of integrality constraints does
not yet exploit automated reasoning tools such as SMT solvers. This seems to
limit both scalability and reach of such tools in practice. We therefore
present a tool, ManyOpt, for MINLP optimization that enables experimentation
with reduction techniques which transform a MINLP problem to feasibility
checking realized by an SMT solver. ManyOpt is similar to the SAT solver
ManySAT in that it runs a specified number of such reduction techniques in
parallel to get the strongest result on a given MINLP problem. The tool is
implemented in layers, which we may see as features and where reduction
techniques are feature vectors. Some of these features are inspired by known
MINLP techniques whereas others are novel and specific to SMT. Our experimental
results on standard benchmarks demonstrate the benefits of this approach. The
tool supports a variety of SMT solvers and is easily extensible with new
features, courtesy of its layered structure. For example, logical formulas for
deductive reasoning are easily added to constrain further the optimization of a
MINLP problem of interest.
Fast and easy blind deblurring using an inverse filter and PROBE
Authors: Naftali Zon, Rana Hanocka, Nahum Kiryati
Date: 2017-02-04 00:00:00
Tasks: Deblurring

PROBE (Progressive Removal of Blur Residual) is a recursive framework for
blind deblurring. Using the elementary modified inverse filter at its core,
PROBE's experimental performance meets or exceeds the state of the art, both
visually and quantitatively. Remarkably, PROBE lends itself to analysis that
reveals its convergence properties. PROBE is motivated by recent ideas on
progressive blind deblurring, but breaks away from previous research by its
simplicity, speed, performance and potential for analysis. PROBE is neither a
functional minimization approach, nor an open-loop sequential method (blur
kernel estimation followed by non-blind deblurring). PROBE is a feedback
scheme, deriving its unique strength from the closed-loop architecture rather
than from the accuracy of its algorithmic components.
Cluster-based Kriging Approximation Algorithms for Complexity Reduction
Authors: Bas van Stein, Hao Wang, Wojtek Kowalczyk, Michael Emmerich, Thomas Bäck
Date: 2017-02-04 00:00:00
Tasks: regression

Kriging or Gaussian Process Regression is applied in many fields as a
non-linear regression model as well as a surrogate model in the field of
evolutionary computation. However, the computational and space complexity of
Kriging, that is cubic and quadratic in the number of data points respectively,
becomes a major bottleneck with more and more data available nowadays. In this
paper, we propose a general methodology for the complexity reduction, called
cluster Kriging, where the whole data set is partitioned into smaller clusters
and multiple Kriging models are built on top of them. In addition, four Kriging
approximation algorithms are proposed as candidate algorithms within the new
framework. Each of these algorithms can be applied to much larger data sets
while maintaining the advantages and power of Kriging. The proposed algorithms
are explained in detail and compared empirically against a broad set of
existing state-of-the-art Kriging approximation methods on a well-defined
testing framework. According to the empirical study, the proposed algorithms
consistently outperform the existing algorithms. Moreover, some practical
suggestions are provided for using the proposed algorithms.
Improving Efficiency of SVM k-fold Cross-validation by Alpha Seeding
Authors: Zeyi Wen, Bin Li, Rao Kotagiri, Jian Chen, Yawen Chen, Rui Zhang
Date: 2016-11-23 00:00:00

The k-fold cross-validation is commonly used to evaluate the effectiveness of
SVMs with the selected hyper-parameters. It is known that the SVM k-fold
cross-validation is expensive, since it requires training k SVMs. However,
little work has explored reusing the h-th SVM for training the (h+1)-th SVM for
improving the efficiency of k-fold cross-validation. In this paper, we propose
three algorithms that reuse the h-th SVM for improving the efficiency of
training the (h+1)-th SVM. Our key idea is to efficiently identify the support
vectors and to accurately estimate their associated weights (also called alpha
values) of the next SVM by using the previous SVM. Our experimental results
show that our algorithms are several times faster than the k-fold
cross-validation which does not make use of the previously trained SVM.
Moreover, our algorithms produce the same results (hence same accuracy) as the
k-fold cross-validation which does not make use of the previously trained SVM.
Communication-Efficient Algorithms for Decentralized and Stochastic Optimization
Authors: Guanghui Lan, Soomin Lee, Yi Zhou
Date: 2017-01-14 00:00:00
Tasks: Stochastic Optimization

We present a new class of decentralized first-order methods for nonsmooth and
stochastic optimization problems defined over multiagent networks. Considering
that communication is a major bottleneck in decentralized optimization, our
main goal in this paper is to develop algorithmic frameworks which can
significantly reduce the number of inter-node communications. We first propose
a decentralized primal-dual method which can find an $\epsilon$-solution both
in terms of functional optimality gap and feasibility residual in
$O(1/\epsilon)$ inter-node communication rounds when the objective functions
are convex and the local primal subproblems are solved exactly. Our major
contribution is to present a new class of decentralized primal-dual type
algorithms, namely the decentralized communication sliding (DCS) methods, which
can skip the inter-node communications while agents solve the primal
subproblems iteratively through linearizations of their local objective
functions. By employing DCS, agents can still find an $\epsilon$-solution in
$O(1/\epsilon)$ (resp., $O(1/\sqrt{\epsilon})$) communication rounds for
general convex functions (resp., strongly convex functions), while maintaining
the $O(1/\epsilon^2)$ (resp., $O(1/\epsilon)$) bound on the total number of
intra-node subgradient evaluations. We also present a stochastic counterpart
for these algorithms, denoted by SDCS, for solving stochastic optimization
problems whose objective function cannot be evaluated exactly. In comparison
with existing results for decentralized nonsmooth and stochastic optimization,
we can reduce the total number of inter-node communication rounds by orders of
magnitude while still maintaining the optimal complexity bounds on intra-node
stochastic subgradient evaluations. The bounds on the subgradient evaluations
are actually comparable to those required for centralized nonsmooth and
stochastic optimization.