pwc-archive/papers-with-abstracts huggingface dataset:
Multi30K: Multilingual English-German Image Descriptions
Authors: Desmond Elliott, Stella Frank, Khalil Sima'an, Lucia Specia
Date: 2016-05-02 00:00:00
Proceeding: WS 2016 8
Tasks: Image Description, Machine Translation, Multimodal Machine Translation, Translation

We introduce the Multi30K dataset to stimulate multilingual multimodal
research. Recent advances in image description have been demonstrated on
English-language datasets almost exclusively, but image description should not
be limited to English. This dataset extends the Flickr30K dataset with i)
German translations created by professional translators over a subset of the
English descriptions, and ii) descriptions crowdsourced independently of the
original English descriptions. We outline how the data can be used for
multilingual image description and multimodal machine translation, but we
anticipate the data will be useful for a broader range of tasks.
Fourier Analysis and q-Gaussian Functions: Analytical and Numerical Results
Authors: Paulo Sérgio Silva Rodrigues, Gilson Antonio Giraldi
Date: 2016-05-02 00:00:00

It is a consensus in signal processing that the Gaussian kernel and its
partial derivatives enable the development of robust algorithms for feature
detection. Fourier analysis and convolution theory have central role in such
development. In this paper we collect theoretical elements to follow this
avenue but using the q-Gaussian kernel that is a nonextensive generalization of
the Gaussian one. Firstly, we review some theoretical elements behind the
one-dimensional q-Gaussian and its Fourier transform. Then, we consider the
two-dimensional q-Gaussian and we highlight the issues behind its analytical
Fourier transform computation. We analyze the q-Gaussian kernel in the space
and Fourier domains using the concepts of space window, cut-off frequency, and
the Heisenberg inequality.
An Enhanced Harmony Search Method for Bangla Handwritten Character Recognition Using Region Sampling
Authors: Ritesh Sarkhel, Amit K Saha, Nibaran Das
Date: 2016-05-02 00:00:00
Tasks: Descriptive

Identification of minimum number of local regions of a handwritten character
image, containing well-defined discriminating features which are sufficient for
a minimal but complete description of the character is a challenging task. A
new region selection technique based on the idea of an enhanced Harmony Search
methodology has been proposed here. The powerful framework of Harmony Search
has been utilized to search the region space and detect only the most
informative regions for correctly recognizing the handwritten character. The
proposed method has been tested on handwritten samples of Bangla Basic,
Compound and mixed (Basic and Compound characters) characters separately with
SVM based classifier using a longest run based feature-set obtained from the
image subregions formed by a CG based quad-tree partitioning approach. Applying
this methodology on the above mentioned three types of datasets, respectively
43.75%, 12.5% and 37.5% gains have been achieved in terms of region reduction
and 2.3%, 0.6% and 1.2% gains have been achieved in terms of recognition
accuracy. The results show a sizeable reduction in the minimal number of
descriptive regions as well a significant increase in recognition accuracy for
all the datasets using the proposed technique. Thus the time and cost related
to feature extraction is decreased without dampening the corresponding
recognition accuracy.
Simple2Complex: Global Optimization by Gradient Descent
Authors: Ming Li
Date: 2016-05-02 00:00:00
Tasks: global-optimization

A method named simple2complex for modeling and training deep neural networks
is proposed. Simple2complex train deep neural networks by smoothly adding more
and more layers to the shallow networks, as the learning procedure going on,
the network is just like growing. Compared with learning by end2end,
simple2complex is with less possibility trapping into local minimal, namely,
owning ability for global optimization. Cifar10 is used for verifying the
superiority of simple2complex.
Revise Saturated Activation Functions
Authors: Bing Xu, Ruitong Huang, Mu Li
Date: 2016-02-18 00:00:00

In this paper, we revise two commonly used saturated functions, the logistic
sigmoid and the hyperbolic tangent (tanh).
  We point out that, besides the well-known non-zero centered property, slope
of the activation function near the origin is another possible reason making
training deep networks with the logistic function difficult to train. We
demonstrate that, with proper rescaling, the logistic sigmoid achieves
comparable results with tanh.
  Then following the same argument, we improve tahn by penalizing in the
negative part. We show that "penalized tanh" is comparable and even outperforms
the state-of-the-art non-saturated functions including ReLU and leaky ReLU on
deep convolution neural networks.
  Our results contradict to the conclusion of previous works that the
saturation property causes the slow convergence. It suggests further
investigation is necessary to better understand activation functions in deep
architectures.
Revisiting Human Action Recognition: Personalization vs. Generalization
Authors: Andrea Zunino, Jacopo Cavazza, Vittorio Murino
Date: 2016-05-02 00:00:00
Tasks: Action Classification, Action Recognition, General Classification, Temporal Action Localization

By thoroughly revisiting the classic human action recognition paradigm, this
paper aims at proposing a new approach for the design of effective action
classification systems. Taking as testbed publicly available three-dimensional
(MoCap) action/activity datasets, we analyzed and validated different
training/testing strategies. In particular, considering that each human action
in the datasets is performed several times by different subjects, we were able
to precisely quantify the effect of inter- and intra-subject variability, so as
to figure out the impact of several learning approaches in terms of
classification performance. The net result is that standard testing strategies
consisting in cross-validating the algorithm using typical splits of the data
(holdout, k-fold, or one-subject-out) is always outperformed by a
"personalization" strategy which learns how a subject is performing an action.
In other words, it is advantageous to customize (i.e., personalize) the method
to learn the actions carried out by each subject, rather than trying to
generalize the actions executions across subjects. Consequently, we finally
propose an action recognition framework consisting of a two-stage
classification approach where, given a test action, the subject is first
identified before the actual recognition of the action takes place. Despite the
basic, off-the-shelf descriptors and standard classifiers adopted, we noted a
relevant increase in performance with respect to standard state-of-the-art
algorithms, so motivating the usage of personalized approaches for designing
effective action recognition systems.
Compression Artifacts Removal Using Convolutional Neural Networks
Authors: Pavel Svoboda, Michal Hradis, David Barina, Pavel Zemcik
Date: 2016-05-02 00:00:00

This paper shows that it is possible to train large and deep convolutional
neural networks (CNN) for JPEG compression artifacts reduction, and that such
networks can provide significantly better reconstruction quality compared to
previously used smaller networks as well as to any other state-of-the-art
methods. We were able to train networks with 8 layers in a single step and in
relatively short time by combining residual learning, skip architecture, and
symmetric weight initialization. We provide further insights into convolution
networks for JPEG artifact reduction by evaluating three different objectives,
generalization with respect to training dataset size, and generalization with
respect to JPEG quality level.
Contrastive Structured Anomaly Detection for Gaussian Graphical Models
Authors: Abhinav Maurya, Mark Cheung
Date: 2016-05-02 00:00:00
Tasks: Anomaly Detection

Gaussian graphical models (GGMs) are probabilistic tools of choice for
analyzing conditional dependencies between variables in complex systems.
Finding changepoints in the structural evolution of a GGM is therefore
essential to detecting anomalies in the underlying system modeled by the GGM.
In order to detect structural anomalies in a GGM, we consider the problem of
estimating changes in the precision matrix of the corresponding Gaussian
distribution. We take a two-step approach to solving this problem:- (i)
estimating a background precision matrix using system observations from the
past without any anomalies, and (ii) estimating a foreground precision matrix
using a sliding temporal window during anomaly monitoring. Our primary
contribution is in estimating the foreground precision using a novel
contrastive inverse covariance estimation procedure. In order to accurately
learn only the structural changes to the GGM, we maximize a penalized
log-likelihood where the penalty is the $l_1$ norm of difference between the
foreground precision being estimated and the already learned background
precision. We modify the alternating direction method of multipliers (ADMM)
algorithm for sparse inverse covariance estimation to perform contrastive
estimation of the foreground precision matrix. Our results on simulated GGM
data show significant improvement in precision and recall for detecting
structural changes to the GGM, compared to a non-contrastive sliding window
baseline.
Predicting distributions with Linearizing Belief Networks
Authors: Yann N. Dauphin, David Grangier
Date: 2015-11-17 00:00:00
Tasks: Denoising, Facial expression generation, Image Denoising, text-to-speech, Text to Speech, valid

Conditional belief networks introduce stochastic binary variables in neural
networks. Contrary to a classical neural network, a belief network can predict
more than the expected value of the output $Y$ given the input $X$. It can
predict a distribution of outputs $Y$ which is useful when an input can admit
multiple outputs whose average is not necessarily a valid answer. Such networks
are particularly relevant to inverse problems such as image prediction for
denoising, or text to speech. However, traditional sigmoid belief networks are
hard to train and are not suited to continuous problems. This work introduces a
new family of networks called linearizing belief nets or LBNs. A LBN decomposes
into a deep linear network where each linear unit can be turned on or off by
non-deterministic binary latent units. It is a universal approximator of
real-valued conditional distributions and can be trained using gradient
descent. Moreover, the linear pathways efficiently propagate continuous
information and they act as multiplicative skip-connections that help
optimization by removing gradient diffusion. This yields a model which trains
efficiently and improves the state-of-the-art on image denoising and facial
expression generation with the Toronto faces dataset.
Revolvable Indoor Panoramas Using a Rectified Azimuthal Projection
Authors: Chamberlain Fong
Date: 2012-06-10 00:00:00

We present an algorithm for converting an indoor spherical panorama into a
photograph with a simulated overhead view. The resulting image will have an
extremely wide field of view covering up to 4{\pi} steradians of the spherical
panorama. We argue that our method complements the stereographic projection
commonly used in the "little planet" effect. The stereographic projection works
well in creating little planets of outdoor scenes; whereas our method is a
well-suited counterpart for indoor scenes. The main innovation of our method is
the introduction of a novel azimuthal map projection that can smoothly blend
between the stereographic projection and the Lambert azimuthal equal-area
projection. Our projection has an adjustable parameter that allows one to
control and compromise between distortions in shape and distortions in size
within the projected panorama. This extra control parameter gives our
projection the ability to produce superior results over the stereographic
projection.
Some Insights into the Geometry and Training of Neural Networks
Authors: Ewout van den Berg
Date: 2016-05-02 00:00:00
Tasks: Classification, General Classification

Neural networks have been successfully used for classification tasks in a
rapidly growing number of practical applications. Despite their popularity and
widespread use, there are still many aspects of training and classification
that are not well understood. In this paper we aim to provide some new insights
into training and classification by analyzing neural networks from a
feature-space perspective. We review and explain the formation of decision
regions and study some of their combinatorial aspects. We place a particular
emphasis on the connections between the neural network weight and bias terms
and properties of decision boundaries and other regions that exhibit varying
levels of classification confidence. We show how the error backpropagates in
these regions and emphasize the important role they have in the formation of
gradients. These findings expose the connections between scaling of the weight
parameters and the density of the training samples. This sheds more light on
the vanishing gradient problem, explains the need for regularization, and
suggests an approach for subsampling training data to improve performance.
Large-scale Binary Quadratic Optimization Using Semidefinite Relaxation and Applications
Authors: Peng Wang, Chunhua Shen, Anton Van Den Hengel, Philip H. S. Torr
Date: 2014-11-27 00:00:00
Tasks: Clustering, Image Segmentation, Scene Parsing, Semantic Segmentation

In computer vision, many problems such as image segmentation, pixel
labelling, and scene parsing can be formulated as binary quadratic programs
(BQPs). For submodular problems, cuts based methods can be employed to
efficiently solve large-scale problems. However, general nonsubmodular problems
are significantly more challenging to solve. Finding a solution when the
problem is of large size to be of practical interest, however, typically
requires relaxation. Two standard relaxation methods are widely used for
solving general BQPs--spectral methods and semidefinite programming (SDP), each
with their own advantages and disadvantages. Spectral relaxation is simple and
easy to implement, but its bound is loose. Semidefinite relaxation has a
tighter bound, but its computational complexity is high, especially for large
scale problems. In this work, we present a new SDP formulation for BQPs, with
two desirable properties. First, it has a similar relaxation bound to
conventional SDP formulations. Second, compared with conventional SDP methods,
the new SDP formulation leads to a significantly more efficient and scalable
dual optimization approach, which has the same degree of complexity as spectral
methods. We then propose two solvers, namely, quasi-Newton and smoothing Newton
methods, for the dual problem. Both of them are significantly more efficiently
than standard interior-point methods. In practice, the smoothing Newton solver
is faster than the quasi-Newton solver for dense or medium-sized problems,
while the quasi-Newton solver is preferable for large sparse/structured
problems. Our experiments on a few computer vision applications including
clustering, image segmentation, co-segmentation and registration show the
potential of our SDP formulation for solving large-scale BQPs.
Dominant Codewords Selection with Topic Model for Action Recognition
Authors: Hirokatsu Kataoka, Masaki Hayashi, Kenji Iwata, Yutaka Satoh, Yoshimitsu Aoki, Slobodan Ilic
Date: 2016-05-01 00:00:00
Tasks: Action Recognition, Temporal Action Localization

In this paper, we propose a framework for recognizing human activities that
uses only in-topic dominant codewords and a mixture of intertopic vectors.
Latent Dirichlet allocation (LDA) is used to develop approximations of human
motion primitives; these are mid-level representations, and they adaptively
integrate dominant vectors when classifying human activities. In LDA topic
modeling, action videos (documents) are represented by a bag-of-words (input
from a dictionary), and these are based on improved dense trajectories. The
output topics correspond to human motion primitives, such as finger moving or
subtle leg motion. We eliminate the impurities, such as missed tracking or
changing light conditions, in each motion primitive. The assembled vector of
motion primitives is an improved representation of the action. We demonstrate
our method on four different datasets.
Directional Statistics in Machine Learning: a Brief Review
Authors: Suvrit Sra
Date: 2016-05-01 00:00:00
Tasks: BIG-bench Machine Learning

The modern data analyst must cope with data encoded in various forms,
vectors, matrices, strings, graphs, or more. Consequently, statistical and
machine learning models tailored to different data encodings are important. We
focus on data encoded as normalized vectors, so that their "direction" is more
important than their magnitude. Specifically, we consider high-dimensional
vectors that lie either on the surface of the unit hypersphere or on the real
projective plane. For such data, we briefly review common mathematical models
prevalent in machine learning, while also outlining some technical aspects,
software, applications, and open mathematical challenges.
A Self-Taught Artificial Agent for Multi-Physics Computational Model Personalization
Authors: Dominik Neumann, Tommaso Mansi, Lucian Itu, Bogdan Georgescu, Elham Kayvanpour, Farbod Sedaghat-Hamedani, Ali Amr, Jan Haas, Hugo Katus, Benjamin Meder, Stefan Steidl, Joachim Hornegger, Dorin Comaniciu
Date: 2016-05-01 00:00:00
Tasks: Reinforcement Learning

Personalization is the process of fitting a model to patient data, a critical
step towards application of multi-physics computational models in clinical
practice. Designing robust personalization algorithms is often a tedious,
time-consuming, model- and data-specific process. We propose to use artificial
intelligence concepts to learn this task, inspired by how human experts
manually perform it. The problem is reformulated in terms of reinforcement
learning. In an off-line phase, Vito, our self-taught artificial agent, learns
a representative decision process model through exploration of the
computational model: it learns how the model behaves under change of
parameters. The agent then automatically learns an optimal strategy for on-line
personalization. The algorithm is model-independent; applying it to a new model
requires only adjusting few hyper-parameters of the agent and defining the
observations to match. The full knowledge of the model itself is not required.
Vito was tested in a synthetic scenario, showing that it could learn how to
optimize cost functions generically. Then Vito was applied to the inverse
problem of cardiac electrophysiology and the personalization of a whole-body
circulation model. The obtained results suggested that Vito could achieve
equivalent, if not better goodness of fit than standard methods, while being
more robust (up to 11% higher success rates) and with faster (up to seven
times) convergence rate. Our artificial intelligence approach could thus make
personalization algorithms generalizable and self-adaptable to any patient and
any model.
Robust Elastic Net Regression
Authors: Weiyang Liu, Rongmei Lin, Meng Yang
Date: 2015-11-15 00:00:00
Tasks: regression

We propose a robust elastic net (REN) model for high-dimensional sparse
regression and give its performance guarantees (both the statistical error
bound and the optimization bound). A simple idea of trimming the inner product
is applied to the elastic net model. Specifically, we robustify the covariance
matrix by trimming the inner product based on the intuition that the trimmed
inner product can not be significant affected by a bounded number of
arbitrarily corrupted points (outliers). The REN model can also derive two
interesting special cases: robust Lasso and robust soft thresholding.
Comprehensive experimental results show that the robustness of the proposed
model consistently outperforms the original elastic net and matches the
performance guarantees nicely.
MODS: Fast and Robust Method for Two-View Matching
Authors: Dmytro Mishkin, Jiri Matas, Michal Perdoch
Date: 2015-03-09 00:00:00
Tasks: Vocal Bursts Valence Prediction

A novel algorithm for wide-baseline matching called MODS - Matching On Demand
with view Synthesis - is presented. The MODS algorithm is experimentally shown
to solve a broader range of wide-baseline problems than the state of the art
while being nearly as fast as standard matchers on simple problems. The
apparent robustness vs. speed trade-off is finessed by the use of progressively
more time-consuming feature detectors and by on-demand generation of
synthesized images that is performed until a reliable estimate of geometry is
obtained.
  We introduce an improved method for tentative correspondence selection,
applicable both with and without view synthesis. A modification of the standard
first to second nearest distance rule increases the number of correct matches
by 5-20% at no additional computational cost.
  Performance of the MODS algorithm is evaluated on several standard publicly
available datasets, and on a new set of geometrically challenging wide baseline
problems that is made public together with the ground truth. Experiments show
that the MODS outperforms the state-of-the-art in robustness and speed.
Moreover, MODS performs well on other classes of difficult two-view problems
like matching of images from different modalities, with wide temporal baseline
or with significant lighting changes.
A vector-contraction inequality for Rademacher complexities
Authors: Andreas Maurer
Date: 2016-05-01 00:00:00
Tasks: Clustering

The contraction inequality for Rademacher averages is extended to Lipschitz
functions with vector-valued domains, and it is also shown that in the bounding
expression the Rademacher variables can be replaced by arbitrary iid symmetric
and sub-gaussian variables. Example applications are given for multi-category
learning, K-means clustering and learning-to-learn.
Clustering Markov Decision Processes For Continual Transfer
Authors: M. M. Hassan Mahmud, Majd Hawasly, Benjamin Rosman, Subramanian Ramamoorthy
Date: 2013-11-15 00:00:00
Tasks: Clustering, Reinforcement Learning, Transfer Learning

We present algorithms to effectively represent a set of Markov decision
processes (MDPs), whose optimal policies have already been learned, by a
smaller source subset for lifelong, policy-reuse-based transfer learning in
reinforcement learning. This is necessary when the number of previous tasks is
large and the cost of measuring similarity counteracts the benefit of transfer.
The source subset forms an `$\epsilon$-net' over the original set of MDPs, in
the sense that for each previous MDP $M_p$, there is a source $M^s$ whose
optimal policy has $<\epsilon$ regret in $M_p$. Our contributions are as
follows. We present EXP-3-Transfer, a principled policy-reuse algorithm that
optimally reuses a given source policy set when learning for a new MDP. We
present a framework to cluster the previous MDPs to extract a source subset.
The framework consists of (i) a distance $d_V$ over MDPs to measure
policy-based similarity between MDPs; (ii) a cost function $g(\cdot)$ that uses
$d_V$ to measure how good a particular clustering is for generating useful
source tasks for EXP-3-Transfer and (iii) a provably convergent algorithm,
MHAV, for finding the optimal clustering. We validate our algorithms through
experiments in a surveillance domain.