pwc-archive/papers-with-abstracts huggingface dataset:
Symbol Grounding Association in Multimodal Sequences with Missing Elements
Authors: Federico Raue, Andreas Dengel, Thomas M. Breuel, Marcus Liwicki
Date: 2015-11-13 00:00:00
Tasks: Dynamic Time Warping, Missing Elements

In this paper, we extend a symbolic association framework for being able to
handle missing elements in multimodal sequences. The general scope of the work
is the symbolic associations of object-word mappings as it happens in language
development in infants. In other words, two different representations of the
same abstract concepts can associate in both directions. This scenario has been
long interested in Artificial Intelligence, Psychology, and Neuroscience. In
this work, we extend a recent approach for multimodal sequences (visual and
audio) to also cope with missing elements in one or both modalities. Our method
uses two parallel Long Short-Term Memories (LSTMs) with a learning rule based
on EM-algorithm. It aligns both LSTM outputs via Dynamic Time Warping (DTW). We
propose to include an extra step for the combination with the max operation for
exploiting the common elements between both sequences. The motivation behind is
that the combination acts as a condition selector for choosing the best
representation from both LSTMs. We evaluated the proposed extension in the
following scenarios: missing elements in one modality (visual or audio) and
missing elements in both modalities (visual and sound). The performance of our
extension reaches better results than the original model and similar results to
individual LSTM trained in each modality.
Compressive Statistical Learning with Random Feature Moments
Authors: Rémi Gribonval, Gilles Blanchard, Nicolas Keriven, Yann Traonmilin
Date: 2017-06-22 00:00:00
Tasks: Clustering

We describe a general framework -- compressive statistical learning -- for resource-efficient large-scale learning: the training collection is compressed in one pass into a low-dimensional sketch (a vector of random empirical generalized moments) that captures the information relevant to the considered learning task. A near-minimizer of the risk is computed from the sketch through the solution of a nonlinear least squares problem. We investigate sufficient sketch sizes to control the generalization error of this procedure. The framework is illustrated on compressive PCA, compressive clustering, and compressive Gaussian mixture Modeling with fixed known variance. The latter two are further developed in a companion paper.
Parallel Coordinate Descent Newton Method for Efficient $\ell_1$-Regularized Minimization
Authors: An Bian, Xiong Li, Yuncai Liu, Ming-Hsuan Yang
Date: 2013-06-18 00:00:00

The recent years have witnessed advances in parallel algorithms for large
scale optimization problems. Notwithstanding demonstrated success, existing
algorithms that parallelize over features are usually limited by divergence
issues under high parallelism or require data preprocessing to alleviate these
problems. In this work, we propose a Parallel Coordinate Descent Newton
algorithm using multidimensional approximate Newton steps (PCDN), where the
off-diagonal elements of the Hessian are set to zero to enable parallelization.
It randomly partitions the feature set into $b$ bundles/subsets with size of
$P$, and sequentially processes each bundle by first computing the descent
directions for each feature in parallel and then conducting $P$-dimensional
line search to obtain the step size. We show that: (1) PCDN is guaranteed to
converge globally despite increasing parallelism; (2) PCDN converges to the
specified accuracy $\epsilon$ within the limited iteration number of
$T_\epsilon$, and $T_\epsilon$ decreases with increasing parallelism (bundle
size $P$). Using the implementation technique of maintaining intermediate
quantities, we minimize the data transfer and synchronization cost of the
$P$-dimensional line search. For concreteness, the proposed PCDN algorithm is
applied to $\ell_1$-regularized logistic regression and $\ell_2$-loss SVM.
Experimental evaluations on six benchmark datasets show that the proposed PCDN
algorithm exploits parallelism well and outperforms the state-of-the-art
methods in speed without losing accuracy.
IKBT: solving closed-form Inverse Kinematics with Behavior Tree
Authors: Dianmu Zhang, Blake Hannaford
Date: 2017-11-15 00:00:00
Tasks: Form

Serial robot arms have complicated kinematic equations which must be solved
to write effective arm planning and control software (the Inverse Kinematics
Problem). Existing software packages for inverse kinematics often rely on
numerical methods which have significant shortcomings. Here we report a new
symbolic inverse kinematics solver which overcomes the limitations of numerical
methods, and the shortcomings of previous symbolic software packages. We
integrate Behavior Trees, an execution planning framework previously used for
controlling intelligent robot behavior, to organize the equation solving
process, and a modular architecture for each solution technique. The system
successfully solved, generated a LaTex report, and generated a Python code
template for 18 out of 19 example robots of 4-6 DOF. The system is readily
extensible, maintainable, and multi-platform with few dependencies. The
complete package is available with a Modified BSD license on Github.
Learning Random Fourier Features by Hybrid Constrained Optimization
Authors: Jianqiao Wangni, Jingwei Zhuo, Jun Zhu
Date: 2017-12-07 00:00:00

The kernel embedding algorithm is an important component for adapting kernel
methods to large datasets. Since the algorithm consumes a major computation
cost in the testing phase, we propose a novel teacher-learner framework of
learning computation-efficient kernel embeddings from specific data. In the
framework, the high-precision embeddings (teacher) transfer the data
information to the computation-efficient kernel embeddings (learner). We
jointly select informative embedding functions and pursue an orthogonal
transformation between two embeddings. We propose a novel approach of
constrained variational expectation maximization (CVEM), where the alternate
direction method of multiplier (ADMM) is applied over a nonconvex domain in the
maximization step. We also propose two specific formulations based on the
prevalent Random Fourier Feature (RFF), the masked and blocked version of
Computation-Efficient RFF (CERF), by imposing a random binary mask or a block
structure on the transformation matrix. By empirical studies of several
applications on different real-world datasets, we demonstrate that the CERF
significantly improves the performance of kernel methods upon the RFF, under
certain arithmetic operation requirements, and suitable for structured matrix
multiplication in Fastfood type algorithms.
Fast Rates for General Unbounded Loss Functions: from ERM to Generalized Bayes
Authors: Peter D. Grünwald, Nishant A. Mehta
Date: 2016-05-01 00:00:00
Tasks: Bayesian Inference

We present new excess risk bounds for general unbounded loss functions including log loss and squared loss, where the distribution of the losses may be heavy-tailed. The bounds hold for general estimators, but they are optimized when applied to $\eta$-generalized Bayesian, MDL, and empirical risk minimization estimators. In the case of log loss, the bounds imply convergence rates for generalized Bayesian inference under misspecification in terms of a generalization of the Hellinger metric as long as the learning rate $\eta$ is set correctly. For general loss functions, our bounds rely on two separate conditions: the $v$-GRIP (generalized reversed information projection) conditions, which control the lower tail of the excess loss; and the newly introduced witness condition, which controls the upper tail. The parameter $v$ in the $v$-GRIP conditions determines the achievable rate and is akin to the exponent in the Tsybakov margin condition and the Bernstein condition for bounded losses, which the $v$-GRIP conditions generalize; favorable $v$ in combination with small model complexity leads to $\tilde{O}(1/n)$ rates. The witness condition allows us to connect the excess risk to an "annealed" version thereof, by which we generalize several previous results connecting Hellinger and R\'enyi divergence to KL divergence.
TV-GAN: Generative Adversarial Network Based Thermal to Visible Face Recognition
Authors: Teng Zhang, Arnold Wiliem, Siqi Yang, Brian C. Lovell
Date: 2017-12-07 00:00:00
Tasks: Face Recognition, Generative Adversarial Network

This work tackles the face recognition task on images captured using thermal
camera sensors which can operate in the non-light environment. While it can
greatly increase the scope and benefits of the current security surveillance
systems, performing such a task using thermal images is a challenging problem
compared to face recognition task in the Visible Light Domain (VLD). This is
partly due to the much smaller amount number of thermal imagery data collected
compared to the VLD data. Unfortunately, direct application of the existing
very strong face recognition models trained using VLD data into the thermal
imagery data will not produce a satisfactory performance. This is due to the
existence of the domain gap between the thermal and VLD images. To this end, we
propose a Thermal-to-Visible Generative Adversarial Network (TV-GAN) that is
able to transform thermal face images into their corresponding VLD images
whilst maintaining identity information which is sufficient enough for the
existing VLD face recognition models to perform recognition. Some examples are
presented in Figure 1. Unlike the previous methods, our proposed TV-GAN uses an
explicit closed-set face recognition loss to regularize the discriminator
network training. This information will then be conveyed into the generator
network in the forms of gradient loss. In the experiment, we show that by using
this additional explicit regularization for the discriminator network, the
TV-GAN is able to preserve more identity information when translating a thermal
image of a person which is not seen before by the TV-GAN.
Gini-regularized Optimal Transport with an Application to Spatio-Temporal Forecasting
Authors: Lucas Roberts, Leo Razoumov, Lin Su, Yuyang Wang
Date: 2017-12-07 00:00:00
Tasks: Open-Ended Question Answering, Spatio-Temporal Forecasting

Rapidly growing product lines and services require a finer-granularity
forecast that considers geographic locales. However the open question remains,
how to assess the quality of a spatio-temporal forecast? In this manuscript we
introduce a metric to evaluate spatio-temporal forecasts. This metric is based
on an Opti- mal Transport (OT) problem. The metric we propose is a constrained
OT objec- tive function using the Gini impurity function as a regularizer. We
demonstrate through computer experiments both the qualitative and the
quantitative charac- teristics of the Gini regularized OT problem. Moreover, we
show that the Gini regularized OT problem converges to the classical OT
problem, when the Gini regularized problem is considered as a function of
{\lambda}, the regularization parame-ter. The convergence to the classical OT
solution is faster than the state-of-the-art Entropic-regularized OT[Cuturi,
2013] and results in a numerically more stable algorithm.
Survival-Supervised Topic Modeling with Anchor Words: Characterizing Pancreatitis Outcomes
Authors: George H. Chen, Jeremy C. Weiss
Date: 2017-12-02 00:00:00
Tasks: Survival Analysis

We introduce a new approach for topic modeling that is supervised by survival
analysis. Specifically, we build on recent work on unsupervised topic modeling
with so-called anchor words by providing supervision through an elastic-net
regularized Cox proportional hazards model. In short, an anchor word being
present in a document provides strong indication that the document is partially
about a specific topic. For example, by seeing "gallstones" in a document, we
are fairly certain that the document is partially about medicine. Our proposed
method alternates between learning a topic model and learning a survival model
to find a local minimum of a block convex optimization problem. We apply our
proposed approach to predicting how long patients with pancreatitis admitted to
an intensive care unit (ICU) will stay in the ICU. Our approach is as accurate
as the best of a variety of baselines while being more interpretable than any
of the baselines.
Semi-Supervised Learning with IPM-based GANs: an Empirical Study
Authors: Tom Sercu, Youssef Mroueh
Date: 2017-12-07 00:00:00
Tasks: General Classification

We present an empirical investigation of a recent class of Generative
Adversarial Networks (GANs) using Integral Probability Metrics (IPM) and their
performance for semi-supervised learning. IPM-based GANs like Wasserstein GAN,
Fisher GAN and Sobolev GAN have desirable properties in terms of theoretical
understanding, training stability, and a meaningful loss. In this work we
investigate how the design of the critic (or discriminator) influences the
performance in semi-supervised learning. We distill three key take-aways which
are important for good SSL performance: (1) the K+1 formulation, (2) avoiding
batch normalization in the critic and (3) avoiding gradient penalty constraints
on the classification layer.
Take it in your stride: Do we need striding in CNNs?
Authors: Chen Kong, Simon Lucey
Date: 2017-12-07 00:00:00

Since their inception, CNNs have utilized some type of striding operator to
reduce the overlap of receptive fields and spatial dimensions. Although having
clear heuristic motivations (i.e. lowering the number of parameters to learn)
the mathematical role of striding within CNN learning remains unclear. This
paper offers a novel and mathematical rigorous perspective on the role of the
striding operator within modern CNNs. Specifically, we demonstrate
theoretically that one can always represent a CNN that incorporates striding
with an equivalent non-striding CNN which has more filters and smaller size.
Through this equivalence we are then able to characterize striding as an
additional mechanism for parameter sharing among channels, thus reducing
training complexity. Finally, the framework presented in this paper offers a
new mathematical perspective on the role of striding which we hope shall
facilitate and simplify the future theoretical analysis of CNNs.
Adversarial Examples that Fool Detectors
Authors: Jiajun Lu, Hussein Sibai, Evan Fabry
Date: 2017-12-07 00:00:00

An adversarial example is an example that has been adjusted to produce a
wrong label when presented to a system at test time. To date, adversarial
example constructions have been demonstrated for classifiers, but not for
detectors. If adversarial examples that could fool a detector exist, they could
be used to (for example) maliciously create security hazards on roads populated
with smart vehicles. In this paper, we demonstrate a construction that
successfully fools two standard detectors, Faster RCNN and YOLO. The existence
of such examples is surprising, as attacking a classifier is very different
from attacking a detector, and that the structure of detectors - which must
search for their own bounding box, and which cannot estimate that box very
accurately - makes it quite likely that adversarial patterns are strongly
disrupted. We show that our construction produces adversarial examples that
generalize well across sequences digitally, even though large perturbations are
needed. We also show that our construction yields physical objects that are
adversarial.
Geometric Semantic Genetic Programming Algorithm and Slump Prediction
Authors: Juncai Xu, Zhenzhong Shen, Qingwen Ren, Xin Xie, Zhengyu Yang
Date: 2017-09-18 00:00:00
Tasks: Prediction

Research on the performance of recycled concrete as building material in the
current world is an important subject. Given the complex composition of
recycled concrete, conventional methods for forecasting slump scarcely obtain
satisfactory results. Based on theory of nonlinear prediction method, we
propose a recycled concrete slump prediction model based on geometric semantic
genetic programming (GSGP) and combined it with recycled concrete features.
Tests show that the model can accurately predict the recycled concrete slump by
using the established prediction model to calculate the recycled concrete slump
with different mixing ratios in practical projects and by comparing the
predicted values with the experimental values. By comparing the model with
several other nonlinear prediction models, we can conclude that GSGP has higher
accuracy and reliability than conventional methods.
Cost-sensitive detection with variational autoencoders for environmental acoustic sensing
Authors: Yunpeng Li, Ivan Kiskin, Davide Zilli, Marianne Sinka, Henry Chan, Kathy Willis, Stephen Roberts
Date: 2017-12-07 00:00:00
Tasks: BIG-bench Machine Learning, General Classification, Retrieval

Environmental acoustic sensing involves the retrieval and processing of audio
signals to better understand our surroundings. While large-scale acoustic data
make manual analysis infeasible, they provide a suitable playground for machine
learning approaches. Most existing machine learning techniques developed for
environmental acoustic sensing do not provide flexible control of the trade-off
between the false positive rate and the false negative rate. This paper
presents a cost-sensitive classification paradigm, in which the
hyper-parameters of classifiers and the structure of variational autoencoders
are selected in a principled Neyman-Pearson framework. We examine the
performance of the proposed approach using a dataset from the HumBug project
which aims to detect the presence of mosquitoes using sound collected by simple
embedded devices.
A Corpus of Deep Argumentative Structures as an Explanation to Argumentative Relations
Authors: Paul Reisert, Naoya Inoue, Naoaki Okazaki, Kentaro Inui
Date: 2017-12-07 00:00:00

In this paper, we compose a new task for deep argumentative structure
analysis that goes beyond shallow discourse structure analysis. The idea is
that argumentative relations can reasonably be represented with a small set of
predefined patterns. For example, using value judgment and bipolar causality,
we can explain a support relation between two argumentative segments as
follows: Segment 1 states that something is good, and Segment 2 states that it
is good because it promotes something good when it happens. We are motivated by
the following questions: (i) how do we formulate the task?, (ii) can a
reasonable pattern set be created?, and (iii) do the patterns work? To examine
the task feasibility, we conduct a three-stage, detailed annotation study using
357 argumentative relations from the argumentative microtext corpus, a small,
but highly reliable corpus. We report the coverage of explanations captured by
our patterns on a test set composed of 270 relations. Our coverage result of
74.6% indicates that argumentative relations can reasonably be explained by our
small pattern set. Our agreement result of 85.9% shows that a reasonable
inter-annotator agreement can be achieved. To assist with future work in
computational argumentation, the annotated corpus is made publicly available.
Stacked Conditional Generative Adversarial Networks for Jointly Learning Shadow Detection and Shadow Removal
Authors: Jifeng Wang, Xiang Li, Le Hui, Jian Yang
Date: 2017-12-07 00:00:00
Proceeding: CVPR 2018 6
Tasks: Generative Adversarial Network, Multi-Task Learning, Shadow Detection, Shadow Removal

Understanding shadows from a single image spontaneously derives into two
types of task in previous studies, containing shadow detection and shadow
removal. In this paper, we present a multi-task perspective, which is not
embraced by any existing work, to jointly learn both detection and removal in
an end-to-end fashion that aims at enjoying the mutually improved benefits from
each other. Our framework is based on a novel STacked Conditional Generative
Adversarial Network (ST-CGAN), which is composed of two stacked CGANs, each
with a generator and a discriminator. Specifically, a shadow image is fed into
the first generator which produces a shadow detection mask. That shadow image,
concatenated with its predicted mask, goes through the second generator in
order to recover its shadow-free image consequently. In addition, the two
corresponding discriminators are very likely to model higher level
relationships and global scene characteristics for the detected shadow region
and reconstruction via removing shadows, respectively. More importantly, for
multi-task learning, our design of stacked paradigm provides a novel view which
is notably different from the commonly used one as the multi-branch version. To
fully evaluate the performance of our proposed framework, we construct the
first large-scale benchmark with 1870 image triplets (shadow image, shadow mask
image, and shadow-free image) under 135 scenes. Extensive experimental results
consistently show the advantages of ST-CGAN over several representative
state-of-the-art methods on two large-scale publicly available datasets and our
newly released one.
Deep Primal-Dual Reinforcement Learning: Accelerating Actor-Critic using Bellman Duality
Authors: Woon Sang Cho, Mengdi Wang
Date: 2017-12-07 00:00:00
Tasks: Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We develop a parameterized Primal-Dual $\pi$ Learning method based on deep
neural networks for Markov decision process with large state space and
off-policy reinforcement learning. In contrast to the popular Q-learning and
actor-critic methods that are based on successive approximations to the
nonlinear Bellman equation, our method makes primal-dual updates to the policy
and value functions utilizing the fundamental linear Bellman duality. Naive
parametrization of the primal-dual $\pi$ learning method using deep neural
networks would encounter two major challenges: (1) each update requires
computing a probability distribution over the state space and is intractable;
(2) the iterates are unstable since the parameterized Lagrangian function is no
longer linear. We address these challenges by proposing a relaxed Lagrangian
formulation with a regularization penalty using the advantage function. We show
that the dual policy update step in our method is equivalent to the policy
gradient update in the actor-critic method in some special case, while the
value updates differ substantially. The main advantage of the primal-dual $\pi$
learning method lies in that the value and policy updates are closely coupled
together using the Bellman duality and therefore more informative. Experiments
on a simple cart-pole problem show that the algorithm significantly outperforms
the one-step temporal-difference actor-critic method, which is the most
relevant benchmark method to compare with. We believe that the primal-dual
updates to the value and policy functions would expedite the learning process.
The proposed methods might open a door to more efficient algorithms and sharper
theoretical analysis.