pwc-archive/papers-with-abstracts huggingface dataset:
E-Generalization Using Grammars
Authors: Jochen Burghardt
Date: 2014-03-28 00:00:00
Tasks: Inductive logic programming

We extend the notion of anti-unification to cover equational theories and
present a method based on regular tree grammars to compute a finite
representation of E-generalization sets. We present a framework to combine
Inductive Logic Programming and E-generalization that includes an extension of
Plotkin's lgg theorem to the equational case. We demonstrate the potential
power of E-generalization by three example applications: computation of
suggestions for auxiliary lemmas in equational inductive proofs, computation of
construction laws for given term sequences, and learning of screen editor
command sequences.
Real-Time Panoramic Tracking for Event Cameras
Authors: Christian Reinbacher, Gottfried Munda, Thomas Pock
Date: 2017-03-15 00:00:00
Tasks: Position, Visual Odometry

Event cameras are a paradigm shift in camera technology. Instead of full
frames, the sensor captures a sparse set of events caused by intensity changes.
Since only the changes are transferred, those cameras are able to capture quick
movements of objects in the scene or of the camera itself. In this work we
propose a novel method to perform camera tracking of event cameras in a
panoramic setting with three degrees of freedom. We propose a direct camera
tracking formulation, similar to state-of-the-art in visual odometry. We show
that the minimal information needed for simultaneous tracking and mapping is
the spatial position of events, without using the appearance of the imaged
scene point. We verify the robustness to fast camera movements and dynamic
objects in the scene on a recently proposed dataset and self-recorded
sequences.
Overcoming model simplifications when quantifying predictive uncertainty
Authors: George M. Mathews, John Vial
Date: 2017-03-21 00:00:00
Tasks: model

It is generally accepted that all models are wrong -- the difficulty is
determining which are useful. Here, a useful model is considered as one that is
capable of combining data and expert knowledge, through an inversion or
calibration process, to adequately characterize the uncertainty in predictions
of interest. This paper derives conditions that specify which simplified models
are useful and how they should be calibrated. To start, the notion of an
optimal simplification is defined. This relates the model simplifications to
the nature of the data and predictions, and determines when a standard
probabilistic calibration scheme is capable of accurately characterizing
uncertainty. Furthermore, two additional conditions are defined for suboptimal
models that determine when the simplifications can be safely ignored. The first
allows a suboptimally simplified model to be used in a way that replicates the
performance of an optimal model. This is achieved through the judicial
selection of a prior term for the calibration process that explicitly includes
the nature of the data, predictions and modelling simplifications. The second
considers the dependency structure between the predictions and the available
data to gain insights into when the simplifications can be overcome by using
the right calibration data. Furthermore, the derived conditions are related to
the commonly used calibration schemes based on Tikhonov and subspace
regularization. To allow concrete insights to be obtained, the analysis is
performed under a linear expansion of the model equations and where the
predictive uncertainty is characterized via second order moments only.
When Slepian Meets Fiedler: Putting a Focus on the Graph Spectrum
Authors: Dimitri Van De Ville, Robin Demesmaeker, Maria Giulia Preti
Date: 2017-01-29 00:00:00
Tasks: Clustering, Graph Clustering

The study of complex systems benefits from graph models and their analysis.
In particular, the eigendecomposition of the graph Laplacian lets emerge
properties of global organization from local interactions; e.g., the Fiedler
vector has the smallest non-zero eigenvalue and plays a key role for graph
clustering. Graph signal processing focusses on the analysis of signals that
are attributed to the graph nodes. The eigendecomposition of the graph
Laplacian allows to define the graph Fourier transform and extend conventional
signal-processing operations to graphs. Here, we introduce the design of
Slepian graph signals, by maximizing energy concentration in a predefined
subgraph for a graph spectral bandlimit. We establish a novel link with
classical Laplacian embedding and graph clustering, which provides a meaning to
localized graph frequencies.
A Deterministic Global Optimization Method for Variational Inference
Authors: Hachem Saddiki, Andrew C. Trapp, Patrick Flaherty
Date: 2017-03-21 00:00:00
Tasks: global-optimization, Variational Inference

Variational inference methods for latent variable statistical models have
gained popularity because they are relatively fast, can handle large data sets,
and have deterministic convergence guarantees. However, in practice it is
unclear whether the fixed point identified by the variational inference
algorithm is a local or a global optimum. Here, we propose a method for
constructing iterative optimization algorithms for variational inference
problems that are guaranteed to converge to the $\epsilon$-global variational
lower bound on the log-likelihood. We derive inference algorithms for two
variational approximations to a standard Bayesian Gaussian mixture model
(BGMM). We present a minimal data set for empirically testing convergence and
show that a variational inference algorithm frequently converges to a local
optimum while our algorithm always converges to the globally optimal
variational lower bound. We characterize the loss incurred by choosing a
non-optimal variational approximation distribution suggesting that selection of
the approximating variational distribution deserves as much attention as the
selection of the original statistical model for a given data set.
Optical Flow-based 3D Human Motion Estimation from Monocular Video
Authors: Thiemo Alldieck, Marc Kassubeck, Marcus Magnor
Date: 2017-03-01 00:00:00
Tasks: Motion Estimation, Optical Flow Estimation

We present a generative method to estimate 3D human motion and body shape
from monocular video. Under the assumption that starting from an initial pose
optical flow constrains subsequent human motion, we exploit flow to find
temporally coherent human poses of a motion sequence. We estimate human motion
by minimizing the difference between computed flow fields and the output of an
artificial flow renderer. A single initialization step is required to estimate
motion over multiple frames. Several regularization functions enhance
robustness over time. Our test scenarios demonstrate that optical flow
effectively regularizes the under-constrained problem of human shape and motion
estimation from monocular video.
Proposal Flow: Semantic Correspondences from Object Proposals
Authors: Bumsub Ham, Minsu Cho, Cordelia Schmid, Jean Ponce
Date: 2017-03-21 00:00:00
Tasks: Object

Finding image correspondences remains a challenging problem in the presence
of intra-class variations and large changes in scene layout. Semantic flow
methods are designed to handle images depicting different instances of the same
object or scene category. We introduce a novel approach to semantic flow,
dubbed proposal flow, that establishes reliable correspondences using object
proposals. Unlike prevailing semantic flow approaches that operate on pixels or
regularly sampled local regions, proposal flow benefits from the
characteristics of modern object proposals, that exhibit high repeatability at
multiple scales, and can take advantage of both local and geometric consistency
constraints among proposals. We also show that the corresponding sparse
proposal flow can effectively be transformed into a conventional dense flow
field. We introduce two new challenging datasets that can be used to evaluate
both general semantic flow techniques and region-based approaches such as
proposal flow. We use these benchmarks to compare different matching
algorithms, object proposals, and region features within proposal flow, to the
state of the art in semantic flow. This comparison, along with experiments on
standard datasets, demonstrates that proposal flow significantly outperforms
existing semantic flow methods in various settings.
Deep generative-contrastive networks for facial expression recognition
Authors: Youngsung Kim, ByungIn Yoo, Youngjun Kwak, Changkyu Choi, Junmo Kim
Date: 2017-03-21 00:00:00
Tasks: Facial Expression Recognition, Facial Expression Recognition (FER)

As the expressive depth of an emotional face differs with individuals or expressions, recognizing an expression using a single facial image at a moment is difficult. A relative expression of a query face compared to a reference face might alleviate this difficulty. In this paper, we propose to utilize contrastive representation that embeds a distinctive expressive factor for a discriminative purpose. The contrastive representation is calculated at the embedding layer of deep networks by comparing a given (query) image with the reference image. We attempt to utilize a generative reference image that is estimated based on the given image. Consequently, we deploy deep neural networks that embed a combination of a generative model, a contrastive model, and a discriminative model with an end-to-end training manner. In our proposed networks, we attempt to disentangle a facial expressive factor in two steps including learning of a generator network and a contrastive encoder network. We conducted extensive experiments on publicly available face expression databases (CK+, MMI, Oulu-CASIA, and in-the-wild databases) that have been widely adopted in the recent literatures. The proposed method outperforms the known state-of-the art methods in terms of the recognition accuracy.
Knowledge distillation using unlabeled mismatched images
Authors: Mandar Kulkarni, Kalpesh Patil, Shirish Karande
Date: 2017-03-21 00:00:00
Tasks: General Classification, image-classification, Image Classification, Knowledge Distillation

Current approaches for Knowledge Distillation (KD) either directly use
training data or sample from the training data distribution. In this paper, we
demonstrate effectiveness of 'mismatched' unlabeled stimulus to perform KD for
image classification networks. For illustration, we consider scenarios where
this is a complete absence of training data, or mismatched stimulus has to be
used for augmenting a small amount of training data. We demonstrate that
stimulus complexity is a key factor for distillation's good performance. Our
examples include use of various datasets for stimulating MNIST and CIFAR
teachers.
Multi Channel-Kernel Canonical Correlation Analysis for Cross-View Person Re-Identification
Authors: Giuseppe Lisanti, Svebor Karaman, Iacopo Masi
Date: 2016-07-08 00:00:00
Tasks: Cross-Modal  Person Re-Identification, Person Re-Identification

In this paper we introduce a method to overcome one of the main challenges of
person re-identification in multi-camera networks, namely cross-view appearance
changes. The proposed solution addresses the extreme variability of person
appearance in different camera views by exploiting multiple feature
representations. For each feature, Kernel Canonical Correlation Analysis (KCCA)
with different kernels is exploited to learn several projection spaces in which
the appearance correlation between samples of the same person observed from
different cameras is maximized. An iterative logistic regression is finally
used to select and weigh the contributions of each feature projections and
perform the matching between the two views. Experimental evaluation shows that
the proposed solution obtains comparable performance on VIPeR and PRID 450s
datasets and improves on PRID and CUHK01 datasets with respect to the state of
the art.
Evolving Parsimonious Networks by Mixing Activation Functions
Authors: Alexander Hagg, Maximilian Mensing, Alexander Asteroth
Date: 2017-03-21 00:00:00

Neuroevolution methods evolve the weights of a neural network, and in some
cases the topology, but little work has been done to analyze the effect of
evolving the activation functions of individual nodes on network size, which is
important when training networks with a small number of samples. In this work
we extend the neuroevolution algorithm NEAT to evolve the activation function
of neurons in addition to the topology and weights of the network. The size and
performance of networks produced using NEAT with uniform activation in all
nodes, or homogenous networks, is compared to networks which contain a mixture
of activation functions, or heterogenous networks. For a number of regression
and classification benchmarks it is shown that, (1) qualitatively different
activation functions lead to different results in homogeneous networks, (2) the
heterogeneous version of NEAT is able to select well performing activation
functions, (3) producing heterogeneous networks that are significantly smaller
than homogeneous networks.
Interest-Driven Discovery of Local Process Models
Authors: Niek Tax, Benjamin Dalmas, Natalia Sidorova, Wil M. P. van der Aalst, Sylvie Norre
Date: 2017-03-21 00:00:00

Local Process Models (LPM) describe structured fragments of process behavior
occurring in the context of less structured business processes. Traditional LPM
discovery aims to generate a collection of process models that describe highly
frequent behavior, but these models do not always provide useful answers for
questions posed by process analysts aiming at business process improvement. We
propose a framework for goal-driven LPM discovery, based on utility functions
and constraints. We describe four scopes on which these utility functions and
constrains can be defined, and show that utility functions and constraints on
different scopes can be combined to form composite utility
functions/constraints. Finally, we demonstrate the applicability of our
approach by presenting several actionable business insights discovered with LPM
discovery on two real life data sets.
Layer-wise training of deep networks using kernel similarity
Authors: Mandar Kulkarni, Shirish Karande
Date: 2017-03-21 00:00:00

Deep learning has shown promising results in many machine learning
applications. The hierarchical feature representation built by deep networks
enable compact and precise encoding of the data. A kernel analysis of the
trained deep networks demonstrated that with deeper layers, more simple and
more accurate data representations are obtained. In this paper, we propose an
approach for layer-wise training of a deep network for the supervised
classification task. A transformation matrix of each layer is obtained by
solving an optimization aimed at a better representation where a subsequent
layer builds its representation on the top of the features produced by a
previous layer. We compared the performance of our approach with a DNN trained
using back-propagation which has same architecture as ours. Experimental
results on the real image datasets demonstrate efficacy of our approach. We
also performed kernel analysis of layer representations to validate the claim
of better feature encoding.
On the Interplay between Strong Regularity and Graph Densification
Authors: Marco Fiorucci, Alessandro Torcinovich, Manuel Curado, Francisco Escolano, Marcello Pelillo
Date: 2017-03-21 00:00:00
Tasks: LEMMA

In this paper we analyze the practical implications of Szemer\'edi's
regularity lemma in the preservation of metric information contained in large
graphs. To this end, we present a heuristic algorithm to find regular
partitions. Our experiments show that this method is quite robust to the
natural sparsification of proximity graphs. In addition, this robustness can be
enforced by graph densification.
Deep LSTM for Large Vocabulary Continuous Speech Recognition
Authors: Xu Tian, Jun Zhang, Zejun Ma, Yi He, Juan Wei, Peihao Wu, Wenchang Situ, Shuai Li, Yang Zhang
Date: 2017-03-21 00:00:00
Tasks: speech-recognition, Speech Recognition, Transfer Learning

Recurrent neural networks (RNNs), especially long short-term memory (LSTM)
RNNs, are effective network for sequential task like speech recognition. Deeper
LSTM models perform well on large vocabulary continuous speech recognition,
because of their impressive learning ability. However, it is more difficult to
train a deeper network. We introduce a training framework with layer-wise
training and exponential moving average methods for deeper LSTM models. It is a
competitive framework that LSTM models of more than 7 layers are successfully
trained on Shenma voice search data in Mandarin and they outperform the deep
LSTM models trained by conventional approach. Moreover, in order for online
streaming speech recognition applications, the shallow model with low real time
factor is distilled from the very deep model. The recognition accuracy have
little loss in the distillation process. Therefore, the model trained with the
proposed training framework reduces relative 14\% character error rate,
compared to original model which has the similar real-time capability.
Furthermore, the novel transfer learning strategy with segmental Minimum
Bayes-Risk is also introduced in the framework. The strategy makes it possible
that training with only a small part of dataset could outperform full dataset
training from the beginning.
Multi-stage Neural Networks with Single-sided Classifiers for False Positive Reduction and its Evaluation using Lung X-ray CT Images
Authors: Masaharu Sakamoto, Hiroki Nakano, Kun Zhao, Taro Sekiyama
Date: 2017-03-01 00:00:00
Tasks: General Classification, Lung Nodule Classification, Sensitivity

Lung nodule classification is a class imbalanced problem because nodules are
found with much lower frequency than non-nodules. In the class imbalanced
problem, conventional classifiers tend to be overwhelmed by the majority class
and ignore the minority class. We therefore propose cascaded convolutional
neural networks to cope with the class imbalanced problem. In the proposed
approach, multi-stage convolutional neural networks that perform as
single-sided classifiers filter out obvious non-nodules. Successively, a
convolutional neural network trained with a balanced data set calculates nodule
probabilities. The proposed method achieved the sensitivity of 92.4\% and 94.5%
at 4 and 8 false positives per scan in Free Receiver Operating Characteristics
(FROC) curve analysis, respectively.
Solving the Goddard problem by an influence diagram
Authors: Jiří Vomlel, Václav Kratochvíl
Date: 2017-03-18 00:00:00

Influence diagrams are a decision-theoretic extension of probabilistic
graphical models. In this paper we show how they can be used to solve the
Goddard problem. We present results of numerical experiments with this problem
and compare the solutions provided by influence diagrams with the optimal
solution.
Revisiting Distributed Synchronous SGD
Authors: Jianmin Chen, Xinghao Pan, Rajat Monga, Samy Bengio, Rafal Jozefowicz
Date: 2016-04-04 00:00:00
Tasks: Stochastic Optimization

Distributed training of deep learning models on large-scale training data is
typically conducted with asynchronous stochastic optimization to maximize the
rate of updates, at the cost of additional noise introduced from asynchrony. In
contrast, the synchronous approach is often thought to be impractical due to
idle time wasted on waiting for straggling workers. We revisit these
conventional beliefs in this paper, and examine the weaknesses of both
approaches. We demonstrate that a third approach, synchronous optimization with
backup workers, can avoid asynchronous noise while mitigating for the worst
stragglers. Our approach is empirically validated and shown to converge faster
and to better test accuracies.
Pseudorehearsal in value function approximation
Authors: Vladimir Marochko, Leonard Johard, Manuel Mazzara
Date: 2017-03-21 00:00:00
Tasks: Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Catastrophic forgetting is of special importance in reinforcement learning,
as the data distribution is generally non-stationary over time. We study and
compare several pseudorehearsal approaches for Q-learning with function
approximation in a pole balancing task. We have found that pseudorehearsal
seems to assist learning even in such very simple problems, given proper
initialization of the rehearsal parameters.
Dual Attention Networks for Multimodal Reasoning and Matching
Authors: Hyeonseob Nam, Jung-Woo Ha, Jeonghee Kim
Date: 2016-11-02 00:00:00
Proceeding: CVPR 2017 7
Tasks: Collaborative Inference, Image-text matching, Multimodal Reasoning, Question Answering, Text Matching, Visual Question Answering, Visual Question Answering (VQA)

We propose Dual Attention Networks (DANs) which jointly leverage visual and
textual attention mechanisms to capture fine-grained interplay between vision
and language. DANs attend to specific regions in images and words in text
through multiple steps and gather essential information from both modalities.
Based on this framework, we introduce two types of DANs for multimodal
reasoning and matching, respectively. The reasoning model allows visual and
textual attentions to steer each other during collaborative inference, which is
useful for tasks such as Visual Question Answering (VQA). In addition, the
matching model exploits the two attention mechanisms to estimate the similarity
between images and sentences by focusing on their shared semantics. Our
extensive experiments validate the effectiveness of DANs in combining vision
and language, achieving the state-of-the-art performance on public benchmarks
for VQA and image-text matching.
Stochastic Primal Dual Coordinate Method with Non-Uniform Sampling Based on Optimality Violations
Authors: Atsushi Shibagaki, Ichiro Takeuchi
Date: 2017-03-21 00:00:00
Tasks: Stochastic Optimization

We study primal-dual type stochastic optimization algorithms with non-uniform
sampling. Our main theoretical contribution in this paper is to present a
convergence analysis of Stochastic Primal Dual Coordinate (SPDC) Method with
arbitrary sampling. Based on this theoretical framework, we propose Optimality
Violation-based Sampling SPDC (ovsSPDC), a non-uniform sampling method based on
Optimality Violation. We also propose two efficient heuristic variants of
ovsSPDC called ovsSDPC+ and ovsSDPC++. Through intensive numerical experiments,
we demonstrate that the proposed method and its variants are faster than other
state-of-the-art primal-dual type stochastic optimization methods.
Investigation of Language Understanding Impact for Reinforcement Learning Based Dialogue Systems
Authors: Xiujun Li, Yun-Nung Chen, Lihong Li, Jianfeng Gao, Asli Celikyilmaz
Date: 2017-03-21 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Language understanding is a key component in a spoken dialogue system. In
this paper, we investigate how the language understanding module influences the
dialogue system performance by conducting a series of systematic experiments on
a task-oriented neural dialogue system in a reinforcement learning based
setting. The empirical study shows that among different types of language
understanding errors, slot-level errors can have more impact on the overall
performance of a dialogue system compared to intent-level errors. In addition,
our experiments demonstrate that the reinforcement learning based dialogue
system is able to learn when and what to confirm in order to achieve better
performance and greater robustness.