pwc-archive/papers-with-abstracts huggingface dataset:
Learning Spatiotemporal Features for Infrared Action Recognition with 3D Convolutional Neural Networks
Authors: Zhuolin Jiang, Viktor Rozgic, Sancar Adali
Date: 2017-05-18 00:00:00
Tasks: Action Recognition, Optical Flow Estimation, Temporal Action Localization

Infrared (IR) imaging has the potential to enable more robust action
recognition systems compared to visible spectrum cameras due to lower
sensitivity to lighting conditions and appearance variability. While the action
recognition task on videos collected from visible spectrum imaging has received
much attention, action recognition in IR videos is significantly less explored.
Our objective is to exploit imaging data in this modality for the action
recognition task. In this work, we propose a novel two-stream 3D convolutional
neural network (CNN) architecture by introducing the discriminative code layer
and the corresponding discriminative code loss function. The proposed network
processes IR image and the IR-based optical flow field sequences. We pretrain
the 3D CNN model on the visible spectrum Sports-1M action dataset and finetune
it on the Infrared Action Recognition (InfAR) dataset. To our best knowledge,
this is the first application of the 3D CNN to action recognition in the IR
domain. We conduct an elaborate analysis of different fusion schemes (weighted
average, single and double-layer neural nets) applied to different 3D CNN
outputs. Experimental results demonstrate that our approach can achieve
state-of-the-art average precision (AP) performances on the InfAR dataset: (1)
the proposed two-stream 3D CNN achieves the best reported 77.5% AP, and (2) our
3D CNN model applied to the optical flow fields achieves the best reported
single stream 75.42% AP.
I Probe, Therefore I Am: Designing a Virtual Journalist with Human Emotions
Authors: Kevin K. Bowden, Tommy Nilsson, Christine P. Spencer, Kubra Cengiz, Alexandru Ghitulescu, Jelte B. van Waterschoot
Date: 2017-05-18 00:00:00
Tasks: Speech Synthesis

By utilizing different communication channels, such as verbal language,
gestures or facial expressions, virtually embodied interactive humans hold a
unique potential to bridge the gap between human-computer interaction and
actual interhuman communication. The use of virtual humans is consequently
becoming increasingly popular in a wide range of areas where such a natural
communication might be beneficial, including entertainment, education, mental
health research and beyond. Behind this development lies a series of
technological advances in a multitude of disciplines, most notably natural
language processing, computer vision, and speech synthesis. In this paper we
discuss a Virtual Human Journalist, a project employing a number of novel
solutions from these disciplines with the goal to demonstrate their viability
by producing a humanoid conversational agent capable of naturally eliciting and
reacting to information from a human user. A set of qualitative and
quantitative evaluation sessions demonstrated the technical feasibility of the
system whilst uncovering a number of deficits in its capacity to engage users
in a way that would be perceived as natural and emotionally engaging. We argue
that naturalness should not always be seen as a desirable goal and suggest that
deliberately suppressing the naturalness of virtual human interactions, such as
by altering its personality cues, might in some cases yield more desirable
results.
Limited-Memory Matrix Adaptation for Large Scale Black-box Optimization
Authors: Ilya Loshchilov, Tobias Glasmachers, Hans-Georg Beyer
Date: 2017-05-18 00:00:00
Tasks: Stochastic Optimization

The Covariance Matrix Adaptation Evolution Strategy (CMA-ES) is a popular
method to deal with nonconvex and/or stochastic optimization problems when the
gradient information is not available. Being based on the CMA-ES, the recently
proposed Matrix Adaptation Evolution Strategy (MA-ES) provides a rather
surprising result that the covariance matrix and all associated operations
(e.g., potentially unstable eigendecomposition) can be replaced in the CMA-ES
by a updated transformation matrix without any loss of performance. In order to
further simplify MA-ES and reduce its $\mathcal{O}\big(n^2\big)$ time and
storage complexity to $\mathcal{O}\big(n\log(n)\big)$, we present the
Limited-Memory Matrix Adaptation Evolution Strategy (LM-MA-ES) for efficient
zeroth order large-scale optimization. The algorithm demonstrates
state-of-the-art performance on a set of established large-scale benchmarks. We
explore the algorithm on the problem of generating adversarial inputs for a
(non-smooth) random forest classifier, demonstrating a surprising vulnerability
of the classifier.
Target-Quality Image Compression with Recurrent, Convolutional Neural Networks
Authors: Michele Covell, Nick Johnston, David Minnen, Sung Jin Hwang, Joel Shor, Saurabh Singh, Damien Vincent, George Toderici
Date: 2017-05-18 00:00:00
Tasks: Image Compression

We introduce a stop-code tolerant (SCT) approach to training recurrent
convolutional neural networks for lossy image compression. Our methods
introduce a multi-pass training method to combine the training goals of
high-quality reconstructions in areas around stop-code masking as well as in
highly-detailed areas. These methods lead to lower true bitrates for a given
recursion count, both pre- and post-entropy coding, even using unstructured
LZ77 code compression. The pre-LZ77 gains are achieved by trimming stop codes.
The post-LZ77 gains are due to the highly unequal distributions of 0/1 codes
from the SCT architectures. With these code compressions, the SCT architecture
maintains or exceeds the image quality at all compression rates compared to
JPEG and to RNN auto-encoders across the Kodak dataset. In addition, the SCT
coding results in lower variance in image quality across the extent of the
image, a characteristic that has been shown to be important in human ratings of
image quality
MUTAN: Multimodal Tucker Fusion for Visual Question Answering
Authors: Hedi Ben-Younes, Rémi Cadene, Matthieu Cord, Nicolas Thome
Date: 2017-05-18 00:00:00
Proceeding: ICCV 2017 10
Tasks: Visual Question Answering, Visual Question Answering (VQA)

Bilinear models provide an appealing framework for mixing and merging
information in Visual Question Answering (VQA) tasks. They help to learn high
level associations between question meaning and visual concepts in the image,
but they suffer from huge dimensionality issues. We introduce MUTAN, a
multimodal tensor-based Tucker decomposition to efficiently parametrize
bilinear interactions between visual and textual representations. Additionally
to the Tucker framework, we design a low-rank matrix-based decomposition to
explicitly constrain the interaction rank. With MUTAN, we control the
complexity of the merging scheme while keeping nice interpretable fusion
relations. We show how our MUTAN model generalizes some of the latest VQA
architectures, providing state-of-the-art results.
Object Proposal with Kernelized Partial Ranking
Authors: Jing Wang, Jie Shen, Ping Li
Date: 2015-02-05 00:00:00
Tasks: Object

Object proposals are an ensemble of bounding boxes with high potential to
contain objects. In order to determine a small set of proposals with a high
recall, a common scheme is extracting multiple features followed by a ranking
algorithm which however, incurs two major challenges: {\bf 1)} The ranking
model often imposes pairwise constraints between each proposal, rendering the
problem away from an efficient training/testing phase; {\bf 2)} Linear kernels
are utilized due to the computational and memory bottleneck of training a
kernelized model.
  In this paper, we remedy these two issues by suggesting a {\em kernelized
partial ranking model}. In particular, we demonstrate that {\bf i)} our partial
ranking model reduces the number of constraints from $O(n^2)$ to $O(nk)$ where
$n$ is the number of all potential proposals for an image but we are only
interested in top-$k$ of them that has the largest overlap with the ground
truth; {\bf ii)} we permit non-linear kernels in our model which is often
superior to the linear classifier in terms of accuracy. For the sake of
mitigating the computational and memory issues, we introduce a consistent
weighted sampling~(CWS) paradigm that approximates the non-linear kernel as
well as facilitates an efficient learning. In fact, as we will show, training a
linear CWS model amounts to learning a kernelized model. Extensive experiments
demonstrate that equipped with the non-linear kernel and the partial ranking
algorithm, recall at top-$k$ proposals can be substantially improved.
Drug-drug Interaction Extraction via Recurrent Neural Network with Multiple Attention Layers
Authors: Zibo Yi, Shasha Li, Jie Yu, Qingbo Wu
Date: 2017-05-09 00:00:00
Tasks: Deep Learning, Drug–drug Interaction Extraction, Feature Engineering, General Classification, Sentence

Drug-drug interaction (DDI) is a vital information when physicians and
pharmacists intend to co-administer two or more drugs. Thus, several DDI
databases are constructed to avoid mistakenly combined use. In recent years,
automatically extracting DDIs from biomedical text has drawn researchers'
attention. However, the existing work utilize either complex feature
engineering or NLP tools, both of which are insufficient for sentence
comprehension. Inspired by the deep learning approaches in natural language
processing, we propose a recur- rent neural network model with multiple
attention layers for DDI classification. We evaluate our model on 2013 SemEval
DDIExtraction dataset. The experiments show that our model classifies most of
the drug pairs into correct DDI categories, which outperforms the existing NLP
or deep learning methods.
A Combinatorial Solution to Non-Rigid 3D Shape-to-Image Matching
Authors: Florian Bernard, Frank R. Schmidt, Johan Thunberg, Daniel Cremers
Date: 2016-11-16 00:00:00
Proceeding: CVPR 2017 7
Tasks: Image Registration

We propose a combinatorial solution for the problem of non-rigidly matching a
3D shape to 3D image data. To this end, we model the shape as a triangular mesh
and allow each triangle of this mesh to be rigidly transformed to achieve a
suitable matching to the image. By penalising the distance and the relative
rotation between neighbouring triangles our matching compromises between image
and shape information. In this paper, we resolve two major challenges: Firstly,
we address the resulting large and NP-hard combinatorial problem with a
suitable graph-theoretic approach. Secondly, we propose an efficient
discretisation of the unbounded 6-dimensional Lie group SE(3). To our knowledge
this is the first combinatorial formulation for non-rigid 3D shape-to-image
matching. In contrast to existing local (gradient descent) optimisation
methods, we obtain solutions that do not require a good initialisation and that
are within a bound of the optimal solution. We evaluate the proposed method on
the two problems of non-rigid 3D shape-to-shape and non-rigid 3D shape-to-image
registration and demonstrate that it provides promising results.
Accelerating Discrete Wavelet Transforms on GPUs
Authors: David Barina, Michal Kula, Michal Matysek, Pavel Zemcik
Date: 2017-05-18 00:00:00

The two-dimensional discrete wavelet transform has a huge number of
applications in image-processing techniques. Until now, several papers compared
the performance of such transform on graphics processing units (GPUs). However,
all of them only dealt with lifting and convolution computation schemes. In
this paper, we show that corresponding horizontal and vertical lifting parts of
the lifting scheme can be merged into non-separable lifting units, which halves
the number of steps. We also discuss an optimization strategy leading to a
reduction in the number of arithmetic operations. The schemes were assessed
using the OpenCL and pixel shaders. The proposed non-separable lifting scheme
outperforms the existing schemes in many cases, irrespective of its higher
complexity.
Model-assisted design of experiments in the presence of network correlated outcomes
Authors: Guillaume W. Basse, Edoardo M. Airoldi
Date: 2015-07-03 00:00:00

We consider the problem of how to assign treatment in a randomized
experiment, in which the correlation among the outcomes is informed by a
network available pre-intervention. Working within the potential outcome causal
framework, we develop a class of models that posit such a correlation structure
among the outcomes. Then we leverage these models to develop restricted
randomization strategies for allocating treatment optimally, by minimizing the
mean square error of the estimated average treatment effect. Analytical
decompositions of the mean square error, due both to the model and to the
randomization distribution, provide insights into aspects of the optimal
designs. In particular, the analysis suggests new notions of balance based on
specific network quantities, in addition to classical covariate balance. The
resulting balanced, optimal restricted randomization strategies are still
design unbiased, in situations where the model used to derive them does not
hold. We illustrate how the proposed treatment allocation strategies improve on
allocations that ignore the network structure, with extensive simulations.
Live Orchestral Piano, a system for real-time orchestral music generation
Authors: Léopold Crestel, Philippe Esling
Date: 2016-09-05 00:00:00
Tasks: Music Generation, Prediction

This paper introduces the first system for performing automatic orchestration
based on a real-time piano input. We believe that it is possible to learn the
underlying regularities existing between piano scores and their orchestrations
by notorious composers, in order to automatically perform this task on novel
piano inputs. To that end, we investigate a class of statistical inference
models called conditional Restricted Boltzmann Machine (cRBM). We introduce a
specific evaluation framework for orchestral generation based on a prediction
task in order to assess the quality of different models. As prediction and
creation are two widely different endeavours, we discuss the potential biases
in evaluating temporal generative models through prediction tasks and their
impact on a creative system. Finally, we introduce an implementation of the
proposed model called Live Orchestral Piano (LOP), which allows to perform
real-time projective orchestration of a MIDI keyboard input.
It's Written All Over Your Face: Full-Face Appearance-Based Gaze Estimation
Authors: Xucong Zhang, Yusuke Sugano, Mario Fritz, Andreas Bulling
Date: 2016-11-27 00:00:00
Tasks: All, Gaze Estimation

Eye gaze is an important non-verbal cue for human affect analysis. Recent gaze estimation work indicated that information from the full face region can benefit performance. Pushing this idea further, we propose an appearance-based method that, in contrast to a long-standing line of work in computer vision, only takes the full face image as input. Our method encodes the face image using a convolutional neural network with spatial weights applied on the feature maps to flexibly suppress or enhance information in different facial regions. Through extensive evaluation, we show that our full-face method significantly outperforms the state of the art for both 2D and 3D gaze estimation, achieving improvements of up to 14.3% on MPIIGaze and 27.7% on EYEDIAP for person-independent 3D gaze estimation. We further show that this improvement is consistent across different illumination conditions and gaze directions and particularly pronounced for the most challenging extreme head poses.
Online learnability of Statistical Relational Learning in anomaly detection
Authors: Magnus Jändel, Pontus Svenson, Niclas Wadströmer
Date: 2017-05-18 00:00:00
Tasks: Anomaly Detection, Common Sense Reasoning, Relational Reasoning

Statistical Relational Learning (SRL) methods for anomaly detection are
introduced via a security-related application. Operational requirements for
online learning stability are outlined and compared to mathematical definitions
as applied to the learning process of a representative SRL method - Bayesian
Logic Programs (BLP). Since a formal proof of online stability appears to be
impossible, tentative common sense requirements are formulated and tested by
theoretical and experimental analysis of a simple and analytically tractable
BLP model. It is found that learning algorithms in initial stages of online
learning can lock on unstable false predictors that nevertheless comply with
our tentative stability requirements and thus masquerade as bona fide
solutions. The very expressiveness of SRL seems to cause significant stability
issues in settings with many variables and scarce data. We conclude that
reliable anomaly detection with SRL-methods requires monitoring by an
overarching framework that may involve a comprehensive context knowledge base
or human supervision.
Stepwise Debugging of Answer-Set Programs
Authors: Johannes Oetsch, Jörg Pührer, Hans Tompits
Date: 2017-05-18 00:00:00

We introduce a stepping methodology for answer-set programming (ASP) that
allows for debugging answer-set programs and is based on the stepwise
application of rules. Similar to debugging in imperative languages, where the
behaviour of a program is observed during a step-by-step execution, stepping
for ASP allows for observing the effects that rule applications have in the
computation of an answer set. While the approach is inspired from debugging in
imperative programming, it is conceptually different to stepping in other
paradigms due to non-determinism and declarativity that are inherent to ASP. In
particular, unlike statements in an imperative program that are executed
following a strict control flow, there is no predetermined order in which to
consider rules in ASP during a computation. In our approach, the user is free
to decide which rule to consider active in the next step following his or her
intuition. This way, one can focus on interesting parts of the debugging search
space. Bugs are detected during stepping by revealing differences between the
actual semantics of the program and the expectations of the user. As a solid
formal basis for stepping, we develop a framework of computations for
answer-set programs. For fully supporting different solver languages, we build
our framework on an abstract ASP language that is sufficiently general to
capture different solver languages. To this end, we make use of abstract
constraints as an established abstraction for popular language constructs such
as aggregates. Stepping has been implemented in SeaLion, an integrated
development environment for ASP. We illustrate stepping using an example
scenario and discuss the stepping plugin of SeaLion. Moreover, we elaborate on
methodological aspects and the embedding of stepping in the ASP development
process.
Agent-Centric Risk Assessment: Accident Anticipation and Risky Region Localization
Authors: Kuo-Hao Zeng, Shih-Han Chou, Fu-Hsiang Chan, Juan Carlos Niebles, Min Sun
Date: 2017-05-18 00:00:00
Proceeding: CVPR 2017 7
Tasks: Accident Anticipation

For survival, a living agent must have the ability to assess risk (1) by
temporally anticipating accidents before they occur, and (2) by spatially
localizing risky regions in the environment to move away from threats. In this
paper, we take an agent-centric approach to study the accident anticipation and
risky region localization tasks. We propose a novel soft-attention Recurrent
Neural Network (RNN) which explicitly models both spatial and appearance-wise
non-linear interaction between the agent triggering the event and another agent
or static-region involved. In order to test our proposed method, we introduce
the Epic Fail (EF) dataset consisting of 3000 viral videos capturing various
accidents. In the experiments, we evaluate the risk assessment accuracy both in
the temporal domain (accident anticipation) and spatial domain (risky region
localization) on our EF dataset and the Street Accident (SA) dataset. Our
method consistently outperforms other baselines on both datasets.
Non-Backtracking Spectrum of Degree-Corrected Stochastic Block Models
Authors: Lennart Gulikers, Marc Lelarge, Laurent Massoulié
Date: 2016-09-08 00:00:00
Tasks: Clustering, Community Detection, Stochastic Block Model

Motivated by community detection, we characterise the spectrum of the
non-backtracking matrix $B$ in the Degree-Corrected Stochastic Block Model.
  Specifically, we consider a random graph on $n$ vertices partitioned into two
equal-sized clusters. The vertices have i.i.d. weights $\{ \phi_u \}_{u=1}^n$
with second moment $\Phi^{(2)}$. The intra-cluster connection probability for
vertices $u$ and $v$ is $\frac{\phi_u \phi_v}{n}a$ and the inter-cluster
connection probability is $\frac{\phi_u \phi_v}{n}b$.
  We show that with high probability, the following holds: The leading
eigenvalue of the non-backtracking matrix $B$ is asymptotic to $\rho =
\frac{a+b}{2} \Phi^{(2)}$. The second eigenvalue is asymptotic to $\mu_2 =
\frac{a-b}{2} \Phi^{(2)}$ when $\mu_2^2 > \rho$, but asymptotically bounded by
$\sqrt{\rho}$ when $\mu_2^2 \leq \rho$. All the remaining eigenvalues are
asymptotically bounded by $\sqrt{\rho}$. As a result, a clustering
positively-correlated with the true communities can be obtained based on the
second eigenvector of $B$ in the regime where $\mu_2^2 > \rho.$
  In a previous work we obtained that detection is impossible when $\mu_2^2 <
\rho,$ meaning that there occurs a phase-transition in the sparse regime of the
Degree-Corrected Stochastic Block Model.
  As a corollary, we obtain that Degree-Corrected Erd\H{o}s-R\'enyi graphs
asymptotically satisfy the graph Riemann hypothesis, a quasi-Ramanujan
property.
  A by-product of our proof is a weak law of large numbers for
local-functionals on Degree-Corrected Stochastic Block Models, which could be
of independent interest.
A fully dense and globally consistent 3D map reconstruction approach for GI tract to enhance therapeutic relevance of the endoscopic capsule robot
Authors: Mehmet Turan, Yusuf Yigit Pilavci, Redhwan Jamiruddin, Helder Araujo, Ender Konukoglu, Metin Sitti
Date: 2017-05-18 00:00:00
Tasks: 3D Reconstruction, Diagnostic, Image Registration, Image Stitching

In the gastrointestinal (GI) tract endoscopy field, ingestible wireless
capsule endoscopy is emerging as a novel, minimally invasive diagnostic
technology for inspection of the GI tract and diagnosis of a wide range of
diseases and pathologies. Since the development of this technology, medical
device companies and many research groups have made substantial progress in
converting passive capsule endoscopes to robotic active capsule endoscopes with
most of the functionality of current active flexible endoscopes. However,
robotic capsule endoscopy still has some challenges. In particular, the use of
such devices to generate a precise three-dimensional (3D) mapping of the entire
inner organ remains an unsolved problem. Such global 3D maps of inner organs
would help doctors to detect the location and size of diseased areas more
accurately and intuitively, thus permitting more reliable diagnoses. To our
knowledge, this paper presents the first complete pipeline for a complete 3D
visual map reconstruction of the stomach. The proposed pipeline is modular and
includes a preprocessing module, an image registration module, and a final
shape-from-shading-based 3D reconstruction module; the 3D map is primarily
generated by a combination of image stitching and shape-from-shading
techniques, and is updated in a frame-by-frame iterative fashion via capsule
motion inside the stomach. A comprehensive quantitative analysis of the
proposed 3D reconstruction method is performed using an esophagus gastro
duodenoscopy simulator, three different endoscopic cameras, and a 3D optical
scanner.